2024-09-10T09:43:21.173 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T09:43:21.194 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T09:43:21.293 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008 branch: wip-rishabh-testing-20240909.165402-squid description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7899008' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI2g5sy5QvepqMrfYO6XjqgwAP3Pc/v3GjgLcEGf0Lc+zn805lrLIch/TiLSoUIwNysf/t7ET1QjFpRE98L8vGM= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFiv7T7h+shSMYEsxZQxxK4FylSDbj28yyJwxH+Hw+YGVdHjNSMw583MQNQ8vqElJSOIZPk0CQTdVD2XUxn/Jvk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T09:43:21.294 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T09:43:21.295 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T09:43:21.295 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T09:43:21.297 INFO:teuthology.task.internal:Checking packages... 2024-09-10T09:43:21.318 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T09:43:21.319 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T09:43:21.319 INFO:teuthology.packaging:ref: None 2024-09-10T09:43:21.319 INFO:teuthology.packaging:tag: None 2024-09-10T09:43:21.319 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:43:21.319 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:43:21.319 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:43:21.470 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:43:21.472 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T09:43:21.479 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T09:43:21.479 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T09:43:21.519 INFO:teuthology.task.internal:Saving configuration 2024-09-10T09:43:21.540 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T09:43:21.553 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T09:43:21.571 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:37:47.151974', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKaZUe6fslW44814AP/F8AOUNuI20+wqizhw+Yq1NWDW'} 2024-09-10T09:43:21.588 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:37:47.153871', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINSExG8cygwfLe2njad0pQqfqrUn7bWfmUMdpAcsTsMW'} 2024-09-10T09:43:21.588 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T09:43:21.595 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T09:43:21.595 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T09:43:21.595 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T09:43:21.669 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f130ed2bbe0>, signals=[15]) 2024-09-10T09:43:21.669 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T09:43:21.674 INFO:teuthology.task.internal:Opening connections... 2024-09-10T09:43:21.675 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:43:21.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:43:21.760 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:43:21.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:43:21.837 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T09:43:21.844 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-10T09:43:21.849 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-10T09:43:21.850 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-10T09:43:21.894 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T09:43:21.894 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Ubuntu" 2024-09-10T09:43:21.894 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="22.04" 2024-09-10T09:43:21.894 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T09:43:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_CODENAME=jammy 2024-09-10T09:43:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:ID=ubuntu 2024-09-10T09:43:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE=debian 2024-09-10T09:43:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T09:43:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T09:43:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T09:43:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T09:43:21.895 INFO:teuthology.orchestra.run.smithi110.stdout:UBUNTU_CODENAME=jammy 2024-09-10T09:43:21.896 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-10T09:43:21.912 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-09-10T09:43:21.917 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-09-10T09:43:21.917 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-09-10T09:43:21.969 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T09:43:21.969 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Ubuntu" 2024-09-10T09:43:21.969 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="22.04" 2024-09-10T09:43:21.969 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T09:43:21.969 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_CODENAME=jammy 2024-09-10T09:43:21.969 INFO:teuthology.orchestra.run.smithi145.stdout:ID=ubuntu 2024-09-10T09:43:21.970 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE=debian 2024-09-10T09:43:21.970 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T09:43:21.970 INFO:teuthology.orchestra.run.smithi145.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T09:43:21.970 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T09:43:21.970 INFO:teuthology.orchestra.run.smithi145.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T09:43:21.970 INFO:teuthology.orchestra.run.smithi145.stdout:UBUNTU_CODENAME=jammy 2024-09-10T09:43:21.970 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-09-10T09:43:21.987 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T09:43:21.996 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T09:43:22.027 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T09:43:22.027 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:43:22.030 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:43:22.035 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T09:43:22.062 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T09:43:22.062 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:43:22.076 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:43:22.149 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T09:43:22.202 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T09:43:22.215 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T09:43:22.215 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-10T09:43:22.215 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-10T09:43:22.216 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:43:22.216 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:43:22.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:43:22.221 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-10T09:43:22.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:43:22.225 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-10T09:43:22.392 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:43:22.392 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T09:43:22.392 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-10T09:43:22.397 INFO:teuthology.orchestra.run.smithi110.stdout:5.15.0-56-generic 2024-09-10T09:43:22.398 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:43:22.398 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T09:43:22.398 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-10T09:43:22.398 DEBUG:teuthology.orchestra.run.smithi110:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T09:43:22.410 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:43:22.411 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T09:43:22.411 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-10T09:43:22.416 INFO:teuthology.orchestra.run.smithi145.stdout:5.15.0-56-generic 2024-09-10T09:43:22.416 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:43:22.416 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T09:43:22.417 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-10T09:43:22.417 DEBUG:teuthology.orchestra.run.smithi145:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T09:43:22.459 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T09:43:22.471 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T09:43:22.575 INFO:teuthology.orchestra.run.smithi110.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: git config --global init.defaultBranch 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: 2024-09-10T09:43:22.576 INFO:teuthology.orchestra.run.smithi110.stderr:hint: git branch -m 2024-09-10T09:43:22.577 INFO:teuthology.orchestra.run.smithi110.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T09:43:22.579 DEBUG:teuthology.orchestra.run.smithi110:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T09:43:22.594 INFO:teuthology.orchestra.run.smithi145.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T09:43:22.594 INFO:teuthology.orchestra.run.smithi145.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T09:43:22.594 INFO:teuthology.orchestra.run.smithi145.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T09:43:22.594 INFO:teuthology.orchestra.run.smithi145.stderr:hint: 2024-09-10T09:43:22.594 INFO:teuthology.orchestra.run.smithi145.stderr:hint: git config --global init.defaultBranch 2024-09-10T09:43:22.594 INFO:teuthology.orchestra.run.smithi145.stderr:hint: 2024-09-10T09:43:22.594 INFO:teuthology.orchestra.run.smithi145.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T09:43:22.595 INFO:teuthology.orchestra.run.smithi145.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T09:43:22.595 INFO:teuthology.orchestra.run.smithi145.stderr:hint: 2024-09-10T09:43:22.595 INFO:teuthology.orchestra.run.smithi145.stderr:hint: git branch -m 2024-09-10T09:43:22.595 INFO:teuthology.orchestra.run.smithi145.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T09:43:22.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T09:43:22.649 DEBUG:teuthology.orchestra.run.smithi110:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T09:43:22.667 DEBUG:teuthology.orchestra.run.smithi145:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T09:43:22.703 DEBUG:teuthology.orchestra.run.smithi110:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T09:43:22.727 DEBUG:teuthology.orchestra.run.smithi145:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T09:43:54.574 INFO:teuthology.orchestra.run.smithi145.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T09:43:54.574 INFO:teuthology.orchestra.run.smithi145.stderr: * [new branch] main -> origin/main 2024-09-10T09:43:54.574 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T09:43:54.611 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T09:43:54.611 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T09:43:54.611 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T09:43:54.611 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T09:43:54.612 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T09:43:54.616 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T09:43:54.616 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T09:43:54.616 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T09:43:54.616 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T09:43:54.617 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T09:43:54.618 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T09:43:54.619 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T09:43:54.619 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T09:43:54.619 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T09:43:54.620 INFO:teuthology.orchestra.run.smithi145.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T09:43:54.855 INFO:teuthology.orchestra.run.smithi110.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T09:43:54.856 INFO:teuthology.orchestra.run.smithi110.stderr: * [new branch] main -> origin/main 2024-09-10T09:43:54.856 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T09:43:54.894 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T09:43:54.894 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T09:43:54.894 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T09:43:54.894 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T09:43:54.894 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T09:43:54.894 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T09:43:54.894 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T09:43:54.894 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T09:43:54.895 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T09:43:54.896 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T09:43:54.896 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T09:43:54.896 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T09:43:54.900 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T09:43:54.900 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T09:43:54.900 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T09:43:54.900 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T09:43:54.900 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T09:43:54.900 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T09:43:54.900 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T09:43:54.900 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T09:43:54.901 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T09:43:54.902 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T09:43:54.903 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T09:43:54.903 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T09:43:54.903 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T09:43:54.903 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T09:43:54.903 INFO:teuthology.orchestra.run.smithi110.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T09:43:59.147 INFO:teuthology.orchestra.run.smithi145.stderr:Updating files: 37% (1326/3499) Updating files: 38% (1330/3499) Updating files: 39% (1365/3499) Updating files: 40% (1400/3499) Updating files: 41% (1435/3499) Updating files: 42% (1470/3499) Updating files: 43% (1505/3499) Updating files: 44% (1540/3499) Updating files: 45% (1575/3499) Updating files: 46% (1610/3499) Updating files: 47% (1645/3499) Updating files: 48% (1680/3499) Updating files: 49% (1715/3499) Updating files: 50% (1750/3499) Updating files: 51% (1785/3499) Updating files: 52% (1820/3499) Updating files: 53% (1855/3499) Updating files: 54% (1890/3499) Updating files: 55% (1925/3499) Updating files: 56% (1960/3499) Updating files: 57% (1995/3499) Updating files: 58% (2030/3499) Updating files: 58% (2063/3499) Updating files: 59% (2065/3499) Updating files: 60% (2100/3499) Updating files: 61% (2135/3499) Updating files: 62% (2170/3499) Updating files: 63% (2205/3499) Updating files: 64% (2240/3499) Updating files: 65% (2275/3499) Updating files: 66% (2310/3499) Updating files: 67% (2345/3499) Updating files: 68% (2380/3499) Updating files: 69% (2415/3499) Updating files: 69% (2422/3499) Updating files: 70% (2450/3499) Updating files: 71% (2485/3499) Updating files: 72% (2520/3499) Updating files: 73% (2555/3499) Updating files: 74% (2590/3499) Updating files: 75% (2625/3499) Updating files: 76% (2660/3499) Updating files: 77% (2695/3499) Updating files: 78% (2730/3499) Updating files: 79% (2765/3499) Updating files: 80% (2800/3499) Updating files: 81% (2835/3499) Updating files: 82% (2870/3499) Updating files: 82% (2889/3499) Updating files: 83% (2905/3499) Updating files: 84% (2940/3499) Updating files: 85% (2975/3499) Updating files: 86% (3010/3499) Updating files: 87% (3045/3499) Updating files: 88% (3080/3499) Updating files: 89% (3115/3499) Updating files: 90% (3150/3499) Updating files: 91% (3185/3499) Updating files: 92% (3220/3499) Updating files: 93% (3255/3499) Updating files: 94% (3290/3499) Updating files: 95% (3325/3499) Updating files: 96% (3360/3499) Updating files: 97% (3395/3499) Updating files: 98% (3430/3499) Updating files: 99% (3465/3499) Updating files: 100% (3499/3499) Updating files: 100% (3499/3499), done. 2024-09-10T09:43:59.152 INFO:teuthology.orchestra.run.smithi145.stdout:HEAD is now at 552ed9b8 Merge branch 'mtl-dmc-v2.23' into 'main' 2024-09-10T09:43:59.173 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T09:43:59.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:43:59.325 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T09:43:59.325 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T09:43:59.328 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi145.front.sepia.ceph.com to complete... 2024-09-10T09:43:59.421 INFO:teuthology.orchestra.run.smithi110.stderr:Updating files: 35% (1231/3499) Updating files: 36% (1260/3499) Updating files: 37% (1295/3499) Updating files: 38% (1330/3499) Updating files: 39% (1365/3499) Updating files: 40% (1400/3499) Updating files: 41% (1435/3499) Updating files: 42% (1470/3499) Updating files: 43% (1505/3499) Updating files: 44% (1540/3499) Updating files: 45% (1575/3499) Updating files: 46% (1610/3499) Updating files: 47% (1645/3499) Updating files: 48% (1680/3499) Updating files: 49% (1715/3499) Updating files: 50% (1750/3499) Updating files: 51% (1785/3499) Updating files: 52% (1820/3499) Updating files: 53% (1855/3499) Updating files: 54% (1890/3499) Updating files: 55% (1925/3499) Updating files: 56% (1960/3499) Updating files: 57% (1995/3499) Updating files: 58% (2030/3499) Updating files: 58% (2062/3499) Updating files: 59% (2065/3499) Updating files: 60% (2100/3499) Updating files: 61% (2135/3499) Updating files: 62% (2170/3499) Updating files: 63% (2205/3499) Updating files: 64% (2240/3499) Updating files: 65% (2275/3499) Updating files: 66% (2310/3499) Updating files: 67% (2345/3499) Updating files: 68% (2380/3499) Updating files: 69% (2415/3499) Updating files: 69% (2423/3499) Updating files: 70% (2450/3499) Updating files: 71% (2485/3499) Updating files: 72% (2520/3499) Updating files: 73% (2555/3499) Updating files: 74% (2590/3499) Updating files: 75% (2625/3499) Updating files: 76% (2660/3499) Updating files: 77% (2695/3499) Updating files: 78% (2730/3499) Updating files: 79% (2765/3499) Updating files: 80% (2800/3499) Updating files: 81% (2835/3499) Updating files: 82% (2870/3499) Updating files: 82% (2889/3499) Updating files: 83% (2905/3499) Updating files: 84% (2940/3499) Updating files: 85% (2975/3499) Updating files: 86% (3010/3499) Updating files: 87% (3045/3499) Updating files: 88% (3080/3499) Updating files: 89% (3115/3499) Updating files: 90% (3150/3499) Updating files: 91% (3185/3499) Updating files: 92% (3220/3499) Updating files: 93% (3255/3499) Updating files: 94% (3290/3499) Updating files: 95% (3325/3499) Updating files: 96% (3360/3499) Updating files: 97% (3395/3499) Updating files: 98% (3430/3499) Updating files: 99% (3465/3499) Updating files: 100% (3499/3499) Updating files: 100% (3499/3499), done. 2024-09-10T09:43:59.426 INFO:teuthology.orchestra.run.smithi110.stdout:HEAD is now at 552ed9b8 Merge branch 'mtl-dmc-v2.23' into 'main' 2024-09-10T09:43:59.448 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T09:43:59.448 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:43:59.600 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T09:43:59.601 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T09:43:59.603 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi110.front.sepia.ceph.com to complete... 2024-09-10T09:44:00.456 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10 09:44:01 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T09:44:00.456 INFO:teuthology.orchestra.run.smithi145.stderr:FINISHED --2024-09-10 09:44:01-- 2024-09-10T09:44:00.456 INFO:teuthology.orchestra.run.smithi145.stderr:Total wall clock time: 1.0s 2024-09-10T09:44:00.456 INFO:teuthology.orchestra.run.smithi145.stderr:Downloaded: 1 files, 43M in 0.8s (51.1 MB/s) 2024-09-10T09:44:00.461 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T09:44:00.461 DEBUG:teuthology.orchestra.run.smithi145:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T09:44:00.706 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T09:44:01.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10 09:44:01 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T09:44:01.775 INFO:teuthology.orchestra.run.smithi110.stderr:FINISHED --2024-09-10 09:44:01-- 2024-09-10T09:44:01.775 INFO:teuthology.orchestra.run.smithi110.stderr:Total wall clock time: 2.0s 2024-09-10T09:44:01.775 INFO:teuthology.orchestra.run.smithi110.stderr:Downloaded: 1 files, 43M in 1.8s (24.1 MB/s) 2024-09-10T09:44:01.779 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T09:44:01.779 DEBUG:teuthology.orchestra.run.smithi110:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T09:44:02.028 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T09:44:02.758 INFO:teuthology.orchestra.run.smithi145.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T09:44:02.762 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T09:44:02.876 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T09:44:03.995 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T09:44:03.999 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T09:44:04.090 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T09:44:06.649 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T09:44:06.748 INFO:teuthology.orchestra.run.smithi145.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:07.873 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T09:44:07.986 INFO:teuthology.orchestra.run.smithi110.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:28.356 INFO:teuthology.orchestra.run.smithi145.stderr:Sourcing file `/etc/default/grub' 2024-09-10T09:44:28.372 INFO:teuthology.orchestra.run.smithi145.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T09:44:28.388 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2024-09-10T09:44:28.695 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:28.715 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:29.004 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T09:44:29.013 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T09:44:29.077 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T09:44:29.082 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T09:44:29.244 INFO:teuthology.orchestra.run.smithi145.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T09:44:29.244 INFO:teuthology.orchestra.run.smithi145.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T09:44:29.244 INFO:teuthology.orchestra.run.smithi145.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T09:44:29.290 INFO:teuthology.orchestra.run.smithi145.stderr:done 2024-09-10T09:44:29.442 DEBUG:teuthology.orchestra.run.smithi145:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T09:44:29.451 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T09:44:29.451 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T09:44:29.451 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:29.466 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:29.466 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:29.486 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T09:44:29.486 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T09:44:29.486 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T09:44:29.486 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T09:44:29.486 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T09:44:29.486 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T09:44:29.486 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T09:44:29.486 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T09:44:29.487 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T09:44:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T09:44:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T09:44:29.490 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T09:44:29.490 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T09:44:29.490 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T09:44:29.490 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T09:44:29.552 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T09:44:29.553 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T09:44:29.554 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T09:44:29.555 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T09:44:29.556 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T09:44:29.557 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T09:44:29.558 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T09:44:29.559 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T09:44:29.560 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T09:44:29.561 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T09:44:29.562 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T09:44:29.563 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T09:44:29.564 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T09:44:29.565 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T09:44:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T09:44:29.567 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T09:44:29.568 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T09:44:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T09:44:29.570 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T09:44:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T09:44:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T09:44:29.573 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T09:44:29.574 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T09:44:29.575 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T09:44:29.576 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T09:44:29.577 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T09:44:29.578 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T09:44:29.579 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T09:44:29.580 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T09:44:29.581 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T09:44:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T09:44:29.583 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T09:44:29.584 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T09:44:29.585 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T09:44:29.586 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T09:44:29.587 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T09:44:29.588 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T09:44:29.589 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T09:44:29.590 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T09:44:29.591 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T09:44:29.592 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T09:44:29.593 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T09:44:29.594 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T09:44:29.595 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T09:44:29.596 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T09:44:29.597 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T09:44:29.598 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T09:44:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T09:44:29.600 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T09:44:29.601 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T09:44:29.602 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T09:44:29.603 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T09:44:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T09:44:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T09:44:29.606 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T09:44:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T09:44:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T09:44:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T09:44:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T09:44:29.611 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T09:44:29.612 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T09:44:29.613 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T09:44:29.614 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T09:44:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T09:44:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T09:44:29.683 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T09:44:29.684 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T09:44:29.685 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T09:44:29.686 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T09:44:29.687 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T09:44:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T09:44:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T09:44:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T09:44:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T09:44:29.692 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T09:44:29.693 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T09:44:29.694 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T09:44:29.695 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T09:44:29.696 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T09:44:29.697 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T09:44:29.698 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T09:44:29.699 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T09:44:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T09:44:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T09:44:29.702 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T09:44:29.703 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T09:44:29.704 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T09:44:29.705 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T09:44:29.706 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T09:44:29.707 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T09:44:29.708 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T09:44:29.709 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T09:44:29.710 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T09:44:29.711 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T09:44:29.712 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T09:44:29.713 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T09:44:29.714 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T09:44:29.715 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T09:44:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T09:44:29.717 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T09:44:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T09:44:29.719 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T09:44:29.720 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T09:44:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T09:44:29.722 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T09:44:29.723 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T09:44:29.724 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T09:44:29.725 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T09:44:29.726 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T09:44:29.727 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T09:44:29.728 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T09:44:29.729 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T09:44:29.730 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T09:44:29.731 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T09:44:29.732 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T09:44:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T09:44:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T09:44:29.735 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T09:44:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T09:44:29.737 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T09:44:29.738 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T09:44:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T09:44:29.740 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T09:44:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T09:44:29.742 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T09:44:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T09:44:29.744 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T09:44:29.745 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T09:44:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T09:44:29.747 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T09:44:29.748 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T09:44:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T09:44:29.750 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T09:44:29.751 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T09:44:29.752 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T09:44:29.753 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T09:44:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T09:44:29.755 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T09:44:29.799 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T09:44:29.799 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T09:44:29.799 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T09:44:29.800 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T09:44:29.801 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T09:44:29.802 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T09:44:29.803 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T09:44:29.804 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T09:44:29.805 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T09:44:29.806 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T09:44:29.807 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T09:44:29.808 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T09:44:29.809 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T09:44:29.810 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T09:44:29.811 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T09:44:29.812 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T09:44:29.813 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T09:44:29.814 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T09:44:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T09:44:29.816 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T09:44:29.817 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T09:44:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T09:44:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T09:44:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T09:44:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T09:44:29.822 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T09:44:29.823 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T09:44:29.824 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T09:44:29.825 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T09:44:29.826 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T09:44:29.827 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T09:44:29.828 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T09:44:29.829 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T09:44:29.830 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T09:44:29.831 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T09:44:29.832 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T09:44:29.833 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T09:44:29.834 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T09:44:29.835 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T09:44:29.836 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T09:44:29.837 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T09:44:29.838 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T09:44:29.839 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T09:44:29.840 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T09:44:29.841 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T09:44:29.842 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T09:44:29.843 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T09:44:29.844 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T09:44:29.845 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T09:44:29.846 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T09:44:29.847 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T09:44:29.848 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T09:44:29.849 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T09:44:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T09:44:29.855 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:29.856 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:29.856 DEBUG:teuthology.orchestra.run.smithi145:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T09:44:29.862 INFO:teuthology.orchestra.run.smithi145.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T09:44:29.862 INFO:teuthology.orchestra.run.smithi145.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T09:44:29.862 INFO:teuthology.orchestra.run.smithi145.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T09:44:29.863 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T09:44:29.863 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:29.863 DEBUG:teuthology.orchestra.run.smithi145:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T09:44:29.906 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi145.front.sepia.ceph.com to complete... 2024-09-10T09:44:30.106 INFO:teuthology.orchestra.run.smithi110.stderr:Sourcing file `/etc/default/grub' 2024-09-10T09:44:30.119 INFO:teuthology.orchestra.run.smithi110.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T09:44:30.131 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-10T09:44:30.285 INFO:teuthology.orchestra.run.smithi145.stderr:Sourcing file `/etc/default/grub' 2024-09-10T09:44:30.286 INFO:teuthology.orchestra.run.smithi145.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T09:44:30.321 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2024-09-10T09:44:30.502 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:30.507 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:30.559 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:30.576 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:30.741 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T09:44:30.747 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T09:44:30.789 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T09:44:30.795 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T09:44:30.909 INFO:teuthology.orchestra.run.smithi145.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T09:44:30.909 INFO:teuthology.orchestra.run.smithi145.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T09:44:30.909 INFO:teuthology.orchestra.run.smithi145.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T09:44:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T09:44:30.926 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T09:44:30.929 INFO:teuthology.orchestra.run.smithi145.stderr:done 2024-09-10T09:44:30.994 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T09:44:31.000 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T09:44:31.153 INFO:teuthology.orchestra.run.smithi110.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T09:44:31.153 INFO:teuthology.orchestra.run.smithi110.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T09:44:31.153 INFO:teuthology.orchestra.run.smithi110.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T09:44:31.191 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-10T09:44:31.366 DEBUG:teuthology.orchestra.run.smithi110:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T09:44:31.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T09:44:31.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T09:44:31.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:31.397 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:31.397 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:31.416 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T09:44:31.417 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T09:44:31.418 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T09:44:31.419 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T09:44:31.420 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T09:44:31.420 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T09:44:31.420 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T09:44:31.420 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T09:44:31.437 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T09:44:31.437 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T09:44:31.437 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T09:44:31.437 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T09:44:31.437 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T09:44:31.438 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T09:44:31.439 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T09:44:31.440 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T09:44:31.441 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T09:44:31.442 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T09:44:31.443 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T09:44:31.444 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T09:44:31.445 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T09:44:31.446 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T09:44:31.447 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T09:44:31.448 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T09:44:31.449 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T09:44:31.450 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T09:44:31.496 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T09:44:31.497 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T09:44:31.498 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T09:44:31.499 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T09:44:31.500 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T09:44:31.501 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T09:44:31.502 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T09:44:31.503 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T09:44:31.504 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T09:44:31.505 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T09:44:31.505 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T09:44:31.505 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T09:44:31.505 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T09:44:31.505 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T09:44:31.505 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T09:44:31.506 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T09:44:31.507 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T09:44:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T09:44:31.509 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T09:44:31.510 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T09:44:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T09:44:31.512 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T09:44:31.513 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T09:44:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T09:44:31.515 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T09:44:31.516 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T09:44:31.517 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T09:44:31.518 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T09:44:31.519 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T09:44:31.520 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T09:44:31.521 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T09:44:31.522 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T09:44:31.523 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T09:44:31.524 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T09:44:31.525 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T09:44:31.526 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T09:44:31.527 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T09:44:31.528 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T09:44:31.529 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T09:44:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T09:44:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T09:44:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T09:44:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T09:44:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T09:44:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T09:44:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T09:44:31.537 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T09:44:31.538 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T09:44:31.539 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T09:44:31.540 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T09:44:31.541 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T09:44:31.542 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T09:44:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T09:44:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T09:44:31.545 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T09:44:31.582 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T09:44:31.582 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T09:44:31.583 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T09:44:31.584 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T09:44:31.585 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T09:44:31.586 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T09:44:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T09:44:31.588 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T09:44:31.589 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T09:44:31.590 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T09:44:31.591 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T09:44:31.592 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T09:44:31.593 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T09:44:31.594 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T09:44:31.595 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T09:44:31.596 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T09:44:31.597 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T09:44:31.598 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T09:44:31.599 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T09:44:31.600 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T09:44:31.601 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T09:44:31.602 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T09:44:31.603 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T09:44:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T09:44:31.605 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T09:44:31.606 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T09:44:31.607 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T09:44:31.608 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T09:44:31.609 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T09:44:31.610 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T09:44:31.611 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T09:44:31.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T09:44:31.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T09:44:31.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T09:44:31.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T09:44:31.616 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T09:44:31.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T09:44:31.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T09:44:31.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T09:44:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T09:44:31.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T09:44:31.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T09:44:31.659 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T09:44:31.660 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T09:44:31.661 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T09:44:31.662 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T09:44:31.663 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T09:44:31.664 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T09:44:31.665 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T09:44:31.666 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T09:44:31.667 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T09:44:31.668 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T09:44:31.669 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T09:44:31.670 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T09:44:31.671 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T09:44:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T09:44:31.673 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T09:44:31.674 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T09:44:31.675 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T09:44:31.676 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T09:44:31.677 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T09:44:31.678 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T09:44:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T09:44:31.680 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T09:44:31.681 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T09:44:31.682 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T09:44:31.683 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T09:44:31.684 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T09:44:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T09:44:31.686 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T09:44:31.687 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T09:44:31.688 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T09:44:31.689 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T09:44:31.690 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T09:44:31.691 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T09:44:31.692 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T09:44:31.693 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T09:44:31.694 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T09:44:31.695 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T09:44:31.696 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T09:44:31.697 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T09:44:31.698 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T09:44:31.699 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T09:44:31.699 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T09:44:31.699 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T09:44:31.699 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T09:44:31.732 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T09:44:31.733 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T09:44:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T09:44:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T09:44:31.736 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T09:44:31.737 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T09:44:31.738 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T09:44:31.739 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T09:44:31.740 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T09:44:31.741 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T09:44:31.742 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T09:44:31.743 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T09:44:31.744 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T09:44:31.745 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T09:44:31.746 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T09:44:31.747 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T09:44:31.748 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T09:44:31.749 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T09:44:31.750 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T09:44:31.751 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T09:44:31.752 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T09:44:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T09:44:31.756 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T09:44:31.757 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T09:44:31.758 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T09:44:31.759 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T09:44:31.760 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T09:44:31.761 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T09:44:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T09:44:31.763 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T09:44:31.764 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T09:44:31.765 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T09:44:31.766 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T09:44:31.767 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T09:44:31.768 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T09:44:31.769 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T09:44:31.770 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T09:44:31.771 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T09:44:31.772 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T09:44:31.774 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T09:44:31.775 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T09:44:31.779 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:31.780 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:31.780 DEBUG:teuthology.orchestra.run.smithi110:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T09:44:31.824 INFO:teuthology.orchestra.run.smithi110.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T09:44:31.824 INFO:teuthology.orchestra.run.smithi110.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T09:44:31.824 INFO:teuthology.orchestra.run.smithi110.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T09:44:31.824 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T09:44:31.824 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:31.825 DEBUG:teuthology.orchestra.run.smithi110:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T09:44:31.868 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi110.front.sepia.ceph.com to complete... 2024-09-10T09:44:32.240 INFO:teuthology.orchestra.run.smithi110.stderr:Sourcing file `/etc/default/grub' 2024-09-10T09:44:32.240 INFO:teuthology.orchestra.run.smithi110.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T09:44:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-10T09:44:32.519 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:32.525 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:44:32.808 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T09:44:32.814 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T09:44:32.855 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T09:44:32.860 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T09:44:32.997 INFO:teuthology.orchestra.run.smithi110.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T09:44:32.997 INFO:teuthology.orchestra.run.smithi110.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T09:44:32.997 INFO:teuthology.orchestra.run.smithi110.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T09:44:33.025 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-10T09:45:02.049 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-10T09:45:02.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T09:45:02.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:45:04.165 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T09:45:04.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-10T09:45:04.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:45:20.449 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-10T09:45:22.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-10T09:45:29.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T09:45:29.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:45:31.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-10T09:45:31.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:45:32.509 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-10T09:45:34.561 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-10T09:45:44.517 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T09:45:44.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:45:46.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-10T09:45:46.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:46:02.941 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-10T09:46:04.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-10T09:46:17.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T09:46:17.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:46:18.215 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-10T09:46:18.750 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T09:46:18.751 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T09:46:18.751 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-10T09:46:18.800 INFO:teuthology.orchestra.run.smithi145.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:46:18.801 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:46:18.801 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T09:46:18.801 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T09:46:19.802 DEBUG:teuthology.parallel:result is None 2024-09-10T09:46:19.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-10T09:46:19.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:46:20.649 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-10T09:46:21.535 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-10T09:46:21.535 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T09:46:21.535 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-10T09:46:21.583 INFO:teuthology.orchestra.run.smithi110.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:46:21.583 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:46:21.583 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T09:46:21.583 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T09:46:22.586 DEBUG:teuthology.parallel:result is None 2024-09-10T09:46:22.586 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T09:46:22.595 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T09:46:22.596 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:46:22.599 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:46:22.607 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T09:46:22.631 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T09:46:22.666 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T09:46:22.666 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:46:22.671 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:46:22.698 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T09:46:22.708 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T09:46:22.708 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:46:22.731 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:46:22.806 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:22.828 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:22.842 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:22.862 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:22.864 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T09:46:22.871 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T09:46:22.872 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:46:22.875 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:46:22.896 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T09:46:22.907 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T09:46:22.907 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:46:22.938 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:46:22.949 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:46:22.991 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:46:23.039 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:46:23.040 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:46:23.109 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:46:23.119 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:46:23.173 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:46:23.173 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:46:23.253 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-10T09:46:23.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-09-10T09:46:23.390 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T09:46:23.398 INFO:teuthology.task.internal:Starting timer... 2024-09-10T09:46:23.398 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T09:46:23.426 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T09:46:23.458 DEBUG:teuthology.task.selinux:Excluding smithi110: OS 'ubuntu' does not support SELinux 2024-09-10T09:46:23.458 DEBUG:teuthology.task.selinux:Excluding smithi145: OS 'ubuntu' does not support SELinux 2024-09-10T09:46:23.458 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T09:46:23.458 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-10T09:46:23.458 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T09:46:23.458 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T09:46:23.491 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T09:46:23.503 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T09:46:23.504 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi110.front.sepia.ceph.com,smithi145.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T09:54:28.952 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com')] 2024-09-10T09:54:28.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-10T09:54:28.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:54:29.042 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-10T09:54:29.176 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-10T09:54:29.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T09:54:29.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:54:29.259 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-10T09:54:29.379 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-10T09:54:29.379 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T09:54:29.390 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T09:54:29.390 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:54:29.391 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:54:29.394 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:54:29.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:54:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T09:54:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Command line: ntpd -gq 2024-09-10T09:54:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: ---------------------------------------------------- 2024-09-10T09:54:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T09:54:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T09:54:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: corporation. Support and training for ntp-4 are 2024-09-10T09:54:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: available at https://www.nwtime.org/support 2024-09-10T09:54:29.430 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: ---------------------------------------------------- 2024-09-10T09:54:29.431 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: proto: precision = 0.043 usec (-24) 2024-09-10T09:54:29.431 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: basedate set to 2022-02-04 2024-09-10T09:54:29.431 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: gps base set to 2022-02-06 (week 2196) 2024-09-10T09:54:29.431 INFO:teuthology.orchestra.run.smithi110.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T09:54:29.432 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T09:54:29.432 INFO:teuthology.orchestra.run.smithi110.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T09:54:29.432 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T09:54:29.433 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T09:54:29.433 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T09:54:29.433 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T09:54:29.434 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Listen normally on 3 enp3s0f1 172.21.15.110:123 2024-09-10T09:54:29.434 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Listen normally on 4 lo [::1]:123 2024-09-10T09:54:29.434 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc35%5]:123 2024-09-10T09:54:29.434 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:29 ntpd[18428]: Listening on routing socket on fd #22 for interface updates 2024-09-10T09:54:29.459 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T09:54:29.459 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Command line: ntpd -gq 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: ---------------------------------------------------- 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: corporation. Support and training for ntp-4 are 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: available at https://www.nwtime.org/support 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: ---------------------------------------------------- 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: proto: precision = 0.042 usec (-24) 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: basedate set to 2022-02-04 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: gps base set to 2022-02-06 (week 2196) 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi145.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi145.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T09:54:29.463 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T09:54:29.463 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T09:54:29.463 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T09:54:29.463 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Listen normally on 3 enp3s0f1 172.21.15.145:123 2024-09-10T09:54:29.463 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Listen normally on 4 lo [::1]:123 2024-09-10T09:54:29.463 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7791%5]:123 2024-09-10T09:54:29.463 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:29 ntpd[18433]: Listening on routing socket on fd #22 for interface updates 2024-09-10T09:54:30.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:30 ntpd[18428]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:32.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:32.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:32.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:32.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:32.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:32.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:32.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:32.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:32.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:32.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:32.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:32.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:32 ntpd[18428]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:32.460 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:32 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:32.460 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:32 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:32.460 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:32 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:32.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:32 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:32.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:32 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:32.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:32 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:32.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:32 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:32.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:32 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:34.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:34.434 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:34 ntpd[18428]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:34.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:34.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:34 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:36.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.431 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:36.432 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:36.432 INFO:teuthology.orchestra.run.smithi110.stdout:10 Sep 09:54:36 ntpd[18428]: ntpd: time slew -0.000979 s 2024-09-10T09:54:36.432 INFO:teuthology.orchestra.run.smithi110.stdout:ntpd: time slew -0.000979s 2024-09-10T09:54:36.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T09:54:36.433 INFO:teuthology.orchestra.run.smithi110.stderr:10 Sep 09:54:36 ntpd[18428]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T09:54:36.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:36.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:36 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:36.530 INFO:teuthology.orchestra.run.smithi110.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T09:54:36.530 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================== 2024-09-10T09:54:36.530 INFO:teuthology.orchestra.run.smithi110.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T09:54:36.530 INFO:teuthology.orchestra.run.smithi110.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T09:54:36.530 INFO:teuthology.orchestra.run.smithi110.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T09:54:36.530 INFO:teuthology.orchestra.run.smithi110.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T09:54:38.461 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:38.462 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:38.463 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:38.463 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T09:54:38.463 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T09:54:38.463 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T09:54:38.463 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T09:54:38.463 INFO:teuthology.orchestra.run.smithi145.stdout:10 Sep 09:54:38 ntpd[18433]: ntpd: time slew -0.000506 s 2024-09-10T09:54:38.464 INFO:teuthology.orchestra.run.smithi145.stdout:ntpd: time slew -0.000506s 2024-09-10T09:54:38.464 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T09:54:38.464 INFO:teuthology.orchestra.run.smithi145.stderr:10 Sep 09:54:38 ntpd[18433]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T09:54:38.570 INFO:teuthology.orchestra.run.smithi145.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T09:54:38.570 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================== 2024-09-10T09:54:38.571 INFO:teuthology.orchestra.run.smithi145.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T09:54:38.571 INFO:teuthology.orchestra.run.smithi145.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T09:54:38.571 INFO:teuthology.orchestra.run.smithi145.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T09:54:38.571 INFO:teuthology.orchestra.run.smithi145.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T09:54:38.572 INFO:teuthology.run_tasks:Running task install... 2024-09-10T09:54:38.581 DEBUG:teuthology.task.install:project ceph 2024-09-10T09:54:38.581 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T09:54:38.582 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T09:54:38.582 INFO:teuthology.task.install:Using flavor: default 2024-09-10T09:54:38.588 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T09:54:38.588 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T09:54:38.588 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-key list | grep Ceph 2024-09-10T09:54:38.589 DEBUG:teuthology.orchestra.run.smithi145:> sudo apt-key list | grep Ceph 2024-09-10T09:54:38.692 INFO:teuthology.orchestra.run.smithi110.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T09:54:38.715 INFO:teuthology.orchestra.run.smithi145.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T09:54:38.744 INFO:teuthology.orchestra.run.smithi110.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T09:54:38.744 INFO:teuthology.orchestra.run.smithi110.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T09:54:38.745 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T09:54:38.745 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T09:54:38.746 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:54:38.765 INFO:teuthology.orchestra.run.smithi145.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T09:54:38.766 INFO:teuthology.orchestra.run.smithi145.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T09:54:38.767 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T09:54:38.767 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T09:54:38.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:54:38.895 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default/ 2024-09-10T09:54:38.895 INFO:teuthology.task.install.deb:Package version is 19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:54:38.907 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default/ 2024-09-10T09:54:38.907 INFO:teuthology.task.install.deb:Package version is 19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:54:39.008 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:54:39.008 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T09:54:39.024 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:54:39.025 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T09:54:39.027 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get update 2024-09-10T09:54:39.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo apt-get update 2024-09-10T09:54:39.210 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T09:54:39.217 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T09:54:39.235 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T09:54:39.259 INFO:teuthology.orchestra.run.smithi110.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T09:54:39.275 INFO:teuthology.orchestra.run.smithi145.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T09:54:39.280 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-10T09:54:39.296 INFO:teuthology.orchestra.run.smithi145.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-10T09:54:39.302 INFO:teuthology.orchestra.run.smithi110.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T09:54:39.318 INFO:teuthology.orchestra.run.smithi145.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T09:54:39.323 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-10T09:54:39.339 INFO:teuthology.orchestra.run.smithi145.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-10T09:54:39.347 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T09:54:39.354 INFO:teuthology.orchestra.run.smithi110.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,404 B] 2024-09-10T09:54:39.360 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T09:54:39.360 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T09:54:39.381 INFO:teuthology.orchestra.run.smithi145.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,404 B] 2024-09-10T09:54:39.425 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T09:54:39.507 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T09:54:39.806 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 28.9 kB in 1s (52.4 kB/s) 2024-09-10T09:54:39.953 INFO:teuthology.orchestra.run.smithi145.stdout:Fetched 28.9 kB in 1s (42.3 kB/s) 2024-09-10T09:54:40.854 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-10T09:54:40.869 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-305-g1cf819b7-1jammy cephadm=19.1.1-305-g1cf819b7-1jammy ceph-mds=19.1.1-305-g1cf819b7-1jammy ceph-mgr=19.1.1-305-g1cf819b7-1jammy ceph-common=19.1.1-305-g1cf819b7-1jammy ceph-fuse=19.1.1-305-g1cf819b7-1jammy ceph-test=19.1.1-305-g1cf819b7-1jammy ceph-volume=19.1.1-305-g1cf819b7-1jammy radosgw=19.1.1-305-g1cf819b7-1jammy python3-rados=19.1.1-305-g1cf819b7-1jammy python3-rgw=19.1.1-305-g1cf819b7-1jammy python3-cephfs=19.1.1-305-g1cf819b7-1jammy python3-rbd=19.1.1-305-g1cf819b7-1jammy libcephfs2=19.1.1-305-g1cf819b7-1jammy libcephfs-dev=19.1.1-305-g1cf819b7-1jammy librados2=19.1.1-305-g1cf819b7-1jammy librbd1=19.1.1-305-g1cf819b7-1jammy rbd-fuse=19.1.1-305-g1cf819b7-1jammy python3-cephfs=19.1.1-305-g1cf819b7-1jammy cephfs-shell=19.1.1-305-g1cf819b7-1jammy cephfs-top=19.1.1-305-g1cf819b7-1jammy cephfs-mirror=19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:54:40.945 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-10T09:54:40.981 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-10T09:54:40.997 DEBUG:teuthology.orchestra.run.smithi145:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-305-g1cf819b7-1jammy cephadm=19.1.1-305-g1cf819b7-1jammy ceph-mds=19.1.1-305-g1cf819b7-1jammy ceph-mgr=19.1.1-305-g1cf819b7-1jammy ceph-common=19.1.1-305-g1cf819b7-1jammy ceph-fuse=19.1.1-305-g1cf819b7-1jammy ceph-test=19.1.1-305-g1cf819b7-1jammy ceph-volume=19.1.1-305-g1cf819b7-1jammy radosgw=19.1.1-305-g1cf819b7-1jammy python3-rados=19.1.1-305-g1cf819b7-1jammy python3-rgw=19.1.1-305-g1cf819b7-1jammy python3-cephfs=19.1.1-305-g1cf819b7-1jammy python3-rbd=19.1.1-305-g1cf819b7-1jammy libcephfs2=19.1.1-305-g1cf819b7-1jammy libcephfs-dev=19.1.1-305-g1cf819b7-1jammy librados2=19.1.1-305-g1cf819b7-1jammy librbd1=19.1.1-305-g1cf819b7-1jammy rbd-fuse=19.1.1-305-g1cf819b7-1jammy python3-cephfs=19.1.1-305-g1cf819b7-1jammy cephfs-shell=19.1.1-305-g1cf819b7-1jammy cephfs-top=19.1.1-305-g1cf819b7-1jammy cephfs-mirror=19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:54:41.074 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-10T09:54:41.127 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-10T09:54:41.128 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-10T09:54:41.265 INFO:teuthology.orchestra.run.smithi145.stdout:Building dependency tree... 2024-09-10T09:54:41.266 INFO:teuthology.orchestra.run.smithi145.stdout:Reading state information... 2024-09-10T09:54:41.312 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T09:54:41.312 INFO:teuthology.orchestra.run.smithi110.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T09:54:41.312 INFO:teuthology.orchestra.run.smithi110.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T09:54:41.312 INFO:teuthology.orchestra.run.smithi110.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T09:54:41.312 INFO:teuthology.orchestra.run.smithi110.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T09:54:41.313 INFO:teuthology.orchestra.run.smithi110.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T09:54:41.313 INFO:teuthology.orchestra.run.smithi110.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T09:54:41.313 INFO:teuthology.orchestra.run.smithi110.stdout:The following additional packages will be installed: 2024-09-10T09:54:41.313 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T09:54:41.313 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T09:54:41.315 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout:Suggested packages: 2024-09-10T09:54:41.316 INFO:teuthology.orchestra.run.smithi110.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T09:54:41.317 INFO:teuthology.orchestra.run.smithi110.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T09:54:41.317 INFO:teuthology.orchestra.run.smithi110.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T09:54:41.317 INFO:teuthology.orchestra.run.smithi110.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T09:54:41.317 INFO:teuthology.orchestra.run.smithi110.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T09:54:41.317 INFO:teuthology.orchestra.run.smithi110.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T09:54:41.317 INFO:teuthology.orchestra.run.smithi110.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T09:54:41.317 INFO:teuthology.orchestra.run.smithi110.stdout:Recommended packages: 2024-09-10T09:54:41.317 INFO:teuthology.orchestra.run.smithi110.stdout: btrfs-tools 2024-09-10T09:54:41.459 INFO:teuthology.orchestra.run.smithi145.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T09:54:41.459 INFO:teuthology.orchestra.run.smithi145.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T09:54:41.459 INFO:teuthology.orchestra.run.smithi145.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T09:54:41.460 INFO:teuthology.orchestra.run.smithi145.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T09:54:41.460 INFO:teuthology.orchestra.run.smithi145.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T09:54:41.460 INFO:teuthology.orchestra.run.smithi145.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T09:54:41.460 INFO:teuthology.orchestra.run.smithi145.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T09:54:41.461 INFO:teuthology.orchestra.run.smithi145.stdout:The following additional packages will be installed: 2024-09-10T09:54:41.461 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T09:54:41.461 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T09:54:41.461 INFO:teuthology.orchestra.run.smithi145.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-10T09:54:41.461 INFO:teuthology.orchestra.run.smithi145.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T09:54:41.463 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T09:54:41.464 INFO:teuthology.orchestra.run.smithi145.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout:Suggested packages: 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout:Recommended packages: 2024-09-10T09:54:41.465 INFO:teuthology.orchestra.run.smithi145.stdout: btrfs-tools 2024-09-10T09:54:41.469 INFO:teuthology.orchestra.run.smithi110.stdout:The following NEW packages will be installed: 2024-09-10T09:54:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T09:54:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T09:54:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T09:54:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T09:54:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-10T09:54:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T09:54:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T09:54:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T09:54:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T09:54:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T09:54:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T09:54:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T09:54:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T09:54:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T09:54:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: socat unzip xmlstarlet zip 2024-09-10T09:54:41.539 INFO:teuthology.orchestra.run.smithi110.stdout:0 upgraded, 124 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T09:54:41.539 INFO:teuthology.orchestra.run.smithi110.stdout:Need to get 181 MB of archives. 2024-09-10T09:54:41.539 INFO:teuthology.orchestra.run.smithi110.stdout:After this operation, 831 MB of additional disk space will be used. 2024-09-10T09:54:41.539 INFO:teuthology.orchestra.run.smithi110.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T09:54:41.557 INFO:teuthology.orchestra.run.smithi110.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T09:54:41.566 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T09:54:41.608 INFO:teuthology.orchestra.run.smithi110.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T09:54:41.619 INFO:teuthology.orchestra.run.smithi145.stdout:The following NEW packages will be installed: 2024-09-10T09:54:41.619 INFO:teuthology.orchestra.run.smithi145.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T09:54:41.619 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T09:54:41.619 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T09:54:41.619 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-305-g1cf819b7-1jammy [3,592 kB] 2024-09-10T09:54:41.621 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T09:54:41.621 INFO:teuthology.orchestra.run.smithi145.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-10T09:54:41.621 INFO:teuthology.orchestra.run.smithi145.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T09:54:41.621 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T09:54:41.621 INFO:teuthology.orchestra.run.smithi145.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T09:54:41.621 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T09:54:41.622 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T09:54:41.623 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T09:54:41.623 INFO:teuthology.orchestra.run.smithi145.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T09:54:41.623 INFO:teuthology.orchestra.run.smithi145.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T09:54:41.623 INFO:teuthology.orchestra.run.smithi145.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T09:54:41.623 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T09:54:41.623 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T09:54:41.623 INFO:teuthology.orchestra.run.smithi145.stdout: socat unzip xmlstarlet zip 2024-09-10T09:54:41.629 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T09:54:41.635 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T09:54:41.642 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T09:54:41.684 INFO:teuthology.orchestra.run.smithi110.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T09:54:41.686 INFO:teuthology.orchestra.run.smithi110.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T09:54:41.690 INFO:teuthology.orchestra.run.smithi145.stdout:0 upgraded, 124 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T09:54:41.690 INFO:teuthology.orchestra.run.smithi145.stdout:Need to get 181 MB of archives. 2024-09-10T09:54:41.690 INFO:teuthology.orchestra.run.smithi145.stdout:After this operation, 831 MB of additional disk space will be used. 2024-09-10T09:54:41.690 INFO:teuthology.orchestra.run.smithi145.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T09:54:41.693 INFO:teuthology.orchestra.run.smithi110.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T09:54:41.696 INFO:teuthology.orchestra.run.smithi110.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T09:54:41.697 INFO:teuthology.orchestra.run.smithi110.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T09:54:41.703 INFO:teuthology.orchestra.run.smithi110.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T09:54:41.705 INFO:teuthology.orchestra.run.smithi110.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T09:54:41.708 INFO:teuthology.orchestra.run.smithi110.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T09:54:41.708 INFO:teuthology.orchestra.run.smithi110.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T09:54:41.709 INFO:teuthology.orchestra.run.smithi110.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T09:54:41.714 INFO:teuthology.orchestra.run.smithi145.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T09:54:41.720 INFO:teuthology.orchestra.run.smithi110.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T09:54:41.722 INFO:teuthology.orchestra.run.smithi110.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T09:54:41.724 INFO:teuthology.orchestra.run.smithi145.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T09:54:41.724 INFO:teuthology.orchestra.run.smithi110.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T09:54:41.724 INFO:teuthology.orchestra.run.smithi110.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T09:54:41.738 INFO:teuthology.orchestra.run.smithi110.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T09:54:41.739 INFO:teuthology.orchestra.run.smithi110.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T09:54:41.739 INFO:teuthology.orchestra.run.smithi110.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T09:54:41.739 INFO:teuthology.orchestra.run.smithi110.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T09:54:41.740 INFO:teuthology.orchestra.run.smithi110.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T09:54:41.740 INFO:teuthology.orchestra.run.smithi110.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T09:54:41.754 INFO:teuthology.orchestra.run.smithi110.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T09:54:41.755 INFO:teuthology.orchestra.run.smithi110.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T09:54:41.757 INFO:teuthology.orchestra.run.smithi110.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T09:54:41.758 INFO:teuthology.orchestra.run.smithi110.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T09:54:41.765 INFO:teuthology.orchestra.run.smithi145.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T09:54:41.772 INFO:teuthology.orchestra.run.smithi110.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T09:54:41.772 INFO:teuthology.orchestra.run.smithi110.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T09:54:41.773 INFO:teuthology.orchestra.run.smithi110.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T09:54:41.774 INFO:teuthology.orchestra.run.smithi110.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T09:54:41.774 INFO:teuthology.orchestra.run.smithi110.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T09:54:41.775 INFO:teuthology.orchestra.run.smithi110.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T09:54:41.781 INFO:teuthology.orchestra.run.smithi145.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-305-g1cf819b7-1jammy [3,592 kB] 2024-09-10T09:54:41.786 INFO:teuthology.orchestra.run.smithi110.stdout:Get:39 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-305-g1cf819b7-1jammy [3,241 kB] 2024-09-10T09:54:41.786 INFO:teuthology.orchestra.run.smithi145.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T09:54:41.788 INFO:teuthology.orchestra.run.smithi110.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T09:54:41.789 INFO:teuthology.orchestra.run.smithi110.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T09:54:41.789 INFO:teuthology.orchestra.run.smithi145.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T09:54:41.793 INFO:teuthology.orchestra.run.smithi110.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T09:54:41.794 INFO:teuthology.orchestra.run.smithi110.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T09:54:41.801 INFO:teuthology.orchestra.run.smithi145.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T09:54:41.806 INFO:teuthology.orchestra.run.smithi110.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T09:54:41.806 INFO:teuthology.orchestra.run.smithi110.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T09:54:41.807 INFO:teuthology.orchestra.run.smithi110.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T09:54:41.809 INFO:teuthology.orchestra.run.smithi110.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T09:54:41.810 INFO:teuthology.orchestra.run.smithi110.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T09:54:41.811 INFO:teuthology.orchestra.run.smithi110.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T09:54:41.823 INFO:teuthology.orchestra.run.smithi110.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T09:54:41.825 INFO:teuthology.orchestra.run.smithi110.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T09:54:41.825 INFO:teuthology.orchestra.run.smithi110.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T09:54:41.826 INFO:teuthology.orchestra.run.smithi110.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T09:54:41.832 INFO:teuthology.orchestra.run.smithi110.stdout:Get:54 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-305-g1cf819b7-1jammy [719 kB] 2024-09-10T09:54:41.839 INFO:teuthology.orchestra.run.smithi145.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T09:54:41.840 INFO:teuthology.orchestra.run.smithi110.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T09:54:41.841 INFO:teuthology.orchestra.run.smithi145.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T09:54:41.843 INFO:teuthology.orchestra.run.smithi110.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-305-g1cf819b7-1jammy [357 kB] 2024-09-10T09:54:41.848 INFO:teuthology.orchestra.run.smithi145.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T09:54:41.849 INFO:teuthology.orchestra.run.smithi110.stdout:Get:57 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-305-g1cf819b7-1jammy [32.8 kB] 2024-09-10T09:54:41.850 INFO:teuthology.orchestra.run.smithi110.stdout:Get:58 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-305-g1cf819b7-1jammy [180 kB] 2024-09-10T09:54:41.851 INFO:teuthology.orchestra.run.smithi145.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T09:54:41.852 INFO:teuthology.orchestra.run.smithi145.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T09:54:41.852 INFO:teuthology.orchestra.run.smithi110.stdout:Get:59 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-305-g1cf819b7-1jammy [67.6 kB] 2024-09-10T09:54:41.853 INFO:teuthology.orchestra.run.smithi110.stdout:Get:60 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-305-g1cf819b7-1jammy [333 kB] 2024-09-10T09:54:41.858 INFO:teuthology.orchestra.run.smithi145.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T09:54:41.860 INFO:teuthology.orchestra.run.smithi145.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T09:54:41.860 INFO:teuthology.orchestra.run.smithi110.stdout:Get:61 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-305-g1cf819b7-1jammy [6,875 kB] 2024-09-10T09:54:41.862 INFO:teuthology.orchestra.run.smithi145.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T09:54:41.863 INFO:teuthology.orchestra.run.smithi145.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T09:54:41.863 INFO:teuthology.orchestra.run.smithi145.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T09:54:41.877 INFO:teuthology.orchestra.run.smithi145.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T09:54:41.879 INFO:teuthology.orchestra.run.smithi145.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi145.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi145.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T09:54:41.886 INFO:teuthology.orchestra.run.smithi110.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T09:54:41.887 INFO:teuthology.orchestra.run.smithi110.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T09:54:41.888 INFO:teuthology.orchestra.run.smithi110.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T09:54:41.889 INFO:teuthology.orchestra.run.smithi110.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T09:54:41.890 INFO:teuthology.orchestra.run.smithi110.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T09:54:41.895 INFO:teuthology.orchestra.run.smithi145.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T09:54:41.896 INFO:teuthology.orchestra.run.smithi145.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T09:54:41.897 INFO:teuthology.orchestra.run.smithi145.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T09:54:41.897 INFO:teuthology.orchestra.run.smithi145.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T09:54:41.898 INFO:teuthology.orchestra.run.smithi145.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T09:54:41.898 INFO:teuthology.orchestra.run.smithi145.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T09:54:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T09:54:41.913 INFO:teuthology.orchestra.run.smithi110.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T09:54:41.913 INFO:teuthology.orchestra.run.smithi110.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T09:54:41.913 INFO:teuthology.orchestra.run.smithi145.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T09:54:41.913 INFO:teuthology.orchestra.run.smithi145.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T09:54:41.915 INFO:teuthology.orchestra.run.smithi145.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T09:54:41.916 INFO:teuthology.orchestra.run.smithi145.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T09:54:41.929 INFO:teuthology.orchestra.run.smithi110.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T09:54:41.929 INFO:teuthology.orchestra.run.smithi110.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T09:54:41.930 INFO:teuthology.orchestra.run.smithi110.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T09:54:41.930 INFO:teuthology.orchestra.run.smithi145.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T09:54:41.930 INFO:teuthology.orchestra.run.smithi145.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T09:54:41.931 INFO:teuthology.orchestra.run.smithi110.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T09:54:41.931 INFO:teuthology.orchestra.run.smithi110.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T09:54:41.931 INFO:teuthology.orchestra.run.smithi145.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T09:54:41.932 INFO:teuthology.orchestra.run.smithi110.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T09:54:41.932 INFO:teuthology.orchestra.run.smithi145.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T09:54:41.932 INFO:teuthology.orchestra.run.smithi145.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T09:54:41.933 INFO:teuthology.orchestra.run.smithi145.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T09:54:41.939 INFO:teuthology.orchestra.run.smithi110.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T09:54:41.940 INFO:teuthology.orchestra.run.smithi110.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T09:54:41.943 INFO:teuthology.orchestra.run.smithi110.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T09:54:41.947 INFO:teuthology.orchestra.run.smithi145.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T09:54:41.947 INFO:teuthology.orchestra.run.smithi145.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T09:54:41.952 INFO:teuthology.orchestra.run.smithi145.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T09:54:41.952 INFO:teuthology.orchestra.run.smithi145.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T09:54:41.956 INFO:teuthology.orchestra.run.smithi110.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T09:54:41.958 INFO:teuthology.orchestra.run.smithi110.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T09:54:41.958 INFO:teuthology.orchestra.run.smithi110.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T09:54:41.962 INFO:teuthology.orchestra.run.smithi110.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T09:54:41.965 INFO:teuthology.orchestra.run.smithi145.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T09:54:41.965 INFO:teuthology.orchestra.run.smithi110.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T09:54:41.965 INFO:teuthology.orchestra.run.smithi145.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T09:54:41.965 INFO:teuthology.orchestra.run.smithi110.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T09:54:41.966 INFO:teuthology.orchestra.run.smithi110.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T09:54:41.966 INFO:teuthology.orchestra.run.smithi145.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T09:54:41.968 INFO:teuthology.orchestra.run.smithi145.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T09:54:41.969 INFO:teuthology.orchestra.run.smithi145.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T09:54:41.970 INFO:teuthology.orchestra.run.smithi145.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T09:54:41.978 INFO:teuthology.orchestra.run.smithi110.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-305-g1cf819b7-1jammy [112 kB] 2024-09-10T09:54:41.979 INFO:teuthology.orchestra.run.smithi110.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-305-g1cf819b7-1jammy [472 kB] 2024-09-10T09:54:41.983 INFO:teuthology.orchestra.run.smithi110.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T09:54:41.984 INFO:teuthology.orchestra.run.smithi145.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T09:54:41.986 INFO:teuthology.orchestra.run.smithi110.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-305-g1cf819b7-1jammy [25.9 MB] 2024-09-10T09:54:41.986 INFO:teuthology.orchestra.run.smithi145.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T09:54:41.987 INFO:teuthology.orchestra.run.smithi145.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T09:54:41.987 INFO:teuthology.orchestra.run.smithi110.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T09:54:41.987 INFO:teuthology.orchestra.run.smithi145.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T09:54:41.988 INFO:teuthology.orchestra.run.smithi110.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T09:54:42.000 INFO:teuthology.orchestra.run.smithi110.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T09:54:42.000 INFO:teuthology.orchestra.run.smithi110.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T09:54:42.001 INFO:teuthology.orchestra.run.smithi110.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T09:54:42.001 INFO:teuthology.orchestra.run.smithi110.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T09:54:42.002 INFO:teuthology.orchestra.run.smithi145.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T09:54:42.003 INFO:teuthology.orchestra.run.smithi110.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T09:54:42.004 INFO:teuthology.orchestra.run.smithi110.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T09:54:42.011 INFO:teuthology.orchestra.run.smithi110.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T09:54:42.017 INFO:teuthology.orchestra.run.smithi110.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T09:54:42.017 INFO:teuthology.orchestra.run.smithi110.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T09:54:42.019 INFO:teuthology.orchestra.run.smithi110.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T09:54:42.034 INFO:teuthology.orchestra.run.smithi110.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T09:54:42.046 INFO:teuthology.orchestra.run.smithi145.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T09:54:42.047 INFO:teuthology.orchestra.run.smithi145.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T09:54:42.048 INFO:teuthology.orchestra.run.smithi145.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T09:54:42.049 INFO:teuthology.orchestra.run.smithi145.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T09:54:42.050 INFO:teuthology.orchestra.run.smithi145.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T09:54:42.068 INFO:teuthology.orchestra.run.smithi145.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T09:54:42.070 INFO:teuthology.orchestra.run.smithi145.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T09:54:42.071 INFO:teuthology.orchestra.run.smithi145.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T09:54:42.087 INFO:teuthology.orchestra.run.smithi145.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T09:54:42.088 INFO:teuthology.orchestra.run.smithi145.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T09:54:42.089 INFO:teuthology.orchestra.run.smithi145.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T09:54:42.089 INFO:teuthology.orchestra.run.smithi145.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T09:54:42.090 INFO:teuthology.orchestra.run.smithi145.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T09:54:42.090 INFO:teuthology.orchestra.run.smithi145.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T09:54:42.101 INFO:teuthology.orchestra.run.smithi145.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T09:54:42.101 INFO:teuthology.orchestra.run.smithi145.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T09:54:42.105 INFO:teuthology.orchestra.run.smithi145.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T09:54:42.118 INFO:teuthology.orchestra.run.smithi145.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T09:54:42.120 INFO:teuthology.orchestra.run.smithi145.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T09:54:42.120 INFO:teuthology.orchestra.run.smithi145.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T09:54:42.124 INFO:teuthology.orchestra.run.smithi145.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T09:54:42.127 INFO:teuthology.orchestra.run.smithi145.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T09:54:42.127 INFO:teuthology.orchestra.run.smithi145.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T09:54:42.128 INFO:teuthology.orchestra.run.smithi145.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T09:54:42.135 INFO:teuthology.orchestra.run.smithi145.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T09:54:42.140 INFO:teuthology.orchestra.run.smithi145.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T09:54:42.141 INFO:teuthology.orchestra.run.smithi145.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T09:54:42.153 INFO:teuthology.orchestra.run.smithi145.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T09:54:42.153 INFO:teuthology.orchestra.run.smithi145.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T09:54:42.154 INFO:teuthology.orchestra.run.smithi145.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T09:54:42.155 INFO:teuthology.orchestra.run.smithi145.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T09:54:42.156 INFO:teuthology.orchestra.run.smithi145.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T09:54:42.157 INFO:teuthology.orchestra.run.smithi145.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T09:54:42.164 INFO:teuthology.orchestra.run.smithi145.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T09:54:42.170 INFO:teuthology.orchestra.run.smithi145.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T09:54:42.171 INFO:teuthology.orchestra.run.smithi145.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T09:54:42.173 INFO:teuthology.orchestra.run.smithi145.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T09:54:42.188 INFO:teuthology.orchestra.run.smithi145.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T09:54:42.437 INFO:teuthology.orchestra.run.smithi110.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-305-g1cf819b7-1jammy [5,145 kB] 2024-09-10T09:54:42.579 INFO:teuthology.orchestra.run.smithi110.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-305-g1cf819b7-1jammy [245 kB] 2024-09-10T09:54:42.582 INFO:teuthology.orchestra.run.smithi110.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-305-g1cf819b7-1jammy [124 kB] 2024-09-10T09:54:42.583 INFO:teuthology.orchestra.run.smithi110.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-305-g1cf819b7-1jammy [1,555 kB] 2024-09-10T09:54:42.595 INFO:teuthology.orchestra.run.smithi110.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-305-g1cf819b7-1jammy [6,142 kB] 2024-09-10T09:54:42.709 INFO:teuthology.orchestra.run.smithi110.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-305-g1cf819b7-1jammy [22.8 MB] 2024-09-10T09:54:42.975 INFO:teuthology.orchestra.run.smithi145.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-305-g1cf819b7-1jammy [3,241 kB] 2024-09-10T09:54:43.083 INFO:teuthology.orchestra.run.smithi110.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-305-g1cf819b7-1jammy [14.1 kB] 2024-09-10T09:54:43.083 INFO:teuthology.orchestra.run.smithi110.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-305-g1cf819b7-1jammy [877 kB] 2024-09-10T09:54:43.095 INFO:teuthology.orchestra.run.smithi110.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-305-g1cf819b7-1jammy [2,462 kB] 2024-09-10T09:54:43.141 INFO:teuthology.orchestra.run.smithi110.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-305-g1cf819b7-1jammy [794 kB] 2024-09-10T09:54:43.153 INFO:teuthology.orchestra.run.smithi110.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-305-g1cf819b7-1jammy [152 kB] 2024-09-10T09:54:43.156 INFO:teuthology.orchestra.run.smithi110.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-305-g1cf819b7-1jammy [2,284 kB] 2024-09-10T09:54:43.199 INFO:teuthology.orchestra.run.smithi110.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-305-g1cf819b7-1jammy [8,625 kB] 2024-09-10T09:54:43.360 INFO:teuthology.orchestra.run.smithi110.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-305-g1cf819b7-1jammy [14.2 kB] 2024-09-10T09:54:43.360 INFO:teuthology.orchestra.run.smithi110.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-305-g1cf819b7-1jammy [51.4 MB] 2024-09-10T09:54:43.492 INFO:teuthology.orchestra.run.smithi145.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-305-g1cf819b7-1jammy [719 kB] 2024-09-10T09:54:43.549 INFO:teuthology.orchestra.run.smithi145.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-305-g1cf819b7-1jammy [357 kB] 2024-09-10T09:54:43.602 INFO:teuthology.orchestra.run.smithi145.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-305-g1cf819b7-1jammy [32.8 kB] 2024-09-10T09:54:43.602 INFO:teuthology.orchestra.run.smithi145.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-305-g1cf819b7-1jammy [180 kB] 2024-09-10T09:54:43.651 INFO:teuthology.orchestra.run.smithi145.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-305-g1cf819b7-1jammy [67.6 kB] 2024-09-10T09:54:43.652 INFO:teuthology.orchestra.run.smithi145.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-305-g1cf819b7-1jammy [333 kB] 2024-09-10T09:54:43.660 INFO:teuthology.orchestra.run.smithi145.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-305-g1cf819b7-1jammy [6,875 kB] 2024-09-10T09:54:44.283 INFO:teuthology.orchestra.run.smithi145.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-305-g1cf819b7-1jammy [112 kB] 2024-09-10T09:54:44.284 INFO:teuthology.orchestra.run.smithi145.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-305-g1cf819b7-1jammy [472 kB] 2024-09-10T09:54:44.338 INFO:teuthology.orchestra.run.smithi145.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-305-g1cf819b7-1jammy [25.9 MB] 2024-09-10T09:54:44.389 INFO:teuthology.orchestra.run.smithi110.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-305-g1cf819b7-1jammy [130 kB] 2024-09-10T09:54:44.390 INFO:teuthology.orchestra.run.smithi110.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.1.1-305-g1cf819b7-1jammy [237 kB] 2024-09-10T09:54:44.392 INFO:teuthology.orchestra.run.smithi110.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.1.1-305-g1cf819b7-1jammy [34.0 kB] 2024-09-10T09:54:44.392 INFO:teuthology.orchestra.run.smithi110.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.1.1-305-g1cf819b7-1jammy [23.5 kB] 2024-09-10T09:54:44.393 INFO:teuthology.orchestra.run.smithi110.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-305-g1cf819b7-1jammy [39.0 kB] 2024-09-10T09:54:44.393 INFO:teuthology.orchestra.run.smithi110.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-305-g1cf819b7-1jammy [13.6 MB] 2024-09-10T09:54:44.689 INFO:teuthology.orchestra.run.smithi110.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-305-g1cf819b7-1jammy [91.4 kB] 2024-09-10T09:54:45.029 INFO:teuthology.orchestra.run.smithi145.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-305-g1cf819b7-1jammy [5,145 kB] 2024-09-10T09:54:45.107 INFO:teuthology.orchestra.run.smithi145.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-305-g1cf819b7-1jammy [245 kB] 2024-09-10T09:54:45.109 INFO:teuthology.orchestra.run.smithi145.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-305-g1cf819b7-1jammy [124 kB] 2024-09-10T09:54:45.111 INFO:teuthology.orchestra.run.smithi145.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-305-g1cf819b7-1jammy [1,555 kB] 2024-09-10T09:54:45.137 INFO:teuthology.orchestra.run.smithi145.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-305-g1cf819b7-1jammy [6,142 kB] 2024-09-10T09:54:45.239 INFO:teuthology.orchestra.run.smithi145.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-305-g1cf819b7-1jammy [22.8 MB] 2024-09-10T09:54:45.341 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 181 MB in 3s (56.6 MB/s) 2024-09-10T09:54:45.513 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T09:54:45.580 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T09:54:45.583 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T09:54:45.619 INFO:teuthology.orchestra.run.smithi145.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-305-g1cf819b7-1jammy [14.1 kB] 2024-09-10T09:54:45.619 INFO:teuthology.orchestra.run.smithi145.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-305-g1cf819b7-1jammy [877 kB] 2024-09-10T09:54:45.626 INFO:teuthology.orchestra.run.smithi145.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-305-g1cf819b7-1jammy [2,462 kB] 2024-09-10T09:54:45.645 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T09:54:45.667 INFO:teuthology.orchestra.run.smithi145.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-305-g1cf819b7-1jammy [794 kB] 2024-09-10T09:54:45.679 INFO:teuthology.orchestra.run.smithi145.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-305-g1cf819b7-1jammy [152 kB] 2024-09-10T09:54:45.682 INFO:teuthology.orchestra.run.smithi145.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-305-g1cf819b7-1jammy [2,284 kB] 2024-09-10T09:54:45.720 INFO:teuthology.orchestra.run.smithi145.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-305-g1cf819b7-1jammy [8,625 kB] 2024-09-10T09:54:45.864 INFO:teuthology.orchestra.run.smithi145.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-305-g1cf819b7-1jammy [14.2 kB] 2024-09-10T09:54:45.864 INFO:teuthology.orchestra.run.smithi145.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-305-g1cf819b7-1jammy [51.4 MB] 2024-09-10T09:54:45.972 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T09:54:45.986 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T09:54:46.031 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T09:54:46.334 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T09:54:46.347 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T09:54:46.392 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T09:54:46.719 INFO:teuthology.orchestra.run.smithi145.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-305-g1cf819b7-1jammy [130 kB] 2024-09-10T09:54:46.720 INFO:teuthology.orchestra.run.smithi145.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.1.1-305-g1cf819b7-1jammy [237 kB] 2024-09-10T09:54:46.722 INFO:teuthology.orchestra.run.smithi145.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.1.1-305-g1cf819b7-1jammy [34.0 kB] 2024-09-10T09:54:46.722 INFO:teuthology.orchestra.run.smithi145.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.1.1-305-g1cf819b7-1jammy [23.5 kB] 2024-09-10T09:54:46.723 INFO:teuthology.orchestra.run.smithi145.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-305-g1cf819b7-1jammy [39.0 kB] 2024-09-10T09:54:46.723 INFO:teuthology.orchestra.run.smithi145.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-305-g1cf819b7-1jammy [13.6 MB] 2024-09-10T09:54:46.728 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T09:54:46.742 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T09:54:46.829 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T09:54:46.939 INFO:teuthology.orchestra.run.smithi145.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-305-g1cf819b7-1jammy [91.4 kB] 2024-09-10T09:54:47.190 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T09:54:47.204 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T09:54:47.249 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T09:54:47.569 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T09:54:47.583 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T09:54:47.587 INFO:teuthology.orchestra.run.smithi145.stdout:Fetched 181 MB in 5s (34.2 MB/s) 2024-09-10T09:54:47.628 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T09:54:47.771 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T09:54:47.836 INFO:teuthology.orchestra.run.smithi145.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T09:54:47.839 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T09:54:47.887 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T09:54:48.030 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T09:54:48.045 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T09:54:48.096 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:54:48.147 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T09:54:48.161 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T09:54:48.197 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T09:54:48.534 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T09:54:48.542 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T09:54:48.548 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T09:54:48.556 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T09:54:48.592 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:54:48.601 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T09:54:48.937 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T09:54:48.945 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T09:54:48.951 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T09:54:48.958 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T09:54:48.986 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:54:49.070 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T09:54:49.306 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T09:54:49.321 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T09:54:49.364 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T09:54:49.406 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T09:54:49.420 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T09:54:49.465 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T09:54:49.726 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librados2. 2024-09-10T09:54:49.741 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../010-librados2_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:49.776 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librados2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:49.810 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T09:54:49.823 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T09:54:49.860 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T09:54:50.220 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librbd1. 2024-09-10T09:54:50.235 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../011-librbd1_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:50.271 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T09:54:50.278 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librbd1 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:50.285 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T09:54:50.362 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:54:50.758 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T09:54:50.771 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T09:54:50.816 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:54:50.898 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T09:54:50.913 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../012-libcephfs2_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:50.956 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libcephfs2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:51.169 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T09:54:51.182 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T09:54:51.227 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:54:51.267 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rados. 2024-09-10T09:54:51.282 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../013-python3-rados_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:51.325 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rados (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:51.645 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T09:54:51.659 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:54:51.720 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-ceph-argparse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:51.906 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T09:54:51.920 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T09:54:51.939 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T09:54:51.954 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../015-python3-cephfs_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:51.964 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T09:54:51.997 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cephfs (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:52.267 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package librados2. 2024-09-10T09:54:52.281 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../010-librados2_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:52.317 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking librados2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:52.333 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T09:54:52.348 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../016-python3-ceph-common_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:54:52.392 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-ceph-common (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:52.745 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T09:54:52.760 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T09:54:52.778 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package librbd1. 2024-09-10T09:54:52.792 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../011-librbd1_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:52.803 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T09:54:52.837 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking librbd1 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:53.081 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T09:54:53.096 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../018-python3-rbd_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:53.131 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rbd (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:53.280 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T09:54:53.294 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../012-libcephfs2_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:53.339 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libcephfs2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:53.486 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T09:54:53.501 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T09:54:53.544 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T09:54:53.608 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-rados. 2024-09-10T09:54:53.622 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../013-python3-rados_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:53.666 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-rados (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:53.805 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T09:54:53.820 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T09:54:53.863 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T09:54:53.994 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T09:54:54.008 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:54:54.052 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-ceph-argparse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:54.191 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T09:54:54.206 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T09:54:54.249 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T09:54:54.288 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T09:54:54.302 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../015-python3-cephfs_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:54.346 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cephfs (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:54.536 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package lua5.1. 2024-09-10T09:54:54.549 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T09:54:54.627 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T09:54:54.682 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T09:54:54.696 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../016-python3-ceph-common_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:54:54.741 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-ceph-common (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:54.914 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package lua-any. 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-10T09:54:54.972 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T09:54:55.060 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T09:54:55.074 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T09:54:55.143 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T09:54:55.241 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package zip. 2024-09-10T09:54:55.255 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-10T09:54:55.300 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T09:54:55.371 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T09:54:55.385 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../018-python3-rbd_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:55.421 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-rbd (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:55.586 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package unzip. 2024-09-10T09:54:55.601 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T09:54:55.636 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T09:54:55.782 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T09:54:55.796 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T09:54:55.841 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T09:54:56.048 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package luarocks. 2024-09-10T09:54:56.063 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T09:54:56.106 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T09:54:56.118 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T09:54:56.132 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T09:54:56.176 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T09:54:56.504 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T09:54:56.518 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T09:54:56.531 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librgw2. 2024-09-10T09:54:56.545 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../027-librgw2_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:56.562 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T09:54:56.589 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librgw2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:56.866 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package lua5.1. 2024-09-10T09:54:56.880 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T09:54:56.925 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T09:54:57.092 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T09:54:57.106 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../028-python3-rgw_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:57.150 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rgw (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:57.261 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package lua-any. 2024-09-10T09:54:57.275 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-10T09:54:57.319 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T09:54:57.487 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T09:54:57.501 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T09:54:57.563 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package zip. 2024-09-10T09:54:57.577 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-10T09:54:57.603 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T09:54:57.613 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T09:54:57.898 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T09:54:57.913 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../030-libradosstriper1_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:57.956 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libradosstriper1 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:57.958 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package unzip. 2024-09-10T09:54:57.972 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T09:54:58.017 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T09:54:58.401 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-common. 2024-09-10T09:54:58.403 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package luarocks. 2024-09-10T09:54:58.416 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../031-ceph-common_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:58.418 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T09:54:58.459 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-common (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:58.487 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T09:54:58.870 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package librgw2. 2024-09-10T09:54:58.884 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../027-librgw2_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:58.928 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking librgw2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:59.438 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T09:54:59.453 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../028-python3-rgw_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:59.497 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-rgw (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:59.606 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-base. 2024-09-10T09:54:59.621 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../032-ceph-base_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:54:59.665 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-base (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:54:59.875 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T09:54:59.890 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T09:54:59.933 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T09:55:00.244 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T09:55:00.259 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../030-libradosstriper1_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:00.269 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T09:55:00.283 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T09:55:00.294 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libradosstriper1 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:00.353 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T09:55:00.655 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T09:55:00.668 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T09:55:00.700 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-common. 2024-09-10T09:55:00.710 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../031-ceph-common_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:00.713 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T09:55:00.747 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-common (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:01.267 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T09:55:01.280 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T09:55:01.317 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T09:55:01.653 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T09:55:01.667 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T09:55:01.711 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T09:55:02.035 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T09:55:02.049 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T09:55:02.097 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T09:55:02.111 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-base. 2024-09-10T09:55:02.124 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../032-ceph-base_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:02.178 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-base (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:02.375 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T09:55:02.390 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T09:55:02.559 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T09:55:02.749 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T09:55:02.763 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T09:55:02.815 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T09:55:02.970 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-portend. 2024-09-10T09:55:02.986 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-10T09:55:03.037 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T09:55:03.077 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T09:55:03.089 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T09:55:03.118 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T09:55:03.415 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T09:55:03.430 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T09:55:03.446 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T09:55:03.458 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T09:55:03.473 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T09:55:03.496 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T09:55:03.782 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T09:55:03.795 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T09:55:03.826 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T09:55:03.832 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T09:55:03.841 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T09:55:03.892 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T09:55:04.160 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T09:55:04.172 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T09:55:04.210 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T09:55:04.221 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T09:55:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T09:55:04.271 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T09:55:04.471 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T09:55:04.483 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T09:55:04.513 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T09:55:04.591 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T09:55:04.606 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T09:55:04.649 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T09:55:04.824 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-portend. 2024-09-10T09:55:04.836 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-10T09:55:04.873 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T09:55:04.969 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T09:55:04.984 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T09:55:05.027 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T09:55:05.177 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T09:55:05.190 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T09:55:05.226 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T09:55:05.330 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-mako. 2024-09-10T09:55:05.345 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T09:55:05.380 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T09:55:05.563 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T09:55:05.576 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T09:55:05.621 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T09:55:05.708 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T09:55:05.725 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T09:55:05.758 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T09:55:06.000 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T09:55:06.013 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T09:55:06.050 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T09:55:06.061 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T09:55:06.077 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T09:55:06.136 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T09:55:06.361 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T09:55:06.374 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T09:55:06.411 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T09:55:06.455 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-webob. 2024-09-10T09:55:06.470 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T09:55:06.514 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T09:55:06.739 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T09:55:06.752 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T09:55:06.789 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T09:55:06.883 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T09:55:06.899 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T09:55:06.944 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T09:55:07.075 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-mako. 2024-09-10T09:55:07.088 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T09:55:07.133 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T09:55:07.278 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T09:55:07.293 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T09:55:07.328 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T09:55:07.495 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T09:55:07.508 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T09:55:07.553 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T09:55:07.606 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-paste. 2024-09-10T09:55:07.621 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T09:55:07.664 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T09:55:07.831 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T09:55:07.844 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T09:55:07.889 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T09:55:08.042 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T09:55:08.058 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T09:55:08.092 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T09:55:08.217 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-webob. 2024-09-10T09:55:08.230 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T09:55:08.275 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T09:55:08.396 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T09:55:08.411 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T09:55:08.453 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T09:55:08.620 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T09:55:08.633 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T09:55:08.681 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T09:55:08.757 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T09:55:08.772 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T09:55:08.815 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T09:55:09.014 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T09:55:09.030 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T09:55:09.072 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T09:55:09.159 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T09:55:09.174 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T09:55:09.217 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T09:55:09.409 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-paste. 2024-09-10T09:55:09.423 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T09:55:09.475 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T09:55:09.606 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T09:55:09.621 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T09:55:09.664 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T09:55:09.787 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T09:55:09.801 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T09:55:09.845 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T09:55:09.942 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-idna. 2024-09-10T09:55:09.957 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T09:55:10.000 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T09:55:10.140 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T09:55:10.154 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T09:55:10.190 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T09:55:10.336 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T09:55:10.351 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T09:55:10.403 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T09:55:10.493 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T09:55:10.507 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T09:55:10.551 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T09:55:10.748 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-requests. 2024-09-10T09:55:10.763 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T09:55:10.806 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T09:55:10.879 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T09:55:10.888 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T09:55:10.929 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T09:55:11.117 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T09:55:11.132 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T09:55:11.176 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T09:55:11.292 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T09:55:11.304 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T09:55:11.342 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T09:55:11.512 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T09:55:11.527 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:11.570 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-modules-core (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:11.703 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-idna. 2024-09-10T09:55:11.713 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T09:55:11.753 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T09:55:11.958 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T09:55:11.972 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:12.017 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:12.239 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T09:55:12.249 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T09:55:12.280 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T09:55:12.311 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T09:55:12.326 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T09:55:12.369 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T09:55:12.601 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-requests. 2024-09-10T09:55:12.610 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T09:55:12.650 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T09:55:12.987 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T09:55:12.997 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T09:55:13.036 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T09:55:13.323 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T09:55:13.332 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:13.373 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-modules-core (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:13.448 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T09:55:13.463 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T09:55:13.497 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T09:55:13.769 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T09:55:13.779 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:13.819 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:13.842 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T09:55:13.857 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../065-ceph-mgr_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:13.900 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:14.147 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T09:55:14.157 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T09:55:14.197 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T09:55:14.287 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T09:55:14.302 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../066-ceph-mon_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:14.345 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mon (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:14.500 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T09:55:14.512 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T09:55:14.725 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T09:55:14.932 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T09:55:14.943 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../067-ceph-osd_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:15.007 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-osd (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:15.653 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T09:55:15.664 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../065-ceph-mgr_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:15.702 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:15.752 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph. 2024-09-10T09:55:15.762 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../068-ceph_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:15.802 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:16.106 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T09:55:16.116 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../066-ceph-mon_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:16.146 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T09:55:16.155 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mon (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:16.157 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../069-ceph-fuse_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:16.196 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-fuse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:16.524 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T09:55:16.534 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../070-ceph-mds_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:16.574 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mds (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:16.768 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T09:55:16.778 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../067-ceph-osd_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:16.817 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-osd (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:16.960 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package cephadm. 2024-09-10T09:55:16.970 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../071-cephadm_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:17.001 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking cephadm (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:17.346 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T09:55:17.357 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T09:55:17.396 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T09:55:17.537 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph. 2024-09-10T09:55:17.547 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../068-ceph_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:17.587 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:17.674 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T09:55:17.684 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T09:55:17.724 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T09:55:17.956 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T09:55:17.966 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../069-ceph-fuse_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:18.006 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-fuse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:18.368 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T09:55:18.378 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../070-ceph-mds_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:18.417 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mds (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:18.754 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-babel. 2024-09-10T09:55:18.764 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T09:55:18.779 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package cephadm. 2024-09-10T09:55:18.789 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../071-cephadm_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:18.795 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T09:55:18.820 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking cephadm (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:19.115 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T09:55:19.123 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T09:55:19.126 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T09:55:19.136 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T09:55:19.164 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T09:55:19.173 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T09:55:19.493 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T09:55:19.504 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T09:55:19.526 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T09:55:19.534 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T09:55:19.537 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:19.576 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-cephadm (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:19.955 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T09:55:19.966 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T09:55:20.004 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T09:55:20.332 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-routes. 2024-09-10T09:55:20.344 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T09:55:20.382 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T09:55:20.480 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-babel. 2024-09-10T09:55:20.490 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T09:55:20.530 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T09:55:20.719 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T09:55:20.731 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:20.769 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-dashboard (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:20.875 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T09:55:20.884 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T09:55:20.925 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T09:55:21.269 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T09:55:21.279 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:21.336 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-cephadm (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:21.673 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T09:55:21.683 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T09:55:21.722 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T09:55:22.044 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-routes. 2024-09-10T09:55:22.055 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T09:55:22.085 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T09:55:22.316 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T09:55:22.330 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T09:55:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T09:55:22.555 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T09:55:22.566 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:22.605 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-dashboard (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:22.886 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T09:55:22.898 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T09:55:22.935 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T09:55:23.314 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T09:55:23.326 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T09:55:23.466 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T09:55:23.786 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T09:55:23.801 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T09:55:23.844 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T09:55:23.869 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T09:55:23.878 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T09:55:23.918 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T09:55:24.414 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T09:55:24.423 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T09:55:24.463 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T09:55:24.470 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T09:55:24.482 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:24.519 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:24.775 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T09:55:24.785 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T09:55:24.902 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T09:55:25.213 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T09:55:25.226 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T09:55:25.263 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T09:55:25.498 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T09:55:25.508 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T09:55:25.539 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T09:55:25.876 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T09:55:25.886 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T09:55:25.925 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T09:55:25.989 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T09:55:26.000 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:26.039 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:26.254 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T09:55:26.264 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T09:55:26.312 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T09:55:26.648 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T09:55:26.661 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-10T09:55:26.706 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T09:55:27.052 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T09:55:27.058 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T09:55:27.063 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T09:55:27.071 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T09:55:27.102 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T09:55:27.108 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T09:55:27.437 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T09:55:27.438 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T09:55:27.449 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T09:55:27.449 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T09:55:27.486 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T09:55:27.488 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T09:55:27.775 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T09:55:27.786 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T09:55:27.815 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T09:55:27.824 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T09:55:27.827 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T09:55:27.856 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T09:55:28.168 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T09:55:28.178 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T09:55:28.179 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-10T09:55:28.189 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T09:55:28.217 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T09:55:28.227 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T09:55:28.555 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T09:55:28.567 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T09:55:28.604 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T09:55:28.924 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T09:55:28.925 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T09:55:28.936 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T09:55:28.938 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:28.966 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:28.991 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T09:55:29.303 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T09:55:29.315 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T09:55:29.336 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T09:55:29.347 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T09:55:29.352 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T09:55:29.386 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T09:55:29.689 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T09:55:29.701 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T09:55:29.731 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T09:55:29.739 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T09:55:29.743 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T09:55:29.781 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T09:55:30.050 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package jq. 2024-09-10T09:55:30.061 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T09:55:30.100 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T09:55:30.353 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package socat. 2024-09-10T09:55:30.363 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T09:55:30.402 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T09:55:30.403 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T09:55:30.414 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:30.453 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:30.739 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T09:55:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T09:55:30.749 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T09:55:30.753 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T09:55:30.788 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T09:55:30.797 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T09:55:31.133 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T09:55:31.144 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T09:55:31.168 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-test. 2024-09-10T09:55:31.180 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../099-ceph-test_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:31.183 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T09:55:31.218 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-test (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:31.427 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package jq. 2024-09-10T09:55:31.439 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T09:55:31.477 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T09:55:31.755 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package socat. 2024-09-10T09:55:31.766 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T09:55:31.805 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T09:55:32.108 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T09:55:32.120 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T09:55:32.158 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T09:55:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-test. 2024-09-10T09:55:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../099-ceph-test_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-test (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:33.174 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T09:55:33.187 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../100-ceph-volume_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:33.224 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-volume (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:33.647 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T09:55:33.659 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../101-cephfs-mirror_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:33.703 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking cephfs-mirror (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:33.981 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T09:55:33.993 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T09:55:34.022 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T09:55:34.317 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T09:55:34.329 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../103-cephfs-shell_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:34.366 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking cephfs-shell (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:34.627 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T09:55:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../104-cephfs-top_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:34.649 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T09:55:34.662 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../100-ceph-volume_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:34.699 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-volume (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:34.710 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking cephfs-top (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:34.963 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T09:55:34.976 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../105-libcephfs-dev_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:35.021 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libcephfs-dev (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:35.104 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T09:55:35.116 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../101-cephfs-mirror_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:35.153 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking cephfs-mirror (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:35.366 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T09:55:35.379 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T09:55:35.407 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T09:55:35.419 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T09:55:35.423 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T09:55:35.473 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T09:55:35.701 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T09:55:35.713 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../103-cephfs-shell_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:35.753 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T09:55:35.765 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T09:55:35.775 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking cephfs-shell (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:35.838 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T09:55:36.029 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T09:55:36.041 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../104-cephfs-top_19.1.1-305-g1cf819b7-1jammy_all.deb ... 2024-09-10T09:55:36.078 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking cephfs-top (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:36.198 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T09:55:36.211 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T09:55:36.256 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T09:55:36.331 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T09:55:36.343 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../105-libcephfs-dev_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:36.380 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libcephfs-dev (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:36.661 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package pkg-config. 2024-09-10T09:55:36.673 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T09:55:36.703 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T09:55:36.717 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T09:55:36.729 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T09:55:36.767 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T09:55:36.979 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T09:55:36.991 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T09:55:37.039 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T09:55:37.113 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T09:55:37.125 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T09:55:37.162 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T09:55:37.442 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T09:55:37.453 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T09:55:37.534 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T09:55:37.546 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T09:55:37.657 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T09:55:37.800 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T09:55:38.102 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T09:55:38.114 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T09:55:38.152 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T09:55:38.239 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package pkg-config. 2024-09-10T09:55:38.251 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T09:55:38.288 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T09:55:38.506 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T09:55:38.519 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T09:55:38.556 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T09:55:38.558 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T09:55:38.570 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T09:55:38.608 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T09:55:38.850 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T09:55:38.863 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T09:55:38.900 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T09:55:39.053 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T09:55:39.064 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T09:55:39.128 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T09:55:39.236 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-py. 2024-09-10T09:55:39.250 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-10T09:55:39.311 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T09:55:39.406 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T09:55:39.418 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T09:55:39.447 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T09:55:39.673 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T09:55:39.686 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T09:55:39.722 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T09:55:39.777 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T09:55:39.789 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T09:55:39.833 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T09:55:40.137 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T09:55:40.149 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T09:55:40.185 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T09:55:40.186 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T09:55:40.197 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T09:55:40.234 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T09:55:40.529 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-toml. 2024-09-10T09:55:40.539 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-py. 2024-09-10T09:55:40.542 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-10T09:55:40.551 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-10T09:55:40.589 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T09:55:40.595 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T09:55:40.890 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T09:55:40.901 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T09:55:40.903 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T09:55:40.916 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T09:55:40.939 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T09:55:40.958 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T09:55:41.318 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T09:55:41.330 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T09:55:41.368 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T09:55:41.429 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T09:55:41.441 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T09:55:41.479 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T09:55:41.604 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T09:55:41.616 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T09:55:41.645 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T09:55:41.816 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-toml. 2024-09-10T09:55:41.827 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-10T09:55:41.865 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T09:55:42.126 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T09:55:42.138 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T09:55:42.176 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T09:55:42.408 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package radosgw. 2024-09-10T09:55:42.420 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../122-radosgw_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:42.457 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking radosgw (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:42.547 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T09:55:42.559 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T09:55:42.596 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T09:55:42.874 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T09:55:42.886 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T09:55:42.924 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T09:55:43.069 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T09:55:43.081 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../123-rbd-fuse_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:43.119 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking rbd-fuse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:43.395 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T09:55:43.698 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T09:55:43.825 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T09:55:43.843 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package radosgw. 2024-09-10T09:55:43.855 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../122-radosgw_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:43.902 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking radosgw (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:44.079 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T09:55:44.486 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T09:55:44.547 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T09:55:44.560 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../123-rbd-fuse_19.1.1-305-g1cf819b7-1jammy_amd64.deb ... 2024-09-10T09:55:44.597 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking rbd-fuse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:44.890 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T09:55:45.096 INFO:teuthology.orchestra.run.smithi110.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T09:55:45.110 INFO:teuthology.orchestra.run.smithi110.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T09:55:45.161 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T09:55:45.201 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up cephadm (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:45.320 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T09:55:45.581 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T09:55:45.845 INFO:teuthology.orchestra.run.smithi110.stdout:Adding system user cephadm....done 2024-09-10T09:55:45.996 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T09:55:46.008 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T09:55:46.639 INFO:teuthology.orchestra.run.smithi145.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T09:55:46.688 INFO:teuthology.orchestra.run.smithi145.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T09:55:46.710 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T09:55:46.823 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up cephadm (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:46.986 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T09:55:47.137 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T09:55:47.308 INFO:teuthology.orchestra.run.smithi145.stdout:Adding system user cephadm....done 2024-09-10T09:55:47.397 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T09:55:47.487 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T09:55:47.700 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T09:55:47.772 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T09:55:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T09:55:48.048 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T09:55:48.117 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T09:55:48.174 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T09:55:48.434 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T09:55:48.485 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T09:55:48.710 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T09:55:48.761 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T09:55:48.811 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T09:55:49.063 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T09:55:49.121 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T09:55:49.347 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T09:55:49.489 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T09:55:49.632 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T09:55:49.757 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T09:55:49.875 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T09:55:50.050 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T09:55:50.168 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T09:55:50.326 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T09:55:50.443 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-ceph-argparse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:50.587 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T09:55:50.728 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T09:55:50.854 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T09:55:50.937 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T09:55:50.980 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T09:55:51.205 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T09:55:51.240 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T09:55:51.340 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T09:55:51.473 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-ceph-argparse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:51.592 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T09:55:51.749 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T09:55:51.875 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T09:55:52.001 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T09:55:52.008 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T09:55:52.039 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T09:55:52.102 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T09:55:52.229 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T09:55:52.285 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T09:55:52.411 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T09:55:52.630 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T09:55:52.679 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T09:55:52.730 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T09:55:52.999 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T09:55:53.130 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T09:55:53.159 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T09:55:53.224 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T09:55:53.250 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T09:55:53.350 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T09:55:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T09:55:53.702 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T09:55:53.743 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T09:55:53.869 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T09:55:54.153 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T09:55:54.387 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T09:55:54.421 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T09:55:54.739 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T09:55:54.881 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T09:55:54.938 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T09:55:55.064 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T09:55:55.216 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T09:55:55.517 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T09:55:55.567 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T09:55:55.835 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T09:55:55.909 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T09:55:55.969 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T09:55:56.095 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T09:55:56.177 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T09:55:56.437 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T09:55:56.562 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T09:55:56.855 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T09:55:56.897 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T09:55:57.115 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T09:55:57.206 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T09:55:57.525 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T09:55:57.541 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T09:55:57.800 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T09:55:57.834 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T09:55:57.929 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-ceph-common (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:58.177 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T09:55:58.252 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T09:55:58.621 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T09:55:58.628 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T09:55:58.888 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T09:55:59.014 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-ceph-common (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:55:59.307 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T09:55:59.658 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T09:55:59.851 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T09:56:00.095 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:56:00.229 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T09:56:00.505 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T09:56:00.630 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T09:56:00.765 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T09:56:00.872 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T09:56:01.066 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T09:56:01.099 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:56:01.233 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T09:56:01.334 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T09:56:01.534 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T09:56:01.602 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:56:01.652 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T09:56:01.728 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T09:56:01.786 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T09:56:02.037 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T09:56:02.088 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T09:56:02.330 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T09:56:02.372 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T09:56:02.589 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T09:56:02.631 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:56:02.757 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T09:56:02.866 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T09:56:03.092 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T09:56:03.134 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T09:56:03.352 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T09:56:03.410 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T09:56:03.535 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T09:56:03.620 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T09:56:03.845 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T09:56:03.887 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T09:56:03.971 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T09:56:04.147 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T09:56:04.272 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T09:56:04.448 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T09:56:04.549 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T09:56:04.573 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T09:56:04.833 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T09:56:04.841 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T09:56:04.984 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T09:56:05.000 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T09:56:05.126 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T09:56:05.215 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T09:56:05.285 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T09:56:05.453 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T09:56:05.595 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T09:56:05.596 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T09:56:05.871 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T09:56:05.989 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T09:56:05.997 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T09:56:06.116 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T09:56:06.186 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T09:56:06.265 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T09:56:06.391 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T09:56:06.458 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T09:56:06.584 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T09:56:06.659 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:56:06.826 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T09:56:06.994 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T09:56:07.152 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T09:56:07.261 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T09:56:07.387 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T09:56:07.462 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T09:56:07.655 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T09:56:07.746 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T09:56:07.781 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T09:56:07.974 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T09:56:08.107 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T09:56:08.259 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T09:56:08.417 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T09:56:08.535 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T09:56:08.704 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T09:56:08.803 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T09:56:08.940 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T09:56:09.188 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T09:56:09.190 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T09:56:09.467 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T09:56:09.472 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T09:56:09.752 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T09:56:09.800 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T09:56:09.926 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librados2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:10.052 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libsqlite3-mod-ceph (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:10.137 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T09:56:10.195 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T09:56:10.472 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T09:56:10.497 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libcephfs2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:10.598 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libradosstriper1 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:10.740 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T09:56:10.816 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T09:56:10.992 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up librados2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:11.026 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librbd1 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libsqlite3-mod-ceph (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:11.152 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-modules-core (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:11.244 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T09:56:11.278 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-fuse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:11.511 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T09:56:11.512 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T09:56:11.597 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libcephfs2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:11.706 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libradosstriper1 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:11.824 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T09:56:12.104 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libcephfs-dev (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:12.117 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up librbd1 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:12.221 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rados (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:12.268 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-modules-core (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:12.331 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librgw2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:12.394 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-fuse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:12.440 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up cephfs-top (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:12.591 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T09:56:12.603 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T09:56:12.603 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T09:56:13.246 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libcephfs-dev (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:13.372 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-rados (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:13.489 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up librgw2 (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:13.615 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up cephfs-top (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:13.741 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T09:56:13.870 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rbd (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:13.996 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up rbd-fuse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:14.097 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rgw (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:14.231 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cephfs (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:14.332 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up cephfs-shell (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:14.458 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-common (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:15.012 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-rbd (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:15.019 INFO:teuthology.orchestra.run.smithi110.stdout:Adding group ceph....done 2024-09-10T09:56:15.138 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up rbd-fuse (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:15.264 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-rgw (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:15.390 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cephfs (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:15.516 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up cephfs-shell (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:15.675 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-common (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:15.688 INFO:teuthology.orchestra.run.smithi110.stdout:Adding system user ceph....done 2024-09-10T09:56:15.857 INFO:teuthology.orchestra.run.smithi110.stdout:Setting system user ceph properties....done 2024-09-10T09:56:15.875 INFO:teuthology.orchestra.run.smithi110.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T09:56:16.015 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T09:56:16.212 INFO:teuthology.orchestra.run.smithi145.stdout:Adding group ceph....done 2024-09-10T09:56:16.411 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T09:56:16.613 INFO:teuthology.orchestra.run.smithi145.stdout:Adding system user ceph....done 2024-09-10T09:56:16.791 INFO:teuthology.orchestra.run.smithi145.stdout:Setting system user ceph properties....done 2024-09-10T09:56:16.806 INFO:teuthology.orchestra.run.smithi145.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T09:56:16.948 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T09:56:17.111 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up cephfs-mirror (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:17.357 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T09:56:17.368 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:56:17.368 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:56:17.995 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-test (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:18.069 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up cephfs-mirror (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:18.146 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up radosgw (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:18.277 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:56:18.277 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:56:18.735 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:56:18.735 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:56:18.912 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-test (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:19.063 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up radosgw (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:19.348 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-base (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:19.609 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:56:19.609 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:56:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T09:56:20.248 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-base (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:20.417 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mds (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:20.632 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T09:56:20.969 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T09:56:20.970 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T09:56:21.300 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mds (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:21.531 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T09:56:21.532 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T09:56:21.597 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:21.827 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T09:56:21.828 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T09:56:22.160 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:22.383 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T09:56:22.383 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T09:56:22.432 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-osd (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:22.779 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T09:56:22.779 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T09:56:23.037 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-osd (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:23.400 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-k8sevents (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:23.413 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T09:56:23.413 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T09:56:23.560 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:23.736 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mon (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:23.944 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T09:56:23.945 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T09:56:24.063 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-k8sevents (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:24.189 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:24.332 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mon (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:24.562 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-cephadm (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:24.566 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T09:56:24.566 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T09:56:24.730 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:24.897 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-dashboard (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:25.065 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-volume (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:25.167 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-cephadm (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:25.292 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:25.419 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-dashboard (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:25.443 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T09:56:25.560 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T09:56:25.578 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-volume (19.1.1-305-g1cf819b7-1jammy) ... 2024-09-10T09:56:25.862 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T09:56:25.939 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T09:56:26.040 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T09:56:26.192 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T09:56:26.999 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T09:56:27.377 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T09:56:27.743 INFO:teuthology.orchestra.run.smithi110.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T09:56:27.748 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T09:56:27.824 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-10T09:56:28.008 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-10T09:56:28.008 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-10T09:56:28.111 INFO:teuthology.orchestra.run.smithi145.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T09:56:28.115 DEBUG:teuthology.orchestra.run.smithi145:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T09:56:28.191 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-10T09:56:28.194 INFO:teuthology.orchestra.run.smithi110.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T09:56:28.195 INFO:teuthology.orchestra.run.smithi110.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T09:56:28.195 INFO:teuthology.orchestra.run.smithi110.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T09:56:28.195 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T09:56:28.195 INFO:teuthology.orchestra.run.smithi110.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T09:56:28.195 INFO:teuthology.orchestra.run.smithi110.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T09:56:28.195 INFO:teuthology.orchestra.run.smithi110.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T09:56:28.196 INFO:teuthology.orchestra.run.smithi110.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T09:56:28.196 INFO:teuthology.orchestra.run.smithi110.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T09:56:28.196 INFO:teuthology.orchestra.run.smithi110.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T09:56:28.197 INFO:teuthology.orchestra.run.smithi110.stdout:The following additional packages will be installed: 2024-09-10T09:56:28.197 INFO:teuthology.orchestra.run.smithi110.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T09:56:28.197 INFO:teuthology.orchestra.run.smithi110.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T09:56:28.197 INFO:teuthology.orchestra.run.smithi110.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T09:56:28.198 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T09:56:28.198 INFO:teuthology.orchestra.run.smithi110.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T09:56:28.199 INFO:teuthology.orchestra.run.smithi110.stdout:Suggested packages: 2024-09-10T09:56:28.199 INFO:teuthology.orchestra.run.smithi110.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T09:56:28.199 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T09:56:28.199 INFO:teuthology.orchestra.run.smithi110.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T09:56:28.255 INFO:teuthology.orchestra.run.smithi110.stdout:The following NEW packages will be installed: 2024-09-10T09:56:28.256 INFO:teuthology.orchestra.run.smithi110.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T09:56:28.256 INFO:teuthology.orchestra.run.smithi110.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T09:56:28.256 INFO:teuthology.orchestra.run.smithi110.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T09:56:28.256 INFO:teuthology.orchestra.run.smithi110.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T09:56:28.256 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T09:56:28.257 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T09:56:28.257 INFO:teuthology.orchestra.run.smithi110.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T09:56:28.258 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages will be upgraded: 2024-09-10T09:56:28.259 INFO:teuthology.orchestra.run.smithi110.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T09:56:28.316 INFO:teuthology.orchestra.run.smithi110.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T09:56:28.316 INFO:teuthology.orchestra.run.smithi110.stdout:Need to get 55.8 MB of archives. 2024-09-10T09:56:28.316 INFO:teuthology.orchestra.run.smithi110.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T09:56:28.316 INFO:teuthology.orchestra.run.smithi110.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T09:56:28.375 INFO:teuthology.orchestra.run.smithi145.stdout:Building dependency tree... 2024-09-10T09:56:28.376 INFO:teuthology.orchestra.run.smithi145.stdout:Reading state information... 2024-09-10T09:56:28.448 INFO:teuthology.orchestra.run.smithi110.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T09:56:28.452 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T09:56:28.479 INFO:teuthology.orchestra.run.smithi110.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T09:56:28.480 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T09:56:28.480 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T09:56:28.487 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T09:56:28.488 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T09:56:28.489 INFO:teuthology.orchestra.run.smithi110.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T09:56:28.490 INFO:teuthology.orchestra.run.smithi110.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T09:56:28.491 INFO:teuthology.orchestra.run.smithi110.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T09:56:28.492 INFO:teuthology.orchestra.run.smithi110.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T09:56:28.500 INFO:teuthology.orchestra.run.smithi110.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T09:56:28.500 INFO:teuthology.orchestra.run.smithi110.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T09:56:28.501 INFO:teuthology.orchestra.run.smithi110.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T09:56:28.517 INFO:teuthology.orchestra.run.smithi110.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T09:56:28.565 INFO:teuthology.orchestra.run.smithi145.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T09:56:28.565 INFO:teuthology.orchestra.run.smithi145.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T09:56:28.565 INFO:teuthology.orchestra.run.smithi145.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T09:56:28.565 INFO:teuthology.orchestra.run.smithi145.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T09:56:28.565 INFO:teuthology.orchestra.run.smithi145.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T09:56:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T09:56:28.566 INFO:teuthology.orchestra.run.smithi145.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T09:56:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T09:56:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T09:56:28.567 INFO:teuthology.orchestra.run.smithi145.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T09:56:28.567 INFO:teuthology.orchestra.run.smithi145.stdout:The following additional packages will be installed: 2024-09-10T09:56:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T09:56:28.567 INFO:teuthology.orchestra.run.smithi145.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T09:56:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T09:56:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T09:56:28.568 INFO:teuthology.orchestra.run.smithi145.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T09:56:28.569 INFO:teuthology.orchestra.run.smithi145.stdout:Suggested packages: 2024-09-10T09:56:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T09:56:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T09:56:28.569 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T09:56:28.625 INFO:teuthology.orchestra.run.smithi145.stdout:The following NEW packages will be installed: 2024-09-10T09:56:28.626 INFO:teuthology.orchestra.run.smithi145.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T09:56:28.626 INFO:teuthology.orchestra.run.smithi145.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T09:56:28.626 INFO:teuthology.orchestra.run.smithi145.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T09:56:28.627 INFO:teuthology.orchestra.run.smithi145.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T09:56:28.627 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T09:56:28.627 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T09:56:28.627 INFO:teuthology.orchestra.run.smithi145.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T09:56:28.628 INFO:teuthology.orchestra.run.smithi145.stdout:The following packages will be upgraded: 2024-09-10T09:56:28.629 INFO:teuthology.orchestra.run.smithi145.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T09:56:28.726 INFO:teuthology.orchestra.run.smithi110.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T09:56:28.728 INFO:teuthology.orchestra.run.smithi110.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T09:56:28.728 INFO:teuthology.orchestra.run.smithi110.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T09:56:28.731 INFO:teuthology.orchestra.run.smithi110.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T09:56:28.731 INFO:teuthology.orchestra.run.smithi110.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T09:56:28.736 INFO:teuthology.orchestra.run.smithi110.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T09:56:28.738 INFO:teuthology.orchestra.run.smithi110.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T09:56:28.745 INFO:teuthology.orchestra.run.smithi110.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T09:56:28.745 INFO:teuthology.orchestra.run.smithi110.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T09:56:28.766 INFO:teuthology.orchestra.run.smithi110.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T09:56:28.766 INFO:teuthology.orchestra.run.smithi110.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T09:56:28.776 INFO:teuthology.orchestra.run.smithi110.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T09:56:28.796 INFO:teuthology.orchestra.run.smithi110.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T09:56:28.799 INFO:teuthology.orchestra.run.smithi110.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T09:56:28.800 INFO:teuthology.orchestra.run.smithi110.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T09:56:28.801 INFO:teuthology.orchestra.run.smithi110.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T09:56:28.801 INFO:teuthology.orchestra.run.smithi110.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T09:56:28.812 INFO:teuthology.orchestra.run.smithi110.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T09:56:28.814 INFO:teuthology.orchestra.run.smithi110.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T09:56:28.814 INFO:teuthology.orchestra.run.smithi145.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T09:56:28.814 INFO:teuthology.orchestra.run.smithi145.stdout:Need to get 55.8 MB of archives. 2024-09-10T09:56:28.814 INFO:teuthology.orchestra.run.smithi145.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T09:56:28.814 INFO:teuthology.orchestra.run.smithi145.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T09:56:28.961 INFO:teuthology.orchestra.run.smithi110.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T09:56:28.961 INFO:teuthology.orchestra.run.smithi110.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T09:56:28.961 INFO:teuthology.orchestra.run.smithi110.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T09:56:28.961 INFO:teuthology.orchestra.run.smithi110.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T09:56:28.962 INFO:teuthology.orchestra.run.smithi110.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T09:56:28.962 INFO:teuthology.orchestra.run.smithi110.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T09:56:28.975 INFO:teuthology.orchestra.run.smithi110.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T09:56:28.975 INFO:teuthology.orchestra.run.smithi110.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T09:56:29.360 INFO:teuthology.orchestra.run.smithi110.stdout:Preconfiguring packages ... 2024-09-10T09:56:29.429 INFO:teuthology.orchestra.run.smithi145.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T09:56:29.445 INFO:teuthology.orchestra.run.smithi145.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T09:56:29.540 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 55.8 MB in 1s (78.8 MB/s) 2024-09-10T09:56:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T09:56:29.569 INFO:teuthology.orchestra.run.smithi145.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T09:56:29.571 INFO:teuthology.orchestra.run.smithi145.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T09:56:29.599 INFO:teuthology.orchestra.run.smithi145.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T09:56:29.604 INFO:teuthology.orchestra.run.smithi145.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T09:56:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T09:56:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T09:56:29.609 INFO:teuthology.orchestra.run.smithi145.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T09:56:29.610 INFO:teuthology.orchestra.run.smithi145.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T09:56:29.672 INFO:teuthology.orchestra.run.smithi145.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T09:56:29.673 INFO:teuthology.orchestra.run.smithi145.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T09:56:29.673 INFO:teuthology.orchestra.run.smithi145.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T09:56:29.754 INFO:teuthology.orchestra.run.smithi145.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T09:56:29.992 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140925 files and directories currently installed.) 2024-09-10T09:56:29.996 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T09:56:30.139 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T09:56:30.432 INFO:teuthology.orchestra.run.smithi145.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T09:56:30.434 INFO:teuthology.orchestra.run.smithi145.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T09:56:30.434 INFO:teuthology.orchestra.run.smithi145.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T09:56:30.437 INFO:teuthology.orchestra.run.smithi145.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T09:56:30.437 INFO:teuthology.orchestra.run.smithi145.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T09:56:30.442 INFO:teuthology.orchestra.run.smithi145.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T09:56:30.444 INFO:teuthology.orchestra.run.smithi145.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T09:56:30.494 INFO:teuthology.orchestra.run.smithi145.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T09:56:30.494 INFO:teuthology.orchestra.run.smithi145.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T09:56:30.528 INFO:teuthology.orchestra.run.smithi145.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T09:56:30.577 INFO:teuthology.orchestra.run.smithi145.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T09:56:30.588 INFO:teuthology.orchestra.run.smithi145.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T09:56:30.664 INFO:teuthology.orchestra.run.smithi145.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T09:56:30.668 INFO:teuthology.orchestra.run.smithi145.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T09:56:30.669 INFO:teuthology.orchestra.run.smithi145.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T09:56:30.669 INFO:teuthology.orchestra.run.smithi145.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T09:56:30.672 INFO:teuthology.orchestra.run.smithi145.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T09:56:30.688 INFO:teuthology.orchestra.run.smithi145.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T09:56:30.690 INFO:teuthology.orchestra.run.smithi145.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T09:56:30.756 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T09:56:31.170 INFO:teuthology.orchestra.run.smithi145.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T09:56:31.170 INFO:teuthology.orchestra.run.smithi145.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T09:56:31.170 INFO:teuthology.orchestra.run.smithi145.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T09:56:31.170 INFO:teuthology.orchestra.run.smithi145.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T09:56:31.171 INFO:teuthology.orchestra.run.smithi145.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T09:56:31.171 INFO:teuthology.orchestra.run.smithi145.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T09:56:31.233 INFO:teuthology.orchestra.run.smithi145.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T09:56:31.234 INFO:teuthology.orchestra.run.smithi145.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T09:56:31.618 INFO:teuthology.orchestra.run.smithi145.stdout:Preconfiguring packages ... 2024-09-10T09:56:31.795 INFO:teuthology.orchestra.run.smithi145.stdout:Fetched 55.8 MB in 3s (21.5 MB/s) 2024-09-10T09:56:31.883 INFO:teuthology.orchestra.run.smithi110.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T09:56:32.061 INFO:teuthology.orchestra.run.smithi145.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140925 files and directories currently installed.) 2024-09-10T09:56:32.065 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T09:56:32.145 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package flex. 2024-09-10T09:56:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T09:56:32.212 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140925 files and directories currently installed.) 2024-09-10T09:56:32.216 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T09:56:32.269 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T09:56:32.780 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T09:56:32.860 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T09:56:32.958 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T09:56:33.331 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T09:56:33.869 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140998 files and directories currently installed.) 2024-09-10T09:56:33.872 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T09:56:33.968 INFO:teuthology.orchestra.run.smithi145.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T09:56:33.991 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T09:56:34.350 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package flex. 2024-09-10T09:56:34.420 INFO:teuthology.orchestra.run.smithi145.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140925 files and directories currently installed.) 2024-09-10T09:56:34.424 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T09:56:34.429 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T09:56:34.473 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T09:56:34.565 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T09:56:34.904 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T09:56:35.018 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T09:56:35.196 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T09:56:35.307 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package bison. 2024-09-10T09:56:35.375 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140998 files and directories currently installed.) 2024-09-10T09:56:35.379 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T09:56:35.435 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T09:56:35.578 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T09:56:35.945 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T09:56:35.959 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T09:56:36.011 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T09:56:36.117 INFO:teuthology.orchestra.run.smithi145.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140998 files and directories currently installed.) 2024-09-10T09:56:36.121 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T09:56:36.246 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T09:56:36.431 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package dump. 2024-09-10T09:56:36.445 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T09:56:36.489 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T09:56:36.675 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T09:56:36.802 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package indent. 2024-09-10T09:56:36.802 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T09:56:36.816 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:36.868 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T09:56:37.178 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T09:56:37.213 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T09:56:37.227 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T09:56:37.280 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T09:56:37.537 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package bison. 2024-09-10T09:56:37.610 INFO:teuthology.orchestra.run.smithi145.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140998 files and directories currently installed.) 2024-09-10T09:56:37.615 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T09:56:37.617 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T09:56:37.632 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T09:56:37.655 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T09:56:37.667 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T09:56:37.920 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T09:56:37.935 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T09:56:37.978 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T09:56:38.040 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T09:56:38.054 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T09:56:38.098 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T09:56:38.332 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T09:56:38.346 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T09:56:38.390 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T09:56:38.434 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package dump. 2024-09-10T09:56:38.450 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T09:56:38.501 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T09:56:38.618 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T09:56:38.633 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T09:56:38.668 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T09:56:38.830 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package indent. 2024-09-10T09:56:38.845 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:38.888 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T09:56:38.946 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T09:56:38.961 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T09:56:39.004 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T09:56:39.150 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T09:56:39.165 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T09:56:39.199 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T09:56:39.349 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T09:56:39.365 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T09:56:39.407 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T09:56:39.580 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T09:56:39.595 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T09:56:39.638 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T09:56:39.941 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T09:56:39.956 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T09:56:39.999 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T09:56:40.336 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T09:56:40.351 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T09:56:40.394 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T09:56:40.672 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T09:56:40.688 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T09:56:40.721 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T09:56:40.777 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T09:56:40.793 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T09:56:40.827 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T09:56:40.958 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T09:56:40.974 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T09:56:41.016 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T09:56:41.097 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T09:56:41.113 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T09:56:41.155 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T09:56:41.336 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T09:56:41.351 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T09:56:41.394 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T09:56:41.475 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T09:56:41.490 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T09:56:41.533 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T09:56:41.895 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T09:56:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T09:56:41.944 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T09:56:42.347 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T09:56:42.363 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T09:56:42.406 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T09:56:42.742 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T09:56:42.758 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:42.780 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T09:56:42.797 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T09:56:42.825 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:42.839 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T09:56:43.083 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T09:56:43.099 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T09:56:43.141 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T09:56:43.162 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T09:56:43.178 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:43.212 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T09:56:43.495 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T09:56:43.510 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T09:56:43.515 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T09:56:43.530 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:43.553 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T09:56:43.573 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T09:56:43.817 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T09:56:43.833 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T09:56:43.875 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T09:56:43.906 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T09:56:43.921 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T09:56:43.964 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T09:56:44.250 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T09:56:44.266 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T09:56:44.308 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T09:56:44.379 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T09:56:44.394 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T09:56:44.438 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T09:56:44.678 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T09:56:44.694 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:44.737 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:44.833 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package modemmanager. 2024-09-10T09:56:44.847 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T09:56:44.891 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T09:56:45.056 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T09:56:45.072 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:45.114 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T09:56:45.392 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T09:56:45.407 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:45.450 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T09:56:45.523 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package network-manager. 2024-09-10T09:56:45.539 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T09:56:45.589 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T09:56:45.678 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T09:56:45.693 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T09:56:45.737 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T09:56:46.206 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ppp. 2024-09-10T09:56:46.224 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T09:56:46.273 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T09:56:46.288 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T09:56:46.331 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T09:56:46.568 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T09:56:46.693 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package modemmanager. 2024-09-10T09:56:46.708 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T09:56:46.751 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T09:56:46.853 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T09:56:46.865 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T09:56:46.945 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T09:56:47.197 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T09:56:47.211 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T09:56:47.256 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T09:56:47.383 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package network-manager. 2024-09-10T09:56:47.398 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T09:56:47.449 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T09:56:47.667 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T09:56:47.680 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T09:56:47.717 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T09:56:47.972 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T09:56:47.984 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:48.013 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:48.193 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ppp. 2024-09-10T09:56:48.207 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T09:56:48.526 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T09:56:48.541 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T09:56:48.566 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T09:56:48.593 INFO:teuthology.orchestra.run.smithi110.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T09:56:48.650 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking postgresql-common (238) ... 2024-09-10T09:56:48.906 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T09:56:48.921 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T09:56:48.964 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T09:56:49.125 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T09:56:49.139 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:49.217 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T09:56:49.218 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:49.232 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T09:56:49.292 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T09:56:49.604 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T09:56:49.618 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T09:56:49.662 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T09:56:49.991 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T09:56:50.008 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:50.049 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:50.321 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package postgresql. 2024-09-10T09:56:50.337 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T09:56:50.379 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking postgresql (14+238) ... 2024-09-10T09:56:50.629 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T09:56:50.643 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T09:56:50.656 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T09:56:50.671 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T09:56:50.704 INFO:teuthology.orchestra.run.smithi145.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T09:56:50.716 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T09:56:50.736 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking postgresql-common (238) ... 2024-09-10T09:56:50.959 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T09:56:50.973 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T09:56:51.042 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T09:56:51.203 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T09:56:51.217 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T09:56:51.289 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:51.328 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T09:56:51.342 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T09:56:51.378 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T09:56:51.689 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package tcl. 2024-09-10T09:56:51.704 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T09:56:51.752 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T09:56:52.101 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T09:56:52.115 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T09:56:52.192 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T09:56:52.398 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package postgresql. 2024-09-10T09:56:52.413 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T09:56:52.456 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking postgresql (14+238) ... 2024-09-10T09:56:52.513 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T09:56:52.528 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T09:56:52.563 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T09:56:52.733 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T09:56:52.748 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T09:56:52.792 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T09:56:52.975 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T09:56:52.990 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T09:56:53.041 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T09:56:53.044 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T09:56:53.059 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T09:56:53.102 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T09:56:53.377 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T09:56:53.438 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T09:56:53.453 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T09:56:53.496 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T09:56:53.590 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T09:56:53.716 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T09:56:53.758 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package tcl. 2024-09-10T09:56:53.772 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T09:56:53.820 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T09:56:53.858 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:54.010 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T09:56:54.127 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T09:56:54.194 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T09:56:54.210 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T09:56:54.228 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:54.252 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T09:56:54.573 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T09:56:54.589 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T09:56:54.631 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T09:56:54.985 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T09:56:55.004 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T09:56:55.051 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T09:56:55.186 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T09:56:55.276 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T09:56:55.330 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T09:56:55.568 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T09:56:55.619 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T09:56:55.619 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T09:56:55.694 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T09:56:55.803 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:55.929 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T09:56:56.089 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T09:56:56.215 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:56:56.288 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T09:56:56.822 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T09:56:56.948 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T09:56:57.074 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T09:56:57.168 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T09:56:57.200 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T09:56:57.262 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T09:56:57.318 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T09:56:57.444 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T09:56:57.545 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T09:56:57.664 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T09:56:57.664 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T09:56:58.301 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T09:56:58.357 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T09:56:58.435 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T09:56:58.483 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T09:56:58.530 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T09:56:58.612 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T09:56:58.625 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T09:56:58.713 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T09:56:58.776 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T09:56:58.831 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T09:56:58.894 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T09:56:58.957 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T09:56:59.003 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T09:56:59.083 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T09:56:59.129 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T09:56:59.206 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T09:56:59.264 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T09:56:59.276 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T09:56:59.403 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T09:56:59.579 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T09:56:59.722 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T09:56:59.831 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T09:56:59.999 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T09:57:00.011 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T09:57:00.134 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T09:57:00.145 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T09:57:00.240 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T09:57:00.305 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T09:57:00.431 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T09:57:00.557 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T09:57:00.683 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T09:57:00.755 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:00.755 INFO:teuthology.orchestra.run.smithi110.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T09:57:00.755 INFO:teuthology.orchestra.run.smithi110.stdout:which means they are currently configured by ifupdown: 2024-09-10T09:57:00.755 INFO:teuthology.orchestra.run.smithi110.stdout:- enp3s0f1 2024-09-10T09:57:00.755 INFO:teuthology.orchestra.run.smithi110.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T09:57:00.755 INFO:teuthology.orchestra.run.smithi110.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T09:57:00.755 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:00.834 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T09:57:00.886 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T09:57:00.924 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T09:57:00.986 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T09:57:01.095 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T09:57:01.272 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T09:57:01.289 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T09:57:01.431 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T09:57:01.557 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T09:57:01.689 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T09:57:01.725 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T09:57:01.851 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T09:57:02.445 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T09:57:02.479 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:02.480 INFO:teuthology.orchestra.run.smithi145.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T09:57:02.480 INFO:teuthology.orchestra.run.smithi145.stdout:which means they are currently configured by ifupdown: 2024-09-10T09:57:02.480 INFO:teuthology.orchestra.run.smithi145.stdout:- enp3s0f1 2024-09-10T09:57:02.480 INFO:teuthology.orchestra.run.smithi145.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T09:57:02.480 INFO:teuthology.orchestra.run.smithi145.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T09:57:02.480 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:02.605 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T09:57:02.611 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T09:57:02.739 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T09:57:02.857 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up postgresql-common (238) ... 2024-09-10T09:57:03.032 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T09:57:03.691 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T09:57:03.804 INFO:teuthology.orchestra.run.smithi110.stdout:Adding user postgres to group ssl-cert 2024-09-10T09:57:04.057 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:04.057 INFO:teuthology.orchestra.run.smithi110.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T09:57:04.329 INFO:teuthology.orchestra.run.smithi110.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T09:57:04.329 INFO:teuthology.orchestra.run.smithi110.stdout:Removing obsolete dictionary files: 2024-09-10T09:57:04.429 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T09:57:04.581 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T09:57:04.706 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T09:57:04.840 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up postgresql-common (238) ... 2024-09-10T09:57:05.012 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T09:57:05.492 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:57:05.816 INFO:teuthology.orchestra.run.smithi145.stdout:Adding user postgres to group ssl-cert 2024-09-10T09:57:05.854 INFO:teuthology.orchestra.run.smithi110.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T09:57:05.872 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T09:57:05.930 INFO:teuthology.orchestra.run.smithi110.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout:This user must also own the server process. 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout:The default text search configuration will be set to "english". 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout:Data page checksums are disabled. 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:05.931 INFO:teuthology.orchestra.run.smithi110.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T09:57:05.932 INFO:teuthology.orchestra.run.smithi110.stdout:creating subdirectories ... ok 2024-09-10T09:57:05.932 INFO:teuthology.orchestra.run.smithi110.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T09:57:05.956 INFO:teuthology.orchestra.run.smithi110.stdout:selecting default max_connections ... 100 2024-09-10T09:57:05.989 INFO:teuthology.orchestra.run.smithi110.stdout:selecting default shared_buffers ... 128MB 2024-09-10T09:57:06.003 INFO:teuthology.orchestra.run.smithi110.stdout:selecting default time zone ... Etc/UTC 2024-09-10T09:57:06.003 INFO:teuthology.orchestra.run.smithi110.stdout:creating configuration files ... ok 2024-09-10T09:57:06.073 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:06.073 INFO:teuthology.orchestra.run.smithi145.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T09:57:06.165 INFO:teuthology.orchestra.run.smithi110.stdout:running bootstrap script ... ok 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi145.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi145.stdout:Removing obsolete dictionary files: 2024-09-10T09:57:06.766 INFO:teuthology.orchestra.run.smithi110.stdout:performing post-bootstrap initialization ... ok 2024-09-10T09:57:07.080 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T09:57:07.312 INFO:teuthology.orchestra.run.smithi110.stdout:syncing data to disk ... ok 2024-09-10T09:57:07.534 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T09:57:07.834 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T09:57:07.898 INFO:teuthology.orchestra.run.smithi145.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T09:57:07.917 INFO:teuthology.orchestra.run.smithi145.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T09:57:07.972 INFO:teuthology.orchestra.run.smithi145.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T09:57:07.972 INFO:teuthology.orchestra.run.smithi145.stdout:This user must also own the server process. 2024-09-10T09:57:07.972 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:07.972 INFO:teuthology.orchestra.run.smithi145.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T09:57:07.972 INFO:teuthology.orchestra.run.smithi145.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T09:57:07.972 INFO:teuthology.orchestra.run.smithi145.stdout:The default text search configuration will be set to "english". 2024-09-10T09:57:07.973 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:07.973 INFO:teuthology.orchestra.run.smithi145.stdout:Data page checksums are disabled. 2024-09-10T09:57:07.973 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:07.973 INFO:teuthology.orchestra.run.smithi145.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T09:57:07.973 INFO:teuthology.orchestra.run.smithi145.stdout:creating subdirectories ... ok 2024-09-10T09:57:07.974 INFO:teuthology.orchestra.run.smithi145.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T09:57:07.997 INFO:teuthology.orchestra.run.smithi145.stdout:selecting default max_connections ... 100 2024-09-10T09:57:08.030 INFO:teuthology.orchestra.run.smithi145.stdout:selecting default shared_buffers ... 128MB 2024-09-10T09:57:08.045 INFO:teuthology.orchestra.run.smithi145.stdout:selecting default time zone ... Etc/UTC 2024-09-10T09:57:08.045 INFO:teuthology.orchestra.run.smithi145.stdout:creating configuration files ... ok 2024-09-10T09:57:08.207 INFO:teuthology.orchestra.run.smithi145.stdout:running bootstrap script ... ok 2024-09-10T09:57:08.814 INFO:teuthology.orchestra.run.smithi145.stdout:performing post-bootstrap initialization ... ok 2024-09-10T09:57:09.344 INFO:teuthology.orchestra.run.smithi145.stdout:syncing data to disk ... ok 2024-09-10T09:57:09.872 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T09:57:10.869 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T09:57:10.995 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T09:57:11.121 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up postgresql (14+238) ... 2024-09-10T09:57:11.247 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T09:57:11.423 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T09:57:11.683 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T09:57:11.683 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T09:57:12.359 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T09:57:12.477 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T09:57:12.770 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T09:57:12.833 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T09:57:12.922 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T09:57:12.959 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T09:57:13.110 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up postgresql (14+238) ... 2024-09-10T09:57:13.236 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T09:57:13.362 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T09:57:13.623 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T09:57:13.623 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T09:57:14.358 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T09:57:14.423 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T09:57:14.450 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T09:57:14.523 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T09:57:14.826 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T09:57:14.961 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T09:57:14.978 INFO:teuthology.orchestra.run.smithi110.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T09:57:14.986 DEBUG:teuthology.parallel:result is None 2024-09-10T09:57:16.430 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T09:57:16.539 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T09:57:17.087 INFO:teuthology.orchestra.run.smithi145.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T09:57:17.093 DEBUG:teuthology.parallel:result is None 2024-09-10T09:57:17.094 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:57:17.247 DEBUG:teuthology.orchestra.run.smithi110:> dpkg-query -W -f '${Version}' ceph 2024-09-10T09:57:17.271 INFO:teuthology.orchestra.run.smithi110.stdout:19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:57:17.272 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:57:17.272 INFO:teuthology.task.install:The correct ceph version 19.1.1-305-g1cf819b7-1jammy is installed. 2024-09-10T09:57:17.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:57:17.418 DEBUG:teuthology.orchestra.run.smithi145:> dpkg-query -W -f '${Version}' ceph 2024-09-10T09:57:17.440 INFO:teuthology.orchestra.run.smithi145.stdout:19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:57:17.440 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305-g1cf819b7-1jammy 2024-09-10T09:57:17.441 INFO:teuthology.task.install:The correct ceph version 19.1.1-305-g1cf819b7-1jammy is installed. 2024-09-10T09:57:17.442 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T09:57:17.443 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:17.443 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:57:17.465 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:17.465 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:57:17.500 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T09:57:17.500 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:17.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:57:17.527 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:57:17.591 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:17.591 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:57:17.610 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:57:17.670 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T09:57:17.670 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:17.670 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:57:17.691 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:57:17.750 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:17.751 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:57:17.769 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:57:17.830 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T09:57:17.830 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:17.830 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:57:17.850 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:57:17.910 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:17.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:57:17.929 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:57:17.990 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T09:57:18.074 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T09:57:18.074 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /var/log/ceph 2024-09-10T09:57:18.077 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /var/log/ceph 2024-09-10T09:57:18.095 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T09:57:18.095 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T09:57:18.138 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T09:57:18.158 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T09:57:18.158 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T09:57:18.197 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T09:57:18.229 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T09:57:18.229 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T09:57:18.229 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T09:57:18.229 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:18.229 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T09:57:18.267 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T09:57:18.362 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:18.363 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T09:57:18.371 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T09:57:18.461 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T09:57:18.461 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T09:57:18.461 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008', 'branch': 'wip-rishabh-testing-20240909.165402-squid', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7899008', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240909.165402-squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240909.165402-squid', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9893, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'sleep_before_teardown': 0, 'subset': '197/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240909.165402-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi110.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI2g5sy5QvepqMrfYO6XjqgwAP3Pc/v3GjgLcEGf0Lc+zn805lrLIch/TiLSoUIwNysf/t7ET1QjFpRE98L8vGM=', 'smithi145.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFiv7T7h+shSMYEsxZQxxK4FylSDbj28yyJwxH+Hw+YGVdHjNSMw583MQNQ8vqElJSOIZPk0CQTdVD2XUxn/Jvk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_06:58:10', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T09:57:18.462 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T09:57:18.465 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T09:57:18.509 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T09:57:18.514 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T09:57:18.571 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:18.571 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:57:18.583 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:57:18.583 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-10T09:57:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:57:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:57:18.635 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 904 Links: 1 2024-09-10T09:57:18.635 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:57:18.636 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-10 09:50:47.107377164 +0000 2024-09-10T09:57:18.636 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-10 09:50:47.103377200 +0000 2024-09-10T09:57:18.636 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-10 09:50:47.103377200 +0000 2024-09-10T09:57:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-10 09:50:47.103377200 +0000 2024-09-10T09:57:18.636 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:57:18.697 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T09:57:18.697 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T09:57:18.697 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000267261 s, 1.9 MB/s 2024-09-10T09:57:18.698 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:57:18.749 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-10T09:57:18.800 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:57:18.800 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:57:18.800 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 918 Links: 1 2024-09-10T09:57:18.800 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:57:18.800 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-10 09:50:47.915369947 +0000 2024-09-10T09:57:18.800 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-10 09:50:47.663372198 +0000 2024-09-10T09:57:18.800 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-10 09:50:47.663372198 +0000 2024-09-10T09:57:18.800 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-10 09:50:47.663372198 +0000 2024-09-10T09:57:18.800 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:57:18.861 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T09:57:18.861 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T09:57:18.862 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000259708 s, 2.0 MB/s 2024-09-10T09:57:18.863 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:57:18.917 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-10T09:57:18.972 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:57:18.972 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:57:18.972 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 935 Links: 1 2024-09-10T09:57:18.972 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:57:18.972 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-10 09:50:48.483364874 +0000 2024-09-10T09:57:18.972 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-10 09:50:48.203367375 +0000 2024-09-10T09:57:18.972 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-10 09:50:48.203367375 +0000 2024-09-10T09:57:18.972 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-10 09:50:48.203367375 +0000 2024-09-10T09:57:18.972 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:57:19.033 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T09:57:19.034 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T09:57:19.034 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000273901 s, 1.9 MB/s 2024-09-10T09:57:19.035 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:57:19.089 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-10T09:57:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:57:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:57:19.144 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 953 Links: 1 2024-09-10T09:57:19.144 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:57:19.144 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-10 09:50:49.071359622 +0000 2024-09-10T09:57:19.144 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-10 09:50:48.787362158 +0000 2024-09-10T09:57:19.144 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-10 09:50:48.787362158 +0000 2024-09-10T09:57:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-10 09:50:48.787362158 +0000 2024-09-10T09:57:19.145 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:57:19.205 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T09:57:19.205 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T09:57:19.205 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.00027779 s, 1.8 MB/s 2024-09-10T09:57:19.207 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:57:19.261 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:19.262 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:19.262 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:57:19.270 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:57:19.270 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-09-10T09:57:19.325 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:57:19.325 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:57:19.325 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 896 Links: 1 2024-09-10T09:57:19.325 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:57:19.325 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-10 09:50:46.551390580 +0000 2024-09-10T09:57:19.325 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-10 09:50:46.547390617 +0000 2024-09-10T09:57:19.325 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-10 09:50:46.547390617 +0000 2024-09-10T09:57:19.325 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-10 09:50:46.547390617 +0000 2024-09-10T09:57:19.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:57:19.390 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-10T09:57:19.390 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-10T09:57:19.390 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000396978 s, 1.3 MB/s 2024-09-10T09:57:19.392 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:57:19.447 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-09-10T09:57:19.501 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:57:19.501 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:57:19.501 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 913 Links: 1 2024-09-10T09:57:19.501 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:57:19.501 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-10 09:50:47.355383396 +0000 2024-09-10T09:57:19.501 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-10 09:50:47.107385612 +0000 2024-09-10T09:57:19.502 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-10 09:50:47.107385612 +0000 2024-09-10T09:57:19.502 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-10 09:50:47.107385612 +0000 2024-09-10T09:57:19.502 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:57:19.563 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-10T09:57:19.563 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-10T09:57:19.563 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000372032 s, 1.4 MB/s 2024-09-10T09:57:19.565 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:57:19.618 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-09-10T09:57:19.669 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:57:19.669 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:57:19.669 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 933 Links: 1 2024-09-10T09:57:19.669 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:57:19.669 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-10 09:50:47.903378503 +0000 2024-09-10T09:57:19.669 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-10 09:50:47.659380683 +0000 2024-09-10T09:57:19.670 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-10 09:50:47.659380683 +0000 2024-09-10T09:57:19.670 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-10 09:50:47.659380683 +0000 2024-09-10T09:57:19.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:57:19.734 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-10T09:57:19.734 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-10T09:57:19.734 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000376779 s, 1.4 MB/s 2024-09-10T09:57:19.736 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:57:19.787 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-09-10T09:57:19.841 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:57:19.841 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:57:19.841 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 949 Links: 1 2024-09-10T09:57:19.841 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:57:19.841 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-09-10 09:50:48.491373254 +0000 2024-09-10T09:57:19.841 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-09-10 09:50:48.227375611 +0000 2024-09-10T09:57:19.841 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-09-10 09:50:48.227375611 +0000 2024-09-10T09:57:19.841 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-09-10 09:50:48.227375611 +0000 2024-09-10T09:57:19.842 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:57:19.907 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-09-10T09:57:19.907 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-09-10T09:57:19.907 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000391607 s, 1.3 MB/s 2024-09-10T09:57:19.908 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:57:19.963 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:19.963 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T09:57:19.963 INFO:tasks.ceph:Generating config... 2024-09-10T09:57:19.964 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-10T09:57:19.964 INFO:tasks.ceph:[client] debug client = 20 2024-09-10T09:57:19.964 INFO:tasks.ceph:[client] debug ms = 1 2024-09-10T09:57:19.964 INFO:tasks.ceph:[client] pid file = 2024-09-10T09:57:19.964 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-10T09:57:19.965 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-10T09:57:19.965 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-10T09:57:19.965 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-10T09:57:19.966 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-10T09:57:19.966 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T09:57:19.966 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T09:57:19.966 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-10T09:57:19.966 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T09:57:19.966 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T09:57:19.966 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T09:57:19.967 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-10T09:57:19.967 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T09:57:19.967 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T09:57:20.015 INFO:teuthology.orchestra.run.smithi110.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T09:57:20.040 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T09:57:20.130 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T09:57:20.149 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.110'), ('mon.b', '172.21.15.145'), ('mon.c', '[v2:172.21.15.145:3301,v1:172.21.15.145:6790]')] 2024-09-10T09:57:20.150 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.110,172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-10T09:57:20.151 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:20.151 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T09:57:20.199 DEBUG:teuthology.orchestra.run.smithi110:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.110 --add b 172.21.15.145 --addv c '[v2:172.21.15.145:3301,v1:172.21.15.145:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:20.289 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:20.289 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool: generated fsid af5c1104-e6d8-4fa7-9635-0686bce1580a 2024-09-10T09:57:20.289 INFO:teuthology.orchestra.run.smithi110.stdout:setting min_mon_release = quincy 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 0 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:fsid af5c1104-e6d8-4fa7-9635-0686bce1580a 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:last_changed 2024-09-10T09:57:20.291728+0000 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-10T09:57:20.291728+0000 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:min_mon_release 17 (quincy) 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:election_strategy: 1 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.b 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:2: [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] mon.c 2024-09-10T09:57:20.290 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T09:57:20.292 DEBUG:teuthology.orchestra.run.smithi110:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T09:57:20.300 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID af5c1104-e6d8-4fa7-9635-0686bce1580a... 2024-09-10T09:57:20.301 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T09:57:20.346 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: chdir = "" 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: auth supported = cephx 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: filestore xattr use omap = true 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: mon clock drift allowed = 1.000 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: osd crush chooseleaf type = 0 2024-09-10T09:57:20.384 INFO:teuthology.orchestra.run.smithi110.stdout: auth debug = true 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on old message = true 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on bug = true 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: mon pg warn max object skew = 0 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default size = 2 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.385 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow primary affinity = true 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow pg remap = true 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on legacy crush tunables = false 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on crush straw calc version zero = false 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on no sortbitwise = false 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on osd down out interval zero = false 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on too few osds = false 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: mon_allow_pool_size_one = true 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: osd default data pool replay window = 5 2024-09-10T09:57:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: mon allow pool delete = true 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: mon cluster log file level = debug 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: debug asserts on shutdown = true 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: mon health detail to clog = false 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: mon host = "172.21.15.110,172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790]" 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on skipped message = False 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = af5c1104-e6d8-4fa7-9635-0686bce1580a 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout:[osd] 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: osd journal size = 100 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub load threshold = 5.0 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub max interval = 600 2024-09-10T09:57:20.387 INFO:teuthology.orchestra.run.smithi110.stdout: osd mclock profile = high_recovery_ops 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd recover clone overlap = true 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd recovery max chunk = 1048576 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug shutdown = true 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug op order = true 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify stray on activate = true 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug trim objects = true 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd open classes on start = true 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug pg log writeout = true 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd deep scrub update digest min age = 30 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.388 INFO:teuthology.orchestra.run.smithi110.stdout: osd map max advance = 10 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: journal zero on create = true 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: filestore ondisk finisher threads = 3 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: filestore apply finisher threads = 3 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: bdev debug aio = true 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug misdirected ops = true 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: bdev async discard = True 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: bdev enable discard = True 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore allocator = bitmap 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore block size = 96636764160 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore fsck on mount = True 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluefs = 1/20 2024-09-10T09:57:20.389 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluestore = 1/20 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: debug osd = 20 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: debug rocksdb = 4/10 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd full ratio = 0.9 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd nearfull ratio = 0.8 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: osd failsafe full ratio = 0.95 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: osd objectstore = bluestore 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: osd op complaint time = 180 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr] 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: debug mgr = 20 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2024-09-10T09:57:20.390 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: mgr/telemetry/nag = false 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: debug client = 20 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: debug finisher = 20 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout:[mon] 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: debug paxos = 20 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: mon data avail warn = 5 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: mon mgr mkfs grace = 240 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2024-09-10T09:57:20.391 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd reporter subtree level = osd 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd prime pg temp = true 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: # 1m isn't quite enough 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: mon_down_mkfs_grace = 2m 2024-09-10T09:57:20.392 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_filestore_osds = false 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: mon down mkfs grace = 300 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: mon op complaint time = 120 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on pool no app = False 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout:[client] 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: rgw cache enabled = true 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable ops log = true 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable usage log = true 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: client mount timeout = 600 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: debug client = 20 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-10T09:57:20.393 INFO:teuthology.orchestra.run.smithi110.stdout: pid file = "" 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: rados mon op timeout = 900 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: rados osd op timeout = 900 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.a] 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.b] 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.c] 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.a] 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.c] 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.b] 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.d] 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout:[mds] 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: debug mds = 20 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: debug mds balancer = 20 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-10T09:57:20.394 INFO:teuthology.orchestra.run.smithi110.stdout: mds debug frag = True 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: mds debug scatterstat = True 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: mds op complaint time = 180 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: mds verify scatter = True 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: osd op complaint time = 180 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: rados mon op timeout = 900 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi110.stdout: rados osd op timeout = 900 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi145.stdout:[global] 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi145.stdout: chdir = "" 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi145.stdout: auth supported = cephx 2024-09-10T09:57:20.395 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: filestore xattr use omap = true 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: mon clock drift allowed = 1.000 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: osd crush chooseleaf type = 0 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: auth debug = true 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on old message = true 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on bug = true 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: mon pg warn max object skew = 0 2024-09-10T09:57:20.396 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default size = 2 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow primary affinity = true 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow pg remap = true 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on legacy crush tunables = false 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on crush straw calc version zero = false 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on no sortbitwise = false 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on osd down out interval zero = false 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on too few osds = false 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T09:57:20.397 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: mon_allow_pool_size_one = true 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: osd default data pool replay window = 5 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: mon allow pool delete = true 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: mon cluster log file level = debug 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: debug asserts on shutdown = true 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: mon health detail to clog = false 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: mon host = "172.21.15.110,172.21.15.145,[v2:172.21.15.145:3301,v1:172.21.15.145:6790]" 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on skipped message = False 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: fsid = af5c1104-e6d8-4fa7-9635-0686bce1580a 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.398 INFO:teuthology.orchestra.run.smithi145.stdout:[osd] 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd journal size = 100 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub load threshold = 5.0 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub max interval = 600 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd mclock profile = high_recovery_ops 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd recover clone overlap = true 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd recovery max chunk = 1048576 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug shutdown = true 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug op order = true 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug verify stray on activate = true 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug trim objects = true 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.399 INFO:teuthology.orchestra.run.smithi145.stdout: osd open classes on start = true 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug pg log writeout = true 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: osd deep scrub update digest min age = 30 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: osd map max advance = 10 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: journal zero on create = true 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: filestore ondisk finisher threads = 3 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: filestore apply finisher threads = 3 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: bdev debug aio = true 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug misdirected ops = true 2024-09-10T09:57:20.400 INFO:teuthology.orchestra.run.smithi145.stdout: bdev async discard = True 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: bdev enable discard = True 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore allocator = bitmap 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore block size = 96636764160 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore fsck on mount = True 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluefs = 1/20 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluestore = 1/20 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: debug osd = 20 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: debug rocksdb = 4/10 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd full ratio = 0.9 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd nearfull ratio = 0.8 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: osd failsafe full ratio = 0.95 2024-09-10T09:57:20.401 INFO:teuthology.orchestra.run.smithi145.stdout: osd objectstore = bluestore 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: osd op complaint time = 180 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr] 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: debug mgr = 20 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: mgr/telemetry/nag = false 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: debug finisher = 20 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout:[mon] 2024-09-10T09:57:20.402 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: debug paxos = 20 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: mon data avail warn = 5 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: mon mgr mkfs grace = 240 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd reporter subtree level = osd 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd prime pg temp = true 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T09:57:20.403 INFO:teuthology.orchestra.run.smithi145.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: # 1m isn't quite enough 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: mon_down_mkfs_grace = 2m 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_filestore_osds = false 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: mon down mkfs grace = 300 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: mon op complaint time = 120 2024-09-10T09:57:20.404 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on pool no app = False 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout:[client] 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: rgw cache enabled = true 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable ops log = true 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable usage log = true 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: client mount timeout = 600 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = "" 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: rados mon op timeout = 900 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout: rados osd op timeout = 900 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.a] 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.b] 2024-09-10T09:57:20.405 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.c] 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a] 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c] 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b] 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d] 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout:[mds] 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: debug mds = 20 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: debug mds balancer = 20 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: mds debug frag = True 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: mds debug scatterstat = True 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: mds op complaint time = 180 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: mds verify scatter = True 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: osd op complaint time = 180 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: rados mon op timeout = 900 2024-09-10T09:57:20.406 INFO:teuthology.orchestra.run.smithi145.stdout: rados osd op timeout = 900 2024-09-10T09:57:20.408 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T09:57:20.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T09:57:20.507 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T09:57:20.507 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:20.507 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T09:57:20.515 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:20.516 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T09:57:20.571 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:57:20.571 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:20.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T09:57:20.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T09:57:20.645 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:20.645 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:20.695 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:57:20.695 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:20.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T09:57:20.695 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T09:57:20.729 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:20.729 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:20.781 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T09:57:20.781 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T09:57:20.781 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T09:57:20.834 INFO:teuthology.orchestra.run.smithi110.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T09:57:20.839 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T09:57:20.899 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T09:57:20.913 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T09:57:20.913 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T09:57:20.966 INFO:teuthology.orchestra.run.smithi110.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T09:57:20.971 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T09:57:20.990 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T09:57:21.084 INFO:teuthology.orchestra.run.smithi110.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T09:57:21.088 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T09:57:21.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T09:57:21.165 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T09:57:21.170 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T09:57:21.190 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T09:57:21.286 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T09:57:21.291 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T09:57:21.310 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T09:57:21.310 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T09:57:21.351 INFO:teuthology.orchestra.run.smithi110.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T09:57:21.368 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T09:57:21.452 INFO:teuthology.orchestra.run.smithi110.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T09:57:21.469 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T09:57:21.510 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T09:57:21.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T09:57:21.613 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T09:57:21.629 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T09:57:21.629 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T09:57:21.629 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T09:57:21.648 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:21.648 INFO:tasks.ceph:role: osd.0 2024-09-10T09:57:21.649 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:57:21.649 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T09:57:21.713 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T09:57:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T09:57:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T09:57:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T09:57:21.713 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T09:57:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-10T09:57:21.713 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T09:57:21.713 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T09:57:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T09:57:21.714 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T09:57:21.737 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-10T09:57:21.740 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-10T09:57:21.740 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T09:57:21.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T09:57:21.870 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T09:57:21.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:57:21.872 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T09:57:21.933 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:21.933 INFO:tasks.ceph:role: osd.1 2024-09-10T09:57:21.934 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:57:21.934 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T09:57:21.993 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T09:57:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T09:57:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T09:57:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T09:57:21.994 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T09:57:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-10T09:57:21.994 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T09:57:21.995 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T09:57:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T09:57:21.995 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T09:57:22.018 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-10T09:57:22.021 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-10T09:57:22.021 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T09:57:22.086 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T09:57:22.142 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T09:57:22.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:57:22.143 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T09:57:22.205 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:22.205 INFO:tasks.ceph:role: osd.2 2024-09-10T09:57:22.205 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:57:22.206 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T09:57:22.266 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T09:57:22.290 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-10T09:57:22.292 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-10T09:57:22.293 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T09:57:22.359 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T09:57:22.414 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T09:57:22.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:57:22.415 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T09:57:22.478 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:22.478 INFO:tasks.ceph:role: osd.3 2024-09-10T09:57:22.478 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:57:22.478 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T09:57:22.537 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T09:57:22.537 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T09:57:22.537 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T09:57:22.537 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T09:57:22.537 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T09:57:22.538 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-10T09:57:22.538 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T09:57:22.538 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T09:57:22.538 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T09:57:22.538 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T09:57:22.561 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-10T09:57:22.564 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-10T09:57:22.564 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T09:57:22.630 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T09:57:22.690 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T09:57:22.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:57:22.691 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:22.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:22.814+0000 7ff9f9c6b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T09:57:22.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:22.814+0000 7ff9f9c6b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T09:57:22.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:22.814+0000 7ff9f9c6b7c0 -1 bdev(0x5614e77bf800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T09:57:22.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:22.814+0000 7ff9f9c6b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T09:57:26.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T09:57:26.593 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:26.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:26.690+0000 7f7489ca27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T09:57:26.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:26.690+0000 7f7489ca27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T09:57:26.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:26.690+0000 7f7489ca27c0 -1 bdev(0x55b257f57800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T09:57:26.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:26.690+0000 7f7489ca27c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T09:57:30.402 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T09:57:30.422 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:30.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:30.514+0000 7fac691137c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T09:57:30.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:30.514+0000 7fac691137c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T09:57:30.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:30.514+0000 7fac691137c0 -1 bdev(0x55eeeab73800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T09:57:30.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:30.514+0000 7fac691137c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T09:57:34.518 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T09:57:34.539 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:34.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:34.634+0000 7ff53873c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T09:57:34.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:34.634+0000 7ff53873c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T09:57:34.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:34.634+0000 7ff53873c7c0 -1 bdev(0x56216825f800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T09:57:34.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:34.634+0000 7ff53873c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T09:57:37.868 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T09:57:37.888 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T09:57:37.909 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:37.909 INFO:tasks.ceph:role: osd.4 2024-09-10T09:57:37.909 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:57:37.909 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T09:57:37.980 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T09:57:37.981 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T09:57:38.391 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-09-10T09:57:38.394 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-09-10T09:57:38.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T09:57:38.518 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T09:57:38.535 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T09:57:38.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:57:38.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T09:57:38.601 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:38.601 INFO:tasks.ceph:role: osd.5 2024-09-10T09:57:38.601 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:57:38.601 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T09:57:38.672 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T09:57:38.672 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T09:57:38.672 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T09:57:38.672 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T09:57:38.672 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T09:57:38.672 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-09-10T09:57:38.673 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T09:57:38.673 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T09:57:38.673 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T09:57:38.673 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T09:57:39.041 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-09-10T09:57:39.044 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-09-10T09:57:39.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T09:57:39.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T09:57:39.180 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T09:57:39.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:57:39.181 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T09:57:39.244 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:39.244 INFO:tasks.ceph:role: osd.6 2024-09-10T09:57:39.245 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:57:39.245 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T09:57:39.315 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T09:57:39.316 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T09:57:39.711 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-09-10T09:57:39.714 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-09-10T09:57:39.714 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T09:57:39.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T09:57:39.827 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T09:57:39.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:57:39.828 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T09:57:39.892 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T09:57:39.892 INFO:tasks.ceph:role: osd.7 2024-09-10T09:57:39.892 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:57:39.892 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T09:57:39.968 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T09:57:39.968 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T09:57:39.969 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T09:57:39.969 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T09:57:39.969 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T09:57:39.969 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-09-10T09:57:39.969 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T09:57:39.969 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T09:57:39.969 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T09:57:39.969 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T09:57:40.370 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-09-10T09:57:40.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-09-10T09:57:40.373 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T09:57:40.496 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T09:57:40.513 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T09:57:40.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:57:40.514 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:40.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:40.639+0000 7fded066b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T09:57:40.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:40.643+0000 7fded066b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T09:57:40.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:40.643+0000 7fded066b7c0 -1 bdev(0x55a419207800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T09:57:40.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:40.643+0000 7fded066b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T09:57:44.610 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T09:57:44.631 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:44.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:44.723+0000 7fb19344a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T09:57:44.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:44.723+0000 7fb19344a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T09:57:44.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:44.723+0000 7fb19344a7c0 -1 bdev(0x55a6c3585800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T09:57:44.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:44.723+0000 7fb19344a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T09:57:48.187 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T09:57:48.206 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:48.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:48.295+0000 7f42365a77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T09:57:48.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:48.295+0000 7f42365a77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T09:57:48.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:48.295+0000 7f42365a77c0 -1 bdev(0x55714dca7800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T09:57:48.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:48.295+0000 7f42365a77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T09:57:48.463 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:57:48.467 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:57:51.758 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T09:57:51.779 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:51.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:51.867+0000 7efcf5a1b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T09:57:51.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:51.871+0000 7efcf5a1b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T09:57:51.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:51.871+0000 7efcf5a1b7c0 -1 bdev(0x563dbcb11800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T09:57:51.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:51.871+0000 7efcf5a1b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T09:57:55.354 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T09:57:55.375 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T09:57:55.375 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:55.375 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T09:57:55.397 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:55.397 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T09:57:55.459 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:55.459 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T09:57:55.527 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:55.527 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T09:57:55.595 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:55.595 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T09:57:55.664 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:55.664 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T09:57:55.731 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:55.732 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T09:57:55.799 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:55.799 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T09:57:55.821 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:55.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T09:57:55.884 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:55.884 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T09:57:55.948 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:55.948 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T09:57:56.008 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:56.008 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T09:57:56.068 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:56.068 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T09:57:56.135 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:56.136 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T09:57:56.195 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:56.196 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T09:57:56.204 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:56.204 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T09:57:56.255 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:56.255 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T09:57:56.264 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:56.264 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T09:57:56.312 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T09:57:56.312 DEBUG:teuthology.orchestra.run.smithi110:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T09:57:56.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.y] 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw== 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.a] 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A== 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.c] 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw== 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.0] 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ== 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.1] 2024-09-10T09:57:56.360 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw== 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.2] 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg== 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.3] 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg== 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.x] 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ== 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.b] 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw== 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.d] 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w== 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.4] 2024-09-10T09:57:56.361 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg== 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.5] 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA== 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.6] 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg== 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.7] 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA== 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout:[client.2] 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg== 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout:[client.1] 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g== 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout:[client.3] 2024-09-10T09:57:56.362 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng== 2024-09-10T09:57:56.374 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.y] 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw== 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a] 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A== 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c] 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw== 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.0] 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ== 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.1] 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw== 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.2] 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg== 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.3] 2024-09-10T09:57:56.375 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg== 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.x] 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ== 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b] 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw== 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d] 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w== 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.4] 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg== 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.5] 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA== 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.6] 2024-09-10T09:57:56.376 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg== 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.7] 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA== 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout:[client.2] 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg== 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout:[client.1] 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g== 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout:[client.3] 2024-09-10T09:57:56.377 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng== 2024-09-10T09:57:56.378 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T09:57:56.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T09:57:56.477 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T09:57:56.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T09:57:56.538 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T09:57:56.541 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T09:57:56.597 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.653 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.656 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.709 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.765 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.768 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.820 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T09:57:56.823 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T09:57:56.875 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T09:57:56.878 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T09:57:56.934 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T09:57:56.937 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T09:57:56.993 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:56.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:57.049 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:57.052 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:57.105 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:57.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:57.161 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:57.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T09:57:57.216 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T09:57:57.220 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T09:57:57.275 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T09:57:57.278 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T09:57:57.334 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T09:57:57.337 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T09:57:57.392 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T09:57:57.395 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T09:57:57.451 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T09:57:57.451 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T09:57:57.469 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T09:57:57.581 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T09:57:57.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T09:57:57.618 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T09:57:57.728 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T09:57:57.747 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T09:57:57.806 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T09:57:57.913 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T09:57:57.932 DEBUG:teuthology.orchestra.run.smithi110:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:57.935 DEBUG:teuthology.orchestra.run.smithi145:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T09:57:57.988 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T09:57:57.988 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T09:57:57.988 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T09:57:57.991 INFO:tasks.ceph.mon.a:Started 2024-09-10T09:57:57.991 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T09:57:57.991 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T09:57:58.031 INFO:tasks.ceph.mon.b:Started 2024-09-10T09:57:58.031 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T09:57:58.031 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T09:57:58.034 INFO:tasks.ceph.mon.c:Started 2024-09-10T09:57:58.034 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T09:57:58.034 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T09:57:58.034 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T09:57:58.037 INFO:tasks.ceph.mgr.y:Started 2024-09-10T09:57:58.037 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T09:57:58.037 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T09:57:58.040 INFO:tasks.ceph.mgr.x:Started 2024-09-10T09:57:58.041 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T09:57:58.041 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T09:57:58.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.174+0000 7fa4d2400640 1 Processor -- start 2024-09-10T09:57:58.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.174+0000 7fa4d2400640 1 -- start start 2024-09-10T09:57:58.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4d2400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa4cc15e830 0x7fa4cc0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:58.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4d2400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 0x7fa4cc0b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4d2400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc0b5f30 con 0x7fa4cc0b2e80 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4d2400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc0b6240 con 0x7fa4cc15e830 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4d2400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b34b0 con 0x7fa4cc06de80 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4caa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa4cc0b2e80 0x7fa4cc06d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52376/0 (socket says 172.21.15.110:52376) 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4caa00640 1 -- 172.21.15.110:0/2791314090 learned_addr learned my addr 172.21.15.110:0/2791314090 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 0x7fa4cc0b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cbe00640 1 -- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 msgr2=0x7fa4cc0b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cbe00640 1 -- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 msgr2=0x7fa4cc0b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 0x7fa4cc0b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 0x7fa4cc0b66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cb400640 1 --2- 172.21.15.110:0/2791314090 >> v2:172.21.15.145:3300/0 conn(0x7fa4cc15e830 0x7fa4cc0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cb400640 1 -- 172.21.15.110:0/2791314090 >> v2:172.21.15.145:3300/0 conn(0x7fa4cc15e830 msgr2=0x7fa4cc0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T09:57:58.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cb400640 1 -- 172.21.15.110:0/2791314090 >> v2:172.21.15.145:3300/0 conn(0x7fa4cc15e830 msgr2=0x7fa4cc0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T09:57:58.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cb400640 1 --2- 172.21.15.110:0/2791314090 >> v2:172.21.15.145:3300/0 conn(0x7fa4cc15e830 0x7fa4cc0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:57:58.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.178+0000 7fa4cb400640 1 --2- 172.21.15.110:0/2791314090 >> v2:172.21.15.145:3300/0 conn(0x7fa4cc15e830 0x7fa4cc0b2940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:57:58.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2143222991 0 0) 0x7fa4b8001120 con 0x7fa4cc0b2e80 2024-09-10T09:57:58.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4b0003530 con 0x7fa4cc0b2e80 2024-09-10T09:57:58.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3278438095 0 0) 0x7fa4b0003530 con 0x7fa4cc0b2e80 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 msgr2=0x7fa4cc0b66d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 0x7fa4cc0b66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 >> v2:172.21.15.145:3300/0 conn(0x7fa4cc15e830 msgr2=0x7fa4cc0b2940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 --2- 172.21.15.110:0/2791314090 >> v2:172.21.15.145:3300/0 conn(0x7fa4cc15e830 0x7fa4cc0b2940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc0b3790 con 0x7fa4cc0b2e80 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fa4b8001850 con 0x7fa4cc0b2e80 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 >> v1:172.21.15.110:6789/0 conn(0x7fa4cc0b2e80 legacy=0x7fa4cc06d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4b00039c0 0x7fa4b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 0x7fa4b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4b0004d50 0x7fa4b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:58.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b6240 con 0x7fa4b00039c0 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b34b0 con 0x7fa4b0004d50 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b000bb30 con 0x7fa4cc06de80 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4cb400640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4b00039c0 0x7fa4b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4b0004d50 0x7fa4b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4caa00640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 0x7fa4b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4cbe00640 1 -- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 msgr2=0x7fa4b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4cc06de80 0x7fa4b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4cbe00640 1 -- 172.21.15.110:0/2791314090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4b00039c0 msgr2=0x7fa4b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:58.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4b00039c0 0x7fa4b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4cbe00640 1 -- 172.21.15.110:0/2791314090 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b0003530 con 0x7fa4b0004d50 2024-09-10T09:57:58.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.330+0000 7fa4cb400640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4b00039c0 0x7fa4b0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:58.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.334+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4b0004d50 0x7fa4b00051d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009610 tx=0x7fa4c401ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8473e4b9978d5d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:58.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40b2070 con 0x7fa4b0004d50 2024-09-10T09:57:58.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c40a5ae0 con 0x7fa4b0004d50 2024-09-10T09:57:58.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4ca000640 1 -- 172.21.15.110:0/2791314090 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4009bd0 con 0x7fa4b0004d50 2024-09-10T09:57:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4d2400640 1 -- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4b0004d50 msgr2=0x7fa4b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4d2400640 1 --2- 172.21.15.110:0/2791314090 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4b0004d50 0x7fa4b00051d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009610 tx=0x7fa4c401ce00 comp rx=0 tx=0).stop 2024-09-10T09:57:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4cbe00640 1 -- 172.21.15.110:0/2791314090 reap_dead start 2024-09-10T09:57:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4d2400640 1 -- 172.21.15.110:0/2791314090 shutdown_connections 2024-09-10T09:57:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4d2400640 1 -- 172.21.15.110:0/2791314090 >> 172.21.15.110:0/2791314090 conn(0x7fa4cc09b9d0 msgr2=0x7fa4cc09bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4d2400640 1 -- 172.21.15.110:0/2791314090 shutdown_connections 2024-09-10T09:57:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4d2400640 1 -- 172.21.15.110:0/2791314090 wait complete. 2024-09-10T09:57:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4d2400640 1 Processor -- start 2024-09-10T09:57:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.338+0000 7fa4d2400640 1 -- start start 2024-09-10T09:57:58.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4b0004d50 0x7fa4cc1f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:58.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4cc1f7270 0x7fa4cc15a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4cc09f600 0x7fa4cc09fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b7040 con 0x7fa4cc1f7270 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b6ec0 con 0x7fa4cc09f600 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b71c0 con 0x7fa4b0004d50 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4cc1f7270 0x7fa4cc15a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4cc1f7270 0x7fa4cc15a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50010/0 (socket says 172.21.15.110:50010) 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 -- 172.21.15.110:0/1644828071 learned_addr learned my addr 172.21.15.110:0/1644828071 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4cc09f600 0x7fa4cc09fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4cb400640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4b0004d50 0x7fa4cc1f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 -- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4b0004d50 msgr2=0x7fa4cc1f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4b0004d50 0x7fa4cc1f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 -- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4cc09f600 msgr2=0x7fa4cc09fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4cc09f600 0x7fa4cc09fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 -- 172.21.15.110:0/1644828071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc15adf0 con 0x7fa4cc1f7270 2024-09-10T09:57:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4cbe00640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4cc09f600 0x7fa4cc09fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4caa00640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4cc1f7270 0x7fa4cc15a480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8007290 tx=0x7fa4b8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4c3400640 1 -- 172.21.15.110:0/1644828071 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b800f940 con 0x7fa4cc1f7270 2024-09-10T09:57:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4c3400640 1 -- 172.21.15.110:0/1644828071 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b8018070 con 0x7fa4cc1f7270 2024-09-10T09:57:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc15b0d0 con 0x7fa4cc1f7270 2024-09-10T09:57:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4c3400640 1 -- 172.21.15.110:0/1644828071 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b800fd00 con 0x7fa4cc1f7270 2024-09-10T09:57:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc15b690 con 0x7fa4cc1f7270 2024-09-10T09:57:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4c3400640 1 -- 172.21.15.110:0/1644828071 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa4b801d460 con 0x7fa4cc1f7270 2024-09-10T09:57:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4c3400640 1 -- 172.21.15.110:0/1644828071 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa4b80137f0 con 0x7fa4cc1f7270 2024-09-10T09:57:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.342+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc0b2940 con 0x7fa4cc1f7270 2024-09-10T09:57:58.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.346+0000 7fa4c3400640 1 -- 172.21.15.110:0/1644828071 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa4cc0b5f30 con 0x7fa4cc1f7270 2024-09-10T09:57:58.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa4cc0b2470 con 0x7fa4cc1f7270 2024-09-10T09:57:58.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4c3400640 1 -- 172.21.15.110:0/1644828071 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fa4b8013a40 con 0x7fa4cc1f7270 2024-09-10T09:57:58.414 INFO:teuthology.orchestra.run.smithi110.stderr:adjusted tunables profile to default 2024-09-10T09:57:58.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4cc1f7270 msgr2=0x7fa4cc15a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:58.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4cc1f7270 0x7fa4cc15a480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8007290 tx=0x7fa4b8007750 comp rx=0 tx=0).stop 2024-09-10T09:57:58.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 shutdown_connections 2024-09-10T09:57:58.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4cc09f600 0x7fa4cc09fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4cc1f7270 0x7fa4cc15a480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 --2- 172.21.15.110:0/1644828071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4b0004d50 0x7fa4cc1f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:58.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 >> 172.21.15.110:0/1644828071 conn(0x7fa4cc09b9d0 msgr2=0x7fa4cc0b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:58.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 shutdown_connections 2024-09-10T09:57:58.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:57:58.410+0000 7fa4d2400640 1 -- 172.21.15.110:0/1644828071 wait complete. 2024-09-10T09:57:58.433 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T09:57:58.433 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T09:57:58.434 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:58.434 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T09:57:58.453 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:58.453 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T09:57:58.518 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:58.518 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T09:57:58.539 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:57:58.534+0000 7f31602d7180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:57:58.552 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:57:58.547+0000 7f966384f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:57:58.557 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:57:58.557 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T09:57:58.618 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:58.618 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T09:57:58.637 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:58.637 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T09:57:58.703 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:58.703 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T09:57:58.763 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:57:58.763 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T09:57:58.827 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 936d59a8-3d51-4d57-b195-7d9187cf8116 0 2024-09-10T09:57:59.006 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.003+0000 7ff3e6000640 1 Processor -- start 2024-09-10T09:57:59.006 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.003+0000 7ff3e6000640 1 -- start start 2024-09-10T09:57:59.009 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.003+0000 7ff3e6000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3e015c820 0x7ff3e0071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.009 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.003+0000 7ff3e6000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff3e0071980 0x7ff3e0073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.009 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.003+0000 7ff3e6000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e00737d0 0x7ff3e0073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.009 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.003+0000 7ff3e6000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e00b5730 con 0x7ff3e0071980 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.003+0000 7ff3e6000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e00b5a10 con 0x7ff3e015c820 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.003+0000 7ff3e6000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e00b5cf0 con 0x7ff3e00737d0 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3e015c820 0x7ff3e0071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3e015c820 0x7ff3e0071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46746/0 (socket says 172.21.15.145:46746) 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 -- 172.21.15.145:0/1102187826 learned_addr learned my addr 172.21.15.145:0/1102187826 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3dfe00640 1 --2- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e00737d0 0x7ff3e0073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3d7e00640 1 --2- 172.21.15.145:0/1102187826 >> v2:172.21.15.110:3300/0 conn(0x7ff3e0071980 0x7ff3e0073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 -- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e00737d0 msgr2=0x7ff3e0073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 --2- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e00737d0 0x7ff3e0073c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 -- 172.21.15.145:0/1102187826 >> v2:172.21.15.110:3300/0 conn(0x7ff3e0071980 msgr2=0x7ff3e0073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 --2- 172.21.15.145:0/1102187826 >> v2:172.21.15.110:3300/0 conn(0x7ff3e0071980 0x7ff3e0073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 -- 172.21.15.145:0/1102187826 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e00b5fd0 con 0x7ff3e015c820 2024-09-10T09:57:59.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3d7e00640 1 --2- 172.21.15.145:0/1102187826 >> v2:172.21.15.110:3300/0 conn(0x7ff3e0071980 0x7ff3e0073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:59.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3dfe00640 1 --2- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e00737d0 0x7ff3e0073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:59.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3df400640 1 --2- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e015c820 0x7ff3e0071440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0069500 tx=0x7ff3d001c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bb37e601b8ea9ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3dea00640 1 -- 172.21.15.145:0/1102187826 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00aa430 con 0x7ff3e015c820 2024-09-10T09:57:59.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3dea00640 1 -- 172.21.15.145:0/1102187826 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d00af070 con 0x7ff3e015c820 2024-09-10T09:57:59.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.007+0000 7ff3dea00640 1 -- 172.21.15.145:0/1102187826 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00aa7f0 con 0x7ff3e015c820 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 -- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e015c820 msgr2=0x7ff3e0071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 --2- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e015c820 0x7ff3e0071440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0069500 tx=0x7ff3d001c4b0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 -- 172.21.15.145:0/1102187826 shutdown_connections 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 --2- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e00737d0 0x7ff3e0073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 --2- 172.21.15.145:0/1102187826 >> v2:172.21.15.110:3300/0 conn(0x7ff3e0071980 0x7ff3e0073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 --2- 172.21.15.145:0/1102187826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e015c820 0x7ff3e0071440 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 -- 172.21.15.145:0/1102187826 >> 172.21.15.145:0/1102187826 conn(0x7ff3e00d25a0 msgr2=0x7ff3e00d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 -- 172.21.15.145:0/1102187826 shutdown_connections 2024-09-10T09:57:59.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 -- 172.21.15.145:0/1102187826 wait complete. 2024-09-10T09:57:59.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 Processor -- start 2024-09-10T09:57:59.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.011+0000 7ff3e6000640 1 -- start start 2024-09-10T09:57:59.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e0071980 0x7ff3e0143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e00737d0 0x7ff3e006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e015c820 0x7ff3e0070830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e00d0e40 con 0x7ff3e00737d0 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e00d0cc0 con 0x7ff3e0071980 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e00d0fc0 con 0x7ff3e015c820 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e0071980 0x7ff3e0143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e0071980 0x7ff3e0143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46756/0 (socket says 172.21.15.145:46756) 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 -- 172.21.15.145:0/1521714085 learned_addr learned my addr 172.21.15.145:0/1521714085 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3dfe00640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e015c820 0x7ff3e0070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3d7e00640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e00737d0 0x7ff3e006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 -- 172.21.15.145:0/1521714085 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e015c820 msgr2=0x7ff3e0070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e015c820 0x7ff3e0070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 -- 172.21.15.145:0/1521714085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e00737d0 msgr2=0x7ff3e006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e00737d0 0x7ff3e006d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 -- 172.21.15.145:0/1521714085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e015f1d0 con 0x7ff3e0071980 2024-09-10T09:57:59.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3df400640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e0071980 0x7ff3e0143ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff3d001ab60 tx=0x7ff3d001acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3d7400640 1 -- 172.21.15.145:0/1521714085 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00bd5f0 con 0x7ff3e0071980 2024-09-10T09:57:59.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3d7400640 1 -- 172.21.15.145:0/1521714085 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d00bd7b0 con 0x7ff3e0071980 2024-09-10T09:57:59.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3d7400640 1 -- 172.21.15.145:0/1521714085 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00bdba0 con 0x7ff3e0071980 2024-09-10T09:57:59.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e015dad0 con 0x7ff3e0071980 2024-09-10T09:57:59.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e015f730 con 0x7ff3e0071980 2024-09-10T09:57:59.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3d7400640 1 -- 172.21.15.145:0/1521714085 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff3d00b3780 con 0x7ff3e0071980 2024-09-10T09:57:59.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3d7400640 1 -- 172.21.15.145:0/1521714085 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7ff3d00c77b0 con 0x7ff3e0071980 2024-09-10T09:57:59.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.015+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e0073290 con 0x7ff3e0071980 2024-09-10T09:57:59.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.019+0000 7ff3d7400640 1 -- 172.21.15.145:0/1521714085 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff3e015f1d0 con 0x7ff3e0071980 2024-09-10T09:57:59.050 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:57:59.046+0000 7f31602d7180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:57:59.082 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:57:59.079+0000 7f966384f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:57:59.091 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.087+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd new", "uuid": "936d59a8-3d51-4d57-b195-7d9187cf8116", "id": 0} v 0) -- 0x7ff3e00cea30 con 0x7ff3e0071980 2024-09-10T09:57:59.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3d7400640 1 -- 172.21.15.145:0/1521714085 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "936d59a8-3d51-4d57-b195-7d9187cf8116", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7ff3d00160d0 con 0x7ff3e0071980 2024-09-10T09:57:59.097 INFO:teuthology.orchestra.run.smithi145.stdout:0 2024-09-10T09:57:59.098 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e0071980 msgr2=0x7ff3e0143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e0071980 0x7ff3e0143ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff3d001ab60 tx=0x7ff3d001acf0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 shutdown_connections 2024-09-10T09:57:59.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e015c820 0x7ff3e0070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e00737d0 0x7ff3e006d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 --2- 172.21.15.145:0/1521714085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e0071980 0x7ff3e0143ee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 >> 172.21.15.145:0/1521714085 conn(0x7ff3e00d25a0 msgr2=0x7ff3e00b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 shutdown_connections 2024-09-10T09:57:59.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.095+0000 7ff3e6000640 1 -- 172.21.15.145:0/1521714085 wait complete. 2024-09-10T09:57:59.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 80b34968-15c0-4a88-864f-d59bb573fc12 1 2024-09-10T09:57:59.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.247+0000 7ff29ca00640 1 Processor -- start 2024-09-10T09:57:59.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.247+0000 7ff29ca00640 1 -- start start 2024-09-10T09:57:59.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff29ca00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff29815e940 0x7ff2980b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff29ca00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff2980d20b0 0x7ff2980b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff29ca00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff298071470 con 0x7ff2980d1b60 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff29ca00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff29806cd60 con 0x7ff2980d20b0 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff29ca00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff2980b7530 con 0x7ff29815e940 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff295e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff29815e940 0x7ff2980b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff295e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff29815e940 0x7ff2980b6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46770/0 (socket says 172.21.15.145:46770) 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff295e00640 1 -- 172.21.15.145:0/2547916577 learned_addr learned my addr 172.21.15.145:0/2547916577 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff28fe00640 1 --2- 172.21.15.145:0/2547916577 >> v2:172.21.15.110:3300/0 conn(0x7ff2980d20b0 0x7ff2980b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff295e00640 1 -- 172.21.15.145:0/2547916577 >> v2:172.21.15.110:3300/0 conn(0x7ff2980d20b0 msgr2=0x7ff2980b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff295e00640 1 --2- 172.21.15.145:0/2547916577 >> v2:172.21.15.110:3300/0 conn(0x7ff2980d20b0 0x7ff2980b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff295e00640 1 -- 172.21.15.145:0/2547916577 >> v1:172.21.15.110:6789/0 conn(0x7ff2980d1b60 legacy=0x7ff298073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff295e00640 1 -- 172.21.15.145:0/2547916577 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff298071710 con 0x7ff29815e940 2024-09-10T09:57:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.251+0000 7ff295e00640 1 --2- 172.21.15.145:0/2547916577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff29815e940 0x7ff2980b6ab0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff2980703e0 tx=0x7ff28801c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee5f0fe58ccb612 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295400640 1 -- 172.21.15.145:0/2547916577 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880b1070 con 0x7ff29815e940 2024-09-10T09:57:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295400640 1 -- 172.21.15.145:0/2547916577 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2880ac600 con 0x7ff29815e940 2024-09-10T09:57:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295400640 1 -- 172.21.15.145:0/2547916577 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880ac9d0 con 0x7ff29815e940 2024-09-10T09:57:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- 172.21.15.145:0/2547916577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff29815e940 msgr2=0x7ff2980b6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 --2- 172.21.15.145:0/2547916577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff29815e940 0x7ff2980b6ab0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff2980703e0 tx=0x7ff28801c490 comp rx=0 tx=0).stop 2024-09-10T09:57:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- 172.21.15.145:0/2547916577 shutdown_connections 2024-09-10T09:57:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 --2- 172.21.15.145:0/2547916577 >> v2:172.21.15.110:3300/0 conn(0x7ff2980d20b0 0x7ff2980b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 --2- 172.21.15.145:0/2547916577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff29815e940 0x7ff2980b6ab0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- 172.21.15.145:0/2547916577 >> 172.21.15.145:0/2547916577 conn(0x7ff2980d25a0 msgr2=0x7ff2980d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- 172.21.15.145:0/2547916577 shutdown_connections 2024-09-10T09:57:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- 172.21.15.145:0/2547916577 wait complete. 2024-09-10T09:57:59.258 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:57:59.254+0000 7f31602d7180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:57:59.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 Processor -- start 2024-09-10T09:57:59.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- start start 2024-09-10T09:57:59.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2980d1b60 0x7ff29814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2980d20b0 0x7ff29814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff29815e940 0x7ff29814a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff29816b4b0 con 0x7ff2980d20b0 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff29816b330 con 0x7ff2980d1b60 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff29ca00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff29816b630 con 0x7ff29815e940 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2980d1b60 0x7ff29814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2980d1b60 0x7ff29814f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46780/0 (socket says 172.21.15.145:46780) 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff296800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff29815e940 0x7ff29814a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295e00640 1 -- 172.21.15.145:0/3342599653 learned_addr learned my addr 172.21.15.145:0/3342599653 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295e00640 1 -- 172.21.15.145:0/3342599653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff29815e940 msgr2=0x7ff29814a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295e00640 1 --2- 172.21.15.145:0/3342599653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff29815e940 0x7ff29814a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295e00640 1 -- 172.21.15.145:0/3342599653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2980d20b0 msgr2=0x7ff29814cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.255+0000 7ff295e00640 1 --2- 172.21.15.145:0/3342599653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2980d20b0 0x7ff29814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff295e00640 1 -- 172.21.15.145:0/3342599653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29814a610 con 0x7ff2980d1b60 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff295e00640 1 --2- 172.21.15.145:0/3342599653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2980d1b60 0x7ff29814f4f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff2880190a0 tx=0x7ff28801acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff28e000640 1 -- 172.21.15.145:0/3342599653 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880bb020 con 0x7ff2980d1b60 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff28e000640 1 -- 172.21.15.145:0/3342599653 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2880b7420 con 0x7ff2980d1b60 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff28e000640 1 -- 172.21.15.145:0/3342599653 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880b1040 con 0x7ff2980d1b60 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29814bfe0 con 0x7ff2980d1b60 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29814c5a0 con 0x7ff2980d1b60 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff28e000640 1 -- 172.21.15.145:0/3342599653 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff2880be020 con 0x7ff2980d1b60 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff28e000640 1 -- 172.21.15.145:0/3342599653 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7ff2880c09f0 con 0x7ff2980d1b60 2024-09-10T09:57:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.259+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff298073290 con 0x7ff2980d1b60 2024-09-10T09:57:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.263+0000 7ff28e000640 1 -- 172.21.15.145:0/3342599653 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff2880c5030 con 0x7ff2980d1b60 2024-09-10T09:57:59.309 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:57:59.307+0000 7f966384f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:57:59.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.331+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd new", "uuid": "80b34968-15c0-4a88-864f-d59bb573fc12", "id": 1} v 0) -- 0x7ff298153ba0 con 0x7ff2980d1b60 2024-09-10T09:57:59.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.331+0000 7ff28e000640 1 -- 172.21.15.145:0/3342599653 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80b34968-15c0-4a88-864f-d59bb573fc12", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7ff29814a610 con 0x7ff2980d1b60 2024-09-10T09:57:59.337 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2980d1b60 msgr2=0x7ff29814f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 --2- 172.21.15.145:0/3342599653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2980d1b60 0x7ff29814f4f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff2880190a0 tx=0x7ff28801acd0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 shutdown_connections 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 --2- 172.21.15.145:0/3342599653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff29815e940 0x7ff29814a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 --2- 172.21.15.145:0/3342599653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2980d20b0 0x7ff29814cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 --2- 172.21.15.145:0/3342599653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2980d1b60 0x7ff29814f4f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 >> 172.21.15.145:0/3342599653 conn(0x7ff2980d25a0 msgr2=0x7ff298152e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 shutdown_connections 2024-09-10T09:57:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.335+0000 7ff29ca00640 1 -- 172.21.15.145:0/3342599653 wait complete. 2024-09-10T09:57:59.356 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 5319529e-6347-4786-9daf-ad1857d2912b 2 2024-09-10T09:57:59.490 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f91a00640 1 Processor -- start 2024-09-10T09:57:59.490 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f91a00640 1 -- start start 2024-09-10T09:57:59.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f91a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c162c30 0x7f6f8c073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f91a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6f8c070a60 0x7f6f8c0737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f91a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c163010 con 0x7f6f8c0b1b90 2024-09-10T09:57:59.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f91a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8c070ef0 con 0x7f6f8c070a60 2024-09-10T09:57:59.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f91a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c0711d0 con 0x7f6f8c162c30 2024-09-10T09:57:59.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f8aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c162c30 0x7f6f8c073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f82000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6f8c070a60 0x7f6f8c0737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f8aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c162c30 0x7f6f8c073290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:41058/0 (socket says 172.21.15.145:41058) 2024-09-10T09:57:59.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f8aa00640 1 -- 172.21.15.145:0/194344619 learned_addr learned my addr 172.21.15.145:0/194344619 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f82000640 1 -- 172.21.15.145:0/194344619 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c162c30 msgr2=0x7f6f8c073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:59.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f82000640 1 --2- 172.21.15.145:0/194344619 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c162c30 0x7f6f8c073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f82000640 1 -- 172.21.15.145:0/194344619 >> v1:172.21.15.145:6789/0 conn(0x7f6f8c0b1b90 legacy=0x7f6f8c16b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f82000640 1 -- 172.21.15.145:0/194344619 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c0714b0 con 0x7f6f8c070a60 2024-09-10T09:57:59.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.487+0000 7f6f8aa00640 1 --2- 172.21.15.145:0/194344619 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c162c30 0x7f6f8c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:59.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f82000640 1 --2- 172.21.15.145:0/194344619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c070a60 0x7f6f8c0737d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f740097c0 tx=0x7f6f7401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22bc5de06e8100d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f8a000640 1 -- 172.21.15.145:0/194344619 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f740b1070 con 0x7f6f8c070a60 2024-09-10T09:57:59.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f8a000640 1 -- 172.21.15.145:0/194344619 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f740ac410 con 0x7f6f8c070a60 2024-09-10T09:57:59.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f8a000640 1 -- 172.21.15.145:0/194344619 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f740ac7e0 con 0x7f6f8c070a60 2024-09-10T09:57:59.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f91a00640 1 -- 172.21.15.145:0/194344619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c070a60 msgr2=0x7f6f8c0737d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f91a00640 1 --2- 172.21.15.145:0/194344619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c070a60 0x7f6f8c0737d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f740097c0 tx=0x7f6f7401c4a0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f91a00640 1 -- 172.21.15.145:0/194344619 shutdown_connections 2024-09-10T09:57:59.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f91a00640 1 --2- 172.21.15.145:0/194344619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c070a60 0x7f6f8c0737d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f91a00640 1 --2- 172.21.15.145:0/194344619 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c162c30 0x7f6f8c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f91a00640 1 -- 172.21.15.145:0/194344619 >> 172.21.15.145:0/194344619 conn(0x7f6f8c0d25a0 msgr2=0x7f6f8c0d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f91a00640 1 -- 172.21.15.145:0/194344619 shutdown_connections 2024-09-10T09:57:59.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.491+0000 7f6f91a00640 1 -- 172.21.15.145:0/194344619 wait complete. 2024-09-10T09:57:59.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 Processor -- start 2024-09-10T09:57:59.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 -- start start 2024-09-10T09:57:59.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0b1b90 0x7f6f8c1fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c162c30 0x7f6f8c146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c15a8a0 0x7f6f8c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c16f8d0 con 0x7f6f8c0b1b90 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0b78b0 con 0x7f6f8c162c30 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c16fa50 con 0x7f6f8c15a8a0 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c162c30 0x7f6f8c146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f8b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c15a8a0 0x7f6f8c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c162c30 0x7f6f8c146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46794/0 (socket says 172.21.15.145:46794) 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 -- 172.21.15.145:0/1885257571 learned_addr learned my addr 172.21.15.145:0/1885257571 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f8aa00640 1 --2- 172.21.15.145:0/1885257571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0b1b90 0x7f6f8c1fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 -- 172.21.15.145:0/1885257571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c15a8a0 msgr2=0x7f6f8c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 --2- 172.21.15.145:0/1885257571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c15a8a0 0x7f6f8c15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 -- 172.21.15.145:0/1885257571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0b1b90 msgr2=0x7f6f8c1fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 --2- 172.21.15.145:0/1885257571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0b1b90 0x7f6f8c1fbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 -- 172.21.15.145:0/1885257571 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c1590b0 con 0x7f6f8c162c30 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f82000640 1 --2- 172.21.15.145:0/1885257571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c162c30 0x7f6f8c146860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f740af040 tx=0x7f6f74019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f83400640 1 -- 172.21.15.145:0/1885257571 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f740b1040 con 0x7f6f8c162c30 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f83400640 1 -- 172.21.15.145:0/1885257571 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f740ba070 con 0x7f6f8c162c30 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f83400640 1 -- 172.21.15.145:0/1885257571 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f740b5690 con 0x7f6f8c162c30 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c159390 con 0x7f6f8c162c30 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.495+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c160760 con 0x7f6f8c162c30 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.499+0000 7f6f83400640 1 -- 172.21.15.145:0/1885257571 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6f740b5830 con 0x7f6f8c162c30 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.499+0000 7f6f83400640 1 -- 172.21.15.145:0/1885257571 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f6f740bf820 con 0x7f6f8c162c30 2024-09-10T09:57:59.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.499+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c073290 con 0x7f6f8c162c30 2024-09-10T09:57:59.504 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.499+0000 7f6f83400640 1 -- 172.21.15.145:0/1885257571 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6f74016090 con 0x7f6f8c162c30 2024-09-10T09:57:59.573 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.571+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5319529e-6347-4786-9daf-ad1857d2912b", "id": 2} v 0) -- 0x7f6f8c0b1970 con 0x7f6f8c162c30 2024-09-10T09:57:59.578 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f83400640 1 -- 172.21.15.145:0/1885257571 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5319529e-6347-4786-9daf-ad1857d2912b", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f6f740b5b20 con 0x7f6f8c162c30 2024-09-10T09:57:59.578 INFO:teuthology.orchestra.run.smithi145.stdout:2 2024-09-10T09:57:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c162c30 msgr2=0x7f6f8c146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 --2- 172.21.15.145:0/1885257571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c162c30 0x7f6f8c146860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f740af040 tx=0x7f6f74019240 comp rx=0 tx=0).stop 2024-09-10T09:57:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 shutdown_connections 2024-09-10T09:57:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 --2- 172.21.15.145:0/1885257571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c15a8a0 0x7f6f8c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 --2- 172.21.15.145:0/1885257571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c162c30 0x7f6f8c146860 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 --2- 172.21.15.145:0/1885257571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0b1b90 0x7f6f8c1fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 >> 172.21.15.145:0/1885257571 conn(0x7f6f8c0d25a0 msgr2=0x7f6f8c163db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 shutdown_connections 2024-09-10T09:57:59.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.575+0000 7f6f91a00640 1 -- 172.21.15.145:0/1885257571 wait complete. 2024-09-10T09:57:59.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 4a19b2ea-bfc7-4cec-a6d2-e837a89b2260 3 2024-09-10T09:57:59.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5341a00640 1 Processor -- start 2024-09-10T09:57:59.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5341a00640 1 -- start start 2024-09-10T09:57:59.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5341a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f533c15e940 0x7f533c0b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5341a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f533c0d20b0 0x7f533c0b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5341a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d1b60 0x7f533c073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5341a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f533c06cd60 con 0x7f533c15e940 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5341a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f533c0b7530 con 0x7f533c0d20b0 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5341a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f533c071470 con 0x7f533c0d1b60 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f533b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d1b60 0x7f533c073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5332000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f533c0d20b0 0x7f533c0b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5332000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f533c0d20b0 0x7f533c0b6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46802/0 (socket says 172.21.15.145:46802) 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5332000640 1 -- 172.21.15.145:0/337865026 learned_addr learned my addr 172.21.15.145:0/337865026 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f533aa00640 1 --2- 172.21.15.145:0/337865026 >> v2:172.21.15.110:3300/0 conn(0x7f533c15e940 0x7f533c0b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5332000640 1 -- 172.21.15.145:0/337865026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d1b60 msgr2=0x7f533c073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5332000640 1 --2- 172.21.15.145:0/337865026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d1b60 0x7f533c073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5332000640 1 -- 172.21.15.145:0/337865026 >> v2:172.21.15.110:3300/0 conn(0x7f533c15e940 msgr2=0x7f533c0b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5332000640 1 --2- 172.21.15.145:0/337865026 >> v2:172.21.15.110:3300/0 conn(0x7f533c15e940 0x7f533c0b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f5332000640 1 -- 172.21.15.145:0/337865026 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c071710 con 0x7f533c0d20b0 2024-09-10T09:57:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.735+0000 7f533b400640 1 --2- 172.21.15.145:0/337865026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d1b60 0x7f533c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:59.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5332000640 1 --2- 172.21.15.145:0/337865026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d20b0 0x7f533c0b6ff0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f533c050030 tx=0x7f532401c1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbb9101f36e4d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f533a000640 1 -- 172.21.15.145:0/337865026 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240b0070 con 0x7f533c0d20b0 2024-09-10T09:57:59.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f533a000640 1 -- 172.21.15.145:0/337865026 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53240ab600 con 0x7f533c0d20b0 2024-09-10T09:57:59.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f533a000640 1 -- 172.21.15.145:0/337865026 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240ab9d0 con 0x7f533c0d20b0 2024-09-10T09:57:59.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 -- 172.21.15.145:0/337865026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d20b0 msgr2=0x7f533c0b6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 --2- 172.21.15.145:0/337865026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d20b0 0x7f533c0b6ff0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f533c050030 tx=0x7f532401c1d0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 -- 172.21.15.145:0/337865026 shutdown_connections 2024-09-10T09:57:59.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 --2- 172.21.15.145:0/337865026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d1b60 0x7f533c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 --2- 172.21.15.145:0/337865026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d20b0 0x7f533c0b6ff0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 --2- 172.21.15.145:0/337865026 >> v2:172.21.15.110:3300/0 conn(0x7f533c15e940 0x7f533c0b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 -- 172.21.15.145:0/337865026 >> 172.21.15.145:0/337865026 conn(0x7f533c0d25a0 msgr2=0x7f533c0d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 -- 172.21.15.145:0/337865026 shutdown_connections 2024-09-10T09:57:59.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.739+0000 7f5341a00640 1 -- 172.21.15.145:0/337865026 wait complete. 2024-09-10T09:57:59.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 Processor -- start 2024-09-10T09:57:59.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 -- start start 2024-09-10T09:57:59.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d1b60 0x7f533c143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d20b0 0x7f533c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f533c15e940 0x7f533c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f533c06d930 con 0x7f533c15e940 2024-09-10T09:57:59.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f533c06d7b0 con 0x7f533c0d1b60 2024-09-10T09:57:59.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f533c06dab0 con 0x7f533c0d20b0 2024-09-10T09:57:59.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d20b0 0x7f533c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f533aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d1b60 0x7f533c143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d20b0 0x7f533c15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:41074/0 (socket says 172.21.15.145:41074) 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 -- 172.21.15.145:0/2257298815 learned_addr learned my addr 172.21.15.145:0/2257298815 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f533b400640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f533c15e940 0x7f533c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 -- 172.21.15.145:0/2257298815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d1b60 msgr2=0x7f533c143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d1b60 0x7f533c143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 -- 172.21.15.145:0/2257298815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f533c15e940 msgr2=0x7f533c15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f533c15e940 0x7f533c15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 -- 172.21.15.145:0/2257298815 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c14bca0 con 0x7f533c0d20b0 2024-09-10T09:57:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f533aa00640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d1b60 0x7f533c143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:59.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5332000640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d20b0 0x7f533c15b6b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f53240ae040 tx=0x7f5324018f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5333400640 1 -- 172.21.15.145:0/2257298815 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240b0040 con 0x7f533c0d20b0 2024-09-10T09:57:59.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5333400640 1 -- 172.21.15.145:0/2257298815 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53240b4420 con 0x7f533c0d20b0 2024-09-10T09:57:59.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5333400640 1 -- 172.21.15.145:0/2257298815 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240b4810 con 0x7f533c0d20b0 2024-09-10T09:57:59.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533c14bf80 con 0x7f533c0d20b0 2024-09-10T09:57:59.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f533c157380 con 0x7f533c0d20b0 2024-09-10T09:57:59.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5333400640 1 -- 172.21.15.145:0/2257298815 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53240b49b0 con 0x7f533c0d20b0 2024-09-10T09:57:59.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.743+0000 7f5333400640 1 -- 172.21.15.145:0/2257298815 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f53240b4ca0 con 0x7f533c0d20b0 2024-09-10T09:57:59.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.747+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f533c073290 con 0x7f533c0d20b0 2024-09-10T09:57:59.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.747+0000 7f5333400640 1 -- 172.21.15.145:0/2257298815 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5324016110 con 0x7f533c0d20b0 2024-09-10T09:57:59.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.815+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4a19b2ea-bfc7-4cec-a6d2-e837a89b2260", "id": 3} v 0) -- 0x7f533c15cff0 con 0x7f533c0d20b0 2024-09-10T09:57:59.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.819+0000 7f5333400640 1 -- 172.21.15.145:0/2257298815 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a19b2ea-bfc7-4cec-a6d2-e837a89b2260", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f53240bd400 con 0x7f533c0d20b0 2024-09-10T09:57:59.824 INFO:teuthology.orchestra.run.smithi145.stdout:3 2024-09-10T09:57:59.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d20b0 msgr2=0x7f533c15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d20b0 0x7f533c15b6b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f53240ae040 tx=0x7f5324018f70 comp rx=0 tx=0).stop 2024-09-10T09:57:59.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 shutdown_connections 2024-09-10T09:57:59.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f533c15e940 0x7f533c15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f533c0d20b0 0x7f533c15b6b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 --2- 172.21.15.145:0/2257298815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f533c0d1b60 0x7f533c143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 >> 172.21.15.145:0/2257298815 conn(0x7f533c0d25a0 msgr2=0x7f533c14c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 shutdown_connections 2024-09-10T09:57:59.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.823+0000 7f5341a00640 1 -- 172.21.15.145:0/2257298815 wait complete. 2024-09-10T09:57:59.844 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 2ff23641-e17c-4abe-a494-ab882bf527b8 4 2024-09-10T09:57:59.897 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:57:59.894+0000 7f31602d7180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:57:59.958 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:57:59.954+0000 7f31602d7180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:57:59.967 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:57:59.963+0000 7f966384f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:57:59.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.975+0000 7fbf76600640 1 Processor -- start 2024-09-10T09:57:59.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.975+0000 7fbf76600640 1 -- start start 2024-09-10T09:57:59.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf76600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700d20b0 0x7fbf7006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf76600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbf700b71c0 0x7fbf7006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf76600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf70071470 con 0x7fbf7015e940 2024-09-10T09:57:59.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf76600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7006cd60 con 0x7fbf700b71c0 2024-09-10T09:57:59.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf76600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbf70071710 con 0x7fbf700d20b0 2024-09-10T09:57:59.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700d20b0 0x7fbf7006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700d20b0 0x7fbf7006d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:41090/0 (socket says 172.21.15.145:41090) 2024-09-10T09:57:59.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf67e00640 1 -- 172.21.15.145:0/4267409125 learned_addr learned my addr 172.21.15.145:0/4267409125 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf6fe00640 1 --2- 172.21.15.145:0/4267409125 >> v2:172.21.15.110:3300/0 conn(0x7fbf700b71c0 0x7fbf7006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf67e00640 1 -- 172.21.15.145:0/4267409125 >> v2:172.21.15.110:3300/0 conn(0x7fbf700b71c0 msgr2=0x7fbf7006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:59.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf67e00640 1 --2- 172.21.15.145:0/4267409125 >> v2:172.21.15.110:3300/0 conn(0x7fbf700b71c0 0x7fbf7006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf67e00640 1 -- 172.21.15.145:0/4267409125 >> v1:172.21.15.110:6789/0 conn(0x7fbf7015e940 legacy=0x7fbf700b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf67e00640 1 -- 172.21.15.145:0/4267409125 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf700719f0 con 0x7fbf700d20b0 2024-09-10T09:57:59.984 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf67e00640 1 --2- 172.21.15.145:0/4267409125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700d20b0 0x7fbf7006d7b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009870 tx=0x7fbf5801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8825dc34baa2dbdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf6ea00640 1 -- 172.21.15.145:0/4267409125 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf580ab660 con 0x7fbf700d20b0 2024-09-10T09:57:59.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf6ea00640 1 -- 172.21.15.145:0/4267409125 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf580b0070 con 0x7fbf700d20b0 2024-09-10T09:57:59.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.979+0000 7fbf6ea00640 1 -- 172.21.15.145:0/4267409125 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf580aba20 con 0x7fbf700d20b0 2024-09-10T09:57:59.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- 172.21.15.145:0/4267409125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700d20b0 msgr2=0x7fbf7006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 --2- 172.21.15.145:0/4267409125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700d20b0 0x7fbf7006d7b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009870 tx=0x7fbf5801c440 comp rx=0 tx=0).stop 2024-09-10T09:57:59.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- 172.21.15.145:0/4267409125 shutdown_connections 2024-09-10T09:57:59.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 --2- 172.21.15.145:0/4267409125 >> v2:172.21.15.110:3300/0 conn(0x7fbf700b71c0 0x7fbf7006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 --2- 172.21.15.145:0/4267409125 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700d20b0 0x7fbf7006d7b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- 172.21.15.145:0/4267409125 >> 172.21.15.145:0/4267409125 conn(0x7fbf700d25a0 msgr2=0x7fbf700d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- 172.21.15.145:0/4267409125 shutdown_connections 2024-09-10T09:57:59.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- 172.21.15.145:0/4267409125 wait complete. 2024-09-10T09:57:59.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 Processor -- start 2024-09-10T09:57:59.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- start start 2024-09-10T09:57:59.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700b71c0 0x7fbf7014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf700d20b0 0x7fbf70158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7015e940 0x7fbf70158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7016b6c0 con 0x7fbf700d20b0 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7016b540 con 0x7fbf7015e940 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf76600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7016b840 con 0x7fbf700b71c0 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf6f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700b71c0 0x7fbf7014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7015e940 0x7fbf70158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf6f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700b71c0 0x7fbf7014f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:41106/0 (socket says 172.21.15.145:41106) 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf6f400640 1 -- 172.21.15.145:0/3793284971 learned_addr learned my addr 172.21.15.145:0/3793284971 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.983+0000 7fbf67e00640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf700d20b0 0x7fbf70158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf6f400640 1 -- 172.21.15.145:0/3793284971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7015e940 msgr2=0x7fbf70158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf6f400640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7015e940 0x7fbf70158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf6f400640 1 -- 172.21.15.145:0/3793284971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf700d20b0 msgr2=0x7fbf70158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf6f400640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf700d20b0 0x7fbf70158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf6f400640 1 -- 172.21.15.145:0/3793284971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf70160470 con 0x7fbf700b71c0 2024-09-10T09:57:59.991 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf67e00640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf700d20b0 0x7fbf70158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:59.991 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf6f400640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700b71c0 0x7fbf7014f0a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c500 tx=0x7fbf6000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.991 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf67400640 1 -- 172.21.15.145:0/3793284971 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60016020 con 0x7fbf700b71c0 2024-09-10T09:57:59.991 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf67400640 1 -- 172.21.15.145:0/3793284971 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf60005220 con 0x7fbf700b71c0 2024-09-10T09:57:59.991 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf67400640 1 -- 172.21.15.145:0/3793284971 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60007030 con 0x7fbf700b71c0 2024-09-10T09:57:59.991 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf70160750 con 0x7fbf700b71c0 2024-09-10T09:57:59.991 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf70160d10 con 0x7fbf700b71c0 2024-09-10T09:57:59.991 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf67400640 1 -- 172.21.15.145:0/3793284971 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf600067e0 con 0x7fbf700b71c0 2024-09-10T09:57:59.992 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf67400640 1 -- 172.21.15.145:0/3793284971 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fbf60007580 con 0x7fbf700b71c0 2024-09-10T09:57:59.992 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.987+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf700b6ab0 con 0x7fbf700b71c0 2024-09-10T09:57:59.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:57:59.991+0000 7fbf67400640 1 -- 172.21.15.145:0/3793284971 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbf60020020 con 0x7fbf700b71c0 2024-09-10T09:58:00.019 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:00.018+0000 7f31602d7180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:58:00.033 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:00.031+0000 7f966384f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:58:00.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.059+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2ff23641-e17c-4abe-a494-ab882bf527b8", "id": 4} v 0) -- 0x7fbf70150310 con 0x7fbf700b71c0 2024-09-10T09:58:00.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.063+0000 7fbf67400640 1 -- 172.21.15.145:0/3793284971 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2ff23641-e17c-4abe-a494-ab882bf527b8", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fbf60012d40 con 0x7fbf700b71c0 2024-09-10T09:58:00.068 INFO:teuthology.orchestra.run.smithi145.stdout:4 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.063+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700b71c0 msgr2=0x7fbf7014f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.063+0000 7fbf76600640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700b71c0 0x7fbf7014f0a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c500 tx=0x7fbf6000c9c0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.067+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 shutdown_connections 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.067+0000 7fbf76600640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf7015e940 0x7fbf70158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.067+0000 7fbf76600640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf700d20b0 0x7fbf70158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.067+0000 7fbf76600640 1 --2- 172.21.15.145:0/3793284971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf700b71c0 0x7fbf7014f0a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.067+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 >> 172.21.15.145:0/3793284971 conn(0x7fbf700d25a0 msgr2=0x7fbf700b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.067+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 shutdown_connections 2024-09-10T09:58:00.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.067+0000 7fbf76600640 1 -- 172.21.15.145:0/3793284971 wait complete. 2024-09-10T09:58:00.087 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new ddbaf2e3-cfba-4c75-a56d-0c32e415dd17 5 2024-09-10T09:58:00.098 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:00.095+0000 7f966384f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:58:00.109 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:00.106+0000 7f31602d7180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:58:00.169 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:00.166+0000 7f31602d7180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:58:00.189 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:00.187+0000 7f966384f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:58:00.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ea00640 1 Processor -- start 2024-09-10T09:58:00.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ea00640 1 -- start start 2024-09-10T09:58:00.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5d780b71c0 0x7f5d7806dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ea00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d78071470 con 0x7f5d7815e940 2024-09-10T09:58:00.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ea00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d78071710 con 0x7f5d780d20b0 2024-09-10T09:58:00.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ea00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5d7806cd60 con 0x7f5d780b71c0 2024-09-10T09:58:00.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ca00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5d780b71c0 0x7f5d7806dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ca00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5d780b71c0 0x7f5d7806dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46820/0 (socket says 172.21.15.145:46820) 2024-09-10T09:58:00.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ca00640 1 -- 172.21.15.145:0/2807682021 learned_addr learned my addr 172.21.15.145:0/2807682021 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:58:00.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ca00640 1 -- 172.21.15.145:0/2807682021 >> v1:172.21.15.145:6789/0 conn(0x7f5d780d20b0 legacy=0x7f5d7806d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ca00640 1 -- 172.21.15.145:0/2807682021 >> v1:172.21.15.110:6789/0 conn(0x7f5d7815e940 legacy=0x7f5d780b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.219+0000 7f5d7ca00640 1 -- 172.21.15.145:0/2807682021 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d780719f0 con 0x7f5d780b71c0 2024-09-10T09:58:00.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d7ca00640 1 --2- 172.21.15.145:0/2807682021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 0x7f5d7806dec0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5d78071640 tx=0x7f5d7806ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc9196230229261 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:00.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d77400640 1 -- 172.21.15.145:0/2807682021 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700a5750 con 0x7f5d780b71c0 2024-09-10T09:58:00.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d77400640 1 -- 172.21.15.145:0/2807682021 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d700a58f0 con 0x7f5d780b71c0 2024-09-10T09:58:00.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d77400640 1 -- 172.21.15.145:0/2807682021 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700a5cc0 con 0x7f5d780b71c0 2024-09-10T09:58:00.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d7ea00640 1 -- 172.21.15.145:0/2807682021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 msgr2=0x7f5d7806dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d7ea00640 1 --2- 172.21.15.145:0/2807682021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 0x7f5d7806dec0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5d78071640 tx=0x7f5d7806ec10 comp rx=0 tx=0).stop 2024-09-10T09:58:00.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d7ea00640 1 -- 172.21.15.145:0/2807682021 shutdown_connections 2024-09-10T09:58:00.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d7ea00640 1 --2- 172.21.15.145:0/2807682021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 0x7f5d7806dec0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d7ea00640 1 -- 172.21.15.145:0/2807682021 >> 172.21.15.145:0/2807682021 conn(0x7f5d780d25a0 msgr2=0x7f5d780d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:00.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d7ea00640 1 -- 172.21.15.145:0/2807682021 shutdown_connections 2024-09-10T09:58:00.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.223+0000 7f5d7ea00640 1 -- 172.21.15.145:0/2807682021 wait complete. 2024-09-10T09:58:00.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 Processor -- start 2024-09-10T09:58:00.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 -- start start 2024-09-10T09:58:00.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 0x7f5d7815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5d780d20b0 0x7f5d78151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d7815e940 0x7f5d781604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7816c6d0 con 0x7f5d7815e940 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7816c550 con 0x7f5d780b71c0 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7816c850 con 0x7f5d780d20b0 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 0x7f5d7815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d6f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5d780d20b0 0x7f5d78151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 0x7f5d7815aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46830/0 (socket says 172.21.15.145:46830) 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d77e00640 1 -- 172.21.15.145:0/3265104024 learned_addr learned my addr 172.21.15.145:0/3265104024 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ca00640 1 --2- 172.21.15.145:0/3265104024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d7815e940 0x7f5d781604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d6f400640 1 -- 172.21.15.145:0/3265104024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 msgr2=0x7f5d7815aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d6f400640 1 --2- 172.21.15.145:0/3265104024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 0x7f5d7815aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d6f400640 1 -- 172.21.15.145:0/3265104024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d7815e940 msgr2=0x7f5d781604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d6f400640 1 --2- 172.21.15.145:0/3265104024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d7815e940 0x7f5d781604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d6f400640 1 -- 172.21.15.145:0/3265104024 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7806ec60 con 0x7f5d780d20b0 2024-09-10T09:58:00.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d6f400640 1 --2- 172.21.15.145:0/3265104024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5d780d20b0 0x7f5d78151350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c390 tx=0x7f5d6000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:00.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d74c00640 1 -- 172.21.15.145:0/3265104024 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60017070 con 0x7f5d780d20b0 2024-09-10T09:58:00.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d74c00640 1 -- 172.21.15.145:0/3265104024 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d600052d0 con 0x7f5d780d20b0 2024-09-10T09:58:00.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d78151890 con 0x7f5d780d20b0 2024-09-10T09:58:00.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d74c00640 1 -- 172.21.15.145:0/3265104024 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60003820 con 0x7f5d780d20b0 2024-09-10T09:58:00.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7815a100 con 0x7f5d780d20b0 2024-09-10T09:58:00.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d74c00640 1 -- 172.21.15.145:0/3265104024 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5d6001a070 con 0x7f5d780d20b0 2024-09-10T09:58:00.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.227+0000 7f5d74c00640 1 -- 172.21.15.145:0/3265104024 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5d60011590 con 0x7f5d780d20b0 2024-09-10T09:58:00.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.231+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d780b6ab0 con 0x7f5d780d20b0 2024-09-10T09:58:00.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.231+0000 7f5d74c00640 1 -- 172.21.15.145:0/3265104024 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5d60021020 con 0x7f5d780d20b0 2024-09-10T09:58:00.253 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:00.251+0000 7f966384f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:58:00.256 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:00.254+0000 7f31602d7180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:58:00.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.299+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ddbaf2e3-cfba-4c75-a56d-0c32e415dd17", "id": 5} v 0) -- 0x7f5d781526c0 con 0x7f5d780d20b0 2024-09-10T09:58:00.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.303+0000 7f5d74c00640 1 -- 172.21.15.145:0/3265104024 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddbaf2e3-cfba-4c75-a56d-0c32e415dd17", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5d60005480 con 0x7f5d780d20b0 2024-09-10T09:58:00.307 INFO:teuthology.orchestra.run.smithi145.stdout:5 2024-09-10T09:58:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.303+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5d780d20b0 msgr2=0x7f5d78151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.303+0000 7f5d7ea00640 1 --2- 172.21.15.145:0/3265104024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5d780d20b0 0x7f5d78151350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c390 tx=0x7f5d6000c850 comp rx=0 tx=0).stop 2024-09-10T09:58:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.307+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 shutdown_connections 2024-09-10T09:58:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.307+0000 7f5d7ea00640 1 --2- 172.21.15.145:0/3265104024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d7815e940 0x7f5d781604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.307+0000 7f5d7ea00640 1 --2- 172.21.15.145:0/3265104024 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5d780d20b0 0x7f5d78151350 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.307+0000 7f5d7ea00640 1 --2- 172.21.15.145:0/3265104024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d780b71c0 0x7f5d7815aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.307+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 >> 172.21.15.145:0/3265104024 conn(0x7f5d780d25a0 msgr2=0x7f5d7806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:00.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.307+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 shutdown_connections 2024-09-10T09:58:00.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.307+0000 7f5d7ea00640 1 -- 172.21.15.145:0/3265104024 wait complete. 2024-09-10T09:58:00.320 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:00.318+0000 7f31602d7180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:58:00.328 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 5150a649-1978-44bc-a1ee-8efa95bad937 6 2024-09-10T09:58:00.347 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:00.343+0000 7f966384f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:58:00.411 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:00.407+0000 7f966384f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:58:00.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff2f200640 1 Processor -- start 2024-09-10T09:58:00.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff2f200640 1 -- start start 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff2f200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7eff2815a080 0x7eff280b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff2f200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7eff280cdd70 0x7eff280b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff2f200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2815a460 con 0x7eff280b6160 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff2f200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7eff2806e5b0 con 0x7eff280cdd70 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff2f200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7eff2815d6c0 con 0x7eff2815a080 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff27e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7eff2815a080 0x7eff280b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff27e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7eff2815a080 0x7eff280b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46846/0 (socket says 172.21.15.145:46846) 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff27e00640 1 -- 172.21.15.145:0/3031263777 learned_addr learned my addr 172.21.15.145:0/3031263777 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff1f400640 1 --2- 172.21.15.145:0/3031263777 >> v2:172.21.15.110:3300/0 conn(0x7eff280cdd70 0x7eff280b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff27e00640 1 -- 172.21.15.145:0/3031263777 >> v2:172.21.15.110:3300/0 conn(0x7eff280cdd70 msgr2=0x7eff280b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff27e00640 1 --2- 172.21.15.145:0/3031263777 >> v2:172.21.15.110:3300/0 conn(0x7eff280cdd70 0x7eff280b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff27e00640 1 -- 172.21.15.145:0/3031263777 >> v1:172.21.15.110:6789/0 conn(0x7eff280b6160 legacy=0x7eff28160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.479+0000 7eff27e00640 1 -- 172.21.15.145:0/3031263777 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2815cd90 con 0x7eff2815a080 2024-09-10T09:58:00.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff27e00640 1 --2- 172.21.15.145:0/3031263777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2815a080 0x7eff280b20b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff2815d840 tx=0x7eff1801c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0ac153ba0ba8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:00.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff27400640 1 -- 172.21.15.145:0/3031263777 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180b0070 con 0x7eff2815a080 2024-09-10T09:58:00.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff27400640 1 -- 172.21.15.145:0/3031263777 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff180ab600 con 0x7eff2815a080 2024-09-10T09:58:00.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff27400640 1 -- 172.21.15.145:0/3031263777 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180ab9d0 con 0x7eff2815a080 2024-09-10T09:58:00.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff2f200640 1 -- 172.21.15.145:0/3031263777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2815a080 msgr2=0x7eff280b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff2f200640 1 --2- 172.21.15.145:0/3031263777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2815a080 0x7eff280b20b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff2815d840 tx=0x7eff1801c400 comp rx=0 tx=0).stop 2024-09-10T09:58:00.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff2f200640 1 -- 172.21.15.145:0/3031263777 shutdown_connections 2024-09-10T09:58:00.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff2f200640 1 --2- 172.21.15.145:0/3031263777 >> v2:172.21.15.110:3300/0 conn(0x7eff280cdd70 0x7eff280b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff2f200640 1 --2- 172.21.15.145:0/3031263777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2815a080 0x7eff280b20b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff2f200640 1 -- 172.21.15.145:0/3031263777 >> 172.21.15.145:0/3031263777 conn(0x7eff280b0580 msgr2=0x7eff280b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:00.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff2f200640 1 -- 172.21.15.145:0/3031263777 shutdown_connections 2024-09-10T09:58:00.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.483+0000 7eff2f200640 1 -- 172.21.15.145:0/3031263777 wait complete. 2024-09-10T09:58:00.489 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:00.486+0000 7f31602d7180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:58:00.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 Processor -- start 2024-09-10T09:58:00.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 -- start start 2024-09-10T09:58:00.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff280b6160 0x7eff281613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff280cdd70 0x7eff2815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff2815a080 0x7eff2814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7eff280ce150 con 0x7eff2815a080 2024-09-10T09:58:00.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7eff2806ea80 con 0x7eff280b6160 2024-09-10T09:58:00.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7eff280ce2d0 con 0x7eff280cdd70 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff1f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff280cdd70 0x7eff2815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff27e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff280b6160 0x7eff281613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff1f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff280cdd70 0x7eff2815bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.145:41130/0 (socket says 172.21.15.145:41130) 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff1f400640 1 -- 172.21.15.145:0/2839328001 learned_addr learned my addr 172.21.15.145:0/2839328001 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2d200640 1 --2- 172.21.15.145:0/2839328001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff2815a080 0x7eff2814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff27e00640 1 -- 172.21.15.145:0/2839328001 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff280cdd70 msgr2=0x7eff2815bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff27e00640 1 --2- 172.21.15.145:0/2839328001 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff280cdd70 0x7eff2815bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff27e00640 1 -- 172.21.15.145:0/2839328001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff2815a080 msgr2=0x7eff2814b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff27e00640 1 --2- 172.21.15.145:0/2839328001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff2815a080 0x7eff2814b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff27e00640 1 -- 172.21.15.145:0/2839328001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2814be90 con 0x7eff280b6160 2024-09-10T09:58:00.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff27e00640 1 --2- 172.21.15.145:0/2839328001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff280b6160 0x7eff281613c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7eff180ab000 tx=0x7eff180191a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:00.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff24c00640 1 -- 172.21.15.145:0/2839328001 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180b3790 con 0x7eff280b6160 2024-09-10T09:58:00.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff24c00640 1 -- 172.21.15.145:0/2839328001 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff180b3930 con 0x7eff280b6160 2024-09-10T09:58:00.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff24c00640 1 -- 172.21.15.145:0/2839328001 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180b3d00 con 0x7eff280b6160 2024-09-10T09:58:00.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2814c170 con 0x7eff280b6160 2024-09-10T09:58:00.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2814c730 con 0x7eff280b6160 2024-09-10T09:58:00.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff24c00640 1 -- 172.21.15.145:0/2839328001 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff180c2060 con 0x7eff280b6160 2024-09-10T09:58:00.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.487+0000 7eff24c00640 1 -- 172.21.15.145:0/2839328001 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7eff180bb9e0 con 0x7eff280b6160 2024-09-10T09:58:00.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.491+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2806e220 con 0x7eff280b6160 2024-09-10T09:58:00.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.491+0000 7eff24c00640 1 -- 172.21.15.145:0/2839328001 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7eff18016110 con 0x7eff280b6160 2024-09-10T09:58:00.558 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:00.554+0000 7f31602d7180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:58:00.562 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.559+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5150a649-1978-44bc-a1ee-8efa95bad937", "id": 6} v 0) -- 0x7eff28075790 con 0x7eff280b6160 2024-09-10T09:58:00.566 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff24c00640 1 -- 172.21.15.145:0/2839328001 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5150a649-1978-44bc-a1ee-8efa95bad937", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7eff180bbbe0 con 0x7eff280b6160 2024-09-10T09:58:00.566 INFO:teuthology.orchestra.run.smithi145.stdout:6 2024-09-10T09:58:00.567 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff280b6160 msgr2=0x7eff281613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.567 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 --2- 172.21.15.145:0/2839328001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff280b6160 0x7eff281613c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7eff180ab000 tx=0x7eff180191a0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.568 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 shutdown_connections 2024-09-10T09:58:00.568 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 --2- 172.21.15.145:0/2839328001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff2815a080 0x7eff2814b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.568 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 --2- 172.21.15.145:0/2839328001 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff280cdd70 0x7eff2815bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.568 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 --2- 172.21.15.145:0/2839328001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff280b6160 0x7eff281613c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.568 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 >> 172.21.15.145:0/2839328001 conn(0x7eff280b0580 msgr2=0x7eff2806ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:00.568 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 shutdown_connections 2024-09-10T09:58:00.568 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.563+0000 7eff2f200640 1 -- 172.21.15.145:0/2839328001 wait complete. 2024-09-10T09:58:00.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd new 1c218083-28ff-425a-bb60-86a19ed07a28 7 2024-09-10T09:58:00.587 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:00.583+0000 7f966384f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:58:00.660 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:00.655+0000 7f966384f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:58:00.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.719+0000 7f3a21000640 1 Processor -- start 2024-09-10T09:58:00.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.719+0000 7f3a21000640 1 -- start start 2024-09-10T09:58:00.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.719+0000 7f3a21000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3a1c0d0460 0x7f3a1c072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.719+0000 7f3a21000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a1c15cde0 con 0x7f3a1c06bc50 2024-09-10T09:58:00.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a21000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a1c0d0b20 con 0x7f3a1c15a0a0 2024-09-10T09:58:00.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a21000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3a1c0d08f0 con 0x7f3a1c0d0460 2024-09-10T09:58:00.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a13e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3a1c0d0460 0x7f3a1c072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a19e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f3a1c15a0a0 0x7f3a1c15d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:51884/0 (socket says 172.21.15.145:51884) 2024-09-10T09:58:00.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a19e00640 1 -- 172.21.15.145:0/3484863667 learned_addr learned my addr 172.21.15.145:0/3484863667 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-09-10T09:58:00.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a13e00640 1 -- 172.21.15.145:0/3484863667 >> v1:172.21.15.145:6789/0 conn(0x7f3a1c15a0a0 legacy=0x7f3a1c15d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a13e00640 1 -- 172.21.15.145:0/3484863667 >> v1:172.21.15.110:6789/0 conn(0x7f3a1c06bc50 legacy=0x7f3a1c06c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a13e00640 1 -- 172.21.15.145:0/3484863667 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c0d0e00 con 0x7f3a1c0d0460 2024-09-10T09:58:00.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a13e00640 1 --2- 172.21.15.145:0/3484863667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c072840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a040097b0 tx=0x7f3a0401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=128e5861345f83b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:00.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a19400640 1 -- 172.21.15.145:0/3484863667 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a040b0070 con 0x7f3a1c0d0460 2024-09-10T09:58:00.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a19400640 1 -- 172.21.15.145:0/3484863667 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a040ab600 con 0x7f3a1c0d0460 2024-09-10T09:58:00.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.723+0000 7f3a19400640 1 -- 172.21.15.145:0/3484863667 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a040ab9d0 con 0x7f3a1c0d0460 2024-09-10T09:58:00.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- 172.21.15.145:0/3484863667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 msgr2=0x7f3a1c072840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 --2- 172.21.15.145:0/3484863667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c072840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a040097b0 tx=0x7f3a0401c4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- 172.21.15.145:0/3484863667 shutdown_connections 2024-09-10T09:58:00.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 --2- 172.21.15.145:0/3484863667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c072840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- 172.21.15.145:0/3484863667 >> 172.21.15.145:0/3484863667 conn(0x7f3a1c06cd60 msgr2=0x7f3a1c06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:00.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- 172.21.15.145:0/3484863667 shutdown_connections 2024-09-10T09:58:00.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- 172.21.15.145:0/3484863667 wait complete. 2024-09-10T09:58:00.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 Processor -- start 2024-09-10T09:58:00.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- start start 2024-09-10T09:58:00.732 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a1c06bc50 0x7f3a1c163510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a1c15a0a0 0x7f3a1c163e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c0cfcc0 con 0x7f3a1c06bc50 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c0cfb40 con 0x7f3a1c0d0460 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a21000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3a1c0cfe40 con 0x7f3a1c15a0a0 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a13e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a13e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46880/0 (socket says 172.21.15.145:46880) 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a13e00640 1 -- 172.21.15.145:0/2192812570 learned_addr learned my addr 172.21.15.145:0/2192812570 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a1a800640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a1c15a0a0 0x7f3a1c163e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a19e00640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a1c06bc50 0x7f3a1c163510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a13e00640 1 -- 172.21.15.145:0/2192812570 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a1c15a0a0 msgr2=0x7f3a1c163e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a13e00640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a1c15a0a0 0x7f3a1c163e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a13e00640 1 -- 172.21.15.145:0/2192812570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a1c06bc50 msgr2=0x7f3a1c163510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a13e00640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a1c06bc50 0x7f3a1c163510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.727+0000 7f3a13e00640 1 -- 172.21.15.145:0/2192812570 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c167510 con 0x7f3a1c0d0460 2024-09-10T09:58:00.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a13e00640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c158ff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a040ae040 tx=0x7f3a04019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:00.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a12000640 1 -- 172.21.15.145:0/2192812570 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a040b0040 con 0x7f3a1c0d0460 2024-09-10T09:58:00.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a12000640 1 -- 172.21.15.145:0/2192812570 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a040ba070 con 0x7f3a1c0d0460 2024-09-10T09:58:00.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a12000640 1 -- 172.21.15.145:0/2192812570 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a040b4890 con 0x7f3a1c0d0460 2024-09-10T09:58:00.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1c166c70 con 0x7f3a1c0d0460 2024-09-10T09:58:00.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1c167230 con 0x7f3a1c0d0460 2024-09-10T09:58:00.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a12000640 1 -- 172.21.15.145:0/2192812570 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a040ab480 con 0x7f3a1c0d0460 2024-09-10T09:58:00.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a12000640 1 -- 172.21.15.145:0/2192812570 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3a040bf460 con 0x7f3a1c0d0460 2024-09-10T09:58:00.736 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a1c072840 con 0x7f3a1c0d0460 2024-09-10T09:58:00.736 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.731+0000 7f3a12000640 1 -- 172.21.15.145:0/2192812570 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3a04016090 con 0x7f3a1c0d0460 2024-09-10T09:58:00.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.799+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1c218083-28ff-425a-bb60-86a19ed07a28", "id": 7} v 0) -- 0x7f3a1c15a7c0 con 0x7f3a1c0d0460 2024-09-10T09:58:00.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a12000640 1 -- 172.21.15.145:0/2192812570 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1c218083-28ff-425a-bb60-86a19ed07a28", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3a040b4a30 con 0x7f3a1c0d0460 2024-09-10T09:58:00.811 INFO:teuthology.orchestra.run.smithi145.stdout:7 2024-09-10T09:58:00.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 msgr2=0x7f3a1c158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c158ff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a040ae040 tx=0x7f3a04019240 comp rx=0 tx=0).stop 2024-09-10T09:58:00.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 shutdown_connections 2024-09-10T09:58:00.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a1c15a0a0 0x7f3a1c163e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a1c0d0460 0x7f3a1c158ff0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 --2- 172.21.15.145:0/2192812570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a1c06bc50 0x7f3a1c163510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 >> 172.21.15.145:0/2192812570 conn(0x7f3a1c06cd60 msgr2=0x7f3a1c15ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:00.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 shutdown_connections 2024-09-10T09:58:00.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:00.807+0000 7f3a21000640 1 -- 172.21.15.145:0/2192812570 wait complete. 2024-09-10T09:58:00.830 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T09:58:00.830 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T09:58:00.834 INFO:tasks.ceph.osd.0:Started 2024-09-10T09:58:00.834 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T09:58:00.834 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T09:58:00.837 INFO:tasks.ceph.osd.1:Started 2024-09-10T09:58:00.837 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T09:58:00.837 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T09:58:00.840 INFO:tasks.ceph.osd.2:Started 2024-09-10T09:58:00.840 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T09:58:00.840 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T09:58:00.844 INFO:tasks.ceph.osd.3:Started 2024-09-10T09:58:00.844 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T09:58:00.845 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T09:58:00.847 INFO:tasks.ceph.osd.4:Started 2024-09-10T09:58:00.848 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T09:58:00.848 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T09:58:00.851 INFO:tasks.ceph.osd.5:Started 2024-09-10T09:58:00.851 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T09:58:00.851 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T09:58:00.854 INFO:tasks.ceph.osd.6:Started 2024-09-10T09:58:00.854 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T09:58:00.854 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T09:58:00.857 INFO:tasks.ceph.osd.7:Started 2024-09-10T09:58:00.857 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:01.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.074+0000 7f5ecfe00640 1 Processor -- start 2024-09-10T09:58:01.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecfe00640 1 -- start start 2024-09-10T09:58:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5ed015e470 0x7f5ed00b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecfe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5ed0162020 0x7f5ed00731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecfe00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed006d020 con 0x7f5ed0073720 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecfe00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed0073b90 con 0x7f5ed0162020 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecfe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed00b6870 con 0x7f5ed015e470 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ece000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5ed0162020 0x7f5ed00731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecf400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5ed0073720 0x7f5ed0070a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53642/0 (socket says 172.21.15.110:53642) 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecf400640 1 -- 172.21.15.110:0/3553783683 learned_addr learned my addr 172.21.15.110:0/3553783683 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ecea00640 1 --2- 172.21.15.110:0/3553783683 >> v2:172.21.15.145:3300/0 conn(0x7f5ed015e470 0x7f5ed00b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ece000640 1 -- 172.21.15.110:0/3553783683 >> v2:172.21.15.145:3300/0 conn(0x7f5ed015e470 msgr2=0x7f5ed00b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ece000640 1 --2- 172.21.15.110:0/3553783683 >> v2:172.21.15.145:3300/0 conn(0x7f5ed015e470 0x7f5ed00b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ece000640 1 -- 172.21.15.110:0/3553783683 >> v1:172.21.15.110:6789/0 conn(0x7f5ed0073720 legacy=0x7f5ed0070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:01.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.078+0000 7f5ece000640 1 -- 172.21.15.110:0/3553783683 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed00b5c60 con 0x7f5ed0162020 2024-09-10T09:58:01.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ece000640 1 --2- 172.21.15.110:0/3553783683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed0162020 0x7f5ed00731e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0070440 tx=0x7f5ebc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909fd9091a46b77e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:01.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecd600640 1 -- 172.21.15.110:0/3553783683 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0b5070 con 0x7f5ed0162020 2024-09-10T09:58:01.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecd600640 1 -- 172.21.15.110:0/3553783683 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc0a8970 con 0x7f5ed0162020 2024-09-10T09:58:01.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecd600640 1 -- 172.21.15.110:0/3553783683 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0a8d40 con 0x7f5ed0162020 2024-09-10T09:58:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 -- 172.21.15.110:0/3553783683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed0162020 msgr2=0x7f5ed00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 --2- 172.21.15.110:0/3553783683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed0162020 0x7f5ed00731e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0070440 tx=0x7f5ebc01ec70 comp rx=0 tx=0).stop 2024-09-10T09:58:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 -- 172.21.15.110:0/3553783683 shutdown_connections 2024-09-10T09:58:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 --2- 172.21.15.110:0/3553783683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed0162020 0x7f5ed00731e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 --2- 172.21.15.110:0/3553783683 >> v2:172.21.15.145:3300/0 conn(0x7f5ed015e470 0x7f5ed00b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 -- 172.21.15.110:0/3553783683 >> 172.21.15.110:0/3553783683 conn(0x7f5ed00b42d0 msgr2=0x7f5ed00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 -- 172.21.15.110:0/3553783683 shutdown_connections 2024-09-10T09:58:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 -- 172.21.15.110:0/3553783683 wait complete. 2024-09-10T09:58:01.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 Processor -- start 2024-09-10T09:58:01.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.082+0000 7f5ecfe00640 1 -- start start 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ed0073720 0x7f5ed01f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed015e470 0x7f5ed01f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ed0162020 0x7f5ed0070870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed00b6500 con 0x7f5ed015e470 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed00b6220 con 0x7f5ed0073720 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed015e850 con 0x7f5ed0162020 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed015e470 0x7f5ed01f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed015e470 0x7f5ed01f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50064/0 (socket says 172.21.15.110:50064) 2024-09-10T09:58:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 -- 172.21.15.110:0/2159492086 learned_addr learned my addr 172.21.15.110:0/2159492086 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecea00640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ed0073720 0x7f5ed01f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecf400640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ed0162020 0x7f5ed0070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 -- 172.21.15.110:0/2159492086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ed0162020 msgr2=0x7f5ed0070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ed0162020 0x7f5ed0070870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 -- 172.21.15.110:0/2159492086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ed0073720 msgr2=0x7f5ed01f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ed0073720 0x7f5ed01f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 -- 172.21.15.110:0/2159492086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed01f8190 con 0x7f5ed015e470 2024-09-10T09:58:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ece000640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed015e470 0x7f5ed01f7890 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0b3040 tx=0x7f5ebc01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:01.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ec6a00640 1 -- 172.21.15.110:0/2159492086 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0bf070 con 0x7f5ed015e470 2024-09-10T09:58:01.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ec6a00640 1 -- 172.21.15.110:0/2159492086 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc0b0720 con 0x7f5ed015e470 2024-09-10T09:58:01.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed01f8470 con 0x7f5ed015e470 2024-09-10T09:58:01.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ec6a00640 1 -- 172.21.15.110:0/2159492086 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0b5040 con 0x7f5ed015e470 2024-09-10T09:58:01.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed00cb9f0 con 0x7f5ed015e470 2024-09-10T09:58:01.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ec6a00640 1 -- 172.21.15.110:0/2159492086 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ebc0c2070 con 0x7f5ed015e470 2024-09-10T09:58:01.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ec6a00640 1 -- 172.21.15.110:0/2159492086 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5ebc0b9ac0 con 0x7f5ed015e470 2024-09-10T09:58:01.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.086+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed00b55d0 con 0x7f5ed015e470 2024-09-10T09:58:01.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.090+0000 7f5ec6a00640 1 -- 172.21.15.110:0/2159492086 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5ebc0187d0 con 0x7f5ed015e470 2024-09-10T09:58:01.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.146+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ed006d870 con 0x7f5ed015e470 2024-09-10T09:58:01.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ec6a00640 1 -- 172.21.15.110:0/2159492086 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5ebc0c4440 con 0x7f5ed015e470 2024-09-10T09:58:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:01.152 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":10,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:00.807874+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:01.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed015e470 msgr2=0x7f5ed01f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:01.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed015e470 0x7f5ed01f7890 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0b3040 tx=0x7f5ebc01ba10 comp rx=0 tx=0).stop 2024-09-10T09:58:01.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 shutdown_connections 2024-09-10T09:58:01.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ed0162020 0x7f5ed0070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:01.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ed015e470 0x7f5ed01f7890 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 --2- 172.21.15.110:0/2159492086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ed0073720 0x7f5ed01f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 >> 172.21.15.110:0/2159492086 conn(0x7f5ed00b42d0 msgr2=0x7f5ed006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 shutdown_connections 2024-09-10T09:58:01.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:01.150+0000 7f5ecfe00640 1 -- 172.21.15.110:0/2159492086 wait complete. 2024-09-10T09:58:01.171 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T09:58:01.172 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T09:58:01.254 INFO:tasks.ceph.mgr.y.smithi110.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T09:58:01.254 INFO:tasks.ceph.mgr.y.smithi110.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T09:58:01.254 INFO:tasks.ceph.mgr.y.smithi110.stderr: from numpy import show_config as show_numpy_config 2024-09-10T09:58:01.272 INFO:tasks.ceph.mgr.x.smithi145.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T09:58:01.272 INFO:tasks.ceph.mgr.x.smithi145.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T09:58:01.272 INFO:tasks.ceph.mgr.x.smithi145.stderr: from numpy import show_config as show_numpy_config 2024-09-10T09:58:01.306 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:01.302+0000 7f31602d7180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:58:01.332 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:01.327+0000 7f966384f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:58:01.455 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:01.450+0000 7f31602d7180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:58:01.480 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:01.475+0000 7f966384f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:58:01.629 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:01.626+0000 7f31602d7180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:58:01.668 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:01.663+0000 7f966384f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:58:01.701 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:01.698+0000 7f31602d7180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:58:01.740 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:01.735+0000 7f966384f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:58:01.851 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:01.846+0000 7f31602d7180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:58:01.891 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:01.887+0000 7f966384f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:58:01.975 DEBUG:teuthology.orchestra.run.smithi110:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:02.101 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22fc00640 1 Processor -- start 2024-09-10T09:58:02.101 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22fc00640 1 -- start start 2024-09-10T09:58:02.102 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22fc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc22815a050 0x7fc2280afb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.102 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22fc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22815a430 con 0x7fc2280b5380 2024-09-10T09:58:02.103 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22fc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2280b2050 con 0x7fc2280b6560 2024-09-10T09:58:02.103 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22fc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc2280b22f0 con 0x7fc22815a050 2024-09-10T09:58:02.103 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22d200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc22815a050 0x7fc2280afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.103 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc227e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc2280b5380 0x7fc2280b5e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53650/0 (socket says 172.21.15.110:53650) 2024-09-10T09:58:02.103 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc227e00640 1 -- 172.21.15.110:0/2817333197 learned_addr learned my addr 172.21.15.110:0/2817333197 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:02.103 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22d200640 1 -- 172.21.15.110:0/2817333197 >> v1:172.21.15.145:6789/0 conn(0x7fc2280b6560 legacy=0x7fc2280b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.103 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22d200640 1 -- 172.21.15.110:0/2817333197 >> v1:172.21.15.110:6789/0 conn(0x7fc2280b5380 legacy=0x7fc2280b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.104 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc22d200640 1 -- 172.21.15.110:0/2817333197 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2280b25d0 con 0x7fc22815a050 2024-09-10T09:58:02.104 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.098+0000 7fc227400640 1 -- 172.21.15.110:0/2817333197 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843212385 0 0) 0x7fc22815a430 con 0x7fc2280b5380 2024-09-10T09:58:02.106 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc22d200640 1 --2- 172.21.15.110:0/2817333197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc2280afb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc2180097c0 tx=0x7fc21801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a166a5002f7d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:02.106 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc227400640 1 -- 172.21.15.110:0/2817333197 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2180aa5f0 con 0x7fc22815a050 2024-09-10T09:58:02.106 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc227400640 1 -- 172.21.15.110:0/2817333197 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2180af070 con 0x7fc22815a050 2024-09-10T09:58:02.106 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc227400640 1 -- 172.21.15.110:0/2817333197 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2180aa9b0 con 0x7fc22815a050 2024-09-10T09:58:02.107 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc22fc00640 1 -- 172.21.15.110:0/2817333197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 msgr2=0x7fc2280afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.107 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc22fc00640 1 --2- 172.21.15.110:0/2817333197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc2280afb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc2180097c0 tx=0x7fc21801c4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.107 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc22fc00640 1 -- 172.21.15.110:0/2817333197 shutdown_connections 2024-09-10T09:58:02.107 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc22fc00640 1 --2- 172.21.15.110:0/2817333197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc2280afb00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.107 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc22fc00640 1 -- 172.21.15.110:0/2817333197 >> 172.21.15.110:0/2817333197 conn(0x7fc22809ba20 msgr2=0x7fc2280b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:02.107 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc22fc00640 1 -- 172.21.15.110:0/2817333197 shutdown_connections 2024-09-10T09:58:02.107 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.102+0000 7fc22fc00640 1 -- 172.21.15.110:0/2817333197 wait complete. 2024-09-10T09:58:02.108 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 Processor -- start 2024-09-10T09:58:02.108 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 -- start start 2024-09-10T09:58:02.111 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2280b5380 0x7fc228166ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.111 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2280b6560 0x7fc22809f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.111 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc22809fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.111 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc22816a360 con 0x7fc22815a050 2024-09-10T09:58:02.111 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc22816a1e0 con 0x7fc2280b6560 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc22816a4e0 con 0x7fc2280b5380 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc22809fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc22809fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50070/0 (socket says 172.21.15.110:50070) 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 -- 172.21.15.110:0/3684798874 learned_addr learned my addr 172.21.15.110:0/3684798874 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22d200640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2280b5380 0x7fc228166ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc227e00640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2280b6560 0x7fc22809f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 -- 172.21.15.110:0/3684798874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2280b5380 msgr2=0x7fc228166ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2280b5380 0x7fc228166ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 -- 172.21.15.110:0/3684798874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2280b6560 msgr2=0x7fc22809f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2280b6560 0x7fc22809f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 -- 172.21.15.110:0/3684798874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2281606f0 con 0x7fc22815a050 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22dc00640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc22809fee0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc22000c770 tx=0x7fc22000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:02.112 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc224c00640 1 -- 172.21.15.110:0/3684798874 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220003b80 con 0x7fc22815a050 2024-09-10T09:58:02.113 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc224c00640 1 -- 172.21.15.110:0/3684798874 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc220003d20 con 0x7fc22815a050 2024-09-10T09:58:02.113 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc224c00640 1 -- 172.21.15.110:0/3684798874 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22000aa90 con 0x7fc22815a050 2024-09-10T09:58:02.113 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281609d0 con 0x7fc22815a050 2024-09-10T09:58:02.113 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc228145140 con 0x7fc22815a050 2024-09-10T09:58:02.113 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc224c00640 1 -- 172.21.15.110:0/3684798874 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc22000ac30 con 0x7fc22815a050 2024-09-10T09:58:02.113 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.106+0000 7fc224c00640 1 -- 172.21.15.110:0/3684798874 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc2200125b0 con 0x7fc22815a050 2024-09-10T09:58:02.113 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.110+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2280afb00 con 0x7fc22815a050 2024-09-10T09:58:02.114 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.110+0000 7fc224c00640 1 -- 172.21.15.110:0/3684798874 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc2200128d0 con 0x7fc22815a050 2024-09-10T09:58:02.172 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc22815b650 con 0x7fc22815a050 2024-09-10T09:58:02.173 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc224c00640 1 -- 172.21.15.110:0/3684798874 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc220012ab0 con 0x7fc22815a050 2024-09-10T09:58:02.173 INFO:teuthology.misc.health.smithi110.stdout: 2024-09-10T09:58:02.173 INFO:teuthology.misc.health.smithi110.stdout:{"epoch":10,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:00.807874+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:02.174 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 msgr2=0x7fc22809fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.174 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc22809fee0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc22000c770 tx=0x7fc22000cc30 comp rx=0 tx=0).stop 2024-09-10T09:58:02.174 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 shutdown_connections 2024-09-10T09:58:02.174 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc22815a050 0x7fc22809fee0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.174 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2280b6560 0x7fc22809f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.174 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 --2- 172.21.15.110:0/3684798874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2280b5380 0x7fc228166ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.174 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 >> 172.21.15.110:0/3684798874 conn(0x7fc22809ba20 msgr2=0x7fc2280ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:02.175 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 shutdown_connections 2024-09-10T09:58:02.175 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:02.170+0000 7fc22fc00640 1 -- 172.21.15.110:0/3684798874 wait complete. 2024-09-10T09:58:02.190 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T09:58:02.554 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:02.550+0000 7f31602d7180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:58:02.581 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:02.575+0000 7f966384f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:58:02.644 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:02.642+0000 7f31602d7180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:58:02.666 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:02.663+0000 7f966384f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:58:02.754 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-09-10T09:58:02.755+0000 7f3de3a957c0 -1 Falling back to public interface 2024-09-10T09:58:02.846 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:02.842+0000 7f31602d7180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:58:02.869 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:02.867+0000 7f966384f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:58:02.931 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:02.926+0000 7f31602d7180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:58:02.952 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:02.951+0000 7f966384f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:58:03.018 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-09-10T09:58:03.019+0000 7f28665967c0 -1 Falling back to public interface 2024-09-10T09:58:03.042 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-09-10T09:58:03.043+0000 7f45b1a127c0 -1 Falling back to public interface 2024-09-10T09:58:03.062 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-09-10T09:58:03.063+0000 7fb70865e7c0 -1 Falling back to public interface 2024-09-10T09:58:03.121 INFO:tasks.ceph.osd.2.smithi110.stderr:2024-09-10T09:58:03.118+0000 7feb34ec17c0 -1 Falling back to public interface 2024-09-10T09:58:03.133 INFO:tasks.ceph.osd.1.smithi110.stderr:2024-09-10T09:58:03.130+0000 7fdfbf6957c0 -1 Falling back to public interface 2024-09-10T09:58:03.136 INFO:tasks.ceph.osd.0.smithi110.stderr:2024-09-10T09:58:03.134+0000 7f6bb715a7c0 -1 Falling back to public interface 2024-09-10T09:58:03.185 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-10T09:58:03.182+0000 7fae0bd437c0 -1 Falling back to public interface 2024-09-10T09:58:03.266 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:03.262+0000 7f31602d7180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:58:03.288 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:03.287+0000 7f966384f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:58:03.336 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:03.334+0000 7f31602d7180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:58:03.363 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T09:58:03.363+0000 7f966384f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:58:04.357 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:04.354+0000 7f3105800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T09:58:04.358 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:58:04.354+0000 7f3105800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T09:58:07.687 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-10T09:58:07.681+0000 7fae0bd437c0 -1 osd.3 0 log_to_monitors true 2024-09-10T09:58:07.812 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-09-10T09:58:07.811+0000 7f28665967c0 -1 osd.5 0 log_to_monitors true 2024-09-10T09:58:07.828 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-09-10T09:58:07.827+0000 7f45b1a127c0 -1 osd.4 0 log_to_monitors true 2024-09-10T09:58:07.857 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-09-10T09:58:07.855+0000 7f3de3a957c0 -1 osd.7 0 log_to_monitors true 2024-09-10T09:58:07.945 INFO:tasks.ceph.osd.2.smithi110.stderr:2024-09-10T09:58:07.941+0000 7feb34ec17c0 -1 osd.2 0 log_to_monitors true 2024-09-10T09:58:07.958 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-09-10T09:58:07.959+0000 7fb70865e7c0 -1 osd.6 0 log_to_monitors true 2024-09-10T09:58:07.965 INFO:tasks.ceph.osd.0.smithi110.stderr:2024-09-10T09:58:07.961+0000 7f6bb715a7c0 -1 osd.0 0 log_to_monitors true 2024-09-10T09:58:08.005 INFO:tasks.ceph.osd.1.smithi110.stderr:2024-09-10T09:58:08.005+0000 7fdfbf6957c0 -1 osd.1 0 log_to_monitors true 2024-09-10T09:58:08.996 DEBUG:teuthology.orchestra.run.smithi110:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:09.136 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.133+0000 7fd852200640 1 Processor -- start 2024-09-10T09:58:09.136 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.133+0000 7fd852200640 1 -- start start 2024-09-10T09:58:09.137 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd852200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 0x7fd84c071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:09.137 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd852200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd84c0cdc90 0x7fd84c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:09.137 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd852200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd84c1461d0 con 0x7fd84c072e40 2024-09-10T09:58:09.138 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd852200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd84c071b80 con 0x7fd84c0cdc90 2024-09-10T09:58:09.138 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd852200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd84c071e60 con 0x7fd84c145df0 2024-09-10T09:58:09.138 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd84be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd84c0cdc90 0x7fd84c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:09.138 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd84b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 0x7fd84c071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:09.138 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd84be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd84c0cdc90 0x7fd84c06d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44668/0 (socket says 172.21.15.110:44668) 2024-09-10T09:58:09.138 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd84be00640 1 -- 172.21.15.110:0/2248026437 learned_addr learned my addr 172.21.15.110:0/2248026437 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:09.138 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd84be00640 1 -- 172.21.15.110:0/2248026437 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 msgr2=0x7fd84c071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:09.139 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd84be00640 1 --2- 172.21.15.110:0/2248026437 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 0x7fd84c071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.139 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd84be00640 1 -- 172.21.15.110:0/2248026437 >> v1:172.21.15.110:6789/0 conn(0x7fd84c072e40 legacy=0x7fd84c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:09.139 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.137+0000 7fd84be00640 1 -- 172.21.15.110:0/2248026437 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c072140 con 0x7fd84c0cdc90 2024-09-10T09:58:09.141 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd84be00640 1 --2- 172.21.15.110:0/2248026437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c0cdc90 0x7fd84c06d1d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd84401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863aa820d0ff901f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:09.141 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd84aa00640 1 -- 172.21.15.110:0/2248026437 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440aa610 con 0x7fd84c0cdc90 2024-09-10T09:58:09.142 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd84aa00640 1 -- 172.21.15.110:0/2248026437 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8440af070 con 0x7fd84c0cdc90 2024-09-10T09:58:09.142 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd84aa00640 1 -- 172.21.15.110:0/2248026437 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440aa9d0 con 0x7fd84c0cdc90 2024-09-10T09:58:09.142 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd852200640 1 -- 172.21.15.110:0/2248026437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c0cdc90 msgr2=0x7fd84c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:09.142 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd852200640 1 --2- 172.21.15.110:0/2248026437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c0cdc90 0x7fd84c06d1d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd84401c4f0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.142 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd852200640 1 -- 172.21.15.110:0/2248026437 shutdown_connections 2024-09-10T09:58:09.143 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd852200640 1 --2- 172.21.15.110:0/2248026437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c0cdc90 0x7fd84c06d1d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.143 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd852200640 1 --2- 172.21.15.110:0/2248026437 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 0x7fd84c071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.143 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd852200640 1 -- 172.21.15.110:0/2248026437 >> 172.21.15.110:0/2248026437 conn(0x7fd84c067980 msgr2=0x7fd84c067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:09.143 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd852200640 1 -- 172.21.15.110:0/2248026437 shutdown_connections 2024-09-10T09:58:09.143 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.141+0000 7fd852200640 1 -- 172.21.15.110:0/2248026437 wait complete. 2024-09-10T09:58:09.144 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 Processor -- start 2024-09-10T09:58:09.144 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 -- start start 2024-09-10T09:58:09.146 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c072e40 0x7fd84c0b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:09.146 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c0cdc90 0x7fd84c141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:09.146 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 0x7fd84c1415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:09.146 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c06e0a0 con 0x7fd84c072e40 2024-09-10T09:58:09.146 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c06df20 con 0x7fd84c0cdc90 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd84c06e220 con 0x7fd84c145df0 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c072e40 0x7fd84c0b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c072e40 0x7fd84c0b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44680/0 (socket says 172.21.15.110:44680) 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 -- 172.21.15.110:0/3647942581 learned_addr learned my addr 172.21.15.110:0/3647942581 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd842a00640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c0cdc90 0x7fd84c141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84be00640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 0x7fd84c1415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 -- 172.21.15.110:0/3647942581 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 msgr2=0x7fd84c1415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 0x7fd84c1415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 -- 172.21.15.110:0/3647942581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c0cdc90 msgr2=0x7fd84c141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c0cdc90 0x7fd84c141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.147 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 -- 172.21.15.110:0/3647942581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c0cb050 con 0x7fd84c072e40 2024-09-10T09:58:09.148 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd84b400640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c072e40 0x7fd84c0b4c20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00efc0 tx=0x7fd83c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:09.148 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd843e00640 1 -- 172.21.15.110:0/3647942581 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c018020 con 0x7fd84c072e40 2024-09-10T09:58:09.148 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd843e00640 1 -- 172.21.15.110:0/3647942581 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd83c014480 con 0x7fd84c072e40 2024-09-10T09:58:09.148 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd843e00640 1 -- 172.21.15.110:0/3647942581 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c014870 con 0x7fd84c072e40 2024-09-10T09:58:09.148 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c1359d0 con 0x7fd84c072e40 2024-09-10T09:58:09.148 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.145+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c0c9f80 con 0x7fd84c072e40 2024-09-10T09:58:09.148 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.149+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84c0b44a0 con 0x7fd84c072e40 2024-09-10T09:58:09.149 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.149+0000 7fd843e00640 1 -- 172.21.15.110:0/3647942581 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7fd83c01b020 con 0x7fd84c072e40 2024-09-10T09:58:09.152 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.149+0000 7fd843e00640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd828067860 0x7fd828069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:09.152 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.149+0000 7fd843e00640 1 -- 172.21.15.110:0/3647942581 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2009+0+0 (secure 0 0 0) 0x7fd83c085670 con 0x7fd84c072e40 2024-09-10T09:58:09.152 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.149+0000 7fd842a00640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd828067860 0x7fd828069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:09.152 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.149+0000 7fd842a00640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd828067860 0x7fd828069d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0af7d0 tx=0x7fd834009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:09.155 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.153+0000 7fd843e00640 1 -- 172.21.15.110:0/3647942581 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd83c022070 con 0x7fd84c072e40 2024-09-10T09:58:09.444 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.445+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd84c0b41c0 con 0x7fd84c072e40 2024-09-10T09:58:09.445 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.445+0000 7fd843e00640 1 -- 172.21.15.110:0/3647942581 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fd83c010040 con 0x7fd84c072e40 2024-09-10T09:58:09.445 INFO:teuthology.misc.health.smithi110.stdout: 2024-09-10T09:58:09.445 INFO:teuthology.misc.health.smithi110.stdout:{"epoch":12,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:08.746533+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:09.448 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd828067860 msgr2=0x7fd828069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd828067860 0x7fd828069d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0af7d0 tx=0x7fd834009040 comp rx=0 tx=0).stop 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c072e40 msgr2=0x7fd84c0b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c072e40 0x7fd84c0b4c20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00efc0 tx=0x7fd83c00c490 comp rx=0 tx=0).stop 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 shutdown_connections 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd828067860 0x7fd828069d20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd84c145df0 0x7fd84c1415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd84c0cdc90 0x7fd84c141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 --2- 172.21.15.110:0/3647942581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd84c072e40 0x7fd84c0b4c20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 >> 172.21.15.110:0/3647942581 conn(0x7fd84c067980 msgr2=0x7fd84c13f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 shutdown_connections 2024-09-10T09:58:09.449 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:09.449+0000 7fd852200640 1 -- 172.21.15.110:0/3647942581 wait complete. 2024-09-10T09:58:09.471 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T09:58:09.730 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-10T09:58:09.729+0000 7fae06200640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T09:58:09.766 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-10T09:58:09.765+0000 7fadfcc00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:11.181 INFO:tasks.ceph.osd.0.smithi110.stderr:2024-09-10T09:58:11.181+0000 7f6bb1600640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T09:58:11.181 INFO:tasks.ceph.osd.1.smithi110.stderr:2024-09-10T09:58:11.181+0000 7fdfb9c00640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T09:58:11.184 INFO:tasks.ceph.osd.2.smithi110.stderr:2024-09-10T09:58:11.185+0000 7feb30800640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T09:58:11.201 INFO:tasks.ceph.osd.0.smithi110.stderr:2024-09-10T09:58:11.201+0000 7f6ba8000640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:11.201 INFO:tasks.ceph.osd.1.smithi110.stderr:2024-09-10T09:58:11.201+0000 7fdfb0600640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:11.201 INFO:tasks.ceph.osd.2.smithi110.stderr:2024-09-10T09:58:11.201+0000 7feb25e00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:11.993 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-09-10T09:58:11.994+0000 7f45abe00640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T09:58:12.021 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-09-10T09:58:12.022+0000 7f3dde000640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T09:58:12.028 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-09-10T09:58:12.026+0000 7f2860a00640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T09:58:12.031 INFO:tasks.ceph.osd.4.smithi145.stderr:2024-09-10T09:58:12.030+0000 7f45a2800640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:12.050 INFO:tasks.ceph.osd.7.smithi145.stderr:2024-09-10T09:58:12.050+0000 7f3dd4a00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:12.058 INFO:tasks.ceph.osd.5.smithi145.stderr:2024-09-10T09:58:12.058+0000 7f2857400640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:12.064 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-09-10T09:58:12.062+0000 7fb702c00640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T09:58:12.098 INFO:tasks.ceph.osd.6.smithi145.stderr:2024-09-10T09:58:12.098+0000 7fb6f9600640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:16.278 DEBUG:teuthology.orchestra.run.smithi110:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:16.411 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.409+0000 7f0517000640 1 Processor -- start 2024-09-10T09:58:16.412 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.409+0000 7f0517000640 1 -- start start 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f0517000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0510071ae0 0x7f0510072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f0517000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510072ba0 0x7f05100b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f0517000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05100b35e0 con 0x7f051015b5f0 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f0517000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f05100b3910 con 0x7f0510071ae0 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f0517000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f05100720a0 con 0x7f0510072ba0 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f050f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0510071ae0 0x7f0510072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f050f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0510071ae0 0x7f0510072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44690/0 (socket says 172.21.15.110:44690) 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f050f400640 1 -- 172.21.15.110:0/3990598954 learned_addr learned my addr 172.21.15.110:0/3990598954 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f0515000640 1 --2- 172.21.15.110:0/3990598954 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510072ba0 0x7f05100b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f050f400640 1 -- 172.21.15.110:0/3990598954 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510072ba0 msgr2=0x7f05100b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f050f400640 1 --2- 172.21.15.110:0/3990598954 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510072ba0 0x7f05100b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f050f400640 1 -- 172.21.15.110:0/3990598954 >> v1:172.21.15.145:6789/0 conn(0x7f051015b5f0 legacy=0x7f051006d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.414 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.413+0000 7f050f400640 1 -- 172.21.15.110:0/3990598954 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0510072380 con 0x7f0510071ae0 2024-09-10T09:58:16.416 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f050f400640 1 --2- 172.21.15.110:0/3990598954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0510071ae0 0x7f0510072660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009870 tx=0x7f04fc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77699edf90d60dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.416 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f050ea00640 1 -- 172.21.15.110:0/3990598954 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0aa600 con 0x7f0510071ae0 2024-09-10T09:58:16.417 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f050ea00640 1 -- 172.21.15.110:0/3990598954 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04fc0af070 con 0x7f0510071ae0 2024-09-10T09:58:16.417 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f050ea00640 1 -- 172.21.15.110:0/3990598954 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0aa9c0 con 0x7f0510071ae0 2024-09-10T09:58:16.417 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- 172.21.15.110:0/3990598954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0510071ae0 msgr2=0x7f0510072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.417 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 --2- 172.21.15.110:0/3990598954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0510071ae0 0x7f0510072660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009870 tx=0x7f04fc01c560 comp rx=0 tx=0).stop 2024-09-10T09:58:16.417 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- 172.21.15.110:0/3990598954 shutdown_connections 2024-09-10T09:58:16.418 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 --2- 172.21.15.110:0/3990598954 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510072ba0 0x7f05100b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.418 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 --2- 172.21.15.110:0/3990598954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0510071ae0 0x7f0510072660 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.418 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- 172.21.15.110:0/3990598954 >> 172.21.15.110:0/3990598954 conn(0x7f0510067980 msgr2=0x7f0510067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.418 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- 172.21.15.110:0/3990598954 shutdown_connections 2024-09-10T09:58:16.418 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- 172.21.15.110:0/3990598954 wait complete. 2024-09-10T09:58:16.419 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 Processor -- start 2024-09-10T09:58:16.419 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- start start 2024-09-10T09:58:16.420 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510071ae0 0x7f0510151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.420 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0510072ba0 0x7f051015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f051015c040 0x7f051013d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05100b4e80 con 0x7f051015c040 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05100b4d00 con 0x7f0510072ba0 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.417+0000 7f0517000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f05101680a0 con 0x7f0510071ae0 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f051015c040 0x7f051013d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f051015c040 0x7f051013d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44696/0 (socket says 172.21.15.110:44696) 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 -- 172.21.15.110:0/2933272165 learned_addr learned my addr 172.21.15.110:0/2933272165 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f050fe00640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510071ae0 0x7f0510151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f050f400640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0510072ba0 0x7f051015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 -- 172.21.15.110:0/2933272165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510071ae0 msgr2=0x7f0510151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510071ae0 0x7f0510151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 -- 172.21.15.110:0/2933272165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0510072ba0 msgr2=0x7f051015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.421 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0510072ba0 0x7f051015bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.422 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 -- 172.21.15.110:0/2933272165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05100b0b00 con 0x7f051015c040 2024-09-10T09:58:16.422 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0515000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f051015c040 0x7f051013d360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0508007e40 tx=0x7f050800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.422 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0507e00640 1 -- 172.21.15.110:0/2933272165 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508014620 con 0x7f051015c040 2024-09-10T09:58:16.423 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0507e00640 1 -- 172.21.15.110:0/2933272165 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0508019070 con 0x7f051015c040 2024-09-10T09:58:16.423 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0507e00640 1 -- 172.21.15.110:0/2933272165 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508014a20 con 0x7f051015c040 2024-09-10T09:58:16.423 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05100b0de0 con 0x7f051015c040 2024-09-10T09:58:16.423 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0510151680 con 0x7f051015c040 2024-09-10T09:58:16.423 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0510072660 con 0x7f051015c040 2024-09-10T09:58:16.423 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.421+0000 7f0507e00640 1 -- 172.21.15.110:0/2933272165 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f0508014bc0 con 0x7f051015c040 2024-09-10T09:58:16.425 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.425+0000 7f0507e00640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f04f0067830 0x7f04f0069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.425 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.425+0000 7f0507e00640 1 -- 172.21.15.110:0/2933272165 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f0508085fc0 con 0x7f051015c040 2024-09-10T09:58:16.430 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.429+0000 7f0507e00640 1 -- 172.21.15.110:0/2933272165 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f05080568d0 con 0x7f051015c040 2024-09-10T09:58:16.468 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.469+0000 7f050fe00640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f04f0067830 0x7f04f0069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.469 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.469+0000 7f050fe00640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f04f0067830 0x7f04f0069cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0510151b70 tx=0x7f0500009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.593 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f051015a7c0 con 0x7f051015c040 2024-09-10T09:58:16.594 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0507e00640 1 -- 172.21.15.110:0/2933272165 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12932 (secure 0 0 0) 0x7f050805abd0 con 0x7f051015c040 2024-09-10T09:58:16.594 INFO:teuthology.misc.health.smithi110.stdout: 2024-09-10T09:58:16.594 INFO:teuthology.misc.health.smithi110.stdout:{"epoch":19,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:16.216646+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:16.596 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f04f0067830 msgr2=0x7f04f0069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.596 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f04f0067830 0x7f04f0069cf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0510151b70 tx=0x7f0500009040 comp rx=0 tx=0).stop 2024-09-10T09:58:16.596 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f051015c040 msgr2=0x7f051013d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.596 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f051015c040 0x7f051013d360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0508007e40 tx=0x7f050800e4b0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.596 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 shutdown_connections 2024-09-10T09:58:16.597 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f04f0067830 0x7f04f0069cf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.597 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f051015c040 0x7f051013d360 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.597 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0510072ba0 0x7f051015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.597 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 --2- 172.21.15.110:0/2933272165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0510071ae0 0x7f0510151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.597 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.593+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 >> 172.21.15.110:0/2933272165 conn(0x7f0510067980 msgr2=0x7f0510158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.597 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.597+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 shutdown_connections 2024-09-10T09:58:16.597 INFO:teuthology.misc.health.smithi110.stderr:2024-09-10T09:58:16.597+0000 7f0517000640 1 -- 172.21.15.110:0/2933272165 wait complete. 2024-09-10T09:58:16.613 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T09:58:16.613 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T09:58:16.613 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T09:58:16.613 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T09:58:16.617 INFO:tasks.ceph.mds.a:Started 2024-09-10T09:58:16.617 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T09:58:16.617 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T09:58:16.620 INFO:tasks.ceph.mds.c:Started 2024-09-10T09:58:16.620 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T09:58:16.620 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T09:58:16.623 INFO:tasks.ceph.mds.b:Started 2024-09-10T09:58:16.623 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T09:58:16.623 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T09:58:16.628 INFO:tasks.ceph.mds.d:Started 2024-09-10T09:58:16.628 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T09:58:16.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:16.715 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T09:58:16.716 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T09:58:16.718 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T09:58:16.718 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T09:58:16.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c6b400640 1 Processor -- start 2024-09-10T09:58:16.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c6b400640 1 -- start start 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c6b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8c64155970 0x7f8c640cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6406f1e0 0x7f8c640ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c6b400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c64155d50 con 0x7f8c6415c560 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c6b400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8c640b4570 con 0x7f8c64155970 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c6b400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640b34d0 con 0x7f8c6406f1e0 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c5be00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f8c6415c560 0x7f8c6406ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:36450/0 (socket says 172.21.15.110:36450) 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c69400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6406f1e0 0x7f8c640ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c5be00640 1 -- 172.21.15.110:0/2302450749 learned_addr learned my addr 172.21.15.110:0/2302450749 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c68a00640 1 --2- 172.21.15.110:0/2302450749 >> v2:172.21.15.110:3300/0 conn(0x7f8c64155970 0x7f8c640cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c69400640 1 -- 172.21.15.110:0/2302450749 >> v2:172.21.15.110:3300/0 conn(0x7f8c64155970 msgr2=0x7f8c640cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c69400640 1 --2- 172.21.15.110:0/2302450749 >> v2:172.21.15.110:3300/0 conn(0x7f8c64155970 0x7f8c640cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c69400640 1 -- 172.21.15.110:0/2302450749 >> v1:172.21.15.145:6789/0 conn(0x7f8c6415c560 legacy=0x7f8c6406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c69400640 1 -- 172.21.15.110:0/2302450749 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c640b37b0 con 0x7f8c6406f1e0 2024-09-10T09:58:16.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.777+0000 7f8c63e00640 1 -- 172.21.15.110:0/2302450749 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4221937717 0 0) 0x7f8c64155d50 con 0x7f8c6415c560 2024-09-10T09:58:16.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c69400640 1 --2- 172.21.15.110:0/2302450749 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6406f1e0 0x7f8c640ce420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009d90 tx=0x7f8c5c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11502b500756c889 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c63e00640 1 -- 172.21.15.110:0/2302450749 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0b2070 con 0x7f8c6406f1e0 2024-09-10T09:58:16.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c63e00640 1 -- 172.21.15.110:0/2302450749 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0a5a00 con 0x7f8c6406f1e0 2024-09-10T09:58:16.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c63e00640 1 -- 172.21.15.110:0/2302450749 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0a5dd0 con 0x7f8c6406f1e0 2024-09-10T09:58:16.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c6b400640 1 -- 172.21.15.110:0/2302450749 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6406f1e0 msgr2=0x7f8c640ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c6b400640 1 --2- 172.21.15.110:0/2302450749 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6406f1e0 0x7f8c640ce420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009d90 tx=0x7f8c5c01c940 comp rx=0 tx=0).stop 2024-09-10T09:58:16.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c6b400640 1 -- 172.21.15.110:0/2302450749 shutdown_connections 2024-09-10T09:58:16.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c6b400640 1 --2- 172.21.15.110:0/2302450749 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6406f1e0 0x7f8c640ce420 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c6b400640 1 --2- 172.21.15.110:0/2302450749 >> v2:172.21.15.110:3300/0 conn(0x7f8c64155970 0x7f8c640cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c6b400640 1 -- 172.21.15.110:0/2302450749 >> 172.21.15.110:0/2302450749 conn(0x7f8c6406bc50 msgr2=0x7f8c6406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c6b400640 1 -- 172.21.15.110:0/2302450749 shutdown_connections 2024-09-10T09:58:16.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.781+0000 7f8c6b400640 1 -- 172.21.15.110:0/2302450749 wait complete. 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 Processor -- start 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 -- start start 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c6406f1e0 0x7f8c64153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c64155970 0x7f8c64164ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6415c560 0x7f8c64162c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640cf170 con 0x7f8c64155970 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640cce00 con 0x7f8c6406f1e0 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640cf2f0 con 0x7f8c6415c560 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c5be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c64155970 0x7f8c64164ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c5be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c64155970 0x7f8c64164ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44756/0 (socket says 172.21.15.110:44756) 2024-09-10T09:58:16.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c5be00640 1 -- 172.21.15.110:0/3800025170 learned_addr learned my addr 172.21.15.110:0/3800025170 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c68a00640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c6406f1e0 0x7f8c64153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c69400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6415c560 0x7f8c64162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c69400640 1 -- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c6406f1e0 msgr2=0x7f8c64153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c69400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c6406f1e0 0x7f8c64153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c69400640 1 -- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c64155970 msgr2=0x7f8c64164ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c69400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c64155970 0x7f8c64164ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c69400640 1 -- 172.21.15.110:0/3800025170 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c64162080 con 0x7f8c6415c560 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c69400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6415c560 0x7f8c64162c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0ac000 tx=0x7f8c5c0196e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c61600640 1 -- 172.21.15.110:0/3800025170 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0b2050 con 0x7f8c6415c560 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c61600640 1 -- 172.21.15.110:0/3800025170 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0b3040 con 0x7f8c6415c560 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c61600640 1 -- 172.21.15.110:0/3800025170 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0ad950 con 0x7f8c6415c560 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.785+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c64163a30 con 0x7f8c6415c560 2024-09-10T09:58:16.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.789+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c64163ff0 con 0x7f8c6415c560 2024-09-10T09:58:16.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.789+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c640ce420 con 0x7f8c6415c560 2024-09-10T09:58:16.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.789+0000 7f8c61600640 1 -- 172.21.15.110:0/3800025170 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f8c5c0c0020 con 0x7f8c6415c560 2024-09-10T09:58:16.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.789+0000 7f8c61600640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8c40067830 0x7f8c40069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.789+0000 7f8c61600640 1 -- 172.21.15.110:0/3800025170 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f8c5c120300 con 0x7f8c6415c560 2024-09-10T09:58:16.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.789+0000 7f8c68a00640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8c40067830 0x7f8c40069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.789+0000 7f8c68a00640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8c40067830 0x7f8c40069cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c54007a60 tx=0x7f8c54008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.793+0000 7f8c61600640 1 -- 172.21.15.110:0/3800025170 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8c5c016560 con 0x7f8c6415c560 2024-09-10T09:58:16.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c6415b690 con 0x7f8c6415c560 2024-09-10T09:58:16.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c61600640 1 -- 172.21.15.110:0/3800025170 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7f8c5c0f0c00 con 0x7f8c6415c560 2024-09-10T09:58:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:16.989 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":2,"btime":"2024-09-10T09:58:16:729838+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T09:58:16.989 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 2 2024-09-10T09:58:16.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8c40067830 msgr2=0x7f8c40069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8c40067830 0x7f8c40069cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c54007a60 tx=0x7f8c54008040 comp rx=0 tx=0).stop 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6415c560 msgr2=0x7f8c64162c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6415c560 0x7f8c64162c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0ac000 tx=0x7f8c5c0196e0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 shutdown_connections 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8c40067830 0x7f8c40069cf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c6415c560 0x7f8c64162c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c64155970 0x7f8c64164ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 --2- 172.21.15.110:0/3800025170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c6406f1e0 0x7f8c64153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 >> 172.21.15.110:0/3800025170 conn(0x7f8c6406bc50 msgr2=0x7f8c640713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 shutdown_connections 2024-09-10T09:58:16.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:16.989+0000 7f8c6b400640 1 -- 172.21.15.110:0/3800025170 wait complete. 2024-09-10T09:58:18.532 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:58:18.536 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:58:19.011 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:19.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4ce400640 1 Processor -- start 2024-09-10T09:58:19.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4ce400640 1 -- start start 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4ce400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd4c8155970 0x7fd4c80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4ce400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c8155d50 con 0x7fd4c806f1e0 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4ce400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c80b34d0 con 0x7fd4c815c560 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4ce400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c80b4570 con 0x7fd4c8155970 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4c7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd4c8155970 0x7fd4c80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4c7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd4c8155970 0x7fd4c80cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46414/0 (socket says 172.21.15.110:46414) 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4c7400640 1 -- 172.21.15.110:0/1031187140 learned_addr learned my addr 172.21.15.110:0/1031187140 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4c7400640 1 -- 172.21.15.110:0/1031187140 >> v1:172.21.15.145:6789/0 conn(0x7fd4c815c560 legacy=0x7fd4c806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4c7400640 1 -- 172.21.15.110:0/1031187140 >> v1:172.21.15.110:6789/0 conn(0x7fd4c806f1e0 legacy=0x7fd4c80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4c7400640 1 -- 172.21.15.110:0/1031187140 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c80b37b0 con 0x7fd4c8155970 2024-09-10T09:58:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.157+0000 7fd4c6000640 1 -- 172.21.15.110:0/1031187140 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988827180 0 0) 0x7fd4c8155d50 con 0x7fd4c806f1e0 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4c7400640 1 --2- 172.21.15.110:0/1031187140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8155970 0x7fd4c80cc8c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009870 tx=0x7fd4b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a8edc1c8816730 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4c6000640 1 -- 172.21.15.110:0/1031187140 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80aa650 con 0x7fd4c8155970 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4c6000640 1 -- 172.21.15.110:0/1031187140 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b80af070 con 0x7fd4c8155970 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4c6000640 1 -- 172.21.15.110:0/1031187140 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80aaa10 con 0x7fd4c8155970 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4ce400640 1 -- 172.21.15.110:0/1031187140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8155970 msgr2=0x7fd4c80cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4ce400640 1 --2- 172.21.15.110:0/1031187140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8155970 0x7fd4c80cc8c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009870 tx=0x7fd4b801c4f0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4ce400640 1 -- 172.21.15.110:0/1031187140 shutdown_connections 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4ce400640 1 --2- 172.21.15.110:0/1031187140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c8155970 0x7fd4c80cc8c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4ce400640 1 -- 172.21.15.110:0/1031187140 >> 172.21.15.110:0/1031187140 conn(0x7fd4c806bc50 msgr2=0x7fd4c806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4ce400640 1 -- 172.21.15.110:0/1031187140 shutdown_connections 2024-09-10T09:58:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4ce400640 1 -- 172.21.15.110:0/1031187140 wait complete. 2024-09-10T09:58:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.161+0000 7fd4ce400640 1 Processor -- start 2024-09-10T09:58:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 -- start start 2024-09-10T09:58:19.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c806f1e0 0x7fd4c815af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd4c8155970 0x7fd4c815ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4c815c560 0x7fd4c8150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c80cf340 con 0x7fd4c806f1e0 2024-09-10T09:58:19.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c80cce00 con 0x7fd4c815c560 2024-09-10T09:58:19.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c80cf4c0 con 0x7fd4c8155970 2024-09-10T09:58:19.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c806f1e0 0x7fd4c815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c806f1e0 0x7fd4c815af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46420/0 (socket says 172.21.15.110:46420) 2024-09-10T09:58:19.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 -- 172.21.15.110:0/503486278 learned_addr learned my addr 172.21.15.110:0/503486278 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 -- 172.21.15.110:0/503486278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd4c8155970 msgr2=0x7fd4c815ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd4c8155970 0x7fd4c815ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 -- 172.21.15.110:0/503486278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4c815c560 msgr2=0x7fd4c8150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4c815c560 0x7fd4c8150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 -- 172.21.15.110:0/503486278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c8159570 con 0x7fd4c806f1e0 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4c7400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c806f1e0 0x7fd4c815af40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80ac040 tx=0x7fd4b801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4bf400640 1 -- 172.21.15.110:0/503486278 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80b9070 con 0x7fd4c806f1e0 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4bf400640 1 -- 172.21.15.110:0/503486278 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b80a6040 con 0x7fd4c806f1e0 2024-09-10T09:58:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c8159850 con 0x7fd4c806f1e0 2024-09-10T09:58:19.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4bf400640 1 -- 172.21.15.110:0/503486278 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8004020 con 0x7fd4c806f1e0 2024-09-10T09:58:19.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c815a0f0 con 0x7fd4c806f1e0 2024-09-10T09:58:19.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.165+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c8159be0 con 0x7fd4c806f1e0 2024-09-10T09:58:19.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.169+0000 7fd4bf400640 1 -- 172.21.15.110:0/503486278 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd4b80041c0 con 0x7fd4c806f1e0 2024-09-10T09:58:19.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.169+0000 7fd4bf400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd4a80678c0 0x7fd4a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.169+0000 7fd4bf400640 1 -- 172.21.15.110:0/503486278 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fd4b811fc00 con 0x7fd4c806f1e0 2024-09-10T09:58:19.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.169+0000 7fd4c6a00640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd4a80678c0 0x7fd4a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.169+0000 7fd4c6a00640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd4a80678c0 0x7fd4a8069d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4c81fa5a0 tx=0x7fd4b4006bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.173+0000 7fd4bf400640 1 -- 172.21.15.110:0/503486278 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd4b8016110 con 0x7fd4c806f1e0 2024-09-10T09:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.361+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4c80ce3e0 con 0x7fd4c806f1e0 2024-09-10T09:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4bf400640 1 -- 172.21.15.110:0/503486278 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3918 (secure 0 0 0) 0x7fd4b80a93b0 con 0x7fd4c806f1e0 2024-09-10T09:58:19.366 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:19.366 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":3,"btime":"2024-09-10T09:58:17:731113+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 3 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd4a80678c0 msgr2=0x7fd4a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd4a80678c0 0x7fd4a8069d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4c81fa5a0 tx=0x7fd4b4006bd0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c806f1e0 msgr2=0x7fd4c815af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c806f1e0 0x7fd4c815af40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80ac040 tx=0x7fd4b801ad30 comp rx=0 tx=0).stop 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 shutdown_connections 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd4a80678c0 0x7fd4a8069d80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4c815c560 0x7fd4c8150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd4c8155970 0x7fd4c815ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 --2- 172.21.15.110:0/503486278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4c806f1e0 0x7fd4c815af40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 >> 172.21.15.110:0/503486278 conn(0x7fd4c806bc50 msgr2=0x7fd4c80ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 shutdown_connections 2024-09-10T09:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.365+0000 7fd4ce400640 1 -- 172.21.15.110:0/503486278 wait complete. 2024-09-10T09:58:19.385 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T09:58:19.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.517+0000 7fa3e2400640 1 Processor -- start 2024-09-10T09:58:19.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e2400640 1 -- start start 2024-09-10T09:58:19.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e2400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa3dc15e470 0x7fa3dc0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e2400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc0738f0 0x7fa3dc06d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e2400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc06d020 con 0x7fa3dc162020 2024-09-10T09:58:19.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e2400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa3dc0b6870 con 0x7fa3dc15e470 2024-09-10T09:58:19.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e2400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa3dc0b5c60 con 0x7fa3dc0738f0 2024-09-10T09:58:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3dbe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa3dc162020 0x7fa3dc0731e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41714/0 (socket says 172.21.15.110:41714) 2024-09-10T09:58:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3dbe00640 1 -- 172.21.15.110:0/1024544508 learned_addr learned my addr 172.21.15.110:0/1024544508 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:19.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e1000640 1 --2- 172.21.15.110:0/1024544508 >> v2:172.21.15.145:3300/0 conn(0x7fa3dc15e470 0x7fa3dc0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e1a00640 1 --2- 172.21.15.110:0/1024544508 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc0738f0 0x7fa3dc06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3db400640 1 -- 172.21.15.110:0/1024544508 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1661157489 0 0) 0x7fa3dc06d020 con 0x7fa3dc162020 2024-09-10T09:58:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3db400640 1 -- 172.21.15.110:0/1024544508 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3c4003530 con 0x7fa3dc162020 2024-09-10T09:58:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e1000640 1 -- 172.21.15.110:0/1024544508 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc0738f0 msgr2=0x7fa3dc06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e1000640 1 --2- 172.21.15.110:0/1024544508 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc0738f0 0x7fa3dc06d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e1000640 1 -- 172.21.15.110:0/1024544508 >> v1:172.21.15.110:6789/0 conn(0x7fa3dc162020 legacy=0x7fa3dc0731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.521+0000 7fa3e1000640 1 -- 172.21.15.110:0/1024544508 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc0b5f40 con 0x7fa3dc15e470 2024-09-10T09:58:19.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e1000640 1 --2- 172.21.15.110:0/1024544508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc15e470 0x7fa3dc0b55d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00097c0 tx=0x7fa3d001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14269eb796621ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3db400640 1 -- 172.21.15.110:0/1024544508 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00b5070 con 0x7fa3dc15e470 2024-09-10T09:58:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3db400640 1 -- 172.21.15.110:0/1024544508 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d00a8990 con 0x7fa3dc15e470 2024-09-10T09:58:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3db400640 1 -- 172.21.15.110:0/1024544508 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00a8d40 con 0x7fa3dc15e470 2024-09-10T09:58:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 -- 172.21.15.110:0/1024544508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc15e470 msgr2=0x7fa3dc0b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 --2- 172.21.15.110:0/1024544508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc15e470 0x7fa3dc0b55d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00097c0 tx=0x7fa3d001ec90 comp rx=0 tx=0).stop 2024-09-10T09:58:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 -- 172.21.15.110:0/1024544508 shutdown_connections 2024-09-10T09:58:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 --2- 172.21.15.110:0/1024544508 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc0738f0 0x7fa3dc06d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 --2- 172.21.15.110:0/1024544508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc15e470 0x7fa3dc0b55d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 -- 172.21.15.110:0/1024544508 >> 172.21.15.110:0/1024544508 conn(0x7fa3dc0b42d0 msgr2=0x7fa3dc0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 -- 172.21.15.110:0/1024544508 shutdown_connections 2024-09-10T09:58:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 -- 172.21.15.110:0/1024544508 wait complete. 2024-09-10T09:58:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 Processor -- start 2024-09-10T09:58:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.525+0000 7fa3e2400640 1 -- start start 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc0738f0 0x7fa3dc152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc15e470 0x7fa3dc15f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3dc162020 0x7fa3dc06df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc15e850 con 0x7fa3dc162020 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc0b6500 con 0x7fa3dc0738f0 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa3dc06dc60 con 0x7fa3dc15e470 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3dc162020 0x7fa3dc06df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3dc162020 0x7fa3dc06df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46422/0 (socket says 172.21.15.110:46422) 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 -- 172.21.15.110:0/3777065662 learned_addr learned my addr 172.21.15.110:0/3777065662 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1000640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc0738f0 0x7fa3dc152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3dbe00640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc15e470 0x7fa3dc15f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 -- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc15e470 msgr2=0x7fa3dc15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc15e470 0x7fa3dc15f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 -- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc0738f0 msgr2=0x7fa3dc152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc0738f0 0x7fa3dc152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 -- 172.21.15.110:0/3777065662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc0cccc0 con 0x7fa3dc162020 2024-09-10T09:58:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3dbe00640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc15e470 0x7fa3dc15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e1a00640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3dc162020 0x7fa3dc06df80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400efc0 tx=0x7fa3d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3d8c00640 1 -- 172.21.15.110:0/3777065662 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4018020 con 0x7fa3dc162020 2024-09-10T09:58:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3d8c00640 1 -- 172.21.15.110:0/3777065662 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d4014480 con 0x7fa3dc162020 2024-09-10T09:58:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc0ccfa0 con 0x7fa3dc162020 2024-09-10T09:58:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3d8c00640 1 -- 172.21.15.110:0/3777065662 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4014870 con 0x7fa3dc162020 2024-09-10T09:58:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc160600 con 0x7fa3dc162020 2024-09-10T09:58:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.529+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3dc06d720 con 0x7fa3dc162020 2024-09-10T09:58:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.533+0000 7fa3d8c00640 1 -- 172.21.15.110:0/3777065662 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa3d401b020 con 0x7fa3dc162020 2024-09-10T09:58:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.533+0000 7fa3d8c00640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3bc0678f0 0x7fa3bc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.533+0000 7fa3d8c00640 1 -- 172.21.15.110:0/3777065662 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa3d4086780 con 0x7fa3dc162020 2024-09-10T09:58:19.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.533+0000 7fa3e1000640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3bc0678f0 0x7fa3bc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.533+0000 7fa3e1000640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3bc0678f0 0x7fa3bc069db0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00b2000 tx=0x7fa3d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.537+0000 7fa3d8c00640 1 -- 172.21.15.110:0/3777065662 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa3d4022030 con 0x7fa3dc162020 2024-09-10T09:58:19.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa3dc159b90 con 0x7fa3dc162020 2024-09-10T09:58:19.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3d8c00640 1 -- 172.21.15.110:0/3777065662 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7fa3d4010040 con 0x7fa3dc162020 2024-09-10T09:58:19.697 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:19.698 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T09:58:19.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:19.698 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T09:58:19.698 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T09:58:19.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:19.698 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T09:58:19.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3bc0678f0 msgr2=0x7fa3bc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3bc0678f0 0x7fa3bc069db0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00b2000 tx=0x7fa3d0008040 comp rx=0 tx=0).stop 2024-09-10T09:58:19.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3dc162020 msgr2=0x7fa3dc06df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3dc162020 0x7fa3dc06df80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400efc0 tx=0x7fa3d400c490 comp rx=0 tx=0).stop 2024-09-10T09:58:19.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 shutdown_connections 2024-09-10T09:58:19.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3bc0678f0 0x7fa3bc069db0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3dc162020 0x7fa3dc06df80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3dc15e470 0x7fa3dc15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 --2- 172.21.15.110:0/3777065662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3dc0738f0 0x7fa3dc152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 >> 172.21.15.110:0/3777065662 conn(0x7fa3dc0b42d0 msgr2=0x7fa3dc0b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:19.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 shutdown_connections 2024-09-10T09:58:19.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.697+0000 7fa3e2400640 1 -- 172.21.15.110:0/3777065662 wait complete. 2024-09-10T09:58:19.717 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T09:58:19.717 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T09:58:19.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb7ea00640 1 Processor -- start 2024-09-10T09:58:19.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb7ea00640 1 -- start start 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb7ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcb780cfe30 0x7fcb78071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb7ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcb7806e7a0 0x7fcb780cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb7ea00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb7806d2d0 con 0x7fcb780cd510 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb7ea00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcb7815d9a0 con 0x7fcb780cfe30 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb7ea00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcb7806d5b0 con 0x7fcb7806e7a0 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb77e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcb780cfe30 0x7fcb78071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb77e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcb780cfe30 0x7fcb78071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46436/0 (socket says 172.21.15.110:46436) 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb77e00640 1 -- 172.21.15.110:0/1061211100 learned_addr learned my addr 172.21.15.110:0/1061211100 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb7ca00640 1 --2- 172.21.15.110:0/1061211100 >> v2:172.21.15.145:3300/0 conn(0x7fcb7806e7a0 0x7fcb780cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb77e00640 1 -- 172.21.15.110:0/1061211100 >> v2:172.21.15.145:3300/0 conn(0x7fcb7806e7a0 msgr2=0x7fcb780cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb77e00640 1 --2- 172.21.15.110:0/1061211100 >> v2:172.21.15.145:3300/0 conn(0x7fcb7806e7a0 0x7fcb780cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb77e00640 1 -- 172.21.15.110:0/1061211100 >> v1:172.21.15.145:6789/0 conn(0x7fcb780cd510 legacy=0x7fcb7806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:19.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.857+0000 7fcb77e00640 1 -- 172.21.15.110:0/1061211100 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7806d890 con 0x7fcb780cfe30 2024-09-10T09:58:19.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb77e00640 1 --2- 172.21.15.110:0/1061211100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb780cfe30 0x7fcb78071a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb7806cb00 tx=0x7fcb6801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2950f6d263a237 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb77400640 1 -- 172.21.15.110:0/1061211100 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb680a56f0 con 0x7fcb780cfe30 2024-09-10T09:58:19.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb77400640 1 -- 172.21.15.110:0/1061211100 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb680a5890 con 0x7fcb780cfe30 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb77400640 1 -- 172.21.15.110:0/1061211100 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb680a5c60 con 0x7fcb780cfe30 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb7ea00640 1 -- 172.21.15.110:0/1061211100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb780cfe30 msgr2=0x7fcb78071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/1061211100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb780cfe30 0x7fcb78071a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb7806cb00 tx=0x7fcb6801c920 comp rx=0 tx=0).stop 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb7ea00640 1 -- 172.21.15.110:0/1061211100 shutdown_connections 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/1061211100 >> v2:172.21.15.145:3300/0 conn(0x7fcb7806e7a0 0x7fcb780cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/1061211100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb780cfe30 0x7fcb78071a40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb7ea00640 1 -- 172.21.15.110:0/1061211100 >> 172.21.15.110:0/1061211100 conn(0x7fcb7806bc50 msgr2=0x7fcb7806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb7ea00640 1 -- 172.21.15.110:0/1061211100 shutdown_connections 2024-09-10T09:58:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.861+0000 7fcb7ea00640 1 -- 172.21.15.110:0/1061211100 wait complete. 2024-09-10T09:58:19.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 Processor -- start 2024-09-10T09:58:19.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 -- start start 2024-09-10T09:58:19.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb7806e7a0 0x7fcb78161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb780cd510 0x7fcb78151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb780cfe30 0x7fcb780b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78072190 con 0x7fcb7806e7a0 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7815d6c0 con 0x7fcb780cd510 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcb78169180 con 0x7fcb780cfe30 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb7806e7a0 0x7fcb78161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb7806e7a0 0x7fcb78161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46446/0 (socket says 172.21.15.110:46446) 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 -- 172.21.15.110:0/3861961146 learned_addr learned my addr 172.21.15.110:0/3861961146 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 -- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb780cfe30 msgr2=0x7fcb780b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ca00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb780cfe30 0x7fcb780b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb6f400640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb780cd510 0x7fcb78151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb780cfe30 0x7fcb780b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 -- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb780cd510 msgr2=0x7fcb78151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb780cd510 0x7fcb78151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 -- 172.21.15.110:0/3861961146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb780af210 con 0x7fcb7806e7a0 2024-09-10T09:58:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb6f400640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb780cd510 0x7fcb78151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb77e00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb7806e7a0 0x7fcb78161120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb680af040 tx=0x7fcb680196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb74c00640 1 -- 172.21.15.110:0/3861961146 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb680bc020 con 0x7fcb7806e7a0 2024-09-10T09:58:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb74c00640 1 -- 172.21.15.110:0/3861961146 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb680b88c0 con 0x7fcb7806e7a0 2024-09-10T09:58:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb74c00640 1 -- 172.21.15.110:0/3861961146 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68009510 con 0x7fcb7806e7a0 2024-09-10T09:58:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb781522d0 con 0x7fcb7806e7a0 2024-09-10T09:58:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.865+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb78152890 con 0x7fcb7806e7a0 2024-09-10T09:58:19.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.869+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7806e2a0 con 0x7fcb7806e7a0 2024-09-10T09:58:19.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.869+0000 7fcb74c00640 1 -- 172.21.15.110:0/3861961146 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb680bf020 con 0x7fcb7806e7a0 2024-09-10T09:58:19.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.869+0000 7fcb74c00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcb540678c0 0x7fcb54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:19.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.869+0000 7fcb74c00640 1 -- 172.21.15.110:0/3861961146 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fcb68120690 con 0x7fcb7806e7a0 2024-09-10T09:58:19.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.869+0000 7fcb6f400640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcb540678c0 0x7fcb54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:19.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.869+0000 7fcb6f400640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcb540678c0 0x7fcb54069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb780ccb70 tx=0x7fcb60008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:19.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:19.873+0000 7fcb74c00640 1 -- 172.21.15.110:0/3861961146 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcb680c6070 con 0x7fcb7806e7a0 2024-09-10T09:58:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.033+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcb7806c280 con 0x7fcb7806e7a0 2024-09-10T09:58:20.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.745+0000 7fcb74c00640 1 -- 172.21.15.110:0/3861961146 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7fcb680165a0 con 0x7fcb7806e7a0 2024-09-10T09:58:20.745 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T09:58:20.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcb540678c0 msgr2=0x7fcb54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:20.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcb540678c0 0x7fcb54069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb780ccb70 tx=0x7fcb60008040 comp rx=0 tx=0).stop 2024-09-10T09:58:20.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb7806e7a0 msgr2=0x7fcb78161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:20.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb7806e7a0 0x7fcb78161120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb680af040 tx=0x7fcb680196c0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 shutdown_connections 2024-09-10T09:58:20.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcb540678c0 0x7fcb54069d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb780cfe30 0x7fcb780b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb780cd510 0x7fcb78151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 --2- 172.21.15.110:0/3861961146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb7806e7a0 0x7fcb78161120 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 >> 172.21.15.110:0/3861961146 conn(0x7fcb7806bc50 msgr2=0x7fcb781543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:20.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 shutdown_connections 2024-09-10T09:58:20.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.749+0000 7fcb7ea00640 1 -- 172.21.15.110:0/3861961146 wait complete. 2024-09-10T09:58:20.777 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T09:58:20.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc83b800640 1 Processor -- start 2024-09-10T09:58:20.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc83b800640 1 -- start start 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc83b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc83406edf0 0x7fc8341693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc83b800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8341482e0 con 0x7fc834147f00 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc83b800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83415ee20 con 0x7fc8340aef50 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc83b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc8340b34d0 con 0x7fc83406edf0 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc838e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc834147f00 0x7fc8340ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41720/0 (socket says 172.21.15.110:41720) 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc838e00640 1 -- 172.21.15.110:0/4172616421 learned_addr learned my addr 172.21.15.110:0/4172616421 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc839800640 1 --2- 172.21.15.110:0/4172616421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc83406edf0 0x7fc8341693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc833400640 1 -- 172.21.15.110:0/4172616421 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3506632458 0 0) 0x7fc8341482e0 con 0x7fc834147f00 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc833400640 1 -- 172.21.15.110:0/4172616421 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc818003530 con 0x7fc834147f00 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc833400640 1 -- 172.21.15.110:0/4172616421 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945896323 0 0) 0x7fc83415ee20 con 0x7fc8340aef50 2024-09-10T09:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc833400640 1 -- 172.21.15.110:0/4172616421 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8341482e0 con 0x7fc8340aef50 2024-09-10T09:58:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc833400640 1 -- 172.21.15.110:0/4172616421 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1329416352 0 0) 0x7fc818003530 con 0x7fc834147f00 2024-09-10T09:58:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc839800640 1 -- 172.21.15.110:0/4172616421 >> v1:172.21.15.145:6789/0 conn(0x7fc8340aef50 legacy=0x7fc83406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc839800640 1 -- 172.21.15.110:0/4172616421 >> v1:172.21.15.110:6789/0 conn(0x7fc834147f00 legacy=0x7fc8340ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.921+0000 7fc839800640 1 -- 172.21.15.110:0/4172616421 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83415f100 con 0x7fc83406edf0 2024-09-10T09:58:20.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc839800640 1 --2- 172.21.15.110:0/4172616421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc83406edf0 0x7fc8341693f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009dc0 tx=0x7fc82c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d227d7a7a2e274f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:20.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc833400640 1 -- 172.21.15.110:0/4172616421 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824001680 con 0x7fc83406edf0 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc833400640 1 -- 172.21.15.110:0/4172616421 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c0a6500 con 0x7fc83406edf0 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc833400640 1 -- 172.21.15.110:0/4172616421 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0a68b0 con 0x7fc83406edf0 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc83b800640 1 -- 172.21.15.110:0/4172616421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc83406edf0 msgr2=0x7fc8341693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc83b800640 1 --2- 172.21.15.110:0/4172616421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc83406edf0 0x7fc8341693f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009dc0 tx=0x7fc82c01c8b0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc83b800640 1 -- 172.21.15.110:0/4172616421 shutdown_connections 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc83b800640 1 --2- 172.21.15.110:0/4172616421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc83406edf0 0x7fc8341693f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc83b800640 1 -- 172.21.15.110:0/4172616421 >> 172.21.15.110:0/4172616421 conn(0x7fc83406bc50 msgr2=0x7fc83406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc83b800640 1 -- 172.21.15.110:0/4172616421 shutdown_connections 2024-09-10T09:58:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.925+0000 7fc83b800640 1 -- 172.21.15.110:0/4172616421 wait complete. 2024-09-10T09:58:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 Processor -- start 2024-09-10T09:58:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 -- start start 2024-09-10T09:58:20.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc83406edf0 0x7fc8340cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:20.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc8340aef50 0x7fc8340ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:20.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc834147f00 0x7fc83415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:20.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc83416c6e0 con 0x7fc834147f00 2024-09-10T09:58:20.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc83416c560 con 0x7fc83406edf0 2024-09-10T09:58:20.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc83416c860 con 0x7fc8340aef50 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc834147f00 0x7fc83415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc834147f00 0x7fc83415ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46462/0 (socket says 172.21.15.110:46462) 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 -- 172.21.15.110:0/2450598464 learned_addr learned my addr 172.21.15.110:0/2450598464 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc833e00640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc8340aef50 0x7fc8340ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc838e00640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc83406edf0 0x7fc8340cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 -- 172.21.15.110:0/2450598464 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc8340aef50 msgr2=0x7fc8340ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc8340aef50 0x7fc8340ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 -- 172.21.15.110:0/2450598464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc83406edf0 msgr2=0x7fc8340cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc83406edf0 0x7fc8340cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 -- 172.21.15.110:0/2450598464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83413db70 con 0x7fc834147f00 2024-09-10T09:58:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc839800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc834147f00 0x7fc83415ce20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc82c01a4e0 tx=0x7fc82c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc830c00640 1 -- 172.21.15.110:0/2450598464 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c009450 con 0x7fc834147f00 2024-09-10T09:58:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc830c00640 1 -- 172.21.15.110:0/2450598464 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c0ba680 con 0x7fc834147f00 2024-09-10T09:58:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc830c00640 1 -- 172.21.15.110:0/2450598464 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0a6e60 con 0x7fc834147f00 2024-09-10T09:58:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83413de10 con 0x7fc834147f00 2024-09-10T09:58:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.929+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83413e3d0 con 0x7fc834147f00 2024-09-10T09:58:20.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.933+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83415ee20 con 0x7fc834147f00 2024-09-10T09:58:20.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.933+0000 7fc830c00640 1 -- 172.21.15.110:0/2450598464 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc82c0c2020 con 0x7fc834147f00 2024-09-10T09:58:20.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.933+0000 7fc830c00640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8140678c0 0x7fc814069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:20.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.933+0000 7fc830c00640 1 -- 172.21.15.110:0/2450598464 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7fc82c1215c0 con 0x7fc834147f00 2024-09-10T09:58:20.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.933+0000 7fc838e00640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8140678c0 0x7fc814069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:20.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.933+0000 7fc838e00640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8140678c0 0x7fc814069d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc824001000 tx=0x7fc824001160 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:20.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:20.937+0000 7fc830c00640 1 -- 172.21.15.110:0/2450598464 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc82c0164a0 con 0x7fc834147f00 2024-09-10T09:58:21.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.093+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc83406dcc0 con 0x7fc834147f00 2024-09-10T09:58:21.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.753+0000 7fc830c00640 1 -- 172.21.15.110:0/2450598464 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7fc82c0f1d00 con 0x7fc834147f00 2024-09-10T09:58:21.752 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T09:58:21.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.753+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8140678c0 msgr2=0x7fc814069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.753+0000 7fc83b800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8140678c0 0x7fc814069d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc824001000 tx=0x7fc824001160 comp rx=0 tx=0).stop 2024-09-10T09:58:21.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.753+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc834147f00 msgr2=0x7fc83415ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.753+0000 7fc83b800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc834147f00 0x7fc83415ce20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc82c01a4e0 tx=0x7fc82c019650 comp rx=0 tx=0).stop 2024-09-10T09:58:21.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.757+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 shutdown_connections 2024-09-10T09:58:21.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.757+0000 7fc83b800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8140678c0 0x7fc814069d80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.757+0000 7fc83b800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc834147f00 0x7fc83415ce20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.757+0000 7fc83b800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc8340aef50 0x7fc8340ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.757+0000 7fc83b800640 1 --2- 172.21.15.110:0/2450598464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc83406edf0 0x7fc8340cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.757+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 >> 172.21.15.110:0/2450598464 conn(0x7fc83406bc50 msgr2=0x7fc8340b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:21.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.757+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 shutdown_connections 2024-09-10T09:58:21.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.757+0000 7fc83b800640 1 -- 172.21.15.110:0/2450598464 wait complete. 2024-09-10T09:58:21.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T09:58:21.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d9e200640 1 Processor -- start 2024-09-10T09:58:21.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d9e200640 1 -- start start 2024-09-10T09:58:21.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d9e200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9d9806ec70 0x7f9d9806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d9e200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9d980b4c80 0x7f9d980b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d9e200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d98072690 con 0x7f9d9815f010 2024-09-10T09:58:21.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d9e200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9d98072970 con 0x7f9d9806ec70 2024-09-10T09:58:21.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d9e200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9d98072c50 con 0x7f9d980b4c80 2024-09-10T09:58:21.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d96a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9d9806ec70 0x7f9d9806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d96a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9d9806ec70 0x7f9d9806f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46478/0 (socket says 172.21.15.110:46478) 2024-09-10T09:58:21.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.933+0000 7f9d96a00640 1 -- 172.21.15.110:0/336125874 learned_addr learned my addr 172.21.15.110:0/336125874 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:21.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.937+0000 7f9d96a00640 1 -- 172.21.15.110:0/336125874 >> v2:172.21.15.145:3300/0 conn(0x7f9d980b4c80 msgr2=0x7f9d980b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:21.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.937+0000 7f9d96a00640 1 --2- 172.21.15.110:0/336125874 >> v2:172.21.15.145:3300/0 conn(0x7f9d980b4c80 0x7f9d980b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.937+0000 7f9d96a00640 1 -- 172.21.15.110:0/336125874 >> v1:172.21.15.110:6789/0 conn(0x7f9d9815f010 legacy=0x7f9d9815f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.937+0000 7f9d96a00640 1 -- 172.21.15.110:0/336125874 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d98072f30 con 0x7f9d9806ec70 2024-09-10T09:58:21.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.937+0000 7f9d96a00640 1 --2- 172.21.15.110:0/336125874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d9806f100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9d98062750 tx=0x7f9d8401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feb79525926fdcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.937+0000 7f9d96000640 1 -- 172.21.15.110:0/336125874 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840b0070 con 0x7f9d9806ec70 2024-09-10T09:58:21.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.937+0000 7f9d96000640 1 -- 172.21.15.110:0/336125874 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d840ab600 con 0x7f9d9806ec70 2024-09-10T09:58:21.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.937+0000 7f9d96000640 1 -- 172.21.15.110:0/336125874 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840ab9d0 con 0x7f9d9806ec70 2024-09-10T09:58:21.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- 172.21.15.110:0/336125874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 msgr2=0x7f9d9806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 --2- 172.21.15.110:0/336125874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d9806f100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9d98062750 tx=0x7f9d8401c490 comp rx=0 tx=0).stop 2024-09-10T09:58:21.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- 172.21.15.110:0/336125874 shutdown_connections 2024-09-10T09:58:21.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 --2- 172.21.15.110:0/336125874 >> v2:172.21.15.145:3300/0 conn(0x7f9d980b4c80 0x7f9d980b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 --2- 172.21.15.110:0/336125874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d9806f100 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- 172.21.15.110:0/336125874 >> 172.21.15.110:0/336125874 conn(0x7f9d9806ca70 msgr2=0x7f9d9806ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:21.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- 172.21.15.110:0/336125874 shutdown_connections 2024-09-10T09:58:21.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- 172.21.15.110:0/336125874 wait complete. 2024-09-10T09:58:21.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 Processor -- start 2024-09-10T09:58:21.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- start start 2024-09-10T09:58:21.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d98150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d980b4c80 0x7f9d981493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d9815f010 0x7f9d980afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98072450 con 0x7f9d9806ec70 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d980722d0 con 0x7f9d9815f010 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d9e200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9816b5f0 con 0x7f9d980b4c80 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d98150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d98150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46486/0 (socket says 172.21.15.110:46486) 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 -- 172.21.15.110:0/1071568477 learned_addr learned my addr 172.21.15.110:0/1071568477 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d96a00640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d980b4c80 0x7f9d981493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97e00640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d9815f010 0x7f9d980afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 -- 172.21.15.110:0/1071568477 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d980b4c80 msgr2=0x7f9d981493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d980b4c80 0x7f9d981493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 -- 172.21.15.110:0/1071568477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d9815f010 msgr2=0x7f9d980afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:21.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d9815f010 0x7f9d980afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:21.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 -- 172.21.15.110:0/1071568477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d980ae970 con 0x7f9d9806ec70 2024-09-10T09:58:21.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.941+0000 7f9d97400640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d98150950 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c420 tx=0x7f9d8800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d8f400640 1 -- 172.21.15.110:0/1071568477 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d880052d0 con 0x7f9d9806ec70 2024-09-10T09:58:21.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d8f400640 1 -- 172.21.15.110:0/1071568477 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d88005470 con 0x7f9d9806ec70 2024-09-10T09:58:21.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d8f400640 1 -- 172.21.15.110:0/1071568477 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88007260 con 0x7f9d9806ec70 2024-09-10T09:58:21.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d980aec50 con 0x7f9d9806ec70 2024-09-10T09:58:21.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d980af210 con 0x7f9d9806ec70 2024-09-10T09:58:21.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9806f100 con 0x7f9d9806ec70 2024-09-10T09:58:21.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d8f400640 1 -- 172.21.15.110:0/1071568477 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9d88007400 con 0x7f9d9806ec70 2024-09-10T09:58:21.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d8f400640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9d780678c0 0x7f9d78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:21.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d8f400640 1 -- 172.21.15.110:0/1071568477 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f9d88085210 con 0x7f9d9806ec70 2024-09-10T09:58:21.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d96a00640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9d780678c0 0x7f9d78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:21.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.945+0000 7f9d96a00640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9d780678c0 0x7f9d78069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d840ad000 tx=0x7f9d84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:21.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:21.953+0000 7f9d8f400640 1 -- 172.21.15.110:0/1071568477 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9d88055d10 con 0x7f9d9806ec70 2024-09-10T09:58:22.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.141+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9d98071ea0 con 0x7f9d9806ec70 2024-09-10T09:58:22.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.765+0000 7f9d8f400640 1 -- 172.21.15.110:0/1071568477 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f9d88055610 con 0x7f9d9806ec70 2024-09-10T09:58:22.767 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T09:58:22.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.765+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9d780678c0 msgr2=0x7f9d78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:22.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.765+0000 7f9d9e200640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9d780678c0 0x7f9d78069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d840ad000 tx=0x7f9d84008040 comp rx=0 tx=0).stop 2024-09-10T09:58:22.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.765+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 msgr2=0x7f9d98150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:22.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.765+0000 7f9d9e200640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d98150950 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c420 tx=0x7f9d8800c8e0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.769+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 shutdown_connections 2024-09-10T09:58:22.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.769+0000 7f9d9e200640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9d780678c0 0x7f9d78069d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.769+0000 7f9d9e200640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d9815f010 0x7f9d980afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.769+0000 7f9d9e200640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9d980b4c80 0x7f9d981493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.769+0000 7f9d9e200640 1 --2- 172.21.15.110:0/1071568477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d9806ec70 0x7f9d98150950 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.769+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 >> 172.21.15.110:0/1071568477 conn(0x7f9d9806ca70 msgr2=0x7f9d9814f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:22.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.769+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 shutdown_connections 2024-09-10T09:58:22.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.769+0000 7f9d9e200640 1 -- 172.21.15.110:0/1071568477 wait complete. 2024-09-10T09:58:22.809 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:22.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25fb800640 1 Processor -- start 2024-09-10T09:58:22.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25fb800640 1 -- start start 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25fb800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f25f4155970 0x7f25f40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25fb800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f406f1e0 0x7f25f40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25fb800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f4155d50 con 0x7f25f415c560 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25fb800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f25f40b4570 con 0x7f25f4155970 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25fb800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f25f40b34d0 con 0x7f25f406f1e0 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f8e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f25f4155970 0x7f25f40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f8e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f25f4155970 0x7f25f40cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46490/0 (socket says 172.21.15.110:46490) 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f8e00640 1 -- 172.21.15.110:0/3229511640 learned_addr learned my addr 172.21.15.110:0/3229511640 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f9800640 1 --2- 172.21.15.110:0/3229511640 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f406f1e0 0x7f25f40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f8e00640 1 -- 172.21.15.110:0/3229511640 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f406f1e0 msgr2=0x7f25f40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f8e00640 1 --2- 172.21.15.110:0/3229511640 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f406f1e0 0x7f25f40ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f8e00640 1 -- 172.21.15.110:0/3229511640 >> v1:172.21.15.145:6789/0 conn(0x7f25f415c560 legacy=0x7f25f406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:22.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f8e00640 1 -- 172.21.15.110:0/3229511640 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f40b37b0 con 0x7f25f4155970 2024-09-10T09:58:22.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.945+0000 7f25f9800640 1 --2- 172.21.15.110:0/3229511640 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f406f1e0 0x7f25f40ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:22.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25f8e00640 1 --2- 172.21.15.110:0/3229511640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f4155970 0x7f25f40cc8c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25f406cc30 tx=0x7f25e401eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c95d1766d7f9552 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:22.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25f3400640 1 -- 172.21.15.110:0/3229511640 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e400b2d0 con 0x7f25f4155970 2024-09-10T09:58:22.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25f3400640 1 -- 172.21.15.110:0/3229511640 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e400b470 con 0x7f25f4155970 2024-09-10T09:58:22.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25f3400640 1 -- 172.21.15.110:0/3229511640 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e40a7c30 con 0x7f25f4155970 2024-09-10T09:58:22.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25fb800640 1 -- 172.21.15.110:0/3229511640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f4155970 msgr2=0x7f25f40cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:22.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25fb800640 1 --2- 172.21.15.110:0/3229511640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f4155970 0x7f25f40cc8c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25f406cc30 tx=0x7f25e401eb80 comp rx=0 tx=0).stop 2024-09-10T09:58:22.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25fb800640 1 -- 172.21.15.110:0/3229511640 shutdown_connections 2024-09-10T09:58:22.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25fb800640 1 --2- 172.21.15.110:0/3229511640 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f406f1e0 0x7f25f40ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25fb800640 1 --2- 172.21.15.110:0/3229511640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f4155970 0x7f25f40cc8c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25fb800640 1 -- 172.21.15.110:0/3229511640 >> 172.21.15.110:0/3229511640 conn(0x7f25f406bc50 msgr2=0x7f25f406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:22.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25fb800640 1 -- 172.21.15.110:0/3229511640 shutdown_connections 2024-09-10T09:58:22.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.949+0000 7f25fb800640 1 -- 172.21.15.110:0/3229511640 wait complete. 2024-09-10T09:58:22.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 Processor -- start 2024-09-10T09:58:22.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 -- start start 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f406f1e0 0x7f25f4154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f4155970 0x7f25f40aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f415c560 0x7f25f4158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25f40cf170 con 0x7f25f415c560 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f25f40cce00 con 0x7f25f406f1e0 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f25f40cf2f0 con 0x7f25f4155970 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f415c560 0x7f25f4158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f415c560 0x7f25f4158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46500/0 (socket says 172.21.15.110:46500) 2024-09-10T09:58:22.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 -- 172.21.15.110:0/4215836001 learned_addr learned my addr 172.21.15.110:0/4215836001 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f3e00640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f4155970 0x7f25f40aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f8e00640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f406f1e0 0x7f25f4154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 -- 172.21.15.110:0/4215836001 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f4155970 msgr2=0x7f25f40aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f4155970 0x7f25f40aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 -- 172.21.15.110:0/4215836001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f406f1e0 msgr2=0x7f25f4154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f406f1e0 0x7f25f4154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 -- 172.21.15.110:0/4215836001 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f40ade30 con 0x7f25f415c560 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f9800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f415c560 0x7f25f4158c50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00c7f0 tx=0x7f25ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f0c00640 1 -- 172.21.15.110:0/4215836001 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec015070 con 0x7f25f415c560 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f0c00640 1 -- 172.21.15.110:0/4215836001 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec007860 con 0x7f25f415c560 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25f0c00640 1 -- 172.21.15.110:0/4215836001 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec007c50 con 0x7f25f415c560 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f40ae110 con 0x7f25f415c560 2024-09-10T09:58:22.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.953+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f40ae6d0 con 0x7f25f415c560 2024-09-10T09:58:22.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.957+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f40ce420 con 0x7f25f415c560 2024-09-10T09:58:22.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.957+0000 7f25f0c00640 1 -- 172.21.15.110:0/4215836001 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f25ec018070 con 0x7f25f415c560 2024-09-10T09:58:22.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.957+0000 7f25f0c00640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25d40678c0 0x7f25d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:22.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.957+0000 7f25f0c00640 1 -- 172.21.15.110:0/4215836001 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f25ec085e40 con 0x7f25f415c560 2024-09-10T09:58:22.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.957+0000 7f25f8e00640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25d40678c0 0x7f25d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:22.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.957+0000 7f25f8e00640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25d40678c0 0x7f25d4069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25e40b0000 tx=0x7f25e40a9070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:22.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:22.961+0000 7f25f0c00640 1 -- 172.21.15.110:0/4215836001 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f25ec056290 con 0x7f25f415c560 2024-09-10T09:58:23.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.121+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25f4076350 con 0x7f25f415c560 2024-09-10T09:58:23.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25f0c00640 1 -- 172.21.15.110:0/4215836001 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16714 (secure 0 0 0) 0x7f25ec05a590 con 0x7f25f415c560 2024-09-10T09:58:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:23.125 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":22,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:22.145867+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:23.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25d40678c0 msgr2=0x7f25d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25d40678c0 0x7f25d4069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25e40b0000 tx=0x7f25e40a9070 comp rx=0 tx=0).stop 2024-09-10T09:58:23.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f415c560 msgr2=0x7f25f4158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f415c560 0x7f25f4158c50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00c7f0 tx=0x7f25ec00ccb0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 shutdown_connections 2024-09-10T09:58:23.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25d40678c0 0x7f25d4069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f415c560 0x7f25f4158c50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25f4155970 0x7f25f40aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 --2- 172.21.15.110:0/4215836001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f406f1e0 0x7f25f4154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 >> 172.21.15.110:0/4215836001 conn(0x7f25f406bc50 msgr2=0x7f25f406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:23.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 shutdown_connections 2024-09-10T09:58:23.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.125+0000 7f25fb800640 1 -- 172.21.15.110:0/4215836001 wait complete. 2024-09-10T09:58:23.147 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:23.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a94800640 1 Processor -- start 2024-09-10T09:58:23.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a94800640 1 -- start start 2024-09-10T09:58:23.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a94800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1a8c0d0cb0 0x7f1a8c163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a94800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1a8c073290 0x7f1a8c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a94800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8c0706a0 con 0x7f1a8c06eb30 2024-09-10T09:58:23.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a94800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8c15ee90 con 0x7f1a8c073290 2024-09-10T09:58:23.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a94800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8c073c20 con 0x7f1a8c0d0cb0 2024-09-10T09:58:23.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a92800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1a8c073290 0x7f1a8c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a92800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1a8c073290 0x7f1a8c0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46502/0 (socket says 172.21.15.110:46502) 2024-09-10T09:58:23.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a91e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1a8c06eb30 0x7f1a8c071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41742/0 (socket says 172.21.15.110:41742) 2024-09-10T09:58:23.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a92800640 1 -- 172.21.15.110:0/3469455612 learned_addr learned my addr 172.21.15.110:0/3469455612 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:23.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a91400640 1 --2- 172.21.15.110:0/3469455612 >> v2:172.21.15.145:3300/0 conn(0x7f1a8c0d0cb0 0x7f1a8c163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a92800640 1 -- 172.21.15.110:0/3469455612 >> v2:172.21.15.145:3300/0 conn(0x7f1a8c0d0cb0 msgr2=0x7f1a8c163000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:23.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a92800640 1 --2- 172.21.15.110:0/3469455612 >> v2:172.21.15.145:3300/0 conn(0x7f1a8c0d0cb0 0x7f1a8c163000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a92800640 1 -- 172.21.15.110:0/3469455612 >> v1:172.21.15.110:6789/0 conn(0x7f1a8c06eb30 legacy=0x7f1a8c071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a90a00640 1 -- 172.21.15.110:0/3469455612 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141237738 0 0) 0x7f1a8c0706a0 con 0x7f1a8c06eb30 2024-09-10T09:58:23.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a92800640 1 -- 172.21.15.110:0/3469455612 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0d10f0 con 0x7f1a8c073290 2024-09-10T09:58:23.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.281+0000 7f1a91400640 1 --2- 172.21.15.110:0/3469455612 >> v2:172.21.15.145:3300/0 conn(0x7f1a8c0d0cb0 0x7f1a8c163000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:23.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a92800640 1 --2- 172.21.15.110:0/3469455612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c073290 0x7f1a8c0736e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c06a060 tx=0x7f1a8801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc94f793fdd9cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a90a00640 1 -- 172.21.15.110:0/3469455612 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880a5620 con 0x7f1a8c073290 2024-09-10T09:58:23.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a90a00640 1 -- 172.21.15.110:0/3469455612 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a880a57c0 con 0x7f1a8c073290 2024-09-10T09:58:23.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a90a00640 1 -- 172.21.15.110:0/3469455612 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880a5b90 con 0x7f1a8c073290 2024-09-10T09:58:23.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a94800640 1 -- 172.21.15.110:0/3469455612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c073290 msgr2=0x7f1a8c0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a94800640 1 --2- 172.21.15.110:0/3469455612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c073290 0x7f1a8c0736e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c06a060 tx=0x7f1a8801c960 comp rx=0 tx=0).stop 2024-09-10T09:58:23.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a94800640 1 -- 172.21.15.110:0/3469455612 shutdown_connections 2024-09-10T09:58:23.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a94800640 1 --2- 172.21.15.110:0/3469455612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c073290 0x7f1a8c0736e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a94800640 1 --2- 172.21.15.110:0/3469455612 >> v2:172.21.15.145:3300/0 conn(0x7f1a8c0d0cb0 0x7f1a8c163000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a94800640 1 -- 172.21.15.110:0/3469455612 >> 172.21.15.110:0/3469455612 conn(0x7f1a8c0d25c0 msgr2=0x7f1a8c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:23.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a94800640 1 -- 172.21.15.110:0/3469455612 shutdown_connections 2024-09-10T09:58:23.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.285+0000 7f1a94800640 1 -- 172.21.15.110:0/3469455612 wait complete. 2024-09-10T09:58:23.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 Processor -- start 2024-09-10T09:58:23.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 -- start start 2024-09-10T09:58:23.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c06eb30 0x7f1a8c0cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a8c073290 0x7f1a8c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a8c0d0cb0 0x7f1a8c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c070d60 con 0x7f1a8c06eb30 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c071c70 con 0x7f1a8c073290 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c06dd60 con 0x7f1a8c0d0cb0 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c06eb30 0x7f1a8c0cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c06eb30 0x7f1a8c0cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46510/0 (socket says 172.21.15.110:46510) 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 -- 172.21.15.110:0/2127239253 learned_addr learned my addr 172.21.15.110:0/2127239253 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a92800640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a8c0d0cb0 0x7f1a8c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91400640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a8c073290 0x7f1a8c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 -- 172.21.15.110:0/2127239253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a8c0d0cb0 msgr2=0x7f1a8c15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a8c0d0cb0 0x7f1a8c15bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 -- 172.21.15.110:0/2127239253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a8c073290 msgr2=0x7f1a8c15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a8c073290 0x7f1a8c15ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 -- 172.21.15.110:0/2127239253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0cd890 con 0x7f1a8c06eb30 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a91e00640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c06eb30 0x7f1a8c0cbb70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1a800011f0 tx=0x7f1a8000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a86000640 1 -- 172.21.15.110:0/2127239253 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80018070 con 0x7f1a8c06eb30 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a86000640 1 -- 172.21.15.110:0/2127239253 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a800074e0 con 0x7f1a8c06eb30 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c0cdb30 con 0x7f1a8c06eb30 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a86000640 1 -- 172.21.15.110:0/2127239253 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80004a10 con 0x7f1a8c06eb30 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c0ce3d0 con 0x7f1a8c06eb30 2024-09-10T09:58:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.289+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8c0cdf20 con 0x7f1a8c06eb30 2024-09-10T09:58:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.293+0000 7f1a86000640 1 -- 172.21.15.110:0/2127239253 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1a8001b070 con 0x7f1a8c06eb30 2024-09-10T09:58:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.293+0000 7f1a86000640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1a700678c0 0x7f1a70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.293+0000 7f1a86000640 1 -- 172.21.15.110:0/2127239253 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1a80084fe0 con 0x7f1a8c06eb30 2024-09-10T09:58:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.293+0000 7f1a91400640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1a700678c0 0x7f1a70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.293+0000 7f1a91400640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1a700678c0 0x7f1a70069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0b1fd0 tx=0x7f1a7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.297+0000 7f1a86000640 1 -- 172.21.15.110:0/2127239253 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1a80055470 con 0x7f1a8c06eb30 2024-09-10T09:58:23.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.457+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a8c0736e0 con 0x7f1a8c06eb30 2024-09-10T09:58:23.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.457+0000 7f1a86000640 1 -- 172.21.15.110:0/2127239253 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16714 (secure 0 0 0) 0x7f1a80059770 con 0x7f1a8c06eb30 2024-09-10T09:58:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:23.462 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":22,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:22.145867+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:23.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1a700678c0 msgr2=0x7f1a70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1a700678c0 0x7f1a70069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0b1fd0 tx=0x7f1a7c008040 comp rx=0 tx=0).stop 2024-09-10T09:58:23.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c06eb30 msgr2=0x7f1a8c0cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c06eb30 0x7f1a8c0cbb70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1a800011f0 tx=0x7f1a8000c9b0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 shutdown_connections 2024-09-10T09:58:23.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1a700678c0 0x7f1a70069d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a8c0d0cb0 0x7f1a8c15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a8c073290 0x7f1a8c15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 --2- 172.21.15.110:0/2127239253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a8c06eb30 0x7f1a8c0cbb70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 >> 172.21.15.110:0/2127239253 conn(0x7f1a8c0d25c0 msgr2=0x7f1a8c0d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:23.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 shutdown_connections 2024-09-10T09:58:23.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.461+0000 7f1a94800640 1 -- 172.21.15.110:0/2127239253 wait complete. 2024-09-10T09:58:23.479 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T09:58:23.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7483200640 1 Processor -- start 2024-09-10T09:58:23.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7483200640 1 -- start start 2024-09-10T09:58:23.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7483200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f747c0b4960 0x7f747c06dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7483200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c0b7340 con 0x7f747c0b1aa0 2024-09-10T09:58:23.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7483200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c0b7620 con 0x7f747c0b4530 2024-09-10T09:58:23.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7483200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f747c06cd60 con 0x7f747c0b4960 2024-09-10T09:58:23.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7481e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f747c0b1aa0 0x7f747c0b6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41748/0 (socket says 172.21.15.110:41748) 2024-09-10T09:58:23.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7481e00640 1 -- 172.21.15.110:0/3179426004 learned_addr learned my addr 172.21.15.110:0/3179426004 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:23.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7482800640 1 --2- 172.21.15.110:0/3179426004 >> v2:172.21.15.145:3300/0 conn(0x7f747c0b4960 0x7f747c06dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2742670974 0 0) 0x7f747c0b7340 con 0x7f747c0b1aa0 2024-09-10T09:58:23.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7464003530 con 0x7f747c0b1aa0 2024-09-10T09:58:23.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269543600 0 0) 0x7f747c0b7620 con 0x7f747c0b4530 2024-09-10T09:58:23.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f747c0b7340 con 0x7f747c0b4530 2024-09-10T09:58:23.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3030565189 0 0) 0x7f7464003530 con 0x7f747c0b1aa0 2024-09-10T09:58:23.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 >> v2:172.21.15.145:3300/0 conn(0x7f747c0b4960 msgr2=0x7f747c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 --2- 172.21.15.110:0/3179426004 >> v2:172.21.15.145:3300/0 conn(0x7f747c0b4960 0x7f747c06dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 >> v1:172.21.15.145:6789/0 conn(0x7f747c0b4530 legacy=0x7f747c06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c06cfb0 con 0x7f747c0b1aa0 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f7470001680 con 0x7f747c0b1aa0 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 >> v1:172.21.15.110:6789/0 conn(0x7f747c0b1aa0 legacy=0x7f747c0b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7464003ba0 0x7f7464003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7464004810 0x7f7464004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464005220 0x7f746400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f747c06cd60 con 0x7f7464004810 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f746400c150 con 0x7f7464003ba0 2024-09-10T09:58:23.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.617+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f746400c350 con 0x7f7464005220 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7481400640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7464004810 0x7f7464004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7481e00640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7464003ba0 0x7f7464003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7482800640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464005220 0x7f746400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7481400640 1 -- 172.21.15.110:0/3179426004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464005220 msgr2=0x7f746400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7481400640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464005220 0x7f746400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7481400640 1 -- 172.21.15.110:0/3179426004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7464003ba0 msgr2=0x7f7464003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7481400640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7464003ba0 0x7f7464003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7481400640 1 -- 172.21.15.110:0/3179426004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7464003530 con 0x7f7464004810 2024-09-10T09:58:23.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7482800640 1 -- 172.21.15.110:0/3179426004 reap_dead start 2024-09-10T09:58:23.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7481400640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7464004810 0x7f7464004c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f746c0018d0 tx=0x7f746c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4eb4fc755eebc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c001520 con 0x7f7464004810 2024-09-10T09:58:23.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746c0052f0 con 0x7f7464004810 2024-09-10T09:58:23.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7480a00640 1 -- 172.21.15.110:0/3179426004 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c003840 con 0x7f7464004810 2024-09-10T09:58:23.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7483200640 1 -- 172.21.15.110:0/3179426004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7464004810 msgr2=0x7f7464004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.621+0000 7f7483200640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7464004810 0x7f7464004c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f746c0018d0 tx=0x7f746c01c4f0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 -- 172.21.15.110:0/3179426004 shutdown_connections 2024-09-10T09:58:23.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 --2- 172.21.15.110:0/3179426004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7464004810 0x7f7464004c90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 -- 172.21.15.110:0/3179426004 >> 172.21.15.110:0/3179426004 conn(0x7f747c0b7a10 msgr2=0x7f747c0b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:23.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 -- 172.21.15.110:0/3179426004 shutdown_connections 2024-09-10T09:58:23.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 -- 172.21.15.110:0/3179426004 wait complete. 2024-09-10T09:58:23.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 Processor -- start 2024-09-10T09:58:23.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 -- start start 2024-09-10T09:58:23.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464004810 0x7f747c150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f747c145f70 0x7f747c0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c1463d0 0x7f747c0b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f747c15edf0 con 0x7f747c1463d0 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f747c15ec70 con 0x7f747c145f70 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7483200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f747c162030 con 0x7f7464004810 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c1463d0 0x7f747c0b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c1463d0 0x7f747c0b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46528/0 (socket says 172.21.15.110:46528) 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 -- 172.21.15.110:0/1366822576 learned_addr learned my addr 172.21.15.110:0/1366822576 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7481400640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f747c145f70 0x7f747c0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7481e00640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464004810 0x7f747c150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 -- 172.21.15.110:0/1366822576 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464004810 msgr2=0x7f747c150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464004810 0x7f747c150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 -- 172.21.15.110:0/1366822576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f747c145f70 msgr2=0x7f747c0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f747c145f70 0x7f747c0b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 -- 172.21.15.110:0/1366822576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c14fbb0 con 0x7f747c1463d0 2024-09-10T09:58:23.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7482800640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c1463d0 0x7f747c0b3120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f747c0d3420 tx=0x7f74780115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7476000640 1 -- 172.21.15.110:0/1366822576 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478011e10 con 0x7f747c1463d0 2024-09-10T09:58:23.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7476000640 1 -- 172.21.15.110:0/1366822576 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747801b070 con 0x7f747c1463d0 2024-09-10T09:58:23.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.625+0000 7f7476000640 1 -- 172.21.15.110:0/1366822576 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74780039a0 con 0x7f747c1463d0 2024-09-10T09:58:23.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.629+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747c14fe90 con 0x7f747c1463d0 2024-09-10T09:58:23.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.629+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747c150450 con 0x7f747c1463d0 2024-09-10T09:58:23.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.629+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f747c150650 con 0x7f747c1463d0 2024-09-10T09:58:23.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.629+0000 7f7476000640 1 -- 172.21.15.110:0/1366822576 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7478003b60 con 0x7f747c1463d0 2024-09-10T09:58:23.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.629+0000 7f7476000640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.629+0000 7f7476000640 1 -- 172.21.15.110:0/1366822576 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f74780865a0 con 0x7f747c1463d0 2024-09-10T09:58:23.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.629+0000 7f7481e00640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.629+0000 7f7481e00640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7470001eb0 tx=0x7f7470009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.633+0000 7f7476000640 1 -- 172.21.15.110:0/1366822576 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f747c150650 con 0x7f747c1463d0 2024-09-10T09:58:23.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:23.829+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f747c1623c0 con 0x7f747c1463d0 2024-09-10T09:58:24.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7476000640 1 -- 172.21.15.110:0/1366822576 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f7478057040 con 0x7f747c1463d0 2024-09-10T09:58:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 msgr2=0x7f7460069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7470001eb0 tx=0x7f7470009040 comp rx=0 tx=0).stop 2024-09-10T09:58:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c1463d0 msgr2=0x7f747c0b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c1463d0 0x7f747c0b3120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f747c0d3420 tx=0x7f74780115a0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 shutdown_connections 2024-09-10T09:58:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f747c1463d0 0x7f747c0b3120 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f747c145f70 0x7f747c0b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 --2- 172.21.15.110:0/1366822576 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7464004810 0x7f747c150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 >> 172.21.15.110:0/1366822576 conn(0x7f747c0b7a10 msgr2=0x7f747c06ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 shutdown_connections 2024-09-10T09:58:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.769+0000 7f7483200640 1 -- 172.21.15.110:0/1366822576 wait complete. 2024-09-10T09:58:24.794 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T09:58:24.795 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T09:58:24.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.933+0000 7ff9a7e00640 1 Processor -- start 2024-09-10T09:58:24.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.933+0000 7ff9a7e00640 1 -- start start 2024-09-10T09:58:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff9a0145f30 0x7ff9a00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a00cfd90 0x7ff9a00b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:24.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a7e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a00b37c0 con 0x7ff9a00b29e0 2024-09-10T09:58:24.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a7e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff9a00b2e50 con 0x7ff9a0145f30 2024-09-10T09:58:24.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a7e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a00b3aa0 con 0x7ff9a00cfd90 2024-09-10T09:58:24.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a5400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff9a0145f30 0x7ff9a00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:24.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a4a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff9a00b29e0 0x7ff9a00cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41750/0 (socket says 172.21.15.110:41750) 2024-09-10T09:58:24.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a4a00640 1 -- 172.21.15.110:0/3566152063 learned_addr learned my addr 172.21.15.110:0/3566152063 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:24.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a5e00640 1 --2- 172.21.15.110:0/3566152063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a00cfd90 0x7ff9a00b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:24.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a5400640 1 -- 172.21.15.110:0/3566152063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a00cfd90 msgr2=0x7ff9a00b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:24.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a5400640 1 --2- 172.21.15.110:0/3566152063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a00cfd90 0x7ff9a00b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a5400640 1 -- 172.21.15.110:0/3566152063 >> v1:172.21.15.110:6789/0 conn(0x7ff9a00b29e0 legacy=0x7ff9a00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:24.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a5400640 1 -- 172.21.15.110:0/3566152063 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a00b3d80 con 0x7ff9a0145f30 2024-09-10T09:58:24.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff99fe00640 1 -- 172.21.15.110:0/3566152063 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902544248 0 0) 0x7ff9a00b37c0 con 0x7ff9a00b29e0 2024-09-10T09:58:24.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.937+0000 7ff9a5e00640 1 --2- 172.21.15.110:0/3566152063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a00cfd90 0x7ff9a00b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:24.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a5400640 1 --2- 172.21.15.110:0/3566152063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a0145f30 0x7ff9a00b24a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff994009d90 tx=0x7ff99401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7596825c8238c519 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:24.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff99fe00640 1 -- 172.21.15.110:0/3566152063 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940b2070 con 0x7ff9a0145f30 2024-09-10T09:58:24.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff99fe00640 1 -- 172.21.15.110:0/3566152063 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9940a5830 con 0x7ff9a0145f30 2024-09-10T09:58:24.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff99fe00640 1 -- 172.21.15.110:0/3566152063 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940a5c00 con 0x7ff9a0145f30 2024-09-10T09:58:24.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 -- 172.21.15.110:0/3566152063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a0145f30 msgr2=0x7ff9a00b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:24.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/3566152063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a0145f30 0x7ff9a00b24a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff994009d90 tx=0x7ff99401c970 comp rx=0 tx=0).stop 2024-09-10T09:58:24.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 -- 172.21.15.110:0/3566152063 shutdown_connections 2024-09-10T09:58:24.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/3566152063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a00cfd90 0x7ff9a00b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/3566152063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a0145f30 0x7ff9a00b24a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 -- 172.21.15.110:0/3566152063 >> 172.21.15.110:0/3566152063 conn(0x7ff9a0062e40 msgr2=0x7ff9a006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:24.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 -- 172.21.15.110:0/3566152063 shutdown_connections 2024-09-10T09:58:24.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 -- 172.21.15.110:0/3566152063 wait complete. 2024-09-10T09:58:24.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 Processor -- start 2024-09-10T09:58:24.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.941+0000 7ff9a7e00640 1 -- start start 2024-09-10T09:58:24.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a00b29e0 0x7ff9a00aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:24.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a00cfd90 0x7ff9a013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:24.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a0145f30 0x7ff9a01434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:24.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a00b5660 con 0x7ff9a00cfd90 2024-09-10T09:58:24.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a00b54e0 con 0x7ff9a00b29e0 2024-09-10T09:58:24.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a00b57e0 con 0x7ff9a0145f30 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a00cfd90 0x7ff9a013dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a00cfd90 0x7ff9a013dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46532/0 (socket says 172.21.15.110:46532) 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 -- 172.21.15.110:0/7611141 learned_addr learned my addr 172.21.15.110:0/7611141 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a5400640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a00b29e0 0x7ff9a00aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a5e00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a0145f30 0x7ff9a01434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 -- 172.21.15.110:0/7611141 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a0145f30 msgr2=0x7ff9a01434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a0145f30 0x7ff9a01434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 -- 172.21.15.110:0/7611141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a00b29e0 msgr2=0x7ff9a00aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a00b29e0 0x7ff9a00aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 -- 172.21.15.110:0/7611141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a013e1d0 con 0x7ff9a00cfd90 2024-09-10T09:58:24.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a4a00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a00cfd90 0x7ff9a013dc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff98c007fd0 tx=0x7ff98c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:24.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff99d600640 1 -- 172.21.15.110:0/7611141 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c017070 con 0x7ff9a00cfd90 2024-09-10T09:58:24.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff99d600640 1 -- 172.21.15.110:0/7611141 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff98c005d10 con 0x7ff9a00cfd90 2024-09-10T09:58:24.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a00af7e0 con 0x7ff9a00cfd90 2024-09-10T09:58:24.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff99d600640 1 -- 172.21.15.110:0/7611141 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c007310 con 0x7ff9a00cfd90 2024-09-10T09:58:24.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a00aae40 con 0x7ff9a00cfd90 2024-09-10T09:58:24.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a00b4560 con 0x7ff9a00cfd90 2024-09-10T09:58:24.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.945+0000 7ff99d600640 1 -- 172.21.15.110:0/7611141 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff98c01a070 con 0x7ff9a00cfd90 2024-09-10T09:58:24.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.949+0000 7ff99d600640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9800678c0 0x7ff980069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:24.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.949+0000 7ff99d600640 1 -- 172.21.15.110:0/7611141 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff98c0850d0 con 0x7ff9a00cfd90 2024-09-10T09:58:24.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.949+0000 7ff9a5400640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9800678c0 0x7ff980069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:24.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.949+0000 7ff9a5400640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9800678c0 0x7ff980069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff9940af000 tx=0x7ff9940a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:24.965+0000 7ff99d600640 1 -- 172.21.15.110:0/7611141 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff98c055520 con 0x7ff9a00cfd90 2024-09-10T09:58:25.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.153+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff9a013f030 con 0x7ff9a00cfd90 2024-09-10T09:58:25.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.777+0000 7ff99d600640 1 -- 172.21.15.110:0/7611141 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7ff98c059820 con 0x7ff9a00cfd90 2024-09-10T09:58:25.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9800678c0 msgr2=0x7ff980069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9800678c0 0x7ff980069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff9940af000 tx=0x7ff9940a7070 comp rx=0 tx=0).stop 2024-09-10T09:58:25.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a00cfd90 msgr2=0x7ff9a013dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a00cfd90 0x7ff9a013dc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff98c007fd0 tx=0x7ff98c00c8e0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 shutdown_connections 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9800678c0 0x7ff980069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a0145f30 0x7ff9a01434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a00cfd90 0x7ff9a013dc90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 --2- 172.21.15.110:0/7611141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a00b29e0 0x7ff9a00aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 >> 172.21.15.110:0/7611141 conn(0x7ff9a0062e40 msgr2=0x7ff9a013f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 shutdown_connections 2024-09-10T09:58:25.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.781+0000 7ff9a7e00640 1 -- 172.21.15.110:0/7611141 wait complete. 2024-09-10T09:58:25.803 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T09:58:25.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:25.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.937+0000 7f9816200640 1 Processor -- start 2024-09-10T09:58:25.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.937+0000 7f9816200640 1 -- start start 2024-09-10T09:58:25.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f9816200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9810147f00 0x7f98100ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f9816200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f981006edf0 0x7f9810072310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f9816200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98100af1a0 con 0x7f98100aed80 2024-09-10T09:58:25.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f9816200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f98100b34d0 con 0x7f981006edf0 2024-09-10T09:58:25.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f9816200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f98101482e0 con 0x7f9810147f00 2024-09-10T09:58:25.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f981006edf0 0x7f9810072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980ea00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f98100aed80 0x7f981006e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41762/0 (socket says 172.21.15.110:41762) 2024-09-10T09:58:25.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980ea00640 1 -- 172.21.15.110:0/81612995 learned_addr learned my addr 172.21.15.110:0/81612995 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:25.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980f400640 1 --2- 172.21.15.110:0/81612995 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9810147f00 0x7f98100ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980fe00640 1 -- 172.21.15.110:0/81612995 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9810147f00 msgr2=0x7f98100ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:25.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980fe00640 1 --2- 172.21.15.110:0/81612995 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9810147f00 0x7f98100ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980fe00640 1 -- 172.21.15.110:0/81612995 >> v1:172.21.15.110:6789/0 conn(0x7f98100aed80 legacy=0x7f981006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980fe00640 1 -- 172.21.15.110:0/81612995 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981015ee20 con 0x7f981006edf0 2024-09-10T09:58:25.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980e000640 1 -- 172.21.15.110:0/81612995 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915044836 0 0) 0x7f98100af1a0 con 0x7f98100aed80 2024-09-10T09:58:25.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.941+0000 7f980fe00640 1 --2- 172.21.15.110:0/81612995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981006edf0 0x7f9810072310 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98080097c0 tx=0x7f980801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d9a6368efa5795 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980e000640 1 -- 172.21.15.110:0/81612995 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080b0070 con 0x7f981006edf0 2024-09-10T09:58:25.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980e000640 1 -- 172.21.15.110:0/81612995 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98080ab600 con 0x7f981006edf0 2024-09-10T09:58:25.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980e000640 1 -- 172.21.15.110:0/81612995 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080ab9d0 con 0x7f981006edf0 2024-09-10T09:58:25.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- 172.21.15.110:0/81612995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981006edf0 msgr2=0x7f9810072310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 --2- 172.21.15.110:0/81612995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981006edf0 0x7f9810072310 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98080097c0 tx=0x7f980801c4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- 172.21.15.110:0/81612995 shutdown_connections 2024-09-10T09:58:25.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 --2- 172.21.15.110:0/81612995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981006edf0 0x7f9810072310 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 --2- 172.21.15.110:0/81612995 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9810147f00 0x7f98100ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- 172.21.15.110:0/81612995 >> 172.21.15.110:0/81612995 conn(0x7f981006bc50 msgr2=0x7f981006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:25.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- 172.21.15.110:0/81612995 shutdown_connections 2024-09-10T09:58:25.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- 172.21.15.110:0/81612995 wait complete. 2024-09-10T09:58:25.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 Processor -- start 2024-09-10T09:58:25.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- start start 2024-09-10T09:58:25.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981006edf0 0x7f981015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98100aed80 0x7f9810139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9810147f00 0x7f9810139d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98100731e0 con 0x7f9810147f00 2024-09-10T09:58:25.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9810073060 con 0x7f981006edf0 2024-09-10T09:58:25.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f9816200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9810073360 con 0x7f98100aed80 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9810147f00 0x7f9810139d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9810147f00 0x7f9810139d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46550/0 (socket says 172.21.15.110:46550) 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980fe00640 1 -- 172.21.15.110:0/3445100709 learned_addr learned my addr 172.21.15.110:0/3445100709 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980f400640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981006edf0 0x7f981015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980ea00640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98100aed80 0x7f9810139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980fe00640 1 -- 172.21.15.110:0/3445100709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98100aed80 msgr2=0x7f9810139800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980fe00640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98100aed80 0x7f9810139800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980fe00640 1 -- 172.21.15.110:0/3445100709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981006edf0 msgr2=0x7f981015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980fe00640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981006edf0 0x7f981015c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.945+0000 7f980fe00640 1 -- 172.21.15.110:0/3445100709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98101610e0 con 0x7f9810147f00 2024-09-10T09:58:25.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f980fe00640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9810147f00 0x7f9810139d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98080097c0 tx=0x7f9808019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9807400640 1 -- 172.21.15.110:0/3445100709 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080b0040 con 0x7f9810147f00 2024-09-10T09:58:25.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9807400640 1 -- 172.21.15.110:0/3445100709 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98080b9070 con 0x7f9810147f00 2024-09-10T09:58:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98101613c0 con 0x7f9810147f00 2024-09-10T09:58:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9807400640 1 -- 172.21.15.110:0/3445100709 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080b4630 con 0x7f9810147f00 2024-09-10T09:58:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981015ffe0 con 0x7f9810147f00 2024-09-10T09:58:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9810072310 con 0x7f9810147f00 2024-09-10T09:58:25.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9807400640 1 -- 172.21.15.110:0/3445100709 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f98080b47d0 con 0x7f9810147f00 2024-09-10T09:58:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9807400640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f97f00678c0 0x7f97f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f9807400640 1 -- 172.21.15.110:0/3445100709 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f980811f990 con 0x7f9810147f00 2024-09-10T09:58:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f980f400640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f97f00678c0 0x7f97f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.949+0000 7f980f400640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f97f00678c0 0x7f97f0069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9800003fd0 tx=0x7f98000039a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:25.953+0000 7f9807400640 1 -- 172.21.15.110:0/3445100709 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9808016090 con 0x7f9810147f00 2024-09-10T09:58:26.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.145+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98100b3c80 con 0x7f9810147f00 2024-09-10T09:58:26.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.145+0000 7f9807400640 1 -- 172.21.15.110:0/3445100709 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5229 (secure 0 0 0) 0x7f98080f04c0 con 0x7f9810147f00 2024-09-10T09:58:26.147 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:26.147 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:58:22.146304+0000","modified":"2024-09-10T09:58:25.155860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-09-10T09:58:26.147 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.145+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f97f00678c0 msgr2=0x7f97f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.145+0000 7f9816200640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f97f00678c0 0x7f97f0069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9800003fd0 tx=0x7f98000039a0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9810147f00 msgr2=0x7f9810139d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9810147f00 0x7f9810139d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98080097c0 tx=0x7f9808019240 comp rx=0 tx=0).stop 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 shutdown_connections 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f97f00678c0 0x7f97f0069d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9810147f00 0x7f9810139d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98100aed80 0x7f9810139800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 --2- 172.21.15.110:0/3445100709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f981006edf0 0x7f981015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 >> 172.21.15.110:0/3445100709 conn(0x7f981006bc50 msgr2=0x7f981015d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 shutdown_connections 2024-09-10T09:58:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.149+0000 7f9816200640 1 -- 172.21.15.110:0/3445100709 wait complete. 2024-09-10T09:58:26.167 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:26.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb254800640 1 Processor -- start 2024-09-10T09:58:26.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb254800640 1 -- start start 2024-09-10T09:58:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb254800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb24c15c960 0x7fb24c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb254800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb24c1603c0 0x7fb24c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb254800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c0706a0 con 0x7fb24c0d04c0 2024-09-10T09:58:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb254800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb24c15cd40 con 0x7fb24c15c960 2024-09-10T09:58:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb254800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb24c0b6370 con 0x7fb24c1603c0 2024-09-10T09:58:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb251e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb24c15c960 0x7fb24c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb251e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb24c15c960 0x7fb24c0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46552/0 (socket says 172.21.15.110:46552) 2024-09-10T09:58:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb251e00640 1 -- 172.21.15.110:0/3004679304 learned_addr learned my addr 172.21.15.110:0/3004679304 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb251400640 1 --2- 172.21.15.110:0/3004679304 >> v2:172.21.15.145:3300/0 conn(0x7fb24c1603c0 0x7fb24c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb251e00640 1 -- 172.21.15.110:0/3004679304 >> v2:172.21.15.145:3300/0 conn(0x7fb24c1603c0 msgr2=0x7fb24c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb251e00640 1 --2- 172.21.15.110:0/3004679304 >> v2:172.21.15.145:3300/0 conn(0x7fb24c1603c0 0x7fb24c0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb251e00640 1 -- 172.21.15.110:0/3004679304 >> v1:172.21.15.145:6789/0 conn(0x7fb24c0d04c0 legacy=0x7fb24c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.301+0000 7fb251e00640 1 -- 172.21.15.110:0/3004679304 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c070a80 con 0x7fb24c15c960 2024-09-10T09:58:26.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb251e00640 1 --2- 172.21.15.110:0/3004679304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c15c960 0x7fb24c0b5e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0b64f0 tx=0x7fb24001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4018d713bd3e6e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb250a00640 1 -- 172.21.15.110:0/3004679304 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400b0070 con 0x7fb24c15c960 2024-09-10T09:58:26.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb250a00640 1 -- 172.21.15.110:0/3004679304 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2400ab600 con 0x7fb24c15c960 2024-09-10T09:58:26.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb250a00640 1 -- 172.21.15.110:0/3004679304 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400ab9d0 con 0x7fb24c15c960 2024-09-10T09:58:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb254800640 1 -- 172.21.15.110:0/3004679304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c15c960 msgr2=0x7fb24c0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb254800640 1 --2- 172.21.15.110:0/3004679304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c15c960 0x7fb24c0b5e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0b64f0 tx=0x7fb24001c390 comp rx=0 tx=0).stop 2024-09-10T09:58:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb254800640 1 -- 172.21.15.110:0/3004679304 shutdown_connections 2024-09-10T09:58:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb254800640 1 --2- 172.21.15.110:0/3004679304 >> v2:172.21.15.145:3300/0 conn(0x7fb24c1603c0 0x7fb24c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb254800640 1 --2- 172.21.15.110:0/3004679304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c15c960 0x7fb24c0b5e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb254800640 1 -- 172.21.15.110:0/3004679304 >> 172.21.15.110:0/3004679304 conn(0x7fb24c0d25c0 msgr2=0x7fb24c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb254800640 1 -- 172.21.15.110:0/3004679304 shutdown_connections 2024-09-10T09:58:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.305+0000 7fb254800640 1 -- 172.21.15.110:0/3004679304 wait complete. 2024-09-10T09:58:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 Processor -- start 2024-09-10T09:58:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 -- start start 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c0d04c0 0x7fb24c153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb24c15c960 0x7fb24c14dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb24c1603c0 0x7fb24c14e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c0d3150 con 0x7fb24c0d04c0 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c1611b0 con 0x7fb24c1603c0 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c0d32d0 con 0x7fb24c15c960 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c0d04c0 0x7fb24c153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c0d04c0 0x7fb24c153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46556/0 (socket says 172.21.15.110:46556) 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 -- 172.21.15.110:0/2665370307 learned_addr learned my addr 172.21.15.110:0/2665370307 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb252800640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb24c1603c0 0x7fb24c14e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251400640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb24c15c960 0x7fb24c14dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 -- 172.21.15.110:0/2665370307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb24c15c960 msgr2=0x7fb24c14dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb24c15c960 0x7fb24c14dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 -- 172.21.15.110:0/2665370307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb24c1603c0 msgr2=0x7fb24c14e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb24c1603c0 0x7fb24c14e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 -- 172.21.15.110:0/2665370307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c06e180 con 0x7fb24c0d04c0 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb251e00640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c0d04c0 0x7fb24c153a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb24001aa40 tx=0x7fb24001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb246000640 1 -- 172.21.15.110:0/2665370307 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400ba020 con 0x7fb24c0d04c0 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb246000640 1 -- 172.21.15.110:0/2665370307 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2400b6640 con 0x7fb24c0d04c0 2024-09-10T09:58:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb246000640 1 -- 172.21.15.110:0/2665370307 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400b0040 con 0x7fb24c0d04c0 2024-09-10T09:58:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c150ba0 con 0x7fb24c0d04c0 2024-09-10T09:58:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c06e700 con 0x7fb24c0d04c0 2024-09-10T09:58:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.309+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c071a80 con 0x7fb24c0d04c0 2024-09-10T09:58:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.313+0000 7fb246000640 1 -- 172.21.15.110:0/2665370307 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb2400bd020 con 0x7fb24c0d04c0 2024-09-10T09:58:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.313+0000 7fb246000640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb2300678c0 0x7fb230069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.313+0000 7fb246000640 1 -- 172.21.15.110:0/2665370307 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb24011fe90 con 0x7fb24c0d04c0 2024-09-10T09:58:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.313+0000 7fb251400640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb2300678c0 0x7fb230069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.313+0000 7fb251400640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb2300678c0 0x7fb230069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb24c1585a0 tx=0x7fb23c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.317+0000 7fb246000640 1 -- 172.21.15.110:0/2665370307 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb2400c4030 con 0x7fb24c0d04c0 2024-09-10T09:58:26.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb24c0d34d0 con 0x7fb24c0d04c0 2024-09-10T09:58:26.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb246000640 1 -- 172.21.15.110:0/2665370307 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16701 (secure 0 0 0) 0x7fb240121020 con 0x7fb24c0d04c0 2024-09-10T09:58:26.478 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:26.478 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:23.766388+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:26.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb2300678c0 msgr2=0x7fb230069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb2300678c0 0x7fb230069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb24c1585a0 tx=0x7fb23c008040 comp rx=0 tx=0).stop 2024-09-10T09:58:26.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c0d04c0 msgr2=0x7fb24c153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c0d04c0 0x7fb24c153a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb24001aa40 tx=0x7fb24001abd0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 shutdown_connections 2024-09-10T09:58:26.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb2300678c0 0x7fb230069d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb24c1603c0 0x7fb24c14e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb24c15c960 0x7fb24c14dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 --2- 172.21.15.110:0/2665370307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb24c0d04c0 0x7fb24c153a50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 >> 172.21.15.110:0/2665370307 conn(0x7fb24c0d25c0 msgr2=0x7fb24c166800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 shutdown_connections 2024-09-10T09:58:26.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.477+0000 7fb254800640 1 -- 172.21.15.110:0/2665370307 wait complete. 2024-09-10T09:58:26.500 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T09:58:26.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:58:26.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.633+0000 7f81af000640 1 Processor -- start 2024-09-10T09:58:26.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.633+0000 7f81af000640 1 -- start start 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f81af000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 0x7f81a806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f81af000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81a806ff60 0x7f81a80af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f81af000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f81a80729b0 0x7f81a8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f81af000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f81a815eae0 con 0x7f81a806ff60 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f81af000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f81a80703a0 con 0x7f81a80729b0 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f81af000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81a806eec0 con 0x7f81a815e700 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f819f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81a806ff60 0x7f81a80af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f819f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81a806ff60 0x7f81a80af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46568/0 (socket says 172.21.15.110:46568) 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f819f400640 1 -- 172.21.15.110:0/2878364614 learned_addr learned my addr 172.21.15.110:0/2878364614 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:26.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f81ad000640 1 --2- 172.21.15.110:0/2878364614 >> v2:172.21.15.145:3300/0 conn(0x7f81a80729b0 0x7f81a8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f81a7e00640 1 --2- 172.21.15.110:0/2878364614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 0x7f81a806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f819f400640 1 -- 172.21.15.110:0/2878364614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 msgr2=0x7f81a806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f819f400640 1 --2- 172.21.15.110:0/2878364614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 0x7f81a806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f819f400640 1 -- 172.21.15.110:0/2878364614 >> v2:172.21.15.145:3300/0 conn(0x7f81a80729b0 msgr2=0x7f81a8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:26.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f819f400640 1 --2- 172.21.15.110:0/2878364614 >> v2:172.21.15.145:3300/0 conn(0x7f81a80729b0 0x7f81a8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.637+0000 7f819f400640 1 -- 172.21.15.110:0/2878364614 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a806f1a0 con 0x7f81a806ff60 2024-09-10T09:58:26.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f819f400640 1 --2- 172.21.15.110:0/2878364614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a806ff60 0x7f81a80af270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f81a80b6500 tx=0x7f819001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12b38b254847e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81a7400640 1 -- 172.21.15.110:0/2878364614 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900b1070 con 0x7f81a806ff60 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81a7400640 1 -- 172.21.15.110:0/2878364614 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81900ac630 con 0x7f81a806ff60 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81a7400640 1 -- 172.21.15.110:0/2878364614 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900a8050 con 0x7f81a806ff60 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- 172.21.15.110:0/2878364614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a806ff60 msgr2=0x7f81a80af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 --2- 172.21.15.110:0/2878364614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a806ff60 0x7f81a80af270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f81a80b6500 tx=0x7f819001c520 comp rx=0 tx=0).stop 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- 172.21.15.110:0/2878364614 shutdown_connections 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 --2- 172.21.15.110:0/2878364614 >> v2:172.21.15.145:3300/0 conn(0x7f81a80729b0 0x7f81a8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 --2- 172.21.15.110:0/2878364614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a806ff60 0x7f81a80af270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 --2- 172.21.15.110:0/2878364614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 0x7f81a806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- 172.21.15.110:0/2878364614 >> 172.21.15.110:0/2878364614 conn(0x7f81a806cb90 msgr2=0x7f81a806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- 172.21.15.110:0/2878364614 shutdown_connections 2024-09-10T09:58:26.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- 172.21.15.110:0/2878364614 wait complete. 2024-09-10T09:58:26.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 Processor -- start 2024-09-10T09:58:26.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- start start 2024-09-10T09:58:26.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a806ff60 0x7f81a8143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a80729b0 0x7f81a8153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 0x7f81a81539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81a80af930 con 0x7f81a80729b0 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81a80af7b0 con 0x7f81a806ff60 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.641+0000 7f81af000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81a80b4660 con 0x7f81a815e700 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a80729b0 0x7f81a8153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a80729b0 0x7f81a8153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46570/0 (socket says 172.21.15.110:46570) 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 -- 172.21.15.110:0/3149429289 learned_addr learned my addr 172.21.15.110:0/3149429289 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81a7e00640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a806ff60 0x7f81a8143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81ad000640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 0x7f81a81539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 -- 172.21.15.110:0/3149429289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 msgr2=0x7f81a81539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 0x7f81a81539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 -- 172.21.15.110:0/3149429289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a806ff60 msgr2=0x7f81a8143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a806ff60 0x7f81a8143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 -- 172.21.15.110:0/3149429289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a80cd040 con 0x7f81a80729b0 2024-09-10T09:58:26.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f819f400640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a80729b0 0x7f81a8153470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f819001abd0 tx=0x7f81900192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81a4c00640 1 -- 172.21.15.110:0/3149429289 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900bc070 con 0x7f81a80729b0 2024-09-10T09:58:26.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81a4c00640 1 -- 172.21.15.110:0/3149429289 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81900ab5e0 con 0x7f81a80729b0 2024-09-10T09:58:26.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81a4c00640 1 -- 172.21.15.110:0/3149429289 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900b1070 con 0x7f81a80729b0 2024-09-10T09:58:26.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a80cd300 con 0x7f81a80729b0 2024-09-10T09:58:26.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a80cd8c0 con 0x7f81a80729b0 2024-09-10T09:58:26.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a80b49f0 con 0x7f81a80729b0 2024-09-10T09:58:26.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.645+0000 7f81a4c00640 1 -- 172.21.15.110:0/3149429289 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81900ab9d0 con 0x7f81a80729b0 2024-09-10T09:58:26.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.649+0000 7f81a4c00640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81840678c0 0x7f8184069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:26.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.649+0000 7f81a4c00640 1 -- 172.21.15.110:0/3149429289 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f819011f5d0 con 0x7f81a80729b0 2024-09-10T09:58:26.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.649+0000 7f81a7e00640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81840678c0 0x7f8184069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:26.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.649+0000 7f81a7e00640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81840678c0 0x7f8184069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81980039f0 tx=0x7f8198009830 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:26.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.653+0000 7f81a4c00640 1 -- 172.21.15.110:0/3149429289 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8190016110 con 0x7f81a80729b0 2024-09-10T09:58:26.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.805+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81a80afb00 con 0x7f81840678c0 2024-09-10T09:58:26.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.813+0000 7f81a4c00640 1 -- 172.21.15.110:0/3149429289 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246050 (secure 0 0 0) 0x7f81a80afb00 con 0x7f81840678c0 2024-09-10T09:58:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:26.819 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81840678c0 msgr2=0x7f8184069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81840678c0 0x7f8184069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81980039f0 tx=0x7f8198009830 comp rx=0 tx=0).stop 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a80729b0 msgr2=0x7f81a8153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a80729b0 0x7f81a8153470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f819001abd0 tx=0x7f81900192c0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 shutdown_connections 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81840678c0 0x7f8184069d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81a815e700 0x7f81a81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81a80729b0 0x7f81a8153470 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 --2- 172.21.15.110:0/3149429289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81a806ff60 0x7f81a8143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 >> 172.21.15.110:0/3149429289 conn(0x7f81a806cb90 msgr2=0x7f81a806cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 shutdown_connections 2024-09-10T09:58:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:26.817+0000 7f81af000640 1 -- 172.21.15.110:0/3149429289 wait complete. 2024-09-10T09:58:26.839 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T09:58:25.355804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235656,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739064,"statfs":{"total":773094113280,"available":772852801536,"internally_reserved":0,"allocated":2834432,"data_stored":1711684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.608542"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.773483+0000","last_change":"2024-09-10T09:58:21.773483+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.773483+0000","last_undegraded":"2024-09-10T09:58:21.773483+0000","last_fullsized":"2024-09-10T09:58:21.773483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758695+0000","last_change":"2024-09-10T09:58:22.758695+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758695+0000","last_undegraded":"2024-09-10T09:58:22.758695+0000","last_fullsized":"2024-09-10T09:58:22.758695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770858+0000","last_change":"2024-09-10T09:58:21.770858+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770858+0000","last_undegraded":"2024-09-10T09:58:21.770858+0000","last_fullsized":"2024-09-10T09:58:21.770858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761137+0000","last_change":"2024-09-10T09:58:22.761137+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761137+0000","last_undegraded":"2024-09-10T09:58:22.761137+0000","last_fullsized":"2024-09-10T09:58:22.761137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770682+0000","last_change":"2024-09-10T09:58:21.770682+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770682+0000","last_undegraded":"2024-09-10T09:58:21.770682+0000","last_fullsized":"2024-09-10T09:58:21.770682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762312+0000","last_change":"2024-09-10T09:58:22.762312+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762312+0000","last_undegraded":"2024-09-10T09:58:22.762312+0000","last_fullsized":"2024-09-10T09:58:22.762312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.780164+0000","last_change":"2024-09-10T09:58:21.780164+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.780164+0000","last_undegraded":"2024-09-10T09:58:21.780164+0000","last_fullsized":"2024-09-10T09:58:21.780164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772032+0000","last_change":"2024-09-10T09:58:21.772032+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772032+0000","last_undegraded":"2024-09-10T09:58:21.772032+0000","last_fullsized":"2024-09-10T09:58:21.772032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760582+0000","last_change":"2024-09-10T09:58:22.760582+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760582+0000","last_undegraded":"2024-09-10T09:58:22.760582+0000","last_fullsized":"2024-09-10T09:58:22.760582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762468+0000","last_change":"2024-09-10T09:58:22.762468+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762468+0000","last_undegraded":"2024-09-10T09:58:22.762468+0000","last_fullsized":"2024-09-10T09:58:22.762468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767842+0000","last_change":"2024-09-10T09:58:21.767842+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767842+0000","last_undegraded":"2024-09-10T09:58:21.767842+0000","last_fullsized":"2024-09-10T09:58:21.767842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.765480+0000","last_change":"2024-09-10T09:58:21.765480+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.765480+0000","last_undegraded":"2024-09-10T09:58:21.765480+0000","last_fullsized":"2024-09-10T09:58:21.765480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758765+0000","last_change":"2024-09-10T09:58:22.758765+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758765+0000","last_undegraded":"2024-09-10T09:58:22.758765+0000","last_fullsized":"2024-09-10T09:58:22.758765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.764622+0000","last_change":"2024-09-10T09:58:22.764622+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.764622+0000","last_undegraded":"2024-09-10T09:58:22.764622+0000","last_fullsized":"2024-09-10T09:58:22.764622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760834+0000","last_change":"2024-09-10T09:58:22.760834+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760834+0000","last_undegraded":"2024-09-10T09:58:22.760834+0000","last_fullsized":"2024-09-10T09:58:22.760834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.762859+0000","last_change":"2024-09-10T09:58:21.762859+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.762859+0000","last_undegraded":"2024-09-10T09:58:21.762859+0000","last_fullsized":"2024-09-10T09:58:21.762859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767720+0000","last_change":"2024-09-10T09:58:21.767720+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767720+0000","last_undegraded":"2024-09-10T09:58:21.767720+0000","last_fullsized":"2024-09-10T09:58:21.767720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.763497+0000","last_change":"2024-09-10T09:58:22.763497+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.763497+0000","last_undegraded":"2024-09-10T09:58:22.763497+0000","last_fullsized":"2024-09-10T09:58:22.763497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.765123+0000","last_change":"2024-09-10T09:58:21.765123+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.765123+0000","last_undegraded":"2024-09-10T09:58:21.765123+0000","last_fullsized":"2024-09-10T09:58:21.765123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762017+0000","last_change":"2024-09-10T09:58:22.762017+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762017+0000","last_undegraded":"2024-09-10T09:58:22.762017+0000","last_fullsized":"2024-09-10T09:58:22.762017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.764229+0000","last_change":"2024-09-10T09:58:22.764229+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.764229+0000","last_undegraded":"2024-09-10T09:58:22.764229+0000","last_fullsized":"2024-09-10T09:58:22.764229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.769476+0000","last_change":"2024-09-10T09:58:22.769476+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.769476+0000","last_undegraded":"2024-09-10T09:58:22.769476+0000","last_fullsized":"2024-09-10T09:58:22.769476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.759733+0000","last_change":"2024-09-10T09:58:22.759733+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.759733+0000","last_undegraded":"2024-09-10T09:58:22.759733+0000","last_fullsized":"2024-09-10T09:58:22.759733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.764586+0000","last_change":"2024-09-10T09:58:21.764586+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.764586+0000","last_undegraded":"2024-09-10T09:58:21.764586+0000","last_fullsized":"2024-09-10T09:58:21.764586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761625+0000","last_change":"2024-09-10T09:58:22.761625+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761625+0000","last_undegraded":"2024-09-10T09:58:22.761625+0000","last_fullsized":"2024-09-10T09:58:22.761625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.762149+0000","last_change":"2024-09-10T09:58:21.762149+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.762149+0000","last_undegraded":"2024-09-10T09:58:21.762149+0000","last_fullsized":"2024-09-10T09:58:21.762149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760420+0000","last_change":"2024-09-10T09:58:22.760420+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760420+0000","last_undegraded":"2024-09-10T09:58:22.760420+0000","last_fullsized":"2024-09-10T09:58:22.760420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.768406+0000","last_change":"2024-09-10T09:58:21.768406+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.768406+0000","last_undegraded":"2024-09-10T09:58:21.768406+0000","last_fullsized":"2024-09-10T09:58:21.768406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T09:58:20.755078+0000","last_change":"2024-09-10T09:58:20.755078+0000","last_active":"2024-09-10T09:58:20.740599+0000","last_peered":"2024-09-10T09:58:20.740599+0000","last_clean":"2024-09-10T09:58:20.740599+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:20.755078+0000","last_undegraded":"2024-09-10T09:58:20.755078+0000","last_fullsized":"2024-09-10T09:58:20.755078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.764021+0000","last_change":"2024-09-10T09:58:21.764021+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.764021+0000","last_undegraded":"2024-09-10T09:58:21.764021+0000","last_fullsized":"2024-09-10T09:58:21.764021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.765118+0000","last_change":"2024-09-10T09:58:21.778244+0000","last_active":"2024-09-10T09:58:22.765118+0000","last_peered":"2024-09-10T09:58:22.765118+0000","last_clean":"2024-09-10T09:58:22.765118+0000","last_became_active":"2024-09-10T09:58:21.777937+0000","last_became_peered":"2024-09-10T09:58:21.777937+0000","last_unstale":"2024-09-10T09:58:22.765118+0000","last_undegraded":"2024-09-10T09:58:22.765118+0000","last_fullsized":"2024-09-10T09:58:22.765118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T09:58:20.755035+0000","last_change":"2024-09-10T09:58:20.755035+0000","last_active":"2024-09-10T09:58:20.740599+0000","last_peered":"2024-09-10T09:58:20.740599+0000","last_clean":"2024-09-10T09:58:20.740599+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:20.755035+0000","last_undegraded":"2024-09-10T09:58:20.755035+0000","last_fullsized":"2024-09-10T09:58:20.755035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.765584+0000","last_change":"2024-09-10T09:58:21.786530+0000","last_active":"2024-09-10T09:58:22.765584+0000","last_peered":"2024-09-10T09:58:22.765584+0000","last_clean":"2024-09-10T09:58:22.765584+0000","last_became_active":"2024-09-10T09:58:21.786157+0000","last_became_peered":"2024-09-10T09:58:21.786157+0000","last_unstale":"2024-09-10T09:58:22.765584+0000","last_undegraded":"2024-09-10T09:58:22.765584+0000","last_fullsized":"2024-09-10T09:58:22.765584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761036+0000","last_change":"2024-09-10T09:58:22.761036+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761036+0000","last_undegraded":"2024-09-10T09:58:22.761036+0000","last_fullsized":"2024-09-10T09:58:22.761036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761809+0000","last_change":"2024-09-10T09:58:21.786956+0000","last_active":"2024-09-10T09:58:22.761809+0000","last_peered":"2024-09-10T09:58:22.761809+0000","last_clean":"2024-09-10T09:58:22.761809+0000","last_became_active":"2024-09-10T09:58:21.786523+0000","last_became_peered":"2024-09-10T09:58:21.786523+0000","last_unstale":"2024-09-10T09:58:22.761809+0000","last_undegraded":"2024-09-10T09:58:22.761809+0000","last_fullsized":"2024-09-10T09:58:22.761809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761458+0000","last_change":"2024-09-10T09:58:22.761458+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761458+0000","last_undegraded":"2024-09-10T09:58:22.761458+0000","last_fullsized":"2024-09-10T09:58:22.761458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761846+0000","last_change":"2024-09-10T09:58:21.787382+0000","last_active":"2024-09-10T09:58:22.761846+0000","last_peered":"2024-09-10T09:58:22.761846+0000","last_clean":"2024-09-10T09:58:22.761846+0000","last_became_active":"2024-09-10T09:58:21.787048+0000","last_became_peered":"2024-09-10T09:58:21.787048+0000","last_unstale":"2024-09-10T09:58:22.761846+0000","last_undegraded":"2024-09-10T09:58:22.761846+0000","last_fullsized":"2024-09-10T09:58:22.761846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772636+0000","last_change":"2024-09-10T09:58:21.772636+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772636+0000","last_undegraded":"2024-09-10T09:58:21.772636+0000","last_fullsized":"2024-09-10T09:58:21.772636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759611+0000","last_change":"2024-09-10T09:58:21.785126+0000","last_active":"2024-09-10T09:58:22.759611+0000","last_peered":"2024-09-10T09:58:22.759611+0000","last_clean":"2024-09-10T09:58:22.759611+0000","last_became_active":"2024-09-10T09:58:21.784839+0000","last_became_peered":"2024-09-10T09:58:21.784839+0000","last_unstale":"2024-09-10T09:58:22.759611+0000","last_undegraded":"2024-09-10T09:58:22.759611+0000","last_fullsized":"2024-09-10T09:58:22.759611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767249+0000","last_change":"2024-09-10T09:58:21.767249+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767249+0000","last_undegraded":"2024-09-10T09:58:21.767249+0000","last_fullsized":"2024-09-10T09:58:21.767249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.763857+0000","last_change":"2024-09-10T09:58:21.776340+0000","last_active":"2024-09-10T09:58:22.763857+0000","last_peered":"2024-09-10T09:58:22.763857+0000","last_clean":"2024-09-10T09:58:22.763857+0000","last_became_active":"2024-09-10T09:58:21.776066+0000","last_became_peered":"2024-09-10T09:58:21.776066+0000","last_unstale":"2024-09-10T09:58:22.763857+0000","last_undegraded":"2024-09-10T09:58:22.763857+0000","last_fullsized":"2024-09-10T09:58:22.763857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772836+0000","last_change":"2024-09-10T09:58:21.772836+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772836+0000","last_undegraded":"2024-09-10T09:58:21.772836+0000","last_fullsized":"2024-09-10T09:58:21.772836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.785464+0000","last_change":"2024-09-10T09:58:21.783558+0000","last_active":"2024-09-10T09:58:21.785464+0000","last_peered":"2024-09-10T09:58:21.785464+0000","last_clean":"2024-09-10T09:58:21.785464+0000","last_became_active":"2024-09-10T09:58:21.782455+0000","last_became_peered":"2024-09-10T09:58:21.782455+0000","last_unstale":"2024-09-10T09:58:21.785464+0000","last_undegraded":"2024-09-10T09:58:21.785464+0000","last_fullsized":"2024-09-10T09:58:21.785464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:26.367199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.763413+0000","last_change":"2024-09-10T09:58:21.763413+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.763413+0000","last_undegraded":"2024-09-10T09:58:21.763413+0000","last_fullsized":"2024-09-10T09:58:21.763413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762503+0000","last_change":"2024-09-10T09:58:21.789974+0000","last_active":"2024-09-10T09:58:22.762503+0000","last_peered":"2024-09-10T09:58:22.762503+0000","last_clean":"2024-09-10T09:58:22.762503+0000","last_became_active":"2024-09-10T09:58:21.788481+0000","last_became_peered":"2024-09-10T09:58:21.788481+0000","last_unstale":"2024-09-10T09:58:22.762503+0000","last_undegraded":"2024-09-10T09:58:22.762503+0000","last_fullsized":"2024-09-10T09:58:22.762503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769168+0000","last_change":"2024-09-10T09:58:21.769168+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769168+0000","last_undegraded":"2024-09-10T09:58:21.769168+0000","last_fullsized":"2024-09-10T09:58:21.769168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.760516+0000","last_change":"2024-09-10T09:58:21.778738+0000","last_active":"2024-09-10T09:58:22.760516+0000","last_peered":"2024-09-10T09:58:22.760516+0000","last_clean":"2024-09-10T09:58:22.760516+0000","last_became_active":"2024-09-10T09:58:21.777982+0000","last_became_peered":"2024-09-10T09:58:21.777982+0000","last_unstale":"2024-09-10T09:58:22.760516+0000","last_undegraded":"2024-09-10T09:58:22.760516+0000","last_fullsized":"2024-09-10T09:58:22.760516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761175+0000","last_change":"2024-09-10T09:58:22.761175+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761175+0000","last_undegraded":"2024-09-10T09:58:22.761175+0000","last_fullsized":"2024-09-10T09:58:22.761175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.782949+0000","last_change":"2024-09-10T09:58:21.779750+0000","last_active":"2024-09-10T09:58:21.782949+0000","last_peered":"2024-09-10T09:58:21.782949+0000","last_clean":"2024-09-10T09:58:21.782949+0000","last_became_active":"2024-09-10T09:58:21.779492+0000","last_became_peered":"2024-09-10T09:58:21.779492+0000","last_unstale":"2024-09-10T09:58:21.782949+0000","last_undegraded":"2024-09-10T09:58:21.782949+0000","last_fullsized":"2024-09-10T09:58:21.782949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:26.367199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772393+0000","last_change":"2024-09-10T09:58:21.772393+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772393+0000","last_undegraded":"2024-09-10T09:58:21.772393+0000","last_fullsized":"2024-09-10T09:58:21.772393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.789668+0000","last_change":"2024-09-10T09:58:21.781792+0000","last_active":"2024-09-10T09:58:21.789668+0000","last_peered":"2024-09-10T09:58:21.789668+0000","last_clean":"2024-09-10T09:58:21.789668+0000","last_became_active":"2024-09-10T09:58:21.781434+0000","last_became_peered":"2024-09-10T09:58:21.781434+0000","last_unstale":"2024-09-10T09:58:21.789668+0000","last_undegraded":"2024-09-10T09:58:21.789668+0000","last_fullsized":"2024-09-10T09:58:21.789668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:42.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767018+0000","last_change":"2024-09-10T09:58:21.767018+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767018+0000","last_undegraded":"2024-09-10T09:58:21.767018+0000","last_fullsized":"2024-09-10T09:58:21.767018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.783020+0000","last_change":"2024-09-10T09:58:21.781287+0000","last_active":"2024-09-10T09:58:21.783020+0000","last_peered":"2024-09-10T09:58:21.783020+0000","last_clean":"2024-09-10T09:58:21.783020+0000","last_became_active":"2024-09-10T09:58:21.780824+0000","last_became_peered":"2024-09-10T09:58:21.780824+0000","last_unstale":"2024-09-10T09:58:21.783020+0000","last_undegraded":"2024-09-10T09:58:21.783020+0000","last_fullsized":"2024-09-10T09:58:21.783020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:11.144974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769098+0000","last_change":"2024-09-10T09:58:21.769098+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769098+0000","last_undegraded":"2024-09-10T09:58:21.769098+0000","last_fullsized":"2024-09-10T09:58:21.769098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759829+0000","last_change":"2024-09-10T09:58:21.776106+0000","last_active":"2024-09-10T09:58:22.759829+0000","last_peered":"2024-09-10T09:58:22.759829+0000","last_clean":"2024-09-10T09:58:22.759829+0000","last_became_active":"2024-09-10T09:58:21.773315+0000","last_became_peered":"2024-09-10T09:58:21.773315+0000","last_unstale":"2024-09-10T09:58:22.759829+0000","last_undegraded":"2024-09-10T09:58:22.759829+0000","last_fullsized":"2024-09-10T09:58:22.759829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760733+0000","last_change":"2024-09-10T09:58:22.760733+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760733+0000","last_undegraded":"2024-09-10T09:58:22.760733+0000","last_fullsized":"2024-09-10T09:58:22.760733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.790976+0000","last_change":"2024-09-10T09:58:21.789986+0000","last_active":"2024-09-10T09:58:21.790976+0000","last_peered":"2024-09-10T09:58:21.790976+0000","last_clean":"2024-09-10T09:58:21.790976+0000","last_became_active":"2024-09-10T09:58:21.789577+0000","last_became_peered":"2024-09-10T09:58:21.789577+0000","last_unstale":"2024-09-10T09:58:21.790976+0000","last_undegraded":"2024-09-10T09:58:21.790976+0000","last_fullsized":"2024-09-10T09:58:21.790976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.768360+0000","last_change":"2024-09-10T09:58:21.768360+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.768360+0000","last_undegraded":"2024-09-10T09:58:21.768360+0000","last_fullsized":"2024-09-10T09:58:21.768360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":166,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.783072+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:58:21.783072+0000","last_peered":"2024-09-10T09:58:21.783072+0000","last_clean":"2024-09-10T09:58:21.783072+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:58:21.783072+0000","last_undegraded":"2024-09-10T09:58:21.783072+0000","last_fullsized":"2024-09-10T09:58:21.783072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762945+0000","last_change":"2024-09-10T09:58:21.789884+0000","last_active":"2024-09-10T09:58:22.762945+0000","last_peered":"2024-09-10T09:58:22.762945+0000","last_clean":"2024-09-10T09:58:22.762945+0000","last_became_active":"2024-09-10T09:58:21.788940+0000","last_became_peered":"2024-09-10T09:58:21.788940+0000","last_unstale":"2024-09-10T09:58:22.762945+0000","last_undegraded":"2024-09-10T09:58:22.762945+0000","last_fullsized":"2024-09-10T09:58:22.762945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792384+0000","last_change":"2024-09-10T09:58:21.791063+0000","last_active":"2024-09-10T09:58:21.792384+0000","last_peered":"2024-09-10T09:58:21.792384+0000","last_clean":"2024-09-10T09:58:21.792384+0000","last_became_active":"2024-09-10T09:58:21.790796+0000","last_became_peered":"2024-09-10T09:58:21.790796+0000","last_unstale":"2024-09-10T09:58:21.792384+0000","last_undegraded":"2024-09-10T09:58:21.792384+0000","last_fullsized":"2024-09-10T09:58:21.792384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.761973+0000","last_change":"2024-09-10T09:58:21.761973+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.761973+0000","last_undegraded":"2024-09-10T09:58:21.761973+0000","last_fullsized":"2024-09-10T09:58:21.761973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.781486+0000","last_change":"2024-09-10T09:58:21.777208+0000","last_active":"2024-09-10T09:58:21.781486+0000","last_peered":"2024-09-10T09:58:21.781486+0000","last_clean":"2024-09-10T09:58:21.781486+0000","last_became_active":"2024-09-10T09:58:21.776820+0000","last_became_peered":"2024-09-10T09:58:21.776820+0000","last_unstale":"2024-09-10T09:58:21.781486+0000","last_undegraded":"2024-09-10T09:58:21.781486+0000","last_fullsized":"2024-09-10T09:58:21.781486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:42.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.759166+0000","last_change":"2024-09-10T09:58:22.759166+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.759166+0000","last_undegraded":"2024-09-10T09:58:22.759166+0000","last_fullsized":"2024-09-10T09:58:22.759166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762585+0000","last_change":"2024-09-10T09:58:21.785388+0000","last_active":"2024-09-10T09:58:22.762585+0000","last_peered":"2024-09-10T09:58:22.762585+0000","last_clean":"2024-09-10T09:58:22.762585+0000","last_became_active":"2024-09-10T09:58:21.785227+0000","last_became_peered":"2024-09-10T09:58:21.785227+0000","last_unstale":"2024-09-10T09:58:22.762585+0000","last_undegraded":"2024-09-10T09:58:22.762585+0000","last_fullsized":"2024-09-10T09:58:22.762585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770833+0000","last_change":"2024-09-10T09:58:21.770833+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770833+0000","last_undegraded":"2024-09-10T09:58:21.770833+0000","last_fullsized":"2024-09-10T09:58:21.770833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.758612+0000","last_change":"2024-09-10T09:58:21.768431+0000","last_active":"2024-09-10T09:58:22.758612+0000","last_peered":"2024-09-10T09:58:22.758612+0000","last_clean":"2024-09-10T09:58:22.758612+0000","last_became_active":"2024-09-10T09:58:21.767826+0000","last_became_peered":"2024-09-10T09:58:21.767826+0000","last_unstale":"2024-09-10T09:58:22.758612+0000","last_undegraded":"2024-09-10T09:58:22.758612+0000","last_fullsized":"2024-09-10T09:58:22.758612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761547+0000","last_change":"2024-09-10T09:58:22.761547+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761547+0000","last_undegraded":"2024-09-10T09:58:22.761547+0000","last_fullsized":"2024-09-10T09:58:22.761547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792043+0000","last_change":"2024-09-10T09:58:21.788480+0000","last_active":"2024-09-10T09:58:21.792043+0000","last_peered":"2024-09-10T09:58:21.792043+0000","last_clean":"2024-09-10T09:58:21.792043+0000","last_became_active":"2024-09-10T09:58:21.788162+0000","last_became_peered":"2024-09-10T09:58:21.788162+0000","last_unstale":"2024-09-10T09:58:21.792043+0000","last_undegraded":"2024-09-10T09:58:21.792043+0000","last_fullsized":"2024-09-10T09:58:21.792043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:05.666246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758668+0000","last_change":"2024-09-10T09:58:22.758668+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758668+0000","last_undegraded":"2024-09-10T09:58:22.758668+0000","last_fullsized":"2024-09-10T09:58:22.758668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.758990+0000","last_change":"2024-09-10T09:58:21.770640+0000","last_active":"2024-09-10T09:58:22.758990+0000","last_peered":"2024-09-10T09:58:22.758990+0000","last_clean":"2024-09-10T09:58:22.758990+0000","last_became_active":"2024-09-10T09:58:21.769111+0000","last_became_peered":"2024-09-10T09:58:21.769111+0000","last_unstale":"2024-09-10T09:58:22.758990+0000","last_undegraded":"2024-09-10T09:58:22.758990+0000","last_fullsized":"2024-09-10T09:58:22.758990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761978+0000","last_change":"2024-09-10T09:58:21.782019+0000","last_active":"2024-09-10T09:58:22.761978+0000","last_peered":"2024-09-10T09:58:22.761978+0000","last_clean":"2024-09-10T09:58:22.761978+0000","last_became_active":"2024-09-10T09:58:21.781750+0000","last_became_peered":"2024-09-10T09:58:21.781750+0000","last_unstale":"2024-09-10T09:58:22.761978+0000","last_undegraded":"2024-09-10T09:58:22.761978+0000","last_fullsized":"2024-09-10T09:58:22.761978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770772+0000","last_change":"2024-09-10T09:58:21.770772+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770772+0000","last_undegraded":"2024-09-10T09:58:21.770772+0000","last_fullsized":"2024-09-10T09:58:21.770772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.791839+0000","last_change":"2024-09-10T09:58:21.788078+0000","last_active":"2024-09-10T09:58:21.791839+0000","last_peered":"2024-09-10T09:58:21.791839+0000","last_clean":"2024-09-10T09:58:21.791839+0000","last_became_active":"2024-09-10T09:58:21.787660+0000","last_became_peered":"2024-09-10T09:58:21.787660+0000","last_unstale":"2024-09-10T09:58:21.791839+0000","last_undegraded":"2024-09-10T09:58:21.791839+0000","last_fullsized":"2024-09-10T09:58:21.791839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769618+0000","last_change":"2024-09-10T09:58:21.769618+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769618+0000","last_undegraded":"2024-09-10T09:58:21.769618+0000","last_fullsized":"2024-09-10T09:58:21.769618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759017+0000","last_change":"2024-09-10T09:58:21.782043+0000","last_active":"2024-09-10T09:58:22.759017+0000","last_peered":"2024-09-10T09:58:22.759017+0000","last_clean":"2024-09-10T09:58:22.759017+0000","last_became_active":"2024-09-10T09:58:21.781762+0000","last_became_peered":"2024-09-10T09:58:21.781762+0000","last_unstale":"2024-09-10T09:58:22.759017+0000","last_undegraded":"2024-09-10T09:58:22.759017+0000","last_fullsized":"2024-09-10T09:58:22.759017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759075+0000","last_change":"2024-09-10T09:58:21.774362+0000","last_active":"2024-09-10T09:58:22.759075+0000","last_peered":"2024-09-10T09:58:22.759075+0000","last_clean":"2024-09-10T09:58:22.759075+0000","last_became_active":"2024-09-10T09:58:21.773133+0000","last_became_peered":"2024-09-10T09:58:21.773133+0000","last_unstale":"2024-09-10T09:58:22.759075+0000","last_undegraded":"2024-09-10T09:58:22.759075+0000","last_fullsized":"2024-09-10T09:58:22.759075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792397+0000","last_change":"2024-09-10T09:58:21.790948+0000","last_active":"2024-09-10T09:58:21.792397+0000","last_peered":"2024-09-10T09:58:21.792397+0000","last_clean":"2024-09-10T09:58:21.792397+0000","last_became_active":"2024-09-10T09:58:21.790543+0000","last_became_peered":"2024-09-10T09:58:21.790543+0000","last_unstale":"2024-09-10T09:58:21.792397+0000","last_undegraded":"2024-09-10T09:58:21.792397+0000","last_fullsized":"2024-09-10T09:58:21.792397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760512+0000","last_change":"2024-09-10T09:58:22.760512+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760512+0000","last_undegraded":"2024-09-10T09:58:22.760512+0000","last_fullsized":"2024-09-10T09:58:22.760512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761985+0000","last_change":"2024-09-10T09:58:22.761985+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761985+0000","last_undegraded":"2024-09-10T09:58:22.761985+0000","last_fullsized":"2024-09-10T09:58:22.761985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.794737+0000","last_change":"2024-09-10T09:58:21.793454+0000","last_active":"2024-09-10T09:58:21.794737+0000","last_peered":"2024-09-10T09:58:21.794737+0000","last_clean":"2024-09-10T09:58:21.794737+0000","last_became_active":"2024-09-10T09:58:21.793058+0000","last_became_peered":"2024-09-10T09:58:21.793058+0000","last_unstale":"2024-09-10T09:58:21.794737+0000","last_undegraded":"2024-09-10T09:58:21.794737+0000","last_fullsized":"2024-09-10T09:58:21.794737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760886+0000","last_change":"2024-09-10T09:58:22.760886+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760886+0000","last_undegraded":"2024-09-10T09:58:22.760886+0000","last_fullsized":"2024-09-10T09:58:22.760886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792296+0000","last_change":"2024-09-10T09:58:21.789114+0000","last_active":"2024-09-10T09:58:21.792296+0000","last_peered":"2024-09-10T09:58:21.792296+0000","last_clean":"2024-09-10T09:58:21.792296+0000","last_became_active":"2024-09-10T09:58:21.788849+0000","last_became_peered":"2024-09-10T09:58:21.788849+0000","last_unstale":"2024-09-10T09:58:21.792296+0000","last_undegraded":"2024-09-10T09:58:21.792296+0000","last_fullsized":"2024-09-10T09:58:21.792296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:00.039731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.773428+0000","last_change":"2024-09-10T09:58:21.773428+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.773428+0000","last_undegraded":"2024-09-10T09:58:21.773428+0000","last_fullsized":"2024-09-10T09:58:21.773428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T09:58:20.755150+0000","last_change":"2024-09-10T09:58:20.755150+0000","last_active":"2024-09-10T09:58:20.740599+0000","last_peered":"2024-09-10T09:58:20.740599+0000","last_clean":"2024-09-10T09:58:20.740599+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:20.755150+0000","last_undegraded":"2024-09-10T09:58:20.755150+0000","last_fullsized":"2024-09-10T09:58:20.755150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.788781+0000","last_change":"2024-09-10T09:58:21.787400+0000","last_active":"2024-09-10T09:58:21.788781+0000","last_peered":"2024-09-10T09:58:21.788781+0000","last_clean":"2024-09-10T09:58:21.788781+0000","last_became_active":"2024-09-10T09:58:21.787111+0000","last_became_peered":"2024-09-10T09:58:21.787111+0000","last_unstale":"2024-09-10T09:58:21.788781+0000","last_undegraded":"2024-09-10T09:58:21.788781+0000","last_fullsized":"2024-09-10T09:58:21.788781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:11.144974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":802816,"data_stored":658248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509443,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":658248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":55788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:58:26.843 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:58:27.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ea00640 1 Processor -- start 2024-09-10T09:58:27.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ea00640 1 -- start start 2024-09-10T09:58:27.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c880cd510 0x7f5c880cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5c8806e090 0x7f5c8806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ea00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c88072150 con 0x7f5c880cfe30 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ea00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5c880cdeb0 con 0x7f5c8806e090 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ea00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8815d9a0 con 0x7f5c880cd510 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ca00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5c8806e090 0x7f5c8806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ca00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5c8806e090 0x7f5c8806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46578/0 (socket says 172.21.15.110:46578) 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c87e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5c880cfe30 0x7f5c88071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41776/0 (socket says 172.21.15.110:41776) 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.013+0000 7f5c8ca00640 1 -- 172.21.15.110:0/1103776099 learned_addr learned my addr 172.21.15.110:0/1103776099 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c7fe00640 1 --2- 172.21.15.110:0/1103776099 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c880cd510 0x7f5c880cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ca00640 1 -- 172.21.15.110:0/1103776099 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c880cd510 msgr2=0x7f5c880cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ca00640 1 --2- 172.21.15.110:0/1103776099 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c880cd510 0x7f5c880cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ca00640 1 -- 172.21.15.110:0/1103776099 >> v1:172.21.15.110:6789/0 conn(0x7f5c880cfe30 legacy=0x7f5c88071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ca00640 1 -- 172.21.15.110:0/1103776099 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8806ea60 con 0x7f5c8806e090 2024-09-10T09:58:27.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c87400640 1 -- 172.21.15.110:0/1103776099 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565239699 0 0) 0x7f5c88072150 con 0x7f5c880cfe30 2024-09-10T09:58:27.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ca00640 1 --2- 172.21.15.110:0/1103776099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c8806e090 0x7f5c8806e520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009e50 tx=0x7f5c8001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e6e4639d9e4ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c87400640 1 -- 172.21.15.110:0/1103776099 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c800a5770 con 0x7f5c8806e090 2024-09-10T09:58:27.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c87400640 1 -- 172.21.15.110:0/1103776099 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c800a5910 con 0x7f5c8806e090 2024-09-10T09:58:27.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c87400640 1 -- 172.21.15.110:0/1103776099 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c800a5ce0 con 0x7f5c8806e090 2024-09-10T09:58:27.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ea00640 1 -- 172.21.15.110:0/1103776099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c8806e090 msgr2=0x7f5c8806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/1103776099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c8806e090 0x7f5c8806e520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009e50 tx=0x7f5c8001c9a0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ea00640 1 -- 172.21.15.110:0/1103776099 shutdown_connections 2024-09-10T09:58:27.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/1103776099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c8806e090 0x7f5c8806e520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/1103776099 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c880cd510 0x7f5c880cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.017+0000 7f5c8ea00640 1 -- 172.21.15.110:0/1103776099 >> 172.21.15.110:0/1103776099 conn(0x7f5c8806bc50 msgr2=0x7f5c8806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 -- 172.21.15.110:0/1103776099 shutdown_connections 2024-09-10T09:58:27.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 -- 172.21.15.110:0/1103776099 wait complete. 2024-09-10T09:58:27.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 Processor -- start 2024-09-10T09:58:27.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 -- start start 2024-09-10T09:58:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c8806e090 0x7f5c88151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c880cd510 0x7f5c88154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880cfe30 0x7f5c88157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8815d5b0 con 0x7f5c880cd510 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8815d430 con 0x7f5c880cfe30 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8815d730 con 0x7f5c8806e090 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c880cd510 0x7f5c88154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c880cd510 0x7f5c88154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46580/0 (socket says 172.21.15.110:46580) 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 -- 172.21.15.110:0/248300644 learned_addr learned my addr 172.21.15.110:0/248300644 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c87e00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c8806e090 0x7f5c88151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 -- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c8806e090 msgr2=0x7f5c88151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c8806e090 0x7f5c88151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 -- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880cfe30 msgr2=0x7f5c88157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ca00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880cfe30 0x7f5c88157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880cfe30 0x7f5c88157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 -- 172.21.15.110:0/248300644 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c881566b0 con 0x7f5c880cd510 2024-09-10T09:58:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c87e00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c8806e090 0x7f5c88151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ca00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880cfe30 0x7f5c88157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c7fe00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c880cd510 0x7f5c88154db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000d620 tx=0x7f5c7000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c84c00640 1 -- 172.21.15.110:0/248300644 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000b620 con 0x7f5c880cd510 2024-09-10T09:58:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c84c00640 1 -- 172.21.15.110:0/248300644 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c70019070 con 0x7f5c880cd510 2024-09-10T09:58:27.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c88156990 con 0x7f5c880cd510 2024-09-10T09:58:27.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c84c00640 1 -- 172.21.15.110:0/248300644 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000b9e0 con 0x7f5c880cd510 2024-09-10T09:58:27.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.021+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c88151540 con 0x7f5c880cd510 2024-09-10T09:58:27.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.025+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c8806e520 con 0x7f5c880cd510 2024-09-10T09:58:27.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.025+0000 7f5c84c00640 1 -- 172.21.15.110:0/248300644 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5c7000bb80 con 0x7f5c880cd510 2024-09-10T09:58:27.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.025+0000 7f5c84c00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5c640678c0 0x7f5c64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.025+0000 7f5c84c00640 1 -- 172.21.15.110:0/248300644 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5c700860b0 con 0x7f5c880cd510 2024-09-10T09:58:27.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.025+0000 7f5c87e00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5c640678c0 0x7f5c64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.025+0000 7f5c87e00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5c640678c0 0x7f5c64069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007fd0 tx=0x7f5c78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.029+0000 7f5c84c00640 1 -- 172.21.15.110:0/248300644 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5c70056bf0 con 0x7f5c880cd510 2024-09-10T09:58:27.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.185+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c881529a0 con 0x7f5c640678c0 2024-09-10T09:58:27.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.193+0000 7f5c84c00640 1 -- 172.21.15.110:0/248300644 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246050 (secure 0 0 0) 0x7f5c881529a0 con 0x7f5c640678c0 2024-09-10T09:58:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:27.196 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:58:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.193+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5c640678c0 msgr2=0x7f5c64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.193+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5c640678c0 0x7f5c64069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007fd0 tx=0x7f5c78009040 comp rx=0 tx=0).stop 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.193+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c880cd510 msgr2=0x7f5c88154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.193+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c880cd510 0x7f5c88154db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000d620 tx=0x7f5c7000dae0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.197+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 shutdown_connections 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.197+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5c640678c0 0x7f5c64069d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.197+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880cfe30 0x7f5c88157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.197+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c880cd510 0x7f5c88154db0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.197+0000 7f5c8ea00640 1 --2- 172.21.15.110:0/248300644 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c8806e090 0x7f5c88151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.197+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 >> 172.21.15.110:0/248300644 conn(0x7f5c8806bc50 msgr2=0x7f5c880ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.197+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 shutdown_connections 2024-09-10T09:58:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.197+0000 7f5c8ea00640 1 -- 172.21.15.110:0/248300644 wait complete. 2024-09-10T09:58:27.215 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T09:58:25.355804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235656,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739064,"statfs":{"total":773094113280,"available":772852801536,"internally_reserved":0,"allocated":2834432,"data_stored":1711684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.608542"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.773483+0000","last_change":"2024-09-10T09:58:21.773483+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.773483+0000","last_undegraded":"2024-09-10T09:58:21.773483+0000","last_fullsized":"2024-09-10T09:58:21.773483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758695+0000","last_change":"2024-09-10T09:58:22.758695+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758695+0000","last_undegraded":"2024-09-10T09:58:22.758695+0000","last_fullsized":"2024-09-10T09:58:22.758695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770858+0000","last_change":"2024-09-10T09:58:21.770858+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770858+0000","last_undegraded":"2024-09-10T09:58:21.770858+0000","last_fullsized":"2024-09-10T09:58:21.770858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761137+0000","last_change":"2024-09-10T09:58:22.761137+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761137+0000","last_undegraded":"2024-09-10T09:58:22.761137+0000","last_fullsized":"2024-09-10T09:58:22.761137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770682+0000","last_change":"2024-09-10T09:58:21.770682+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770682+0000","last_undegraded":"2024-09-10T09:58:21.770682+0000","last_fullsized":"2024-09-10T09:58:21.770682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762312+0000","last_change":"2024-09-10T09:58:22.762312+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762312+0000","last_undegraded":"2024-09-10T09:58:22.762312+0000","last_fullsized":"2024-09-10T09:58:22.762312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.780164+0000","last_change":"2024-09-10T09:58:21.780164+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.780164+0000","last_undegraded":"2024-09-10T09:58:21.780164+0000","last_fullsized":"2024-09-10T09:58:21.780164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772032+0000","last_change":"2024-09-10T09:58:21.772032+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772032+0000","last_undegraded":"2024-09-10T09:58:21.772032+0000","last_fullsized":"2024-09-10T09:58:21.772032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760582+0000","last_change":"2024-09-10T09:58:22.760582+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760582+0000","last_undegraded":"2024-09-10T09:58:22.760582+0000","last_fullsized":"2024-09-10T09:58:22.760582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762468+0000","last_change":"2024-09-10T09:58:22.762468+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762468+0000","last_undegraded":"2024-09-10T09:58:22.762468+0000","last_fullsized":"2024-09-10T09:58:22.762468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767842+0000","last_change":"2024-09-10T09:58:21.767842+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767842+0000","last_undegraded":"2024-09-10T09:58:21.767842+0000","last_fullsized":"2024-09-10T09:58:21.767842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.765480+0000","last_change":"2024-09-10T09:58:21.765480+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.765480+0000","last_undegraded":"2024-09-10T09:58:21.765480+0000","last_fullsized":"2024-09-10T09:58:21.765480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758765+0000","last_change":"2024-09-10T09:58:22.758765+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758765+0000","last_undegraded":"2024-09-10T09:58:22.758765+0000","last_fullsized":"2024-09-10T09:58:22.758765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.764622+0000","last_change":"2024-09-10T09:58:22.764622+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.764622+0000","last_undegraded":"2024-09-10T09:58:22.764622+0000","last_fullsized":"2024-09-10T09:58:22.764622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760834+0000","last_change":"2024-09-10T09:58:22.760834+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760834+0000","last_undegraded":"2024-09-10T09:58:22.760834+0000","last_fullsized":"2024-09-10T09:58:22.760834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.762859+0000","last_change":"2024-09-10T09:58:21.762859+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.762859+0000","last_undegraded":"2024-09-10T09:58:21.762859+0000","last_fullsized":"2024-09-10T09:58:21.762859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767720+0000","last_change":"2024-09-10T09:58:21.767720+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767720+0000","last_undegraded":"2024-09-10T09:58:21.767720+0000","last_fullsized":"2024-09-10T09:58:21.767720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.763497+0000","last_change":"2024-09-10T09:58:22.763497+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.763497+0000","last_undegraded":"2024-09-10T09:58:22.763497+0000","last_fullsized":"2024-09-10T09:58:22.763497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.765123+0000","last_change":"2024-09-10T09:58:21.765123+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.765123+0000","last_undegraded":"2024-09-10T09:58:21.765123+0000","last_fullsized":"2024-09-10T09:58:21.765123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762017+0000","last_change":"2024-09-10T09:58:22.762017+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762017+0000","last_undegraded":"2024-09-10T09:58:22.762017+0000","last_fullsized":"2024-09-10T09:58:22.762017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.764229+0000","last_change":"2024-09-10T09:58:22.764229+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.764229+0000","last_undegraded":"2024-09-10T09:58:22.764229+0000","last_fullsized":"2024-09-10T09:58:22.764229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.769476+0000","last_change":"2024-09-10T09:58:22.769476+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.769476+0000","last_undegraded":"2024-09-10T09:58:22.769476+0000","last_fullsized":"2024-09-10T09:58:22.769476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.759733+0000","last_change":"2024-09-10T09:58:22.759733+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.759733+0000","last_undegraded":"2024-09-10T09:58:22.759733+0000","last_fullsized":"2024-09-10T09:58:22.759733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.764586+0000","last_change":"2024-09-10T09:58:21.764586+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.764586+0000","last_undegraded":"2024-09-10T09:58:21.764586+0000","last_fullsized":"2024-09-10T09:58:21.764586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761625+0000","last_change":"2024-09-10T09:58:22.761625+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761625+0000","last_undegraded":"2024-09-10T09:58:22.761625+0000","last_fullsized":"2024-09-10T09:58:22.761625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.762149+0000","last_change":"2024-09-10T09:58:21.762149+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.762149+0000","last_undegraded":"2024-09-10T09:58:21.762149+0000","last_fullsized":"2024-09-10T09:58:21.762149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760420+0000","last_change":"2024-09-10T09:58:22.760420+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760420+0000","last_undegraded":"2024-09-10T09:58:22.760420+0000","last_fullsized":"2024-09-10T09:58:22.760420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.768406+0000","last_change":"2024-09-10T09:58:21.768406+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.768406+0000","last_undegraded":"2024-09-10T09:58:21.768406+0000","last_fullsized":"2024-09-10T09:58:21.768406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T09:58:20.755078+0000","last_change":"2024-09-10T09:58:20.755078+0000","last_active":"2024-09-10T09:58:20.740599+0000","last_peered":"2024-09-10T09:58:20.740599+0000","last_clean":"2024-09-10T09:58:20.740599+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:20.755078+0000","last_undegraded":"2024-09-10T09:58:20.755078+0000","last_fullsized":"2024-09-10T09:58:20.755078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.764021+0000","last_change":"2024-09-10T09:58:21.764021+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.764021+0000","last_undegraded":"2024-09-10T09:58:21.764021+0000","last_fullsized":"2024-09-10T09:58:21.764021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.765118+0000","last_change":"2024-09-10T09:58:21.778244+0000","last_active":"2024-09-10T09:58:22.765118+0000","last_peered":"2024-09-10T09:58:22.765118+0000","last_clean":"2024-09-10T09:58:22.765118+0000","last_became_active":"2024-09-10T09:58:21.777937+0000","last_became_peered":"2024-09-10T09:58:21.777937+0000","last_unstale":"2024-09-10T09:58:22.765118+0000","last_undegraded":"2024-09-10T09:58:22.765118+0000","last_fullsized":"2024-09-10T09:58:22.765118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T09:58:20.755035+0000","last_change":"2024-09-10T09:58:20.755035+0000","last_active":"2024-09-10T09:58:20.740599+0000","last_peered":"2024-09-10T09:58:20.740599+0000","last_clean":"2024-09-10T09:58:20.740599+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:20.755035+0000","last_undegraded":"2024-09-10T09:58:20.755035+0000","last_fullsized":"2024-09-10T09:58:20.755035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.765584+0000","last_change":"2024-09-10T09:58:21.786530+0000","last_active":"2024-09-10T09:58:22.765584+0000","last_peered":"2024-09-10T09:58:22.765584+0000","last_clean":"2024-09-10T09:58:22.765584+0000","last_became_active":"2024-09-10T09:58:21.786157+0000","last_became_peered":"2024-09-10T09:58:21.786157+0000","last_unstale":"2024-09-10T09:58:22.765584+0000","last_undegraded":"2024-09-10T09:58:22.765584+0000","last_fullsized":"2024-09-10T09:58:22.765584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761036+0000","last_change":"2024-09-10T09:58:22.761036+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761036+0000","last_undegraded":"2024-09-10T09:58:22.761036+0000","last_fullsized":"2024-09-10T09:58:22.761036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761809+0000","last_change":"2024-09-10T09:58:21.786956+0000","last_active":"2024-09-10T09:58:22.761809+0000","last_peered":"2024-09-10T09:58:22.761809+0000","last_clean":"2024-09-10T09:58:22.761809+0000","last_became_active":"2024-09-10T09:58:21.786523+0000","last_became_peered":"2024-09-10T09:58:21.786523+0000","last_unstale":"2024-09-10T09:58:22.761809+0000","last_undegraded":"2024-09-10T09:58:22.761809+0000","last_fullsized":"2024-09-10T09:58:22.761809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761458+0000","last_change":"2024-09-10T09:58:22.761458+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761458+0000","last_undegraded":"2024-09-10T09:58:22.761458+0000","last_fullsized":"2024-09-10T09:58:22.761458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761846+0000","last_change":"2024-09-10T09:58:21.787382+0000","last_active":"2024-09-10T09:58:22.761846+0000","last_peered":"2024-09-10T09:58:22.761846+0000","last_clean":"2024-09-10T09:58:22.761846+0000","last_became_active":"2024-09-10T09:58:21.787048+0000","last_became_peered":"2024-09-10T09:58:21.787048+0000","last_unstale":"2024-09-10T09:58:22.761846+0000","last_undegraded":"2024-09-10T09:58:22.761846+0000","last_fullsized":"2024-09-10T09:58:22.761846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772636+0000","last_change":"2024-09-10T09:58:21.772636+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772636+0000","last_undegraded":"2024-09-10T09:58:21.772636+0000","last_fullsized":"2024-09-10T09:58:21.772636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759611+0000","last_change":"2024-09-10T09:58:21.785126+0000","last_active":"2024-09-10T09:58:22.759611+0000","last_peered":"2024-09-10T09:58:22.759611+0000","last_clean":"2024-09-10T09:58:22.759611+0000","last_became_active":"2024-09-10T09:58:21.784839+0000","last_became_peered":"2024-09-10T09:58:21.784839+0000","last_unstale":"2024-09-10T09:58:22.759611+0000","last_undegraded":"2024-09-10T09:58:22.759611+0000","last_fullsized":"2024-09-10T09:58:22.759611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767249+0000","last_change":"2024-09-10T09:58:21.767249+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767249+0000","last_undegraded":"2024-09-10T09:58:21.767249+0000","last_fullsized":"2024-09-10T09:58:21.767249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.763857+0000","last_change":"2024-09-10T09:58:21.776340+0000","last_active":"2024-09-10T09:58:22.763857+0000","last_peered":"2024-09-10T09:58:22.763857+0000","last_clean":"2024-09-10T09:58:22.763857+0000","last_became_active":"2024-09-10T09:58:21.776066+0000","last_became_peered":"2024-09-10T09:58:21.776066+0000","last_unstale":"2024-09-10T09:58:22.763857+0000","last_undegraded":"2024-09-10T09:58:22.763857+0000","last_fullsized":"2024-09-10T09:58:22.763857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772836+0000","last_change":"2024-09-10T09:58:21.772836+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772836+0000","last_undegraded":"2024-09-10T09:58:21.772836+0000","last_fullsized":"2024-09-10T09:58:21.772836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.785464+0000","last_change":"2024-09-10T09:58:21.783558+0000","last_active":"2024-09-10T09:58:21.785464+0000","last_peered":"2024-09-10T09:58:21.785464+0000","last_clean":"2024-09-10T09:58:21.785464+0000","last_became_active":"2024-09-10T09:58:21.782455+0000","last_became_peered":"2024-09-10T09:58:21.782455+0000","last_unstale":"2024-09-10T09:58:21.785464+0000","last_undegraded":"2024-09-10T09:58:21.785464+0000","last_fullsized":"2024-09-10T09:58:21.785464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:26.367199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.763413+0000","last_change":"2024-09-10T09:58:21.763413+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.763413+0000","last_undegraded":"2024-09-10T09:58:21.763413+0000","last_fullsized":"2024-09-10T09:58:21.763413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762503+0000","last_change":"2024-09-10T09:58:21.789974+0000","last_active":"2024-09-10T09:58:22.762503+0000","last_peered":"2024-09-10T09:58:22.762503+0000","last_clean":"2024-09-10T09:58:22.762503+0000","last_became_active":"2024-09-10T09:58:21.788481+0000","last_became_peered":"2024-09-10T09:58:21.788481+0000","last_unstale":"2024-09-10T09:58:22.762503+0000","last_undegraded":"2024-09-10T09:58:22.762503+0000","last_fullsized":"2024-09-10T09:58:22.762503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769168+0000","last_change":"2024-09-10T09:58:21.769168+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769168+0000","last_undegraded":"2024-09-10T09:58:21.769168+0000","last_fullsized":"2024-09-10T09:58:21.769168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.760516+0000","last_change":"2024-09-10T09:58:21.778738+0000","last_active":"2024-09-10T09:58:22.760516+0000","last_peered":"2024-09-10T09:58:22.760516+0000","last_clean":"2024-09-10T09:58:22.760516+0000","last_became_active":"2024-09-10T09:58:21.777982+0000","last_became_peered":"2024-09-10T09:58:21.777982+0000","last_unstale":"2024-09-10T09:58:22.760516+0000","last_undegraded":"2024-09-10T09:58:22.760516+0000","last_fullsized":"2024-09-10T09:58:22.760516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761175+0000","last_change":"2024-09-10T09:58:22.761175+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761175+0000","last_undegraded":"2024-09-10T09:58:22.761175+0000","last_fullsized":"2024-09-10T09:58:22.761175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.782949+0000","last_change":"2024-09-10T09:58:21.779750+0000","last_active":"2024-09-10T09:58:21.782949+0000","last_peered":"2024-09-10T09:58:21.782949+0000","last_clean":"2024-09-10T09:58:21.782949+0000","last_became_active":"2024-09-10T09:58:21.779492+0000","last_became_peered":"2024-09-10T09:58:21.779492+0000","last_unstale":"2024-09-10T09:58:21.782949+0000","last_undegraded":"2024-09-10T09:58:21.782949+0000","last_fullsized":"2024-09-10T09:58:21.782949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:26.367199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772393+0000","last_change":"2024-09-10T09:58:21.772393+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772393+0000","last_undegraded":"2024-09-10T09:58:21.772393+0000","last_fullsized":"2024-09-10T09:58:21.772393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.789668+0000","last_change":"2024-09-10T09:58:21.781792+0000","last_active":"2024-09-10T09:58:21.789668+0000","last_peered":"2024-09-10T09:58:21.789668+0000","last_clean":"2024-09-10T09:58:21.789668+0000","last_became_active":"2024-09-10T09:58:21.781434+0000","last_became_peered":"2024-09-10T09:58:21.781434+0000","last_unstale":"2024-09-10T09:58:21.789668+0000","last_undegraded":"2024-09-10T09:58:21.789668+0000","last_fullsized":"2024-09-10T09:58:21.789668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:42.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767018+0000","last_change":"2024-09-10T09:58:21.767018+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767018+0000","last_undegraded":"2024-09-10T09:58:21.767018+0000","last_fullsized":"2024-09-10T09:58:21.767018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.783020+0000","last_change":"2024-09-10T09:58:21.781287+0000","last_active":"2024-09-10T09:58:21.783020+0000","last_peered":"2024-09-10T09:58:21.783020+0000","last_clean":"2024-09-10T09:58:21.783020+0000","last_became_active":"2024-09-10T09:58:21.780824+0000","last_became_peered":"2024-09-10T09:58:21.780824+0000","last_unstale":"2024-09-10T09:58:21.783020+0000","last_undegraded":"2024-09-10T09:58:21.783020+0000","last_fullsized":"2024-09-10T09:58:21.783020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:11.144974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769098+0000","last_change":"2024-09-10T09:58:21.769098+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769098+0000","last_undegraded":"2024-09-10T09:58:21.769098+0000","last_fullsized":"2024-09-10T09:58:21.769098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759829+0000","last_change":"2024-09-10T09:58:21.776106+0000","last_active":"2024-09-10T09:58:22.759829+0000","last_peered":"2024-09-10T09:58:22.759829+0000","last_clean":"2024-09-10T09:58:22.759829+0000","last_became_active":"2024-09-10T09:58:21.773315+0000","last_became_peered":"2024-09-10T09:58:21.773315+0000","last_unstale":"2024-09-10T09:58:22.759829+0000","last_undegraded":"2024-09-10T09:58:22.759829+0000","last_fullsized":"2024-09-10T09:58:22.759829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760733+0000","last_change":"2024-09-10T09:58:22.760733+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760733+0000","last_undegraded":"2024-09-10T09:58:22.760733+0000","last_fullsized":"2024-09-10T09:58:22.760733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.790976+0000","last_change":"2024-09-10T09:58:21.789986+0000","last_active":"2024-09-10T09:58:21.790976+0000","last_peered":"2024-09-10T09:58:21.790976+0000","last_clean":"2024-09-10T09:58:21.790976+0000","last_became_active":"2024-09-10T09:58:21.789577+0000","last_became_peered":"2024-09-10T09:58:21.789577+0000","last_unstale":"2024-09-10T09:58:21.790976+0000","last_undegraded":"2024-09-10T09:58:21.790976+0000","last_fullsized":"2024-09-10T09:58:21.790976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.768360+0000","last_change":"2024-09-10T09:58:21.768360+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.768360+0000","last_undegraded":"2024-09-10T09:58:21.768360+0000","last_fullsized":"2024-09-10T09:58:21.768360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":166,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.783072+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:58:21.783072+0000","last_peered":"2024-09-10T09:58:21.783072+0000","last_clean":"2024-09-10T09:58:21.783072+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:58:21.783072+0000","last_undegraded":"2024-09-10T09:58:21.783072+0000","last_fullsized":"2024-09-10T09:58:21.783072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762945+0000","last_change":"2024-09-10T09:58:21.789884+0000","last_active":"2024-09-10T09:58:22.762945+0000","last_peered":"2024-09-10T09:58:22.762945+0000","last_clean":"2024-09-10T09:58:22.762945+0000","last_became_active":"2024-09-10T09:58:21.788940+0000","last_became_peered":"2024-09-10T09:58:21.788940+0000","last_unstale":"2024-09-10T09:58:22.762945+0000","last_undegraded":"2024-09-10T09:58:22.762945+0000","last_fullsized":"2024-09-10T09:58:22.762945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792384+0000","last_change":"2024-09-10T09:58:21.791063+0000","last_active":"2024-09-10T09:58:21.792384+0000","last_peered":"2024-09-10T09:58:21.792384+0000","last_clean":"2024-09-10T09:58:21.792384+0000","last_became_active":"2024-09-10T09:58:21.790796+0000","last_became_peered":"2024-09-10T09:58:21.790796+0000","last_unstale":"2024-09-10T09:58:21.792384+0000","last_undegraded":"2024-09-10T09:58:21.792384+0000","last_fullsized":"2024-09-10T09:58:21.792384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.761973+0000","last_change":"2024-09-10T09:58:21.761973+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.761973+0000","last_undegraded":"2024-09-10T09:58:21.761973+0000","last_fullsized":"2024-09-10T09:58:21.761973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.781486+0000","last_change":"2024-09-10T09:58:21.777208+0000","last_active":"2024-09-10T09:58:21.781486+0000","last_peered":"2024-09-10T09:58:21.781486+0000","last_clean":"2024-09-10T09:58:21.781486+0000","last_became_active":"2024-09-10T09:58:21.776820+0000","last_became_peered":"2024-09-10T09:58:21.776820+0000","last_unstale":"2024-09-10T09:58:21.781486+0000","last_undegraded":"2024-09-10T09:58:21.781486+0000","last_fullsized":"2024-09-10T09:58:21.781486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:42.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.759166+0000","last_change":"2024-09-10T09:58:22.759166+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.759166+0000","last_undegraded":"2024-09-10T09:58:22.759166+0000","last_fullsized":"2024-09-10T09:58:22.759166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762585+0000","last_change":"2024-09-10T09:58:21.785388+0000","last_active":"2024-09-10T09:58:22.762585+0000","last_peered":"2024-09-10T09:58:22.762585+0000","last_clean":"2024-09-10T09:58:22.762585+0000","last_became_active":"2024-09-10T09:58:21.785227+0000","last_became_peered":"2024-09-10T09:58:21.785227+0000","last_unstale":"2024-09-10T09:58:22.762585+0000","last_undegraded":"2024-09-10T09:58:22.762585+0000","last_fullsized":"2024-09-10T09:58:22.762585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770833+0000","last_change":"2024-09-10T09:58:21.770833+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770833+0000","last_undegraded":"2024-09-10T09:58:21.770833+0000","last_fullsized":"2024-09-10T09:58:21.770833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.758612+0000","last_change":"2024-09-10T09:58:21.768431+0000","last_active":"2024-09-10T09:58:22.758612+0000","last_peered":"2024-09-10T09:58:22.758612+0000","last_clean":"2024-09-10T09:58:22.758612+0000","last_became_active":"2024-09-10T09:58:21.767826+0000","last_became_peered":"2024-09-10T09:58:21.767826+0000","last_unstale":"2024-09-10T09:58:22.758612+0000","last_undegraded":"2024-09-10T09:58:22.758612+0000","last_fullsized":"2024-09-10T09:58:22.758612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761547+0000","last_change":"2024-09-10T09:58:22.761547+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761547+0000","last_undegraded":"2024-09-10T09:58:22.761547+0000","last_fullsized":"2024-09-10T09:58:22.761547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792043+0000","last_change":"2024-09-10T09:58:21.788480+0000","last_active":"2024-09-10T09:58:21.792043+0000","last_peered":"2024-09-10T09:58:21.792043+0000","last_clean":"2024-09-10T09:58:21.792043+0000","last_became_active":"2024-09-10T09:58:21.788162+0000","last_became_peered":"2024-09-10T09:58:21.788162+0000","last_unstale":"2024-09-10T09:58:21.792043+0000","last_undegraded":"2024-09-10T09:58:21.792043+0000","last_fullsized":"2024-09-10T09:58:21.792043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:05.666246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758668+0000","last_change":"2024-09-10T09:58:22.758668+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758668+0000","last_undegraded":"2024-09-10T09:58:22.758668+0000","last_fullsized":"2024-09-10T09:58:22.758668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.758990+0000","last_change":"2024-09-10T09:58:21.770640+0000","last_active":"2024-09-10T09:58:22.758990+0000","last_peered":"2024-09-10T09:58:22.758990+0000","last_clean":"2024-09-10T09:58:22.758990+0000","last_became_active":"2024-09-10T09:58:21.769111+0000","last_became_peered":"2024-09-10T09:58:21.769111+0000","last_unstale":"2024-09-10T09:58:22.758990+0000","last_undegraded":"2024-09-10T09:58:22.758990+0000","last_fullsized":"2024-09-10T09:58:22.758990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761978+0000","last_change":"2024-09-10T09:58:21.782019+0000","last_active":"2024-09-10T09:58:22.761978+0000","last_peered":"2024-09-10T09:58:22.761978+0000","last_clean":"2024-09-10T09:58:22.761978+0000","last_became_active":"2024-09-10T09:58:21.781750+0000","last_became_peered":"2024-09-10T09:58:21.781750+0000","last_unstale":"2024-09-10T09:58:22.761978+0000","last_undegraded":"2024-09-10T09:58:22.761978+0000","last_fullsized":"2024-09-10T09:58:22.761978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770772+0000","last_change":"2024-09-10T09:58:21.770772+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770772+0000","last_undegraded":"2024-09-10T09:58:21.770772+0000","last_fullsized":"2024-09-10T09:58:21.770772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.791839+0000","last_change":"2024-09-10T09:58:21.788078+0000","last_active":"2024-09-10T09:58:21.791839+0000","last_peered":"2024-09-10T09:58:21.791839+0000","last_clean":"2024-09-10T09:58:21.791839+0000","last_became_active":"2024-09-10T09:58:21.787660+0000","last_became_peered":"2024-09-10T09:58:21.787660+0000","last_unstale":"2024-09-10T09:58:21.791839+0000","last_undegraded":"2024-09-10T09:58:21.791839+0000","last_fullsized":"2024-09-10T09:58:21.791839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769618+0000","last_change":"2024-09-10T09:58:21.769618+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769618+0000","last_undegraded":"2024-09-10T09:58:21.769618+0000","last_fullsized":"2024-09-10T09:58:21.769618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759017+0000","last_change":"2024-09-10T09:58:21.782043+0000","last_active":"2024-09-10T09:58:22.759017+0000","last_peered":"2024-09-10T09:58:22.759017+0000","last_clean":"2024-09-10T09:58:22.759017+0000","last_became_active":"2024-09-10T09:58:21.781762+0000","last_became_peered":"2024-09-10T09:58:21.781762+0000","last_unstale":"2024-09-10T09:58:22.759017+0000","last_undegraded":"2024-09-10T09:58:22.759017+0000","last_fullsized":"2024-09-10T09:58:22.759017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759075+0000","last_change":"2024-09-10T09:58:21.774362+0000","last_active":"2024-09-10T09:58:22.759075+0000","last_peered":"2024-09-10T09:58:22.759075+0000","last_clean":"2024-09-10T09:58:22.759075+0000","last_became_active":"2024-09-10T09:58:21.773133+0000","last_became_peered":"2024-09-10T09:58:21.773133+0000","last_unstale":"2024-09-10T09:58:22.759075+0000","last_undegraded":"2024-09-10T09:58:22.759075+0000","last_fullsized":"2024-09-10T09:58:22.759075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792397+0000","last_change":"2024-09-10T09:58:21.790948+0000","last_active":"2024-09-10T09:58:21.792397+0000","last_peered":"2024-09-10T09:58:21.792397+0000","last_clean":"2024-09-10T09:58:21.792397+0000","last_became_active":"2024-09-10T09:58:21.790543+0000","last_became_peered":"2024-09-10T09:58:21.790543+0000","last_unstale":"2024-09-10T09:58:21.792397+0000","last_undegraded":"2024-09-10T09:58:21.792397+0000","last_fullsized":"2024-09-10T09:58:21.792397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760512+0000","last_change":"2024-09-10T09:58:22.760512+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760512+0000","last_undegraded":"2024-09-10T09:58:22.760512+0000","last_fullsized":"2024-09-10T09:58:22.760512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761985+0000","last_change":"2024-09-10T09:58:22.761985+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761985+0000","last_undegraded":"2024-09-10T09:58:22.761985+0000","last_fullsized":"2024-09-10T09:58:22.761985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.794737+0000","last_change":"2024-09-10T09:58:21.793454+0000","last_active":"2024-09-10T09:58:21.794737+0000","last_peered":"2024-09-10T09:58:21.794737+0000","last_clean":"2024-09-10T09:58:21.794737+0000","last_became_active":"2024-09-10T09:58:21.793058+0000","last_became_peered":"2024-09-10T09:58:21.793058+0000","last_unstale":"2024-09-10T09:58:21.794737+0000","last_undegraded":"2024-09-10T09:58:21.794737+0000","last_fullsized":"2024-09-10T09:58:21.794737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760886+0000","last_change":"2024-09-10T09:58:22.760886+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760886+0000","last_undegraded":"2024-09-10T09:58:22.760886+0000","last_fullsized":"2024-09-10T09:58:22.760886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792296+0000","last_change":"2024-09-10T09:58:21.789114+0000","last_active":"2024-09-10T09:58:21.792296+0000","last_peered":"2024-09-10T09:58:21.792296+0000","last_clean":"2024-09-10T09:58:21.792296+0000","last_became_active":"2024-09-10T09:58:21.788849+0000","last_became_peered":"2024-09-10T09:58:21.788849+0000","last_unstale":"2024-09-10T09:58:21.792296+0000","last_undegraded":"2024-09-10T09:58:21.792296+0000","last_fullsized":"2024-09-10T09:58:21.792296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:00.039731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.773428+0000","last_change":"2024-09-10T09:58:21.773428+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.773428+0000","last_undegraded":"2024-09-10T09:58:21.773428+0000","last_fullsized":"2024-09-10T09:58:21.773428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T09:58:20.755150+0000","last_change":"2024-09-10T09:58:20.755150+0000","last_active":"2024-09-10T09:58:20.740599+0000","last_peered":"2024-09-10T09:58:20.740599+0000","last_clean":"2024-09-10T09:58:20.740599+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:20.755150+0000","last_undegraded":"2024-09-10T09:58:20.755150+0000","last_fullsized":"2024-09-10T09:58:20.755150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:58:21.747139+0000","last_change":"2024-09-10T09:58:21.747139+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.747139+0000","last_undegraded":"2024-09-10T09:58:21.747139+0000","last_fullsized":"2024-09-10T09:58:21.747139+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.788781+0000","last_change":"2024-09-10T09:58:21.787400+0000","last_active":"2024-09-10T09:58:21.788781+0000","last_peered":"2024-09-10T09:58:21.788781+0000","last_clean":"2024-09-10T09:58:21.788781+0000","last_became_active":"2024-09-10T09:58:21.787111+0000","last_became_peered":"2024-09-10T09:58:21.787111+0000","last_unstale":"2024-09-10T09:58:21.788781+0000","last_undegraded":"2024-09-10T09:58:21.788781+0000","last_fullsized":"2024-09-10T09:58:21.788781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:11.144974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":802816,"data_stored":658248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509443,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":658248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":55788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:58:27.220 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:58:27.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.393+0000 7f7113000640 1 Processor -- start 2024-09-10T09:58:27.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.393+0000 7f7113000640 1 -- start start 2024-09-10T09:58:27.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f7113000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f710c155970 0x7f710c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f7113000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f710c155d50 con 0x7f710c06f1e0 2024-09-10T09:58:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f7113000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f710c0b34d0 con 0x7f710c15c560 2024-09-10T09:58:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f7113000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f710c0b4570 con 0x7f710c155970 2024-09-10T09:58:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f710be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f710c155970 0x7f710c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f710be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f710c155970 0x7f710c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46590/0 (socket says 172.21.15.110:46590) 2024-09-10T09:58:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f710be00640 1 -- 172.21.15.110:0/31097 learned_addr learned my addr 172.21.15.110:0/31097 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f710be00640 1 -- 172.21.15.110:0/31097 >> v1:172.21.15.145:6789/0 conn(0x7f710c15c560 legacy=0x7f710c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f710be00640 1 -- 172.21.15.110:0/31097 >> v1:172.21.15.110:6789/0 conn(0x7f710c06f1e0 legacy=0x7f710c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f710be00640 1 -- 172.21.15.110:0/31097 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c0b37b0 con 0x7f710c155970 2024-09-10T09:58:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f710aa00640 1 -- 172.21.15.110:0/31097 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3628550756 0 0) 0x7f710c155d50 con 0x7f710c06f1e0 2024-09-10T09:58:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.397+0000 7f710be00640 1 --2- 172.21.15.110:0/31097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c155970 0x7f710c0cc8c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009870 tx=0x7f70fc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576d9e9e5ddc5598 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f710aa00640 1 -- 172.21.15.110:0/31097 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0a95f0 con 0x7f710c155970 2024-09-10T09:58:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f710aa00640 1 -- 172.21.15.110:0/31097 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70fc0a9790 con 0x7f710c155970 2024-09-10T09:58:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f710aa00640 1 -- 172.21.15.110:0/31097 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0a9b60 con 0x7f710c155970 2024-09-10T09:58:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- 172.21.15.110:0/31097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c155970 msgr2=0x7f710c0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 --2- 172.21.15.110:0/31097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c155970 0x7f710c0cc8c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009870 tx=0x7f70fc01c530 comp rx=0 tx=0).stop 2024-09-10T09:58:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- 172.21.15.110:0/31097 shutdown_connections 2024-09-10T09:58:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 --2- 172.21.15.110:0/31097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c155970 0x7f710c0cc8c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- 172.21.15.110:0/31097 >> 172.21.15.110:0/31097 conn(0x7f710c06bc50 msgr2=0x7f710c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- 172.21.15.110:0/31097 shutdown_connections 2024-09-10T09:58:27.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- 172.21.15.110:0/31097 wait complete. 2024-09-10T09:58:27.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 Processor -- start 2024-09-10T09:58:27.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- start start 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f710c06f1e0 0x7f710c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f710c155970 0x7f710c14dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c15c560 0x7f710c15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f710c0cf340 con 0x7f710c15c560 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f710c0cce00 con 0x7f710c06f1e0 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7113000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f710c0cf4c0 con 0x7f710c155970 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7111000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c15c560 0x7f710c15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7111000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c15c560 0x7f710c15a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46598/0 (socket says 172.21.15.110:46598) 2024-09-10T09:58:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.401+0000 7f7111000640 1 -- 172.21.15.110:0/506452048 learned_addr learned my addr 172.21.15.110:0/506452048 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f710be00640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f710c06f1e0 0x7f710c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f710b400640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f710c155970 0x7f710c14dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7111000640 1 -- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f710c155970 msgr2=0x7f710c14dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7111000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f710c155970 0x7f710c14dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7111000640 1 -- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f710c06f1e0 msgr2=0x7f710c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7111000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f710c06f1e0 0x7f710c151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7111000640 1 -- 172.21.15.110:0/506452048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c151120 con 0x7f710c15c560 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f710be00640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f710c06f1e0 0x7f710c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f710b400640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f710c155970 0x7f710c14dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7111000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c15c560 0x7f710c15a520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f710400c780 tx=0x7f710400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7103e00640 1 -- 172.21.15.110:0/506452048 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104003db0 con 0x7f710c15c560 2024-09-10T09:58:27.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7103e00640 1 -- 172.21.15.110:0/506452048 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7104005070 con 0x7f710c15c560 2024-09-10T09:58:27.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7103e00640 1 -- 172.21.15.110:0/506452048 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104005460 con 0x7f710c15c560 2024-09-10T09:58:27.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c151400 con 0x7f710c15c560 2024-09-10T09:58:27.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c14e1e0 con 0x7f710c15c560 2024-09-10T09:58:27.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710c0cc8c0 con 0x7f710c15c560 2024-09-10T09:58:27.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7103e00640 1 -- 172.21.15.110:0/506452048 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7104005600 con 0x7f710c15c560 2024-09-10T09:58:27.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7103e00640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70ec0678c0 0x7f70ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f7103e00640 1 -- 172.21.15.110:0/506452048 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7104085bf0 con 0x7f710c15c560 2024-09-10T09:58:27.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.405+0000 7f710be00640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70ec0678c0 0x7f70ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.409+0000 7f710be00640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70ec0678c0 0x7f70ec069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0ae000 tx=0x7f70fc0b3040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.413+0000 7f7103e00640 1 -- 172.21.15.110:0/506452048 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f71040567c0 con 0x7f710c15c560 2024-09-10T09:58:27.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.565+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f710c06c310 con 0x7f70ec0678c0 2024-09-10T09:58:27.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.569+0000 7f7103e00640 1 -- 172.21.15.110:0/506452048 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247805 (secure 0 0 0) 0x7f710c06c310 con 0x7f70ec0678c0 2024-09-10T09:58:27.573 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:27.575 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:58:27.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70ec0678c0 msgr2=0x7f70ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70ec0678c0 0x7f70ec069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0ae000 tx=0x7f70fc0b3040 comp rx=0 tx=0).stop 2024-09-10T09:58:27.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c15c560 msgr2=0x7f710c15a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c15c560 0x7f710c15a520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f710400c780 tx=0x7f710400cc40 comp rx=0 tx=0).stop 2024-09-10T09:58:27.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 shutdown_connections 2024-09-10T09:58:27.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70ec0678c0 0x7f70ec069d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f710c15c560 0x7f710c15a520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f710c155970 0x7f710c14dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 --2- 172.21.15.110:0/506452048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f710c06f1e0 0x7f710c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 >> 172.21.15.110:0/506452048 conn(0x7f710c06bc50 msgr2=0x7f710c0cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 shutdown_connections 2024-09-10T09:58:27.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.573+0000 7f7113000640 1 -- 172.21.15.110:0/506452048 wait complete. 2024-09-10T09:58:27.593 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T09:58:27.356156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235716,"kb_used_data":2828,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739004,"statfs":{"total":773094113280,"available":772852740096,"internally_reserved":0,"allocated":2895872,"data_stored":1738463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001171"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.773483+0000","last_change":"2024-09-10T09:58:21.773483+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.773483+0000","last_undegraded":"2024-09-10T09:58:21.773483+0000","last_fullsized":"2024-09-10T09:58:21.773483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758695+0000","last_change":"2024-09-10T09:58:22.758695+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758695+0000","last_undegraded":"2024-09-10T09:58:22.758695+0000","last_fullsized":"2024-09-10T09:58:22.758695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770858+0000","last_change":"2024-09-10T09:58:21.770858+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770858+0000","last_undegraded":"2024-09-10T09:58:21.770858+0000","last_fullsized":"2024-09-10T09:58:21.770858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761137+0000","last_change":"2024-09-10T09:58:22.761137+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761137+0000","last_undegraded":"2024-09-10T09:58:22.761137+0000","last_fullsized":"2024-09-10T09:58:22.761137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770682+0000","last_change":"2024-09-10T09:58:21.770682+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770682+0000","last_undegraded":"2024-09-10T09:58:21.770682+0000","last_fullsized":"2024-09-10T09:58:21.770682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762312+0000","last_change":"2024-09-10T09:58:22.762312+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762312+0000","last_undegraded":"2024-09-10T09:58:22.762312+0000","last_fullsized":"2024-09-10T09:58:22.762312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.780164+0000","last_change":"2024-09-10T09:58:21.780164+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.780164+0000","last_undegraded":"2024-09-10T09:58:21.780164+0000","last_fullsized":"2024-09-10T09:58:21.780164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737654+0000","last_change":"2024-09-10T09:58:22.790696+0000","last_active":"2024-09-10T09:58:24.737654+0000","last_peered":"2024-09-10T09:58:24.737654+0000","last_clean":"2024-09-10T09:58:24.737654+0000","last_became_active":"2024-09-10T09:58:22.790265+0000","last_became_peered":"2024-09-10T09:58:22.790265+0000","last_unstale":"2024-09-10T09:58:24.737654+0000","last_undegraded":"2024-09-10T09:58:24.737654+0000","last_fullsized":"2024-09-10T09:58:24.737654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:20.430762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772032+0000","last_change":"2024-09-10T09:58:21.772032+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772032+0000","last_undegraded":"2024-09-10T09:58:21.772032+0000","last_fullsized":"2024-09-10T09:58:21.772032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760582+0000","last_change":"2024-09-10T09:58:22.760582+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760582+0000","last_undegraded":"2024-09-10T09:58:22.760582+0000","last_fullsized":"2024-09-10T09:58:22.760582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762468+0000","last_change":"2024-09-10T09:58:22.762468+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762468+0000","last_undegraded":"2024-09-10T09:58:22.762468+0000","last_fullsized":"2024-09-10T09:58:22.762468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767842+0000","last_change":"2024-09-10T09:58:21.767842+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767842+0000","last_undegraded":"2024-09-10T09:58:21.767842+0000","last_fullsized":"2024-09-10T09:58:21.767842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738573+0000","last_change":"2024-09-10T09:58:22.768744+0000","last_active":"2024-09-10T09:58:24.738573+0000","last_peered":"2024-09-10T09:58:24.738573+0000","last_clean":"2024-09-10T09:58:24.738573+0000","last_became_active":"2024-09-10T09:58:22.768515+0000","last_became_peered":"2024-09-10T09:58:22.768515+0000","last_unstale":"2024-09-10T09:58:24.738573+0000","last_undegraded":"2024-09-10T09:58:24.738573+0000","last_fullsized":"2024-09-10T09:58:24.738573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:37.491552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.765480+0000","last_change":"2024-09-10T09:58:21.765480+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.765480+0000","last_undegraded":"2024-09-10T09:58:21.765480+0000","last_fullsized":"2024-09-10T09:58:21.765480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758765+0000","last_change":"2024-09-10T09:58:22.758765+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758765+0000","last_undegraded":"2024-09-10T09:58:22.758765+0000","last_fullsized":"2024-09-10T09:58:22.758765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.764622+0000","last_change":"2024-09-10T09:58:22.764622+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.764622+0000","last_undegraded":"2024-09-10T09:58:22.764622+0000","last_fullsized":"2024-09-10T09:58:22.764622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760834+0000","last_change":"2024-09-10T09:58:22.760834+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760834+0000","last_undegraded":"2024-09-10T09:58:22.760834+0000","last_fullsized":"2024-09-10T09:58:22.760834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.762859+0000","last_change":"2024-09-10T09:58:21.762859+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.762859+0000","last_undegraded":"2024-09-10T09:58:21.762859+0000","last_fullsized":"2024-09-10T09:58:21.762859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767720+0000","last_change":"2024-09-10T09:58:21.767720+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767720+0000","last_undegraded":"2024-09-10T09:58:21.767720+0000","last_fullsized":"2024-09-10T09:58:21.767720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.763497+0000","last_change":"2024-09-10T09:58:22.763497+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.763497+0000","last_undegraded":"2024-09-10T09:58:22.763497+0000","last_fullsized":"2024-09-10T09:58:22.763497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.765123+0000","last_change":"2024-09-10T09:58:21.765123+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.765123+0000","last_undegraded":"2024-09-10T09:58:21.765123+0000","last_fullsized":"2024-09-10T09:58:21.765123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762017+0000","last_change":"2024-09-10T09:58:22.762017+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762017+0000","last_undegraded":"2024-09-10T09:58:22.762017+0000","last_fullsized":"2024-09-10T09:58:22.762017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738073+0000","last_change":"2024-09-10T09:58:22.799342+0000","last_active":"2024-09-10T09:58:24.738073+0000","last_peered":"2024-09-10T09:58:24.738073+0000","last_clean":"2024-09-10T09:58:24.738073+0000","last_became_active":"2024-09-10T09:58:22.799049+0000","last_became_peered":"2024-09-10T09:58:22.799049+0000","last_unstale":"2024-09-10T09:58:24.738073+0000","last_undegraded":"2024-09-10T09:58:24.738073+0000","last_fullsized":"2024-09-10T09:58:24.738073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:24.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.764229+0000","last_change":"2024-09-10T09:58:22.764229+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.764229+0000","last_undegraded":"2024-09-10T09:58:22.764229+0000","last_fullsized":"2024-09-10T09:58:22.764229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737548+0000","last_change":"2024-09-10T09:58:22.789676+0000","last_active":"2024-09-10T09:58:24.737548+0000","last_peered":"2024-09-10T09:58:24.737548+0000","last_clean":"2024-09-10T09:58:24.737548+0000","last_became_active":"2024-09-10T09:58:22.789249+0000","last_became_peered":"2024-09-10T09:58:22.789249+0000","last_unstale":"2024-09-10T09:58:24.737548+0000","last_undegraded":"2024-09-10T09:58:24.737548+0000","last_fullsized":"2024-09-10T09:58:24.737548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:03.582635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.769476+0000","last_change":"2024-09-10T09:58:22.769476+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.769476+0000","last_undegraded":"2024-09-10T09:58:22.769476+0000","last_fullsized":"2024-09-10T09:58:22.769476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.759733+0000","last_change":"2024-09-10T09:58:22.759733+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.759733+0000","last_undegraded":"2024-09-10T09:58:22.759733+0000","last_fullsized":"2024-09-10T09:58:22.759733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738554+0000","last_change":"2024-09-10T09:58:22.796593+0000","last_active":"2024-09-10T09:58:24.738554+0000","last_peered":"2024-09-10T09:58:24.738554+0000","last_clean":"2024-09-10T09:58:24.738554+0000","last_became_active":"2024-09-10T09:58:22.796294+0000","last_became_peered":"2024-09-10T09:58:22.796294+0000","last_unstale":"2024-09-10T09:58:24.738554+0000","last_undegraded":"2024-09-10T09:58:24.738554+0000","last_fullsized":"2024-09-10T09:58:24.738554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:43.026541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.764586+0000","last_change":"2024-09-10T09:58:21.764586+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.764586+0000","last_undegraded":"2024-09-10T09:58:21.764586+0000","last_fullsized":"2024-09-10T09:58:21.764586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761625+0000","last_change":"2024-09-10T09:58:22.761625+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761625+0000","last_undegraded":"2024-09-10T09:58:22.761625+0000","last_fullsized":"2024-09-10T09:58:22.761625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.762149+0000","last_change":"2024-09-10T09:58:21.762149+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.762149+0000","last_undegraded":"2024-09-10T09:58:21.762149+0000","last_fullsized":"2024-09-10T09:58:21.762149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760420+0000","last_change":"2024-09-10T09:58:22.760420+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760420+0000","last_undegraded":"2024-09-10T09:58:22.760420+0000","last_fullsized":"2024-09-10T09:58:22.760420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.768406+0000","last_change":"2024-09-10T09:58:21.768406+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.768406+0000","last_undegraded":"2024-09-10T09:58:21.768406+0000","last_fullsized":"2024-09-10T09:58:21.768406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738624+0000","last_change":"2024-09-10T09:58:21.783061+0000","last_active":"2024-09-10T09:58:24.738624+0000","last_peered":"2024-09-10T09:58:24.738624+0000","last_clean":"2024-09-10T09:58:24.738624+0000","last_became_active":"2024-09-10T09:58:21.782756+0000","last_became_peered":"2024-09-10T09:58:21.782756+0000","last_unstale":"2024-09-10T09:58:24.738624+0000","last_undegraded":"2024-09-10T09:58:24.738624+0000","last_fullsized":"2024-09-10T09:58:24.738624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:39.702619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.764021+0000","last_change":"2024-09-10T09:58:21.764021+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.764021+0000","last_undegraded":"2024-09-10T09:58:21.764021+0000","last_fullsized":"2024-09-10T09:58:21.764021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.765118+0000","last_change":"2024-09-10T09:58:21.778244+0000","last_active":"2024-09-10T09:58:22.765118+0000","last_peered":"2024-09-10T09:58:22.765118+0000","last_clean":"2024-09-10T09:58:22.765118+0000","last_became_active":"2024-09-10T09:58:21.777937+0000","last_became_peered":"2024-09-10T09:58:21.777937+0000","last_unstale":"2024-09-10T09:58:22.765118+0000","last_undegraded":"2024-09-10T09:58:22.765118+0000","last_fullsized":"2024-09-10T09:58:22.765118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738137+0000","last_change":"2024-09-10T09:58:21.775701+0000","last_active":"2024-09-10T09:58:24.738137+0000","last_peered":"2024-09-10T09:58:24.738137+0000","last_clean":"2024-09-10T09:58:24.738137+0000","last_became_active":"2024-09-10T09:58:21.775431+0000","last_became_peered":"2024-09-10T09:58:21.775431+0000","last_unstale":"2024-09-10T09:58:24.738137+0000","last_undegraded":"2024-09-10T09:58:24.738137+0000","last_fullsized":"2024-09-10T09:58:24.738137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:03.845811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737371+0000","last_change":"2024-09-10T09:58:22.772570+0000","last_active":"2024-09-10T09:58:24.737371+0000","last_peered":"2024-09-10T09:58:24.737371+0000","last_clean":"2024-09-10T09:58:24.737371+0000","last_became_active":"2024-09-10T09:58:22.772248+0000","last_became_peered":"2024-09-10T09:58:22.772248+0000","last_unstale":"2024-09-10T09:58:24.737371+0000","last_undegraded":"2024-09-10T09:58:24.737371+0000","last_fullsized":"2024-09-10T09:58:24.737371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:55.062755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.765584+0000","last_change":"2024-09-10T09:58:21.786530+0000","last_active":"2024-09-10T09:58:22.765584+0000","last_peered":"2024-09-10T09:58:22.765584+0000","last_clean":"2024-09-10T09:58:22.765584+0000","last_became_active":"2024-09-10T09:58:21.786157+0000","last_became_peered":"2024-09-10T09:58:21.786157+0000","last_unstale":"2024-09-10T09:58:22.765584+0000","last_undegraded":"2024-09-10T09:58:22.765584+0000","last_fullsized":"2024-09-10T09:58:22.765584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761036+0000","last_change":"2024-09-10T09:58:22.761036+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761036+0000","last_undegraded":"2024-09-10T09:58:22.761036+0000","last_fullsized":"2024-09-10T09:58:22.761036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761809+0000","last_change":"2024-09-10T09:58:21.786956+0000","last_active":"2024-09-10T09:58:22.761809+0000","last_peered":"2024-09-10T09:58:22.761809+0000","last_clean":"2024-09-10T09:58:22.761809+0000","last_became_active":"2024-09-10T09:58:21.786523+0000","last_became_peered":"2024-09-10T09:58:21.786523+0000","last_unstale":"2024-09-10T09:58:22.761809+0000","last_undegraded":"2024-09-10T09:58:22.761809+0000","last_fullsized":"2024-09-10T09:58:22.761809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761458+0000","last_change":"2024-09-10T09:58:22.761458+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761458+0000","last_undegraded":"2024-09-10T09:58:22.761458+0000","last_fullsized":"2024-09-10T09:58:22.761458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761846+0000","last_change":"2024-09-10T09:58:21.787382+0000","last_active":"2024-09-10T09:58:22.761846+0000","last_peered":"2024-09-10T09:58:22.761846+0000","last_clean":"2024-09-10T09:58:22.761846+0000","last_became_active":"2024-09-10T09:58:21.787048+0000","last_became_peered":"2024-09-10T09:58:21.787048+0000","last_unstale":"2024-09-10T09:58:22.761846+0000","last_undegraded":"2024-09-10T09:58:22.761846+0000","last_fullsized":"2024-09-10T09:58:22.761846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772636+0000","last_change":"2024-09-10T09:58:21.772636+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772636+0000","last_undegraded":"2024-09-10T09:58:21.772636+0000","last_fullsized":"2024-09-10T09:58:21.772636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759611+0000","last_change":"2024-09-10T09:58:21.785126+0000","last_active":"2024-09-10T09:58:22.759611+0000","last_peered":"2024-09-10T09:58:22.759611+0000","last_clean":"2024-09-10T09:58:22.759611+0000","last_became_active":"2024-09-10T09:58:21.784839+0000","last_became_peered":"2024-09-10T09:58:21.784839+0000","last_unstale":"2024-09-10T09:58:22.759611+0000","last_undegraded":"2024-09-10T09:58:22.759611+0000","last_fullsized":"2024-09-10T09:58:22.759611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767249+0000","last_change":"2024-09-10T09:58:21.767249+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767249+0000","last_undegraded":"2024-09-10T09:58:21.767249+0000","last_fullsized":"2024-09-10T09:58:21.767249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.763857+0000","last_change":"2024-09-10T09:58:21.776340+0000","last_active":"2024-09-10T09:58:22.763857+0000","last_peered":"2024-09-10T09:58:22.763857+0000","last_clean":"2024-09-10T09:58:22.763857+0000","last_became_active":"2024-09-10T09:58:21.776066+0000","last_became_peered":"2024-09-10T09:58:21.776066+0000","last_unstale":"2024-09-10T09:58:22.763857+0000","last_undegraded":"2024-09-10T09:58:22.763857+0000","last_fullsized":"2024-09-10T09:58:22.763857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772836+0000","last_change":"2024-09-10T09:58:21.772836+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772836+0000","last_undegraded":"2024-09-10T09:58:21.772836+0000","last_fullsized":"2024-09-10T09:58:21.772836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.785464+0000","last_change":"2024-09-10T09:58:21.783558+0000","last_active":"2024-09-10T09:58:21.785464+0000","last_peered":"2024-09-10T09:58:21.785464+0000","last_clean":"2024-09-10T09:58:21.785464+0000","last_became_active":"2024-09-10T09:58:21.782455+0000","last_became_peered":"2024-09-10T09:58:21.782455+0000","last_unstale":"2024-09-10T09:58:21.785464+0000","last_undegraded":"2024-09-10T09:58:21.785464+0000","last_fullsized":"2024-09-10T09:58:21.785464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:26.367199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.763413+0000","last_change":"2024-09-10T09:58:21.763413+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.763413+0000","last_undegraded":"2024-09-10T09:58:21.763413+0000","last_fullsized":"2024-09-10T09:58:21.763413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762503+0000","last_change":"2024-09-10T09:58:21.789974+0000","last_active":"2024-09-10T09:58:22.762503+0000","last_peered":"2024-09-10T09:58:22.762503+0000","last_clean":"2024-09-10T09:58:22.762503+0000","last_became_active":"2024-09-10T09:58:21.788481+0000","last_became_peered":"2024-09-10T09:58:21.788481+0000","last_unstale":"2024-09-10T09:58:22.762503+0000","last_undegraded":"2024-09-10T09:58:22.762503+0000","last_fullsized":"2024-09-10T09:58:22.762503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769168+0000","last_change":"2024-09-10T09:58:21.769168+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769168+0000","last_undegraded":"2024-09-10T09:58:21.769168+0000","last_fullsized":"2024-09-10T09:58:21.769168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.760516+0000","last_change":"2024-09-10T09:58:21.778738+0000","last_active":"2024-09-10T09:58:22.760516+0000","last_peered":"2024-09-10T09:58:22.760516+0000","last_clean":"2024-09-10T09:58:22.760516+0000","last_became_active":"2024-09-10T09:58:21.777982+0000","last_became_peered":"2024-09-10T09:58:21.777982+0000","last_unstale":"2024-09-10T09:58:22.760516+0000","last_undegraded":"2024-09-10T09:58:22.760516+0000","last_fullsized":"2024-09-10T09:58:22.760516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761175+0000","last_change":"2024-09-10T09:58:22.761175+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761175+0000","last_undegraded":"2024-09-10T09:58:22.761175+0000","last_fullsized":"2024-09-10T09:58:22.761175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.782949+0000","last_change":"2024-09-10T09:58:21.779750+0000","last_active":"2024-09-10T09:58:21.782949+0000","last_peered":"2024-09-10T09:58:21.782949+0000","last_clean":"2024-09-10T09:58:21.782949+0000","last_became_active":"2024-09-10T09:58:21.779492+0000","last_became_peered":"2024-09-10T09:58:21.779492+0000","last_unstale":"2024-09-10T09:58:21.782949+0000","last_undegraded":"2024-09-10T09:58:21.782949+0000","last_fullsized":"2024-09-10T09:58:21.782949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:26.367199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772393+0000","last_change":"2024-09-10T09:58:21.772393+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772393+0000","last_undegraded":"2024-09-10T09:58:21.772393+0000","last_fullsized":"2024-09-10T09:58:21.772393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.789668+0000","last_change":"2024-09-10T09:58:21.781792+0000","last_active":"2024-09-10T09:58:21.789668+0000","last_peered":"2024-09-10T09:58:21.789668+0000","last_clean":"2024-09-10T09:58:21.789668+0000","last_became_active":"2024-09-10T09:58:21.781434+0000","last_became_peered":"2024-09-10T09:58:21.781434+0000","last_unstale":"2024-09-10T09:58:21.789668+0000","last_undegraded":"2024-09-10T09:58:21.789668+0000","last_fullsized":"2024-09-10T09:58:21.789668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:42.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767018+0000","last_change":"2024-09-10T09:58:21.767018+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767018+0000","last_undegraded":"2024-09-10T09:58:21.767018+0000","last_fullsized":"2024-09-10T09:58:21.767018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.783020+0000","last_change":"2024-09-10T09:58:21.781287+0000","last_active":"2024-09-10T09:58:21.783020+0000","last_peered":"2024-09-10T09:58:21.783020+0000","last_clean":"2024-09-10T09:58:21.783020+0000","last_became_active":"2024-09-10T09:58:21.780824+0000","last_became_peered":"2024-09-10T09:58:21.780824+0000","last_unstale":"2024-09-10T09:58:21.783020+0000","last_undegraded":"2024-09-10T09:58:21.783020+0000","last_fullsized":"2024-09-10T09:58:21.783020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:11.144974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769098+0000","last_change":"2024-09-10T09:58:21.769098+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769098+0000","last_undegraded":"2024-09-10T09:58:21.769098+0000","last_fullsized":"2024-09-10T09:58:21.769098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759829+0000","last_change":"2024-09-10T09:58:21.776106+0000","last_active":"2024-09-10T09:58:22.759829+0000","last_peered":"2024-09-10T09:58:22.759829+0000","last_clean":"2024-09-10T09:58:22.759829+0000","last_became_active":"2024-09-10T09:58:21.773315+0000","last_became_peered":"2024-09-10T09:58:21.773315+0000","last_unstale":"2024-09-10T09:58:22.759829+0000","last_undegraded":"2024-09-10T09:58:22.759829+0000","last_fullsized":"2024-09-10T09:58:22.759829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760733+0000","last_change":"2024-09-10T09:58:22.760733+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760733+0000","last_undegraded":"2024-09-10T09:58:22.760733+0000","last_fullsized":"2024-09-10T09:58:22.760733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.790976+0000","last_change":"2024-09-10T09:58:21.789986+0000","last_active":"2024-09-10T09:58:21.790976+0000","last_peered":"2024-09-10T09:58:21.790976+0000","last_clean":"2024-09-10T09:58:21.790976+0000","last_became_active":"2024-09-10T09:58:21.789577+0000","last_became_peered":"2024-09-10T09:58:21.789577+0000","last_unstale":"2024-09-10T09:58:21.790976+0000","last_undegraded":"2024-09-10T09:58:21.790976+0000","last_fullsized":"2024-09-10T09:58:21.790976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.768360+0000","last_change":"2024-09-10T09:58:21.768360+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.768360+0000","last_undegraded":"2024-09-10T09:58:21.768360+0000","last_fullsized":"2024-09-10T09:58:21.768360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":166,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.783072+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:58:21.783072+0000","last_peered":"2024-09-10T09:58:21.783072+0000","last_clean":"2024-09-10T09:58:21.783072+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:58:21.783072+0000","last_undegraded":"2024-09-10T09:58:21.783072+0000","last_fullsized":"2024-09-10T09:58:21.783072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762945+0000","last_change":"2024-09-10T09:58:21.789884+0000","last_active":"2024-09-10T09:58:22.762945+0000","last_peered":"2024-09-10T09:58:22.762945+0000","last_clean":"2024-09-10T09:58:22.762945+0000","last_became_active":"2024-09-10T09:58:21.788940+0000","last_became_peered":"2024-09-10T09:58:21.788940+0000","last_unstale":"2024-09-10T09:58:22.762945+0000","last_undegraded":"2024-09-10T09:58:22.762945+0000","last_fullsized":"2024-09-10T09:58:22.762945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737853+0000","last_change":"2024-09-10T09:58:22.792386+0000","last_active":"2024-09-10T09:58:24.737853+0000","last_peered":"2024-09-10T09:58:24.737853+0000","last_clean":"2024-09-10T09:58:24.737853+0000","last_became_active":"2024-09-10T09:58:22.792151+0000","last_became_peered":"2024-09-10T09:58:22.792151+0000","last_unstale":"2024-09-10T09:58:24.737853+0000","last_undegraded":"2024-09-10T09:58:24.737853+0000","last_fullsized":"2024-09-10T09:58:24.737853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.810620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792384+0000","last_change":"2024-09-10T09:58:21.791063+0000","last_active":"2024-09-10T09:58:21.792384+0000","last_peered":"2024-09-10T09:58:21.792384+0000","last_clean":"2024-09-10T09:58:21.792384+0000","last_became_active":"2024-09-10T09:58:21.790796+0000","last_became_peered":"2024-09-10T09:58:21.790796+0000","last_unstale":"2024-09-10T09:58:21.792384+0000","last_undegraded":"2024-09-10T09:58:21.792384+0000","last_fullsized":"2024-09-10T09:58:21.792384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.761973+0000","last_change":"2024-09-10T09:58:21.761973+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.761973+0000","last_undegraded":"2024-09-10T09:58:21.761973+0000","last_fullsized":"2024-09-10T09:58:21.761973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.781486+0000","last_change":"2024-09-10T09:58:21.777208+0000","last_active":"2024-09-10T09:58:21.781486+0000","last_peered":"2024-09-10T09:58:21.781486+0000","last_clean":"2024-09-10T09:58:21.781486+0000","last_became_active":"2024-09-10T09:58:21.776820+0000","last_became_peered":"2024-09-10T09:58:21.776820+0000","last_unstale":"2024-09-10T09:58:21.781486+0000","last_undegraded":"2024-09-10T09:58:21.781486+0000","last_fullsized":"2024-09-10T09:58:21.781486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:42.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.759166+0000","last_change":"2024-09-10T09:58:22.759166+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.759166+0000","last_undegraded":"2024-09-10T09:58:22.759166+0000","last_fullsized":"2024-09-10T09:58:22.759166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762585+0000","last_change":"2024-09-10T09:58:21.785388+0000","last_active":"2024-09-10T09:58:22.762585+0000","last_peered":"2024-09-10T09:58:22.762585+0000","last_clean":"2024-09-10T09:58:22.762585+0000","last_became_active":"2024-09-10T09:58:21.785227+0000","last_became_peered":"2024-09-10T09:58:21.785227+0000","last_unstale":"2024-09-10T09:58:22.762585+0000","last_undegraded":"2024-09-10T09:58:22.762585+0000","last_fullsized":"2024-09-10T09:58:22.762585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770833+0000","last_change":"2024-09-10T09:58:21.770833+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770833+0000","last_undegraded":"2024-09-10T09:58:21.770833+0000","last_fullsized":"2024-09-10T09:58:21.770833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.758612+0000","last_change":"2024-09-10T09:58:21.768431+0000","last_active":"2024-09-10T09:58:22.758612+0000","last_peered":"2024-09-10T09:58:22.758612+0000","last_clean":"2024-09-10T09:58:22.758612+0000","last_became_active":"2024-09-10T09:58:21.767826+0000","last_became_peered":"2024-09-10T09:58:21.767826+0000","last_unstale":"2024-09-10T09:58:22.758612+0000","last_undegraded":"2024-09-10T09:58:22.758612+0000","last_fullsized":"2024-09-10T09:58:22.758612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761547+0000","last_change":"2024-09-10T09:58:22.761547+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761547+0000","last_undegraded":"2024-09-10T09:58:22.761547+0000","last_fullsized":"2024-09-10T09:58:22.761547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792043+0000","last_change":"2024-09-10T09:58:21.788480+0000","last_active":"2024-09-10T09:58:21.792043+0000","last_peered":"2024-09-10T09:58:21.792043+0000","last_clean":"2024-09-10T09:58:21.792043+0000","last_became_active":"2024-09-10T09:58:21.788162+0000","last_became_peered":"2024-09-10T09:58:21.788162+0000","last_unstale":"2024-09-10T09:58:21.792043+0000","last_undegraded":"2024-09-10T09:58:21.792043+0000","last_fullsized":"2024-09-10T09:58:21.792043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:05.666246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758668+0000","last_change":"2024-09-10T09:58:22.758668+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758668+0000","last_undegraded":"2024-09-10T09:58:22.758668+0000","last_fullsized":"2024-09-10T09:58:22.758668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.758990+0000","last_change":"2024-09-10T09:58:21.770640+0000","last_active":"2024-09-10T09:58:22.758990+0000","last_peered":"2024-09-10T09:58:22.758990+0000","last_clean":"2024-09-10T09:58:22.758990+0000","last_became_active":"2024-09-10T09:58:21.769111+0000","last_became_peered":"2024-09-10T09:58:21.769111+0000","last_unstale":"2024-09-10T09:58:22.758990+0000","last_undegraded":"2024-09-10T09:58:22.758990+0000","last_fullsized":"2024-09-10T09:58:22.758990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738499+0000","last_change":"2024-09-10T09:58:22.789520+0000","last_active":"2024-09-10T09:58:24.738499+0000","last_peered":"2024-09-10T09:58:24.738499+0000","last_clean":"2024-09-10T09:58:24.738499+0000","last_became_active":"2024-09-10T09:58:22.788875+0000","last_became_peered":"2024-09-10T09:58:22.788875+0000","last_unstale":"2024-09-10T09:58:24.738499+0000","last_undegraded":"2024-09-10T09:58:24.738499+0000","last_fullsized":"2024-09-10T09:58:24.738499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:00.959927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761978+0000","last_change":"2024-09-10T09:58:21.782019+0000","last_active":"2024-09-10T09:58:22.761978+0000","last_peered":"2024-09-10T09:58:22.761978+0000","last_clean":"2024-09-10T09:58:22.761978+0000","last_became_active":"2024-09-10T09:58:21.781750+0000","last_became_peered":"2024-09-10T09:58:21.781750+0000","last_unstale":"2024-09-10T09:58:22.761978+0000","last_undegraded":"2024-09-10T09:58:22.761978+0000","last_fullsized":"2024-09-10T09:58:22.761978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770772+0000","last_change":"2024-09-10T09:58:21.770772+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770772+0000","last_undegraded":"2024-09-10T09:58:21.770772+0000","last_fullsized":"2024-09-10T09:58:21.770772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.791839+0000","last_change":"2024-09-10T09:58:21.788078+0000","last_active":"2024-09-10T09:58:21.791839+0000","last_peered":"2024-09-10T09:58:21.791839+0000","last_clean":"2024-09-10T09:58:21.791839+0000","last_became_active":"2024-09-10T09:58:21.787660+0000","last_became_peered":"2024-09-10T09:58:21.787660+0000","last_unstale":"2024-09-10T09:58:21.791839+0000","last_undegraded":"2024-09-10T09:58:21.791839+0000","last_fullsized":"2024-09-10T09:58:21.791839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769618+0000","last_change":"2024-09-10T09:58:21.769618+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769618+0000","last_undegraded":"2024-09-10T09:58:21.769618+0000","last_fullsized":"2024-09-10T09:58:21.769618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759017+0000","last_change":"2024-09-10T09:58:21.782043+0000","last_active":"2024-09-10T09:58:22.759017+0000","last_peered":"2024-09-10T09:58:22.759017+0000","last_clean":"2024-09-10T09:58:22.759017+0000","last_became_active":"2024-09-10T09:58:21.781762+0000","last_became_peered":"2024-09-10T09:58:21.781762+0000","last_unstale":"2024-09-10T09:58:22.759017+0000","last_undegraded":"2024-09-10T09:58:22.759017+0000","last_fullsized":"2024-09-10T09:58:22.759017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738162+0000","last_change":"2024-09-10T09:58:22.772755+0000","last_active":"2024-09-10T09:58:24.738162+0000","last_peered":"2024-09-10T09:58:24.738162+0000","last_clean":"2024-09-10T09:58:24.738162+0000","last_became_active":"2024-09-10T09:58:22.772505+0000","last_became_peered":"2024-09-10T09:58:22.772505+0000","last_unstale":"2024-09-10T09:58:24.738162+0000","last_undegraded":"2024-09-10T09:58:24.738162+0000","last_fullsized":"2024-09-10T09:58:24.738162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:48.318092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759075+0000","last_change":"2024-09-10T09:58:21.774362+0000","last_active":"2024-09-10T09:58:22.759075+0000","last_peered":"2024-09-10T09:58:22.759075+0000","last_clean":"2024-09-10T09:58:22.759075+0000","last_became_active":"2024-09-10T09:58:21.773133+0000","last_became_peered":"2024-09-10T09:58:21.773133+0000","last_unstale":"2024-09-10T09:58:22.759075+0000","last_undegraded":"2024-09-10T09:58:22.759075+0000","last_fullsized":"2024-09-10T09:58:22.759075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738587+0000","last_change":"2024-09-10T09:58:22.791719+0000","last_active":"2024-09-10T09:58:24.738587+0000","last_peered":"2024-09-10T09:58:24.738587+0000","last_clean":"2024-09-10T09:58:24.738587+0000","last_became_active":"2024-09-10T09:58:22.790615+0000","last_became_peered":"2024-09-10T09:58:22.790615+0000","last_unstale":"2024-09-10T09:58:24.738587+0000","last_undegraded":"2024-09-10T09:58:24.738587+0000","last_fullsized":"2024-09-10T09:58:24.738587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:59.165906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792397+0000","last_change":"2024-09-10T09:58:21.790948+0000","last_active":"2024-09-10T09:58:21.792397+0000","last_peered":"2024-09-10T09:58:21.792397+0000","last_clean":"2024-09-10T09:58:21.792397+0000","last_became_active":"2024-09-10T09:58:21.790543+0000","last_became_peered":"2024-09-10T09:58:21.790543+0000","last_unstale":"2024-09-10T09:58:21.792397+0000","last_undegraded":"2024-09-10T09:58:21.792397+0000","last_fullsized":"2024-09-10T09:58:21.792397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760512+0000","last_change":"2024-09-10T09:58:22.760512+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760512+0000","last_undegraded":"2024-09-10T09:58:22.760512+0000","last_fullsized":"2024-09-10T09:58:22.760512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761985+0000","last_change":"2024-09-10T09:58:22.761985+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761985+0000","last_undegraded":"2024-09-10T09:58:22.761985+0000","last_fullsized":"2024-09-10T09:58:22.761985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.794737+0000","last_change":"2024-09-10T09:58:21.793454+0000","last_active":"2024-09-10T09:58:21.794737+0000","last_peered":"2024-09-10T09:58:21.794737+0000","last_clean":"2024-09-10T09:58:21.794737+0000","last_became_active":"2024-09-10T09:58:21.793058+0000","last_became_peered":"2024-09-10T09:58:21.793058+0000","last_unstale":"2024-09-10T09:58:21.794737+0000","last_undegraded":"2024-09-10T09:58:21.794737+0000","last_fullsized":"2024-09-10T09:58:21.794737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760886+0000","last_change":"2024-09-10T09:58:22.760886+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760886+0000","last_undegraded":"2024-09-10T09:58:22.760886+0000","last_fullsized":"2024-09-10T09:58:22.760886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792296+0000","last_change":"2024-09-10T09:58:21.789114+0000","last_active":"2024-09-10T09:58:21.792296+0000","last_peered":"2024-09-10T09:58:21.792296+0000","last_clean":"2024-09-10T09:58:21.792296+0000","last_became_active":"2024-09-10T09:58:21.788849+0000","last_became_peered":"2024-09-10T09:58:21.788849+0000","last_unstale":"2024-09-10T09:58:21.792296+0000","last_undegraded":"2024-09-10T09:58:21.792296+0000","last_fullsized":"2024-09-10T09:58:21.792296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:00.039731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.773428+0000","last_change":"2024-09-10T09:58:21.773428+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.773428+0000","last_undegraded":"2024-09-10T09:58:21.773428+0000","last_fullsized":"2024-09-10T09:58:21.773428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738067+0000","last_change":"2024-09-10T09:58:21.779788+0000","last_active":"2024-09-10T09:58:24.738067+0000","last_peered":"2024-09-10T09:58:24.738067+0000","last_clean":"2024-09-10T09:58:24.738067+0000","last_became_active":"2024-09-10T09:58:21.779521+0000","last_became_peered":"2024-09-10T09:58:21.779521+0000","last_unstale":"2024-09-10T09:58:24.738067+0000","last_undegraded":"2024-09-10T09:58:24.738067+0000","last_fullsized":"2024-09-10T09:58:24.738067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:33.355227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737993+0000","last_change":"2024-09-10T09:58:22.777347+0000","last_active":"2024-09-10T09:58:24.737993+0000","last_peered":"2024-09-10T09:58:24.737993+0000","last_clean":"2024-09-10T09:58:24.737993+0000","last_became_active":"2024-09-10T09:58:22.777042+0000","last_became_peered":"2024-09-10T09:58:22.777042+0000","last_unstale":"2024-09-10T09:58:24.737993+0000","last_undegraded":"2024-09-10T09:58:24.737993+0000","last_fullsized":"2024-09-10T09:58:24.737993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:31.068726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.788781+0000","last_change":"2024-09-10T09:58:21.787400+0000","last_active":"2024-09-10T09:58:21.788781+0000","last_peered":"2024-09-10T09:58:21.788781+0000","last_clean":"2024-09-10T09:58:21.788781+0000","last_became_active":"2024-09-10T09:58:21.787111+0000","last_became_peered":"2024-09-10T09:58:21.787111+0000","last_unstale":"2024-09-10T09:58:21.788781+0000","last_undegraded":"2024-09-10T09:58:21.788781+0000","last_fullsized":"2024-09-10T09:58:21.788781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:11.144974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":802816,"data_stored":658248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509443,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":658248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:58:27.596 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.773483+0000', 'last_change': '2024-09-10T09:58:21.773483+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.773483+0000', 'last_undegraded': '2024-09-10T09:58:21.773483+0000', 'last_fullsized': '2024-09-10T09:58:21.773483+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.758695+0000', 'last_change': '2024-09-10T09:58:22.758695+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.758695+0000', 'last_undegraded': '2024-09-10T09:58:22.758695+0000', 'last_fullsized': '2024-09-10T09:58:22.758695+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': ['1', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.770858+0000', 'last_change': '2024-09-10T09:58:21.770858+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.770858+0000', 'last_undegraded': '2024-09-10T09:58:21.770858+0000', 'last_fullsized': '2024-09-10T09:58:21.770858+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.761137+0000', 'last_change': '2024-09-10T09:58:22.761137+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.761137+0000', 'last_undegraded': '2024-09-10T09:58:22.761137+0000', 'last_fullsized': '2024-09-10T09:58:22.761137+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-10T09:58:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.770682+0000', 'last_change': '2024-09-10T09:58:21.770682+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.770682+0000', 'last_undegraded': '2024-09-10T09:58:21.770682+0000', 'last_fullsized': '2024-09-10T09:58:21.770682+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.762312+0000', 'last_change': '2024-09-10T09:58:22.762312+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.762312+0000', 'last_undegraded': '2024-09-10T09:58:22.762312+0000', 'last_fullsized': '2024-09-10T09:58:22.762312+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.780164+0000', 'last_change': '2024-09-10T09:58:21.780164+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.780164+0000', 'last_undegraded': '2024-09-10T09:58:21.780164+0000', 'last_fullsized': '2024-09-10T09:58:21.780164+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.772032+0000', 'last_change': '2024-09-10T09:58:21.772032+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.772032+0000', 'last_undegraded': '2024-09-10T09:58:21.772032+0000', 'last_fullsized': '2024-09-10T09:58:21.772032+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.760582+0000', 'last_change': '2024-09-10T09:58:22.760582+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.760582+0000', 'last_undegraded': '2024-09-10T09:58:22.760582+0000', 'last_fullsized': '2024-09-10T09:58:22.760582+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': ['6', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.762468+0000', 'last_change': '2024-09-10T09:58:22.762468+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.762468+0000', 'last_undegraded': '2024-09-10T09:58:22.762468+0000', 'last_fullsized': '2024-09-10T09:58:22.762468+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': ['6', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:58:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.767842+0000', 'last_change': '2024-09-10T09:58:21.767842+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.767842+0000', 'last_undegraded': '2024-09-10T09:58:21.767842+0000', 'last_fullsized': '2024-09-10T09:58:21.767842+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.765480+0000', 'last_change': '2024-09-10T09:58:21.765480+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.765480+0000', 'last_undegraded': '2024-09-10T09:58:21.765480+0000', 'last_fullsized': '2024-09-10T09:58:21.765480+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.758765+0000', 'last_change': '2024-09-10T09:58:22.758765+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.758765+0000', 'last_undegraded': '2024-09-10T09:58:22.758765+0000', 'last_fullsized': '2024-09-10T09:58:22.758765+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': ['1', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.764622+0000', 'last_change': '2024-09-10T09:58:22.764622+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.764622+0000', 'last_undegraded': '2024-09-10T09:58:22.764622+0000', 'last_fullsized': '2024-09-10T09:58:22.764622+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.760834+0000', 'last_change': '2024-09-10T09:58:22.760834+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.760834+0000', 'last_undegraded': '2024-09-10T09:58:22.760834+0000', 'last_fullsized': '2024-09-10T09:58:22.760834+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-10T09:58:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.762859+0000', 'last_change': '2024-09-10T09:58:21.762859+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.762859+0000', 'last_undegraded': '2024-09-10T09:58:21.762859+0000', 'last_fullsized': '2024-09-10T09:58:21.762859+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.767720+0000', 'last_change': '2024-09-10T09:58:21.767720+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.767720+0000', 'last_undegraded': '2024-09-10T09:58:21.767720+0000', 'last_fullsized': '2024-09-10T09:58:21.767720+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.763497+0000', 'last_change': '2024-09-10T09:58:22.763497+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.763497+0000', 'last_undegraded': '2024-09-10T09:58:22.763497+0000', 'last_fullsized': '2024-09-10T09:58:22.763497+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.765123+0000', 'last_change': '2024-09-10T09:58:21.765123+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.765123+0000', 'last_undegraded': '2024-09-10T09:58:21.765123+0000', 'last_fullsized': '2024-09-10T09:58:21.765123+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.762017+0000', 'last_change': '2024-09-10T09:58:22.762017+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.762017+0000', 'last_undegraded': '2024-09-10T09:58:22.762017+0000', 'last_fullsized': '2024-09-10T09:58:22.762017+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.764229+0000', 'last_change': '2024-09-10T09:58:22.764229+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.764229+0000', 'last_undegraded': '2024-09-10T09:58:22.764229+0000', 'last_fullsized': '2024-09-10T09:58:22.764229+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.769476+0000', 'last_change': '2024-09-10T09:58:22.769476+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.769476+0000', 'last_undegraded': '2024-09-10T09:58:22.769476+0000', 'last_fullsized': '2024-09-10T09:58:22.769476+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.759733+0000', 'last_change': '2024-09-10T09:58:22.759733+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.759733+0000', 'last_undegraded': '2024-09-10T09:58:22.759733+0000', 'last_fullsized': '2024-09-10T09:58:22.759733+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': ['1', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.764586+0000', 'last_change': '2024-09-10T09:58:21.764586+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.764586+0000', 'last_undegraded': '2024-09-10T09:58:21.764586+0000', 'last_fullsized': '2024-09-10T09:58:21.764586+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.761625+0000', 'last_change': '2024-09-10T09:58:22.761625+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.761625+0000', 'last_undegraded': '2024-09-10T09:58:22.761625+0000', 'last_fullsized': '2024-09-10T09:58:22.761625+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.762149+0000', 'last_change': '2024-09-10T09:58:21.762149+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.762149+0000', 'last_undegraded': '2024-09-10T09:58:21.762149+0000', 'last_fullsized': '2024-09-10T09:58:21.762149+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.760420+0000', 'last_change': '2024-09-10T09:58:22.760420+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.760420+0000', 'last_undegraded': '2024-09-10T09:58:22.760420+0000', 'last_fullsized': '2024-09-10T09:58:22.760420+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': ['6', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.768406+0000', 'last_change': '2024-09-10T09:58:21.768406+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.768406+0000', 'last_undegraded': '2024-09-10T09:58:21.768406+0000', 'last_fullsized': '2024-09-10T09:58:21.768406+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.764021+0000', 'last_change': '2024-09-10T09:58:21.764021+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.764021+0000', 'last_undegraded': '2024-09-10T09:58:21.764021+0000', 'last_fullsized': '2024-09-10T09:58:21.764021+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.761036+0000', 'last_change': '2024-09-10T09:58:22.761036+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.761036+0000', 'last_undegraded': '2024-09-10T09:58:22.761036+0000', 'last_fullsized': '2024-09-10T09:58:22.761036+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': ['1', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-10T09:58:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.761458+0000', 'last_change': '2024-09-10T09:58:22.761458+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.761458+0000', 'last_undegraded': '2024-09-10T09:58:22.761458+0000', 'last_fullsized': '2024-09-10T09:58:22.761458+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.772636+0000', 'last_change': '2024-09-10T09:58:21.772636+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.772636+0000', 'last_undegraded': '2024-09-10T09:58:21.772636+0000', 'last_fullsized': '2024-09-10T09:58:21.772636+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.767249+0000', 'last_change': '2024-09-10T09:58:21.767249+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.767249+0000', 'last_undegraded': '2024-09-10T09:58:21.767249+0000', 'last_fullsized': '2024-09-10T09:58:21.767249+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.772836+0000', 'last_change': '2024-09-10T09:58:21.772836+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.772836+0000', 'last_undegraded': '2024-09-10T09:58:21.772836+0000', 'last_fullsized': '2024-09-10T09:58:21.772836+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.763413+0000', 'last_change': '2024-09-10T09:58:21.763413+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.763413+0000', 'last_undegraded': '2024-09-10T09:58:21.763413+0000', 'last_fullsized': '2024-09-10T09:58:21.763413+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-10T09:58:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.769168+0000', 'last_change': '2024-09-10T09:58:21.769168+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.769168+0000', 'last_undegraded': '2024-09-10T09:58:21.769168+0000', 'last_fullsized': '2024-09-10T09:58:21.769168+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.761175+0000', 'last_change': '2024-09-10T09:58:22.761175+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.761175+0000', 'last_undegraded': '2024-09-10T09:58:22.761175+0000', 'last_fullsized': '2024-09-10T09:58:22.761175+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': ['6', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.772393+0000', 'last_change': '2024-09-10T09:58:21.772393+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.772393+0000', 'last_undegraded': '2024-09-10T09:58:21.772393+0000', 'last_fullsized': '2024-09-10T09:58:21.772393+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.767018+0000', 'last_change': '2024-09-10T09:58:21.767018+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.767018+0000', 'last_undegraded': '2024-09-10T09:58:21.767018+0000', 'last_fullsized': '2024-09-10T09:58:21.767018+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.769098+0000', 'last_change': '2024-09-10T09:58:21.769098+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.769098+0000', 'last_undegraded': '2024-09-10T09:58:21.769098+0000', 'last_fullsized': '2024-09-10T09:58:21.769098+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.760733+0000', 'last_change': '2024-09-10T09:58:22.760733+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.760733+0000', 'last_undegraded': '2024-09-10T09:58:22.760733+0000', 'last_fullsized': '2024-09-10T09:58:22.760733+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.768360+0000', 'last_change': '2024-09-10T09:58:21.768360+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.768360+0000', 'last_undegraded': '2024-09-10T09:58:21.768360+0000', 'last_fullsized': '2024-09-10T09:58:21.768360+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.761973+0000', 'last_change': '2024-09-10T09:58:21.761973+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.761973+0000', 'last_undegraded': '2024-09-10T09:58:21.761973+0000', 'last_fullsized': '2024-09-10T09:58:21.761973+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.759166+0000', 'last_change': '2024-09-10T09:58:22.759166+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.759166+0000', 'last_undegraded': '2024-09-10T09:58:22.759166+0000', 'last_fullsized': '2024-09-10T09:58:22.759166+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': ['1', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.770833+0000', 'last_change': '2024-09-10T09:58:21.770833+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.770833+0000', 'last_undegraded': '2024-09-10T09:58:21.770833+0000', 'last_fullsized': '2024-09-10T09:58:21.770833+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.761547+0000', 'last_change': '2024-09-10T09:58:22.761547+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.761547+0000', 'last_undegraded': '2024-09-10T09:58:22.761547+0000', 'last_fullsized': '2024-09-10T09:58:22.761547+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.758668+0000', 'last_change': '2024-09-10T09:58:22.758668+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.758668+0000', 'last_undegraded': '2024-09-10T09:58:22.758668+0000', 'last_fullsized': '2024-09-10T09:58:22.758668+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': ['1', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.770772+0000', 'last_change': '2024-09-10T09:58:21.770772+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.770772+0000', 'last_undegraded': '2024-09-10T09:58:21.770772+0000', 'last_fullsized': '2024-09-10T09:58:21.770772+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.769618+0000', 'last_change': '2024-09-10T09:58:21.769618+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.769618+0000', 'last_undegraded': '2024-09-10T09:58:21.769618+0000', 'last_fullsized': '2024-09-10T09:58:21.769618+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.760512+0000', 'last_change': '2024-09-10T09:58:22.760512+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.760512+0000', 'last_undegraded': '2024-09-10T09:58:22.760512+0000', 'last_fullsized': '2024-09-10T09:58:22.760512+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.761985+0000', 'last_change': '2024-09-10T09:58:22.761985+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.761985+0000', 'last_undegraded': '2024-09-10T09:58:22.761985+0000', 'last_fullsized': '2024-09-10T09:58:22.761985+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-10T09:58:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 22, 'state': 'creating+activating', 'last_fresh': '2024-09-10T09:58:22.760886+0000', 'last_change': '2024-09-10T09:58:22.760886+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:22.760886+0000', 'last_undegraded': '2024-09-10T09:58:22.760886+0000', 'last_fullsized': '2024-09-10T09:58:22.760886+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:58:27.607 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-10T09:58:27.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:58:21.773428+0000', 'last_change': '2024-09-10T09:58:21.773428+0000', 'last_active': '2024-09-10T09:58:21.747139+0000', 'last_peered': '2024-09-10T09:58:21.747139+0000', 'last_clean': '2024-09-10T09:58:21.747139+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:58:21.773428+0000', 'last_undegraded': '2024-09-10T09:58:21.773428+0000', 'last_fullsized': '2024-09-10T09:58:21.773428+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'last_clean_scrub_stamp': '2024-09-10T09:58:21.747139+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:58:27.607 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T09:58:27.739 INFO:tasks.ceph.mds.b.smithi145.stderr:2024-09-10T09:58:27.738+0000 7fe673a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T09:58:27.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d4e200640 1 Processor -- start 2024-09-10T09:58:27.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d4e200640 1 -- start start 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d4e200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7d4806e890 0x7f7d4806ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d4e200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7d480729b0 0x7f7d48072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d4e200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d48073300 con 0x7f7d4815e700 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d4e200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f7d4815eae0 con 0x7f7d4806e890 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d4e200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f7d480af980 con 0x7f7d480729b0 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d47400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f7d4815e700 0x7f7d480af270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41804/0 (socket says 172.21.15.110:41804) 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d47400640 1 -- 172.21.15.110:0/640403414 learned_addr learned my addr 172.21.15.110:0/640403414 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d46a00640 1 --2- 172.21.15.110:0/640403414 >> v2:172.21.15.110:3300/0 conn(0x7f7d4806e890 0x7f7d4806ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.769+0000 7f7d47e00640 1 --2- 172.21.15.110:0/640403414 >> v2:172.21.15.145:3300/0 conn(0x7f7d480729b0 0x7f7d48072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46a00640 1 -- 172.21.15.110:0/640403414 >> v2:172.21.15.145:3300/0 conn(0x7f7d480729b0 msgr2=0x7f7d48072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46a00640 1 --2- 172.21.15.110:0/640403414 >> v2:172.21.15.145:3300/0 conn(0x7f7d480729b0 0x7f7d48072dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46a00640 1 -- 172.21.15.110:0/640403414 >> v1:172.21.15.110:6789/0 conn(0x7f7d4815e700 legacy=0x7f7d480af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46a00640 1 -- 172.21.15.110:0/640403414 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4806eec0 con 0x7f7d4806e890 2024-09-10T09:58:27.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46000640 1 -- 172.21.15.110:0/640403414 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 265125399 0 0) 0x7f7d38001120 con 0x7f7d4815e700 2024-09-10T09:58:27.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46a00640 1 --2- 172.21.15.110:0/640403414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4806e890 0x7f7d4806ff60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d48062890 tx=0x7f7d3401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa11c03a3e88220 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46000640 1 -- 172.21.15.110:0/640403414 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d340b0070 con 0x7f7d4806e890 2024-09-10T09:58:27.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46000640 1 -- 172.21.15.110:0/640403414 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d340ab600 con 0x7f7d4806e890 2024-09-10T09:58:27.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d46000640 1 -- 172.21.15.110:0/640403414 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d340ab9d0 con 0x7f7d4806e890 2024-09-10T09:58:27.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d4e200640 1 -- 172.21.15.110:0/640403414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4806e890 msgr2=0x7f7d4806ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d4e200640 1 --2- 172.21.15.110:0/640403414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4806e890 0x7f7d4806ff60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d48062890 tx=0x7f7d3401c480 comp rx=0 tx=0).stop 2024-09-10T09:58:27.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d4e200640 1 -- 172.21.15.110:0/640403414 shutdown_connections 2024-09-10T09:58:27.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d4e200640 1 --2- 172.21.15.110:0/640403414 >> v2:172.21.15.145:3300/0 conn(0x7f7d480729b0 0x7f7d48072dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d4e200640 1 --2- 172.21.15.110:0/640403414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4806e890 0x7f7d4806ff60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d4e200640 1 -- 172.21.15.110:0/640403414 >> 172.21.15.110:0/640403414 conn(0x7f7d4806cb90 msgr2=0x7f7d4806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d4e200640 1 -- 172.21.15.110:0/640403414 shutdown_connections 2024-09-10T09:58:27.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.773+0000 7f7d4e200640 1 -- 172.21.15.110:0/640403414 wait complete. 2024-09-10T09:58:27.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 Processor -- start 2024-09-10T09:58:27.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 -- start start 2024-09-10T09:58:27.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4806e890 0x7f7d48148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d480729b0 0x7f7d4813f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4815e700 0x7f7d48159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d480b4660 con 0x7f7d4815e700 2024-09-10T09:58:27.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4806fa40 con 0x7f7d480729b0 2024-09-10T09:58:27.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7d480b47e0 con 0x7f7d4806e890 2024-09-10T09:58:27.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4815e700 0x7f7d48159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4815e700 0x7f7d48159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46618/0 (socket says 172.21.15.110:46618) 2024-09-10T09:58:27.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 -- 172.21.15.110:0/985820856 learned_addr learned my addr 172.21.15.110:0/985820856 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:27.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d46a00640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d480729b0 0x7f7d4813f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47400640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4806e890 0x7f7d48148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 -- 172.21.15.110:0/985820856 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4806e890 msgr2=0x7f7d48148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4806e890 0x7f7d48148870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 -- 172.21.15.110:0/985820856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d480729b0 msgr2=0x7f7d4813f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d480729b0 0x7f7d4813f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 -- 172.21.15.110:0/985820856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d48153200 con 0x7f7d4815e700 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d47e00640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4815e700 0x7f7d48159f90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000efc0 tx=0x7f7d4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d3f400640 1 -- 172.21.15.110:0/985820856 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40018020 con 0x7f7d4815e700 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d3f400640 1 -- 172.21.15.110:0/985820856 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d40014480 con 0x7f7d4815e700 2024-09-10T09:58:27.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d3f400640 1 -- 172.21.15.110:0/985820856 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40014870 con 0x7f7d4815e700 2024-09-10T09:58:27.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.777+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4813fdc0 con 0x7f7d4815e700 2024-09-10T09:58:27.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.781+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4814bc10 con 0x7f7d4815e700 2024-09-10T09:58:27.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.781+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d48072dc0 con 0x7f7d4815e700 2024-09-10T09:58:27.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.781+0000 7f7d3f400640 1 -- 172.21.15.110:0/985820856 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7d4001b020 con 0x7f7d4815e700 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.781+0000 7f7d3f400640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7d280678c0 0x7f7d28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.781+0000 7f7d3f400640 1 -- 172.21.15.110:0/985820856 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7d40086690 con 0x7f7d4815e700 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.781+0000 7f7d47400640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7d280678c0 0x7f7d28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.781+0000 7f7d47400640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7d280678c0 0x7f7d28069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000f70 tx=0x7f7d38007df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.785+0000 7f7d3f400640 1 -- 172.21.15.110:0/985820856 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7d40022070 con 0x7f7d4815e700 2024-09-10T09:58:28.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.997+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7d480afb00 con 0x7f7d4815e700 2024-09-10T09:58:28.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:27.997+0000 7f7d3f400640 1 -- 172.21.15.110:0/985820856 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1122 (secure 0 0 0) 0x7f7d40010040 con 0x7f7d4815e700 2024-09-10T09:58:28.001 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:28.001 INFO:teuthology.orchestra.run.smithi110.stdout:{"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1725962292,"num_in_osds":8,"osd_in_since":1725962280,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":44},{"state_name":"creating+peering","count":28},{"state_name":"creating+activating","count":25}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":241373184,"bytes_avail":772852740096,"bytes_total":773094113280,"inactive_pgs_ratio":0.54639172554016113},"fsmap":{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4270}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T09:57:58.314137+0000","services":{}},"progress_events":{}} 2024-09-10T09:58:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7d280678c0 msgr2=0x7f7d28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:28.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7d280678c0 0x7f7d28069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000f70 tx=0x7f7d38007df0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4815e700 msgr2=0x7f7d48159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4815e700 0x7f7d48159f90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000efc0 tx=0x7f7d4000c490 comp rx=0 tx=0).stop 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 shutdown_connections 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7d280678c0 0x7f7d28069d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d4815e700 0x7f7d48159f90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d480729b0 0x7f7d4813f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 --2- 172.21.15.110:0/985820856 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7d4806e890 0x7f7d48148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 >> 172.21.15.110:0/985820856 conn(0x7f7d4806cb90 msgr2=0x7f7d480b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:28.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 shutdown_connections 2024-09-10T09:58:28.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.001+0000 7f7d4e200640 1 -- 172.21.15.110:0/985820856 wait complete. 2024-09-10T09:58:28.020 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T09:58:28.021 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:58:28.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dafc00640 1 Processor -- start 2024-09-10T09:58:28.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dafc00640 1 -- start start 2024-09-10T09:58:28.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dafc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1da80af710 0x7f1da80b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:28.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dafc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1da8070470 0x7f1da80b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:28.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dafc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1da806ce00 con 0x7f1da806f980 2024-09-10T09:58:28.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dafc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1da806da10 con 0x7f1da80af710 2024-09-10T09:58:28.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dafc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1da806d0e0 con 0x7f1da8070470 2024-09-10T09:58:28.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dad200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1da806f980 0x7f1da806fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41806/0 (socket says 172.21.15.110:41806) 2024-09-10T09:58:28.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1dad200640 1 -- 172.21.15.110:0/1598231811 learned_addr learned my addr 172.21.15.110:0/1598231811 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:28.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.153+0000 7f1da7e00640 1 --2- 172.21.15.110:0/1598231811 >> v2:172.21.15.110:3300/0 conn(0x7f1da80af710 0x7f1da80b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:28.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1dadc00640 1 --2- 172.21.15.110:0/1598231811 >> v2:172.21.15.145:3300/0 conn(0x7f1da8070470 0x7f1da80b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:28.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7400640 1 -- 172.21.15.110:0/1598231811 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760262112 0 0) 0x7f1da806ce00 con 0x7f1da806f980 2024-09-10T09:58:28.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7400640 1 -- 172.21.15.110:0/1598231811 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d8c003530 con 0x7f1da806f980 2024-09-10T09:58:28.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7e00640 1 -- 172.21.15.110:0/1598231811 >> v2:172.21.15.145:3300/0 conn(0x7f1da8070470 msgr2=0x7f1da80b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:28.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7e00640 1 --2- 172.21.15.110:0/1598231811 >> v2:172.21.15.145:3300/0 conn(0x7f1da8070470 0x7f1da80b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7e00640 1 -- 172.21.15.110:0/1598231811 >> v1:172.21.15.110:6789/0 conn(0x7f1da806f980 legacy=0x7f1da806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:28.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7e00640 1 -- 172.21.15.110:0/1598231811 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da806d3c0 con 0x7f1da80af710 2024-09-10T09:58:28.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7e00640 1 --2- 172.21.15.110:0/1598231811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da80af710 0x7f1da80b6ae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1da80b2310 tx=0x7f1d9401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575d5542039ccb4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:28.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7400640 1 -- 172.21.15.110:0/1598231811 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b1070 con 0x7f1da80af710 2024-09-10T09:58:28.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7400640 1 -- 172.21.15.110:0/1598231811 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d940a8050 con 0x7f1da80af710 2024-09-10T09:58:28.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1da7400640 1 -- 172.21.15.110:0/1598231811 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940ac860 con 0x7f1da80af710 2024-09-10T09:58:28.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1dafc00640 1 -- 172.21.15.110:0/1598231811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da80af710 msgr2=0x7f1da80b6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:28.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1dafc00640 1 --2- 172.21.15.110:0/1598231811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da80af710 0x7f1da80b6ae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1da80b2310 tx=0x7f1d9401c3c0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1dafc00640 1 -- 172.21.15.110:0/1598231811 shutdown_connections 2024-09-10T09:58:28.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1dafc00640 1 --2- 172.21.15.110:0/1598231811 >> v2:172.21.15.145:3300/0 conn(0x7f1da8070470 0x7f1da80b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1dafc00640 1 --2- 172.21.15.110:0/1598231811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da80af710 0x7f1da80b6ae0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.157+0000 7f1dafc00640 1 -- 172.21.15.110:0/1598231811 >> 172.21.15.110:0/1598231811 conn(0x7f1da8063160 msgr2=0x7f1da809b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:28.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 -- 172.21.15.110:0/1598231811 shutdown_connections 2024-09-10T09:58:28.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 -- 172.21.15.110:0/1598231811 wait complete. 2024-09-10T09:58:28.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 Processor -- start 2024-09-10T09:58:28.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 -- start start 2024-09-10T09:58:28.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1da806f980 0x7f1da815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:28.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da8070470 0x7f1da809d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:28.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da80af710 0x7f1da815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1da816ae20 con 0x7f1da8070470 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1da816aca0 con 0x7f1da80af710 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dafc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1da816afa0 con 0x7f1da806f980 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da8070470 0x7f1da809d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da8070470 0x7f1da809d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46636/0 (socket says 172.21.15.110:46636) 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 -- 172.21.15.110:0/4255874092 learned_addr learned my addr 172.21.15.110:0/4255874092 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dad200640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1da806f980 0x7f1da815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 -- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1da806f980 msgr2=0x7f1da815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dadc00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da80af710 0x7f1da815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1da806f980 0x7f1da815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 -- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da80af710 msgr2=0x7f1da815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:28.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da80af710 0x7f1da815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 -- 172.21.15.110:0/4255874092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da8153c90 con 0x7f1da8070470 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dad200640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1da806f980 0x7f1da815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1dadc00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da80af710 0x7f1da815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da7e00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da8070470 0x7f1da809d870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1d94018fd0 tx=0x7f1d94019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da4c00640 1 -- 172.21.15.110:0/4255874092 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940bb070 con 0x7f1da8070470 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da4c00640 1 -- 172.21.15.110:0/4255874092 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d940acd20 con 0x7f1da8070470 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.161+0000 7f1da4c00640 1 -- 172.21.15.110:0/4255874092 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b1040 con 0x7f1da8070470 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.165+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da8153f70 con 0x7f1da8070470 2024-09-10T09:58:28.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.165+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da815c190 con 0x7f1da8070470 2024-09-10T09:58:28.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.165+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da806fd60 con 0x7f1da8070470 2024-09-10T09:58:28.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.165+0000 7f1da4c00640 1 -- 172.21.15.110:0/4255874092 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1d940be070 con 0x7f1da8070470 2024-09-10T09:58:28.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.165+0000 7f1da4c00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1d880678f0 0x7f1d88069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:28.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.165+0000 7f1da4c00640 1 -- 172.21.15.110:0/4255874092 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1d94120070 con 0x7f1da8070470 2024-09-10T09:58:28.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.165+0000 7f1dad200640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1d880678f0 0x7f1d88069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:28.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.165+0000 7f1dad200640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1d880678f0 0x7f1d88069db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005890 tx=0x7f1d98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:28.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.169+0000 7f1da4c00640 1 -- 172.21.15.110:0/4255874092 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1d94016070 con 0x7f1da8070470 2024-09-10T09:58:28.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.325+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1da80b0330 con 0x7f1d880678f0 2024-09-10T09:58:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.333+0000 7f1da4c00640 1 -- 172.21.15.110:0/4255874092 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247805 (secure 0 0 0) 0x7f1da80b0330 con 0x7f1d880678f0 2024-09-10T09:58:28.336 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:28.337 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:58:28.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1d880678f0 msgr2=0x7f1d88069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:28.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1d880678f0 0x7f1d88069db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005890 tx=0x7f1d98009040 comp rx=0 tx=0).stop 2024-09-10T09:58:28.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da8070470 msgr2=0x7f1da809d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:28.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da8070470 0x7f1da809d870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1d94018fd0 tx=0x7f1d94019160 comp rx=0 tx=0).stop 2024-09-10T09:58:28.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 shutdown_connections 2024-09-10T09:58:28.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1d880678f0 0x7f1d88069db0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1da80af710 0x7f1da815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1da8070470 0x7f1da809d870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 --2- 172.21.15.110:0/4255874092 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1da806f980 0x7f1da815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:28.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 >> 172.21.15.110:0/4255874092 conn(0x7f1da8063160 msgr2=0x7f1da80b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:28.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 shutdown_connections 2024-09-10T09:58:28.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:28.337+0000 7f1dafc00640 1 -- 172.21.15.110:0/4255874092 wait complete. 2024-09-10T09:58:28.356 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T09:58:27.356156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235716,"kb_used_data":2828,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739004,"statfs":{"total":773094113280,"available":772852740096,"internally_reserved":0,"allocated":2895872,"data_stored":1738463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001171"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.773483+0000","last_change":"2024-09-10T09:58:21.773483+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.773483+0000","last_undegraded":"2024-09-10T09:58:21.773483+0000","last_fullsized":"2024-09-10T09:58:21.773483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758695+0000","last_change":"2024-09-10T09:58:22.758695+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758695+0000","last_undegraded":"2024-09-10T09:58:22.758695+0000","last_fullsized":"2024-09-10T09:58:22.758695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770858+0000","last_change":"2024-09-10T09:58:21.770858+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770858+0000","last_undegraded":"2024-09-10T09:58:21.770858+0000","last_fullsized":"2024-09-10T09:58:21.770858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761137+0000","last_change":"2024-09-10T09:58:22.761137+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761137+0000","last_undegraded":"2024-09-10T09:58:22.761137+0000","last_fullsized":"2024-09-10T09:58:22.761137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770682+0000","last_change":"2024-09-10T09:58:21.770682+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770682+0000","last_undegraded":"2024-09-10T09:58:21.770682+0000","last_fullsized":"2024-09-10T09:58:21.770682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762312+0000","last_change":"2024-09-10T09:58:22.762312+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762312+0000","last_undegraded":"2024-09-10T09:58:22.762312+0000","last_fullsized":"2024-09-10T09:58:22.762312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.780164+0000","last_change":"2024-09-10T09:58:21.780164+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.780164+0000","last_undegraded":"2024-09-10T09:58:21.780164+0000","last_fullsized":"2024-09-10T09:58:21.780164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737654+0000","last_change":"2024-09-10T09:58:22.790696+0000","last_active":"2024-09-10T09:58:24.737654+0000","last_peered":"2024-09-10T09:58:24.737654+0000","last_clean":"2024-09-10T09:58:24.737654+0000","last_became_active":"2024-09-10T09:58:22.790265+0000","last_became_peered":"2024-09-10T09:58:22.790265+0000","last_unstale":"2024-09-10T09:58:24.737654+0000","last_undegraded":"2024-09-10T09:58:24.737654+0000","last_fullsized":"2024-09-10T09:58:24.737654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:20.430762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772032+0000","last_change":"2024-09-10T09:58:21.772032+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772032+0000","last_undegraded":"2024-09-10T09:58:21.772032+0000","last_fullsized":"2024-09-10T09:58:21.772032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760582+0000","last_change":"2024-09-10T09:58:22.760582+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760582+0000","last_undegraded":"2024-09-10T09:58:22.760582+0000","last_fullsized":"2024-09-10T09:58:22.760582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762468+0000","last_change":"2024-09-10T09:58:22.762468+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762468+0000","last_undegraded":"2024-09-10T09:58:22.762468+0000","last_fullsized":"2024-09-10T09:58:22.762468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767842+0000","last_change":"2024-09-10T09:58:21.767842+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767842+0000","last_undegraded":"2024-09-10T09:58:21.767842+0000","last_fullsized":"2024-09-10T09:58:21.767842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738573+0000","last_change":"2024-09-10T09:58:22.768744+0000","last_active":"2024-09-10T09:58:24.738573+0000","last_peered":"2024-09-10T09:58:24.738573+0000","last_clean":"2024-09-10T09:58:24.738573+0000","last_became_active":"2024-09-10T09:58:22.768515+0000","last_became_peered":"2024-09-10T09:58:22.768515+0000","last_unstale":"2024-09-10T09:58:24.738573+0000","last_undegraded":"2024-09-10T09:58:24.738573+0000","last_fullsized":"2024-09-10T09:58:24.738573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:37.491552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.765480+0000","last_change":"2024-09-10T09:58:21.765480+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.765480+0000","last_undegraded":"2024-09-10T09:58:21.765480+0000","last_fullsized":"2024-09-10T09:58:21.765480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758765+0000","last_change":"2024-09-10T09:58:22.758765+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758765+0000","last_undegraded":"2024-09-10T09:58:22.758765+0000","last_fullsized":"2024-09-10T09:58:22.758765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.764622+0000","last_change":"2024-09-10T09:58:22.764622+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.764622+0000","last_undegraded":"2024-09-10T09:58:22.764622+0000","last_fullsized":"2024-09-10T09:58:22.764622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760834+0000","last_change":"2024-09-10T09:58:22.760834+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760834+0000","last_undegraded":"2024-09-10T09:58:22.760834+0000","last_fullsized":"2024-09-10T09:58:22.760834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.762859+0000","last_change":"2024-09-10T09:58:21.762859+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.762859+0000","last_undegraded":"2024-09-10T09:58:21.762859+0000","last_fullsized":"2024-09-10T09:58:21.762859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767720+0000","last_change":"2024-09-10T09:58:21.767720+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767720+0000","last_undegraded":"2024-09-10T09:58:21.767720+0000","last_fullsized":"2024-09-10T09:58:21.767720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.763497+0000","last_change":"2024-09-10T09:58:22.763497+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.763497+0000","last_undegraded":"2024-09-10T09:58:22.763497+0000","last_fullsized":"2024-09-10T09:58:22.763497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.765123+0000","last_change":"2024-09-10T09:58:21.765123+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.765123+0000","last_undegraded":"2024-09-10T09:58:21.765123+0000","last_fullsized":"2024-09-10T09:58:21.765123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.762017+0000","last_change":"2024-09-10T09:58:22.762017+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.762017+0000","last_undegraded":"2024-09-10T09:58:22.762017+0000","last_fullsized":"2024-09-10T09:58:22.762017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738073+0000","last_change":"2024-09-10T09:58:22.799342+0000","last_active":"2024-09-10T09:58:24.738073+0000","last_peered":"2024-09-10T09:58:24.738073+0000","last_clean":"2024-09-10T09:58:24.738073+0000","last_became_active":"2024-09-10T09:58:22.799049+0000","last_became_peered":"2024-09-10T09:58:22.799049+0000","last_unstale":"2024-09-10T09:58:24.738073+0000","last_undegraded":"2024-09-10T09:58:24.738073+0000","last_fullsized":"2024-09-10T09:58:24.738073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:24.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.764229+0000","last_change":"2024-09-10T09:58:22.764229+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.764229+0000","last_undegraded":"2024-09-10T09:58:22.764229+0000","last_fullsized":"2024-09-10T09:58:22.764229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737548+0000","last_change":"2024-09-10T09:58:22.789676+0000","last_active":"2024-09-10T09:58:24.737548+0000","last_peered":"2024-09-10T09:58:24.737548+0000","last_clean":"2024-09-10T09:58:24.737548+0000","last_became_active":"2024-09-10T09:58:22.789249+0000","last_became_peered":"2024-09-10T09:58:22.789249+0000","last_unstale":"2024-09-10T09:58:24.737548+0000","last_undegraded":"2024-09-10T09:58:24.737548+0000","last_fullsized":"2024-09-10T09:58:24.737548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:03.582635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.769476+0000","last_change":"2024-09-10T09:58:22.769476+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.769476+0000","last_undegraded":"2024-09-10T09:58:22.769476+0000","last_fullsized":"2024-09-10T09:58:22.769476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.759733+0000","last_change":"2024-09-10T09:58:22.759733+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.759733+0000","last_undegraded":"2024-09-10T09:58:22.759733+0000","last_fullsized":"2024-09-10T09:58:22.759733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738554+0000","last_change":"2024-09-10T09:58:22.796593+0000","last_active":"2024-09-10T09:58:24.738554+0000","last_peered":"2024-09-10T09:58:24.738554+0000","last_clean":"2024-09-10T09:58:24.738554+0000","last_became_active":"2024-09-10T09:58:22.796294+0000","last_became_peered":"2024-09-10T09:58:22.796294+0000","last_unstale":"2024-09-10T09:58:24.738554+0000","last_undegraded":"2024-09-10T09:58:24.738554+0000","last_fullsized":"2024-09-10T09:58:24.738554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:43.026541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.764586+0000","last_change":"2024-09-10T09:58:21.764586+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.764586+0000","last_undegraded":"2024-09-10T09:58:21.764586+0000","last_fullsized":"2024-09-10T09:58:21.764586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761625+0000","last_change":"2024-09-10T09:58:22.761625+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761625+0000","last_undegraded":"2024-09-10T09:58:22.761625+0000","last_fullsized":"2024-09-10T09:58:22.761625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.762149+0000","last_change":"2024-09-10T09:58:21.762149+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.762149+0000","last_undegraded":"2024-09-10T09:58:21.762149+0000","last_fullsized":"2024-09-10T09:58:21.762149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760420+0000","last_change":"2024-09-10T09:58:22.760420+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760420+0000","last_undegraded":"2024-09-10T09:58:22.760420+0000","last_fullsized":"2024-09-10T09:58:22.760420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.768406+0000","last_change":"2024-09-10T09:58:21.768406+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.768406+0000","last_undegraded":"2024-09-10T09:58:21.768406+0000","last_fullsized":"2024-09-10T09:58:21.768406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738624+0000","last_change":"2024-09-10T09:58:21.783061+0000","last_active":"2024-09-10T09:58:24.738624+0000","last_peered":"2024-09-10T09:58:24.738624+0000","last_clean":"2024-09-10T09:58:24.738624+0000","last_became_active":"2024-09-10T09:58:21.782756+0000","last_became_peered":"2024-09-10T09:58:21.782756+0000","last_unstale":"2024-09-10T09:58:24.738624+0000","last_undegraded":"2024-09-10T09:58:24.738624+0000","last_fullsized":"2024-09-10T09:58:24.738624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:39.702619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.764021+0000","last_change":"2024-09-10T09:58:21.764021+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.764021+0000","last_undegraded":"2024-09-10T09:58:21.764021+0000","last_fullsized":"2024-09-10T09:58:21.764021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.765118+0000","last_change":"2024-09-10T09:58:21.778244+0000","last_active":"2024-09-10T09:58:22.765118+0000","last_peered":"2024-09-10T09:58:22.765118+0000","last_clean":"2024-09-10T09:58:22.765118+0000","last_became_active":"2024-09-10T09:58:21.777937+0000","last_became_peered":"2024-09-10T09:58:21.777937+0000","last_unstale":"2024-09-10T09:58:22.765118+0000","last_undegraded":"2024-09-10T09:58:22.765118+0000","last_fullsized":"2024-09-10T09:58:22.765118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738137+0000","last_change":"2024-09-10T09:58:21.775701+0000","last_active":"2024-09-10T09:58:24.738137+0000","last_peered":"2024-09-10T09:58:24.738137+0000","last_clean":"2024-09-10T09:58:24.738137+0000","last_became_active":"2024-09-10T09:58:21.775431+0000","last_became_peered":"2024-09-10T09:58:21.775431+0000","last_unstale":"2024-09-10T09:58:24.738137+0000","last_undegraded":"2024-09-10T09:58:24.738137+0000","last_fullsized":"2024-09-10T09:58:24.738137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:03.845811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737371+0000","last_change":"2024-09-10T09:58:22.772570+0000","last_active":"2024-09-10T09:58:24.737371+0000","last_peered":"2024-09-10T09:58:24.737371+0000","last_clean":"2024-09-10T09:58:24.737371+0000","last_became_active":"2024-09-10T09:58:22.772248+0000","last_became_peered":"2024-09-10T09:58:22.772248+0000","last_unstale":"2024-09-10T09:58:24.737371+0000","last_undegraded":"2024-09-10T09:58:24.737371+0000","last_fullsized":"2024-09-10T09:58:24.737371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:55.062755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.765584+0000","last_change":"2024-09-10T09:58:21.786530+0000","last_active":"2024-09-10T09:58:22.765584+0000","last_peered":"2024-09-10T09:58:22.765584+0000","last_clean":"2024-09-10T09:58:22.765584+0000","last_became_active":"2024-09-10T09:58:21.786157+0000","last_became_peered":"2024-09-10T09:58:21.786157+0000","last_unstale":"2024-09-10T09:58:22.765584+0000","last_undegraded":"2024-09-10T09:58:22.765584+0000","last_fullsized":"2024-09-10T09:58:22.765584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761036+0000","last_change":"2024-09-10T09:58:22.761036+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761036+0000","last_undegraded":"2024-09-10T09:58:22.761036+0000","last_fullsized":"2024-09-10T09:58:22.761036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761809+0000","last_change":"2024-09-10T09:58:21.786956+0000","last_active":"2024-09-10T09:58:22.761809+0000","last_peered":"2024-09-10T09:58:22.761809+0000","last_clean":"2024-09-10T09:58:22.761809+0000","last_became_active":"2024-09-10T09:58:21.786523+0000","last_became_peered":"2024-09-10T09:58:21.786523+0000","last_unstale":"2024-09-10T09:58:22.761809+0000","last_undegraded":"2024-09-10T09:58:22.761809+0000","last_fullsized":"2024-09-10T09:58:22.761809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761458+0000","last_change":"2024-09-10T09:58:22.761458+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761458+0000","last_undegraded":"2024-09-10T09:58:22.761458+0000","last_fullsized":"2024-09-10T09:58:22.761458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761846+0000","last_change":"2024-09-10T09:58:21.787382+0000","last_active":"2024-09-10T09:58:22.761846+0000","last_peered":"2024-09-10T09:58:22.761846+0000","last_clean":"2024-09-10T09:58:22.761846+0000","last_became_active":"2024-09-10T09:58:21.787048+0000","last_became_peered":"2024-09-10T09:58:21.787048+0000","last_unstale":"2024-09-10T09:58:22.761846+0000","last_undegraded":"2024-09-10T09:58:22.761846+0000","last_fullsized":"2024-09-10T09:58:22.761846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772636+0000","last_change":"2024-09-10T09:58:21.772636+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772636+0000","last_undegraded":"2024-09-10T09:58:21.772636+0000","last_fullsized":"2024-09-10T09:58:21.772636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759611+0000","last_change":"2024-09-10T09:58:21.785126+0000","last_active":"2024-09-10T09:58:22.759611+0000","last_peered":"2024-09-10T09:58:22.759611+0000","last_clean":"2024-09-10T09:58:22.759611+0000","last_became_active":"2024-09-10T09:58:21.784839+0000","last_became_peered":"2024-09-10T09:58:21.784839+0000","last_unstale":"2024-09-10T09:58:22.759611+0000","last_undegraded":"2024-09-10T09:58:22.759611+0000","last_fullsized":"2024-09-10T09:58:22.759611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767249+0000","last_change":"2024-09-10T09:58:21.767249+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767249+0000","last_undegraded":"2024-09-10T09:58:21.767249+0000","last_fullsized":"2024-09-10T09:58:21.767249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.763857+0000","last_change":"2024-09-10T09:58:21.776340+0000","last_active":"2024-09-10T09:58:22.763857+0000","last_peered":"2024-09-10T09:58:22.763857+0000","last_clean":"2024-09-10T09:58:22.763857+0000","last_became_active":"2024-09-10T09:58:21.776066+0000","last_became_peered":"2024-09-10T09:58:21.776066+0000","last_unstale":"2024-09-10T09:58:22.763857+0000","last_undegraded":"2024-09-10T09:58:22.763857+0000","last_fullsized":"2024-09-10T09:58:22.763857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772836+0000","last_change":"2024-09-10T09:58:21.772836+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772836+0000","last_undegraded":"2024-09-10T09:58:21.772836+0000","last_fullsized":"2024-09-10T09:58:21.772836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.785464+0000","last_change":"2024-09-10T09:58:21.783558+0000","last_active":"2024-09-10T09:58:21.785464+0000","last_peered":"2024-09-10T09:58:21.785464+0000","last_clean":"2024-09-10T09:58:21.785464+0000","last_became_active":"2024-09-10T09:58:21.782455+0000","last_became_peered":"2024-09-10T09:58:21.782455+0000","last_unstale":"2024-09-10T09:58:21.785464+0000","last_undegraded":"2024-09-10T09:58:21.785464+0000","last_fullsized":"2024-09-10T09:58:21.785464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:26.367199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.763413+0000","last_change":"2024-09-10T09:58:21.763413+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.763413+0000","last_undegraded":"2024-09-10T09:58:21.763413+0000","last_fullsized":"2024-09-10T09:58:21.763413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762503+0000","last_change":"2024-09-10T09:58:21.789974+0000","last_active":"2024-09-10T09:58:22.762503+0000","last_peered":"2024-09-10T09:58:22.762503+0000","last_clean":"2024-09-10T09:58:22.762503+0000","last_became_active":"2024-09-10T09:58:21.788481+0000","last_became_peered":"2024-09-10T09:58:21.788481+0000","last_unstale":"2024-09-10T09:58:22.762503+0000","last_undegraded":"2024-09-10T09:58:22.762503+0000","last_fullsized":"2024-09-10T09:58:22.762503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769168+0000","last_change":"2024-09-10T09:58:21.769168+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769168+0000","last_undegraded":"2024-09-10T09:58:21.769168+0000","last_fullsized":"2024-09-10T09:58:21.769168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.760516+0000","last_change":"2024-09-10T09:58:21.778738+0000","last_active":"2024-09-10T09:58:22.760516+0000","last_peered":"2024-09-10T09:58:22.760516+0000","last_clean":"2024-09-10T09:58:22.760516+0000","last_became_active":"2024-09-10T09:58:21.777982+0000","last_became_peered":"2024-09-10T09:58:21.777982+0000","last_unstale":"2024-09-10T09:58:22.760516+0000","last_undegraded":"2024-09-10T09:58:22.760516+0000","last_fullsized":"2024-09-10T09:58:22.760516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761175+0000","last_change":"2024-09-10T09:58:22.761175+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761175+0000","last_undegraded":"2024-09-10T09:58:22.761175+0000","last_fullsized":"2024-09-10T09:58:22.761175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.782949+0000","last_change":"2024-09-10T09:58:21.779750+0000","last_active":"2024-09-10T09:58:21.782949+0000","last_peered":"2024-09-10T09:58:21.782949+0000","last_clean":"2024-09-10T09:58:21.782949+0000","last_became_active":"2024-09-10T09:58:21.779492+0000","last_became_peered":"2024-09-10T09:58:21.779492+0000","last_unstale":"2024-09-10T09:58:21.782949+0000","last_undegraded":"2024-09-10T09:58:21.782949+0000","last_fullsized":"2024-09-10T09:58:21.782949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:26.367199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.772393+0000","last_change":"2024-09-10T09:58:21.772393+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.772393+0000","last_undegraded":"2024-09-10T09:58:21.772393+0000","last_fullsized":"2024-09-10T09:58:21.772393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.789668+0000","last_change":"2024-09-10T09:58:21.781792+0000","last_active":"2024-09-10T09:58:21.789668+0000","last_peered":"2024-09-10T09:58:21.789668+0000","last_clean":"2024-09-10T09:58:21.789668+0000","last_became_active":"2024-09-10T09:58:21.781434+0000","last_became_peered":"2024-09-10T09:58:21.781434+0000","last_unstale":"2024-09-10T09:58:21.789668+0000","last_undegraded":"2024-09-10T09:58:21.789668+0000","last_fullsized":"2024-09-10T09:58:21.789668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:42.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.767018+0000","last_change":"2024-09-10T09:58:21.767018+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.767018+0000","last_undegraded":"2024-09-10T09:58:21.767018+0000","last_fullsized":"2024-09-10T09:58:21.767018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.783020+0000","last_change":"2024-09-10T09:58:21.781287+0000","last_active":"2024-09-10T09:58:21.783020+0000","last_peered":"2024-09-10T09:58:21.783020+0000","last_clean":"2024-09-10T09:58:21.783020+0000","last_became_active":"2024-09-10T09:58:21.780824+0000","last_became_peered":"2024-09-10T09:58:21.780824+0000","last_unstale":"2024-09-10T09:58:21.783020+0000","last_undegraded":"2024-09-10T09:58:21.783020+0000","last_fullsized":"2024-09-10T09:58:21.783020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:11.144974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769098+0000","last_change":"2024-09-10T09:58:21.769098+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769098+0000","last_undegraded":"2024-09-10T09:58:21.769098+0000","last_fullsized":"2024-09-10T09:58:21.769098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759829+0000","last_change":"2024-09-10T09:58:21.776106+0000","last_active":"2024-09-10T09:58:22.759829+0000","last_peered":"2024-09-10T09:58:22.759829+0000","last_clean":"2024-09-10T09:58:22.759829+0000","last_became_active":"2024-09-10T09:58:21.773315+0000","last_became_peered":"2024-09-10T09:58:21.773315+0000","last_unstale":"2024-09-10T09:58:22.759829+0000","last_undegraded":"2024-09-10T09:58:22.759829+0000","last_fullsized":"2024-09-10T09:58:22.759829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760733+0000","last_change":"2024-09-10T09:58:22.760733+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760733+0000","last_undegraded":"2024-09-10T09:58:22.760733+0000","last_fullsized":"2024-09-10T09:58:22.760733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.790976+0000","last_change":"2024-09-10T09:58:21.789986+0000","last_active":"2024-09-10T09:58:21.790976+0000","last_peered":"2024-09-10T09:58:21.790976+0000","last_clean":"2024-09-10T09:58:21.790976+0000","last_became_active":"2024-09-10T09:58:21.789577+0000","last_became_peered":"2024-09-10T09:58:21.789577+0000","last_unstale":"2024-09-10T09:58:21.790976+0000","last_undegraded":"2024-09-10T09:58:21.790976+0000","last_fullsized":"2024-09-10T09:58:21.790976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.768360+0000","last_change":"2024-09-10T09:58:21.768360+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.768360+0000","last_undegraded":"2024-09-10T09:58:21.768360+0000","last_fullsized":"2024-09-10T09:58:21.768360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":166,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.783072+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:58:21.783072+0000","last_peered":"2024-09-10T09:58:21.783072+0000","last_clean":"2024-09-10T09:58:21.783072+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:58:21.783072+0000","last_undegraded":"2024-09-10T09:58:21.783072+0000","last_fullsized":"2024-09-10T09:58:21.783072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762945+0000","last_change":"2024-09-10T09:58:21.789884+0000","last_active":"2024-09-10T09:58:22.762945+0000","last_peered":"2024-09-10T09:58:22.762945+0000","last_clean":"2024-09-10T09:58:22.762945+0000","last_became_active":"2024-09-10T09:58:21.788940+0000","last_became_peered":"2024-09-10T09:58:21.788940+0000","last_unstale":"2024-09-10T09:58:22.762945+0000","last_undegraded":"2024-09-10T09:58:22.762945+0000","last_fullsized":"2024-09-10T09:58:22.762945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737853+0000","last_change":"2024-09-10T09:58:22.792386+0000","last_active":"2024-09-10T09:58:24.737853+0000","last_peered":"2024-09-10T09:58:24.737853+0000","last_clean":"2024-09-10T09:58:24.737853+0000","last_became_active":"2024-09-10T09:58:22.792151+0000","last_became_peered":"2024-09-10T09:58:22.792151+0000","last_unstale":"2024-09-10T09:58:24.737853+0000","last_undegraded":"2024-09-10T09:58:24.737853+0000","last_fullsized":"2024-09-10T09:58:24.737853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.810620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792384+0000","last_change":"2024-09-10T09:58:21.791063+0000","last_active":"2024-09-10T09:58:21.792384+0000","last_peered":"2024-09-10T09:58:21.792384+0000","last_clean":"2024-09-10T09:58:21.792384+0000","last_became_active":"2024-09-10T09:58:21.790796+0000","last_became_peered":"2024-09-10T09:58:21.790796+0000","last_unstale":"2024-09-10T09:58:21.792384+0000","last_undegraded":"2024-09-10T09:58:21.792384+0000","last_fullsized":"2024-09-10T09:58:21.792384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.761973+0000","last_change":"2024-09-10T09:58:21.761973+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.761973+0000","last_undegraded":"2024-09-10T09:58:21.761973+0000","last_fullsized":"2024-09-10T09:58:21.761973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.781486+0000","last_change":"2024-09-10T09:58:21.777208+0000","last_active":"2024-09-10T09:58:21.781486+0000","last_peered":"2024-09-10T09:58:21.781486+0000","last_clean":"2024-09-10T09:58:21.781486+0000","last_became_active":"2024-09-10T09:58:21.776820+0000","last_became_peered":"2024-09-10T09:58:21.776820+0000","last_unstale":"2024-09-10T09:58:21.781486+0000","last_undegraded":"2024-09-10T09:58:21.781486+0000","last_fullsized":"2024-09-10T09:58:21.781486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:42.983386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.759166+0000","last_change":"2024-09-10T09:58:22.759166+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.759166+0000","last_undegraded":"2024-09-10T09:58:22.759166+0000","last_fullsized":"2024-09-10T09:58:22.759166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.762585+0000","last_change":"2024-09-10T09:58:21.785388+0000","last_active":"2024-09-10T09:58:22.762585+0000","last_peered":"2024-09-10T09:58:22.762585+0000","last_clean":"2024-09-10T09:58:22.762585+0000","last_became_active":"2024-09-10T09:58:21.785227+0000","last_became_peered":"2024-09-10T09:58:21.785227+0000","last_unstale":"2024-09-10T09:58:22.762585+0000","last_undegraded":"2024-09-10T09:58:22.762585+0000","last_fullsized":"2024-09-10T09:58:22.762585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770833+0000","last_change":"2024-09-10T09:58:21.770833+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770833+0000","last_undegraded":"2024-09-10T09:58:21.770833+0000","last_fullsized":"2024-09-10T09:58:21.770833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.758612+0000","last_change":"2024-09-10T09:58:21.768431+0000","last_active":"2024-09-10T09:58:22.758612+0000","last_peered":"2024-09-10T09:58:22.758612+0000","last_clean":"2024-09-10T09:58:22.758612+0000","last_became_active":"2024-09-10T09:58:21.767826+0000","last_became_peered":"2024-09-10T09:58:21.767826+0000","last_unstale":"2024-09-10T09:58:22.758612+0000","last_undegraded":"2024-09-10T09:58:22.758612+0000","last_fullsized":"2024-09-10T09:58:22.758612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761547+0000","last_change":"2024-09-10T09:58:22.761547+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761547+0000","last_undegraded":"2024-09-10T09:58:22.761547+0000","last_fullsized":"2024-09-10T09:58:22.761547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792043+0000","last_change":"2024-09-10T09:58:21.788480+0000","last_active":"2024-09-10T09:58:21.792043+0000","last_peered":"2024-09-10T09:58:21.792043+0000","last_clean":"2024-09-10T09:58:21.792043+0000","last_became_active":"2024-09-10T09:58:21.788162+0000","last_became_peered":"2024-09-10T09:58:21.788162+0000","last_unstale":"2024-09-10T09:58:21.792043+0000","last_undegraded":"2024-09-10T09:58:21.792043+0000","last_fullsized":"2024-09-10T09:58:21.792043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:05.666246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.758668+0000","last_change":"2024-09-10T09:58:22.758668+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.758668+0000","last_undegraded":"2024-09-10T09:58:22.758668+0000","last_fullsized":"2024-09-10T09:58:22.758668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.758990+0000","last_change":"2024-09-10T09:58:21.770640+0000","last_active":"2024-09-10T09:58:22.758990+0000","last_peered":"2024-09-10T09:58:22.758990+0000","last_clean":"2024-09-10T09:58:22.758990+0000","last_became_active":"2024-09-10T09:58:21.769111+0000","last_became_peered":"2024-09-10T09:58:21.769111+0000","last_unstale":"2024-09-10T09:58:22.758990+0000","last_undegraded":"2024-09-10T09:58:22.758990+0000","last_fullsized":"2024-09-10T09:58:22.758990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738499+0000","last_change":"2024-09-10T09:58:22.789520+0000","last_active":"2024-09-10T09:58:24.738499+0000","last_peered":"2024-09-10T09:58:24.738499+0000","last_clean":"2024-09-10T09:58:24.738499+0000","last_became_active":"2024-09-10T09:58:22.788875+0000","last_became_peered":"2024-09-10T09:58:22.788875+0000","last_unstale":"2024-09-10T09:58:24.738499+0000","last_undegraded":"2024-09-10T09:58:24.738499+0000","last_fullsized":"2024-09-10T09:58:24.738499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:00.959927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.761978+0000","last_change":"2024-09-10T09:58:21.782019+0000","last_active":"2024-09-10T09:58:22.761978+0000","last_peered":"2024-09-10T09:58:22.761978+0000","last_clean":"2024-09-10T09:58:22.761978+0000","last_became_active":"2024-09-10T09:58:21.781750+0000","last_became_peered":"2024-09-10T09:58:21.781750+0000","last_unstale":"2024-09-10T09:58:22.761978+0000","last_undegraded":"2024-09-10T09:58:22.761978+0000","last_fullsized":"2024-09-10T09:58:22.761978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.770772+0000","last_change":"2024-09-10T09:58:21.770772+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.770772+0000","last_undegraded":"2024-09-10T09:58:21.770772+0000","last_fullsized":"2024-09-10T09:58:21.770772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.791839+0000","last_change":"2024-09-10T09:58:21.788078+0000","last_active":"2024-09-10T09:58:21.791839+0000","last_peered":"2024-09-10T09:58:21.791839+0000","last_clean":"2024-09-10T09:58:21.791839+0000","last_became_active":"2024-09-10T09:58:21.787660+0000","last_became_peered":"2024-09-10T09:58:21.787660+0000","last_unstale":"2024-09-10T09:58:21.791839+0000","last_undegraded":"2024-09-10T09:58:21.791839+0000","last_fullsized":"2024-09-10T09:58:21.791839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.769618+0000","last_change":"2024-09-10T09:58:21.769618+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.769618+0000","last_undegraded":"2024-09-10T09:58:21.769618+0000","last_fullsized":"2024-09-10T09:58:21.769618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759017+0000","last_change":"2024-09-10T09:58:21.782043+0000","last_active":"2024-09-10T09:58:22.759017+0000","last_peered":"2024-09-10T09:58:22.759017+0000","last_clean":"2024-09-10T09:58:22.759017+0000","last_became_active":"2024-09-10T09:58:21.781762+0000","last_became_peered":"2024-09-10T09:58:21.781762+0000","last_unstale":"2024-09-10T09:58:22.759017+0000","last_undegraded":"2024-09-10T09:58:22.759017+0000","last_fullsized":"2024-09-10T09:58:22.759017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738162+0000","last_change":"2024-09-10T09:58:22.772755+0000","last_active":"2024-09-10T09:58:24.738162+0000","last_peered":"2024-09-10T09:58:24.738162+0000","last_clean":"2024-09-10T09:58:24.738162+0000","last_became_active":"2024-09-10T09:58:22.772505+0000","last_became_peered":"2024-09-10T09:58:22.772505+0000","last_unstale":"2024-09-10T09:58:24.738162+0000","last_undegraded":"2024-09-10T09:58:24.738162+0000","last_fullsized":"2024-09-10T09:58:24.738162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:48.318092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T09:58:22.759075+0000","last_change":"2024-09-10T09:58:21.774362+0000","last_active":"2024-09-10T09:58:22.759075+0000","last_peered":"2024-09-10T09:58:22.759075+0000","last_clean":"2024-09-10T09:58:22.759075+0000","last_became_active":"2024-09-10T09:58:21.773133+0000","last_became_peered":"2024-09-10T09:58:21.773133+0000","last_unstale":"2024-09-10T09:58:22.759075+0000","last_undegraded":"2024-09-10T09:58:22.759075+0000","last_fullsized":"2024-09-10T09:58:22.759075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738587+0000","last_change":"2024-09-10T09:58:22.791719+0000","last_active":"2024-09-10T09:58:24.738587+0000","last_peered":"2024-09-10T09:58:24.738587+0000","last_clean":"2024-09-10T09:58:24.738587+0000","last_became_active":"2024-09-10T09:58:22.790615+0000","last_became_peered":"2024-09-10T09:58:22.790615+0000","last_unstale":"2024-09-10T09:58:24.738587+0000","last_undegraded":"2024-09-10T09:58:24.738587+0000","last_fullsized":"2024-09-10T09:58:24.738587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:59.165906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792397+0000","last_change":"2024-09-10T09:58:21.790948+0000","last_active":"2024-09-10T09:58:21.792397+0000","last_peered":"2024-09-10T09:58:21.792397+0000","last_clean":"2024-09-10T09:58:21.792397+0000","last_became_active":"2024-09-10T09:58:21.790543+0000","last_became_peered":"2024-09-10T09:58:21.790543+0000","last_unstale":"2024-09-10T09:58:21.792397+0000","last_undegraded":"2024-09-10T09:58:21.792397+0000","last_fullsized":"2024-09-10T09:58:21.792397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760512+0000","last_change":"2024-09-10T09:58:22.760512+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760512+0000","last_undegraded":"2024-09-10T09:58:22.760512+0000","last_fullsized":"2024-09-10T09:58:22.760512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.761985+0000","last_change":"2024-09-10T09:58:22.761985+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.761985+0000","last_undegraded":"2024-09-10T09:58:22.761985+0000","last_fullsized":"2024-09-10T09:58:22.761985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.794737+0000","last_change":"2024-09-10T09:58:21.793454+0000","last_active":"2024-09-10T09:58:21.794737+0000","last_peered":"2024-09-10T09:58:21.794737+0000","last_clean":"2024-09-10T09:58:21.794737+0000","last_became_active":"2024-09-10T09:58:21.793058+0000","last_became_peered":"2024-09-10T09:58:21.793058+0000","last_unstale":"2024-09-10T09:58:21.794737+0000","last_undegraded":"2024-09-10T09:58:21.794737+0000","last_fullsized":"2024-09-10T09:58:21.794737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":3,"reported_epoch":22,"state":"creating+activating","last_fresh":"2024-09-10T09:58:22.760886+0000","last_change":"2024-09-10T09:58:22.760886+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:22.760886+0000","last_undegraded":"2024-09-10T09:58:22.760886+0000","last_fullsized":"2024-09-10T09:58:22.760886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.792296+0000","last_change":"2024-09-10T09:58:21.789114+0000","last_active":"2024-09-10T09:58:21.792296+0000","last_peered":"2024-09-10T09:58:21.792296+0000","last_clean":"2024-09-10T09:58:21.792296+0000","last_became_active":"2024-09-10T09:58:21.788849+0000","last_became_peered":"2024-09-10T09:58:21.788849+0000","last_unstale":"2024-09-10T09:58:21.792296+0000","last_undegraded":"2024-09-10T09:58:21.792296+0000","last_fullsized":"2024-09-10T09:58:21.792296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:00.039731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-10T09:58:21.773428+0000","last_change":"2024-09-10T09:58:21.773428+0000","last_active":"2024-09-10T09:58:21.747139+0000","last_peered":"2024-09-10T09:58:21.747139+0000","last_clean":"2024-09-10T09:58:21.747139+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:58:21.773428+0000","last_undegraded":"2024-09-10T09:58:21.773428+0000","last_fullsized":"2024-09-10T09:58:21.773428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738067+0000","last_change":"2024-09-10T09:58:21.779788+0000","last_active":"2024-09-10T09:58:24.738067+0000","last_peered":"2024-09-10T09:58:24.738067+0000","last_clean":"2024-09-10T09:58:24.738067+0000","last_became_active":"2024-09-10T09:58:21.779521+0000","last_became_peered":"2024-09-10T09:58:21.779521+0000","last_unstale":"2024-09-10T09:58:24.738067+0000","last_undegraded":"2024-09-10T09:58:24.738067+0000","last_fullsized":"2024-09-10T09:58:24.738067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:33.355227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737993+0000","last_change":"2024-09-10T09:58:22.777347+0000","last_active":"2024-09-10T09:58:24.737993+0000","last_peered":"2024-09-10T09:58:24.737993+0000","last_clean":"2024-09-10T09:58:24.737993+0000","last_became_active":"2024-09-10T09:58:22.777042+0000","last_became_peered":"2024-09-10T09:58:22.777042+0000","last_unstale":"2024-09-10T09:58:24.737993+0000","last_undegraded":"2024-09-10T09:58:24.737993+0000","last_fullsized":"2024-09-10T09:58:24.737993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:31.068726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T09:58:21.788781+0000","last_change":"2024-09-10T09:58:21.787400+0000","last_active":"2024-09-10T09:58:21.788781+0000","last_peered":"2024-09-10T09:58:21.788781+0000","last_clean":"2024-09-10T09:58:21.788781+0000","last_became_active":"2024-09-10T09:58:21.787111+0000","last_became_peered":"2024-09-10T09:58:21.787111+0000","last_unstale":"2024-09-10T09:58:21.788781+0000","last_undegraded":"2024-09-10T09:58:21.788781+0000","last_fullsized":"2024-09-10T09:58:21.788781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:09:11.144974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":802816,"data_stored":658248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509443,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":67880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":658248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:58:31.360 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:58:31.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.505+0000 7f4580000640 1 Processor -- start 2024-09-10T09:58:31.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.505+0000 7f4580000640 1 -- start start 2024-09-10T09:58:31.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.505+0000 7f4580000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f45780cfe30 0x7f4578071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.505+0000 7f4580000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f457806e7a0 0x7f45780cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.505+0000 7f4580000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457806d2d0 con 0x7f45780cd510 2024-09-10T09:58:31.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.505+0000 7f4580000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f457815d9a0 con 0x7f45780cfe30 2024-09-10T09:58:31.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.505+0000 7f4580000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f457806d5b0 con 0x7f457806e7a0 2024-09-10T09:58:31.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457d600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f45780cfe30 0x7f4578071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457d600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f45780cfe30 0x7f4578071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53028/0 (socket says 172.21.15.110:53028) 2024-09-10T09:58:31.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457d600640 1 -- 172.21.15.110:0/2452728344 learned_addr learned my addr 172.21.15.110:0/2452728344 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:31.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457e000640 1 --2- 172.21.15.110:0/2452728344 >> v2:172.21.15.145:3300/0 conn(0x7f457806e7a0 0x7f45780cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457d600640 1 -- 172.21.15.110:0/2452728344 >> v2:172.21.15.145:3300/0 conn(0x7f457806e7a0 msgr2=0x7f45780cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457d600640 1 --2- 172.21.15.110:0/2452728344 >> v2:172.21.15.145:3300/0 conn(0x7f457806e7a0 0x7f45780cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457d600640 1 -- 172.21.15.110:0/2452728344 >> v1:172.21.15.145:6789/0 conn(0x7f45780cd510 legacy=0x7f457806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:31.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457d600640 1 -- 172.21.15.110:0/2452728344 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457806d890 con 0x7f45780cfe30 2024-09-10T09:58:31.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.509+0000 7f457d600640 1 --2- 172.21.15.110:0/2452728344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cfe30 0x7f4578071a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f457806cb00 tx=0x7f456c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518ee538b29b8616 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4577e00640 1 -- 172.21.15.110:0/2452728344 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c0a56f0 con 0x7f45780cfe30 2024-09-10T09:58:31.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4577e00640 1 -- 172.21.15.110:0/2452728344 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456c0a5890 con 0x7f45780cfe30 2024-09-10T09:58:31.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4577e00640 1 -- 172.21.15.110:0/2452728344 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c0a5c60 con 0x7f45780cfe30 2024-09-10T09:58:31.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- 172.21.15.110:0/2452728344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cfe30 msgr2=0x7f4578071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 --2- 172.21.15.110:0/2452728344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cfe30 0x7f4578071a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f457806cb00 tx=0x7f456c01c920 comp rx=0 tx=0).stop 2024-09-10T09:58:31.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- 172.21.15.110:0/2452728344 shutdown_connections 2024-09-10T09:58:31.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 --2- 172.21.15.110:0/2452728344 >> v2:172.21.15.145:3300/0 conn(0x7f457806e7a0 0x7f45780cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 --2- 172.21.15.110:0/2452728344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cfe30 0x7f4578071a40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- 172.21.15.110:0/2452728344 >> 172.21.15.110:0/2452728344 conn(0x7f457806bc50 msgr2=0x7f457806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- 172.21.15.110:0/2452728344 shutdown_connections 2024-09-10T09:58:31.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- 172.21.15.110:0/2452728344 wait complete. 2024-09-10T09:58:31.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 Processor -- start 2024-09-10T09:58:31.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- start start 2024-09-10T09:58:31.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f457806e7a0 0x7f4578157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cd510 0x7f45780aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45780cfe30 0x7f45780aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4578072190 con 0x7f45780cd510 2024-09-10T09:58:31.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f457815d6c0 con 0x7f457806e7a0 2024-09-10T09:58:31.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f4580000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4578169180 con 0x7f45780cfe30 2024-09-10T09:58:31.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f457cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cd510 0x7f45780aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.513+0000 7f457cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cd510 0x7f45780aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53042/0 (socket says 172.21.15.110:53042) 2024-09-10T09:58:31.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457cc00640 1 -- 172.21.15.110:0/2351419547 learned_addr learned my addr 172.21.15.110:0/2351419547 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:31.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457d600640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f457806e7a0 0x7f4578157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457e000640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45780cfe30 0x7f45780aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457cc00640 1 -- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45780cfe30 msgr2=0x7f45780aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457cc00640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45780cfe30 0x7f45780aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457cc00640 1 -- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f457806e7a0 msgr2=0x7f4578157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457cc00640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f457806e7a0 0x7f4578157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457cc00640 1 -- 172.21.15.110:0/2351419547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457806ecd0 con 0x7f45780cd510 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457e000640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45780cfe30 0x7f45780aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f457cc00640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cd510 0x7f45780aa430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f456400b440 tx=0x7f456400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f4575600640 1 -- 172.21.15.110:0/2351419547 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564015020 con 0x7f45780cd510 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f4575600640 1 -- 172.21.15.110:0/2351419547 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4564011890 con 0x7f45780cd510 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4578152bf0 con 0x7f45780cd510 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f4575600640 1 -- 172.21.15.110:0/2351419547 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564011c80 con 0x7f45780cd510 2024-09-10T09:58:31.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4578158440 con 0x7f45780cd510 2024-09-10T09:58:31.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457815c260 con 0x7f45780cd510 2024-09-10T09:58:31.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.517+0000 7f4575600640 1 -- 172.21.15.110:0/2351419547 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4564018020 con 0x7f45780cd510 2024-09-10T09:58:31.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.521+0000 7f4575600640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f45580678f0 0x7f4558069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.521+0000 7f4575600640 1 -- 172.21.15.110:0/2351419547 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4564085a10 con 0x7f45780cd510 2024-09-10T09:58:31.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.521+0000 7f457d600640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f45580678f0 0x7f4558069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.521+0000 7f457d600640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f45580678f0 0x7f4558069db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f456c0ae000 tx=0x7f456c0a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.525+0000 7f4575600640 1 -- 172.21.15.110:0/2351419547 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f456401f030 con 0x7f45780cd510 2024-09-10T09:58:31.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.677+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45780719c0 con 0x7f45580678f0 2024-09-10T09:58:31.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.681+0000 7f4575600640 1 -- 172.21.15.110:0/2351419547 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f45780719c0 con 0x7f45580678f0 2024-09-10T09:58:31.687 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:58:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f45580678f0 msgr2=0x7f4558069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f45580678f0 0x7f4558069db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f456c0ae000 tx=0x7f456c0a7070 comp rx=0 tx=0).stop 2024-09-10T09:58:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cd510 msgr2=0x7f45780aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cd510 0x7f45780aa430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f456400b440 tx=0x7f456400b900 comp rx=0 tx=0).stop 2024-09-10T09:58:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 shutdown_connections 2024-09-10T09:58:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f45580678f0 0x7f4558069db0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45780cfe30 0x7f45780aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45780cd510 0x7f45780aa430 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 --2- 172.21.15.110:0/2351419547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f457806e7a0 0x7f4578157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 >> 172.21.15.110:0/2351419547 conn(0x7f457806bc50 msgr2=0x7f4578072340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 shutdown_connections 2024-09-10T09:58:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.685+0000 7f4580000640 1 -- 172.21.15.110:0/2351419547 wait complete. 2024-09-10T09:58:31.707 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T09:58:31.357169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235916,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738804,"statfs":{"total":773094113280,"available":772852535296,"internally_reserved":0,"allocated":3121152,"data_stored":1833196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.600326"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739344+0000","last_change":"2024-09-10T09:58:22.803813+0000","last_active":"2024-09-10T09:58:24.739344+0000","last_peered":"2024-09-10T09:58:24.739344+0000","last_clean":"2024-09-10T09:58:24.739344+0000","last_became_active":"2024-09-10T09:58:22.803440+0000","last_became_peered":"2024-09-10T09:58:22.803440+0000","last_unstale":"2024-09-10T09:58:24.739344+0000","last_undegraded":"2024-09-10T09:58:24.739344+0000","last_fullsized":"2024-09-10T09:58:24.739344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:41.057291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782118+0000","last_change":"2024-09-10T09:58:22.786389+0000","last_active":"2024-09-10T09:58:23.782118+0000","last_peered":"2024-09-10T09:58:23.782118+0000","last_clean":"2024-09-10T09:58:23.782118+0000","last_became_active":"2024-09-10T09:58:22.786093+0000","last_became_peered":"2024-09-10T09:58:22.786093+0000","last_unstale":"2024-09-10T09:58:23.782118+0000","last_undegraded":"2024-09-10T09:58:23.782118+0000","last_fullsized":"2024-09-10T09:58:23.782118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738194+0000","last_change":"2024-09-10T09:58:22.782747+0000","last_active":"2024-09-10T09:58:24.738194+0000","last_peered":"2024-09-10T09:58:24.738194+0000","last_clean":"2024-09-10T09:58:24.738194+0000","last_became_active":"2024-09-10T09:58:22.782493+0000","last_became_peered":"2024-09-10T09:58:22.782493+0000","last_unstale":"2024-09-10T09:58:24.738194+0000","last_undegraded":"2024-09-10T09:58:24.738194+0000","last_fullsized":"2024-09-10T09:58:24.738194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738202+0000","last_change":"2024-09-10T09:58:22.780919+0000","last_active":"2024-09-10T09:58:24.738202+0000","last_peered":"2024-09-10T09:58:24.738202+0000","last_clean":"2024-09-10T09:58:24.738202+0000","last_became_active":"2024-09-10T09:58:22.779369+0000","last_became_peered":"2024-09-10T09:58:22.779369+0000","last_unstale":"2024-09-10T09:58:24.738202+0000","last_undegraded":"2024-09-10T09:58:24.738202+0000","last_fullsized":"2024-09-10T09:58:24.738202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:17.698136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738364+0000","last_change":"2024-09-10T09:58:22.803835+0000","last_active":"2024-09-10T09:58:24.738364+0000","last_peered":"2024-09-10T09:58:24.738364+0000","last_clean":"2024-09-10T09:58:24.738364+0000","last_became_active":"2024-09-10T09:58:22.803465+0000","last_became_peered":"2024-09-10T09:58:22.803465+0000","last_unstale":"2024-09-10T09:58:24.738364+0000","last_undegraded":"2024-09-10T09:58:24.738364+0000","last_fullsized":"2024-09-10T09:58:24.738364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:17.698136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737987+0000","last_change":"2024-09-10T09:58:22.804773+0000","last_active":"2024-09-10T09:58:24.737987+0000","last_peered":"2024-09-10T09:58:24.737987+0000","last_clean":"2024-09-10T09:58:24.737987+0000","last_became_active":"2024-09-10T09:58:22.801323+0000","last_became_peered":"2024-09-10T09:58:22.801323+0000","last_unstale":"2024-09-10T09:58:24.737987+0000","last_undegraded":"2024-09-10T09:58:24.737987+0000","last_fullsized":"2024-09-10T09:58:24.737987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:36.116325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737772+0000","last_change":"2024-09-10T09:58:22.790265+0000","last_active":"2024-09-10T09:58:24.737772+0000","last_peered":"2024-09-10T09:58:24.737772+0000","last_clean":"2024-09-10T09:58:24.737772+0000","last_became_active":"2024-09-10T09:58:22.789359+0000","last_became_peered":"2024-09-10T09:58:22.789359+0000","last_unstale":"2024-09-10T09:58:24.737772+0000","last_undegraded":"2024-09-10T09:58:24.737772+0000","last_fullsized":"2024-09-10T09:58:24.737772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:36.116325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737654+0000","last_change":"2024-09-10T09:58:22.790696+0000","last_active":"2024-09-10T09:58:24.737654+0000","last_peered":"2024-09-10T09:58:24.737654+0000","last_clean":"2024-09-10T09:58:24.737654+0000","last_became_active":"2024-09-10T09:58:22.790265+0000","last_became_peered":"2024-09-10T09:58:22.790265+0000","last_unstale":"2024-09-10T09:58:24.737654+0000","last_undegraded":"2024-09-10T09:58:24.737654+0000","last_fullsized":"2024-09-10T09:58:24.737654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:20.430762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739480+0000","last_change":"2024-09-10T09:58:22.806568+0000","last_active":"2024-09-10T09:58:24.739480+0000","last_peered":"2024-09-10T09:58:24.739480+0000","last_clean":"2024-09-10T09:58:24.739480+0000","last_became_active":"2024-09-10T09:58:22.806203+0000","last_became_peered":"2024-09-10T09:58:22.806203+0000","last_unstale":"2024-09-10T09:58:24.739480+0000","last_undegraded":"2024-09-10T09:58:24.739480+0000","last_fullsized":"2024-09-10T09:58:24.739480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:17.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773231+0000","last_change":"2024-09-10T09:58:22.785593+0000","last_active":"2024-09-10T09:58:23.773231+0000","last_peered":"2024-09-10T09:58:23.773231+0000","last_clean":"2024-09-10T09:58:23.773231+0000","last_became_active":"2024-09-10T09:58:22.785227+0000","last_became_peered":"2024-09-10T09:58:22.785227+0000","last_unstale":"2024-09-10T09:58:23.773231+0000","last_undegraded":"2024-09-10T09:58:23.773231+0000","last_fullsized":"2024-09-10T09:58:23.773231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772786+0000","last_change":"2024-09-10T09:58:22.792764+0000","last_active":"2024-09-10T09:58:23.772786+0000","last_peered":"2024-09-10T09:58:23.772786+0000","last_clean":"2024-09-10T09:58:23.772786+0000","last_became_active":"2024-09-10T09:58:22.791958+0000","last_became_peered":"2024-09-10T09:58:22.791958+0000","last_unstale":"2024-09-10T09:58:23.772786+0000","last_undegraded":"2024-09-10T09:58:23.772786+0000","last_fullsized":"2024-09-10T09:58:23.772786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773090+0000","last_change":"2024-09-10T09:58:22.774413+0000","last_active":"2024-09-10T09:58:23.773090+0000","last_peered":"2024-09-10T09:58:23.773090+0000","last_clean":"2024-09-10T09:58:23.773090+0000","last_became_active":"2024-09-10T09:58:22.774110+0000","last_became_peered":"2024-09-10T09:58:22.774110+0000","last_unstale":"2024-09-10T09:58:23.773090+0000","last_undegraded":"2024-09-10T09:58:23.773090+0000","last_fullsized":"2024-09-10T09:58:23.773090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:27.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738573+0000","last_change":"2024-09-10T09:58:22.768744+0000","last_active":"2024-09-10T09:58:24.738573+0000","last_peered":"2024-09-10T09:58:24.738573+0000","last_clean":"2024-09-10T09:58:24.738573+0000","last_became_active":"2024-09-10T09:58:22.768515+0000","last_became_peered":"2024-09-10T09:58:22.768515+0000","last_unstale":"2024-09-10T09:58:24.738573+0000","last_undegraded":"2024-09-10T09:58:24.738573+0000","last_fullsized":"2024-09-10T09:58:24.738573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:37.491552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737347+0000","last_change":"2024-09-10T09:58:22.797642+0000","last_active":"2024-09-10T09:58:24.737347+0000","last_peered":"2024-09-10T09:58:24.737347+0000","last_clean":"2024-09-10T09:58:24.737347+0000","last_became_active":"2024-09-10T09:58:22.797304+0000","last_became_peered":"2024-09-10T09:58:22.797304+0000","last_unstale":"2024-09-10T09:58:24.737347+0000","last_undegraded":"2024-09-10T09:58:24.737347+0000","last_fullsized":"2024-09-10T09:58:24.737347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:59.080268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781863+0000","last_change":"2024-09-10T09:58:22.799237+0000","last_active":"2024-09-10T09:58:23.781863+0000","last_peered":"2024-09-10T09:58:23.781863+0000","last_clean":"2024-09-10T09:58:23.781863+0000","last_became_active":"2024-09-10T09:58:22.798951+0000","last_became_peered":"2024-09-10T09:58:22.798951+0000","last_unstale":"2024-09-10T09:58:23.781863+0000","last_undegraded":"2024-09-10T09:58:23.781863+0000","last_fullsized":"2024-09-10T09:58:23.781863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738069+0000","last_change":"2024-09-10T09:58:22.801607+0000","last_active":"2024-09-10T09:58:24.738069+0000","last_peered":"2024-09-10T09:58:24.738069+0000","last_clean":"2024-09-10T09:58:24.738069+0000","last_became_active":"2024-09-10T09:58:22.801252+0000","last_became_peered":"2024-09-10T09:58:22.801252+0000","last_unstale":"2024-09-10T09:58:24.738069+0000","last_undegraded":"2024-09-10T09:58:24.738069+0000","last_fullsized":"2024-09-10T09:58:24.738069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739077+0000","last_change":"2024-09-10T09:58:22.800761+0000","last_active":"2024-09-10T09:58:24.739077+0000","last_peered":"2024-09-10T09:58:24.739077+0000","last_clean":"2024-09-10T09:58:24.739077+0000","last_became_active":"2024-09-10T09:58:22.800421+0000","last_became_peered":"2024-09-10T09:58:22.800421+0000","last_unstale":"2024-09-10T09:58:24.739077+0000","last_undegraded":"2024-09-10T09:58:24.739077+0000","last_fullsized":"2024-09-10T09:58:24.739077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:17.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770538+0000","last_change":"2024-09-10T09:58:22.780000+0000","last_active":"2024-09-10T09:58:23.770538+0000","last_peered":"2024-09-10T09:58:23.770538+0000","last_clean":"2024-09-10T09:58:23.770538+0000","last_became_active":"2024-09-10T09:58:22.779739+0000","last_became_peered":"2024-09-10T09:58:22.779739+0000","last_unstale":"2024-09-10T09:58:23.770538+0000","last_undegraded":"2024-09-10T09:58:23.770538+0000","last_fullsized":"2024-09-10T09:58:23.770538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:48.157725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739018+0000","last_change":"2024-09-10T09:58:22.789943+0000","last_active":"2024-09-10T09:58:24.739018+0000","last_peered":"2024-09-10T09:58:24.739018+0000","last_clean":"2024-09-10T09:58:24.739018+0000","last_became_active":"2024-09-10T09:58:22.789657+0000","last_became_peered":"2024-09-10T09:58:22.789657+0000","last_unstale":"2024-09-10T09:58:24.739018+0000","last_undegraded":"2024-09-10T09:58:24.739018+0000","last_fullsized":"2024-09-10T09:58:24.739018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:17.386153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772836+0000","last_change":"2024-09-10T09:58:22.795150+0000","last_active":"2024-09-10T09:58:23.772836+0000","last_peered":"2024-09-10T09:58:23.772836+0000","last_clean":"2024-09-10T09:58:23.772836+0000","last_became_active":"2024-09-10T09:58:22.794828+0000","last_became_peered":"2024-09-10T09:58:22.794828+0000","last_unstale":"2024-09-10T09:58:23.772836+0000","last_undegraded":"2024-09-10T09:58:23.772836+0000","last_fullsized":"2024-09-10T09:58:23.772836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737641+0000","last_change":"2024-09-10T09:58:22.804668+0000","last_active":"2024-09-10T09:58:24.737641+0000","last_peered":"2024-09-10T09:58:24.737641+0000","last_clean":"2024-09-10T09:58:24.737641+0000","last_became_active":"2024-09-10T09:58:22.804314+0000","last_became_peered":"2024-09-10T09:58:22.804314+0000","last_unstale":"2024-09-10T09:58:24.737641+0000","last_undegraded":"2024-09-10T09:58:24.737641+0000","last_fullsized":"2024-09-10T09:58:24.737641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:34.641953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738097+0000","last_change":"2024-09-10T09:58:22.800575+0000","last_active":"2024-09-10T09:58:24.738097+0000","last_peered":"2024-09-10T09:58:24.738097+0000","last_clean":"2024-09-10T09:58:24.738097+0000","last_became_active":"2024-09-10T09:58:22.800254+0000","last_became_peered":"2024-09-10T09:58:22.800254+0000","last_unstale":"2024-09-10T09:58:24.738097+0000","last_undegraded":"2024-09-10T09:58:24.738097+0000","last_fullsized":"2024-09-10T09:58:24.738097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:12.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738073+0000","last_change":"2024-09-10T09:58:22.799342+0000","last_active":"2024-09-10T09:58:24.738073+0000","last_peered":"2024-09-10T09:58:24.738073+0000","last_clean":"2024-09-10T09:58:24.738073+0000","last_became_active":"2024-09-10T09:58:22.799049+0000","last_became_peered":"2024-09-10T09:58:22.799049+0000","last_unstale":"2024-09-10T09:58:24.738073+0000","last_undegraded":"2024-09-10T09:58:24.738073+0000","last_fullsized":"2024-09-10T09:58:24.738073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:24.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737824+0000","last_change":"2024-09-10T09:58:22.786910+0000","last_active":"2024-09-10T09:58:24.737824+0000","last_peered":"2024-09-10T09:58:24.737824+0000","last_clean":"2024-09-10T09:58:24.737824+0000","last_became_active":"2024-09-10T09:58:22.784972+0000","last_became_peered":"2024-09-10T09:58:22.784972+0000","last_unstale":"2024-09-10T09:58:24.737824+0000","last_undegraded":"2024-09-10T09:58:24.737824+0000","last_fullsized":"2024-09-10T09:58:24.737824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737548+0000","last_change":"2024-09-10T09:58:22.789676+0000","last_active":"2024-09-10T09:58:24.737548+0000","last_peered":"2024-09-10T09:58:24.737548+0000","last_clean":"2024-09-10T09:58:24.737548+0000","last_became_active":"2024-09-10T09:58:22.789249+0000","last_became_peered":"2024-09-10T09:58:22.789249+0000","last_unstale":"2024-09-10T09:58:24.737548+0000","last_undegraded":"2024-09-10T09:58:24.737548+0000","last_fullsized":"2024-09-10T09:58:24.737548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:03.582635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773346+0000","last_change":"2024-09-10T09:58:22.793654+0000","last_active":"2024-09-10T09:58:23.773346+0000","last_peered":"2024-09-10T09:58:23.773346+0000","last_clean":"2024-09-10T09:58:23.773346+0000","last_became_active":"2024-09-10T09:58:22.793394+0000","last_became_peered":"2024-09-10T09:58:22.793394+0000","last_unstale":"2024-09-10T09:58:23.773346+0000","last_undegraded":"2024-09-10T09:58:23.773346+0000","last_fullsized":"2024-09-10T09:58:23.773346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782057+0000","last_change":"2024-09-10T09:58:22.766616+0000","last_active":"2024-09-10T09:58:23.782057+0000","last_peered":"2024-09-10T09:58:23.782057+0000","last_clean":"2024-09-10T09:58:23.782057+0000","last_became_active":"2024-09-10T09:58:22.766321+0000","last_became_peered":"2024-09-10T09:58:22.766321+0000","last_unstale":"2024-09-10T09:58:23.782057+0000","last_undegraded":"2024-09-10T09:58:23.782057+0000","last_fullsized":"2024-09-10T09:58:23.782057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738554+0000","last_change":"2024-09-10T09:58:22.796593+0000","last_active":"2024-09-10T09:58:24.738554+0000","last_peered":"2024-09-10T09:58:24.738554+0000","last_clean":"2024-09-10T09:58:24.738554+0000","last_became_active":"2024-09-10T09:58:22.796294+0000","last_became_peered":"2024-09-10T09:58:22.796294+0000","last_unstale":"2024-09-10T09:58:24.738554+0000","last_undegraded":"2024-09-10T09:58:24.738554+0000","last_fullsized":"2024-09-10T09:58:24.738554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:43.026541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737668+0000","last_change":"2024-09-10T09:58:22.790593+0000","last_active":"2024-09-10T09:58:24.737668+0000","last_peered":"2024-09-10T09:58:24.737668+0000","last_clean":"2024-09-10T09:58:24.737668+0000","last_became_active":"2024-09-10T09:58:22.790244+0000","last_became_peered":"2024-09-10T09:58:22.790244+0000","last_unstale":"2024-09-10T09:58:24.737668+0000","last_undegraded":"2024-09-10T09:58:24.737668+0000","last_fullsized":"2024-09-10T09:58:24.737668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:18.106190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738741+0000","last_change":"2024-09-10T09:58:22.784426+0000","last_active":"2024-09-10T09:58:24.738741+0000","last_peered":"2024-09-10T09:58:24.738741+0000","last_clean":"2024-09-10T09:58:24.738741+0000","last_became_active":"2024-09-10T09:58:22.784101+0000","last_became_peered":"2024-09-10T09:58:22.784101+0000","last_unstale":"2024-09-10T09:58:24.738741+0000","last_undegraded":"2024-09-10T09:58:24.738741+0000","last_fullsized":"2024-09-10T09:58:24.738741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:39.940923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770388+0000","last_change":"2024-09-10T09:58:22.785741+0000","last_active":"2024-09-10T09:58:23.770388+0000","last_peered":"2024-09-10T09:58:23.770388+0000","last_clean":"2024-09-10T09:58:23.770388+0000","last_became_active":"2024-09-10T09:58:22.785508+0000","last_became_peered":"2024-09-10T09:58:22.785508+0000","last_unstale":"2024-09-10T09:58:23.770388+0000","last_undegraded":"2024-09-10T09:58:23.770388+0000","last_fullsized":"2024-09-10T09:58:23.770388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:49.472990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772262+0000","last_change":"2024-09-10T09:58:22.776565+0000","last_active":"2024-09-10T09:58:23.772262+0000","last_peered":"2024-09-10T09:58:23.772262+0000","last_clean":"2024-09-10T09:58:23.772262+0000","last_became_active":"2024-09-10T09:58:22.776167+0000","last_became_peered":"2024-09-10T09:58:22.776167+0000","last_unstale":"2024-09-10T09:58:23.772262+0000","last_undegraded":"2024-09-10T09:58:23.772262+0000","last_fullsized":"2024-09-10T09:58:23.772262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772714+0000","last_change":"2024-09-10T09:58:22.776703+0000","last_active":"2024-09-10T09:58:23.772714+0000","last_peered":"2024-09-10T09:58:23.772714+0000","last_clean":"2024-09-10T09:58:23.772714+0000","last_became_active":"2024-09-10T09:58:22.776418+0000","last_became_peered":"2024-09-10T09:58:22.776418+0000","last_unstale":"2024-09-10T09:58:23.772714+0000","last_undegraded":"2024-09-10T09:58:23.772714+0000","last_fullsized":"2024-09-10T09:58:23.772714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:59.498264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738624+0000","last_change":"2024-09-10T09:58:21.783061+0000","last_active":"2024-09-10T09:58:24.738624+0000","last_peered":"2024-09-10T09:58:24.738624+0000","last_clean":"2024-09-10T09:58:24.738624+0000","last_became_active":"2024-09-10T09:58:21.782756+0000","last_became_peered":"2024-09-10T09:58:21.782756+0000","last_unstale":"2024-09-10T09:58:24.738624+0000","last_undegraded":"2024-09-10T09:58:24.738624+0000","last_fullsized":"2024-09-10T09:58:24.738624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:39.702619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737843+0000","last_change":"2024-09-10T09:58:22.800136+0000","last_active":"2024-09-10T09:58:24.737843+0000","last_peered":"2024-09-10T09:58:24.737843+0000","last_clean":"2024-09-10T09:58:24.737843+0000","last_became_active":"2024-09-10T09:58:22.799760+0000","last_became_peered":"2024-09-10T09:58:22.799760+0000","last_unstale":"2024-09-10T09:58:24.737843+0000","last_undegraded":"2024-09-10T09:58:24.737843+0000","last_fullsized":"2024-09-10T09:58:24.737843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772303+0000","last_change":"2024-09-10T09:58:21.778244+0000","last_active":"2024-09-10T09:58:23.772303+0000","last_peered":"2024-09-10T09:58:23.772303+0000","last_clean":"2024-09-10T09:58:23.772303+0000","last_became_active":"2024-09-10T09:58:21.777937+0000","last_became_peered":"2024-09-10T09:58:21.777937+0000","last_unstale":"2024-09-10T09:58:23.772303+0000","last_undegraded":"2024-09-10T09:58:23.772303+0000","last_fullsized":"2024-09-10T09:58:23.772303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738137+0000","last_change":"2024-09-10T09:58:21.775701+0000","last_active":"2024-09-10T09:58:24.738137+0000","last_peered":"2024-09-10T09:58:24.738137+0000","last_clean":"2024-09-10T09:58:24.738137+0000","last_became_active":"2024-09-10T09:58:21.775431+0000","last_became_peered":"2024-09-10T09:58:21.775431+0000","last_unstale":"2024-09-10T09:58:24.738137+0000","last_undegraded":"2024-09-10T09:58:24.738137+0000","last_fullsized":"2024-09-10T09:58:24.738137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:03.845811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737371+0000","last_change":"2024-09-10T09:58:22.772570+0000","last_active":"2024-09-10T09:58:24.737371+0000","last_peered":"2024-09-10T09:58:24.737371+0000","last_clean":"2024-09-10T09:58:24.737371+0000","last_became_active":"2024-09-10T09:58:22.772248+0000","last_became_peered":"2024-09-10T09:58:22.772248+0000","last_unstale":"2024-09-10T09:58:24.737371+0000","last_undegraded":"2024-09-10T09:58:24.737371+0000","last_fullsized":"2024-09-10T09:58:24.737371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:55.062755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737937+0000","last_change":"2024-09-10T09:58:21.786530+0000","last_active":"2024-09-10T09:58:24.737937+0000","last_peered":"2024-09-10T09:58:24.737937+0000","last_clean":"2024-09-10T09:58:24.737937+0000","last_became_active":"2024-09-10T09:58:21.786157+0000","last_became_peered":"2024-09-10T09:58:21.786157+0000","last_unstale":"2024-09-10T09:58:24.737937+0000","last_undegraded":"2024-09-10T09:58:24.737937+0000","last_fullsized":"2024-09-10T09:58:24.737937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781837+0000","last_change":"2024-09-10T09:58:22.767449+0000","last_active":"2024-09-10T09:58:23.781837+0000","last_peered":"2024-09-10T09:58:23.781837+0000","last_clean":"2024-09-10T09:58:23.781837+0000","last_became_active":"2024-09-10T09:58:22.767182+0000","last_became_peered":"2024-09-10T09:58:22.767182+0000","last_unstale":"2024-09-10T09:58:23.781837+0000","last_undegraded":"2024-09-10T09:58:23.781837+0000","last_fullsized":"2024-09-10T09:58:23.781837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773334+0000","last_change":"2024-09-10T09:58:21.786956+0000","last_active":"2024-09-10T09:58:23.773334+0000","last_peered":"2024-09-10T09:58:23.773334+0000","last_clean":"2024-09-10T09:58:23.773334+0000","last_became_active":"2024-09-10T09:58:21.786523+0000","last_became_peered":"2024-09-10T09:58:21.786523+0000","last_unstale":"2024-09-10T09:58:23.773334+0000","last_undegraded":"2024-09-10T09:58:23.773334+0000","last_fullsized":"2024-09-10T09:58:23.773334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738905+0000","last_change":"2024-09-10T09:58:22.801352+0000","last_active":"2024-09-10T09:58:24.738905+0000","last_peered":"2024-09-10T09:58:24.738905+0000","last_clean":"2024-09-10T09:58:24.738905+0000","last_became_active":"2024-09-10T09:58:22.800233+0000","last_became_peered":"2024-09-10T09:58:22.800233+0000","last_unstale":"2024-09-10T09:58:24.738905+0000","last_undegraded":"2024-09-10T09:58:24.738905+0000","last_fullsized":"2024-09-10T09:58:24.738905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:41.057291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739047+0000","last_change":"2024-09-10T09:58:21.787382+0000","last_active":"2024-09-10T09:58:24.739047+0000","last_peered":"2024-09-10T09:58:24.739047+0000","last_clean":"2024-09-10T09:58:24.739047+0000","last_became_active":"2024-09-10T09:58:21.787048+0000","last_became_peered":"2024-09-10T09:58:21.787048+0000","last_unstale":"2024-09-10T09:58:24.739047+0000","last_undegraded":"2024-09-10T09:58:24.739047+0000","last_fullsized":"2024-09-10T09:58:24.739047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738181+0000","last_change":"2024-09-10T09:58:22.792565+0000","last_active":"2024-09-10T09:58:24.738181+0000","last_peered":"2024-09-10T09:58:24.738181+0000","last_clean":"2024-09-10T09:58:24.738181+0000","last_became_active":"2024-09-10T09:58:22.792257+0000","last_became_peered":"2024-09-10T09:58:22.792257+0000","last_unstale":"2024-09-10T09:58:24.738181+0000","last_undegraded":"2024-09-10T09:58:24.738181+0000","last_fullsized":"2024-09-10T09:58:24.738181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782153+0000","last_change":"2024-09-10T09:58:21.785126+0000","last_active":"2024-09-10T09:58:23.782153+0000","last_peered":"2024-09-10T09:58:23.782153+0000","last_clean":"2024-09-10T09:58:23.782153+0000","last_became_active":"2024-09-10T09:58:21.784839+0000","last_became_peered":"2024-09-10T09:58:21.784839+0000","last_unstale":"2024-09-10T09:58:23.782153+0000","last_undegraded":"2024-09-10T09:58:23.782153+0000","last_fullsized":"2024-09-10T09:58:23.782153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773200+0000","last_change":"2024-09-10T09:58:22.786530+0000","last_active":"2024-09-10T09:58:23.773200+0000","last_peered":"2024-09-10T09:58:23.773200+0000","last_clean":"2024-09-10T09:58:23.773200+0000","last_became_active":"2024-09-10T09:58:22.782828+0000","last_became_peered":"2024-09-10T09:58:22.782828+0000","last_unstale":"2024-09-10T09:58:23.773200+0000","last_undegraded":"2024-09-10T09:58:23.773200+0000","last_fullsized":"2024-09-10T09:58:23.773200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:18.151980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772938+0000","last_change":"2024-09-10T09:58:21.776340+0000","last_active":"2024-09-10T09:58:23.772938+0000","last_peered":"2024-09-10T09:58:23.772938+0000","last_clean":"2024-09-10T09:58:23.772938+0000","last_became_active":"2024-09-10T09:58:21.776066+0000","last_became_peered":"2024-09-10T09:58:21.776066+0000","last_unstale":"2024-09-10T09:58:23.772938+0000","last_undegraded":"2024-09-10T09:58:23.772938+0000","last_fullsized":"2024-09-10T09:58:23.772938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738448+0000","last_change":"2024-09-10T09:58:22.785299+0000","last_active":"2024-09-10T09:58:24.738448+0000","last_peered":"2024-09-10T09:58:24.738448+0000","last_clean":"2024-09-10T09:58:24.738448+0000","last_became_active":"2024-09-10T09:58:22.784958+0000","last_became_peered":"2024-09-10T09:58:22.784958+0000","last_unstale":"2024-09-10T09:58:24.738448+0000","last_undegraded":"2024-09-10T09:58:24.738448+0000","last_fullsized":"2024-09-10T09:58:24.738448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:26.963772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737976+0000","last_change":"2024-09-10T09:58:21.783558+0000","last_active":"2024-09-10T09:58:24.737976+0000","last_peered":"2024-09-10T09:58:24.737976+0000","last_clean":"2024-09-10T09:58:24.737976+0000","last_became_active":"2024-09-10T09:58:21.782455+0000","last_became_peered":"2024-09-10T09:58:21.782455+0000","last_unstale":"2024-09-10T09:58:24.737976+0000","last_undegraded":"2024-09-10T09:58:24.737976+0000","last_fullsized":"2024-09-10T09:58:24.737976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737404+0000","last_change":"2024-09-10T09:58:22.796235+0000","last_active":"2024-09-10T09:58:24.737404+0000","last_peered":"2024-09-10T09:58:24.737404+0000","last_clean":"2024-09-10T09:58:24.737404+0000","last_became_active":"2024-09-10T09:58:22.793164+0000","last_became_peered":"2024-09-10T09:58:22.793164+0000","last_unstale":"2024-09-10T09:58:24.737404+0000","last_undegraded":"2024-09-10T09:58:24.737404+0000","last_fullsized":"2024-09-10T09:58:24.737404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738233+0000","last_change":"2024-09-10T09:58:21.789974+0000","last_active":"2024-09-10T09:58:24.738233+0000","last_peered":"2024-09-10T09:58:24.738233+0000","last_clean":"2024-09-10T09:58:24.738233+0000","last_became_active":"2024-09-10T09:58:21.788481+0000","last_became_peered":"2024-09-10T09:58:21.788481+0000","last_unstale":"2024-09-10T09:58:24.738233+0000","last_undegraded":"2024-09-10T09:58:24.738233+0000","last_fullsized":"2024-09-10T09:58:24.738233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738521+0000","last_change":"2024-09-10T09:58:22.801237+0000","last_active":"2024-09-10T09:58:24.738521+0000","last_peered":"2024-09-10T09:58:24.738521+0000","last_clean":"2024-09-10T09:58:24.738521+0000","last_became_active":"2024-09-10T09:58:22.800975+0000","last_became_peered":"2024-09-10T09:58:22.800975+0000","last_unstale":"2024-09-10T09:58:24.738521+0000","last_undegraded":"2024-09-10T09:58:24.738521+0000","last_fullsized":"2024-09-10T09:58:24.738521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:26.981758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772229+0000","last_change":"2024-09-10T09:58:21.778738+0000","last_active":"2024-09-10T09:58:23.772229+0000","last_peered":"2024-09-10T09:58:23.772229+0000","last_clean":"2024-09-10T09:58:23.772229+0000","last_became_active":"2024-09-10T09:58:21.777982+0000","last_became_peered":"2024-09-10T09:58:21.777982+0000","last_unstale":"2024-09-10T09:58:23.772229+0000","last_undegraded":"2024-09-10T09:58:23.772229+0000","last_fullsized":"2024-09-10T09:58:23.772229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772461+0000","last_change":"2024-09-10T09:58:22.775403+0000","last_active":"2024-09-10T09:58:23.772461+0000","last_peered":"2024-09-10T09:58:23.772461+0000","last_clean":"2024-09-10T09:58:23.772461+0000","last_became_active":"2024-09-10T09:58:22.775043+0000","last_became_peered":"2024-09-10T09:58:22.775043+0000","last_unstale":"2024-09-10T09:58:23.772461+0000","last_undegraded":"2024-09-10T09:58:23.772461+0000","last_fullsized":"2024-09-10T09:58:23.772461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770292+0000","last_change":"2024-09-10T09:58:21.779750+0000","last_active":"2024-09-10T09:58:23.770292+0000","last_peered":"2024-09-10T09:58:23.770292+0000","last_clean":"2024-09-10T09:58:23.770292+0000","last_became_active":"2024-09-10T09:58:21.779492+0000","last_became_peered":"2024-09-10T09:58:21.779492+0000","last_unstale":"2024-09-10T09:58:23.770292+0000","last_undegraded":"2024-09-10T09:58:23.770292+0000","last_fullsized":"2024-09-10T09:58:23.770292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737852+0000","last_change":"2024-09-10T09:58:22.801768+0000","last_active":"2024-09-10T09:58:24.737852+0000","last_peered":"2024-09-10T09:58:24.737852+0000","last_clean":"2024-09-10T09:58:24.737852+0000","last_became_active":"2024-09-10T09:58:22.801503+0000","last_became_peered":"2024-09-10T09:58:22.801503+0000","last_unstale":"2024-09-10T09:58:24.737852+0000","last_undegraded":"2024-09-10T09:58:24.737852+0000","last_fullsized":"2024-09-10T09:58:24.737852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:12.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772798+0000","last_change":"2024-09-10T09:58:21.781792+0000","last_active":"2024-09-10T09:58:23.772798+0000","last_peered":"2024-09-10T09:58:23.772798+0000","last_clean":"2024-09-10T09:58:23.772798+0000","last_became_active":"2024-09-10T09:58:21.781434+0000","last_became_peered":"2024-09-10T09:58:21.781434+0000","last_unstale":"2024-09-10T09:58:23.772798+0000","last_undegraded":"2024-09-10T09:58:23.772798+0000","last_fullsized":"2024-09-10T09:58:23.772798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738537+0000","last_change":"2024-09-10T09:58:22.804739+0000","last_active":"2024-09-10T09:58:24.738537+0000","last_peered":"2024-09-10T09:58:24.738537+0000","last_clean":"2024-09-10T09:58:24.738537+0000","last_became_active":"2024-09-10T09:58:22.804451+0000","last_became_peered":"2024-09-10T09:58:22.804451+0000","last_unstale":"2024-09-10T09:58:24.738537+0000","last_undegraded":"2024-09-10T09:58:24.738537+0000","last_fullsized":"2024-09-10T09:58:24.738537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:39.940923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770561+0000","last_change":"2024-09-10T09:58:21.781287+0000","last_active":"2024-09-10T09:58:23.770561+0000","last_peered":"2024-09-10T09:58:23.770561+0000","last_clean":"2024-09-10T09:58:23.770561+0000","last_became_active":"2024-09-10T09:58:21.780824+0000","last_became_peered":"2024-09-10T09:58:21.780824+0000","last_unstale":"2024-09-10T09:58:23.770561+0000","last_undegraded":"2024-09-10T09:58:23.770561+0000","last_fullsized":"2024-09-10T09:58:23.770561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:00.039731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739601+0000","last_change":"2024-09-10T09:58:22.802541+0000","last_active":"2024-09-10T09:58:24.739601+0000","last_peered":"2024-09-10T09:58:24.739601+0000","last_clean":"2024-09-10T09:58:24.739601+0000","last_became_active":"2024-09-10T09:58:22.802216+0000","last_became_peered":"2024-09-10T09:58:22.802216+0000","last_unstale":"2024-09-10T09:58:24.739601+0000","last_undegraded":"2024-09-10T09:58:24.739601+0000","last_fullsized":"2024-09-10T09:58:24.739601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:56.685285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782186+0000","last_change":"2024-09-10T09:58:21.776106+0000","last_active":"2024-09-10T09:58:23.782186+0000","last_peered":"2024-09-10T09:58:23.782186+0000","last_clean":"2024-09-10T09:58:23.782186+0000","last_became_active":"2024-09-10T09:58:21.773315+0000","last_became_peered":"2024-09-10T09:58:21.773315+0000","last_unstale":"2024-09-10T09:58:23.782186+0000","last_undegraded":"2024-09-10T09:58:23.782186+0000","last_fullsized":"2024-09-10T09:58:23.782186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738418+0000","last_change":"2024-09-10T09:58:22.779675+0000","last_active":"2024-09-10T09:58:24.738418+0000","last_peered":"2024-09-10T09:58:24.738418+0000","last_clean":"2024-09-10T09:58:24.738418+0000","last_became_active":"2024-09-10T09:58:22.779088+0000","last_became_peered":"2024-09-10T09:58:22.779088+0000","last_unstale":"2024-09-10T09:58:24.738418+0000","last_undegraded":"2024-09-10T09:58:24.738418+0000","last_fullsized":"2024-09-10T09:58:24.738418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:26.981758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770459+0000","last_change":"2024-09-10T09:58:21.789986+0000","last_active":"2024-09-10T09:58:23.770459+0000","last_peered":"2024-09-10T09:58:23.770459+0000","last_clean":"2024-09-10T09:58:23.770459+0000","last_became_active":"2024-09-10T09:58:21.789577+0000","last_became_peered":"2024-09-10T09:58:21.789577+0000","last_unstale":"2024-09-10T09:58:23.770459+0000","last_undegraded":"2024-09-10T09:58:23.770459+0000","last_fullsized":"2024-09-10T09:58:23.770459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738630+0000","last_change":"2024-09-10T09:58:22.806455+0000","last_active":"2024-09-10T09:58:24.738630+0000","last_peered":"2024-09-10T09:58:24.738630+0000","last_clean":"2024-09-10T09:58:24.738630+0000","last_became_active":"2024-09-10T09:58:22.806181+0000","last_became_peered":"2024-09-10T09:58:22.806181+0000","last_unstale":"2024-09-10T09:58:24.738630+0000","last_undegraded":"2024-09-10T09:58:24.738630+0000","last_fullsized":"2024-09-10T09:58:24.738630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:32.590352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739373+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:58:24.739373+0000","last_peered":"2024-09-10T09:58:24.739373+0000","last_clean":"2024-09-10T09:58:24.739373+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:58:24.739373+0000","last_undegraded":"2024-09-10T09:58:24.739373+0000","last_fullsized":"2024-09-10T09:58:24.739373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737745+0000","last_change":"2024-09-10T09:58:21.789884+0000","last_active":"2024-09-10T09:58:24.737745+0000","last_peered":"2024-09-10T09:58:24.737745+0000","last_clean":"2024-09-10T09:58:24.737745+0000","last_became_active":"2024-09-10T09:58:21.788940+0000","last_became_peered":"2024-09-10T09:58:21.788940+0000","last_unstale":"2024-09-10T09:58:24.737745+0000","last_undegraded":"2024-09-10T09:58:24.737745+0000","last_fullsized":"2024-09-10T09:58:24.737745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737853+0000","last_change":"2024-09-10T09:58:22.792386+0000","last_active":"2024-09-10T09:58:24.737853+0000","last_peered":"2024-09-10T09:58:24.737853+0000","last_clean":"2024-09-10T09:58:24.737853+0000","last_became_active":"2024-09-10T09:58:22.792151+0000","last_became_peered":"2024-09-10T09:58:22.792151+0000","last_unstale":"2024-09-10T09:58:24.737853+0000","last_undegraded":"2024-09-10T09:58:24.737853+0000","last_fullsized":"2024-09-10T09:58:24.737853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.810620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737643+0000","last_change":"2024-09-10T09:58:21.791063+0000","last_active":"2024-09-10T09:58:24.737643+0000","last_peered":"2024-09-10T09:58:24.737643+0000","last_clean":"2024-09-10T09:58:24.737643+0000","last_became_active":"2024-09-10T09:58:21.790796+0000","last_became_peered":"2024-09-10T09:58:21.790796+0000","last_unstale":"2024-09-10T09:58:24.737643+0000","last_undegraded":"2024-09-10T09:58:24.737643+0000","last_fullsized":"2024-09-10T09:58:24.737643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738301+0000","last_change":"2024-09-10T09:58:22.798826+0000","last_active":"2024-09-10T09:58:24.738301+0000","last_peered":"2024-09-10T09:58:24.738301+0000","last_clean":"2024-09-10T09:58:24.738301+0000","last_became_active":"2024-09-10T09:58:22.798497+0000","last_became_peered":"2024-09-10T09:58:22.798497+0000","last_unstale":"2024-09-10T09:58:24.738301+0000","last_undegraded":"2024-09-10T09:58:24.738301+0000","last_fullsized":"2024-09-10T09:58:24.738301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770375+0000","last_change":"2024-09-10T09:58:21.777208+0000","last_active":"2024-09-10T09:58:23.770375+0000","last_peered":"2024-09-10T09:58:23.770375+0000","last_clean":"2024-09-10T09:58:23.770375+0000","last_became_active":"2024-09-10T09:58:21.776820+0000","last_became_peered":"2024-09-10T09:58:21.776820+0000","last_unstale":"2024-09-10T09:58:23.770375+0000","last_undegraded":"2024-09-10T09:58:23.770375+0000","last_fullsized":"2024-09-10T09:58:23.770375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781864+0000","last_change":"2024-09-10T09:58:22.771438+0000","last_active":"2024-09-10T09:58:23.781864+0000","last_peered":"2024-09-10T09:58:23.781864+0000","last_clean":"2024-09-10T09:58:23.781864+0000","last_became_active":"2024-09-10T09:58:22.771255+0000","last_became_peered":"2024-09-10T09:58:22.771255+0000","last_unstale":"2024-09-10T09:58:23.781864+0000","last_undegraded":"2024-09-10T09:58:23.781864+0000","last_fullsized":"2024-09-10T09:58:23.781864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738323+0000","last_change":"2024-09-10T09:58:21.785388+0000","last_active":"2024-09-10T09:58:24.738323+0000","last_peered":"2024-09-10T09:58:24.738323+0000","last_clean":"2024-09-10T09:58:24.738323+0000","last_became_active":"2024-09-10T09:58:21.785227+0000","last_became_peered":"2024-09-10T09:58:21.785227+0000","last_unstale":"2024-09-10T09:58:24.738323+0000","last_undegraded":"2024-09-10T09:58:24.738323+0000","last_fullsized":"2024-09-10T09:58:24.738323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772655+0000","last_change":"2024-09-10T09:58:22.798575+0000","last_active":"2024-09-10T09:58:23.772655+0000","last_peered":"2024-09-10T09:58:23.772655+0000","last_clean":"2024-09-10T09:58:23.772655+0000","last_became_active":"2024-09-10T09:58:22.794489+0000","last_became_peered":"2024-09-10T09:58:22.794489+0000","last_unstale":"2024-09-10T09:58:23.772655+0000","last_undegraded":"2024-09-10T09:58:23.772655+0000","last_fullsized":"2024-09-10T09:58:23.772655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:30.932236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782053+0000","last_change":"2024-09-10T09:58:21.768431+0000","last_active":"2024-09-10T09:58:23.782053+0000","last_peered":"2024-09-10T09:58:23.782053+0000","last_clean":"2024-09-10T09:58:23.782053+0000","last_became_active":"2024-09-10T09:58:21.767826+0000","last_became_peered":"2024-09-10T09:58:21.767826+0000","last_unstale":"2024-09-10T09:58:23.782053+0000","last_undegraded":"2024-09-10T09:58:23.782053+0000","last_fullsized":"2024-09-10T09:58:23.782053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738857+0000","last_change":"2024-09-10T09:58:22.791075+0000","last_active":"2024-09-10T09:58:24.738857+0000","last_peered":"2024-09-10T09:58:24.738857+0000","last_clean":"2024-09-10T09:58:24.738857+0000","last_became_active":"2024-09-10T09:58:22.790778+0000","last_became_peered":"2024-09-10T09:58:22.790778+0000","last_unstale":"2024-09-10T09:58:24.738857+0000","last_undegraded":"2024-09-10T09:58:24.738857+0000","last_fullsized":"2024-09-10T09:58:24.738857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:17.386153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738892+0000","last_change":"2024-09-10T09:58:21.788480+0000","last_active":"2024-09-10T09:58:24.738892+0000","last_peered":"2024-09-10T09:58:24.738892+0000","last_clean":"2024-09-10T09:58:24.738892+0000","last_became_active":"2024-09-10T09:58:21.788162+0000","last_became_peered":"2024-09-10T09:58:21.788162+0000","last_unstale":"2024-09-10T09:58:24.738892+0000","last_undegraded":"2024-09-10T09:58:24.738892+0000","last_fullsized":"2024-09-10T09:58:24.738892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:47.151185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781960+0000","last_change":"2024-09-10T09:58:22.778100+0000","last_active":"2024-09-10T09:58:23.781960+0000","last_peered":"2024-09-10T09:58:23.781960+0000","last_clean":"2024-09-10T09:58:23.781960+0000","last_became_active":"2024-09-10T09:58:22.777764+0000","last_became_peered":"2024-09-10T09:58:22.777764+0000","last_unstale":"2024-09-10T09:58:23.781960+0000","last_undegraded":"2024-09-10T09:58:23.781960+0000","last_fullsized":"2024-09-10T09:58:23.781960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782165+0000","last_change":"2024-09-10T09:58:21.770640+0000","last_active":"2024-09-10T09:58:23.782165+0000","last_peered":"2024-09-10T09:58:23.782165+0000","last_clean":"2024-09-10T09:58:23.782165+0000","last_became_active":"2024-09-10T09:58:21.769111+0000","last_became_peered":"2024-09-10T09:58:21.769111+0000","last_unstale":"2024-09-10T09:58:23.782165+0000","last_undegraded":"2024-09-10T09:58:23.782165+0000","last_fullsized":"2024-09-10T09:58:23.782165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738499+0000","last_change":"2024-09-10T09:58:22.789520+0000","last_active":"2024-09-10T09:58:24.738499+0000","last_peered":"2024-09-10T09:58:24.738499+0000","last_clean":"2024-09-10T09:58:24.738499+0000","last_became_active":"2024-09-10T09:58:22.788875+0000","last_became_peered":"2024-09-10T09:58:22.788875+0000","last_unstale":"2024-09-10T09:58:24.738499+0000","last_undegraded":"2024-09-10T09:58:24.738499+0000","last_fullsized":"2024-09-10T09:58:24.738499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:00.959927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773408+0000","last_change":"2024-09-10T09:58:21.782019+0000","last_active":"2024-09-10T09:58:23.773408+0000","last_peered":"2024-09-10T09:58:23.773408+0000","last_clean":"2024-09-10T09:58:23.773408+0000","last_became_active":"2024-09-10T09:58:21.781750+0000","last_became_peered":"2024-09-10T09:58:21.781750+0000","last_unstale":"2024-09-10T09:58:23.773408+0000","last_undegraded":"2024-09-10T09:58:23.773408+0000","last_fullsized":"2024-09-10T09:58:23.773408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738027+0000","last_change":"2024-09-10T09:58:22.781562+0000","last_active":"2024-09-10T09:58:24.738027+0000","last_peered":"2024-09-10T09:58:24.738027+0000","last_clean":"2024-09-10T09:58:24.738027+0000","last_became_active":"2024-09-10T09:58:22.781242+0000","last_became_peered":"2024-09-10T09:58:22.781242+0000","last_unstale":"2024-09-10T09:58:24.738027+0000","last_undegraded":"2024-09-10T09:58:24.738027+0000","last_fullsized":"2024-09-10T09:58:24.738027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737634+0000","last_change":"2024-09-10T09:58:21.788078+0000","last_active":"2024-09-10T09:58:24.737634+0000","last_peered":"2024-09-10T09:58:24.737634+0000","last_clean":"2024-09-10T09:58:24.737634+0000","last_became_active":"2024-09-10T09:58:21.787660+0000","last_became_peered":"2024-09-10T09:58:21.787660+0000","last_unstale":"2024-09-10T09:58:24.737634+0000","last_undegraded":"2024-09-10T09:58:24.737634+0000","last_fullsized":"2024-09-10T09:58:24.737634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772631+0000","last_change":"2024-09-10T09:58:22.798059+0000","last_active":"2024-09-10T09:58:23.772631+0000","last_peered":"2024-09-10T09:58:23.772631+0000","last_clean":"2024-09-10T09:58:23.772631+0000","last_became_active":"2024-09-10T09:58:22.797346+0000","last_became_peered":"2024-09-10T09:58:22.797346+0000","last_unstale":"2024-09-10T09:58:23.772631+0000","last_undegraded":"2024-09-10T09:58:23.772631+0000","last_fullsized":"2024-09-10T09:58:23.772631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:35.690818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781938+0000","last_change":"2024-09-10T09:58:21.782043+0000","last_active":"2024-09-10T09:58:23.781938+0000","last_peered":"2024-09-10T09:58:23.781938+0000","last_clean":"2024-09-10T09:58:23.781938+0000","last_became_active":"2024-09-10T09:58:21.781762+0000","last_became_peered":"2024-09-10T09:58:21.781762+0000","last_unstale":"2024-09-10T09:58:23.781938+0000","last_undegraded":"2024-09-10T09:58:23.781938+0000","last_fullsized":"2024-09-10T09:58:23.781938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738162+0000","last_change":"2024-09-10T09:58:22.772755+0000","last_active":"2024-09-10T09:58:24.738162+0000","last_peered":"2024-09-10T09:58:24.738162+0000","last_clean":"2024-09-10T09:58:24.738162+0000","last_became_active":"2024-09-10T09:58:22.772505+0000","last_became_peered":"2024-09-10T09:58:22.772505+0000","last_unstale":"2024-09-10T09:58:24.738162+0000","last_undegraded":"2024-09-10T09:58:24.738162+0000","last_fullsized":"2024-09-10T09:58:24.738162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:48.318092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782006+0000","last_change":"2024-09-10T09:58:21.774362+0000","last_active":"2024-09-10T09:58:23.782006+0000","last_peered":"2024-09-10T09:58:23.782006+0000","last_clean":"2024-09-10T09:58:23.782006+0000","last_became_active":"2024-09-10T09:58:21.773133+0000","last_became_peered":"2024-09-10T09:58:21.773133+0000","last_unstale":"2024-09-10T09:58:23.782006+0000","last_undegraded":"2024-09-10T09:58:23.782006+0000","last_fullsized":"2024-09-10T09:58:23.782006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738587+0000","last_change":"2024-09-10T09:58:22.791719+0000","last_active":"2024-09-10T09:58:24.738587+0000","last_peered":"2024-09-10T09:58:24.738587+0000","last_clean":"2024-09-10T09:58:24.738587+0000","last_became_active":"2024-09-10T09:58:22.790615+0000","last_became_peered":"2024-09-10T09:58:22.790615+0000","last_unstale":"2024-09-10T09:58:24.738587+0000","last_undegraded":"2024-09-10T09:58:24.738587+0000","last_fullsized":"2024-09-10T09:58:24.738587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:59.165906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738554+0000","last_change":"2024-09-10T09:58:21.790948+0000","last_active":"2024-09-10T09:58:24.738554+0000","last_peered":"2024-09-10T09:58:24.738554+0000","last_clean":"2024-09-10T09:58:24.738554+0000","last_became_active":"2024-09-10T09:58:21.790543+0000","last_became_peered":"2024-09-10T09:58:21.790543+0000","last_unstale":"2024-09-10T09:58:24.738554+0000","last_undegraded":"2024-09-10T09:58:24.738554+0000","last_fullsized":"2024-09-10T09:58:24.738554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:26.671745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772287+0000","last_change":"2024-09-10T09:58:22.790405+0000","last_active":"2024-09-10T09:58:23.772287+0000","last_peered":"2024-09-10T09:58:23.772287+0000","last_clean":"2024-09-10T09:58:23.772287+0000","last_became_active":"2024-09-10T09:58:22.788732+0000","last_became_peered":"2024-09-10T09:58:22.788732+0000","last_unstale":"2024-09-10T09:58:23.772287+0000","last_undegraded":"2024-09-10T09:58:23.772287+0000","last_fullsized":"2024-09-10T09:58:23.772287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738773+0000","last_change":"2024-09-10T09:58:22.795014+0000","last_active":"2024-09-10T09:58:24.738773+0000","last_peered":"2024-09-10T09:58:24.738773+0000","last_clean":"2024-09-10T09:58:24.738773+0000","last_became_active":"2024-09-10T09:58:22.794752+0000","last_became_peered":"2024-09-10T09:58:22.794752+0000","last_unstale":"2024-09-10T09:58:24.738773+0000","last_undegraded":"2024-09-10T09:58:24.738773+0000","last_fullsized":"2024-09-10T09:58:24.738773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:32.590352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738737+0000","last_change":"2024-09-10T09:58:21.793454+0000","last_active":"2024-09-10T09:58:24.738737+0000","last_peered":"2024-09-10T09:58:24.738737+0000","last_clean":"2024-09-10T09:58:24.738737+0000","last_became_active":"2024-09-10T09:58:21.793058+0000","last_became_peered":"2024-09-10T09:58:21.793058+0000","last_unstale":"2024-09-10T09:58:24.738737+0000","last_undegraded":"2024-09-10T09:58:24.738737+0000","last_fullsized":"2024-09-10T09:58:24.738737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738316+0000","last_change":"2024-09-10T09:58:22.804585+0000","last_active":"2024-09-10T09:58:24.738316+0000","last_peered":"2024-09-10T09:58:24.738316+0000","last_clean":"2024-09-10T09:58:24.738316+0000","last_became_active":"2024-09-10T09:58:22.804350+0000","last_became_peered":"2024-09-10T09:58:22.804350+0000","last_unstale":"2024-09-10T09:58:24.738316+0000","last_undegraded":"2024-09-10T09:58:24.738316+0000","last_fullsized":"2024-09-10T09:58:24.738316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:26.963772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738735+0000","last_change":"2024-09-10T09:58:21.789114+0000","last_active":"2024-09-10T09:58:24.738735+0000","last_peered":"2024-09-10T09:58:24.738735+0000","last_clean":"2024-09-10T09:58:24.738735+0000","last_became_active":"2024-09-10T09:58:21.788849+0000","last_became_peered":"2024-09-10T09:58:21.788849+0000","last_unstale":"2024-09-10T09:58:24.738735+0000","last_undegraded":"2024-09-10T09:58:24.738735+0000","last_fullsized":"2024-09-10T09:58:24.738735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737991+0000","last_change":"2024-09-10T09:58:22.792540+0000","last_active":"2024-09-10T09:58:24.737991+0000","last_peered":"2024-09-10T09:58:24.737991+0000","last_clean":"2024-09-10T09:58:24.737991+0000","last_became_active":"2024-09-10T09:58:22.792265+0000","last_became_peered":"2024-09-10T09:58:22.792265+0000","last_unstale":"2024-09-10T09:58:24.737991+0000","last_undegraded":"2024-09-10T09:58:24.737991+0000","last_fullsized":"2024-09-10T09:58:24.737991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738067+0000","last_change":"2024-09-10T09:58:21.779788+0000","last_active":"2024-09-10T09:58:24.738067+0000","last_peered":"2024-09-10T09:58:24.738067+0000","last_clean":"2024-09-10T09:58:24.738067+0000","last_became_active":"2024-09-10T09:58:21.779521+0000","last_became_peered":"2024-09-10T09:58:21.779521+0000","last_unstale":"2024-09-10T09:58:24.738067+0000","last_undegraded":"2024-09-10T09:58:24.738067+0000","last_fullsized":"2024-09-10T09:58:24.738067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:33.355227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737993+0000","last_change":"2024-09-10T09:58:22.777347+0000","last_active":"2024-09-10T09:58:24.737993+0000","last_peered":"2024-09-10T09:58:24.737993+0000","last_clean":"2024-09-10T09:58:24.737993+0000","last_became_active":"2024-09-10T09:58:22.777042+0000","last_became_peered":"2024-09-10T09:58:22.777042+0000","last_unstale":"2024-09-10T09:58:24.737993+0000","last_undegraded":"2024-09-10T09:58:24.737993+0000","last_fullsized":"2024-09-10T09:58:24.737993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:31.068726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738287+0000","last_change":"2024-09-10T09:58:21.787400+0000","last_active":"2024-09-10T09:58:24.738287+0000","last_peered":"2024-09-10T09:58:24.738287+0000","last_clean":"2024-09-10T09:58:24.738287+0000","last_became_active":"2024-09-10T09:58:21.787111+0000","last_became_peered":"2024-09-10T09:58:21.787111+0000","last_unstale":"2024-09-10T09:58:24.738287+0000","last_undegraded":"2024-09-10T09:58:24.738287+0000","last_fullsized":"2024-09-10T09:58:24.738287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":671905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":81081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":80955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":241664,"data_stored":81115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:58:31.711 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T09:58:31.711 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T09:58:31.711 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T09:58:31.711 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T09:58:31.712 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T09:58:31.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.885+0000 7f01b8200640 1 Processor -- start 2024-09-10T09:58:31.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.885+0000 7f01b8200640 1 -- start start 2024-09-10T09:58:31.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b8200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f01b00cfe30 0x7f01b0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b8200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b006d2d0 con 0x7f01b006e7a0 2024-09-10T09:58:31.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b8200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b006d5b0 con 0x7f01b00cd510 2024-09-10T09:58:31.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b8200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f01b015d9a0 con 0x7f01b00cfe30 2024-09-10T09:58:31.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b6200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f01b006e7a0 0x7f01b015cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38154/0 (socket says 172.21.15.110:38154) 2024-09-10T09:58:31.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b6200640 1 -- 172.21.15.110:0/3031863776 learned_addr learned my addr 172.21.15.110:0/3031863776 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:31.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b5800640 1 --2- 172.21.15.110:0/3031863776 >> v2:172.21.15.145:3300/0 conn(0x7f01b00cfe30 0x7f01b0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01afe00640 1 -- 172.21.15.110:0/3031863776 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884882446 0 0) 0x7f01b006d2d0 con 0x7f01b006e7a0 2024-09-10T09:58:31.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01afe00640 1 -- 172.21.15.110:0/3031863776 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0198003530 con 0x7f01b006e7a0 2024-09-10T09:58:31.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b5800640 1 -- 172.21.15.110:0/3031863776 >> v1:172.21.15.145:6789/0 conn(0x7f01b00cd510 legacy=0x7f01b006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b5800640 1 -- 172.21.15.110:0/3031863776 >> v1:172.21.15.110:6789/0 conn(0x7f01b006e7a0 legacy=0x7f01b015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.889+0000 7f01b5800640 1 -- 172.21.15.110:0/3031863776 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b006d890 con 0x7f01b00cfe30 2024-09-10T09:58:31.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b5800640 1 --2- 172.21.15.110:0/3031863776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cfe30 0x7f01b0071a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009d90 tx=0x7f01a401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950f6f04fc480ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01afe00640 1 -- 172.21.15.110:0/3031863776 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40b2070 con 0x7f01b00cfe30 2024-09-10T09:58:31.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01afe00640 1 -- 172.21.15.110:0/3031863776 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a40a58b0 con 0x7f01b00cfe30 2024-09-10T09:58:31.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01afe00640 1 -- 172.21.15.110:0/3031863776 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40a5c60 con 0x7f01b00cfe30 2024-09-10T09:58:31.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- 172.21.15.110:0/3031863776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cfe30 msgr2=0x7f01b0071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 --2- 172.21.15.110:0/3031863776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cfe30 0x7f01b0071a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009d90 tx=0x7f01a401c8e0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- 172.21.15.110:0/3031863776 shutdown_connections 2024-09-10T09:58:31.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 --2- 172.21.15.110:0/3031863776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cfe30 0x7f01b0071a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- 172.21.15.110:0/3031863776 >> 172.21.15.110:0/3031863776 conn(0x7f01b006bc50 msgr2=0x7f01b006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:31.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- 172.21.15.110:0/3031863776 shutdown_connections 2024-09-10T09:58:31.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- 172.21.15.110:0/3031863776 wait complete. 2024-09-10T09:58:31.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 Processor -- start 2024-09-10T09:58:31.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- start start 2024-09-10T09:58:31.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01b006e7a0 0x7f01b00b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cd510 0x7f01b00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00cfe30 0x7f01b00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f01b015d740 con 0x7f01b00cfe30 2024-09-10T09:58:31.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01b0072190 con 0x7f01b00cd510 2024-09-10T09:58:31.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.893+0000 7f01b8200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f01b016abb0 con 0x7f01b006e7a0 2024-09-10T09:58:31.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00cfe30 0x7f01b00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00cfe30 0x7f01b00ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53048/0 (socket says 172.21.15.110:53048) 2024-09-10T09:58:31.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 -- 172.21.15.110:0/1679972415 learned_addr learned my addr 172.21.15.110:0/1679972415 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:31.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b5800640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01b006e7a0 0x7f01b00b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b4e00640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cd510 0x7f01b00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 -- 172.21.15.110:0/1679972415 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01b006e7a0 msgr2=0x7f01b00b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01b006e7a0 0x7f01b00b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 -- 172.21.15.110:0/1679972415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cd510 msgr2=0x7f01b00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cd510 0x7f01b00b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 -- 172.21.15.110:0/1679972415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b00b3e20 con 0x7f01b00cfe30 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b6200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00cfe30 0x7f01b00ae840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01a800c3c0 tx=0x7f01a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01ad600640 1 -- 172.21.15.110:0/1679972415 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8017070 con 0x7f01b00cfe30 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01ad600640 1 -- 172.21.15.110:0/1679972415 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8005cf0 con 0x7f01b00cfe30 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01ad600640 1 -- 172.21.15.110:0/1679972415 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8007310 con 0x7f01b00cfe30 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b00b4100 con 0x7f01b00cfe30 2024-09-10T09:58:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b00b49a0 con 0x7f01b00cfe30 2024-09-10T09:58:31.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b00b42d0 con 0x7f01b00cfe30 2024-09-10T09:58:31.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.897+0000 7f01ad600640 1 -- 172.21.15.110:0/1679972415 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f01a80074b0 con 0x7f01b00cfe30 2024-09-10T09:58:31.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.901+0000 7f01ad600640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01900678f0 0x7f0190069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:31.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.901+0000 7f01ad600640 1 -- 172.21.15.110:0/1679972415 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f01a8084ee0 con 0x7f01b00cfe30 2024-09-10T09:58:31.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.901+0000 7f01b5800640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01900678f0 0x7f0190069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:31.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.901+0000 7f01b5800640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01900678f0 0x7f0190069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01a40af000 tx=0x7f01a40a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:31.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:31.905+0000 7f01ad600640 1 -- 172.21.15.110:0/1679972415 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f01a80553c0 con 0x7f01b00cfe30 2024-09-10T09:58:32.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.096+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f01b006d1a0 con 0x7f01b00cfe30 2024-09-10T09:58:32.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01ad600640 1 -- 172.21.15.110:0/1679972415 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166891 (secure 0 0 0) 0x7f01a80596c0 con 0x7f01b00cfe30 2024-09-10T09:58:32.103 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01900678f0 msgr2=0x7f0190069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01900678f0 0x7f0190069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01a40af000 tx=0x7f01a40a7070 comp rx=0 tx=0).stop 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00cfe30 msgr2=0x7f01b00ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00cfe30 0x7f01b00ae840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01a800c3c0 tx=0x7f01a800c880 comp rx=0 tx=0).stop 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 shutdown_connections 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01900678f0 0x7f0190069db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00cfe30 0x7f01b00ae840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01b00cd510 0x7f01b00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 --2- 172.21.15.110:0/1679972415 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01b006e7a0 0x7f01b00b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 >> 172.21.15.110:0/1679972415 conn(0x7f01b006bc50 msgr2=0x7f01b015bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.100+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 shutdown_connections 2024-09-10T09:58:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.104+0000 7f01b8200640 1 -- 172.21.15.110:0/1679972415 wait complete. 2024-09-10T09:58:32.123 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":952833175},{"type":"v1","addr":"172.21.15.110:6833","nonce":952833175}]},"active_addr":"172.21.15.110:6833/952833175","active_change":"2024-09-10T09:58:03.341235+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":2970528962}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3823528576}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3209652493}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3716204142}]}]} 2024-09-10T09:58:32.126 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T09:58:32.126 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T09:58:32.126 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:32.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f1083600640 1 Processor -- start 2024-09-10T09:58:32.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f1083600640 1 -- start start 2024-09-10T09:58:32.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f1083600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f107c0cfe30 0x7f107c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f1083600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f107c0cd510 0x7f107c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f1083600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f107c06d2d0 con 0x7f107c06e090 2024-09-10T09:58:32.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f1083600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f107c15d9a0 con 0x7f107c0cd510 2024-09-10T09:58:32.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f1083600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f107c06d5b0 con 0x7f107c0cfe30 2024-09-10T09:58:32.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f107be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f107c0cd510 0x7f107c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f107be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f107c0cd510 0x7f107c0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53052/0 (socket says 172.21.15.110:53052) 2024-09-10T09:58:32.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f107be00640 1 -- 172.21.15.110:0/1028443967 learned_addr learned my addr 172.21.15.110:0/1028443967 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:32.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f1080c00640 1 --2- 172.21.15.110:0/1028443967 >> v2:172.21.15.145:3300/0 conn(0x7f107c0cfe30 0x7f107c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f107be00640 1 -- 172.21.15.110:0/1028443967 >> v2:172.21.15.145:3300/0 conn(0x7f107c0cfe30 msgr2=0x7f107c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f107be00640 1 --2- 172.21.15.110:0/1028443967 >> v2:172.21.15.145:3300/0 conn(0x7f107c0cfe30 0x7f107c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f107be00640 1 -- 172.21.15.110:0/1028443967 >> v1:172.21.15.145:6789/0 conn(0x7f107c06e090 legacy=0x7f107c06e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.260+0000 7f107be00640 1 -- 172.21.15.110:0/1028443967 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c06d890 con 0x7f107c0cd510 2024-09-10T09:58:32.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f107be00640 1 --2- 172.21.15.110:0/1028443967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c0cd9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f107c0ce120 tx=0x7f106801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e40e57494457d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f107b400640 1 -- 172.21.15.110:0/1028443967 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680b1070 con 0x7f107c0cd510 2024-09-10T09:58:32.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f107b400640 1 -- 172.21.15.110:0/1028443967 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10680ac600 con 0x7f107c0cd510 2024-09-10T09:58:32.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f107b400640 1 -- 172.21.15.110:0/1028443967 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680ac9d0 con 0x7f107c0cd510 2024-09-10T09:58:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f1083600640 1 -- 172.21.15.110:0/1028443967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 msgr2=0x7f107c0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f1083600640 1 --2- 172.21.15.110:0/1028443967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c0cd9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f107c0ce120 tx=0x7f106801c490 comp rx=0 tx=0).stop 2024-09-10T09:58:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f1083600640 1 -- 172.21.15.110:0/1028443967 shutdown_connections 2024-09-10T09:58:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f1083600640 1 --2- 172.21.15.110:0/1028443967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c0cd9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f1083600640 1 --2- 172.21.15.110:0/1028443967 >> v2:172.21.15.145:3300/0 conn(0x7f107c0cfe30 0x7f107c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f1083600640 1 -- 172.21.15.110:0/1028443967 >> 172.21.15.110:0/1028443967 conn(0x7f107c06bc50 msgr2=0x7f107c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f1083600640 1 -- 172.21.15.110:0/1028443967 shutdown_connections 2024-09-10T09:58:32.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.264+0000 7f1083600640 1 -- 172.21.15.110:0/1028443967 wait complete. 2024-09-10T09:58:32.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 Processor -- start 2024-09-10T09:58:32.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 -- start start 2024-09-10T09:58:32.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107c06e090 0x7f107c166820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c1fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107c0cfe30 0x7f107c1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f107c072190 con 0x7f107c0cd510 2024-09-10T09:58:32.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f107c15d6c0 con 0x7f107c06e090 2024-09-10T09:58:32.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f107c169180 con 0x7f107c0cfe30 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c1fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c1fadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53066/0 (socket says 172.21.15.110:53066) 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 -- 172.21.15.110:0/3196572154 learned_addr learned my addr 172.21.15.110:0/3196572154 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1080c00640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107c06e090 0x7f107c166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1081600640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107c0cfe30 0x7f107c1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 -- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107c0cfe30 msgr2=0x7f107c1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107c0cfe30 0x7f107c1572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 -- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107c06e090 msgr2=0x7f107c166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107c06e090 0x7f107c166820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 -- 172.21.15.110:0/3196572154 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c1fb330 con 0x7f107c0cd510 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1081600640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107c0cfe30 0x7f107c1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:32.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f107be00640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c1fadf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f10680190a0 tx=0x7f106801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1078c00640 1 -- 172.21.15.110:0/3196572154 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680b1040 con 0x7f107c0cd510 2024-09-10T09:58:32.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1078c00640 1 -- 172.21.15.110:0/3196572154 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10680bc070 con 0x7f107c0cd510 2024-09-10T09:58:32.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107c1f7d80 con 0x7f107c0cd510 2024-09-10T09:58:32.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1078c00640 1 -- 172.21.15.110:0/3196572154 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680b78b0 con 0x7f107c0cd510 2024-09-10T09:58:32.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107c1f74e0 con 0x7f107c0cd510 2024-09-10T09:58:32.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.268+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107c0b4b30 con 0x7f107c0cd510 2024-09-10T09:58:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.272+0000 7f1078c00640 1 -- 172.21.15.110:0/3196572154 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f10680b7a50 con 0x7f107c0cd510 2024-09-10T09:58:32.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.272+0000 7f1078c00640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f105c0678c0 0x7f105c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.272+0000 7f1078c00640 1 -- 172.21.15.110:0/3196572154 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f106811fde0 con 0x7f107c0cd510 2024-09-10T09:58:32.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.272+0000 7f1080c00640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f105c0678c0 0x7f105c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.272+0000 7f1080c00640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f105c0678c0 0x7f105c069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f107c1564e0 tx=0x7f1070009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.276+0000 7f1078c00640 1 -- 172.21.15.110:0/3196572154 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f107c1fb330 con 0x7f107c0cd510 2024-09-10T09:58:32.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f107c0719c0 con 0x7f107c0cd510 2024-09-10T09:58:32.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1078c00640 1 -- 172.21.15.110:0/3196572154 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16701 (secure 0 0 0) 0x7f1068016110 con 0x7f107c0cd510 2024-09-10T09:58:32.438 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:32.438 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:23.766388+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:32.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f105c0678c0 msgr2=0x7f105c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f105c0678c0 0x7f105c069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f107c1564e0 tx=0x7f1070009190 comp rx=0 tx=0).stop 2024-09-10T09:58:32.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 msgr2=0x7f107c1fadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c1fadf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f10680190a0 tx=0x7f106801acd0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 shutdown_connections 2024-09-10T09:58:32.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f105c0678c0 0x7f105c069d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107c0cfe30 0x7f107c1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f107c0cd510 0x7f107c1fadf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 --2- 172.21.15.110:0/3196572154 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107c06e090 0x7f107c166820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 >> 172.21.15.110:0/3196572154 conn(0x7f107c06bc50 msgr2=0x7f107c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 shutdown_connections 2024-09-10T09:58:32.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.436+0000 7f1083600640 1 -- 172.21.15.110:0/3196572154 wait complete. 2024-09-10T09:58:32.460 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T09:58:32.460 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:32.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4643000640 1 Processor -- start 2024-09-10T09:58:32.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4643000640 1 -- start start 2024-09-10T09:58:32.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4643000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f463c1625e0 0x7f463c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4643000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c06c360 0x7f463c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4643000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463c15f0e0 con 0x7f463c1629c0 2024-09-10T09:58:32.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4643000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f463c1419d0 con 0x7f463c1625e0 2024-09-10T09:58:32.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4643000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f463c0b3b10 con 0x7f463c06c360 2024-09-10T09:58:32.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4633400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f463c1629c0 0x7f463c06bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38168/0 (socket says 172.21.15.110:38168) 2024-09-10T09:58:32.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4633400640 1 -- 172.21.15.110:0/2075429312 learned_addr learned my addr 172.21.15.110:0/2075429312 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:32.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f463be00640 1 --2- 172.21.15.110:0/2075429312 >> v2:172.21.15.145:3300/0 conn(0x7f463c1625e0 0x7f463c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f4641000640 1 --2- 172.21.15.110:0/2075429312 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c06c360 0x7f463c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f463b400640 1 -- 172.21.15.110:0/2075429312 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963990128 0 0) 0x7f463c15f0e0 con 0x7f463c1629c0 2024-09-10T09:58:32.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.592+0000 7f463b400640 1 -- 172.21.15.110:0/2075429312 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4620003530 con 0x7f463c1629c0 2024-09-10T09:58:32.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f463be00640 1 -- 172.21.15.110:0/2075429312 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c06c360 msgr2=0x7f463c0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f463be00640 1 --2- 172.21.15.110:0/2075429312 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c06c360 0x7f463c0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f463be00640 1 -- 172.21.15.110:0/2075429312 >> v1:172.21.15.110:6789/0 conn(0x7f463c1629c0 legacy=0x7f463c06bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f463be00640 1 -- 172.21.15.110:0/2075429312 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c15f3c0 con 0x7f463c1625e0 2024-09-10T09:58:32.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f463be00640 1 --2- 172.21.15.110:0/2075429312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c1625e0 0x7f463c15eb00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f462c009d90 tx=0x7f462c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0960337cf065f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f463b400640 1 -- 172.21.15.110:0/2075429312 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0b2070 con 0x7f463c1625e0 2024-09-10T09:58:32.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f463b400640 1 -- 172.21.15.110:0/2075429312 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462c0a5920 con 0x7f463c1625e0 2024-09-10T09:58:32.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f463b400640 1 -- 172.21.15.110:0/2075429312 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0a5cd0 con 0x7f463c1625e0 2024-09-10T09:58:32.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f4643000640 1 -- 172.21.15.110:0/2075429312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c1625e0 msgr2=0x7f463c15eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f4643000640 1 --2- 172.21.15.110:0/2075429312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c1625e0 0x7f463c15eb00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f462c009d90 tx=0x7f462c01c970 comp rx=0 tx=0).stop 2024-09-10T09:58:32.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f4643000640 1 -- 172.21.15.110:0/2075429312 shutdown_connections 2024-09-10T09:58:32.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f4643000640 1 --2- 172.21.15.110:0/2075429312 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c06c360 0x7f463c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f4643000640 1 --2- 172.21.15.110:0/2075429312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c1625e0 0x7f463c15eb00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.596+0000 7f4643000640 1 -- 172.21.15.110:0/2075429312 >> 172.21.15.110:0/2075429312 conn(0x7f463c0b24e0 msgr2=0x7f463c0b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 -- 172.21.15.110:0/2075429312 shutdown_connections 2024-09-10T09:58:32.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 -- 172.21.15.110:0/2075429312 wait complete. 2024-09-10T09:58:32.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 Processor -- start 2024-09-10T09:58:32.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 -- start start 2024-09-10T09:58:32.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c06c360 0x7f463c13f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f463c1625e0 0x7f463c1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c1629c0 0x7f463c13b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f463c0afdb0 con 0x7f463c1625e0 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f463c0afc30 con 0x7f463c06c360 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f463c0aff30 con 0x7f463c1629c0 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f463c1625e0 0x7f463c1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f463c1625e0 0x7f463c1562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53080/0 (socket says 172.21.15.110:53080) 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 -- 172.21.15.110:0/1401654662 learned_addr learned my addr 172.21.15.110:0/1401654662 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f463be00640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c06c360 0x7f463c13f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4641000640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c1629c0 0x7f463c13b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 -- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c1629c0 msgr2=0x7f463c13b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c1629c0 0x7f463c13b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 -- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c06c360 msgr2=0x7f463c13f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c06c360 0x7f463c13f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 -- 172.21.15.110:0/1401654662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c156b30 con 0x7f463c1625e0 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4641000640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c1629c0 0x7f463c13b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4633400640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f463c1625e0 0x7f463c1562e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f462400c450 tx=0x7f462400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4638c00640 1 -- 172.21.15.110:0/1401654662 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624017070 con 0x7f463c1625e0 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4638c00640 1 -- 172.21.15.110:0/1401654662 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4624005cf0 con 0x7f463c1625e0 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.600+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c156e10 con 0x7f463c1625e0 2024-09-10T09:58:32.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.604+0000 7f4638c00640 1 -- 172.21.15.110:0/1401654662 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624007290 con 0x7f463c1625e0 2024-09-10T09:58:32.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.604+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c0ce980 con 0x7f463c1625e0 2024-09-10T09:58:32.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.604+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c0aecd0 con 0x7f463c1625e0 2024-09-10T09:58:32.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.604+0000 7f4638c00640 1 -- 172.21.15.110:0/1401654662 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f462401a070 con 0x7f463c1625e0 2024-09-10T09:58:32.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.604+0000 7f4638c00640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46180678c0 0x7f4618069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.604+0000 7f4638c00640 1 -- 172.21.15.110:0/1401654662 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f46240850e0 con 0x7f463c1625e0 2024-09-10T09:58:32.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.604+0000 7f463be00640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46180678c0 0x7f4618069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.604+0000 7f463be00640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46180678c0 0x7f4618069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f462c0af000 tx=0x7f462c0a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.608+0000 7f4638c00640 1 -- 172.21.15.110:0/1401654662 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4624055b80 con 0x7f463c1625e0 2024-09-10T09:58:32.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.768+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f463c0ccaf0 con 0x7f463c1625e0 2024-09-10T09:58:32.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4638c00640 1 -- 172.21.15.110:0/1401654662 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16701 (secure 0 0 0) 0x7f4624055480 con 0x7f463c1625e0 2024-09-10T09:58:32.773 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:32.773 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:23.766388+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:32.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46180678c0 msgr2=0x7f4618069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46180678c0 0x7f4618069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f462c0af000 tx=0x7f462c0a7070 comp rx=0 tx=0).stop 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f463c1625e0 msgr2=0x7f463c1562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f463c1625e0 0x7f463c1562e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f462400c450 tx=0x7f462400c910 comp rx=0 tx=0).stop 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 shutdown_connections 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46180678c0 0x7f4618069d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f463c1629c0 0x7f463c13b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f463c1625e0 0x7f463c1562e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 --2- 172.21.15.110:0/1401654662 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f463c06c360 0x7f463c13f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 >> 172.21.15.110:0/1401654662 conn(0x7f463c0b24e0 msgr2=0x7f463c168360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 shutdown_connections 2024-09-10T09:58:32.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:32.772+0000 7f4643000640 1 -- 172.21.15.110:0/1401654662 wait complete. 2024-09-10T09:58:32.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T09:58:32.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T09:58:32.795 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T09:58:32.795 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T09:58:32.796 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T09:58:32.796 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T09:58:32.797 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T09:58:32.797 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T09:58:33.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.032+0000 7f0026000640 1 Processor -- start 2024-09-10T09:58:33.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.032+0000 7f0026000640 1 -- start start 2024-09-10T09:58:33.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0041e00640 1 Processor -- start 2024-09-10T09:58:33.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0041e00640 1 -- start start 2024-09-10T09:58:33.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0026000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f002006e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0026000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f002006d6d0 con 0x7f00201602c0 2024-09-10T09:58:33.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0026000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f002006d9b0 con 0x7f0020072230 2024-09-10T09:58:33.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0026000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f00200cd020 con 0x7f00200cff50 2024-09-10T09:58:33.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0024c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f00201602c0 0x7f00201637c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38176/0 (socket says 172.21.15.110:38176) 2024-09-10T09:58:33.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0024c00640 1 -- 172.21.15.110:0/800334597 learned_addr learned my addr 172.21.15.110:0/800334597 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f0041e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f003c0d0fe0 0x7f003c15f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f001fe00640 1 --2- 172.21.15.110:0/800334597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f002006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f001f400640 1 -- 172.21.15.110:0/800334597 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 34454470 0 0) 0x7f002006d6d0 con 0x7f00201602c0 2024-09-10T09:58:33.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f001f400640 1 -- 172.21.15.110:0/800334597 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0008003530 con 0x7f00201602c0 2024-09-10T09:58:33.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f001f400640 1 -- 172.21.15.110:0/800334597 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653989890 0 0) 0x7f002006d9b0 con 0x7f0020072230 2024-09-10T09:58:33.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f001f400640 1 -- 172.21.15.110:0/800334597 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f002006d6d0 con 0x7f0020072230 2024-09-10T09:58:33.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f001fe00640 1 -- 172.21.15.110:0/800334597 >> v1:172.21.15.145:6789/0 conn(0x7f0020072230 legacy=0x7f00200b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f001fe00640 1 -- 172.21.15.110:0/800334597 >> v1:172.21.15.110:6789/0 conn(0x7f00201602c0 legacy=0x7f00201637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f001fe00640 1 -- 172.21.15.110:0/800334597 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002006dc90 con 0x7f00200cff50 2024-09-10T09:58:33.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f003a000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f003c073fb0 0x7f003c0d2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38192/0 (socket says 172.21.15.110:38192) 2024-09-10T09:58:33.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f003a000640 1 -- 172.21.15.110:0/2463406020 learned_addr learned my addr 172.21.15.110:0/2463406020 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f003aa00640 1 --2- 172.21.15.110:0/2463406020 >> v2:172.21.15.110:3300/0 conn(0x7f003c0d0fe0 0x7f003c15f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f0041e00640 1 -- 172.21.15.110:0/2463406020 --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003c0d13c0 con 0x7f003c073fb0 2024-09-10T09:58:33.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f0041e00640 1 -- 172.21.15.110:0/2463406020 --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003c073430 con 0x7f003c0d2770 2024-09-10T09:58:33.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f0041e00640 1 -- 172.21.15.110:0/2463406020 --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f003c15f1f0 con 0x7f003c0d0fe0 2024-09-10T09:58:33.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f003aa00640 1 -- 172.21.15.110:0/2463406020 >> v1:172.21.15.145:6789/0 conn(0x7f003c0d2770 legacy=0x7f003c06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f003aa00640 1 -- 172.21.15.110:0/2463406020 >> v1:172.21.15.110:6789/0 conn(0x7f003c073fb0 legacy=0x7f003c0d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.040+0000 7f003aa00640 1 -- 172.21.15.110:0/2463406020 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c073710 con 0x7f003c0d0fe0 2024-09-10T09:58:33.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.036+0000 7f4c4e400640 1 Processor -- start 2024-09-10T09:58:33.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4e400640 1 -- start start 2024-09-10T09:58:33.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f001fe00640 1 --2- 172.21.15.110:0/800334597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f002006e770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f00140097f0 tx=0x7f001401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12586f498e831c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f036a600640 1 Processor -- start 2024-09-10T09:58:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f001f400640 1 -- 172.21.15.110:0/800334597 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140a6040 con 0x7f00200cff50 2024-09-10T09:58:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f001f400640 1 -- 172.21.15.110:0/800334597 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00140af070 con 0x7f00200cff50 2024-09-10T09:58:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f001f400640 1 -- 172.21.15.110:0/800334597 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140aa860 con 0x7f00200cff50 2024-09-10T09:58:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4e400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4c400e2180 0x7f4c4000a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4e400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c40052710 0x7f4c40007250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4e400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4003a7c0 con 0x7f4c40004ce0 2024-09-10T09:58:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4e400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4c4000a700 con 0x7f4c400e2180 2024-09-10T09:58:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4e400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4c4003a290 con 0x7f4c40052710 2024-09-10T09:58:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0026000640 1 -- 172.21.15.110:0/800334597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 msgr2=0x7f002006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0026000640 1 --2- 172.21.15.110:0/800334597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f002006e770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f00140097f0 tx=0x7f001401c4e0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c47400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f4c40004ce0 0x7f4c4000bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38194/0 (socket says 172.21.15.110:38194) 2024-09-10T09:58:33.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c47400640 1 -- 172.21.15.110:0/3324099450 learned_addr learned my addr 172.21.15.110:0/3324099450 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4ce00640 1 --2- 172.21.15.110:0/3324099450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c40052710 0x7f4c40007250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f036a600640 1 -- start start 2024-09-10T09:58:33.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f036a600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 0x7f03640d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f036a600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03640d2d10 con 0x7f03640d1650 2024-09-10T09:58:33.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f036a600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03640d2ff0 con 0x7f0364073910 2024-09-10T09:58:33.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f036a600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f03640d0630 con 0x7f036415c570 2024-09-10T09:58:33.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c46a00640 1 --2- 172.21.15.110:0/3324099450 >> v2:172.21.15.110:3300/0 conn(0x7f4c400e2180 0x7f4c4000a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0026000640 1 -- 172.21.15.110:0/800334597 shutdown_connections 2024-09-10T09:58:33.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0026000640 1 --2- 172.21.15.110:0/800334597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f002006e770 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0026000640 1 -- 172.21.15.110:0/800334597 >> 172.21.15.110:0/800334597 conn(0x7f002006c050 msgr2=0x7f002006c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0363e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f0364073910 0x7f03640b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:46878/0 (socket says 172.21.15.110:46878) 2024-09-10T09:58:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0026000640 1 -- 172.21.15.110:0/800334597 shutdown_connections 2024-09-10T09:58:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0026000640 1 -- 172.21.15.110:0/800334597 wait complete. 2024-09-10T09:58:33.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0363e00640 1 -- 172.21.15.110:0/230425594 learned_addr learned my addr 172.21.15.110:0/230425594 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f0363400640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 0x7f03640d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f003aa00640 1 --2- 172.21.15.110:0/2463406020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c0d0fe0 0x7f003c15f680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f003c15f370 tx=0x7f002c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d06555c01b0f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0039600640 1 -- 172.21.15.110:0/2463406020 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0b0070 con 0x7f003c0d0fe0 2024-09-10T09:58:33.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345642027 0 0) 0x7f03640d2ff0 con 0x7f0364073910 2024-09-10T09:58:33.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0348003530 con 0x7f0364073910 2024-09-10T09:58:33.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2bf000640 1 Processor -- start 2024-09-10T09:58:33.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0039600640 1 -- 172.21.15.110:0/2463406020 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c0ab600 con 0x7f003c0d0fe0 2024-09-10T09:58:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334355962 0 0) 0x7f03640d2d10 con 0x7f03640d1650 2024-09-10T09:58:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03640d2ff0 con 0x7f03640d1650 2024-09-10T09:58:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 <== mon.1 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1452068449 0 0) 0x7f0348003530 con 0x7f0364073910 2024-09-10T09:58:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 msgr2=0x7f03640d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 0x7f03640d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 >> v1:172.21.15.110:6789/0 conn(0x7f03640d1650 legacy=0x7f0364073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03640d0910 con 0x7f0364073910 2024-09-10T09:58:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0039600640 1 -- 172.21.15.110:0/2463406020 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0ab9d0 con 0x7f003c0d0fe0 2024-09-10T09:58:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4ce00640 1 -- 172.21.15.110:0/3324099450 >> v2:172.21.15.110:3300/0 conn(0x7f4c400e2180 msgr2=0x7f4c4000a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4ce00640 1 --2- 172.21.15.110:0/3324099450 >> v2:172.21.15.110:3300/0 conn(0x7f4c400e2180 0x7f4c4000a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4ce00640 1 -- 172.21.15.110:0/3324099450 >> v1:172.21.15.110:6789/0 conn(0x7f4c40004ce0 legacy=0x7f4c4000bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.044+0000 7f4c4ce00640 1 -- 172.21.15.110:0/3324099450 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c40055480 con 0x7f4c40052710 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 <== mon.1 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f035c001980 con 0x7f0364073910 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 >> v1:172.21.15.145:6789/0 conn(0x7f0364073910 legacy=0x7f03640b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0348003b50 0x7f0348003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348004810 0x7f0348004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 0x7f03480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03640d0630 con 0x7f0348003b50 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03480079b0 con 0x7f0348004810 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0348007b80 con 0x7f036415c570 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2bf000640 1 -- start start 2024-09-10T09:58:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2bf000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 0x7fc2b0009110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2bf000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc2b0038950 0x7fc2b00512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0041e00640 1 -- 172.21.15.110:0/2463406020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c0d0fe0 msgr2=0x7f003c15f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0041e00640 1 --2- 172.21.15.110:0/2463406020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c0d0fe0 0x7f003c15f680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f003c15f370 tx=0x7f002c01c3e0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2bf000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2b000bec0 con 0x7fc2b0057350 2024-09-10T09:58:33.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc1d7400640 1 Processor -- start 2024-09-10T09:58:33.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2bf000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b0050e10 con 0x7fc2b0038950 2024-09-10T09:58:33.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2bf000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b000c1a0 con 0x7fc2b0035b90 2024-09-10T09:58:33.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2be600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc2b0038950 0x7fc2b00512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc1d7400640 1 -- start start 2024-09-10T09:58:33.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0026000640 1 Processor -- start 2024-09-10T09:58:33.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2be600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc2b0038950 0x7fc2b00512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:40998/0 (socket says 172.21.15.110:40998) 2024-09-10T09:58:33.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0363e00640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 0x7f03480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0026000640 1 -- start start 2024-09-10T09:58:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2be600640 1 -- 172.21.15.110:0/1584561794 learned_addr learned my addr 172.21.15.110:0/1584561794 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2be600640 1 -- 172.21.15.110:0/1584561794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 msgr2=0x7fc2b0009110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2be600640 1 --2- 172.21.15.110:0/1584561794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 0x7fc2b0009110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2be600640 1 -- 172.21.15.110:0/1584561794 >> v1:172.21.15.110:6789/0 conn(0x7fc2b0057350 legacy=0x7fc2b0038240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7fc2be600640 1 -- 172.21.15.110:0/1584561794 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b000c480 con 0x7fc2b0038950 2024-09-10T09:58:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.048+0000 7f0362a00640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348004810 0x7f0348004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0363e00640 1 -- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348004810 msgr2=0x7f0348004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0363e00640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348004810 0x7f0348004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0363e00640 1 -- 172.21.15.110:0/230425594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0348003b50 msgr2=0x7f0348003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0363e00640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0348003b50 0x7f0348003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0363e00640 1 -- 172.21.15.110:0/230425594 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03640d0910 con 0x7f036415c570 2024-09-10T09:58:33.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c4ce00640 1 --2- 172.21.15.110:0/3324099450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c40052710 0x7f4c40007250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4c340097c0 tx=0x7f4c3401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b15f504f0bcbca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0362a00640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348004810 0x7f0348004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:33.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0026000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0020072230 0x7f00201663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0026000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f0020153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0026000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00201602c0 0x7f0020157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0026000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0020160820 con 0x7f00201602c0 2024-09-10T09:58:33.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0026000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f00201606a0 con 0x7f0020072230 2024-09-10T09:58:33.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0026000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f002006ecb0 con 0x7f00200cff50 2024-09-10T09:58:33.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c46000640 1 -- 172.21.15.110:0/3324099450 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c340b0070 con 0x7f4c40052710 2024-09-10T09:58:33.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c46000640 1 -- 172.21.15.110:0/3324099450 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c340ab600 con 0x7f4c40052710 2024-09-10T09:58:33.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1d7400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc1d00aeaa0 0x7fc1d0071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1d7400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d00aee80 0x7fc1d006e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0041e00640 1 -- 172.21.15.110:0/2463406020 shutdown_connections 2024-09-10T09:58:33.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0041e00640 1 --2- 172.21.15.110:0/2463406020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c0d0fe0 0x7f003c15f680 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0041e00640 1 -- 172.21.15.110:0/2463406020 >> 172.21.15.110:0/2463406020 conn(0x7f003c06c050 msgr2=0x7f003c0626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1d7400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d006f340 con 0x7fc1d006ed30 2024-09-10T09:58:33.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1d7400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d0071c70 con 0x7fc1d00aeaa0 2024-09-10T09:58:33.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1d7400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d006f620 con 0x7fc1d00aee80 2024-09-10T09:58:33.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1d5400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc1d006ed30 0x7fc1d0146d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:46890/0 (socket says 172.21.15.110:46890) 2024-09-10T09:58:33.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1d5400640 1 -- 172.21.15.110:0/1901329130 learned_addr learned my addr 172.21.15.110:0/1901329130 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1d4a00640 1 --2- 172.21.15.110:0/1901329130 >> v2:172.21.15.145:3300/0 conn(0x7fc1d00aeaa0 0x7fc1d0071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f6e10e00640 1 Processor -- start 2024-09-10T09:58:33.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0024c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0020072230 0x7f00201663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0024c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0020072230 0x7f00201663f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:41004/0 (socket says 172.21.15.110:41004) 2024-09-10T09:58:33.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0024c00640 1 -- 172.21.15.110:0/1590812555 learned_addr learned my addr 172.21.15.110:0/1590812555 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001fe00640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f0020153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1cfe00640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d00aee80 0x7fc1d006e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c46000640 1 -- 172.21.15.110:0/3324099450 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c340ab9d0 con 0x7f4c40052710 2024-09-10T09:58:33.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0025600640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00201602c0 0x7f0020157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976277715 0 0) 0x7fc1d006f340 con 0x7fc1d006ed30 2024-09-10T09:58:33.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c4e400640 1 -- 172.21.15.110:0/3324099450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c40052710 msgr2=0x7f4c40007250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c4e400640 1 --2- 172.21.15.110:0/3324099450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c40052710 0x7f4c40007250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4c340097c0 tx=0x7f4c3401c4c0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1b4003530 con 0x7fc1d006ed30 2024-09-10T09:58:33.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001fe00640 1 -- 172.21.15.110:0/1590812555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0020072230 msgr2=0x7f00201663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001fe00640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0020072230 0x7f00201663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001fe00640 1 -- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00201602c0 msgr2=0x7f0020157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001fe00640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00201602c0 0x7f0020157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001fe00640 1 -- 172.21.15.110:0/1590812555 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0020156d90 con 0x7f00200cff50 2024-09-10T09:58:33.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c4e400640 1 -- 172.21.15.110:0/3324099450 shutdown_connections 2024-09-10T09:58:33.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c4e400640 1 --2- 172.21.15.110:0/3324099450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c40052710 0x7f4c40007250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c4e400640 1 --2- 172.21.15.110:0/3324099450 >> v2:172.21.15.110:3300/0 conn(0x7f4c400e2180 0x7f4c4000a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c4e400640 1 -- 172.21.15.110:0/3324099450 >> 172.21.15.110:0/3324099450 conn(0x7f4c4001df60 msgr2=0x7f4c40054bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f4c4e400640 1 -- 172.21.15.110:0/3324099450 shutdown_connections 2024-09-10T09:58:33.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001fe00640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f0020153880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00140ab000 tx=0x7f001401ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 <== mon.1 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3589023993 0 0) 0x7fc1b4003530 con 0x7fc1d006ed30 2024-09-10T09:58:33.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d00aee80 msgr2=0x7fc1d006e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7fc1cf400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d00aee80 0x7fc1d006e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0041e00640 1 -- 172.21.15.110:0/2463406020 shutdown_connections 2024-09-10T09:58:33.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f0041e00640 1 -- 172.21.15.110:0/2463406020 wait complete. 2024-09-10T09:58:33.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140bd3f0 con 0x7f00200cff50 2024-09-10T09:58:33.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00140bd590 con 0x7f00200cff50 2024-09-10T09:58:33.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140bd980 con 0x7f00200cff50 2024-09-10T09:58:33.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.052+0000 7f6e10e00640 1 -- start start 2024-09-10T09:58:33.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002015ea00 con 0x7f00200cff50 2024-09-10T09:58:33.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0020157310 con 0x7f00200cff50 2024-09-10T09:58:33.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2be600640 1 --2- 172.21.15.110:0/1584561794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b0038950 0x7fc2b00512a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0050f90 tx=0x7fc2b8098880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e63cf2e1bdf723d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e10e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6e0c0d0000 0x7f6e0c06d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2b7e00640 1 -- 172.21.15.110:0/1584561794 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8122750 con 0x7fc2b0038950 2024-09-10T09:58:33.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f002006e770 con 0x7f00200cff50 2024-09-10T09:58:33.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2b7e00640 1 -- 172.21.15.110:0/1584561794 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b81228f0 con 0x7fc2b0038950 2024-09-10T09:58:33.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e10e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c162ce0 0x7f6e0c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2b7e00640 1 -- 172.21.15.110:0/1584561794 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8122d20 con 0x7fc2b0038950 2024-09-10T09:58:33.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e09e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6e0c0b4040 0x7f6e0c1625d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38204/0 (socket says 172.21.15.110:38204) 2024-09-10T09:58:33.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e09e00640 1 -- 172.21.15.110:0/2548898502 learned_addr learned my addr 172.21.15.110:0/2548898502 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e10e00640 1 -- 172.21.15.110:0/2548898502 --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e0c0b3720 con 0x7f6e0c0b4040 2024-09-10T09:58:33.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e09400640 1 --2- 172.21.15.110:0/2548898502 >> v2:172.21.15.110:3300/0 conn(0x7f6e0c0d0000 0x7f6e0c06d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e0a800640 1 --2- 172.21.15.110:0/2548898502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c162ce0 0x7f6e0c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f0041e00640 1 Processor -- start 2024-09-10T09:58:33.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e10e00640 1 -- 172.21.15.110:0/2548898502 --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6e0c0b34d0 con 0x7f6e0c0d0000 2024-09-10T09:58:33.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e10e00640 1 -- 172.21.15.110:0/2548898502 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c0b3a00 con 0x7f6e0c162ce0 2024-09-10T09:58:33.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 >> v2:172.21.15.145:3300/0 conn(0x7fc1d00aeaa0 msgr2=0x7fc1d0071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 --2- 172.21.15.110:0/1901329130 >> v2:172.21.15.145:3300/0 conn(0x7fc1d00aeaa0 0x7fc1d0071730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d006f900 con 0x7fc1d006ed30 2024-09-10T09:58:33.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 <== mon.1 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fc1c8001a00 con 0x7fc1d006ed30 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 >> v1:172.21.15.145:6789/0 conn(0x7fc1d006ed30 legacy=0x7fc1d0146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d00aee80 0x7fc1b40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1b4003ff0 0x7fc1b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 0x7fc1b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0071c70 con 0x7fc1b4004d50 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d006f620 con 0x7fc1b4003ff0 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b400bad0 con 0x7fc1d00aee80 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1d4a00640 1 --2- 172.21.15.110:0/1901329130 >> v2:172.21.15.145:3300/0 conn(0x7fc1d00aeaa0 0x7fc1d0071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:33.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f4c4e400640 1 -- 172.21.15.110:0/3324099450 wait complete. 2024-09-10T09:58:33.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e08a00640 1 -- 172.21.15.110:0/2548898502 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569244733 0 0) 0x7f6e0c0b3720 con 0x7f6e0c0b4040 2024-09-10T09:58:33.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e08a00640 1 -- 172.21.15.110:0/2548898502 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df0003530 con 0x7f6e0c0b4040 2024-09-10T09:58:33.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f0363e00640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 0x7f03480051d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f035c00a7b0 tx=0x7f035c01d290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9dc136c979192d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1cfe00640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1b4003ff0 0x7fc1b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1d4a00640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d00aee80 0x7fc1b40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc1d5400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 0x7fc1b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2bf000640 1 -- 172.21.15.110:0/1584561794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b0038950 msgr2=0x7fc2b00512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2bf000640 1 --2- 172.21.15.110:0/1584561794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b0038950 0x7fc2b00512a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0050f90 tx=0x7fc2b8098880 comp rx=0 tx=0).stop 2024-09-10T09:58:33.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2bf000640 1 -- 172.21.15.110:0/1584561794 shutdown_connections 2024-09-10T09:58:33.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2bf000640 1 --2- 172.21.15.110:0/1584561794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b0038950 0x7fc2b00512a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2bf000640 1 --2- 172.21.15.110:0/1584561794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 0x7fc2b0009110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7fc2bf000640 1 -- 172.21.15.110:0/1584561794 >> 172.21.15.110:0/1584561794 conn(0x7fc2b00567c0 msgr2=0x7fc2b0056bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc1d5400640 1 -- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d00aee80 msgr2=0x7fc1b40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc1d5400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d00aee80 0x7fc1b40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc1d5400640 1 -- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1b4003ff0 msgr2=0x7fc1b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc1d5400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1b4003ff0 0x7fc1b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc1d5400640 1 -- 172.21.15.110:0/1901329130 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d006f900 con 0x7fc1b4004d50 2024-09-10T09:58:33.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c00a3b0 con 0x7f036415c570 2024-09-10T09:58:33.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0362000640 1 -- 172.21.15.110:0/230425594 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035c0ad630 con 0x7f036415c570 2024-09-10T09:58:33.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e0a800640 1 -- 172.21.15.110:0/2548898502 >> v2:172.21.15.110:3300/0 conn(0x7f6e0c0d0000 msgr2=0x7f6e0c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e0a800640 1 --2- 172.21.15.110:0/2548898502 >> v2:172.21.15.110:3300/0 conn(0x7f6e0c0d0000 0x7f6e0c06d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f0041e00640 1 -- start start 2024-09-10T09:58:33.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00140b33f0 con 0x7f00200cff50 2024-09-10T09:58:33.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e0a800640 1 -- 172.21.15.110:0/2548898502 >> v1:172.21.15.110:6789/0 conn(0x7f6e0c0b4040 legacy=0x7f6e0c1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f6e0a800640 1 -- 172.21.15.110:0/2548898502 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c0726a0 con 0x7f6e0c162ce0 2024-09-10T09:58:33.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.056+0000 7f001cc00640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00040678c0 0x7f0004069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f001411f990 con 0x7f00200cff50 2024-09-10T09:58:33.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f001cc00640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f000406fa60 0x7f0004071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 --> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00140bee50 con 0x7f000406fa60 2024-09-10T09:58:33.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f00141242c0 con 0x7f00200cff50 2024-09-10T09:58:33.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0041e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c073fb0 0x7f003c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f4c4e400640 1 Processor -- start 2024-09-10T09:58:33.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0025600640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f000406fa60 0x7f0004071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c073fb0 0x7f003c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c073fb0 0x7f003c15c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53138/0 (socket says 172.21.15.110:53138) 2024-09-10T09:58:33.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0024c00640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00040678c0 0x7f0004069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 -- 172.21.15.110:0/798908164 learned_addr learned my addr 172.21.15.110:0/798908164 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f036a600640 1 -- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 msgr2=0x7f03480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f036a600640 1 --2- 172.21.15.110:0/230425594 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415c570 0x7f03480051d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f035c00a7b0 tx=0x7f035c01d290 comp rx=0 tx=0).stop 2024-09-10T09:58:33.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0041e00640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f003c0d0fe0 0x7f003c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0025600640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f000406fa60 0x7f0004071e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0041e00640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f003c0d2770 0x7f003c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0041e00640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f003c16c1d0 con 0x7f003c073fb0 2024-09-10T09:58:33.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0041e00640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f003c0730c0 con 0x7f003c0d2770 2024-09-10T09:58:33.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0041e00640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f003c16c350 con 0x7f003c0d0fe0 2024-09-10T09:58:33.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fbb13000640 1 Processor -- start 2024-09-10T09:58:33.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003a000640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f003c0d0fe0 0x7f003c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc2bf000640 1 -- 172.21.15.110:0/1584561794 shutdown_connections 2024-09-10T09:58:33.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc2bf000640 1 -- 172.21.15.110:0/1584561794 wait complete. 2024-09-10T09:58:33.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fbb13000640 1 -- start start 2024-09-10T09:58:33.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003b400640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f003c0d2770 0x7f003c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 -- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f003c0d0fe0 msgr2=0x7f003c15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f003c0d0fe0 0x7f003c15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc2bf000640 1 Processor -- start 2024-09-10T09:58:33.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 -- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f003c0d2770 msgr2=0x7f003c0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f003c0d2770 0x7f003c0b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c15d390 con 0x7f003c073fb0 2024-09-10T09:58:33.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003b400640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f003c0d2770 0x7f003c0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:33.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0024c00640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00040678c0 0x7f0004069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0010000f70 tx=0x7f0010007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0363e00640 1 -- 172.21.15.110:0/230425594 reap_dead start 2024-09-10T09:58:33.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7fc2bf000640 1 -- start start 2024-09-10T09:58:33.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f003aa00640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c073fb0 0x7f003c15c400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f002c018ff0 tx=0x7f002c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 <== osd.2 v2:172.21.15.110:6800/1201840781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f00140bee50 con 0x7f000406fa60 2024-09-10T09:58:33.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f036a600640 1 -- 172.21.15.110:0/230425594 shutdown_connections 2024-09-10T09:58:33.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f036a600640 1 -- 172.21.15.110:0/230425594 >> 172.21.15.110:0/230425594 conn(0x7f036406f440 msgr2=0x7f036406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f036a600640 1 -- 172.21.15.110:0/230425594 shutdown_connections 2024-09-10T09:58:33.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0b9070 con 0x7f003c073fb0 2024-09-10T09:58:33.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c0b5680 con 0x7f003c073fb0 2024-09-10T09:58:33.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0b0040 con 0x7f003c073fb0 2024-09-10T09:58:33.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f0041e00640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c15b920 con 0x7f003c073fb0 2024-09-10T09:58:33.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f0041e00640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c15cf30 con 0x7f003c073fb0 2024-09-10T09:58:33.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f036a600640 1 -- 172.21.15.110:0/230425594 wait complete. 2024-09-10T09:58:33.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb13000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0d0000 0x7fbb0c06d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb13000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0c0b3720 con 0x7fbb0c162ce0 2024-09-10T09:58:33.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb13000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0c0b3a00 con 0x7fbb0c0b4040 2024-09-10T09:58:33.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb13000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0c0b34d0 con 0x7fbb0c0d0000 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.060+0000 7f4c4e400640 1 -- start start 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c4e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c40004ce0 0x7f4c40051b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c4e400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c40052710 0x7f4c400d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c4e400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c400e2180 0x7f4c400d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c4e400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c400e5a40 con 0x7f4c40052710 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c4e400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c400e58c0 con 0x7f4c40004ce0 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c4e400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4c400e5bc0 con 0x7f4c400e2180 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c40004ce0 0x7f4c40051b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bf000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 0x7fc2b0037870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bf000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2b0038950 0x7fc2b003b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bf000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b0057350 0x7fc2b00df490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bf000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0052170 con 0x7fc2b0038950 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bf000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0051ff0 con 0x7fc2b0057350 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bf000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b00522f0 con 0x7fc2b0035b90 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 0x7fc2b0037870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 0x7fc2b0037870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:46628/0 (socket says 172.21.15.110:46628) 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 -- 172.21.15.110:0/3348421014 learned_addr learned my addr 172.21.15.110:0/3348421014 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2be600640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b0057350 0x7fc2b00df490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb11000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fbb0c162ce0 0x7fbb0c06d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38208/0 (socket says 172.21.15.110:38208) 2024-09-10T09:58:33.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb11000640 1 -- 172.21.15.110:0/2098153102 learned_addr learned my addr 172.21.15.110:0/2098153102 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c40004ce0 0x7f4c40051b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:41050/0 (socket says 172.21.15.110:41050) 2024-09-10T09:58:33.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 -- 172.21.15.110:0/3942560058 learned_addr learned my addr 172.21.15.110:0/3942560058 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c4ce00640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c400e2180 0x7f4c400d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 -- 172.21.15.110:0/3942560058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c400e2180 msgr2=0x7f4c400d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4c400e2180 0x7f4c400d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 -- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c40052710 msgr2=0x7f4c400d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c40052710 0x7f4c400d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 -- 172.21.15.110:0/3942560058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4000b420 con 0x7f4c40004ce0 2024-09-10T09:58:33.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bd200640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2b0038950 0x7fc2b003b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 -- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b0057350 msgr2=0x7fc2b00df490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b0057350 0x7fc2b00df490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 -- 172.21.15.110:0/3348421014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2b0038950 msgr2=0x7fc2b003b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2b0038950 0x7fc2b003b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f6e0a800640 1 --2- 172.21.15.110:0/2548898502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c162ce0 0x7f6e0c06d670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009d90 tx=0x7f6e0401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fe56b7f7e54a9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb0b400640 1 --2- 172.21.15.110:0/2098153102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0d0000 0x7fbb0c06d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f003c15bfe0 con 0x7f003c073fb0 2024-09-10T09:58:33.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c47400640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c40004ce0 0x7f4c40051b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c48074670 tx=0x7f4c48069860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc1d5400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 0x7fc1b40051d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800a8e0 tx=0x7fc1c801d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff4884e1e726448 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 -- 172.21.15.110:0/3348421014 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b00362c0 con 0x7fc2b0035b90 2024-09-10T09:58:33.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f6e08a00640 1 -- 172.21.15.110:0/2548898502 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040b2070 con 0x7f6e0c162ce0 2024-09-10T09:58:33.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f6e08a00640 1 -- 172.21.15.110:0/2548898502 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e040a5990 con 0x7f6e0c162ce0 2024-09-10T09:58:33.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f6e08a00640 1 -- 172.21.15.110:0/2548898502 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040a5d40 con 0x7f6e0c162ce0 2024-09-10T09:58:33.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb0aa00640 1 -- 172.21.15.110:0/2098153102 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091360237 0 0) 0x7fbb0c0b3a00 con 0x7fbb0c0b4040 2024-09-10T09:58:33.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb0aa00640 1 -- 172.21.15.110:0/2098153102 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaf0003530 con 0x7fbb0c0b4040 2024-09-10T09:58:33.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48071630 con 0x7f4c40004ce0 2024-09-10T09:58:33.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb0b400640 1 -- 172.21.15.110:0/2098153102 >> v1:172.21.15.145:6789/0 conn(0x7fbb0c0b4040 legacy=0x7fbb0c1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb0b400640 1 -- 172.21.15.110:0/2098153102 >> v1:172.21.15.110:6789/0 conn(0x7fbb0c162ce0 legacy=0x7fbb0c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c480717d0 con 0x7f4c40004ce0 2024-09-10T09:58:33.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48071630 con 0x7f4c40004ce0 2024-09-10T09:58:33.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fbb0b400640 1 -- 172.21.15.110:0/2098153102 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c0726a0 con 0x7fbb0c0d0000 2024-09-10T09:58:33.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bdc00640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 0x7fc2b0037870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00b4a0 tx=0x7fc2ac00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac014070 con 0x7fc2b0035b90 2024-09-10T09:58:33.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2ac005a20 con 0x7fc2b0035b90 2024-09-10T09:58:33.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac005e50 con 0x7fc2b0035b90 2024-09-10T09:58:33.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7f036a600640 1 Processor -- start 2024-09-10T09:58:33.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.064+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b0036560 con 0x7fc2b0035b90 2024-09-10T09:58:33.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f036a600640 1 -- start start 2024-09-10T09:58:33.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 -- 172.21.15.110:0/2548898502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c162ce0 msgr2=0x7f6e0c06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 --2- 172.21.15.110:0/2548898502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c162ce0 0x7f6e0c06d670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009d90 tx=0x7f6e0401c8b0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b0038210 con 0x7fc2b0035b90 2024-09-10T09:58:33.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800a4e0 con 0x7fc1b4004d50 2024-09-10T09:58:33.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f4c4e400640 1 -- 172.21.15.110:0/3942560058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4017fa40 con 0x7f4c40004ce0 2024-09-10T09:58:33.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f4c4e400640 1 -- 172.21.15.110:0/3942560058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c40180000 con 0x7f4c40004ce0 2024-09-10T09:58:33.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c80a9020 con 0x7fc1b4004d50 2024-09-10T09:58:33.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc1cf400640 1 -- 172.21.15.110:0/1901329130 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800a4e0 con 0x7fc1b4004d50 2024-09-10T09:58:33.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 -- 172.21.15.110:0/2548898502 shutdown_connections 2024-09-10T09:58:33.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 --2- 172.21.15.110:0/2548898502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c162ce0 0x7f6e0c06d670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 --2- 172.21.15.110:0/2548898502 >> v2:172.21.15.110:3300/0 conn(0x7f6e0c0d0000 0x7f6e0c06d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 -- 172.21.15.110:0/2548898502 >> 172.21.15.110:0/2548898502 conn(0x7f6e0c062d10 msgr2=0x7f6e0c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 -- 172.21.15.110:0/2548898502 shutdown_connections 2024-09-10T09:58:33.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 -- 172.21.15.110:0/2548898502 wait complete. 2024-09-10T09:58:33.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc2b0009110 con 0x7fc2b0035b90 2024-09-10T09:58:33.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f4c4e400640 1 -- 172.21.15.110:0/3942560058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c40033660 con 0x7f4c40004ce0 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc1d7400640 1 -- 172.21.15.110:0/1901329130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 msgr2=0x7fc1b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc1d7400640 1 --2- 172.21.15.110:0/1901329130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 0x7fc1b40051d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800a8e0 tx=0x7fc1c801d3c0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc1d5400640 1 -- 172.21.15.110:0/1901329130 reap_dead start 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f036a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036415c570 0x7f03640b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f036a600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415cb60 0x7f03640b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f0363400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036415c570 0x7f03640b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f0363400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036415c570 0x7f03640b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:41056/0 (socket says 172.21.15.110:41056) 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f036a600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f036415a5a0 0x7f036415aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f0363400640 1 -- 172.21.15.110:0/1326881636 learned_addr learned my addr 172.21.15.110:0/1326881636 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f0362a00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415cb60 0x7f03640b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f036a600640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0364071a80 con 0x7f036415a5a0 2024-09-10T09:58:33.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f036a600640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0364071900 con 0x7f036415c570 2024-09-10T09:58:33.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f036a600640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0364071c00 con 0x7f036415cb60 2024-09-10T09:58:33.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc2ac0327b0 con 0x7fc2b0035b90 2024-09-10T09:58:33.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f002c0bc070 con 0x7f003c073fb0 2024-09-10T09:58:33.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f0033400640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f001c0678f0 0x7f001c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f002c120590 con 0x7f003c073fb0 2024-09-10T09:58:33.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0363e00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f036415a5a0 0x7f036415aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0363400640 1 -- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415cb60 msgr2=0x7f03640b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0363400640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f036415cb60 0x7f03640b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0363400640 1 -- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f036415a5a0 msgr2=0x7f036415aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7fc2b5600640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc298067ab0 0x7fc298069f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc2ac0850e0 con 0x7fc2b0035b90 2024-09-10T09:58:33.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.068+0000 7f6e10e00640 1 Processor -- start 2024-09-10T09:58:33.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e10e00640 1 -- start start 2024-09-10T09:58:33.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0033400640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f001c06fb10 0x7f001c071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f001c0725e0 con 0x7f001c06fb10 2024-09-10T09:58:33.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f002c120980 con 0x7f003c073fb0 2024-09-10T09:58:33.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f003b400640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f001c06fb10 0x7f001c071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f003b400640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f001c06fb10 0x7f001c071f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4c48095060 con 0x7f4c40004ce0 2024-09-10T09:58:33.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f4c3b400640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4c28067ac0 0x7f4c28069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc1d7400640 1 -- 172.21.15.110:0/1901329130 shutdown_connections 2024-09-10T09:58:33.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc1d7400640 1 -- 172.21.15.110:0/1901329130 >> 172.21.15.110:0/1901329130 conn(0x7fc1d006be20 msgr2=0x7fc1d006c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc1d7400640 1 -- 172.21.15.110:0/1901329130 shutdown_connections 2024-09-10T09:58:33.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc1d7400640 1 -- 172.21.15.110:0/1901329130 wait complete. 2024-09-10T09:58:33.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0363400640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f036415a5a0 0x7f036415aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0363400640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03640b38e0 con 0x7f036415c570 2024-09-10T09:58:33.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0363e00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f036415a5a0 0x7f036415aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2b5600640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7fc29806fcd0 0x7fc2980720f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 --> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2ac01be70 con 0x7fc29806fcd0 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac0855c0 con 0x7fc2b0035b90 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2bd200640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc298067ab0 0x7fc298069f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2be600640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7fc29806fcd0 0x7fc2980720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2be600640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7fc29806fcd0 0x7fc2980720f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e10e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c0b4040 0x7f6e0c0ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e10e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e0c0d0000 0x7f6e0c0cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e10e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e0c162ce0 0x7f6e0c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e10e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c16f9a0 con 0x7f6e0c0d0000 2024-09-10T09:58:33.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e10e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c16f820 con 0x7f6e0c162ce0 2024-09-10T09:58:33.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e10e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c16fb20 con 0x7f6e0c0b4040 2024-09-10T09:58:33.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fbb0b400640 1 --2- 172.21.15.110:0/2098153102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0d0000 0x7fbb0c06d130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0b3650 tx=0x7fbafc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e097475655627b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f003a000640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f001c0678f0 0x7f001c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4c48100560 con 0x7f4c40004ce0 2024-09-10T09:58:33.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2bd200640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc298067ab0 0x7fc298069f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0050870 tx=0x7fc2a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f4c3b400640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f4c2806fcc0 0x7f4c280720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 --> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c28072790 con 0x7f4c2806fcc0 2024-09-10T09:58:33.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4c48107050 con 0x7f4c40004ce0 2024-09-10T09:58:33.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0363400640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036415c570 0x7f03640b3020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f035000e7c0 tx=0x7f035000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c0b4040 0x7f6e0c0ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c0b4040 0x7f6e0c0ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:46658/0 (socket says 172.21.15.110:46658) 2024-09-10T09:58:33.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 -- 172.21.15.110:0/3247140692 learned_addr learned my addr 172.21.15.110:0/3247140692 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e0a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e0c162ce0 0x7f6e0c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fbb0aa00640 1 -- 172.21.15.110:0/2098153102 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0ab620 con 0x7fbb0c0d0000 2024-09-10T09:58:33.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fbb0aa00640 1 -- 172.21.15.110:0/2098153102 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc0b0070 con 0x7fbb0c0d0000 2024-09-10T09:58:33.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fbb0aa00640 1 -- 172.21.15.110:0/2098153102 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0ab9e0 con 0x7fbb0c0d0000 2024-09-10T09:58:33.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09400640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e0c0d0000 0x7f6e0c0cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc1d7400640 1 Processor -- start 2024-09-10T09:58:33.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 <== osd.0 v2:172.21.15.110:6814/321203351 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f001c0725e0 con 0x7f001c06fb10 2024-09-10T09:58:33.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f4c4ce00640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f4c2806fcc0 0x7f4c280720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f4c46a00640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4c28067ac0 0x7f4c28069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc1d7400640 1 -- start start 2024-09-10T09:58:33.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 <== osd.5 v2:172.21.15.145:6808/1843342183 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2ac01be70 con 0x7fc29806fcd0 2024-09-10T09:58:33.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 -- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e0c162ce0 msgr2=0x7f6e0c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e0c162ce0 0x7f6e0c15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 -- 172.21.15.110:0/3247140692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e0c0d0000 msgr2=0x7f6e0c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e0c0d0000 0x7f6e0c0cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 -- 172.21.15.110:0/3247140692 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c15f790 con 0x7f6e0c0b4040 2024-09-10T09:58:33.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f6e09e00640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c0b4040 0x7f6e0c0ac370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c450 tx=0x7f6df800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350017070 con 0x7f036415c570 2024-09-10T09:58:33.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8017070 con 0x7f6e0c0b4040 2024-09-10T09:58:33.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df8005cf0 con 0x7f6e0c0b4040 2024-09-10T09:58:33.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 0x7fc1d0160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d7400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d015eab0 0x7fc1d00adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f4c4ce00640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f4c2806fcc0 0x7f4c280720e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035000ee70 con 0x7f036415c570 2024-09-10T09:58:33.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035000cb40 con 0x7f036415c570 2024-09-10T09:58:33.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fbb13000640 1 -- 172.21.15.110:0/2098153102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0d0000 msgr2=0x7fbb0c06d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.072+0000 7fbb13000640 1 --2- 172.21.15.110:0/2098153102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0d0000 0x7fbb0c06d130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0b3650 tx=0x7fbafc01c400 comp rx=0 tx=0).stop 2024-09-10T09:58:33.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fbb13000640 1 -- 172.21.15.110:0/2098153102 shutdown_connections 2024-09-10T09:58:33.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fbb13000640 1 --2- 172.21.15.110:0/2098153102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0d0000 0x7fbb0c06d130 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0b3650 tx=0x7fbafc01c400 comp rx=0 tx=0).stop 2024-09-10T09:58:33.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fbb13000640 1 -- 172.21.15.110:0/2098153102 >> 172.21.15.110:0/2098153102 conn(0x7fbb0c062d10 msgr2=0x7fbb0c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fbb13000640 1 -- 172.21.15.110:0/2098153102 shutdown_connections 2024-09-10T09:58:33.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fbb13000640 1 -- 172.21.15.110:0/2098153102 wait complete. 2024-09-10T09:58:33.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8017070 con 0x7f6e0c0b4040 2024-09-10T09:58:33.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f003a000640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f001c0678f0 0x7f001c069db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f003c0b6220 tx=0x7f0028009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f6e10e00640 1 -- 172.21.15.110:0/3247140692 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c0cf170 con 0x7f6e0c0b4040 2024-09-10T09:58:33.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1cfe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d015eab0 0x7fc1d00adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1cfe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d015eab0 0x7fc1d00adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:46664/0 (socket says 172.21.15.110:46664) 2024-09-10T09:58:33.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f6e10e00640 1 -- 172.21.15.110:0/3247140692 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c0ab5b0 con 0x7f6e0c0b4040 2024-09-10T09:58:33.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e0c1625d0 con 0x7f6e0c0b4040 2024-09-10T09:58:33.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fbb13000640 1 Processor -- start 2024-09-10T09:58:33.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fbb13000640 1 -- start start 2024-09-10T09:58:33.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d01418d0 0x7fc1d01407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d7400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01481d0 con 0x7fc1b4004d50 2024-09-10T09:58:33.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d7400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0148050 con 0x7fc1d01418d0 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d7400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d0147430 con 0x7fc1d015eab0 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d01418d0 0x7fc1d01407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d01418d0 0x7fc1d01407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:41074/0 (socket says 172.21.15.110:41074) 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d5400640 1 -- 172.21.15.110:0/679461667 learned_addr learned my addr 172.21.15.110:0/679461667 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d4a00640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 0x7fc1d0160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d5400640 1 -- 172.21.15.110:0/679461667 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d015eab0 msgr2=0x7fc1d00adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d5400640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d015eab0 0x7fc1d00adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d5400640 1 -- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 msgr2=0x7fc1d0160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d5400640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1b4004d50 0x7fc1d0160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1d5400640 1 -- 172.21.15.110:0/679461667 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d01415f0 con 0x7fc1d01418d0 2024-09-10T09:58:33.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7fc1cfe00640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1d015eab0 0x7fc1d00adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T09:58:33.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f036a600640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03640b3600 con 0x7f036415c570 2024-09-10T09:58:33.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.076+0000 7f036a600640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03641559a0 con 0x7f036415c570 2024-09-10T09:58:33.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7f036a600640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03640d2c70 con 0x7f036415c570 2024-09-10T09:58:33.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 <== osd.1 v2:172.21.15.110:6808/3662700555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c28072790 con 0x7f4c2806fcc0 2024-09-10T09:58:33.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7f4c46a00640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4c28067ac0 0x7f4c28069f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c400d1220 tx=0x7f4c3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fc1d5400640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d01418d0 0x7fc1d01407c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1c801d1f0 tx=0x7fc1b4005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb13000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0b4040 0x7fbb0c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb13000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb0c162ce0 0x7fbb0c15a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb13000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c150dd0 0x7fbb0c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb0c162ce0 0x7fbb0c15a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6df801a070 con 0x7f6e0c0b4040 2024-09-10T09:58:33.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb0c162ce0 0x7fbb0c15a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:41084/0 (socket says 172.21.15.110:41084) 2024-09-10T09:58:33.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb11000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c150dd0 0x7fbb0c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb11000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c150dd0 0x7fbb0c0cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53208/0 (socket says 172.21.15.110:53208) 2024-09-10T09:58:33.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800a4e0 con 0x7fc1d01418d0 2024-09-10T09:58:33.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0b400640 1 -- 172.21.15.110:0/156216349 learned_addr learned my addr 172.21.15.110:0/156216349 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c80ae670 con 0x7fc1d01418d0 2024-09-10T09:58:33.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80aea20 con 0x7fc1d01418d0 2024-09-10T09:58:33.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0be00640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0b4040 0x7fbb0c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c170bc0 con 0x7fbb0c150dd0 2024-09-10T09:58:33.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c170a40 con 0x7fbb0c162ce0 2024-09-10T09:58:33.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0c170d40 con 0x7fbb0c0b4040 2024-09-10T09:58:33.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0b400640 1 -- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0b4040 msgr2=0x7fbb0c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f035001a070 con 0x7f036415c570 2024-09-10T09:58:33.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0b400640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0b4040 0x7fbb0c1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d0143db0 con 0x7fc1d01418d0 2024-09-10T09:58:33.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d0144370 con 0x7fc1d01418d0 2024-09-10T09:58:33.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0b400640 1 -- 172.21.15.110:0/156216349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c150dd0 msgr2=0x7fbb0c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0b400640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c150dd0 0x7fbb0c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0b400640 1 -- 172.21.15.110:0/156216349 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c151820 con 0x7fbb0c162ce0 2024-09-10T09:58:33.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb11000640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c150dd0 0x7fbb0c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:33.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1d01413a0 con 0x7fc1d01418d0 2024-09-10T09:58:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7fbb0be00640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbb0c0b4040 0x7fbb0c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7f6e02000640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6dec067ac0 0x7f6dec069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.080+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6df8084ea0 con 0x7f6e0c0b4040 2024-09-10T09:58:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f6e02000640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f6dec06fc60 0x7f6dec072080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f6e09400640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6dec067ac0 0x7f6dec069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 --> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6dec072730 con 0x7f6dec06fc60 2024-09-10T09:58:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f035b400640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0344067990 0x7f0344069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f6e0a800640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f6dec06fc60 0x7f6dec072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6df8085290 con 0x7f6e0c0b4040 2024-09-10T09:58:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f03500861a0 con 0x7f036415c570 2024-09-10T09:58:33.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f035b400640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7f034406fbb0 0x7f0344071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0344072680 con 0x7f034406fbb0 2024-09-10T09:58:33.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0350086590 con 0x7f036415c570 2024-09-10T09:58:33.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fbb0b400640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb0c162ce0 0x7fbb0c15a020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009740 tx=0x7fbafc01ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f0363e00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7f034406fbb0 0x7f0344071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f6e0a800640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f6dec06fc60 0x7f6dec072080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f0362a00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0344067990 0x7f0344069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fbb03e00640 1 -- 172.21.15.110:0/156216349 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc004220 con 0x7fbb0c162ce0 2024-09-10T09:58:33.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc1c80aebc0 con 0x7fc1d01418d0 2024-09-10T09:58:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f6e09400640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6dec067ac0 0x7f6dec069f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0cebc0 tx=0x7f6dfc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fc1ccc00640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1b0067990 0x7fc1b0069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc1c8131e10 con 0x7fc1d01418d0 2024-09-10T09:58:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fc1ccc00640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7fc1b006fbb0 0x7fc1b0071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 --> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1b0072680 con 0x7fc1b006fbb0 2024-09-10T09:58:33.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1c8132200 con 0x7fc1d01418d0 2024-09-10T09:58:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7f0363e00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7f034406fbb0 0x7f0344071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fbb03e00640 1 -- 172.21.15.110:0/156216349 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc0043c0 con 0x7fbb0c162ce0 2024-09-10T09:58:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fbb03e00640 1 -- 172.21.15.110:0/156216349 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0abb40 con 0x7fbb0c162ce0 2024-09-10T09:58:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.084+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c151b00 con 0x7fbb0c162ce0 2024-09-10T09:58:33.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.088+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 <== osd.6 v2:172.21.15.145:6824/3155222893 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6dec072730 con 0x7f6dec06fc60 2024-09-10T09:58:33.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.088+0000 7fc1d4a00640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1b0067990 0x7fc1b0069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.088+0000 7fc1cfe00640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7fc1b006fbb0 0x7fc1b0071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.088+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c1520c0 con 0x7fbb0c162ce0 2024-09-10T09:58:33.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.088+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 <== osd.4 v2:172.21.15.145:6816/927410831 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0344072680 con 0x7f034406fbb0 2024-09-10T09:58:33.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.088+0000 7fc1d4a00640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1b0067990 0x7fc1b0069e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00aa630 tx=0x7fc1c0018040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.088+0000 7fc1cfe00640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7fc1b006fbb0 0x7fc1b0071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.088+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 <== osd.3 v2:172.21.15.110:6824/4112166746 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc1b0072680 con 0x7fc1b006fbb0 2024-09-10T09:58:33.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.092+0000 7f0362a00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0344067990 0x7f0344069e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f036415cf80 tx=0x7f0354009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.092+0000 7fbb03e00640 1 -- 172.21.15.110:0/156216349 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbafc0bd070 con 0x7fbb0c162ce0 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.092+0000 7fbb03e00640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbaec0679c0 0x7fbaec069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fbb03e00640 1 -- 172.21.15.110:0/156216349 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbafc11f1b0 con 0x7fbb0c162ce0 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fbb0be00640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbaec0679c0 0x7fbaec069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fbb13000640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7fbb0c152300 0x7fbb0c0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 --> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb0c1625d0 con 0x7fbb0c152300 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fbb11000640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7fbb0c152300 0x7fbb0c0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fbb0be00640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbaec0679c0 0x7fbaec069e80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb04001570 tx=0x7fbb04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fbb11000640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7fbb0c152300 0x7fbb0c0ce090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 --> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc2b0055e10 con 0x7fc29806fcd0 2024-09-10T09:58:33.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fbb03e00640 1 -- 172.21.15.110:0/156216349 <== osd.7 v2:172.21.15.145:6800/938429327 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb0c1625d0 con 0x7fbb0c152300 2024-09-10T09:58:33.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.096+0000 7fc2b5600640 1 -- 172.21.15.110:0/3348421014 <== osd.5 v2:172.21.15.145:6808/1843342183 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc2b0055e10 con 0x7fc29806fcd0 2024-09-10T09:58:33.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.100+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 --> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0020073090 con 0x7f000406fa60 2024-09-10T09:58:33.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7f001cc00640 1 -- 172.21.15.110:0/1590812555 <== osd.2 v2:172.21.15.110:6800/1201840781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0020073090 con 0x7f000406fa60 2024-09-10T09:58:33.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7fc29806fcd0 msgr2=0x7fc2980720f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7fc29806fcd0 0x7fc2980720f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc298067ab0 msgr2=0x7fc298069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc298067ab0 0x7fc298069f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0050870 tx=0x7fc2a4008040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 msgr2=0x7fc2b0037870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 --2- 172.21.15.110:0/3348421014 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc2b0035b90 0x7fc2b0037870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00b4a0 tx=0x7fc2ac00b960 comp rx=0 tx=0).stop 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2be600640 1 -- 172.21.15.110:0/3348421014 reap_dead start 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 shutdown_connections 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 >> 172.21.15.110:0/3348421014 conn(0x7fc2b00567c0 msgr2=0x7fc2b0054e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f000406fa60 msgr2=0x7f0004071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7f0026000640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f000406fa60 0x7f0004071e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 shutdown_connections 2024-09-10T09:58:33.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7fc2bf000640 1 -- 172.21.15.110:0/3348421014 wait complete. 2024-09-10T09:58:33.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 --> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f003c0b33d0 con 0x7f001c06fb10 2024-09-10T09:58:33.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.104+0000 7f0033400640 1 -- 172.21.15.110:0/798908164 <== osd.0 v2:172.21.15.110:6814/321203351 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f003c0b33d0 con 0x7f001c06fb10 2024-09-10T09:58:33.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f001c06fb10 msgr2=0x7f001c071f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f001c06fb10 0x7f001c071f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f001c0678f0 msgr2=0x7f001c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f001c0678f0 0x7f001c069db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f003c0b6220 tx=0x7f0028009040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c073fb0 msgr2=0x7f003c15c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 --2- 172.21.15.110:0/798908164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f003c073fb0 0x7f003c15c400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f002c018ff0 tx=0x7f002c019180 comp rx=0 tx=0).stop 2024-09-10T09:58:33.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f003b400640 1 -- 172.21.15.110:0/798908164 reap_dead start 2024-09-10T09:58:33.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 shutdown_connections 2024-09-10T09:58:33.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00040678c0 msgr2=0x7f0004069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0026000640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00040678c0 0x7f0004069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0010000f70 tx=0x7f0010007e20 comp rx=0 tx=0).stop 2024-09-10T09:58:33.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 msgr2=0x7f0020153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0026000640 1 --2- 172.21.15.110:0/1590812555 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00200cff50 0x7f0020153880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00140ab000 tx=0x7f001401ad20 comp rx=0 tx=0).stop 2024-09-10T09:58:33.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0025600640 1 -- 172.21.15.110:0/1590812555 reap_dead start 2024-09-10T09:58:33.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 shutdown_connections 2024-09-10T09:58:33.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 >> 172.21.15.110:0/1590812555 conn(0x7f002006c050 msgr2=0x7f00200b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 >> 172.21.15.110:0/798908164 conn(0x7f003c06c050 msgr2=0x7f003c0b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 shutdown_connections 2024-09-10T09:58:33.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0030c00640 1 -- 172.21.15.110:0/798908164 wait complete. 2024-09-10T09:58:33.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 shutdown_connections 2024-09-10T09:58:33.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c4e400640 1 -- 172.21.15.110:0/3942560058 --> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4c40007250 con 0x7f4c2806fcc0 2024-09-10T09:58:33.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.108+0000 7f0026000640 1 -- 172.21.15.110:0/1590812555 wait complete. 2024-09-10T09:58:33.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c3b400640 1 -- 172.21.15.110:0/3942560058 <== osd.1 v2:172.21.15.110:6808/3662700555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4c40007250 con 0x7f4c2806fcc0 2024-09-10T09:58:33.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 -- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f4c2806fcc0 msgr2=0x7f4c280720e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f4c2806fcc0 0x7f4c280720e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 -- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4c28067ac0 msgr2=0x7f4c28069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4c28067ac0 0x7f4c28069f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c400d1220 tx=0x7f4c3c008040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 -- 172.21.15.110:0/3942560058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c40004ce0 msgr2=0x7f4c40051b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 --2- 172.21.15.110:0/3942560058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c40004ce0 0x7f4c40051b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c48074670 tx=0x7f4c48069860 comp rx=0 tx=0).stop 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c4ce00640 1 -- 172.21.15.110:0/3942560058 reap_dead start 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 -- 172.21.15.110:0/3942560058 shutdown_connections 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 -- 172.21.15.110:0/3942560058 >> 172.21.15.110:0/3942560058 conn(0x7f4c4001df60 msgr2=0x7f4c400534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 -- 172.21.15.110:0/3942560058 shutdown_connections 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.112+0000 7f4c38c00640 1 -- 172.21.15.110:0/3942560058 wait complete. 2024-09-10T09:58:33.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 --> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e0c06f070 con 0x7f6dec06fc60 2024-09-10T09:58:33.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f6e02000640 1 -- 172.21.15.110:0/3247140692 <== osd.6 v2:172.21.15.145:6824/3155222893 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6e0c06f070 con 0x7f6dec06fc60 2024-09-10T09:58:33.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f6dec06fc60 msgr2=0x7f6dec072080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f6df7400640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f6dec06fc60 0x7f6dec072080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f036a600640 1 -- 172.21.15.110:0/1326881636 --> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03640762d0 con 0x7f034406fbb0 2024-09-10T09:58:33.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f035b400640 1 -- 172.21.15.110:0/1326881636 <== osd.4 v2:172.21.15.145:6816/927410831 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f03640762d0 con 0x7f034406fbb0 2024-09-10T09:58:33.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6dec067ac0 msgr2=0x7f6dec069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f6df7400640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6dec067ac0 0x7f6dec069f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0cebc0 tx=0x7f6dfc008040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c0b4040 msgr2=0x7f6e0c0ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.116+0000 7f6df7400640 1 --2- 172.21.15.110:0/3247140692 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e0c0b4040 0x7f6e0c0ac370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c450 tx=0x7f6df800c910 comp rx=0 tx=0).stop 2024-09-10T09:58:33.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f6e0a800640 1 -- 172.21.15.110:0/3247140692 reap_dead start 2024-09-10T09:58:33.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 shutdown_connections 2024-09-10T09:58:33.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 >> 172.21.15.110:0/3247140692 conn(0x7f6e0c062d10 msgr2=0x7f6e0c0cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 -- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7f034406fbb0 msgr2=0x7f0344071fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7f034406fbb0 0x7f0344071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 shutdown_connections 2024-09-10T09:58:33.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 -- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0344067990 msgr2=0x7f0344069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0344067990 0x7f0344069e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f036415cf80 tx=0x7f0354009040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 -- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036415c570 msgr2=0x7f03640b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 --2- 172.21.15.110:0/1326881636 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036415c570 0x7f03640b3020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f035000e7c0 tx=0x7f035000ec80 comp rx=0 tx=0).stop 2024-09-10T09:58:33.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0363e00640 1 -- 172.21.15.110:0/1326881636 reap_dead start 2024-09-10T09:58:33.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 -- 172.21.15.110:0/1326881636 shutdown_connections 2024-09-10T09:58:33.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 -- 172.21.15.110:0/1326881636 >> 172.21.15.110:0/1326881636 conn(0x7f036406f440 msgr2=0x7f03640b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f6df7400640 1 -- 172.21.15.110:0/3247140692 wait complete. 2024-09-10T09:58:33.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 -- 172.21.15.110:0/1326881636 shutdown_connections 2024-09-10T09:58:33.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.120+0000 7f0358c00640 1 -- 172.21.15.110:0/1326881636 wait complete. 2024-09-10T09:58:33.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.124+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 --> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc1d01607d0 con 0x7fc1b006fbb0 2024-09-10T09:58:33.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1ccc00640 1 -- 172.21.15.110:0/679461667 <== osd.3 v2:172.21.15.110:6824/4112166746 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc1d01607d0 con 0x7fc1b006fbb0 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7fc1b006fbb0 msgr2=0x7fc1b0071fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7fc1b006fbb0 0x7fc1b0071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1b0067990 msgr2=0x7fc1b0069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1b0067990 0x7fc1b0069e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00aa630 tx=0x7fc1c0018040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d01418d0 msgr2=0x7fc1d01407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 --2- 172.21.15.110:0/679461667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d01418d0 0x7fc1d01407c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1c801d1f0 tx=0x7fc1b4005840 comp rx=0 tx=0).stop 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d5400640 1 -- 172.21.15.110:0/679461667 reap_dead start 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 shutdown_connections 2024-09-10T09:58:33.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 >> 172.21.15.110:0/679461667 conn(0x7fc1d006be20 msgr2=0x7fc1d0146b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 --> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb0c0af510 con 0x7fbb0c152300 2024-09-10T09:58:33.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 shutdown_connections 2024-09-10T09:58:33.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fc1d7400640 1 -- 172.21.15.110:0/679461667 wait complete. 2024-09-10T09:58:33.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fbb03e00640 1 -- 172.21.15.110:0/156216349 <== osd.7 v2:172.21.15.145:6800/938429327 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbb0c0af510 con 0x7fbb0c152300 2024-09-10T09:58:33.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7fbb0c152300 msgr2=0x7fbb0c0ce090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.128+0000 7fbb13000640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7fbb0c152300 0x7fbb0c0ce090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbaec0679c0 msgr2=0x7fbaec069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb13000640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbaec0679c0 0x7fbaec069e80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb04001570 tx=0x7fbb04009040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb0c162ce0 msgr2=0x7fbb0c15a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb13000640 1 --2- 172.21.15.110:0/156216349 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbb0c162ce0 0x7fbb0c15a020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009740 tx=0x7fbafc01ac40 comp rx=0 tx=0).stop 2024-09-10T09:58:33.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb11000640 1 -- 172.21.15.110:0/156216349 reap_dead start 2024-09-10T09:58:33.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 shutdown_connections 2024-09-10T09:58:33.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 >> 172.21.15.110:0/156216349 conn(0x7fbb0c062d10 msgr2=0x7fbb0c0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 shutdown_connections 2024-09-10T09:58:33.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.132+0000 7fbb13000640 1 -- 172.21.15.110:0/156216349 wait complete. 2024-09-10T09:58:33.167 INFO:teuthology.orchestra.run.smithi110.stdout:64424509446 2024-09-10T09:58:33.167 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T09:58:33.168 INFO:teuthology.orchestra.run.smithi110.stdout:64424509447 2024-09-10T09:58:33.168 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T09:58:33.169 INFO:teuthology.orchestra.run.smithi110.stdout:64424509446 2024-09-10T09:58:33.169 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T09:58:33.170 INFO:teuthology.orchestra.run.smithi110.stdout:64424509447 2024-09-10T09:58:33.170 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T09:58:33.171 INFO:teuthology.orchestra.run.smithi110.stdout:64424509446 2024-09-10T09:58:33.171 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T09:58:33.172 INFO:teuthology.orchestra.run.smithi110.stdout:64424509447 2024-09-10T09:58:33.172 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T09:58:33.173 INFO:teuthology.orchestra.run.smithi110.stdout:64424509446 2024-09-10T09:58:33.173 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T09:58:33.174 INFO:teuthology.orchestra.run.smithi110.stdout:60129542152 2024-09-10T09:58:33.174 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T09:58:33.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.448+0000 7fa4e1200640 1 Processor -- start 2024-09-10T09:58:33.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.448+0000 7fa4e1200640 1 -- start start 2024-09-10T09:58:33.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4188000640 1 Processor -- start 2024-09-10T09:58:33.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4188000640 1 -- start start 2024-09-10T09:58:33.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4e1200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4dc0718d0 0x7fa4dc0b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4e1200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa4dc06f1a0 0x7fa4dc1651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4e1200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4dc0cd6c0 con 0x7fa4dc071d10 2024-09-10T09:58:33.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4e1200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa4dc06d130 con 0x7fa4dc0718d0 2024-09-10T09:58:33.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4e1200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa4dc0b4730 con 0x7fa4dc06f1a0 2024-09-10T09:58:33.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4d9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4dc0718d0 0x7fa4dc0b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4d9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4dc0718d0 0x7fa4dc0b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53220/0 (socket says 172.21.15.110:53220) 2024-09-10T09:58:33.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4d9e00640 1 -- 172.21.15.110:0/2859841285 learned_addr learned my addr 172.21.15.110:0/2859841285 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4188000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4180158230 0x7f41800d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4188000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41800d0b10 con 0x7f418006d130 2024-09-10T09:58:33.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4188000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f418015ace0 con 0x7f41800d06c0 2024-09-10T09:58:33.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4188000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f418015afc0 con 0x7f4180158230 2024-09-10T09:58:33.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4184c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4180158230 0x7f41800d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4d9e00640 1 -- 172.21.15.110:0/2859841285 >> v2:172.21.15.145:3300/0 conn(0x7fa4dc06f1a0 msgr2=0x7fa4dc1651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4d9e00640 1 --2- 172.21.15.110:0/2859841285 >> v2:172.21.15.145:3300/0 conn(0x7fa4dc06f1a0 0x7fa4dc1651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4d9e00640 1 -- 172.21.15.110:0/2859841285 >> v1:172.21.15.145:6789/0 conn(0x7fa4dc071d10 legacy=0x7fa4dc072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7fa4d9e00640 1 -- 172.21.15.110:0/2859841285 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc0b4a10 con 0x7fa4dc0718d0 2024-09-10T09:58:33.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4184c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4180158230 0x7f41800d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53232/0 (socket says 172.21.15.110:53232) 2024-09-10T09:58:33.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4184c00640 1 -- 172.21.15.110:0/94751511 learned_addr learned my addr 172.21.15.110:0/94751511 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4184c00640 1 -- 172.21.15.110:0/94751511 >> v1:172.21.15.145:6789/0 conn(0x7f41800d06c0 legacy=0x7f4180165190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:58:33.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4184c00640 1 -- 172.21.15.110:0/94751511 >> v1:172.21.15.110:6789/0 conn(0x7f418006d130 legacy=0x7f41800b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.452+0000 7f4184c00640 1 -- 172.21.15.110:0/94751511 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418015b2a0 con 0x7f4180158230 2024-09-10T09:58:33.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4d9e00640 1 --2- 172.21.15.110:0/2859841285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4dc0718d0 0x7fa4dc0b3ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc072890 tx=0x7fa4c8020680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecbb35a43aba680 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4d8a00640 1 -- 172.21.15.110:0/2859841285 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c80ae5f0 con 0x7fa4dc0718d0 2024-09-10T09:58:33.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4d8a00640 1 -- 172.21.15.110:0/2859841285 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c80b3070 con 0x7fa4dc0718d0 2024-09-10T09:58:33.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4d8a00640 1 -- 172.21.15.110:0/2859841285 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c80ae9b0 con 0x7fa4dc0718d0 2024-09-10T09:58:33.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f4184c00640 1 --2- 172.21.15.110:0/94751511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180158230 0x7f41800d0180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f417400f9b0 tx=0x7f4174020760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311a4309a1e5f2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f7c84e00640 1 Processor -- start 2024-09-10T09:58:33.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f417fe00640 1 -- 172.21.15.110:0/94751511 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41740b6070 con 0x7f4180158230 2024-09-10T09:58:33.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f417fe00640 1 -- 172.21.15.110:0/94751511 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41740a9740 con 0x7f4180158230 2024-09-10T09:58:33.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4e1200640 1 -- 172.21.15.110:0/2859841285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4dc0718d0 msgr2=0x7fa4dc0b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2859841285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4dc0718d0 0x7fa4dc0b3ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc072890 tx=0x7fa4c8020680 comp rx=0 tx=0).stop 2024-09-10T09:58:33.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f7c84e00640 1 -- start start 2024-09-10T09:58:33.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f4188000640 1 -- 172.21.15.110:0/94751511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180158230 msgr2=0x7f41800d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4e1200640 1 -- 172.21.15.110:0/2859841285 shutdown_connections 2024-09-10T09:58:33.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2859841285 >> v2:172.21.15.145:3300/0 conn(0x7fa4dc06f1a0 0x7fa4dc1651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2859841285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4dc0718d0 0x7fa4dc0b3ee0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4e1200640 1 -- 172.21.15.110:0/2859841285 >> 172.21.15.110:0/2859841285 conn(0x7fa4dc062d10 msgr2=0x7fa4dc06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4e1200640 1 -- 172.21.15.110:0/2859841285 shutdown_connections 2024-09-10T09:58:33.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f417fe00640 1 -- 172.21.15.110:0/94751511 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41740b6070 con 0x7f4180158230 2024-09-10T09:58:33.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7fa4e1200640 1 -- 172.21.15.110:0/2859841285 wait complete. 2024-09-10T09:58:33.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f4188000640 1 --2- 172.21.15.110:0/94751511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180158230 0x7f41800d0180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f417400f9b0 tx=0x7f4174020760 comp rx=0 tx=0).stop 2024-09-10T09:58:33.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f4188000640 1 -- 172.21.15.110:0/94751511 shutdown_connections 2024-09-10T09:58:33.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f4188000640 1 --2- 172.21.15.110:0/94751511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180158230 0x7f41800d0180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f4188000640 1 -- 172.21.15.110:0/94751511 >> 172.21.15.110:0/94751511 conn(0x7f41800626e0 msgr2=0x7f4180062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f4188000640 1 -- 172.21.15.110:0/94751511 shutdown_connections 2024-09-10T09:58:33.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c84e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7c800b75c0 0x7f7c80163e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c84e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c80167280 0x7f7c8006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c84e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c801676f0 con 0x7f7c801607d0 2024-09-10T09:58:33.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c84e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f7c800b1cb0 con 0x7f7c800b75c0 2024-09-10T09:58:33.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c84e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7c800b1f90 con 0x7f7c80167280 2024-09-10T09:58:33.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c7de00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f7c801607d0 0x7f7c800b6eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:46934/0 (socket says 172.21.15.110:46934) 2024-09-10T09:58:33.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4e1200640 1 Processor -- start 2024-09-10T09:58:33.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c7de00640 1 -- 172.21.15.110:0/3599971784 learned_addr learned my addr 172.21.15.110:0/3599971784 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c7e800640 1 --2- 172.21.15.110:0/3599971784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c80167280 0x7f7c8006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4e1200640 1 -- start start 2024-09-10T09:58:33.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.460+0000 7f4188000640 1 -- 172.21.15.110:0/94751511 wait complete. 2024-09-10T09:58:33.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f4188000640 1 Processor -- start 2024-09-10T09:58:33.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f2203000640 1 Processor -- start 2024-09-10T09:58:33.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa6f8e00640 1 Processor -- start 2024-09-10T09:58:33.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f4188000640 1 -- start start 2024-09-10T09:58:33.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f2203000640 1 -- start start 2024-09-10T09:58:33.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4e1200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4dc06f1a0 0x7fa4dc159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c7e800640 1 -- 172.21.15.110:0/3599971784 >> v2:172.21.15.110:3300/0 conn(0x7f7c800b75c0 msgr2=0x7f7c80163e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c7e800640 1 --2- 172.21.15.110:0/3599971784 >> v2:172.21.15.110:3300/0 conn(0x7f7c800b75c0 0x7f7c80163e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c7e800640 1 -- 172.21.15.110:0/3599971784 >> v1:172.21.15.145:6789/0 conn(0x7f7c801607d0 legacy=0x7f7c800b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7f7c7e800640 1 -- 172.21.15.110:0/3599971784 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c80163390 con 0x7f7c80167280 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4e1200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4dc0718d0 0x7fa4dc156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4e1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4dc071d10 0x7fa4dc156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4e1200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc06e0b0 con 0x7fa4dc06f1a0 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4e1200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc0b4420 con 0x7fa4dc071d10 2024-09-10T09:58:33.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4e1200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4dc06e230 con 0x7fa4dc0718d0 2024-09-10T09:58:33.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa6f8e00640 1 -- start start 2024-09-10T09:58:33.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.464+0000 7fa4da800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4dc071d10 0x7fa4dc156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4da800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4dc071d10 0x7fa4dc156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:41100/0 (socket says 172.21.15.110:41100) 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4da800640 1 -- 172.21.15.110:0/2680953781 learned_addr learned my addr 172.21.15.110:0/2680953781 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4d9400640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4dc0718d0 0x7fa4dc156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4da800640 1 -- 172.21.15.110:0/2680953781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4dc0718d0 msgr2=0x7fa4dc156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4da800640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4dc0718d0 0x7fa4dc156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4188000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f418006d130 0x7f4180204400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4da800640 1 -- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4dc06f1a0 msgr2=0x7fa4dc159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4da800640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4dc06f1a0 0x7fa4dc159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4da800640 1 -- 172.21.15.110:0/2680953781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc1554a0 con 0x7fa4dc071d10 2024-09-10T09:58:33.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4185600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f418006d130 0x7f4180204400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4185600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f418006d130 0x7f4180204400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53256/0 (socket says 172.21.15.110:53256) 2024-09-10T09:58:33.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fc5c4200640 1 Processor -- start 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fc5c4200640 1 -- start start 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4185600640 1 -- 172.21.15.110:0/2522760820 learned_addr learned my addr 172.21.15.110:0/2522760820 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4188000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41800d06c0 0x7f4180071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4da800640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4dc071d10 0x7fa4dc156860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400b9a0 tx=0x7fa4d400be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4188000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4180158230 0x7f418015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4184c00640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41800d06c0 0x7f4180071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f2203000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f21fc0b34c0 0x7f21fc0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f2203000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f21fc0cfd80 0x7f21fc0d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f2203000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21fc0d2f10 con 0x7f21fc15e850 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f2203000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f21fc0d31f0 con 0x7f21fc0cfd80 2024-09-10T09:58:33.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f2203000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f21fc0d34d0 con 0x7f21fc0b34c0 2024-09-10T09:58:33.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f8e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f40ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f8e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa6f40ce830 0x7fa6f4163f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f2a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f40ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f3400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa6f415d040 0x7fa6f4163830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38222/0 (socket says 172.21.15.110:38222) 2024-09-10T09:58:33.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f3400640 1 -- 172.21.15.110:0/2578114846 learned_addr learned my addr 172.21.15.110:0/2578114846 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4d2000640 1 -- 172.21.15.110:0/2680953781 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4019070 con 0x7fa4dc071d10 2024-09-10T09:58:33.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4d2000640 1 -- 172.21.15.110:0/2680953781 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d400cbc0 con 0x7fa4dc071d10 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa4d2000640 1 -- 172.21.15.110:0/2680953781 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4014440 con 0x7fa4dc071d10 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f2201000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f21fc0cfd80 0x7f21fc0d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f2201000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f21fc0cfd80 0x7f21fc0d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53272/0 (socket says 172.21.15.110:53272) 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f2201000640 1 -- 172.21.15.110:0/428453585 learned_addr learned my addr 172.21.15.110:0/428453585 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f8e00640 1 -- 172.21.15.110:0/2578114846 --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f40b5460 con 0x7fa6f415d040 2024-09-10T09:58:33.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f8e00640 1 -- 172.21.15.110:0/2578114846 --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f4071030 con 0x7fa6f40ce830 2024-09-10T09:58:33.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f8e00640 1 -- 172.21.15.110:0/2578114846 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f40ce0c0 con 0x7fa6f40b47e0 2024-09-10T09:58:33.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4186000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4180158230 0x7f418015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f21fb400640 1 --2- 172.21.15.110:0/428453585 >> v2:172.21.15.145:3300/0 conn(0x7f21fc0b34c0 0x7f21fc0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f41800ceb80 con 0x7f418006d130 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f41800cea00 con 0x7f4180158230 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f41800ced00 con 0x7f41800d06c0 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f3e00640 1 --2- 172.21.15.110:0/2578114846 >> v2:172.21.15.110:3300/0 conn(0x7fa6f40ce830 0x7fa6f4163f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f2000640 1 -- 172.21.15.110:0/2578114846 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2600544600 0 0) 0x7fa6f40b5460 con 0x7fa6f415d040 2024-09-10T09:58:33.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.468+0000 7fa6f2000640 1 -- 172.21.15.110:0/2578114846 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6d8003530 con 0x7fa6f415d040 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc155780 con 0x7fa4dc071d10 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fa6f2a00640 1 -- 172.21.15.110:0/2578114846 >> v2:172.21.15.110:3300/0 conn(0x7fa6f40ce830 msgr2=0x7fa6f4163f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fa6f2a00640 1 --2- 172.21.15.110:0/2578114846 >> v2:172.21.15.110:3300/0 conn(0x7fa6f40ce830 0x7fa6f4163f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f21fb400640 1 -- 172.21.15.110:0/428453585 >> v2:172.21.15.110:3300/0 conn(0x7f21fc0cfd80 msgr2=0x7f21fc0d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fa6f2a00640 1 -- 172.21.15.110:0/2578114846 >> v1:172.21.15.110:6789/0 conn(0x7fa6f415d040 legacy=0x7fa6f4163830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc155d40 con 0x7fa4dc071d10 2024-09-10T09:58:33.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f7c7e800640 1 --2- 172.21.15.110:0/3599971784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c80167280 0x7f7c8006cd60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097c0 tx=0x7f7c7001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f64215c30061738 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4186000640 1 -- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41800d06c0 msgr2=0x7f4180071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4186000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41800d06c0 0x7f4180071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4186000640 1 -- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f418006d130 msgr2=0x7f4180204400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4186000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f418006d130 0x7f4180204400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4186000640 1 -- 172.21.15.110:0/2522760820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4180072a50 con 0x7f4180158230 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4185600640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f418006d130 0x7f4180204400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4184c00640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41800d06c0 0x7f4180071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4186000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4180158230 0x7f418015c2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f417800be20 tx=0x7f417800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f417d600640 1 -- 172.21.15.110:0/2522760820 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178019070 con 0x7f4180158230 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4180072d30 con 0x7f4180158230 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4180203cc0 con 0x7f4180158230 2024-09-10T09:58:33.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f417d600640 1 -- 172.21.15.110:0/2522760820 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4178014440 con 0x7f4180158230 2024-09-10T09:58:33.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f21fb400640 1 --2- 172.21.15.110:0/428453585 >> v2:172.21.15.110:3300/0 conn(0x7f21fc0cfd80 0x7f21fc0d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f21fb400640 1 -- 172.21.15.110:0/428453585 >> v1:172.21.15.110:6789/0 conn(0x7f21fc15e850 legacy=0x7f21fc0d3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f21fb400640 1 -- 172.21.15.110:0/428453585 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc0d37b0 con 0x7f21fc0b34c0 2024-09-10T09:58:33.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fa6f2a00640 1 -- 172.21.15.110:0/2578114846 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f40b3740 con 0x7fa6f40b47e0 2024-09-10T09:58:33.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fc5c4200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc5bc15c820 0x7fc5bc15dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fc5c4200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5bc15ce00 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fc5c4200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5bc15d0e0 con 0x7fc5bc15aac0 2024-09-10T09:58:33.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fc5c4200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc5bc15d3c0 con 0x7fc5bc15c820 2024-09-10T09:58:33.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fc5c1800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc5bc15aac0 0x7fc5bc0b5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:46942/0 (socket says 172.21.15.110:46942) 2024-09-10T09:58:33.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fc5c1800640 1 -- 172.21.15.110:0/1574440752 learned_addr learned my addr 172.21.15.110:0/1574440752 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f417d600640 1 -- 172.21.15.110:0/2522760820 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178019070 con 0x7f4180158230 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4dc06e5e0 con 0x7fa4dc071d10 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f7c7ca00640 1 -- 172.21.15.110:0/3599971784 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c700b1070 con 0x7f7c80167280 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f7c7ca00640 1 -- 172.21.15.110:0/3599971784 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c700a8050 con 0x7f7c80167280 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f7c7ca00640 1 -- 172.21.15.110:0/3599971784 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c700ac860 con 0x7f7c80167280 2024-09-10T09:58:33.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7fa4d2000640 1 -- 172.21.15.110:0/2680953781 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4d401c070 con 0x7fa4dc071d10 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f7c84e00640 1 -- 172.21.15.110:0/3599971784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c80167280 msgr2=0x7f7c8006cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.472+0000 7f7c84e00640 1 --2- 172.21.15.110:0/3599971784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c80167280 0x7f7c8006cd60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097c0 tx=0x7f7c7001c3e0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d9e00640 1 Processor -- start 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5bbe00640 1 -- 172.21.15.110:0/1574440752 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781407423 0 0) 0x7fc5bc15d0e0 con 0x7fc5bc15aac0 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5bbe00640 1 -- 172.21.15.110:0/1574440752 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5a4003450 con 0x7fc5bc15aac0 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5c2200640 1 --2- 172.21.15.110:0/1574440752 >> v2:172.21.15.110:3300/0 conn(0x7fc5bc15c820 0x7fc5bc15dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f7c84e00640 1 -- 172.21.15.110:0/3599971784 shutdown_connections 2024-09-10T09:58:33.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d9e00640 1 -- start start 2024-09-10T09:58:33.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f7c84e00640 1 --2- 172.21.15.110:0/3599971784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c80167280 0x7f7c8006cd60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f7c84e00640 1 --2- 172.21.15.110:0/3599971784 >> v2:172.21.15.110:3300/0 conn(0x7f7c800b75c0 0x7f7c80163e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f7c84e00640 1 -- 172.21.15.110:0/3599971784 >> 172.21.15.110:0/3599971784 conn(0x7f7c8006f440 msgr2=0x7f7c8006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa4d2000640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4bc067910 0x7fa4bc069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa4d2000640 1 -- 172.21.15.110:0/2680953781 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa4d40857f0 con 0x7fa4dc071d10 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1ba6600640 1 Processor -- start 2024-09-10T09:58:33.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f7c84e00640 1 -- 172.21.15.110:0/3599971784 shutdown_connections 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f21fb400640 1 --2- 172.21.15.110:0/428453585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 0x7f21fc0cf840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0ba970 tx=0x7f21e801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=499b651dcbaaa520 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1ba6600640 1 -- start start 2024-09-10T09:58:33.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa4d9e00640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4bc067910 0x7fa4bc069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f7c84e00640 1 -- 172.21.15.110:0/3599971784 wait complete. 2024-09-10T09:58:33.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415c0052a0 con 0x7f4180158230 2024-09-10T09:58:33.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f2a00640 1 --2- 172.21.15.110:0/2578114846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f40ce2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00097c0 tx=0x7fa6e001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e93106478043a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f417d600640 1 -- 172.21.15.110:0/2522760820 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f417801c070 con 0x7f4180158230 2024-09-10T09:58:33.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5bbe00640 1 -- 172.21.15.110:0/1574440752 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302883639 0 0) 0x7fc5bc15ce00 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5bbe00640 1 -- 172.21.15.110:0/1574440752 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5bc15d0e0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f2000640 1 -- 172.21.15.110:0/2578114846 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b5070 con 0x7fa6f40b47e0 2024-09-10T09:58:33.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f2000640 1 -- 172.21.15.110:0/2578114846 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e00a8990 con 0x7fa6f40b47e0 2024-09-10T09:58:33.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa4d9e00640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4bc067910 0x7fa4bc069dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80b2000 tx=0x7fa4c8004740 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d9e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f47d406fb50 0x7f47d40cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5c2200640 1 -- 172.21.15.110:0/1574440752 >> v1:172.21.15.145:6789/0 conn(0x7fc5bc15aac0 legacy=0x7fc5bc0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5c2200640 1 -- 172.21.15.110:0/1574440752 >> v1:172.21.15.110:6789/0 conn(0x7fc5bc0b5f10 legacy=0x7fc5bc15c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5bbe00640 1 -- 172.21.15.110:0/1574440752 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1586127344 0 0) 0x7fc5bc15d0e0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fc5c2200640 1 -- 172.21.15.110:0/1574440752 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc15d6a0 con 0x7fc5bc15c820 2024-09-10T09:58:33.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d40d02c0 0x7f47d415ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d9e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d4071510 con 0x7f47d415c300 2024-09-10T09:58:33.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d9e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f47d40717f0 con 0x7f47d406fb50 2024-09-10T09:58:33.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d9e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f47d4070bd0 con 0x7f47d40d02c0 2024-09-10T09:58:33.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1ba6600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1ba015e850 0x7f1ba00d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1ba6600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba00d2f10 con 0x7f1ba00d10b0 2024-09-10T09:58:33.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1ba6600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba00d31f0 con 0x7f1ba00b34c0 2024-09-10T09:58:33.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1ba6600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba00d34d0 con 0x7f1ba015e850 2024-09-10T09:58:33.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d8a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f47d415c300 0x7f47d406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:46956/0 (socket says 172.21.15.110:46956) 2024-09-10T09:58:33.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d8a00640 1 -- 172.21.15.110:0/3629649513 learned_addr learned my addr 172.21.15.110:0/3629649513 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:33.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d3e00640 1 --2- 172.21.15.110:0/3629649513 >> v2:172.21.15.145:3300/0 conn(0x7f47d406fb50 0x7f47d40cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f2000640 1 -- 172.21.15.110:0/2578114846 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b5070 con 0x7fa6f40b47e0 2024-09-10T09:58:33.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f8e00640 1 -- 172.21.15.110:0/2578114846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 msgr2=0x7fa6f40ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/2578114846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f40ce2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00097c0 tx=0x7fa6e001ec90 comp rx=0 tx=0).stop 2024-09-10T09:58:33.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f47d9400640 1 --2- 172.21.15.110:0/3629649513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d40d02c0 0x7f47d415ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f417d600640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4160067cc0 0x7f416006a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f8e00640 1 -- 172.21.15.110:0/2578114846 shutdown_connections 2024-09-10T09:58:33.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/2578114846 >> v2:172.21.15.110:3300/0 conn(0x7fa6f40ce830 0x7fa6f4163f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/2578114846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f40ce2f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7fa6f8e00640 1 -- 172.21.15.110:0/2578114846 >> 172.21.15.110:0/2578114846 conn(0x7fa6f406bc50 msgr2=0x7fa6f406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1b9f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1ba015e850 0x7f1ba00d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1b9f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1ba015e850 0x7f1ba00d3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53292/0 (socket says 172.21.15.110:53292) 2024-09-10T09:58:33.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.476+0000 7f1b9f400640 1 -- 172.21.15.110:0/93689196 learned_addr learned my addr 172.21.15.110:0/93689196 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f417d600640 1 -- 172.21.15.110:0/2522760820 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f417800cc70 con 0x7f4180158230 2024-09-10T09:58:33.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7fa6f8e00640 1 -- 172.21.15.110:0/2578114846 shutdown_connections 2024-09-10T09:58:33.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f47d3e00640 1 -- 172.21.15.110:0/3629649513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d40d02c0 msgr2=0x7f47d415ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:33.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f47d3e00640 1 --2- 172.21.15.110:0/3629649513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d40d02c0 0x7f47d415ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f47d3e00640 1 -- 172.21.15.110:0/3629649513 >> v1:172.21.15.145:6789/0 conn(0x7f47d415c300 legacy=0x7f47d406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f47d3e00640 1 -- 172.21.15.110:0/3629649513 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d4070eb0 con 0x7f47d406fb50 2024-09-10T09:58:33.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f4185600640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4160067cc0 0x7f416006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7fa6f8e00640 1 -- 172.21.15.110:0/2578114846 wait complete. 2024-09-10T09:58:33.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f1b9f400640 1 -- 172.21.15.110:0/93689196 >> v1:172.21.15.145:6789/0 conn(0x7f1ba00b34c0 legacy=0x7f1ba00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f1b9f400640 1 -- 172.21.15.110:0/93689196 >> v1:172.21.15.110:6789/0 conn(0x7f1ba00d10b0 legacy=0x7f1ba00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f1b9f400640 1 -- 172.21.15.110:0/93689196 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba00d37b0 con 0x7f1ba015e850 2024-09-10T09:58:33.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c84e00640 1 Processor -- start 2024-09-10T09:58:33.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c84e00640 1 -- start start 2024-09-10T09:58:33.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f4185600640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4160067cc0 0x7f416006a180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4180073a70 tx=0x7f416c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7fa6f8e00640 1 Processor -- start 2024-09-10T09:58:33.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c84e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c800b75c0 0x7f7c80156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c84e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c801607d0 0x7f7c801543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c84e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c80167280 0x7f7c80154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c84e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c800b5780 con 0x7f7c80167280 2024-09-10T09:58:33.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c84e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7c80160bb0 con 0x7f7c800b75c0 2024-09-10T09:58:33.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c84e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8016fa00 con 0x7f7c801607d0 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7fa6f8e00640 1 -- start start 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c7e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c80167280 0x7f7c80154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c7e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c80167280 0x7f7c80154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53308/0 (socket says 172.21.15.110:53308) 2024-09-10T09:58:33.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c7e800640 1 -- 172.21.15.110:0/973049132 learned_addr learned my addr 172.21.15.110:0/973049132 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7fa6f8e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f415e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c7d400640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c801607d0 0x7f7c801543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.480+0000 7f7c7de00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c800b75c0 0x7f7c80156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c7e800640 1 -- 172.21.15.110:0/973049132 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c801607d0 msgr2=0x7f7c801543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c7e800640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c801607d0 0x7f7c801543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c7e800640 1 -- 172.21.15.110:0/973049132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c800b75c0 msgr2=0x7f7c80156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c7e800640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c800b75c0 0x7f7c80156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c7e800640 1 -- 172.21.15.110:0/973049132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8015b260 con 0x7f7c80167280 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6f40ce830 0x7fa6f4160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f8e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f415d040 0x7fa6f4160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f3400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f415e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f3400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f415e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:46744/0 (socket says 172.21.15.110:46744) 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f3400640 1 -- 172.21.15.110:0/3437718684 learned_addr learned my addr 172.21.15.110:0/3437718684 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5c2200640 1 --2- 172.21.15.110:0/1574440752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc15c820 0x7fc5bc15dea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009d90 tx=0x7fc5b401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efb67063a648d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f40b3fe0 con 0x7fa6f415d040 2024-09-10T09:58:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f40b3d00 con 0x7fa6f40ce830 2024-09-10T09:58:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f40b42c0 con 0x7fa6f40b47e0 2024-09-10T09:58:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f2a00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6f40ce830 0x7fa6f4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1b9f400640 1 --2- 172.21.15.110:0/93689196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba00d3fb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009870 tx=0x7f1b9001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9823d264cf06e7cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f47d3e00640 1 --2- 172.21.15.110:0/3629649513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47d406fb50 0x7f47d40cf780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47c00097c0 tx=0x7f47c001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7477303aa845fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f3e00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f415d040 0x7fa6f4160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5bbe00640 1 -- 172.21.15.110:0/1574440752 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40b2070 con 0x7fc5bc15c820 2024-09-10T09:58:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5bbe00640 1 -- 172.21.15.110:0/1574440752 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b40a5560 con 0x7fc5bc15c820 2024-09-10T09:58:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5bbe00640 1 -- 172.21.15.110:0/1574440752 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8001680 con 0x7fc5bc15c820 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c7e800640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c80167280 0x7f7c80154900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c7001a010 tx=0x7f7c70019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c76000640 1 -- 172.21.15.110:0/973049132 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c700bb070 con 0x7f7c80167280 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c76000640 1 -- 172.21.15.110:0/973049132 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c700b6460 con 0x7f7c80167280 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f2a00640 1 -- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 msgr2=0x7fa6f415e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c76000640 1 -- 172.21.15.110:0/973049132 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c700b1040 con 0x7f7c80167280 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f2a00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f415e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f2a00640 1 -- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f415d040 msgr2=0x7fa6f4160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f2a00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f415d040 0x7fa6f4160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f2a00640 1 -- 172.21.15.110:0/3437718684 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f4162480 con 0x7fa6f40ce830 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f3400640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f415e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:33.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1b9ea00640 1 -- 172.21.15.110:0/93689196 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900aa430 con 0x7f1ba015e850 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8015b500 con 0x7f7c80167280 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1b9ea00640 1 -- 172.21.15.110:0/93689196 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b900af070 con 0x7f1ba015e850 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1b9ea00640 1 -- 172.21.15.110:0/93689196 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900aa7f0 con 0x7f1ba015e850 2024-09-10T09:58:33.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fa6f2a00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6f40ce830 0x7fa6f4160230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00af000 tx=0x7fa6e001ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5c4200640 1 -- 172.21.15.110:0/1574440752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc15c820 msgr2=0x7fc5bc15dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c80156f60 con 0x7f7c80167280 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1ba6600640 1 -- 172.21.15.110:0/93689196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 msgr2=0x7f1ba00d3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1ba6600640 1 --2- 172.21.15.110:0/93689196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba00d3fb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009870 tx=0x7f1b9001c530 comp rx=0 tx=0).stop 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5c4200640 1 --2- 172.21.15.110:0/1574440752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc15c820 0x7fc5bc15dea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009d90 tx=0x7fc5b401c890 comp rx=0 tx=0).stop 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5c4200640 1 -- 172.21.15.110:0/1574440752 shutdown_connections 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5c4200640 1 --2- 172.21.15.110:0/1574440752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc15c820 0x7fc5bc15dea0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5c4200640 1 -- 172.21.15.110:0/1574440752 >> 172.21.15.110:0/1574440752 conn(0x7fc5bc0b0970 msgr2=0x7fc5bc0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5c4200640 1 -- 172.21.15.110:0/1574440752 shutdown_connections 2024-09-10T09:58:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7fc5c4200640 1 -- 172.21.15.110:0/1574440752 wait complete. 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1ba6600640 1 -- 172.21.15.110:0/93689196 shutdown_connections 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1ba6600640 1 --2- 172.21.15.110:0/93689196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba00d3fb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1ba6600640 1 -- 172.21.15.110:0/93689196 >> 172.21.15.110:0/93689196 conn(0x7f1ba006db90 msgr2=0x7f1ba006dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1ba6600640 1 -- 172.21.15.110:0/93689196 shutdown_connections 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f1ba6600640 1 -- 172.21.15.110:0/93689196 wait complete. 2024-09-10T09:58:33.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.484+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c80163e70 con 0x7f7c80167280 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c4200640 1 Processor -- start 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f7c76000640 1 -- 172.21.15.110:0/973049132 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c700be070 con 0x7f7c80167280 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1ba6600640 1 Processor -- start 2024-09-10T09:58:33.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1ba6600640 1 -- start start 2024-09-10T09:58:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c4200640 1 -- start start 2024-09-10T09:58:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c4200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc0b5f10 0x7fc5bc0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c4200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5bc15aac0 0x7fc5bc14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c4200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5bc15c820 0x7fc5bc14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c4200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc0ceef0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c4200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc0ced70 con 0x7fc5bc15c820 2024-09-10T09:58:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c4200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc5bc0cf070 con 0x7fc5bc15aac0 2024-09-10T09:58:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc0b5f10 0x7fc5bc0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc0b5f10 0x7fc5bc0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53318/0 (socket says 172.21.15.110:53318) 2024-09-10T09:58:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 -- 172.21.15.110:0/2439320639 learned_addr learned my addr 172.21.15.110:0/2439320639 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1ba6600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ba00b34c0 0x7f1ba00c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1ba6600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba00d10b0 0x7f1ba00b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1ba6600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba0158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1ba6600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba015edb0 con 0x7f1ba015e850 2024-09-10T09:58:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1ba6600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba015ec30 con 0x7f1ba00d10b0 2024-09-10T09:58:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1ba6600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba00b81b0 con 0x7f1ba00b34c0 2024-09-10T09:58:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba0158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f7c76000640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7c60067910 0x7f7c60069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f7c76000640 1 -- 172.21.15.110:0/973049132 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7c70120080 con 0x7f7c80167280 2024-09-10T09:58:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba0158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53328/0 (socket says 172.21.15.110:53328) 2024-09-10T09:58:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 -- 172.21.15.110:0/2282244223 learned_addr learned my addr 172.21.15.110:0/2282244223 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c0e00640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5bc15aac0 0x7fc5bc14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c2200640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5bc15c820 0x7fc5bc14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b97e00640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba00d10b0 0x7f1ba00b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9f400640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ba00b34c0 0x7f1ba00c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 -- 172.21.15.110:0/2439320639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5bc15aac0 msgr2=0x7fc5bc14dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5bc15aac0 0x7fc5bc14dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 -- 172.21.15.110:0/2439320639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5bc15c820 msgr2=0x7fc5bc14e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5bc15c820 0x7fc5bc14e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 -- 172.21.15.110:0/2439320639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc14e6f0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f7c7de00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7c60067910 0x7f7c60069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 -- 172.21.15.110:0/2282244223 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ba00b34c0 msgr2=0x7f1ba00c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ba00b34c0 0x7f1ba00c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 -- 172.21.15.110:0/2282244223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba00d10b0 msgr2=0x7f1ba00b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba00d10b0 0x7f1ba00b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 -- 172.21.15.110:0/2282244223 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba0159500 con 0x7f1ba015e850 2024-09-10T09:58:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7fc5c1800640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc0b5f10 0x7fc5bc0aeef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c4d0 tx=0x7fc5b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f1b9fe00640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba0158ab0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c710 tx=0x7f1b9800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.488+0000 7f7c7de00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7c60067910 0x7f7c60069dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c801631c0 tx=0x7f7c6c007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7fc5b9600640 1 -- 172.21.15.110:0/2439320639 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0017070 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7fc5b9600640 1 -- 172.21.15.110:0/2439320639 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b0005cf0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7fc5b9600640 1 -- 172.21.15.110:0/2439320639 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0007270 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc1525d0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7f1b97400640 1 -- 172.21.15.110:0/2282244223 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98016020 con 0x7f1ba015e850 2024-09-10T09:58:33.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7f1b97400640 1 -- 172.21.15.110:0/2282244223 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b98006d60 con 0x7f1ba015e850 2024-09-10T09:58:33.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7f1b97400640 1 -- 172.21.15.110:0/2282244223 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98003ba0 con 0x7f1ba015e850 2024-09-10T09:58:33.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba01597e0 con 0x7f1ba015e850 2024-09-10T09:58:33.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc152b50 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7f7c76000640 1 -- 172.21.15.110:0/973049132 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7c70016090 con 0x7f7c80167280 2024-09-10T09:58:33.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.492+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba00b6fb0 con 0x7f1ba015e850 2024-09-10T09:58:33.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5bc15dea0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba00d3fb0 con 0x7f1ba015e850 2024-09-10T09:58:33.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7fc5b9600640 1 -- 172.21.15.110:0/2439320639 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc5b001a070 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7f1b97400640 1 -- 172.21.15.110:0/2282244223 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1b9800a7f0 con 0x7f1ba015e850 2024-09-10T09:58:33.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7fc5b9600640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc59c0678f0 0x7fc59c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7fc5b9600640 1 -- 172.21.15.110:0/2439320639 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc5b0053440 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7fc5c0e00640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc59c0678f0 0x7fc59c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7f1b97400640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1b7c067910 0x7f1b7c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7f1b97400640 1 -- 172.21.15.110:0/2282244223 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1b98085800 con 0x7f1ba015e850 2024-09-10T09:58:33.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7fa4d2000640 1 -- 172.21.15.110:0/2680953781 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa4d4055e50 con 0x7fa4dc071d10 2024-09-10T09:58:33.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7f21faa00640 1 -- 172.21.15.110:0/428453585 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80b0070 con 0x7f21fc0b34c0 2024-09-10T09:58:33.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.496+0000 7f1b9f400640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1b7c067910 0x7f1b7c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7fc5c0e00640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc59c0678f0 0x7fc59c069db0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8001390 tx=0x7fc5a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7f1b9f400640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1b7c067910 0x7f1b7c069dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b900ae000 tx=0x7f1b900a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7f21faa00640 1 -- 172.21.15.110:0/428453585 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e80ab600 con 0x7f21fc0b34c0 2024-09-10T09:58:33.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7f21faa00640 1 -- 172.21.15.110:0/428453585 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80ab9d0 con 0x7f21fc0b34c0 2024-09-10T09:58:33.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7f47d3400640 1 -- 172.21.15.110:0/3629649513 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00b0070 con 0x7f47d406fb50 2024-09-10T09:58:33.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7f47d3400640 1 -- 172.21.15.110:0/3629649513 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c00ab600 con 0x7f47d406fb50 2024-09-10T09:58:33.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7f47d3400640 1 -- 172.21.15.110:0/3629649513 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00ab9d0 con 0x7f47d406fb50 2024-09-10T09:58:33.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7fa6eb400640 1 -- 172.21.15.110:0/3437718684 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b5040 con 0x7fa6f40ce830 2024-09-10T09:58:33.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7fa6eb400640 1 -- 172.21.15.110:0/3437718684 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e0004200 con 0x7fa6f40ce830 2024-09-10T09:58:33.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.500+0000 7fa6eb400640 1 -- 172.21.15.110:0/3437718684 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0003620 con 0x7fa6f40ce830 2024-09-10T09:58:33.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f415d840 con 0x7fa6f40ce830 2024-09-10T09:58:33.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f415de00 con 0x7fa6f40ce830 2024-09-10T09:58:33.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f2203000640 1 -- 172.21.15.110:0/428453585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 msgr2=0x7f21fc0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 -- 172.21.15.110:0/3629649513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47d406fb50 msgr2=0x7f47d40cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 --2- 172.21.15.110:0/3629649513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47d406fb50 0x7f47d40cf780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47c00097c0 tx=0x7f47c001c380 comp rx=0 tx=0).stop 2024-09-10T09:58:33.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f417d600640 1 -- 172.21.15.110:0/2522760820 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4178055a30 con 0x7f4180158230 2024-09-10T09:58:33.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f2203000640 1 --2- 172.21.15.110:0/428453585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 0x7f21fc0cf840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0ba970 tx=0x7f21e801c4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 -- 172.21.15.110:0/3629649513 shutdown_connections 2024-09-10T09:58:33.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 --2- 172.21.15.110:0/3629649513 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d40d02c0 0x7f47d415ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 --2- 172.21.15.110:0/3629649513 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47d406fb50 0x7f47d40cf780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 -- 172.21.15.110:0/3629649513 >> 172.21.15.110:0/3629649513 conn(0x7f47d40b4110 msgr2=0x7f47d40b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f2203000640 1 -- 172.21.15.110:0/428453585 shutdown_connections 2024-09-10T09:58:33.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f2203000640 1 --2- 172.21.15.110:0/428453585 >> v2:172.21.15.110:3300/0 conn(0x7f21fc0cfd80 0x7f21fc0d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f2203000640 1 --2- 172.21.15.110:0/428453585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 0x7f21fc0cf840 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f2203000640 1 -- 172.21.15.110:0/428453585 >> 172.21.15.110:0/428453585 conn(0x7f21fc06db90 msgr2=0x7f21fc06dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 -- 172.21.15.110:0/3629649513 shutdown_connections 2024-09-10T09:58:33.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 -- 172.21.15.110:0/3629649513 wait complete. 2024-09-10T09:58:33.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f2203000640 1 -- 172.21.15.110:0/428453585 shutdown_connections 2024-09-10T09:58:33.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f2203000640 1 -- 172.21.15.110:0/428453585 wait complete. 2024-09-10T09:58:33.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f40b3310 con 0x7fa6f40ce830 2024-09-10T09:58:33.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f1b97400640 1 -- 172.21.15.110:0/2282244223 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1b98020020 con 0x7f1ba015e850 2024-09-10T09:58:33.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 Processor -- start 2024-09-10T09:58:33.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.504+0000 7f47d9e00640 1 -- start start 2024-09-10T09:58:33.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2203000640 1 Processor -- start 2024-09-10T09:58:33.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d406fb50 0x7f47d415d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47d40d02c0 0x7f47d415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47d415c300 0x7f47d41ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41693d0 con 0x7f47d415c300 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4169250 con 0x7f47d40d02c0 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f47d4169550 con 0x7f47d406fb50 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2203000640 1 -- start start 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2203000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 0x7f21fc154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2203000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21fc0cfd80 0x7f21fc0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2203000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21fc15e850 0x7f21fc0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2203000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc1694a0 con 0x7f21fc15e850 2024-09-10T09:58:33.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2203000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc0b8440 con 0x7f21fc0b34c0 2024-09-10T09:58:33.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2203000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc169620 con 0x7f21fc0cfd80 2024-09-10T09:58:33.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7fa6eb400640 1 -- 172.21.15.110:0/3437718684 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6e00c1030 con 0x7fa6f40ce830 2024-09-10T09:58:33.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47d415c300 0x7f47d41ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47d415c300 0x7f47d41ef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53336/0 (socket says 172.21.15.110:53336) 2024-09-10T09:58:33.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9400640 1 -- 172.21.15.110:0/2557755297 learned_addr learned my addr 172.21.15.110:0/2557755297 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7fa6eb400640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa6d4067910 0x7fa6d4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2201000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21fc15e850 0x7f21fc0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2201000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21fc15e850 0x7f21fc0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53338/0 (socket says 172.21.15.110:53338) 2024-09-10T09:58:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f2201000640 1 -- 172.21.15.110:0/1253032739 learned_addr learned my addr 172.21.15.110:0/1253032739 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f21fbe00640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 0x7f21fc154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f21fb400640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21fc0cfd80 0x7f21fc0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7fa6eb400640 1 -- 172.21.15.110:0/3437718684 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e01225d0 con 0x7fa6f40ce830 2024-09-10T09:58:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7fa6f3400640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa6d4067910 0x7fa6d4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d8a00640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d406fb50 0x7f47d415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f2201000640 1 -- 172.21.15.110:0/1253032739 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21fc0cfd80 msgr2=0x7f21fc0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f2201000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21fc0cfd80 0x7f21fc0b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f2201000640 1 -- 172.21.15.110:0/1253032739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 msgr2=0x7f21fc154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f2201000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 0x7f21fc154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f2201000640 1 -- 172.21.15.110:0/1253032739 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc0b6200 con 0x7f21fc15e850 2024-09-10T09:58:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7fa6f3400640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa6d4067910 0x7fa6d4069dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007fd0 tx=0x7fa6e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9400640 1 -- 172.21.15.110:0/2557755297 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d406fb50 msgr2=0x7f47d415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9400640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d406fb50 0x7f47d415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9400640 1 -- 172.21.15.110:0/2557755297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47d40d02c0 msgr2=0x7f47d415abb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:58:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9400640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47d40d02c0 0x7f47d415abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.508+0000 7f47d9400640 1 -- 172.21.15.110:0/2557755297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d415d840 con 0x7f47d415c300 2024-09-10T09:58:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f47d9400640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47d415c300 0x7f47d41ef8d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f47cc007e40 tx=0x7f47cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f2201000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21fc15e850 0x7f21fc0ce180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21f400efc0 tx=0x7f21f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f47d0c00640 1 -- 172.21.15.110:0/2557755297 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc014620 con 0x7f47d415c300 2024-09-10T09:58:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f47d0c00640 1 -- 172.21.15.110:0/2557755297 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc019070 con 0x7f47d415c300 2024-09-10T09:58:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f47d0c00640 1 -- 172.21.15.110:0/2557755297 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc014a20 con 0x7f47d415c300 2024-09-10T09:58:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7fa6eb400640 1 -- 172.21.15.110:0/3437718684 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa6e00c8060 con 0x7fa6f40ce830 2024-09-10T09:58:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f21f3e00640 1 -- 172.21.15.110:0/1253032739 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f400cd20 con 0x7f21fc15e850 2024-09-10T09:58:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f21f3e00640 1 -- 172.21.15.110:0/1253032739 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f40039a0 con 0x7f21fc15e850 2024-09-10T09:58:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f21f3e00640 1 -- 172.21.15.110:0/1253032739 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4003d90 con 0x7f21fc15e850 2024-09-10T09:58:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f47d9e00640 1 -- 172.21.15.110:0/2557755297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d4157af0 con 0x7f47d415c300 2024-09-10T09:58:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.512+0000 7f47d9e00640 1 -- 172.21.15.110:0/2557755297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d41eeb10 con 0x7f47d415c300 2024-09-10T09:58:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.516+0000 7fc5b9600640 1 -- 172.21.15.110:0/2439320639 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc5bc14e6f0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.516+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc0b64e0 con 0x7f21fc15e850 2024-09-10T09:58:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.516+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc0b6aa0 con 0x7f21fc15e850 2024-09-10T09:58:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.516+0000 7f47ca000640 1 -- 172.21.15.110:0/2557755297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a80052a0 con 0x7f47d415c300 2024-09-10T09:58:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.516+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc0cf840 con 0x7f21fc15e850 2024-09-10T09:58:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.520+0000 7f47d0c00640 1 -- 172.21.15.110:0/2557755297 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47cc014bc0 con 0x7f47d415c300 2024-09-10T09:58:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.520+0000 7f47d0c00640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f47b40678c0 0x7f47b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.524+0000 7f47d8a00640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f47b40678c0 0x7f47b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.524+0000 7f47d0c00640 1 -- 172.21.15.110:0/2557755297 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f47cc0559a0 con 0x7f47d415c300 2024-09-10T09:58:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.524+0000 7f21f3e00640 1 -- 172.21.15.110:0/1253032739 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f21f4012050 con 0x7f21fc15e850 2024-09-10T09:58:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.524+0000 7f21f3e00640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21dc067940 0x7f21dc069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:33.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.528+0000 7f47d8a00640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f47b40678c0 0x7f47b4069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47c4007fd0 tx=0x7f47c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.528+0000 7f21fbe00640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21dc067940 0x7f21dc069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:33.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.528+0000 7f21fbe00640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21dc067940 0x7f21dc069e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21ec007fd0 tx=0x7f21ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:33.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.532+0000 7f21f3e00640 1 -- 172.21.15.110:0/1253032739 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f21f40554d0 con 0x7f21fc15e850 2024-09-10T09:58:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.532+0000 7f47d0c00640 1 -- 172.21.15.110:0/2557755297 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f47d415d840 con 0x7f47d415c300 2024-09-10T09:58:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.552+0000 7f21f3e00640 1 -- 172.21.15.110:0/1253032739 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f21f4010040 con 0x7f21fc15e850 2024-09-10T09:58:33.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.816+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7c8015a0e0 con 0x7f7c80167280 2024-09-10T09:58:33.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.820+0000 7f7c76000640 1 -- 172.21.15.110:0/973049132 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7c700f06e0 con 0x7f7c80167280 2024-09-10T09:58:33.823 INFO:teuthology.orchestra.run.smithi110.stdout:64424509446 2024-09-10T09:58:33.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7c60067910 msgr2=0x7f7c60069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7c60067910 0x7f7c60069dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c801631c0 tx=0x7f7c6c007e90 comp rx=0 tx=0).stop 2024-09-10T09:58:33.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c80167280 msgr2=0x7f7c80154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c80167280 0x7f7c80154900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c7001a010 tx=0x7f7c70019180 comp rx=0 tx=0).stop 2024-09-10T09:58:33.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 shutdown_connections 2024-09-10T09:58:33.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7c60067910 0x7f7c60069dd0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c801631c0 tx=0x7f7c6c007e90 comp rx=0 tx=0).stop 2024-09-10T09:58:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c80167280 0x7f7c80154900 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7c801607d0 0x7f7c801543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 --2- 172.21.15.110:0/973049132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c800b75c0 0x7f7c80156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.824+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 >> 172.21.15.110:0/973049132 conn(0x7f7c8006f440 msgr2=0x7f7c8006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.828+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 shutdown_connections 2024-09-10T09:58:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.828+0000 7f7c84e00640 1 -- 172.21.15.110:0/973049132 wait complete. 2024-09-10T09:58:33.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.828+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f415c005860 con 0x7f4180158230 2024-09-10T09:58:33.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.832+0000 7f417d600640 1 -- 172.21.15.110:0/2522760820 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4178059d30 con 0x7f4180158230 2024-09-10T09:58:33.833 INFO:teuthology.orchestra.run.smithi110.stdout:64424509446 2024-09-10T09:58:33.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4160067cc0 msgr2=0x7f416006a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4160067cc0 0x7f416006a180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4180073a70 tx=0x7f416c009040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4180158230 msgr2=0x7f418015c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4180158230 0x7f418015c2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f417800be20 tx=0x7f417800bf10 comp rx=0 tx=0).stop 2024-09-10T09:58:33.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 shutdown_connections 2024-09-10T09:58:33.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4160067cc0 0x7f416006a180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4180158230 0x7f418015c2e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41800d06c0 0x7f4180071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 --2- 172.21.15.110:0/2522760820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f418006d130 0x7f4180204400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 >> 172.21.15.110:0/2522760820 conn(0x7f41800626e0 msgr2=0x7f41800b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 shutdown_connections 2024-09-10T09:58:33.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.836+0000 7f4188000640 1 -- 172.21.15.110:0/2522760820 wait complete. 2024-09-10T09:58:33.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.844+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa4dc155f40 con 0x7fa4dc071d10 2024-09-10T09:58:33.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.848+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1ba00d0810 con 0x7f1ba015e850 2024-09-10T09:58:33.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.848+0000 7f1b97400640 1 -- 172.21.15.110:0/2282244223 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b98083e20 con 0x7f1ba015e850 2024-09-10T09:58:33.850 INFO:teuthology.orchestra.run.smithi110.stdout:64424509447 2024-09-10T09:58:33.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7fa4d2000640 1 -- 172.21.15.110:0/2680953781 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa4d405a150 con 0x7fa4dc071d10 2024-09-10T09:58:33.853 INFO:teuthology.orchestra.run.smithi110.stdout:64424509447 2024-09-10T09:58:33.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa6f4165fb0 con 0x7fa6f40ce830 2024-09-10T09:58:33.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1b7c067910 msgr2=0x7f1b7c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1b7c067910 0x7f1b7c069dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b900ae000 tx=0x7f1b900a7000 comp rx=0 tx=0).stop 2024-09-10T09:58:33.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 msgr2=0x7f1ba0158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba0158ab0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c710 tx=0x7f1b9800cbd0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 shutdown_connections 2024-09-10T09:58:33.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1b7c067910 0x7f1b7c069dd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ba015e850 0x7f1ba0158ab0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba00d10b0 0x7f1ba00b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 --2- 172.21.15.110:0/2282244223 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ba00b34c0 0x7f1ba00c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 >> 172.21.15.110:0/2282244223 conn(0x7f1ba006db90 msgr2=0x7f1ba0154c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 shutdown_connections 2024-09-10T09:58:33.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7f1ba6600640 1 -- 172.21.15.110:0/2282244223 wait complete. 2024-09-10T09:58:33.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4bc067910 msgr2=0x7fa4bc069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4bc067910 0x7fa4bc069dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80b2000 tx=0x7fa4c8004740 comp rx=0 tx=0).stop 2024-09-10T09:58:33.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4dc071d10 msgr2=0x7fa4dc156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4dc071d10 0x7fa4dc156860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400b9a0 tx=0x7fa4d400be60 comp rx=0 tx=0).stop 2024-09-10T09:58:33.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 shutdown_connections 2024-09-10T09:58:33.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4bc067910 0x7fa4bc069dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4dc071d10 0x7fa4dc156860 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4dc0718d0 0x7fa4dc156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 --2- 172.21.15.110:0/2680953781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4dc06f1a0 0x7fa4dc159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 >> 172.21.15.110:0/2680953781 conn(0x7fa4dc062d10 msgr2=0x7fa4dc0ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.852+0000 7fa6eb400640 1 -- 172.21.15.110:0/3437718684 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa6e00187f0 con 0x7fa6f40ce830 2024-09-10T09:58:33.858 INFO:teuthology.orchestra.run.smithi110.stdout:64424509446 2024-09-10T09:58:33.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 shutdown_connections 2024-09-10T09:58:33.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa4e1200640 1 -- 172.21.15.110:0/2680953781 wait complete. 2024-09-10T09:58:33.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa6d4067910 msgr2=0x7fa6d4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa6d4067910 0x7fa6d4069dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007fd0 tx=0x7fa6e4009040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6f40ce830 msgr2=0x7fa6f4160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.856+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6f40ce830 0x7fa6f4160230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00af000 tx=0x7fa6e001ba30 comp rx=0 tx=0).stop 2024-09-10T09:58:33.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.860+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 shutdown_connections 2024-09-10T09:58:33.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.860+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa6d4067910 0x7fa6d4069dd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.860+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6f415d040 0x7fa6f4160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.860+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6f40ce830 0x7fa6f4160230 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.860+0000 7fa6f8e00640 1 --2- 172.21.15.110:0/3437718684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa6f40b47e0 0x7fa6f415e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.860+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 >> 172.21.15.110:0/3437718684 conn(0x7fa6f406bc50 msgr2=0x7fa6f40b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.860+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 shutdown_connections 2024-09-10T09:58:33.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.860+0000 7fa6f8e00640 1 -- 172.21.15.110:0/3437718684 wait complete. 2024-09-10T09:58:33.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-09-10T09:58:33.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.864+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc5bc0b4280 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.866 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:33.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.864+0000 7fc5b9600640 1 -- 172.21.15.110:0/2439320639 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc5b00553a0 con 0x7fc5bc0b5f10 2024-09-10T09:58:33.866 INFO:teuthology.orchestra.run.smithi110.stdout:64424509446 2024-09-10T09:58:33.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc59c0678f0 msgr2=0x7fc59c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc59c0678f0 0x7fc59c069db0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8001390 tx=0x7fc5a8009040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc0b5f10 msgr2=0x7fc5bc0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc0b5f10 0x7fc5bc0aeef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c4d0 tx=0x7fc5b000c990 comp rx=0 tx=0).stop 2024-09-10T09:58:33.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 shutdown_connections 2024-09-10T09:58:33.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc59c0678f0 0x7fc59c069db0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5bc15c820 0x7fc5bc14e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5bc15aac0 0x7fc5bc14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 --2- 172.21.15.110:0/2439320639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5bc0b5f10 0x7fc5bc0aeef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 >> 172.21.15.110:0/2439320639 conn(0x7fc5bc0b0970 msgr2=0x7fc5bc1590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 shutdown_connections 2024-09-10T09:58:33.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.868+0000 7fc5c4200640 1 -- 172.21.15.110:0/2439320639 wait complete. 2024-09-10T09:58:33.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-10T09:58:33.879 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:33.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.880+0000 7f47ca000640 1 -- 172.21.15.110:0/2557755297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f47a8005860 con 0x7f47d415c300 2024-09-10T09:58:33.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.880+0000 7f47d0c00640 1 -- 172.21.15.110:0/2557755297 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47cc058000 con 0x7f47d415c300 2024-09-10T09:58:33.883 INFO:teuthology.orchestra.run.smithi110.stdout:60129542152 2024-09-10T09:58:33.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 -- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f47b40678c0 msgr2=0x7f47b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f47b40678c0 0x7f47b4069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47c4007fd0 tx=0x7f47c4009040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 -- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47d415c300 msgr2=0x7f47d41ef8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47d415c300 0x7f47d41ef8d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f47cc007e40 tx=0x7f47cc00e4b0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 -- 172.21.15.110:0/2557755297 shutdown_connections 2024-09-10T09:58:33.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f47b40678c0 0x7f47b4069d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47d415c300 0x7f47d41ef8d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47d40d02c0 0x7f47d415abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 --2- 172.21.15.110:0/2557755297 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47d406fb50 0x7f47d415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 -- 172.21.15.110:0/2557755297 >> 172.21.15.110:0/2557755297 conn(0x7f47d40b4110 msgr2=0x7f47d40b6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 -- 172.21.15.110:0/2557755297 shutdown_connections 2024-09-10T09:58:33.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.884+0000 7f47ca000640 1 -- 172.21.15.110:0/2557755297 wait complete. 2024-09-10T09:58:33.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.888+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f21fc15b640 con 0x7f21fc15e850 2024-09-10T09:58:33.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-09-10T09:58:33.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.888+0000 7f21f3e00640 1 -- 172.21.15.110:0/1253032739 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f21f4057b30 con 0x7f21fc15e850 2024-09-10T09:58:33.893 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:33.893 INFO:teuthology.orchestra.run.smithi110.stdout:64424509447 2024-09-10T09:58:33.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-09-10T09:58:33.894 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21dc067940 msgr2=0x7f21dc069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21dc067940 0x7f21dc069e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21ec007fd0 tx=0x7f21ec009040 comp rx=0 tx=0).stop 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21fc15e850 msgr2=0x7f21fc0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21fc15e850 0x7f21fc0ce180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21f400efc0 tx=0x7f21f400c490 comp rx=0 tx=0).stop 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 shutdown_connections 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21dc067940 0x7f21dc069e00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21fc15e850 0x7f21fc0ce180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21fc0cfd80 0x7f21fc0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 --2- 172.21.15.110:0/1253032739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21fc0b34c0 0x7f21fc154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 >> 172.21.15.110:0/1253032739 conn(0x7f21fc06db90 msgr2=0x7f21fc0b9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 shutdown_connections 2024-09-10T09:58:33.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:33.892+0000 7f2203000640 1 -- 172.21.15.110:0/1253032739 wait complete. 2024-09-10T09:58:33.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2024-09-10T09:58:33.900 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:33.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-09-10T09:58:33.908 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:33.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.3 2024-09-10T09:58:33.913 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:33.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-09-10T09:58:33.918 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:33.919 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T09:58:33.919 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:58:34.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.092+0000 7f7e18200640 1 Processor -- start 2024-09-10T09:58:34.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.092+0000 7f7e18200640 1 -- start start 2024-09-10T09:58:34.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.092+0000 7f7e18200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1006f1e0 0x7f7e10166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.092+0000 7f7e18200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e10155d50 con 0x7f7e10155970 2024-09-10T09:58:34.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.092+0000 7f7e18200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e100b34d0 con 0x7f7e1015c560 2024-09-10T09:58:34.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.092+0000 7f7e18200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7e100b4570 con 0x7f7e1006f1e0 2024-09-10T09:58:34.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e15800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f7e10155970 0x7f7e100cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38268/0 (socket says 172.21.15.110:38268) 2024-09-10T09:58:34.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e15800640 1 -- 172.21.15.110:0/2186391715 learned_addr learned my addr 172.21.15.110:0/2186391715 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2023170554 0 0) 0x7f7e10155d50 con 0x7f7e10155970 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7df8005740 con 0x7f7e10155970 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e16200640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1006f1e0 0x7f7e10166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1670967866 0 0) 0x7f7df8005740 con 0x7f7e10155970 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1006f1e0 msgr2=0x7f7e10166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1006f1e0 0x7f7e10166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 >> v1:172.21.15.145:6789/0 conn(0x7f7e1015c560 legacy=0x7f7e1006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e100b37b0 con 0x7f7e10155970 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f7e04001a20 con 0x7f7e10155970 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 >> v1:172.21.15.110:6789/0 conn(0x7f7e10155970 legacy=0x7f7e100cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8005c00 0x7f7df8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df8006a20 0x7f7df8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1006f1e0 0x7f7df80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7df800dc90 con 0x7f7df8006a20 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7df800de40 con 0x7f7df8005c00 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7df800e040 con 0x7f7e1006f1e0 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e14e00640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df8006a20 0x7f7df8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e15800640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8005c00 0x7f7df8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e16200640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1006f1e0 0x7f7df80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e14e00640 1 -- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1006f1e0 msgr2=0x7f7df80073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e14e00640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1006f1e0 0x7f7df80073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e14e00640 1 -- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8005c00 msgr2=0x7f7df8005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e14e00640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8005c00 0x7f7df8005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.096+0000 7f7e14e00640 1 -- 172.21.15.110:0/2186391715 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df8005740 con 0x7f7df8006a20 2024-09-10T09:58:34.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e14e00640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df8006a20 0x7f7df8006ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc007fd0 tx=0x7f7dfc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ab35dfe00f8aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0b5070 con 0x7f7df8006a20 2024-09-10T09:58:34.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dfc0a8830 con 0x7f7df8006a20 2024-09-10T09:58:34.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e0fe00640 1 -- 172.21.15.110:0/2186391715 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc001460 con 0x7f7df8006a20 2024-09-10T09:58:34.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e18200640 1 -- 172.21.15.110:0/2186391715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df8006a20 msgr2=0x7f7df8006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e18200640 1 --2- 172.21.15.110:0/2186391715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7df8006a20 0x7f7df8006ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc007fd0 tx=0x7f7dfc01ec10 comp rx=0 tx=0).stop 2024-09-10T09:58:34.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e16200640 1 -- 172.21.15.110:0/2186391715 reap_dead start 2024-09-10T09:58:34.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e18200640 1 -- 172.21.15.110:0/2186391715 shutdown_connections 2024-09-10T09:58:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e18200640 1 -- 172.21.15.110:0/2186391715 >> 172.21.15.110:0/2186391715 conn(0x7f7e1006bc50 msgr2=0x7f7e1006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e18200640 1 -- 172.21.15.110:0/2186391715 shutdown_connections 2024-09-10T09:58:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e18200640 1 -- 172.21.15.110:0/2186391715 wait complete. 2024-09-10T09:58:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e18200640 1 Processor -- start 2024-09-10T09:58:34.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.100+0000 7f7e18200640 1 -- start start 2024-09-10T09:58:34.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8006a20 0x7f7e10152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e1015ca50 0x7f7e1015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1015f5a0 0x7f7e10157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7e100b4050 con 0x7f7e1015ca50 2024-09-10T09:58:34.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7e100b3d70 con 0x7f7df8006a20 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7e100cf430 con 0x7f7e1015f5a0 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e1015ca50 0x7f7e1015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e1015ca50 0x7f7e1015f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53366/0 (socket says 172.21.15.110:53366) 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 -- 172.21.15.110:0/1553692255 learned_addr learned my addr 172.21.15.110:0/1553692255 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e15800640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8006a20 0x7f7e10152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e16200640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1015f5a0 0x7f7e10157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 -- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1015f5a0 msgr2=0x7f7e10157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1015f5a0 0x7f7e10157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 -- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8006a20 msgr2=0x7f7e10152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8006a20 0x7f7e10152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 -- 172.21.15.110:0/1553692255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1015fab0 con 0x7f7e1015ca50 2024-09-10T09:58:34.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e16200640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1015f5a0 0x7f7e10157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:34.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e15800640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8006a20 0x7f7e10152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:34.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e14e00640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e1015ca50 0x7f7e1015f060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc01ea40 tx=0x7f7df8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e0d600640 1 -- 172.21.15.110:0/1553692255 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0c1070 con 0x7f7e1015ca50 2024-09-10T09:58:34.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e0d600640 1 -- 172.21.15.110:0/1553692255 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dfc0a89e0 con 0x7f7e1015ca50 2024-09-10T09:58:34.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e10159b10 con 0x7f7e1015ca50 2024-09-10T09:58:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e0d600640 1 -- 172.21.15.110:0/1553692255 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0b5040 con 0x7f7e1015ca50 2024-09-10T09:58:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1015a0d0 con 0x7f7e1015ca50 2024-09-10T09:58:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e100cc8c0 con 0x7f7e1015ca50 2024-09-10T09:58:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.104+0000 7f7e0d600640 1 -- 172.21.15.110:0/1553692255 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7dfc0c4070 con 0x7f7e1015ca50 2024-09-10T09:58:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.108+0000 7f7e0d600640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7df00678c0 0x7f7df0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.108+0000 7f7e0d600640 1 -- 172.21.15.110:0/1553692255 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7dfc122be0 con 0x7f7e1015ca50 2024-09-10T09:58:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.108+0000 7f7e15800640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7df00678c0 0x7f7df0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.108+0000 7f7e15800640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7df00678c0 0x7f7df0069d80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e100733a0 tx=0x7f7e04006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.116+0000 7f7e0d600640 1 -- 172.21.15.110:0/1553692255 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7dfc018920 con 0x7f7e1015ca50 2024-09-10T09:58:34.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.272+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e1015bbd0 con 0x7f7df00678c0 2024-09-10T09:58:34.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.276+0000 7f7e0d600640 1 -- 172.21.15.110:0/1553692255 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f7e1015bbd0 con 0x7f7df00678c0 2024-09-10T09:58:34.282 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:34.283 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:58:34.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7df00678c0 msgr2=0x7f7df0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7df00678c0 0x7f7df0069d80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e100733a0 tx=0x7f7e04006040 comp rx=0 tx=0).stop 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e1015ca50 msgr2=0x7f7e1015f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e1015ca50 0x7f7e1015f060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc01ea40 tx=0x7f7df8007a50 comp rx=0 tx=0).stop 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 shutdown_connections 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7df00678c0 0x7f7df0069d80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e1015f5a0 0x7f7e10157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e1015ca50 0x7f7e1015f060 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 --2- 172.21.15.110:0/1553692255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7df8006a20 0x7f7e10152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 >> 172.21.15.110:0/1553692255 conn(0x7f7e1006bc50 msgr2=0x7f7e100b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 shutdown_connections 2024-09-10T09:58:34.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.280+0000 7f7e18200640 1 -- 172.21.15.110:0/1553692255 wait complete. 2024-09-10T09:58:34.302 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T09:58:33.357719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235916,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738804,"statfs":{"total":773094113280,"available":772852535296,"internally_reserved":0,"allocated":3121152,"data_stored":1833196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002379"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739344+0000","last_change":"2024-09-10T09:58:22.803813+0000","last_active":"2024-09-10T09:58:24.739344+0000","last_peered":"2024-09-10T09:58:24.739344+0000","last_clean":"2024-09-10T09:58:24.739344+0000","last_became_active":"2024-09-10T09:58:22.803440+0000","last_became_peered":"2024-09-10T09:58:22.803440+0000","last_unstale":"2024-09-10T09:58:24.739344+0000","last_undegraded":"2024-09-10T09:58:24.739344+0000","last_fullsized":"2024-09-10T09:58:24.739344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:41.057291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782118+0000","last_change":"2024-09-10T09:58:22.786389+0000","last_active":"2024-09-10T09:58:23.782118+0000","last_peered":"2024-09-10T09:58:23.782118+0000","last_clean":"2024-09-10T09:58:23.782118+0000","last_became_active":"2024-09-10T09:58:22.786093+0000","last_became_peered":"2024-09-10T09:58:22.786093+0000","last_unstale":"2024-09-10T09:58:23.782118+0000","last_undegraded":"2024-09-10T09:58:23.782118+0000","last_fullsized":"2024-09-10T09:58:23.782118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738194+0000","last_change":"2024-09-10T09:58:22.782747+0000","last_active":"2024-09-10T09:58:24.738194+0000","last_peered":"2024-09-10T09:58:24.738194+0000","last_clean":"2024-09-10T09:58:24.738194+0000","last_became_active":"2024-09-10T09:58:22.782493+0000","last_became_peered":"2024-09-10T09:58:22.782493+0000","last_unstale":"2024-09-10T09:58:24.738194+0000","last_undegraded":"2024-09-10T09:58:24.738194+0000","last_fullsized":"2024-09-10T09:58:24.738194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738202+0000","last_change":"2024-09-10T09:58:22.780919+0000","last_active":"2024-09-10T09:58:24.738202+0000","last_peered":"2024-09-10T09:58:24.738202+0000","last_clean":"2024-09-10T09:58:24.738202+0000","last_became_active":"2024-09-10T09:58:22.779369+0000","last_became_peered":"2024-09-10T09:58:22.779369+0000","last_unstale":"2024-09-10T09:58:24.738202+0000","last_undegraded":"2024-09-10T09:58:24.738202+0000","last_fullsized":"2024-09-10T09:58:24.738202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:17.698136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738364+0000","last_change":"2024-09-10T09:58:22.803835+0000","last_active":"2024-09-10T09:58:24.738364+0000","last_peered":"2024-09-10T09:58:24.738364+0000","last_clean":"2024-09-10T09:58:24.738364+0000","last_became_active":"2024-09-10T09:58:22.803465+0000","last_became_peered":"2024-09-10T09:58:22.803465+0000","last_unstale":"2024-09-10T09:58:24.738364+0000","last_undegraded":"2024-09-10T09:58:24.738364+0000","last_fullsized":"2024-09-10T09:58:24.738364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:17.698136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737987+0000","last_change":"2024-09-10T09:58:22.804773+0000","last_active":"2024-09-10T09:58:24.737987+0000","last_peered":"2024-09-10T09:58:24.737987+0000","last_clean":"2024-09-10T09:58:24.737987+0000","last_became_active":"2024-09-10T09:58:22.801323+0000","last_became_peered":"2024-09-10T09:58:22.801323+0000","last_unstale":"2024-09-10T09:58:24.737987+0000","last_undegraded":"2024-09-10T09:58:24.737987+0000","last_fullsized":"2024-09-10T09:58:24.737987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:36.116325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737772+0000","last_change":"2024-09-10T09:58:22.790265+0000","last_active":"2024-09-10T09:58:24.737772+0000","last_peered":"2024-09-10T09:58:24.737772+0000","last_clean":"2024-09-10T09:58:24.737772+0000","last_became_active":"2024-09-10T09:58:22.789359+0000","last_became_peered":"2024-09-10T09:58:22.789359+0000","last_unstale":"2024-09-10T09:58:24.737772+0000","last_undegraded":"2024-09-10T09:58:24.737772+0000","last_fullsized":"2024-09-10T09:58:24.737772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:36.116325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737654+0000","last_change":"2024-09-10T09:58:22.790696+0000","last_active":"2024-09-10T09:58:24.737654+0000","last_peered":"2024-09-10T09:58:24.737654+0000","last_clean":"2024-09-10T09:58:24.737654+0000","last_became_active":"2024-09-10T09:58:22.790265+0000","last_became_peered":"2024-09-10T09:58:22.790265+0000","last_unstale":"2024-09-10T09:58:24.737654+0000","last_undegraded":"2024-09-10T09:58:24.737654+0000","last_fullsized":"2024-09-10T09:58:24.737654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:20.430762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739480+0000","last_change":"2024-09-10T09:58:22.806568+0000","last_active":"2024-09-10T09:58:24.739480+0000","last_peered":"2024-09-10T09:58:24.739480+0000","last_clean":"2024-09-10T09:58:24.739480+0000","last_became_active":"2024-09-10T09:58:22.806203+0000","last_became_peered":"2024-09-10T09:58:22.806203+0000","last_unstale":"2024-09-10T09:58:24.739480+0000","last_undegraded":"2024-09-10T09:58:24.739480+0000","last_fullsized":"2024-09-10T09:58:24.739480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:17.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773231+0000","last_change":"2024-09-10T09:58:22.785593+0000","last_active":"2024-09-10T09:58:23.773231+0000","last_peered":"2024-09-10T09:58:23.773231+0000","last_clean":"2024-09-10T09:58:23.773231+0000","last_became_active":"2024-09-10T09:58:22.785227+0000","last_became_peered":"2024-09-10T09:58:22.785227+0000","last_unstale":"2024-09-10T09:58:23.773231+0000","last_undegraded":"2024-09-10T09:58:23.773231+0000","last_fullsized":"2024-09-10T09:58:23.773231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772786+0000","last_change":"2024-09-10T09:58:22.792764+0000","last_active":"2024-09-10T09:58:23.772786+0000","last_peered":"2024-09-10T09:58:23.772786+0000","last_clean":"2024-09-10T09:58:23.772786+0000","last_became_active":"2024-09-10T09:58:22.791958+0000","last_became_peered":"2024-09-10T09:58:22.791958+0000","last_unstale":"2024-09-10T09:58:23.772786+0000","last_undegraded":"2024-09-10T09:58:23.772786+0000","last_fullsized":"2024-09-10T09:58:23.772786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773090+0000","last_change":"2024-09-10T09:58:22.774413+0000","last_active":"2024-09-10T09:58:23.773090+0000","last_peered":"2024-09-10T09:58:23.773090+0000","last_clean":"2024-09-10T09:58:23.773090+0000","last_became_active":"2024-09-10T09:58:22.774110+0000","last_became_peered":"2024-09-10T09:58:22.774110+0000","last_unstale":"2024-09-10T09:58:23.773090+0000","last_undegraded":"2024-09-10T09:58:23.773090+0000","last_fullsized":"2024-09-10T09:58:23.773090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:27.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738573+0000","last_change":"2024-09-10T09:58:22.768744+0000","last_active":"2024-09-10T09:58:24.738573+0000","last_peered":"2024-09-10T09:58:24.738573+0000","last_clean":"2024-09-10T09:58:24.738573+0000","last_became_active":"2024-09-10T09:58:22.768515+0000","last_became_peered":"2024-09-10T09:58:22.768515+0000","last_unstale":"2024-09-10T09:58:24.738573+0000","last_undegraded":"2024-09-10T09:58:24.738573+0000","last_fullsized":"2024-09-10T09:58:24.738573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:37.491552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737347+0000","last_change":"2024-09-10T09:58:22.797642+0000","last_active":"2024-09-10T09:58:24.737347+0000","last_peered":"2024-09-10T09:58:24.737347+0000","last_clean":"2024-09-10T09:58:24.737347+0000","last_became_active":"2024-09-10T09:58:22.797304+0000","last_became_peered":"2024-09-10T09:58:22.797304+0000","last_unstale":"2024-09-10T09:58:24.737347+0000","last_undegraded":"2024-09-10T09:58:24.737347+0000","last_fullsized":"2024-09-10T09:58:24.737347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:59.080268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781863+0000","last_change":"2024-09-10T09:58:22.799237+0000","last_active":"2024-09-10T09:58:23.781863+0000","last_peered":"2024-09-10T09:58:23.781863+0000","last_clean":"2024-09-10T09:58:23.781863+0000","last_became_active":"2024-09-10T09:58:22.798951+0000","last_became_peered":"2024-09-10T09:58:22.798951+0000","last_unstale":"2024-09-10T09:58:23.781863+0000","last_undegraded":"2024-09-10T09:58:23.781863+0000","last_fullsized":"2024-09-10T09:58:23.781863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738069+0000","last_change":"2024-09-10T09:58:22.801607+0000","last_active":"2024-09-10T09:58:24.738069+0000","last_peered":"2024-09-10T09:58:24.738069+0000","last_clean":"2024-09-10T09:58:24.738069+0000","last_became_active":"2024-09-10T09:58:22.801252+0000","last_became_peered":"2024-09-10T09:58:22.801252+0000","last_unstale":"2024-09-10T09:58:24.738069+0000","last_undegraded":"2024-09-10T09:58:24.738069+0000","last_fullsized":"2024-09-10T09:58:24.738069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739077+0000","last_change":"2024-09-10T09:58:22.800761+0000","last_active":"2024-09-10T09:58:24.739077+0000","last_peered":"2024-09-10T09:58:24.739077+0000","last_clean":"2024-09-10T09:58:24.739077+0000","last_became_active":"2024-09-10T09:58:22.800421+0000","last_became_peered":"2024-09-10T09:58:22.800421+0000","last_unstale":"2024-09-10T09:58:24.739077+0000","last_undegraded":"2024-09-10T09:58:24.739077+0000","last_fullsized":"2024-09-10T09:58:24.739077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:17.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770538+0000","last_change":"2024-09-10T09:58:22.780000+0000","last_active":"2024-09-10T09:58:23.770538+0000","last_peered":"2024-09-10T09:58:23.770538+0000","last_clean":"2024-09-10T09:58:23.770538+0000","last_became_active":"2024-09-10T09:58:22.779739+0000","last_became_peered":"2024-09-10T09:58:22.779739+0000","last_unstale":"2024-09-10T09:58:23.770538+0000","last_undegraded":"2024-09-10T09:58:23.770538+0000","last_fullsized":"2024-09-10T09:58:23.770538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:48.157725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739018+0000","last_change":"2024-09-10T09:58:22.789943+0000","last_active":"2024-09-10T09:58:24.739018+0000","last_peered":"2024-09-10T09:58:24.739018+0000","last_clean":"2024-09-10T09:58:24.739018+0000","last_became_active":"2024-09-10T09:58:22.789657+0000","last_became_peered":"2024-09-10T09:58:22.789657+0000","last_unstale":"2024-09-10T09:58:24.739018+0000","last_undegraded":"2024-09-10T09:58:24.739018+0000","last_fullsized":"2024-09-10T09:58:24.739018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:17.386153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772836+0000","last_change":"2024-09-10T09:58:22.795150+0000","last_active":"2024-09-10T09:58:23.772836+0000","last_peered":"2024-09-10T09:58:23.772836+0000","last_clean":"2024-09-10T09:58:23.772836+0000","last_became_active":"2024-09-10T09:58:22.794828+0000","last_became_peered":"2024-09-10T09:58:22.794828+0000","last_unstale":"2024-09-10T09:58:23.772836+0000","last_undegraded":"2024-09-10T09:58:23.772836+0000","last_fullsized":"2024-09-10T09:58:23.772836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737641+0000","last_change":"2024-09-10T09:58:22.804668+0000","last_active":"2024-09-10T09:58:24.737641+0000","last_peered":"2024-09-10T09:58:24.737641+0000","last_clean":"2024-09-10T09:58:24.737641+0000","last_became_active":"2024-09-10T09:58:22.804314+0000","last_became_peered":"2024-09-10T09:58:22.804314+0000","last_unstale":"2024-09-10T09:58:24.737641+0000","last_undegraded":"2024-09-10T09:58:24.737641+0000","last_fullsized":"2024-09-10T09:58:24.737641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:34.641953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738097+0000","last_change":"2024-09-10T09:58:22.800575+0000","last_active":"2024-09-10T09:58:24.738097+0000","last_peered":"2024-09-10T09:58:24.738097+0000","last_clean":"2024-09-10T09:58:24.738097+0000","last_became_active":"2024-09-10T09:58:22.800254+0000","last_became_peered":"2024-09-10T09:58:22.800254+0000","last_unstale":"2024-09-10T09:58:24.738097+0000","last_undegraded":"2024-09-10T09:58:24.738097+0000","last_fullsized":"2024-09-10T09:58:24.738097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:12.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738073+0000","last_change":"2024-09-10T09:58:22.799342+0000","last_active":"2024-09-10T09:58:24.738073+0000","last_peered":"2024-09-10T09:58:24.738073+0000","last_clean":"2024-09-10T09:58:24.738073+0000","last_became_active":"2024-09-10T09:58:22.799049+0000","last_became_peered":"2024-09-10T09:58:22.799049+0000","last_unstale":"2024-09-10T09:58:24.738073+0000","last_undegraded":"2024-09-10T09:58:24.738073+0000","last_fullsized":"2024-09-10T09:58:24.738073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:24.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737824+0000","last_change":"2024-09-10T09:58:22.786910+0000","last_active":"2024-09-10T09:58:24.737824+0000","last_peered":"2024-09-10T09:58:24.737824+0000","last_clean":"2024-09-10T09:58:24.737824+0000","last_became_active":"2024-09-10T09:58:22.784972+0000","last_became_peered":"2024-09-10T09:58:22.784972+0000","last_unstale":"2024-09-10T09:58:24.737824+0000","last_undegraded":"2024-09-10T09:58:24.737824+0000","last_fullsized":"2024-09-10T09:58:24.737824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737548+0000","last_change":"2024-09-10T09:58:22.789676+0000","last_active":"2024-09-10T09:58:24.737548+0000","last_peered":"2024-09-10T09:58:24.737548+0000","last_clean":"2024-09-10T09:58:24.737548+0000","last_became_active":"2024-09-10T09:58:22.789249+0000","last_became_peered":"2024-09-10T09:58:22.789249+0000","last_unstale":"2024-09-10T09:58:24.737548+0000","last_undegraded":"2024-09-10T09:58:24.737548+0000","last_fullsized":"2024-09-10T09:58:24.737548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:03.582635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773346+0000","last_change":"2024-09-10T09:58:22.793654+0000","last_active":"2024-09-10T09:58:23.773346+0000","last_peered":"2024-09-10T09:58:23.773346+0000","last_clean":"2024-09-10T09:58:23.773346+0000","last_became_active":"2024-09-10T09:58:22.793394+0000","last_became_peered":"2024-09-10T09:58:22.793394+0000","last_unstale":"2024-09-10T09:58:23.773346+0000","last_undegraded":"2024-09-10T09:58:23.773346+0000","last_fullsized":"2024-09-10T09:58:23.773346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782057+0000","last_change":"2024-09-10T09:58:22.766616+0000","last_active":"2024-09-10T09:58:23.782057+0000","last_peered":"2024-09-10T09:58:23.782057+0000","last_clean":"2024-09-10T09:58:23.782057+0000","last_became_active":"2024-09-10T09:58:22.766321+0000","last_became_peered":"2024-09-10T09:58:22.766321+0000","last_unstale":"2024-09-10T09:58:23.782057+0000","last_undegraded":"2024-09-10T09:58:23.782057+0000","last_fullsized":"2024-09-10T09:58:23.782057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738554+0000","last_change":"2024-09-10T09:58:22.796593+0000","last_active":"2024-09-10T09:58:24.738554+0000","last_peered":"2024-09-10T09:58:24.738554+0000","last_clean":"2024-09-10T09:58:24.738554+0000","last_became_active":"2024-09-10T09:58:22.796294+0000","last_became_peered":"2024-09-10T09:58:22.796294+0000","last_unstale":"2024-09-10T09:58:24.738554+0000","last_undegraded":"2024-09-10T09:58:24.738554+0000","last_fullsized":"2024-09-10T09:58:24.738554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:43.026541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737668+0000","last_change":"2024-09-10T09:58:22.790593+0000","last_active":"2024-09-10T09:58:24.737668+0000","last_peered":"2024-09-10T09:58:24.737668+0000","last_clean":"2024-09-10T09:58:24.737668+0000","last_became_active":"2024-09-10T09:58:22.790244+0000","last_became_peered":"2024-09-10T09:58:22.790244+0000","last_unstale":"2024-09-10T09:58:24.737668+0000","last_undegraded":"2024-09-10T09:58:24.737668+0000","last_fullsized":"2024-09-10T09:58:24.737668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:18.106190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738741+0000","last_change":"2024-09-10T09:58:22.784426+0000","last_active":"2024-09-10T09:58:24.738741+0000","last_peered":"2024-09-10T09:58:24.738741+0000","last_clean":"2024-09-10T09:58:24.738741+0000","last_became_active":"2024-09-10T09:58:22.784101+0000","last_became_peered":"2024-09-10T09:58:22.784101+0000","last_unstale":"2024-09-10T09:58:24.738741+0000","last_undegraded":"2024-09-10T09:58:24.738741+0000","last_fullsized":"2024-09-10T09:58:24.738741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:39.940923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770388+0000","last_change":"2024-09-10T09:58:22.785741+0000","last_active":"2024-09-10T09:58:23.770388+0000","last_peered":"2024-09-10T09:58:23.770388+0000","last_clean":"2024-09-10T09:58:23.770388+0000","last_became_active":"2024-09-10T09:58:22.785508+0000","last_became_peered":"2024-09-10T09:58:22.785508+0000","last_unstale":"2024-09-10T09:58:23.770388+0000","last_undegraded":"2024-09-10T09:58:23.770388+0000","last_fullsized":"2024-09-10T09:58:23.770388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:49.472990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772262+0000","last_change":"2024-09-10T09:58:22.776565+0000","last_active":"2024-09-10T09:58:23.772262+0000","last_peered":"2024-09-10T09:58:23.772262+0000","last_clean":"2024-09-10T09:58:23.772262+0000","last_became_active":"2024-09-10T09:58:22.776167+0000","last_became_peered":"2024-09-10T09:58:22.776167+0000","last_unstale":"2024-09-10T09:58:23.772262+0000","last_undegraded":"2024-09-10T09:58:23.772262+0000","last_fullsized":"2024-09-10T09:58:23.772262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772714+0000","last_change":"2024-09-10T09:58:22.776703+0000","last_active":"2024-09-10T09:58:23.772714+0000","last_peered":"2024-09-10T09:58:23.772714+0000","last_clean":"2024-09-10T09:58:23.772714+0000","last_became_active":"2024-09-10T09:58:22.776418+0000","last_became_peered":"2024-09-10T09:58:22.776418+0000","last_unstale":"2024-09-10T09:58:23.772714+0000","last_undegraded":"2024-09-10T09:58:23.772714+0000","last_fullsized":"2024-09-10T09:58:23.772714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:59.498264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738624+0000","last_change":"2024-09-10T09:58:21.783061+0000","last_active":"2024-09-10T09:58:24.738624+0000","last_peered":"2024-09-10T09:58:24.738624+0000","last_clean":"2024-09-10T09:58:24.738624+0000","last_became_active":"2024-09-10T09:58:21.782756+0000","last_became_peered":"2024-09-10T09:58:21.782756+0000","last_unstale":"2024-09-10T09:58:24.738624+0000","last_undegraded":"2024-09-10T09:58:24.738624+0000","last_fullsized":"2024-09-10T09:58:24.738624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:39.702619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737843+0000","last_change":"2024-09-10T09:58:22.800136+0000","last_active":"2024-09-10T09:58:24.737843+0000","last_peered":"2024-09-10T09:58:24.737843+0000","last_clean":"2024-09-10T09:58:24.737843+0000","last_became_active":"2024-09-10T09:58:22.799760+0000","last_became_peered":"2024-09-10T09:58:22.799760+0000","last_unstale":"2024-09-10T09:58:24.737843+0000","last_undegraded":"2024-09-10T09:58:24.737843+0000","last_fullsized":"2024-09-10T09:58:24.737843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772303+0000","last_change":"2024-09-10T09:58:21.778244+0000","last_active":"2024-09-10T09:58:23.772303+0000","last_peered":"2024-09-10T09:58:23.772303+0000","last_clean":"2024-09-10T09:58:23.772303+0000","last_became_active":"2024-09-10T09:58:21.777937+0000","last_became_peered":"2024-09-10T09:58:21.777937+0000","last_unstale":"2024-09-10T09:58:23.772303+0000","last_undegraded":"2024-09-10T09:58:23.772303+0000","last_fullsized":"2024-09-10T09:58:23.772303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738137+0000","last_change":"2024-09-10T09:58:21.775701+0000","last_active":"2024-09-10T09:58:24.738137+0000","last_peered":"2024-09-10T09:58:24.738137+0000","last_clean":"2024-09-10T09:58:24.738137+0000","last_became_active":"2024-09-10T09:58:21.775431+0000","last_became_peered":"2024-09-10T09:58:21.775431+0000","last_unstale":"2024-09-10T09:58:24.738137+0000","last_undegraded":"2024-09-10T09:58:24.738137+0000","last_fullsized":"2024-09-10T09:58:24.738137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:03.845811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737371+0000","last_change":"2024-09-10T09:58:22.772570+0000","last_active":"2024-09-10T09:58:24.737371+0000","last_peered":"2024-09-10T09:58:24.737371+0000","last_clean":"2024-09-10T09:58:24.737371+0000","last_became_active":"2024-09-10T09:58:22.772248+0000","last_became_peered":"2024-09-10T09:58:22.772248+0000","last_unstale":"2024-09-10T09:58:24.737371+0000","last_undegraded":"2024-09-10T09:58:24.737371+0000","last_fullsized":"2024-09-10T09:58:24.737371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:55.062755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737937+0000","last_change":"2024-09-10T09:58:21.786530+0000","last_active":"2024-09-10T09:58:24.737937+0000","last_peered":"2024-09-10T09:58:24.737937+0000","last_clean":"2024-09-10T09:58:24.737937+0000","last_became_active":"2024-09-10T09:58:21.786157+0000","last_became_peered":"2024-09-10T09:58:21.786157+0000","last_unstale":"2024-09-10T09:58:24.737937+0000","last_undegraded":"2024-09-10T09:58:24.737937+0000","last_fullsized":"2024-09-10T09:58:24.737937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781837+0000","last_change":"2024-09-10T09:58:22.767449+0000","last_active":"2024-09-10T09:58:23.781837+0000","last_peered":"2024-09-10T09:58:23.781837+0000","last_clean":"2024-09-10T09:58:23.781837+0000","last_became_active":"2024-09-10T09:58:22.767182+0000","last_became_peered":"2024-09-10T09:58:22.767182+0000","last_unstale":"2024-09-10T09:58:23.781837+0000","last_undegraded":"2024-09-10T09:58:23.781837+0000","last_fullsized":"2024-09-10T09:58:23.781837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773334+0000","last_change":"2024-09-10T09:58:21.786956+0000","last_active":"2024-09-10T09:58:23.773334+0000","last_peered":"2024-09-10T09:58:23.773334+0000","last_clean":"2024-09-10T09:58:23.773334+0000","last_became_active":"2024-09-10T09:58:21.786523+0000","last_became_peered":"2024-09-10T09:58:21.786523+0000","last_unstale":"2024-09-10T09:58:23.773334+0000","last_undegraded":"2024-09-10T09:58:23.773334+0000","last_fullsized":"2024-09-10T09:58:23.773334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738905+0000","last_change":"2024-09-10T09:58:22.801352+0000","last_active":"2024-09-10T09:58:24.738905+0000","last_peered":"2024-09-10T09:58:24.738905+0000","last_clean":"2024-09-10T09:58:24.738905+0000","last_became_active":"2024-09-10T09:58:22.800233+0000","last_became_peered":"2024-09-10T09:58:22.800233+0000","last_unstale":"2024-09-10T09:58:24.738905+0000","last_undegraded":"2024-09-10T09:58:24.738905+0000","last_fullsized":"2024-09-10T09:58:24.738905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:41.057291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739047+0000","last_change":"2024-09-10T09:58:21.787382+0000","last_active":"2024-09-10T09:58:24.739047+0000","last_peered":"2024-09-10T09:58:24.739047+0000","last_clean":"2024-09-10T09:58:24.739047+0000","last_became_active":"2024-09-10T09:58:21.787048+0000","last_became_peered":"2024-09-10T09:58:21.787048+0000","last_unstale":"2024-09-10T09:58:24.739047+0000","last_undegraded":"2024-09-10T09:58:24.739047+0000","last_fullsized":"2024-09-10T09:58:24.739047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738181+0000","last_change":"2024-09-10T09:58:22.792565+0000","last_active":"2024-09-10T09:58:24.738181+0000","last_peered":"2024-09-10T09:58:24.738181+0000","last_clean":"2024-09-10T09:58:24.738181+0000","last_became_active":"2024-09-10T09:58:22.792257+0000","last_became_peered":"2024-09-10T09:58:22.792257+0000","last_unstale":"2024-09-10T09:58:24.738181+0000","last_undegraded":"2024-09-10T09:58:24.738181+0000","last_fullsized":"2024-09-10T09:58:24.738181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782153+0000","last_change":"2024-09-10T09:58:21.785126+0000","last_active":"2024-09-10T09:58:23.782153+0000","last_peered":"2024-09-10T09:58:23.782153+0000","last_clean":"2024-09-10T09:58:23.782153+0000","last_became_active":"2024-09-10T09:58:21.784839+0000","last_became_peered":"2024-09-10T09:58:21.784839+0000","last_unstale":"2024-09-10T09:58:23.782153+0000","last_undegraded":"2024-09-10T09:58:23.782153+0000","last_fullsized":"2024-09-10T09:58:23.782153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773200+0000","last_change":"2024-09-10T09:58:22.786530+0000","last_active":"2024-09-10T09:58:23.773200+0000","last_peered":"2024-09-10T09:58:23.773200+0000","last_clean":"2024-09-10T09:58:23.773200+0000","last_became_active":"2024-09-10T09:58:22.782828+0000","last_became_peered":"2024-09-10T09:58:22.782828+0000","last_unstale":"2024-09-10T09:58:23.773200+0000","last_undegraded":"2024-09-10T09:58:23.773200+0000","last_fullsized":"2024-09-10T09:58:23.773200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:18.151980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772938+0000","last_change":"2024-09-10T09:58:21.776340+0000","last_active":"2024-09-10T09:58:23.772938+0000","last_peered":"2024-09-10T09:58:23.772938+0000","last_clean":"2024-09-10T09:58:23.772938+0000","last_became_active":"2024-09-10T09:58:21.776066+0000","last_became_peered":"2024-09-10T09:58:21.776066+0000","last_unstale":"2024-09-10T09:58:23.772938+0000","last_undegraded":"2024-09-10T09:58:23.772938+0000","last_fullsized":"2024-09-10T09:58:23.772938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738448+0000","last_change":"2024-09-10T09:58:22.785299+0000","last_active":"2024-09-10T09:58:24.738448+0000","last_peered":"2024-09-10T09:58:24.738448+0000","last_clean":"2024-09-10T09:58:24.738448+0000","last_became_active":"2024-09-10T09:58:22.784958+0000","last_became_peered":"2024-09-10T09:58:22.784958+0000","last_unstale":"2024-09-10T09:58:24.738448+0000","last_undegraded":"2024-09-10T09:58:24.738448+0000","last_fullsized":"2024-09-10T09:58:24.738448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:26.963772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737976+0000","last_change":"2024-09-10T09:58:21.783558+0000","last_active":"2024-09-10T09:58:24.737976+0000","last_peered":"2024-09-10T09:58:24.737976+0000","last_clean":"2024-09-10T09:58:24.737976+0000","last_became_active":"2024-09-10T09:58:21.782455+0000","last_became_peered":"2024-09-10T09:58:21.782455+0000","last_unstale":"2024-09-10T09:58:24.737976+0000","last_undegraded":"2024-09-10T09:58:24.737976+0000","last_fullsized":"2024-09-10T09:58:24.737976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737404+0000","last_change":"2024-09-10T09:58:22.796235+0000","last_active":"2024-09-10T09:58:24.737404+0000","last_peered":"2024-09-10T09:58:24.737404+0000","last_clean":"2024-09-10T09:58:24.737404+0000","last_became_active":"2024-09-10T09:58:22.793164+0000","last_became_peered":"2024-09-10T09:58:22.793164+0000","last_unstale":"2024-09-10T09:58:24.737404+0000","last_undegraded":"2024-09-10T09:58:24.737404+0000","last_fullsized":"2024-09-10T09:58:24.737404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738233+0000","last_change":"2024-09-10T09:58:21.789974+0000","last_active":"2024-09-10T09:58:24.738233+0000","last_peered":"2024-09-10T09:58:24.738233+0000","last_clean":"2024-09-10T09:58:24.738233+0000","last_became_active":"2024-09-10T09:58:21.788481+0000","last_became_peered":"2024-09-10T09:58:21.788481+0000","last_unstale":"2024-09-10T09:58:24.738233+0000","last_undegraded":"2024-09-10T09:58:24.738233+0000","last_fullsized":"2024-09-10T09:58:24.738233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738521+0000","last_change":"2024-09-10T09:58:22.801237+0000","last_active":"2024-09-10T09:58:24.738521+0000","last_peered":"2024-09-10T09:58:24.738521+0000","last_clean":"2024-09-10T09:58:24.738521+0000","last_became_active":"2024-09-10T09:58:22.800975+0000","last_became_peered":"2024-09-10T09:58:22.800975+0000","last_unstale":"2024-09-10T09:58:24.738521+0000","last_undegraded":"2024-09-10T09:58:24.738521+0000","last_fullsized":"2024-09-10T09:58:24.738521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:26.981758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772229+0000","last_change":"2024-09-10T09:58:21.778738+0000","last_active":"2024-09-10T09:58:23.772229+0000","last_peered":"2024-09-10T09:58:23.772229+0000","last_clean":"2024-09-10T09:58:23.772229+0000","last_became_active":"2024-09-10T09:58:21.777982+0000","last_became_peered":"2024-09-10T09:58:21.777982+0000","last_unstale":"2024-09-10T09:58:23.772229+0000","last_undegraded":"2024-09-10T09:58:23.772229+0000","last_fullsized":"2024-09-10T09:58:23.772229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772461+0000","last_change":"2024-09-10T09:58:22.775403+0000","last_active":"2024-09-10T09:58:23.772461+0000","last_peered":"2024-09-10T09:58:23.772461+0000","last_clean":"2024-09-10T09:58:23.772461+0000","last_became_active":"2024-09-10T09:58:22.775043+0000","last_became_peered":"2024-09-10T09:58:22.775043+0000","last_unstale":"2024-09-10T09:58:23.772461+0000","last_undegraded":"2024-09-10T09:58:23.772461+0000","last_fullsized":"2024-09-10T09:58:23.772461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770292+0000","last_change":"2024-09-10T09:58:21.779750+0000","last_active":"2024-09-10T09:58:23.770292+0000","last_peered":"2024-09-10T09:58:23.770292+0000","last_clean":"2024-09-10T09:58:23.770292+0000","last_became_active":"2024-09-10T09:58:21.779492+0000","last_became_peered":"2024-09-10T09:58:21.779492+0000","last_unstale":"2024-09-10T09:58:23.770292+0000","last_undegraded":"2024-09-10T09:58:23.770292+0000","last_fullsized":"2024-09-10T09:58:23.770292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737852+0000","last_change":"2024-09-10T09:58:22.801768+0000","last_active":"2024-09-10T09:58:24.737852+0000","last_peered":"2024-09-10T09:58:24.737852+0000","last_clean":"2024-09-10T09:58:24.737852+0000","last_became_active":"2024-09-10T09:58:22.801503+0000","last_became_peered":"2024-09-10T09:58:22.801503+0000","last_unstale":"2024-09-10T09:58:24.737852+0000","last_undegraded":"2024-09-10T09:58:24.737852+0000","last_fullsized":"2024-09-10T09:58:24.737852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:12.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772798+0000","last_change":"2024-09-10T09:58:21.781792+0000","last_active":"2024-09-10T09:58:23.772798+0000","last_peered":"2024-09-10T09:58:23.772798+0000","last_clean":"2024-09-10T09:58:23.772798+0000","last_became_active":"2024-09-10T09:58:21.781434+0000","last_became_peered":"2024-09-10T09:58:21.781434+0000","last_unstale":"2024-09-10T09:58:23.772798+0000","last_undegraded":"2024-09-10T09:58:23.772798+0000","last_fullsized":"2024-09-10T09:58:23.772798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738537+0000","last_change":"2024-09-10T09:58:22.804739+0000","last_active":"2024-09-10T09:58:24.738537+0000","last_peered":"2024-09-10T09:58:24.738537+0000","last_clean":"2024-09-10T09:58:24.738537+0000","last_became_active":"2024-09-10T09:58:22.804451+0000","last_became_peered":"2024-09-10T09:58:22.804451+0000","last_unstale":"2024-09-10T09:58:24.738537+0000","last_undegraded":"2024-09-10T09:58:24.738537+0000","last_fullsized":"2024-09-10T09:58:24.738537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:39.940923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770561+0000","last_change":"2024-09-10T09:58:21.781287+0000","last_active":"2024-09-10T09:58:23.770561+0000","last_peered":"2024-09-10T09:58:23.770561+0000","last_clean":"2024-09-10T09:58:23.770561+0000","last_became_active":"2024-09-10T09:58:21.780824+0000","last_became_peered":"2024-09-10T09:58:21.780824+0000","last_unstale":"2024-09-10T09:58:23.770561+0000","last_undegraded":"2024-09-10T09:58:23.770561+0000","last_fullsized":"2024-09-10T09:58:23.770561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:00.039731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739601+0000","last_change":"2024-09-10T09:58:22.802541+0000","last_active":"2024-09-10T09:58:24.739601+0000","last_peered":"2024-09-10T09:58:24.739601+0000","last_clean":"2024-09-10T09:58:24.739601+0000","last_became_active":"2024-09-10T09:58:22.802216+0000","last_became_peered":"2024-09-10T09:58:22.802216+0000","last_unstale":"2024-09-10T09:58:24.739601+0000","last_undegraded":"2024-09-10T09:58:24.739601+0000","last_fullsized":"2024-09-10T09:58:24.739601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:56.685285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782186+0000","last_change":"2024-09-10T09:58:21.776106+0000","last_active":"2024-09-10T09:58:23.782186+0000","last_peered":"2024-09-10T09:58:23.782186+0000","last_clean":"2024-09-10T09:58:23.782186+0000","last_became_active":"2024-09-10T09:58:21.773315+0000","last_became_peered":"2024-09-10T09:58:21.773315+0000","last_unstale":"2024-09-10T09:58:23.782186+0000","last_undegraded":"2024-09-10T09:58:23.782186+0000","last_fullsized":"2024-09-10T09:58:23.782186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738418+0000","last_change":"2024-09-10T09:58:22.779675+0000","last_active":"2024-09-10T09:58:24.738418+0000","last_peered":"2024-09-10T09:58:24.738418+0000","last_clean":"2024-09-10T09:58:24.738418+0000","last_became_active":"2024-09-10T09:58:22.779088+0000","last_became_peered":"2024-09-10T09:58:22.779088+0000","last_unstale":"2024-09-10T09:58:24.738418+0000","last_undegraded":"2024-09-10T09:58:24.738418+0000","last_fullsized":"2024-09-10T09:58:24.738418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:26.981758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770459+0000","last_change":"2024-09-10T09:58:21.789986+0000","last_active":"2024-09-10T09:58:23.770459+0000","last_peered":"2024-09-10T09:58:23.770459+0000","last_clean":"2024-09-10T09:58:23.770459+0000","last_became_active":"2024-09-10T09:58:21.789577+0000","last_became_peered":"2024-09-10T09:58:21.789577+0000","last_unstale":"2024-09-10T09:58:23.770459+0000","last_undegraded":"2024-09-10T09:58:23.770459+0000","last_fullsized":"2024-09-10T09:58:23.770459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738630+0000","last_change":"2024-09-10T09:58:22.806455+0000","last_active":"2024-09-10T09:58:24.738630+0000","last_peered":"2024-09-10T09:58:24.738630+0000","last_clean":"2024-09-10T09:58:24.738630+0000","last_became_active":"2024-09-10T09:58:22.806181+0000","last_became_peered":"2024-09-10T09:58:22.806181+0000","last_unstale":"2024-09-10T09:58:24.738630+0000","last_undegraded":"2024-09-10T09:58:24.738630+0000","last_fullsized":"2024-09-10T09:58:24.738630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:32.590352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739373+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:58:24.739373+0000","last_peered":"2024-09-10T09:58:24.739373+0000","last_clean":"2024-09-10T09:58:24.739373+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:58:24.739373+0000","last_undegraded":"2024-09-10T09:58:24.739373+0000","last_fullsized":"2024-09-10T09:58:24.739373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737745+0000","last_change":"2024-09-10T09:58:21.789884+0000","last_active":"2024-09-10T09:58:24.737745+0000","last_peered":"2024-09-10T09:58:24.737745+0000","last_clean":"2024-09-10T09:58:24.737745+0000","last_became_active":"2024-09-10T09:58:21.788940+0000","last_became_peered":"2024-09-10T09:58:21.788940+0000","last_unstale":"2024-09-10T09:58:24.737745+0000","last_undegraded":"2024-09-10T09:58:24.737745+0000","last_fullsized":"2024-09-10T09:58:24.737745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737853+0000","last_change":"2024-09-10T09:58:22.792386+0000","last_active":"2024-09-10T09:58:24.737853+0000","last_peered":"2024-09-10T09:58:24.737853+0000","last_clean":"2024-09-10T09:58:24.737853+0000","last_became_active":"2024-09-10T09:58:22.792151+0000","last_became_peered":"2024-09-10T09:58:22.792151+0000","last_unstale":"2024-09-10T09:58:24.737853+0000","last_undegraded":"2024-09-10T09:58:24.737853+0000","last_fullsized":"2024-09-10T09:58:24.737853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.810620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737643+0000","last_change":"2024-09-10T09:58:21.791063+0000","last_active":"2024-09-10T09:58:24.737643+0000","last_peered":"2024-09-10T09:58:24.737643+0000","last_clean":"2024-09-10T09:58:24.737643+0000","last_became_active":"2024-09-10T09:58:21.790796+0000","last_became_peered":"2024-09-10T09:58:21.790796+0000","last_unstale":"2024-09-10T09:58:24.737643+0000","last_undegraded":"2024-09-10T09:58:24.737643+0000","last_fullsized":"2024-09-10T09:58:24.737643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738301+0000","last_change":"2024-09-10T09:58:22.798826+0000","last_active":"2024-09-10T09:58:24.738301+0000","last_peered":"2024-09-10T09:58:24.738301+0000","last_clean":"2024-09-10T09:58:24.738301+0000","last_became_active":"2024-09-10T09:58:22.798497+0000","last_became_peered":"2024-09-10T09:58:22.798497+0000","last_unstale":"2024-09-10T09:58:24.738301+0000","last_undegraded":"2024-09-10T09:58:24.738301+0000","last_fullsized":"2024-09-10T09:58:24.738301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770375+0000","last_change":"2024-09-10T09:58:21.777208+0000","last_active":"2024-09-10T09:58:23.770375+0000","last_peered":"2024-09-10T09:58:23.770375+0000","last_clean":"2024-09-10T09:58:23.770375+0000","last_became_active":"2024-09-10T09:58:21.776820+0000","last_became_peered":"2024-09-10T09:58:21.776820+0000","last_unstale":"2024-09-10T09:58:23.770375+0000","last_undegraded":"2024-09-10T09:58:23.770375+0000","last_fullsized":"2024-09-10T09:58:23.770375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781864+0000","last_change":"2024-09-10T09:58:22.771438+0000","last_active":"2024-09-10T09:58:23.781864+0000","last_peered":"2024-09-10T09:58:23.781864+0000","last_clean":"2024-09-10T09:58:23.781864+0000","last_became_active":"2024-09-10T09:58:22.771255+0000","last_became_peered":"2024-09-10T09:58:22.771255+0000","last_unstale":"2024-09-10T09:58:23.781864+0000","last_undegraded":"2024-09-10T09:58:23.781864+0000","last_fullsized":"2024-09-10T09:58:23.781864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738323+0000","last_change":"2024-09-10T09:58:21.785388+0000","last_active":"2024-09-10T09:58:24.738323+0000","last_peered":"2024-09-10T09:58:24.738323+0000","last_clean":"2024-09-10T09:58:24.738323+0000","last_became_active":"2024-09-10T09:58:21.785227+0000","last_became_peered":"2024-09-10T09:58:21.785227+0000","last_unstale":"2024-09-10T09:58:24.738323+0000","last_undegraded":"2024-09-10T09:58:24.738323+0000","last_fullsized":"2024-09-10T09:58:24.738323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772655+0000","last_change":"2024-09-10T09:58:22.798575+0000","last_active":"2024-09-10T09:58:23.772655+0000","last_peered":"2024-09-10T09:58:23.772655+0000","last_clean":"2024-09-10T09:58:23.772655+0000","last_became_active":"2024-09-10T09:58:22.794489+0000","last_became_peered":"2024-09-10T09:58:22.794489+0000","last_unstale":"2024-09-10T09:58:23.772655+0000","last_undegraded":"2024-09-10T09:58:23.772655+0000","last_fullsized":"2024-09-10T09:58:23.772655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:30.932236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782053+0000","last_change":"2024-09-10T09:58:21.768431+0000","last_active":"2024-09-10T09:58:23.782053+0000","last_peered":"2024-09-10T09:58:23.782053+0000","last_clean":"2024-09-10T09:58:23.782053+0000","last_became_active":"2024-09-10T09:58:21.767826+0000","last_became_peered":"2024-09-10T09:58:21.767826+0000","last_unstale":"2024-09-10T09:58:23.782053+0000","last_undegraded":"2024-09-10T09:58:23.782053+0000","last_fullsized":"2024-09-10T09:58:23.782053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738857+0000","last_change":"2024-09-10T09:58:22.791075+0000","last_active":"2024-09-10T09:58:24.738857+0000","last_peered":"2024-09-10T09:58:24.738857+0000","last_clean":"2024-09-10T09:58:24.738857+0000","last_became_active":"2024-09-10T09:58:22.790778+0000","last_became_peered":"2024-09-10T09:58:22.790778+0000","last_unstale":"2024-09-10T09:58:24.738857+0000","last_undegraded":"2024-09-10T09:58:24.738857+0000","last_fullsized":"2024-09-10T09:58:24.738857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:17.386153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738892+0000","last_change":"2024-09-10T09:58:21.788480+0000","last_active":"2024-09-10T09:58:24.738892+0000","last_peered":"2024-09-10T09:58:24.738892+0000","last_clean":"2024-09-10T09:58:24.738892+0000","last_became_active":"2024-09-10T09:58:21.788162+0000","last_became_peered":"2024-09-10T09:58:21.788162+0000","last_unstale":"2024-09-10T09:58:24.738892+0000","last_undegraded":"2024-09-10T09:58:24.738892+0000","last_fullsized":"2024-09-10T09:58:24.738892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:47.151185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781960+0000","last_change":"2024-09-10T09:58:22.778100+0000","last_active":"2024-09-10T09:58:23.781960+0000","last_peered":"2024-09-10T09:58:23.781960+0000","last_clean":"2024-09-10T09:58:23.781960+0000","last_became_active":"2024-09-10T09:58:22.777764+0000","last_became_peered":"2024-09-10T09:58:22.777764+0000","last_unstale":"2024-09-10T09:58:23.781960+0000","last_undegraded":"2024-09-10T09:58:23.781960+0000","last_fullsized":"2024-09-10T09:58:23.781960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782165+0000","last_change":"2024-09-10T09:58:21.770640+0000","last_active":"2024-09-10T09:58:23.782165+0000","last_peered":"2024-09-10T09:58:23.782165+0000","last_clean":"2024-09-10T09:58:23.782165+0000","last_became_active":"2024-09-10T09:58:21.769111+0000","last_became_peered":"2024-09-10T09:58:21.769111+0000","last_unstale":"2024-09-10T09:58:23.782165+0000","last_undegraded":"2024-09-10T09:58:23.782165+0000","last_fullsized":"2024-09-10T09:58:23.782165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738499+0000","last_change":"2024-09-10T09:58:22.789520+0000","last_active":"2024-09-10T09:58:24.738499+0000","last_peered":"2024-09-10T09:58:24.738499+0000","last_clean":"2024-09-10T09:58:24.738499+0000","last_became_active":"2024-09-10T09:58:22.788875+0000","last_became_peered":"2024-09-10T09:58:22.788875+0000","last_unstale":"2024-09-10T09:58:24.738499+0000","last_undegraded":"2024-09-10T09:58:24.738499+0000","last_fullsized":"2024-09-10T09:58:24.738499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:00.959927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773408+0000","last_change":"2024-09-10T09:58:21.782019+0000","last_active":"2024-09-10T09:58:23.773408+0000","last_peered":"2024-09-10T09:58:23.773408+0000","last_clean":"2024-09-10T09:58:23.773408+0000","last_became_active":"2024-09-10T09:58:21.781750+0000","last_became_peered":"2024-09-10T09:58:21.781750+0000","last_unstale":"2024-09-10T09:58:23.773408+0000","last_undegraded":"2024-09-10T09:58:23.773408+0000","last_fullsized":"2024-09-10T09:58:23.773408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738027+0000","last_change":"2024-09-10T09:58:22.781562+0000","last_active":"2024-09-10T09:58:24.738027+0000","last_peered":"2024-09-10T09:58:24.738027+0000","last_clean":"2024-09-10T09:58:24.738027+0000","last_became_active":"2024-09-10T09:58:22.781242+0000","last_became_peered":"2024-09-10T09:58:22.781242+0000","last_unstale":"2024-09-10T09:58:24.738027+0000","last_undegraded":"2024-09-10T09:58:24.738027+0000","last_fullsized":"2024-09-10T09:58:24.738027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737634+0000","last_change":"2024-09-10T09:58:21.788078+0000","last_active":"2024-09-10T09:58:24.737634+0000","last_peered":"2024-09-10T09:58:24.737634+0000","last_clean":"2024-09-10T09:58:24.737634+0000","last_became_active":"2024-09-10T09:58:21.787660+0000","last_became_peered":"2024-09-10T09:58:21.787660+0000","last_unstale":"2024-09-10T09:58:24.737634+0000","last_undegraded":"2024-09-10T09:58:24.737634+0000","last_fullsized":"2024-09-10T09:58:24.737634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772631+0000","last_change":"2024-09-10T09:58:22.798059+0000","last_active":"2024-09-10T09:58:23.772631+0000","last_peered":"2024-09-10T09:58:23.772631+0000","last_clean":"2024-09-10T09:58:23.772631+0000","last_became_active":"2024-09-10T09:58:22.797346+0000","last_became_peered":"2024-09-10T09:58:22.797346+0000","last_unstale":"2024-09-10T09:58:23.772631+0000","last_undegraded":"2024-09-10T09:58:23.772631+0000","last_fullsized":"2024-09-10T09:58:23.772631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:35.690818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781938+0000","last_change":"2024-09-10T09:58:21.782043+0000","last_active":"2024-09-10T09:58:23.781938+0000","last_peered":"2024-09-10T09:58:23.781938+0000","last_clean":"2024-09-10T09:58:23.781938+0000","last_became_active":"2024-09-10T09:58:21.781762+0000","last_became_peered":"2024-09-10T09:58:21.781762+0000","last_unstale":"2024-09-10T09:58:23.781938+0000","last_undegraded":"2024-09-10T09:58:23.781938+0000","last_fullsized":"2024-09-10T09:58:23.781938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738162+0000","last_change":"2024-09-10T09:58:22.772755+0000","last_active":"2024-09-10T09:58:24.738162+0000","last_peered":"2024-09-10T09:58:24.738162+0000","last_clean":"2024-09-10T09:58:24.738162+0000","last_became_active":"2024-09-10T09:58:22.772505+0000","last_became_peered":"2024-09-10T09:58:22.772505+0000","last_unstale":"2024-09-10T09:58:24.738162+0000","last_undegraded":"2024-09-10T09:58:24.738162+0000","last_fullsized":"2024-09-10T09:58:24.738162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:48.318092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782006+0000","last_change":"2024-09-10T09:58:21.774362+0000","last_active":"2024-09-10T09:58:23.782006+0000","last_peered":"2024-09-10T09:58:23.782006+0000","last_clean":"2024-09-10T09:58:23.782006+0000","last_became_active":"2024-09-10T09:58:21.773133+0000","last_became_peered":"2024-09-10T09:58:21.773133+0000","last_unstale":"2024-09-10T09:58:23.782006+0000","last_undegraded":"2024-09-10T09:58:23.782006+0000","last_fullsized":"2024-09-10T09:58:23.782006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738587+0000","last_change":"2024-09-10T09:58:22.791719+0000","last_active":"2024-09-10T09:58:24.738587+0000","last_peered":"2024-09-10T09:58:24.738587+0000","last_clean":"2024-09-10T09:58:24.738587+0000","last_became_active":"2024-09-10T09:58:22.790615+0000","last_became_peered":"2024-09-10T09:58:22.790615+0000","last_unstale":"2024-09-10T09:58:24.738587+0000","last_undegraded":"2024-09-10T09:58:24.738587+0000","last_fullsized":"2024-09-10T09:58:24.738587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:59.165906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738554+0000","last_change":"2024-09-10T09:58:21.790948+0000","last_active":"2024-09-10T09:58:24.738554+0000","last_peered":"2024-09-10T09:58:24.738554+0000","last_clean":"2024-09-10T09:58:24.738554+0000","last_became_active":"2024-09-10T09:58:21.790543+0000","last_became_peered":"2024-09-10T09:58:21.790543+0000","last_unstale":"2024-09-10T09:58:24.738554+0000","last_undegraded":"2024-09-10T09:58:24.738554+0000","last_fullsized":"2024-09-10T09:58:24.738554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:26.671745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772287+0000","last_change":"2024-09-10T09:58:22.790405+0000","last_active":"2024-09-10T09:58:23.772287+0000","last_peered":"2024-09-10T09:58:23.772287+0000","last_clean":"2024-09-10T09:58:23.772287+0000","last_became_active":"2024-09-10T09:58:22.788732+0000","last_became_peered":"2024-09-10T09:58:22.788732+0000","last_unstale":"2024-09-10T09:58:23.772287+0000","last_undegraded":"2024-09-10T09:58:23.772287+0000","last_fullsized":"2024-09-10T09:58:23.772287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738773+0000","last_change":"2024-09-10T09:58:22.795014+0000","last_active":"2024-09-10T09:58:24.738773+0000","last_peered":"2024-09-10T09:58:24.738773+0000","last_clean":"2024-09-10T09:58:24.738773+0000","last_became_active":"2024-09-10T09:58:22.794752+0000","last_became_peered":"2024-09-10T09:58:22.794752+0000","last_unstale":"2024-09-10T09:58:24.738773+0000","last_undegraded":"2024-09-10T09:58:24.738773+0000","last_fullsized":"2024-09-10T09:58:24.738773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:32.590352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738737+0000","last_change":"2024-09-10T09:58:21.793454+0000","last_active":"2024-09-10T09:58:24.738737+0000","last_peered":"2024-09-10T09:58:24.738737+0000","last_clean":"2024-09-10T09:58:24.738737+0000","last_became_active":"2024-09-10T09:58:21.793058+0000","last_became_peered":"2024-09-10T09:58:21.793058+0000","last_unstale":"2024-09-10T09:58:24.738737+0000","last_undegraded":"2024-09-10T09:58:24.738737+0000","last_fullsized":"2024-09-10T09:58:24.738737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738316+0000","last_change":"2024-09-10T09:58:22.804585+0000","last_active":"2024-09-10T09:58:24.738316+0000","last_peered":"2024-09-10T09:58:24.738316+0000","last_clean":"2024-09-10T09:58:24.738316+0000","last_became_active":"2024-09-10T09:58:22.804350+0000","last_became_peered":"2024-09-10T09:58:22.804350+0000","last_unstale":"2024-09-10T09:58:24.738316+0000","last_undegraded":"2024-09-10T09:58:24.738316+0000","last_fullsized":"2024-09-10T09:58:24.738316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:26.963772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738735+0000","last_change":"2024-09-10T09:58:21.789114+0000","last_active":"2024-09-10T09:58:24.738735+0000","last_peered":"2024-09-10T09:58:24.738735+0000","last_clean":"2024-09-10T09:58:24.738735+0000","last_became_active":"2024-09-10T09:58:21.788849+0000","last_became_peered":"2024-09-10T09:58:21.788849+0000","last_unstale":"2024-09-10T09:58:24.738735+0000","last_undegraded":"2024-09-10T09:58:24.738735+0000","last_fullsized":"2024-09-10T09:58:24.738735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737991+0000","last_change":"2024-09-10T09:58:22.792540+0000","last_active":"2024-09-10T09:58:24.737991+0000","last_peered":"2024-09-10T09:58:24.737991+0000","last_clean":"2024-09-10T09:58:24.737991+0000","last_became_active":"2024-09-10T09:58:22.792265+0000","last_became_peered":"2024-09-10T09:58:22.792265+0000","last_unstale":"2024-09-10T09:58:24.737991+0000","last_undegraded":"2024-09-10T09:58:24.737991+0000","last_fullsized":"2024-09-10T09:58:24.737991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738067+0000","last_change":"2024-09-10T09:58:21.779788+0000","last_active":"2024-09-10T09:58:24.738067+0000","last_peered":"2024-09-10T09:58:24.738067+0000","last_clean":"2024-09-10T09:58:24.738067+0000","last_became_active":"2024-09-10T09:58:21.779521+0000","last_became_peered":"2024-09-10T09:58:21.779521+0000","last_unstale":"2024-09-10T09:58:24.738067+0000","last_undegraded":"2024-09-10T09:58:24.738067+0000","last_fullsized":"2024-09-10T09:58:24.738067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:33.355227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737993+0000","last_change":"2024-09-10T09:58:22.777347+0000","last_active":"2024-09-10T09:58:24.737993+0000","last_peered":"2024-09-10T09:58:24.737993+0000","last_clean":"2024-09-10T09:58:24.737993+0000","last_became_active":"2024-09-10T09:58:22.777042+0000","last_became_peered":"2024-09-10T09:58:22.777042+0000","last_unstale":"2024-09-10T09:58:24.737993+0000","last_undegraded":"2024-09-10T09:58:24.737993+0000","last_fullsized":"2024-09-10T09:58:24.737993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:31.068726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738287+0000","last_change":"2024-09-10T09:58:21.787400+0000","last_active":"2024-09-10T09:58:24.738287+0000","last_peered":"2024-09-10T09:58:24.738287+0000","last_clean":"2024-09-10T09:58:24.738287+0000","last_became_active":"2024-09-10T09:58:21.787111+0000","last_became_peered":"2024-09-10T09:58:21.787111+0000","last_unstale":"2024-09-10T09:58:24.738287+0000","last_undegraded":"2024-09-10T09:58:24.738287+0000","last_fullsized":"2024-09-10T09:58:24.738287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":671905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":81081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":80955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":241664,"data_stored":81115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:58:34.308 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:58:34.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab08a00640 1 Processor -- start 2024-09-10T09:58:34.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab08a00640 1 -- start start 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab08a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fab04155970 0x7fab040cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab08a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab0415c560 0x7fab0406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab08a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab0406f010 0x7fab0415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab08a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fab040b4570 con 0x7fab0415c560 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab08a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fab0406f430 con 0x7fab04155970 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab08a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fab04155d50 con 0x7fab0406f010 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab0415c560 0x7fab0406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab0415c560 0x7fab0406ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53382/0 (socket says 172.21.15.110:53382) 2024-09-10T09:58:34.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01400640 1 -- 172.21.15.110:0/2611292652 learned_addr learned my addr 172.21.15.110:0/2611292652 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:34.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01e00640 1 --2- 172.21.15.110:0/2611292652 >> v2:172.21.15.145:3300/0 conn(0x7fab04155970 0x7fab040cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab02800640 1 --2- 172.21.15.110:0/2611292652 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab0406f010 0x7fab0415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01400640 1 -- 172.21.15.110:0/2611292652 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab0406f010 msgr2=0x7fab0415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:34.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01400640 1 --2- 172.21.15.110:0/2611292652 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab0406f010 0x7fab0415eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01400640 1 -- 172.21.15.110:0/2611292652 >> v2:172.21.15.145:3300/0 conn(0x7fab04155970 msgr2=0x7fab040cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:34.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01400640 1 --2- 172.21.15.110:0/2611292652 >> v2:172.21.15.145:3300/0 conn(0x7fab04155970 0x7fab040cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01400640 1 -- 172.21.15.110:0/2611292652 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab040b34d0 con 0x7fab0415c560 2024-09-10T09:58:34.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.476+0000 7fab01e00640 1 --2- 172.21.15.110:0/2611292652 >> v2:172.21.15.145:3300/0 conn(0x7fab04155970 0x7fab040cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:34.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab01400640 1 --2- 172.21.15.110:0/2611292652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab0406ead0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab0406cc30 tx=0x7faaf001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6649173a78055793 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab00a00640 1 -- 172.21.15.110:0/2611292652 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf00b0070 con 0x7fab0415c560 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab00a00640 1 -- 172.21.15.110:0/2611292652 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf00ab600 con 0x7fab0415c560 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab00a00640 1 -- 172.21.15.110:0/2611292652 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf00ab9d0 con 0x7fab0415c560 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 -- 172.21.15.110:0/2611292652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 msgr2=0x7fab0406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 --2- 172.21.15.110:0/2611292652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab0406ead0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab0406cc30 tx=0x7faaf001c230 comp rx=0 tx=0).stop 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 -- 172.21.15.110:0/2611292652 shutdown_connections 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 --2- 172.21.15.110:0/2611292652 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab0406f010 0x7fab0415eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 --2- 172.21.15.110:0/2611292652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab0406ead0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 --2- 172.21.15.110:0/2611292652 >> v2:172.21.15.145:3300/0 conn(0x7fab04155970 0x7fab040cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 -- 172.21.15.110:0/2611292652 >> 172.21.15.110:0/2611292652 conn(0x7fab0406bc50 msgr2=0x7fab0406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 -- 172.21.15.110:0/2611292652 shutdown_connections 2024-09-10T09:58:34.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.480+0000 7fab08a00640 1 -- 172.21.15.110:0/2611292652 wait complete. 2024-09-10T09:58:34.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 Processor -- start 2024-09-10T09:58:34.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 -- start start 2024-09-10T09:58:34.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab0406f010 0x7fab04145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab04155970 0x7fab04158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab041547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab040cce00 con 0x7fab0415c560 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab040b4050 con 0x7fab0406f010 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fab040ce420 con 0x7fab04155970 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab041547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab041547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53384/0 (socket says 172.21.15.110:53384) 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 -- 172.21.15.110:0/699871248 learned_addr learned my addr 172.21.15.110:0/699871248 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab01400640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab04155970 0x7fab04158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab01e00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab0406f010 0x7fab04145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 -- 172.21.15.110:0/699871248 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab04155970 msgr2=0x7fab04158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab04155970 0x7fab04158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 -- 172.21.15.110:0/699871248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab0406f010 msgr2=0x7fab04145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab0406f010 0x7fab04145e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 -- 172.21.15.110:0/699871248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab041570f0 con 0x7fab0415c560 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab02800640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab041547f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faafc007e60 tx=0x7faafc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7faafa000640 1 -- 172.21.15.110:0/699871248 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc014620 con 0x7fab0415c560 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7faafa000640 1 -- 172.21.15.110:0/699871248 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc019070 con 0x7fab0415c560 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7faafa000640 1 -- 172.21.15.110:0/699871248 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc014a20 con 0x7fab0415c560 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab041573d0 con 0x7fab0415c560 2024-09-10T09:58:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.484+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab04157990 con 0x7fab0415c560 2024-09-10T09:58:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.488+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab040cc8c0 con 0x7fab0415c560 2024-09-10T09:58:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.488+0000 7faafa000640 1 -- 172.21.15.110:0/699871248 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faafc014bc0 con 0x7fab0415c560 2024-09-10T09:58:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.488+0000 7faafa000640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7faae40678c0 0x7faae4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.488+0000 7faafa000640 1 -- 172.21.15.110:0/699871248 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7faafc0864f0 con 0x7fab0415c560 2024-09-10T09:58:34.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.488+0000 7fab01e00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7faae40678c0 0x7faae4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.488+0000 7fab01e00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7faae40678c0 0x7faae4069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009950 tx=0x7faaf4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.492+0000 7faafa000640 1 -- 172.21.15.110:0/699871248 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7faafc0568b0 con 0x7fab0415c560 2024-09-10T09:58:34.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.648+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab040cfe70 con 0x7faae40678c0 2024-09-10T09:58:34.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.652+0000 7faafa000640 1 -- 172.21.15.110:0/699871248 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fab040cfe70 con 0x7faae40678c0 2024-09-10T09:58:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:34.659 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:58:34.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7faae40678c0 msgr2=0x7faae4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7faae40678c0 0x7faae4069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009950 tx=0x7faaf4008040 comp rx=0 tx=0).stop 2024-09-10T09:58:34.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 msgr2=0x7fab041547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab041547f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faafc007e60 tx=0x7faafc00e4b0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 shutdown_connections 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7faae40678c0 0x7faae4069d80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab0415c560 0x7fab041547f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab04155970 0x7fab04158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 --2- 172.21.15.110:0/699871248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab0406f010 0x7fab04145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 >> 172.21.15.110:0/699871248 conn(0x7fab0406bc50 msgr2=0x7fab04071180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 shutdown_connections 2024-09-10T09:58:34.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.656+0000 7fab08a00640 1 -- 172.21.15.110:0/699871248 wait complete. 2024-09-10T09:58:34.678 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T09:58:33.357719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235916,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738804,"statfs":{"total":773094113280,"available":772852535296,"internally_reserved":0,"allocated":3121152,"data_stored":1833196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002379"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739344+0000","last_change":"2024-09-10T09:58:22.803813+0000","last_active":"2024-09-10T09:58:24.739344+0000","last_peered":"2024-09-10T09:58:24.739344+0000","last_clean":"2024-09-10T09:58:24.739344+0000","last_became_active":"2024-09-10T09:58:22.803440+0000","last_became_peered":"2024-09-10T09:58:22.803440+0000","last_unstale":"2024-09-10T09:58:24.739344+0000","last_undegraded":"2024-09-10T09:58:24.739344+0000","last_fullsized":"2024-09-10T09:58:24.739344+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:41.057291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782118+0000","last_change":"2024-09-10T09:58:22.786389+0000","last_active":"2024-09-10T09:58:23.782118+0000","last_peered":"2024-09-10T09:58:23.782118+0000","last_clean":"2024-09-10T09:58:23.782118+0000","last_became_active":"2024-09-10T09:58:22.786093+0000","last_became_peered":"2024-09-10T09:58:22.786093+0000","last_unstale":"2024-09-10T09:58:23.782118+0000","last_undegraded":"2024-09-10T09:58:23.782118+0000","last_fullsized":"2024-09-10T09:58:23.782118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738194+0000","last_change":"2024-09-10T09:58:22.782747+0000","last_active":"2024-09-10T09:58:24.738194+0000","last_peered":"2024-09-10T09:58:24.738194+0000","last_clean":"2024-09-10T09:58:24.738194+0000","last_became_active":"2024-09-10T09:58:22.782493+0000","last_became_peered":"2024-09-10T09:58:22.782493+0000","last_unstale":"2024-09-10T09:58:24.738194+0000","last_undegraded":"2024-09-10T09:58:24.738194+0000","last_fullsized":"2024-09-10T09:58:24.738194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738202+0000","last_change":"2024-09-10T09:58:22.780919+0000","last_active":"2024-09-10T09:58:24.738202+0000","last_peered":"2024-09-10T09:58:24.738202+0000","last_clean":"2024-09-10T09:58:24.738202+0000","last_became_active":"2024-09-10T09:58:22.779369+0000","last_became_peered":"2024-09-10T09:58:22.779369+0000","last_unstale":"2024-09-10T09:58:24.738202+0000","last_undegraded":"2024-09-10T09:58:24.738202+0000","last_fullsized":"2024-09-10T09:58:24.738202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:17.698136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738364+0000","last_change":"2024-09-10T09:58:22.803835+0000","last_active":"2024-09-10T09:58:24.738364+0000","last_peered":"2024-09-10T09:58:24.738364+0000","last_clean":"2024-09-10T09:58:24.738364+0000","last_became_active":"2024-09-10T09:58:22.803465+0000","last_became_peered":"2024-09-10T09:58:22.803465+0000","last_unstale":"2024-09-10T09:58:24.738364+0000","last_undegraded":"2024-09-10T09:58:24.738364+0000","last_fullsized":"2024-09-10T09:58:24.738364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:17.698136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737987+0000","last_change":"2024-09-10T09:58:22.804773+0000","last_active":"2024-09-10T09:58:24.737987+0000","last_peered":"2024-09-10T09:58:24.737987+0000","last_clean":"2024-09-10T09:58:24.737987+0000","last_became_active":"2024-09-10T09:58:22.801323+0000","last_became_peered":"2024-09-10T09:58:22.801323+0000","last_unstale":"2024-09-10T09:58:24.737987+0000","last_undegraded":"2024-09-10T09:58:24.737987+0000","last_fullsized":"2024-09-10T09:58:24.737987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:36.116325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737772+0000","last_change":"2024-09-10T09:58:22.790265+0000","last_active":"2024-09-10T09:58:24.737772+0000","last_peered":"2024-09-10T09:58:24.737772+0000","last_clean":"2024-09-10T09:58:24.737772+0000","last_became_active":"2024-09-10T09:58:22.789359+0000","last_became_peered":"2024-09-10T09:58:22.789359+0000","last_unstale":"2024-09-10T09:58:24.737772+0000","last_undegraded":"2024-09-10T09:58:24.737772+0000","last_fullsized":"2024-09-10T09:58:24.737772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:36.116325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737654+0000","last_change":"2024-09-10T09:58:22.790696+0000","last_active":"2024-09-10T09:58:24.737654+0000","last_peered":"2024-09-10T09:58:24.737654+0000","last_clean":"2024-09-10T09:58:24.737654+0000","last_became_active":"2024-09-10T09:58:22.790265+0000","last_became_peered":"2024-09-10T09:58:22.790265+0000","last_unstale":"2024-09-10T09:58:24.737654+0000","last_undegraded":"2024-09-10T09:58:24.737654+0000","last_fullsized":"2024-09-10T09:58:24.737654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:20.430762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739480+0000","last_change":"2024-09-10T09:58:22.806568+0000","last_active":"2024-09-10T09:58:24.739480+0000","last_peered":"2024-09-10T09:58:24.739480+0000","last_clean":"2024-09-10T09:58:24.739480+0000","last_became_active":"2024-09-10T09:58:22.806203+0000","last_became_peered":"2024-09-10T09:58:22.806203+0000","last_unstale":"2024-09-10T09:58:24.739480+0000","last_undegraded":"2024-09-10T09:58:24.739480+0000","last_fullsized":"2024-09-10T09:58:24.739480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:17.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773231+0000","last_change":"2024-09-10T09:58:22.785593+0000","last_active":"2024-09-10T09:58:23.773231+0000","last_peered":"2024-09-10T09:58:23.773231+0000","last_clean":"2024-09-10T09:58:23.773231+0000","last_became_active":"2024-09-10T09:58:22.785227+0000","last_became_peered":"2024-09-10T09:58:22.785227+0000","last_unstale":"2024-09-10T09:58:23.773231+0000","last_undegraded":"2024-09-10T09:58:23.773231+0000","last_fullsized":"2024-09-10T09:58:23.773231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772786+0000","last_change":"2024-09-10T09:58:22.792764+0000","last_active":"2024-09-10T09:58:23.772786+0000","last_peered":"2024-09-10T09:58:23.772786+0000","last_clean":"2024-09-10T09:58:23.772786+0000","last_became_active":"2024-09-10T09:58:22.791958+0000","last_became_peered":"2024-09-10T09:58:22.791958+0000","last_unstale":"2024-09-10T09:58:23.772786+0000","last_undegraded":"2024-09-10T09:58:23.772786+0000","last_fullsized":"2024-09-10T09:58:23.772786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773090+0000","last_change":"2024-09-10T09:58:22.774413+0000","last_active":"2024-09-10T09:58:23.773090+0000","last_peered":"2024-09-10T09:58:23.773090+0000","last_clean":"2024-09-10T09:58:23.773090+0000","last_became_active":"2024-09-10T09:58:22.774110+0000","last_became_peered":"2024-09-10T09:58:22.774110+0000","last_unstale":"2024-09-10T09:58:23.773090+0000","last_undegraded":"2024-09-10T09:58:23.773090+0000","last_fullsized":"2024-09-10T09:58:23.773090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:27.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738573+0000","last_change":"2024-09-10T09:58:22.768744+0000","last_active":"2024-09-10T09:58:24.738573+0000","last_peered":"2024-09-10T09:58:24.738573+0000","last_clean":"2024-09-10T09:58:24.738573+0000","last_became_active":"2024-09-10T09:58:22.768515+0000","last_became_peered":"2024-09-10T09:58:22.768515+0000","last_unstale":"2024-09-10T09:58:24.738573+0000","last_undegraded":"2024-09-10T09:58:24.738573+0000","last_fullsized":"2024-09-10T09:58:24.738573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:37.491552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737347+0000","last_change":"2024-09-10T09:58:22.797642+0000","last_active":"2024-09-10T09:58:24.737347+0000","last_peered":"2024-09-10T09:58:24.737347+0000","last_clean":"2024-09-10T09:58:24.737347+0000","last_became_active":"2024-09-10T09:58:22.797304+0000","last_became_peered":"2024-09-10T09:58:22.797304+0000","last_unstale":"2024-09-10T09:58:24.737347+0000","last_undegraded":"2024-09-10T09:58:24.737347+0000","last_fullsized":"2024-09-10T09:58:24.737347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:59.080268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781863+0000","last_change":"2024-09-10T09:58:22.799237+0000","last_active":"2024-09-10T09:58:23.781863+0000","last_peered":"2024-09-10T09:58:23.781863+0000","last_clean":"2024-09-10T09:58:23.781863+0000","last_became_active":"2024-09-10T09:58:22.798951+0000","last_became_peered":"2024-09-10T09:58:22.798951+0000","last_unstale":"2024-09-10T09:58:23.781863+0000","last_undegraded":"2024-09-10T09:58:23.781863+0000","last_fullsized":"2024-09-10T09:58:23.781863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738069+0000","last_change":"2024-09-10T09:58:22.801607+0000","last_active":"2024-09-10T09:58:24.738069+0000","last_peered":"2024-09-10T09:58:24.738069+0000","last_clean":"2024-09-10T09:58:24.738069+0000","last_became_active":"2024-09-10T09:58:22.801252+0000","last_became_peered":"2024-09-10T09:58:22.801252+0000","last_unstale":"2024-09-10T09:58:24.738069+0000","last_undegraded":"2024-09-10T09:58:24.738069+0000","last_fullsized":"2024-09-10T09:58:24.738069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739077+0000","last_change":"2024-09-10T09:58:22.800761+0000","last_active":"2024-09-10T09:58:24.739077+0000","last_peered":"2024-09-10T09:58:24.739077+0000","last_clean":"2024-09-10T09:58:24.739077+0000","last_became_active":"2024-09-10T09:58:22.800421+0000","last_became_peered":"2024-09-10T09:58:22.800421+0000","last_unstale":"2024-09-10T09:58:24.739077+0000","last_undegraded":"2024-09-10T09:58:24.739077+0000","last_fullsized":"2024-09-10T09:58:24.739077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:17.515999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770538+0000","last_change":"2024-09-10T09:58:22.780000+0000","last_active":"2024-09-10T09:58:23.770538+0000","last_peered":"2024-09-10T09:58:23.770538+0000","last_clean":"2024-09-10T09:58:23.770538+0000","last_became_active":"2024-09-10T09:58:22.779739+0000","last_became_peered":"2024-09-10T09:58:22.779739+0000","last_unstale":"2024-09-10T09:58:23.770538+0000","last_undegraded":"2024-09-10T09:58:23.770538+0000","last_fullsized":"2024-09-10T09:58:23.770538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:48.157725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739018+0000","last_change":"2024-09-10T09:58:22.789943+0000","last_active":"2024-09-10T09:58:24.739018+0000","last_peered":"2024-09-10T09:58:24.739018+0000","last_clean":"2024-09-10T09:58:24.739018+0000","last_became_active":"2024-09-10T09:58:22.789657+0000","last_became_peered":"2024-09-10T09:58:22.789657+0000","last_unstale":"2024-09-10T09:58:24.739018+0000","last_undegraded":"2024-09-10T09:58:24.739018+0000","last_fullsized":"2024-09-10T09:58:24.739018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:17.386153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772836+0000","last_change":"2024-09-10T09:58:22.795150+0000","last_active":"2024-09-10T09:58:23.772836+0000","last_peered":"2024-09-10T09:58:23.772836+0000","last_clean":"2024-09-10T09:58:23.772836+0000","last_became_active":"2024-09-10T09:58:22.794828+0000","last_became_peered":"2024-09-10T09:58:22.794828+0000","last_unstale":"2024-09-10T09:58:23.772836+0000","last_undegraded":"2024-09-10T09:58:23.772836+0000","last_fullsized":"2024-09-10T09:58:23.772836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737641+0000","last_change":"2024-09-10T09:58:22.804668+0000","last_active":"2024-09-10T09:58:24.737641+0000","last_peered":"2024-09-10T09:58:24.737641+0000","last_clean":"2024-09-10T09:58:24.737641+0000","last_became_active":"2024-09-10T09:58:22.804314+0000","last_became_peered":"2024-09-10T09:58:22.804314+0000","last_unstale":"2024-09-10T09:58:24.737641+0000","last_undegraded":"2024-09-10T09:58:24.737641+0000","last_fullsized":"2024-09-10T09:58:24.737641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:34.641953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738097+0000","last_change":"2024-09-10T09:58:22.800575+0000","last_active":"2024-09-10T09:58:24.738097+0000","last_peered":"2024-09-10T09:58:24.738097+0000","last_clean":"2024-09-10T09:58:24.738097+0000","last_became_active":"2024-09-10T09:58:22.800254+0000","last_became_peered":"2024-09-10T09:58:22.800254+0000","last_unstale":"2024-09-10T09:58:24.738097+0000","last_undegraded":"2024-09-10T09:58:24.738097+0000","last_fullsized":"2024-09-10T09:58:24.738097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:12.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738073+0000","last_change":"2024-09-10T09:58:22.799342+0000","last_active":"2024-09-10T09:58:24.738073+0000","last_peered":"2024-09-10T09:58:24.738073+0000","last_clean":"2024-09-10T09:58:24.738073+0000","last_became_active":"2024-09-10T09:58:22.799049+0000","last_became_peered":"2024-09-10T09:58:22.799049+0000","last_unstale":"2024-09-10T09:58:24.738073+0000","last_undegraded":"2024-09-10T09:58:24.738073+0000","last_fullsized":"2024-09-10T09:58:24.738073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:24.891454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737824+0000","last_change":"2024-09-10T09:58:22.786910+0000","last_active":"2024-09-10T09:58:24.737824+0000","last_peered":"2024-09-10T09:58:24.737824+0000","last_clean":"2024-09-10T09:58:24.737824+0000","last_became_active":"2024-09-10T09:58:22.784972+0000","last_became_peered":"2024-09-10T09:58:22.784972+0000","last_unstale":"2024-09-10T09:58:24.737824+0000","last_undegraded":"2024-09-10T09:58:24.737824+0000","last_fullsized":"2024-09-10T09:58:24.737824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:54.142589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737548+0000","last_change":"2024-09-10T09:58:22.789676+0000","last_active":"2024-09-10T09:58:24.737548+0000","last_peered":"2024-09-10T09:58:24.737548+0000","last_clean":"2024-09-10T09:58:24.737548+0000","last_became_active":"2024-09-10T09:58:22.789249+0000","last_became_peered":"2024-09-10T09:58:22.789249+0000","last_unstale":"2024-09-10T09:58:24.737548+0000","last_undegraded":"2024-09-10T09:58:24.737548+0000","last_fullsized":"2024-09-10T09:58:24.737548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:03.582635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773346+0000","last_change":"2024-09-10T09:58:22.793654+0000","last_active":"2024-09-10T09:58:23.773346+0000","last_peered":"2024-09-10T09:58:23.773346+0000","last_clean":"2024-09-10T09:58:23.773346+0000","last_became_active":"2024-09-10T09:58:22.793394+0000","last_became_peered":"2024-09-10T09:58:22.793394+0000","last_unstale":"2024-09-10T09:58:23.773346+0000","last_undegraded":"2024-09-10T09:58:23.773346+0000","last_fullsized":"2024-09-10T09:58:23.773346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782057+0000","last_change":"2024-09-10T09:58:22.766616+0000","last_active":"2024-09-10T09:58:23.782057+0000","last_peered":"2024-09-10T09:58:23.782057+0000","last_clean":"2024-09-10T09:58:23.782057+0000","last_became_active":"2024-09-10T09:58:22.766321+0000","last_became_peered":"2024-09-10T09:58:22.766321+0000","last_unstale":"2024-09-10T09:58:23.782057+0000","last_undegraded":"2024-09-10T09:58:23.782057+0000","last_fullsized":"2024-09-10T09:58:23.782057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738554+0000","last_change":"2024-09-10T09:58:22.796593+0000","last_active":"2024-09-10T09:58:24.738554+0000","last_peered":"2024-09-10T09:58:24.738554+0000","last_clean":"2024-09-10T09:58:24.738554+0000","last_became_active":"2024-09-10T09:58:22.796294+0000","last_became_peered":"2024-09-10T09:58:22.796294+0000","last_unstale":"2024-09-10T09:58:24.738554+0000","last_undegraded":"2024-09-10T09:58:24.738554+0000","last_fullsized":"2024-09-10T09:58:24.738554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:43.026541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737668+0000","last_change":"2024-09-10T09:58:22.790593+0000","last_active":"2024-09-10T09:58:24.737668+0000","last_peered":"2024-09-10T09:58:24.737668+0000","last_clean":"2024-09-10T09:58:24.737668+0000","last_became_active":"2024-09-10T09:58:22.790244+0000","last_became_peered":"2024-09-10T09:58:22.790244+0000","last_unstale":"2024-09-10T09:58:24.737668+0000","last_undegraded":"2024-09-10T09:58:24.737668+0000","last_fullsized":"2024-09-10T09:58:24.737668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:18.106190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738741+0000","last_change":"2024-09-10T09:58:22.784426+0000","last_active":"2024-09-10T09:58:24.738741+0000","last_peered":"2024-09-10T09:58:24.738741+0000","last_clean":"2024-09-10T09:58:24.738741+0000","last_became_active":"2024-09-10T09:58:22.784101+0000","last_became_peered":"2024-09-10T09:58:22.784101+0000","last_unstale":"2024-09-10T09:58:24.738741+0000","last_undegraded":"2024-09-10T09:58:24.738741+0000","last_fullsized":"2024-09-10T09:58:24.738741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:39.940923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770388+0000","last_change":"2024-09-10T09:58:22.785741+0000","last_active":"2024-09-10T09:58:23.770388+0000","last_peered":"2024-09-10T09:58:23.770388+0000","last_clean":"2024-09-10T09:58:23.770388+0000","last_became_active":"2024-09-10T09:58:22.785508+0000","last_became_peered":"2024-09-10T09:58:22.785508+0000","last_unstale":"2024-09-10T09:58:23.770388+0000","last_undegraded":"2024-09-10T09:58:23.770388+0000","last_fullsized":"2024-09-10T09:58:23.770388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:49.472990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772262+0000","last_change":"2024-09-10T09:58:22.776565+0000","last_active":"2024-09-10T09:58:23.772262+0000","last_peered":"2024-09-10T09:58:23.772262+0000","last_clean":"2024-09-10T09:58:23.772262+0000","last_became_active":"2024-09-10T09:58:22.776167+0000","last_became_peered":"2024-09-10T09:58:22.776167+0000","last_unstale":"2024-09-10T09:58:23.772262+0000","last_undegraded":"2024-09-10T09:58:23.772262+0000","last_fullsized":"2024-09-10T09:58:23.772262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772714+0000","last_change":"2024-09-10T09:58:22.776703+0000","last_active":"2024-09-10T09:58:23.772714+0000","last_peered":"2024-09-10T09:58:23.772714+0000","last_clean":"2024-09-10T09:58:23.772714+0000","last_became_active":"2024-09-10T09:58:22.776418+0000","last_became_peered":"2024-09-10T09:58:22.776418+0000","last_unstale":"2024-09-10T09:58:23.772714+0000","last_undegraded":"2024-09-10T09:58:23.772714+0000","last_fullsized":"2024-09-10T09:58:23.772714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:59.498264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738624+0000","last_change":"2024-09-10T09:58:21.783061+0000","last_active":"2024-09-10T09:58:24.738624+0000","last_peered":"2024-09-10T09:58:24.738624+0000","last_clean":"2024-09-10T09:58:24.738624+0000","last_became_active":"2024-09-10T09:58:21.782756+0000","last_became_peered":"2024-09-10T09:58:21.782756+0000","last_unstale":"2024-09-10T09:58:24.738624+0000","last_undegraded":"2024-09-10T09:58:24.738624+0000","last_fullsized":"2024-09-10T09:58:24.738624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:39.702619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737843+0000","last_change":"2024-09-10T09:58:22.800136+0000","last_active":"2024-09-10T09:58:24.737843+0000","last_peered":"2024-09-10T09:58:24.737843+0000","last_clean":"2024-09-10T09:58:24.737843+0000","last_became_active":"2024-09-10T09:58:22.799760+0000","last_became_peered":"2024-09-10T09:58:22.799760+0000","last_unstale":"2024-09-10T09:58:24.737843+0000","last_undegraded":"2024-09-10T09:58:24.737843+0000","last_fullsized":"2024-09-10T09:58:24.737843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772303+0000","last_change":"2024-09-10T09:58:21.778244+0000","last_active":"2024-09-10T09:58:23.772303+0000","last_peered":"2024-09-10T09:58:23.772303+0000","last_clean":"2024-09-10T09:58:23.772303+0000","last_became_active":"2024-09-10T09:58:21.777937+0000","last_became_peered":"2024-09-10T09:58:21.777937+0000","last_unstale":"2024-09-10T09:58:23.772303+0000","last_undegraded":"2024-09-10T09:58:23.772303+0000","last_fullsized":"2024-09-10T09:58:23.772303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738137+0000","last_change":"2024-09-10T09:58:21.775701+0000","last_active":"2024-09-10T09:58:24.738137+0000","last_peered":"2024-09-10T09:58:24.738137+0000","last_clean":"2024-09-10T09:58:24.738137+0000","last_became_active":"2024-09-10T09:58:21.775431+0000","last_became_peered":"2024-09-10T09:58:21.775431+0000","last_unstale":"2024-09-10T09:58:24.738137+0000","last_undegraded":"2024-09-10T09:58:24.738137+0000","last_fullsized":"2024-09-10T09:58:24.738137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:03.845811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737371+0000","last_change":"2024-09-10T09:58:22.772570+0000","last_active":"2024-09-10T09:58:24.737371+0000","last_peered":"2024-09-10T09:58:24.737371+0000","last_clean":"2024-09-10T09:58:24.737371+0000","last_became_active":"2024-09-10T09:58:22.772248+0000","last_became_peered":"2024-09-10T09:58:22.772248+0000","last_unstale":"2024-09-10T09:58:24.737371+0000","last_undegraded":"2024-09-10T09:58:24.737371+0000","last_fullsized":"2024-09-10T09:58:24.737371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:55.062755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737937+0000","last_change":"2024-09-10T09:58:21.786530+0000","last_active":"2024-09-10T09:58:24.737937+0000","last_peered":"2024-09-10T09:58:24.737937+0000","last_clean":"2024-09-10T09:58:24.737937+0000","last_became_active":"2024-09-10T09:58:21.786157+0000","last_became_peered":"2024-09-10T09:58:21.786157+0000","last_unstale":"2024-09-10T09:58:24.737937+0000","last_undegraded":"2024-09-10T09:58:24.737937+0000","last_fullsized":"2024-09-10T09:58:24.737937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781837+0000","last_change":"2024-09-10T09:58:22.767449+0000","last_active":"2024-09-10T09:58:23.781837+0000","last_peered":"2024-09-10T09:58:23.781837+0000","last_clean":"2024-09-10T09:58:23.781837+0000","last_became_active":"2024-09-10T09:58:22.767182+0000","last_became_peered":"2024-09-10T09:58:22.767182+0000","last_unstale":"2024-09-10T09:58:23.781837+0000","last_undegraded":"2024-09-10T09:58:23.781837+0000","last_fullsized":"2024-09-10T09:58:23.781837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773334+0000","last_change":"2024-09-10T09:58:21.786956+0000","last_active":"2024-09-10T09:58:23.773334+0000","last_peered":"2024-09-10T09:58:23.773334+0000","last_clean":"2024-09-10T09:58:23.773334+0000","last_became_active":"2024-09-10T09:58:21.786523+0000","last_became_peered":"2024-09-10T09:58:21.786523+0000","last_unstale":"2024-09-10T09:58:23.773334+0000","last_undegraded":"2024-09-10T09:58:23.773334+0000","last_fullsized":"2024-09-10T09:58:23.773334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738905+0000","last_change":"2024-09-10T09:58:22.801352+0000","last_active":"2024-09-10T09:58:24.738905+0000","last_peered":"2024-09-10T09:58:24.738905+0000","last_clean":"2024-09-10T09:58:24.738905+0000","last_became_active":"2024-09-10T09:58:22.800233+0000","last_became_peered":"2024-09-10T09:58:22.800233+0000","last_unstale":"2024-09-10T09:58:24.738905+0000","last_undegraded":"2024-09-10T09:58:24.738905+0000","last_fullsized":"2024-09-10T09:58:24.738905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:41.057291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739047+0000","last_change":"2024-09-10T09:58:21.787382+0000","last_active":"2024-09-10T09:58:24.739047+0000","last_peered":"2024-09-10T09:58:24.739047+0000","last_clean":"2024-09-10T09:58:24.739047+0000","last_became_active":"2024-09-10T09:58:21.787048+0000","last_became_peered":"2024-09-10T09:58:21.787048+0000","last_unstale":"2024-09-10T09:58:24.739047+0000","last_undegraded":"2024-09-10T09:58:24.739047+0000","last_fullsized":"2024-09-10T09:58:24.739047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738181+0000","last_change":"2024-09-10T09:58:22.792565+0000","last_active":"2024-09-10T09:58:24.738181+0000","last_peered":"2024-09-10T09:58:24.738181+0000","last_clean":"2024-09-10T09:58:24.738181+0000","last_became_active":"2024-09-10T09:58:22.792257+0000","last_became_peered":"2024-09-10T09:58:22.792257+0000","last_unstale":"2024-09-10T09:58:24.738181+0000","last_undegraded":"2024-09-10T09:58:24.738181+0000","last_fullsized":"2024-09-10T09:58:24.738181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782153+0000","last_change":"2024-09-10T09:58:21.785126+0000","last_active":"2024-09-10T09:58:23.782153+0000","last_peered":"2024-09-10T09:58:23.782153+0000","last_clean":"2024-09-10T09:58:23.782153+0000","last_became_active":"2024-09-10T09:58:21.784839+0000","last_became_peered":"2024-09-10T09:58:21.784839+0000","last_unstale":"2024-09-10T09:58:23.782153+0000","last_undegraded":"2024-09-10T09:58:23.782153+0000","last_fullsized":"2024-09-10T09:58:23.782153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773200+0000","last_change":"2024-09-10T09:58:22.786530+0000","last_active":"2024-09-10T09:58:23.773200+0000","last_peered":"2024-09-10T09:58:23.773200+0000","last_clean":"2024-09-10T09:58:23.773200+0000","last_became_active":"2024-09-10T09:58:22.782828+0000","last_became_peered":"2024-09-10T09:58:22.782828+0000","last_unstale":"2024-09-10T09:58:23.773200+0000","last_undegraded":"2024-09-10T09:58:23.773200+0000","last_fullsized":"2024-09-10T09:58:23.773200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:18.151980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772938+0000","last_change":"2024-09-10T09:58:21.776340+0000","last_active":"2024-09-10T09:58:23.772938+0000","last_peered":"2024-09-10T09:58:23.772938+0000","last_clean":"2024-09-10T09:58:23.772938+0000","last_became_active":"2024-09-10T09:58:21.776066+0000","last_became_peered":"2024-09-10T09:58:21.776066+0000","last_unstale":"2024-09-10T09:58:23.772938+0000","last_undegraded":"2024-09-10T09:58:23.772938+0000","last_fullsized":"2024-09-10T09:58:23.772938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738448+0000","last_change":"2024-09-10T09:58:22.785299+0000","last_active":"2024-09-10T09:58:24.738448+0000","last_peered":"2024-09-10T09:58:24.738448+0000","last_clean":"2024-09-10T09:58:24.738448+0000","last_became_active":"2024-09-10T09:58:22.784958+0000","last_became_peered":"2024-09-10T09:58:22.784958+0000","last_unstale":"2024-09-10T09:58:24.738448+0000","last_undegraded":"2024-09-10T09:58:24.738448+0000","last_fullsized":"2024-09-10T09:58:24.738448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:26.963772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737976+0000","last_change":"2024-09-10T09:58:21.783558+0000","last_active":"2024-09-10T09:58:24.737976+0000","last_peered":"2024-09-10T09:58:24.737976+0000","last_clean":"2024-09-10T09:58:24.737976+0000","last_became_active":"2024-09-10T09:58:21.782455+0000","last_became_peered":"2024-09-10T09:58:21.782455+0000","last_unstale":"2024-09-10T09:58:24.737976+0000","last_undegraded":"2024-09-10T09:58:24.737976+0000","last_fullsized":"2024-09-10T09:58:24.737976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737404+0000","last_change":"2024-09-10T09:58:22.796235+0000","last_active":"2024-09-10T09:58:24.737404+0000","last_peered":"2024-09-10T09:58:24.737404+0000","last_clean":"2024-09-10T09:58:24.737404+0000","last_became_active":"2024-09-10T09:58:22.793164+0000","last_became_peered":"2024-09-10T09:58:22.793164+0000","last_unstale":"2024-09-10T09:58:24.737404+0000","last_undegraded":"2024-09-10T09:58:24.737404+0000","last_fullsized":"2024-09-10T09:58:24.737404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:30.250228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738233+0000","last_change":"2024-09-10T09:58:21.789974+0000","last_active":"2024-09-10T09:58:24.738233+0000","last_peered":"2024-09-10T09:58:24.738233+0000","last_clean":"2024-09-10T09:58:24.738233+0000","last_became_active":"2024-09-10T09:58:21.788481+0000","last_became_peered":"2024-09-10T09:58:21.788481+0000","last_unstale":"2024-09-10T09:58:24.738233+0000","last_undegraded":"2024-09-10T09:58:24.738233+0000","last_fullsized":"2024-09-10T09:58:24.738233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738521+0000","last_change":"2024-09-10T09:58:22.801237+0000","last_active":"2024-09-10T09:58:24.738521+0000","last_peered":"2024-09-10T09:58:24.738521+0000","last_clean":"2024-09-10T09:58:24.738521+0000","last_became_active":"2024-09-10T09:58:22.800975+0000","last_became_peered":"2024-09-10T09:58:22.800975+0000","last_unstale":"2024-09-10T09:58:24.738521+0000","last_undegraded":"2024-09-10T09:58:24.738521+0000","last_fullsized":"2024-09-10T09:58:24.738521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:26.981758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772229+0000","last_change":"2024-09-10T09:58:21.778738+0000","last_active":"2024-09-10T09:58:23.772229+0000","last_peered":"2024-09-10T09:58:23.772229+0000","last_clean":"2024-09-10T09:58:23.772229+0000","last_became_active":"2024-09-10T09:58:21.777982+0000","last_became_peered":"2024-09-10T09:58:21.777982+0000","last_unstale":"2024-09-10T09:58:23.772229+0000","last_undegraded":"2024-09-10T09:58:23.772229+0000","last_fullsized":"2024-09-10T09:58:23.772229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772461+0000","last_change":"2024-09-10T09:58:22.775403+0000","last_active":"2024-09-10T09:58:23.772461+0000","last_peered":"2024-09-10T09:58:23.772461+0000","last_clean":"2024-09-10T09:58:23.772461+0000","last_became_active":"2024-09-10T09:58:22.775043+0000","last_became_peered":"2024-09-10T09:58:22.775043+0000","last_unstale":"2024-09-10T09:58:23.772461+0000","last_undegraded":"2024-09-10T09:58:23.772461+0000","last_fullsized":"2024-09-10T09:58:23.772461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770292+0000","last_change":"2024-09-10T09:58:21.779750+0000","last_active":"2024-09-10T09:58:23.770292+0000","last_peered":"2024-09-10T09:58:23.770292+0000","last_clean":"2024-09-10T09:58:23.770292+0000","last_became_active":"2024-09-10T09:58:21.779492+0000","last_became_peered":"2024-09-10T09:58:21.779492+0000","last_unstale":"2024-09-10T09:58:23.770292+0000","last_undegraded":"2024-09-10T09:58:23.770292+0000","last_fullsized":"2024-09-10T09:58:23.770292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737852+0000","last_change":"2024-09-10T09:58:22.801768+0000","last_active":"2024-09-10T09:58:24.737852+0000","last_peered":"2024-09-10T09:58:24.737852+0000","last_clean":"2024-09-10T09:58:24.737852+0000","last_became_active":"2024-09-10T09:58:22.801503+0000","last_became_peered":"2024-09-10T09:58:22.801503+0000","last_unstale":"2024-09-10T09:58:24.737852+0000","last_undegraded":"2024-09-10T09:58:24.737852+0000","last_fullsized":"2024-09-10T09:58:24.737852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:12.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772798+0000","last_change":"2024-09-10T09:58:21.781792+0000","last_active":"2024-09-10T09:58:23.772798+0000","last_peered":"2024-09-10T09:58:23.772798+0000","last_clean":"2024-09-10T09:58:23.772798+0000","last_became_active":"2024-09-10T09:58:21.781434+0000","last_became_peered":"2024-09-10T09:58:21.781434+0000","last_unstale":"2024-09-10T09:58:23.772798+0000","last_undegraded":"2024-09-10T09:58:23.772798+0000","last_fullsized":"2024-09-10T09:58:23.772798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738537+0000","last_change":"2024-09-10T09:58:22.804739+0000","last_active":"2024-09-10T09:58:24.738537+0000","last_peered":"2024-09-10T09:58:24.738537+0000","last_clean":"2024-09-10T09:58:24.738537+0000","last_became_active":"2024-09-10T09:58:22.804451+0000","last_became_peered":"2024-09-10T09:58:22.804451+0000","last_unstale":"2024-09-10T09:58:24.738537+0000","last_undegraded":"2024-09-10T09:58:24.738537+0000","last_fullsized":"2024-09-10T09:58:24.738537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:39.940923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770561+0000","last_change":"2024-09-10T09:58:21.781287+0000","last_active":"2024-09-10T09:58:23.770561+0000","last_peered":"2024-09-10T09:58:23.770561+0000","last_clean":"2024-09-10T09:58:23.770561+0000","last_became_active":"2024-09-10T09:58:21.780824+0000","last_became_peered":"2024-09-10T09:58:21.780824+0000","last_unstale":"2024-09-10T09:58:23.770561+0000","last_undegraded":"2024-09-10T09:58:23.770561+0000","last_fullsized":"2024-09-10T09:58:23.770561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:00.039731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739601+0000","last_change":"2024-09-10T09:58:22.802541+0000","last_active":"2024-09-10T09:58:24.739601+0000","last_peered":"2024-09-10T09:58:24.739601+0000","last_clean":"2024-09-10T09:58:24.739601+0000","last_became_active":"2024-09-10T09:58:22.802216+0000","last_became_peered":"2024-09-10T09:58:22.802216+0000","last_unstale":"2024-09-10T09:58:24.739601+0000","last_undegraded":"2024-09-10T09:58:24.739601+0000","last_fullsized":"2024-09-10T09:58:24.739601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:56.685285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782186+0000","last_change":"2024-09-10T09:58:21.776106+0000","last_active":"2024-09-10T09:58:23.782186+0000","last_peered":"2024-09-10T09:58:23.782186+0000","last_clean":"2024-09-10T09:58:23.782186+0000","last_became_active":"2024-09-10T09:58:21.773315+0000","last_became_peered":"2024-09-10T09:58:21.773315+0000","last_unstale":"2024-09-10T09:58:23.782186+0000","last_undegraded":"2024-09-10T09:58:23.782186+0000","last_fullsized":"2024-09-10T09:58:23.782186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738418+0000","last_change":"2024-09-10T09:58:22.779675+0000","last_active":"2024-09-10T09:58:24.738418+0000","last_peered":"2024-09-10T09:58:24.738418+0000","last_clean":"2024-09-10T09:58:24.738418+0000","last_became_active":"2024-09-10T09:58:22.779088+0000","last_became_peered":"2024-09-10T09:58:22.779088+0000","last_unstale":"2024-09-10T09:58:24.738418+0000","last_undegraded":"2024-09-10T09:58:24.738418+0000","last_fullsized":"2024-09-10T09:58:24.738418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:26.981758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770459+0000","last_change":"2024-09-10T09:58:21.789986+0000","last_active":"2024-09-10T09:58:23.770459+0000","last_peered":"2024-09-10T09:58:23.770459+0000","last_clean":"2024-09-10T09:58:23.770459+0000","last_became_active":"2024-09-10T09:58:21.789577+0000","last_became_peered":"2024-09-10T09:58:21.789577+0000","last_unstale":"2024-09-10T09:58:23.770459+0000","last_undegraded":"2024-09-10T09:58:23.770459+0000","last_fullsized":"2024-09-10T09:58:23.770459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738630+0000","last_change":"2024-09-10T09:58:22.806455+0000","last_active":"2024-09-10T09:58:24.738630+0000","last_peered":"2024-09-10T09:58:24.738630+0000","last_clean":"2024-09-10T09:58:24.738630+0000","last_became_active":"2024-09-10T09:58:22.806181+0000","last_became_peered":"2024-09-10T09:58:22.806181+0000","last_unstale":"2024-09-10T09:58:24.738630+0000","last_undegraded":"2024-09-10T09:58:24.738630+0000","last_fullsized":"2024-09-10T09:58:24.738630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:32.590352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.739373+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:58:24.739373+0000","last_peered":"2024-09-10T09:58:24.739373+0000","last_clean":"2024-09-10T09:58:24.739373+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:58:24.739373+0000","last_undegraded":"2024-09-10T09:58:24.739373+0000","last_fullsized":"2024-09-10T09:58:24.739373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737745+0000","last_change":"2024-09-10T09:58:21.789884+0000","last_active":"2024-09-10T09:58:24.737745+0000","last_peered":"2024-09-10T09:58:24.737745+0000","last_clean":"2024-09-10T09:58:24.737745+0000","last_became_active":"2024-09-10T09:58:21.788940+0000","last_became_peered":"2024-09-10T09:58:21.788940+0000","last_unstale":"2024-09-10T09:58:24.737745+0000","last_undegraded":"2024-09-10T09:58:24.737745+0000","last_fullsized":"2024-09-10T09:58:24.737745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737853+0000","last_change":"2024-09-10T09:58:22.792386+0000","last_active":"2024-09-10T09:58:24.737853+0000","last_peered":"2024-09-10T09:58:24.737853+0000","last_clean":"2024-09-10T09:58:24.737853+0000","last_became_active":"2024-09-10T09:58:22.792151+0000","last_became_peered":"2024-09-10T09:58:22.792151+0000","last_unstale":"2024-09-10T09:58:24.737853+0000","last_undegraded":"2024-09-10T09:58:24.737853+0000","last_fullsized":"2024-09-10T09:58:24.737853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.810620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737643+0000","last_change":"2024-09-10T09:58:21.791063+0000","last_active":"2024-09-10T09:58:24.737643+0000","last_peered":"2024-09-10T09:58:24.737643+0000","last_clean":"2024-09-10T09:58:24.737643+0000","last_became_active":"2024-09-10T09:58:21.790796+0000","last_became_peered":"2024-09-10T09:58:21.790796+0000","last_unstale":"2024-09-10T09:58:24.737643+0000","last_undegraded":"2024-09-10T09:58:24.737643+0000","last_fullsized":"2024-09-10T09:58:24.737643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:17.145440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738301+0000","last_change":"2024-09-10T09:58:22.798826+0000","last_active":"2024-09-10T09:58:24.738301+0000","last_peered":"2024-09-10T09:58:24.738301+0000","last_clean":"2024-09-10T09:58:24.738301+0000","last_became_active":"2024-09-10T09:58:22.798497+0000","last_became_peered":"2024-09-10T09:58:22.798497+0000","last_unstale":"2024-09-10T09:58:24.738301+0000","last_undegraded":"2024-09-10T09:58:24.738301+0000","last_fullsized":"2024-09-10T09:58:24.738301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:24.717356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.770375+0000","last_change":"2024-09-10T09:58:21.777208+0000","last_active":"2024-09-10T09:58:23.770375+0000","last_peered":"2024-09-10T09:58:23.770375+0000","last_clean":"2024-09-10T09:58:23.770375+0000","last_became_active":"2024-09-10T09:58:21.776820+0000","last_became_peered":"2024-09-10T09:58:21.776820+0000","last_unstale":"2024-09-10T09:58:23.770375+0000","last_undegraded":"2024-09-10T09:58:23.770375+0000","last_fullsized":"2024-09-10T09:58:23.770375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:42.810983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781864+0000","last_change":"2024-09-10T09:58:22.771438+0000","last_active":"2024-09-10T09:58:23.781864+0000","last_peered":"2024-09-10T09:58:23.781864+0000","last_clean":"2024-09-10T09:58:23.781864+0000","last_became_active":"2024-09-10T09:58:22.771255+0000","last_became_peered":"2024-09-10T09:58:22.771255+0000","last_unstale":"2024-09-10T09:58:23.781864+0000","last_undegraded":"2024-09-10T09:58:23.781864+0000","last_fullsized":"2024-09-10T09:58:23.781864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:31.806636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738323+0000","last_change":"2024-09-10T09:58:21.785388+0000","last_active":"2024-09-10T09:58:24.738323+0000","last_peered":"2024-09-10T09:58:24.738323+0000","last_clean":"2024-09-10T09:58:24.738323+0000","last_became_active":"2024-09-10T09:58:21.785227+0000","last_became_peered":"2024-09-10T09:58:21.785227+0000","last_unstale":"2024-09-10T09:58:24.738323+0000","last_undegraded":"2024-09-10T09:58:24.738323+0000","last_fullsized":"2024-09-10T09:58:24.738323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772655+0000","last_change":"2024-09-10T09:58:22.798575+0000","last_active":"2024-09-10T09:58:23.772655+0000","last_peered":"2024-09-10T09:58:23.772655+0000","last_clean":"2024-09-10T09:58:23.772655+0000","last_became_active":"2024-09-10T09:58:22.794489+0000","last_became_peered":"2024-09-10T09:58:22.794489+0000","last_unstale":"2024-09-10T09:58:23.772655+0000","last_undegraded":"2024-09-10T09:58:23.772655+0000","last_fullsized":"2024-09-10T09:58:23.772655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:30.932236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782053+0000","last_change":"2024-09-10T09:58:21.768431+0000","last_active":"2024-09-10T09:58:23.782053+0000","last_peered":"2024-09-10T09:58:23.782053+0000","last_clean":"2024-09-10T09:58:23.782053+0000","last_became_active":"2024-09-10T09:58:21.767826+0000","last_became_peered":"2024-09-10T09:58:21.767826+0000","last_unstale":"2024-09-10T09:58:23.782053+0000","last_undegraded":"2024-09-10T09:58:23.782053+0000","last_fullsized":"2024-09-10T09:58:23.782053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:34.684278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738857+0000","last_change":"2024-09-10T09:58:22.791075+0000","last_active":"2024-09-10T09:58:24.738857+0000","last_peered":"2024-09-10T09:58:24.738857+0000","last_clean":"2024-09-10T09:58:24.738857+0000","last_became_active":"2024-09-10T09:58:22.790778+0000","last_became_peered":"2024-09-10T09:58:22.790778+0000","last_unstale":"2024-09-10T09:58:24.738857+0000","last_undegraded":"2024-09-10T09:58:24.738857+0000","last_fullsized":"2024-09-10T09:58:24.738857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:17.386153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738892+0000","last_change":"2024-09-10T09:58:21.788480+0000","last_active":"2024-09-10T09:58:24.738892+0000","last_peered":"2024-09-10T09:58:24.738892+0000","last_clean":"2024-09-10T09:58:24.738892+0000","last_became_active":"2024-09-10T09:58:21.788162+0000","last_became_peered":"2024-09-10T09:58:21.788162+0000","last_unstale":"2024-09-10T09:58:24.738892+0000","last_undegraded":"2024-09-10T09:58:24.738892+0000","last_fullsized":"2024-09-10T09:58:24.738892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:47.151185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781960+0000","last_change":"2024-09-10T09:58:22.778100+0000","last_active":"2024-09-10T09:58:23.781960+0000","last_peered":"2024-09-10T09:58:23.781960+0000","last_clean":"2024-09-10T09:58:23.781960+0000","last_became_active":"2024-09-10T09:58:22.777764+0000","last_became_peered":"2024-09-10T09:58:22.777764+0000","last_unstale":"2024-09-10T09:58:23.781960+0000","last_undegraded":"2024-09-10T09:58:23.781960+0000","last_fullsized":"2024-09-10T09:58:23.781960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782165+0000","last_change":"2024-09-10T09:58:21.770640+0000","last_active":"2024-09-10T09:58:23.782165+0000","last_peered":"2024-09-10T09:58:23.782165+0000","last_clean":"2024-09-10T09:58:23.782165+0000","last_became_active":"2024-09-10T09:58:21.769111+0000","last_became_peered":"2024-09-10T09:58:21.769111+0000","last_unstale":"2024-09-10T09:58:23.782165+0000","last_undegraded":"2024-09-10T09:58:23.782165+0000","last_fullsized":"2024-09-10T09:58:23.782165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738499+0000","last_change":"2024-09-10T09:58:22.789520+0000","last_active":"2024-09-10T09:58:24.738499+0000","last_peered":"2024-09-10T09:58:24.738499+0000","last_clean":"2024-09-10T09:58:24.738499+0000","last_became_active":"2024-09-10T09:58:22.788875+0000","last_became_peered":"2024-09-10T09:58:22.788875+0000","last_unstale":"2024-09-10T09:58:24.738499+0000","last_undegraded":"2024-09-10T09:58:24.738499+0000","last_fullsized":"2024-09-10T09:58:24.738499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:00.959927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.773408+0000","last_change":"2024-09-10T09:58:21.782019+0000","last_active":"2024-09-10T09:58:23.773408+0000","last_peered":"2024-09-10T09:58:23.773408+0000","last_clean":"2024-09-10T09:58:23.773408+0000","last_became_active":"2024-09-10T09:58:21.781750+0000","last_became_peered":"2024-09-10T09:58:21.781750+0000","last_unstale":"2024-09-10T09:58:23.773408+0000","last_undegraded":"2024-09-10T09:58:23.773408+0000","last_fullsized":"2024-09-10T09:58:23.773408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738027+0000","last_change":"2024-09-10T09:58:22.781562+0000","last_active":"2024-09-10T09:58:24.738027+0000","last_peered":"2024-09-10T09:58:24.738027+0000","last_clean":"2024-09-10T09:58:24.738027+0000","last_became_active":"2024-09-10T09:58:22.781242+0000","last_became_peered":"2024-09-10T09:58:22.781242+0000","last_unstale":"2024-09-10T09:58:24.738027+0000","last_undegraded":"2024-09-10T09:58:24.738027+0000","last_fullsized":"2024-09-10T09:58:24.738027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:27.362331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737634+0000","last_change":"2024-09-10T09:58:21.788078+0000","last_active":"2024-09-10T09:58:24.737634+0000","last_peered":"2024-09-10T09:58:24.737634+0000","last_clean":"2024-09-10T09:58:24.737634+0000","last_became_active":"2024-09-10T09:58:21.787660+0000","last_became_peered":"2024-09-10T09:58:21.787660+0000","last_unstale":"2024-09-10T09:58:24.737634+0000","last_undegraded":"2024-09-10T09:58:24.737634+0000","last_fullsized":"2024-09-10T09:58:24.737634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772631+0000","last_change":"2024-09-10T09:58:22.798059+0000","last_active":"2024-09-10T09:58:23.772631+0000","last_peered":"2024-09-10T09:58:23.772631+0000","last_clean":"2024-09-10T09:58:23.772631+0000","last_became_active":"2024-09-10T09:58:22.797346+0000","last_became_peered":"2024-09-10T09:58:22.797346+0000","last_unstale":"2024-09-10T09:58:23.772631+0000","last_undegraded":"2024-09-10T09:58:23.772631+0000","last_fullsized":"2024-09-10T09:58:23.772631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:35.690818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.781938+0000","last_change":"2024-09-10T09:58:21.782043+0000","last_active":"2024-09-10T09:58:23.781938+0000","last_peered":"2024-09-10T09:58:23.781938+0000","last_clean":"2024-09-10T09:58:23.781938+0000","last_became_active":"2024-09-10T09:58:21.781762+0000","last_became_peered":"2024-09-10T09:58:21.781762+0000","last_unstale":"2024-09-10T09:58:23.781938+0000","last_undegraded":"2024-09-10T09:58:23.781938+0000","last_fullsized":"2024-09-10T09:58:23.781938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738162+0000","last_change":"2024-09-10T09:58:22.772755+0000","last_active":"2024-09-10T09:58:24.738162+0000","last_peered":"2024-09-10T09:58:24.738162+0000","last_clean":"2024-09-10T09:58:24.738162+0000","last_became_active":"2024-09-10T09:58:22.772505+0000","last_became_peered":"2024-09-10T09:58:22.772505+0000","last_unstale":"2024-09-10T09:58:24.738162+0000","last_undegraded":"2024-09-10T09:58:24.738162+0000","last_fullsized":"2024-09-10T09:58:24.738162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:48.318092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.782006+0000","last_change":"2024-09-10T09:58:21.774362+0000","last_active":"2024-09-10T09:58:23.782006+0000","last_peered":"2024-09-10T09:58:23.782006+0000","last_clean":"2024-09-10T09:58:23.782006+0000","last_became_active":"2024-09-10T09:58:21.773133+0000","last_became_peered":"2024-09-10T09:58:21.773133+0000","last_unstale":"2024-09-10T09:58:23.782006+0000","last_undegraded":"2024-09-10T09:58:23.782006+0000","last_fullsized":"2024-09-10T09:58:23.782006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:26.476416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738587+0000","last_change":"2024-09-10T09:58:22.791719+0000","last_active":"2024-09-10T09:58:24.738587+0000","last_peered":"2024-09-10T09:58:24.738587+0000","last_clean":"2024-09-10T09:58:24.738587+0000","last_became_active":"2024-09-10T09:58:22.790615+0000","last_became_peered":"2024-09-10T09:58:22.790615+0000","last_unstale":"2024-09-10T09:58:24.738587+0000","last_undegraded":"2024-09-10T09:58:24.738587+0000","last_fullsized":"2024-09-10T09:58:24.738587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:59.165906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738554+0000","last_change":"2024-09-10T09:58:21.790948+0000","last_active":"2024-09-10T09:58:24.738554+0000","last_peered":"2024-09-10T09:58:24.738554+0000","last_clean":"2024-09-10T09:58:24.738554+0000","last_became_active":"2024-09-10T09:58:21.790543+0000","last_became_peered":"2024-09-10T09:58:21.790543+0000","last_unstale":"2024-09-10T09:58:24.738554+0000","last_undegraded":"2024-09-10T09:58:24.738554+0000","last_fullsized":"2024-09-10T09:58:24.738554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:26.671745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:23.772287+0000","last_change":"2024-09-10T09:58:22.790405+0000","last_active":"2024-09-10T09:58:23.772287+0000","last_peered":"2024-09-10T09:58:23.772287+0000","last_clean":"2024-09-10T09:58:23.772287+0000","last_became_active":"2024-09-10T09:58:22.788732+0000","last_became_peered":"2024-09-10T09:58:22.788732+0000","last_unstale":"2024-09-10T09:58:23.772287+0000","last_undegraded":"2024-09-10T09:58:23.772287+0000","last_fullsized":"2024-09-10T09:58:23.772287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:45.571011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738773+0000","last_change":"2024-09-10T09:58:22.795014+0000","last_active":"2024-09-10T09:58:24.738773+0000","last_peered":"2024-09-10T09:58:24.738773+0000","last_clean":"2024-09-10T09:58:24.738773+0000","last_became_active":"2024-09-10T09:58:22.794752+0000","last_became_peered":"2024-09-10T09:58:22.794752+0000","last_unstale":"2024-09-10T09:58:24.738773+0000","last_undegraded":"2024-09-10T09:58:24.738773+0000","last_fullsized":"2024-09-10T09:58:24.738773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:32.590352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738737+0000","last_change":"2024-09-10T09:58:21.793454+0000","last_active":"2024-09-10T09:58:24.738737+0000","last_peered":"2024-09-10T09:58:24.738737+0000","last_clean":"2024-09-10T09:58:24.738737+0000","last_became_active":"2024-09-10T09:58:21.793058+0000","last_became_peered":"2024-09-10T09:58:21.793058+0000","last_unstale":"2024-09-10T09:58:24.738737+0000","last_undegraded":"2024-09-10T09:58:24.738737+0000","last_fullsized":"2024-09-10T09:58:24.738737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:58.491724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738316+0000","last_change":"2024-09-10T09:58:22.804585+0000","last_active":"2024-09-10T09:58:24.738316+0000","last_peered":"2024-09-10T09:58:24.738316+0000","last_clean":"2024-09-10T09:58:24.738316+0000","last_became_active":"2024-09-10T09:58:22.804350+0000","last_became_peered":"2024-09-10T09:58:22.804350+0000","last_unstale":"2024-09-10T09:58:24.738316+0000","last_undegraded":"2024-09-10T09:58:24.738316+0000","last_fullsized":"2024-09-10T09:58:24.738316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:26.963772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738735+0000","last_change":"2024-09-10T09:58:21.789114+0000","last_active":"2024-09-10T09:58:24.738735+0000","last_peered":"2024-09-10T09:58:24.738735+0000","last_clean":"2024-09-10T09:58:24.738735+0000","last_became_active":"2024-09-10T09:58:21.788849+0000","last_became_peered":"2024-09-10T09:58:21.788849+0000","last_unstale":"2024-09-10T09:58:24.738735+0000","last_undegraded":"2024-09-10T09:58:24.738735+0000","last_fullsized":"2024-09-10T09:58:24.738735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:48.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737991+0000","last_change":"2024-09-10T09:58:22.792540+0000","last_active":"2024-09-10T09:58:24.737991+0000","last_peered":"2024-09-10T09:58:24.737991+0000","last_clean":"2024-09-10T09:58:24.737991+0000","last_became_active":"2024-09-10T09:58:22.792265+0000","last_became_peered":"2024-09-10T09:58:22.792265+0000","last_unstale":"2024-09-10T09:58:24.737991+0000","last_undegraded":"2024-09-10T09:58:24.737991+0000","last_fullsized":"2024-09-10T09:58:24.737991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.806201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738067+0000","last_change":"2024-09-10T09:58:21.779788+0000","last_active":"2024-09-10T09:58:24.738067+0000","last_peered":"2024-09-10T09:58:24.738067+0000","last_clean":"2024-09-10T09:58:24.738067+0000","last_became_active":"2024-09-10T09:58:21.779521+0000","last_became_peered":"2024-09-10T09:58:21.779521+0000","last_unstale":"2024-09-10T09:58:24.738067+0000","last_undegraded":"2024-09-10T09:58:24.738067+0000","last_fullsized":"2024-09-10T09:58:24.738067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:33.355227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.737993+0000","last_change":"2024-09-10T09:58:22.777347+0000","last_active":"2024-09-10T09:58:24.737993+0000","last_peered":"2024-09-10T09:58:24.737993+0000","last_clean":"2024-09-10T09:58:24.737993+0000","last_became_active":"2024-09-10T09:58:22.777042+0000","last_became_peered":"2024-09-10T09:58:22.777042+0000","last_unstale":"2024-09-10T09:58:24.737993+0000","last_undegraded":"2024-09-10T09:58:24.737993+0000","last_fullsized":"2024-09-10T09:58:24.737993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:21.747139+0000","last_clean_scrub_stamp":"2024-09-10T09:58:21.747139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:31.068726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-10T09:58:24.738287+0000","last_change":"2024-09-10T09:58:21.787400+0000","last_active":"2024-09-10T09:58:24.738287+0000","last_peered":"2024-09-10T09:58:24.738287+0000","last_clean":"2024-09-10T09:58:24.738287+0000","last_became_active":"2024-09-10T09:58:21.787111+0000","last_became_peered":"2024-09-10T09:58:21.787111+0000","last_unstale":"2024-09-10T09:58:24.738287+0000","last_undegraded":"2024-09-10T09:58:24.738287+0000","last_fullsized":"2024-09-10T09:58:24.738287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:20.740599+0000","last_clean_scrub_stamp":"2024-09-10T09:58:20.740599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:21.856988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":671905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":81081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":80955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":241664,"data_stored":81115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:58:34.681 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T09:58:34.681 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T09:58:34.681 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T09:58:34.682 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T09:58:34.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.852+0000 7f23ab400640 1 Processor -- start 2024-09-10T09:58:34.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.852+0000 7f23ab400640 1 -- start start 2024-09-10T09:58:34.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.852+0000 7f23ab400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a415c560 0x7f23a406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.852+0000 7f23ab400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a406f430 con 0x7f23a4155970 2024-09-10T09:58:34.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.852+0000 7f23ab400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a4155d50 con 0x7f23a406f010 2024-09-10T09:58:34.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.852+0000 7f23ab400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f23a40b4570 con 0x7f23a415c560 2024-09-10T09:58:34.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a8a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f23a4155970 0x7f23a40cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38272/0 (socket says 172.21.15.110:38272) 2024-09-10T09:58:34.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a8a00640 1 -- 172.21.15.110:0/2772402577 learned_addr learned my addr 172.21.15.110:0/2772402577 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3e00640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a415c560 0x7f23a406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1726788933 0 0) 0x7f23a406f430 con 0x7f23a4155970 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2388003530 con 0x7f23a4155970 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 346254488 0 0) 0x7f2388003530 con 0x7f23a4155970 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a415c560 msgr2=0x7f23a406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a415c560 0x7f23a406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 >> v1:172.21.15.145:6789/0 conn(0x7f23a406f010 legacy=0x7f23a40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a40b34d0 con 0x7f23a4155970 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f2394001680 con 0x7f23a4155970 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 >> v1:172.21.15.110:6789/0 conn(0x7f23a4155970 legacy=0x7f23a40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23880039f0 0x7f2388003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f2388004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a415c560 0x7f2388005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f238800ba80 con 0x7f23880043a0 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f238800bc00 con 0x7f23880039f0 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f238800be00 con 0x7f23a415c560 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3e00640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f2388004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a8a00640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23880039f0 0x7f2388003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a9400640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a415c560 0x7f2388005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3e00640 1 -- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a415c560 msgr2=0x7f2388005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3e00640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a415c560 0x7f2388005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3e00640 1 -- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23880039f0 msgr2=0x7f2388003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3e00640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23880039f0 0x7f2388003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a3e00640 1 -- 172.21.15.110:0/2772402577 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2388003530 con 0x7f23880043a0 2024-09-10T09:58:34.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.856+0000 7f23a8a00640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23880039f0 0x7f2388003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:34.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23a3e00640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f2388004870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2390009ae0 tx=0x7f239001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f22e42fdadec04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900b2070 con 0x7f23880043a0 2024-09-10T09:58:34.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23900a5660 con 0x7f23880043a0 2024-09-10T09:58:34.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23a3400640 1 -- 172.21.15.110:0/2772402577 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900096b0 con 0x7f23880043a0 2024-09-10T09:58:34.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23ab400640 1 -- 172.21.15.110:0/2772402577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 msgr2=0x7f2388004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23ab400640 1 --2- 172.21.15.110:0/2772402577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f2388004870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2390009ae0 tx=0x7f239001ca70 comp rx=0 tx=0).stop 2024-09-10T09:58:34.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23a9400640 1 -- 172.21.15.110:0/2772402577 reap_dead start 2024-09-10T09:58:34.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23ab400640 1 -- 172.21.15.110:0/2772402577 shutdown_connections 2024-09-10T09:58:34.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23ab400640 1 -- 172.21.15.110:0/2772402577 >> 172.21.15.110:0/2772402577 conn(0x7f23a406bc50 msgr2=0x7f23a406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:34.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23ab400640 1 -- 172.21.15.110:0/2772402577 shutdown_connections 2024-09-10T09:58:34.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23ab400640 1 -- 172.21.15.110:0/2772402577 wait complete. 2024-09-10T09:58:34.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23ab400640 1 Processor -- start 2024-09-10T09:58:34.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.860+0000 7f23ab400640 1 -- start start 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f23a4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a406d9c0 0x7f23a406de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23a4154350 0x7f23a41547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23a40b3d70 con 0x7f23880043a0 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f23a40b3a90 con 0x7f23a4154350 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f23a40b4050 con 0x7f23a406d9c0 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f23a4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f23a4152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53396/0 (socket says 172.21.15.110:53396) 2024-09-10T09:58:34.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 -- 172.21.15.110:0/2917872492 learned_addr learned my addr 172.21.15.110:0/2917872492 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a3e00640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a406d9c0 0x7f23a406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a9400640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23a4154350 0x7f23a41547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 -- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a406d9c0 msgr2=0x7f23a406de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a406d9c0 0x7f23a406de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 -- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23a4154350 msgr2=0x7f23a41547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23a4154350 0x7f23a41547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 -- 172.21.15.110:0/2917872492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a4154d10 con 0x7f23880043a0 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a9400640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23a4154350 0x7f23a41547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a8a00640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f23a4152280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23940072c0 tx=0x7f2394007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a0c00640 1 -- 172.21.15.110:0/2917872492 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239400f8e0 con 0x7f23880043a0 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a0c00640 1 -- 172.21.15.110:0/2917872492 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2394018070 con 0x7f23880043a0 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a0c00640 1 -- 172.21.15.110:0/2917872492 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239400fce0 con 0x7f23880043a0 2024-09-10T09:58:34.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a4152b10 con 0x7f23880043a0 2024-09-10T09:58:34.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a41530b0 con 0x7f23880043a0 2024-09-10T09:58:34.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a406ead0 con 0x7f23880043a0 2024-09-10T09:58:34.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.864+0000 7f23a0c00640 1 -- 172.21.15.110:0/2917872492 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f239401c410 con 0x7f23880043a0 2024-09-10T09:58:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.868+0000 7f23a0c00640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23840678f0 0x7f2384069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.868+0000 7f23a0c00640 1 -- 172.21.15.110:0/2917872492 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2394085820 con 0x7f23880043a0 2024-09-10T09:58:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.868+0000 7f23a3e00640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23840678f0 0x7f2384069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.868+0000 7f23a3e00640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23840678f0 0x7f2384069db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23900af000 tx=0x7f23900046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:34.872+0000 7f23a0c00640 1 -- 172.21.15.110:0/2917872492 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2394056350 con 0x7f23880043a0 2024-09-10T09:58:35.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.060+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f23a40cf970 con 0x7f23880043a0 2024-09-10T09:58:35.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.060+0000 7f23a0c00640 1 -- 172.21.15.110:0/2917872492 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2394055c50 con 0x7f23880043a0 2024-09-10T09:58:35.063 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:35.063 INFO:teuthology.orchestra.run.smithi110.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.060+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23840678f0 msgr2=0x7f2384069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.060+0000 7f23ab400640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23840678f0 0x7f2384069db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f23900af000 tx=0x7f23900046d0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.060+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 msgr2=0x7f23a4152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.060+0000 7f23ab400640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f23a4152280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23940072c0 tx=0x7f2394007780 comp rx=0 tx=0).stop 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.064+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 shutdown_connections 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.064+0000 7f23ab400640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23840678f0 0x7f2384069db0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.064+0000 7f23ab400640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23a4154350 0x7f23a41547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.064+0000 7f23ab400640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23a406d9c0 0x7f23a406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.064+0000 7f23ab400640 1 --2- 172.21.15.110:0/2917872492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23880043a0 0x7f23a4152280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.064+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 >> 172.21.15.110:0/2917872492 conn(0x7f23a406bc50 msgr2=0x7f23a40b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.064+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 shutdown_connections 2024-09-10T09:58:35.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.064+0000 7f23ab400640 1 -- 172.21.15.110:0/2917872492 wait complete. 2024-09-10T09:58:35.082 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T09:58:35.083 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:35.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54de400640 1 Processor -- start 2024-09-10T09:58:35.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54de400640 1 -- start start 2024-09-10T09:58:35.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54de400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f54d815c3d0 0x7f54d815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54de400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d1660 0x7f54d80d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54de400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d80d1fb0 con 0x7f54d80d0bd0 2024-09-10T09:58:35.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54de400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f54d80d1040 con 0x7f54d815c3d0 2024-09-10T09:58:35.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54de400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f54d815c7b0 con 0x7f54d80d1660 2024-09-10T09:58:35.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54d7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f54d815c3d0 0x7f54d815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54d7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f54d815c3d0 0x7f54d815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53400/0 (socket says 172.21.15.110:53400) 2024-09-10T09:58:35.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54d7400640 1 -- 172.21.15.110:0/2774883645 learned_addr learned my addr 172.21.15.110:0/2774883645 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:35.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54d7e00640 1 --2- 172.21.15.110:0/2774883645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d1660 0x7f54d80d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54d7400640 1 -- 172.21.15.110:0/2774883645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d1660 msgr2=0x7f54d80d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54d7400640 1 --2- 172.21.15.110:0/2774883645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d1660 0x7f54d80d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54d7400640 1 -- 172.21.15.110:0/2774883645 >> v1:172.21.15.110:6789/0 conn(0x7f54d80d0bd0 legacy=0x7f54d80cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.216+0000 7f54d7400640 1 -- 172.21.15.110:0/2774883645 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d80cfb30 con 0x7f54d815c3d0 2024-09-10T09:58:35.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54d7400640 1 --2- 172.21.15.110:0/2774883645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 0x7f54d815d4b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009e50 tx=0x7f54c801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cac77edab39e446 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54d6000640 1 -- 172.21.15.110:0/2774883645 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80a58d0 con 0x7f54d815c3d0 2024-09-10T09:58:35.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54d6000640 1 -- 172.21.15.110:0/2774883645 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c80a5a70 con 0x7f54d815c3d0 2024-09-10T09:58:35.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54d6000640 1 -- 172.21.15.110:0/2774883645 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80a5e40 con 0x7f54d815c3d0 2024-09-10T09:58:35.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54de400640 1 -- 172.21.15.110:0/2774883645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 msgr2=0x7f54d815d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54de400640 1 --2- 172.21.15.110:0/2774883645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 0x7f54d815d4b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009e50 tx=0x7f54c801ca00 comp rx=0 tx=0).stop 2024-09-10T09:58:35.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54de400640 1 -- 172.21.15.110:0/2774883645 shutdown_connections 2024-09-10T09:58:35.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54de400640 1 --2- 172.21.15.110:0/2774883645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d1660 0x7f54d80d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54de400640 1 --2- 172.21.15.110:0/2774883645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 0x7f54d815d4b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54de400640 1 -- 172.21.15.110:0/2774883645 >> 172.21.15.110:0/2774883645 conn(0x7f54d806cf60 msgr2=0x7f54d806d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54de400640 1 -- 172.21.15.110:0/2774883645 shutdown_connections 2024-09-10T09:58:35.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.220+0000 7f54de400640 1 -- 172.21.15.110:0/2774883645 wait complete. 2024-09-10T09:58:35.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 Processor -- start 2024-09-10T09:58:35.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 -- start start 2024-09-10T09:58:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d0bd0 0x7f54d8148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54d80d1660 0x7f54d80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 0x7f54d806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54d815e300 con 0x7f54d815c3d0 2024-09-10T09:58:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54d80d06b0 con 0x7f54d80d1660 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f54d815e480 con 0x7f54d80d0bd0 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d0bd0 0x7f54d8148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54d80d1660 0x7f54d80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d0bd0 0x7f54d8148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:46884/0 (socket says 172.21.15.110:46884) 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 -- 172.21.15.110:0/1190444612 learned_addr learned my addr 172.21.15.110:0/1190444612 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7e00640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 0x7f54d806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 -- 172.21.15.110:0/1190444612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54d80d1660 msgr2=0x7f54d80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54d80d1660 0x7f54d80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 -- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 msgr2=0x7f54d806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 0x7f54d806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 -- 172.21.15.110:0/1190444612 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d806ef70 con 0x7f54d80d0bd0 2024-09-10T09:58:35.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54d7400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d0bd0 0x7f54d8148fa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54c801a630 tx=0x7f54c80197a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54cf400640 1 -- 172.21.15.110:0/1190444612 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80bd020 con 0x7f54d80d0bd0 2024-09-10T09:58:35.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54cf400640 1 -- 172.21.15.110:0/1190444612 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c80b8470 con 0x7f54d80d0bd0 2024-09-10T09:58:35.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d806e9b0 con 0x7f54d80d0bd0 2024-09-10T09:58:35.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54cf400640 1 -- 172.21.15.110:0/1190444612 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80a58d0 con 0x7f54d80d0bd0 2024-09-10T09:58:35.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.224+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d80b5740 con 0x7f54d80d0bd0 2024-09-10T09:58:35.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.228+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d80d1a70 con 0x7f54d80d0bd0 2024-09-10T09:58:35.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.228+0000 7f54cf400640 1 -- 172.21.15.110:0/1190444612 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f54c80b8860 con 0x7f54d80d0bd0 2024-09-10T09:58:35.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.228+0000 7f54cf400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f54b80678c0 0x7f54b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.228+0000 7f54cf400640 1 -- 172.21.15.110:0/1190444612 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f54c8120cc0 con 0x7f54d80d0bd0 2024-09-10T09:58:35.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.228+0000 7f54d6a00640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f54b80678c0 0x7f54b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.228+0000 7f54d6a00640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f54b80678c0 0x7f54b8069d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54d806d9f0 tx=0x7f54c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.232+0000 7f54cf400640 1 -- 172.21.15.110:0/1190444612 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f54c8016620 con 0x7f54d80d0bd0 2024-09-10T09:58:35.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.424+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54d8070c00 con 0x7f54d80d0bd0 2024-09-10T09:58:35.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54cf400640 1 -- 172.21.15.110:0/1190444612 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5229 (secure 0 0 0) 0x7f54c80f1250 con 0x7f54d80d0bd0 2024-09-10T09:58:35.429 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:35.429 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:58:22.146304+0000","modified":"2024-09-10T09:58:25.155860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-09-10T09:58:35.430 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-09-10T09:58:35.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f54b80678c0 msgr2=0x7f54b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f54b80678c0 0x7f54b8069d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54d806d9f0 tx=0x7f54c4009040 comp rx=0 tx=0).stop 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d0bd0 msgr2=0x7f54d8148fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d0bd0 0x7f54d8148fa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54c801a630 tx=0x7f54c80197a0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 shutdown_connections 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f54b80678c0 0x7f54b8069d80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54d815c3d0 0x7f54d806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54d80d1660 0x7f54d80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 --2- 172.21.15.110:0/1190444612 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54d80d0bd0 0x7f54d8148fa0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 >> 172.21.15.110:0/1190444612 conn(0x7f54d806cf60 msgr2=0x7f54d80b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 shutdown_connections 2024-09-10T09:58:35.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.428+0000 7f54de400640 1 -- 172.21.15.110:0/1190444612 wait complete. 2024-09-10T09:58:35.450 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:35.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab92a00640 1 Processor -- start 2024-09-10T09:58:35.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab92a00640 1 -- start start 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab92a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c15c960 0x7fab8c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab92a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab8c1603c0 0x7fab8c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab92a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab8c0706a0 con 0x7fab8c0d04c0 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab92a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c15cd40 con 0x7fab8c1603c0 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab92a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c0b6370 con 0x7fab8c15c960 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab8c1603c0 0x7fab8c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab8c1603c0 0x7fab8c0d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53428/0 (socket says 172.21.15.110:53428) 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8b400640 1 -- 172.21.15.110:0/545464566 learned_addr learned my addr 172.21.15.110:0/545464566 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8be00640 1 --2- 172.21.15.110:0/545464566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c15c960 0x7fab8c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8b400640 1 -- 172.21.15.110:0/545464566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c15c960 msgr2=0x7fab8c0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8b400640 1 --2- 172.21.15.110:0/545464566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c15c960 0x7fab8c0b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8b400640 1 -- 172.21.15.110:0/545464566 >> v1:172.21.15.145:6789/0 conn(0x7fab8c0d04c0 legacy=0x7fab8c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8b400640 1 -- 172.21.15.110:0/545464566 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c070a80 con 0x7fab8c1603c0 2024-09-10T09:58:35.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.584+0000 7fab8be00640 1 --2- 172.21.15.110:0/545464566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c15c960 0x7fab8c0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:35.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab8b400640 1 --2- 172.21.15.110:0/545464566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c1603c0 0x7fab8c0d0cd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab78009870 tx=0x7fab7801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb7adba6aff5894 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab8aa00640 1 -- 172.21.15.110:0/545464566 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab780b0070 con 0x7fab8c1603c0 2024-09-10T09:58:35.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab8aa00640 1 -- 172.21.15.110:0/545464566 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab780ab600 con 0x7fab8c1603c0 2024-09-10T09:58:35.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab8aa00640 1 -- 172.21.15.110:0/545464566 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab780ab9d0 con 0x7fab8c1603c0 2024-09-10T09:58:35.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab92a00640 1 -- 172.21.15.110:0/545464566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c1603c0 msgr2=0x7fab8c0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab92a00640 1 --2- 172.21.15.110:0/545464566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c1603c0 0x7fab8c0d0cd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab78009870 tx=0x7fab7801c3d0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab92a00640 1 -- 172.21.15.110:0/545464566 shutdown_connections 2024-09-10T09:58:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab92a00640 1 --2- 172.21.15.110:0/545464566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c1603c0 0x7fab8c0d0cd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab92a00640 1 --2- 172.21.15.110:0/545464566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c15c960 0x7fab8c0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab92a00640 1 -- 172.21.15.110:0/545464566 >> 172.21.15.110:0/545464566 conn(0x7fab8c0d25c0 msgr2=0x7fab8c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab92a00640 1 -- 172.21.15.110:0/545464566 shutdown_connections 2024-09-10T09:58:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.588+0000 7fab92a00640 1 -- 172.21.15.110:0/545464566 wait complete. 2024-09-10T09:58:35.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 Processor -- start 2024-09-10T09:58:35.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 -- start start 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c0d04c0 0x7fab8c15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab8c15c960 0x7fab8c06e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c1603c0 0x7fab8c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c0d3150 con 0x7fab8c0d04c0 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1611b0 con 0x7fab8c15c960 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c0d32d0 con 0x7fab8c1603c0 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c0d04c0 0x7fab8c15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c0d04c0 0x7fab8c15e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53440/0 (socket says 172.21.15.110:53440) 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 -- 172.21.15.110:0/3922905934 learned_addr learned my addr 172.21.15.110:0/3922905934 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8b400640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab8c15c960 0x7fab8c06e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab90a00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c1603c0 0x7fab8c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 -- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c1603c0 msgr2=0x7fab8c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c1603c0 0x7fab8c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 -- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab8c15c960 msgr2=0x7fab8c06e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab8c15c960 0x7fab8c06e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 -- 172.21.15.110:0/3922905934 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c15d060 con 0x7fab8c0d04c0 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8b400640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab8c15c960 0x7fab8c06e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab8be00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c0d04c0 0x7fab8c15e100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00c7f0 tx=0x7fab7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab83e00640 1 -- 172.21.15.110:0/3922905934 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c007a30 con 0x7fab8c0d04c0 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab83e00640 1 -- 172.21.15.110:0/3922905934 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7c007bd0 con 0x7fab8c0d04c0 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c15d340 con 0x7fab8c0d04c0 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab83e00640 1 -- 172.21.15.110:0/3922905934 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0113f0 con 0x7fab8c0d04c0 2024-09-10T09:58:35.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.592+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c15d900 con 0x7fab8c0d04c0 2024-09-10T09:58:35.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.596+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c071a80 con 0x7fab8c0d04c0 2024-09-10T09:58:35.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.596+0000 7fab83e00640 1 -- 172.21.15.110:0/3922905934 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab7c011590 con 0x7fab8c0d04c0 2024-09-10T09:58:35.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.596+0000 7fab83e00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab6c0678c0 0x7fab6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.596+0000 7fab83e00640 1 -- 172.21.15.110:0/3922905934 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fab7c085ea0 con 0x7fab8c0d04c0 2024-09-10T09:58:35.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.596+0000 7fab8b400640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab6c0678c0 0x7fab6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.596+0000 7fab8b400640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab6c0678c0 0x7fab6c069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab780ad000 tx=0x7fab78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.600+0000 7fab83e00640 1 -- 172.21.15.110:0/3922905934 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fab7c015020 con 0x7fab8c0d04c0 2024-09-10T09:58:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.788+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab8c0d34d0 con 0x7fab8c0d04c0 2024-09-10T09:58:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab83e00640 1 -- 172.21.15.110:0/3922905934 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5229 (secure 0 0 0) 0x7fab7c0569d0 con 0x7fab8c0d04c0 2024-09-10T09:58:35.793 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:35.793 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:58:22.146304+0000","modified":"2024-09-10T09:58:25.155860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-09-10T09:58:35.793 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab6c0678c0 msgr2=0x7fab6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab6c0678c0 0x7fab6c069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab780ad000 tx=0x7fab78008040 comp rx=0 tx=0).stop 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c0d04c0 msgr2=0x7fab8c15e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c0d04c0 0x7fab8c15e100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00c7f0 tx=0x7fab7c00ccb0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 shutdown_connections 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab6c0678c0 0x7fab6c069d80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab8c1603c0 0x7fab8c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab8c15c960 0x7fab8c06e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 --2- 172.21.15.110:0/3922905934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab8c0d04c0 0x7fab8c15e100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 >> 172.21.15.110:0/3922905934 conn(0x7fab8c0d25c0 msgr2=0x7fab8c074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 shutdown_connections 2024-09-10T09:58:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.792+0000 7fab92a00640 1 -- 172.21.15.110:0/3922905934 wait complete. 2024-09-10T09:58:35.814 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:35.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1c400640 1 Processor -- start 2024-09-10T09:58:35.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1c400640 1 -- start start 2024-09-10T09:58:35.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1c400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2c14147f00 0x7f2c140ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1c400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2c1406edf0 0x7f2c14072310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1c400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c140af1a0 con 0x7f2c140aed80 2024-09-10T09:58:35.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1c400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2c140b34d0 con 0x7f2c14147f00 2024-09-10T09:58:35.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1c400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2c141482e0 con 0x7f2c1406edf0 2024-09-10T09:58:35.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c19a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2c14147f00 0x7f2c140ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c19000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2c140aed80 0x7f2c1406e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38296/0 (socket says 172.21.15.110:38296) 2024-09-10T09:58:35.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c19000640 1 -- 172.21.15.110:0/2262748954 learned_addr learned my addr 172.21.15.110:0/2262748954 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:35.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c13e00640 1 -- 172.21.15.110:0/2262748954 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014142482 0 0) 0x7f2c140af1a0 con 0x7f2c140aed80 2024-09-10T09:58:35.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c13e00640 1 -- 172.21.15.110:0/2262748954 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bfc003530 con 0x7f2c140aed80 2024-09-10T09:58:35.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1a400640 1 --2- 172.21.15.110:0/2262748954 >> v2:172.21.15.145:3300/0 conn(0x7f2c1406edf0 0x7f2c14072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c19a00640 1 -- 172.21.15.110:0/2262748954 >> v2:172.21.15.145:3300/0 conn(0x7f2c1406edf0 msgr2=0x7f2c14072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:35.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c19a00640 1 --2- 172.21.15.110:0/2262748954 >> v2:172.21.15.145:3300/0 conn(0x7f2c1406edf0 0x7f2c14072310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c19a00640 1 -- 172.21.15.110:0/2262748954 >> v1:172.21.15.110:6789/0 conn(0x7f2c140aed80 legacy=0x7f2c1406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c19a00640 1 -- 172.21.15.110:0/2262748954 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1415ee20 con 0x7f2c14147f00 2024-09-10T09:58:35.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.948+0000 7f2c1a400640 1 --2- 172.21.15.110:0/2262748954 >> v2:172.21.15.145:3300/0 conn(0x7f2c1406edf0 0x7f2c14072310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:35.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c19a00640 1 --2- 172.21.15.110:0/2262748954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140ae840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2c140ccde0 tx=0x7f2c0801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890997e38581aa5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c13e00640 1 -- 172.21.15.110:0/2262748954 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080b2070 con 0x7f2c14147f00 2024-09-10T09:58:35.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c13e00640 1 -- 172.21.15.110:0/2262748954 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c080a5900 con 0x7f2c14147f00 2024-09-10T09:58:35.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c13e00640 1 -- 172.21.15.110:0/2262748954 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080a5cd0 con 0x7f2c14147f00 2024-09-10T09:58:35.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c1c400640 1 -- 172.21.15.110:0/2262748954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 msgr2=0x7f2c140ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2262748954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140ae840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2c140ccde0 tx=0x7f2c0801c930 comp rx=0 tx=0).stop 2024-09-10T09:58:35.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c1c400640 1 -- 172.21.15.110:0/2262748954 shutdown_connections 2024-09-10T09:58:35.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2262748954 >> v2:172.21.15.145:3300/0 conn(0x7f2c1406edf0 0x7f2c14072310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2262748954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140ae840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c1c400640 1 -- 172.21.15.110:0/2262748954 >> 172.21.15.110:0/2262748954 conn(0x7f2c1406bc50 msgr2=0x7f2c1406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:35.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c1c400640 1 -- 172.21.15.110:0/2262748954 shutdown_connections 2024-09-10T09:58:35.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.952+0000 7f2c1c400640 1 -- 172.21.15.110:0/2262748954 wait complete. 2024-09-10T09:58:35.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 Processor -- start 2024-09-10T09:58:35.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 -- start start 2024-09-10T09:58:35.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c1406edf0 0x7f2c140cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c140aed80 0x7f2c140b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c140731e0 con 0x7f2c14147f00 2024-09-10T09:58:35.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c14073060 con 0x7f2c1406edf0 2024-09-10T09:58:35.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2c14073360 con 0x7f2c140aed80 2024-09-10T09:58:35.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53454/0 (socket says 172.21.15.110:53454) 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 -- 172.21.15.110:0/2111517908 learned_addr learned my addr 172.21.15.110:0/2111517908 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c19000640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c140aed80 0x7f2c140b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c19a00640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c1406edf0 0x7f2c140cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 -- 172.21.15.110:0/2111517908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c140aed80 msgr2=0x7f2c140b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c140aed80 0x7f2c140b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 -- 172.21.15.110:0/2111517908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c1406edf0 msgr2=0x7f2c140cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c1406edf0 0x7f2c140cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 -- 172.21.15.110:0/2111517908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c14143c10 con 0x7f2c14147f00 2024-09-10T09:58:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1a400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140b1670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00c7f0 tx=0x7f2c0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c11600640 1 -- 172.21.15.110:0/2111517908 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c015020 con 0x7f2c14147f00 2024-09-10T09:58:35.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c11600640 1 -- 172.21.15.110:0/2111517908 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0c004250 con 0x7f2c14147f00 2024-09-10T09:58:35.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c11600640 1 -- 172.21.15.110:0/2111517908 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c003620 con 0x7f2c14147f00 2024-09-10T09:58:35.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c14143eb0 con 0x7f2c14147f00 2024-09-10T09:58:35.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.956+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c14144470 con 0x7f2c14147f00 2024-09-10T09:58:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.960+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c141453a0 con 0x7f2c14147f00 2024-09-10T09:58:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.960+0000 7f2c11600640 1 -- 172.21.15.110:0/2111517908 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2c0c018020 con 0x7f2c14147f00 2024-09-10T09:58:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.960+0000 7f2c11600640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2bf40678c0 0x7f2bf4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.960+0000 7f2c11600640 1 -- 172.21.15.110:0/2111517908 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2c0c085c30 con 0x7f2c14147f00 2024-09-10T09:58:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.960+0000 7f2c19a00640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2bf40678c0 0x7f2bf4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:35.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.960+0000 7f2c19a00640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2bf40678c0 0x7f2bf4069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2c080af000 tx=0x7f2c080a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:35.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:35.964+0000 7f2c11600640 1 -- 172.21.15.110:0/2111517908 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2c0c01f030 con 0x7f2c14147f00 2024-09-10T09:58:36.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.124+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c140b4ac0 con 0x7f2c14147f00 2024-09-10T09:58:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.124+0000 7f2c11600640 1 -- 172.21.15.110:0/2111517908 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16701 (secure 0 0 0) 0x7f2c0c056760 con 0x7f2c14147f00 2024-09-10T09:58:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:36.127 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:23.766388+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2bf40678c0 msgr2=0x7f2bf4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2bf40678c0 0x7f2bf4069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2c080af000 tx=0x7f2c080a7070 comp rx=0 tx=0).stop 2024-09-10T09:58:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 msgr2=0x7f2c140b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140b1670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00c7f0 tx=0x7f2c0c00ccb0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 shutdown_connections 2024-09-10T09:58:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2bf40678c0 0x7f2bf4069d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c14147f00 0x7f2c140b1670 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c140aed80 0x7f2c140b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 --2- 172.21.15.110:0/2111517908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c1406edf0 0x7f2c140cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 >> 172.21.15.110:0/2111517908 conn(0x7f2c1406bc50 msgr2=0x7f2c14144990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 shutdown_connections 2024-09-10T09:58:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.128+0000 7f2c1c400640 1 -- 172.21.15.110:0/2111517908 wait complete. 2024-09-10T09:58:36.148 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:36.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.280+0000 7f66c0400640 1 Processor -- start 2024-09-10T09:58:36.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.280+0000 7f66c0400640 1 -- start start 2024-09-10T09:58:36.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66c0400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f66b8072840 0x7f66b8140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:36.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66c0400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80ccf40 con 0x7f66b80ccb60 2024-09-10T09:58:36.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66c0400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b81412a0 con 0x7f66b8140a60 2024-09-10T09:58:36.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66c0400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f66b815e760 con 0x7f66b8072840 2024-09-10T09:58:36.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66bd000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f66b8072840 0x7f66b8140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:36.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66bda00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f66b80ccb60 0x7f66b8072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38304/0 (socket says 172.21.15.110:38304) 2024-09-10T09:58:36.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66bda00640 1 -- 172.21.15.110:0/1161833560 learned_addr learned my addr 172.21.15.110:0/1161833560 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:36.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66bd000640 1 -- 172.21.15.110:0/1161833560 >> v1:172.21.15.145:6789/0 conn(0x7f66b8140a60 legacy=0x7f66b815b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66bd000640 1 -- 172.21.15.110:0/1161833560 >> v1:172.21.15.110:6789/0 conn(0x7f66b80ccb60 legacy=0x7f66b8072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66bd000640 1 -- 172.21.15.110:0/1161833560 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b815ea40 con 0x7f66b8072840 2024-09-10T09:58:36.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66b7e00640 1 -- 172.21.15.110:0/1161833560 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325298369 0 0) 0x7f66b80ccf40 con 0x7f66b80ccb60 2024-09-10T09:58:36.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.284+0000 7f66bd000640 1 --2- 172.21.15.110:0/1161833560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8140520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f66a40097c0 tx=0x7f66a401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332b2a23e10b5231 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:36.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66b7e00640 1 -- 172.21.15.110:0/1161833560 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a40a7070 con 0x7f66b8072840 2024-09-10T09:58:36.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66b7e00640 1 -- 172.21.15.110:0/1161833560 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a40b0070 con 0x7f66b8072840 2024-09-10T09:58:36.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66b7e00640 1 -- 172.21.15.110:0/1161833560 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a40ab890 con 0x7f66b8072840 2024-09-10T09:58:36.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- 172.21.15.110:0/1161833560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 msgr2=0x7f66b8140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 --2- 172.21.15.110:0/1161833560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8140520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f66a40097c0 tx=0x7f66a401c470 comp rx=0 tx=0).stop 2024-09-10T09:58:36.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- 172.21.15.110:0/1161833560 shutdown_connections 2024-09-10T09:58:36.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 --2- 172.21.15.110:0/1161833560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8140520 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- 172.21.15.110:0/1161833560 >> 172.21.15.110:0/1161833560 conn(0x7f66b806bc50 msgr2=0x7f66b806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:36.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- 172.21.15.110:0/1161833560 shutdown_connections 2024-09-10T09:58:36.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- 172.21.15.110:0/1161833560 wait complete. 2024-09-10T09:58:36.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 Processor -- start 2024-09-10T09:58:36.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- start start 2024-09-10T09:58:36.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:36.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b80ccb60 0x7f66b80b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:36.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66b8140a60 0x7f66b813baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:36.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8140ff0 con 0x7f66b8072840 2024-09-10T09:58:36.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8140e70 con 0x7f66b80ccb60 2024-09-10T09:58:36.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66c0400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f66b815b830 con 0x7f66b8140a60 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66bda00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66bda00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53468/0 (socket says 172.21.15.110:53468) 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66bda00640 1 -- 172.21.15.110:0/3352605335 learned_addr learned my addr 172.21.15.110:0/3352605335 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66bd000640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b80ccb60 0x7f66b80b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.288+0000 7f66be400640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66b8140a60 0x7f66b813baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66bda00640 1 -- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66b8140a60 msgr2=0x7f66b813baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66bda00640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66b8140a60 0x7f66b813baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66bda00640 1 -- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b80ccb60 msgr2=0x7f66b80b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66bda00640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b80ccb60 0x7f66b80b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66bda00640 1 -- 172.21.15.110:0/3352605335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b80b27c0 con 0x7f66b8072840 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66be400640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66b8140a60 0x7f66b813baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66bda00640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8155470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c8c0 tx=0x7f66ac00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66b5600640 1 -- 172.21.15.110:0/3352605335 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac018070 con 0x7f66b8072840 2024-09-10T09:58:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66b5600640 1 -- 172.21.15.110:0/3352605335 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac00a640 con 0x7f66b8072840 2024-09-10T09:58:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b81537c0 con 0x7f66b8072840 2024-09-10T09:58:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66b5600640 1 -- 172.21.15.110:0/3352605335 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00aa70 con 0x7f66b8072840 2024-09-10T09:58:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b8153d80 con 0x7f66b8072840 2024-09-10T09:58:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8140520 con 0x7f66b8072840 2024-09-10T09:58:36.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66b5600640 1 -- 172.21.15.110:0/3352605335 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f66ac01b070 con 0x7f66b8072840 2024-09-10T09:58:36.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66b5600640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66980678c0 0x7f6698069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:36.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66b5600640 1 -- 172.21.15.110:0/3352605335 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f66ac085320 con 0x7f66b8072840 2024-09-10T09:58:36.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.292+0000 7f66bd000640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66980678c0 0x7f6698069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:36.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.296+0000 7f66bd000640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66980678c0 0x7f6698069d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66a40af000 tx=0x7f66a40a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:36.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.296+0000 7f66b5600640 1 -- 172.21.15.110:0/3352605335 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f66ac055770 con 0x7f66b8072840 2024-09-10T09:58:36.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.484+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66b8156130 con 0x7f66b8072840 2024-09-10T09:58:36.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66b5600640 1 -- 172.21.15.110:0/3352605335 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5229 (secure 0 0 0) 0x7f66ac059a70 con 0x7f66b8072840 2024-09-10T09:58:36.489 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:36.489 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:58:22.146304+0000","modified":"2024-09-10T09:58:25.155860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-09-10T09:58:36.489 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-09-10T09:58:36.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66980678c0 msgr2=0x7f6698069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66980678c0 0x7f6698069d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66a40af000 tx=0x7f66a40a7000 comp rx=0 tx=0).stop 2024-09-10T09:58:36.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 msgr2=0x7f66b8155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:36.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8155470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c8c0 tx=0x7f66ac00cd80 comp rx=0 tx=0).stop 2024-09-10T09:58:36.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 shutdown_connections 2024-09-10T09:58:36.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66980678c0 0x7f6698069d80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66b8140a60 0x7f66b813baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66b80ccb60 0x7f66b80b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 --2- 172.21.15.110:0/3352605335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8072840 0x7f66b8155470 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:36.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 >> 172.21.15.110:0/3352605335 conn(0x7f66b806bc50 msgr2=0x7f66b80b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:36.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 shutdown_connections 2024-09-10T09:58:36.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:36.488+0000 7f66c0400640 1 -- 172.21.15.110:0/3352605335 wait complete. 2024-09-10T09:58:36.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T09:58:22.146304+0000', 'modified': '2024-09-10T09:58:25.155860+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.145:6834/2931287411', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6832', 'nonce': 2931287411}, {'type': 'v1', 'addr': '172.21.15.145:6834', 'nonce': 2931287411}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4270, 'qdb_cluster': [4270]} 2024-09-10T09:58:36.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T09:58:36.912 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T09:58:37.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb530800640 1 Processor -- start 2024-09-10T09:58:37.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb530800640 1 -- start start 2024-09-10T09:58:37.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb530800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5280b5600 0x7fb5280cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb530800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb52815fda0 con 0x7fb52806efa0 2024-09-10T09:58:37.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb530800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5280ce520 con 0x7fb5280726e0 2024-09-10T09:58:37.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb530800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb5280cee50 con 0x7fb5280b5600 2024-09-10T09:58:37.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52e800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb52806efa0 0x7fb5280b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:38306/0 (socket says 172.21.15.110:38306) 2024-09-10T09:58:37.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52e800640 1 -- 172.21.15.110:0/2558155120 learned_addr learned my addr 172.21.15.110:0/2558155120 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:37.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52d400640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5280b5600 0x7fb5280cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 151304522 0 0) 0x7fb52815fda0 con 0x7fb52806efa0 2024-09-10T09:58:37.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb510005740 con 0x7fb52806efa0 2024-09-10T09:58:37.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 603312632 0 0) 0x7fb510005740 con 0x7fb52806efa0 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5280b5600 msgr2=0x7fb5280cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5280b5600 0x7fb5280cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 >> v1:172.21.15.145:6789/0 conn(0x7fb5280726e0 legacy=0x7fb5280b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5280ce800 con 0x7fb52806efa0 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fb524001a00 con 0x7fb52806efa0 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 >> v1:172.21.15.110:6789/0 conn(0x7fb52806efa0 legacy=0x7fb5280b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb510006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb510006a20 0x7fb510006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5280b5600 0x7fb5100073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5280cee50 con 0x7fb510005c00 2024-09-10T09:58:37.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb51000dc90 con 0x7fb510006a20 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb51000de90 con 0x7fb5280b5600 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52de00640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb510006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52d400640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb510006a20 0x7fb510006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52e800640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5280b5600 0x7fb5100073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52de00640 1 -- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5280b5600 msgr2=0x7fb5100073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52de00640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5280b5600 0x7fb5100073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52de00640 1 -- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb510006a20 msgr2=0x7fb510006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52de00640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb510006a20 0x7fb510006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.048+0000 7fb52de00640 1 -- 172.21.15.110:0/2558155120 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb510005740 con 0x7fb510005c00 2024-09-10T09:58:37.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb52de00640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb510006020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb51c001750 tx=0x7fb51c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31fcb29ac2b142 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0013a0 con 0x7fb510005c00 2024-09-10T09:58:37.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51c0052f0 con 0x7fb510005c00 2024-09-10T09:58:37.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb52ca00640 1 -- 172.21.15.110:0/2558155120 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c003840 con 0x7fb510005c00 2024-09-10T09:58:37.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb530800640 1 -- 172.21.15.110:0/2558155120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 msgr2=0x7fb510006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb530800640 1 --2- 172.21.15.110:0/2558155120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb510006020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb51c001750 tx=0x7fb51c01c390 comp rx=0 tx=0).stop 2024-09-10T09:58:37.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb52e800640 1 -- 172.21.15.110:0/2558155120 reap_dead start 2024-09-10T09:58:37.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb530800640 1 -- 172.21.15.110:0/2558155120 shutdown_connections 2024-09-10T09:58:37.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb530800640 1 -- 172.21.15.110:0/2558155120 >> 172.21.15.110:0/2558155120 conn(0x7fb528062d10 msgr2=0x7fb52806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb530800640 1 -- 172.21.15.110:0/2558155120 shutdown_connections 2024-09-10T09:58:37.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.052+0000 7fb530800640 1 -- 172.21.15.110:0/2558155120 wait complete. 2024-09-10T09:58:37.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 Processor -- start 2024-09-10T09:58:37.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 -- start start 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb52814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5281485a0 0x7fb5280b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5280b1c70 0x7fb528153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5280b3ff0 con 0x7fb510005c00 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb528073270 con 0x7fb5280b1c70 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb5280b4170 con 0x7fb5281485a0 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb52814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb52814afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53488/0 (socket says 172.21.15.110:53488) 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 -- 172.21.15.110:0/298160843 learned_addr learned my addr 172.21.15.110:0/298160843 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52e800640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5280b1c70 0x7fb528153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52d400640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5281485a0 0x7fb5280b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 -- 172.21.15.110:0/298160843 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5281485a0 msgr2=0x7fb5280b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5281485a0 0x7fb5280b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 -- 172.21.15.110:0/298160843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5280b1c70 msgr2=0x7fb528153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5280b1c70 0x7fb528153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 -- 172.21.15.110:0/298160843 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb528154680 con 0x7fb510005c00 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb52de00640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb52814afc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb51c001750 tx=0x7fb510007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb522000640 1 -- 172.21.15.110:0/298160843 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0ba030 con 0x7fb510005c00 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb522000640 1 -- 172.21.15.110:0/298160843 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51c000ee0 con 0x7fb510005c00 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb522000640 1 -- 172.21.15.110:0/298160843 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c007a40 con 0x7fb510005c00 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 -- 172.21.15.110:0/298160843 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb528154960 con 0x7fb510005c00 2024-09-10T09:58:37.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.056+0000 7fb530800640 1 -- 172.21.15.110:0/298160843 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb528154f20 con 0x7fb510005c00 2024-09-10T09:58:37.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb522000640 1 -- 172.21.15.110:0/298160843 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb51c0bd070 con 0x7fb510005c00 2024-09-10T09:58:37.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb522000640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb50c0678f0 0x7fb50c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb522000640 1 -- 172.21.15.110:0/298160843 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb51c11f7e0 con 0x7fb510005c00 2024-09-10T09:58:37.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb52d400640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb50c0678f0 0x7fb50c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb533bb5000 1 Processor -- start 2024-09-10T09:58:37.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb533bb5000 1 -- start start 2024-09-10T09:58:37.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb52d400640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb50c0678f0 0x7fb50c069db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cfc50 tx=0x7fb518008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.060+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654caaefd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caaf0290 0x5654caaf0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654ca9d7b60 con 0x5654cab39b20 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5654ca93ec70 con 0x5654cab3a440 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5654ca8e4ed0 con 0x5654caaf0290 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab39f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53494/0 (socket says 172.21.15.110:53494) 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 -- 172.21.15.110:0/2808996476 learned_addr learned my addr 172.21.15.110:0/2808996476 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52d400640 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654caaefd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52e800640 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caaf0290 0x5654caaf0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 -- 172.21.15.110:0/2808996476 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caaf0290 msgr2=0x5654caaf0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caaf0290 0x5654caaf0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 -- 172.21.15.110:0/2808996476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 msgr2=0x5654caaefd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654caaefd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 -- 172.21.15.110:0/2808996476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ca8f6e50 con 0x5654cab39b20 2024-09-10T09:58:37.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab39f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0ec070 tx=0x7fb51c0ef570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac79c22829a7f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb516a00640 1 -- 172.21.15.110:0/2808996476 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0f3fc0 con 0x5654cab39b20 2024-09-10T09:58:37.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb516a00640 1 -- 172.21.15.110:0/2808996476 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51c0f87f0 con 0x5654cab39b20 2024-09-10T09:58:37.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb516a00640 1 -- 172.21.15.110:0/2808996476 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c005350 con 0x5654cab39b20 2024-09-10T09:58:37.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- 172.21.15.110:0/2808996476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 msgr2=0x5654cab39f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab39f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0ec070 tx=0x7fb51c0ef570 comp rx=0 tx=0).stop 2024-09-10T09:58:37.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- 172.21.15.110:0/2808996476 shutdown_connections 2024-09-10T09:58:37.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caaf0290 0x5654caaf0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654caaefd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2808996476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab39f00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- 172.21.15.110:0/2808996476 >> 172.21.15.110:0/2808996476 conn(0x5654caaf6600 msgr2=0x5654caaee630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- 172.21.15.110:0/2808996476 shutdown_connections 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- 172.21.15.110:0/2808996476 wait complete. 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- start start 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 Processor -- start 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654cab39b20 0x5654caaf4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab3a440 0x5654caaf5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654caafacf0 0x5654caafb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654ca9a0b00 con 0x5654cab3a440 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5654ca919360 con 0x5654caafacf0 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5654ca9162d0 con 0x5654cab39b20 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb533bb5000 10 client.0 _ll_register_callbacks cb 0x5654caadb820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab3a440 0x5654caaf5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab3a440 0x5654caaf5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53508/0 (socket says 172.21.15.110:53508) 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52d400640 1 -- 172.21.15.110:0/2106755290 learned_addr learned my addr 172.21.15.110:0/2106755290 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.064+0000 7fb52de00640 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654cab39b20 0x5654caaf4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52e800640 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654caafacf0 0x5654caafb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52d400640 1 -- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654cab39b20 msgr2=0x5654caaf4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52d400640 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654cab39b20 0x5654caaf4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52d400640 1 -- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654caafacf0 msgr2=0x5654caafb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52d400640 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654caafacf0 0x5654caafb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52d400640 1 -- 172.21.15.110:0/2106755290 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ca8e5120 con 0x5654cab3a440 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52d400640 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab3a440 0x5654caaf5150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb5180169e0 tx=0x7fb518004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba3f970260d79af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 1 -- 172.21.15.110:0/2106755290 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518015b50 con 0x5654cab3a440 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 1 -- 172.21.15.110:0/2106755290 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb51c0f87f0 con 0x5654cab3a440 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 1 -- 172.21.15.110:0/2106755290 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb518015cf0 con 0x5654cab3a440 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 1 -- 172.21.15.110:0/2106755290 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fb518034050 con 0x5654cab3a440 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 10 client.4578 fetch_fsmap learned FSMap version 7 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 1 -- 172.21.15.110:0/2106755290 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5654ca8de450 con 0x5654cab3a440 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 1 -- 172.21.15.110:0/2106755290 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51c0f87f0 con 0x5654cab3a440 2024-09-10T09:58:37.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 1 -- 172.21.15.110:0/2106755290 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fb518037020 con 0x5654cab3a440 2024-09-10T09:58:37.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 10 client.4578 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T09:58:37.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 10 client.4578 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:58:37.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 20 client.4578 populate_metadata read hostname 'smithi110' 2024-09-10T09:58:37.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] conn(0x5654cab48280 0x5654cab4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 4 client.4578 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:37.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 1 -- 172.21.15.110:0/2106755290 --> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654ca8d8c00 con 0x5654cab48280 2024-09-10T09:58:37.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52de00640 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] conn(0x5654cab48280 0x5654cab4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb52de00640 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] conn(0x5654cab48280 0x5654cab4a670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b328e0e8b4eb0351 server_cookie=dd92b40e9e8f2d1b in_seq=0 out_seq=0 2024-09-10T09:58:37.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 10 client.4578 ms_handle_connect on v2:172.21.15.145:6832/2931287411 2024-09-10T09:58:37.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 1 -- 172.21.15.110:0/2106755290 <== mds.0 v2:172.21.15.145:6832/2931287411 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654ca8d8c00 con 0x5654cab48280 2024-09-10T09:58:37.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb50b400640 10 client.4578 handle_command_reply: tid=0 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 1 client.4578 shutdown 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 1 -- 172.21.15.110:0/2106755290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab3a440 msgr2=0x5654caaf5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.068+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab3a440 0x5654caaf5150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb5180169e0 tx=0x7fb518004740 comp rx=0 tx=0).stop 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 1 -- 172.21.15.110:0/2106755290 shutdown_connections 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654caafacf0 0x5654caafb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab3a440 0x5654caaf5150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] conn(0x5654cab48280 0x5654cab4a670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2106755290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654cab39b20 0x5654caaf4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 1 -- 172.21.15.110:0/2106755290 >> 172.21.15.110:0/2106755290 conn(0x5654caaf0290 msgr2=0x5654caaeed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb50b400640 0 client.4578 ms_handle_reset on v2:172.21.15.145:6832/2931287411 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 1 -- 172.21.15.110:0/2106755290 shutdown_connections 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 1 -- 172.21.15.110:0/2106755290 wait complete. 2024-09-10T09:58:37.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.072+0000 7fb533bb5000 20 client.4578 trim_cache size 0 max 16384 2024-09-10T09:58:37.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb533bb5000 1 Processor -- start 2024-09-10T09:58:37.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb533bb5000 1 -- start start 2024-09-10T09:58:37.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caafacf0 0x5654cab38b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654cab4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb533bb5000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb528203bf0 con 0x5654cab39b20 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb533bb5000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb528159ff0 con 0x5654cab3a440 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb533bb5000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb5282035a0 con 0x5654caafacf0 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb52d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb52d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab390a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53522/0 (socket says 172.21.15.110:53522) 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb52d400640 1 -- 172.21.15.110:0/2952536878 learned_addr learned my addr 172.21.15.110:0/2952536878 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb52d400640 1 -- 172.21.15.110:0/2952536878 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caafacf0 msgr2=0x5654cab38b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb52d400640 1 --2- 172.21.15.110:0/2952536878 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caafacf0 0x5654cab38b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb52d400640 1 -- 172.21.15.110:0/2952536878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 msgr2=0x5654cab4a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb52d400640 1 --2- 172.21.15.110:0/2952536878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654cab4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.088+0000 7fb52d400640 1 -- 172.21.15.110:0/2952536878 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ca8c5fd0 con 0x5654cab39b20 2024-09-10T09:58:37.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 --2- 172.21.15.110:0/2952536878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab390a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb518014790 tx=0x7fb518025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80f07ace5498f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb516000640 1 -- 172.21.15.110:0/2952536878 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518031070 con 0x5654cab39b20 2024-09-10T09:58:37.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb516000640 1 -- 172.21.15.110:0/2952536878 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51803d070 con 0x5654cab39b20 2024-09-10T09:58:37.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb516000640 1 -- 172.21.15.110:0/2952536878 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518024430 con 0x5654cab39b20 2024-09-10T09:58:37.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- 172.21.15.110:0/2952536878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 msgr2=0x5654cab390a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2952536878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab390a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb518014790 tx=0x7fb518025000 comp rx=0 tx=0).stop 2024-09-10T09:58:37.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- 172.21.15.110:0/2952536878 shutdown_connections 2024-09-10T09:58:37.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2952536878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654cab4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2952536878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab390a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 --2- 172.21.15.110:0/2952536878 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caafacf0 0x5654cab38b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- 172.21.15.110:0/2952536878 >> 172.21.15.110:0/2952536878 conn(0x5654caaf0290 msgr2=0x5654caaf1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- 172.21.15.110:0/2952536878 shutdown_connections 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- 172.21.15.110:0/2952536878 wait complete. 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- start start 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 Processor -- start 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caafacf0 0x5654cab4d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab68960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654cab68ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654ca93df80 con 0x5654cab39b20 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5654ca8f6ca0 con 0x5654cab3a440 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5654ca9432a0 con 0x5654caafacf0 2024-09-10T09:58:37.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 10 client.0 _ll_register_callbacks cb 0x7fb51801a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab68960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab68960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53536/0 (socket says 172.21.15.110:53536) 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 -- 172.21.15.110:0/1465118510 learned_addr learned my addr 172.21.15.110:0/1465118510 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 -- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caafacf0 msgr2=0x5654cab4d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caafacf0 0x5654cab4d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 -- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 msgr2=0x5654cab68ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52e800640 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654cab68ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654cab68ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 -- 172.21.15.110:0/1465118510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ca9c1750 con 0x5654cab39b20 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52e800640 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654cab68ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb52d400640 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab68960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb518014910 tx=0x7fb518019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cb96d07d089b7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb50b400640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T09:58:37.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb50b400640 1 -- 172.21.15.110:0/1465118510 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518047020 con 0x5654cab39b20 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb50b400640 1 -- 172.21.15.110:0/1465118510 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb51802e6f0 con 0x5654cab39b20 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.092+0000 7fb533bb5000 1 -- 172.21.15.110:0/1465118510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb51803d070 con 0x5654cab39b20 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb50b400640 1 -- 172.21.15.110:0/1465118510 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fb51802e0c0 con 0x5654cab39b20 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 10 client.4584 fetch_fsmap learned FSMap version 7 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 -- 172.21.15.110:0/1465118510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5654ca8cca30 con 0x5654cab39b20 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb50b400640 1 -- 172.21.15.110:0/1465118510 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51803d070 con 0x5654cab39b20 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb50b400640 1 -- 172.21.15.110:0/1465118510 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fb518037070 con 0x5654cab39b20 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 10 client.4584 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 10 client.4584 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 20 client.4584 populate_metadata read hostname 'smithi110' 2024-09-10T09:58:37.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] conn(0x5654cab693e0 0x5654cabac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 4 client.4584 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:58:37.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 -- 172.21.15.110:0/1465118510 --> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] -- command(tid 0: {"prefix": "status"}) -- 0x5654ca94c3a0 con 0x5654cab693e0 2024-09-10T09:58:37.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb52de00640 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] conn(0x5654cab693e0 0x5654cabac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb52de00640 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] conn(0x5654cab693e0 0x5654cabac9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8590a3f39fb7536f server_cookie=9a7b884bde76b3f9 in_seq=0 out_seq=0 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb50b400640 10 client.4584 ms_handle_connect on v2:172.21.15.145:6832/2931287411 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb50b400640 1 -- 172.21.15.110:0/1465118510 <== mds.0 v2:172.21.15.145:6832/2931287411 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5654ca94c3a0 con 0x5654cab693e0 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb50b400640 10 client.4584 handle_command_reply: tid=0 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 client.4584 shutdown 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 -- 172.21.15.110:0/1465118510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 msgr2=0x5654cab68960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab68960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb518014910 tx=0x7fb518019000 comp rx=0 tx=0).stop 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 -- 172.21.15.110:0/1465118510 shutdown_connections 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5654cab3a440 0x5654cab68ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654cab39b20 0x5654cab68960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:6832/2931287411,v1:172.21.15.145:6834/2931287411] conn(0x5654cab693e0 0x5654cabac9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 --2- 172.21.15.110:0/1465118510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5654caafacf0 0x5654cab4d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb50b400640 0 client.4584 ms_handle_reset on v2:172.21.15.145:6832/2931287411 2024-09-10T09:58:37.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 -- 172.21.15.110:0/1465118510 >> 172.21.15.110:0/1465118510 conn(0x5654caaf0290 msgr2=0x5654cab37b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 -- 172.21.15.110:0/1465118510 shutdown_connections 2024-09-10T09:58:37.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 1 -- 172.21.15.110:0/1465118510 wait complete. 2024-09-10T09:58:37.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb533bb5000 20 client.4584 trim_cache size 0 max 16384 2024-09-10T09:58:37.103 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 4270, 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 14.329055442, 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 7, 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 23, 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 23, 2024-09-10T09:58:37.104 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 20.375748725000001 2024-09-10T09:58:37.105 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T09:58:37.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 -- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb50c0678f0 msgr2=0x7fb50c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb50c0678f0 0x7fb50c069db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cfc50 tx=0x7fb518008040 comp rx=0 tx=0).stop 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 -- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 msgr2=0x7fb52814afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb52814afc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb51c001750 tx=0x7fb510007a50 comp rx=0 tx=0).stop 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 -- 172.21.15.110:0/298160843 shutdown_connections 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb50c0678f0 0x7fb50c069db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5280b1c70 0x7fb528153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5281485a0 0x7fb5280b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 --2- 172.21.15.110:0/298160843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510005c00 0x7fb52814afc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 -- 172.21.15.110:0/298160843 >> 172.21.15.110:0/298160843 conn(0x7fb528062d10 msgr2=0x7fb5280ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 -- 172.21.15.110:0/298160843 shutdown_connections 2024-09-10T09:58:37.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.096+0000 7fb530800640 1 -- 172.21.15.110:0/298160843 wait complete. 2024-09-10T09:58:37.118 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T09:58:37.128 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T09:58:37.128 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T09:58:37.129 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T09:58:37.129 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T09:58:37.177 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T09:58:37.177 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T09:58:37.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.178 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T09:58:37.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.187 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-09-10T09:58:37.249 INFO:teuthology.orchestra.run.smithi110.stderr:Cannot find device "ceph-brx" 2024-09-10T09:58:37.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:58:37.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.252 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T09:58:37.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.304 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-09-10T09:58:37.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:58:37.373 INFO:teuthology.orchestra.run.smithi110.stderr:Cannot find device "ceph-brx" 2024-09-10T09:58:37.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.373 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T09:58:37.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.384 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2024-09-10T09:58:37.449 INFO:teuthology.orchestra.run.smithi145.stderr:Cannot find device "ceph-brx" 2024-09-10T09:58:37.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:58:37.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.452 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T09:58:37.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.506 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2024-09-10T09:58:37.572 INFO:teuthology.orchestra.run.smithi145.stderr:Cannot find device "ceph-brx" 2024-09-10T09:58:37.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:58:37.575 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-10T09:58:37.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T09:58:37.575 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T09:58:37.575 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T09:58:37.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f9865000640 1 Processor -- start 2024-09-10T09:58:37.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f9865000640 1 -- start start 2024-09-10T09:58:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f9865000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9860155970 0x7f98600cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f9865000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f986006f1e0 0x7f98600ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f9865000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9860155d50 con 0x7f986015c560 2024-09-10T09:58:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f9865000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f98600b4570 con 0x7f986006f1e0 2024-09-10T09:58:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f9865000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f98600b34d0 con 0x7f9860155970 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f985e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f986006f1e0 0x7f98600ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f985de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9860155970 0x7f98600cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f985de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9860155970 0x7f98600cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:46988/0 (socket says 172.21.15.110:46988) 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f985de00640 1 -- 172.21.15.110:0/2446183371 learned_addr learned my addr 172.21.15.110:0/2446183371 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.708+0000 7f985d400640 1 --1- 172.21.15.110:0/2446183371 >> v1:172.21.15.145:6789/0 conn(0x7f986015c560 0x7f986006ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:47022/0 (socket says 172.21.15.110:47022) 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985de00640 1 -- 172.21.15.110:0/2446183371 >> v2:172.21.15.145:3300/0 conn(0x7f986006f1e0 msgr2=0x7f98600ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985de00640 1 --2- 172.21.15.110:0/2446183371 >> v2:172.21.15.145:3300/0 conn(0x7f986006f1e0 0x7f98600ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985de00640 1 -- 172.21.15.110:0/2446183371 >> v1:172.21.15.145:6789/0 conn(0x7f986015c560 legacy=0x7f986006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985de00640 1 -- 172.21.15.110:0/2446183371 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98600b37b0 con 0x7f9860155970 2024-09-10T09:58:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985e800640 1 --2- 172.21.15.110:0/2446183371 >> v2:172.21.15.145:3300/0 conn(0x7f986006f1e0 0x7f98600ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:37.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985de00640 1 --2- 172.21.15.110:0/2446183371 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9860155970 0x7f98600cc8c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9850009e50 tx=0x7f985001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f960f45eb32a4e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985ca00640 1 -- 172.21.15.110:0/2446183371 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500a5830 con 0x7f9860155970 2024-09-10T09:58:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985ca00640 1 -- 172.21.15.110:0/2446183371 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98500a59d0 con 0x7f9860155970 2024-09-10T09:58:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.712+0000 7f985ca00640 1 -- 172.21.15.110:0/2446183371 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500a5da0 con 0x7f9860155970 2024-09-10T09:58:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- 172.21.15.110:0/2446183371 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9860155970 msgr2=0x7f98600cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 --2- 172.21.15.110:0/2446183371 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9860155970 0x7f98600cc8c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9850009e50 tx=0x7f985001c970 comp rx=0 tx=0).stop 2024-09-10T09:58:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- 172.21.15.110:0/2446183371 shutdown_connections 2024-09-10T09:58:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 --2- 172.21.15.110:0/2446183371 >> v2:172.21.15.145:3300/0 conn(0x7f986006f1e0 0x7f98600ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 --2- 172.21.15.110:0/2446183371 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9860155970 0x7f98600cc8c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- 172.21.15.110:0/2446183371 >> 172.21.15.110:0/2446183371 conn(0x7f986006bc50 msgr2=0x7f986006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- 172.21.15.110:0/2446183371 shutdown_connections 2024-09-10T09:58:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- 172.21.15.110:0/2446183371 wait complete. 2024-09-10T09:58:37.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 Processor -- start 2024-09-10T09:58:37.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- start start 2024-09-10T09:58:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f986006f1e0 0x7f986013f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9860155970 0x7f986015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f986015c560 0x7f9860159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98600cf170 con 0x7f986015c560 2024-09-10T09:58:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98600cce00 con 0x7f9860155970 2024-09-10T09:58:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f9865000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f98600cf2f0 con 0x7f986006f1e0 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f986015c560 0x7f9860159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f986015c560 0x7f9860159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53542/0 (socket says 172.21.15.110:53542) 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 -- 172.21.15.110:0/1415096304 learned_addr learned my addr 172.21.15.110:0/1415096304 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985de00640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f986006f1e0 0x7f986013f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985d400640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9860155970 0x7f986015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 -- 172.21.15.110:0/1415096304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f986006f1e0 msgr2=0x7f986013f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f986006f1e0 0x7f986013f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 -- 172.21.15.110:0/1415096304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9860155970 msgr2=0x7f986015cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9860155970 0x7f986015cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 -- 172.21.15.110:0/1415096304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98600d00a0 con 0x7f986015c560 2024-09-10T09:58:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.716+0000 7f985e800640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f986015c560 0x7f9860159b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f985800e950 tx=0x7f985800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9856000640 1 -- 172.21.15.110:0/1415096304 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858017070 con 0x7f986015c560 2024-09-10T09:58:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9856000640 1 -- 172.21.15.110:0/1415096304 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985800ca00 con 0x7f986015c560 2024-09-10T09:58:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986015eb60 con 0x7f986015c560 2024-09-10T09:58:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9856000640 1 -- 172.21.15.110:0/1415096304 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985800cdd0 con 0x7f986015c560 2024-09-10T09:58:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986015f0e0 con 0x7f986015c560 2024-09-10T09:58:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98600ce420 con 0x7f986015c560 2024-09-10T09:58:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9856000640 1 -- 172.21.15.110:0/1415096304 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9858013470 con 0x7f986015c560 2024-09-10T09:58:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9856000640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f98400678c0 0x7f9840069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f9856000640 1 -- 172.21.15.110:0/1415096304 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f98580858a0 con 0x7f986015c560 2024-09-10T09:58:37.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f985de00640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f98400678c0 0x7f9840069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:37.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.720+0000 7f985de00640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f98400678c0 0x7f9840069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98500ae000 tx=0x7f98500a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:37.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.724+0000 7f9856000640 1 -- 172.21.15.110:0/1415096304 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9858056350 con 0x7f986015c560 2024-09-10T09:58:37.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.920+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9860072dd0 con 0x7f986015c560 2024-09-10T09:58:37.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9856000640 1 -- 172.21.15.110:0/1415096304 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f9858055c50 con 0x7f986015c560 2024-09-10T09:58:37.925 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T09:58:37.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f98400678c0 msgr2=0x7f9840069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f98400678c0 0x7f9840069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98500ae000 tx=0x7f98500a7070 comp rx=0 tx=0).stop 2024-09-10T09:58:37.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f986015c560 msgr2=0x7f9860159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f986015c560 0x7f9860159b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f985800e950 tx=0x7f985800ee10 comp rx=0 tx=0).stop 2024-09-10T09:58:37.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 shutdown_connections 2024-09-10T09:58:37.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f98400678c0 0x7f9840069d80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f986015c560 0x7f9860159b90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9860155970 0x7f986015cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 --2- 172.21.15.110:0/1415096304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f986006f1e0 0x7f986013f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 >> 172.21.15.110:0/1415096304 conn(0x7f986006bc50 msgr2=0x7f986015b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 shutdown_connections 2024-09-10T09:58:37.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:37.924+0000 7f9865000640 1 -- 172.21.15.110:0/1415096304 wait complete. 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:58:37.946 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:58:37.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:37.947 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T09:58:37.955 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:58:37.955 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:58:37.955 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:58:37.955 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:37.955 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T09:58:37.956 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42457sec preferred_lft 42457sec 2024-09-10T09:58:37.957 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T09:58:37.957 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:37.957 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T09:58:37.957 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:37.957 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T09:58:37.957 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:37.958 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T09:58:37.958 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:37.958 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:37.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add name ceph-brx type bridge 2024-09-10T09:58:37.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip addr flush dev ceph-brx 2024-09-10T09:58:37.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set ceph-brx up 2024-09-10T09:58:37.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T09:58:37.959 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:38.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:38.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:38.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:38.112 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:58:38.130 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T09:58:38.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:38.132 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T09:58:38.205 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:38.206 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:58:38.207 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:38.207 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:38.207 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:58:38.207 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:58:38.207 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:58:38.207 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:38.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:38.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:38.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:38.341 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T09:58:38.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:38.393 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list-id 2024-09-10T09:58:38.444 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:38.444 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:38.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T09:58:38.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T09:58:38.444 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:38.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:38.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:38.570 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T09:58:38.570 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:38.570 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:38.570 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T09:58:38.570 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T09:58:38.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T09:58:38.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T09:58:38.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T09:58:38.571 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:38.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:38.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:38.744 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:38.744 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:38.745 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T09:58:38.745 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T09:58:38.745 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:38.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:38.873 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:58:38.873 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T09:58:38.924 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:58:38.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:58:38.924 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:58:38.979 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T09:58:38.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:38.980 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:39.071 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:39.071 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:58:39.742 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:39.742 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:58:39.742 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T09:58:39.742 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T09:58:39.742 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T09:58:39.746 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:58:39.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:39.747 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:58:39.767 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:39.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:39.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:39.892 INFO:teuthology.orchestra.run.smithi110.stdout:client4590 2024-09-10T09:58:39.893 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:58:39.893 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:39.901 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:39.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:40.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:40.009 INFO:teuthology.orchestra.run.smithi110.stdout:client4590 2024-09-10T09:58:40.010 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:40.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:40.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:40.076 INFO:teuthology.orchestra.run.smithi110.stdout:client4590 2024-09-10T09:58:40.077 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/af5c1104-e6d8-4fa7-9635-0686bce1580a.client4590/status') 2024-09-10T09:58:40.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi110.stdout:instance: client.4590 (3)192.168.0.1:0/1848453376 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi110.stdout:blocklisted: false 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi110.stderr:0+1 records in 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi110.stderr:0+1 records out 2024-09-10T09:58:40.190 INFO:teuthology.orchestra.run.smithi110.stderr:69 bytes copied, 4.8601e-05 s, 1.4 MB/s 2024-09-10T09:58:40.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:40.198 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-10T09:58:40.198 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T09:58:40.198 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T09:58:40.198 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T09:58:40.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5683000640 1 Processor -- start 2024-09-10T09:58:40.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5683000640 1 -- start start 2024-09-10T09:58:40.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5683000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f567c0afea0 0x7f567c072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5683000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f567c0b0300 con 0x7f567c072d80 2024-09-10T09:58:40.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5683000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f567c160c90 con 0x7f567c153210 2024-09-10T09:58:40.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5683000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f567c15d9d0 con 0x7f567c0afea0 2024-09-10T09:58:40.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5682600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f567c072d80 0x7f567c0cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51012/0 (socket says 172.21.15.110:51012) 2024-09-10T09:58:40.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5682600640 1 -- 172.21.15.110:0/3537280417 learned_addr learned my addr 172.21.15.110:0/3537280417 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:40.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5681200640 1 --2- 172.21.15.110:0/3537280417 >> v2:172.21.15.145:3300/0 conn(0x7f567c0afea0 0x7f567c072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925518801 0 0) 0x7f567c0b0300 con 0x7f567c072d80 2024-09-10T09:58:40.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5664003530 con 0x7f567c072d80 2024-09-10T09:58:40.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1178774441 0 0) 0x7f5664003530 con 0x7f567c072d80 2024-09-10T09:58:40.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 >> v2:172.21.15.145:3300/0 conn(0x7f567c0afea0 msgr2=0x7f567c072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 --2- 172.21.15.110:0/3537280417 >> v2:172.21.15.145:3300/0 conn(0x7f567c0afea0 0x7f567c072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 >> v1:172.21.15.145:6789/0 conn(0x7f567c153210 legacy=0x7f567c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c0b05a0 con 0x7f567c072d80 2024-09-10T09:58:40.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f56740016a0 con 0x7f567c072d80 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 >> v1:172.21.15.110:6789/0 conn(0x7f567c072d80 legacy=0x7f567c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56640039c0 0x7f5664003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f5664004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56640051d0 0x7f566400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f567c15d9d0 con 0x7f5664004810 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f566400c070 con 0x7f56640039c0 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f566400c270 con 0x7f56640051d0 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5681200640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f5664004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5681c00640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56640039c0 0x7f5664003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.332+0000 7f5682600640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56640051d0 0x7f566400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5681200640 1 -- 172.21.15.110:0/3537280417 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56640051d0 msgr2=0x7f566400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5681200640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56640051d0 0x7f566400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5681200640 1 -- 172.21.15.110:0/3537280417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56640039c0 msgr2=0x7f5664003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:40.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5681200640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56640039c0 0x7f5664003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5681200640 1 -- 172.21.15.110:0/3537280417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5664003530 con 0x7f5664004810 2024-09-10T09:58:40.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5682600640 1 -- 172.21.15.110:0/3537280417 reap_dead start 2024-09-10T09:58:40.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5681200640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f5664004c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5668009ae0 tx=0x7f566801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6d872717d51871 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56680b0070 con 0x7f5664004810 2024-09-10T09:58:40.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56680ab400 con 0x7f5664004810 2024-09-10T09:58:40.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f567be00640 1 -- 172.21.15.110:0/3537280417 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56680096b0 con 0x7f5664004810 2024-09-10T09:58:40.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5683000640 1 -- 172.21.15.110:0/3537280417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 msgr2=0x7f5664004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5683000640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f5664004c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5668009ae0 tx=0x7f566801c750 comp rx=0 tx=0).stop 2024-09-10T09:58:40.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5683000640 1 -- 172.21.15.110:0/3537280417 shutdown_connections 2024-09-10T09:58:40.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5683000640 1 --2- 172.21.15.110:0/3537280417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f5664004c90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.336+0000 7f5683000640 1 -- 172.21.15.110:0/3537280417 >> 172.21.15.110:0/3537280417 conn(0x7f567c06be20 msgr2=0x7f567c06c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:40.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 -- 172.21.15.110:0/3537280417 shutdown_connections 2024-09-10T09:58:40.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 -- 172.21.15.110:0/3537280417 wait complete. 2024-09-10T09:58:40.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 Processor -- start 2024-09-10T09:58:40.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 -- start start 2024-09-10T09:58:40.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f567c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f567c1507b0 0x7f567c14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f567c13cd90 0x7f567c13d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f567c160490 con 0x7f5664004810 2024-09-10T09:58:40.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f567c1601b0 con 0x7f567c13cd90 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f567c160770 con 0x7f567c1507b0 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f567c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f567c153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59448/0 (socket says 172.21.15.110:59448) 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 -- 172.21.15.110:0/1343064774 learned_addr learned my addr 172.21.15.110:0/1343064774 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681200640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f567c1507b0 0x7f567c14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5682600640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f567c13cd90 0x7f567c13d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 -- 172.21.15.110:0/1343064774 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f567c1507b0 msgr2=0x7f567c14dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f567c1507b0 0x7f567c14dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 -- 172.21.15.110:0/1343064774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f567c13cd90 msgr2=0x7f567c13d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f567c13cd90 0x7f567c13d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 -- 172.21.15.110:0/1343064774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c13d760 con 0x7f5664004810 2024-09-10T09:58:40.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5681c00640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f567c153800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f567000f980 tx=0x7f567000fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5679600640 1 -- 172.21.15.110:0/1343064774 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670003b90 con 0x7f5664004810 2024-09-10T09:58:40.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5679600640 1 -- 172.21.15.110:0/1343064774 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670018070 con 0x7f5664004810 2024-09-10T09:58:40.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5679600640 1 -- 172.21.15.110:0/1343064774 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567000d7f0 con 0x7f5664004810 2024-09-10T09:58:40.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c153d40 con 0x7f5664004810 2024-09-10T09:58:40.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.340+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c14fcd0 con 0x7f5664004810 2024-09-10T09:58:40.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.344+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567c1535f0 con 0x7f5664004810 2024-09-10T09:58:40.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.344+0000 7f5679600640 1 -- 172.21.15.110:0/1343064774 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f567000d990 con 0x7f5664004810 2024-09-10T09:58:40.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.344+0000 7f5679600640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f565c0678f0 0x7f565c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:40.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.344+0000 7f5679600640 1 -- 172.21.15.110:0/1343064774 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5670085da0 con 0x7f5664004810 2024-09-10T09:58:40.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.344+0000 7f5681200640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f565c0678f0 0x7f565c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:40.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.344+0000 7f5681200640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f565c0678f0 0x7f565c069db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f56680ad000 tx=0x7f56680046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:40.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.348+0000 7f5679600640 1 -- 172.21.15.110:0/1343064774 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f567c13d760 con 0x7f5664004810 2024-09-10T09:58:40.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f567c14faa0 con 0x7f5664004810 2024-09-10T09:58:40.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5679600640 1 -- 172.21.15.110:0/1343064774 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f56700561f0 con 0x7f5664004810 2024-09-10T09:58:40.546 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f565c0678f0 msgr2=0x7f565c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f565c0678f0 0x7f565c069db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f56680ad000 tx=0x7f56680046d0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 msgr2=0x7f567c153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f567c153800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f567000f980 tx=0x7f567000fe40 comp rx=0 tx=0).stop 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 shutdown_connections 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f565c0678f0 0x7f565c069db0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f567c13cd90 0x7f567c13d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f567c1507b0 0x7f567c14dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 --2- 172.21.15.110:0/1343064774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5664004810 0x7f567c153800 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:40.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 >> 172.21.15.110:0/1343064774 conn(0x7f567c06be20 msgr2=0x7f567c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:40.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 shutdown_connections 2024-09-10T09:58:40.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40.544+0000 7f5683000640 1 -- 172.21.15.110:0/1343064774 wait complete. 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:58:40.567 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T09:58:40.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:40.568 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T09:58:40.576 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:58:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:58:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:40.577 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42454sec preferred_lft 42454sec 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T09:58:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:40.579 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:58:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T09:58:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-09-10T09:58:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:40.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:40.580 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:58:40.637 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T09:58:40.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:40.639 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T09:58:40.691 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T09:58:40.691 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:40.692 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:58:40.692 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:40.693 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:40.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:58:40.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:58:40.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:58:40.693 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:40.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:40.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:40.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:40.824 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T09:58:40.834 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T09:58:40.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:40.834 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list-id 2024-09-10T09:58:40.888 INFO:teuthology.orchestra.run.smithi110.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T09:58:40.888 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:40.888 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:40.888 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T09:58:40.889 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T09:58:40.889 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:40.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:41.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:41.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:41.018 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T09:58:41.038 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:58:41.038 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:58:41.038 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:58:41.038 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:41.038 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T09:58:41.038 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:41.038 INFO:teuthology.orchestra.run.smithi110.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:58:41.039 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 46:73:63:34:cf:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T09:58:41.039 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T09:58:41.039 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:41.039 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::4473:63ff:fe34:cf52/64 scope link 2024-09-10T09:58:41.039 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:41.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:41.040 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 46:73:63:34:cf:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::4473:63ff:fe34:cf52/64 scope link 2024-09-10T09:58:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:41.110 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T09:58:41.110 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:41.110 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:41.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T09:58:41.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T09:58:41.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T09:58:41.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T09:58:41.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T09:58:41.110 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:41.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:41.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:41.287 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:41.287 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:58:41.287 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.1 up 2024-09-10T09:58:41.287 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T09:58:41.287 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:58:41.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:41.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:41.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:58:41.370 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T09:58:41.420 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T09:58:41.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:58:41.420 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T09:58:41.471 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-10T09:58:41.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:41.471 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:41.559 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:41.559 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:58:41.574 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:41.574 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:58:41.574 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T09:58:41.574 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 2@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T09:58:41.574 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T09:58:41.578 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:58:41.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:41.578 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T09:58:41.642 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:41.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:41.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:41.744 INFO:teuthology.orchestra.run.smithi110.stdout:client4513 2024-09-10T09:58:41.745 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:58:41.745 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:41.796 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:41.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:41.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:41.899 INFO:teuthology.orchestra.run.smithi110.stdout:client4513 2024-09-10T09:58:41.899 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:41.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:41.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:41.995 INFO:teuthology.orchestra.run.smithi110.stdout:client4513 2024-09-10T09:58:41.996 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/af5c1104-e6d8-4fa7-9635-0686bce1580a.client4513/status') 2024-09-10T09:58:42.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:42.093 INFO:teuthology.orchestra.run.smithi110.stdout:instance: client.4513 (3)172.21.15.110:0/4030790561 2024-09-10T09:58:42.094 INFO:teuthology.orchestra.run.smithi110.stdout:blocklisted: false 2024-09-10T09:58:42.094 INFO:teuthology.orchestra.run.smithi110.stderr:0+1 records in 2024-09-10T09:58:42.094 INFO:teuthology.orchestra.run.smithi110.stderr:0+1 records out 2024-09-10T09:58:42.094 INFO:teuthology.orchestra.run.smithi110.stderr:71 bytes copied, 6.0657e-05 s, 1.2 MB/s 2024-09-10T09:58:42.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:42.102 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-10T09:58:42.102 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T09:58:42.102 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T09:58:42.102 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T09:58:42.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b26000640 1 Processor -- start 2024-09-10T09:58:42.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b26000640 1 -- start start 2024-09-10T09:58:42.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b26000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9b200cfe30 0x7f9b20071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:42.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b26000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2006d2d0 con 0x7f9b200cd510 2024-09-10T09:58:42.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b26000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2006d5b0 con 0x7f9b2006e7a0 2024-09-10T09:58:42.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b26000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9b2015d9a0 con 0x7f9b200cfe30 2024-09-10T09:58:42.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b1ea00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9b200cd510 0x7f9b2006e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51026/0 (socket says 172.21.15.110:51026) 2024-09-10T09:58:42.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b1f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9b200cfe30 0x7f9b20071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:42.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.280+0000 7f9b1ea00640 1 -- 172.21.15.110:0/3642859092 learned_addr learned my addr 172.21.15.110:0/3642859092 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:42.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.284+0000 7f9b1f400640 1 -- 172.21.15.110:0/3642859092 >> v1:172.21.15.145:6789/0 conn(0x7f9b2006e7a0 legacy=0x7f9b2015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:42.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.284+0000 7f9b1f400640 1 -- 172.21.15.110:0/3642859092 >> v1:172.21.15.110:6789/0 conn(0x7f9b200cd510 legacy=0x7f9b2006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:42.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.284+0000 7f9b1f400640 1 -- 172.21.15.110:0/3642859092 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2006d890 con 0x7f9b200cfe30 2024-09-10T09:58:42.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.284+0000 7f9b1f400640 1 --2- 172.21.15.110:0/3642859092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b200cfe30 0x7f9b20071a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009d90 tx=0x7f9b1001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4580a745ecca7a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:42.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.284+0000 7f9b1e000640 1 -- 172.21.15.110:0/3642859092 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100b2070 con 0x7f9b200cfe30 2024-09-10T09:58:42.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.284+0000 7f9b1e000640 1 -- 172.21.15.110:0/3642859092 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b100a5870 con 0x7f9b200cfe30 2024-09-10T09:58:42.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.284+0000 7f9b1e000640 1 -- 172.21.15.110:0/3642859092 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100a5c40 con 0x7f9b200cfe30 2024-09-10T09:58:42.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- 172.21.15.110:0/3642859092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b200cfe30 msgr2=0x7f9b20071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:42.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 --2- 172.21.15.110:0/3642859092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b200cfe30 0x7f9b20071a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009d90 tx=0x7f9b1001c8a0 comp rx=0 tx=0).stop 2024-09-10T09:58:42.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- 172.21.15.110:0/3642859092 shutdown_connections 2024-09-10T09:58:42.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 --2- 172.21.15.110:0/3642859092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b200cfe30 0x7f9b20071a40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:42.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- 172.21.15.110:0/3642859092 >> 172.21.15.110:0/3642859092 conn(0x7f9b2006bc50 msgr2=0x7f9b2006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:42.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- 172.21.15.110:0/3642859092 shutdown_connections 2024-09-10T09:58:42.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- 172.21.15.110:0/3642859092 wait complete. 2024-09-10T09:58:42.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 Processor -- start 2024-09-10T09:58:42.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- start start 2024-09-10T09:58:42.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b2006e7a0 0x7f9b2014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:42.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b200cd510 0x7f9b200ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:42.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b200cfe30 0x7f9b200aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2015d740 con 0x7f9b2006e7a0 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9b20072190 con 0x7f9b200cd510 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b26000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2016abb0 con 0x7f9b200cfe30 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b2006e7a0 0x7f9b2014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b2006e7a0 0x7f9b2014fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59454/0 (socket says 172.21.15.110:59454) 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 -- 172.21.15.110:0/1358345551 learned_addr learned my addr 172.21.15.110:0/1358345551 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1ea00640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b200cd510 0x7f9b200ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 -- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b200cfe30 msgr2=0x7f9b200aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1fe00640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b200cfe30 0x7f9b200aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b200cfe30 0x7f9b200aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 -- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b200cd510 msgr2=0x7f9b200ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b200cd510 0x7f9b200ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:42.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 -- 172.21.15.110:0/1358345551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2015a750 con 0x7f9b2006e7a0 2024-09-10T09:58:42.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1ea00640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b200cd510 0x7f9b200ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:42.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1fe00640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b200cfe30 0x7f9b200aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:42.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b1f400640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b2006e7a0 0x7f9b2014fe60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009d90 tx=0x7f9b10019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:42.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.288+0000 7f9b17400640 1 -- 172.21.15.110:0/1358345551 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100bd020 con 0x7f9b2006e7a0 2024-09-10T09:58:42.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b17400640 1 -- 172.21.15.110:0/1358345551 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b100ad870 con 0x7f9b2006e7a0 2024-09-10T09:58:42.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b17400640 1 -- 172.21.15.110:0/1358345551 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100b2040 con 0x7f9b2006e7a0 2024-09-10T09:58:42.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b201503a0 con 0x7f9b2006e7a0 2024-09-10T09:58:42.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b200af740 con 0x7f9b2006e7a0 2024-09-10T09:58:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b200b2790 con 0x7f9b2006e7a0 2024-09-10T09:58:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b17400640 1 -- 172.21.15.110:0/1358345551 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9b100c0020 con 0x7f9b2006e7a0 2024-09-10T09:58:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b17400640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9b000678f0 0x7f9b00069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b17400640 1 -- 172.21.15.110:0/1358345551 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9b10120bd0 con 0x7f9b2006e7a0 2024-09-10T09:58:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b1ea00640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9b000678f0 0x7f9b00069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:42.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.292+0000 7f9b1ea00640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9b000678f0 0x7f9b00069db0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c007fd0 tx=0x7f9b0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.296+0000 7f9b17400640 1 -- 172.21.15.110:0/1358345551 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9b100c7070 con 0x7f9b2006e7a0 2024-09-10T09:58:42.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.488+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9b200719c0 con 0x7f9b2006e7a0 2024-09-10T09:58:42.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.488+0000 7f9b17400640 1 -- 172.21.15.110:0/1358345551 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f9b10016520 con 0x7f9b2006e7a0 2024-09-10T09:58:42.492 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T09:58:42.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9b000678f0 msgr2=0x7f9b00069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:42.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9b000678f0 0x7f9b00069db0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c007fd0 tx=0x7f9b0c009040 comp rx=0 tx=0).stop 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b2006e7a0 msgr2=0x7f9b2014fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b2006e7a0 0x7f9b2014fe60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009d90 tx=0x7f9b10019640 comp rx=0 tx=0).stop 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 shutdown_connections 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9b000678f0 0x7f9b00069db0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b200cfe30 0x7f9b200aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b200cd510 0x7f9b200ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 --2- 172.21.15.110:0/1358345551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b2006e7a0 0x7f9b2014fe60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 >> 172.21.15.110:0/1358345551 conn(0x7f9b2006bc50 msgr2=0x7f9b2015aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 shutdown_connections 2024-09-10T09:58:42.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:42.492+0000 7f9b26000640 1 -- 172.21.15.110:0/1358345551 wait complete. 2024-09-10T09:58:42.513 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T09:58:42.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:58:42.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T09:58:42.513 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T09:58:42.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T09:58:42.513 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T09:58:42.514 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T09:58:42.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:58:42.514 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T09:58:42.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:42.514 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T09:58:42.524 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:58:42.524 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:58:42.524 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:58:42.524 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:42.524 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T09:58:42.524 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:42.524 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42449sec preferred_lft 42449sec 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T09:58:42.525 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:42.526 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T09:58:42.526 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:42.526 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T09:58:42.526 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:42.527 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T09:58:42.527 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:42.527 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:42.527 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add name ceph-brx type bridge 2024-09-10T09:58:42.527 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip addr flush dev ceph-brx 2024-09-10T09:58:42.527 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set ceph-brx up 2024-09-10T09:58:42.527 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T09:58:42.527 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:42.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:42.684 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:42.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:42.691 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:58:42.710 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T09:58:42.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:42.712 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T09:58:42.810 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T09:58:42.810 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:58:42.810 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T09:58:42.810 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:58:42.811 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T09:58:42.811 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T09:58:42.811 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:58:42.811 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:42.811 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:42.811 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:42.811 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:58:42.811 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:42.811 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:42.812 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:58:42.812 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:58:42.812 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:58:42.812 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:42.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:42.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:42.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:42.906 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T09:58:42.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:42.962 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list-id 2024-09-10T09:58:43.013 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:43.013 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:43.013 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T09:58:43.013 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T09:58:43.014 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:43.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:43.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:43.139 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T09:58:43.139 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:43.139 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:43.139 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T09:58:43.140 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T09:58:43.140 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T09:58:43.140 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T09:58:43.140 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T09:58:43.140 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:43.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:43.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:43.301 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:43.301 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:43.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T09:58:43.301 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T09:58:43.301 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:43.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:43.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:43.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:58:43.386 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T09:58:43.437 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T09:58:43.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:58:43.438 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T09:58:43.488 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T09:58:43.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:43.488 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:43.578 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:43.578 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:58:44.246 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:44.246 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:58:44.246 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T09:58:44.246 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T09:58:44.246 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T09:58:44.250 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:58:44.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:44.250 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T09:58:44.270 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:44.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:44.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:44.380 INFO:teuthology.orchestra.run.smithi145.stdout:client4516 2024-09-10T09:58:44.380 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:58:44.380 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:44.388 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:44.474 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:44.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:44.483 INFO:teuthology.orchestra.run.smithi145.stdout:client4516 2024-09-10T09:58:44.484 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:44.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:44.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:44.540 INFO:teuthology.orchestra.run.smithi145.stdout:client4516 2024-09-10T09:58:44.540 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/af5c1104-e6d8-4fa7-9635-0686bce1580a.client4516/status') 2024-09-10T09:58:44.627 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:44.639 INFO:teuthology.orchestra.run.smithi145.stdout:instance: client.4516 (3)192.168.0.1:0/2857898996 2024-09-10T09:58:44.639 INFO:teuthology.orchestra.run.smithi145.stdout:blocklisted: false 2024-09-10T09:58:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:0+1 records in 2024-09-10T09:58:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:0+1 records out 2024-09-10T09:58:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:69 bytes copied, 6.5803e-05 s, 1.0 MB/s 2024-09-10T09:58:44.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:44.647 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-10T09:58:44.647 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T09:58:44.647 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T09:58:44.648 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T09:58:44.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.792+0000 7fcc52c00640 1 Processor -- start 2024-09-10T09:58:44.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.792+0000 7fcc52c00640 1 -- start start 2024-09-10T09:58:44.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc52c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cfe30 0x7fcc4c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:44.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc52c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc4c0cd510 0x7fcc4c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:44.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc52c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc4c06d2d0 con 0x7fcc4c06e090 2024-09-10T09:58:44.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc52c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcc4c0cdeb0 con 0x7fcc4c0cd510 2024-09-10T09:58:44.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc52c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4c15d9a0 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc4b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc4c0cd510 0x7fcc4c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:44.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc50c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcc4c06e090 0x7fcc4c06e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51032/0 (socket says 172.21.15.110:51032) 2024-09-10T09:58:44.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc50c00640 1 -- 172.21.15.110:0/3740202088 learned_addr learned my addr 172.21.15.110:0/3740202088 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:44.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc4be00640 1 --2- 172.21.15.110:0/3740202088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cfe30 0x7fcc4c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:44.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc4b400640 1 -- 172.21.15.110:0/3740202088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cfe30 msgr2=0x7fcc4c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:44.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc4b400640 1 --2- 172.21.15.110:0/3740202088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cfe30 0x7fcc4c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:44.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc4b400640 1 -- 172.21.15.110:0/3740202088 >> v1:172.21.15.110:6789/0 conn(0x7fcc4c06e090 legacy=0x7fcc4c06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:44.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.796+0000 7fcc4b400640 1 -- 172.21.15.110:0/3740202088 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c06d5b0 con 0x7fcc4c0cd510 2024-09-10T09:58:44.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc4b400640 1 --2- 172.21.15.110:0/3740202088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cd510 0x7fcc4c0cd970 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc380097c0 tx=0x7fcc3801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5cb761daa799d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:44.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc4aa00640 1 -- 172.21.15.110:0/3740202088 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc380b0070 con 0x7fcc4c0cd510 2024-09-10T09:58:44.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc4aa00640 1 -- 172.21.15.110:0/3740202088 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc380ab600 con 0x7fcc4c0cd510 2024-09-10T09:58:44.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc4aa00640 1 -- 172.21.15.110:0/3740202088 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc380ab9d0 con 0x7fcc4c0cd510 2024-09-10T09:58:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- 172.21.15.110:0/3740202088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cd510 msgr2=0x7fcc4c0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 --2- 172.21.15.110:0/3740202088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cd510 0x7fcc4c0cd970 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc380097c0 tx=0x7fcc3801c410 comp rx=0 tx=0).stop 2024-09-10T09:58:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- 172.21.15.110:0/3740202088 shutdown_connections 2024-09-10T09:58:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 --2- 172.21.15.110:0/3740202088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cd510 0x7fcc4c0cd970 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 --2- 172.21.15.110:0/3740202088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cfe30 0x7fcc4c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- 172.21.15.110:0/3740202088 >> 172.21.15.110:0/3740202088 conn(0x7fcc4c06bc50 msgr2=0x7fcc4c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- 172.21.15.110:0/3740202088 shutdown_connections 2024-09-10T09:58:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- 172.21.15.110:0/3740202088 wait complete. 2024-09-10T09:58:44.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 Processor -- start 2024-09-10T09:58:44.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- start start 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc4c06e090 0x7fcc4c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cd510 0x7fcc4c157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cfe30 0x7fcc4c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c072190 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c15d640 con 0x7fcc4c06e090 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc52c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4c169180 con 0x7fcc4c0cd510 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.800+0000 7fcc50c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cfe30 0x7fcc4c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc50c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cfe30 0x7fcc4c157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59466/0 (socket says 172.21.15.110:59466) 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc50c00640 1 -- 172.21.15.110:0/1477663885 learned_addr learned my addr 172.21.15.110:0/1477663885 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc4be00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc4c06e090 0x7fcc4c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:44.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc4b400640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cd510 0x7fcc4c157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:44.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc50c00640 1 -- 172.21.15.110:0/1477663885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cd510 msgr2=0x7fcc4c157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:44.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc50c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cd510 0x7fcc4c157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:44.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc50c00640 1 -- 172.21.15.110:0/1477663885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc4c06e090 msgr2=0x7fcc4c1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:44.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc50c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc4c06e090 0x7fcc4c1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:44.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc50c00640 1 -- 172.21.15.110:0/1477663885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c153b00 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc50c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cfe30 0x7fcc4c157690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400cab0 tx=0x7fcc4400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:44.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc43e00640 1 -- 172.21.15.110:0/1477663885 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44017070 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc43e00640 1 -- 172.21.15.110:0/1477663885 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4400ac30 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c153de0 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc43e00640 1 -- 172.21.15.110:0/1477663885 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44012400 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c154820 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c0cd970 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc43e00640 1 -- 172.21.15.110:0/1477663885 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcc4401a070 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc43e00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc2c0678c0 0x7fcc2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.804+0000 7fcc43e00640 1 -- 172.21.15.110:0/1477663885 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcc440858d0 con 0x7fcc4c0cfe30 2024-09-10T09:58:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.808+0000 7fcc4be00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc2c0678c0 0x7fcc2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:44.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.808+0000 7fcc4be00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc2c0678c0 0x7fcc2c069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c156d70 tx=0x7fcc3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:44.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:44.812+0000 7fcc43e00640 1 -- 172.21.15.110:0/1477663885 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcc44055db0 con 0x7fcc4c0cfe30 2024-09-10T09:58:45.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.000+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcc4c0af740 con 0x7fcc4c0cfe30 2024-09-10T09:58:45.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc43e00640 1 -- 172.21.15.110:0/1477663885 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fcc4405a0b0 con 0x7fcc4c0cfe30 2024-09-10T09:58:45.005 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T09:58:45.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc2c0678c0 msgr2=0x7fcc2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:45.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc2c0678c0 0x7fcc2c069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c156d70 tx=0x7fcc3c008040 comp rx=0 tx=0).stop 2024-09-10T09:58:45.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cfe30 msgr2=0x7fcc4c157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:45.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cfe30 0x7fcc4c157690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400cab0 tx=0x7fcc4400cf70 comp rx=0 tx=0).stop 2024-09-10T09:58:45.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 shutdown_connections 2024-09-10T09:58:45.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc2c0678c0 0x7fcc2c069d80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:45.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc4c0cfe30 0x7fcc4c157690 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:45.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc4c0cd510 0x7fcc4c157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:45.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 --2- 172.21.15.110:0/1477663885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc4c06e090 0x7fcc4c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:45.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 >> 172.21.15.110:0/1477663885 conn(0x7fcc4c06bc50 msgr2=0x7fcc4c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:45.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 shutdown_connections 2024-09-10T09:58:45.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:45.004+0000 7fcc52c00640 1 -- 172.21.15.110:0/1477663885 wait complete. 2024-09-10T09:58:45.026 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T09:58:45.026 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:58:45.026 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T09:58:45.027 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T09:58:45.027 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T09:58:45.027 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T09:58:45.027 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T09:58:45.027 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:58:45.027 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T09:58:45.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.027 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:45.037 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42446sec preferred_lft 42446sec 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:58:45.038 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.039 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:58:45.098 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T09:58:45.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.101 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T09:58:45.152 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:58:45.153 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:58:45.153 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:45.154 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:45.154 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:58:45.154 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:58:45.154 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:58:45.154 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:45.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:45.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:45.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.290 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T09:58:45.299 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T09:58:45.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.300 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list-id 2024-09-10T09:58:45.352 INFO:teuthology.orchestra.run.smithi145.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T09:58:45.353 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:45.353 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:45.354 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T09:58:45.354 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T09:58:45.354 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:45.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:45.473 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:45.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T09:58:45.500 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:58:45.500 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:58:45.500 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:58:45.500 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.500 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T09:58:45.500 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.501 INFO:teuthology.orchestra.run.smithi145.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:58:45.501 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether fe:be:a3:5b:16:b0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T09:58:45.501 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T09:58:45.501 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.501 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::fcbe:a3ff:fe5b:16b0/64 scope link 2024-09-10T09:58:45.501 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.502 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T09:58:45.564 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether fe:be:a3:5b:16:b0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.565 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::fcbe:a3ff:fe5b:16b0/64 scope link 2024-09-10T09:58:45.566 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:58:45.567 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T09:58:45.567 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:45.567 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:45.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T09:58:45.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T09:58:45.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T09:58:45.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T09:58:45.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T09:58:45.567 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:45.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:45.745 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:58:45.745 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:58:45.745 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.1 up 2024-09-10T09:58:45.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T09:58:45.746 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:58:45.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:45.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:45.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:58:45.831 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T09:58:45.840 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T09:58:45.841 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:58:45.841 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T09:58:45.888 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-10T09:58:45.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.889 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:45.976 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:45.976 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:58:45.989 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:58:45.990 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:58:45.990 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T09:58:45.990 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 3@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T09:58:45.990 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T09:58:45.993 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:58:45.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:45.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T09:58:46.012 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:46.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:46.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:46.110 INFO:teuthology.orchestra.run.smithi145.stdout:client4442 2024-09-10T09:58:46.110 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:58:46.110 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:46.165 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:46.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:46.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:46.268 INFO:teuthology.orchestra.run.smithi145.stdout:client4442 2024-09-10T09:58:46.268 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:58:46.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:58:46.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:46.323 INFO:teuthology.orchestra.run.smithi145.stdout:client4442 2024-09-10T09:58:46.323 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/af5c1104-e6d8-4fa7-9635-0686bce1580a.client4442/status') 2024-09-10T09:58:46.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:58:46.422 INFO:teuthology.orchestra.run.smithi145.stdout:instance: client.4442 (3)192.168.0.2:0/546066027 2024-09-10T09:58:46.423 INFO:teuthology.orchestra.run.smithi145.stdout:blocklisted: false 2024-09-10T09:58:46.423 INFO:teuthology.orchestra.run.smithi145.stderr:0+1 records in 2024-09-10T09:58:46.423 INFO:teuthology.orchestra.run.smithi145.stderr:0+1 records out 2024-09-10T09:58:46.423 INFO:teuthology.orchestra.run.smithi145.stderr:68 bytes copied, 5.045e-05 s, 1.3 MB/s 2024-09-10T09:58:46.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:58:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:58:46.431 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T09:58:46.515 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-10T09:58:46.516 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T09:58:46.516 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:46.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb201200640 1 Processor -- start 2024-09-10T09:58:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb201200640 1 -- start start 2024-09-10T09:58:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb201200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb1fc06ea50 0x7fb1fc06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb201200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc06bd30 0x7fb1fc06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb201200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1fc0b5f50 con 0x7fb1fc15e8f0 2024-09-10T09:58:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb201200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb1fc06f3d0 con 0x7fb1fc06ea50 2024-09-10T09:58:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb201200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc0b6230 con 0x7fb1fc06bd30 2024-09-10T09:58:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb1f3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb1fc06ea50 0x7fb1fc06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb1f3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb1fc06ea50 0x7fb1fc06ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59468/0 (socket says 172.21.15.110:59468) 2024-09-10T09:58:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb1f3e00640 1 -- 172.21.15.110:0/4041516418 learned_addr learned my addr 172.21.15.110:0/4041516418 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb1fbe00640 1 --2- 172.21.15.110:0/4041516418 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc06bd30 0x7fb1fc06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb1f3e00640 1 -- 172.21.15.110:0/4041516418 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc06bd30 msgr2=0x7fb1fc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb1f3e00640 1 --2- 172.21.15.110:0/4041516418 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc06bd30 0x7fb1fc06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb1f3e00640 1 -- 172.21.15.110:0/4041516418 >> v1:172.21.15.145:6789/0 conn(0x7fb1fc15e8f0 legacy=0x7fb1fc0b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.660+0000 7fb1f3e00640 1 -- 172.21.15.110:0/4041516418 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc0b6510 con 0x7fb1fc06ea50 2024-09-10T09:58:46.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb1f3e00640 1 --2- 172.21.15.110:0/4041516418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06ea50 0x7fb1fc06ee90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009870 tx=0x7fb1e401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c678af2725d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:46.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb1faa00640 1 -- 172.21.15.110:0/4041516418 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40b1070 con 0x7fb1fc06ea50 2024-09-10T09:58:46.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb1faa00640 1 -- 172.21.15.110:0/4041516418 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e40ac620 con 0x7fb1fc06ea50 2024-09-10T09:58:46.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb1faa00640 1 -- 172.21.15.110:0/4041516418 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40ac9f0 con 0x7fb1fc06ea50 2024-09-10T09:58:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb201200640 1 -- 172.21.15.110:0/4041516418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06ea50 msgr2=0x7fb1fc06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb201200640 1 --2- 172.21.15.110:0/4041516418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06ea50 0x7fb1fc06ee90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009870 tx=0x7fb1e401c4d0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb201200640 1 -- 172.21.15.110:0/4041516418 shutdown_connections 2024-09-10T09:58:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb201200640 1 --2- 172.21.15.110:0/4041516418 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc06bd30 0x7fb1fc06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb201200640 1 --2- 172.21.15.110:0/4041516418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06ea50 0x7fb1fc06ee90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb201200640 1 -- 172.21.15.110:0/4041516418 >> 172.21.15.110:0/4041516418 conn(0x7fb1fc0d1480 msgr2=0x7fb1fc0d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb201200640 1 -- 172.21.15.110:0/4041516418 shutdown_connections 2024-09-10T09:58:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.664+0000 7fb201200640 1 -- 172.21.15.110:0/4041516418 wait complete. 2024-09-10T09:58:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 Processor -- start 2024-09-10T09:58:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 -- start start 2024-09-10T09:58:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06bd30 0x7fb1fc13eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1fc06ea50 0x7fb1fc156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc15e8f0 0x7fb1fc0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0cddc0 con 0x7fb1fc06bd30 2024-09-10T09:58:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0cdc40 con 0x7fb1fc06ea50 2024-09-10T09:58:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc16b330 con 0x7fb1fc15e8f0 2024-09-10T09:58:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06bd30 0x7fb1fc13eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06bd30 0x7fb1fc13eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59472/0 (socket says 172.21.15.110:59472) 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 -- 172.21.15.110:0/288974021 learned_addr learned my addr 172.21.15.110:0/288974021 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1f3e00640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1fc06ea50 0x7fb1fc156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fbe00640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc15e8f0 0x7fb1fc0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 -- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc15e8f0 msgr2=0x7fb1fc0cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc15e8f0 0x7fb1fc0cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 -- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1fc06ea50 msgr2=0x7fb1fc156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1fc06ea50 0x7fb1fc156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 -- 172.21.15.110:0/288974021 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc154840 con 0x7fb1fc06bd30 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1f3e00640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1fc06ea50 0x7fb1fc156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1fb400640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06bd30 0x7fb1fc13eef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c3f0 tx=0x7fb1ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1f3400640 1 -- 172.21.15.110:0/288974021 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec017070 con 0x7fb1fc06bd30 2024-09-10T09:58:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1f3400640 1 -- 172.21.15.110:0/288974021 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1ec0052d0 con 0x7fb1fc06bd30 2024-09-10T09:58:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb1f3400640 1 -- 172.21.15.110:0/288974021 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec003840 con 0x7fb1fc06bd30 2024-09-10T09:58:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc154b20 con 0x7fb1fc06bd30 2024-09-10T09:58:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.668+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc13f430 con 0x7fb1fc06bd30 2024-09-10T09:58:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.672+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1fc06c1c0 con 0x7fb1fc06bd30 2024-09-10T09:58:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.672+0000 7fb1f3400640 1 -- 172.21.15.110:0/288974021 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb1ec01a070 con 0x7fb1fc06bd30 2024-09-10T09:58:46.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.672+0000 7fb1f3400640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb1d80678c0 0x7fb1d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:46.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.672+0000 7fb1f3400640 1 -- 172.21.15.110:0/288974021 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb1ec084ff0 con 0x7fb1fc06bd30 2024-09-10T09:58:46.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.672+0000 7fb1f3e00640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb1d80678c0 0x7fb1d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:46.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.672+0000 7fb1f3e00640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb1d80678c0 0x7fb1d8069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ae000 tx=0x7fb1e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:46.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.676+0000 7fb1f3400640 1 -- 172.21.15.110:0/288974021 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb1ec005480 con 0x7fb1fc06bd30 2024-09-10T09:58:46.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.840+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1fc153170 con 0x7fb1fc06bd30 2024-09-10T09:58:46.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb1f3400640 1 -- 172.21.15.110:0/288974021 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16701 (secure 0 0 0) 0x7fb1ec07ce20 con 0x7fb1fc06bd30 2024-09-10T09:58:46.845 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:46.846 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:23.766388+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:46.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb1d80678c0 msgr2=0x7fb1d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb1d80678c0 0x7fb1d8069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ae000 tx=0x7fb1e4008040 comp rx=0 tx=0).stop 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06bd30 msgr2=0x7fb1fc13eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06bd30 0x7fb1fc13eef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c3f0 tx=0x7fb1ec00c8b0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 shutdown_connections 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb1d80678c0 0x7fb1d8069d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1fc15e8f0 0x7fb1fc0cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1fc06ea50 0x7fb1fc156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 --2- 172.21.15.110:0/288974021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1fc06bd30 0x7fb1fc13eef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 >> 172.21.15.110:0/288974021 conn(0x7fb1fc0d1480 msgr2=0x7fb1fc15dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 shutdown_connections 2024-09-10T09:58:46.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:46.844+0000 7fb201200640 1 -- 172.21.15.110:0/288974021 wait complete. 2024-09-10T09:58:46.867 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T09:58:20.077745+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T09:58:21.140873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T09:58:46.867 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T09:58:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b0a000640 1 Processor -- start 2024-09-10T09:58:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b0a000640 1 -- start start 2024-09-10T09:58:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b0415e520 0x7f9b040cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b0a000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9b040b5f90 0x7f9b040b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b0a000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9b04070970 0x7f9b04070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b0a000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9b0406cd60 con 0x7f9b04070970 2024-09-10T09:58:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b0a000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9b0406cfb0 con 0x7f9b040b5f90 2024-09-10T09:58:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b0a000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0406d290 con 0x7f9b0415e520 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b09600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9b04070970 0x7f9b04070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b09600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9b04070970 0x7f9b04070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59476/0 (socket says 172.21.15.110:59476) 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b09600640 1 -- 172.21.15.110:0/4039664161 learned_addr learned my addr 172.21.15.110:0/4039664161 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b08c00640 1 --2- 172.21.15.110:0/4039664161 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b0415e520 0x7f9b040cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b03e00640 1 --2- 172.21.15.110:0/4039664161 >> v2:172.21.15.145:3300/0 conn(0x7f9b040b5f90 0x7f9b040b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b09600640 1 -- 172.21.15.110:0/4039664161 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b0415e520 msgr2=0x7f9b040cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b09600640 1 --2- 172.21.15.110:0/4039664161 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b0415e520 0x7f9b040cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b09600640 1 -- 172.21.15.110:0/4039664161 >> v2:172.21.15.145:3300/0 conn(0x7f9b040b5f90 msgr2=0x7f9b040b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b09600640 1 --2- 172.21.15.110:0/4039664161 >> v2:172.21.15.145:3300/0 conn(0x7f9b040b5f90 0x7f9b040b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b09600640 1 -- 172.21.15.110:0/4039664161 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0406d570 con 0x7f9b04070970 2024-09-10T09:58:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b08c00640 1 --2- 172.21.15.110:0/4039664161 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b0415e520 0x7f9b040cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:47.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.004+0000 7f9b03e00640 1 --2- 172.21.15.110:0/4039664161 >> v2:172.21.15.145:3300/0 conn(0x7f9b040b5f90 0x7f9b040b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:47.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b09600640 1 --2- 172.21.15.110:0/4039664161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b04070970 0x7f9b04070e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9b040636c0 tx=0x7f9afc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef411175f4c235a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b03400640 1 -- 172.21.15.110:0/4039664161 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0a55e0 con 0x7f9b04070970 2024-09-10T09:58:47.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b03400640 1 -- 172.21.15.110:0/4039664161 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc0a5780 con 0x7f9b04070970 2024-09-10T09:58:47.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b03400640 1 -- 172.21.15.110:0/4039664161 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0a5b50 con 0x7f9b04070970 2024-09-10T09:58:47.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 -- 172.21.15.110:0/4039664161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b04070970 msgr2=0x7f9b04070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 --2- 172.21.15.110:0/4039664161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b04070970 0x7f9b04070e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9b040636c0 tx=0x7f9afc01c960 comp rx=0 tx=0).stop 2024-09-10T09:58:47.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 -- 172.21.15.110:0/4039664161 shutdown_connections 2024-09-10T09:58:47.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 --2- 172.21.15.110:0/4039664161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b04070970 0x7f9b04070e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 --2- 172.21.15.110:0/4039664161 >> v2:172.21.15.145:3300/0 conn(0x7f9b040b5f90 0x7f9b040b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 --2- 172.21.15.110:0/4039664161 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b0415e520 0x7f9b040cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 -- 172.21.15.110:0/4039664161 >> 172.21.15.110:0/4039664161 conn(0x7f9b040b41f0 msgr2=0x7f9b040b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:47.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 -- 172.21.15.110:0/4039664161 shutdown_connections 2024-09-10T09:58:47.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.008+0000 7f9b0a000640 1 -- 172.21.15.110:0/4039664161 wait complete. 2024-09-10T09:58:47.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 Processor -- start 2024-09-10T09:58:47.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 -- start start 2024-09-10T09:58:47.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b04070970 0x7f9b040b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b040b5f90 0x7f9b0414e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b0415e520 0x7f9b0415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04162d50 con 0x7f9b0415e520 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04162bd0 con 0x7f9b04070970 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9b04162ed0 con 0x7f9b040b5f90 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b0415e520 0x7f9b0415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b0415e520 0x7f9b0415cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59480/0 (socket says 172.21.15.110:59480) 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 -- 172.21.15.110:0/1857293569 learned_addr learned my addr 172.21.15.110:0/1857293569 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b03e00640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b040b5f90 0x7f9b0414e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 -- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b040b5f90 msgr2=0x7f9b0414e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b040b5f90 0x7f9b0414e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b08c00640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b04070970 0x7f9b040b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 -- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b04070970 msgr2=0x7f9b040b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b04070970 0x7f9b040b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 -- 172.21.15.110:0/1857293569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b04155b40 con 0x7f9b0415e520 2024-09-10T09:58:47.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b08c00640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b04070970 0x7f9b040b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:47.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b09600640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b0415e520 0x7f9b0415cf40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0af040 tx=0x7f9afc019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b00c00640 1 -- 172.21.15.110:0/1857293569 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0bc020 con 0x7f9b0415e520 2024-09-10T09:58:47.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b00c00640 1 -- 172.21.15.110:0/1857293569 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc0b8690 con 0x7f9b0415e520 2024-09-10T09:58:47.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b04155de0 con 0x7f9b0415e520 2024-09-10T09:58:47.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b00c00640 1 -- 172.21.15.110:0/1857293569 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc009510 con 0x7f9b0415e520 2024-09-10T09:58:47.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041563a0 con 0x7f9b0415e520 2024-09-10T09:58:47.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.012+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b04070e20 con 0x7f9b0415e520 2024-09-10T09:58:47.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.016+0000 7f9b00c00640 1 -- 172.21.15.110:0/1857293569 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9afc0bf020 con 0x7f9b0415e520 2024-09-10T09:58:47.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.016+0000 7f9b00c00640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ae40678c0 0x7f9ae4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.016+0000 7f9b00c00640 1 -- 172.21.15.110:0/1857293569 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9afc120b80 con 0x7f9b0415e520 2024-09-10T09:58:47.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.016+0000 7f9b08c00640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ae40678c0 0x7f9ae4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.016+0000 7f9b08c00640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ae40678c0 0x7f9ae4069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009980 tx=0x7f9af4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.020+0000 7f9b00c00640 1 -- 172.21.15.110:0/1857293569 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9afc0c6030 con 0x7f9b0415e520 2024-09-10T09:58:47.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.176+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b040d1740 con 0x7f9b0415e520 2024-09-10T09:58:47.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.176+0000 7f9b00c00640 1 -- 172.21.15.110:0/1857293569 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f9afc0165e0 con 0x7f9b0415e520 2024-09-10T09:58:47.179 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T09:58:47.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ae40678c0 msgr2=0x7f9ae4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ae40678c0 0x7f9ae4069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009980 tx=0x7f9af4008040 comp rx=0 tx=0).stop 2024-09-10T09:58:47.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b0415e520 msgr2=0x7f9b0415cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b0415e520 0x7f9b0415cf40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0af040 tx=0x7f9afc019700 comp rx=0 tx=0).stop 2024-09-10T09:58:47.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 shutdown_connections 2024-09-10T09:58:47.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ae40678c0 0x7f9ae4069d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b0415e520 0x7f9b0415cf40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9b040b5f90 0x7f9b0414e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 --2- 172.21.15.110:0/1857293569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b04070970 0x7f9b040b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 >> 172.21.15.110:0/1857293569 conn(0x7f9b040b41f0 msgr2=0x7f9b040cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:47.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 shutdown_connections 2024-09-10T09:58:47.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.180+0000 7f9b0a000640 1 -- 172.21.15.110:0/1857293569 wait complete. 2024-09-10T09:58:47.200 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T09:58:47.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1771600640 1 Processor -- start 2024-09-10T09:58:47.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1771600640 1 -- start start 2024-09-10T09:58:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1771600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f176c0d20b0 0x7f176c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1771600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f176c0b71c0 0x7f176c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1771600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176c071470 con 0x7f176c15e940 2024-09-10T09:58:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1771600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f176c06cd60 con 0x7f176c0b71c0 2024-09-10T09:58:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1771600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f176c071710 con 0x7f176c0d20b0 2024-09-10T09:58:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176aa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f176c15e940 0x7f176c0b6ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51038/0 (socket says 172.21.15.110:51038) 2024-09-10T09:58:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176aa00640 1 -- 172.21.15.110:0/3187965843 learned_addr learned my addr 172.21.15.110:0/3187965843 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176b400640 1 --2- 172.21.15.110:0/3187965843 >> v2:172.21.15.110:3300/0 conn(0x7f176c0b71c0 0x7f176c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176a000640 1 --2- 172.21.15.110:0/3187965843 >> v2:172.21.15.145:3300/0 conn(0x7f176c0d20b0 0x7f176c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1769600640 1 -- 172.21.15.110:0/3187965843 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3657441181 0 0) 0x7f176c071470 con 0x7f176c15e940 2024-09-10T09:58:47.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f1769600640 1 -- 172.21.15.110:0/3187965843 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1750003530 con 0x7f176c15e940 2024-09-10T09:58:47.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176b400640 1 -- 172.21.15.110:0/3187965843 >> v2:172.21.15.145:3300/0 conn(0x7f176c0d20b0 msgr2=0x7f176c06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:47.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176b400640 1 --2- 172.21.15.110:0/3187965843 >> v2:172.21.15.145:3300/0 conn(0x7f176c0d20b0 0x7f176c06d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176b400640 1 -- 172.21.15.110:0/3187965843 >> v1:172.21.15.110:6789/0 conn(0x7f176c15e940 legacy=0x7f176c0b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176b400640 1 -- 172.21.15.110:0/3187965843 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c0719f0 con 0x7f176c0b71c0 2024-09-10T09:58:47.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.336+0000 7f176a000640 1 --2- 172.21.15.110:0/3187965843 >> v2:172.21.15.145:3300/0 conn(0x7f176c0d20b0 0x7f176c06d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:47.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f176b400640 1 --2- 172.21.15.110:0/3187965843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0b71c0 0x7f176c06dcf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f176c050030 tx=0x7f176401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d9c3c9e0256c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1769600640 1 -- 172.21.15.110:0/3187965843 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640b0070 con 0x7f176c0b71c0 2024-09-10T09:58:47.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1769600640 1 -- 172.21.15.110:0/3187965843 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17640ab600 con 0x7f176c0b71c0 2024-09-10T09:58:47.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1769600640 1 -- 172.21.15.110:0/3187965843 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640ab9b0 con 0x7f176c0b71c0 2024-09-10T09:58:47.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1771600640 1 -- 172.21.15.110:0/3187965843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0b71c0 msgr2=0x7f176c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1771600640 1 --2- 172.21.15.110:0/3187965843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0b71c0 0x7f176c06dcf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f176c050030 tx=0x7f176401c3f0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1771600640 1 -- 172.21.15.110:0/3187965843 shutdown_connections 2024-09-10T09:58:47.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1771600640 1 --2- 172.21.15.110:0/3187965843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0b71c0 0x7f176c06dcf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1771600640 1 --2- 172.21.15.110:0/3187965843 >> v2:172.21.15.145:3300/0 conn(0x7f176c0d20b0 0x7f176c06d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1771600640 1 -- 172.21.15.110:0/3187965843 >> 172.21.15.110:0/3187965843 conn(0x7f176c0d25a0 msgr2=0x7f176c0d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:47.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1771600640 1 -- 172.21.15.110:0/3187965843 shutdown_connections 2024-09-10T09:58:47.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.340+0000 7f1771600640 1 -- 172.21.15.110:0/3187965843 wait complete. 2024-09-10T09:58:47.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 Processor -- start 2024-09-10T09:58:47.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 -- start start 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176c0b71c0 0x7f176c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0d20b0 0x7f176c073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f176c15e940 0x7f176c073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f176c16b6c0 con 0x7f176c0d20b0 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f176c16b540 con 0x7f176c0b71c0 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f176c16b840 con 0x7f176c15e940 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0d20b0 0x7f176c073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0d20b0 0x7f176c073850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59498/0 (socket says 172.21.15.110:59498) 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 -- 172.21.15.110:0/4084370321 learned_addr learned my addr 172.21.15.110:0/4084370321 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176aa00640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176c0b71c0 0x7f176c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176b400640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f176c15e940 0x7f176c073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 -- 172.21.15.110:0/4084370321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f176c15e940 msgr2=0x7f176c073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f176c15e940 0x7f176c073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 -- 172.21.15.110:0/4084370321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176c0b71c0 msgr2=0x7f176c14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176c0b71c0 0x7f176c14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 -- 172.21.15.110:0/4084370321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c14e450 con 0x7f176c0d20b0 2024-09-10T09:58:47.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f176a000640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0d20b0 0x7f176c073850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f175800e840 tx=0x7f175800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1762a00640 1 -- 172.21.15.110:0/4084370321 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758017070 con 0x7f176c0d20b0 2024-09-10T09:58:47.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1762a00640 1 -- 172.21.15.110:0/4084370321 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175800c8d0 con 0x7f176c0d20b0 2024-09-10T09:58:47.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c14e730 con 0x7f176c0d20b0 2024-09-10T09:58:47.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1762a00640 1 -- 172.21.15.110:0/4084370321 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175800f040 con 0x7f176c0d20b0 2024-09-10T09:58:47.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.344+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c14ecf0 con 0x7f176c0d20b0 2024-09-10T09:58:47.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.348+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176c0b6ab0 con 0x7f176c0d20b0 2024-09-10T09:58:47.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.348+0000 7f1762a00640 1 -- 172.21.15.110:0/4084370321 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f175801a070 con 0x7f176c0d20b0 2024-09-10T09:58:47.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.348+0000 7f1762a00640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f174c0678f0 0x7f174c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.348+0000 7f1762a00640 1 -- 172.21.15.110:0/4084370321 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1758085890 con 0x7f176c0d20b0 2024-09-10T09:58:47.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.348+0000 7f176aa00640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f174c0678f0 0x7f174c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.348+0000 7f176aa00640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f174c0678f0 0x7f174c069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f175c005c10 tx=0x7f175c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.352+0000 7f1762a00640 1 -- 172.21.15.110:0/4084370321 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1758055ce0 con 0x7f176c0d20b0 2024-09-10T09:58:47.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f176c15cc90 con 0x7f176c0d20b0 2024-09-10T09:58:47.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1762a00640 1 -- 172.21.15.110:0/4084370321 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f1758059fe0 con 0x7f176c0d20b0 2024-09-10T09:58:47.513 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-09-10T09:58:47.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f174c0678f0 msgr2=0x7f174c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f174c0678f0 0x7f174c069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f175c005c10 tx=0x7f175c009040 comp rx=0 tx=0).stop 2024-09-10T09:58:47.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0d20b0 msgr2=0x7f176c073850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0d20b0 0x7f176c073850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f175800e840 tx=0x7f175800ed00 comp rx=0 tx=0).stop 2024-09-10T09:58:47.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 shutdown_connections 2024-09-10T09:58:47.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f174c0678f0 0x7f174c069db0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f176c15e940 0x7f176c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f176c0d20b0 0x7f176c073850 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 --2- 172.21.15.110:0/4084370321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176c0b71c0 0x7f176c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 >> 172.21.15.110:0/4084370321 conn(0x7f176c0d25a0 msgr2=0x7f176c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:47.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 shutdown_connections 2024-09-10T09:58:47.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.512+0000 7f1771600640 1 -- 172.21.15.110:0/4084370321 wait complete. 2024-09-10T09:58:47.535 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T09:58:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90b0600640 1 Processor -- start 2024-09-10T09:58:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90b0600640 1 -- start start 2024-09-10T09:58:47.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90b0600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f90a815f010 0x7f90a815f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90b0600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a806f380 0x7f90a80b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90b0600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a8072690 con 0x7f90a815f930 2024-09-10T09:58:47.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90b0600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f90a8072970 con 0x7f90a815f010 2024-09-10T09:58:47.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90b0600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f90a8072c50 con 0x7f90a806f380 2024-09-10T09:58:47.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90ae600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a806f380 0x7f90a80b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90adc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f90a815f010 0x7f90a815f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90ad200640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f90a815f930 0x7f90a806ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:48742/0 (socket says 172.21.15.110:48742) 2024-09-10T09:58:47.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90ae600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a806f380 0x7f90a80b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:49418/0 (socket says 172.21.15.110:49418) 2024-09-10T09:58:47.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90ad200640 1 -- 172.21.15.110:0/3986940546 learned_addr learned my addr 172.21.15.110:0/3986940546 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:47.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90adc00640 1 -- 172.21.15.110:0/3986940546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a806f380 msgr2=0x7f90a80b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:47.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90adc00640 1 --2- 172.21.15.110:0/3986940546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a806f380 0x7f90a80b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90adc00640 1 -- 172.21.15.110:0/3986940546 >> v1:172.21.15.145:6789/0 conn(0x7f90a815f930 legacy=0x7f90a806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.672+0000 7f90adc00640 1 -- 172.21.15.110:0/3986940546 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a8072f30 con 0x7f90a815f010 2024-09-10T09:58:47.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90adc00640 1 --2- 172.21.15.110:0/3986940546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 0x7f90a815f3f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097c0 tx=0x7f909c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=462e63c28676ac4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90a7e00640 1 -- 172.21.15.110:0/3986940546 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0b0070 con 0x7f90a815f010 2024-09-10T09:58:47.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90a7e00640 1 -- 172.21.15.110:0/3986940546 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909c0ab600 con 0x7f90a815f010 2024-09-10T09:58:47.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90a7e00640 1 -- 172.21.15.110:0/3986940546 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0ab9d0 con 0x7f90a815f010 2024-09-10T09:58:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 -- 172.21.15.110:0/3986940546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 msgr2=0x7f90a815f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 --2- 172.21.15.110:0/3986940546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 0x7f90a815f3f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097c0 tx=0x7f909c01c3e0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 -- 172.21.15.110:0/3986940546 shutdown_connections 2024-09-10T09:58:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 --2- 172.21.15.110:0/3986940546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a806f380 0x7f90a80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 --2- 172.21.15.110:0/3986940546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 0x7f90a815f3f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 -- 172.21.15.110:0/3986940546 >> 172.21.15.110:0/3986940546 conn(0x7f90a806ca70 msgr2=0x7f90a806ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 -- 172.21.15.110:0/3986940546 shutdown_connections 2024-09-10T09:58:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 -- 172.21.15.110:0/3986940546 wait complete. 2024-09-10T09:58:47.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 Processor -- start 2024-09-10T09:58:47.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.676+0000 7f90b0600640 1 -- start start 2024-09-10T09:58:47.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90a806f380 0x7f90a814ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 0x7f90a8155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a815f930 0x7f90a8156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90a80b5b50 con 0x7f90a806f380 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f90a80b59d0 con 0x7f90a815f010 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f90a80b5cd0 con 0x7f90a815f930 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90a806f380 0x7f90a814ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90a806f380 0x7f90a814ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59502/0 (socket says 172.21.15.110:59502) 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 -- 172.21.15.110:0/2774566857 learned_addr learned my addr 172.21.15.110:0/2774566857 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90ad200640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 0x7f90a8155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90ae600640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a815f930 0x7f90a8156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 -- 172.21.15.110:0/2774566857 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a815f930 msgr2=0x7f90a8156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a815f930 0x7f90a8156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 -- 172.21.15.110:0/2774566857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 msgr2=0x7f90a8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 0x7f90a8155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 -- 172.21.15.110:0/2774566857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a81558e0 con 0x7f90a806f380 2024-09-10T09:58:47.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90adc00640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90a806f380 0x7f90a814ac60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f909c0ae040 tx=0x7f909c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90a5600640 1 -- 172.21.15.110:0/2774566857 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0b0050 con 0x7f90a806f380 2024-09-10T09:58:47.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90a5600640 1 -- 172.21.15.110:0/2774566857 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909c0bc070 con 0x7f90a806f380 2024-09-10T09:58:47.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a814bda0 con 0x7f90a806f380 2024-09-10T09:58:47.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90a5600640 1 -- 172.21.15.110:0/2774566857 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0b5870 con 0x7f90a806f380 2024-09-10T09:58:47.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a814c360 con 0x7f90a806f380 2024-09-10T09:58:47.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a814c5a0 con 0x7f90a806f380 2024-09-10T09:58:47.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.680+0000 7f90a5600640 1 -- 172.21.15.110:0/2774566857 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f909c0b5a10 con 0x7f90a806f380 2024-09-10T09:58:47.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.684+0000 7f90a5600640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f90880678c0 0x7f9088069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:47.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.684+0000 7f90a5600640 1 -- 172.21.15.110:0/2774566857 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f909c120560 con 0x7f90a806f380 2024-09-10T09:58:47.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.684+0000 7f90ad200640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f90880678c0 0x7f9088069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:47.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.684+0000 7f90ad200640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f90880678c0 0x7f9088069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90a81faac0 tx=0x7f9094009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:47.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.688+0000 7f90a5600640 1 -- 172.21.15.110:0/2774566857 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f909c016090 con 0x7f90a806f380 2024-09-10T09:58:47.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.844+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90a80b4c80 con 0x7f90a806f380 2024-09-10T09:58:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90a5600640 1 -- 172.21.15.110:0/2774566857 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f909c0f11b0 con 0x7f90a806f380 2024-09-10T09:58:47.849 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 64 2024-09-10T09:58:47.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f90880678c0 msgr2=0x7f9088069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f90880678c0 0x7f9088069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90a81faac0 tx=0x7f9094009040 comp rx=0 tx=0).stop 2024-09-10T09:58:47.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90a806f380 msgr2=0x7f90a814ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:47.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90a806f380 0x7f90a814ac60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f909c0ae040 tx=0x7f909c01ac20 comp rx=0 tx=0).stop 2024-09-10T09:58:47.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 shutdown_connections 2024-09-10T09:58:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f90880678c0 0x7f9088069d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90a815f930 0x7f90a8156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90a815f010 0x7f90a8155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 --2- 172.21.15.110:0/2774566857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90a806f380 0x7f90a814ac60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 >> 172.21.15.110:0/2774566857 conn(0x7f90a806ca70 msgr2=0x7f90a815b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 shutdown_connections 2024-09-10T09:58:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:47.848+0000 7f90b0600640 1 -- 172.21.15.110:0/2774566857 wait complete. 2024-09-10T09:58:47.870 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-10T09:58:48.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42e00640 1 Processor -- start 2024-09-10T09:58:48.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42e00640 1 -- start start 2024-09-10T09:58:48.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbe3c06f200 0x7fbe3c0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:48.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fbe3c15c340 0x7fbe3c0d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:48.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:48.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbe3c15c7d0 con 0x7fbe3c06f200 2024-09-10T09:58:48.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fbe3c0b6780 con 0x7fbe3c15c340 2024-09-10T09:58:48.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3c0ce730 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbe3c06f200 0x7fbe3c0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:48.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbe3c06f200 0x7fbe3c0cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59516/0 (socket says 172.21.15.110:59516) 2024-09-10T09:58:48.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41a00640 1 -- 172.21.15.110:0/2239242712 learned_addr learned my addr 172.21.15.110:0/2239242712 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:48.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41000640 1 --2- 172.21.15.110:0/2239242712 >> v2:172.21.15.145:3300/0 conn(0x7fbe3c15c340 0x7fbe3c0d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:48.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe42400640 1 --2- 172.21.15.110:0/2239242712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:48.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41a00640 1 -- 172.21.15.110:0/2239242712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 msgr2=0x7fbe3c0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:48.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41a00640 1 --2- 172.21.15.110:0/2239242712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c0b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41a00640 1 -- 172.21.15.110:0/2239242712 >> v2:172.21.15.145:3300/0 conn(0x7fbe3c15c340 msgr2=0x7fbe3c0d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:48.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41a00640 1 --2- 172.21.15.110:0/2239242712 >> v2:172.21.15.145:3300/0 conn(0x7fbe3c15c340 0x7fbe3c0d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41a00640 1 -- 172.21.15.110:0/2239242712 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c0ce9d0 con 0x7fbe3c06f200 2024-09-10T09:58:48.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.004+0000 7fbe41000640 1 --2- 172.21.15.110:0/2239242712 >> v2:172.21.15.145:3300/0 conn(0x7fbe3c15c340 0x7fbe3c0d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:48.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe41a00640 1 --2- 172.21.15.110:0/2239242712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c06f200 0x7fbe3c0cf790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0717e0 tx=0x7fbe3001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110b0d0814dd2f72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:48.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe3be00640 1 -- 172.21.15.110:0/2239242712 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300b0070 con 0x7fbe3c06f200 2024-09-10T09:58:48.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe3be00640 1 -- 172.21.15.110:0/2239242712 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe300ab600 con 0x7fbe3c06f200 2024-09-10T09:58:48.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe3be00640 1 -- 172.21.15.110:0/2239242712 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300ab9d0 con 0x7fbe3c06f200 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 -- 172.21.15.110:0/2239242712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c06f200 msgr2=0x7fbe3c0cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 --2- 172.21.15.110:0/2239242712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c06f200 0x7fbe3c0cf790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0717e0 tx=0x7fbe3001c520 comp rx=0 tx=0).stop 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 -- 172.21.15.110:0/2239242712 shutdown_connections 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 --2- 172.21.15.110:0/2239242712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c0b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 --2- 172.21.15.110:0/2239242712 >> v2:172.21.15.145:3300/0 conn(0x7fbe3c15c340 0x7fbe3c0d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 --2- 172.21.15.110:0/2239242712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c06f200 0x7fbe3c0cf790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 -- 172.21.15.110:0/2239242712 >> 172.21.15.110:0/2239242712 conn(0x7fbe3c06caa0 msgr2=0x7fbe3c06ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 -- 172.21.15.110:0/2239242712 shutdown_connections 2024-09-10T09:58:48.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 -- 172.21.15.110:0/2239242712 wait complete. 2024-09-10T09:58:48.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.008+0000 7fbe42e00640 1 Processor -- start 2024-09-10T09:58:48.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 -- start start 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe3c06f200 0x7fbe3c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c14be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c15c340 0x7fbe3c15c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c0b4940 con 0x7fbe3c15c340 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c06f5e0 con 0x7fbe3c06f200 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3c0b4ac0 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe3c06f200 0x7fbe3c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c14be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c15c340 0x7fbe3c15c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c14be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:49436/0 (socket says 172.21.15.110:49436) 2024-09-10T09:58:48.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 -- 172.21.15.110:0/515184109 learned_addr learned my addr 172.21.15.110:0/515184109 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:48.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe3c06f200 0x7fbe3c15e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:33758/0 (socket says 172.21.15.110:33758) 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 -- 172.21.15.110:0/515184109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe3c06f200 msgr2=0x7fbe3c15e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe3c06f200 0x7fbe3c15e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 -- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c15c340 msgr2=0x7fbe3c15c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c15c340 0x7fbe3c15c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 -- 172.21.15.110:0/515184109 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c15d200 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42400640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c15c340 0x7fbe3c15c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41a00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe3c06f200 0x7fbe3c15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe41000640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c14be40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800d800 tx=0x7fbe2800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:48.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe39600640 1 -- 172.21.15.110:0/515184109 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28018020 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe39600640 1 -- 172.21.15.110:0/515184109 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe280096c0 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c15d4e0 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe39600640 1 -- 172.21.15.110:0/515184109 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28009ab0 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c15daa0 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.012+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c0cf790 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.016+0000 7fbe39600640 1 -- 172.21.15.110:0/515184109 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbe2801b020 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.016+0000 7fbe39600640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe1c0678c0 0x7fbe1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:48.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.016+0000 7fbe39600640 1 -- 172.21.15.110:0/515184109 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbe28086560 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.016+0000 7fbe41a00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe1c0678c0 0x7fbe1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:48.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.016+0000 7fbe41a00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe1c0678c0 0x7fbe1c069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe300ad000 tx=0x7fbe30008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.020+0000 7fbe39600640 1 -- 172.21.15.110:0/515184109 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbe28022020 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.232+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fbe3c06d290 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.589 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:58:48.593 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:58:48.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.852+0000 7fbe39600640 1 -- 172.21.15.110:0/515184109 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v43) ==== 161+0+0 (secure 0 0 0) 0x7fbe28056a30 con 0x7fbe3c0b5e30 2024-09-10T09:58:48.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe1c0678c0 msgr2=0x7fbe1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:48.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe1c0678c0 0x7fbe1c069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe300ad000 tx=0x7fbe30008040 comp rx=0 tx=0).stop 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 msgr2=0x7fbe3c14be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c14be40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800d800 tx=0x7fbe2800dcc0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 shutdown_connections 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe1c0678c0 0x7fbe1c069d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe3c15c340 0x7fbe3c15c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe3c0b5e30 0x7fbe3c14be40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 --2- 172.21.15.110:0/515184109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe3c06f200 0x7fbe3c15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 >> 172.21.15.110:0/515184109 conn(0x7fbe3c06caa0 msgr2=0x7fbe3c1591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 shutdown_connections 2024-09-10T09:58:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:48.856+0000 7fbe42e00640 1 -- 172.21.15.110:0/515184109 wait complete. 2024-09-10T09:58:48.878 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T09:58:49.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd150400640 1 Processor -- start 2024-09-10T09:58:49.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd150400640 1 -- start start 2024-09-10T09:58:49.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd150400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd1480b2bb0 0x7fd1480cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd150400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1480b37c0 con 0x7fd1480cfbc0 2024-09-10T09:58:49.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd150400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1480b3aa0 con 0x7fd148145f30 2024-09-10T09:58:49.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd150400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd1480b3d80 con 0x7fd1480b2bb0 2024-09-10T09:58:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd14e400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fd1480cfbc0 0x7fd1480b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57778/0 (socket says 172.21.15.110:57778) 2024-09-10T09:58:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd14e400640 1 -- 172.21.15.110:0/3031053531 learned_addr learned my addr 172.21.15.110:0/3031053531 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd14d000640 1 --2- 172.21.15.110:0/3031053531 >> v2:172.21.15.145:3300/0 conn(0x7fd1480b2bb0 0x7fd1480cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd147e00640 1 -- 172.21.15.110:0/3031053531 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2784974051 0 0) 0x7fd1480b37c0 con 0x7fd1480cfbc0 2024-09-10T09:58:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd147e00640 1 -- 172.21.15.110:0/3031053531 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd130003530 con 0x7fd1480cfbc0 2024-09-10T09:58:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd14d000640 1 -- 172.21.15.110:0/3031053531 >> v1:172.21.15.145:6789/0 conn(0x7fd148145f30 legacy=0x7fd1480b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd14d000640 1 -- 172.21.15.110:0/3031053531 >> v1:172.21.15.110:6789/0 conn(0x7fd1480cfbc0 legacy=0x7fd1480b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.016+0000 7fd14d000640 1 -- 172.21.15.110:0/3031053531 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd148146310 con 0x7fd1480b2bb0 2024-09-10T09:58:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd14d000640 1 --2- 172.21.15.110:0/3031053531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 0x7fd1480cf680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd1340097c0 tx=0x7fd13401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a155e1941544a376 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd147e00640 1 -- 172.21.15.110:0/3031053531 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1340b0070 con 0x7fd1480b2bb0 2024-09-10T09:58:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd147e00640 1 -- 172.21.15.110:0/3031053531 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1340ab600 con 0x7fd1480b2bb0 2024-09-10T09:58:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd147e00640 1 -- 172.21.15.110:0/3031053531 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1340ab9d0 con 0x7fd1480b2bb0 2024-09-10T09:58:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd150400640 1 -- 172.21.15.110:0/3031053531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 msgr2=0x7fd1480cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd150400640 1 --2- 172.21.15.110:0/3031053531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 0x7fd1480cf680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd1340097c0 tx=0x7fd13401c4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd150400640 1 -- 172.21.15.110:0/3031053531 shutdown_connections 2024-09-10T09:58:49.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd150400640 1 --2- 172.21.15.110:0/3031053531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 0x7fd1480cf680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd150400640 1 -- 172.21.15.110:0/3031053531 >> 172.21.15.110:0/3031053531 conn(0x7fd148062e40 msgr2=0x7fd14806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:49.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd150400640 1 -- 172.21.15.110:0/3031053531 shutdown_connections 2024-09-10T09:58:49.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.020+0000 7fd150400640 1 -- 172.21.15.110:0/3031053531 wait complete. 2024-09-10T09:58:49.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 Processor -- start 2024-09-10T09:58:49.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 -- start start 2024-09-10T09:58:49.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 0x7fd148160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd1480cfbc0 0x7fd1480b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd148145f30 0x7fd1480b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd1480b58b0 con 0x7fd148145f30 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1480b5730 con 0x7fd1480b2bb0 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd14816a1b0 con 0x7fd1480cfbc0 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd148145f30 0x7fd1480b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd148145f30 0x7fd1480b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56440/0 (socket says 172.21.15.110:56440) 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 -- 172.21.15.110:0/3419676336 learned_addr learned my addr 172.21.15.110:0/3419676336 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14da00640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 0x7fd148160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14d000640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd1480cfbc0 0x7fd1480b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 -- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd1480cfbc0 msgr2=0x7fd1480b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd1480cfbc0 0x7fd1480b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 -- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 msgr2=0x7fd148160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 0x7fd148160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 -- 172.21.15.110:0/3419676336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1480b1b00 con 0x7fd148145f30 2024-09-10T09:58:49.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14d000640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd1480cfbc0 0x7fd1480b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:49.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd14e400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd148145f30 0x7fd1480b0e90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd14000caa0 tx=0x7fd14000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:49.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd145600640 1 -- 172.21.15.110:0/3419676336 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140017070 con 0x7fd148145f30 2024-09-10T09:58:49.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd145600640 1 -- 172.21.15.110:0/3419676336 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14000ac00 con 0x7fd148145f30 2024-09-10T09:58:49.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd145600640 1 -- 172.21.15.110:0/3419676336 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140012400 con 0x7fd148145f30 2024-09-10T09:58:49.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1480b1de0 con 0x7fd148145f30 2024-09-10T09:58:49.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.024+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1480b0430 con 0x7fd148145f30 2024-09-10T09:58:49.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.028+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1480cf680 con 0x7fd148145f30 2024-09-10T09:58:49.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.028+0000 7fd145600640 1 -- 172.21.15.110:0/3419676336 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd14001a070 con 0x7fd148145f30 2024-09-10T09:58:49.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.028+0000 7fd145600640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd1280678c0 0x7fd128069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.028+0000 7fd145600640 1 -- 172.21.15.110:0/3419676336 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd140085b30 con 0x7fd148145f30 2024-09-10T09:58:49.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.028+0000 7fd14da00640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd1280678c0 0x7fd128069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.028+0000 7fd14da00640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd1280678c0 0x7fd128069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1481610d0 tx=0x7fd13c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:49.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.032+0000 7fd145600640 1 -- 172.21.15.110:0/3419676336 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd1400565d0 con 0x7fd148145f30 2024-09-10T09:58:49.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.188+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd1480b2300 con 0x7fd148145f30 2024-09-10T09:58:49.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.192+0000 7fd145600640 1 -- 172.21.15.110:0/3419676336 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fd140055ed0 con 0x7fd148145f30 2024-09-10T09:58:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd1280678c0 msgr2=0x7fd128069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd1280678c0 0x7fd128069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1481610d0 tx=0x7fd13c009040 comp rx=0 tx=0).stop 2024-09-10T09:58:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd148145f30 msgr2=0x7fd1480b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd148145f30 0x7fd1480b0e90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd14000caa0 tx=0x7fd14000cf60 comp rx=0 tx=0).stop 2024-09-10T09:58:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 shutdown_connections 2024-09-10T09:58:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd1280678c0 0x7fd128069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd148145f30 0x7fd1480b0e90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd1480cfbc0 0x7fd1480b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 --2- 172.21.15.110:0/3419676336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1480b2bb0 0x7fd148160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 >> 172.21.15.110:0/3419676336 conn(0x7fd148062e40 msgr2=0x7fd148071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 shutdown_connections 2024-09-10T09:58:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.196+0000 7fd150400640 1 -- 172.21.15.110:0/3419676336 wait complete. 2024-09-10T09:58:49.218 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T09:58:49.218 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T09:58:49.315 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T09:58:49.420 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T09:58:49.420 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:58:49.421 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:49.430 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:58:49.430 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:49.480 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:58:49.480 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:49.532 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T09:58:49.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.708+0000 7fdbd3800640 1 Processor -- start 2024-09-10T09:58:49.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.708+0000 7fdbd3800640 1 -- start start 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd3800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0726e0 0x7fdbcc0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd3800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdbcc0b5600 0x7fdbcc0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd3800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbcc06f3e0 con 0x7fdbcc06efa0 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd3800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdbcc0cee50 con 0x7fdbcc0b5600 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd3800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdbcc15fda0 con 0x7fdbcc0726e0 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd1800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdbcc0b5600 0x7fdbcc0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd1800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdbcc0b5600 0x7fdbcc0b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56442/0 (socket says 172.21.15.110:56442) 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd1800640 1 -- 172.21.15.110:0/2631424753 learned_addr learned my addr 172.21.15.110:0/2631424753 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd0e00640 1 --2- 172.21.15.110:0/2631424753 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0726e0 0x7fdbcc0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd1800640 1 -- 172.21.15.110:0/2631424753 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0726e0 msgr2=0x7fdbcc0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:49.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd1800640 1 --2- 172.21.15.110:0/2631424753 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0726e0 0x7fdbcc0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd1800640 1 -- 172.21.15.110:0/2631424753 >> v1:172.21.15.145:6789/0 conn(0x7fdbcc06efa0 legacy=0x7fdbcc0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.712+0000 7fdbd1800640 1 -- 172.21.15.110:0/2631424753 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc0ce520 con 0x7fdbcc0b5600 2024-09-10T09:58:49.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd1800640 1 --2- 172.21.15.110:0/2631424753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc0b5600 0x7fdbcc0b34d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009870 tx=0x7fdbc401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fabf8c54804b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:49.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbcb400640 1 -- 172.21.15.110:0/2631424753 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40a95f0 con 0x7fdbcc0b5600 2024-09-10T09:58:49.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbcb400640 1 -- 172.21.15.110:0/2631424753 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc40a9790 con 0x7fdbcc0b5600 2024-09-10T09:58:49.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbcb400640 1 -- 172.21.15.110:0/2631424753 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40a9b60 con 0x7fdbcc0b5600 2024-09-10T09:58:49.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 -- 172.21.15.110:0/2631424753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc0b5600 msgr2=0x7fdbcc0b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 --2- 172.21.15.110:0/2631424753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc0b5600 0x7fdbcc0b34d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009870 tx=0x7fdbc401c530 comp rx=0 tx=0).stop 2024-09-10T09:58:49.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 -- 172.21.15.110:0/2631424753 shutdown_connections 2024-09-10T09:58:49.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 --2- 172.21.15.110:0/2631424753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc0b5600 0x7fdbcc0b34d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 --2- 172.21.15.110:0/2631424753 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0726e0 0x7fdbcc0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 -- 172.21.15.110:0/2631424753 >> 172.21.15.110:0/2631424753 conn(0x7fdbcc062d10 msgr2=0x7fdbcc06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:49.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 -- 172.21.15.110:0/2631424753 shutdown_connections 2024-09-10T09:58:49.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 -- 172.21.15.110:0/2631424753 wait complete. 2024-09-10T09:58:49.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 Processor -- start 2024-09-10T09:58:49.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.716+0000 7fdbd3800640 1 -- start start 2024-09-10T09:58:49.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc06efa0 0x7fdbcc152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbcc0726e0 0x7fdbcc14c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0b5600 0x7fdbcc14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc072c40 con 0x7fdbcc06efa0 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc072ac0 con 0x7fdbcc0726e0 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdbcc15f950 con 0x7fdbcc0b5600 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc06efa0 0x7fdbcc152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc06efa0 0x7fdbcc152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56450/0 (socket says 172.21.15.110:56450) 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 -- 172.21.15.110:0/1597891519 learned_addr learned my addr 172.21.15.110:0/1597891519 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbcbe00640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbcc0726e0 0x7fdbcc14c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd1800640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0b5600 0x7fdbcc14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 -- 172.21.15.110:0/1597891519 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0b5600 msgr2=0x7fdbcc14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0b5600 0x7fdbcc14bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 -- 172.21.15.110:0/1597891519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbcc0726e0 msgr2=0x7fdbcc14c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbcc0726e0 0x7fdbcc14c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 -- 172.21.15.110:0/1597891519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc153370 con 0x7fdbcc06efa0 2024-09-10T09:58:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd0e00640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc06efa0 0x7fdbcc152d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc007ef0 tx=0x7fdbbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbc8c00640 1 -- 172.21.15.110:0/1597891519 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc014620 con 0x7fdbcc06efa0 2024-09-10T09:58:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbc8c00640 1 -- 172.21.15.110:0/1597891519 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbbc019070 con 0x7fdbcc06efa0 2024-09-10T09:58:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbc8c00640 1 -- 172.21.15.110:0/1597891519 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc014a00 con 0x7fdbcc06efa0 2024-09-10T09:58:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc14e440 con 0x7fdbcc06efa0 2024-09-10T09:58:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc14c970 con 0x7fdbcc06efa0 2024-09-10T09:58:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbcc0b34d0 con 0x7fdbcc06efa0 2024-09-10T09:58:49.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.720+0000 7fdbc8c00640 1 -- 172.21.15.110:0/1597891519 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdbbc014ba0 con 0x7fdbcc06efa0 2024-09-10T09:58:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.724+0000 7fdbc8c00640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdbac0678c0 0x7fdbac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.724+0000 7fdbc8c00640 1 -- 172.21.15.110:0/1597891519 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdbbc086480 con 0x7fdbcc06efa0 2024-09-10T09:58:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.724+0000 7fdbcbe00640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdbac0678c0 0x7fdbac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:49.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.724+0000 7fdbcbe00640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdbac0678c0 0x7fdbac069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc1f8720 tx=0x7fdbb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:49.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.728+0000 7fdbc8c00640 1 -- 172.21.15.110:0/1597891519 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdbbc056960 con 0x7fdbcc06efa0 2024-09-10T09:58:49.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.896+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdbcc159f10 con 0x7fdbcc06efa0 2024-09-10T09:58:49.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.896+0000 7fdbc8c00640 1 -- 172.21.15.110:0/1597891519 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fdbbc05ac60 con 0x7fdbcc06efa0 2024-09-10T09:58:49.899 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T09:58:49.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdbac0678c0 msgr2=0x7fdbac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdbac0678c0 0x7fdbac069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc1f8720 tx=0x7fdbb8009040 comp rx=0 tx=0).stop 2024-09-10T09:58:49.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc06efa0 msgr2=0x7fdbcc152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc06efa0 0x7fdbcc152d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc007ef0 tx=0x7fdbbc00e4b0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 shutdown_connections 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdbac0678c0 0x7fdbac069d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbcc0b5600 0x7fdbcc14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbcc0726e0 0x7fdbcc14c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 --2- 172.21.15.110:0/1597891519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbcc06efa0 0x7fdbcc152d90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 >> 172.21.15.110:0/1597891519 conn(0x7fdbcc062d10 msgr2=0x7fdbcc0b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 shutdown_connections 2024-09-10T09:58:49.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:49.900+0000 7fdbd3800640 1 -- 172.21.15.110:0/1597891519 wait complete. 2024-09-10T09:58:49.920 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T09:58:49.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:49.920 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T09:58:49.954 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:58:49.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:49.955 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T09:58:50.011 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:58:50.011 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:50.021 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:58:50.021 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:50.069 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:58:50.069 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:50.121 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T09:58:50.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.256+0000 7fcc96000640 1 Processor -- start 2024-09-10T09:58:50.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.256+0000 7fcc96000640 1 -- start start 2024-09-10T09:58:50.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.256+0000 7fcc96000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc900cfb90 0x7fcc9015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.256+0000 7fcc96000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 0x7fcc900cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc96000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc900cff70 0x7fcc900cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc96000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcc9006d130 con 0x7fcc900cff70 2024-09-10T09:58:50.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc96000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcc9015d4d0 con 0x7fcc900cfb90 2024-09-10T09:58:50.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc96000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc9015d7b0 con 0x7fcc900b3f40 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc900cff70 0x7fcc900cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc900cff70 0x7fcc900cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56456/0 (socket says 172.21.15.110:56456) 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 -- 172.21.15.110:0/2004052321 learned_addr learned my addr 172.21.15.110:0/2004052321 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8f400640 1 --2- 172.21.15.110:0/2004052321 >> v2:172.21.15.145:3300/0 conn(0x7fcc900cfb90 0x7fcc9015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8ea00640 1 --2- 172.21.15.110:0/2004052321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 0x7fcc900cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 -- 172.21.15.110:0/2004052321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 msgr2=0x7fcc900cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 --2- 172.21.15.110:0/2004052321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 0x7fcc900cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 -- 172.21.15.110:0/2004052321 >> v2:172.21.15.145:3300/0 conn(0x7fcc900cfb90 msgr2=0x7fcc9015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 --2- 172.21.15.110:0/2004052321 >> v2:172.21.15.145:3300/0 conn(0x7fcc900cfb90 0x7fcc9015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 -- 172.21.15.110:0/2004052321 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9006d380 con 0x7fcc900cff70 2024-09-10T09:58:50.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8fe00640 1 --2- 172.21.15.110:0/2004052321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc900cdae0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc90073450 tx=0x7fcc8801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae3e406e391e972 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:50.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8e000640 1 -- 172.21.15.110:0/2004052321 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc880b0070 con 0x7fcc900cff70 2024-09-10T09:58:50.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8e000640 1 -- 172.21.15.110:0/2004052321 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc880ab600 con 0x7fcc900cff70 2024-09-10T09:58:50.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.260+0000 7fcc8e000640 1 -- 172.21.15.110:0/2004052321 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc880ab9d0 con 0x7fcc900cff70 2024-09-10T09:58:50.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- 172.21.15.110:0/2004052321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 msgr2=0x7fcc900cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:50.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 --2- 172.21.15.110:0/2004052321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc900cdae0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc90073450 tx=0x7fcc8801c490 comp rx=0 tx=0).stop 2024-09-10T09:58:50.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- 172.21.15.110:0/2004052321 shutdown_connections 2024-09-10T09:58:50.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 --2- 172.21.15.110:0/2004052321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc900cdae0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 --2- 172.21.15.110:0/2004052321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 0x7fcc900cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 --2- 172.21.15.110:0/2004052321 >> v2:172.21.15.145:3300/0 conn(0x7fcc900cfb90 0x7fcc9015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- 172.21.15.110:0/2004052321 >> 172.21.15.110:0/2004052321 conn(0x7fcc90062d10 msgr2=0x7fcc9006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:50.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- 172.21.15.110:0/2004052321 shutdown_connections 2024-09-10T09:58:50.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- 172.21.15.110:0/2004052321 wait complete. 2024-09-10T09:58:50.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 Processor -- start 2024-09-10T09:58:50.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- start start 2024-09-10T09:58:50.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 0x7fcc901649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc900cfb90 0x7fcc90150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc9015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90160ba0 con 0x7fcc900cff70 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90160a20 con 0x7fcc900cfb90 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc96000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc90160d20 con 0x7fcc900b3f40 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc9015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc9015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56466/0 (socket says 172.21.15.110:56466) 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8fe00640 1 -- 172.21.15.110:0/2515298679 learned_addr learned my addr 172.21.15.110:0/2515298679 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8ea00640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc900cfb90 0x7fcc90150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8f400640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 0x7fcc901649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8fe00640 1 -- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 msgr2=0x7fcc901649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8fe00640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 0x7fcc901649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8fe00640 1 -- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc900cfb90 msgr2=0x7fcc90150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8fe00640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc900cfb90 0x7fcc90150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8fe00640 1 -- 172.21.15.110:0/2515298679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc90151140 con 0x7fcc900cff70 2024-09-10T09:58:50.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.264+0000 7fcc8ea00640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc900cfb90 0x7fcc90150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:50.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc8fe00640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc9015b7f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc880ae040 tx=0x7fcc88019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:50.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc87400640 1 -- 172.21.15.110:0/2515298679 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc880b0040 con 0x7fcc900cff70 2024-09-10T09:58:50.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc87400640 1 -- 172.21.15.110:0/2515298679 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc880b9070 con 0x7fcc900cff70 2024-09-10T09:58:50.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc90151410 con 0x7fcc900cff70 2024-09-10T09:58:50.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc87400640 1 -- 172.21.15.110:0/2515298679 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc880b4850 con 0x7fcc900cff70 2024-09-10T09:58:50.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc90161ef0 con 0x7fcc900cff70 2024-09-10T09:58:50.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc9015ca10 con 0x7fcc900cff70 2024-09-10T09:58:50.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc87400640 1 -- 172.21.15.110:0/2515298679 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcc880ab480 con 0x7fcc900cff70 2024-09-10T09:58:50.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc87400640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc700678c0 0x7fcc70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc87400640 1 -- 172.21.15.110:0/2515298679 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcc8811f810 con 0x7fcc900cff70 2024-09-10T09:58:50.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc8f400640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc700678c0 0x7fcc70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.268+0000 7fcc8f400640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc700678c0 0x7fcc70069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc901642f0 tx=0x7fcc8000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:50.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.272+0000 7fcc87400640 1 -- 172.21.15.110:0/2515298679 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcc88016110 con 0x7fcc900cff70 2024-09-10T09:58:50.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.440+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcc900b3b00 con 0x7fcc900cff70 2024-09-10T09:58:50.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.440+0000 7fcc87400640 1 -- 172.21.15.110:0/2515298679 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fcc880efe10 con 0x7fcc900cff70 2024-09-10T09:58:50.444 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T09:58:50.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc700678c0 msgr2=0x7fcc70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc700678c0 0x7fcc70069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc901642f0 tx=0x7fcc8000b040 comp rx=0 tx=0).stop 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 msgr2=0x7fcc9015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc9015b7f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc880ae040 tx=0x7fcc88019230 comp rx=0 tx=0).stop 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 shutdown_connections 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcc700678c0 0x7fcc70069d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc900cff70 0x7fcc9015b7f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc900cfb90 0x7fcc90150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 --2- 172.21.15.110:0/2515298679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc900b3f40 0x7fcc901649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 >> 172.21.15.110:0/2515298679 conn(0x7fcc90062d10 msgr2=0x7fcc9015f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 shutdown_connections 2024-09-10T09:58:50.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.444+0000 7fcc96000640 1 -- 172.21.15.110:0/2515298679 wait complete. 2024-09-10T09:58:50.466 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T09:58:50.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:50.466 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-10T09:58:50.499 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:58:50.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:50.499 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T09:58:50.560 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:58:50.560 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:50.569 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:58:50.569 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:50.623 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:58:50.624 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:50.676 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T09:58:50.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.852+0000 7fa2f8000640 1 Processor -- start 2024-09-10T09:58:50.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.852+0000 7fa2f8000640 1 -- start start 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f8000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2f00cfb90 0x7fa2f015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f8000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa2f00b3f40 0x7fa2f00cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f8000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cff70 0x7fa2f00cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f8000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa2f006d130 con 0x7fa2f00cfb90 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f8000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa2f015d4d0 con 0x7fa2f00b3f40 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f8000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f015d7b0 con 0x7fa2f00cff70 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f5600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2f00cfb90 0x7fa2f015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f5600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2f00cfb90 0x7fa2f015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56470/0 (socket says 172.21.15.110:56470) 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f5600640 1 -- 172.21.15.110:0/2515495958 learned_addr learned my addr 172.21.15.110:0/2515495958 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f6000640 1 --2- 172.21.15.110:0/2515495958 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cff70 0x7fa2f00cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f4c00640 1 --2- 172.21.15.110:0/2515495958 >> v2:172.21.15.145:3300/0 conn(0x7fa2f00b3f40 0x7fa2f00cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f5600640 1 -- 172.21.15.110:0/2515495958 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cff70 msgr2=0x7fa2f00cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:50.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f5600640 1 --2- 172.21.15.110:0/2515495958 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cff70 0x7fa2f00cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f5600640 1 -- 172.21.15.110:0/2515495958 >> v2:172.21.15.145:3300/0 conn(0x7fa2f00b3f40 msgr2=0x7fa2f00cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:50.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f5600640 1 --2- 172.21.15.110:0/2515495958 >> v2:172.21.15.145:3300/0 conn(0x7fa2f00b3f40 0x7fa2f00cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.856+0000 7fa2f5600640 1 -- 172.21.15.110:0/2515495958 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f006d380 con 0x7fa2f00cfb90 2024-09-10T09:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f5600640 1 --2- 172.21.15.110:0/2515495958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00cfb90 0x7fa2f015ca10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00cf000 tx=0x7fa2e401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bb62658386c2de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2efe00640 1 -- 172.21.15.110:0/2515495958 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40b0070 con 0x7fa2f00cfb90 2024-09-10T09:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2efe00640 1 -- 172.21.15.110:0/2515495958 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e40ab600 con 0x7fa2f00cfb90 2024-09-10T09:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2efe00640 1 -- 172.21.15.110:0/2515495958 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40ab9d0 con 0x7fa2f00cfb90 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- 172.21.15.110:0/2515495958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00cfb90 msgr2=0x7fa2f015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 --2- 172.21.15.110:0/2515495958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00cfb90 0x7fa2f015ca10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00cf000 tx=0x7fa2e401c490 comp rx=0 tx=0).stop 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- 172.21.15.110:0/2515495958 shutdown_connections 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 --2- 172.21.15.110:0/2515495958 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cff70 0x7fa2f00cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 --2- 172.21.15.110:0/2515495958 >> v2:172.21.15.145:3300/0 conn(0x7fa2f00b3f40 0x7fa2f00cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 --2- 172.21.15.110:0/2515495958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00cfb90 0x7fa2f015ca10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- 172.21.15.110:0/2515495958 >> 172.21.15.110:0/2515495958 conn(0x7fa2f0062d10 msgr2=0x7fa2f006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- 172.21.15.110:0/2515495958 shutdown_connections 2024-09-10T09:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- 172.21.15.110:0/2515495958 wait complete. 2024-09-10T09:58:50.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 Processor -- start 2024-09-10T09:58:50.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- start start 2024-09-10T09:58:50.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00b3f40 0x7fa2f01508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cfb90 0x7fa2f01f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2f00cff70 0x7fa2f01f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0160ba0 con 0x7fa2f00b3f40 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0160a20 con 0x7fa2f00cff70 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.860+0000 7fa2f8000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f0160d20 con 0x7fa2f00cfb90 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00b3f40 0x7fa2f01508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00b3f40 0x7fa2f01508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56472/0 (socket says 172.21.15.110:56472) 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 -- 172.21.15.110:0/625480043 learned_addr learned my addr 172.21.15.110:0/625480043 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f6000640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2f00cff70 0x7fa2f01f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f4c00640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cfb90 0x7fa2f01f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 -- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cfb90 msgr2=0x7fa2f01f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cfb90 0x7fa2f01f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 -- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2f00cff70 msgr2=0x7fa2f01f9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2f00cff70 0x7fa2f01f9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 -- 172.21.15.110:0/625480043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f0150ff0 con 0x7fa2f00b3f40 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f4c00640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cfb90 0x7fa2f01f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f5600640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00b3f40 0x7fa2f01508f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40ae040 tx=0x7fa2e4019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2ed600640 1 -- 172.21.15.110:0/625480043 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40b0040 con 0x7fa2f00b3f40 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2ed600640 1 -- 172.21.15.110:0/625480043 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e40b9070 con 0x7fa2f00b3f40 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f01512d0 con 0x7fa2f00b3f40 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2ed600640 1 -- 172.21.15.110:0/625480043 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40b4860 con 0x7fa2f00b3f40 2024-09-10T09:58:50.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f0151890 con 0x7fa2f00b3f40 2024-09-10T09:58:50.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f015ca10 con 0x7fa2f00b3f40 2024-09-10T09:58:50.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2ed600640 1 -- 172.21.15.110:0/625480043 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa2e40ab4a0 con 0x7fa2f00b3f40 2024-09-10T09:58:50.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2ed600640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2d00678c0 0x7fa2d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:50.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.864+0000 7fa2ed600640 1 -- 172.21.15.110:0/625480043 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa2e411f9d0 con 0x7fa2f00b3f40 2024-09-10T09:58:50.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.868+0000 7fa2f4c00640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2d00678c0 0x7fa2d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:50.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.868+0000 7fa2f4c00640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2d00678c0 0x7fa2d0069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00b5bc0 tx=0x7fa2dc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:50.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:50.872+0000 7fa2ed600640 1 -- 172.21.15.110:0/625480043 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa2e4016110 con 0x7fa2f00b3f40 2024-09-10T09:58:51.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2f01f7b30 con 0x7fa2f00b3f40 2024-09-10T09:58:51.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2ed600640 1 -- 172.21.15.110:0/625480043 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fa2e40f0620 con 0x7fa2f00b3f40 2024-09-10T09:58:51.041 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2d00678c0 msgr2=0x7fa2d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2d00678c0 0x7fa2d0069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00b5bc0 tx=0x7fa2dc008040 comp rx=0 tx=0).stop 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00b3f40 msgr2=0x7fa2f01508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00b3f40 0x7fa2f01508f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40ae040 tx=0x7fa2e4019230 comp rx=0 tx=0).stop 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 shutdown_connections 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2d00678c0 0x7fa2d0069d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2f00cff70 0x7fa2f01f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2f00cfb90 0x7fa2f01f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 --2- 172.21.15.110:0/625480043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2f00b3f40 0x7fa2f01508f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 >> 172.21.15.110:0/625480043 conn(0x7fa2f0062d10 msgr2=0x7fa2f015db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:51.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 shutdown_connections 2024-09-10T09:58:51.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.040+0000 7fa2f8000640 1 -- 172.21.15.110:0/625480043 wait complete. 2024-09-10T09:58:51.063 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-10T09:58:51.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:51.064 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-10T09:58:51.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:58:51.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:51.086 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T09:58:51.137 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:58:51.137 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:51.146 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:58:51.146 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:51.201 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:58:51.201 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:58:51.257 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T09:58:51.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb79ba00640 1 Processor -- start 2024-09-10T09:58:51.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb79ba00640 1 -- start start 2024-09-10T09:58:51.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb79ba00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb7940b5600 0x7fb7940cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb79ba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79406efa0 0x7fb7940b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb79ba00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79406f3e0 con 0x7fb7940726e0 2024-09-10T09:58:51.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb79ba00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb7940cee50 con 0x7fb7940b5600 2024-09-10T09:58:51.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb79ba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb79415fda0 con 0x7fb79406efa0 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb799000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb7940726e0 0x7fb7940b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57788/0 (socket says 172.21.15.110:57788) 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb799000640 1 -- 172.21.15.110:0/584164208 learned_addr learned my addr 172.21.15.110:0/584164208 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb793e00640 1 --2- 172.21.15.110:0/584164208 >> v2:172.21.15.110:3300/0 conn(0x7fb7940b5600 0x7fb7940cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb799a00640 1 --2- 172.21.15.110:0/584164208 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79406efa0 0x7fb7940b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb793400640 1 -- 172.21.15.110:0/584164208 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907566198 0 0) 0x7fb79406f3e0 con 0x7fb7940726e0 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb793400640 1 -- 172.21.15.110:0/584164208 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb778005740 con 0x7fb7940726e0 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb793e00640 1 -- 172.21.15.110:0/584164208 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79406efa0 msgr2=0x7fb7940b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb793e00640 1 --2- 172.21.15.110:0/584164208 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79406efa0 0x7fb7940b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb793e00640 1 -- 172.21.15.110:0/584164208 >> v1:172.21.15.110:6789/0 conn(0x7fb7940726e0 legacy=0x7fb7940b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.392+0000 7fb793e00640 1 -- 172.21.15.110:0/584164208 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7940ce520 con 0x7fb7940b5600 2024-09-10T09:58:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb793e00640 1 --2- 172.21.15.110:0/584164208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940b5600 0x7fb7940cf2e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7800097c0 tx=0x7fb78001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a588a3794528e2f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb793400640 1 -- 172.21.15.110:0/584164208 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7800a7070 con 0x7fb7940b5600 2024-09-10T09:58:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb793400640 1 -- 172.21.15.110:0/584164208 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7800b0070 con 0x7fb7940b5600 2024-09-10T09:58:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb793400640 1 -- 172.21.15.110:0/584164208 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7800ab870 con 0x7fb7940b5600 2024-09-10T09:58:51.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb79ba00640 1 -- 172.21.15.110:0/584164208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940b5600 msgr2=0x7fb7940cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb79ba00640 1 --2- 172.21.15.110:0/584164208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940b5600 0x7fb7940cf2e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7800097c0 tx=0x7fb78001c470 comp rx=0 tx=0).stop 2024-09-10T09:58:51.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb79ba00640 1 -- 172.21.15.110:0/584164208 shutdown_connections 2024-09-10T09:58:51.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb79ba00640 1 --2- 172.21.15.110:0/584164208 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79406efa0 0x7fb7940b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb79ba00640 1 --2- 172.21.15.110:0/584164208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940b5600 0x7fb7940cf2e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb79ba00640 1 -- 172.21.15.110:0/584164208 >> 172.21.15.110:0/584164208 conn(0x7fb794062d10 msgr2=0x7fb79406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:51.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb79ba00640 1 -- 172.21.15.110:0/584164208 shutdown_connections 2024-09-10T09:58:51.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.396+0000 7fb79ba00640 1 -- 172.21.15.110:0/584164208 wait complete. 2024-09-10T09:58:51.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 Processor -- start 2024-09-10T09:58:51.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 -- start start 2024-09-10T09:58:51.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79406efa0 0x7fb79414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940726e0 0x7fb7940b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7940b5600 0x7fb7940b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb794072c40 con 0x7fb7940726e0 2024-09-10T09:58:51.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb794072ac0 con 0x7fb79406efa0 2024-09-10T09:58:51.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb79415f950 con 0x7fb7940b5600 2024-09-10T09:58:51.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940726e0 0x7fb7940b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940726e0 0x7fb7940b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56484/0 (socket says 172.21.15.110:56484) 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 -- 172.21.15.110:0/3914046217 learned_addr learned my addr 172.21.15.110:0/3914046217 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb799000640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79406efa0 0x7fb79414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb799a00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7940b5600 0x7fb7940b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 -- 172.21.15.110:0/3914046217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7940b5600 msgr2=0x7fb7940b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7940b5600 0x7fb7940b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 -- 172.21.15.110:0/3914046217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79406efa0 msgr2=0x7fb79414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79406efa0 0x7fb79414a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 -- 172.21.15.110:0/3914046217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79415bee0 con 0x7fb7940726e0 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb793e00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940726e0 0x7fb7940b0140 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb7800ad040 tx=0x7fb78001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb790c00640 1 -- 172.21.15.110:0/3914046217 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7800bf610 con 0x7fb7940726e0 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb790c00640 1 -- 172.21.15.110:0/3914046217 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7800abe50 con 0x7fb7940726e0 2024-09-10T09:58:51.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb790c00640 1 -- 172.21.15.110:0/3914046217 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7800bf9b0 con 0x7fb7940726e0 2024-09-10T09:58:51.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79415c1c0 con 0x7fb7940726e0 2024-09-10T09:58:51.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.400+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb794158390 con 0x7fb7940726e0 2024-09-10T09:58:51.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.404+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7940b34d0 con 0x7fb7940726e0 2024-09-10T09:58:51.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.404+0000 7fb790c00640 1 -- 172.21.15.110:0/3914046217 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7800bd020 con 0x7fb7940726e0 2024-09-10T09:58:51.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.404+0000 7fb790c00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb7740678c0 0x7fb774069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.404+0000 7fb790c00640 1 -- 172.21.15.110:0/3914046217 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb78011ffc0 con 0x7fb7940726e0 2024-09-10T09:58:51.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.404+0000 7fb799000640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb7740678c0 0x7fb774069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.404+0000 7fb799000640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb7740678c0 0x7fb774069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb784007fd0 tx=0x7fb784009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:51.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.408+0000 7fb790c00640 1 -- 172.21.15.110:0/3914046217 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb7800c6030 con 0x7fb7940726e0 2024-09-10T09:58:51.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.576+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb794153de0 con 0x7fb7940726e0 2024-09-10T09:58:51.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.576+0000 7fb790c00640 1 -- 172.21.15.110:0/3914046217 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fb780016090 con 0x7fb7940726e0 2024-09-10T09:58:51.580 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb7740678c0 msgr2=0x7fb774069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb7740678c0 0x7fb774069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb784007fd0 tx=0x7fb784009040 comp rx=0 tx=0).stop 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940726e0 msgr2=0x7fb7940b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940726e0 0x7fb7940b0140 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb7800ad040 tx=0x7fb78001acb0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 shutdown_connections 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb7740678c0 0x7fb774069d80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7940b5600 0x7fb7940b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7940726e0 0x7fb7940b0140 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 --2- 172.21.15.110:0/3914046217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79406efa0 0x7fb79414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 >> 172.21.15.110:0/3914046217 conn(0x7fb794062d10 msgr2=0x7fb794154300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:51.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 shutdown_connections 2024-09-10T09:58:51.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.580+0000 7fb79ba00640 1 -- 172.21.15.110:0/3914046217 wait complete. 2024-09-10T09:58:51.601 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-10T09:58:51.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:51.601 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-10T09:58:51.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:58:51.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:58:51.625 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T09:58:51.674 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T09:58:51.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:51.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f6e00640 1 Processor -- start 2024-09-10T09:58:51.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f6e00640 1 -- start start 2024-09-10T09:58:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f6e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f006eb30 0x7f73f00d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f6e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f73f0163000 0x7f73f0071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f6e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f73f0073290 0x7f73f0073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f6e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f73f015ee90 con 0x7f73f0163000 2024-09-10T09:58:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f6e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f73f00710d0 con 0x7f73f0073290 2024-09-10T09:58:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f6e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73f00d1370 con 0x7f73f006eb30 2024-09-10T09:58:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73ef400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f73f0163000 0x7f73f0071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73ef400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f73f0163000 0x7f73f0071560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56500/0 (socket says 172.21.15.110:56500) 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73ef400640 1 -- 172.21.15.110:0/92233167 learned_addr learned my addr 172.21.15.110:0/92233167 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73efe00640 1 --2- 172.21.15.110:0/92233167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f006eb30 0x7f73f00d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73f4e00640 1 --2- 172.21.15.110:0/92233167 >> v2:172.21.15.145:3300/0 conn(0x7f73f0073290 0x7f73f0073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73ef400640 1 -- 172.21.15.110:0/92233167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f006eb30 msgr2=0x7f73f00d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73ef400640 1 --2- 172.21.15.110:0/92233167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f006eb30 0x7f73f00d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73ef400640 1 -- 172.21.15.110:0/92233167 >> v2:172.21.15.145:3300/0 conn(0x7f73f0073290 msgr2=0x7f73f0073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73ef400640 1 --2- 172.21.15.110:0/92233167 >> v2:172.21.15.145:3300/0 conn(0x7f73f0073290 0x7f73f0073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.808+0000 7f73ef400640 1 -- 172.21.15.110:0/92233167 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f00706a0 con 0x7f73f0163000 2024-09-10T09:58:51.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73ef400640 1 --2- 172.21.15.110:0/92233167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0163000 0x7f73f0071560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73f0063650 tx=0x7f73dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2ca01a24d4eb46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:51.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73eea00640 1 -- 172.21.15.110:0/92233167 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0b0070 con 0x7f73f0163000 2024-09-10T09:58:51.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73eea00640 1 -- 172.21.15.110:0/92233167 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73dc0ab600 con 0x7f73f0163000 2024-09-10T09:58:51.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73eea00640 1 -- 172.21.15.110:0/92233167 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0ab9d0 con 0x7f73f0163000 2024-09-10T09:58:51.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 -- 172.21.15.110:0/92233167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0163000 msgr2=0x7f73f0071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 --2- 172.21.15.110:0/92233167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0163000 0x7f73f0071560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73f0063650 tx=0x7f73dc01c490 comp rx=0 tx=0).stop 2024-09-10T09:58:51.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 -- 172.21.15.110:0/92233167 shutdown_connections 2024-09-10T09:58:51.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 --2- 172.21.15.110:0/92233167 >> v2:172.21.15.145:3300/0 conn(0x7f73f0073290 0x7f73f0073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 --2- 172.21.15.110:0/92233167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0163000 0x7f73f0071560 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 --2- 172.21.15.110:0/92233167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f006eb30 0x7f73f00d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 -- 172.21.15.110:0/92233167 >> 172.21.15.110:0/92233167 conn(0x7f73f00d25c0 msgr2=0x7f73f00d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:51.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 -- 172.21.15.110:0/92233167 shutdown_connections 2024-09-10T09:58:51.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.812+0000 7f73f6e00640 1 -- 172.21.15.110:0/92233167 wait complete. 2024-09-10T09:58:51.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 Processor -- start 2024-09-10T09:58:51.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 -- start start 2024-09-10T09:58:51.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f006eb30 0x7f73f01552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0073290 0x7f73f0156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f0163000 0x7f73f015f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0071aa0 con 0x7f73f0073290 2024-09-10T09:58:51.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0163440 con 0x7f73f006eb30 2024-09-10T09:58:51.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73f0071c20 con 0x7f73f0163000 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0073290 0x7f73f0156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0073290 0x7f73f0156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56506/0 (socket says 172.21.15.110:56506) 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 -- 172.21.15.110:0/2817606108 learned_addr learned my addr 172.21.15.110:0/2817606108 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f4e00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f0163000 0x7f73f015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73efe00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f006eb30 0x7f73f01552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 -- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f0163000 msgr2=0x7f73f015f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f0163000 0x7f73f015f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 -- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f006eb30 msgr2=0x7f73f01552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f006eb30 0x7f73f01552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 -- 172.21.15.110:0/2817606108 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f015df10 con 0x7f73f0073290 2024-09-10T09:58:51.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73efe00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f006eb30 0x7f73f01552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:51.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73ef400640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0073290 0x7f73f0156ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73dc01cd30 tx=0x7f73dc019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:51.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73e7e00640 1 -- 172.21.15.110:0/2817606108 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0b0040 con 0x7f73f0073290 2024-09-10T09:58:51.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73e7e00640 1 -- 172.21.15.110:0/2817606108 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73dc0b6440 con 0x7f73f0073290 2024-09-10T09:58:51.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f015d670 con 0x7f73f0073290 2024-09-10T09:58:51.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73e7e00640 1 -- 172.21.15.110:0/2817606108 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0b67f0 con 0x7f73f0073290 2024-09-10T09:58:51.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f015dc30 con 0x7f73f0073290 2024-09-10T09:58:51.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.816+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f006dd60 con 0x7f73f0073290 2024-09-10T09:58:51.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.820+0000 7f73e7e00640 1 -- 172.21.15.110:0/2817606108 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f73dc0bd060 con 0x7f73f0073290 2024-09-10T09:58:51.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.820+0000 7f73e7e00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f73d00678c0 0x7f73d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:51.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.820+0000 7f73e7e00640 1 -- 172.21.15.110:0/2817606108 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f73dc11f570 con 0x7f73f0073290 2024-09-10T09:58:51.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.820+0000 7f73efe00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f73d00678c0 0x7f73d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:51.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.820+0000 7f73efe00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f73d00678c0 0x7f73d0069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73f0162430 tx=0x7f73e000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:51.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:51.824+0000 7f73e7e00640 1 -- 172.21.15.110:0/2817606108 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f73dc016110 con 0x7f73f0073290 2024-09-10T09:58:52.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73f00b1c40 con 0x7f73f0073290 2024-09-10T09:58:52.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73e7e00640 1 -- 172.21.15.110:0/2817606108 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5229 (secure 0 0 0) 0x7f73dc0f00a0 con 0x7f73f0073290 2024-09-10T09:58:52.022 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:52.022 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:58:22.146304+0000","modified":"2024-09-10T09:58:25.155860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-09-10T09:58:52.022 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f73d00678c0 msgr2=0x7f73d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f73d00678c0 0x7f73d0069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73f0162430 tx=0x7f73e000b040 comp rx=0 tx=0).stop 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0073290 msgr2=0x7f73f0156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0073290 0x7f73f0156ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73dc01cd30 tx=0x7f73dc019230 comp rx=0 tx=0).stop 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 shutdown_connections 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f73d00678c0 0x7f73d0069d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73f0163000 0x7f73f015f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73f0073290 0x7f73f0156ea0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 --2- 172.21.15.110:0/2817606108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73f006eb30 0x7f73f01552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 >> 172.21.15.110:0/2817606108 conn(0x7f73f00d25c0 msgr2=0x7f73f00b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 shutdown_connections 2024-09-10T09:58:52.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.020+0000 7f73f6e00640 1 -- 172.21.15.110:0/2817606108 wait complete. 2024-09-10T09:58:52.042 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:52.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa561600640 1 Processor -- start 2024-09-10T09:58:52.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa561600640 1 -- start start 2024-09-10T09:58:52.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa561600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 0x7fa55c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa561600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa55c06e7a0 0x7fa55c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa561600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa55c06d2d0 con 0x7fa55c0cd510 2024-09-10T09:58:52.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa561600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa55c0cdeb0 con 0x7fa55c06e7a0 2024-09-10T09:58:52.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa561600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c15d9a0 con 0x7fa55c0cfe30 2024-09-10T09:58:52.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa55b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa55c06e7a0 0x7fa55c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa553e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa55c0cd510 0x7fa55c06e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57798/0 (socket says 172.21.15.110:57798) 2024-09-10T09:58:52.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa553e00640 1 -- 172.21.15.110:0/4004711804 learned_addr learned my addr 172.21.15.110:0/4004711804 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:52.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa55aa00640 1 --2- 172.21.15.110:0/4004711804 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 0x7fa55c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa55b400640 1 -- 172.21.15.110:0/4004711804 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 msgr2=0x7fa55c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:52.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa55b400640 1 --2- 172.21.15.110:0/4004711804 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 0x7fa55c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa55b400640 1 -- 172.21.15.110:0/4004711804 >> v1:172.21.15.110:6789/0 conn(0x7fa55c0cd510 legacy=0x7fa55c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa55b400640 1 -- 172.21.15.110:0/4004711804 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c06d5b0 con 0x7fa55c06e7a0 2024-09-10T09:58:52.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.176+0000 7fa55a000640 1 -- 172.21.15.110:0/4004711804 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735573906 0 0) 0x7fa55c06d2d0 con 0x7fa55c0cd510 2024-09-10T09:58:52.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa55b400640 1 --2- 172.21.15.110:0/4004711804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c0cd970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa554009d90 tx=0x7fa55401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f933664e56398ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa55a000640 1 -- 172.21.15.110:0/4004711804 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5540b2070 con 0x7fa55c06e7a0 2024-09-10T09:58:52.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa55a000640 1 -- 172.21.15.110:0/4004711804 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5540a58b0 con 0x7fa55c06e7a0 2024-09-10T09:58:52.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa55a000640 1 -- 172.21.15.110:0/4004711804 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5540a5c80 con 0x7fa55c06e7a0 2024-09-10T09:58:52.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 -- 172.21.15.110:0/4004711804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 msgr2=0x7fa55c0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 --2- 172.21.15.110:0/4004711804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c0cd970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa554009d90 tx=0x7fa55401c8e0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 -- 172.21.15.110:0/4004711804 shutdown_connections 2024-09-10T09:58:52.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 --2- 172.21.15.110:0/4004711804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c0cd970 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 --2- 172.21.15.110:0/4004711804 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 0x7fa55c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 -- 172.21.15.110:0/4004711804 >> 172.21.15.110:0/4004711804 conn(0x7fa55c06bc50 msgr2=0x7fa55c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:52.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 -- 172.21.15.110:0/4004711804 shutdown_connections 2024-09-10T09:58:52.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 -- 172.21.15.110:0/4004711804 wait complete. 2024-09-10T09:58:52.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 Processor -- start 2024-09-10T09:58:52.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.180+0000 7fa561600640 1 -- start start 2024-09-10T09:58:52.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa55c0cd510 0x7fa55c151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 0x7fa55c15c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c072190 con 0x7fa55c06e7a0 2024-09-10T09:58:52.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c15d640 con 0x7fa55c0cd510 2024-09-10T09:58:52.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c169180 con 0x7fa55c0cfe30 2024-09-10T09:58:52.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56522/0 (socket says 172.21.15.110:56522) 2024-09-10T09:58:52.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 -- 172.21.15.110:0/1868284158 learned_addr learned my addr 172.21.15.110:0/1868284158 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:52.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55b400640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 0x7fa55c15c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa553e00640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa55c0cd510 0x7fa55c151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 -- 172.21.15.110:0/1868284158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 msgr2=0x7fa55c15c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 0x7fa55c15c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 -- 172.21.15.110:0/1868284158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa55c0cd510 msgr2=0x7fa55c151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa55c0cd510 0x7fa55c151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 -- 172.21.15.110:0/1868284158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c071400 con 0x7fa55c06e7a0 2024-09-10T09:58:52.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa55aa00640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c154680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00e9b0 tx=0x7fa54c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa552a00640 1 -- 172.21.15.110:0/1868284158 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c017070 con 0x7fa55c06e7a0 2024-09-10T09:58:52.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa552a00640 1 -- 172.21.15.110:0/1868284158 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c00c9a0 con 0x7fa55c06e7a0 2024-09-10T09:58:52.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa552a00640 1 -- 172.21.15.110:0/1868284158 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c00cd70 con 0x7fa55c06e7a0 2024-09-10T09:58:52.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c155370 con 0x7fa55c06e7a0 2024-09-10T09:58:52.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c155930 con 0x7fa55c06e7a0 2024-09-10T09:58:52.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55c0cd970 con 0x7fa55c06e7a0 2024-09-10T09:58:52.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.184+0000 7fa552a00640 1 -- 172.21.15.110:0/1868284158 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa54c01a070 con 0x7fa55c06e7a0 2024-09-10T09:58:52.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.188+0000 7fa552a00640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa5380678c0 0x7fa538069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.188+0000 7fa552a00640 1 -- 172.21.15.110:0/1868284158 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa54c085f90 con 0x7fa55c06e7a0 2024-09-10T09:58:52.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.188+0000 7fa553e00640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa5380678c0 0x7fa538069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.188+0000 7fa553e00640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa5380678c0 0x7fa538069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0b2ab0 tx=0x7fa544007df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.192+0000 7fa552a00640 1 -- 172.21.15.110:0/1868284158 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa54c021030 con 0x7fa55c06e7a0 2024-09-10T09:58:52.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.380+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa55c15f6f0 con 0x7fa55c06e7a0 2024-09-10T09:58:52.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.380+0000 7fa552a00640 1 -- 172.21.15.110:0/1868284158 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5229 (secure 0 0 0) 0x7fa54c0563e0 con 0x7fa55c06e7a0 2024-09-10T09:58:52.383 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:52.383 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:58:22.146304+0000","modified":"2024-09-10T09:58:25.155860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-09-10T09:58:52.383 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-09-10T09:58:52.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.380+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa5380678c0 msgr2=0x7fa538069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.380+0000 7fa561600640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa5380678c0 0x7fa538069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0b2ab0 tx=0x7fa544007df0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.380+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 msgr2=0x7fa55c154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.380+0000 7fa561600640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c154680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00e9b0 tx=0x7fa54c00ee70 comp rx=0 tx=0).stop 2024-09-10T09:58:52.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.384+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 shutdown_connections 2024-09-10T09:58:52.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.384+0000 7fa561600640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa5380678c0 0x7fa538069d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.384+0000 7fa561600640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa55c0cfe30 0x7fa55c15c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.384+0000 7fa561600640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa55c0cd510 0x7fa55c151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.384+0000 7fa561600640 1 --2- 172.21.15.110:0/1868284158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c06e7a0 0x7fa55c154680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.384+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 >> 172.21.15.110:0/1868284158 conn(0x7fa55c06bc50 msgr2=0x7fa55c152970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:52.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.384+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 shutdown_connections 2024-09-10T09:58:52.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.384+0000 7fa561600640 1 -- 172.21.15.110:0/1868284158 wait complete. 2024-09-10T09:58:52.404 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:52.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.536+0000 7fcafa000640 1 Processor -- start 2024-09-10T09:58:52.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.536+0000 7fcafa000640 1 -- start start 2024-09-10T09:58:52.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcafa000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcafa000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcaf406f010 0x7fcaf40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcafa000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf406f430 con 0x7fcaf4155970 2024-09-10T09:58:52.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcafa000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf40b4570 con 0x7fcaf406f010 2024-09-10T09:58:52.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcafa000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf4155d50 con 0x7fcaf415c560 2024-09-10T09:58:52.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf3400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcaf4155970 0x7fcaf40cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57814/0 (socket says 172.21.15.110:57814) 2024-09-10T09:58:52.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf3400640 1 -- 172.21.15.110:0/126701058 learned_addr learned my addr 172.21.15.110:0/126701058 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:52.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf3e00640 1 --2- 172.21.15.110:0/126701058 >> v2:172.21.15.110:3300/0 conn(0x7fcaf406f010 0x7fcaf40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf2a00640 1 --2- 172.21.15.110:0/126701058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf3e00640 1 -- 172.21.15.110:0/126701058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 msgr2=0x7fcaf406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:52.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf3e00640 1 --2- 172.21.15.110:0/126701058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf3e00640 1 -- 172.21.15.110:0/126701058 >> v1:172.21.15.110:6789/0 conn(0x7fcaf4155970 legacy=0x7fcaf40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf3e00640 1 -- 172.21.15.110:0/126701058 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf40b34d0 con 0x7fcaf406f010 2024-09-10T09:58:52.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf2000640 1 -- 172.21.15.110:0/126701058 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3507715906 0 0) 0x7fcaf406f430 con 0x7fcaf4155970 2024-09-10T09:58:52.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.540+0000 7fcaf2a00640 1 --2- 172.21.15.110:0/126701058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:52.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcaf3e00640 1 --2- 172.21.15.110:0/126701058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf406f010 0x7fcaf40ce420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0097c0 tx=0x7fcaec01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3348d9391d2ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcaf2000640 1 -- 172.21.15.110:0/126701058 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0b1070 con 0x7fcaf406f010 2024-09-10T09:58:52.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcaf2000640 1 -- 172.21.15.110:0/126701058 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec0a8050 con 0x7fcaf406f010 2024-09-10T09:58:52.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcaf2000640 1 -- 172.21.15.110:0/126701058 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0ac860 con 0x7fcaf406f010 2024-09-10T09:58:52.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- 172.21.15.110:0/126701058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf406f010 msgr2=0x7fcaf40ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 --2- 172.21.15.110:0/126701058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf406f010 0x7fcaf40ce420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0097c0 tx=0x7fcaec01c3e0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- 172.21.15.110:0/126701058 shutdown_connections 2024-09-10T09:58:52.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 --2- 172.21.15.110:0/126701058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf406f010 0x7fcaf40ce420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 --2- 172.21.15.110:0/126701058 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- 172.21.15.110:0/126701058 >> 172.21.15.110:0/126701058 conn(0x7fcaf406bc50 msgr2=0x7fcaf406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:52.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- 172.21.15.110:0/126701058 shutdown_connections 2024-09-10T09:58:52.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- 172.21.15.110:0/126701058 wait complete. 2024-09-10T09:58:52.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 Processor -- start 2024-09-10T09:58:52.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- start start 2024-09-10T09:58:52.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcaf406f010 0x7fcaf4157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf4155970 0x7fcaf414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf4158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40cf170 con 0x7fcaf4155970 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40b4050 con 0x7fcaf406f010 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcafa000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf40cf2f0 con 0x7fcaf415c560 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.544+0000 7fcaf2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf4155970 0x7fcaf414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf4155970 0x7fcaf414dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56534/0 (socket says 172.21.15.110:56534) 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf2a00640 1 -- 172.21.15.110:0/784500808 learned_addr learned my addr 172.21.15.110:0/784500808 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf3e00640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf4158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf3400640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcaf406f010 0x7fcaf4157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf2a00640 1 -- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 msgr2=0x7fcaf4158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf2a00640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf4158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf2a00640 1 -- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcaf406f010 msgr2=0x7fcaf4157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf2a00640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcaf406f010 0x7fcaf4157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf2a00640 1 -- 172.21.15.110:0/784500808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf4168520 con 0x7fcaf4155970 2024-09-10T09:58:52.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf3400640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcaf406f010 0x7fcaf4157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:52.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaf2a00640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf4155970 0x7fcaf414dff0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c7f0 tx=0x7fcae000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaeb400640 1 -- 172.21.15.110:0/784500808 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0015070 con 0x7fcaf4155970 2024-09-10T09:58:52.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaeb400640 1 -- 172.21.15.110:0/784500808 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae0007820 con 0x7fcaf4155970 2024-09-10T09:58:52.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaeb400640 1 -- 172.21.15.110:0/784500808 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0007c10 con 0x7fcaf4155970 2024-09-10T09:58:52.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf4168800 con 0x7fcaf4155970 2024-09-10T09:58:52.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf4158ec0 con 0x7fcaf4155970 2024-09-10T09:58:52.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf4069bb0 con 0x7fcaf4155970 2024-09-10T09:58:52.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaeb400640 1 -- 172.21.15.110:0/784500808 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcae0018070 con 0x7fcaf4155970 2024-09-10T09:58:52.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.548+0000 7fcaeb400640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcad40678c0 0x7fcad4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.552+0000 7fcaeb400640 1 -- 172.21.15.110:0/784500808 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcae0085d70 con 0x7fcaf4155970 2024-09-10T09:58:52.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.552+0000 7fcaf3400640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcad40678c0 0x7fcad4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.552+0000 7fcaf3400640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcad40678c0 0x7fcad4069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcae4005fd0 tx=0x7fcae4005b80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.556+0000 7fcaeb400640 1 -- 172.21.15.110:0/784500808 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcae0056250 con 0x7fcaf4155970 2024-09-10T09:58:52.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.712+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcaf40cfe40 con 0x7fcaf4155970 2024-09-10T09:58:52.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.712+0000 7fcaeb400640 1 -- 172.21.15.110:0/784500808 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16701 (secure 0 0 0) 0x7fcae005a550 con 0x7fcaf4155970 2024-09-10T09:58:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:52.716 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:23.766388+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:52.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcad40678c0 msgr2=0x7fcad4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcad40678c0 0x7fcad4069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcae4005fd0 tx=0x7fcae4005b80 comp rx=0 tx=0).stop 2024-09-10T09:58:52.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf4155970 msgr2=0x7fcaf414dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf4155970 0x7fcaf414dff0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c7f0 tx=0x7fcae000ccb0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 shutdown_connections 2024-09-10T09:58:52.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcad40678c0 0x7fcad4069d80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcaf415c560 0x7fcaf4158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaf4155970 0x7fcaf414dff0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 --2- 172.21.15.110:0/784500808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcaf406f010 0x7fcaf4157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 >> 172.21.15.110:0/784500808 conn(0x7fcaf406bc50 msgr2=0x7fcaf40b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:52.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 shutdown_connections 2024-09-10T09:58:52.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.716+0000 7fcafa000640 1 -- 172.21.15.110:0/784500808 wait complete. 2024-09-10T09:58:52.738 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T09:58:52.738 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:52.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb605e00640 1 Processor -- start 2024-09-10T09:58:52.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb605e00640 1 -- start start 2024-09-10T09:58:52.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb605e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb60006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb605e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb6000af710 0x7fb6000702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb605e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb60006ce00 con 0x7fb6000b6ae0 2024-09-10T09:58:52.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb605e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb6000b76a0 con 0x7fb6000af710 2024-09-10T09:58:52.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb605e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb60006da10 con 0x7fb60006f980 2024-09-10T09:58:52.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb5fe000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb6000af710 0x7fb6000702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb5fe000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb6000af710 0x7fb6000702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56542/0 (socket says 172.21.15.110:56542) 2024-09-10T09:58:52.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb5fe000640 1 -- 172.21.15.110:0/2874862695 learned_addr learned my addr 172.21.15.110:0/2874862695 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:52.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb5fea00640 1 --2- 172.21.15.110:0/2874862695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb60006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb5fe000640 1 -- 172.21.15.110:0/2874862695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 msgr2=0x7fb60006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:52.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb5fe000640 1 --2- 172.21.15.110:0/2874862695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb60006fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb5fe000640 1 -- 172.21.15.110:0/2874862695 >> v1:172.21.15.145:6789/0 conn(0x7fb6000b6ae0 legacy=0x7fb6000b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.872+0000 7fb5fe000640 1 -- 172.21.15.110:0/2874862695 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60006d0e0 con 0x7fb6000af710 2024-09-10T09:58:52.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb5fe000640 1 --2- 172.21.15.110:0/2874862695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000702a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009870 tx=0x7fb5ec01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7152cc9aa0a3f0a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb5fd600640 1 -- 172.21.15.110:0/2874862695 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0ab650 con 0x7fb6000af710 2024-09-10T09:58:52.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb5fd600640 1 -- 172.21.15.110:0/2874862695 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ec0b0070 con 0x7fb6000af710 2024-09-10T09:58:52.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb5fd600640 1 -- 172.21.15.110:0/2874862695 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0aba10 con 0x7fb6000af710 2024-09-10T09:58:52.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb605e00640 1 -- 172.21.15.110:0/2874862695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 msgr2=0x7fb6000702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb605e00640 1 --2- 172.21.15.110:0/2874862695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000702a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009870 tx=0x7fb5ec01c560 comp rx=0 tx=0).stop 2024-09-10T09:58:52.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb605e00640 1 -- 172.21.15.110:0/2874862695 shutdown_connections 2024-09-10T09:58:52.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb605e00640 1 --2- 172.21.15.110:0/2874862695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000702a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb605e00640 1 --2- 172.21.15.110:0/2874862695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb60006fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb605e00640 1 -- 172.21.15.110:0/2874862695 >> 172.21.15.110:0/2874862695 conn(0x7fb600063160 msgr2=0x7fb60009b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:52.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb605e00640 1 -- 172.21.15.110:0/2874862695 shutdown_connections 2024-09-10T09:58:52.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.876+0000 7fb605e00640 1 -- 172.21.15.110:0/2874862695 wait complete. 2024-09-10T09:58:52.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 Processor -- start 2024-09-10T09:58:52.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 -- start start 2024-09-10T09:58:52.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb600144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6000b6ae0 0x7fb6000a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb60016ae20 con 0x7fb6000af710 2024-09-10T09:58:52.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb60016aca0 con 0x7fb6000b6ae0 2024-09-10T09:58:52.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb60016afa0 con 0x7fb60006f980 2024-09-10T09:58:52.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56554/0 (socket says 172.21.15.110:56554) 2024-09-10T09:58:52.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 -- 172.21.15.110:0/1883587891 learned_addr learned my addr 172.21.15.110:0/1883587891 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:52.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fea00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb600144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5ff400640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6000b6ae0 0x7fb6000a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 -- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 msgr2=0x7fb600144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb600144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 -- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6000b6ae0 msgr2=0x7fb6000a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:52.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6000b6ae0 0x7fb6000a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:52.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 -- 172.21.15.110:0/1883587891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60015a010 con 0x7fb6000af710 2024-09-10T09:58:52.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fea00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb600144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:52.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5fe000640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000a00c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0ad040 tx=0x7fb5ec01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5f7400640 1 -- 172.21.15.110:0/1883587891 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0be5f0 con 0x7fb6000af710 2024-09-10T09:58:52.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5f7400640 1 -- 172.21.15.110:0/1883587891 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ec0be7b0 con 0x7fb6000af710 2024-09-10T09:58:52.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb5f7400640 1 -- 172.21.15.110:0/1883587891 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0beba0 con 0x7fb6000af710 2024-09-10T09:58:52.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60015a2f0 con 0x7fb6000af710 2024-09-10T09:58:52.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.880+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb600145520 con 0x7fb6000af710 2024-09-10T09:58:52.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.884+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6000b6f90 con 0x7fb6000af710 2024-09-10T09:58:52.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.884+0000 7fb5f7400640 1 -- 172.21.15.110:0/1883587891 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5ec0c9430 con 0x7fb6000af710 2024-09-10T09:58:52.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.884+0000 7fb5f7400640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb5e00678c0 0x7fb5e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:52.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.884+0000 7fb5f7400640 1 -- 172.21.15.110:0/1883587891 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb5ec1206e0 con 0x7fb6000af710 2024-09-10T09:58:52.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.884+0000 7fb5fea00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb5e00678c0 0x7fb5e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:52.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.884+0000 7fb5fea00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb5e00678c0 0x7fb5e0069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009b10 tx=0x7fb5f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:52.888+0000 7fb5f7400640 1 -- 172.21.15.110:0/1883587891 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb5ec016150 con 0x7fb6000af710 2024-09-10T09:58:53.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.076+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb60015a4c0 con 0x7fb6000af710 2024-09-10T09:58:53.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.076+0000 7fb5f7400640 1 -- 172.21.15.110:0/1883587891 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5229 (secure 0 0 0) 0x7fb5ec0f0cb0 con 0x7fb6000af710 2024-09-10T09:58:53.079 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:53.079 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:58:22.146304+0000","modified":"2024-09-10T09:58:25.155860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-09-10T09:58:53.080 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-09-10T09:58:53.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb5e00678c0 msgr2=0x7fb5e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb5e00678c0 0x7fb5e0069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009b10 tx=0x7fb5f0008040 comp rx=0 tx=0).stop 2024-09-10T09:58:53.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 msgr2=0x7fb6000a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000a00c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0ad040 tx=0x7fb5ec01ada0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 shutdown_connections 2024-09-10T09:58:53.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb5e00678c0 0x7fb5e0069d80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6000b6ae0 0x7fb6000a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6000af710 0x7fb6000a00c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 --2- 172.21.15.110:0/1883587891 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb60006f980 0x7fb600144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 >> 172.21.15.110:0/1883587891 conn(0x7fb600063160 msgr2=0x7fb6000b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:53.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 shutdown_connections 2024-09-10T09:58:53.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.080+0000 7fb605e00640 1 -- 172.21.15.110:0/1883587891 wait complete. 2024-09-10T09:58:53.100 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:58:53.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.232+0000 7f6e87400640 1 Processor -- start 2024-09-10T09:58:53.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.232+0000 7f6e87400640 1 -- start start 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e87400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e80155970 0x7f6e800cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e87400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6e8015c560 0x7f6e8006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e87400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6e8006f010 0x7f6e8015eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e87400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6e800b4570 con 0x7f6e8015c560 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e87400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6e8006f450 con 0x7f6e8006f010 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e87400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6e80155d50 con 0x7f6e80155970 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e7fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6e8015c560 0x7f6e8006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e7fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6e8015c560 0x7f6e8006ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56566/0 (socket says 172.21.15.110:56566) 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e7fe00640 1 -- 172.21.15.110:0/3651705096 learned_addr learned my addr 172.21.15.110:0/3651705096 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:53.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e85400640 1 --2- 172.21.15.110:0/3651705096 >> v2:172.21.15.145:3300/0 conn(0x7f6e8006f010 0x7f6e8015eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e84a00640 1 --2- 172.21.15.110:0/3651705096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e80155970 0x7f6e800cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e7fe00640 1 -- 172.21.15.110:0/3651705096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e80155970 msgr2=0x7f6e800cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:53.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e7fe00640 1 --2- 172.21.15.110:0/3651705096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e80155970 0x7f6e800cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e7fe00640 1 -- 172.21.15.110:0/3651705096 >> v2:172.21.15.145:3300/0 conn(0x7f6e8006f010 msgr2=0x7f6e8015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:53.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e7fe00640 1 --2- 172.21.15.110:0/3651705096 >> v2:172.21.15.145:3300/0 conn(0x7f6e8006f010 0x7f6e8015eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.236+0000 7f6e7fe00640 1 -- 172.21.15.110:0/3651705096 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e800b34d0 con 0x7f6e8015c560 2024-09-10T09:58:53.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e7fe00640 1 --2- 172.21.15.110:0/3651705096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e8015c560 0x7f6e8006ead0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e8006d4d0 tx=0x7f6e6c01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae83b79dd926595 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e7f400640 1 -- 172.21.15.110:0/3651705096 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0b0070 con 0x7f6e8015c560 2024-09-10T09:58:53.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e7f400640 1 -- 172.21.15.110:0/3651705096 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e6c0ab600 con 0x7f6e8015c560 2024-09-10T09:58:53.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e7f400640 1 -- 172.21.15.110:0/3651705096 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0ab9d0 con 0x7f6e8015c560 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- 172.21.15.110:0/3651705096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e8015c560 msgr2=0x7f6e8006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 --2- 172.21.15.110:0/3651705096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e8015c560 0x7f6e8006ead0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e8006d4d0 tx=0x7f6e6c01c1a0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- 172.21.15.110:0/3651705096 shutdown_connections 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 --2- 172.21.15.110:0/3651705096 >> v2:172.21.15.145:3300/0 conn(0x7f6e8006f010 0x7f6e8015eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 --2- 172.21.15.110:0/3651705096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e8015c560 0x7f6e8006ead0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 --2- 172.21.15.110:0/3651705096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e80155970 0x7f6e800cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- 172.21.15.110:0/3651705096 >> 172.21.15.110:0/3651705096 conn(0x7f6e8006bc50 msgr2=0x7f6e8006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- 172.21.15.110:0/3651705096 shutdown_connections 2024-09-10T09:58:53.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- 172.21.15.110:0/3651705096 wait complete. 2024-09-10T09:58:53.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 Processor -- start 2024-09-10T09:58:53.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- start start 2024-09-10T09:58:53.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e8006f010 0x7f6e80150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e80155970 0x7f6e801648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e8015c560 0x7f6e800ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e800cce00 con 0x7f6e80155970 2024-09-10T09:58:53.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e800b4050 con 0x7f6e8015c560 2024-09-10T09:58:53.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.240+0000 7f6e87400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8006d9c0 con 0x7f6e8006f010 2024-09-10T09:58:53.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e80155970 0x7f6e801648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e80155970 0x7f6e801648f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56580/0 (socket says 172.21.15.110:56580) 2024-09-10T09:58:53.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 -- 172.21.15.110:0/2867078560 learned_addr learned my addr 172.21.15.110:0/2867078560 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:53.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e85400640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e8015c560 0x7f6e800ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e84a00640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e8006f010 0x7f6e80150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 -- 172.21.15.110:0/2867078560 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e8006f010 msgr2=0x7f6e80150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e8006f010 0x7f6e80150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 -- 172.21.15.110:0/2867078560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e8015c560 msgr2=0x7f6e800ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e8015c560 0x7f6e800ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 -- 172.21.15.110:0/2867078560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e80068e80 con 0x7f6e80155970 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7fe00640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e80155970 0x7f6e801648f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c01a850 tx=0x7f6e6c018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7cc00640 1 -- 172.21.15.110:0/2867078560 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0b0040 con 0x7f6e80155970 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7cc00640 1 -- 172.21.15.110:0/2867078560 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e6c0b4620 con 0x7f6e80155970 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7cc00640 1 -- 172.21.15.110:0/2867078560 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0b4a10 con 0x7f6e80155970 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e80153cf0 con 0x7f6e80155970 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e801542b0 con 0x7f6e80155970 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8015eea0 con 0x7f6e80155970 2024-09-10T09:58:53.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7cc00640 1 -- 172.21.15.110:0/2867078560 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6e6c0ab4c0 con 0x7f6e80155970 2024-09-10T09:58:53.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7cc00640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6e600678c0 0x7f6e60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.244+0000 7f6e7cc00640 1 -- 172.21.15.110:0/2867078560 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6e6c11f9f0 con 0x7f6e80155970 2024-09-10T09:58:53.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.248+0000 7f6e84a00640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6e600678c0 0x7f6e60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.248+0000 7f6e84a00640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6e600678c0 0x7f6e60069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e800af440 tx=0x7f6e74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.252+0000 7f6e7cc00640 1 -- 172.21.15.110:0/2867078560 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6e6c016110 con 0x7f6e80155970 2024-09-10T09:58:53.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.440+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e800ce8e0 con 0x7f6e80155970 2024-09-10T09:58:53.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.440+0000 7f6e7cc00640 1 -- 172.21.15.110:0/2867078560 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5229 (secure 0 0 0) 0x7f6e6c0ba2b0 con 0x7f6e80155970 2024-09-10T09:58:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:53.445 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"btime":"2024-09-10T09:58:25:774533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1070311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1070311334},{"type":"v1","addr":"172.21.15.145:6835","nonce":1070311334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/3939816490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":3939816490},{"type":"v1","addr":"172.21.15.110:6836","nonce":3939816490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2395818459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2395818459},{"type":"v1","addr":"172.21.15.110:6837","nonce":2395818459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:58:22.146304+0000","modified":"2024-09-10T09:58:25.155860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6834/2931287411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2931287411},{"type":"v1","addr":"172.21.15.145:6834","nonce":2931287411}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-09-10T09:58:53.445 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-09-10T09:58:53.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6e600678c0 msgr2=0x7f6e60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6e600678c0 0x7f6e60069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e800af440 tx=0x7f6e74008040 comp rx=0 tx=0).stop 2024-09-10T09:58:53.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e80155970 msgr2=0x7f6e801648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e80155970 0x7f6e801648f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c01a850 tx=0x7f6e6c018f40 comp rx=0 tx=0).stop 2024-09-10T09:58:53.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 shutdown_connections 2024-09-10T09:58:53.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6e600678c0 0x7f6e60069d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e8015c560 0x7f6e800ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e80155970 0x7f6e801648f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 --2- 172.21.15.110:0/2867078560 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e8006f010 0x7f6e80150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 >> 172.21.15.110:0/2867078560 conn(0x7f6e8006bc50 msgr2=0x7f6e800cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:53.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 shutdown_connections 2024-09-10T09:58:53.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.444+0000 7f6e87400640 1 -- 172.21.15.110:0/2867078560 wait complete. 2024-09-10T09:58:53.465 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:58:53.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.596+0000 7fd422a00640 1 Processor -- start 2024-09-10T09:58:53.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd422a00640 1 -- start start 2024-09-10T09:58:53.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd422a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd41c0cfe30 0x7fd41c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd422a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd41c06e7a0 0x7fd41c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd422a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd41c06d2d0 con 0x7fd41c0cd510 2024-09-10T09:58:53.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd422a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd41c15d9a0 con 0x7fd41c0cfe30 2024-09-10T09:58:53.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd422a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd41c06d5b0 con 0x7fd41c06e7a0 2024-09-10T09:58:53.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd41be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd41c0cfe30 0x7fd41c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd41be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd41c0cfe30 0x7fd41c071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56596/0 (socket says 172.21.15.110:56596) 2024-09-10T09:58:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd41be00640 1 -- 172.21.15.110:0/428532032 learned_addr learned my addr 172.21.15.110:0/428532032 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd420a00640 1 --2- 172.21.15.110:0/428532032 >> v2:172.21.15.145:3300/0 conn(0x7fd41c06e7a0 0x7fd41c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd41be00640 1 -- 172.21.15.110:0/428532032 >> v2:172.21.15.145:3300/0 conn(0x7fd41c06e7a0 msgr2=0x7fd41c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd41be00640 1 --2- 172.21.15.110:0/428532032 >> v2:172.21.15.145:3300/0 conn(0x7fd41c06e7a0 0x7fd41c0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd41be00640 1 -- 172.21.15.110:0/428532032 >> v1:172.21.15.145:6789/0 conn(0x7fd41c0cd510 legacy=0x7fd41c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.600+0000 7fd41be00640 1 -- 172.21.15.110:0/428532032 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c06d890 con 0x7fd41c0cfe30 2024-09-10T09:58:53.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd41be00640 1 --2- 172.21.15.110:0/428532032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cfe30 0x7fd41c071a40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd41c06cb00 tx=0x7fd40c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aeff535f3fe897 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd41b400640 1 -- 172.21.15.110:0/428532032 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0b1070 con 0x7fd41c0cfe30 2024-09-10T09:58:53.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd41b400640 1 -- 172.21.15.110:0/428532032 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40c0ac600 con 0x7fd41c0cfe30 2024-09-10T09:58:53.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd41b400640 1 -- 172.21.15.110:0/428532032 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0ac9d0 con 0x7fd41c0cfe30 2024-09-10T09:58:53.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 -- 172.21.15.110:0/428532032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cfe30 msgr2=0x7fd41c071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 --2- 172.21.15.110:0/428532032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cfe30 0x7fd41c071a40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd41c06cb00 tx=0x7fd40c01c490 comp rx=0 tx=0).stop 2024-09-10T09:58:53.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 -- 172.21.15.110:0/428532032 shutdown_connections 2024-09-10T09:58:53.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 --2- 172.21.15.110:0/428532032 >> v2:172.21.15.145:3300/0 conn(0x7fd41c06e7a0 0x7fd41c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 --2- 172.21.15.110:0/428532032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cfe30 0x7fd41c071a40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 -- 172.21.15.110:0/428532032 >> 172.21.15.110:0/428532032 conn(0x7fd41c06bc50 msgr2=0x7fd41c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:53.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 -- 172.21.15.110:0/428532032 shutdown_connections 2024-09-10T09:58:53.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 -- 172.21.15.110:0/428532032 wait complete. 2024-09-10T09:58:53.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 Processor -- start 2024-09-10T09:58:53.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.604+0000 7fd422a00640 1 -- start start 2024-09-10T09:58:53.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd41c06e7a0 0x7fd41c1491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cd510 0x7fd41c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd41c0cfe30 0x7fd41c0af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c072190 con 0x7fd41c0cd510 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c15d6c0 con 0x7fd41c0cfe30 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd41c169180 con 0x7fd41c06e7a0 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cd510 0x7fd41c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cd510 0x7fd41c0b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56602/0 (socket says 172.21.15.110:56602) 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 -- 172.21.15.110:0/1081453275 learned_addr learned my addr 172.21.15.110:0/1081453275 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd41be00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd41c06e7a0 0x7fd41c1491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd420a00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd41c0cfe30 0x7fd41c0af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 -- 172.21.15.110:0/1081453275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd41c06e7a0 msgr2=0x7fd41c1491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd41c06e7a0 0x7fd41c1491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 -- 172.21.15.110:0/1081453275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd41c0cfe30 msgr2=0x7fd41c0af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd41c0cfe30 0x7fd41c0af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 -- 172.21.15.110:0/1081453275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c149dd0 con 0x7fd41c0cd510 2024-09-10T09:58:53.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd413e00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cd510 0x7fd41c0b20b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c370 tx=0x7fd40400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd418c00640 1 -- 172.21.15.110:0/1081453275 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040052d0 con 0x7fd41c0cd510 2024-09-10T09:58:53.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd418c00640 1 -- 172.21.15.110:0/1081453275 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd404005470 con 0x7fd41c0cd510 2024-09-10T09:58:53.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c14a0b0 con 0x7fd41c0cd510 2024-09-10T09:58:53.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd418c00640 1 -- 172.21.15.110:0/1081453275 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404007260 con 0x7fd41c0cd510 2024-09-10T09:58:53.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c14a670 con 0x7fd41c0cd510 2024-09-10T09:58:53.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41c06e190 con 0x7fd41c0cd510 2024-09-10T09:58:53.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.608+0000 7fd418c00640 1 -- 172.21.15.110:0/1081453275 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd404007400 con 0x7fd41c0cd510 2024-09-10T09:58:53.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.612+0000 7fd418c00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd3f80678c0 0x7fd3f8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.612+0000 7fd418c00640 1 -- 172.21.15.110:0/1081453275 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd404085070 con 0x7fd41c0cd510 2024-09-10T09:58:53.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.612+0000 7fd41be00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd3f80678c0 0x7fd3f8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.612+0000 7fd41be00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd3f80678c0 0x7fd3f8069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0ae000 tx=0x7fd40c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.616+0000 7fd418c00640 1 -- 172.21.15.110:0/1081453275 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd404055550 con 0x7fd41c0cd510 2024-09-10T09:58:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.772+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd41c15f710 con 0x7fd41c0cd510 2024-09-10T09:58:53.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd418c00640 1 -- 172.21.15.110:0/1081453275 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16701 (secure 0 0 0) 0x7fd404059850 con 0x7fd41c0cd510 2024-09-10T09:58:53.777 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:53.777 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:58:23.766388+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:58:20.077745+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:58:21.140873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:58:53.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd3f80678c0 msgr2=0x7fd3f8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd3f80678c0 0x7fd3f8069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0ae000 tx=0x7fd40c0a7000 comp rx=0 tx=0).stop 2024-09-10T09:58:53.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cd510 msgr2=0x7fd41c0b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cd510 0x7fd41c0b20b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c370 tx=0x7fd40400c830 comp rx=0 tx=0).stop 2024-09-10T09:58:53.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 shutdown_connections 2024-09-10T09:58:53.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd3f80678c0 0x7fd3f8069d80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd41c0cfe30 0x7fd41c0af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd41c0cd510 0x7fd41c0b20b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 --2- 172.21.15.110:0/1081453275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd41c06e7a0 0x7fd41c1491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 >> 172.21.15.110:0/1081453275 conn(0x7fd41c06bc50 msgr2=0x7fd41c15bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 shutdown_connections 2024-09-10T09:58:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.776+0000 7fd422a00640 1 -- 172.21.15.110:0/1081453275 wait complete. 2024-09-10T09:58:53.799 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T09:58:53.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.932+0000 7f7484e00640 1 Processor -- start 2024-09-10T09:58:53.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.932+0000 7f7484e00640 1 -- start start 2024-09-10T09:58:53.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f7484e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f748015e450 0x7f7480071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f7484e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f74800d2ec0 0x7f74800b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f7484e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7480073380 con 0x7f74800b7010 2024-09-10T09:58:53.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f7484e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f7480074140 con 0x7f74800d2ec0 2024-09-10T09:58:53.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f7484e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f74800b7900 con 0x7f748015e450 2024-09-10T09:58:53.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f74800d2ec0 0x7f74800b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747fe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f74800b7010 0x7f7480162060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57816/0 (socket says 172.21.15.110:57816) 2024-09-10T09:58:53.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747fe00640 1 -- 172.21.15.110:0/4294135999 learned_addr learned my addr 172.21.15.110:0/4294135999 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:53.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747f400640 1 --2- 172.21.15.110:0/4294135999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f748015e450 0x7f7480071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747ea00640 1 -- 172.21.15.110:0/4294135999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f748015e450 msgr2=0x7f7480071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:53.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747ea00640 1 --2- 172.21.15.110:0/4294135999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f748015e450 0x7f7480071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747ea00640 1 -- 172.21.15.110:0/4294135999 >> v1:172.21.15.110:6789/0 conn(0x7f74800b7010 legacy=0x7f7480162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747ea00640 1 -- 172.21.15.110:0/4294135999 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7480073660 con 0x7f74800d2ec0 2024-09-10T09:58:53.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.936+0000 7f747f400640 1 --2- 172.21.15.110:0/4294135999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f748015e450 0x7f7480071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f747ea00640 1 --2- 172.21.15.110:0/4294135999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800d2ec0 0x7f74800b6ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f746c0097c0 tx=0x7f746c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e36a258f0e0426f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f747e000640 1 -- 172.21.15.110:0/4294135999 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c0b0070 con 0x7f74800d2ec0 2024-09-10T09:58:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f747e000640 1 -- 172.21.15.110:0/4294135999 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746c0ab600 con 0x7f74800d2ec0 2024-09-10T09:58:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f747e000640 1 -- 172.21.15.110:0/4294135999 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c0ab9d0 con 0x7f74800d2ec0 2024-09-10T09:58:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 -- 172.21.15.110:0/4294135999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800d2ec0 msgr2=0x7f74800b6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 --2- 172.21.15.110:0/4294135999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800d2ec0 0x7f74800b6ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f746c0097c0 tx=0x7f746c01c4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 -- 172.21.15.110:0/4294135999 shutdown_connections 2024-09-10T09:58:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 --2- 172.21.15.110:0/4294135999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800d2ec0 0x7f74800b6ad0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 --2- 172.21.15.110:0/4294135999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f748015e450 0x7f7480071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 -- 172.21.15.110:0/4294135999 >> 172.21.15.110:0/4294135999 conn(0x7f74800d23a0 msgr2=0x7f74800d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:53.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 -- 172.21.15.110:0/4294135999 shutdown_connections 2024-09-10T09:58:53.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 -- 172.21.15.110:0/4294135999 wait complete. 2024-09-10T09:58:53.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 Processor -- start 2024-09-10T09:58:53.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.940+0000 7f7484e00640 1 -- start start 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800b7010 0x7f74801585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f74800d2ec0 0x7f74801f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f748015e450 0x7f74800b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f748006dd70 con 0x7f74800b7010 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f74800b7440 con 0x7f748015e450 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f748006def0 con 0x7f74800d2ec0 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800b7010 0x7f74801585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800b7010 0x7f74801585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56624/0 (socket says 172.21.15.110:56624) 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 -- 172.21.15.110:0/3783924901 learned_addr learned my addr 172.21.15.110:0/3783924901 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:53.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747ea00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f74800d2ec0 0x7f74801f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747fe00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f748015e450 0x7f74800b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 -- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f74800d2ec0 msgr2=0x7f74801f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f74800d2ec0 0x7f74801f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 -- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f748015e450 msgr2=0x7f74800b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f748015e450 0x7f74800b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 -- 172.21.15.110:0/3783924901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7480158b90 con 0x7f74800b7010 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747fe00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f748015e450 0x7f74800b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f747f400640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800b7010 0x7f74801585b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f747000ce00 tx=0x7f747000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7477400640 1 -- 172.21.15.110:0/3783924901 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747000f890 con 0x7f74800b7010 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7477400640 1 -- 172.21.15.110:0/3783924901 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747000fa30 con 0x7f74800b7010 2024-09-10T09:58:53.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74801f7b60 con 0x7f74800b7010 2024-09-10T09:58:53.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7477400640 1 -- 172.21.15.110:0/3783924901 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747000fe40 con 0x7f74800b7010 2024-09-10T09:58:53.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74801f7810 con 0x7f74800b7010 2024-09-10T09:58:53.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74800b6ad0 con 0x7f74800b7010 2024-09-10T09:58:53.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.944+0000 7f7477400640 1 -- 172.21.15.110:0/3783924901 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7470018400 con 0x7f74800b7010 2024-09-10T09:58:53.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.948+0000 7f7477400640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:53.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.948+0000 7f7477400640 1 -- 172.21.15.110:0/3783924901 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7470086a60 con 0x7f74800b7010 2024-09-10T09:58:53.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.948+0000 7f747ea00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:53.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.948+0000 7f747ea00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f746c0ad000 tx=0x7f746c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:53.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:53.952+0000 7f7477400640 1 -- 172.21.15.110:0/3783924901 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7470014030 con 0x7f74800b7010 2024-09-10T09:58:54.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.140+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f748015c490 con 0x7f74800b7010 2024-09-10T09:58:54.144 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T09:58:54.140+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T09:58:54.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.144+0000 7f7477400640 1 -- 172.21.15.110:0/3783924901 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f747000ae20 con 0x7f74800b7010 2024-09-10T09:58:54.146 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T09:58:54.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 msgr2=0x7f7460069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f746c0ad000 tx=0x7f746c008040 comp rx=0 tx=0).stop 2024-09-10T09:58:54.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800b7010 msgr2=0x7f74801585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800b7010 0x7f74801585b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f747000ce00 tx=0x7f747000a650 comp rx=0 tx=0).stop 2024-09-10T09:58:54.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 shutdown_connections 2024-09-10T09:58:54.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f74600678c0 0x7f7460069d80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f748015e450 0x7f74800b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f74800d2ec0 0x7f74801f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 --2- 172.21.15.110:0/3783924901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f74800b7010 0x7f74801585b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 >> 172.21.15.110:0/3783924901 conn(0x7f74800d23a0 msgr2=0x7f7480153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:54.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 shutdown_connections 2024-09-10T09:58:54.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.148+0000 7f7484e00640 1 -- 172.21.15.110:0/3783924901 wait complete. 2024-09-10T09:58:54.162 INFO:tasks.ceph.mds.b.smithi145.stderr: -6696> 2024-09-10T09:58:27.738+0000 7fe673a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T09:58:54.169 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T09:58:54.215 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T09:58:54.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.304+0000 7f4171600640 1 Processor -- start 2024-09-10T09:58:54.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f4171600640 1 -- start start 2024-09-10T09:58:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f4171600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 0x7f416c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f4171600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f416c0ce4d0 0x7f416c06e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f4171600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f416c0b1c60 0x7f416c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f4171600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f416c0b5570 con 0x7f416c0b1c60 2024-09-10T09:58:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f4171600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f416c0b4960 con 0x7f416c0ce4d0 2024-09-10T09:58:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f4171600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f416c0b4c40 con 0x7f416c15b330 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f416c0b1c60 0x7f416c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f416c0b1c60 0x7f416c06e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56626/0 (socket says 172.21.15.110:56626) 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416b400640 1 -- 172.21.15.110:0/3761973656 learned_addr learned my addr 172.21.15.110:0/3761973656 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416a000640 1 --2- 172.21.15.110:0/3761973656 >> v2:172.21.15.145:3300/0 conn(0x7f416c0ce4d0 0x7f416c06e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416aa00640 1 --2- 172.21.15.110:0/3761973656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 0x7f416c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416b400640 1 -- 172.21.15.110:0/3761973656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 msgr2=0x7f416c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416b400640 1 --2- 172.21.15.110:0/3761973656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 0x7f416c06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416b400640 1 -- 172.21.15.110:0/3761973656 >> v2:172.21.15.145:3300/0 conn(0x7f416c0ce4d0 msgr2=0x7f416c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416b400640 1 --2- 172.21.15.110:0/3761973656 >> v2:172.21.15.145:3300/0 conn(0x7f416c0ce4d0 0x7f416c06e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.308+0000 7f416b400640 1 -- 172.21.15.110:0/3761973656 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c0b4f20 con 0x7f416c0b1c60 2024-09-10T09:58:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f416b400640 1 --2- 172.21.15.110:0/3761973656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c06e680 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f416c06f070 tx=0x7f416401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95da96b3ac50114a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4169600640 1 -- 172.21.15.110:0/3761973656 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640b4070 con 0x7f416c0b1c60 2024-09-10T09:58:54.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4169600640 1 -- 172.21.15.110:0/3761973656 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41640a7b90 con 0x7f416c0b1c60 2024-09-10T09:58:54.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4169600640 1 -- 172.21.15.110:0/3761973656 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640af440 con 0x7f416c0b1c60 2024-09-10T09:58:54.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 -- 172.21.15.110:0/3761973656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 msgr2=0x7f416c06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 --2- 172.21.15.110:0/3761973656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c06e680 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f416c06f070 tx=0x7f416401ebf0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 -- 172.21.15.110:0/3761973656 shutdown_connections 2024-09-10T09:58:54.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 --2- 172.21.15.110:0/3761973656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c06e680 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 --2- 172.21.15.110:0/3761973656 >> v2:172.21.15.145:3300/0 conn(0x7f416c0ce4d0 0x7f416c06e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 --2- 172.21.15.110:0/3761973656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 0x7f416c06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 -- 172.21.15.110:0/3761973656 >> 172.21.15.110:0/3761973656 conn(0x7f416c062e40 msgr2=0x7f416c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:54.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 -- 172.21.15.110:0/3761973656 shutdown_connections 2024-09-10T09:58:54.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 -- 172.21.15.110:0/3761973656 wait complete. 2024-09-10T09:58:54.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 Processor -- start 2024-09-10T09:58:54.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.312+0000 7f4171600640 1 -- start start 2024-09-10T09:58:54.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 0x7f416c15c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416c0722e0 0x7f416c072790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f416c15fae0 con 0x7f416c0b1c60 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f416c15f960 con 0x7f416c0722e0 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f416c15fc60 con 0x7f416c15b330 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56630/0 (socket says 172.21.15.110:56630) 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 -- 172.21.15.110:0/1927313243 learned_addr learned my addr 172.21.15.110:0/1927313243 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416b400640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416c0722e0 0x7f416c072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416a000640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 0x7f416c15c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 -- 172.21.15.110:0/1927313243 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 msgr2=0x7f416c15c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 0x7f416c15c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 -- 172.21.15.110:0/1927313243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416c0722e0 msgr2=0x7f416c072790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416c0722e0 0x7f416c072790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 -- 172.21.15.110:0/1927313243 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1f73e0 con 0x7f416c0b1c60 2024-09-10T09:58:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f416aa00640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c144f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f415c00efc0 tx=0x7f415c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4162a00640 1 -- 172.21.15.110:0/1927313243 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c018020 con 0x7f416c0b1c60 2024-09-10T09:58:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1f76a0 con 0x7f416c0b1c60 2024-09-10T09:58:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4162a00640 1 -- 172.21.15.110:0/1927313243 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f415c014480 con 0x7f416c0b1c60 2024-09-10T09:58:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4162a00640 1 -- 172.21.15.110:0/1927313243 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c014870 con 0x7f416c0b1c60 2024-09-10T09:58:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1f7c60 con 0x7f416c0b1c60 2024-09-10T09:58:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c1f8320 con 0x7f416c0b1c60 2024-09-10T09:58:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.316+0000 7f4162a00640 1 -- 172.21.15.110:0/1927313243 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f415c01b020 con 0x7f416c0b1c60 2024-09-10T09:58:54.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.320+0000 7f4162a00640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f414c0678c0 0x7f414c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.320+0000 7f4162a00640 1 -- 172.21.15.110:0/1927313243 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5795+0+0 (secure 0 0 0) 0x7f415c086970 con 0x7f416c0b1c60 2024-09-10T09:58:54.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.320+0000 7f416a000640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f414c0678c0 0x7f414c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.320+0000 7f416a000640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f414c0678c0 0x7f414c069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f416c15ad50 tx=0x7f4158009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:54.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.324+0000 7f4162a00640 1 -- 172.21.15.110:0/1927313243 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f416c1f8320 con 0x7f416c0b1c60 2024-09-10T09:58:54.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:54.516+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f416c06c6a0 con 0x7f416c0b1c60 2024-09-10T09:58:55.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.144+0000 7f4162a00640 1 -- 172.21.15.110:0/1927313243 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f415c022030 con 0x7f416c0b1c60 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f414c0678c0 msgr2=0x7f414c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f414c0678c0 0x7f414c069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f416c15ad50 tx=0x7f4158009190 comp rx=0 tx=0).stop 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 msgr2=0x7f416c144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c144f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f415c00efc0 tx=0x7f415c00c490 comp rx=0 tx=0).stop 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 shutdown_connections 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f414c0678c0 0x7f414c069d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416c0722e0 0x7f416c072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f416c15b330 0x7f416c15c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 --2- 172.21.15.110:0/1927313243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c0b1c60 0x7f416c144f20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 >> 172.21.15.110:0/1927313243 conn(0x7f416c062e40 msgr2=0x7f416c154840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:55.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 shutdown_connections 2024-09-10T09:58:55.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.148+0000 7f4171600640 1 -- 172.21.15.110:0/1927313243 wait complete. 2024-09-10T09:58:55.171 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T09:58:55.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.308+0000 7f46ff600640 1 Processor -- start 2024-09-10T09:58:55.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.308+0000 7f46ff600640 1 -- start start 2024-09-10T09:58:55.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.308+0000 7f46ff600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8072130 0x7f46f8072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:55.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.308+0000 7f46ff600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f80ccf40 con 0x7f46f8140520 2024-09-10T09:58:55.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.308+0000 7f46ff600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f81412a0 con 0x7f46f80ccb60 2024-09-10T09:58:55.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.308+0000 7f46ff600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f46f8072ad0 con 0x7f46f8072130 2024-09-10T09:58:55.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46fd600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f46f8140520 0x7f46f81409b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57824/0 (socket says 172.21.15.110:57824) 2024-09-10T09:58:55.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46fd600640 1 -- 172.21.15.110:0/2805059949 learned_addr learned my addr 172.21.15.110:0/2805059949 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:55.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7e00640 1 --2- 172.21.15.110:0/2805059949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8072130 0x7f46f8072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:55.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7400640 1 -- 172.21.15.110:0/2805059949 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 963116963 0 0) 0x7f46f80ccf40 con 0x7f46f8140520 2024-09-10T09:58:55.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7400640 1 -- 172.21.15.110:0/2805059949 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46dc003530 con 0x7f46f8140520 2024-09-10T09:58:55.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7400640 1 -- 172.21.15.110:0/2805059949 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 56243034 0 0) 0x7f46f81412a0 con 0x7f46f80ccb60 2024-09-10T09:58:55.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7400640 1 -- 172.21.15.110:0/2805059949 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46f80ccf40 con 0x7f46f80ccb60 2024-09-10T09:58:55.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7e00640 1 -- 172.21.15.110:0/2805059949 >> v1:172.21.15.145:6789/0 conn(0x7f46f80ccb60 legacy=0x7f46f806d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:55.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7e00640 1 -- 172.21.15.110:0/2805059949 >> v1:172.21.15.110:6789/0 conn(0x7f46f8140520 legacy=0x7f46f81409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:55.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7e00640 1 -- 172.21.15.110:0/2805059949 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f815e760 con 0x7f46f8072130 2024-09-10T09:58:55.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.312+0000 7f46f7e00640 1 --2- 172.21.15.110:0/2805059949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8072130 0x7f46f8072590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46e40097f0 tx=0x7f46e401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d768f234fb8867 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:55.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46f7400640 1 -- 172.21.15.110:0/2805059949 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e40b0070 con 0x7f46f8072130 2024-09-10T09:58:55.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46f7400640 1 -- 172.21.15.110:0/2805059949 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e40ab600 con 0x7f46f8072130 2024-09-10T09:58:55.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46f7400640 1 -- 172.21.15.110:0/2805059949 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e40ab9d0 con 0x7f46f8072130 2024-09-10T09:58:55.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- 172.21.15.110:0/2805059949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8072130 msgr2=0x7f46f8072590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:55.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 --2- 172.21.15.110:0/2805059949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8072130 0x7f46f8072590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46e40097f0 tx=0x7f46e401c450 comp rx=0 tx=0).stop 2024-09-10T09:58:55.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- 172.21.15.110:0/2805059949 shutdown_connections 2024-09-10T09:58:55.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 --2- 172.21.15.110:0/2805059949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8072130 0x7f46f8072590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:55.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- 172.21.15.110:0/2805059949 >> 172.21.15.110:0/2805059949 conn(0x7f46f806bc50 msgr2=0x7f46f806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:55.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- 172.21.15.110:0/2805059949 shutdown_connections 2024-09-10T09:58:55.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- 172.21.15.110:0/2805059949 wait complete. 2024-09-10T09:58:55.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 Processor -- start 2024-09-10T09:58:55.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- start start 2024-09-10T09:58:55.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46f8072130 0x7f46f813fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:55.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46f80ccb60 0x7f46f80ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8140520 0x7f46f80aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f46f815f520 con 0x7f46f8072130 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46f806d840 con 0x7f46f80ccb60 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.316+0000 7f46ff600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f46f815d060 con 0x7f46f8140520 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46f8072130 0x7f46f813fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46f8072130 0x7f46f813fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56640/0 (socket says 172.21.15.110:56640) 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 -- 172.21.15.110:0/2026818004 learned_addr learned my addr 172.21.15.110:0/2026818004 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fd600640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8140520 0x7f46f80aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46f7e00640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46f80ccb60 0x7f46f80ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 -- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8140520 msgr2=0x7f46f80aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8140520 0x7f46f80aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 -- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46f80ccb60 msgr2=0x7f46f80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:55.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46f80ccb60 0x7f46f80ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 -- 172.21.15.110:0/2026818004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f815d340 con 0x7f46f8072130 2024-09-10T09:58:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fd600640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8140520 0x7f46f80aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46fcc00640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46f8072130 0x7f46f813fdf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c770 tx=0x7f46e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46f4c00640 1 -- 172.21.15.110:0/2026818004 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8003b80 con 0x7f46f8072130 2024-09-10T09:58:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46f4c00640 1 -- 172.21.15.110:0/2026818004 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8003d20 con 0x7f46f8072130 2024-09-10T09:58:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46f4c00640 1 -- 172.21.15.110:0/2026818004 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e800aa90 con 0x7f46f8072130 2024-09-10T09:58:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f815d620 con 0x7f46f8072130 2024-09-10T09:58:55.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f8158200 con 0x7f46f8072130 2024-09-10T09:58:55.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f8072590 con 0x7f46f8072130 2024-09-10T09:58:55.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.320+0000 7f46f4c00640 1 -- 172.21.15.110:0/2026818004 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46e800ac30 con 0x7f46f8072130 2024-09-10T09:58:55.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.324+0000 7f46f4c00640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46d80678c0 0x7f46d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:55.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.324+0000 7f46f4c00640 1 -- 172.21.15.110:0/2026818004 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5795+0+0 (secure 0 0 0) 0x7f46e8085d20 con 0x7f46f8072130 2024-09-10T09:58:55.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.324+0000 7f46f7e00640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46d80678c0 0x7f46d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:55.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.324+0000 7f46f7e00640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46d80678c0 0x7f46d8069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46f806cc00 tx=0x7f46e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:55.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.328+0000 7f46f4c00640 1 -- 172.21.15.110:0/2026818004 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f46e8056760 con 0x7f46f8072130 2024-09-10T09:58:55.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:55.488+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46f8154990 con 0x7f46f8072130 2024-09-10T09:58:56.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.152+0000 7f46f4c00640 1 -- 172.21.15.110:0/2026818004 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7f46e8056060 con 0x7f46f8072130 2024-09-10T09:58:56.155 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T09:58:56.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46d80678c0 msgr2=0x7f46d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46d80678c0 0x7f46d8069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46f806cc00 tx=0x7f46e4008040 comp rx=0 tx=0).stop 2024-09-10T09:58:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46f8072130 msgr2=0x7f46f813fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46f8072130 0x7f46f813fdf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c770 tx=0x7f46e800cc30 comp rx=0 tx=0).stop 2024-09-10T09:58:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 shutdown_connections 2024-09-10T09:58:56.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46d80678c0 0x7f46d8069d80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46f8140520 0x7f46f80aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46f80ccb60 0x7f46f80ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 --2- 172.21.15.110:0/2026818004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46f8072130 0x7f46f813fdf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 >> 172.21.15.110:0/2026818004 conn(0x7f46f806bc50 msgr2=0x7f46f80ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:56.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 shutdown_connections 2024-09-10T09:58:56.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.156+0000 7f46ff600640 1 -- 172.21.15.110:0/2026818004 wait complete. 2024-09-10T09:58:56.181 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T09:58:56.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f420ce00640 1 Processor -- start 2024-09-10T09:58:56.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f420ce00640 1 -- start start 2024-09-10T09:58:56.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f420ce00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f42080d1e40 0x7f420815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:56.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f420ce00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4208073200 0x7f4208073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:56.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f420ce00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42080b60b0 con 0x7f42080b6b70 2024-09-10T09:58:56.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f420ce00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4208073bb0 con 0x7f4208073200 2024-09-10T09:58:56.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f420ce00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4208160490 con 0x7f42080d1e40 2024-09-10T09:58:56.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f4207400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f42080b6b70 0x7f42080b59a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57832/0 (socket says 172.21.15.110:57832) 2024-09-10T09:58:56.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f4207400640 1 -- 172.21.15.110:0/1120552890 learned_addr learned my addr 172.21.15.110:0/1120552890 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:56.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f4207e00640 1 --2- 172.21.15.110:0/1120552890 >> v2:172.21.15.110:3300/0 conn(0x7f4208073200 0x7f4208073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:56.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f41ffe00640 1 --2- 172.21.15.110:0/1120552890 >> v2:172.21.15.145:3300/0 conn(0x7f42080d1e40 0x7f420815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:56.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f4207e00640 1 -- 172.21.15.110:0/1120552890 >> v2:172.21.15.145:3300/0 conn(0x7f42080d1e40 msgr2=0x7f420815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:56.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f4207e00640 1 --2- 172.21.15.110:0/1120552890 >> v2:172.21.15.145:3300/0 conn(0x7f42080d1e40 0x7f420815ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f4207e00640 1 -- 172.21.15.110:0/1120552890 >> v1:172.21.15.110:6789/0 conn(0x7f42080b6b70 legacy=0x7f42080b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:56.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f4207e00640 1 -- 172.21.15.110:0/1120552890 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42081606f0 con 0x7f4208073200 2024-09-10T09:58:56.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.324+0000 7f41ffe00640 1 --2- 172.21.15.110:0/1120552890 >> v2:172.21.15.145:3300/0 conn(0x7f42080d1e40 0x7f420815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:56.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f4207e00640 1 --2- 172.21.15.110:0/1120552890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f4208073670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f42080731b0 tx=0x7f420001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331fad3e7ccfc74e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:56.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f4206a00640 1 -- 172.21.15.110:0/1120552890 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42000b2070 con 0x7f4208073200 2024-09-10T09:58:56.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f4206a00640 1 -- 172.21.15.110:0/1120552890 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42000a57f0 con 0x7f4208073200 2024-09-10T09:58:56.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f4206a00640 1 -- 172.21.15.110:0/1120552890 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42000a5bc0 con 0x7f4208073200 2024-09-10T09:58:56.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 -- 172.21.15.110:0/1120552890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 msgr2=0x7f4208073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:56.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 --2- 172.21.15.110:0/1120552890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f4208073670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f42080731b0 tx=0x7f420001c930 comp rx=0 tx=0).stop 2024-09-10T09:58:56.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 -- 172.21.15.110:0/1120552890 shutdown_connections 2024-09-10T09:58:56.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 --2- 172.21.15.110:0/1120552890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f4208073670 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 --2- 172.21.15.110:0/1120552890 >> v2:172.21.15.145:3300/0 conn(0x7f42080d1e40 0x7f420815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 -- 172.21.15.110:0/1120552890 >> 172.21.15.110:0/1120552890 conn(0x7f420806c050 msgr2=0x7f42080626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:56.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 -- 172.21.15.110:0/1120552890 shutdown_connections 2024-09-10T09:58:56.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 -- 172.21.15.110:0/1120552890 wait complete. 2024-09-10T09:58:56.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 Processor -- start 2024-09-10T09:58:56.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.328+0000 7f420ce00640 1 -- start start 2024-09-10T09:58:56.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f420815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:56.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42080b6b70 0x7f42080b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:56.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f42080d1e40 0x7f420815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:56.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f42081690e0 con 0x7f4208073200 2024-09-10T09:58:56.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42080b6f50 con 0x7f42080b6b70 2024-09-10T09:58:56.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4208169260 con 0x7f42080d1e40 2024-09-10T09:58:56.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f420815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:56.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f420815b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56650/0 (socket says 172.21.15.110:56650) 2024-09-10T09:58:56.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 -- 172.21.15.110:0/372302983 learned_addr learned my addr 172.21.15.110:0/372302983 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:56.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207e00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f42080d1e40 0x7f420815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:56.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f41ffe00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42080b6b70 0x7f42080b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:56.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 -- 172.21.15.110:0/372302983 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f42080d1e40 msgr2=0x7f420815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:56.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f42080d1e40 0x7f420815e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 -- 172.21.15.110:0/372302983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42080b6b70 msgr2=0x7f42080b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:56.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42080b6b70 0x7f42080b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:56.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 -- 172.21.15.110:0/372302983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42080b2d90 con 0x7f4208073200 2024-09-10T09:58:56.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f4207400640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f420815b3b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c450 tx=0x7f41f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:56.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f41ff400640 1 -- 172.21.15.110:0/372302983 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8017070 con 0x7f4208073200 2024-09-10T09:58:56.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f41ff400640 1 -- 172.21.15.110:0/372302983 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f8005cf0 con 0x7f4208073200 2024-09-10T09:58:56.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42080b3070 con 0x7f4208073200 2024-09-10T09:58:56.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f41ff400640 1 -- 172.21.15.110:0/372302983 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8007290 con 0x7f4208073200 2024-09-10T09:58:56.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42080b3630 con 0x7f4208073200 2024-09-10T09:58:56.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.332+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4208073670 con 0x7f4208073200 2024-09-10T09:58:56.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.336+0000 7f41ff400640 1 -- 172.21.15.110:0/372302983 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f41f801a070 con 0x7f4208073200 2024-09-10T09:58:56.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.336+0000 7f41ff400640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f41e40678c0 0x7f41e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:56.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.336+0000 7f41ff400640 1 -- 172.21.15.110:0/372302983 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5357+0+0 (secure 0 0 0) 0x7f41f8084e30 con 0x7f4208073200 2024-09-10T09:58:56.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.336+0000 7f41ffe00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f41e40678c0 0x7f41e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:56.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.336+0000 7f41ffe00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f41e40678c0 0x7f41e4069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f42080aab00 tx=0x7f41f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:56.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.340+0000 7f41ff400640 1 -- 172.21.15.110:0/372302983 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f41f80553e0 con 0x7f4208073200 2024-09-10T09:58:56.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:56.508+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f420815e0b0 con 0x7f4208073200 2024-09-10T09:58:57.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.160+0000 7f41ff400640 1 -- 172.21.15.110:0/372302983 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f41f80596e0 con 0x7f4208073200 2024-09-10T09:58:57.161 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T09:58:57.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f41e40678c0 msgr2=0x7f41e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f41e40678c0 0x7f41e4069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f42080aab00 tx=0x7f41f0008040 comp rx=0 tx=0).stop 2024-09-10T09:58:57.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 msgr2=0x7f420815b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f420815b3b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c450 tx=0x7f41f800c910 comp rx=0 tx=0).stop 2024-09-10T09:58:57.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 shutdown_connections 2024-09-10T09:58:57.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f41e40678c0 0x7f41e4069d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f42080d1e40 0x7f420815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42080b6b70 0x7f42080b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 --2- 172.21.15.110:0/372302983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4208073200 0x7f420815b3b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 >> 172.21.15.110:0/372302983 conn(0x7f420806c050 msgr2=0x7f420806d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:57.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 shutdown_connections 2024-09-10T09:58:57.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.164+0000 7f420ce00640 1 -- 172.21.15.110:0/372302983 wait complete. 2024-09-10T09:58:57.191 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T09:58:57.191 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T09:58:57.192 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T09:58:57.192 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:58:57.192 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T09:58:57.192 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T09:58:57.192 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T09:58:57.192 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:58:57.192 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T09:58:57.193 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T09:58:57.193 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T09:58:57.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:58:57.193 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T09:58:57.193 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T09:58:57.193 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T09:58:57.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:58:57.207 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T09:58:57.207 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T09:58:57.208 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T09:58:57.208 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T09:58:57.209 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T09:58:57.209 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T09:58:57.210 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T09:58:57.210 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T09:58:57.255 INFO:tasks.ceph.mds.a:Started 2024-09-10T09:58:57.256 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:57.256 INFO:tasks.ceph.mds.b:Started 2024-09-10T09:58:57.256 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:57.257 INFO:tasks.ceph.mds.c:Started 2024-09-10T09:58:57.257 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:57.258 INFO:tasks.ceph.mds.d:Started 2024-09-10T09:58:57.258 DEBUG:teuthology.parallel:result is None 2024-09-10T09:58:57.258 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T09:58:57.346 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T09:58:57.347 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T09:58:57.347 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T09:58:57.348 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T09:58:57.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff322c00640 1 Processor -- start 2024-09-10T09:58:57.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff322c00640 1 -- start start 2024-09-10T09:58:57.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff322c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff31c0cf280 0x7ff31c06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff322c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff31c06e4b0 con 0x7ff31c0cf660 2024-09-10T09:58:57.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff322c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff31c0d0b70 con 0x7ff31c0d0700 2024-09-10T09:58:57.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff322c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff31c0707b0 con 0x7ff31c0cf280 2024-09-10T09:58:57.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff31be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff31c0cf280 0x7ff31c06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff31b400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff31c0cf660 0x7ff31c06ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57848/0 (socket says 172.21.15.110:57848) 2024-09-10T09:58:57.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff31b400640 1 -- 172.21.15.110:0/3043052679 learned_addr learned my addr 172.21.15.110:0/3043052679 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:57.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff31be00640 1 -- 172.21.15.110:0/3043052679 >> v1:172.21.15.145:6789/0 conn(0x7ff31c0d0700 legacy=0x7ff31c15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:57.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff31be00640 1 -- 172.21.15.110:0/3043052679 >> v1:172.21.15.110:6789/0 conn(0x7ff31c0cf660 legacy=0x7ff31c06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff31be00640 1 -- 172.21.15.110:0/3043052679 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c06def0 con 0x7ff31c0cf280 2024-09-10T09:58:57.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.404+0000 7ff31aa00640 1 -- 172.21.15.110:0/3043052679 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416265343 0 0) 0x7ff31c06e4b0 con 0x7ff31c0cf660 2024-09-10T09:58:57.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff31be00640 1 --2- 172.21.15.110:0/3043052679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0cf280 0x7ff31c06e9b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d90 tx=0x7ff30c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80465e56169f95fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff31aa00640 1 -- 172.21.15.110:0/3043052679 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0b2070 con 0x7ff31c0cf280 2024-09-10T09:58:57.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff31aa00640 1 -- 172.21.15.110:0/3043052679 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c0a59c0 con 0x7ff31c0cf280 2024-09-10T09:58:57.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff31aa00640 1 -- 172.21.15.110:0/3043052679 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0a5d90 con 0x7ff31c0cf280 2024-09-10T09:58:57.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff322c00640 1 -- 172.21.15.110:0/3043052679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0cf280 msgr2=0x7ff31c06e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff322c00640 1 --2- 172.21.15.110:0/3043052679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0cf280 0x7ff31c06e9b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d90 tx=0x7ff30c01c900 comp rx=0 tx=0).stop 2024-09-10T09:58:57.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff322c00640 1 -- 172.21.15.110:0/3043052679 shutdown_connections 2024-09-10T09:58:57.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff322c00640 1 --2- 172.21.15.110:0/3043052679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0cf280 0x7ff31c06e9b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff322c00640 1 -- 172.21.15.110:0/3043052679 >> 172.21.15.110:0/3043052679 conn(0x7ff31c09b9d0 msgr2=0x7ff31c09bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:57.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff322c00640 1 -- 172.21.15.110:0/3043052679 shutdown_connections 2024-09-10T09:58:57.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.408+0000 7ff322c00640 1 -- 172.21.15.110:0/3043052679 wait complete. 2024-09-10T09:58:57.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 Processor -- start 2024-09-10T09:58:57.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 -- start start 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff31c0cf280 0x7ff31c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff31c0cf660 0x7ff31c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0d0700 0x7ff31c0c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c0ce9d0 con 0x7ff31c0d0700 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c0ce850 con 0x7ff31c0cf280 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff31c0ceb50 con 0x7ff31c0cf660 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0d0700 0x7ff31c0c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff31be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff31c0cf280 0x7ff31c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0d0700 0x7ff31c0c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56706/0 (socket says 172.21.15.110:56706) 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 -- 172.21.15.110:0/3137566765 learned_addr learned my addr 172.21.15.110:0/3137566765 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:57.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff31b400640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff31c0cf660 0x7ff31c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 -- 172.21.15.110:0/3137566765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff31c0cf660 msgr2=0x7ff31c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff31c0cf660 0x7ff31c15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 -- 172.21.15.110:0/3137566765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff31c0cf280 msgr2=0x7ff31c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff31c0cf280 0x7ff31c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 -- 172.21.15.110:0/3137566765 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c0b39e0 con 0x7ff31c0d0700 2024-09-10T09:58:57.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff320c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0d0700 0x7ff31c0c6e80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff31400b440 tx=0x7ff31400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff313e00640 1 -- 172.21.15.110:0/3137566765 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314015020 con 0x7ff31c0d0700 2024-09-10T09:58:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff313e00640 1 -- 172.21.15.110:0/3137566765 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff314011850 con 0x7ff31c0d0700 2024-09-10T09:58:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff313e00640 1 -- 172.21.15.110:0/3137566765 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314011c40 con 0x7ff31c0d0700 2024-09-10T09:58:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c15d360 con 0x7ff31c0d0700 2024-09-10T09:58:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c15d920 con 0x7ff31c0d0700 2024-09-10T09:58:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.412+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31c15d180 con 0x7ff31c0d0700 2024-09-10T09:58:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.416+0000 7ff313e00640 1 -- 172.21.15.110:0/3137566765 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff314018020 con 0x7ff31c0d0700 2024-09-10T09:58:57.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.416+0000 7ff313e00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff2fc0678c0 0x7ff2fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.416+0000 7ff313e00640 1 -- 172.21.15.110:0/3137566765 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7ff314085440 con 0x7ff31c0d0700 2024-09-10T09:58:57.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.416+0000 7ff31be00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff2fc0678c0 0x7ff2fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.416+0000 7ff31be00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff2fc0678c0 0x7ff2fc069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0af000 tx=0x7ff30c0a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.420+0000 7ff313e00640 1 -- 172.21.15.110:0/3137566765 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff31c15d180 con 0x7ff31c0d0700 2024-09-10T09:58:57.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:57.588+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff31c0d19e0 con 0x7ff31c0d0700 2024-09-10T09:58:58.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff313e00640 1 -- 172.21.15.110:0/3137566765 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7ff31401f030 con 0x7ff31c0d0700 2024-09-10T09:58:58.173 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T09:58:58.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff2fc0678c0 msgr2=0x7ff2fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff2fc0678c0 0x7ff2fc069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0af000 tx=0x7ff30c0a7070 comp rx=0 tx=0).stop 2024-09-10T09:58:58.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0d0700 msgr2=0x7ff31c0c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0d0700 0x7ff31c0c6e80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff31400b440 tx=0x7ff31400b900 comp rx=0 tx=0).stop 2024-09-10T09:58:58.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 shutdown_connections 2024-09-10T09:58:58.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff2fc0678c0 0x7ff2fc069d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff31c0d0700 0x7ff31c0c6e80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff31c0cf660 0x7ff31c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 --2- 172.21.15.110:0/3137566765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff31c0cf280 0x7ff31c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 >> 172.21.15.110:0/3137566765 conn(0x7ff31c09b9d0 msgr2=0x7ff31c0b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:58.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 shutdown_connections 2024-09-10T09:58:58.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.172+0000 7ff322c00640 1 -- 172.21.15.110:0/3137566765 wait complete. 2024-09-10T09:58:58.204 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T09:58:58.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.336+0000 7fab44200640 1 Processor -- start 2024-09-10T09:58:58.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.336+0000 7fab44200640 1 -- start start 2024-09-10T09:58:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab44200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fab3c0cfe30 0x7fab3c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab44200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab3c06e7a0 0x7fab3c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab44200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3c06d2d0 con 0x7fab3c0cd510 2024-09-10T09:58:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab44200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fab3c15d9a0 con 0x7fab3c06e7a0 2024-09-10T09:58:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab44200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fab3c06d5b0 con 0x7fab3c0cfe30 2024-09-10T09:58:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab42200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab3c06e7a0 0x7fab3c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab42200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fab3c06e7a0 0x7fab3c0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36392/0 (socket says 172.21.15.110:36392) 2024-09-10T09:58:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab42200640 1 -- 172.21.15.110:0/2034514824 learned_addr learned my addr 172.21.15.110:0/2034514824 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab40e00640 1 --1- 172.21.15.110:0/2034514824 >> v1:172.21.15.110:6789/0 conn(0x7fab3c0cd510 0x7fab3c06e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43800/0 (socket says 172.21.15.110:43800) 2024-09-10T09:58:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab41800640 1 --2- 172.21.15.110:0/2034514824 >> v2:172.21.15.145:3300/0 conn(0x7fab3c0cfe30 0x7fab3c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab42200640 1 -- 172.21.15.110:0/2034514824 >> v2:172.21.15.145:3300/0 conn(0x7fab3c0cfe30 msgr2=0x7fab3c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:58.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab42200640 1 --2- 172.21.15.110:0/2034514824 >> v2:172.21.15.145:3300/0 conn(0x7fab3c0cfe30 0x7fab3c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab42200640 1 -- 172.21.15.110:0/2034514824 >> v1:172.21.15.110:6789/0 conn(0x7fab3c0cd510 legacy=0x7fab3c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab42200640 1 -- 172.21.15.110:0/2034514824 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c06d890 con 0x7fab3c06e7a0 2024-09-10T09:58:58.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.340+0000 7fab3be00640 1 -- 172.21.15.110:0/2034514824 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 499278945 0 0) 0x7fab3c06d2d0 con 0x7fab3c0cd510 2024-09-10T09:58:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab42200640 1 --2- 172.21.15.110:0/2034514824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c06e7a0 0x7fab3c0cd9a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0ce120 tx=0x7fab3401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fab36810487765 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab3be00640 1 -- 172.21.15.110:0/2034514824 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab340a5730 con 0x7fab3c06e7a0 2024-09-10T09:58:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab3be00640 1 -- 172.21.15.110:0/2034514824 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab340a58d0 con 0x7fab3c06e7a0 2024-09-10T09:58:58.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab3be00640 1 -- 172.21.15.110:0/2034514824 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab340a5ca0 con 0x7fab3c06e7a0 2024-09-10T09:58:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- 172.21.15.110:0/2034514824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c06e7a0 msgr2=0x7fab3c0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 --2- 172.21.15.110:0/2034514824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c06e7a0 0x7fab3c0cd9a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0ce120 tx=0x7fab3401c960 comp rx=0 tx=0).stop 2024-09-10T09:58:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- 172.21.15.110:0/2034514824 shutdown_connections 2024-09-10T09:58:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 --2- 172.21.15.110:0/2034514824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c06e7a0 0x7fab3c0cd9a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 --2- 172.21.15.110:0/2034514824 >> v2:172.21.15.145:3300/0 conn(0x7fab3c0cfe30 0x7fab3c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- 172.21.15.110:0/2034514824 >> 172.21.15.110:0/2034514824 conn(0x7fab3c06bc50 msgr2=0x7fab3c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- 172.21.15.110:0/2034514824 shutdown_connections 2024-09-10T09:58:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- 172.21.15.110:0/2034514824 wait complete. 2024-09-10T09:58:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 Processor -- start 2024-09-10T09:58:58.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- start start 2024-09-10T09:58:58.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab3c06e7a0 0x7fab3c14e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c0cd510 0x7fab3c149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab3c0cfe30 0x7fab3c1493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c072190 con 0x7fab3c0cd510 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c15d6c0 con 0x7fab3c06e7a0 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab44200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fab3c169180 con 0x7fab3c0cfe30 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.344+0000 7fab40e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c0cd510 0x7fab3c149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab40e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c0cd510 0x7fab3c149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36402/0 (socket says 172.21.15.110:36402) 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab40e00640 1 -- 172.21.15.110:0/1787297716 learned_addr learned my addr 172.21.15.110:0/1787297716 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab41800640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab3c06e7a0 0x7fab3c14e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab42200640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab3c0cfe30 0x7fab3c1493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab40e00640 1 -- 172.21.15.110:0/1787297716 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab3c0cfe30 msgr2=0x7fab3c1493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab40e00640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab3c0cfe30 0x7fab3c1493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab40e00640 1 -- 172.21.15.110:0/1787297716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab3c06e7a0 msgr2=0x7fab3c14e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab40e00640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab3c06e7a0 0x7fab3c14e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab40e00640 1 -- 172.21.15.110:0/1787297716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c14f290 con 0x7fab3c0cd510 2024-09-10T09:58:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab40e00640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c0cd510 0x7fab3c149b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab2800c4b0 tx=0x7fab2800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab39600640 1 -- 172.21.15.110:0/1787297716 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28018070 con 0x7fab3c0cd510 2024-09-10T09:58:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab39600640 1 -- 172.21.15.110:0/1787297716 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab28007500 con 0x7fab3c0cd510 2024-09-10T09:58:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab39600640 1 -- 172.21.15.110:0/1787297716 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28004a30 con 0x7fab3c0cd510 2024-09-10T09:58:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c14f570 con 0x7fab3c0cd510 2024-09-10T09:58:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3c14a0d0 con 0x7fab3c0cd510 2024-09-10T09:58:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3c06c240 con 0x7fab3c0cd510 2024-09-10T09:58:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab39600640 1 -- 172.21.15.110:0/1787297716 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab28004bd0 con 0x7fab3c0cd510 2024-09-10T09:58:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab39600640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab1c0678f0 0x7fab1c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.348+0000 7fab39600640 1 -- 172.21.15.110:0/1787297716 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fab28084b30 con 0x7fab3c0cd510 2024-09-10T09:58:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.352+0000 7fab41800640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab1c0678f0 0x7fab1c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.352+0000 7fab41800640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab1c0678f0 0x7fab1c069db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab30009ac0 tx=0x7fab30009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.356+0000 7fab39600640 1 -- 172.21.15.110:0/1787297716 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fab280552d0 con 0x7fab3c0cd510 2024-09-10T09:58:58.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fab3c149910 con 0x7fab3c0cd510 2024-09-10T09:58:58.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab39600640 1 -- 172.21.15.110:0/1787297716 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fab280595d0 con 0x7fab3c0cd510 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:58:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:58:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.578 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.580 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T09:58:58.580 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T09:58:58.580 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.580 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:58:58.580 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:58:58.580 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T09:58:58.581 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.583 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T09:58:58.584 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T09:58:58.585 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.586 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.bootstrap-mds", 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAmGOBmJBGvEhAAowgFJfVDjyFL+g21voGHwQ==", 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAmGOBmvRuvEhAACQp3sJrV+tJKO2slNaOL5Q==", 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.bootstrap-osd", 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAmGOBmxCSvEhAA/t3GHcYv3x9XPZXywgY81w==", 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAmGOBmeS2vEhAAgOxHY7kJnJmr3JhCScF4/Q==", 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAmGOBmQTavEhAA3khx44iylQg0qS1qvpe3QA==", 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAmGOBmXD+vEhAAf3e/kzcJjyp0g+omL6DTiQ==", 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T09:58:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T09:58:58.590 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T09:58:58.591 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:58:58.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:58:58.591 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T09:58:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab1c0678f0 msgr2=0x7fab1c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab1c0678f0 0x7fab1c069db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab30009ac0 tx=0x7fab30009190 comp rx=0 tx=0).stop 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c0cd510 msgr2=0x7fab3c149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c0cd510 0x7fab3c149b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab2800c4b0 tx=0x7fab2800c970 comp rx=0 tx=0).stop 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 shutdown_connections 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fab1c0678f0 0x7fab1c069db0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fab3c0cfe30 0x7fab3c1493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab3c0cd510 0x7fab3c149b90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 --2- 172.21.15.110:0/1787297716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab3c06e7a0 0x7fab3c14e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 >> 172.21.15.110:0/1787297716 conn(0x7fab3c06bc50 msgr2=0x7fab3c074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 shutdown_connections 2024-09-10T09:58:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.572+0000 7fab44200640 1 -- 172.21.15.110:0/1787297716 wait complete. 2024-09-10T09:58:58.594 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T09:58:58.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.728+0000 7f68d4000640 1 Processor -- start 2024-09-10T09:58:58.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.728+0000 7f68d4000640 1 -- start start 2024-09-10T09:58:58.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d4000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc0726e0 0x7f68cc0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d4000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f68cc06efa0 0x7f68cc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d4000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68cc06f3e0 con 0x7f68cc0b5430 2024-09-10T09:58:58.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d4000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f68cc0b5880 con 0x7f68cc06efa0 2024-09-10T09:58:58.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d4000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f68cc0cee50 con 0x7f68cc0726e0 2024-09-10T09:58:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d0c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f68cc06efa0 0x7f68cc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d0c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f68cc06efa0 0x7f68cc0b4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36412/0 (socket says 172.21.15.110:36412) 2024-09-10T09:58:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d0c00640 1 -- 172.21.15.110:0/1314058826 learned_addr learned my addr 172.21.15.110:0/1314058826 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d1600640 1 --2- 172.21.15.110:0/1314058826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc0726e0 0x7f68cc0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d0c00640 1 -- 172.21.15.110:0/1314058826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc0726e0 msgr2=0x7f68cc0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d0c00640 1 --2- 172.21.15.110:0/1314058826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc0726e0 0x7f68cc0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d0c00640 1 -- 172.21.15.110:0/1314058826 >> v1:172.21.15.145:6789/0 conn(0x7f68cc0b5430 legacy=0x7f68cc0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d0c00640 1 -- 172.21.15.110:0/1314058826 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc15fda0 con 0x7f68cc06efa0 2024-09-10T09:58:58.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.732+0000 7f68d0c00640 1 --2- 172.21.15.110:0/1314058826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc06efa0 0x7f68cc0b4ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009870 tx=0x7f68b801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6231c5b9dabccf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:58.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68cbe00640 1 -- 172.21.15.110:0/1314058826 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b80a95f0 con 0x7f68cc06efa0 2024-09-10T09:58:58.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68cbe00640 1 -- 172.21.15.110:0/1314058826 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b80a9790 con 0x7f68cc06efa0 2024-09-10T09:58:58.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68cbe00640 1 -- 172.21.15.110:0/1314058826 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b80a9b60 con 0x7f68cc06efa0 2024-09-10T09:58:58.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- 172.21.15.110:0/1314058826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc06efa0 msgr2=0x7f68cc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 --2- 172.21.15.110:0/1314058826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc06efa0 0x7f68cc0b4ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009870 tx=0x7f68b801c530 comp rx=0 tx=0).stop 2024-09-10T09:58:58.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- 172.21.15.110:0/1314058826 shutdown_connections 2024-09-10T09:58:58.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 --2- 172.21.15.110:0/1314058826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc06efa0 0x7f68cc0b4ef0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 --2- 172.21.15.110:0/1314058826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc0726e0 0x7f68cc0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- 172.21.15.110:0/1314058826 >> 172.21.15.110:0/1314058826 conn(0x7f68cc062d10 msgr2=0x7f68cc06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:58.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- 172.21.15.110:0/1314058826 shutdown_connections 2024-09-10T09:58:58.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- 172.21.15.110:0/1314058826 wait complete. 2024-09-10T09:58:58.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 Processor -- start 2024-09-10T09:58:58.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- start start 2024-09-10T09:58:58.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc06efa0 0x7f68cc158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc0726e0 0x7f68cc155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68cc0b5430 0x7f68cc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc0b3ff0 con 0x7f68cc0726e0 2024-09-10T09:58:58.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc073270 con 0x7f68cc0b5430 2024-09-10T09:58:58.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d4000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f68cc0b4170 con 0x7f68cc06efa0 2024-09-10T09:58:58.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc0726e0 0x7f68cc155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc0726e0 0x7f68cc155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36414/0 (socket says 172.21.15.110:36414) 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d0c00640 1 -- 172.21.15.110:0/125281986 learned_addr learned my addr 172.21.15.110:0/125281986 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d1600640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc06efa0 0x7f68cc158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.736+0000 7f68d2000640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68cc0b5430 0x7f68cc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d0c00640 1 -- 172.21.15.110:0/125281986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc06efa0 msgr2=0x7f68cc158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d0c00640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc06efa0 0x7f68cc158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d0c00640 1 -- 172.21.15.110:0/125281986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68cc0b5430 msgr2=0x7f68cc1529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d0c00640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68cc0b5430 0x7f68cc1529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d0c00640 1 -- 172.21.15.110:0/125281986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc165e90 con 0x7f68cc0726e0 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d0c00640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc0726e0 0x7f68cc155090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f68b80ab040 tx=0x7f68b801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68c9600640 1 -- 172.21.15.110:0/125281986 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b80b1800 con 0x7f68cc0726e0 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68c9600640 1 -- 172.21.15.110:0/125281986 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b80b19a0 con 0x7f68cc0726e0 2024-09-10T09:58:58.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68c9600640 1 -- 172.21.15.110:0/125281986 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b80b1d90 con 0x7f68cc0726e0 2024-09-10T09:58:58.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68cc165bb0 con 0x7f68cc0726e0 2024-09-10T09:58:58.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68cc1582d0 con 0x7f68cc0726e0 2024-09-10T09:58:58.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68cc072ac0 con 0x7f68cc0726e0 2024-09-10T09:58:58.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68c9600640 1 -- 172.21.15.110:0/125281986 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f68b80a9650 con 0x7f68cc0726e0 2024-09-10T09:58:58.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68c9600640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f68ac0678c0 0x7f68ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:58.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68c9600640 1 -- 172.21.15.110:0/125281986 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f68b811fcc0 con 0x7f68cc0726e0 2024-09-10T09:58:58.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.740+0000 7f68d1600640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f68ac0678c0 0x7f68ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:58.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.744+0000 7f68d1600640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f68ac0678c0 0x7f68ac069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0cc9d0 tx=0x7f68c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:58.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.744+0000 7f68c9600640 1 -- 172.21.15.110:0/125281986 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f68b8126080 con 0x7f68cc0726e0 2024-09-10T09:58:58.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.956+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f68cc1555d0 con 0x7f68cc0726e0 2024-09-10T09:58:58.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68c9600640 1 -- 172.21.15.110:0/125281986 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f68b80c3070 con 0x7f68cc0726e0 2024-09-10T09:58:58.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f68ac0678c0 msgr2=0x7f68ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f68ac0678c0 0x7f68ac069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0cc9d0 tx=0x7f68c0008040 comp rx=0 tx=0).stop 2024-09-10T09:58:58.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc0726e0 msgr2=0x7f68cc155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc0726e0 0x7f68cc155090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f68b80ab040 tx=0x7f68b801ad70 comp rx=0 tx=0).stop 2024-09-10T09:58:58.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 shutdown_connections 2024-09-10T09:58:58.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f68ac0678c0 0x7f68ac069d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68cc0b5430 0x7f68cc1529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68cc0726e0 0x7f68cc155090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 --2- 172.21.15.110:0/125281986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68cc06efa0 0x7f68cc158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 >> 172.21.15.110:0/125281986 conn(0x7f68cc062d10 msgr2=0x7f68cc0b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:58.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 shutdown_connections 2024-09-10T09:58:58.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:58.960+0000 7f68d4000640 1 -- 172.21.15.110:0/125281986 wait complete. 2024-09-10T09:58:58.982 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T09:58:59.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff45a800640 1 Processor -- start 2024-09-10T09:58:59.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff45a800640 1 -- start start 2024-09-10T09:58:59.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff45a800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4540b5600 0x7ff4540cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff45a800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45415fda0 con 0x7ff45406efa0 2024-09-10T09:58:59.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff45a800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4540ce520 con 0x7ff4540726e0 2024-09-10T09:58:59.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff45a800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff4540cee50 con 0x7ff4540b5600 2024-09-10T09:58:59.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff453e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff45406efa0 0x7ff4540b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43814/0 (socket says 172.21.15.110:43814) 2024-09-10T09:58:59.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff453e00640 1 -- 172.21.15.110:0/72718097 learned_addr learned my addr 172.21.15.110:0/72718097 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:59.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.116+0000 7ff452a00640 1 --2- 172.21.15.110:0/72718097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4540b5600 0x7ff4540cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452000640 1 -- 172.21.15.110:0/72718097 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675619077 0 0) 0x7ff45415fda0 con 0x7ff45406efa0 2024-09-10T09:58:59.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452000640 1 -- 172.21.15.110:0/72718097 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff438003530 con 0x7ff45406efa0 2024-09-10T09:58:59.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452000640 1 -- 172.21.15.110:0/72718097 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3306755088 0 0) 0x7ff4540ce520 con 0x7ff4540726e0 2024-09-10T09:58:59.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452000640 1 -- 172.21.15.110:0/72718097 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff45415fda0 con 0x7ff4540726e0 2024-09-10T09:58:59.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452a00640 1 -- 172.21.15.110:0/72718097 >> v1:172.21.15.145:6789/0 conn(0x7ff4540726e0 legacy=0x7ff4540b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452a00640 1 -- 172.21.15.110:0/72718097 >> v1:172.21.15.110:6789/0 conn(0x7ff45406efa0 legacy=0x7ff4540b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452a00640 1 -- 172.21.15.110:0/72718097 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4540ce800 con 0x7ff4540b5600 2024-09-10T09:58:59.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452a00640 1 --2- 172.21.15.110:0/72718097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4540b5600 0x7ff4540cf2e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4400097f0 tx=0x7ff44001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf2a656f346605c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452000640 1 -- 172.21.15.110:0/72718097 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4400a7070 con 0x7ff4540b5600 2024-09-10T09:58:59.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452000640 1 -- 172.21.15.110:0/72718097 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4400b0070 con 0x7ff4540b5600 2024-09-10T09:58:59.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.120+0000 7ff452000640 1 -- 172.21.15.110:0/72718097 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4400ab870 con 0x7ff4540b5600 2024-09-10T09:58:59.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- 172.21.15.110:0/72718097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4540b5600 msgr2=0x7ff4540cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 --2- 172.21.15.110:0/72718097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4540b5600 0x7ff4540cf2e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4400097f0 tx=0x7ff44001c4b0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- 172.21.15.110:0/72718097 shutdown_connections 2024-09-10T09:58:59.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 --2- 172.21.15.110:0/72718097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4540b5600 0x7ff4540cf2e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- 172.21.15.110:0/72718097 >> 172.21.15.110:0/72718097 conn(0x7ff454062d10 msgr2=0x7ff45406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:59.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- 172.21.15.110:0/72718097 shutdown_connections 2024-09-10T09:58:59.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- 172.21.15.110:0/72718097 wait complete. 2024-09-10T09:58:59.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 Processor -- start 2024-09-10T09:58:59.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- start start 2024-09-10T09:58:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff45406efa0 0x7ff45414ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4540726e0 0x7ff4541557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4540b5600 0x7ff45414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540b4170 con 0x7ff4540b5600 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540b3ff0 con 0x7ff4540726e0 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff45a800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff454072ac0 con 0x7ff45406efa0 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4540b5600 0x7ff45414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff452a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4540726e0 0x7ff4541557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4540b5600 0x7ff45414fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36424/0 (socket says 172.21.15.110:36424) 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 -- 172.21.15.110:0/215875494 learned_addr learned my addr 172.21.15.110:0/215875494 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453400640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff45406efa0 0x7ff45414ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 -- 172.21.15.110:0/215875494 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff45406efa0 msgr2=0x7ff45414ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff45406efa0 0x7ff45414ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 -- 172.21.15.110:0/215875494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4540726e0 msgr2=0x7ff4541557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4540726e0 0x7ff4541557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 -- 172.21.15.110:0/215875494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45414f350 con 0x7ff4540b5600 2024-09-10T09:58:59.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff453e00640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4540b5600 0x7ff45414fc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c480 tx=0x7ff44c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff44b400640 1 -- 172.21.15.110:0/215875494 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c017070 con 0x7ff4540b5600 2024-09-10T09:58:59.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.124+0000 7ff44b400640 1 -- 172.21.15.110:0/215875494 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff44c005d10 con 0x7ff4540b5600 2024-09-10T09:58:59.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4541551d0 con 0x7ff4540b5600 2024-09-10T09:58:59.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff44b400640 1 -- 172.21.15.110:0/215875494 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0072b0 con 0x7ff4540b5600 2024-09-10T09:58:59.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff454157b20 con 0x7ff4540b5600 2024-09-10T09:58:59.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45415f950 con 0x7ff4540b5600 2024-09-10T09:58:59.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff44b400640 1 -- 172.21.15.110:0/215875494 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff44c01a070 con 0x7ff4540b5600 2024-09-10T09:58:59.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff44b400640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff4340678c0 0x7ff434069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff44b400640 1 -- 172.21.15.110:0/215875494 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7ff44c084d10 con 0x7ff4540b5600 2024-09-10T09:58:59.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff453400640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff4340678c0 0x7ff434069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.128+0000 7ff453400640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff4340678c0 0x7ff434069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff444007fd0 tx=0x7ff444009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.132+0000 7ff44b400640 1 -- 172.21.15.110:0/215875494 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff44c08a020 con 0x7ff4540b5600 2024-09-10T09:58:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.352+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7ff4540cd090 con 0x7ff4540b5600 2024-09-10T09:58:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.352+0000 7ff44b400640 1 -- 172.21.15.110:0/215875494 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7ff44c055540 con 0x7ff4540b5600 2024-09-10T09:58:59.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff4340678c0 msgr2=0x7ff434069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff4340678c0 0x7ff434069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff444007fd0 tx=0x7ff444009040 comp rx=0 tx=0).stop 2024-09-10T09:58:59.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4540b5600 msgr2=0x7ff45414fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4540b5600 0x7ff45414fc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c480 tx=0x7ff44c00c940 comp rx=0 tx=0).stop 2024-09-10T09:58:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 shutdown_connections 2024-09-10T09:58:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff4340678c0 0x7ff434069d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4540b5600 0x7ff45414fc70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4540726e0 0x7ff4541557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 --2- 172.21.15.110:0/215875494 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff45406efa0 0x7ff45414ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 >> 172.21.15.110:0/215875494 conn(0x7ff454062d10 msgr2=0x7ff45415bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 shutdown_connections 2024-09-10T09:58:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.356+0000 7ff45a800640 1 -- 172.21.15.110:0/215875494 wait complete. 2024-09-10T09:58:59.378 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T09:58:59.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.512+0000 7f44e3400640 1 Processor -- start 2024-09-10T09:58:59.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.512+0000 7f44e3400640 1 -- start start 2024-09-10T09:58:59.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44e3400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f44dc06c360 0x7f44dc0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44e3400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44dc1632f0 con 0x7f44dc162390 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44e3400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44dc163010 con 0x7f44dc0d0b90 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44e3400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f44dc0cdf30 con 0x7f44dc06c360 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44dbe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f44dc06c360 0x7f44dc0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44dbe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f44dc06c360 0x7f44dc0cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36428/0 (socket says 172.21.15.110:36428) 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44e1400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f44dc162390 0x7f44dc162840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43822/0 (socket says 172.21.15.110:43822) 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44dbe00640 1 -- 172.21.15.110:0/2707612636 learned_addr learned my addr 172.21.15.110:0/2707612636 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44dbe00640 1 -- 172.21.15.110:0/2707612636 >> v1:172.21.15.145:6789/0 conn(0x7f44dc0d0b90 legacy=0x7f44dc06bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44dbe00640 1 -- 172.21.15.110:0/2707612636 >> v1:172.21.15.110:6789/0 conn(0x7f44dc162390 legacy=0x7f44dc162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.516+0000 7f44dbe00640 1 -- 172.21.15.110:0/2707612636 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc0ce210 con 0x7f44dc06c360 2024-09-10T09:58:59.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44dbe00640 1 --2- 172.21.15.110:0/2707612636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0cea10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009870 tx=0x7f44c801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfbdb09cbe6c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44db400640 1 -- 172.21.15.110:0/2707612636 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80a9610 con 0x7f44dc06c360 2024-09-10T09:58:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44db400640 1 -- 172.21.15.110:0/2707612636 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c80a97b0 con 0x7f44dc06c360 2024-09-10T09:58:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44db400640 1 -- 172.21.15.110:0/2707612636 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80a9b80 con 0x7f44dc06c360 2024-09-10T09:58:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- 172.21.15.110:0/2707612636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 msgr2=0x7f44dc0cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 --2- 172.21.15.110:0/2707612636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0cea10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009870 tx=0x7f44c801c530 comp rx=0 tx=0).stop 2024-09-10T09:58:59.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- 172.21.15.110:0/2707612636 shutdown_connections 2024-09-10T09:58:59.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 --2- 172.21.15.110:0/2707612636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0cea10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- 172.21.15.110:0/2707612636 >> 172.21.15.110:0/2707612636 conn(0x7f44dc0b3100 msgr2=0x7f44dc0b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:59.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- 172.21.15.110:0/2707612636 shutdown_connections 2024-09-10T09:58:59.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- 172.21.15.110:0/2707612636 wait complete. 2024-09-10T09:58:59.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 Processor -- start 2024-09-10T09:58:59.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- start start 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44dc0d0b90 0x7f44dc15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f44dc162390 0x7f44dc0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc072dc0 con 0x7f44dc06c360 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc072c40 con 0x7f44dc0d0b90 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e3400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f44dc16c740 con 0x7f44dc162390 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e0a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e0a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36436/0 (socket says 172.21.15.110:36436) 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.520+0000 7f44e0a00640 1 -- 172.21.15.110:0/4160335666 learned_addr learned my addr 172.21.15.110:0/4160335666 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e1400640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f44dc162390 0x7f44dc0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44dbe00640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44dc0d0b90 0x7f44dc15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e0a00640 1 -- 172.21.15.110:0/4160335666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f44dc162390 msgr2=0x7f44dc0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e0a00640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f44dc162390 0x7f44dc0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e0a00640 1 -- 172.21.15.110:0/4160335666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44dc0d0b90 msgr2=0x7f44dc15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e0a00640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44dc0d0b90 0x7f44dc15d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e0a00640 1 -- 172.21.15.110:0/4160335666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc0ae840 con 0x7f44dc06c360 2024-09-10T09:58:59.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e0a00640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0af300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44d000c530 tx=0x7f44d000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44d8c00640 1 -- 172.21.15.110:0/4160335666 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0016020 con 0x7f44dc06c360 2024-09-10T09:58:59.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44d8c00640 1 -- 172.21.15.110:0/4160335666 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d0005220 con 0x7f44dc06c360 2024-09-10T09:58:59.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44d8c00640 1 -- 172.21.15.110:0/4160335666 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0007030 con 0x7f44dc06c360 2024-09-10T09:58:59.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44dc0b0940 con 0x7f44dc06c360 2024-09-10T09:58:59.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44dc0aeb00 con 0x7f44dc06c360 2024-09-10T09:58:59.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc0cea10 con 0x7f44dc06c360 2024-09-10T09:58:59.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44d8c00640 1 -- 172.21.15.110:0/4160335666 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f44d00071d0 con 0x7f44dc06c360 2024-09-10T09:58:59.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44d8c00640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f44bc0678c0 0x7f44bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44d8c00640 1 -- 172.21.15.110:0/4160335666 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f44d0085190 con 0x7f44dc06c360 2024-09-10T09:58:59.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.524+0000 7f44dbe00640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f44bc0678c0 0x7f44bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.528+0000 7f44dbe00640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f44bc0678c0 0x7f44bc069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f44c80ae000 tx=0x7f44c80b3040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.528+0000 7f44d8c00640 1 -- 172.21.15.110:0/4160335666 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f44d0020020 con 0x7f44dc06c360 2024-09-10T09:58:59.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.740+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f44dc06ca90 con 0x7f44dc06c360 2024-09-10T09:58:59.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44d8c00640 1 -- 172.21.15.110:0/4160335666 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f44d00074c0 con 0x7f44dc06c360 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f44bc0678c0 msgr2=0x7f44bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f44bc0678c0 0x7f44bc069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f44c80ae000 tx=0x7f44c80b3040 comp rx=0 tx=0).stop 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 msgr2=0x7f44dc0af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0af300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44d000c530 tx=0x7f44d000c9f0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 shutdown_connections 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f44bc0678c0 0x7f44bc069d80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f44dc162390 0x7f44dc0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44dc0d0b90 0x7f44dc15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 --2- 172.21.15.110:0/4160335666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44dc06c360 0x7f44dc0af300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 >> 172.21.15.110:0/4160335666 conn(0x7f44dc0b3100 msgr2=0x7f44dc0d1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:59.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 shutdown_connections 2024-09-10T09:58:59.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.744+0000 7f44e3400640 1 -- 172.21.15.110:0/4160335666 wait complete. 2024-09-10T09:58:59.765 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T09:58:59.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f3526000640 1 Processor -- start 2024-09-10T09:58:59.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f3526000640 1 -- start start 2024-09-10T09:58:59.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f3526000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f35200cf2e0 0x7f352006efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f3526000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352015fda0 con 0x7f35200726e0 2024-09-10T09:58:59.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f3526000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35200ce520 con 0x7f35200b5600 2024-09-10T09:58:59.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f3526000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f35200cee50 con 0x7f35200cf2e0 2024-09-10T09:58:59.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f351f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f35200726e0 0x7f35200b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43828/0 (socket says 172.21.15.110:43828) 2024-09-10T09:58:59.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f351fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f35200cf2e0 0x7f352006efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f351f400640 1 -- 172.21.15.110:0/3458830494 learned_addr learned my addr 172.21.15.110:0/3458830494 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:58:59.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f351fe00640 1 -- 172.21.15.110:0/3458830494 >> v1:172.21.15.145:6789/0 conn(0x7f35200b5600 legacy=0x7f35200b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:59.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f351fe00640 1 -- 172.21.15.110:0/3458830494 >> v1:172.21.15.110:6789/0 conn(0x7f35200726e0 legacy=0x7f35200b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.900+0000 7f351fe00640 1 -- 172.21.15.110:0/3458830494 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35200ce800 con 0x7f35200cf2e0 2024-09-10T09:58:59.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f351fe00640 1 --2- 172.21.15.110:0/3458830494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200cf2e0 0x7f352006efa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35180097c0 tx=0x7f351801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49c3924a669ff18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f351e000640 1 -- 172.21.15.110:0/3458830494 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35180b0070 con 0x7f35200cf2e0 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f351e000640 1 -- 172.21.15.110:0/3458830494 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35180ab600 con 0x7f35200cf2e0 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f351e000640 1 -- 172.21.15.110:0/3458830494 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35180ab9d0 con 0x7f35200cf2e0 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f3526000640 1 -- 172.21.15.110:0/3458830494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200cf2e0 msgr2=0x7f352006efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f3526000640 1 --2- 172.21.15.110:0/3458830494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200cf2e0 0x7f352006efa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35180097c0 tx=0x7f351801c460 comp rx=0 tx=0).stop 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f3526000640 1 -- 172.21.15.110:0/3458830494 shutdown_connections 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f3526000640 1 --2- 172.21.15.110:0/3458830494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200cf2e0 0x7f352006efa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f3526000640 1 -- 172.21.15.110:0/3458830494 >> 172.21.15.110:0/3458830494 conn(0x7f3520062d10 msgr2=0x7f352006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:59.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f3526000640 1 -- 172.21.15.110:0/3458830494 shutdown_connections 2024-09-10T09:58:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.904+0000 7f3526000640 1 -- 172.21.15.110:0/3458830494 wait complete. 2024-09-10T09:58:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 Processor -- start 2024-09-10T09:58:59.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 -- start start 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35200726e0 0x7f35200b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200b5600 0x7f35200b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35200cf2e0 0x7f352015d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3520072c40 con 0x7f35200b5600 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3520072ac0 con 0x7f35200726e0 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f352015f950 con 0x7f35200cf2e0 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200b5600 0x7f35200b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200b5600 0x7f35200b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36456/0 (socket says 172.21.15.110:36456) 2024-09-10T09:58:59.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 -- 172.21.15.110:0/3939085450 learned_addr learned my addr 172.21.15.110:0/3939085450 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351f400640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35200726e0 0x7f35200b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351fe00640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35200cf2e0 0x7f352015d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 -- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35200cf2e0 msgr2=0x7f352015d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35200cf2e0 0x7f352015d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 -- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35200726e0 msgr2=0x7f35200b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35200726e0 0x7f35200b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 -- 172.21.15.110:0/3939085450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35200afe70 con 0x7f35200b5600 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351f400640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35200726e0 0x7f35200b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351fe00640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35200cf2e0 0x7f352015d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f351ea00640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200b5600 0x7f35200b0c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c410 tx=0x7f350c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3517400640 1 -- 172.21.15.110:0/3939085450 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c017070 con 0x7f35200b5600 2024-09-10T09:58:59.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3517400640 1 -- 172.21.15.110:0/3939085450 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350c003600 con 0x7f35200b5600 2024-09-10T09:58:59.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35200b0130 con 0x7f35200b5600 2024-09-10T09:58:59.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3517400640 1 -- 172.21.15.110:0/3939085450 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c0039f0 con 0x7f35200b5600 2024-09-10T09:58:59.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.908+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3520154150 con 0x7f35200b5600 2024-09-10T09:58:59.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.912+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352006efa0 con 0x7f35200b5600 2024-09-10T09:58:59.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.912+0000 7f3517400640 1 -- 172.21.15.110:0/3939085450 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f350c01a070 con 0x7f35200b5600 2024-09-10T09:58:59.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.912+0000 7f3517400640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f35000678f0 0x7f3500069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:59.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.912+0000 7f3517400640 1 -- 172.21.15.110:0/3939085450 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f350c084b90 con 0x7f35200b5600 2024-09-10T09:58:59.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.912+0000 7f351f400640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f35000678f0 0x7f3500069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:59.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.912+0000 7f351f400640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f35000678f0 0x7f3500069db0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35100012f0 tx=0x7f3510009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:59.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:58:59.916+0000 7f3517400640 1 -- 172.21.15.110:0/3939085450 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f350c055460 con 0x7f35200b5600 2024-09-10T09:59:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.136+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f35200b3650 con 0x7f35200b5600 2024-09-10T09:59:00.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3517400640 1 -- 172.21.15.110:0/3939085450 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f350c059760 con 0x7f35200b5600 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f35000678f0 msgr2=0x7f3500069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f35000678f0 0x7f3500069db0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35100012f0 tx=0x7f3510009040 comp rx=0 tx=0).stop 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200b5600 msgr2=0x7f35200b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200b5600 0x7f35200b0c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c410 tx=0x7f350c00c8d0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 shutdown_connections 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f35000678f0 0x7f3500069db0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35200cf2e0 0x7f352015d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35200b5600 0x7f35200b0c10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 --2- 172.21.15.110:0/3939085450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35200726e0 0x7f35200b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 >> 172.21.15.110:0/3939085450 conn(0x7f3520062d10 msgr2=0x7f352006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:00.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 shutdown_connections 2024-09-10T09:59:00.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.140+0000 7f3526000640 1 -- 172.21.15.110:0/3939085450 wait complete. 2024-09-10T09:59:00.161 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T09:59:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.296+0000 7f0209a00640 1 Processor -- start 2024-09-10T09:59:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.296+0000 7f0209a00640 1 -- start start 2024-09-10T09:59:00.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.296+0000 7f0209a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f02040cdcb0 0x7f0204167160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.296+0000 7f0209a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020415d4d0 con 0x7f02040b3f40 2024-09-10T09:59:00.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.296+0000 7f0209a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020415d7b0 con 0x7f02040cfb90 2024-09-10T09:59:00.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.296+0000 7f0209a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f020406d130 con 0x7f02040cdcb0 2024-09-10T09:59:00.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.296+0000 7f0202000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f02040b3f40 0x7f02040cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43832/0 (socket says 172.21.15.110:43832) 2024-09-10T09:59:00.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.296+0000 7f0202000640 1 -- 172.21.15.110:0/1998283748 learned_addr learned my addr 172.21.15.110:0/1998283748 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:00.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0203400640 1 --2- 172.21.15.110:0/1998283748 >> v2:172.21.15.145:3300/0 conn(0x7f02040cdcb0 0x7f0204167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0201600640 1 -- 172.21.15.110:0/1998283748 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2904127942 0 0) 0x7f020415d4d0 con 0x7f02040b3f40 2024-09-10T09:59:00.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0201600640 1 -- 172.21.15.110:0/1998283748 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01e8003530 con 0x7f02040b3f40 2024-09-10T09:59:00.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0203400640 1 -- 172.21.15.110:0/1998283748 >> v1:172.21.15.145:6789/0 conn(0x7f02040cfb90 legacy=0x7f020415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0203400640 1 -- 172.21.15.110:0/1998283748 >> v1:172.21.15.110:6789/0 conn(0x7f02040b3f40 legacy=0x7f02040cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0203400640 1 -- 172.21.15.110:0/1998283748 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020406d380 con 0x7f02040cdcb0 2024-09-10T09:59:00.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0203400640 1 --2- 172.21.15.110:0/1998283748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 0x7f0204167160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b369389234601712 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:00.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0201600640 1 -- 172.21.15.110:0/1998283748 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0b0070 con 0x7f02040cdcb0 2024-09-10T09:59:00.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0201600640 1 -- 172.21.15.110:0/1998283748 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc0ab600 con 0x7f02040cdcb0 2024-09-10T09:59:00.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.300+0000 7f0201600640 1 -- 172.21.15.110:0/1998283748 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0ab9b0 con 0x7f02040cdcb0 2024-09-10T09:59:00.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- 172.21.15.110:0/1998283748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 msgr2=0x7f0204167160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 --2- 172.21.15.110:0/1998283748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 0x7f0204167160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc01c410 comp rx=0 tx=0).stop 2024-09-10T09:59:00.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- 172.21.15.110:0/1998283748 shutdown_connections 2024-09-10T09:59:00.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 --2- 172.21.15.110:0/1998283748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 0x7f0204167160 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- 172.21.15.110:0/1998283748 >> 172.21.15.110:0/1998283748 conn(0x7f0204062d10 msgr2=0x7f020406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:00.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- 172.21.15.110:0/1998283748 shutdown_connections 2024-09-10T09:59:00.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- 172.21.15.110:0/1998283748 wait complete. 2024-09-10T09:59:00.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 Processor -- start 2024-09-10T09:59:00.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- start start 2024-09-10T09:59:00.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02040b3f40 0x7f02041466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 0x7f020415e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02040cfb90 0x7f02041495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f020416a250 con 0x7f02040cfb90 2024-09-10T09:59:00.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f020416a0d0 con 0x7f02040cdcb0 2024-09-10T09:59:00.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0209a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f020416a3d0 con 0x7f02040b3f40 2024-09-10T09:59:00.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02040cfb90 0x7f02041495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02040cfb90 0x7f02041495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36468/0 (socket says 172.21.15.110:36468) 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 -- 172.21.15.110:0/4102893691 learned_addr learned my addr 172.21.15.110:0/4102893691 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0202a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02040b3f40 0x7f02041466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0202000640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 0x7f020415e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 -- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02040b3f40 msgr2=0x7f02041466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02040b3f40 0x7f02041466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 -- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 msgr2=0x7f020415e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 0x7f020415e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 -- 172.21.15.110:0/4102893691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0204146c00 con 0x7f02040cfb90 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0202000640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 0x7f020415e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:00.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.304+0000 7f0203400640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02040cfb90 0x7f02041495b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:00.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f01faa00640 1 -- 172.21.15.110:0/4102893691 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0b0070 con 0x7f02040cfb90 2024-09-10T09:59:00.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f01faa00640 1 -- 172.21.15.110:0/4102893691 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc0ba070 con 0x7f02040cfb90 2024-09-10T09:59:00.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f01faa00640 1 -- 172.21.15.110:0/4102893691 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0abbc0 con 0x7f02040cfb90 2024-09-10T09:59:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020415f090 con 0x7f02040cfb90 2024-09-10T09:59:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020415f650 con 0x7f02040cfb90 2024-09-10T09:59:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020415d4d0 con 0x7f02040cfb90 2024-09-10T09:59:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f01faa00640 1 -- 172.21.15.110:0/4102893691 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f01fc003620 con 0x7f02040cfb90 2024-09-10T09:59:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f01faa00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01e40678c0 0x7f01e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f01faa00640 1 -- 172.21.15.110:0/4102893691 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f01fc11f8f0 con 0x7f02040cfb90 2024-09-10T09:59:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f0202a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01e40678c0 0x7f01e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.308+0000 7f0202a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01e40678c0 0x7f01e4069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f020414b9f0 tx=0x7f01f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:00.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.312+0000 7f01faa00640 1 -- 172.21.15.110:0/4102893691 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f01fc016090 con 0x7f02040cfb90 2024-09-10T09:59:00.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.524+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f02041524a0 con 0x7f02040cfb90 2024-09-10T09:59:00.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f01faa00640 1 -- 172.21.15.110:0/4102893691 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f01fc0f07f0 con 0x7f02040cfb90 2024-09-10T09:59:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01e40678c0 msgr2=0x7f01e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01e40678c0 0x7f01e4069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f020414b9f0 tx=0x7f01f4009040 comp rx=0 tx=0).stop 2024-09-10T09:59:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02040cfb90 msgr2=0x7f02041495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02040cfb90 0x7f02041495b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc01ac50 comp rx=0 tx=0).stop 2024-09-10T09:59:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 shutdown_connections 2024-09-10T09:59:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f01e40678c0 0x7f01e4069d80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02040cfb90 0x7f02041495b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02040cdcb0 0x7f020415e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 --2- 172.21.15.110:0/4102893691 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02040b3f40 0x7f02041466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 >> 172.21.15.110:0/4102893691 conn(0x7f0204062d10 msgr2=0x7f0204071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:00.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 shutdown_connections 2024-09-10T09:59:00.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.528+0000 7f0209a00640 1 -- 172.21.15.110:0/4102893691 wait complete. 2024-09-10T09:59:00.551 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T09:59:00.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.684+0000 7ff847000640 1 Processor -- start 2024-09-10T09:59:00.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.684+0000 7ff847000640 1 -- start start 2024-09-10T09:59:00.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff847000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff8400cfb90 0x7ff84015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff847000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff84015d4d0 con 0x7ff8400b3f40 2024-09-10T09:59:00.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff847000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff84015d7b0 con 0x7ff8400cdcb0 2024-09-10T09:59:00.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff847000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff84006d130 con 0x7ff8400cfb90 2024-09-10T09:59:00.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff83fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff8400cfb90 0x7ff84015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff837e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff8400b3f40 0x7ff8400cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43836/0 (socket says 172.21.15.110:43836) 2024-09-10T09:59:00.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff837e00640 1 -- 172.21.15.110:0/3692569342 learned_addr learned my addr 172.21.15.110:0/3692569342 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:00.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff83fe00640 1 -- 172.21.15.110:0/3692569342 >> v1:172.21.15.145:6789/0 conn(0x7ff8400cdcb0 legacy=0x7ff84006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff83fe00640 1 -- 172.21.15.110:0/3692569342 >> v1:172.21.15.110:6789/0 conn(0x7ff8400b3f40 legacy=0x7ff8400cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.688+0000 7ff83fe00640 1 -- 172.21.15.110:0/3692569342 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84006d380 con 0x7ff8400cfb90 2024-09-10T09:59:00.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff83fe00640 1 --2- 172.21.15.110:0/3692569342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400cfb90 0x7ff84015ca10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff8300097c0 tx=0x7ff83001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88597482b07ccca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:00.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff83f400640 1 -- 172.21.15.110:0/3692569342 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8300b0070 con 0x7ff8400cfb90 2024-09-10T09:59:00.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff83f400640 1 -- 172.21.15.110:0/3692569342 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8300ab600 con 0x7ff8400cfb90 2024-09-10T09:59:00.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff83f400640 1 -- 172.21.15.110:0/3692569342 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8300ab9d0 con 0x7ff8400cfb90 2024-09-10T09:59:00.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- 172.21.15.110:0/3692569342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400cfb90 msgr2=0x7ff84015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 --2- 172.21.15.110:0/3692569342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400cfb90 0x7ff84015ca10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff8300097c0 tx=0x7ff83001c410 comp rx=0 tx=0).stop 2024-09-10T09:59:00.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- 172.21.15.110:0/3692569342 shutdown_connections 2024-09-10T09:59:00.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 --2- 172.21.15.110:0/3692569342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400cfb90 0x7ff84015ca10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- 172.21.15.110:0/3692569342 >> 172.21.15.110:0/3692569342 conn(0x7ff840062d10 msgr2=0x7ff84006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:00.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- 172.21.15.110:0/3692569342 shutdown_connections 2024-09-10T09:59:00.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- 172.21.15.110:0/3692569342 wait complete. 2024-09-10T09:59:00.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 Processor -- start 2024-09-10T09:59:00.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- start start 2024-09-10T09:59:00.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400b3f40 0x7ff840156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8400cdcb0 0x7ff84014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8400cfb90 0x7ff84015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff84016a390 con 0x7ff8400b3f40 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff84006eaa0 con 0x7ff8400cfb90 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff847000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff84016a510 con 0x7ff8400cdcb0 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400b3f40 0x7ff840156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400b3f40 0x7ff840156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36490/0 (socket says 172.21.15.110:36490) 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.692+0000 7ff83fe00640 1 -- 172.21.15.110:0/1644834356 learned_addr learned my addr 172.21.15.110:0/1644834356 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff837e00640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8400cdcb0 0x7ff84014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff845000640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8400cfb90 0x7ff84015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83fe00640 1 -- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8400cdcb0 msgr2=0x7ff84014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83fe00640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8400cdcb0 0x7ff84014cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83fe00640 1 -- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8400cfb90 msgr2=0x7ff84015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83fe00640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8400cfb90 0x7ff84015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83fe00640 1 -- 172.21.15.110:0/1644834356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff840156580 con 0x7ff8400b3f40 2024-09-10T09:59:00.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff845000640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8400cfb90 0x7ff84015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:00.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83fe00640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400b3f40 0x7ff840156040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8300097c0 tx=0x7ff83001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:00.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83cc00640 1 -- 172.21.15.110:0/1644834356 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8300b0050 con 0x7ff8400b3f40 2024-09-10T09:59:00.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83cc00640 1 -- 172.21.15.110:0/1644834356 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8300bc070 con 0x7ff8400b3f40 2024-09-10T09:59:00.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83cc00640 1 -- 172.21.15.110:0/1644834356 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8300aac00 con 0x7ff8400b3f40 2024-09-10T09:59:00.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff840153e60 con 0x7ff8400b3f40 2024-09-10T09:59:00.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff840154420 con 0x7ff8400b3f40 2024-09-10T09:59:00.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84015ca10 con 0x7ff8400b3f40 2024-09-10T09:59:00.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83cc00640 1 -- 172.21.15.110:0/1644834356 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff8300c1460 con 0x7ff8400b3f40 2024-09-10T09:59:00.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83cc00640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff81c0678f0 0x7ff81c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:00.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff83cc00640 1 -- 172.21.15.110:0/1644834356 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7ff830120450 con 0x7ff8400b3f40 2024-09-10T09:59:00.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.696+0000 7ff837e00640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff81c0678f0 0x7ff81c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:00.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.700+0000 7ff837e00640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff81c0678f0 0x7ff81c069db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cecf0 tx=0x7ff828009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.700+0000 7ff83cc00640 1 -- 172.21.15.110:0/1644834356 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff830016090 con 0x7ff8400b3f40 2024-09-10T09:59:00.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.916+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7ff84014d490 con 0x7ff8400b3f40 2024-09-10T09:59:00.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff83cc00640 1 -- 172.21.15.110:0/1644834356 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7ff8300f1240 con 0x7ff8400b3f40 2024-09-10T09:59:00.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff81c0678f0 msgr2=0x7ff81c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff81c0678f0 0x7ff81c069db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cecf0 tx=0x7ff828009040 comp rx=0 tx=0).stop 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400b3f40 msgr2=0x7ff840156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400b3f40 0x7ff840156040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8300097c0 tx=0x7ff83001ac50 comp rx=0 tx=0).stop 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 shutdown_connections 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff81c0678f0 0x7ff81c069db0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8400cfb90 0x7ff84015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8400cdcb0 0x7ff84014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 --2- 172.21.15.110:0/1644834356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8400b3f40 0x7ff840156040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 >> 172.21.15.110:0/1644834356 conn(0x7ff840062d10 msgr2=0x7ff84015f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.920+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 shutdown_connections 2024-09-10T09:59:00.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:00.924+0000 7ff847000640 1 -- 172.21.15.110:0/1644834356 wait complete. 2024-09-10T09:59:00.943 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T09:59:01.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a9200640 1 Processor -- start 2024-09-10T09:59:01.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a9200640 1 -- start start 2024-09-10T09:59:01.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a9200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f80a40b5270 0x7f80a40729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a9200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f80a40b56e0 0x7f80a4072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a9200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a40b41d0 con 0x7f80a415aa30 2024-09-10T09:59:01.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a9200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f80a40b44b0 con 0x7f80a40b56e0 2024-09-10T09:59:01.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a9200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f80a40b4790 con 0x7f80a40b5270 2024-09-10T09:59:01.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a2800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f80a40b56e0 0x7f80a4072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a2800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f80a40b56e0 0x7f80a4072ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36492/0 (socket says 172.21.15.110:36492) 2024-09-10T09:59:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a2800640 1 -- 172.21.15.110:0/967091131 learned_addr learned my addr 172.21.15.110:0/967091131 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a1400640 1 --2- 172.21.15.110:0/967091131 >> v2:172.21.15.145:3300/0 conn(0x7f80a40b5270 0x7f80a40729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a2800640 1 -- 172.21.15.110:0/967091131 >> v2:172.21.15.145:3300/0 conn(0x7f80a40b5270 msgr2=0x7f80a40729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a2800640 1 --2- 172.21.15.110:0/967091131 >> v2:172.21.15.145:3300/0 conn(0x7f80a40b5270 0x7f80a40729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a2800640 1 -- 172.21.15.110:0/967091131 >> v1:172.21.15.145:6789/0 conn(0x7f80a415aa30 legacy=0x7f80a415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a2800640 1 -- 172.21.15.110:0/967091131 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a40b4a70 con 0x7f80a40b56e0 2024-09-10T09:59:01.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.076+0000 7f80a1400640 1 --2- 172.21.15.110:0/967091131 >> v2:172.21.15.145:3300/0 conn(0x7f80a40b5270 0x7f80a40729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:01.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a2800640 1 --2- 172.21.15.110:0/967091131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b56e0 0x7f80a4072ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80a40cd620 tx=0x7f809c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddd71be7338c372 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:01.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a0a00640 1 -- 172.21.15.110:0/967091131 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c0b4070 con 0x7f80a40b56e0 2024-09-10T09:59:01.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a0a00640 1 -- 172.21.15.110:0/967091131 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809c0a7ba0 con 0x7f80a40b56e0 2024-09-10T09:59:01.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a0a00640 1 -- 172.21.15.110:0/967091131 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c0af440 con 0x7f80a40b56e0 2024-09-10T09:59:01.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a9200640 1 -- 172.21.15.110:0/967091131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b56e0 msgr2=0x7f80a4072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a9200640 1 --2- 172.21.15.110:0/967091131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b56e0 0x7f80a4072ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80a40cd620 tx=0x7f809c01eb30 comp rx=0 tx=0).stop 2024-09-10T09:59:01.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a9200640 1 -- 172.21.15.110:0/967091131 shutdown_connections 2024-09-10T09:59:01.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a9200640 1 --2- 172.21.15.110:0/967091131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b56e0 0x7f80a4072ef0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a9200640 1 --2- 172.21.15.110:0/967091131 >> v2:172.21.15.145:3300/0 conn(0x7f80a40b5270 0x7f80a40729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a9200640 1 -- 172.21.15.110:0/967091131 >> 172.21.15.110:0/967091131 conn(0x7f80a406cb90 msgr2=0x7f80a406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:01.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a9200640 1 -- 172.21.15.110:0/967091131 shutdown_connections 2024-09-10T09:59:01.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.080+0000 7f80a9200640 1 -- 172.21.15.110:0/967091131 wait complete. 2024-09-10T09:59:01.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 Processor -- start 2024-09-10T09:59:01.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 -- start start 2024-09-10T09:59:01.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b5270 0x7f80a4161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a40b56e0 0x7f80a41557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80a415aa30 0x7f80a415d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80a415af90 con 0x7f80a40b5270 2024-09-10T09:59:01.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f80a415ae10 con 0x7f80a40b56e0 2024-09-10T09:59:01.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f80a40ce4b0 con 0x7f80a415aa30 2024-09-10T09:59:01.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b5270 0x7f80a4161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b5270 0x7f80a4161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36500/0 (socket says 172.21.15.110:36500) 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 -- 172.21.15.110:0/3931540463 learned_addr learned my addr 172.21.15.110:0/3931540463 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1400640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a40b56e0 0x7f80a41557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a2800640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80a415aa30 0x7f80a415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 -- 172.21.15.110:0/3931540463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80a415aa30 msgr2=0x7f80a415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80a415aa30 0x7f80a415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 -- 172.21.15.110:0/3931540463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a40b56e0 msgr2=0x7f80a41557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a40b56e0 0x7f80a41557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 -- 172.21.15.110:0/3931540463 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a41600e0 con 0x7f80a40b5270 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a1e00640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b5270 0x7f80a4161180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f809400c730 tx=0x7f809400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f809a000640 1 -- 172.21.15.110:0/3931540463 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094016020 con 0x7f80a40b5270 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f809a000640 1 -- 172.21.15.110:0/3931540463 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8094006d60 con 0x7f80a40b5270 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f809a000640 1 -- 172.21.15.110:0/3931540463 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094003bc0 con 0x7f80a40b5270 2024-09-10T09:59:01.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a41603c0 con 0x7f80a40b5270 2024-09-10T09:59:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.084+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a4160980 con 0x7f80a40b5270 2024-09-10T09:59:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.088+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a415f100 con 0x7f80a40b5270 2024-09-10T09:59:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.088+0000 7f809a000640 1 -- 172.21.15.110:0/3931540463 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f809400a850 con 0x7f80a40b5270 2024-09-10T09:59:01.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.088+0000 7f809a000640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f80840678c0 0x7f8084069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.088+0000 7f809a000640 1 -- 172.21.15.110:0/3931540463 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f8094085620 con 0x7f80a40b5270 2024-09-10T09:59:01.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.088+0000 7f80a1400640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f80840678c0 0x7f8084069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.088+0000 7f80a1400640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f80840678c0 0x7f8084069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f80a4165270 tx=0x7f8090008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:01.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.092+0000 7f809a000640 1 -- 172.21.15.110:0/3931540463 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8094020020 con 0x7f80a40b5270 2024-09-10T09:59:01.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.252+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f80a40d00b0 con 0x7f80a40b5270 2024-09-10T09:59:01.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.252+0000 7f809a000640 1 -- 172.21.15.110:0/3931540463 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7f80940564a0 con 0x7f80a40b5270 2024-09-10T09:59:01.255 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:01.255 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T09:59:01.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:01.255 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T09:59:01.256 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T09:59:01.256 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:01.256 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f80840678c0 msgr2=0x7f8084069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f80840678c0 0x7f8084069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f80a4165270 tx=0x7f8090008040 comp rx=0 tx=0).stop 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b5270 msgr2=0x7f80a4161180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b5270 0x7f80a4161180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f809400c730 tx=0x7f809400cbf0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 shutdown_connections 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f80840678c0 0x7f8084069d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80a415aa30 0x7f80a415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80a40b56e0 0x7f80a41557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 --2- 172.21.15.110:0/3931540463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80a40b5270 0x7f80a4161180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 >> 172.21.15.110:0/3931540463 conn(0x7f80a406cb90 msgr2=0x7f80a40ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:01.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 shutdown_connections 2024-09-10T09:59:01.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.256+0000 7f80a9200640 1 -- 172.21.15.110:0/3931540463 wait complete. 2024-09-10T09:59:01.278 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T09:59:01.278 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T09:59:01.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.416+0000 7f9e82200640 1 Processor -- start 2024-09-10T09:59:01.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.416+0000 7f9e82200640 1 -- start start 2024-09-10T09:59:01.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.416+0000 7f9e82200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 0x7f9e7c0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.416+0000 7f9e82200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e7c0aed80 0x7f9e7c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.416+0000 7f9e82200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e7c06ec20 0x7f9e7c06d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.416+0000 7f9e82200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7c0b34d0 con 0x7f9e7c06ec20 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.416+0000 7f9e82200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7c0af1a0 con 0x7f9e7c0aed80 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.416+0000 7f9e82200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c1482e0 con 0x7f9e7c147f00 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e7c06ec20 0x7f9e7c06d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e7c06ec20 0x7f9e7c06d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36502/0 (socket says 172.21.15.110:36502) 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 -- 172.21.15.110:0/3660869203 learned_addr learned my addr 172.21.15.110:0/3660869203 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e73e00640 1 --2- 172.21.15.110:0/3660869203 >> v2:172.21.15.145:3300/0 conn(0x7f9e7c0aed80 0x7f9e7c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7b400640 1 --2- 172.21.15.110:0/3660869203 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 0x7f9e7c0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 -- 172.21.15.110:0/3660869203 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 msgr2=0x7f9e7c0ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:01.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 --2- 172.21.15.110:0/3660869203 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 0x7f9e7c0ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 -- 172.21.15.110:0/3660869203 >> v2:172.21.15.145:3300/0 conn(0x7f9e7c0aed80 msgr2=0x7f9e7c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:01.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 --2- 172.21.15.110:0/3660869203 >> v2:172.21.15.145:3300/0 conn(0x7f9e7c0aed80 0x7f9e7c06e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 -- 172.21.15.110:0/3660869203 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c15ee20 con 0x7f9e7c06ec20 2024-09-10T09:59:01.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7be00640 1 --2- 172.21.15.110:0/3660869203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c06ec20 0x7f9e7c06d130 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c06c8b0 tx=0x7f9e7401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9db12add3e53c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:01.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7aa00640 1 -- 172.21.15.110:0/3660869203 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e740ac070 con 0x7f9e7c06ec20 2024-09-10T09:59:01.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7aa00640 1 -- 172.21.15.110:0/3660869203 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e740a8420 con 0x7f9e7c06ec20 2024-09-10T09:59:01.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.420+0000 7f9e7aa00640 1 -- 172.21.15.110:0/3660869203 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e740a87f0 con 0x7f9e7c06ec20 2024-09-10T09:59:01.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- 172.21.15.110:0/3660869203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c06ec20 msgr2=0x7f9e7c06d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 --2- 172.21.15.110:0/3660869203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c06ec20 0x7f9e7c06d130 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c06c8b0 tx=0x7f9e7401c730 comp rx=0 tx=0).stop 2024-09-10T09:59:01.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- 172.21.15.110:0/3660869203 shutdown_connections 2024-09-10T09:59:01.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 --2- 172.21.15.110:0/3660869203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c06ec20 0x7f9e7c06d130 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 --2- 172.21.15.110:0/3660869203 >> v2:172.21.15.145:3300/0 conn(0x7f9e7c0aed80 0x7f9e7c06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 --2- 172.21.15.110:0/3660869203 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 0x7f9e7c0ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- 172.21.15.110:0/3660869203 >> 172.21.15.110:0/3660869203 conn(0x7f9e7c06bc50 msgr2=0x7f9e7c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:01.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- 172.21.15.110:0/3660869203 shutdown_connections 2024-09-10T09:59:01.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- 172.21.15.110:0/3660869203 wait complete. 2024-09-10T09:59:01.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 Processor -- start 2024-09-10T09:59:01.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- start start 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e7c06ec20 0x7f9e7c142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c0aed80 0x7f9e7c139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 0x7f9e7c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c0722f0 con 0x7f9e7c0aed80 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c06f0d0 con 0x7f9e7c06ec20 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e82200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c072470 con 0x7f9e7c147f00 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e73e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c0aed80 0x7f9e7c139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e73e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c0aed80 0x7f9e7c139eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36510/0 (socket says 172.21.15.110:36510) 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e73e00640 1 -- 172.21.15.110:0/172290598 learned_addr learned my addr 172.21.15.110:0/172290598 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:01.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e7b400640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e7c06ec20 0x7f9e7c142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e7be00640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 0x7f9e7c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e73e00640 1 -- 172.21.15.110:0/172290598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 msgr2=0x7f9e7c143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e73e00640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 0x7f9e7c143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e73e00640 1 -- 172.21.15.110:0/172290598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e7c06ec20 msgr2=0x7f9e7c142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e73e00640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e7c06ec20 0x7f9e7c142120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.424+0000 7f9e73e00640 1 -- 172.21.15.110:0/172290598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c141360 con 0x7f9e7c0aed80 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e73e00640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c0aed80 0x7f9e7c139eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400e8b0 tx=0x7f9e6400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e73400640 1 -- 172.21.15.110:0/172290598 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64017070 con 0x7f9e7c0aed80 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e73400640 1 -- 172.21.15.110:0/172290598 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6400c940 con 0x7f9e7c0aed80 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c141640 con 0x7f9e7c0aed80 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e73400640 1 -- 172.21.15.110:0/172290598 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6400cd10 con 0x7f9e7c0aed80 2024-09-10T09:59:01.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c13a3f0 con 0x7f9e7c0aed80 2024-09-10T09:59:01.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c06e6e0 con 0x7f9e7c0aed80 2024-09-10T09:59:01.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e73400640 1 -- 172.21.15.110:0/172290598 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e6401a070 con 0x7f9e7c0aed80 2024-09-10T09:59:01.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e73400640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e580678c0 0x7f9e58069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:01.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e73400640 1 -- 172.21.15.110:0/172290598 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f9e64085420 con 0x7f9e7c0aed80 2024-09-10T09:59:01.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.428+0000 7f9e7b400640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e580678c0 0x7f9e58069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:01.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.432+0000 7f9e7b400640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e580678c0 0x7f9e58069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009ac0 tx=0x7f9e6c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:01.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.432+0000 7f9e73400640 1 -- 172.21.15.110:0/172290598 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9e64055cf0 con 0x7f9e7c0aed80 2024-09-10T09:59:01.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:01.592+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9e7c075020 con 0x7f9e7c0aed80 2024-09-10T09:59:02.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.180+0000 7f9e73400640 1 -- 172.21.15.110:0/172290598 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7f9e64059ff0 con 0x7f9e7c0aed80 2024-09-10T09:59:02.184 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e580678c0 msgr2=0x7f9e58069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e580678c0 0x7f9e58069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009ac0 tx=0x7f9e6c009190 comp rx=0 tx=0).stop 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c0aed80 msgr2=0x7f9e7c139eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c0aed80 0x7f9e7c139eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400e8b0 tx=0x7f9e6400ed70 comp rx=0 tx=0).stop 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 shutdown_connections 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e580678c0 0x7f9e58069d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e7c147f00 0x7f9e7c143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e7c0aed80 0x7f9e7c139eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 --2- 172.21.15.110:0/172290598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e7c06ec20 0x7f9e7c142120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 >> 172.21.15.110:0/172290598 conn(0x7f9e7c06bc50 msgr2=0x7f9e7c0aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:02.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 shutdown_connections 2024-09-10T09:59:02.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.184+0000 7f9e82200640 1 -- 172.21.15.110:0/172290598 wait complete. 2024-09-10T09:59:02.218 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T09:59:02.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.360+0000 7f1cb2a00640 1 Processor -- start 2024-09-10T09:59:02.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.360+0000 7f1cb2a00640 1 -- start start 2024-09-10T09:59:02.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb2a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1cac15b170 0x7f1cac073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:02.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb2a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac0cf2e0 0x7f1cac0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:02.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb2a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cac0b3b40 con 0x7f1cac15ed40 2024-09-10T09:59:02.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb2a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1cac0b3910 con 0x7f1cac15b170 2024-09-10T09:59:02.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb2a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac0cfc60 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb1600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1cac15b170 0x7f1cac073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:02.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb0c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac0cf2e0 0x7f1cac0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:02.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb1600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1cac15b170 0x7f1cac073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:52670/0 (socket says 172.21.15.110:52670) 2024-09-10T09:59:02.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb1600640 1 -- 172.21.15.110:0/3254222873 learned_addr learned my addr 172.21.15.110:0/3254222873 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:02.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb0c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac0cf2e0 0x7f1cac0cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:34244/0 (socket says 172.21.15.110:34244) 2024-09-10T09:59:02.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb1600640 1 -- 172.21.15.110:0/3254222873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac0cf2e0 msgr2=0x7f1cac0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:02.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb1600640 1 --2- 172.21.15.110:0/3254222873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac0cf2e0 0x7f1cac0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb1600640 1 -- 172.21.15.110:0/3254222873 >> v1:172.21.15.145:6789/0 conn(0x7f1cac15ed40 legacy=0x7f1cac15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:02.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.364+0000 7f1cb1600640 1 -- 172.21.15.110:0/3254222873 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac0b3e20 con 0x7f1cac15b170 2024-09-10T09:59:02.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb1600640 1 --2- 172.21.15.110:0/3254222873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 0x7f1cac073390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009e50 tx=0x7f1ca001ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15c041272aea63e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:02.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cabe00640 1 -- 172.21.15.110:0/3254222873 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00a57d0 con 0x7f1cac15b170 2024-09-10T09:59:02.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cabe00640 1 -- 172.21.15.110:0/3254222873 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca00a5970 con 0x7f1cac15b170 2024-09-10T09:59:02.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cabe00640 1 -- 172.21.15.110:0/3254222873 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00a5d40 con 0x7f1cac15b170 2024-09-10T09:59:02.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 -- 172.21.15.110:0/3254222873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 msgr2=0x7f1cac073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:02.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/3254222873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 0x7f1cac073390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009e50 tx=0x7f1ca001ca00 comp rx=0 tx=0).stop 2024-09-10T09:59:02.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 -- 172.21.15.110:0/3254222873 shutdown_connections 2024-09-10T09:59:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/3254222873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac0cf2e0 0x7f1cac0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/3254222873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 0x7f1cac073390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 -- 172.21.15.110:0/3254222873 >> 172.21.15.110:0/3254222873 conn(0x7f1cac06d0f0 msgr2=0x7f1cac06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 -- 172.21.15.110:0/3254222873 shutdown_connections 2024-09-10T09:59:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 -- 172.21.15.110:0/3254222873 wait complete. 2024-09-10T09:59:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 Processor -- start 2024-09-10T09:59:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.368+0000 7f1cb2a00640 1 -- start start 2024-09-10T09:59:02.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cac0cf2e0 0x7f1cac0aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:02.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 0x7f1cac155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:02.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac15ed40 0x7f1cac156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:02.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac072680 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac15b550 con 0x7f1cac15b170 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac072800 con 0x7f1cac15ed40 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cac0cf2e0 0x7f1cac0aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cac0cf2e0 0x7f1cac0aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36520/0 (socket says 172.21.15.110:36520) 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 -- 172.21.15.110:0/768641397 learned_addr learned my addr 172.21.15.110:0/768641397 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb0c00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 0x7f1cac155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2000640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac15ed40 0x7f1cac156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 -- 172.21.15.110:0/768641397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac15ed40 msgr2=0x7f1cac156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac15ed40 0x7f1cac156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 -- 172.21.15.110:0/768641397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 msgr2=0x7f1cac155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 0x7f1cac155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 -- 172.21.15.110:0/768641397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac160dd0 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb1600640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cac0cf2e0 0x7f1cac0aa690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ca001a630 tx=0x7f1ca00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:02.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1ca9600640 1 -- 172.21.15.110:0/768641397 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00bc020 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1ca9600640 1 -- 172.21.15.110:0/768641397 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca00b8670 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cac1610b0 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1ca9600640 1 -- 172.21.15.110:0/768641397 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00b8a60 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cac161670 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.372+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cac0cf720 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.376+0000 7f1ca9600640 1 -- 172.21.15.110:0/768641397 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ca00bf020 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.376+0000 7f1ca9600640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1c8c0678c0 0x7f1c8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.376+0000 7f1ca9600640 1 -- 172.21.15.110:0/768641397 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7f1ca0121180 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.376+0000 7f1cb0c00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1c8c0678c0 0x7f1c8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.376+0000 7f1cb0c00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1c8c0678c0 0x7f1c8c069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1cac1628a0 tx=0x7f1c9800b3a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:02.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.380+0000 7f1ca9600640 1 -- 172.21.15.110:0/768641397 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1ca00c6030 con 0x7f1cac0cf2e0 2024-09-10T09:59:02.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:02.536+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1cac06f290 con 0x7f1cac0cf2e0 2024-09-10T09:59:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.184+0000 7f1ca9600640 1 -- 172.21.15.110:0/768641397 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7f1ca0016620 con 0x7f1cac0cf2e0 2024-09-10T09:59:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1c8c0678c0 msgr2=0x7f1c8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1c8c0678c0 0x7f1c8c069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1cac1628a0 tx=0x7f1c9800b3a0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cac0cf2e0 msgr2=0x7f1cac0aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cac0cf2e0 0x7f1cac0aa690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ca001a630 tx=0x7f1ca00197a0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 shutdown_connections 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1c8c0678c0 0x7f1c8c069d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1cac15ed40 0x7f1cac156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cac15b170 0x7f1cac155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 --2- 172.21.15.110:0/768641397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cac0cf2e0 0x7f1cac0aa690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 >> 172.21.15.110:0/768641397 conn(0x7f1cac06d0f0 msgr2=0x7f1cac1653d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 shutdown_connections 2024-09-10T09:59:03.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.188+0000 7f1cb2a00640 1 -- 172.21.15.110:0/768641397 wait complete. 2024-09-10T09:59:03.224 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T09:59:03.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1e000640 1 Processor -- start 2024-09-10T09:59:03.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1e000640 1 -- start start 2024-09-10T09:59:03.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18070660 0x7f9c180b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:03.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1e000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c180b2690 con 0x7f9c1806bc50 2024-09-10T09:59:03.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1e000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c180b2970 con 0x7f9c18148000 2024-09-10T09:59:03.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9c180b2c50 con 0x7f9c18070660 2024-09-10T09:59:03.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1cc00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9c1806bc50 0x7f9c1806c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43838/0 (socket says 172.21.15.110:43838) 2024-09-10T09:59:03.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1cc00640 1 -- 172.21.15.110:0/2338606043 learned_addr learned my addr 172.21.15.110:0/2338606043 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:03.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.360+0000 7f9c1d600640 1 --2- 172.21.15.110:0/2338606043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18070660 0x7f9c180b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:03.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c17e00640 1 -- 172.21.15.110:0/2338606043 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410621835 0 0) 0x7f9c180b2690 con 0x7f9c1806bc50 2024-09-10T09:59:03.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c17e00640 1 -- 172.21.15.110:0/2338606043 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bfc003530 con 0x7f9c1806bc50 2024-09-10T09:59:03.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c17e00640 1 -- 172.21.15.110:0/2338606043 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342571505 0 0) 0x7f9c180b2970 con 0x7f9c18148000 2024-09-10T09:59:03.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c17e00640 1 -- 172.21.15.110:0/2338606043 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c180b2690 con 0x7f9c18148000 2024-09-10T09:59:03.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c1d600640 1 -- 172.21.15.110:0/2338606043 >> v1:172.21.15.145:6789/0 conn(0x7f9c18148000 legacy=0x7f9c180b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:03.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c1d600640 1 -- 172.21.15.110:0/2338606043 >> v1:172.21.15.110:6789/0 conn(0x7f9c1806bc50 legacy=0x7f9c1806c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:03.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c1d600640 1 -- 172.21.15.110:0/2338606043 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c180b2f30 con 0x7f9c18070660 2024-09-10T09:59:03.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c1d600640 1 --2- 172.21.15.110:0/2338606043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18070660 0x7f9c180b5be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009dc0 tx=0x7f9c1001c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3827fc44b39ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:03.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c17e00640 1 -- 172.21.15.110:0/2338606043 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c100b2070 con 0x7f9c18070660 2024-09-10T09:59:03.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c17e00640 1 -- 172.21.15.110:0/2338606043 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c100a5730 con 0x7f9c18070660 2024-09-10T09:59:03.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.364+0000 7f9c17e00640 1 -- 172.21.15.110:0/2338606043 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c100a5ae0 con 0x7f9c18070660 2024-09-10T09:59:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- 172.21.15.110:0/2338606043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18070660 msgr2=0x7f9c180b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 --2- 172.21.15.110:0/2338606043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18070660 0x7f9c180b5be0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009dc0 tx=0x7f9c1001c870 comp rx=0 tx=0).stop 2024-09-10T09:59:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- 172.21.15.110:0/2338606043 shutdown_connections 2024-09-10T09:59:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 --2- 172.21.15.110:0/2338606043 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18070660 0x7f9c180b5be0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- 172.21.15.110:0/2338606043 >> 172.21.15.110:0/2338606043 conn(0x7f9c1806f7e0 msgr2=0x7f9c1806fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- 172.21.15.110:0/2338606043 shutdown_connections 2024-09-10T09:59:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- 172.21.15.110:0/2338606043 wait complete. 2024-09-10T09:59:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 Processor -- start 2024-09-10T09:59:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- start start 2024-09-10T09:59:03.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c1806bc50 0x7f9c1815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:03.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c18070660 0x7f9c181444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:03.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18148000 0x7f9c1813eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:03.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1816cc90 con 0x7f9c1806bc50 2024-09-10T09:59:03.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1816cb10 con 0x7f9c18070660 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1816ce10 con 0x7f9c18148000 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c1806bc50 0x7f9c1815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c1806bc50 0x7f9c1815d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36524/0 (socket says 172.21.15.110:36524) 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1cc00640 1 -- 172.21.15.110:0/2355910013 learned_addr learned my addr 172.21.15.110:0/2355910013 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c0fe00640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c18070660 0x7f9c181444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1d600640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18148000 0x7f9c1813eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1cc00640 1 -- 172.21.15.110:0/2355910013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18148000 msgr2=0x7f9c1813eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1cc00640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18148000 0x7f9c1813eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1cc00640 1 -- 172.21.15.110:0/2355910013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c18070660 msgr2=0x7f9c181444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1cc00640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c18070660 0x7f9c181444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.368+0000 7f9c1cc00640 1 -- 172.21.15.110:0/2355910013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1813db20 con 0x7f9c1806bc50 2024-09-10T09:59:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c1cc00640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c1806bc50 0x7f9c1815d6a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c4d0 tx=0x7f9c0800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:03.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c15600640 1 -- 172.21.15.110:0/2355910013 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08017070 con 0x7f9c1806bc50 2024-09-10T09:59:03.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c15600640 1 -- 172.21.15.110:0/2355910013 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c08005cf0 con 0x7f9c1806bc50 2024-09-10T09:59:03.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c15600640 1 -- 172.21.15.110:0/2355910013 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080072b0 con 0x7f9c1806bc50 2024-09-10T09:59:03.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1813ddc0 con 0x7f9c1806bc50 2024-09-10T09:59:03.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1813e380 con 0x7f9c1806bc50 2024-09-10T09:59:03.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1806c030 con 0x7f9c1806bc50 2024-09-10T09:59:03.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c15600640 1 -- 172.21.15.110:0/2355910013 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9c0801a070 con 0x7f9c1806bc50 2024-09-10T09:59:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c15600640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9bf40678f0 0x7f9bf4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c15600640 1 -- 172.21.15.110:0/2355910013 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7f9c08084ff0 con 0x7f9c1806bc50 2024-09-10T09:59:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.372+0000 7f9c0fe00640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9bf40678f0 0x7f9bf4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.376+0000 7f9c0fe00640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9bf40678f0 0x7f9bf4069db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9c1813c2c0 tx=0x7f9c00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:03.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.380+0000 7f9c15600640 1 -- 172.21.15.110:0/2355910013 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9c080554a0 con 0x7f9c1806bc50 2024-09-10T09:59:03.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:03.568+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9c180d1280 con 0x7f9c1806bc50 2024-09-10T09:59:04.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.204+0000 7f9c15600640 1 -- 172.21.15.110:0/2355910013 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f9c080597a0 con 0x7f9c1806bc50 2024-09-10T09:59:04.209 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 4 and data pool 5 2024-09-10T09:59:04.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.208+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9bf40678f0 msgr2=0x7f9bf4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.208+0000 7f9c1e000640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9bf40678f0 0x7f9bf4069db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9c1813c2c0 tx=0x7f9c00009040 comp rx=0 tx=0).stop 2024-09-10T09:59:04.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.208+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c1806bc50 msgr2=0x7f9c1815d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.208+0000 7f9c1e000640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c1806bc50 0x7f9c1815d6a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c4d0 tx=0x7f9c0800c990 comp rx=0 tx=0).stop 2024-09-10T09:59:04.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.212+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 shutdown_connections 2024-09-10T09:59:04.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.212+0000 7f9c1e000640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9bf40678f0 0x7f9bf4069db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.212+0000 7f9c1e000640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c18148000 0x7f9c1813eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.212+0000 7f9c1e000640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c18070660 0x7f9c181444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.212+0000 7f9c1e000640 1 --2- 172.21.15.110:0/2355910013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c1806bc50 0x7f9c1815d6a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.212+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 >> 172.21.15.110:0/2355910013 conn(0x7f9c1806f7e0 msgr2=0x7f9c180b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.212+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 shutdown_connections 2024-09-10T09:59:04.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.212+0000 7f9c1e000640 1 -- 172.21.15.110:0/2355910013 wait complete. 2024-09-10T09:59:04.261 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:04.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.396+0000 7f93a3c00640 1 Processor -- start 2024-09-10T09:59:04.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.396+0000 7f93a3c00640 1 -- start start 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.396+0000 7f93a3c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f939c0cfe30 0x7f939c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.396+0000 7f93a3c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cd510 0x7f939c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.396+0000 7f93a3c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f939c06d2d0 con 0x7f939c06e090 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.396+0000 7f93a3c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f939c0cdeb0 con 0x7f939c0cfe30 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.396+0000 7f93a3c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f939c15d9a0 con 0x7f939c0cd510 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f93a1200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f939c0cfe30 0x7f939c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f9393e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cd510 0x7f939c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f93a1200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f939c0cfe30 0x7f939c071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:52696/0 (socket says 172.21.15.110:52696) 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f93a1200640 1 -- 172.21.15.110:0/390393167 learned_addr learned my addr 172.21.15.110:0/390393167 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f93a1c00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f939c06e090 0x7f939c06e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:38638/0 (socket says 172.21.15.110:38638) 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f9393e00640 1 -- 172.21.15.110:0/390393167 >> v2:172.21.15.145:3300/0 conn(0x7f939c0cfe30 msgr2=0x7f939c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f9393e00640 1 --2- 172.21.15.110:0/390393167 >> v2:172.21.15.145:3300/0 conn(0x7f939c0cfe30 0x7f939c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f9393e00640 1 -- 172.21.15.110:0/390393167 >> v1:172.21.15.145:6789/0 conn(0x7f939c06e090 legacy=0x7f939c06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f9393e00640 1 -- 172.21.15.110:0/390393167 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c06d5b0 con 0x7f939c0cd510 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f939be00640 1 -- 172.21.15.110:0/390393167 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 116437448 0 0) 0x7f939c06d2d0 con 0x7f939c06e090 2024-09-10T09:59:04.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f93a1200640 1 --2- 172.21.15.110:0/390393167 >> v2:172.21.15.145:3300/0 conn(0x7f939c0cfe30 0x7f939c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.400+0000 7f9393e00640 1 --2- 172.21.15.110:0/390393167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cd510 0x7f939c0cd970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f93840097c0 tx=0x7f938401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a325577c44927261 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f939be00640 1 -- 172.21.15.110:0/390393167 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93840b0070 con 0x7f939c0cd510 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f939be00640 1 -- 172.21.15.110:0/390393167 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93840ab600 con 0x7f939c0cd510 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f939be00640 1 -- 172.21.15.110:0/390393167 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93840ab9d0 con 0x7f939c0cd510 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- 172.21.15.110:0/390393167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cd510 msgr2=0x7f939c0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 --2- 172.21.15.110:0/390393167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cd510 0x7f939c0cd970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f93840097c0 tx=0x7f938401c410 comp rx=0 tx=0).stop 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- 172.21.15.110:0/390393167 shutdown_connections 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 --2- 172.21.15.110:0/390393167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cd510 0x7f939c0cd970 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 --2- 172.21.15.110:0/390393167 >> v2:172.21.15.145:3300/0 conn(0x7f939c0cfe30 0x7f939c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- 172.21.15.110:0/390393167 >> 172.21.15.110:0/390393167 conn(0x7f939c06bc50 msgr2=0x7f939c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- 172.21.15.110:0/390393167 shutdown_connections 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- 172.21.15.110:0/390393167 wait complete. 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 Processor -- start 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- start start 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f939c06e090 0x7f939c154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c0cd510 0x7f939c14f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cfe30 0x7f939c072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f939c072190 con 0x7f939c0cd510 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f939c15d640 con 0x7f939c06e090 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a3c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f939c0b35d0 con 0x7f939c0cfe30 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f9393e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c0cd510 0x7f939c14f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f9393e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c0cd510 0x7f939c14f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36530/0 (socket says 172.21.15.110:36530) 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f9393e00640 1 -- 172.21.15.110:0/2577286362 learned_addr learned my addr 172.21.15.110:0/2577286362 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a1200640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f939c06e090 0x7f939c154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f93a1c00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cfe30 0x7f939c072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f9393e00640 1 -- 172.21.15.110:0/2577286362 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cfe30 msgr2=0x7f939c072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f9393e00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cfe30 0x7f939c072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f9393e00640 1 -- 172.21.15.110:0/2577286362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f939c06e090 msgr2=0x7f939c154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.404+0000 7f9393e00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f939c06e090 0x7f939c154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f9393e00640 1 -- 172.21.15.110:0/2577286362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c15faa0 con 0x7f939c0cd510 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f9393e00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c0cd510 0x7f939c14f310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f93840ae040 tx=0x7f938401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f9399600640 1 -- 172.21.15.110:0/2577286362 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93840b0050 con 0x7f939c0cd510 2024-09-10T09:59:04.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f9399600640 1 -- 172.21.15.110:0/2577286362 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93840b1040 con 0x7f939c0cd510 2024-09-10T09:59:04.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f9399600640 1 -- 172.21.15.110:0/2577286362 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93840b5610 con 0x7f939c0cd510 2024-09-10T09:59:04.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c0b5040 con 0x7f939c0cd510 2024-09-10T09:59:04.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c1567a0 con 0x7f939c0cd510 2024-09-10T09:59:04.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939c0cd970 con 0x7f939c0cd510 2024-09-10T09:59:04.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f9399600640 1 -- 172.21.15.110:0/2577286362 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f93840be030 con 0x7f939c0cd510 2024-09-10T09:59:04.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f9399600640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93780678f0 0x7f9378069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f9399600640 1 -- 172.21.15.110:0/2577286362 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f938411f7f0 con 0x7f939c0cd510 2024-09-10T09:59:04.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.408+0000 7f93a1200640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93780678f0 0x7f9378069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.412+0000 7f93a1200640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93780678f0 0x7f9378069db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f938c006cb0 tx=0x7f938c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.412+0000 7f9399600640 1 -- 172.21.15.110:0/2577286362 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f93840c5020 con 0x7f939c0cd510 2024-09-10T09:59:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.572+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f939c075560 con 0x7f939c0cd510 2024-09-10T09:59:04.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.572+0000 7f9399600640 1 -- 172.21.15.110:0/2577286362 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16778 (secure 0 0 0) 0x7f9384016090 con 0x7f939c0cd510 2024-09-10T09:59:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:04.575 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":33,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:03.572755+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:01.639334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:02.583881+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:04.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93780678f0 msgr2=0x7f9378069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93780678f0 0x7f9378069db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f938c006cb0 tx=0x7f938c009240 comp rx=0 tx=0).stop 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c0cd510 msgr2=0x7f939c14f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c0cd510 0x7f939c14f310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f93840ae040 tx=0x7f938401ac50 comp rx=0 tx=0).stop 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 shutdown_connections 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93780678f0 0x7f9378069db0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f939c0cfe30 0x7f939c072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f939c0cd510 0x7f939c14f310 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 --2- 172.21.15.110:0/2577286362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f939c06e090 0x7f939c154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 >> 172.21.15.110:0/2577286362 conn(0x7f939c06bc50 msgr2=0x7f939c15db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 shutdown_connections 2024-09-10T09:59:04.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.576+0000 7f93a3c00640 1 -- 172.21.15.110:0/2577286362 wait complete. 2024-09-10T09:59:04.596 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:04.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.728+0000 7f65fdc00640 1 Processor -- start 2024-09-10T09:59:04.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.728+0000 7f65fdc00640 1 -- start start 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65fdc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f65f815c560 0x7f65f806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65fdc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f806f010 0x7f65f80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65fdc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f806f430 con 0x7f65f8155970 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65fdc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f65f80b4570 con 0x7f65f815c560 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65fdc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f65f8155d50 con 0x7f65f806f010 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f6a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f65f8155970 0x7f65f80cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43852/0 (socket says 172.21.15.110:43852) 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f6a00640 1 -- 172.21.15.110:0/4117811116 learned_addr learned my addr 172.21.15.110:0/4117811116 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f6000640 1 --2- 172.21.15.110:0/4117811116 >> v2:172.21.15.145:3300/0 conn(0x7f65f815c560 0x7f65f806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f7400640 1 --2- 172.21.15.110:0/4117811116 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f806f010 0x7f65f80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f6000640 1 -- 172.21.15.110:0/4117811116 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f806f010 msgr2=0x7f65f80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f6000640 1 --2- 172.21.15.110:0/4117811116 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f806f010 0x7f65f80ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f6000640 1 -- 172.21.15.110:0/4117811116 >> v1:172.21.15.110:6789/0 conn(0x7f65f8155970 legacy=0x7f65f80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f6000640 1 -- 172.21.15.110:0/4117811116 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f80b34d0 con 0x7f65f815c560 2024-09-10T09:59:04.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.732+0000 7f65f5600640 1 -- 172.21.15.110:0/4117811116 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988577635 0 0) 0x7f65f806f430 con 0x7f65f8155970 2024-09-10T09:59:04.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65f6000640 1 --2- 172.21.15.110:0/4117811116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f815c560 0x7f65f806ead0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f65e40097c0 tx=0x7f65e401e680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e1669d74aa4915e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65f5600640 1 -- 172.21.15.110:0/4117811116 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40ae070 con 0x7f65f815c560 2024-09-10T09:59:04.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65f5600640 1 -- 172.21.15.110:0/4117811116 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e40a8650 con 0x7f65f815c560 2024-09-10T09:59:04.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65f5600640 1 -- 172.21.15.110:0/4117811116 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40a8a20 con 0x7f65f815c560 2024-09-10T09:59:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 -- 172.21.15.110:0/4117811116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f815c560 msgr2=0x7f65f806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4117811116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f815c560 0x7f65f806ead0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f65e40097c0 tx=0x7f65e401e680 comp rx=0 tx=0).stop 2024-09-10T09:59:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 -- 172.21.15.110:0/4117811116 shutdown_connections 2024-09-10T09:59:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4117811116 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f806f010 0x7f65f80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4117811116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f815c560 0x7f65f806ead0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 -- 172.21.15.110:0/4117811116 >> 172.21.15.110:0/4117811116 conn(0x7f65f806bc50 msgr2=0x7f65f806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 -- 172.21.15.110:0/4117811116 shutdown_connections 2024-09-10T09:59:04.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 -- 172.21.15.110:0/4117811116 wait complete. 2024-09-10T09:59:04.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 Processor -- start 2024-09-10T09:59:04.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.736+0000 7f65fdc00640 1 -- start start 2024-09-10T09:59:04.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f806f010 0x7f65f8154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f8155970 0x7f65f8142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f815c560 0x7f65f8141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80cf170 con 0x7f65f815c560 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80b4050 con 0x7f65f806f010 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f65f80cf2f0 con 0x7f65f8155970 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f815c560 0x7f65f8141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f815c560 0x7f65f8141660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36546/0 (socket says 172.21.15.110:36546) 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 -- 172.21.15.110:0/4195719601 learned_addr learned my addr 172.21.15.110:0/4195719601 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f6a00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f806f010 0x7f65f8154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f6000640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f8155970 0x7f65f8142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 -- 172.21.15.110:0/4195719601 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f8155970 msgr2=0x7f65f8142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f8155970 0x7f65f8142290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 -- 172.21.15.110:0/4195719601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f806f010 msgr2=0x7f65f8154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f806f010 0x7f65f8154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 -- 172.21.15.110:0/4195719601 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f8142d50 con 0x7f65f815c560 2024-09-10T09:59:04.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65f7400640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f815c560 0x7f65f8141660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f65f000e930 tx=0x7f65f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65eea00640 1 -- 172.21.15.110:0/4195719601 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0017070 con 0x7f65f815c560 2024-09-10T09:59:04.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65eea00640 1 -- 172.21.15.110:0/4195719601 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f000c9a0 con 0x7f65f815c560 2024-09-10T09:59:04.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f8143030 con 0x7f65f815c560 2024-09-10T09:59:04.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65eea00640 1 -- 172.21.15.110:0/4195719601 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f000cd70 con 0x7f65f815c560 2024-09-10T09:59:04.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f81435f0 con 0x7f65f815c560 2024-09-10T09:59:04.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f80ce420 con 0x7f65f815c560 2024-09-10T09:59:04.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.740+0000 7f65eea00640 1 -- 172.21.15.110:0/4195719601 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f65f001a070 con 0x7f65f815c560 2024-09-10T09:59:04.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.744+0000 7f65eea00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f65d80678c0 0x7f65d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.744+0000 7f65eea00640 1 -- 172.21.15.110:0/4195719601 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f65f0086110 con 0x7f65f815c560 2024-09-10T09:59:04.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.744+0000 7f65f6a00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f65d80678c0 0x7f65d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.744+0000 7f65f6a00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f65d80678c0 0x7f65d8069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f65e80012f0 tx=0x7f65e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.748+0000 7f65eea00640 1 -- 172.21.15.110:0/4195719601 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f65f0056c70 con 0x7f65f815c560 2024-09-10T09:59:04.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.904+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65f80d0170 con 0x7f65f815c560 2024-09-10T09:59:04.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65eea00640 1 -- 172.21.15.110:0/4195719601 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16778 (secure 0 0 0) 0x7f65f0056570 con 0x7f65f815c560 2024-09-10T09:59:04.910 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:04.910 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":33,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:03.572755+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:01.639334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:02.583881+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f65d80678c0 msgr2=0x7f65d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f65d80678c0 0x7f65d8069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f65e80012f0 tx=0x7f65e8009040 comp rx=0 tx=0).stop 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f815c560 msgr2=0x7f65f8141660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f815c560 0x7f65f8141660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f65f000e930 tx=0x7f65f000edf0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 shutdown_connections 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f65d80678c0 0x7f65d8069d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f815c560 0x7f65f8141660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f65f8155970 0x7f65f8142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 --2- 172.21.15.110:0/4195719601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f65f806f010 0x7f65f8154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 >> 172.21.15.110:0/4195719601 conn(0x7f65f806bc50 msgr2=0x7f65f80b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 shutdown_connections 2024-09-10T09:59:04.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:04.908+0000 7f65fdc00640 1 -- 172.21.15.110:0/4195719601 wait complete. 2024-09-10T09:59:04.931 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T09:59:05.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f773b400640 1 Processor -- start 2024-09-10T09:59:05.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f773b400640 1 -- start start 2024-09-10T09:59:05.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f773b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f77340cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:05.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f773b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f77340cfbc0 0x7f77340b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:05.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f773b400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77340b37c0 con 0x7f7734145f30 2024-09-10T09:59:05.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f773b400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f77340b3aa0 con 0x7f77340cfbc0 2024-09-10T09:59:05.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f773b400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f77340b3d80 con 0x7f77340b2bb0 2024-09-10T09:59:05.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7733e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f77340cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7739400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f77340cfbc0 0x7f77340b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7733e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f77340cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:34312/0 (socket says 172.21.15.110:34312) 2024-09-10T09:59:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7733e00640 1 -- 172.21.15.110:0/3476730060 learned_addr learned my addr 172.21.15.110:0/3476730060 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7733e00640 1 -- 172.21.15.110:0/3476730060 >> v2:172.21.15.145:3300/0 conn(0x7f77340cfbc0 msgr2=0x7f77340b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7733e00640 1 --2- 172.21.15.110:0/3476730060 >> v2:172.21.15.145:3300/0 conn(0x7f77340cfbc0 0x7f77340b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7733e00640 1 -- 172.21.15.110:0/3476730060 >> v1:172.21.15.145:6789/0 conn(0x7f7734145f30 legacy=0x7f77340b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7733e00640 1 -- 172.21.15.110:0/3476730060 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7734146310 con 0x7f77340b2bb0 2024-09-10T09:59:05.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.068+0000 7f7739400640 1 --2- 172.21.15.110:0/3476730060 >> v2:172.21.15.145:3300/0 conn(0x7f77340cfbc0 0x7f77340b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:05.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f7733e00640 1 --2- 172.21.15.110:0/3476730060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f77340cf680 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7720009870 tx=0x7f772001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb1c511973f0498 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:05.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f7733400640 1 -- 172.21.15.110:0/3476730060 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77200b2070 con 0x7f77340b2bb0 2024-09-10T09:59:05.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f7733400640 1 -- 172.21.15.110:0/3476730060 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77200a9070 con 0x7f77340b2bb0 2024-09-10T09:59:05.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f7733400640 1 -- 172.21.15.110:0/3476730060 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77200ad880 con 0x7f77340b2bb0 2024-09-10T09:59:05.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f773b400640 1 -- 172.21.15.110:0/3476730060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 msgr2=0x7f77340cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:05.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f773b400640 1 --2- 172.21.15.110:0/3476730060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f77340cf680 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7720009870 tx=0x7f772001c560 comp rx=0 tx=0).stop 2024-09-10T09:59:05.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f773b400640 1 -- 172.21.15.110:0/3476730060 shutdown_connections 2024-09-10T09:59:05.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f773b400640 1 --2- 172.21.15.110:0/3476730060 >> v2:172.21.15.145:3300/0 conn(0x7f77340cfbc0 0x7f77340b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:05.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f773b400640 1 --2- 172.21.15.110:0/3476730060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f77340cf680 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:05.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f773b400640 1 -- 172.21.15.110:0/3476730060 >> 172.21.15.110:0/3476730060 conn(0x7f7734062e40 msgr2=0x7f773406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:05.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f773b400640 1 -- 172.21.15.110:0/3476730060 shutdown_connections 2024-09-10T09:59:05.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.072+0000 7f773b400640 1 -- 172.21.15.110:0/3476730060 wait complete. 2024-09-10T09:59:05.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 Processor -- start 2024-09-10T09:59:05.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 -- start start 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f773413b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77340cfbc0 0x7f773414f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7734145f30 0x7f773413d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77340b56a0 con 0x7f77340cfbc0 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77340b5520 con 0x7f7734145f30 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f77340b5820 con 0x7f77340b2bb0 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77340cfbc0 0x7f773414f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77340cfbc0 0x7f773414f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36560/0 (socket says 172.21.15.110:36560) 2024-09-10T09:59:05.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 -- 172.21.15.110:0/2419386497 learned_addr learned my addr 172.21.15.110:0/2419386497 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:05.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7739400640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7734145f30 0x7f773413d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:05.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7738a00640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f773413b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:05.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 -- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 msgr2=0x7f773413b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:05.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f773413b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:05.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 -- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7734145f30 msgr2=0x7f773413d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:05.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7734145f30 0x7f773413d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 -- 172.21.15.110:0/2419386497 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773413b5e0 con 0x7f77340cfbc0 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7739400640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7734145f30 0x7f773413d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7733e00640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77340cfbc0 0x7f773414f6f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f772001a190 tx=0x7f7720019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7730c00640 1 -- 172.21.15.110:0/2419386497 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77200bc070 con 0x7f77340cfbc0 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7730c00640 1 -- 172.21.15.110:0/2419386497 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77200b6620 con 0x7f77340cfbc0 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773413e180 con 0x7f77340cfbc0 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f7730c00640 1 -- 172.21.15.110:0/2419386497 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77200b2040 con 0x7f77340cfbc0 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773413bb50 con 0x7f77340cfbc0 2024-09-10T09:59:05.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.076+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77340cf680 con 0x7f77340cfbc0 2024-09-10T09:59:05.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.080+0000 7f7730c00640 1 -- 172.21.15.110:0/2419386497 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f77200bf070 con 0x7f77340cfbc0 2024-09-10T09:59:05.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.080+0000 7f7730c00640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f77140678c0 0x7f7714069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:05.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.080+0000 7f7730c00640 1 -- 172.21.15.110:0/2419386497 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f772011f130 con 0x7f77340cfbc0 2024-09-10T09:59:05.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.080+0000 7f7738a00640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f77140678c0 0x7f7714069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:05.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.080+0000 7f7738a00640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f77140678c0 0x7f7714069d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77240012f0 tx=0x7f7724009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:05.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.084+0000 7f7730c00640 1 -- 172.21.15.110:0/2419386497 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7720016150 con 0x7f77340cfbc0 2024-09-10T09:59:05.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:05.280+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7734160bd0 con 0x7f77340cfbc0 2024-09-10T09:59:06.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f7730c00640 1 -- 172.21.15.110:0/2419386497 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f77200efbd0 con 0x7f77340cfbc0 2024-09-10T09:59:06.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f77140678c0 msgr2=0x7f7714069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:06.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f77140678c0 0x7f7714069d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77240012f0 tx=0x7f7724009040 comp rx=0 tx=0).stop 2024-09-10T09:59:06.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77340cfbc0 msgr2=0x7f773414f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:06.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77340cfbc0 0x7f773414f6f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f772001a190 tx=0x7f7720019300 comp rx=0 tx=0).stop 2024-09-10T09:59:06.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 shutdown_connections 2024-09-10T09:59:06.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f77140678c0 0x7f7714069d80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7734145f30 0x7f773413d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77340cfbc0 0x7f773414f6f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 --2- 172.21.15.110:0/2419386497 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f77340b2bb0 0x7f773413b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 >> 172.21.15.110:0/2419386497 conn(0x7f7734062e40 msgr2=0x7f77340cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:06.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 shutdown_connections 2024-09-10T09:59:06.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.248+0000 7f773b400640 1 -- 172.21.15.110:0/2419386497 wait complete. 2024-09-10T09:59:06.275 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T09:59:06.275 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T09:59:06.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.416+0000 7f2e37e00640 1 Processor -- start 2024-09-10T09:59:06.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.416+0000 7f2e37e00640 1 -- start start 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e37e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2e38070870 0x7f2e380d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e37e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2e3806f330 0x7f2e38070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e37e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 0x7f2e3806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e37e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3815e7e0 con 0x7f2e3806f330 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e37e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2e38071190 con 0x7f2e38070870 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e37e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2e38073760 con 0x7f2e380731a0 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2e3806f330 0x7f2e38070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2e3806f330 0x7f2e38070c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36574/0 (socket says 172.21.15.110:36574) 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 -- 172.21.15.110:0/4084185481 learned_addr learned my addr 172.21.15.110:0/4084185481 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:06.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e37400640 1 --2- 172.21.15.110:0/4084185481 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 0x7f2e3806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36a00640 1 --2- 172.21.15.110:0/4084185481 >> v2:172.21.15.145:3300/0 conn(0x7f2e38070870 0x7f2e380d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 -- 172.21.15.110:0/4084185481 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 msgr2=0x7f2e3806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:06.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 --2- 172.21.15.110:0/4084185481 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 0x7f2e3806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 -- 172.21.15.110:0/4084185481 >> v2:172.21.15.145:3300/0 conn(0x7f2e38070870 msgr2=0x7f2e380d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:06.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 --2- 172.21.15.110:0/4084185481 >> v2:172.21.15.145:3300/0 conn(0x7f2e38070870 0x7f2e380d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 -- 172.21.15.110:0/4084185481 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e38073a40 con 0x7f2e3806f330 2024-09-10T09:59:06.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.420+0000 7f2e36000640 1 --2- 172.21.15.110:0/4084185481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e38070c50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e380b85e0 tx=0x7f2e2401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd973932702b311 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:06.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e35600640 1 -- 172.21.15.110:0/4084185481 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e240b0070 con 0x7f2e3806f330 2024-09-10T09:59:06.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e35600640 1 -- 172.21.15.110:0/4084185481 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e240ab600 con 0x7f2e3806f330 2024-09-10T09:59:06.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e35600640 1 -- 172.21.15.110:0/4084185481 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e240ab9d0 con 0x7f2e3806f330 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- 172.21.15.110:0/4084185481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 msgr2=0x7f2e38070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 --2- 172.21.15.110:0/4084185481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e38070c50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2e380b85e0 tx=0x7f2e2401c490 comp rx=0 tx=0).stop 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- 172.21.15.110:0/4084185481 shutdown_connections 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 --2- 172.21.15.110:0/4084185481 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 0x7f2e3806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 --2- 172.21.15.110:0/4084185481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e38070c50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 --2- 172.21.15.110:0/4084185481 >> v2:172.21.15.145:3300/0 conn(0x7f2e38070870 0x7f2e380d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- 172.21.15.110:0/4084185481 >> 172.21.15.110:0/4084185481 conn(0x7f2e380cedf0 msgr2=0x7f2e380cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- 172.21.15.110:0/4084185481 shutdown_connections 2024-09-10T09:59:06.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- 172.21.15.110:0/4084185481 wait complete. 2024-09-10T09:59:06.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 Processor -- start 2024-09-10T09:59:06.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- start start 2024-09-10T09:59:06.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e3815bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e38070870 0x7f2e3815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 0x7f2e3815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e380d1c90 con 0x7f2e3806f330 2024-09-10T09:59:06.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2e380d1b10 con 0x7f2e38070870 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2e38169f70 con 0x7f2e380731a0 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e3815bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e3815bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36582/0 (socket says 172.21.15.110:36582) 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e36a00640 1 -- 172.21.15.110:0/2069365105 learned_addr learned my addr 172.21.15.110:0/2069365105 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e37400640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 0x7f2e3815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.424+0000 7f2e36000640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e38070870 0x7f2e3815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e36a00640 1 -- 172.21.15.110:0/2069365105 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 msgr2=0x7f2e3815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e36a00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 0x7f2e3815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e36a00640 1 -- 172.21.15.110:0/2069365105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e38070870 msgr2=0x7f2e3815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e36a00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e38070870 0x7f2e3815b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e36a00640 1 -- 172.21.15.110:0/2069365105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3815f870 con 0x7f2e3806f330 2024-09-10T09:59:06.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e36a00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e3815bdf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800d800 tx=0x7f2e2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:06.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e2ea00640 1 -- 172.21.15.110:0/2069365105 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800b6c0 con 0x7f2e3806f330 2024-09-10T09:59:06.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e2ea00640 1 -- 172.21.15.110:0/2069365105 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e28019070 con 0x7f2e3806f330 2024-09-10T09:59:06.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e2ea00640 1 -- 172.21.15.110:0/2069365105 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800ba60 con 0x7f2e3806f330 2024-09-10T09:59:06.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e380b39a0 con 0x7f2e3806f330 2024-09-10T09:59:06.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e38155cb0 con 0x7f2e3806f330 2024-09-10T09:59:06.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e380d08c0 con 0x7f2e3806f330 2024-09-10T09:59:06.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e2ea00640 1 -- 172.21.15.110:0/2069365105 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e2800bc00 con 0x7f2e3806f330 2024-09-10T09:59:06.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e2ea00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e180678c0 0x7f2e18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e2ea00640 1 -- 172.21.15.110:0/2069365105 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2e28085fe0 con 0x7f2e3806f330 2024-09-10T09:59:06.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.428+0000 7f2e36000640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e180678c0 0x7f2e18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.432+0000 7f2e36000640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e180678c0 0x7f2e18069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2e380cedc0 tx=0x7f2e24008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:06.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.432+0000 7f2e2ea00640 1 -- 172.21.15.110:0/2069365105 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2e28056500 con 0x7f2e3806f330 2024-09-10T09:59:06.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:06.624+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2e380b1ac0 con 0x7f2e3806f330 2024-09-10T09:59:07.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.248+0000 7f2e2ea00640 1 -- 172.21.15.110:0/2069365105 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f2e2805a800 con 0x7f2e3806f330 2024-09-10T09:59:07.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e180678c0 msgr2=0x7f2e18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e180678c0 0x7f2e18069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2e380cedc0 tx=0x7f2e24008040 comp rx=0 tx=0).stop 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 msgr2=0x7f2e3815bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e3815bdf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800d800 tx=0x7f2e2800dcc0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 shutdown_connections 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e180678c0 0x7f2e18069d80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e380731a0 0x7f2e3815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e38070870 0x7f2e3815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 --2- 172.21.15.110:0/2069365105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3806f330 0x7f2e3815bdf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 >> 172.21.15.110:0/2069365105 conn(0x7f2e380cedf0 msgr2=0x7f2e3815d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 shutdown_connections 2024-09-10T09:59:07.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.252+0000 7f2e37e00640 1 -- 172.21.15.110:0/2069365105 wait complete. 2024-09-10T09:59:07.274 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T09:59:07.274 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:07.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee1400640 1 Processor -- start 2024-09-10T09:59:07.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee1400640 1 -- start start 2024-09-10T09:59:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee1400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc155680 0x7f5edc0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5edc0cf2e0 0x7f5edc0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee1400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc0cd7e0 con 0x7f5edc0cd260 2024-09-10T09:59:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee1400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5edc0cdac0 con 0x7f5edc0cf2e0 2024-09-10T09:59:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee1400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc072770 con 0x7f5edc155680 2024-09-10T09:59:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee0a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5edc0cf2e0 0x7f5edc0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee0a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5edc0cf2e0 0x7f5edc0cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36586/0 (socket says 172.21.15.110:36586) 2024-09-10T09:59:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee0a00640 1 -- 172.21.15.110:0/1010650451 learned_addr learned my addr 172.21.15.110:0/1010650451 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5edbe00640 1 --2- 172.21.15.110:0/1010650451 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc155680 0x7f5edc0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee0a00640 1 -- 172.21.15.110:0/1010650451 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc155680 msgr2=0x7f5edc0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee0a00640 1 --2- 172.21.15.110:0/1010650451 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc155680 0x7f5edc0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee0a00640 1 -- 172.21.15.110:0/1010650451 >> v1:172.21.15.145:6789/0 conn(0x7f5edc0cd260 legacy=0x7f5edc072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.411+0000 7f5ee0a00640 1 -- 172.21.15.110:0/1010650451 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc072a50 con 0x7f5edc0cf2e0 2024-09-10T09:59:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee0a00640 1 --2- 172.21.15.110:0/1010650451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc0cf2e0 0x7f5edc0cf770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009870 tx=0x7f5ed401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4381396123b884e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5edaa00640 1 -- 172.21.15.110:0/1010650451 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40ab5f0 con 0x7f5edc0cf2e0 2024-09-10T09:59:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5edaa00640 1 -- 172.21.15.110:0/1010650451 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed40b0070 con 0x7f5edc0cf2e0 2024-09-10T09:59:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5edaa00640 1 -- 172.21.15.110:0/1010650451 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40ab9b0 con 0x7f5edc0cf2e0 2024-09-10T09:59:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee1400640 1 -- 172.21.15.110:0/1010650451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc0cf2e0 msgr2=0x7f5edc0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee1400640 1 --2- 172.21.15.110:0/1010650451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc0cf2e0 0x7f5edc0cf770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009870 tx=0x7f5ed401c520 comp rx=0 tx=0).stop 2024-09-10T09:59:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee1400640 1 -- 172.21.15.110:0/1010650451 shutdown_connections 2024-09-10T09:59:07.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee1400640 1 --2- 172.21.15.110:0/1010650451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc0cf2e0 0x7f5edc0cf770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee1400640 1 --2- 172.21.15.110:0/1010650451 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc155680 0x7f5edc0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee1400640 1 -- 172.21.15.110:0/1010650451 >> 172.21.15.110:0/1010650451 conn(0x7f5edc06d0f0 msgr2=0x7f5edc06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:07.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee1400640 1 -- 172.21.15.110:0/1010650451 shutdown_connections 2024-09-10T09:59:07.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.415+0000 7f5ee1400640 1 -- 172.21.15.110:0/1010650451 wait complete. 2024-09-10T09:59:07.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 Processor -- start 2024-09-10T09:59:07.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 -- start start 2024-09-10T09:59:07.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc0cd260 0x7f5edc168e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc0cf2e0 0x7f5edc154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc155680 0x7f5edc14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc073800 con 0x7f5edc155680 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc073680 con 0x7f5edc0cf2e0 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc0b56b0 con 0x7f5edc0cd260 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc155680 0x7f5edc14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc155680 0x7f5edc14fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36592/0 (socket says 172.21.15.110:36592) 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 -- 172.21.15.110:0/665487092 learned_addr learned my addr 172.21.15.110:0/665487092 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 -- 172.21.15.110:0/665487092 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc0cd260 msgr2=0x7f5edc168e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc0cd260 0x7f5edc168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 -- 172.21.15.110:0/665487092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc0cf2e0 msgr2=0x7f5edc154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc0cf2e0 0x7f5edc154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 -- 172.21.15.110:0/665487092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc15abb0 con 0x7f5edc155680 2024-09-10T09:59:07.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee0a00640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc155680 0x7f5edc14fea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ed401abd0 tx=0x7f5ed401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:07.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ed3e00640 1 -- 172.21.15.110:0/665487092 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40be3f0 con 0x7f5edc155680 2024-09-10T09:59:07.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ed3e00640 1 -- 172.21.15.110:0/665487092 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed40be590 con 0x7f5edc155680 2024-09-10T09:59:07.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc168050 con 0x7f5edc155680 2024-09-10T09:59:07.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ed3e00640 1 -- 172.21.15.110:0/665487092 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40be940 con 0x7f5edc155680 2024-09-10T09:59:07.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc168610 con 0x7f5edc155680 2024-09-10T09:59:07.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.419+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc0cf770 con 0x7f5edc155680 2024-09-10T09:59:07.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.423+0000 7f5ed3e00640 1 -- 172.21.15.110:0/665487092 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ed40b43f0 con 0x7f5edc155680 2024-09-10T09:59:07.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.423+0000 7f5ed3e00640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5ebc0678c0 0x7f5ebc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.423+0000 7f5ed3e00640 1 -- 172.21.15.110:0/665487092 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5ed411fe90 con 0x7f5edc155680 2024-09-10T09:59:07.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.423+0000 7f5edbe00640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5ebc0678c0 0x7f5ebc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.423+0000 7f5edbe00640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5ebc0678c0 0x7f5ebc069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0070b0 tx=0x7f5ecc007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:07.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.427+0000 7f5ed3e00640 1 -- 172.21.15.110:0/665487092 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5ed40c4060 con 0x7f5edc155680 2024-09-10T09:59:07.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.615+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5edc15b540 con 0x7f5edc155680 2024-09-10T09:59:07.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.615+0000 7f5ed3e00640 1 -- 172.21.15.110:0/665487092 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7f5ed4016110 con 0x7f5edc155680 2024-09-10T09:59:07.620 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:07.620 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":16,"btime":"2024-09-10T09:59:07:250101+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4556,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1141773053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1141773053},{"type":"v1","addr":"172.21.15.145:6834","nonce":1141773053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/656217845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":656217845},{"type":"v1","addr":"172.21.15.145:6835","nonce":656217845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2244539155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2244539155},{"type":"v1","addr":"172.21.15.110:6835","nonce":2244539155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:03.573094+0000","modified":"2024-09-10T09:59:06.627577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1972132715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1972132715},{"type":"v1","addr":"172.21.15.110:6837","nonce":1972132715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-09-10T09:59:07.620 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 16 2024-09-10T09:59:07.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5ebc0678c0 msgr2=0x7f5ebc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5ebc0678c0 0x7f5ebc069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0070b0 tx=0x7f5ecc007040 comp rx=0 tx=0).stop 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc155680 msgr2=0x7f5edc14fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc155680 0x7f5edc14fea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ed401abd0 tx=0x7f5ed401ad60 comp rx=0 tx=0).stop 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 shutdown_connections 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5ebc0678c0 0x7f5ebc069d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5edc155680 0x7f5edc14fea0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc0cf2e0 0x7f5edc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 --2- 172.21.15.110:0/665487092 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5edc0cd260 0x7f5edc168e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 >> 172.21.15.110:0/665487092 conn(0x7f5edc06d0f0 msgr2=0x7f5edc073980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 shutdown_connections 2024-09-10T09:59:07.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.619+0000 7f5ee1400640 1 -- 172.21.15.110:0/665487092 wait complete. 2024-09-10T09:59:07.642 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:07.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b9600640 1 Processor -- start 2024-09-10T09:59:07.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b9600640 1 -- start start 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b9600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f96b406eb30 0x7f96b40d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b9600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f96b4073290 0x7f96b40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b9600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b40706a0 con 0x7f96b4163000 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b9600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f96b415ee90 con 0x7f96b406eb30 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b9600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f96b4073c20 con 0x7f96b4073290 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b2a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f96b406eb30 0x7f96b40d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b2a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f96b406eb30 0x7f96b40d0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36594/0 (socket says 172.21.15.110:36594) 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b2a00640 1 -- 172.21.15.110:0/2681310000 learned_addr learned my addr 172.21.15.110:0/2681310000 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:07.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b3400640 1 --2- 172.21.15.110:0/2681310000 >> v2:172.21.15.145:3300/0 conn(0x7f96b4073290 0x7f96b40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b2a00640 1 -- 172.21.15.110:0/2681310000 >> v2:172.21.15.145:3300/0 conn(0x7f96b4073290 msgr2=0x7f96b40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:07.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b2a00640 1 --2- 172.21.15.110:0/2681310000 >> v2:172.21.15.145:3300/0 conn(0x7f96b4073290 0x7f96b40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b2a00640 1 -- 172.21.15.110:0/2681310000 >> v1:172.21.15.110:6789/0 conn(0x7f96b4163000 legacy=0x7f96b4071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.775+0000 7f96b2a00640 1 -- 172.21.15.110:0/2681310000 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b40d1370 con 0x7f96b406eb30 2024-09-10T09:59:07.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b2a00640 1 --2- 172.21.15.110:0/2681310000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b406eb30 0x7f96b40d0cb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f96b40b8010 tx=0x7f96a401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2cd93b8e22044a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:07.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b1600640 1 -- 172.21.15.110:0/2681310000 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a40a5620 con 0x7f96b406eb30 2024-09-10T09:59:07.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b1600640 1 -- 172.21.15.110:0/2681310000 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a40a57c0 con 0x7f96b406eb30 2024-09-10T09:59:07.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b1600640 1 -- 172.21.15.110:0/2681310000 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a40a5b90 con 0x7f96b406eb30 2024-09-10T09:59:07.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b9600640 1 -- 172.21.15.110:0/2681310000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b406eb30 msgr2=0x7f96b40d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b9600640 1 --2- 172.21.15.110:0/2681310000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b406eb30 0x7f96b40d0cb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f96b40b8010 tx=0x7f96a401c960 comp rx=0 tx=0).stop 2024-09-10T09:59:07.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b9600640 1 -- 172.21.15.110:0/2681310000 shutdown_connections 2024-09-10T09:59:07.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b9600640 1 --2- 172.21.15.110:0/2681310000 >> v2:172.21.15.145:3300/0 conn(0x7f96b4073290 0x7f96b40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b9600640 1 --2- 172.21.15.110:0/2681310000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b406eb30 0x7f96b40d0cb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b9600640 1 -- 172.21.15.110:0/2681310000 >> 172.21.15.110:0/2681310000 conn(0x7f96b40d25c0 msgr2=0x7f96b40d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:07.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b9600640 1 -- 172.21.15.110:0/2681310000 shutdown_connections 2024-09-10T09:59:07.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.779+0000 7f96b9600640 1 -- 172.21.15.110:0/2681310000 wait complete. 2024-09-10T09:59:07.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 Processor -- start 2024-09-10T09:59:07.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 -- start start 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96b406eb30 0x7f96b414dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b4073290 0x7f96b414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96b4163000 0x7f96b415f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f96b406dd60 con 0x7f96b4073290 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41633e0 con 0x7f96b406eb30 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f96b406dee0 con 0x7f96b4163000 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b4073290 0x7f96b414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b4073290 0x7f96b414cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36596/0 (socket says 172.21.15.110:36596) 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 -- 172.21.15.110:0/4125600873 learned_addr learned my addr 172.21.15.110:0/4125600873 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:07.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2a00640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96b406eb30 0x7f96b414dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b3400640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96b4163000 0x7f96b415f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 -- 172.21.15.110:0/4125600873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96b4163000 msgr2=0x7f96b415f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96b4163000 0x7f96b415f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 -- 172.21.15.110:0/4125600873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96b406eb30 msgr2=0x7f96b414dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96b406eb30 0x7f96b414dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 -- 172.21.15.110:0/4125600873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b414e250 con 0x7f96b4073290 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b2000640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b4073290 0x7f96b414cf00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96a000c3a0 tx=0x7f96a000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96aaa00640 1 -- 172.21.15.110:0/4125600873 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a00052d0 con 0x7f96b4073290 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96aaa00640 1 -- 172.21.15.110:0/4125600873 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a0005470 con 0x7f96b4073290 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96aaa00640 1 -- 172.21.15.110:0/4125600873 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0007260 con 0x7f96b4073290 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b40b2bd0 con 0x7f96b4073290 2024-09-10T09:59:07.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b40b3190 con 0x7f96b4073290 2024-09-10T09:59:07.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.783+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b40b3390 con 0x7f96b4073290 2024-09-10T09:59:07.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.787+0000 7f96aaa00640 1 -- 172.21.15.110:0/4125600873 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f96a0007400 con 0x7f96b4073290 2024-09-10T09:59:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.787+0000 7f96aaa00640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f96940678c0 0x7f9694069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.787+0000 7f96aaa00640 1 -- 172.21.15.110:0/4125600873 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f96a0085070 con 0x7f96b4073290 2024-09-10T09:59:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.787+0000 7f96b2a00640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f96940678c0 0x7f9694069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.787+0000 7f96b2a00640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f96940678c0 0x7f9694069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96a40ae000 tx=0x7f96a40a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:07.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.791+0000 7f96aaa00640 1 -- 172.21.15.110:0/4125600873 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f96b40b3390 con 0x7f96b4073290 2024-09-10T09:59:07.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.947+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96b40b68d0 con 0x7f96b4073290 2024-09-10T09:59:07.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96aaa00640 1 -- 172.21.15.110:0/4125600873 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16765 (secure 0 0 0) 0x7f96a0055550 con 0x7f96b4073290 2024-09-10T09:59:07.953 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:07.953 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":34,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:05.220702+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:01.639334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:02.583881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:07.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f96940678c0 msgr2=0x7f9694069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f96940678c0 0x7f9694069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96a40ae000 tx=0x7f96a40a7070 comp rx=0 tx=0).stop 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b4073290 msgr2=0x7f96b414cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b4073290 0x7f96b414cf00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96a000c3a0 tx=0x7f96a000c860 comp rx=0 tx=0).stop 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 shutdown_connections 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f96940678c0 0x7f9694069d80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f96b4163000 0x7f96b415f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96b4073290 0x7f96b414cf00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 --2- 172.21.15.110:0/4125600873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96b406eb30 0x7f96b414dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 >> 172.21.15.110:0/4125600873 conn(0x7f96b40d25c0 msgr2=0x7f96b4071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 shutdown_connections 2024-09-10T09:59:07.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:07.951+0000 7f96b9600640 1 -- 172.21.15.110:0/4125600873 wait complete. 2024-09-10T09:59:07.975 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T09:59:07.975 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:59:08.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc03ea00640 1 Processor -- start 2024-09-10T09:59:08.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc03ea00640 1 -- start start 2024-09-10T09:59:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc03ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc0380cfe30 0x7fc038071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc03ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc03806e7a0 0x7fc0380cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc03ea00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03806d2d0 con 0x7fc0380cd510 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc03ea00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc03815d9a0 con 0x7fc0380cfe30 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc03ea00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc03806d5b0 con 0x7fc03806e7a0 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc037e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc0380cfe30 0x7fc038071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc037e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc0380cfe30 0x7fc038071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36612/0 (socket says 172.21.15.110:36612) 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc037e00640 1 -- 172.21.15.110:0/2190457438 learned_addr learned my addr 172.21.15.110:0/2190457438 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc037400640 1 --1- 172.21.15.110:0/2190457438 >> v1:172.21.15.110:6789/0 conn(0x7fc0380cd510 0x7fc03806e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43870/0 (socket says 172.21.15.110:43870) 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc03ca00640 1 --2- 172.21.15.110:0/2190457438 >> v2:172.21.15.145:3300/0 conn(0x7fc03806e7a0 0x7fc0380cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc037e00640 1 -- 172.21.15.110:0/2190457438 >> v2:172.21.15.145:3300/0 conn(0x7fc03806e7a0 msgr2=0x7fc0380cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc037e00640 1 --2- 172.21.15.110:0/2190457438 >> v2:172.21.15.145:3300/0 conn(0x7fc03806e7a0 0x7fc0380cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc037e00640 1 -- 172.21.15.110:0/2190457438 >> v1:172.21.15.110:6789/0 conn(0x7fc0380cd510 legacy=0x7fc03806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.107+0000 7fc037e00640 1 -- 172.21.15.110:0/2190457438 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03806d890 con 0x7fc0380cfe30 2024-09-10T09:59:08.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc037e00640 1 --2- 172.21.15.110:0/2190457438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0380cfe30 0x7fc038071a40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc03806cb00 tx=0x7fc02801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5f1fbec0befd80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc036a00640 1 -- 172.21.15.110:0/2190457438 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0280a5730 con 0x7fc0380cfe30 2024-09-10T09:59:08.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc036a00640 1 -- 172.21.15.110:0/2190457438 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0280a58d0 con 0x7fc0380cfe30 2024-09-10T09:59:08.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc036a00640 1 -- 172.21.15.110:0/2190457438 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0280a5ca0 con 0x7fc0380cfe30 2024-09-10T09:59:08.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc03ea00640 1 -- 172.21.15.110:0/2190457438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0380cfe30 msgr2=0x7fc038071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc03ea00640 1 --2- 172.21.15.110:0/2190457438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0380cfe30 0x7fc038071a40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc03806cb00 tx=0x7fc02801c960 comp rx=0 tx=0).stop 2024-09-10T09:59:08.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc03ea00640 1 -- 172.21.15.110:0/2190457438 shutdown_connections 2024-09-10T09:59:08.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc03ea00640 1 --2- 172.21.15.110:0/2190457438 >> v2:172.21.15.145:3300/0 conn(0x7fc03806e7a0 0x7fc0380cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc03ea00640 1 --2- 172.21.15.110:0/2190457438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0380cfe30 0x7fc038071a40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc03ea00640 1 -- 172.21.15.110:0/2190457438 >> 172.21.15.110:0/2190457438 conn(0x7fc03806bc50 msgr2=0x7fc03806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:08.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc03ea00640 1 -- 172.21.15.110:0/2190457438 shutdown_connections 2024-09-10T09:59:08.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.111+0000 7fc03ea00640 1 -- 172.21.15.110:0/2190457438 wait complete. 2024-09-10T09:59:08.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 Processor -- start 2024-09-10T09:59:08.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 -- start start 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc03806e7a0 0x7fc0381488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0380cd510 0x7fc0380b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0380cfe30 0x7fc0380b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc038072190 con 0x7fc03806e7a0 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc03815d6c0 con 0x7fc0380cfe30 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc038169180 con 0x7fc0380cd510 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc03806e7a0 0x7fc0381488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc03806e7a0 0x7fc0381488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36628/0 (socket says 172.21.15.110:36628) 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 -- 172.21.15.110:0/3518531271 learned_addr learned my addr 172.21.15.110:0/3518531271 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ca00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0380cfe30 0x7fc0380b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 -- 172.21.15.110:0/3518531271 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0380cd510 msgr2=0x7fc0380b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0380cd510 0x7fc0380b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 -- 172.21.15.110:0/3518531271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0380cfe30 msgr2=0x7fc0380b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0380cfe30 0x7fc0380b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 -- 172.21.15.110:0/3518531271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380b21d0 con 0x7fc03806e7a0 2024-09-10T09:59:08.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ca00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0380cfe30 0x7fc0380b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:08.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc037e00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc03806e7a0 0x7fc0381488f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc0280af040 tx=0x7fc028019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc02fe00640 1 -- 172.21.15.110:0/3518531271 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028009510 con 0x7fc03806e7a0 2024-09-10T09:59:08.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc02fe00640 1 -- 172.21.15.110:0/3518531271 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0280be070 con 0x7fc03806e7a0 2024-09-10T09:59:08.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc038154d40 con 0x7fc03806e7a0 2024-09-10T09:59:08.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc02fe00640 1 -- 172.21.15.110:0/3518531271 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0280b8b00 con 0x7fc03806e7a0 2024-09-10T09:59:08.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03815a800 con 0x7fc03806e7a0 2024-09-10T09:59:08.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.115+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc038149390 con 0x7fc03806e7a0 2024-09-10T09:59:08.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.119+0000 7fc02fe00640 1 -- 172.21.15.110:0/3518531271 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc0280b8ca0 con 0x7fc03806e7a0 2024-09-10T09:59:08.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.119+0000 7fc02fe00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0180678c0 0x7fc018069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.119+0000 7fc02fe00640 1 -- 172.21.15.110:0/3518531271 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc028120fe0 con 0x7fc03806e7a0 2024-09-10T09:59:08.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.119+0000 7fc037400640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0180678c0 0x7fc018069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.119+0000 7fc037400640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0180678c0 0x7fc018069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc024000f70 tx=0x7fc024007a10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.123+0000 7fc02fe00640 1 -- 172.21.15.110:0/3518531271 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc0280165e0 con 0x7fc03806e7a0 2024-09-10T09:59:08.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.279+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0380b39e0 con 0x7fc0180678c0 2024-09-10T09:59:08.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.287+0000 7fc02fe00640 1 -- 172.21.15.110:0/3518531271 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241840 (secure 0 0 0) 0x7fc0380b39e0 con 0x7fc0180678c0 2024-09-10T09:59:08.292 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:08.293 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:59:08.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0180678c0 msgr2=0x7fc018069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0180678c0 0x7fc018069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc024000f70 tx=0x7fc024007a10 comp rx=0 tx=0).stop 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc03806e7a0 msgr2=0x7fc0381488f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc03806e7a0 0x7fc0381488f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc0280af040 tx=0x7fc028019700 comp rx=0 tx=0).stop 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 shutdown_connections 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0180678c0 0x7fc018069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0380cfe30 0x7fc0380b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0380cd510 0x7fc0380b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 --2- 172.21.15.110:0/3518531271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc03806e7a0 0x7fc0381488f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 >> 172.21.15.110:0/3518531271 conn(0x7fc03806bc50 msgr2=0x7fc038152b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 shutdown_connections 2024-09-10T09:59:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.291+0000 7fc03ea00640 1 -- 172.21.15.110:0/3518531271 wait complete. 2024-09-10T09:59:08.312 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-10T09:59:07.364370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236720,"kb_used_data":3808,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738000,"statfs":{"total":773094113280,"available":772851712000,"internally_reserved":0,"allocated":3899392,"data_stored":2171067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.178349"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282874+0000","last_change":"2024-09-10T09:59:04.248176+0000","last_active":"2024-09-10T09:59:05.282874+0000","last_peered":"2024-09-10T09:59:05.282874+0000","last_clean":"2024-09-10T09:59:05.282874+0000","last_became_active":"2024-09-10T09:59:04.247900+0000","last_became_peered":"2024-09-10T09:59:04.247900+0000","last_unstale":"2024-09-10T09:59:05.282874+0000","last_undegraded":"2024-09-10T09:59:05.282874+0000","last_fullsized":"2024-09-10T09:59:05.282874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:33.156203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281677+0000","last_change":"2024-09-10T09:59:04.244110+0000","last_active":"2024-09-10T09:59:05.281677+0000","last_peered":"2024-09-10T09:59:05.281677+0000","last_clean":"2024-09-10T09:59:05.281677+0000","last_became_active":"2024-09-10T09:59:04.243830+0000","last_became_peered":"2024-09-10T09:59:04.243830+0000","last_unstale":"2024-09-10T09:59:05.281677+0000","last_undegraded":"2024-09-10T09:59:05.281677+0000","last_fullsized":"2024-09-10T09:59:05.281677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:01.116838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281767+0000","last_change":"2024-09-10T09:59:04.247517+0000","last_active":"2024-09-10T09:59:05.281767+0000","last_peered":"2024-09-10T09:59:05.281767+0000","last_clean":"2024-09-10T09:59:05.281767+0000","last_became_active":"2024-09-10T09:59:04.247232+0000","last_became_peered":"2024-09-10T09:59:04.247232+0000","last_unstale":"2024-09-10T09:59:05.281767+0000","last_undegraded":"2024-09-10T09:59:05.281767+0000","last_fullsized":"2024-09-10T09:59:05.281767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:31.397569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283009+0000","last_change":"2024-09-10T09:59:04.257031+0000","last_active":"2024-09-10T09:59:05.283009+0000","last_peered":"2024-09-10T09:59:05.283009+0000","last_clean":"2024-09-10T09:59:05.283009+0000","last_became_active":"2024-09-10T09:59:04.256552+0000","last_became_peered":"2024-09-10T09:59:04.256552+0000","last_unstale":"2024-09-10T09:59:05.283009+0000","last_undegraded":"2024-09-10T09:59:05.283009+0000","last_fullsized":"2024-09-10T09:59:05.283009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:02.774151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282345+0000","last_change":"2024-09-10T09:59:04.256930+0000","last_active":"2024-09-10T09:59:05.282345+0000","last_peered":"2024-09-10T09:59:05.282345+0000","last_clean":"2024-09-10T09:59:05.282345+0000","last_became_active":"2024-09-10T09:59:04.256147+0000","last_became_peered":"2024-09-10T09:59:04.256147+0000","last_unstale":"2024-09-10T09:59:05.282345+0000","last_undegraded":"2024-09-10T09:59:05.282345+0000","last_fullsized":"2024-09-10T09:59:05.282345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:17.464661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283126+0000","last_change":"2024-09-10T09:59:04.235085+0000","last_active":"2024-09-10T09:59:05.283126+0000","last_peered":"2024-09-10T09:59:05.283126+0000","last_clean":"2024-09-10T09:59:05.283126+0000","last_became_active":"2024-09-10T09:59:04.232953+0000","last_became_peered":"2024-09-10T09:59:04.232953+0000","last_unstale":"2024-09-10T09:59:05.283126+0000","last_undegraded":"2024-09-10T09:59:05.283126+0000","last_fullsized":"2024-09-10T09:59:05.283126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:57.657194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190226+0000","last_change":"2024-09-10T09:59:02.190226+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190226+0000","last_undegraded":"2024-09-10T09:59:02.190226+0000","last_fullsized":"2024-09-10T09:59:02.190226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.191133+0000","last_change":"2024-09-10T09:59:02.191133+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.191133+0000","last_undegraded":"2024-09-10T09:59:02.191133+0000","last_fullsized":"2024-09-10T09:59:02.191133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282590+0000","last_change":"2024-09-10T09:59:03.218809+0000","last_active":"2024-09-10T09:59:05.282590+0000","last_peered":"2024-09-10T09:59:05.282590+0000","last_clean":"2024-09-10T09:59:05.282590+0000","last_became_active":"2024-09-10T09:59:03.218587+0000","last_became_peered":"2024-09-10T09:59:03.218587+0000","last_unstale":"2024-09-10T09:59:05.282590+0000","last_undegraded":"2024-09-10T09:59:05.282590+0000","last_fullsized":"2024-09-10T09:59:05.282590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:33.139925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192556+0000","last_change":"2024-09-10T09:59:02.192556+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192556+0000","last_undegraded":"2024-09-10T09:59:02.192556+0000","last_fullsized":"2024-09-10T09:59:02.192556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189712+0000","last_change":"2024-09-10T09:59:02.189712+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189712+0000","last_undegraded":"2024-09-10T09:59:02.189712+0000","last_fullsized":"2024-09-10T09:59:02.189712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188828+0000","last_change":"2024-09-10T09:59:02.188828+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188828+0000","last_undegraded":"2024-09-10T09:59:02.188828+0000","last_fullsized":"2024-09-10T09:59:02.188828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281756+0000","last_change":"2024-09-10T09:59:03.210986+0000","last_active":"2024-09-10T09:59:05.281756+0000","last_peered":"2024-09-10T09:59:05.281756+0000","last_clean":"2024-09-10T09:59:05.281756+0000","last_became_active":"2024-09-10T09:59:03.210736+0000","last_became_peered":"2024-09-10T09:59:03.210736+0000","last_unstale":"2024-09-10T09:59:05.281756+0000","last_undegraded":"2024-09-10T09:59:05.281756+0000","last_fullsized":"2024-09-10T09:59:05.281756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:01.201854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188854+0000","last_change":"2024-09-10T09:59:02.188854+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188854+0000","last_undegraded":"2024-09-10T09:59:02.188854+0000","last_fullsized":"2024-09-10T09:59:02.188854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283365+0000","last_change":"2024-09-10T09:59:03.208433+0000","last_active":"2024-09-10T09:59:05.283365+0000","last_peered":"2024-09-10T09:59:05.283365+0000","last_clean":"2024-09-10T09:59:05.283365+0000","last_became_active":"2024-09-10T09:59:03.207106+0000","last_became_peered":"2024-09-10T09:59:03.207106+0000","last_unstale":"2024-09-10T09:59:05.283365+0000","last_undegraded":"2024-09-10T09:59:05.283365+0000","last_fullsized":"2024-09-10T09:59:05.283365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:13.286437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283272+0000","last_change":"2024-09-10T09:59:04.257770+0000","last_active":"2024-09-10T09:59:05.283272+0000","last_peered":"2024-09-10T09:59:05.283272+0000","last_clean":"2024-09-10T09:59:05.283272+0000","last_became_active":"2024-09-10T09:59:04.257535+0000","last_became_peered":"2024-09-10T09:59:04.257535+0000","last_unstale":"2024-09-10T09:59:05.283272+0000","last_undegraded":"2024-09-10T09:59:05.283272+0000","last_fullsized":"2024-09-10T09:59:05.283272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:27.392572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190726+0000","last_change":"2024-09-10T09:59:02.190726+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190726+0000","last_undegraded":"2024-09-10T09:59:02.190726+0000","last_fullsized":"2024-09-10T09:59:02.190726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281596+0000","last_change":"2024-09-10T09:59:04.234326+0000","last_active":"2024-09-10T09:59:05.281596+0000","last_peered":"2024-09-10T09:59:05.281596+0000","last_clean":"2024-09-10T09:59:05.281596+0000","last_became_active":"2024-09-10T09:59:04.234025+0000","last_became_peered":"2024-09-10T09:59:04.234025+0000","last_unstale":"2024-09-10T09:59:05.281596+0000","last_undegraded":"2024-09-10T09:59:05.281596+0000","last_fullsized":"2024-09-10T09:59:05.281596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:09.398256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192908+0000","last_change":"2024-09-10T09:59:02.192908+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192908+0000","last_undegraded":"2024-09-10T09:59:02.192908+0000","last_fullsized":"2024-09-10T09:59:02.192908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281699+0000","last_change":"2024-09-10T09:59:03.219831+0000","last_active":"2024-09-10T09:59:05.281699+0000","last_peered":"2024-09-10T09:59:05.281699+0000","last_clean":"2024-09-10T09:59:05.281699+0000","last_became_active":"2024-09-10T09:59:03.219272+0000","last_became_peered":"2024-09-10T09:59:03.219272+0000","last_unstale":"2024-09-10T09:59:05.281699+0000","last_undegraded":"2024-09-10T09:59:05.281699+0000","last_fullsized":"2024-09-10T09:59:05.281699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:03.029812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189265+0000","last_change":"2024-09-10T09:59:02.189265+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189265+0000","last_undegraded":"2024-09-10T09:59:02.189265+0000","last_fullsized":"2024-09-10T09:59:02.189265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281688+0000","last_change":"2024-09-10T09:59:04.233774+0000","last_active":"2024-09-10T09:59:05.281688+0000","last_peered":"2024-09-10T09:59:05.281688+0000","last_clean":"2024-09-10T09:59:05.281688+0000","last_became_active":"2024-09-10T09:59:04.233387+0000","last_became_peered":"2024-09-10T09:59:04.233387+0000","last_unstale":"2024-09-10T09:59:05.281688+0000","last_undegraded":"2024-09-10T09:59:05.281688+0000","last_fullsized":"2024-09-10T09:59:05.281688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:49.890997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.195166+0000","last_change":"2024-09-10T09:59:02.195166+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.195166+0000","last_undegraded":"2024-09-10T09:59:02.195166+0000","last_fullsized":"2024-09-10T09:59:02.195166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281843+0000","last_change":"2024-09-10T09:59:03.208322+0000","last_active":"2024-09-10T09:59:05.281843+0000","last_peered":"2024-09-10T09:59:05.281843+0000","last_clean":"2024-09-10T09:59:05.281843+0000","last_became_active":"2024-09-10T09:59:03.207255+0000","last_became_peered":"2024-09-10T09:59:03.207255+0000","last_unstale":"2024-09-10T09:59:05.281843+0000","last_undegraded":"2024-09-10T09:59:05.281843+0000","last_fullsized":"2024-09-10T09:59:05.281843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:37.076444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281289+0000","last_change":"2024-09-10T09:59:03.219531+0000","last_active":"2024-09-10T09:59:05.281289+0000","last_peered":"2024-09-10T09:59:05.281289+0000","last_clean":"2024-09-10T09:59:05.281289+0000","last_became_active":"2024-09-10T09:59:03.219216+0000","last_became_peered":"2024-09-10T09:59:03.219216+0000","last_unstale":"2024-09-10T09:59:05.281289+0000","last_undegraded":"2024-09-10T09:59:05.281289+0000","last_fullsized":"2024-09-10T09:59:05.281289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:03.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.194881+0000","last_change":"2024-09-10T09:59:02.194881+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.194881+0000","last_undegraded":"2024-09-10T09:59:02.194881+0000","last_fullsized":"2024-09-10T09:59:02.194881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189160+0000","last_change":"2024-09-10T09:59:02.189160+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189160+0000","last_undegraded":"2024-09-10T09:59:02.189160+0000","last_fullsized":"2024-09-10T09:59:02.189160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189685+0000","last_change":"2024-09-10T09:59:02.189685+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189685+0000","last_undegraded":"2024-09-10T09:59:02.189685+0000","last_fullsized":"2024-09-10T09:59:02.189685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189092+0000","last_change":"2024-09-10T09:59:02.189092+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189092+0000","last_undegraded":"2024-09-10T09:59:02.189092+0000","last_fullsized":"2024-09-10T09:59:02.189092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190612+0000","last_change":"2024-09-10T09:59:02.190612+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190612+0000","last_undegraded":"2024-09-10T09:59:02.190612+0000","last_fullsized":"2024-09-10T09:59:02.190612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190111+0000","last_change":"2024-09-10T09:59:02.190111+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190111+0000","last_undegraded":"2024-09-10T09:59:02.190111+0000","last_fullsized":"2024-09-10T09:59:02.190111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190721+0000","last_change":"2024-09-10T09:59:02.190721+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190721+0000","last_undegraded":"2024-09-10T09:59:02.190721+0000","last_fullsized":"2024-09-10T09:59:02.190721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192879+0000","last_change":"2024-09-10T09:59:02.192879+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192879+0000","last_undegraded":"2024-09-10T09:59:02.192879+0000","last_fullsized":"2024-09-10T09:59:02.192879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193914+0000","last_change":"2024-09-10T09:59:02.193914+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193914+0000","last_undegraded":"2024-09-10T09:59:02.193914+0000","last_fullsized":"2024-09-10T09:59:02.193914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189993+0000","last_change":"2024-09-10T09:59:02.189993+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189993+0000","last_undegraded":"2024-09-10T09:59:02.189993+0000","last_fullsized":"2024-09-10T09:59:02.189993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193859+0000","last_change":"2024-09-10T09:59:02.193859+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193859+0000","last_undegraded":"2024-09-10T09:59:02.193859+0000","last_fullsized":"2024-09-10T09:59:02.193859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189987+0000","last_change":"2024-09-10T09:59:02.189987+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189987+0000","last_undegraded":"2024-09-10T09:59:02.189987+0000","last_fullsized":"2024-09-10T09:59:02.189987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193929+0000","last_change":"2024-09-10T09:59:02.193929+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193929+0000","last_undegraded":"2024-09-10T09:59:02.193929+0000","last_fullsized":"2024-09-10T09:59:02.193929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281192+0000","last_change":"2024-09-10T09:59:04.231281+0000","last_active":"2024-09-10T09:59:05.281192+0000","last_peered":"2024-09-10T09:59:05.281192+0000","last_clean":"2024-09-10T09:59:05.281192+0000","last_became_active":"2024-09-10T09:59:04.231070+0000","last_became_peered":"2024-09-10T09:59:04.231070+0000","last_unstale":"2024-09-10T09:59:05.281192+0000","last_undegraded":"2024-09-10T09:59:05.281192+0000","last_fullsized":"2024-09-10T09:59:05.281192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:34.062362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188810+0000","last_change":"2024-09-10T09:59:02.188810+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188810+0000","last_undegraded":"2024-09-10T09:59:02.188810+0000","last_fullsized":"2024-09-10T09:59:02.188810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281381+0000","last_change":"2024-09-10T09:59:03.208224+0000","last_active":"2024-09-10T09:59:05.281381+0000","last_peered":"2024-09-10T09:59:05.281381+0000","last_clean":"2024-09-10T09:59:05.281381+0000","last_became_active":"2024-09-10T09:59:03.207076+0000","last_became_peered":"2024-09-10T09:59:03.207076+0000","last_unstale":"2024-09-10T09:59:05.281381+0000","last_undegraded":"2024-09-10T09:59:05.281381+0000","last_fullsized":"2024-09-10T09:59:05.281381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:59.885497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189619+0000","last_change":"2024-09-10T09:59:02.189619+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189619+0000","last_undegraded":"2024-09-10T09:59:02.189619+0000","last_fullsized":"2024-09-10T09:59:02.189619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":184,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T09:59:02.196835+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:59:02.196835+0000","last_peered":"2024-09-10T09:59:02.196835+0000","last_clean":"2024-09-10T09:59:02.196835+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:59:02.196835+0000","last_undegraded":"2024-09-10T09:59:02.196835+0000","last_fullsized":"2024-09-10T09:59:02.196835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":921600,"data_stored":709356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":124805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":124805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":921600,"data_stored":709356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:59:08.316 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:59:08.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a9fc00640 1 Processor -- start 2024-09-10T09:59:08.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a9fc00640 1 -- start start 2024-09-10T09:59:08.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a9fc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8a9815c560 0x7f8a9806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a9fc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8a9806f010 0x7f8a980ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a9fc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a9806f430 con 0x7f8a98155970 2024-09-10T09:59:08.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a9fc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8a980b4570 con 0x7f8a9815c560 2024-09-10T09:59:08.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a9fc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8a98155d50 con 0x7f8a9806f010 2024-09-10T09:59:08.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a97e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8a9815c560 0x7f8a9806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a97e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8a9815c560 0x7f8a9806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46206/0 (socket says 172.21.15.110:46206) 2024-09-10T09:59:08.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a97e00640 1 -- 172.21.15.110:0/1847419942 learned_addr learned my addr 172.21.15.110:0/1847419942 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:08.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a9dc00640 1 --2- 172.21.15.110:0/1847419942 >> v2:172.21.15.145:3300/0 conn(0x7f8a9806f010 0x7f8a980ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a97e00640 1 -- 172.21.15.110:0/1847419942 >> v2:172.21.15.145:3300/0 conn(0x7f8a9806f010 msgr2=0x7f8a980ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:08.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a97e00640 1 --2- 172.21.15.110:0/1847419942 >> v2:172.21.15.145:3300/0 conn(0x7f8a9806f010 0x7f8a980ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a97e00640 1 -- 172.21.15.110:0/1847419942 >> v1:172.21.15.145:6789/0 conn(0x7f8a98155970 legacy=0x7f8a980cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.487+0000 7f8a97e00640 1 -- 172.21.15.110:0/1847419942 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a980b34d0 con 0x7f8a9815c560 2024-09-10T09:59:08.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a97e00640 1 --2- 172.21.15.110:0/1847419942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a9815c560 0x7f8a9806ead0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a980735e0 tx=0x7f8a8401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444a081d9f8eff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a97400640 1 -- 172.21.15.110:0/1847419942 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a840aa650 con 0x7f8a9815c560 2024-09-10T09:59:08.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a97400640 1 -- 172.21.15.110:0/1847419942 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a840af070 con 0x7f8a9815c560 2024-09-10T09:59:08.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a97400640 1 -- 172.21.15.110:0/1847419942 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a840aaa10 con 0x7f8a9815c560 2024-09-10T09:59:08.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a9fc00640 1 -- 172.21.15.110:0/1847419942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a9815c560 msgr2=0x7f8a9806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/1847419942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a9815c560 0x7f8a9806ead0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a980735e0 tx=0x7f8a8401c4f0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a9fc00640 1 -- 172.21.15.110:0/1847419942 shutdown_connections 2024-09-10T09:59:08.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/1847419942 >> v2:172.21.15.145:3300/0 conn(0x7f8a9806f010 0x7f8a980ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/1847419942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a9815c560 0x7f8a9806ead0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a9fc00640 1 -- 172.21.15.110:0/1847419942 >> 172.21.15.110:0/1847419942 conn(0x7f8a9806bc50 msgr2=0x7f8a9806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:08.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a9fc00640 1 -- 172.21.15.110:0/1847419942 shutdown_connections 2024-09-10T09:59:08.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.491+0000 7f8a9fc00640 1 -- 172.21.15.110:0/1847419942 wait complete. 2024-09-10T09:59:08.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 Processor -- start 2024-09-10T09:59:08.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 -- start start 2024-09-10T09:59:08.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8a9806f010 0x7f8a9815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a98155970 0x7f8a981483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a9815c560 0x7f8a981580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8a980cf170 con 0x7f8a98155970 2024-09-10T09:59:08.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a980b4050 con 0x7f8a9815c560 2024-09-10T09:59:08.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8a980cf2f0 con 0x7f8a9806f010 2024-09-10T09:59:08.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a98155970 0x7f8a981483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a98155970 0x7f8a981483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46208/0 (socket says 172.21.15.110:46208) 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 -- 172.21.15.110:0/2561477814 learned_addr learned my addr 172.21.15.110:0/2561477814 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9d200640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8a9806f010 0x7f8a9815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9dc00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a9815c560 0x7f8a981580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 -- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8a9806f010 msgr2=0x7f8a9815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8a9806f010 0x7f8a9815f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 -- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a9815c560 msgr2=0x7f8a981580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a9815c560 0x7f8a981580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 -- 172.21.15.110:0/2561477814 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a98148900 con 0x7f8a98155970 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9dc00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a9815c560 0x7f8a981580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a97e00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a98155970 0x7f8a981483c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a84019100 tx=0x7f8a8401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a94c00640 1 -- 172.21.15.110:0/2561477814 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a840bd5f0 con 0x7f8a98155970 2024-09-10T09:59:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a94c00640 1 -- 172.21.15.110:0/2561477814 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a840a6040 con 0x7f8a98155970 2024-09-10T09:59:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a94c00640 1 -- 172.21.15.110:0/2561477814 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a840bd9d0 con 0x7f8a98155970 2024-09-10T09:59:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9815d6b0 con 0x7f8a98155970 2024-09-10T09:59:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.495+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9815dc70 con 0x7f8a98155970 2024-09-10T09:59:08.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.499+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9815d440 con 0x7f8a98155970 2024-09-10T09:59:08.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.499+0000 7f8a94c00640 1 -- 172.21.15.110:0/2561477814 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8a840b33f0 con 0x7f8a98155970 2024-09-10T09:59:08.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.499+0000 7f8a94c00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8a780678c0 0x7f8a78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.499+0000 7f8a94c00640 1 -- 172.21.15.110:0/2561477814 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8a8411fe50 con 0x7f8a98155970 2024-09-10T09:59:08.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.499+0000 7f8a9d200640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8a780678c0 0x7f8a78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.499+0000 7f8a9d200640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8a780678c0 0x7f8a78069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a88001810 tx=0x7f8a88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.503+0000 7f8a94c00640 1 -- 172.21.15.110:0/2561477814 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8a98148900 con 0x7f8a98155970 2024-09-10T09:59:08.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.655+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a98069bb0 con 0x7f8a780678c0 2024-09-10T09:59:08.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.663+0000 7f8a94c00640 1 -- 172.21.15.110:0/2561477814 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241840 (secure 0 0 0) 0x7f8a98069bb0 con 0x7f8a780678c0 2024-09-10T09:59:08.668 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:08.671 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:59:08.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8a780678c0 msgr2=0x7f8a78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8a780678c0 0x7f8a78069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a88001810 tx=0x7f8a88009040 comp rx=0 tx=0).stop 2024-09-10T09:59:08.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a98155970 msgr2=0x7f8a981483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a98155970 0x7f8a981483c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a84019100 tx=0x7f8a8401ad30 comp rx=0 tx=0).stop 2024-09-10T09:59:08.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 shutdown_connections 2024-09-10T09:59:08.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8a780678c0 0x7f8a78069d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a9815c560 0x7f8a981580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a98155970 0x7f8a981483c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 --2- 172.21.15.110:0/2561477814 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8a9806f010 0x7f8a9815f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 >> 172.21.15.110:0/2561477814 conn(0x7f8a9806bc50 msgr2=0x7f8a980710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:08.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 shutdown_connections 2024-09-10T09:59:08.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.667+0000 7f8a9fc00640 1 -- 172.21.15.110:0/2561477814 wait complete. 2024-09-10T09:59:08.690 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-10T09:59:07.364370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236720,"kb_used_data":3808,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738000,"statfs":{"total":773094113280,"available":772851712000,"internally_reserved":0,"allocated":3899392,"data_stored":2171067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.178349"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282874+0000","last_change":"2024-09-10T09:59:04.248176+0000","last_active":"2024-09-10T09:59:05.282874+0000","last_peered":"2024-09-10T09:59:05.282874+0000","last_clean":"2024-09-10T09:59:05.282874+0000","last_became_active":"2024-09-10T09:59:04.247900+0000","last_became_peered":"2024-09-10T09:59:04.247900+0000","last_unstale":"2024-09-10T09:59:05.282874+0000","last_undegraded":"2024-09-10T09:59:05.282874+0000","last_fullsized":"2024-09-10T09:59:05.282874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:33.156203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281677+0000","last_change":"2024-09-10T09:59:04.244110+0000","last_active":"2024-09-10T09:59:05.281677+0000","last_peered":"2024-09-10T09:59:05.281677+0000","last_clean":"2024-09-10T09:59:05.281677+0000","last_became_active":"2024-09-10T09:59:04.243830+0000","last_became_peered":"2024-09-10T09:59:04.243830+0000","last_unstale":"2024-09-10T09:59:05.281677+0000","last_undegraded":"2024-09-10T09:59:05.281677+0000","last_fullsized":"2024-09-10T09:59:05.281677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:01.116838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281767+0000","last_change":"2024-09-10T09:59:04.247517+0000","last_active":"2024-09-10T09:59:05.281767+0000","last_peered":"2024-09-10T09:59:05.281767+0000","last_clean":"2024-09-10T09:59:05.281767+0000","last_became_active":"2024-09-10T09:59:04.247232+0000","last_became_peered":"2024-09-10T09:59:04.247232+0000","last_unstale":"2024-09-10T09:59:05.281767+0000","last_undegraded":"2024-09-10T09:59:05.281767+0000","last_fullsized":"2024-09-10T09:59:05.281767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:31.397569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283009+0000","last_change":"2024-09-10T09:59:04.257031+0000","last_active":"2024-09-10T09:59:05.283009+0000","last_peered":"2024-09-10T09:59:05.283009+0000","last_clean":"2024-09-10T09:59:05.283009+0000","last_became_active":"2024-09-10T09:59:04.256552+0000","last_became_peered":"2024-09-10T09:59:04.256552+0000","last_unstale":"2024-09-10T09:59:05.283009+0000","last_undegraded":"2024-09-10T09:59:05.283009+0000","last_fullsized":"2024-09-10T09:59:05.283009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:02.774151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282345+0000","last_change":"2024-09-10T09:59:04.256930+0000","last_active":"2024-09-10T09:59:05.282345+0000","last_peered":"2024-09-10T09:59:05.282345+0000","last_clean":"2024-09-10T09:59:05.282345+0000","last_became_active":"2024-09-10T09:59:04.256147+0000","last_became_peered":"2024-09-10T09:59:04.256147+0000","last_unstale":"2024-09-10T09:59:05.282345+0000","last_undegraded":"2024-09-10T09:59:05.282345+0000","last_fullsized":"2024-09-10T09:59:05.282345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:17.464661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283126+0000","last_change":"2024-09-10T09:59:04.235085+0000","last_active":"2024-09-10T09:59:05.283126+0000","last_peered":"2024-09-10T09:59:05.283126+0000","last_clean":"2024-09-10T09:59:05.283126+0000","last_became_active":"2024-09-10T09:59:04.232953+0000","last_became_peered":"2024-09-10T09:59:04.232953+0000","last_unstale":"2024-09-10T09:59:05.283126+0000","last_undegraded":"2024-09-10T09:59:05.283126+0000","last_fullsized":"2024-09-10T09:59:05.283126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:57.657194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190226+0000","last_change":"2024-09-10T09:59:02.190226+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190226+0000","last_undegraded":"2024-09-10T09:59:02.190226+0000","last_fullsized":"2024-09-10T09:59:02.190226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.191133+0000","last_change":"2024-09-10T09:59:02.191133+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.191133+0000","last_undegraded":"2024-09-10T09:59:02.191133+0000","last_fullsized":"2024-09-10T09:59:02.191133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282590+0000","last_change":"2024-09-10T09:59:03.218809+0000","last_active":"2024-09-10T09:59:05.282590+0000","last_peered":"2024-09-10T09:59:05.282590+0000","last_clean":"2024-09-10T09:59:05.282590+0000","last_became_active":"2024-09-10T09:59:03.218587+0000","last_became_peered":"2024-09-10T09:59:03.218587+0000","last_unstale":"2024-09-10T09:59:05.282590+0000","last_undegraded":"2024-09-10T09:59:05.282590+0000","last_fullsized":"2024-09-10T09:59:05.282590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:33.139925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192556+0000","last_change":"2024-09-10T09:59:02.192556+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192556+0000","last_undegraded":"2024-09-10T09:59:02.192556+0000","last_fullsized":"2024-09-10T09:59:02.192556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189712+0000","last_change":"2024-09-10T09:59:02.189712+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189712+0000","last_undegraded":"2024-09-10T09:59:02.189712+0000","last_fullsized":"2024-09-10T09:59:02.189712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188828+0000","last_change":"2024-09-10T09:59:02.188828+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188828+0000","last_undegraded":"2024-09-10T09:59:02.188828+0000","last_fullsized":"2024-09-10T09:59:02.188828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281756+0000","last_change":"2024-09-10T09:59:03.210986+0000","last_active":"2024-09-10T09:59:05.281756+0000","last_peered":"2024-09-10T09:59:05.281756+0000","last_clean":"2024-09-10T09:59:05.281756+0000","last_became_active":"2024-09-10T09:59:03.210736+0000","last_became_peered":"2024-09-10T09:59:03.210736+0000","last_unstale":"2024-09-10T09:59:05.281756+0000","last_undegraded":"2024-09-10T09:59:05.281756+0000","last_fullsized":"2024-09-10T09:59:05.281756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:01.201854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188854+0000","last_change":"2024-09-10T09:59:02.188854+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188854+0000","last_undegraded":"2024-09-10T09:59:02.188854+0000","last_fullsized":"2024-09-10T09:59:02.188854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283365+0000","last_change":"2024-09-10T09:59:03.208433+0000","last_active":"2024-09-10T09:59:05.283365+0000","last_peered":"2024-09-10T09:59:05.283365+0000","last_clean":"2024-09-10T09:59:05.283365+0000","last_became_active":"2024-09-10T09:59:03.207106+0000","last_became_peered":"2024-09-10T09:59:03.207106+0000","last_unstale":"2024-09-10T09:59:05.283365+0000","last_undegraded":"2024-09-10T09:59:05.283365+0000","last_fullsized":"2024-09-10T09:59:05.283365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:13.286437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283272+0000","last_change":"2024-09-10T09:59:04.257770+0000","last_active":"2024-09-10T09:59:05.283272+0000","last_peered":"2024-09-10T09:59:05.283272+0000","last_clean":"2024-09-10T09:59:05.283272+0000","last_became_active":"2024-09-10T09:59:04.257535+0000","last_became_peered":"2024-09-10T09:59:04.257535+0000","last_unstale":"2024-09-10T09:59:05.283272+0000","last_undegraded":"2024-09-10T09:59:05.283272+0000","last_fullsized":"2024-09-10T09:59:05.283272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:27.392572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190726+0000","last_change":"2024-09-10T09:59:02.190726+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190726+0000","last_undegraded":"2024-09-10T09:59:02.190726+0000","last_fullsized":"2024-09-10T09:59:02.190726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281596+0000","last_change":"2024-09-10T09:59:04.234326+0000","last_active":"2024-09-10T09:59:05.281596+0000","last_peered":"2024-09-10T09:59:05.281596+0000","last_clean":"2024-09-10T09:59:05.281596+0000","last_became_active":"2024-09-10T09:59:04.234025+0000","last_became_peered":"2024-09-10T09:59:04.234025+0000","last_unstale":"2024-09-10T09:59:05.281596+0000","last_undegraded":"2024-09-10T09:59:05.281596+0000","last_fullsized":"2024-09-10T09:59:05.281596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:09.398256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192908+0000","last_change":"2024-09-10T09:59:02.192908+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192908+0000","last_undegraded":"2024-09-10T09:59:02.192908+0000","last_fullsized":"2024-09-10T09:59:02.192908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281699+0000","last_change":"2024-09-10T09:59:03.219831+0000","last_active":"2024-09-10T09:59:05.281699+0000","last_peered":"2024-09-10T09:59:05.281699+0000","last_clean":"2024-09-10T09:59:05.281699+0000","last_became_active":"2024-09-10T09:59:03.219272+0000","last_became_peered":"2024-09-10T09:59:03.219272+0000","last_unstale":"2024-09-10T09:59:05.281699+0000","last_undegraded":"2024-09-10T09:59:05.281699+0000","last_fullsized":"2024-09-10T09:59:05.281699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:03.029812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189265+0000","last_change":"2024-09-10T09:59:02.189265+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189265+0000","last_undegraded":"2024-09-10T09:59:02.189265+0000","last_fullsized":"2024-09-10T09:59:02.189265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281688+0000","last_change":"2024-09-10T09:59:04.233774+0000","last_active":"2024-09-10T09:59:05.281688+0000","last_peered":"2024-09-10T09:59:05.281688+0000","last_clean":"2024-09-10T09:59:05.281688+0000","last_became_active":"2024-09-10T09:59:04.233387+0000","last_became_peered":"2024-09-10T09:59:04.233387+0000","last_unstale":"2024-09-10T09:59:05.281688+0000","last_undegraded":"2024-09-10T09:59:05.281688+0000","last_fullsized":"2024-09-10T09:59:05.281688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:49.890997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.195166+0000","last_change":"2024-09-10T09:59:02.195166+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.195166+0000","last_undegraded":"2024-09-10T09:59:02.195166+0000","last_fullsized":"2024-09-10T09:59:02.195166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281843+0000","last_change":"2024-09-10T09:59:03.208322+0000","last_active":"2024-09-10T09:59:05.281843+0000","last_peered":"2024-09-10T09:59:05.281843+0000","last_clean":"2024-09-10T09:59:05.281843+0000","last_became_active":"2024-09-10T09:59:03.207255+0000","last_became_peered":"2024-09-10T09:59:03.207255+0000","last_unstale":"2024-09-10T09:59:05.281843+0000","last_undegraded":"2024-09-10T09:59:05.281843+0000","last_fullsized":"2024-09-10T09:59:05.281843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:37.076444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281289+0000","last_change":"2024-09-10T09:59:03.219531+0000","last_active":"2024-09-10T09:59:05.281289+0000","last_peered":"2024-09-10T09:59:05.281289+0000","last_clean":"2024-09-10T09:59:05.281289+0000","last_became_active":"2024-09-10T09:59:03.219216+0000","last_became_peered":"2024-09-10T09:59:03.219216+0000","last_unstale":"2024-09-10T09:59:05.281289+0000","last_undegraded":"2024-09-10T09:59:05.281289+0000","last_fullsized":"2024-09-10T09:59:05.281289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:03.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.194881+0000","last_change":"2024-09-10T09:59:02.194881+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.194881+0000","last_undegraded":"2024-09-10T09:59:02.194881+0000","last_fullsized":"2024-09-10T09:59:02.194881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189160+0000","last_change":"2024-09-10T09:59:02.189160+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189160+0000","last_undegraded":"2024-09-10T09:59:02.189160+0000","last_fullsized":"2024-09-10T09:59:02.189160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189685+0000","last_change":"2024-09-10T09:59:02.189685+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189685+0000","last_undegraded":"2024-09-10T09:59:02.189685+0000","last_fullsized":"2024-09-10T09:59:02.189685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189092+0000","last_change":"2024-09-10T09:59:02.189092+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189092+0000","last_undegraded":"2024-09-10T09:59:02.189092+0000","last_fullsized":"2024-09-10T09:59:02.189092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190612+0000","last_change":"2024-09-10T09:59:02.190612+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190612+0000","last_undegraded":"2024-09-10T09:59:02.190612+0000","last_fullsized":"2024-09-10T09:59:02.190612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190111+0000","last_change":"2024-09-10T09:59:02.190111+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190111+0000","last_undegraded":"2024-09-10T09:59:02.190111+0000","last_fullsized":"2024-09-10T09:59:02.190111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190721+0000","last_change":"2024-09-10T09:59:02.190721+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190721+0000","last_undegraded":"2024-09-10T09:59:02.190721+0000","last_fullsized":"2024-09-10T09:59:02.190721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192879+0000","last_change":"2024-09-10T09:59:02.192879+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192879+0000","last_undegraded":"2024-09-10T09:59:02.192879+0000","last_fullsized":"2024-09-10T09:59:02.192879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193914+0000","last_change":"2024-09-10T09:59:02.193914+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193914+0000","last_undegraded":"2024-09-10T09:59:02.193914+0000","last_fullsized":"2024-09-10T09:59:02.193914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189993+0000","last_change":"2024-09-10T09:59:02.189993+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189993+0000","last_undegraded":"2024-09-10T09:59:02.189993+0000","last_fullsized":"2024-09-10T09:59:02.189993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193859+0000","last_change":"2024-09-10T09:59:02.193859+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193859+0000","last_undegraded":"2024-09-10T09:59:02.193859+0000","last_fullsized":"2024-09-10T09:59:02.193859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189987+0000","last_change":"2024-09-10T09:59:02.189987+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189987+0000","last_undegraded":"2024-09-10T09:59:02.189987+0000","last_fullsized":"2024-09-10T09:59:02.189987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193929+0000","last_change":"2024-09-10T09:59:02.193929+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193929+0000","last_undegraded":"2024-09-10T09:59:02.193929+0000","last_fullsized":"2024-09-10T09:59:02.193929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281192+0000","last_change":"2024-09-10T09:59:04.231281+0000","last_active":"2024-09-10T09:59:05.281192+0000","last_peered":"2024-09-10T09:59:05.281192+0000","last_clean":"2024-09-10T09:59:05.281192+0000","last_became_active":"2024-09-10T09:59:04.231070+0000","last_became_peered":"2024-09-10T09:59:04.231070+0000","last_unstale":"2024-09-10T09:59:05.281192+0000","last_undegraded":"2024-09-10T09:59:05.281192+0000","last_fullsized":"2024-09-10T09:59:05.281192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:34.062362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188810+0000","last_change":"2024-09-10T09:59:02.188810+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188810+0000","last_undegraded":"2024-09-10T09:59:02.188810+0000","last_fullsized":"2024-09-10T09:59:02.188810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281381+0000","last_change":"2024-09-10T09:59:03.208224+0000","last_active":"2024-09-10T09:59:05.281381+0000","last_peered":"2024-09-10T09:59:05.281381+0000","last_clean":"2024-09-10T09:59:05.281381+0000","last_became_active":"2024-09-10T09:59:03.207076+0000","last_became_peered":"2024-09-10T09:59:03.207076+0000","last_unstale":"2024-09-10T09:59:05.281381+0000","last_undegraded":"2024-09-10T09:59:05.281381+0000","last_fullsized":"2024-09-10T09:59:05.281381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:59.885497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189619+0000","last_change":"2024-09-10T09:59:02.189619+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189619+0000","last_undegraded":"2024-09-10T09:59:02.189619+0000","last_fullsized":"2024-09-10T09:59:02.189619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":184,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T09:59:02.196835+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:59:02.196835+0000","last_peered":"2024-09-10T09:59:02.196835+0000","last_clean":"2024-09-10T09:59:02.196835+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:59:02.196835+0000","last_undegraded":"2024-09-10T09:59:02.196835+0000","last_fullsized":"2024-09-10T09:59:02.196835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":921600,"data_stored":709356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":124805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":124805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":921600,"data_stored":709356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:59:08.694 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:59:08.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.863+0000 7f4bd6e00640 1 Processor -- start 2024-09-10T09:59:08.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.863+0000 7f4bd6e00640 1 -- start start 2024-09-10T09:59:08.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.863+0000 7f4bd6e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4bd015c570 0x7f4bd006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bd6e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd0072270 0x7f4bd006bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bd6e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd006ea70 con 0x7f4bd006c190 2024-09-10T09:59:08.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bd6e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd006c5e0 con 0x7f4bd015c570 2024-09-10T09:59:08.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bd6e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd015c950 con 0x7f4bd0072270 2024-09-10T09:59:08.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcf400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd0072270 0x7f4bd006bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4bd015c570 0x7f4bd006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4bd015c570 0x7f4bd006e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:51856/0 (socket says 172.21.15.110:51856) 2024-09-10T09:59:08.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcf400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd0072270 0x7f4bd006bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:37848/0 (socket says 172.21.15.110:37848) 2024-09-10T09:59:08.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcfe00640 1 -- 172.21.15.110:0/3447950542 learned_addr learned my addr 172.21.15.110:0/3447950542 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:08.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bd4e00640 1 --1- 172.21.15.110:0/3447950542 >> v1:172.21.15.145:6789/0 conn(0x7f4bd006c190 0x7f4bd006fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:37960/0 (socket says 172.21.15.110:37960) 2024-09-10T09:59:08.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcfe00640 1 -- 172.21.15.110:0/3447950542 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd0072270 msgr2=0x7f4bd006bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:08.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcfe00640 1 --2- 172.21.15.110:0/3447950542 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd0072270 0x7f4bd006bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcfe00640 1 -- 172.21.15.110:0/3447950542 >> v1:172.21.15.145:6789/0 conn(0x7f4bd006c190 legacy=0x7f4bd006fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.867+0000 7f4bcfe00640 1 -- 172.21.15.110:0/3447950542 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd006ed50 con 0x7f4bd015c570 2024-09-10T09:59:08.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bcfe00640 1 --2- 172.21.15.110:0/3447950542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd015c570 0x7f4bd006e240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009e50 tx=0x7f4bc001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78e3ac01730a918 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bcea00640 1 -- 172.21.15.110:0/3447950542 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00a5750 con 0x7f4bd015c570 2024-09-10T09:59:08.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bcea00640 1 -- 172.21.15.110:0/3447950542 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc00a58f0 con 0x7f4bd015c570 2024-09-10T09:59:08.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bcea00640 1 -- 172.21.15.110:0/3447950542 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00a5cc0 con 0x7f4bd015c570 2024-09-10T09:59:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- 172.21.15.110:0/3447950542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd015c570 msgr2=0x7f4bd006e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/3447950542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd015c570 0x7f4bd006e240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009e50 tx=0x7f4bc001c9a0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- 172.21.15.110:0/3447950542 shutdown_connections 2024-09-10T09:59:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/3447950542 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd0072270 0x7f4bd006bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/3447950542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd015c570 0x7f4bd006e240 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- 172.21.15.110:0/3447950542 >> 172.21.15.110:0/3447950542 conn(0x7f4bd00d1530 msgr2=0x7f4bd00d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- 172.21.15.110:0/3447950542 shutdown_connections 2024-09-10T09:59:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- 172.21.15.110:0/3447950542 wait complete. 2024-09-10T09:59:08.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 Processor -- start 2024-09-10T09:59:08.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- start start 2024-09-10T09:59:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd006c190 0x7f4bd014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd0072270 0x7f4bd0159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd015c570 0x7f4bd0153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00d0620 con 0x7f4bd015c570 2024-09-10T09:59:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd006e780 con 0x7f4bd0072270 2024-09-10T09:59:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd6e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd00d07a0 con 0x7f4bd006c190 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd4e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd015c570 0x7f4bd0153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd4e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd015c570 0x7f4bd0153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46224/0 (socket says 172.21.15.110:46224) 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.871+0000 7f4bd4e00640 1 -- 172.21.15.110:0/1383830367 learned_addr learned my addr 172.21.15.110:0/1383830367 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bcf400640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd0072270 0x7f4bd0159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bcfe00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd006c190 0x7f4bd014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd4e00640 1 -- 172.21.15.110:0/1383830367 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd006c190 msgr2=0x7f4bd014ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd4e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd006c190 0x7f4bd014ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd4e00640 1 -- 172.21.15.110:0/1383830367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd0072270 msgr2=0x7f4bd0159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd4e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd0072270 0x7f4bd0159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd4e00640 1 -- 172.21.15.110:0/1383830367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd014a1a0 con 0x7f4bd015c570 2024-09-10T09:59:08.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd4e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd015c570 0x7f4bd0153350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c3c0 tx=0x7f4bc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bc7e00640 1 -- 172.21.15.110:0/1383830367 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80074e0 con 0x7f4bd015c570 2024-09-10T09:59:08.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bc7e00640 1 -- 172.21.15.110:0/1383830367 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc8007680 con 0x7f4bd015c570 2024-09-10T09:59:08.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bc7e00640 1 -- 172.21.15.110:0/1383830367 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8004a10 con 0x7f4bd015c570 2024-09-10T09:59:08.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd01556f0 con 0x7f4bd015c570 2024-09-10T09:59:08.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd0155cb0 con 0x7f4bd015c570 2024-09-10T09:59:08.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd00697d0 con 0x7f4bd015c570 2024-09-10T09:59:08.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bc7e00640 1 -- 172.21.15.110:0/1383830367 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4bc801a020 con 0x7f4bd015c570 2024-09-10T09:59:08.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bc7e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4bb00678f0 0x7f4bb0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:08.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bc7e00640 1 -- 172.21.15.110:0/1383830367 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4bc8085270 con 0x7f4bd015c570 2024-09-10T09:59:08.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.875+0000 7f4bcfe00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4bb00678f0 0x7f4bb0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:08.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.879+0000 7f4bcfe00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4bb00678f0 0x7f4bb0069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00ae000 tx=0x7f4bc00a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:08.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:08.879+0000 7f4bc7e00640 1 -- 172.21.15.110:0/1383830367 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4bc8055750 con 0x7f4bd015c570 2024-09-10T09:59:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.035+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bd0156ce0 con 0x7f4bb00678f0 2024-09-10T09:59:09.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.039+0000 7f4bc7e00640 1 -- 172.21.15.110:0/1383830367 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241840 (secure 0 0 0) 0x7f4bd0156ce0 con 0x7f4bb00678f0 2024-09-10T09:59:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:59:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4bb00678f0 msgr2=0x7f4bb0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4bb00678f0 0x7f4bb0069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00ae000 tx=0x7f4bc00a7070 comp rx=0 tx=0).stop 2024-09-10T09:59:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd015c570 msgr2=0x7f4bd0153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd015c570 0x7f4bd0153350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c3c0 tx=0x7f4bc800c880 comp rx=0 tx=0).stop 2024-09-10T09:59:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 shutdown_connections 2024-09-10T09:59:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4bb00678f0 0x7f4bb0069db0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd015c570 0x7f4bd0153350 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bd0072270 0x7f4bd0159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 --2- 172.21.15.110:0/1383830367 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bd006c190 0x7f4bd014ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 >> 172.21.15.110:0/1383830367 conn(0x7f4bd00d1530 msgr2=0x7f4bd00b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:09.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 shutdown_connections 2024-09-10T09:59:09.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.043+0000 7f4bd6e00640 1 -- 172.21.15.110:0/1383830367 wait complete. 2024-09-10T09:59:09.067 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-10T09:59:07.364370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236720,"kb_used_data":3808,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738000,"statfs":{"total":773094113280,"available":772851712000,"internally_reserved":0,"allocated":3899392,"data_stored":2171067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.178349"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282874+0000","last_change":"2024-09-10T09:59:04.248176+0000","last_active":"2024-09-10T09:59:05.282874+0000","last_peered":"2024-09-10T09:59:05.282874+0000","last_clean":"2024-09-10T09:59:05.282874+0000","last_became_active":"2024-09-10T09:59:04.247900+0000","last_became_peered":"2024-09-10T09:59:04.247900+0000","last_unstale":"2024-09-10T09:59:05.282874+0000","last_undegraded":"2024-09-10T09:59:05.282874+0000","last_fullsized":"2024-09-10T09:59:05.282874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:33.156203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281677+0000","last_change":"2024-09-10T09:59:04.244110+0000","last_active":"2024-09-10T09:59:05.281677+0000","last_peered":"2024-09-10T09:59:05.281677+0000","last_clean":"2024-09-10T09:59:05.281677+0000","last_became_active":"2024-09-10T09:59:04.243830+0000","last_became_peered":"2024-09-10T09:59:04.243830+0000","last_unstale":"2024-09-10T09:59:05.281677+0000","last_undegraded":"2024-09-10T09:59:05.281677+0000","last_fullsized":"2024-09-10T09:59:05.281677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:01.116838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281767+0000","last_change":"2024-09-10T09:59:04.247517+0000","last_active":"2024-09-10T09:59:05.281767+0000","last_peered":"2024-09-10T09:59:05.281767+0000","last_clean":"2024-09-10T09:59:05.281767+0000","last_became_active":"2024-09-10T09:59:04.247232+0000","last_became_peered":"2024-09-10T09:59:04.247232+0000","last_unstale":"2024-09-10T09:59:05.281767+0000","last_undegraded":"2024-09-10T09:59:05.281767+0000","last_fullsized":"2024-09-10T09:59:05.281767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:31.397569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283009+0000","last_change":"2024-09-10T09:59:04.257031+0000","last_active":"2024-09-10T09:59:05.283009+0000","last_peered":"2024-09-10T09:59:05.283009+0000","last_clean":"2024-09-10T09:59:05.283009+0000","last_became_active":"2024-09-10T09:59:04.256552+0000","last_became_peered":"2024-09-10T09:59:04.256552+0000","last_unstale":"2024-09-10T09:59:05.283009+0000","last_undegraded":"2024-09-10T09:59:05.283009+0000","last_fullsized":"2024-09-10T09:59:05.283009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:02.774151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282345+0000","last_change":"2024-09-10T09:59:04.256930+0000","last_active":"2024-09-10T09:59:05.282345+0000","last_peered":"2024-09-10T09:59:05.282345+0000","last_clean":"2024-09-10T09:59:05.282345+0000","last_became_active":"2024-09-10T09:59:04.256147+0000","last_became_peered":"2024-09-10T09:59:04.256147+0000","last_unstale":"2024-09-10T09:59:05.282345+0000","last_undegraded":"2024-09-10T09:59:05.282345+0000","last_fullsized":"2024-09-10T09:59:05.282345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:17.464661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283126+0000","last_change":"2024-09-10T09:59:04.235085+0000","last_active":"2024-09-10T09:59:05.283126+0000","last_peered":"2024-09-10T09:59:05.283126+0000","last_clean":"2024-09-10T09:59:05.283126+0000","last_became_active":"2024-09-10T09:59:04.232953+0000","last_became_peered":"2024-09-10T09:59:04.232953+0000","last_unstale":"2024-09-10T09:59:05.283126+0000","last_undegraded":"2024-09-10T09:59:05.283126+0000","last_fullsized":"2024-09-10T09:59:05.283126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:57.657194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190226+0000","last_change":"2024-09-10T09:59:02.190226+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190226+0000","last_undegraded":"2024-09-10T09:59:02.190226+0000","last_fullsized":"2024-09-10T09:59:02.190226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.191133+0000","last_change":"2024-09-10T09:59:02.191133+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.191133+0000","last_undegraded":"2024-09-10T09:59:02.191133+0000","last_fullsized":"2024-09-10T09:59:02.191133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282590+0000","last_change":"2024-09-10T09:59:03.218809+0000","last_active":"2024-09-10T09:59:05.282590+0000","last_peered":"2024-09-10T09:59:05.282590+0000","last_clean":"2024-09-10T09:59:05.282590+0000","last_became_active":"2024-09-10T09:59:03.218587+0000","last_became_peered":"2024-09-10T09:59:03.218587+0000","last_unstale":"2024-09-10T09:59:05.282590+0000","last_undegraded":"2024-09-10T09:59:05.282590+0000","last_fullsized":"2024-09-10T09:59:05.282590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:33.139925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192556+0000","last_change":"2024-09-10T09:59:02.192556+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192556+0000","last_undegraded":"2024-09-10T09:59:02.192556+0000","last_fullsized":"2024-09-10T09:59:02.192556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189712+0000","last_change":"2024-09-10T09:59:02.189712+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189712+0000","last_undegraded":"2024-09-10T09:59:02.189712+0000","last_fullsized":"2024-09-10T09:59:02.189712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188828+0000","last_change":"2024-09-10T09:59:02.188828+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188828+0000","last_undegraded":"2024-09-10T09:59:02.188828+0000","last_fullsized":"2024-09-10T09:59:02.188828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281756+0000","last_change":"2024-09-10T09:59:03.210986+0000","last_active":"2024-09-10T09:59:05.281756+0000","last_peered":"2024-09-10T09:59:05.281756+0000","last_clean":"2024-09-10T09:59:05.281756+0000","last_became_active":"2024-09-10T09:59:03.210736+0000","last_became_peered":"2024-09-10T09:59:03.210736+0000","last_unstale":"2024-09-10T09:59:05.281756+0000","last_undegraded":"2024-09-10T09:59:05.281756+0000","last_fullsized":"2024-09-10T09:59:05.281756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:01.201854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188854+0000","last_change":"2024-09-10T09:59:02.188854+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188854+0000","last_undegraded":"2024-09-10T09:59:02.188854+0000","last_fullsized":"2024-09-10T09:59:02.188854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283365+0000","last_change":"2024-09-10T09:59:03.208433+0000","last_active":"2024-09-10T09:59:05.283365+0000","last_peered":"2024-09-10T09:59:05.283365+0000","last_clean":"2024-09-10T09:59:05.283365+0000","last_became_active":"2024-09-10T09:59:03.207106+0000","last_became_peered":"2024-09-10T09:59:03.207106+0000","last_unstale":"2024-09-10T09:59:05.283365+0000","last_undegraded":"2024-09-10T09:59:05.283365+0000","last_fullsized":"2024-09-10T09:59:05.283365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:13.286437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283272+0000","last_change":"2024-09-10T09:59:04.257770+0000","last_active":"2024-09-10T09:59:05.283272+0000","last_peered":"2024-09-10T09:59:05.283272+0000","last_clean":"2024-09-10T09:59:05.283272+0000","last_became_active":"2024-09-10T09:59:04.257535+0000","last_became_peered":"2024-09-10T09:59:04.257535+0000","last_unstale":"2024-09-10T09:59:05.283272+0000","last_undegraded":"2024-09-10T09:59:05.283272+0000","last_fullsized":"2024-09-10T09:59:05.283272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:27.392572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190726+0000","last_change":"2024-09-10T09:59:02.190726+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190726+0000","last_undegraded":"2024-09-10T09:59:02.190726+0000","last_fullsized":"2024-09-10T09:59:02.190726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281596+0000","last_change":"2024-09-10T09:59:04.234326+0000","last_active":"2024-09-10T09:59:05.281596+0000","last_peered":"2024-09-10T09:59:05.281596+0000","last_clean":"2024-09-10T09:59:05.281596+0000","last_became_active":"2024-09-10T09:59:04.234025+0000","last_became_peered":"2024-09-10T09:59:04.234025+0000","last_unstale":"2024-09-10T09:59:05.281596+0000","last_undegraded":"2024-09-10T09:59:05.281596+0000","last_fullsized":"2024-09-10T09:59:05.281596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:09.398256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192908+0000","last_change":"2024-09-10T09:59:02.192908+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192908+0000","last_undegraded":"2024-09-10T09:59:02.192908+0000","last_fullsized":"2024-09-10T09:59:02.192908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281699+0000","last_change":"2024-09-10T09:59:03.219831+0000","last_active":"2024-09-10T09:59:05.281699+0000","last_peered":"2024-09-10T09:59:05.281699+0000","last_clean":"2024-09-10T09:59:05.281699+0000","last_became_active":"2024-09-10T09:59:03.219272+0000","last_became_peered":"2024-09-10T09:59:03.219272+0000","last_unstale":"2024-09-10T09:59:05.281699+0000","last_undegraded":"2024-09-10T09:59:05.281699+0000","last_fullsized":"2024-09-10T09:59:05.281699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:03.029812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189265+0000","last_change":"2024-09-10T09:59:02.189265+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189265+0000","last_undegraded":"2024-09-10T09:59:02.189265+0000","last_fullsized":"2024-09-10T09:59:02.189265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281688+0000","last_change":"2024-09-10T09:59:04.233774+0000","last_active":"2024-09-10T09:59:05.281688+0000","last_peered":"2024-09-10T09:59:05.281688+0000","last_clean":"2024-09-10T09:59:05.281688+0000","last_became_active":"2024-09-10T09:59:04.233387+0000","last_became_peered":"2024-09-10T09:59:04.233387+0000","last_unstale":"2024-09-10T09:59:05.281688+0000","last_undegraded":"2024-09-10T09:59:05.281688+0000","last_fullsized":"2024-09-10T09:59:05.281688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:49.890997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.195166+0000","last_change":"2024-09-10T09:59:02.195166+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.195166+0000","last_undegraded":"2024-09-10T09:59:02.195166+0000","last_fullsized":"2024-09-10T09:59:02.195166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281843+0000","last_change":"2024-09-10T09:59:03.208322+0000","last_active":"2024-09-10T09:59:05.281843+0000","last_peered":"2024-09-10T09:59:05.281843+0000","last_clean":"2024-09-10T09:59:05.281843+0000","last_became_active":"2024-09-10T09:59:03.207255+0000","last_became_peered":"2024-09-10T09:59:03.207255+0000","last_unstale":"2024-09-10T09:59:05.281843+0000","last_undegraded":"2024-09-10T09:59:05.281843+0000","last_fullsized":"2024-09-10T09:59:05.281843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:37.076444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281289+0000","last_change":"2024-09-10T09:59:03.219531+0000","last_active":"2024-09-10T09:59:05.281289+0000","last_peered":"2024-09-10T09:59:05.281289+0000","last_clean":"2024-09-10T09:59:05.281289+0000","last_became_active":"2024-09-10T09:59:03.219216+0000","last_became_peered":"2024-09-10T09:59:03.219216+0000","last_unstale":"2024-09-10T09:59:05.281289+0000","last_undegraded":"2024-09-10T09:59:05.281289+0000","last_fullsized":"2024-09-10T09:59:05.281289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:03.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.194881+0000","last_change":"2024-09-10T09:59:02.194881+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.194881+0000","last_undegraded":"2024-09-10T09:59:02.194881+0000","last_fullsized":"2024-09-10T09:59:02.194881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189160+0000","last_change":"2024-09-10T09:59:02.189160+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189160+0000","last_undegraded":"2024-09-10T09:59:02.189160+0000","last_fullsized":"2024-09-10T09:59:02.189160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189685+0000","last_change":"2024-09-10T09:59:02.189685+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189685+0000","last_undegraded":"2024-09-10T09:59:02.189685+0000","last_fullsized":"2024-09-10T09:59:02.189685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189092+0000","last_change":"2024-09-10T09:59:02.189092+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189092+0000","last_undegraded":"2024-09-10T09:59:02.189092+0000","last_fullsized":"2024-09-10T09:59:02.189092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190612+0000","last_change":"2024-09-10T09:59:02.190612+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190612+0000","last_undegraded":"2024-09-10T09:59:02.190612+0000","last_fullsized":"2024-09-10T09:59:02.190612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190111+0000","last_change":"2024-09-10T09:59:02.190111+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190111+0000","last_undegraded":"2024-09-10T09:59:02.190111+0000","last_fullsized":"2024-09-10T09:59:02.190111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.190721+0000","last_change":"2024-09-10T09:59:02.190721+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.190721+0000","last_undegraded":"2024-09-10T09:59:02.190721+0000","last_fullsized":"2024-09-10T09:59:02.190721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.192879+0000","last_change":"2024-09-10T09:59:02.192879+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.192879+0000","last_undegraded":"2024-09-10T09:59:02.192879+0000","last_fullsized":"2024-09-10T09:59:02.192879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193914+0000","last_change":"2024-09-10T09:59:02.193914+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193914+0000","last_undegraded":"2024-09-10T09:59:02.193914+0000","last_fullsized":"2024-09-10T09:59:02.193914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189993+0000","last_change":"2024-09-10T09:59:02.189993+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189993+0000","last_undegraded":"2024-09-10T09:59:02.189993+0000","last_fullsized":"2024-09-10T09:59:02.189993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193859+0000","last_change":"2024-09-10T09:59:02.193859+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193859+0000","last_undegraded":"2024-09-10T09:59:02.193859+0000","last_fullsized":"2024-09-10T09:59:02.193859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189987+0000","last_change":"2024-09-10T09:59:02.189987+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189987+0000","last_undegraded":"2024-09-10T09:59:02.189987+0000","last_fullsized":"2024-09-10T09:59:02.189987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.193929+0000","last_change":"2024-09-10T09:59:02.193929+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.193929+0000","last_undegraded":"2024-09-10T09:59:02.193929+0000","last_fullsized":"2024-09-10T09:59:02.193929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281192+0000","last_change":"2024-09-10T09:59:04.231281+0000","last_active":"2024-09-10T09:59:05.281192+0000","last_peered":"2024-09-10T09:59:05.281192+0000","last_clean":"2024-09-10T09:59:05.281192+0000","last_became_active":"2024-09-10T09:59:04.231070+0000","last_became_peered":"2024-09-10T09:59:04.231070+0000","last_unstale":"2024-09-10T09:59:05.281192+0000","last_undegraded":"2024-09-10T09:59:05.281192+0000","last_fullsized":"2024-09-10T09:59:05.281192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:34.062362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.188810+0000","last_change":"2024-09-10T09:59:02.188810+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.188810+0000","last_undegraded":"2024-09-10T09:59:02.188810+0000","last_fullsized":"2024-09-10T09:59:02.188810+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281381+0000","last_change":"2024-09-10T09:59:03.208224+0000","last_active":"2024-09-10T09:59:05.281381+0000","last_peered":"2024-09-10T09:59:05.281381+0000","last_clean":"2024-09-10T09:59:05.281381+0000","last_became_active":"2024-09-10T09:59:03.207076+0000","last_became_peered":"2024-09-10T09:59:03.207076+0000","last_unstale":"2024-09-10T09:59:05.281381+0000","last_undegraded":"2024-09-10T09:59:05.281381+0000","last_fullsized":"2024-09-10T09:59:05.281381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:59.885497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T09:59:03.185155+0000","last_change":"2024-09-10T09:59:03.185155+0000","last_active":"2024-09-10T09:59:03.185155+0000","last_peered":"2024-09-10T09:59:03.185155+0000","last_clean":"2024-09-10T09:59:03.185155+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:03.185155+0000","last_undegraded":"2024-09-10T09:59:03.185155+0000","last_fullsized":"2024-09-10T09:59:03.185155+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-10T09:59:02.189619+0000","last_change":"2024-09-10T09:59:02.189619+0000","last_active":"2024-09-10T09:59:02.179205+0000","last_peered":"2024-09-10T09:59:02.179205+0000","last_clean":"2024-09-10T09:59:02.179205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T09:59:02.189619+0000","last_undegraded":"2024-09-10T09:59:02.189619+0000","last_fullsized":"2024-09-10T09:59:02.189619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":184,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T09:59:02.196835+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:59:02.196835+0000","last_peered":"2024-09-10T09:59:02.196835+0000","last_clean":"2024-09-10T09:59:02.196835+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:59:02.196835+0000","last_undegraded":"2024-09-10T09:59:02.196835+0000","last_fullsized":"2024-09-10T09:59:02.196835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":921600,"data_stored":709356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":124805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":124805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":118988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":921600,"data_stored":709356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:59:09.071 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3f is not active+clean 2024-09-10T09:59:09.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.071 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-10T09:59:09.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.071 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3c is not active+clean 2024-09-10T09:59:09.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:PG 5.39 is not active+clean 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:PG 5.38 is not active+clean 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:PG 5.36 is not active+clean 2024-09-10T09:59:09.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:PG 5.35 is not active+clean 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:PG 5.33 is not active+clean 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:PG 5.31 is not active+clean 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:PG 5.30 is not active+clean 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.073 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2f is not active+clean 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2c is not active+clean 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:PG 5.29 is not active+clean 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:PG 5.28 is not active+clean 2024-09-10T09:59:09.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:PG 5.27 is not active+clean 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:PG 5.25 is not active+clean 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:PG 5.24 is not active+clean 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:PG 5.22 is not active+clean 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:PG 5.21 is not active+clean 2024-09-10T09:59:09.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:PG 5.20 is not active+clean 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1f is not active+clean 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.190226+0000', 'last_change': '2024-09-10T09:59:02.190226+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.190226+0000', 'last_undegraded': '2024-09-10T09:59:02.190226+0000', 'last_fullsized': '2024-09-10T09:59:02.190226+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1e is not active+clean 2024-09-10T09:59:09.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.191133+0000', 'last_change': '2024-09-10T09:59:02.191133+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.191133+0000', 'last_undegraded': '2024-09-10T09:59:02.191133+0000', 'last_fullsized': '2024-09-10T09:59:02.191133+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:59:09.077 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-10T09:59:09.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.077 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1d is not active+clean 2024-09-10T09:59:09.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.077 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1c is not active+clean 2024-09-10T09:59:09.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.192556+0000', 'last_change': '2024-09-10T09:59:02.192556+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.192556+0000', 'last_undegraded': '2024-09-10T09:59:02.192556+0000', 'last_fullsized': '2024-09-10T09:59:02.192556+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T09:59:09.077 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-10T09:59:09.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1b is not active+clean 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.189712+0000', 'last_change': '2024-09-10T09:59:02.189712+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.189712+0000', 'last_undegraded': '2024-09-10T09:59:02.189712+0000', 'last_fullsized': '2024-09-10T09:59:02.189712+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1a is not active+clean 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.188828+0000', 'last_change': '2024-09-10T09:59:02.188828+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.188828+0000', 'last_undegraded': '2024-09-10T09:59:02.188828+0000', 'last_fullsized': '2024-09-10T09:59:02.188828+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:PG 5.19 is not active+clean 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:PG 4.18 is not active+clean 2024-09-10T09:59:09.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.188854+0000', 'last_change': '2024-09-10T09:59:02.188854+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.188854+0000', 'last_undegraded': '2024-09-10T09:59:02.188854+0000', 'last_fullsized': '2024-09-10T09:59:02.188854+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:PG 4.16 is not active+clean 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.190726+0000', 'last_change': '2024-09-10T09:59:02.190726+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.190726+0000', 'last_undegraded': '2024-09-10T09:59:02.190726+0000', 'last_fullsized': '2024-09-10T09:59:02.190726+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:PG 4.15 is not active+clean 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.192908+0000', 'last_change': '2024-09-10T09:59:02.192908+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.192908+0000', 'last_undegraded': '2024-09-10T09:59:02.192908+0000', 'last_fullsized': '2024-09-10T09:59:02.192908+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:PG 5.12 is not active+clean 2024-09-10T09:59:09.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:PG 4.13 is not active+clean 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.189265+0000', 'last_change': '2024-09-10T09:59:02.189265+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.189265+0000', 'last_undegraded': '2024-09-10T09:59:02.189265+0000', 'last_fullsized': '2024-09-10T09:59:02.189265+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:PG 4.12 is not active+clean 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.195166+0000', 'last_change': '2024-09-10T09:59:02.195166+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.195166+0000', 'last_undegraded': '2024-09-10T09:59:02.195166+0000', 'last_fullsized': '2024-09-10T09:59:02.195166+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-10T09:59:09.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:PG 4.f is not active+clean 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.194881+0000', 'last_change': '2024-09-10T09:59:02.194881+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.194881+0000', 'last_undegraded': '2024-09-10T09:59:02.194881+0000', 'last_fullsized': '2024-09-10T09:59:02.194881+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:PG 4.e is not active+clean 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.189160+0000', 'last_change': '2024-09-10T09:59:02.189160+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.189160+0000', 'last_undegraded': '2024-09-10T09:59:02.189160+0000', 'last_fullsized': '2024-09-10T09:59:02.189160+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:PG 4.d is not active+clean 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.189685+0000', 'last_change': '2024-09-10T09:59:02.189685+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.189685+0000', 'last_undegraded': '2024-09-10T09:59:02.189685+0000', 'last_fullsized': '2024-09-10T09:59:02.189685+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-09-10T09:59:09.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:PG 4.c is not active+clean 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.189092+0000', 'last_change': '2024-09-10T09:59:02.189092+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.189092+0000', 'last_undegraded': '2024-09-10T09:59:02.189092+0000', 'last_fullsized': '2024-09-10T09:59:02.189092+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:PG 4.b is not active+clean 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.190612+0000', 'last_change': '2024-09-10T09:59:02.190612+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.190612+0000', 'last_undegraded': '2024-09-10T09:59:02.190612+0000', 'last_fullsized': '2024-09-10T09:59:02.190612+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:PG 4.a is not active+clean 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.190111+0000', 'last_change': '2024-09-10T09:59:02.190111+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.190111+0000', 'last_undegraded': '2024-09-10T09:59:02.190111+0000', 'last_fullsized': '2024-09-10T09:59:02.190111+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:59:09.082 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:PG 4.9 is not active+clean 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.190721+0000', 'last_change': '2024-09-10T09:59:02.190721+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.190721+0000', 'last_undegraded': '2024-09-10T09:59:02.190721+0000', 'last_fullsized': '2024-09-10T09:59:02.190721+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:PG 4.8 is not active+clean 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.192879+0000', 'last_change': '2024-09-10T09:59:02.192879+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.192879+0000', 'last_undegraded': '2024-09-10T09:59:02.192879+0000', 'last_fullsized': '2024-09-10T09:59:02.192879+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.083 INFO:tasks.ceph.ceph_manager.ceph:PG 4.7 is not active+clean 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.193914+0000', 'last_change': '2024-09-10T09:59:02.193914+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.193914+0000', 'last_undegraded': '2024-09-10T09:59:02.193914+0000', 'last_fullsized': '2024-09-10T09:59:02.193914+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:PG 4.6 is not active+clean 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.189993+0000', 'last_change': '2024-09-10T09:59:02.189993+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.189993+0000', 'last_undegraded': '2024-09-10T09:59:02.189993+0000', 'last_fullsized': '2024-09-10T09:59:02.189993+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:PG 4.4 is not active+clean 2024-09-10T09:59:09.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.193859+0000', 'last_change': '2024-09-10T09:59:02.193859+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.193859+0000', 'last_undegraded': '2024-09-10T09:59:02.193859+0000', 'last_fullsized': '2024-09-10T09:59:02.193859+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:PG 4.3 is not active+clean 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.189987+0000', 'last_change': '2024-09-10T09:59:02.189987+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.189987+0000', 'last_undegraded': '2024-09-10T09:59:02.189987+0000', 'last_fullsized': '2024-09-10T09:59:02.189987+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:PG 4.2 is not active+clean 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.193929+0000', 'last_change': '2024-09-10T09:59:02.193929+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.193929+0000', 'last_undegraded': '2024-09-10T09:59:02.193929+0000', 'last_fullsized': '2024-09-10T09:59:02.193929+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1 is not active+clean 2024-09-10T09:59:09.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.188810+0000', 'last_change': '2024-09-10T09:59:02.188810+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.188810+0000', 'last_undegraded': '2024-09-10T09:59:02.188810+0000', 'last_fullsized': '2024-09-10T09:59:02.188810+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T09:59:09.086 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-10T09:59:09.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.086 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-10T09:59:09.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T09:59:03.185155+0000', 'last_change': '2024-09-10T09:59:03.185155+0000', 'last_active': '2024-09-10T09:59:03.185155+0000', 'last_peered': '2024-09-10T09:59:03.185155+0000', 'last_clean': '2024-09-10T09:59:03.185155+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:03.185155+0000', 'last_undegraded': '2024-09-10T09:59:03.185155+0000', 'last_fullsized': '2024-09-10T09:59:03.185155+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:03.185155+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T09:59:09.086 INFO:tasks.ceph.ceph_manager.ceph:PG 4.5 is not active+clean 2024-09-10T09:59:09.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-10T09:59:02.189619+0000', 'last_change': '2024-09-10T09:59:02.189619+0000', 'last_active': '2024-09-10T09:59:02.179205+0000', 'last_peered': '2024-09-10T09:59:02.179205+0000', 'last_clean': '2024-09-10T09:59:02.179205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T09:59:02.189619+0000', 'last_undegraded': '2024-09-10T09:59:02.189619+0000', 'last_fullsized': '2024-09-10T09:59:02.189619+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'last_clean_scrub_stamp': '2024-09-10T09:59:02.179205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T09:59:09.087 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T09:59:09.180 INFO:tasks.ceph.mds.c.smithi110.stderr:2024-09-10T09:59:09.175+0000 7fbfe8600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T09:59:09.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2e1000640 1 Processor -- start 2024-09-10T09:59:09.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2e1000640 1 -- start start 2024-09-10T09:59:09.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2e1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc155970 0x7fa2dc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2e1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2dc06f1e0 0x7fa2dc0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2e1000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2dc155d50 con 0x7fa2dc15c560 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2e1000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa2dc0b4570 con 0x7fa2dc06f1e0 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2e1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc0b34d0 con 0x7fa2dc155970 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2da800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2dc06f1e0 0x7fa2dc0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2d1400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa2dc15c560 0x7fa2dc06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52624/0 (socket says 172.21.15.110:52624) 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2da800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2dc06f1e0 0x7fa2dc0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46230/0 (socket says 172.21.15.110:46230) 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2d1400640 1 -- 172.21.15.110:0/1008969794 learned_addr learned my addr 172.21.15.110:0/1008969794 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.239+0000 7fa2d9e00640 1 --2- 172.21.15.110:0/1008969794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc155970 0x7fa2dc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2da800640 1 -- 172.21.15.110:0/1008969794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc155970 msgr2=0x7fa2dc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2da800640 1 --2- 172.21.15.110:0/1008969794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc155970 0x7fa2dc0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2da800640 1 -- 172.21.15.110:0/1008969794 >> v1:172.21.15.110:6789/0 conn(0x7fa2dc15c560 legacy=0x7fa2dc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2da800640 1 -- 172.21.15.110:0/1008969794 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc0b37b0 con 0x7fa2dc06f1e0 2024-09-10T09:59:09.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2d9400640 1 -- 172.21.15.110:0/1008969794 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3674892056 0 0) 0x7fa2dc155d50 con 0x7fa2dc15c560 2024-09-10T09:59:09.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2da800640 1 --2- 172.21.15.110:0/1008969794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc06f1e0 0x7fa2dc0ce420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009d90 tx=0x7fa2d401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596d25e633f34a75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:09.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2d9400640 1 -- 172.21.15.110:0/1008969794 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40b2070 con 0x7fa2dc06f1e0 2024-09-10T09:59:09.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2d9400640 1 -- 172.21.15.110:0/1008969794 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d40a5970 con 0x7fa2dc06f1e0 2024-09-10T09:59:09.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2d9400640 1 -- 172.21.15.110:0/1008969794 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40a5d40 con 0x7fa2dc06f1e0 2024-09-10T09:59:09.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2e1000640 1 -- 172.21.15.110:0/1008969794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc06f1e0 msgr2=0x7fa2dc0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2e1000640 1 --2- 172.21.15.110:0/1008969794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc06f1e0 0x7fa2dc0ce420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009d90 tx=0x7fa2d401c8b0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2e1000640 1 -- 172.21.15.110:0/1008969794 shutdown_connections 2024-09-10T09:59:09.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2e1000640 1 --2- 172.21.15.110:0/1008969794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc06f1e0 0x7fa2dc0ce420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2e1000640 1 --2- 172.21.15.110:0/1008969794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc155970 0x7fa2dc0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.243+0000 7fa2e1000640 1 -- 172.21.15.110:0/1008969794 >> 172.21.15.110:0/1008969794 conn(0x7fa2dc06bc50 msgr2=0x7fa2dc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:09.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 -- 172.21.15.110:0/1008969794 shutdown_connections 2024-09-10T09:59:09.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 -- 172.21.15.110:0/1008969794 wait complete. 2024-09-10T09:59:09.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 Processor -- start 2024-09-10T09:59:09.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 -- start start 2024-09-10T09:59:09.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc06f1e0 0x7fa2dc15d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc155970 0x7fa2dc1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2dc15c560 0x7fa2dc14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0cf170 con 0x7fa2dc155970 2024-09-10T09:59:09.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc0cce00 con 0x7fa2dc15c560 2024-09-10T09:59:09.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc0cf2f0 con 0x7fa2dc06f1e0 2024-09-10T09:59:09.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc155970 0x7fa2dc1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc155970 0x7fa2dc1516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46236/0 (socket says 172.21.15.110:46236) 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 -- 172.21.15.110:0/3130080083 learned_addr learned my addr 172.21.15.110:0/3130080083 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2da800640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2dc15c560 0x7fa2dc14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d9e00640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc06f1e0 0x7fa2dc15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 -- 172.21.15.110:0/3130080083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc06f1e0 msgr2=0x7fa2dc15d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc06f1e0 0x7fa2dc15d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 -- 172.21.15.110:0/3130080083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2dc15c560 msgr2=0x7fa2dc14d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2dc15c560 0x7fa2dc14d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 -- 172.21.15.110:0/3130080083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc15cbd0 con 0x7fa2dc155970 2024-09-10T09:59:09.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d1400640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc155970 0x7fa2dc1516a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400c450 tx=0x7fa2c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:09.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d2a00640 1 -- 172.21.15.110:0/3130080083 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4018070 con 0x7fa2dc155970 2024-09-10T09:59:09.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d2a00640 1 -- 172.21.15.110:0/3130080083 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c40074e0 con 0x7fa2dc155970 2024-09-10T09:59:09.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2d2a00640 1 -- 172.21.15.110:0/3130080083 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c40049f0 con 0x7fa2dc155970 2024-09-10T09:59:09.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc15ceb0 con 0x7fa2dc155970 2024-09-10T09:59:09.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.247+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc15d470 con 0x7fa2dc155970 2024-09-10T09:59:09.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.251+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc0ce420 con 0x7fa2dc155970 2024-09-10T09:59:09.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.251+0000 7fa2d2a00640 1 -- 172.21.15.110:0/3130080083 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa2c4004b90 con 0x7fa2dc155970 2024-09-10T09:59:09.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.251+0000 7fa2d2a00640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2b80678c0 0x7fa2b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.251+0000 7fa2d2a00640 1 -- 172.21.15.110:0/3130080083 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa2c4084fc0 con 0x7fa2dc155970 2024-09-10T09:59:09.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.251+0000 7fa2d9e00640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2b80678c0 0x7fa2b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.251+0000 7fa2d9e00640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2b80678c0 0x7fa2b8069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009b60 tx=0x7fa2cc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:09.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.255+0000 7fa2d2a00640 1 -- 172.21.15.110:0/3130080083 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa2c4055410 con 0x7fa2dc155970 2024-09-10T09:59:09.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa2dc15bd70 con 0x7fa2dc155970 2024-09-10T09:59:09.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2d2a00640 1 -- 172.21.15.110:0/3130080083 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2828 (secure 0 0 0) 0x7fa2c4059710 con 0x7fa2dc155970 2024-09-10T09:59:09.478 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:09.478 INFO:teuthology.orchestra.run.smithi110.stdout:{"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":71,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1725962292,"num_in_osds":8,"osd_in_since":1725962280,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":54},{"state_name":"creating+peering","count":25},{"state_name":"active+clean","count":18}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":591980,"bytes_used":242401280,"bytes_avail":772851712000,"bytes_total":773094113280,"unknown_pgs_ratio":0.55670100450515747,"inactive_pgs_ratio":0.25773194432258606,"write_bytes_sec":593,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":16,"btime":"2024-09-10T09:59:07:250101+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4776}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-10T09:59:05.364064+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T09:59:09.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2b80678c0 msgr2=0x7fa2b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2b80678c0 0x7fa2b8069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009b60 tx=0x7fa2cc009190 comp rx=0 tx=0).stop 2024-09-10T09:59:09.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc155970 msgr2=0x7fa2dc1516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc155970 0x7fa2dc1516a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400c450 tx=0x7fa2c400c910 comp rx=0 tx=0).stop 2024-09-10T09:59:09.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 shutdown_connections 2024-09-10T09:59:09.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2b80678c0 0x7fa2b8069d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2dc15c560 0x7fa2dc14d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2dc155970 0x7fa2dc1516a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 --2- 172.21.15.110:0/3130080083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2dc06f1e0 0x7fa2dc15d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 >> 172.21.15.110:0/3130080083 conn(0x7fa2dc06bc50 msgr2=0x7fa2dc0cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:09.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 shutdown_connections 2024-09-10T09:59:09.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.475+0000 7fa2e1000640 1 -- 172.21.15.110:0/3130080083 wait complete. 2024-09-10T09:59:09.499 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T09:59:09.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:59:09.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e1ec00640 1 Processor -- start 2024-09-10T09:59:09.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e1ec00640 1 -- start start 2024-09-10T09:59:09.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e1ec00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3e180cd510 0x7f3e180cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e1ec00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e1ec00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e18072150 con 0x7f3e180cfe30 2024-09-10T09:59:09.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e1ec00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3e1815d9a0 con 0x7f3e180cd510 2024-09-10T09:59:09.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e1ec00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1806ea60 con 0x7f3e1806e090 2024-09-10T09:59:09.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e17400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3e180cd510 0x7f3e180cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e17400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3e180cd510 0x7f3e180cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46238/0 (socket says 172.21.15.110:46238) 2024-09-10T09:59:09.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e17400640 1 -- 172.21.15.110:0/2387210902 learned_addr learned my addr 172.21.15.110:0/2387210902 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:09.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.631+0000 7f3e17e00640 1 --1- 172.21.15.110:0/2387210902 >> v1:172.21.15.110:6789/0 conn(0x7f3e180cfe30 0x7f3e18071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52640/0 (socket says 172.21.15.110:52640) 2024-09-10T09:59:09.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1cc00640 1 --2- 172.21.15.110:0/2387210902 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e17400640 1 -- 172.21.15.110:0/2387210902 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 msgr2=0x7f3e1806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:09.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e17400640 1 --2- 172.21.15.110:0/2387210902 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1806e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e17400640 1 -- 172.21.15.110:0/2387210902 >> v1:172.21.15.110:6789/0 conn(0x7f3e180cfe30 legacy=0x7f3e18071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e17400640 1 -- 172.21.15.110:0/2387210902 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1806d2d0 con 0x7f3e180cd510 2024-09-10T09:59:09.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e16a00640 1 -- 172.21.15.110:0/2387210902 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862833495 0 0) 0x7f3e18072150 con 0x7f3e180cfe30 2024-09-10T09:59:09.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1cc00640 1 --2- 172.21.15.110:0/2387210902 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1806e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:09.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e17400640 1 --2- 172.21.15.110:0/2387210902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cd510 0x7f3e180cd9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009870 tx=0x7f3e0401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e886daea392afbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e16a00640 1 -- 172.21.15.110:0/2387210902 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e040b1070 con 0x7f3e180cd510 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e16a00640 1 -- 172.21.15.110:0/2387210902 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e040a8050 con 0x7f3e180cd510 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e16a00640 1 -- 172.21.15.110:0/2387210902 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e040ac860 con 0x7f3e180cd510 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2387210902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cd510 msgr2=0x7f3e180cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2387210902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cd510 0x7f3e180cd9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009870 tx=0x7f3e0401c4d0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2387210902 shutdown_connections 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2387210902 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1806e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2387210902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cd510 0x7f3e180cd9a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2387210902 >> 172.21.15.110:0/2387210902 conn(0x7f3e1806bc50 msgr2=0x7f3e1806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2387210902 shutdown_connections 2024-09-10T09:59:09.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.635+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2387210902 wait complete. 2024-09-10T09:59:09.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 Processor -- start 2024-09-10T09:59:09.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 -- start start 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1814a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e180cd510 0x7f3e1815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cfe30 0x7f3e18155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18169180 con 0x7f3e180cfe30 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1815d6c0 con 0x7f3e180cd510 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3e18169300 con 0x7f3e1806e090 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cfe30 0x7f3e18155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cfe30 0x7f3e18155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46244/0 (socket says 172.21.15.110:46244) 2024-09-10T09:59:09.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 -- 172.21.15.110:0/2619015136 learned_addr learned my addr 172.21.15.110:0/2619015136 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e17400640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e180cd510 0x7f3e1815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e17e00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1814a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 -- 172.21.15.110:0/2619015136 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 msgr2=0x7f3e1814a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1814a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 -- 172.21.15.110:0/2619015136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e180cd510 msgr2=0x7f3e1815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e180cd510 0x7f3e1815a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 -- 172.21.15.110:0/2619015136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1815b050 con 0x7f3e180cfe30 2024-09-10T09:59:09.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1cc00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cfe30 0x7f3e18155a30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c820 tx=0x7f3e1000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:09.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e0fe00640 1 -- 172.21.15.110:0/2619015136 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10015020 con 0x7f3e180cfe30 2024-09-10T09:59:09.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e0fe00640 1 -- 172.21.15.110:0/2619015136 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e10004240 con 0x7f3e180cfe30 2024-09-10T09:59:09.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e0fe00640 1 -- 172.21.15.110:0/2619015136 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10003620 con 0x7f3e180cfe30 2024-09-10T09:59:09.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1815b330 con 0x7f3e180cfe30 2024-09-10T09:59:09.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.639+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e18155510 con 0x7f3e180cfe30 2024-09-10T09:59:09.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.643+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1806e4a0 con 0x7f3e180cfe30 2024-09-10T09:59:09.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.643+0000 7f3e0fe00640 1 -- 172.21.15.110:0/2619015136 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3e10018020 con 0x7f3e180cfe30 2024-09-10T09:59:09.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.643+0000 7f3e0fe00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3df80678c0 0x7f3df8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:09.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.643+0000 7f3e0fe00640 1 -- 172.21.15.110:0/2619015136 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3e10085c10 con 0x7f3e180cfe30 2024-09-10T09:59:09.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.643+0000 7f3e17e00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3df80678c0 0x7f3df8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:09.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.643+0000 7f3e17e00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3df80678c0 0x7f3df8069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e080012f0 tx=0x7f3e08001690 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:09.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.647+0000 7f3e0fe00640 1 -- 172.21.15.110:0/2619015136 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3e1001f020 con 0x7f3e180cfe30 2024-09-10T09:59:09.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.803+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e1815f7a0 con 0x7f3df80678c0 2024-09-10T09:59:09.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.807+0000 7f3e0fe00640 1 -- 172.21.15.110:0/2619015136 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f3e1815f7a0 con 0x7f3df80678c0 2024-09-10T09:59:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:09.815 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3df80678c0 msgr2=0x7f3df8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3df80678c0 0x7f3df8069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e080012f0 tx=0x7f3e08001690 comp rx=0 tx=0).stop 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cfe30 msgr2=0x7f3e18155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cfe30 0x7f3e18155a30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c820 tx=0x7f3e1000cce0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 shutdown_connections 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3df80678c0 0x7f3df8069d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e180cfe30 0x7f3e18155a30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e180cd510 0x7f3e1815a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 --2- 172.21.15.110:0/2619015136 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3e1806e090 0x7f3e1814a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 >> 172.21.15.110:0/2619015136 conn(0x7f3e1806bc50 msgr2=0x7f3e1806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 shutdown_connections 2024-09-10T09:59:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:09.811+0000 7f3e1ec00640 1 -- 172.21.15.110:0/2619015136 wait complete. 2024-09-10T09:59:09.835 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-10T09:59:09.364686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237048,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737672,"statfs":{"total":773094113280,"available":772851376128,"internally_reserved":0,"allocated":4268032,"data_stored":2338732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.175583"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238936+0000","last_change":"2024-09-10T09:59:04.228967+0000","last_active":"2024-09-10T09:59:05.238936+0000","last_peered":"2024-09-10T09:59:05.238936+0000","last_clean":"2024-09-10T09:59:05.238936+0000","last_became_active":"2024-09-10T09:59:04.228750+0000","last_became_peered":"2024-09-10T09:59:04.228750+0000","last_unstale":"2024-09-10T09:59:05.238936+0000","last_undegraded":"2024-09-10T09:59:05.238936+0000","last_fullsized":"2024-09-10T09:59:05.238936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:44.726674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282874+0000","last_change":"2024-09-10T09:59:04.248176+0000","last_active":"2024-09-10T09:59:05.282874+0000","last_peered":"2024-09-10T09:59:05.282874+0000","last_clean":"2024-09-10T09:59:05.282874+0000","last_became_active":"2024-09-10T09:59:04.247900+0000","last_became_peered":"2024-09-10T09:59:04.247900+0000","last_unstale":"2024-09-10T09:59:05.282874+0000","last_undegraded":"2024-09-10T09:59:05.282874+0000","last_fullsized":"2024-09-10T09:59:05.282874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:33.156203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282745+0000","last_change":"2024-09-10T09:59:04.239138+0000","last_active":"2024-09-10T09:59:05.282745+0000","last_peered":"2024-09-10T09:59:05.282745+0000","last_clean":"2024-09-10T09:59:05.282745+0000","last_became_active":"2024-09-10T09:59:04.238635+0000","last_became_peered":"2024-09-10T09:59:04.238635+0000","last_unstale":"2024-09-10T09:59:05.282745+0000","last_undegraded":"2024-09-10T09:59:05.282745+0000","last_fullsized":"2024-09-10T09:59:05.282745+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:02.620413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282877+0000","last_change":"2024-09-10T09:59:04.245525+0000","last_active":"2024-09-10T09:59:05.282877+0000","last_peered":"2024-09-10T09:59:05.282877+0000","last_clean":"2024-09-10T09:59:05.282877+0000","last_became_active":"2024-09-10T09:59:04.245226+0000","last_became_peered":"2024-09-10T09:59:04.245226+0000","last_unstale":"2024-09-10T09:59:05.282877+0000","last_undegraded":"2024-09-10T09:59:05.282877+0000","last_fullsized":"2024-09-10T09:59:05.282877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:59.181205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281986+0000","last_change":"2024-09-10T09:59:04.238166+0000","last_active":"2024-09-10T09:59:05.281986+0000","last_peered":"2024-09-10T09:59:05.281986+0000","last_clean":"2024-09-10T09:59:05.281986+0000","last_became_active":"2024-09-10T09:59:04.237855+0000","last_became_peered":"2024-09-10T09:59:04.237855+0000","last_unstale":"2024-09-10T09:59:05.281986+0000","last_undegraded":"2024-09-10T09:59:05.281986+0000","last_fullsized":"2024-09-10T09:59:05.281986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:58.954014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281677+0000","last_change":"2024-09-10T09:59:04.244110+0000","last_active":"2024-09-10T09:59:05.281677+0000","last_peered":"2024-09-10T09:59:05.281677+0000","last_clean":"2024-09-10T09:59:05.281677+0000","last_became_active":"2024-09-10T09:59:04.243830+0000","last_became_peered":"2024-09-10T09:59:04.243830+0000","last_unstale":"2024-09-10T09:59:05.281677+0000","last_undegraded":"2024-09-10T09:59:05.281677+0000","last_fullsized":"2024-09-10T09:59:05.281677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:01.116838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282524+0000","last_change":"2024-09-10T09:59:04.237928+0000","last_active":"2024-09-10T09:59:05.282524+0000","last_peered":"2024-09-10T09:59:05.282524+0000","last_clean":"2024-09-10T09:59:05.282524+0000","last_became_active":"2024-09-10T09:59:04.237485+0000","last_became_peered":"2024-09-10T09:59:04.237485+0000","last_unstale":"2024-09-10T09:59:05.282524+0000","last_undegraded":"2024-09-10T09:59:05.282524+0000","last_fullsized":"2024-09-10T09:59:05.282524+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:06.851574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282259+0000","last_change":"2024-09-10T09:59:04.254726+0000","last_active":"2024-09-10T09:59:05.282259+0000","last_peered":"2024-09-10T09:59:05.282259+0000","last_clean":"2024-09-10T09:59:05.282259+0000","last_became_active":"2024-09-10T09:59:04.254503+0000","last_became_peered":"2024-09-10T09:59:04.254503+0000","last_unstale":"2024-09-10T09:59:05.282259+0000","last_undegraded":"2024-09-10T09:59:05.282259+0000","last_fullsized":"2024-09-10T09:59:05.282259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:56.008342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238963+0000","last_change":"2024-09-10T09:59:04.248675+0000","last_active":"2024-09-10T09:59:05.238963+0000","last_peered":"2024-09-10T09:59:05.238963+0000","last_clean":"2024-09-10T09:59:05.238963+0000","last_became_active":"2024-09-10T09:59:04.248367+0000","last_became_peered":"2024-09-10T09:59:04.248367+0000","last_unstale":"2024-09-10T09:59:05.238963+0000","last_undegraded":"2024-09-10T09:59:05.238963+0000","last_fullsized":"2024-09-10T09:59:05.238963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.785736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283201+0000","last_change":"2024-09-10T09:59:04.234085+0000","last_active":"2024-09-10T09:59:05.283201+0000","last_peered":"2024-09-10T09:59:05.283201+0000","last_clean":"2024-09-10T09:59:05.283201+0000","last_became_active":"2024-09-10T09:59:04.233765+0000","last_became_peered":"2024-09-10T09:59:04.233765+0000","last_unstale":"2024-09-10T09:59:05.283201+0000","last_undegraded":"2024-09-10T09:59:05.283201+0000","last_fullsized":"2024-09-10T09:59:05.283201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:50.791729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282976+0000","last_change":"2024-09-10T09:59:04.242186+0000","last_active":"2024-09-10T09:59:05.282976+0000","last_peered":"2024-09-10T09:59:05.282976+0000","last_clean":"2024-09-10T09:59:05.282976+0000","last_became_active":"2024-09-10T09:59:04.241941+0000","last_became_peered":"2024-09-10T09:59:04.241941+0000","last_unstale":"2024-09-10T09:59:05.282976+0000","last_undegraded":"2024-09-10T09:59:05.282976+0000","last_fullsized":"2024-09-10T09:59:05.282976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:25.498252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282404+0000","last_change":"2024-09-10T09:59:04.234960+0000","last_active":"2024-09-10T09:59:05.282404+0000","last_peered":"2024-09-10T09:59:05.282404+0000","last_clean":"2024-09-10T09:59:05.282404+0000","last_became_active":"2024-09-10T09:59:04.234681+0000","last_became_peered":"2024-09-10T09:59:04.234681+0000","last_unstale":"2024-09-10T09:59:05.282404+0000","last_undegraded":"2024-09-10T09:59:05.282404+0000","last_fullsized":"2024-09-10T09:59:05.282404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:53.610398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282124+0000","last_change":"2024-09-10T09:59:04.249214+0000","last_active":"2024-09-10T09:59:05.282124+0000","last_peered":"2024-09-10T09:59:05.282124+0000","last_clean":"2024-09-10T09:59:05.282124+0000","last_became_active":"2024-09-10T09:59:04.248871+0000","last_became_peered":"2024-09-10T09:59:04.248871+0000","last_unstale":"2024-09-10T09:59:05.282124+0000","last_undegraded":"2024-09-10T09:59:05.282124+0000","last_fullsized":"2024-09-10T09:59:05.282124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:44.565690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281767+0000","last_change":"2024-09-10T09:59:04.247517+0000","last_active":"2024-09-10T09:59:05.281767+0000","last_peered":"2024-09-10T09:59:05.281767+0000","last_clean":"2024-09-10T09:59:05.281767+0000","last_became_active":"2024-09-10T09:59:04.247232+0000","last_became_peered":"2024-09-10T09:59:04.247232+0000","last_unstale":"2024-09-10T09:59:05.281767+0000","last_undegraded":"2024-09-10T09:59:05.281767+0000","last_fullsized":"2024-09-10T09:59:05.281767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:31.397569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282058+0000","last_change":"2024-09-10T09:59:04.233979+0000","last_active":"2024-09-10T09:59:05.282058+0000","last_peered":"2024-09-10T09:59:05.282058+0000","last_clean":"2024-09-10T09:59:05.282058+0000","last_became_active":"2024-09-10T09:59:04.233741+0000","last_became_peered":"2024-09-10T09:59:04.233741+0000","last_unstale":"2024-09-10T09:59:05.282058+0000","last_undegraded":"2024-09-10T09:59:05.282058+0000","last_fullsized":"2024-09-10T09:59:05.282058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:56.008342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282881+0000","last_change":"2024-09-10T09:59:04.248489+0000","last_active":"2024-09-10T09:59:05.282881+0000","last_peered":"2024-09-10T09:59:05.282881+0000","last_clean":"2024-09-10T09:59:05.282881+0000","last_became_active":"2024-09-10T09:59:04.248249+0000","last_became_peered":"2024-09-10T09:59:04.248249+0000","last_unstale":"2024-09-10T09:59:05.282881+0000","last_undegraded":"2024-09-10T09:59:05.282881+0000","last_fullsized":"2024-09-10T09:59:05.282881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:02.620413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283193+0000","last_change":"2024-09-10T09:59:04.237672+0000","last_active":"2024-09-10T09:59:05.283193+0000","last_peered":"2024-09-10T09:59:05.283193+0000","last_clean":"2024-09-10T09:59:05.283193+0000","last_became_active":"2024-09-10T09:59:04.236129+0000","last_became_peered":"2024-09-10T09:59:04.236129+0000","last_unstale":"2024-09-10T09:59:05.283193+0000","last_undegraded":"2024-09-10T09:59:05.283193+0000","last_fullsized":"2024-09-10T09:59:05.283193+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:09.621494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283009+0000","last_change":"2024-09-10T09:59:04.257031+0000","last_active":"2024-09-10T09:59:05.283009+0000","last_peered":"2024-09-10T09:59:05.283009+0000","last_clean":"2024-09-10T09:59:05.283009+0000","last_became_active":"2024-09-10T09:59:04.256552+0000","last_became_peered":"2024-09-10T09:59:04.256552+0000","last_unstale":"2024-09-10T09:59:05.283009+0000","last_undegraded":"2024-09-10T09:59:05.283009+0000","last_fullsized":"2024-09-10T09:59:05.283009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:02.774151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282345+0000","last_change":"2024-09-10T09:59:04.256930+0000","last_active":"2024-09-10T09:59:05.282345+0000","last_peered":"2024-09-10T09:59:05.282345+0000","last_clean":"2024-09-10T09:59:05.282345+0000","last_became_active":"2024-09-10T09:59:04.256147+0000","last_became_peered":"2024-09-10T09:59:04.256147+0000","last_unstale":"2024-09-10T09:59:05.282345+0000","last_undegraded":"2024-09-10T09:59:05.282345+0000","last_fullsized":"2024-09-10T09:59:05.282345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:17.464661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282613+0000","last_change":"2024-09-10T09:59:04.241976+0000","last_active":"2024-09-10T09:59:05.282613+0000","last_peered":"2024-09-10T09:59:05.282613+0000","last_clean":"2024-09-10T09:59:05.282613+0000","last_became_active":"2024-09-10T09:59:04.241667+0000","last_became_peered":"2024-09-10T09:59:04.241667+0000","last_unstale":"2024-09-10T09:59:05.282613+0000","last_undegraded":"2024-09-10T09:59:05.282613+0000","last_fullsized":"2024-09-10T09:59:05.282613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:43.713238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239003+0000","last_change":"2024-09-10T09:59:04.232895+0000","last_active":"2024-09-10T09:59:05.239003+0000","last_peered":"2024-09-10T09:59:05.239003+0000","last_clean":"2024-09-10T09:59:05.239003+0000","last_became_active":"2024-09-10T09:59:04.232628+0000","last_became_peered":"2024-09-10T09:59:04.232628+0000","last_unstale":"2024-09-10T09:59:05.239003+0000","last_undegraded":"2024-09-10T09:59:05.239003+0000","last_fullsized":"2024-09-10T09:59:05.239003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:36.495442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239585+0000","last_change":"2024-09-10T09:59:04.235308+0000","last_active":"2024-09-10T09:59:05.239585+0000","last_peered":"2024-09-10T09:59:05.239585+0000","last_clean":"2024-09-10T09:59:05.239585+0000","last_became_active":"2024-09-10T09:59:04.233274+0000","last_became_peered":"2024-09-10T09:59:04.233274+0000","last_unstale":"2024-09-10T09:59:05.239585+0000","last_undegraded":"2024-09-10T09:59:05.239585+0000","last_fullsized":"2024-09-10T09:59:05.239585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:39.465659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234473+0000","last_change":"2024-09-10T09:59:04.232149+0000","last_active":"2024-09-10T09:59:05.234473+0000","last_peered":"2024-09-10T09:59:05.234473+0000","last_clean":"2024-09-10T09:59:05.234473+0000","last_became_active":"2024-09-10T09:59:04.231828+0000","last_became_peered":"2024-09-10T09:59:04.231828+0000","last_unstale":"2024-09-10T09:59:05.234473+0000","last_undegraded":"2024-09-10T09:59:05.234473+0000","last_fullsized":"2024-09-10T09:59:05.234473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:44.726674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281997+0000","last_change":"2024-09-10T09:59:04.244645+0000","last_active":"2024-09-10T09:59:05.281997+0000","last_peered":"2024-09-10T09:59:05.281997+0000","last_clean":"2024-09-10T09:59:05.281997+0000","last_became_active":"2024-09-10T09:59:04.244315+0000","last_became_peered":"2024-09-10T09:59:04.244315+0000","last_unstale":"2024-09-10T09:59:05.281997+0000","last_undegraded":"2024-09-10T09:59:05.281997+0000","last_fullsized":"2024-09-10T09:59:05.281997+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:35.416635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283299+0000","last_change":"2024-09-10T09:59:04.247910+0000","last_active":"2024-09-10T09:59:05.283299+0000","last_peered":"2024-09-10T09:59:05.283299+0000","last_clean":"2024-09-10T09:59:05.283299+0000","last_became_active":"2024-09-10T09:59:04.247541+0000","last_became_peered":"2024-09-10T09:59:04.247541+0000","last_unstale":"2024-09-10T09:59:05.283299+0000","last_undegraded":"2024-09-10T09:59:05.283299+0000","last_fullsized":"2024-09-10T09:59:05.283299+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:35.416635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283126+0000","last_change":"2024-09-10T09:59:04.235085+0000","last_active":"2024-09-10T09:59:05.283126+0000","last_peered":"2024-09-10T09:59:05.283126+0000","last_clean":"2024-09-10T09:59:05.283126+0000","last_became_active":"2024-09-10T09:59:04.232953+0000","last_became_peered":"2024-09-10T09:59:04.232953+0000","last_unstale":"2024-09-10T09:59:05.283126+0000","last_undegraded":"2024-09-10T09:59:05.283126+0000","last_fullsized":"2024-09-10T09:59:05.283126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:57.657194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282615+0000","last_change":"2024-09-10T09:59:04.245634+0000","last_active":"2024-09-10T09:59:05.282615+0000","last_peered":"2024-09-10T09:59:05.282615+0000","last_clean":"2024-09-10T09:59:05.282615+0000","last_became_active":"2024-09-10T09:59:04.245286+0000","last_became_peered":"2024-09-10T09:59:04.245286+0000","last_unstale":"2024-09-10T09:59:05.282615+0000","last_undegraded":"2024-09-10T09:59:05.282615+0000","last_fullsized":"2024-09-10T09:59:05.282615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:36.422543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282574+0000","last_change":"2024-09-10T09:59:04.235825+0000","last_active":"2024-09-10T09:59:05.282574+0000","last_peered":"2024-09-10T09:59:05.282574+0000","last_clean":"2024-09-10T09:59:05.282574+0000","last_became_active":"2024-09-10T09:59:04.235305+0000","last_became_peered":"2024-09-10T09:59:04.235305+0000","last_unstale":"2024-09-10T09:59:05.282574+0000","last_undegraded":"2024-09-10T09:59:05.282574+0000","last_fullsized":"2024-09-10T09:59:05.282574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:50.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239170+0000","last_change":"2024-09-10T09:59:04.233036+0000","last_active":"2024-09-10T09:59:05.239170+0000","last_peered":"2024-09-10T09:59:05.239170+0000","last_clean":"2024-09-10T09:59:05.239170+0000","last_became_active":"2024-09-10T09:59:04.232793+0000","last_became_peered":"2024-09-10T09:59:04.232793+0000","last_unstale":"2024-09-10T09:59:05.239170+0000","last_undegraded":"2024-09-10T09:59:05.239170+0000","last_fullsized":"2024-09-10T09:59:05.239170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:36.422543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282455+0000","last_change":"2024-09-10T09:59:04.233379+0000","last_active":"2024-09-10T09:59:05.282455+0000","last_peered":"2024-09-10T09:59:05.282455+0000","last_clean":"2024-09-10T09:59:05.282455+0000","last_became_active":"2024-09-10T09:59:04.232985+0000","last_became_peered":"2024-09-10T09:59:04.232985+0000","last_unstale":"2024-09-10T09:59:05.282455+0000","last_undegraded":"2024-09-10T09:59:05.282455+0000","last_fullsized":"2024-09-10T09:59:05.282455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:53.610398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238690+0000","last_change":"2024-09-10T09:59:04.244628+0000","last_active":"2024-09-10T09:59:05.238690+0000","last_peered":"2024-09-10T09:59:05.238690+0000","last_clean":"2024-09-10T09:59:05.238690+0000","last_became_active":"2024-09-10T09:59:04.244218+0000","last_became_peered":"2024-09-10T09:59:04.244218+0000","last_unstale":"2024-09-10T09:59:05.238690+0000","last_undegraded":"2024-09-10T09:59:05.238690+0000","last_fullsized":"2024-09-10T09:59:05.238690+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:02.539176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283138+0000","last_change":"2024-09-10T09:59:04.242293+0000","last_active":"2024-09-10T09:59:05.283138+0000","last_peered":"2024-09-10T09:59:05.283138+0000","last_clean":"2024-09-10T09:59:05.283138+0000","last_became_active":"2024-09-10T09:59:04.238754+0000","last_became_peered":"2024-09-10T09:59:04.238754+0000","last_unstale":"2024-09-10T09:59:05.283138+0000","last_undegraded":"2024-09-10T09:59:05.283138+0000","last_fullsized":"2024-09-10T09:59:05.283138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:12.332951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234909+0000","last_change":"2024-09-10T09:59:04.237127+0000","last_active":"2024-09-10T09:59:05.234909+0000","last_peered":"2024-09-10T09:59:05.234909+0000","last_clean":"2024-09-10T09:59:05.234909+0000","last_became_active":"2024-09-10T09:59:04.236856+0000","last_became_peered":"2024-09-10T09:59:04.236856+0000","last_unstale":"2024-09-10T09:59:05.234909+0000","last_undegraded":"2024-09-10T09:59:05.234909+0000","last_fullsized":"2024-09-10T09:59:05.234909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.785736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239151+0000","last_change":"2024-09-10T09:59:03.217173+0000","last_active":"2024-09-10T09:59:05.239151+0000","last_peered":"2024-09-10T09:59:05.239151+0000","last_clean":"2024-09-10T09:59:05.239151+0000","last_became_active":"2024-09-10T09:59:03.216896+0000","last_became_peered":"2024-09-10T09:59:03.216896+0000","last_unstale":"2024-09-10T09:59:05.239151+0000","last_undegraded":"2024-09-10T09:59:05.239151+0000","last_fullsized":"2024-09-10T09:59:05.239151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:46.302427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239090+0000","last_change":"2024-09-10T09:59:04.232436+0000","last_active":"2024-09-10T09:59:05.239090+0000","last_peered":"2024-09-10T09:59:05.239090+0000","last_clean":"2024-09-10T09:59:05.239090+0000","last_became_active":"2024-09-10T09:59:04.229805+0000","last_became_peered":"2024-09-10T09:59:04.229805+0000","last_unstale":"2024-09-10T09:59:05.239090+0000","last_undegraded":"2024-09-10T09:59:05.239090+0000","last_fullsized":"2024-09-10T09:59:05.239090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:30.880271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234937+0000","last_change":"2024-09-10T09:59:03.204423+0000","last_active":"2024-09-10T09:59:05.234937+0000","last_peered":"2024-09-10T09:59:05.234937+0000","last_clean":"2024-09-10T09:59:05.234937+0000","last_became_active":"2024-09-10T09:59:03.204117+0000","last_became_peered":"2024-09-10T09:59:03.204117+0000","last_unstale":"2024-09-10T09:59:05.234937+0000","last_undegraded":"2024-09-10T09:59:05.234937+0000","last_fullsized":"2024-09-10T09:59:05.234937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:42.707288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239394+0000","last_change":"2024-09-10T09:59:04.235085+0000","last_active":"2024-09-10T09:59:05.239394+0000","last_peered":"2024-09-10T09:59:05.239394+0000","last_clean":"2024-09-10T09:59:05.239394+0000","last_became_active":"2024-09-10T09:59:04.234827+0000","last_became_peered":"2024-09-10T09:59:04.234827+0000","last_unstale":"2024-09-10T09:59:05.239394+0000","last_undegraded":"2024-09-10T09:59:05.239394+0000","last_fullsized":"2024-09-10T09:59:05.239394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:59.181205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282590+0000","last_change":"2024-09-10T09:59:03.218809+0000","last_active":"2024-09-10T09:59:05.282590+0000","last_peered":"2024-09-10T09:59:05.282590+0000","last_clean":"2024-09-10T09:59:05.282590+0000","last_became_active":"2024-09-10T09:59:03.218587+0000","last_became_peered":"2024-09-10T09:59:03.218587+0000","last_unstale":"2024-09-10T09:59:05.282590+0000","last_undegraded":"2024-09-10T09:59:05.282590+0000","last_fullsized":"2024-09-10T09:59:05.282590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:33.139925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283206+0000","last_change":"2024-09-10T09:59:04.236025+0000","last_active":"2024-09-10T09:59:05.283206+0000","last_peered":"2024-09-10T09:59:05.283206+0000","last_clean":"2024-09-10T09:59:05.283206+0000","last_became_active":"2024-09-10T09:59:04.235729+0000","last_became_peered":"2024-09-10T09:59:04.235729+0000","last_unstale":"2024-09-10T09:59:05.283206+0000","last_undegraded":"2024-09-10T09:59:05.283206+0000","last_fullsized":"2024-09-10T09:59:05.283206+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:49.686495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282555+0000","last_change":"2024-09-10T09:59:03.219613+0000","last_active":"2024-09-10T09:59:05.282555+0000","last_peered":"2024-09-10T09:59:05.282555+0000","last_clean":"2024-09-10T09:59:05.282555+0000","last_became_active":"2024-09-10T09:59:03.219323+0000","last_became_peered":"2024-09-10T09:59:03.219323+0000","last_unstale":"2024-09-10T09:59:05.282555+0000","last_undegraded":"2024-09-10T09:59:05.282555+0000","last_fullsized":"2024-09-10T09:59:05.282555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:16.548391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282810+0000","last_change":"2024-09-10T09:59:04.238511+0000","last_active":"2024-09-10T09:59:05.282810+0000","last_peered":"2024-09-10T09:59:05.282810+0000","last_clean":"2024-09-10T09:59:05.282810+0000","last_became_active":"2024-09-10T09:59:04.238179+0000","last_became_peered":"2024-09-10T09:59:04.238179+0000","last_unstale":"2024-09-10T09:59:05.282810+0000","last_undegraded":"2024-09-10T09:59:05.282810+0000","last_fullsized":"2024-09-10T09:59:05.282810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:12.920550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239229+0000","last_change":"2024-09-10T09:59:03.214550+0000","last_active":"2024-09-10T09:59:05.239229+0000","last_peered":"2024-09-10T09:59:05.239229+0000","last_clean":"2024-09-10T09:59:05.239229+0000","last_became_active":"2024-09-10T09:59:03.213507+0000","last_became_peered":"2024-09-10T09:59:03.213507+0000","last_unstale":"2024-09-10T09:59:05.239229+0000","last_undegraded":"2024-09-10T09:59:05.239229+0000","last_fullsized":"2024-09-10T09:59:05.239229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:21.489357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282249+0000","last_change":"2024-09-10T09:59:04.234812+0000","last_active":"2024-09-10T09:59:05.282249+0000","last_peered":"2024-09-10T09:59:05.282249+0000","last_clean":"2024-09-10T09:59:05.282249+0000","last_became_active":"2024-09-10T09:59:04.234515+0000","last_became_peered":"2024-09-10T09:59:04.234515+0000","last_unstale":"2024-09-10T09:59:05.282249+0000","last_undegraded":"2024-09-10T09:59:05.282249+0000","last_fullsized":"2024-09-10T09:59:05.282249+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:54.921113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238659+0000","last_change":"2024-09-10T09:59:03.210463+0000","last_active":"2024-09-10T09:59:05.238659+0000","last_peered":"2024-09-10T09:59:05.238659+0000","last_clean":"2024-09-10T09:59:05.238659+0000","last_became_active":"2024-09-10T09:59:03.210157+0000","last_became_peered":"2024-09-10T09:59:03.210157+0000","last_unstale":"2024-09-10T09:59:05.238659+0000","last_undegraded":"2024-09-10T09:59:05.238659+0000","last_fullsized":"2024-09-10T09:59:05.238659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:13.022417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238459+0000","last_change":"2024-09-10T09:59:04.223832+0000","last_active":"2024-09-10T09:59:05.238459+0000","last_peered":"2024-09-10T09:59:05.238459+0000","last_clean":"2024-09-10T09:59:05.238459+0000","last_became_active":"2024-09-10T09:59:04.223512+0000","last_became_peered":"2024-09-10T09:59:04.223512+0000","last_unstale":"2024-09-10T09:59:05.238459+0000","last_undegraded":"2024-09-10T09:59:05.238459+0000","last_fullsized":"2024-09-10T09:59:05.238459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:44.726674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281756+0000","last_change":"2024-09-10T09:59:03.210986+0000","last_active":"2024-09-10T09:59:05.281756+0000","last_peered":"2024-09-10T09:59:05.281756+0000","last_clean":"2024-09-10T09:59:05.281756+0000","last_became_active":"2024-09-10T09:59:03.210736+0000","last_became_peered":"2024-09-10T09:59:03.210736+0000","last_unstale":"2024-09-10T09:59:05.281756+0000","last_undegraded":"2024-09-10T09:59:05.281756+0000","last_fullsized":"2024-09-10T09:59:05.281756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:01.201854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281572+0000","last_change":"2024-09-10T09:59:04.233989+0000","last_active":"2024-09-10T09:59:05.281572+0000","last_peered":"2024-09-10T09:59:05.281572+0000","last_clean":"2024-09-10T09:59:05.281572+0000","last_became_active":"2024-09-10T09:59:04.233630+0000","last_became_peered":"2024-09-10T09:59:04.233630+0000","last_unstale":"2024-09-10T09:59:05.281572+0000","last_undegraded":"2024-09-10T09:59:05.281572+0000","last_fullsized":"2024-09-10T09:59:05.281572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.785736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238515+0000","last_change":"2024-09-10T09:59:03.215583+0000","last_active":"2024-09-10T09:59:05.238515+0000","last_peered":"2024-09-10T09:59:05.238515+0000","last_clean":"2024-09-10T09:59:05.238515+0000","last_became_active":"2024-09-10T09:59:03.215296+0000","last_became_peered":"2024-09-10T09:59:03.215296+0000","last_unstale":"2024-09-10T09:59:05.238515+0000","last_undegraded":"2024-09-10T09:59:05.238515+0000","last_fullsized":"2024-09-10T09:59:05.238515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:57.818219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282564+0000","last_change":"2024-09-10T09:59:04.247805+0000","last_active":"2024-09-10T09:59:05.282564+0000","last_peered":"2024-09-10T09:59:05.282564+0000","last_clean":"2024-09-10T09:59:05.282564+0000","last_became_active":"2024-09-10T09:59:04.247479+0000","last_became_peered":"2024-09-10T09:59:04.247479+0000","last_unstale":"2024-09-10T09:59:05.282564+0000","last_undegraded":"2024-09-10T09:59:05.282564+0000","last_fullsized":"2024-09-10T09:59:05.282564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:06.851574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283365+0000","last_change":"2024-09-10T09:59:03.208433+0000","last_active":"2024-09-10T09:59:05.283365+0000","last_peered":"2024-09-10T09:59:05.283365+0000","last_clean":"2024-09-10T09:59:05.283365+0000","last_became_active":"2024-09-10T09:59:03.207106+0000","last_became_peered":"2024-09-10T09:59:03.207106+0000","last_unstale":"2024-09-10T09:59:05.283365+0000","last_undegraded":"2024-09-10T09:59:05.283365+0000","last_fullsized":"2024-09-10T09:59:05.283365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:13.286437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283272+0000","last_change":"2024-09-10T09:59:04.257770+0000","last_active":"2024-09-10T09:59:05.283272+0000","last_peered":"2024-09-10T09:59:05.283272+0000","last_clean":"2024-09-10T09:59:05.283272+0000","last_became_active":"2024-09-10T09:59:04.257535+0000","last_became_peered":"2024-09-10T09:59:04.257535+0000","last_unstale":"2024-09-10T09:59:05.283272+0000","last_undegraded":"2024-09-10T09:59:05.283272+0000","last_fullsized":"2024-09-10T09:59:05.283272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:27.392572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234963+0000","last_change":"2024-09-10T09:59:03.204371+0000","last_active":"2024-09-10T09:59:05.234963+0000","last_peered":"2024-09-10T09:59:05.234963+0000","last_clean":"2024-09-10T09:59:05.234963+0000","last_became_active":"2024-09-10T09:59:03.204087+0000","last_became_peered":"2024-09-10T09:59:03.204087+0000","last_unstale":"2024-09-10T09:59:05.234963+0000","last_undegraded":"2024-09-10T09:59:05.234963+0000","last_fullsized":"2024-09-10T09:59:05.234963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:49.036675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281596+0000","last_change":"2024-09-10T09:59:04.234326+0000","last_active":"2024-09-10T09:59:05.281596+0000","last_peered":"2024-09-10T09:59:05.281596+0000","last_clean":"2024-09-10T09:59:05.281596+0000","last_became_active":"2024-09-10T09:59:04.234025+0000","last_became_peered":"2024-09-10T09:59:04.234025+0000","last_unstale":"2024-09-10T09:59:05.281596+0000","last_undegraded":"2024-09-10T09:59:05.281596+0000","last_fullsized":"2024-09-10T09:59:05.281596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:09.398256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282688+0000","last_change":"2024-09-10T09:59:03.206547+0000","last_active":"2024-09-10T09:59:05.282688+0000","last_peered":"2024-09-10T09:59:05.282688+0000","last_clean":"2024-09-10T09:59:05.282688+0000","last_became_active":"2024-09-10T09:59:03.206237+0000","last_became_peered":"2024-09-10T09:59:03.206237+0000","last_unstale":"2024-09-10T09:59:05.282688+0000","last_undegraded":"2024-09-10T09:59:05.282688+0000","last_fullsized":"2024-09-10T09:59:05.282688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:01.533226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282632+0000","last_change":"2024-09-10T09:59:04.239390+0000","last_active":"2024-09-10T09:59:05.282632+0000","last_peered":"2024-09-10T09:59:05.282632+0000","last_clean":"2024-09-10T09:59:05.282632+0000","last_became_active":"2024-09-10T09:59:04.238904+0000","last_became_peered":"2024-09-10T09:59:04.238904+0000","last_unstale":"2024-09-10T09:59:05.282632+0000","last_undegraded":"2024-09-10T09:59:05.282632+0000","last_fullsized":"2024-09-10T09:59:05.282632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:12.920550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281699+0000","last_change":"2024-09-10T09:59:03.219831+0000","last_active":"2024-09-10T09:59:05.281699+0000","last_peered":"2024-09-10T09:59:05.281699+0000","last_clean":"2024-09-10T09:59:05.281699+0000","last_became_active":"2024-09-10T09:59:03.219272+0000","last_became_peered":"2024-09-10T09:59:03.219272+0000","last_unstale":"2024-09-10T09:59:05.281699+0000","last_undegraded":"2024-09-10T09:59:05.281699+0000","last_fullsized":"2024-09-10T09:59:05.281699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:03.029812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281706+0000","last_change":"2024-09-10T09:59:04.241369+0000","last_active":"2024-09-10T09:59:05.281706+0000","last_peered":"2024-09-10T09:59:05.281706+0000","last_clean":"2024-09-10T09:59:05.281706+0000","last_became_active":"2024-09-10T09:59:04.241100+0000","last_became_peered":"2024-09-10T09:59:04.241100+0000","last_unstale":"2024-09-10T09:59:05.281706+0000","last_undegraded":"2024-09-10T09:59:05.281706+0000","last_fullsized":"2024-09-10T09:59:05.281706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:36.495442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239393+0000","last_change":"2024-09-10T09:59:03.216285+0000","last_active":"2024-09-10T09:59:05.239393+0000","last_peered":"2024-09-10T09:59:05.239393+0000","last_clean":"2024-09-10T09:59:05.239393+0000","last_became_active":"2024-09-10T09:59:03.215878+0000","last_became_peered":"2024-09-10T09:59:03.215878+0000","last_unstale":"2024-09-10T09:59:05.239393+0000","last_undegraded":"2024-09-10T09:59:05.239393+0000","last_fullsized":"2024-09-10T09:59:05.239393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:43.559741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281688+0000","last_change":"2024-09-10T09:59:04.233774+0000","last_active":"2024-09-10T09:59:05.281688+0000","last_peered":"2024-09-10T09:59:05.281688+0000","last_clean":"2024-09-10T09:59:05.281688+0000","last_became_active":"2024-09-10T09:59:04.233387+0000","last_became_peered":"2024-09-10T09:59:04.233387+0000","last_unstale":"2024-09-10T09:59:05.281688+0000","last_undegraded":"2024-09-10T09:59:05.281688+0000","last_fullsized":"2024-09-10T09:59:05.281688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:49.890997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281796+0000","last_change":"2024-09-10T09:59:03.217487+0000","last_active":"2024-09-10T09:59:05.281796+0000","last_peered":"2024-09-10T09:59:05.281796+0000","last_clean":"2024-09-10T09:59:05.281796+0000","last_became_active":"2024-09-10T09:59:03.217236+0000","last_became_peered":"2024-09-10T09:59:03.217236+0000","last_unstale":"2024-09-10T09:59:05.281796+0000","last_undegraded":"2024-09-10T09:59:05.281796+0000","last_fullsized":"2024-09-10T09:59:05.281796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:42.707288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282716+0000","last_change":"2024-09-10T09:59:04.238031+0000","last_active":"2024-09-10T09:59:05.282716+0000","last_peered":"2024-09-10T09:59:05.282716+0000","last_clean":"2024-09-10T09:59:05.282716+0000","last_became_active":"2024-09-10T09:59:04.237680+0000","last_became_peered":"2024-09-10T09:59:04.237680+0000","last_unstale":"2024-09-10T09:59:05.282716+0000","last_undegraded":"2024-09-10T09:59:05.282716+0000","last_fullsized":"2024-09-10T09:59:05.282716+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:50.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281843+0000","last_change":"2024-09-10T09:59:03.208322+0000","last_active":"2024-09-10T09:59:05.281843+0000","last_peered":"2024-09-10T09:59:05.281843+0000","last_clean":"2024-09-10T09:59:05.281843+0000","last_became_active":"2024-09-10T09:59:03.207255+0000","last_became_peered":"2024-09-10T09:59:03.207255+0000","last_unstale":"2024-09-10T09:59:05.281843+0000","last_undegraded":"2024-09-10T09:59:05.281843+0000","last_fullsized":"2024-09-10T09:59:05.281843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:37.076444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238532+0000","last_change":"2024-09-10T09:59:04.231783+0000","last_active":"2024-09-10T09:59:05.238532+0000","last_peered":"2024-09-10T09:59:05.238532+0000","last_clean":"2024-09-10T09:59:05.238532+0000","last_became_active":"2024-09-10T09:59:04.231510+0000","last_became_peered":"2024-09-10T09:59:04.231510+0000","last_unstale":"2024-09-10T09:59:05.238532+0000","last_undegraded":"2024-09-10T09:59:05.238532+0000","last_fullsized":"2024-09-10T09:59:05.238532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:22.377203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281289+0000","last_change":"2024-09-10T09:59:03.219531+0000","last_active":"2024-09-10T09:59:05.281289+0000","last_peered":"2024-09-10T09:59:05.281289+0000","last_clean":"2024-09-10T09:59:05.281289+0000","last_became_active":"2024-09-10T09:59:03.219216+0000","last_became_peered":"2024-09-10T09:59:03.219216+0000","last_unstale":"2024-09-10T09:59:05.281289+0000","last_undegraded":"2024-09-10T09:59:05.281289+0000","last_fullsized":"2024-09-10T09:59:05.281289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:03.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238874+0000","last_change":"2024-09-10T09:59:04.232545+0000","last_active":"2024-09-10T09:59:05.238874+0000","last_peered":"2024-09-10T09:59:05.238874+0000","last_clean":"2024-09-10T09:59:05.238874+0000","last_became_active":"2024-09-10T09:59:04.232160+0000","last_became_peered":"2024-09-10T09:59:04.232160+0000","last_unstale":"2024-09-10T09:59:05.238874+0000","last_undegraded":"2024-09-10T09:59:05.238874+0000","last_fullsized":"2024-09-10T09:59:05.238874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:26.363067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283359+0000","last_change":"2024-09-10T09:59:03.216864+0000","last_active":"2024-09-10T09:59:05.283359+0000","last_peered":"2024-09-10T09:59:05.283359+0000","last_clean":"2024-09-10T09:59:05.283359+0000","last_became_active":"2024-09-10T09:59:03.216585+0000","last_became_peered":"2024-09-10T09:59:03.216585+0000","last_unstale":"2024-09-10T09:59:05.283359+0000","last_undegraded":"2024-09-10T09:59:05.283359+0000","last_fullsized":"2024-09-10T09:59:05.283359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.310845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282843+0000","last_change":"2024-09-10T09:59:04.232330+0000","last_active":"2024-09-10T09:59:05.282843+0000","last_peered":"2024-09-10T09:59:05.282843+0000","last_clean":"2024-09-10T09:59:05.282843+0000","last_became_active":"2024-09-10T09:59:04.232058+0000","last_became_peered":"2024-09-10T09:59:04.232058+0000","last_unstale":"2024-09-10T09:59:05.282843+0000","last_undegraded":"2024-09-10T09:59:05.282843+0000","last_fullsized":"2024-09-10T09:59:05.282843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:25.498252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238928+0000","last_change":"2024-09-10T09:59:03.216300+0000","last_active":"2024-09-10T09:59:05.238928+0000","last_peered":"2024-09-10T09:59:05.238928+0000","last_clean":"2024-09-10T09:59:05.238928+0000","last_became_active":"2024-09-10T09:59:03.215913+0000","last_became_peered":"2024-09-10T09:59:03.215913+0000","last_unstale":"2024-09-10T09:59:05.238928+0000","last_undegraded":"2024-09-10T09:59:05.238928+0000","last_fullsized":"2024-09-10T09:59:05.238928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:57.948064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282949+0000","last_change":"2024-09-10T09:59:04.245576+0000","last_active":"2024-09-10T09:59:05.282949+0000","last_peered":"2024-09-10T09:59:05.282949+0000","last_clean":"2024-09-10T09:59:05.282949+0000","last_became_active":"2024-09-10T09:59:04.245256+0000","last_became_peered":"2024-09-10T09:59:04.245256+0000","last_unstale":"2024-09-10T09:59:05.282949+0000","last_undegraded":"2024-09-10T09:59:05.282949+0000","last_fullsized":"2024-09-10T09:59:05.282949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:39.465659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239689+0000","last_change":"2024-09-10T09:59:03.219779+0000","last_active":"2024-09-10T09:59:05.239689+0000","last_peered":"2024-09-10T09:59:05.239689+0000","last_clean":"2024-09-10T09:59:05.239689+0000","last_became_active":"2024-09-10T09:59:03.219222+0000","last_became_peered":"2024-09-10T09:59:03.219222+0000","last_unstale":"2024-09-10T09:59:05.239689+0000","last_undegraded":"2024-09-10T09:59:05.239689+0000","last_fullsized":"2024-09-10T09:59:05.239689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:37.117350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282324+0000","last_change":"2024-09-10T09:59:04.248514+0000","last_active":"2024-09-10T09:59:05.282324+0000","last_peered":"2024-09-10T09:59:05.282324+0000","last_clean":"2024-09-10T09:59:05.282324+0000","last_became_active":"2024-09-10T09:59:04.236015+0000","last_became_peered":"2024-09-10T09:59:04.236015+0000","last_unstale":"2024-09-10T09:59:05.282324+0000","last_undegraded":"2024-09-10T09:59:05.282324+0000","last_fullsized":"2024-09-10T09:59:05.282324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:22.316795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239391+0000","last_change":"2024-09-10T09:59:03.213361+0000","last_active":"2024-09-10T09:59:05.239391+0000","last_peered":"2024-09-10T09:59:05.239391+0000","last_clean":"2024-09-10T09:59:05.239391+0000","last_became_active":"2024-09-10T09:59:03.213126+0000","last_became_peered":"2024-09-10T09:59:03.213126+0000","last_unstale":"2024-09-10T09:59:05.239391+0000","last_undegraded":"2024-09-10T09:59:05.239391+0000","last_fullsized":"2024-09-10T09:59:05.239391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:11.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282158+0000","last_change":"2024-09-10T09:59:04.244079+0000","last_active":"2024-09-10T09:59:05.282158+0000","last_peered":"2024-09-10T09:59:05.282158+0000","last_clean":"2024-09-10T09:59:05.282158+0000","last_became_active":"2024-09-10T09:59:04.243851+0000","last_became_peered":"2024-09-10T09:59:04.243851+0000","last_unstale":"2024-09-10T09:59:05.282158+0000","last_undegraded":"2024-09-10T09:59:05.282158+0000","last_fullsized":"2024-09-10T09:59:05.282158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:12.897693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234727+0000","last_change":"2024-09-10T09:59:03.209652+0000","last_active":"2024-09-10T09:59:05.234727+0000","last_peered":"2024-09-10T09:59:05.234727+0000","last_clean":"2024-09-10T09:59:05.234727+0000","last_became_active":"2024-09-10T09:59:03.209369+0000","last_became_peered":"2024-09-10T09:59:03.209369+0000","last_unstale":"2024-09-10T09:59:05.234727+0000","last_undegraded":"2024-09-10T09:59:05.234727+0000","last_fullsized":"2024-09-10T09:59:05.234727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.310845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282269+0000","last_change":"2024-09-10T09:59:04.239608+0000","last_active":"2024-09-10T09:59:05.282269+0000","last_peered":"2024-09-10T09:59:05.282269+0000","last_clean":"2024-09-10T09:59:05.282269+0000","last_became_active":"2024-09-10T09:59:04.239262+0000","last_became_peered":"2024-09-10T09:59:04.239262+0000","last_unstale":"2024-09-10T09:59:05.282269+0000","last_undegraded":"2024-09-10T09:59:05.282269+0000","last_fullsized":"2024-09-10T09:59:05.282269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:47.308377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238528+0000","last_change":"2024-09-10T09:59:03.219990+0000","last_active":"2024-09-10T09:59:05.238528+0000","last_peered":"2024-09-10T09:59:05.238528+0000","last_clean":"2024-09-10T09:59:05.238528+0000","last_became_active":"2024-09-10T09:59:03.219678+0000","last_became_peered":"2024-09-10T09:59:03.219678+0000","last_unstale":"2024-09-10T09:59:05.238528+0000","last_undegraded":"2024-09-10T09:59:05.238528+0000","last_fullsized":"2024-09-10T09:59:05.238528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:43.559741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282017+0000","last_change":"2024-09-10T09:59:04.239523+0000","last_active":"2024-09-10T09:59:05.282017+0000","last_peered":"2024-09-10T09:59:05.282017+0000","last_clean":"2024-09-10T09:59:05.282017+0000","last_became_active":"2024-09-10T09:59:04.239246+0000","last_became_peered":"2024-09-10T09:59:04.239246+0000","last_unstale":"2024-09-10T09:59:05.282017+0000","last_undegraded":"2024-09-10T09:59:05.282017+0000","last_fullsized":"2024-09-10T09:59:05.282017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:38.123300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239462+0000","last_change":"2024-09-10T09:59:03.210723+0000","last_active":"2024-09-10T09:59:05.239462+0000","last_peered":"2024-09-10T09:59:05.239462+0000","last_clean":"2024-09-10T09:59:05.239462+0000","last_became_active":"2024-09-10T09:59:03.210441+0000","last_became_peered":"2024-09-10T09:59:03.210441+0000","last_unstale":"2024-09-10T09:59:05.239462+0000","last_undegraded":"2024-09-10T09:59:05.239462+0000","last_fullsized":"2024-09-10T09:59:05.239462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.310845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282398+0000","last_change":"2024-09-10T09:59:04.237462+0000","last_active":"2024-09-10T09:59:05.282398+0000","last_peered":"2024-09-10T09:59:05.282398+0000","last_clean":"2024-09-10T09:59:05.282398+0000","last_became_active":"2024-09-10T09:59:04.235950+0000","last_became_peered":"2024-09-10T09:59:04.235950+0000","last_unstale":"2024-09-10T09:59:05.282398+0000","last_undegraded":"2024-09-10T09:59:05.282398+0000","last_fullsized":"2024-09-10T09:59:05.282398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:54.921113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281874+0000","last_change":"2024-09-10T09:59:03.206594+0000","last_active":"2024-09-10T09:59:05.281874+0000","last_peered":"2024-09-10T09:59:05.281874+0000","last_clean":"2024-09-10T09:59:05.281874+0000","last_became_active":"2024-09-10T09:59:03.206278+0000","last_became_peered":"2024-09-10T09:59:03.206278+0000","last_unstale":"2024-09-10T09:59:05.281874+0000","last_undegraded":"2024-09-10T09:59:05.281874+0000","last_fullsized":"2024-09-10T09:59:05.281874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:21.371253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282486+0000","last_change":"2024-09-10T09:59:04.228620+0000","last_active":"2024-09-10T09:59:05.282486+0000","last_peered":"2024-09-10T09:59:05.282486+0000","last_clean":"2024-09-10T09:59:05.282486+0000","last_became_active":"2024-09-10T09:59:04.228327+0000","last_became_peered":"2024-09-10T09:59:04.228327+0000","last_unstale":"2024-09-10T09:59:05.282486+0000","last_undegraded":"2024-09-10T09:59:05.282486+0000","last_fullsized":"2024-09-10T09:59:05.282486+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:18.633489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283407+0000","last_change":"2024-09-10T09:59:03.218569+0000","last_active":"2024-09-10T09:59:05.283407+0000","last_peered":"2024-09-10T09:59:05.283407+0000","last_clean":"2024-09-10T09:59:05.283407+0000","last_became_active":"2024-09-10T09:59:03.218234+0000","last_became_peered":"2024-09-10T09:59:03.218234+0000","last_unstale":"2024-09-10T09:59:05.283407+0000","last_undegraded":"2024-09-10T09:59:05.283407+0000","last_fullsized":"2024-09-10T09:59:05.283407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:17.627540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282978+0000","last_change":"2024-09-10T09:59:04.249788+0000","last_active":"2024-09-10T09:59:05.282978+0000","last_peered":"2024-09-10T09:59:05.282978+0000","last_clean":"2024-09-10T09:59:05.282978+0000","last_became_active":"2024-09-10T09:59:04.249254+0000","last_became_peered":"2024-09-10T09:59:04.249254+0000","last_unstale":"2024-09-10T09:59:05.282978+0000","last_undegraded":"2024-09-10T09:59:05.282978+0000","last_fullsized":"2024-09-10T09:59:05.282978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:54.301065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.235142+0000","last_change":"2024-09-10T09:59:03.209630+0000","last_active":"2024-09-10T09:59:05.235142+0000","last_peered":"2024-09-10T09:59:05.235142+0000","last_clean":"2024-09-10T09:59:05.235142+0000","last_became_active":"2024-09-10T09:59:03.209346+0000","last_became_peered":"2024-09-10T09:59:03.209346+0000","last_unstale":"2024-09-10T09:59:05.235142+0000","last_undegraded":"2024-09-10T09:59:05.235142+0000","last_fullsized":"2024-09-10T09:59:05.235142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:44.038105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234792+0000","last_change":"2024-09-10T09:59:04.240645+0000","last_active":"2024-09-10T09:59:05.234792+0000","last_peered":"2024-09-10T09:59:05.234792+0000","last_clean":"2024-09-10T09:59:05.234792+0000","last_became_active":"2024-09-10T09:59:04.239034+0000","last_became_peered":"2024-09-10T09:59:04.239034+0000","last_unstale":"2024-09-10T09:59:05.234792+0000","last_undegraded":"2024-09-10T09:59:05.234792+0000","last_fullsized":"2024-09-10T09:59:05.234792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:26.363067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282974+0000","last_change":"2024-09-10T09:59:03.215506+0000","last_active":"2024-09-10T09:59:05.282974+0000","last_peered":"2024-09-10T09:59:05.282974+0000","last_clean":"2024-09-10T09:59:05.282974+0000","last_became_active":"2024-09-10T09:59:03.214501+0000","last_became_peered":"2024-09-10T09:59:03.214501+0000","last_unstale":"2024-09-10T09:59:05.282974+0000","last_undegraded":"2024-09-10T09:59:05.282974+0000","last_fullsized":"2024-09-10T09:59:05.282974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:46.302427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238455+0000","last_change":"2024-09-10T09:59:04.229435+0000","last_active":"2024-09-10T09:59:05.238455+0000","last_peered":"2024-09-10T09:59:05.238455+0000","last_clean":"2024-09-10T09:59:05.238455+0000","last_became_active":"2024-09-10T09:59:04.229158+0000","last_became_peered":"2024-09-10T09:59:04.229158+0000","last_unstale":"2024-09-10T09:59:05.238455+0000","last_undegraded":"2024-09-10T09:59:05.238455+0000","last_fullsized":"2024-09-10T09:59:05.238455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:48.367623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234803+0000","last_change":"2024-09-10T09:59:03.207503+0000","last_active":"2024-09-10T09:59:05.234803+0000","last_peered":"2024-09-10T09:59:05.234803+0000","last_clean":"2024-09-10T09:59:05.234803+0000","last_became_active":"2024-09-10T09:59:03.206392+0000","last_became_peered":"2024-09-10T09:59:03.206392+0000","last_unstale":"2024-09-10T09:59:05.234803+0000","last_undegraded":"2024-09-10T09:59:05.234803+0000","last_fullsized":"2024-09-10T09:59:05.234803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:17.627540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234721+0000","last_change":"2024-09-10T09:59:04.231822+0000","last_active":"2024-09-10T09:59:05.234721+0000","last_peered":"2024-09-10T09:59:05.234721+0000","last_clean":"2024-09-10T09:59:05.234721+0000","last_became_active":"2024-09-10T09:59:04.231491+0000","last_became_peered":"2024-09-10T09:59:04.231491+0000","last_unstale":"2024-09-10T09:59:05.234721+0000","last_undegraded":"2024-09-10T09:59:05.234721+0000","last_fullsized":"2024-09-10T09:59:05.234721+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:30.880271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282146+0000","last_change":"2024-09-10T09:59:03.218500+0000","last_active":"2024-09-10T09:59:05.282146+0000","last_peered":"2024-09-10T09:59:05.282146+0000","last_clean":"2024-09-10T09:59:05.282146+0000","last_became_active":"2024-09-10T09:59:03.218191+0000","last_became_peered":"2024-09-10T09:59:03.218191+0000","last_unstale":"2024-09-10T09:59:05.282146+0000","last_undegraded":"2024-09-10T09:59:05.282146+0000","last_fullsized":"2024-09-10T09:59:05.282146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:49.036675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281192+0000","last_change":"2024-09-10T09:59:04.231281+0000","last_active":"2024-09-10T09:59:05.281192+0000","last_peered":"2024-09-10T09:59:05.281192+0000","last_clean":"2024-09-10T09:59:05.281192+0000","last_became_active":"2024-09-10T09:59:04.231070+0000","last_became_peered":"2024-09-10T09:59:04.231070+0000","last_unstale":"2024-09-10T09:59:05.281192+0000","last_undegraded":"2024-09-10T09:59:05.281192+0000","last_fullsized":"2024-09-10T09:59:05.281192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:34.062362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238657+0000","last_change":"2024-09-10T09:59:03.207218+0000","last_active":"2024-09-10T09:59:05.238657+0000","last_peered":"2024-09-10T09:59:05.238657+0000","last_clean":"2024-09-10T09:59:05.238657+0000","last_became_active":"2024-09-10T09:59:03.207032+0000","last_became_peered":"2024-09-10T09:59:03.207032+0000","last_unstale":"2024-09-10T09:59:05.238657+0000","last_undegraded":"2024-09-10T09:59:05.238657+0000","last_fullsized":"2024-09-10T09:59:05.238657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:46.302427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238564+0000","last_change":"2024-09-10T09:59:04.233898+0000","last_active":"2024-09-10T09:59:05.238564+0000","last_peered":"2024-09-10T09:59:05.238564+0000","last_clean":"2024-09-10T09:59:05.238564+0000","last_became_active":"2024-09-10T09:59:04.233583+0000","last_became_peered":"2024-09-10T09:59:04.233583+0000","last_unstale":"2024-09-10T09:59:05.238564+0000","last_undegraded":"2024-09-10T09:59:05.238564+0000","last_fullsized":"2024-09-10T09:59:05.238564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:30.880271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281381+0000","last_change":"2024-09-10T09:59:03.208224+0000","last_active":"2024-09-10T09:59:05.281381+0000","last_peered":"2024-09-10T09:59:05.281381+0000","last_clean":"2024-09-10T09:59:05.281381+0000","last_became_active":"2024-09-10T09:59:03.207076+0000","last_became_peered":"2024-09-10T09:59:03.207076+0000","last_unstale":"2024-09-10T09:59:05.281381+0000","last_undegraded":"2024-09-10T09:59:05.281381+0000","last_fullsized":"2024-09-10T09:59:05.281381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:59.885497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283110+0000","last_change":"2024-09-10T09:59:04.251728+0000","last_active":"2024-09-10T09:59:05.283110+0000","last_peered":"2024-09-10T09:59:05.283110+0000","last_clean":"2024-09-10T09:59:05.283110+0000","last_became_active":"2024-09-10T09:59:04.251481+0000","last_became_peered":"2024-09-10T09:59:04.251481+0000","last_unstale":"2024-09-10T09:59:05.283110+0000","last_undegraded":"2024-09-10T09:59:05.283110+0000","last_fullsized":"2024-09-10T09:59:05.283110+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:54.301065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238901+0000","last_change":"2024-09-10T09:59:03.219002+0000","last_active":"2024-09-10T09:59:05.238901+0000","last_peered":"2024-09-10T09:59:05.238901+0000","last_clean":"2024-09-10T09:59:05.238901+0000","last_became_active":"2024-09-10T09:59:03.218552+0000","last_became_peered":"2024-09-10T09:59:03.218552+0000","last_unstale":"2024-09-10T09:59:05.238901+0000","last_undegraded":"2024-09-10T09:59:05.238901+0000","last_fullsized":"2024-09-10T09:59:05.238901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:11.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":190,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282945+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:59:05.282945+0000","last_peered":"2024-09-10T09:59:05.282945+0000","last_clean":"2024-09-10T09:59:05.282945+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:59:05.282945+0000","last_undegraded":"2024-09-10T09:59:05.282945+0000","last_fullsized":"2024-09-10T09:59:05.282945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":734605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":144763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509453,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":144819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":144251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":380928,"data_stored":144217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":970752,"data_stored":734515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:59:12.840 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:59:12.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.979+0000 7fb113000640 1 Processor -- start 2024-09-10T09:59:12.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.979+0000 7fb113000640 1 -- start start 2024-09-10T09:59:12.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb113000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb10c0cfe30 0x7fb10c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb113000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c0cd510 0x7fb10c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb113000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb10c06e090 0x7fb10c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb113000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb10c0cdeb0 con 0x7fb10c06e090 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb113000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb10c15d9a0 con 0x7fb10c0cfe30 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb113000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb10c06ea60 con 0x7fb10c0cd510 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb111000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb10c06e090 0x7fb10c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb111000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb10c06e090 0x7fb10c06e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46258/0 (socket says 172.21.15.110:46258) 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb111000640 1 -- 172.21.15.110:0/4006102249 learned_addr learned my addr 172.21.15.110:0/4006102249 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb10be00640 1 --2- 172.21.15.110:0/4006102249 >> v2:172.21.15.145:3300/0 conn(0x7fb10c0cfe30 0x7fb10c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb10b400640 1 --2- 172.21.15.110:0/4006102249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c0cd510 0x7fb10c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb111000640 1 -- 172.21.15.110:0/4006102249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c0cd510 msgr2=0x7fb10c0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb111000640 1 --2- 172.21.15.110:0/4006102249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c0cd510 0x7fb10c0cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb111000640 1 -- 172.21.15.110:0/4006102249 >> v2:172.21.15.145:3300/0 conn(0x7fb10c0cfe30 msgr2=0x7fb10c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb111000640 1 --2- 172.21.15.110:0/4006102249 >> v2:172.21.15.145:3300/0 conn(0x7fb10c0cfe30 0x7fb10c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.983+0000 7fb111000640 1 -- 172.21.15.110:0/4006102249 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c06d2d0 con 0x7fb10c06e090 2024-09-10T09:59:12.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb111000640 1 --2- 172.21.15.110:0/4006102249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c06e090 0x7fb10c06e520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0ce120 tx=0x7fb10401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a31b85cc3be0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb10aa00640 1 -- 172.21.15.110:0/4006102249 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1040a56f0 con 0x7fb10c06e090 2024-09-10T09:59:12.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb10aa00640 1 -- 172.21.15.110:0/4006102249 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1040a5890 con 0x7fb10c06e090 2024-09-10T09:59:12.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb10aa00640 1 -- 172.21.15.110:0/4006102249 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1040a5c60 con 0x7fb10c06e090 2024-09-10T09:59:12.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 -- 172.21.15.110:0/4006102249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c06e090 msgr2=0x7fb10c06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 --2- 172.21.15.110:0/4006102249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c06e090 0x7fb10c06e520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0ce120 tx=0x7fb10401c960 comp rx=0 tx=0).stop 2024-09-10T09:59:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 -- 172.21.15.110:0/4006102249 shutdown_connections 2024-09-10T09:59:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 --2- 172.21.15.110:0/4006102249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c06e090 0x7fb10c06e520 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 --2- 172.21.15.110:0/4006102249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c0cd510 0x7fb10c0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 --2- 172.21.15.110:0/4006102249 >> v2:172.21.15.145:3300/0 conn(0x7fb10c0cfe30 0x7fb10c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 -- 172.21.15.110:0/4006102249 >> 172.21.15.110:0/4006102249 conn(0x7fb10c06bc50 msgr2=0x7fb10c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 -- 172.21.15.110:0/4006102249 shutdown_connections 2024-09-10T09:59:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 -- 172.21.15.110:0/4006102249 wait complete. 2024-09-10T09:59:12.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 Processor -- start 2024-09-10T09:59:12.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.987+0000 7fb113000640 1 -- start start 2024-09-10T09:59:12.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c06e090 0x7fb10c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c0cd510 0x7fb10c148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb10c0cfe30 0x7fb10c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c15d430 con 0x7fb10c0cd510 2024-09-10T09:59:12.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c072190 con 0x7fb10c0cfe30 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb10c15d5b0 con 0x7fb10c06e090 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c0cd510 0x7fb10c148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c0cd510 0x7fb10c148930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46274/0 (socket says 172.21.15.110:46274) 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 -- 172.21.15.110:0/2867054443 learned_addr learned my addr 172.21.15.110:0/2867054443 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10be00640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c06e090 0x7fb10c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb111000640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb10c0cfe30 0x7fb10c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 -- 172.21.15.110:0/2867054443 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c06e090 msgr2=0x7fb10c14ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c06e090 0x7fb10c14ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 -- 172.21.15.110:0/2867054443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb10c0cfe30 msgr2=0x7fb10c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb10c0cfe30 0x7fb10c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 -- 172.21.15.110:0/2867054443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c069bb0 con 0x7fb10c0cd510 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb10b400640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c0cd510 0x7fb10c148930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d830 tx=0x7fb0f800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb103e00640 1 -- 172.21.15.110:0/2867054443 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8018020 con 0x7fb10c0cd510 2024-09-10T09:59:12.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb103e00640 1 -- 172.21.15.110:0/2867054443 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f80096c0 con 0x7fb10c0cd510 2024-09-10T09:59:12.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb103e00640 1 -- 172.21.15.110:0/2867054443 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8009ab0 con 0x7fb10c0cd510 2024-09-10T09:59:12.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10c073150 con 0x7fb10c0cd510 2024-09-10T09:59:12.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10c14df60 con 0x7fb10c0cd510 2024-09-10T09:59:12.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10c06e520 con 0x7fb10c0cd510 2024-09-10T09:59:12.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.991+0000 7fb103e00640 1 -- 172.21.15.110:0/2867054443 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb0f801b020 con 0x7fb10c0cd510 2024-09-10T09:59:12.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.995+0000 7fb103e00640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb0ec0678c0 0x7fb0ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.995+0000 7fb103e00640 1 -- 172.21.15.110:0/2867054443 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb0f80863c0 con 0x7fb10c0cd510 2024-09-10T09:59:12.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.995+0000 7fb10be00640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb0ec0678c0 0x7fb0ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.995+0000 7fb10be00640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb0ec0678c0 0x7fb0ec069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb10c150a70 tx=0x7fb0fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:12.999+0000 7fb103e00640 1 -- 172.21.15.110:0/2867054443 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb0f8022020 con 0x7fb10c0cd510 2024-09-10T09:59:13.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.151+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb10c148e70 con 0x7fb0ec0678c0 2024-09-10T09:59:13.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb103e00640 1 -- 172.21.15.110:0/2867054443 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7fb10c148e70 con 0x7fb0ec0678c0 2024-09-10T09:59:13.163 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:13.165 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:59:13.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb0ec0678c0 msgr2=0x7fb0ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb0ec0678c0 0x7fb0ec069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb10c150a70 tx=0x7fb0fc008040 comp rx=0 tx=0).stop 2024-09-10T09:59:13.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c0cd510 msgr2=0x7fb10c148930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c0cd510 0x7fb10c148930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d830 tx=0x7fb0f800dcf0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 shutdown_connections 2024-09-10T09:59:13.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb0ec0678c0 0x7fb0ec069d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb10c0cfe30 0x7fb10c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb10c0cd510 0x7fb10c148930 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 --2- 172.21.15.110:0/2867054443 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb10c06e090 0x7fb10c14ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 >> 172.21.15.110:0/2867054443 conn(0x7fb10c06bc50 msgr2=0x7fb10c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:13.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 shutdown_connections 2024-09-10T09:59:13.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.159+0000 7fb113000640 1 -- 172.21.15.110:0/2867054443 wait complete. 2024-09-10T09:59:13.185 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-10T09:59:11.365194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237048,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737672,"statfs":{"total":773094113280,"available":772851376128,"internally_reserved":0,"allocated":4268032,"data_stored":2338732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001503"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238936+0000","last_change":"2024-09-10T09:59:04.228967+0000","last_active":"2024-09-10T09:59:05.238936+0000","last_peered":"2024-09-10T09:59:05.238936+0000","last_clean":"2024-09-10T09:59:05.238936+0000","last_became_active":"2024-09-10T09:59:04.228750+0000","last_became_peered":"2024-09-10T09:59:04.228750+0000","last_unstale":"2024-09-10T09:59:05.238936+0000","last_undegraded":"2024-09-10T09:59:05.238936+0000","last_fullsized":"2024-09-10T09:59:05.238936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:44.726674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282874+0000","last_change":"2024-09-10T09:59:04.248176+0000","last_active":"2024-09-10T09:59:05.282874+0000","last_peered":"2024-09-10T09:59:05.282874+0000","last_clean":"2024-09-10T09:59:05.282874+0000","last_became_active":"2024-09-10T09:59:04.247900+0000","last_became_peered":"2024-09-10T09:59:04.247900+0000","last_unstale":"2024-09-10T09:59:05.282874+0000","last_undegraded":"2024-09-10T09:59:05.282874+0000","last_fullsized":"2024-09-10T09:59:05.282874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:33.156203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282745+0000","last_change":"2024-09-10T09:59:04.239138+0000","last_active":"2024-09-10T09:59:05.282745+0000","last_peered":"2024-09-10T09:59:05.282745+0000","last_clean":"2024-09-10T09:59:05.282745+0000","last_became_active":"2024-09-10T09:59:04.238635+0000","last_became_peered":"2024-09-10T09:59:04.238635+0000","last_unstale":"2024-09-10T09:59:05.282745+0000","last_undegraded":"2024-09-10T09:59:05.282745+0000","last_fullsized":"2024-09-10T09:59:05.282745+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:02.620413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282877+0000","last_change":"2024-09-10T09:59:04.245525+0000","last_active":"2024-09-10T09:59:05.282877+0000","last_peered":"2024-09-10T09:59:05.282877+0000","last_clean":"2024-09-10T09:59:05.282877+0000","last_became_active":"2024-09-10T09:59:04.245226+0000","last_became_peered":"2024-09-10T09:59:04.245226+0000","last_unstale":"2024-09-10T09:59:05.282877+0000","last_undegraded":"2024-09-10T09:59:05.282877+0000","last_fullsized":"2024-09-10T09:59:05.282877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:59.181205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281986+0000","last_change":"2024-09-10T09:59:04.238166+0000","last_active":"2024-09-10T09:59:05.281986+0000","last_peered":"2024-09-10T09:59:05.281986+0000","last_clean":"2024-09-10T09:59:05.281986+0000","last_became_active":"2024-09-10T09:59:04.237855+0000","last_became_peered":"2024-09-10T09:59:04.237855+0000","last_unstale":"2024-09-10T09:59:05.281986+0000","last_undegraded":"2024-09-10T09:59:05.281986+0000","last_fullsized":"2024-09-10T09:59:05.281986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:58.954014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281677+0000","last_change":"2024-09-10T09:59:04.244110+0000","last_active":"2024-09-10T09:59:05.281677+0000","last_peered":"2024-09-10T09:59:05.281677+0000","last_clean":"2024-09-10T09:59:05.281677+0000","last_became_active":"2024-09-10T09:59:04.243830+0000","last_became_peered":"2024-09-10T09:59:04.243830+0000","last_unstale":"2024-09-10T09:59:05.281677+0000","last_undegraded":"2024-09-10T09:59:05.281677+0000","last_fullsized":"2024-09-10T09:59:05.281677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:01.116838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282524+0000","last_change":"2024-09-10T09:59:04.237928+0000","last_active":"2024-09-10T09:59:05.282524+0000","last_peered":"2024-09-10T09:59:05.282524+0000","last_clean":"2024-09-10T09:59:05.282524+0000","last_became_active":"2024-09-10T09:59:04.237485+0000","last_became_peered":"2024-09-10T09:59:04.237485+0000","last_unstale":"2024-09-10T09:59:05.282524+0000","last_undegraded":"2024-09-10T09:59:05.282524+0000","last_fullsized":"2024-09-10T09:59:05.282524+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:06.851574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282259+0000","last_change":"2024-09-10T09:59:04.254726+0000","last_active":"2024-09-10T09:59:05.282259+0000","last_peered":"2024-09-10T09:59:05.282259+0000","last_clean":"2024-09-10T09:59:05.282259+0000","last_became_active":"2024-09-10T09:59:04.254503+0000","last_became_peered":"2024-09-10T09:59:04.254503+0000","last_unstale":"2024-09-10T09:59:05.282259+0000","last_undegraded":"2024-09-10T09:59:05.282259+0000","last_fullsized":"2024-09-10T09:59:05.282259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:56.008342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238963+0000","last_change":"2024-09-10T09:59:04.248675+0000","last_active":"2024-09-10T09:59:05.238963+0000","last_peered":"2024-09-10T09:59:05.238963+0000","last_clean":"2024-09-10T09:59:05.238963+0000","last_became_active":"2024-09-10T09:59:04.248367+0000","last_became_peered":"2024-09-10T09:59:04.248367+0000","last_unstale":"2024-09-10T09:59:05.238963+0000","last_undegraded":"2024-09-10T09:59:05.238963+0000","last_fullsized":"2024-09-10T09:59:05.238963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.785736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283201+0000","last_change":"2024-09-10T09:59:04.234085+0000","last_active":"2024-09-10T09:59:05.283201+0000","last_peered":"2024-09-10T09:59:05.283201+0000","last_clean":"2024-09-10T09:59:05.283201+0000","last_became_active":"2024-09-10T09:59:04.233765+0000","last_became_peered":"2024-09-10T09:59:04.233765+0000","last_unstale":"2024-09-10T09:59:05.283201+0000","last_undegraded":"2024-09-10T09:59:05.283201+0000","last_fullsized":"2024-09-10T09:59:05.283201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:50.791729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282976+0000","last_change":"2024-09-10T09:59:04.242186+0000","last_active":"2024-09-10T09:59:05.282976+0000","last_peered":"2024-09-10T09:59:05.282976+0000","last_clean":"2024-09-10T09:59:05.282976+0000","last_became_active":"2024-09-10T09:59:04.241941+0000","last_became_peered":"2024-09-10T09:59:04.241941+0000","last_unstale":"2024-09-10T09:59:05.282976+0000","last_undegraded":"2024-09-10T09:59:05.282976+0000","last_fullsized":"2024-09-10T09:59:05.282976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:25.498252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282404+0000","last_change":"2024-09-10T09:59:04.234960+0000","last_active":"2024-09-10T09:59:05.282404+0000","last_peered":"2024-09-10T09:59:05.282404+0000","last_clean":"2024-09-10T09:59:05.282404+0000","last_became_active":"2024-09-10T09:59:04.234681+0000","last_became_peered":"2024-09-10T09:59:04.234681+0000","last_unstale":"2024-09-10T09:59:05.282404+0000","last_undegraded":"2024-09-10T09:59:05.282404+0000","last_fullsized":"2024-09-10T09:59:05.282404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:53.610398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282124+0000","last_change":"2024-09-10T09:59:04.249214+0000","last_active":"2024-09-10T09:59:05.282124+0000","last_peered":"2024-09-10T09:59:05.282124+0000","last_clean":"2024-09-10T09:59:05.282124+0000","last_became_active":"2024-09-10T09:59:04.248871+0000","last_became_peered":"2024-09-10T09:59:04.248871+0000","last_unstale":"2024-09-10T09:59:05.282124+0000","last_undegraded":"2024-09-10T09:59:05.282124+0000","last_fullsized":"2024-09-10T09:59:05.282124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:44.565690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281767+0000","last_change":"2024-09-10T09:59:04.247517+0000","last_active":"2024-09-10T09:59:05.281767+0000","last_peered":"2024-09-10T09:59:05.281767+0000","last_clean":"2024-09-10T09:59:05.281767+0000","last_became_active":"2024-09-10T09:59:04.247232+0000","last_became_peered":"2024-09-10T09:59:04.247232+0000","last_unstale":"2024-09-10T09:59:05.281767+0000","last_undegraded":"2024-09-10T09:59:05.281767+0000","last_fullsized":"2024-09-10T09:59:05.281767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:31.397569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282058+0000","last_change":"2024-09-10T09:59:04.233979+0000","last_active":"2024-09-10T09:59:05.282058+0000","last_peered":"2024-09-10T09:59:05.282058+0000","last_clean":"2024-09-10T09:59:05.282058+0000","last_became_active":"2024-09-10T09:59:04.233741+0000","last_became_peered":"2024-09-10T09:59:04.233741+0000","last_unstale":"2024-09-10T09:59:05.282058+0000","last_undegraded":"2024-09-10T09:59:05.282058+0000","last_fullsized":"2024-09-10T09:59:05.282058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:56.008342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282881+0000","last_change":"2024-09-10T09:59:04.248489+0000","last_active":"2024-09-10T09:59:05.282881+0000","last_peered":"2024-09-10T09:59:05.282881+0000","last_clean":"2024-09-10T09:59:05.282881+0000","last_became_active":"2024-09-10T09:59:04.248249+0000","last_became_peered":"2024-09-10T09:59:04.248249+0000","last_unstale":"2024-09-10T09:59:05.282881+0000","last_undegraded":"2024-09-10T09:59:05.282881+0000","last_fullsized":"2024-09-10T09:59:05.282881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:02.620413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283193+0000","last_change":"2024-09-10T09:59:04.237672+0000","last_active":"2024-09-10T09:59:05.283193+0000","last_peered":"2024-09-10T09:59:05.283193+0000","last_clean":"2024-09-10T09:59:05.283193+0000","last_became_active":"2024-09-10T09:59:04.236129+0000","last_became_peered":"2024-09-10T09:59:04.236129+0000","last_unstale":"2024-09-10T09:59:05.283193+0000","last_undegraded":"2024-09-10T09:59:05.283193+0000","last_fullsized":"2024-09-10T09:59:05.283193+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:09.621494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283009+0000","last_change":"2024-09-10T09:59:04.257031+0000","last_active":"2024-09-10T09:59:05.283009+0000","last_peered":"2024-09-10T09:59:05.283009+0000","last_clean":"2024-09-10T09:59:05.283009+0000","last_became_active":"2024-09-10T09:59:04.256552+0000","last_became_peered":"2024-09-10T09:59:04.256552+0000","last_unstale":"2024-09-10T09:59:05.283009+0000","last_undegraded":"2024-09-10T09:59:05.283009+0000","last_fullsized":"2024-09-10T09:59:05.283009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:02.774151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282345+0000","last_change":"2024-09-10T09:59:04.256930+0000","last_active":"2024-09-10T09:59:05.282345+0000","last_peered":"2024-09-10T09:59:05.282345+0000","last_clean":"2024-09-10T09:59:05.282345+0000","last_became_active":"2024-09-10T09:59:04.256147+0000","last_became_peered":"2024-09-10T09:59:04.256147+0000","last_unstale":"2024-09-10T09:59:05.282345+0000","last_undegraded":"2024-09-10T09:59:05.282345+0000","last_fullsized":"2024-09-10T09:59:05.282345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:17.464661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282613+0000","last_change":"2024-09-10T09:59:04.241976+0000","last_active":"2024-09-10T09:59:05.282613+0000","last_peered":"2024-09-10T09:59:05.282613+0000","last_clean":"2024-09-10T09:59:05.282613+0000","last_became_active":"2024-09-10T09:59:04.241667+0000","last_became_peered":"2024-09-10T09:59:04.241667+0000","last_unstale":"2024-09-10T09:59:05.282613+0000","last_undegraded":"2024-09-10T09:59:05.282613+0000","last_fullsized":"2024-09-10T09:59:05.282613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:43.713238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239003+0000","last_change":"2024-09-10T09:59:04.232895+0000","last_active":"2024-09-10T09:59:05.239003+0000","last_peered":"2024-09-10T09:59:05.239003+0000","last_clean":"2024-09-10T09:59:05.239003+0000","last_became_active":"2024-09-10T09:59:04.232628+0000","last_became_peered":"2024-09-10T09:59:04.232628+0000","last_unstale":"2024-09-10T09:59:05.239003+0000","last_undegraded":"2024-09-10T09:59:05.239003+0000","last_fullsized":"2024-09-10T09:59:05.239003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:36.495442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239585+0000","last_change":"2024-09-10T09:59:04.235308+0000","last_active":"2024-09-10T09:59:05.239585+0000","last_peered":"2024-09-10T09:59:05.239585+0000","last_clean":"2024-09-10T09:59:05.239585+0000","last_became_active":"2024-09-10T09:59:04.233274+0000","last_became_peered":"2024-09-10T09:59:04.233274+0000","last_unstale":"2024-09-10T09:59:05.239585+0000","last_undegraded":"2024-09-10T09:59:05.239585+0000","last_fullsized":"2024-09-10T09:59:05.239585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:39.465659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234473+0000","last_change":"2024-09-10T09:59:04.232149+0000","last_active":"2024-09-10T09:59:05.234473+0000","last_peered":"2024-09-10T09:59:05.234473+0000","last_clean":"2024-09-10T09:59:05.234473+0000","last_became_active":"2024-09-10T09:59:04.231828+0000","last_became_peered":"2024-09-10T09:59:04.231828+0000","last_unstale":"2024-09-10T09:59:05.234473+0000","last_undegraded":"2024-09-10T09:59:05.234473+0000","last_fullsized":"2024-09-10T09:59:05.234473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:44.726674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281997+0000","last_change":"2024-09-10T09:59:04.244645+0000","last_active":"2024-09-10T09:59:05.281997+0000","last_peered":"2024-09-10T09:59:05.281997+0000","last_clean":"2024-09-10T09:59:05.281997+0000","last_became_active":"2024-09-10T09:59:04.244315+0000","last_became_peered":"2024-09-10T09:59:04.244315+0000","last_unstale":"2024-09-10T09:59:05.281997+0000","last_undegraded":"2024-09-10T09:59:05.281997+0000","last_fullsized":"2024-09-10T09:59:05.281997+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:35.416635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283299+0000","last_change":"2024-09-10T09:59:04.247910+0000","last_active":"2024-09-10T09:59:05.283299+0000","last_peered":"2024-09-10T09:59:05.283299+0000","last_clean":"2024-09-10T09:59:05.283299+0000","last_became_active":"2024-09-10T09:59:04.247541+0000","last_became_peered":"2024-09-10T09:59:04.247541+0000","last_unstale":"2024-09-10T09:59:05.283299+0000","last_undegraded":"2024-09-10T09:59:05.283299+0000","last_fullsized":"2024-09-10T09:59:05.283299+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:35.416635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283126+0000","last_change":"2024-09-10T09:59:04.235085+0000","last_active":"2024-09-10T09:59:05.283126+0000","last_peered":"2024-09-10T09:59:05.283126+0000","last_clean":"2024-09-10T09:59:05.283126+0000","last_became_active":"2024-09-10T09:59:04.232953+0000","last_became_peered":"2024-09-10T09:59:04.232953+0000","last_unstale":"2024-09-10T09:59:05.283126+0000","last_undegraded":"2024-09-10T09:59:05.283126+0000","last_fullsized":"2024-09-10T09:59:05.283126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:57.657194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282615+0000","last_change":"2024-09-10T09:59:04.245634+0000","last_active":"2024-09-10T09:59:05.282615+0000","last_peered":"2024-09-10T09:59:05.282615+0000","last_clean":"2024-09-10T09:59:05.282615+0000","last_became_active":"2024-09-10T09:59:04.245286+0000","last_became_peered":"2024-09-10T09:59:04.245286+0000","last_unstale":"2024-09-10T09:59:05.282615+0000","last_undegraded":"2024-09-10T09:59:05.282615+0000","last_fullsized":"2024-09-10T09:59:05.282615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:36.422543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282574+0000","last_change":"2024-09-10T09:59:04.235825+0000","last_active":"2024-09-10T09:59:05.282574+0000","last_peered":"2024-09-10T09:59:05.282574+0000","last_clean":"2024-09-10T09:59:05.282574+0000","last_became_active":"2024-09-10T09:59:04.235305+0000","last_became_peered":"2024-09-10T09:59:04.235305+0000","last_unstale":"2024-09-10T09:59:05.282574+0000","last_undegraded":"2024-09-10T09:59:05.282574+0000","last_fullsized":"2024-09-10T09:59:05.282574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:50.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239170+0000","last_change":"2024-09-10T09:59:04.233036+0000","last_active":"2024-09-10T09:59:05.239170+0000","last_peered":"2024-09-10T09:59:05.239170+0000","last_clean":"2024-09-10T09:59:05.239170+0000","last_became_active":"2024-09-10T09:59:04.232793+0000","last_became_peered":"2024-09-10T09:59:04.232793+0000","last_unstale":"2024-09-10T09:59:05.239170+0000","last_undegraded":"2024-09-10T09:59:05.239170+0000","last_fullsized":"2024-09-10T09:59:05.239170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:36.422543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282455+0000","last_change":"2024-09-10T09:59:04.233379+0000","last_active":"2024-09-10T09:59:05.282455+0000","last_peered":"2024-09-10T09:59:05.282455+0000","last_clean":"2024-09-10T09:59:05.282455+0000","last_became_active":"2024-09-10T09:59:04.232985+0000","last_became_peered":"2024-09-10T09:59:04.232985+0000","last_unstale":"2024-09-10T09:59:05.282455+0000","last_undegraded":"2024-09-10T09:59:05.282455+0000","last_fullsized":"2024-09-10T09:59:05.282455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:53.610398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238690+0000","last_change":"2024-09-10T09:59:04.244628+0000","last_active":"2024-09-10T09:59:05.238690+0000","last_peered":"2024-09-10T09:59:05.238690+0000","last_clean":"2024-09-10T09:59:05.238690+0000","last_became_active":"2024-09-10T09:59:04.244218+0000","last_became_peered":"2024-09-10T09:59:04.244218+0000","last_unstale":"2024-09-10T09:59:05.238690+0000","last_undegraded":"2024-09-10T09:59:05.238690+0000","last_fullsized":"2024-09-10T09:59:05.238690+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:02.539176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283138+0000","last_change":"2024-09-10T09:59:04.242293+0000","last_active":"2024-09-10T09:59:05.283138+0000","last_peered":"2024-09-10T09:59:05.283138+0000","last_clean":"2024-09-10T09:59:05.283138+0000","last_became_active":"2024-09-10T09:59:04.238754+0000","last_became_peered":"2024-09-10T09:59:04.238754+0000","last_unstale":"2024-09-10T09:59:05.283138+0000","last_undegraded":"2024-09-10T09:59:05.283138+0000","last_fullsized":"2024-09-10T09:59:05.283138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:12.332951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234909+0000","last_change":"2024-09-10T09:59:04.237127+0000","last_active":"2024-09-10T09:59:05.234909+0000","last_peered":"2024-09-10T09:59:05.234909+0000","last_clean":"2024-09-10T09:59:05.234909+0000","last_became_active":"2024-09-10T09:59:04.236856+0000","last_became_peered":"2024-09-10T09:59:04.236856+0000","last_unstale":"2024-09-10T09:59:05.234909+0000","last_undegraded":"2024-09-10T09:59:05.234909+0000","last_fullsized":"2024-09-10T09:59:05.234909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.785736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239151+0000","last_change":"2024-09-10T09:59:03.217173+0000","last_active":"2024-09-10T09:59:05.239151+0000","last_peered":"2024-09-10T09:59:05.239151+0000","last_clean":"2024-09-10T09:59:05.239151+0000","last_became_active":"2024-09-10T09:59:03.216896+0000","last_became_peered":"2024-09-10T09:59:03.216896+0000","last_unstale":"2024-09-10T09:59:05.239151+0000","last_undegraded":"2024-09-10T09:59:05.239151+0000","last_fullsized":"2024-09-10T09:59:05.239151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:46.302427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239090+0000","last_change":"2024-09-10T09:59:04.232436+0000","last_active":"2024-09-10T09:59:05.239090+0000","last_peered":"2024-09-10T09:59:05.239090+0000","last_clean":"2024-09-10T09:59:05.239090+0000","last_became_active":"2024-09-10T09:59:04.229805+0000","last_became_peered":"2024-09-10T09:59:04.229805+0000","last_unstale":"2024-09-10T09:59:05.239090+0000","last_undegraded":"2024-09-10T09:59:05.239090+0000","last_fullsized":"2024-09-10T09:59:05.239090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:30.880271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234937+0000","last_change":"2024-09-10T09:59:03.204423+0000","last_active":"2024-09-10T09:59:05.234937+0000","last_peered":"2024-09-10T09:59:05.234937+0000","last_clean":"2024-09-10T09:59:05.234937+0000","last_became_active":"2024-09-10T09:59:03.204117+0000","last_became_peered":"2024-09-10T09:59:03.204117+0000","last_unstale":"2024-09-10T09:59:05.234937+0000","last_undegraded":"2024-09-10T09:59:05.234937+0000","last_fullsized":"2024-09-10T09:59:05.234937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:42.707288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239394+0000","last_change":"2024-09-10T09:59:04.235085+0000","last_active":"2024-09-10T09:59:05.239394+0000","last_peered":"2024-09-10T09:59:05.239394+0000","last_clean":"2024-09-10T09:59:05.239394+0000","last_became_active":"2024-09-10T09:59:04.234827+0000","last_became_peered":"2024-09-10T09:59:04.234827+0000","last_unstale":"2024-09-10T09:59:05.239394+0000","last_undegraded":"2024-09-10T09:59:05.239394+0000","last_fullsized":"2024-09-10T09:59:05.239394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:59.181205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282590+0000","last_change":"2024-09-10T09:59:03.218809+0000","last_active":"2024-09-10T09:59:05.282590+0000","last_peered":"2024-09-10T09:59:05.282590+0000","last_clean":"2024-09-10T09:59:05.282590+0000","last_became_active":"2024-09-10T09:59:03.218587+0000","last_became_peered":"2024-09-10T09:59:03.218587+0000","last_unstale":"2024-09-10T09:59:05.282590+0000","last_undegraded":"2024-09-10T09:59:05.282590+0000","last_fullsized":"2024-09-10T09:59:05.282590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:33.139925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283206+0000","last_change":"2024-09-10T09:59:04.236025+0000","last_active":"2024-09-10T09:59:05.283206+0000","last_peered":"2024-09-10T09:59:05.283206+0000","last_clean":"2024-09-10T09:59:05.283206+0000","last_became_active":"2024-09-10T09:59:04.235729+0000","last_became_peered":"2024-09-10T09:59:04.235729+0000","last_unstale":"2024-09-10T09:59:05.283206+0000","last_undegraded":"2024-09-10T09:59:05.283206+0000","last_fullsized":"2024-09-10T09:59:05.283206+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:49.686495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282555+0000","last_change":"2024-09-10T09:59:03.219613+0000","last_active":"2024-09-10T09:59:05.282555+0000","last_peered":"2024-09-10T09:59:05.282555+0000","last_clean":"2024-09-10T09:59:05.282555+0000","last_became_active":"2024-09-10T09:59:03.219323+0000","last_became_peered":"2024-09-10T09:59:03.219323+0000","last_unstale":"2024-09-10T09:59:05.282555+0000","last_undegraded":"2024-09-10T09:59:05.282555+0000","last_fullsized":"2024-09-10T09:59:05.282555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:16.548391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282810+0000","last_change":"2024-09-10T09:59:04.238511+0000","last_active":"2024-09-10T09:59:05.282810+0000","last_peered":"2024-09-10T09:59:05.282810+0000","last_clean":"2024-09-10T09:59:05.282810+0000","last_became_active":"2024-09-10T09:59:04.238179+0000","last_became_peered":"2024-09-10T09:59:04.238179+0000","last_unstale":"2024-09-10T09:59:05.282810+0000","last_undegraded":"2024-09-10T09:59:05.282810+0000","last_fullsized":"2024-09-10T09:59:05.282810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:12.920550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239229+0000","last_change":"2024-09-10T09:59:03.214550+0000","last_active":"2024-09-10T09:59:05.239229+0000","last_peered":"2024-09-10T09:59:05.239229+0000","last_clean":"2024-09-10T09:59:05.239229+0000","last_became_active":"2024-09-10T09:59:03.213507+0000","last_became_peered":"2024-09-10T09:59:03.213507+0000","last_unstale":"2024-09-10T09:59:05.239229+0000","last_undegraded":"2024-09-10T09:59:05.239229+0000","last_fullsized":"2024-09-10T09:59:05.239229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:21.489357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282249+0000","last_change":"2024-09-10T09:59:04.234812+0000","last_active":"2024-09-10T09:59:05.282249+0000","last_peered":"2024-09-10T09:59:05.282249+0000","last_clean":"2024-09-10T09:59:05.282249+0000","last_became_active":"2024-09-10T09:59:04.234515+0000","last_became_peered":"2024-09-10T09:59:04.234515+0000","last_unstale":"2024-09-10T09:59:05.282249+0000","last_undegraded":"2024-09-10T09:59:05.282249+0000","last_fullsized":"2024-09-10T09:59:05.282249+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:54.921113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238659+0000","last_change":"2024-09-10T09:59:03.210463+0000","last_active":"2024-09-10T09:59:05.238659+0000","last_peered":"2024-09-10T09:59:05.238659+0000","last_clean":"2024-09-10T09:59:05.238659+0000","last_became_active":"2024-09-10T09:59:03.210157+0000","last_became_peered":"2024-09-10T09:59:03.210157+0000","last_unstale":"2024-09-10T09:59:05.238659+0000","last_undegraded":"2024-09-10T09:59:05.238659+0000","last_fullsized":"2024-09-10T09:59:05.238659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:13.022417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238459+0000","last_change":"2024-09-10T09:59:04.223832+0000","last_active":"2024-09-10T09:59:05.238459+0000","last_peered":"2024-09-10T09:59:05.238459+0000","last_clean":"2024-09-10T09:59:05.238459+0000","last_became_active":"2024-09-10T09:59:04.223512+0000","last_became_peered":"2024-09-10T09:59:04.223512+0000","last_unstale":"2024-09-10T09:59:05.238459+0000","last_undegraded":"2024-09-10T09:59:05.238459+0000","last_fullsized":"2024-09-10T09:59:05.238459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:44.726674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281756+0000","last_change":"2024-09-10T09:59:03.210986+0000","last_active":"2024-09-10T09:59:05.281756+0000","last_peered":"2024-09-10T09:59:05.281756+0000","last_clean":"2024-09-10T09:59:05.281756+0000","last_became_active":"2024-09-10T09:59:03.210736+0000","last_became_peered":"2024-09-10T09:59:03.210736+0000","last_unstale":"2024-09-10T09:59:05.281756+0000","last_undegraded":"2024-09-10T09:59:05.281756+0000","last_fullsized":"2024-09-10T09:59:05.281756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:01.201854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281572+0000","last_change":"2024-09-10T09:59:04.233989+0000","last_active":"2024-09-10T09:59:05.281572+0000","last_peered":"2024-09-10T09:59:05.281572+0000","last_clean":"2024-09-10T09:59:05.281572+0000","last_became_active":"2024-09-10T09:59:04.233630+0000","last_became_peered":"2024-09-10T09:59:04.233630+0000","last_unstale":"2024-09-10T09:59:05.281572+0000","last_undegraded":"2024-09-10T09:59:05.281572+0000","last_fullsized":"2024-09-10T09:59:05.281572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.785736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238515+0000","last_change":"2024-09-10T09:59:03.215583+0000","last_active":"2024-09-10T09:59:05.238515+0000","last_peered":"2024-09-10T09:59:05.238515+0000","last_clean":"2024-09-10T09:59:05.238515+0000","last_became_active":"2024-09-10T09:59:03.215296+0000","last_became_peered":"2024-09-10T09:59:03.215296+0000","last_unstale":"2024-09-10T09:59:05.238515+0000","last_undegraded":"2024-09-10T09:59:05.238515+0000","last_fullsized":"2024-09-10T09:59:05.238515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:57.818219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282564+0000","last_change":"2024-09-10T09:59:04.247805+0000","last_active":"2024-09-10T09:59:05.282564+0000","last_peered":"2024-09-10T09:59:05.282564+0000","last_clean":"2024-09-10T09:59:05.282564+0000","last_became_active":"2024-09-10T09:59:04.247479+0000","last_became_peered":"2024-09-10T09:59:04.247479+0000","last_unstale":"2024-09-10T09:59:05.282564+0000","last_undegraded":"2024-09-10T09:59:05.282564+0000","last_fullsized":"2024-09-10T09:59:05.282564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:06.851574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283365+0000","last_change":"2024-09-10T09:59:03.208433+0000","last_active":"2024-09-10T09:59:05.283365+0000","last_peered":"2024-09-10T09:59:05.283365+0000","last_clean":"2024-09-10T09:59:05.283365+0000","last_became_active":"2024-09-10T09:59:03.207106+0000","last_became_peered":"2024-09-10T09:59:03.207106+0000","last_unstale":"2024-09-10T09:59:05.283365+0000","last_undegraded":"2024-09-10T09:59:05.283365+0000","last_fullsized":"2024-09-10T09:59:05.283365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:13.286437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283272+0000","last_change":"2024-09-10T09:59:04.257770+0000","last_active":"2024-09-10T09:59:05.283272+0000","last_peered":"2024-09-10T09:59:05.283272+0000","last_clean":"2024-09-10T09:59:05.283272+0000","last_became_active":"2024-09-10T09:59:04.257535+0000","last_became_peered":"2024-09-10T09:59:04.257535+0000","last_unstale":"2024-09-10T09:59:05.283272+0000","last_undegraded":"2024-09-10T09:59:05.283272+0000","last_fullsized":"2024-09-10T09:59:05.283272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:09:27.392572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234963+0000","last_change":"2024-09-10T09:59:03.204371+0000","last_active":"2024-09-10T09:59:05.234963+0000","last_peered":"2024-09-10T09:59:05.234963+0000","last_clean":"2024-09-10T09:59:05.234963+0000","last_became_active":"2024-09-10T09:59:03.204087+0000","last_became_peered":"2024-09-10T09:59:03.204087+0000","last_unstale":"2024-09-10T09:59:05.234963+0000","last_undegraded":"2024-09-10T09:59:05.234963+0000","last_fullsized":"2024-09-10T09:59:05.234963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:49.036675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281596+0000","last_change":"2024-09-10T09:59:04.234326+0000","last_active":"2024-09-10T09:59:05.281596+0000","last_peered":"2024-09-10T09:59:05.281596+0000","last_clean":"2024-09-10T09:59:05.281596+0000","last_became_active":"2024-09-10T09:59:04.234025+0000","last_became_peered":"2024-09-10T09:59:04.234025+0000","last_unstale":"2024-09-10T09:59:05.281596+0000","last_undegraded":"2024-09-10T09:59:05.281596+0000","last_fullsized":"2024-09-10T09:59:05.281596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:09.398256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282688+0000","last_change":"2024-09-10T09:59:03.206547+0000","last_active":"2024-09-10T09:59:05.282688+0000","last_peered":"2024-09-10T09:59:05.282688+0000","last_clean":"2024-09-10T09:59:05.282688+0000","last_became_active":"2024-09-10T09:59:03.206237+0000","last_became_peered":"2024-09-10T09:59:03.206237+0000","last_unstale":"2024-09-10T09:59:05.282688+0000","last_undegraded":"2024-09-10T09:59:05.282688+0000","last_fullsized":"2024-09-10T09:59:05.282688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:01.533226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282632+0000","last_change":"2024-09-10T09:59:04.239390+0000","last_active":"2024-09-10T09:59:05.282632+0000","last_peered":"2024-09-10T09:59:05.282632+0000","last_clean":"2024-09-10T09:59:05.282632+0000","last_became_active":"2024-09-10T09:59:04.238904+0000","last_became_peered":"2024-09-10T09:59:04.238904+0000","last_unstale":"2024-09-10T09:59:05.282632+0000","last_undegraded":"2024-09-10T09:59:05.282632+0000","last_fullsized":"2024-09-10T09:59:05.282632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:12.920550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281699+0000","last_change":"2024-09-10T09:59:03.219831+0000","last_active":"2024-09-10T09:59:05.281699+0000","last_peered":"2024-09-10T09:59:05.281699+0000","last_clean":"2024-09-10T09:59:05.281699+0000","last_became_active":"2024-09-10T09:59:03.219272+0000","last_became_peered":"2024-09-10T09:59:03.219272+0000","last_unstale":"2024-09-10T09:59:05.281699+0000","last_undegraded":"2024-09-10T09:59:05.281699+0000","last_fullsized":"2024-09-10T09:59:05.281699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:03.029812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281706+0000","last_change":"2024-09-10T09:59:04.241369+0000","last_active":"2024-09-10T09:59:05.281706+0000","last_peered":"2024-09-10T09:59:05.281706+0000","last_clean":"2024-09-10T09:59:05.281706+0000","last_became_active":"2024-09-10T09:59:04.241100+0000","last_became_peered":"2024-09-10T09:59:04.241100+0000","last_unstale":"2024-09-10T09:59:05.281706+0000","last_undegraded":"2024-09-10T09:59:05.281706+0000","last_fullsized":"2024-09-10T09:59:05.281706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:36.495442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239393+0000","last_change":"2024-09-10T09:59:03.216285+0000","last_active":"2024-09-10T09:59:05.239393+0000","last_peered":"2024-09-10T09:59:05.239393+0000","last_clean":"2024-09-10T09:59:05.239393+0000","last_became_active":"2024-09-10T09:59:03.215878+0000","last_became_peered":"2024-09-10T09:59:03.215878+0000","last_unstale":"2024-09-10T09:59:05.239393+0000","last_undegraded":"2024-09-10T09:59:05.239393+0000","last_fullsized":"2024-09-10T09:59:05.239393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:43.559741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281688+0000","last_change":"2024-09-10T09:59:04.233774+0000","last_active":"2024-09-10T09:59:05.281688+0000","last_peered":"2024-09-10T09:59:05.281688+0000","last_clean":"2024-09-10T09:59:05.281688+0000","last_became_active":"2024-09-10T09:59:04.233387+0000","last_became_peered":"2024-09-10T09:59:04.233387+0000","last_unstale":"2024-09-10T09:59:05.281688+0000","last_undegraded":"2024-09-10T09:59:05.281688+0000","last_fullsized":"2024-09-10T09:59:05.281688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:49.890997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281796+0000","last_change":"2024-09-10T09:59:03.217487+0000","last_active":"2024-09-10T09:59:05.281796+0000","last_peered":"2024-09-10T09:59:05.281796+0000","last_clean":"2024-09-10T09:59:05.281796+0000","last_became_active":"2024-09-10T09:59:03.217236+0000","last_became_peered":"2024-09-10T09:59:03.217236+0000","last_unstale":"2024-09-10T09:59:05.281796+0000","last_undegraded":"2024-09-10T09:59:05.281796+0000","last_fullsized":"2024-09-10T09:59:05.281796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:42.707288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282716+0000","last_change":"2024-09-10T09:59:04.238031+0000","last_active":"2024-09-10T09:59:05.282716+0000","last_peered":"2024-09-10T09:59:05.282716+0000","last_clean":"2024-09-10T09:59:05.282716+0000","last_became_active":"2024-09-10T09:59:04.237680+0000","last_became_peered":"2024-09-10T09:59:04.237680+0000","last_unstale":"2024-09-10T09:59:05.282716+0000","last_undegraded":"2024-09-10T09:59:05.282716+0000","last_fullsized":"2024-09-10T09:59:05.282716+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:50.560127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281843+0000","last_change":"2024-09-10T09:59:03.208322+0000","last_active":"2024-09-10T09:59:05.281843+0000","last_peered":"2024-09-10T09:59:05.281843+0000","last_clean":"2024-09-10T09:59:05.281843+0000","last_became_active":"2024-09-10T09:59:03.207255+0000","last_became_peered":"2024-09-10T09:59:03.207255+0000","last_unstale":"2024-09-10T09:59:05.281843+0000","last_undegraded":"2024-09-10T09:59:05.281843+0000","last_fullsized":"2024-09-10T09:59:05.281843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:37.076444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238532+0000","last_change":"2024-09-10T09:59:04.231783+0000","last_active":"2024-09-10T09:59:05.238532+0000","last_peered":"2024-09-10T09:59:05.238532+0000","last_clean":"2024-09-10T09:59:05.238532+0000","last_became_active":"2024-09-10T09:59:04.231510+0000","last_became_peered":"2024-09-10T09:59:04.231510+0000","last_unstale":"2024-09-10T09:59:05.238532+0000","last_undegraded":"2024-09-10T09:59:05.238532+0000","last_fullsized":"2024-09-10T09:59:05.238532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:22.377203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281289+0000","last_change":"2024-09-10T09:59:03.219531+0000","last_active":"2024-09-10T09:59:05.281289+0000","last_peered":"2024-09-10T09:59:05.281289+0000","last_clean":"2024-09-10T09:59:05.281289+0000","last_became_active":"2024-09-10T09:59:03.219216+0000","last_became_peered":"2024-09-10T09:59:03.219216+0000","last_unstale":"2024-09-10T09:59:05.281289+0000","last_undegraded":"2024-09-10T09:59:05.281289+0000","last_fullsized":"2024-09-10T09:59:05.281289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:03.964831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238874+0000","last_change":"2024-09-10T09:59:04.232545+0000","last_active":"2024-09-10T09:59:05.238874+0000","last_peered":"2024-09-10T09:59:05.238874+0000","last_clean":"2024-09-10T09:59:05.238874+0000","last_became_active":"2024-09-10T09:59:04.232160+0000","last_became_peered":"2024-09-10T09:59:04.232160+0000","last_unstale":"2024-09-10T09:59:05.238874+0000","last_undegraded":"2024-09-10T09:59:05.238874+0000","last_fullsized":"2024-09-10T09:59:05.238874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:26.363067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283359+0000","last_change":"2024-09-10T09:59:03.216864+0000","last_active":"2024-09-10T09:59:05.283359+0000","last_peered":"2024-09-10T09:59:05.283359+0000","last_clean":"2024-09-10T09:59:05.283359+0000","last_became_active":"2024-09-10T09:59:03.216585+0000","last_became_peered":"2024-09-10T09:59:03.216585+0000","last_unstale":"2024-09-10T09:59:05.283359+0000","last_undegraded":"2024-09-10T09:59:05.283359+0000","last_fullsized":"2024-09-10T09:59:05.283359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.310845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282843+0000","last_change":"2024-09-10T09:59:04.232330+0000","last_active":"2024-09-10T09:59:05.282843+0000","last_peered":"2024-09-10T09:59:05.282843+0000","last_clean":"2024-09-10T09:59:05.282843+0000","last_became_active":"2024-09-10T09:59:04.232058+0000","last_became_peered":"2024-09-10T09:59:04.232058+0000","last_unstale":"2024-09-10T09:59:05.282843+0000","last_undegraded":"2024-09-10T09:59:05.282843+0000","last_fullsized":"2024-09-10T09:59:05.282843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:25.498252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238928+0000","last_change":"2024-09-10T09:59:03.216300+0000","last_active":"2024-09-10T09:59:05.238928+0000","last_peered":"2024-09-10T09:59:05.238928+0000","last_clean":"2024-09-10T09:59:05.238928+0000","last_became_active":"2024-09-10T09:59:03.215913+0000","last_became_peered":"2024-09-10T09:59:03.215913+0000","last_unstale":"2024-09-10T09:59:05.238928+0000","last_undegraded":"2024-09-10T09:59:05.238928+0000","last_fullsized":"2024-09-10T09:59:05.238928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:57.948064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282949+0000","last_change":"2024-09-10T09:59:04.245576+0000","last_active":"2024-09-10T09:59:05.282949+0000","last_peered":"2024-09-10T09:59:05.282949+0000","last_clean":"2024-09-10T09:59:05.282949+0000","last_became_active":"2024-09-10T09:59:04.245256+0000","last_became_peered":"2024-09-10T09:59:04.245256+0000","last_unstale":"2024-09-10T09:59:05.282949+0000","last_undegraded":"2024-09-10T09:59:05.282949+0000","last_fullsized":"2024-09-10T09:59:05.282949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:39.465659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239689+0000","last_change":"2024-09-10T09:59:03.219779+0000","last_active":"2024-09-10T09:59:05.239689+0000","last_peered":"2024-09-10T09:59:05.239689+0000","last_clean":"2024-09-10T09:59:05.239689+0000","last_became_active":"2024-09-10T09:59:03.219222+0000","last_became_peered":"2024-09-10T09:59:03.219222+0000","last_unstale":"2024-09-10T09:59:05.239689+0000","last_undegraded":"2024-09-10T09:59:05.239689+0000","last_fullsized":"2024-09-10T09:59:05.239689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:37.117350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282324+0000","last_change":"2024-09-10T09:59:04.248514+0000","last_active":"2024-09-10T09:59:05.282324+0000","last_peered":"2024-09-10T09:59:05.282324+0000","last_clean":"2024-09-10T09:59:05.282324+0000","last_became_active":"2024-09-10T09:59:04.236015+0000","last_became_peered":"2024-09-10T09:59:04.236015+0000","last_unstale":"2024-09-10T09:59:05.282324+0000","last_undegraded":"2024-09-10T09:59:05.282324+0000","last_fullsized":"2024-09-10T09:59:05.282324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:22.316795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239391+0000","last_change":"2024-09-10T09:59:03.213361+0000","last_active":"2024-09-10T09:59:05.239391+0000","last_peered":"2024-09-10T09:59:05.239391+0000","last_clean":"2024-09-10T09:59:05.239391+0000","last_became_active":"2024-09-10T09:59:03.213126+0000","last_became_peered":"2024-09-10T09:59:03.213126+0000","last_unstale":"2024-09-10T09:59:05.239391+0000","last_undegraded":"2024-09-10T09:59:05.239391+0000","last_fullsized":"2024-09-10T09:59:05.239391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:11.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282158+0000","last_change":"2024-09-10T09:59:04.244079+0000","last_active":"2024-09-10T09:59:05.282158+0000","last_peered":"2024-09-10T09:59:05.282158+0000","last_clean":"2024-09-10T09:59:05.282158+0000","last_became_active":"2024-09-10T09:59:04.243851+0000","last_became_peered":"2024-09-10T09:59:04.243851+0000","last_unstale":"2024-09-10T09:59:05.282158+0000","last_undegraded":"2024-09-10T09:59:05.282158+0000","last_fullsized":"2024-09-10T09:59:05.282158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:12.897693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234727+0000","last_change":"2024-09-10T09:59:03.209652+0000","last_active":"2024-09-10T09:59:05.234727+0000","last_peered":"2024-09-10T09:59:05.234727+0000","last_clean":"2024-09-10T09:59:05.234727+0000","last_became_active":"2024-09-10T09:59:03.209369+0000","last_became_peered":"2024-09-10T09:59:03.209369+0000","last_unstale":"2024-09-10T09:59:05.234727+0000","last_undegraded":"2024-09-10T09:59:05.234727+0000","last_fullsized":"2024-09-10T09:59:05.234727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.310845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282269+0000","last_change":"2024-09-10T09:59:04.239608+0000","last_active":"2024-09-10T09:59:05.282269+0000","last_peered":"2024-09-10T09:59:05.282269+0000","last_clean":"2024-09-10T09:59:05.282269+0000","last_became_active":"2024-09-10T09:59:04.239262+0000","last_became_peered":"2024-09-10T09:59:04.239262+0000","last_unstale":"2024-09-10T09:59:05.282269+0000","last_undegraded":"2024-09-10T09:59:05.282269+0000","last_fullsized":"2024-09-10T09:59:05.282269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:47.308377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238528+0000","last_change":"2024-09-10T09:59:03.219990+0000","last_active":"2024-09-10T09:59:05.238528+0000","last_peered":"2024-09-10T09:59:05.238528+0000","last_clean":"2024-09-10T09:59:05.238528+0000","last_became_active":"2024-09-10T09:59:03.219678+0000","last_became_peered":"2024-09-10T09:59:03.219678+0000","last_unstale":"2024-09-10T09:59:05.238528+0000","last_undegraded":"2024-09-10T09:59:05.238528+0000","last_fullsized":"2024-09-10T09:59:05.238528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:43.559741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282017+0000","last_change":"2024-09-10T09:59:04.239523+0000","last_active":"2024-09-10T09:59:05.282017+0000","last_peered":"2024-09-10T09:59:05.282017+0000","last_clean":"2024-09-10T09:59:05.282017+0000","last_became_active":"2024-09-10T09:59:04.239246+0000","last_became_peered":"2024-09-10T09:59:04.239246+0000","last_unstale":"2024-09-10T09:59:05.282017+0000","last_undegraded":"2024-09-10T09:59:05.282017+0000","last_fullsized":"2024-09-10T09:59:05.282017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:38.123300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.239462+0000","last_change":"2024-09-10T09:59:03.210723+0000","last_active":"2024-09-10T09:59:05.239462+0000","last_peered":"2024-09-10T09:59:05.239462+0000","last_clean":"2024-09-10T09:59:05.239462+0000","last_became_active":"2024-09-10T09:59:03.210441+0000","last_became_peered":"2024-09-10T09:59:03.210441+0000","last_unstale":"2024-09-10T09:59:05.239462+0000","last_undegraded":"2024-09-10T09:59:05.239462+0000","last_fullsized":"2024-09-10T09:59:05.239462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.310845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282398+0000","last_change":"2024-09-10T09:59:04.237462+0000","last_active":"2024-09-10T09:59:05.282398+0000","last_peered":"2024-09-10T09:59:05.282398+0000","last_clean":"2024-09-10T09:59:05.282398+0000","last_became_active":"2024-09-10T09:59:04.235950+0000","last_became_peered":"2024-09-10T09:59:04.235950+0000","last_unstale":"2024-09-10T09:59:05.282398+0000","last_undegraded":"2024-09-10T09:59:05.282398+0000","last_fullsized":"2024-09-10T09:59:05.282398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:54.921113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281874+0000","last_change":"2024-09-10T09:59:03.206594+0000","last_active":"2024-09-10T09:59:05.281874+0000","last_peered":"2024-09-10T09:59:05.281874+0000","last_clean":"2024-09-10T09:59:05.281874+0000","last_became_active":"2024-09-10T09:59:03.206278+0000","last_became_peered":"2024-09-10T09:59:03.206278+0000","last_unstale":"2024-09-10T09:59:05.281874+0000","last_undegraded":"2024-09-10T09:59:05.281874+0000","last_fullsized":"2024-09-10T09:59:05.281874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:21.371253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282486+0000","last_change":"2024-09-10T09:59:04.228620+0000","last_active":"2024-09-10T09:59:05.282486+0000","last_peered":"2024-09-10T09:59:05.282486+0000","last_clean":"2024-09-10T09:59:05.282486+0000","last_became_active":"2024-09-10T09:59:04.228327+0000","last_became_peered":"2024-09-10T09:59:04.228327+0000","last_unstale":"2024-09-10T09:59:05.282486+0000","last_undegraded":"2024-09-10T09:59:05.282486+0000","last_fullsized":"2024-09-10T09:59:05.282486+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:18.633489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283407+0000","last_change":"2024-09-10T09:59:03.218569+0000","last_active":"2024-09-10T09:59:05.283407+0000","last_peered":"2024-09-10T09:59:05.283407+0000","last_clean":"2024-09-10T09:59:05.283407+0000","last_became_active":"2024-09-10T09:59:03.218234+0000","last_became_peered":"2024-09-10T09:59:03.218234+0000","last_unstale":"2024-09-10T09:59:05.283407+0000","last_undegraded":"2024-09-10T09:59:05.283407+0000","last_fullsized":"2024-09-10T09:59:05.283407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:17.627540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282978+0000","last_change":"2024-09-10T09:59:04.249788+0000","last_active":"2024-09-10T09:59:05.282978+0000","last_peered":"2024-09-10T09:59:05.282978+0000","last_clean":"2024-09-10T09:59:05.282978+0000","last_became_active":"2024-09-10T09:59:04.249254+0000","last_became_peered":"2024-09-10T09:59:04.249254+0000","last_unstale":"2024-09-10T09:59:05.282978+0000","last_undegraded":"2024-09-10T09:59:05.282978+0000","last_fullsized":"2024-09-10T09:59:05.282978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:54.301065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.235142+0000","last_change":"2024-09-10T09:59:03.209630+0000","last_active":"2024-09-10T09:59:05.235142+0000","last_peered":"2024-09-10T09:59:05.235142+0000","last_clean":"2024-09-10T09:59:05.235142+0000","last_became_active":"2024-09-10T09:59:03.209346+0000","last_became_peered":"2024-09-10T09:59:03.209346+0000","last_unstale":"2024-09-10T09:59:05.235142+0000","last_undegraded":"2024-09-10T09:59:05.235142+0000","last_fullsized":"2024-09-10T09:59:05.235142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:44.038105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234792+0000","last_change":"2024-09-10T09:59:04.240645+0000","last_active":"2024-09-10T09:59:05.234792+0000","last_peered":"2024-09-10T09:59:05.234792+0000","last_clean":"2024-09-10T09:59:05.234792+0000","last_became_active":"2024-09-10T09:59:04.239034+0000","last_became_peered":"2024-09-10T09:59:04.239034+0000","last_unstale":"2024-09-10T09:59:05.234792+0000","last_undegraded":"2024-09-10T09:59:05.234792+0000","last_fullsized":"2024-09-10T09:59:05.234792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:26.363067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282974+0000","last_change":"2024-09-10T09:59:03.215506+0000","last_active":"2024-09-10T09:59:05.282974+0000","last_peered":"2024-09-10T09:59:05.282974+0000","last_clean":"2024-09-10T09:59:05.282974+0000","last_became_active":"2024-09-10T09:59:03.214501+0000","last_became_peered":"2024-09-10T09:59:03.214501+0000","last_unstale":"2024-09-10T09:59:05.282974+0000","last_undegraded":"2024-09-10T09:59:05.282974+0000","last_fullsized":"2024-09-10T09:59:05.282974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:46.302427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238455+0000","last_change":"2024-09-10T09:59:04.229435+0000","last_active":"2024-09-10T09:59:05.238455+0000","last_peered":"2024-09-10T09:59:05.238455+0000","last_clean":"2024-09-10T09:59:05.238455+0000","last_became_active":"2024-09-10T09:59:04.229158+0000","last_became_peered":"2024-09-10T09:59:04.229158+0000","last_unstale":"2024-09-10T09:59:05.238455+0000","last_undegraded":"2024-09-10T09:59:05.238455+0000","last_fullsized":"2024-09-10T09:59:05.238455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:48.367623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234803+0000","last_change":"2024-09-10T09:59:03.207503+0000","last_active":"2024-09-10T09:59:05.234803+0000","last_peered":"2024-09-10T09:59:05.234803+0000","last_clean":"2024-09-10T09:59:05.234803+0000","last_became_active":"2024-09-10T09:59:03.206392+0000","last_became_peered":"2024-09-10T09:59:03.206392+0000","last_unstale":"2024-09-10T09:59:05.234803+0000","last_undegraded":"2024-09-10T09:59:05.234803+0000","last_fullsized":"2024-09-10T09:59:05.234803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:17.627540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.234721+0000","last_change":"2024-09-10T09:59:04.231822+0000","last_active":"2024-09-10T09:59:05.234721+0000","last_peered":"2024-09-10T09:59:05.234721+0000","last_clean":"2024-09-10T09:59:05.234721+0000","last_became_active":"2024-09-10T09:59:04.231491+0000","last_became_peered":"2024-09-10T09:59:04.231491+0000","last_unstale":"2024-09-10T09:59:05.234721+0000","last_undegraded":"2024-09-10T09:59:05.234721+0000","last_fullsized":"2024-09-10T09:59:05.234721+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:30.880271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282146+0000","last_change":"2024-09-10T09:59:03.218500+0000","last_active":"2024-09-10T09:59:05.282146+0000","last_peered":"2024-09-10T09:59:05.282146+0000","last_clean":"2024-09-10T09:59:05.282146+0000","last_became_active":"2024-09-10T09:59:03.218191+0000","last_became_peered":"2024-09-10T09:59:03.218191+0000","last_unstale":"2024-09-10T09:59:05.282146+0000","last_undegraded":"2024-09-10T09:59:05.282146+0000","last_fullsized":"2024-09-10T09:59:05.282146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:49.036675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281192+0000","last_change":"2024-09-10T09:59:04.231281+0000","last_active":"2024-09-10T09:59:05.281192+0000","last_peered":"2024-09-10T09:59:05.281192+0000","last_clean":"2024-09-10T09:59:05.281192+0000","last_became_active":"2024-09-10T09:59:04.231070+0000","last_became_peered":"2024-09-10T09:59:04.231070+0000","last_unstale":"2024-09-10T09:59:05.281192+0000","last_undegraded":"2024-09-10T09:59:05.281192+0000","last_fullsized":"2024-09-10T09:59:05.281192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:34.062362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238657+0000","last_change":"2024-09-10T09:59:03.207218+0000","last_active":"2024-09-10T09:59:05.238657+0000","last_peered":"2024-09-10T09:59:05.238657+0000","last_clean":"2024-09-10T09:59:05.238657+0000","last_became_active":"2024-09-10T09:59:03.207032+0000","last_became_peered":"2024-09-10T09:59:03.207032+0000","last_unstale":"2024-09-10T09:59:05.238657+0000","last_undegraded":"2024-09-10T09:59:05.238657+0000","last_fullsized":"2024-09-10T09:59:05.238657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:46.302427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238564+0000","last_change":"2024-09-10T09:59:04.233898+0000","last_active":"2024-09-10T09:59:05.238564+0000","last_peered":"2024-09-10T09:59:05.238564+0000","last_clean":"2024-09-10T09:59:05.238564+0000","last_became_active":"2024-09-10T09:59:04.233583+0000","last_became_peered":"2024-09-10T09:59:04.233583+0000","last_unstale":"2024-09-10T09:59:05.238564+0000","last_undegraded":"2024-09-10T09:59:05.238564+0000","last_fullsized":"2024-09-10T09:59:05.238564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:30.880271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.281381+0000","last_change":"2024-09-10T09:59:03.208224+0000","last_active":"2024-09-10T09:59:05.281381+0000","last_peered":"2024-09-10T09:59:05.281381+0000","last_clean":"2024-09-10T09:59:05.281381+0000","last_became_active":"2024-09-10T09:59:03.207076+0000","last_became_peered":"2024-09-10T09:59:03.207076+0000","last_unstale":"2024-09-10T09:59:05.281381+0000","last_undegraded":"2024-09-10T09:59:05.281381+0000","last_fullsized":"2024-09-10T09:59:05.281381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:59.885497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.283110+0000","last_change":"2024-09-10T09:59:04.251728+0000","last_active":"2024-09-10T09:59:05.283110+0000","last_peered":"2024-09-10T09:59:05.283110+0000","last_clean":"2024-09-10T09:59:05.283110+0000","last_became_active":"2024-09-10T09:59:04.251481+0000","last_became_peered":"2024-09-10T09:59:04.251481+0000","last_unstale":"2024-09-10T09:59:05.283110+0000","last_undegraded":"2024-09-10T09:59:05.283110+0000","last_fullsized":"2024-09-10T09:59:05.283110+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:03.185155+0000","last_clean_scrub_stamp":"2024-09-10T09:59:03.185155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:54.301065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.238901+0000","last_change":"2024-09-10T09:59:03.219002+0000","last_active":"2024-09-10T09:59:05.238901+0000","last_peered":"2024-09-10T09:59:05.238901+0000","last_clean":"2024-09-10T09:59:05.238901+0000","last_became_active":"2024-09-10T09:59:03.218552+0000","last_became_peered":"2024-09-10T09:59:03.218552+0000","last_unstale":"2024-09-10T09:59:05.238901+0000","last_undegraded":"2024-09-10T09:59:05.238901+0000","last_fullsized":"2024-09-10T09:59:05.238901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:02.179205+0000","last_clean_scrub_stamp":"2024-09-10T09:59:02.179205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:11.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":190,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-10T09:59:05.282945+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:59:05.282945+0000","last_peered":"2024-09-10T09:59:05.282945+0000","last_clean":"2024-09-10T09:59:05.282945+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:59:05.282945+0000","last_undegraded":"2024-09-10T09:59:05.282945+0000","last_fullsized":"2024-09-10T09:59:05.282945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":734605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":144763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509453,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":144819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":144251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":380928,"data_stored":144217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":970752,"data_stored":734515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:59:13.189 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T09:59:13.189 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T09:59:13.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.367+0000 7fbc32c00640 1 Processor -- start 2024-09-10T09:59:13.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.367+0000 7fbc32c00640 1 -- start start 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc32c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc2c15c080 0x7fbc2c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc32c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fbc2c0b6fe0 0x7fbc2c15fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc32c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2c0b0c10 con 0x7fbc2c0720e0 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc32c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2c0b09c0 con 0x7fbc2c15c080 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc32c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2c06cb70 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc31800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc2c15c080 0x7fbc2c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc31800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc2c15c080 0x7fbc2c06c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46276/0 (socket says 172.21.15.110:46276) 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc30e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fbc2c0720e0 0x7fbc2c0b68d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52644/0 (socket says 172.21.15.110:52644) 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc31800640 1 -- 172.21.15.110:0/1110342399 learned_addr learned my addr 172.21.15.110:0/1110342399 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc32200640 1 --2- 172.21.15.110:0/1110342399 >> v2:172.21.15.145:3300/0 conn(0x7fbc2c0b6fe0 0x7fbc2c15fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc31800640 1 -- 172.21.15.110:0/1110342399 >> v2:172.21.15.145:3300/0 conn(0x7fbc2c0b6fe0 msgr2=0x7fbc2c15fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:13.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc31800640 1 --2- 172.21.15.110:0/1110342399 >> v2:172.21.15.145:3300/0 conn(0x7fbc2c0b6fe0 0x7fbc2c15fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc31800640 1 -- 172.21.15.110:0/1110342399 >> v1:172.21.15.110:6789/0 conn(0x7fbc2c0720e0 legacy=0x7fbc2c0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc31800640 1 -- 172.21.15.110:0/1110342399 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c0b0ef0 con 0x7fbc2c15c080 2024-09-10T09:59:13.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc2be00640 1 -- 172.21.15.110:0/1110342399 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180718402 0 0) 0x7fbc2c0b0c10 con 0x7fbc2c0720e0 2024-09-10T09:59:13.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.371+0000 7fbc31800640 1 --2- 172.21.15.110:0/1110342399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c15c080 0x7fbc2c06c630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0b4320 tx=0x7fbc2001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf8d554ab1bfe2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc2be00640 1 -- 172.21.15.110:0/1110342399 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200a5710 con 0x7fbc2c15c080 2024-09-10T09:59:13.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc2be00640 1 -- 172.21.15.110:0/1110342399 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc200a58b0 con 0x7fbc2c15c080 2024-09-10T09:59:13.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc2be00640 1 -- 172.21.15.110:0/1110342399 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200a5c80 con 0x7fbc2c15c080 2024-09-10T09:59:13.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- 172.21.15.110:0/1110342399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c15c080 msgr2=0x7fbc2c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 --2- 172.21.15.110:0/1110342399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c15c080 0x7fbc2c06c630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0b4320 tx=0x7fbc2001c960 comp rx=0 tx=0).stop 2024-09-10T09:59:13.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- 172.21.15.110:0/1110342399 shutdown_connections 2024-09-10T09:59:13.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 --2- 172.21.15.110:0/1110342399 >> v2:172.21.15.145:3300/0 conn(0x7fbc2c0b6fe0 0x7fbc2c15fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 --2- 172.21.15.110:0/1110342399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c15c080 0x7fbc2c06c630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- 172.21.15.110:0/1110342399 >> 172.21.15.110:0/1110342399 conn(0x7fbc2c06cd60 msgr2=0x7fbc2c06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:13.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- 172.21.15.110:0/1110342399 shutdown_connections 2024-09-10T09:59:13.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- 172.21.15.110:0/1110342399 wait complete. 2024-09-10T09:59:13.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 Processor -- start 2024-09-10T09:59:13.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- start start 2024-09-10T09:59:13.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc2c0720e0 0x7fbc2c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c0b6fe0 0x7fbc2c0d0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2c15c080 0x7fbc2c143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c15c5e0 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c15c460 con 0x7fbc2c15c080 2024-09-10T09:59:13.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c06e290 con 0x7fbc2c0720e0 2024-09-10T09:59:13.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc30e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c0b6fe0 0x7fbc2c0d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc30e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c0b6fe0 0x7fbc2c0d0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46282/0 (socket says 172.21.15.110:46282) 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc30e00640 1 -- 172.21.15.110:0/3408374205 learned_addr learned my addr 172.21.15.110:0/3408374205 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc31800640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc2c0720e0 0x7fbc2c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.375+0000 7fbc32200640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2c15c080 0x7fbc2c143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc30e00640 1 -- 172.21.15.110:0/3408374205 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc2c0720e0 msgr2=0x7fbc2c0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc30e00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc2c0720e0 0x7fbc2c0d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc30e00640 1 -- 172.21.15.110:0/3408374205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2c15c080 msgr2=0x7fbc2c143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc30e00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2c15c080 0x7fbc2c143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc30e00640 1 -- 172.21.15.110:0/3408374205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c0b3560 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc30e00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c0b6fe0 0x7fbc2c0d0630 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800c4b0 tx=0x7fbc1800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc29600640 1 -- 172.21.15.110:0/3408374205 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18018070 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc29600640 1 -- 172.21.15.110:0/3408374205 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc18007500 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c14c730 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc29600640 1 -- 172.21.15.110:0/3408374205 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18004a30 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c14ccf0 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c15fde0 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc29600640 1 -- 172.21.15.110:0/3408374205 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbc18004bd0 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc29600640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbc0c0678c0 0x7fbc0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc29600640 1 -- 172.21.15.110:0/3408374205 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbc180850a0 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.379+0000 7fbc31800640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbc0c0678c0 0x7fbc0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.383+0000 7fbc31800640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbc0c0678c0 0x7fbc0c069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc200ae000 tx=0x7fbc200a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.387+0000 7fbc29600640 1 -- 172.21.15.110:0/3408374205 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbc18055530 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.599+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbc2c151f10 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.599+0000 7fbc29600640 1 -- 172.21.15.110:0/3408374205 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fbc18059830 con 0x7fbc2c0b6fe0 2024-09-10T09:59:13.605 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T09:59:13.605 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T09:59:13.605 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T09:59:13.605 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T09:59:13.605 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T09:59:13.605 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T09:59:13.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbc0c0678c0 msgr2=0x7fbc0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbc0c0678c0 0x7fbc0c069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc200ae000 tx=0x7fbc200a7070 comp rx=0 tx=0).stop 2024-09-10T09:59:13.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c0b6fe0 msgr2=0x7fbc2c0d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c0b6fe0 0x7fbc2c0d0630 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800c4b0 tx=0x7fbc1800c970 comp rx=0 tx=0).stop 2024-09-10T09:59:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 shutdown_connections 2024-09-10T09:59:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbc0c0678c0 0x7fbc0c069d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc2c15c080 0x7fbc2c143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc2c0b6fe0 0x7fbc2c0d0630 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 --2- 172.21.15.110:0/3408374205 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc2c0720e0 0x7fbc2c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 >> 172.21.15.110:0/3408374205 conn(0x7fbc2c06cd60 msgr2=0x7fbc2c0ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 shutdown_connections 2024-09-10T09:59:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.603+0000 7fbc32c00640 1 -- 172.21.15.110:0/3408374205 wait complete. 2024-09-10T09:59:13.627 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d8b800640 1 Processor -- start 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d8b800640 1 -- start start 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d8b800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6d8415c960 0x7f6d840b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d8b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d84070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d8b800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d840706a0 con 0x7f6d841603c0 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d8b800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8415cd40 con 0x7f6d8415c960 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d8b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6d840b6370 con 0x7f6d840d13e0 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d88e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6d8415c960 0x7f6d840b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d88e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6d8415c960 0x7f6d840b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46298/0 (socket says 172.21.15.110:46298) 2024-09-10T09:59:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d88e00640 1 -- 172.21.15.110:0/669999838 learned_addr learned my addr 172.21.15.110:0/669999838 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:13.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d89800640 1 --2- 172.21.15.110:0/669999838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d84070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d88e00640 1 -- 172.21.15.110:0/669999838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 msgr2=0x7f6d84070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:13.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d88e00640 1 --2- 172.21.15.110:0/669999838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d84070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d88e00640 1 -- 172.21.15.110:0/669999838 >> v1:172.21.15.110:6789/0 conn(0x7f6d841603c0 legacy=0x7f6d840d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d88e00640 1 -- 172.21.15.110:0/669999838 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d84070a80 con 0x7f6d8415c960 2024-09-10T09:59:13.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.759+0000 7f6d89800640 1 --2- 172.21.15.110:0/669999838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d84070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:13.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d88e00640 1 --2- 172.21.15.110:0/669999838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d840b5e30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6d840b64f0 tx=0x7f6d7401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c14729b14911f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d83400640 1 -- 172.21.15.110:0/669999838 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740ab5f0 con 0x7f6d8415c960 2024-09-10T09:59:13.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d83400640 1 -- 172.21.15.110:0/669999838 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d740b0070 con 0x7f6d8415c960 2024-09-10T09:59:13.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d83400640 1 -- 172.21.15.110:0/669999838 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740ab9b0 con 0x7f6d8415c960 2024-09-10T09:59:13.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 -- 172.21.15.110:0/669999838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 msgr2=0x7f6d840b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 --2- 172.21.15.110:0/669999838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d840b5e30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6d840b64f0 tx=0x7f6d7401c3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 -- 172.21.15.110:0/669999838 shutdown_connections 2024-09-10T09:59:13.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 --2- 172.21.15.110:0/669999838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d84070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 --2- 172.21.15.110:0/669999838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d840b5e30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 -- 172.21.15.110:0/669999838 >> 172.21.15.110:0/669999838 conn(0x7f6d840d25c0 msgr2=0x7f6d840d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:13.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 -- 172.21.15.110:0/669999838 shutdown_connections 2024-09-10T09:59:13.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 -- 172.21.15.110:0/669999838 wait complete. 2024-09-10T09:59:13.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.763+0000 7f6d8b800640 1 Processor -- start 2024-09-10T09:59:13.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 -- start start 2024-09-10T09:59:13.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d841473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d84159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d841603c0 0x7f6d84151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6d840d3150 con 0x7f6d8415c960 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d841611b0 con 0x7f6d841603c0 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6d840d32d0 con 0x7f6d840d13e0 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d84159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d88e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d841473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d84159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46300/0 (socket says 172.21.15.110:46300) 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 -- 172.21.15.110:0/4046150298 learned_addr learned my addr 172.21.15.110:0/4046150298 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d89800640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d841603c0 0x7f6d84151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 -- 172.21.15.110:0/4046150298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 msgr2=0x7f6d841473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d841473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 -- 172.21.15.110:0/4046150298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d841603c0 msgr2=0x7f6d84151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d841603c0 0x7f6d84151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 -- 172.21.15.110:0/4046150298 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d841f5d90 con 0x7f6d8415c960 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d89800640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d841603c0 0x7f6d84151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d83e00640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d84159570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c370 tx=0x7f6d7000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d80c00640 1 -- 172.21.15.110:0/4046150298 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d700052d0 con 0x7f6d8415c960 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d80c00640 1 -- 172.21.15.110:0/4046150298 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d70005470 con 0x7f6d8415c960 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d841f6070 con 0x7f6d8415c960 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d80c00640 1 -- 172.21.15.110:0/4046150298 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70007260 con 0x7f6d8415c960 2024-09-10T09:59:13.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d841f6630 con 0x7f6d8415c960 2024-09-10T09:59:13.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.767+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d840d34d0 con 0x7f6d8415c960 2024-09-10T09:59:13.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.771+0000 7f6d80c00640 1 -- 172.21.15.110:0/4046150298 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d70007400 con 0x7f6d8415c960 2024-09-10T09:59:13.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.771+0000 7f6d80c00640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6d640678c0 0x7f6d64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:13.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.771+0000 7f6d80c00640 1 -- 172.21.15.110:0/4046150298 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6d700850c0 con 0x7f6d8415c960 2024-09-10T09:59:13.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.771+0000 7f6d88e00640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6d640678c0 0x7f6d64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:13.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.771+0000 7f6d88e00640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6d640678c0 0x7f6d64069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d740af000 tx=0x7f6d740a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:13.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.775+0000 7f6d80c00640 1 -- 172.21.15.110:0/4046150298 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6d70055510 con 0x7f6d8415c960 2024-09-10T09:59:13.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.963+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d840b6f10 con 0x7f6d8415c960 2024-09-10T09:59:13.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.963+0000 7f6d80c00640 1 -- 172.21.15.110:0/4046150298 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7f6d70059810 con 0x7f6d8415c960 2024-09-10T09:59:13.967 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:13.967 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":16,"btime":"2024-09-10T09:59:07:250101+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4556,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1141773053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1141773053},{"type":"v1","addr":"172.21.15.145:6834","nonce":1141773053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/656217845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":656217845},{"type":"v1","addr":"172.21.15.145:6835","nonce":656217845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2244539155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2244539155},{"type":"v1","addr":"172.21.15.110:6835","nonce":2244539155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:03.573094+0000","modified":"2024-09-10T09:59:06.627577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1972132715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1972132715},{"type":"v1","addr":"172.21.15.110:6837","nonce":1972132715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-09-10T09:59:13.968 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 16 2024-09-10T09:59:13.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6d640678c0 msgr2=0x7f6d64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6d640678c0 0x7f6d64069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d740af000 tx=0x7f6d740a7000 comp rx=0 tx=0).stop 2024-09-10T09:59:13.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 msgr2=0x7f6d84159570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:13.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d84159570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c370 tx=0x7f6d7000c830 comp rx=0 tx=0).stop 2024-09-10T09:59:13.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 shutdown_connections 2024-09-10T09:59:13.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6d640678c0 0x7f6d64069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d841603c0 0x7f6d84151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d8415c960 0x7f6d84159570 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 --2- 172.21.15.110:0/4046150298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d840d13e0 0x7f6d841473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:13.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 >> 172.21.15.110:0/4046150298 conn(0x7f6d840d25c0 msgr2=0x7f6d84153160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:13.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 shutdown_connections 2024-09-10T09:59:13.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:13.967+0000 7f6d8b800640 1 -- 172.21.15.110:0/4046150298 wait complete. 2024-09-10T09:59:14.392 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T09:59:03.573094+0000', 'modified': '2024-09-10T09:59:06.627577+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4776}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4776': {'gid': 4776, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6837/1972132715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 1972132715}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 1972132715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4776, 'qdb_cluster': [4776]} 2024-09-10T09:59:14.392 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T09:59:14.392 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T09:59:14.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.523+0000 7f0005200640 1 Processor -- start 2024-09-10T09:59:14.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.523+0000 7f0005200640 1 -- start start 2024-09-10T09:59:14.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7f0005200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f00000b4ce0 0x7f0000165690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7f0005200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000015ae90 con 0x7f000015aab0 2024-09-10T09:59:14.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7f0005200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000015b170 con 0x7f00000b4170 2024-09-10T09:59:14.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7f0005200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f000015b450 con 0x7f00000b4ce0 2024-09-10T09:59:14.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7efffde00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f000015aab0 0x7f0000072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52660/0 (socket says 172.21.15.110:52660) 2024-09-10T09:59:14.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7efffde00640 1 -- 172.21.15.110:0/659230006 learned_addr learned my addr 172.21.15.110:0/659230006 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:14.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7efffe800640 1 --2- 172.21.15.110:0/659230006 >> v2:172.21.15.110:3300/0 conn(0x7f00000b4ce0 0x7f0000165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7efffe800640 1 -- 172.21.15.110:0/659230006 >> v1:172.21.15.145:6789/0 conn(0x7f00000b4170 legacy=0x7f00000b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7efffe800640 1 -- 172.21.15.110:0/659230006 >> v1:172.21.15.110:6789/0 conn(0x7f000015aab0 legacy=0x7f0000072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7efffe800640 1 -- 172.21.15.110:0/659230006 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000015b730 con 0x7f00000b4ce0 2024-09-10T09:59:14.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.527+0000 7efffca00640 1 -- 172.21.15.110:0/659230006 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2564687352 0 0) 0x7f000015ae90 con 0x7f000015aab0 2024-09-10T09:59:14.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7efffe800640 1 --2- 172.21.15.110:0/659230006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4ce0 0x7f0000165690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efff800a960 tx=0x7efff801d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f721b12daded12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7efffca00640 1 -- 172.21.15.110:0/659230006 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff80b3070 con 0x7f00000b4ce0 2024-09-10T09:59:14.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7efffca00640 1 -- 172.21.15.110:0/659230006 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff80a6750 con 0x7f00000b4ce0 2024-09-10T09:59:14.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7efffca00640 1 -- 172.21.15.110:0/659230006 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff800b040 con 0x7f00000b4ce0 2024-09-10T09:59:14.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 -- 172.21.15.110:0/659230006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4ce0 msgr2=0x7f0000165690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 --2- 172.21.15.110:0/659230006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4ce0 0x7f0000165690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efff800a960 tx=0x7efff801d830 comp rx=0 tx=0).stop 2024-09-10T09:59:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 -- 172.21.15.110:0/659230006 shutdown_connections 2024-09-10T09:59:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 --2- 172.21.15.110:0/659230006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4ce0 0x7f0000165690 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 -- 172.21.15.110:0/659230006 >> 172.21.15.110:0/659230006 conn(0x7f0000062d10 msgr2=0x7f000006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 -- 172.21.15.110:0/659230006 shutdown_connections 2024-09-10T09:59:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 -- 172.21.15.110:0/659230006 wait complete. 2024-09-10T09:59:14.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 Processor -- start 2024-09-10T09:59:14.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.531+0000 7f0005200640 1 -- start start 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7f0005200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4170 0x7f00000b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7f0005200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00000b4ce0 0x7f000014f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7f0005200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000015aab0 0x7f00000ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7f0005200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0000167f70 con 0x7f00000b4170 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7f0005200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0000167df0 con 0x7f00000b4ce0 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7f0005200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f00001680f0 con 0x7f000015aab0 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4170 0x7f00000b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4170 0x7f00000b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46318/0 (socket says 172.21.15.110:46318) 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 -- 172.21.15.110:0/609489765 learned_addr learned my addr 172.21.15.110:0/609489765 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffd400640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00000b4ce0 0x7f000014f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffe800640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000015aab0 0x7f00000ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 -- 172.21.15.110:0/609489765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000015aab0 msgr2=0x7f00000ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000015aab0 0x7f00000ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 -- 172.21.15.110:0/609489765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00000b4ce0 msgr2=0x7f000014f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00000b4ce0 0x7f000014f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 -- 172.21.15.110:0/609489765 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00000b1cc0 con 0x7f00000b4170 2024-09-10T09:59:14.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efffde00640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4170 0x7f00000b07c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efff000c7c0 tx=0x7efff000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efff6000640 1 -- 172.21.15.110:0/609489765 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0018070 con 0x7f00000b4170 2024-09-10T09:59:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efff6000640 1 -- 172.21.15.110:0/609489765 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff000a640 con 0x7f00000b4170 2024-09-10T09:59:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7efff6000640 1 -- 172.21.15.110:0/609489765 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff000aa30 con 0x7f00000b4170 2024-09-10T09:59:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7f0005200640 1 -- 172.21.15.110:0/609489765 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00000b1700 con 0x7f00000b4170 2024-09-10T09:59:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.535+0000 7f0005200640 1 -- 172.21.15.110:0/609489765 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00000affe0 con 0x7f00000b4170 2024-09-10T09:59:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efff6000640 1 -- 172.21.15.110:0/609489765 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efff000abd0 con 0x7f00000b4170 2024-09-10T09:59:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efff6000640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7effe00678c0 0x7effe0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efff6000640 1 -- 172.21.15.110:0/609489765 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7efff00853c0 con 0x7f00000b4170 2024-09-10T09:59:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7f000865f000 1 Processor -- start 2024-09-10T09:59:14.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7effe00678c0 0x7effe0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7f000865f000 1 -- start start 2024-09-10T09:59:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7effe00678c0 0x7effe0069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0000155ab0 tx=0x7effec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7f000865f000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450357f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7f000865f000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a45030dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7f000865f000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a45030e290 0x55a45030e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7f000865f000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a4501f5b60 con 0x55a450358440 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7f000865f000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a45015cc70 con 0x55a450357b20 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7f000865f000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55a450102ed0 con 0x55a45030e290 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a45030dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a45030dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46324/0 (socket says 172.21.15.110:46324) 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 -- 172.21.15.110:0/1676969096 learned_addr learned my addr 172.21.15.110:0/1676969096 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffe800640 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a45030e290 0x55a45030e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffde00640 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450357f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 -- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a45030e290 msgr2=0x55a45030e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a45030e290 0x55a45030e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 -- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 msgr2=0x55a450357f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450357f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 -- 172.21.15.110:0/1676969096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a450114e50 con 0x55a450358440 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffde00640 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450357f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7efffd400640 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a45030dd50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7effec0055a0 tx=0x7effec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a71505120db48d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7effeaa00640 1 -- 172.21.15.110:0/1676969096 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec025070 con 0x55a450358440 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7effeaa00640 1 -- 172.21.15.110:0/1676969096 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effec015800 con 0x55a450358440 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.539+0000 7effeaa00640 1 -- 172.21.15.110:0/1676969096 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec015bb0 con 0x55a450358440 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- 172.21.15.110:0/1676969096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 msgr2=0x55a45030dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a45030dd50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7effec0055a0 tx=0x7effec011000 comp rx=0 tx=0).stop 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- 172.21.15.110:0/1676969096 shutdown_connections 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a45030e290 0x55a45030e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a45030dd50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 --2- 172.21.15.110:0/1676969096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450357f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- 172.21.15.110:0/1676969096 >> 172.21.15.110:0/1676969096 conn(0x55a450314600 msgr2=0x55a45030c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- 172.21.15.110:0/1676969096 shutdown_connections 2024-09-10T09:59:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- 172.21.15.110:0/1676969096 wait complete. 2024-09-10T09:59:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- start start 2024-09-10T09:59:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 Processor -- start 2024-09-10T09:59:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450312e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450358440 0x55a450318cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a450319230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a4501beb00 con 0x55a450313390 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a450137360 con 0x55a450357b20 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55a4501342d0 con 0x55a450358440 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 10 client.0 _ll_register_callbacks cb 0x55a4502f9820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a450319230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a450319230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46326/0 (socket says 172.21.15.110:46326) 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 -- 172.21.15.110:0/3351280684 learned_addr learned my addr 172.21.15.110:0/3351280684 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffd400640 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450358440 0x55a450318cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffde00640 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450312e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 -- 172.21.15.110:0/3351280684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450358440 msgr2=0x55a450318cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450358440 0x55a450318cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 -- 172.21.15.110:0/3351280684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 msgr2=0x55a450312e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450312e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 -- 172.21.15.110:0/3351280684 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a450103120 con 0x55a450313390 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7efffe800640 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a450319230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efff801b460 tx=0x7efff80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb674cab081a77d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7effdf400640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7effdf400640 1 -- 172.21.15.110:0/3351280684 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff80b3040 con 0x55a450313390 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- 172.21.15.110:0/3351280684 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effec015800 con 0x55a450313390 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7effdf400640 1 -- 172.21.15.110:0/3351280684 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7efff8003820 con 0x55a450313390 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7effdf400640 1 -- 172.21.15.110:0/3351280684 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7efff80c7050 con 0x55a450313390 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 10 client.4968 fetch_fsmap learned FSMap version 16 2024-09-10T09:59:14.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.543+0000 7f000865f000 1 -- 172.21.15.110:0/3351280684 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a4500fc450 con 0x55a450313390 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7effdf400640 1 -- 172.21.15.110:0/3351280684 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effec015800 con 0x55a450313390 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7effdf400640 1 -- 172.21.15.110:0/3351280684 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7efff80ca020 con 0x55a450313390 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 10 client.4968 fetch_fsmap finished waiting for FSMap version 16 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 10 client.4968 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 20 client.4968 populate_metadata read hostname 'smithi110' 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] conn(0x55a450366280 0x55a450368670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 4 client.4968 mds_command: new command op to 4776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 -- 172.21.15.110:0/3351280684 --> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4500f6c00 con 0x55a450366280 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7efffde00640 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] conn(0x55a450366280 0x55a450368670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7efffde00640 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] conn(0x55a450366280 0x55a450368670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a4b42a27cdbbd47 server_cookie=e53df178f67d3a4 in_seq=0 out_seq=0 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7effdf400640 10 client.4968 ms_handle_connect on v2:172.21.15.110:6836/1972132715 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7effdf400640 1 -- 172.21.15.110:0/3351280684 <== mds.0 v2:172.21.15.110:6836/1972132715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4500f6c00 con 0x55a450366280 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7effdf400640 10 client.4968 handle_command_reply: tid=0 2024-09-10T09:59:14.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 client.4968 shutdown 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 -- 172.21.15.110:0/3351280684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 msgr2=0x55a450319230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a450319230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efff801b460 tx=0x7efff80046d0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 -- 172.21.15.110:0/3351280684 shutdown_connections 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] conn(0x55a450366280 0x55a450368670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a450319230 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450358440 0x55a450318cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 --2- 172.21.15.110:0/3351280684 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450357b20 0x55a450312e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 -- 172.21.15.110:0/3351280684 >> 172.21.15.110:0/3351280684 conn(0x55a45030e290 msgr2=0x55a45030cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7effdf400640 0 client.4968 ms_handle_reset on v2:172.21.15.110:6836/1972132715 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 -- 172.21.15.110:0/3351280684 shutdown_connections 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 1 -- 172.21.15.110:0/3351280684 wait complete. 2024-09-10T09:59:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.547+0000 7f000865f000 20 client.4968 trim_cache size 0 max 16384 2024-09-10T09:59:14.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 Processor -- start 2024-09-10T09:59:14.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- start start 2024-09-10T09:59:14.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450313390 0x55a450356cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a450357230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a450368670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0000156fa0 con 0x55a450358440 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0000156300 con 0x55a450313390 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0000156950 con 0x55a450357b20 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a450368670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a450368670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46338/0 (socket says 172.21.15.110:46338) 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 -- 172.21.15.110:0/952963301 learned_addr learned my addr 172.21.15.110:0/952963301 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffde00640 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450313390 0x55a450356cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffd400640 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a450357230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 -- 172.21.15.110:0/952963301 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 msgr2=0x55a450357230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a450357230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 -- 172.21.15.110:0/952963301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450313390 msgr2=0x55a450356cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450313390 0x55a450356cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 -- 172.21.15.110:0/952963301 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4500e3fd0 con 0x55a450358440 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7efffe800640 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a450368670 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efff8004780 tx=0x7efff80a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f3f9955d3f64e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7effea000640 1 -- 172.21.15.110:0/952963301 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff80b3040 con 0x55a450358440 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7effea000640 1 -- 172.21.15.110:0/952963301 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff8000e50 con 0x55a450358440 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7effea000640 1 -- 172.21.15.110:0/952963301 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8003e10 con 0x55a450358440 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- 172.21.15.110:0/952963301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 msgr2=0x55a450368670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a450368670 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efff8004780 tx=0x7efff80a8000 comp rx=0 tx=0).stop 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- 172.21.15.110:0/952963301 shutdown_connections 2024-09-10T09:59:14.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450358440 0x55a450368670 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a450357230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 --2- 172.21.15.110:0/952963301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450313390 0x55a450356cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- 172.21.15.110:0/952963301 >> 172.21.15.110:0/952963301 conn(0x55a45030e290 msgr2=0x55a450312650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- 172.21.15.110:0/952963301 shutdown_connections 2024-09-10T09:59:14.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- 172.21.15.110:0/952963301 wait complete. 2024-09-10T09:59:14.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.567+0000 7f000865f000 1 -- start start 2024-09-10T09:59:14.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 Processor -- start 2024-09-10T09:59:14.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a45036b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a4503d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450358440 0x55a4503d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a45015bf80 con 0x55a450313390 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a450114ca0 con 0x55a450358440 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55a4501612a0 con 0x55a450357b20 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 10 client.0 _ll_register_callbacks cb 0x7efff80b1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a45036b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a45036b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46350/0 (socket says 172.21.15.110:46350) 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 -- 172.21.15.110:0/4082965787 learned_addr learned my addr 172.21.15.110:0/4082965787 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffd400640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a4503d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffe800640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450358440 0x55a4503d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 -- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 msgr2=0x55a4503d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a4503d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 -- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450358440 msgr2=0x55a4503d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450358440 0x55a4503d2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 -- 172.21.15.110:0/4082965787 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4501df750 con 0x55a450313390 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffd400640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a4503d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffe800640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450358440 0x55a4503d2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffde00640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a45036b590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efff000cfd0 tx=0x7efff000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8bdb9bfdd2223a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 1 -- 172.21.15.110:0/4082965787 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff005e340 con 0x55a450313390 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 1 -- 172.21.15.110:0/4082965787 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7efff00850b0 con 0x55a450313390 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- 172.21.15.110:0/4082965787 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff8000e50 con 0x55a450313390 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 1 -- 172.21.15.110:0/4082965787 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7efff0051dd0 con 0x55a450313390 2024-09-10T09:59:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 10 client.4974 fetch_fsmap learned FSMap version 16 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- 172.21.15.110:0/4082965787 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a4500eaa30 con 0x55a450313390 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 1 -- 172.21.15.110:0/4082965787 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff8000e50 con 0x55a450313390 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 1 -- 172.21.15.110:0/4082965787 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7efff0054bb0 con 0x55a450313390 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 10 client.4974 fetch_fsmap finished waiting for FSMap version 16 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 10 client.4974 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 20 client.4974 populate_metadata read hostname 'smithi110' 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] conn(0x55a45038f170 0x55a45038f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 4 client.4974 mds_command: new command op to 4776 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- 172.21.15.110:0/4082965787 --> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] -- command(tid 0: {"prefix": "status"}) -- 0x55a45016a3a0 con 0x55a45038f170 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffd400640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] conn(0x55a45038f170 0x55a45038f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7efffd400640 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] conn(0x55a45038f170 0x55a45038f550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5337248745d8da3 server_cookie=436c5dadf1442868 in_seq=0 out_seq=0 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 10 client.4974 ms_handle_connect on v2:172.21.15.110:6836/1972132715 2024-09-10T09:59:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 1 -- 172.21.15.110:0/4082965787 <== mds.0 v2:172.21.15.110:6836/1972132715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a45016a3a0 con 0x55a45038f170 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 10 client.4974 handle_command_reply: tid=0 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 client.4974 shutdown 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- 172.21.15.110:0/4082965787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 msgr2=0x55a45036b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a45036b590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efff000cfd0 tx=0x7efff000c740 comp rx=0 tx=0).stop 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- 172.21.15.110:0/4082965787 shutdown_connections 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.110:6836/1972132715,v1:172.21.15.110:6837/1972132715] conn(0x55a45038f170 0x55a45038f550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a450358440 0x55a4503d2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55a450357b20 0x55a4503d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 --2- 172.21.15.110:0/4082965787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a450313390 0x55a45036b590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- 172.21.15.110:0/4082965787 >> 172.21.15.110:0/4082965787 conn(0x55a45030e290 msgr2=0x55a450355ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7effdf400640 0 client.4974 ms_handle_reset on v2:172.21.15.110:6836/1972132715 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.571+0000 7f000865f000 1 -- 172.21.15.110:0/4082965787 shutdown_connections 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f000865f000 1 -- 172.21.15.110:0/4082965787 wait complete. 2024-09-10T09:59:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f000865f000 20 client.4974 trim_cache size 0 max 16384 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 4776, 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 10.366776550000001, 2024-09-10T09:59:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 16, 2024-09-10T09:59:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 34, 2024-09-10T09:59:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 34, 2024-09-10T09:59:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 17.223397521999999 2024-09-10T09:59:14.588 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T09:59:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 -- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7effe00678c0 msgr2=0x7effe0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7effe00678c0 0x7effe0069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0000155ab0 tx=0x7effec009040 comp rx=0 tx=0).stop 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 -- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4170 msgr2=0x7f00000b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4170 0x7f00000b07c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efff000c7c0 tx=0x7efff000cc80 comp rx=0 tx=0).stop 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 -- 172.21.15.110:0/609489765 shutdown_connections 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7effe00678c0 0x7effe0069d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000015aab0 0x7f00000ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00000b4ce0 0x7f000014f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 --2- 172.21.15.110:0/609489765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00000b4170 0x7f00000b07c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 -- 172.21.15.110:0/609489765 >> 172.21.15.110:0/609489765 conn(0x7f0000062d10 msgr2=0x7f0000159690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 -- 172.21.15.110:0/609489765 shutdown_connections 2024-09-10T09:59:14.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.575+0000 7f0005200640 1 -- 172.21.15.110:0/609489765 wait complete. 2024-09-10T09:59:14.597 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T09:59:14.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.727+0000 7f62d4a00640 1 Processor -- start 2024-09-10T09:59:14.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.727+0000 7f62d4a00640 1 -- start start 2024-09-10T09:59:14.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62d4a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f62d015e700 0x7f62d006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62d4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d006ff60 0x7f62d00af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62d4a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d00703a0 con 0x7f62d00729b0 2024-09-10T09:59:14.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62d4a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f62d015eae0 con 0x7f62d015e700 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62d4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f62d006eec0 con 0x7f62d006ff60 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f62d015e700 0x7f62d006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f62d015e700 0x7f62d006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46364/0 (socket says 172.21.15.110:46364) 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cde00640 1 -- 172.21.15.110:0/275464790 learned_addr learned my addr 172.21.15.110:0/275464790 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62ce800640 1 --1- 172.21.15.110:0/275464790 >> v1:172.21.15.110:6789/0 conn(0x7f62d00729b0 0x7f62d00b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52668/0 (socket says 172.21.15.110:52668) 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cd400640 1 --2- 172.21.15.110:0/275464790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d006ff60 0x7f62d00af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cde00640 1 -- 172.21.15.110:0/275464790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d006ff60 msgr2=0x7f62d00af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cde00640 1 --2- 172.21.15.110:0/275464790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d006ff60 0x7f62d00af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cde00640 1 -- 172.21.15.110:0/275464790 >> v1:172.21.15.110:6789/0 conn(0x7f62d00729b0 legacy=0x7f62d00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cde00640 1 -- 172.21.15.110:0/275464790 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d006f1a0 con 0x7f62d015e700 2024-09-10T09:59:14.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.731+0000 7f62cd400640 1 --2- 172.21.15.110:0/275464790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d006ff60 0x7f62d00af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:14.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62cde00640 1 --2- 172.21.15.110:0/275464790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d015e700 0x7f62d006e890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009870 tx=0x7f62c001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff12a6ea46e7a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62cca00640 1 -- 172.21.15.110:0/275464790 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00a95f0 con 0x7f62d015e700 2024-09-10T09:59:14.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62cca00640 1 -- 172.21.15.110:0/275464790 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c00a9790 con 0x7f62d015e700 2024-09-10T09:59:14.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62cca00640 1 -- 172.21.15.110:0/275464790 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00a9b60 con 0x7f62d015e700 2024-09-10T09:59:14.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- 172.21.15.110:0/275464790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d015e700 msgr2=0x7f62d006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 --2- 172.21.15.110:0/275464790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d015e700 0x7f62d006e890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009870 tx=0x7f62c001c530 comp rx=0 tx=0).stop 2024-09-10T09:59:14.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- 172.21.15.110:0/275464790 shutdown_connections 2024-09-10T09:59:14.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 --2- 172.21.15.110:0/275464790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d006ff60 0x7f62d00af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 --2- 172.21.15.110:0/275464790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d015e700 0x7f62d006e890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- 172.21.15.110:0/275464790 >> 172.21.15.110:0/275464790 conn(0x7f62d006cb90 msgr2=0x7f62d006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- 172.21.15.110:0/275464790 shutdown_connections 2024-09-10T09:59:14.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- 172.21.15.110:0/275464790 wait complete. 2024-09-10T09:59:14.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 Processor -- start 2024-09-10T09:59:14.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- start start 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d006ff60 0x7f62d014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d00729b0 0x7f62d00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62d015e700 0x7f62d00719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62d00731d0 con 0x7f62d006ff60 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62d00af7b0 con 0x7f62d015e700 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62d4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f62d014bcd0 con 0x7f62d00729b0 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.735+0000 7f62cde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d006ff60 0x7f62d014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d006ff60 0x7f62d014b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46372/0 (socket says 172.21.15.110:46372) 2024-09-10T09:59:14.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cde00640 1 -- 172.21.15.110:0/2817893104 learned_addr learned my addr 172.21.15.110:0/2817893104 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cd400640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d00729b0 0x7f62d00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62ce800640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62d015e700 0x7f62d00719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cde00640 1 -- 172.21.15.110:0/2817893104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d00729b0 msgr2=0x7f62d00cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cde00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d00729b0 0x7f62d00cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cde00640 1 -- 172.21.15.110:0/2817893104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62d015e700 msgr2=0x7f62d00719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cde00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62d015e700 0x7f62d00719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cde00640 1 -- 172.21.15.110:0/2817893104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d0153410 con 0x7f62d006ff60 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62cde00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d006ff60 0x7f62d014b790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ab040 tx=0x7f62c001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62c6000640 1 -- 172.21.15.110:0/2817893104 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00b1800 con 0x7f62d006ff60 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62c6000640 1 -- 172.21.15.110:0/2817893104 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c00b19c0 con 0x7f62d006ff60 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d01536f0 con 0x7f62d006ff60 2024-09-10T09:59:14.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62c6000640 1 -- 172.21.15.110:0/2817893104 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00b1db0 con 0x7f62d006ff60 2024-09-10T09:59:14.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d0153cb0 con 0x7f62d006ff60 2024-09-10T09:59:14.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d00b4970 con 0x7f62d006ff60 2024-09-10T09:59:14.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62c6000640 1 -- 172.21.15.110:0/2817893104 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f62c00bc450 con 0x7f62d006ff60 2024-09-10T09:59:14.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62c6000640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f62b00678f0 0x7f62b0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.739+0000 7f62c6000640 1 -- 172.21.15.110:0/2817893104 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f62c01207b0 con 0x7f62d006ff60 2024-09-10T09:59:14.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.743+0000 7f62cd400640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f62b00678f0 0x7f62b0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.743+0000 7f62cd400640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f62b00678f0 0x7f62b0069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f62d0154890 tx=0x7f62bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.743+0000 7f62c6000640 1 -- 172.21.15.110:0/2817893104 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f62c00ad020 con 0x7f62d006ff60 2024-09-10T09:59:14.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.935+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62d00afb00 con 0x7f62d006ff60 2024-09-10T09:59:14.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.935+0000 7f62c6000640 1 -- 172.21.15.110:0/2817893104 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f62c0016150 con 0x7f62d006ff60 2024-09-10T09:59:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f62b00678f0 msgr2=0x7f62b0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f62b00678f0 0x7f62b0069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f62d0154890 tx=0x7f62bc008040 comp rx=0 tx=0).stop 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d006ff60 msgr2=0x7f62d014b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d006ff60 0x7f62d014b790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ab040 tx=0x7f62c001ad70 comp rx=0 tx=0).stop 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 shutdown_connections 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f62b00678f0 0x7f62b0069db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62d015e700 0x7f62d00719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62d00729b0 0x7f62d00cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 --2- 172.21.15.110:0/2817893104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62d006ff60 0x7f62d014b790 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 >> 172.21.15.110:0/2817893104 conn(0x7f62d006cb90 msgr2=0x7f62d015ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 shutdown_connections 2024-09-10T09:59:14.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:14.939+0000 7f62d4a00640 1 -- 172.21.15.110:0/2817893104 wait complete. 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:59:14.962 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:59:14.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:14.963 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T09:59:14.972 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:59:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:59:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:59:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42420sec preferred_lft 42420sec 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:14.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:14.976 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:59:15.033 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T09:59:15.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:15.036 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T09:59:15.091 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T09:59:15.091 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:59:15.091 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T09:59:15.092 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:59:15.092 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T09:59:15.092 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T09:59:15.092 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:59:15.092 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:15.092 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:15.092 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:15.092 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:59:15.092 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:59:15.093 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:59:15.093 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:59:15.093 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:59:15.093 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:59:15.093 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:59:15.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:59:15.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:15.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:15.229 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T09:59:15.237 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T09:59:15.237 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T09:59:15.238 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:59:15.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:15.238 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T09:59:15.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:59:15.304 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:15.352 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:59:15.352 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:59:15.353 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:15.404 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T09:59:15.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:15.404 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:15.495 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:15.496 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:59:15.508 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:15.508 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:59:15.508 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T09:59:15.508 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T09:59:15.508 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T09:59:15.512 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:59:15.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:15.512 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:15.531 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:59:15.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:59:15.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:15.635 INFO:teuthology.orchestra.run.smithi110.stdout:client4694 2024-09-10T09:59:15.635 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:15.635 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:15.644 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T09:59:15.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e72200640 1 Processor -- start 2024-09-10T09:59:15.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e72200640 1 -- start start 2024-09-10T09:59:15.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e72200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c155970 0x7f9e6c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:15.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e72200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e6c06f1e0 0x7f9e6c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:15.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e72200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e6c155d50 con 0x7f9e6c15c560 2024-09-10T09:59:15.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e72200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9e6c0b4570 con 0x7f9e6c06f1e0 2024-09-10T09:59:15.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e72200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c0b34d0 con 0x7f9e6c155970 2024-09-10T09:59:15.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e6be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e6c06f1e0 0x7f9e6c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:15.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e6aa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9e6c15c560 0x7f9e6c06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52684/0 (socket says 172.21.15.110:52684) 2024-09-10T09:59:15.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.815+0000 7f9e6aa00640 1 -- 172.21.15.110:0/3647351359 learned_addr learned my addr 172.21.15.110:0/3647351359 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:15.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6b400640 1 --2- 172.21.15.110:0/3647351359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c155970 0x7f9e6c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:15.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6be00640 1 -- 172.21.15.110:0/3647351359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c155970 msgr2=0x7f9e6c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:15.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6be00640 1 --2- 172.21.15.110:0/3647351359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c155970 0x7f9e6c0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:15.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6be00640 1 -- 172.21.15.110:0/3647351359 >> v1:172.21.15.110:6789/0 conn(0x7f9e6c15c560 legacy=0x7f9e6c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:15.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6be00640 1 -- 172.21.15.110:0/3647351359 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c0b37b0 con 0x7f9e6c06f1e0 2024-09-10T09:59:15.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6b400640 1 --2- 172.21.15.110:0/3647351359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c155970 0x7f9e6c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:15.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6be00640 1 --2- 172.21.15.110:0/3647351359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c06f1e0 0x7f9e6c0ce420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e640097c0 tx=0x7f9e6401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1336ee3aa9fceed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:15.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6a000640 1 -- 172.21.15.110:0/3647351359 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640b1070 con 0x7f9e6c06f1e0 2024-09-10T09:59:15.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6a000640 1 -- 172.21.15.110:0/3647351359 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e640a8050 con 0x7f9e6c06f1e0 2024-09-10T09:59:15.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e6a000640 1 -- 172.21.15.110:0/3647351359 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640ac860 con 0x7f9e6c06f1e0 2024-09-10T09:59:15.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e72200640 1 -- 172.21.15.110:0/3647351359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c06f1e0 msgr2=0x7f9e6c0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:15.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e72200640 1 --2- 172.21.15.110:0/3647351359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c06f1e0 0x7f9e6c0ce420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e640097c0 tx=0x7f9e6401c3e0 comp rx=0 tx=0).stop 2024-09-10T09:59:15.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e72200640 1 -- 172.21.15.110:0/3647351359 shutdown_connections 2024-09-10T09:59:15.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e72200640 1 --2- 172.21.15.110:0/3647351359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c06f1e0 0x7f9e6c0ce420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:15.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e72200640 1 --2- 172.21.15.110:0/3647351359 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c155970 0x7f9e6c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:15.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e72200640 1 -- 172.21.15.110:0/3647351359 >> 172.21.15.110:0/3647351359 conn(0x7f9e6c06bc50 msgr2=0x7f9e6c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:15.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e72200640 1 -- 172.21.15.110:0/3647351359 shutdown_connections 2024-09-10T09:59:15.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.819+0000 7f9e72200640 1 -- 172.21.15.110:0/3647351359 wait complete. 2024-09-10T09:59:15.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 Processor -- start 2024-09-10T09:59:15.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 -- start start 2024-09-10T09:59:15.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e6c06f1e0 0x7f9e6c152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:15.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c155970 0x7f9e6c15b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:15.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c15c560 0x7f9e6c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:15.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c0cf170 con 0x7f9e6c155970 2024-09-10T09:59:15.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c0cce00 con 0x7f9e6c06f1e0 2024-09-10T09:59:15.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c0cf2f0 con 0x7f9e6c15c560 2024-09-10T09:59:15.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c155970 0x7f9e6c15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:15.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c155970 0x7f9e6c15b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46394/0 (socket says 172.21.15.110:46394) 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 -- 172.21.15.110:0/1521327761 learned_addr learned my addr 172.21.15.110:0/1521327761 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6be00640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c15c560 0x7f9e6c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6b400640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e6c06f1e0 0x7f9e6c152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 -- 172.21.15.110:0/1521327761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c15c560 msgr2=0x7f9e6c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c15c560 0x7f9e6c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 -- 172.21.15.110:0/1521327761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e6c06f1e0 msgr2=0x7f9e6c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e6c06f1e0 0x7f9e6c152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 -- 172.21.15.110:0/1521327761 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c1594d0 con 0x7f9e6c155970 2024-09-10T09:59:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e6aa00640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c155970 0x7f9e6c15b000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800c370 tx=0x7f9e5800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:15.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e63400640 1 -- 172.21.15.110:0/1521327761 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58017070 con 0x7f9e6c155970 2024-09-10T09:59:15.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e63400640 1 -- 172.21.15.110:0/1521327761 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e58005cf0 con 0x7f9e6c155970 2024-09-10T09:59:15.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c1597b0 con 0x7f9e6c155970 2024-09-10T09:59:15.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e63400640 1 -- 172.21.15.110:0/1521327761 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e580072f0 con 0x7f9e6c155970 2024-09-10T09:59:15.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.823+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c159d70 con 0x7f9e6c155970 2024-09-10T09:59:15.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.827+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c0ce420 con 0x7f9e6c155970 2024-09-10T09:59:15.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.827+0000 7f9e63400640 1 -- 172.21.15.110:0/1521327761 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e5801a070 con 0x7f9e6c155970 2024-09-10T09:59:15.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.827+0000 7f9e63400640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e4c0678c0 0x7f9e4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:15.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.827+0000 7f9e63400640 1 -- 172.21.15.110:0/1521327761 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9e58084f20 con 0x7f9e6c155970 2024-09-10T09:59:15.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.827+0000 7f9e6b400640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e4c0678c0 0x7f9e4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:15.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.827+0000 7f9e6b400640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e4c0678c0 0x7f9e4c069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d620 tx=0x7f9e5c007810 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:15.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:15.831+0000 7f9e63400640 1 -- 172.21.15.110:0/1521327761 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9e58055400 con 0x7f9e6c155970 2024-09-10T09:59:16.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9e6c154940 con 0x7f9e6c155970 2024-09-10T09:59:16.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e63400640 1 -- 172.21.15.110:0/1521327761 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f9e58059700 con 0x7f9e6c155970 2024-09-10T09:59:16.027 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T09:59:16.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e4c0678c0 msgr2=0x7f9e4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e4c0678c0 0x7f9e4c069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d620 tx=0x7f9e5c007810 comp rx=0 tx=0).stop 2024-09-10T09:59:16.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c155970 msgr2=0x7f9e6c15b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c155970 0x7f9e6c15b000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800c370 tx=0x7f9e5800c830 comp rx=0 tx=0).stop 2024-09-10T09:59:16.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 shutdown_connections 2024-09-10T09:59:16.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9e4c0678c0 0x7f9e4c069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e6c15c560 0x7f9e6c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e6c155970 0x7f9e6c15b000 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 --2- 172.21.15.110:0/1521327761 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e6c06f1e0 0x7f9e6c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.023+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 >> 172.21.15.110:0/1521327761 conn(0x7f9e6c06bc50 msgr2=0x7f9e6c154e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.027+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 shutdown_connections 2024-09-10T09:59:16.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.027+0000 7f9e72200640 1 -- 172.21.15.110:0/1521327761 wait complete. 2024-09-10T09:59:16.047 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T09:59:16.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:59:16.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T09:59:16.047 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T09:59:16.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:16.048 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T09:59:16.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T09:59:16.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:59:16.048 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T09:59:16.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:16.048 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T09:59:16.058 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:59:16.058 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:59:16.058 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:59:16.058 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:59:16.059 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42415sec preferred_lft 42415sec 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:16.060 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T09:59:16.061 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:16.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:16.062 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:59:16.125 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T09:59:16.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:16.128 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T09:59:16.181 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T09:59:16.181 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:16.182 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:59:16.182 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:59:16.183 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:59:16.183 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:59:16.183 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:59:16.183 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:59:16.183 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:59:16.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:59:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:59:16.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:16.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:16.329 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T09:59:16.338 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T09:59:16.338 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T09:59:16.338 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T09:59:16.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:16.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T09:59:16.405 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:59:16.406 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:16.457 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T09:59:16.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:59:16.458 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:16.509 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T09:59:16.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:16.509 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:16.597 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:16.597 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:59:16.611 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:16.611 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:59:16.611 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T09:59:16.611 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T09:59:16.611 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T09:59:16.615 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:59:16.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:16.615 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:16.676 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:59:16.764 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:59:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:59:16.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:16.773 INFO:teuthology.orchestra.run.smithi145.stdout:client4822 2024-09-10T09:59:16.773 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:16.773 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:16.782 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T09:59:16.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff997a00640 1 Processor -- start 2024-09-10T09:59:16.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff997a00640 1 -- start start 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff997a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff9900cfe30 0x7ff990071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff997a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff9900cd510 0x7ff9900cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff997a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff99006e090 0x7ff99006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff997a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff99015d9a0 con 0x7ff9900cd510 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff997a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff99006ea60 con 0x7ff9900cfe30 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff997a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff99006d2d0 con 0x7ff99006e090 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff98fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff9900cd510 0x7ff9900cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff98fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff9900cd510 0x7ff9900cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46408/0 (socket says 172.21.15.110:46408) 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.919+0000 7ff98fe00640 1 -- 172.21.15.110:0/849616523 learned_addr learned my addr 172.21.15.110:0/849616523 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff995a00640 1 --2- 172.21.15.110:0/849616523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff99006e090 0x7ff99006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff995000640 1 --2- 172.21.15.110:0/849616523 >> v2:172.21.15.145:3300/0 conn(0x7ff9900cfe30 0x7ff990071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98fe00640 1 -- 172.21.15.110:0/849616523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff99006e090 msgr2=0x7ff99006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98fe00640 1 --2- 172.21.15.110:0/849616523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff99006e090 0x7ff99006e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98fe00640 1 -- 172.21.15.110:0/849616523 >> v2:172.21.15.145:3300/0 conn(0x7ff9900cfe30 msgr2=0x7ff990071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98fe00640 1 --2- 172.21.15.110:0/849616523 >> v2:172.21.15.145:3300/0 conn(0x7ff9900cfe30 0x7ff990071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98fe00640 1 -- 172.21.15.110:0/849616523 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99006d5b0 con 0x7ff9900cd510 2024-09-10T09:59:16.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff995a00640 1 --2- 172.21.15.110:0/849616523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff99006e090 0x7ff99006e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:16.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98fe00640 1 --2- 172.21.15.110:0/849616523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9900cd510 0x7ff9900cd9a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff9900ce120 tx=0x7ff97c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb213097f3799742 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98f400640 1 -- 172.21.15.110:0/849616523 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0b2070 con 0x7ff9900cd510 2024-09-10T09:59:16.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98f400640 1 -- 172.21.15.110:0/849616523 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97c0a9070 con 0x7ff9900cd510 2024-09-10T09:59:16.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff98f400640 1 -- 172.21.15.110:0/849616523 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0ad880 con 0x7ff9900cd510 2024-09-10T09:59:16.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff997a00640 1 -- 172.21.15.110:0/849616523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9900cd510 msgr2=0x7ff9900cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.923+0000 7ff997a00640 1 --2- 172.21.15.110:0/849616523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9900cd510 0x7ff9900cd9a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff9900ce120 tx=0x7ff97c01c490 comp rx=0 tx=0).stop 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- 172.21.15.110:0/849616523 shutdown_connections 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 --2- 172.21.15.110:0/849616523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff99006e090 0x7ff99006e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 --2- 172.21.15.110:0/849616523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9900cd510 0x7ff9900cd9a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 --2- 172.21.15.110:0/849616523 >> v2:172.21.15.145:3300/0 conn(0x7ff9900cfe30 0x7ff990071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- 172.21.15.110:0/849616523 >> 172.21.15.110:0/849616523 conn(0x7ff99006bc50 msgr2=0x7ff99006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- 172.21.15.110:0/849616523 shutdown_connections 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- 172.21.15.110:0/849616523 wait complete. 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 Processor -- start 2024-09-10T09:59:16.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- start start 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff99006e090 0x7ff99014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9900cd510 0x7ff990149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9900cfe30 0x7ff990149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff990072190 con 0x7ff99006e090 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff99015d680 con 0x7ff9900cd510 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff990160010 con 0x7ff9900cfe30 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff99006e090 0x7ff99014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff99006e090 0x7ff99014bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46418/0 (socket says 172.21.15.110:46418) 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 -- 172.21.15.110:0/2249673035 learned_addr learned my addr 172.21.15.110:0/2249673035 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff98fe00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9900cd510 0x7ff990149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995a00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9900cfe30 0x7ff990149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 -- 172.21.15.110:0/2249673035 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9900cfe30 msgr2=0x7ff990149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9900cfe30 0x7ff990149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 -- 172.21.15.110:0/2249673035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9900cd510 msgr2=0x7ff990149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9900cd510 0x7ff990149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 -- 172.21.15.110:0/2249673035 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff990149cc0 con 0x7ff99006e090 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff995000640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff99006e090 0x7ff99014bb90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff980007e40 tx=0x7ff98000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff98cc00640 1 -- 172.21.15.110:0/2249673035 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980014620 con 0x7ff99006e090 2024-09-10T09:59:16.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff990149fa0 con 0x7ff99006e090 2024-09-10T09:59:16.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff98cc00640 1 -- 172.21.15.110:0/2249673035 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff980019070 con 0x7ff99006e090 2024-09-10T09:59:16.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff98cc00640 1 -- 172.21.15.110:0/2249673035 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980014a00 con 0x7ff99006e090 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.927+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99014a560 con 0x7ff99006e090 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.931+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff990071380 con 0x7ff99006e090 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.931+0000 7ff98cc00640 1 -- 172.21.15.110:0/2249673035 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff980014ba0 con 0x7ff99006e090 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.931+0000 7ff98cc00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9700678f0 0x7ff970069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.931+0000 7ff98cc00640 1 -- 172.21.15.110:0/2249673035 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff980086620 con 0x7ff99006e090 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.931+0000 7ff98fe00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9700678f0 0x7ff970069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.931+0000 7ff98fe00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9700678f0 0x7ff970069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0af000 tx=0x7ff97c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:16.935+0000 7ff98cc00640 1 -- 172.21.15.110:0/2249673035 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff980057040 con 0x7ff99006e090 2024-09-10T09:59:17.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.127+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff99006c3b0 con 0x7ff99006e090 2024-09-10T09:59:17.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff98cc00640 1 -- 172.21.15.110:0/2249673035 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) ==== 136+0+0 (secure 0 0 0) 0x7ff980056940 con 0x7ff99006e090 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9700678f0 msgr2=0x7ff970069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9700678f0 0x7ff970069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0af000 tx=0x7ff97c008040 comp rx=0 tx=0).stop 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff99006e090 msgr2=0x7ff99014bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff99006e090 0x7ff99014bb90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff980007e40 tx=0x7ff98000e4b0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 shutdown_connections 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff9700678f0 0x7ff970069db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9900cfe30 0x7ff990149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9900cd510 0x7ff990149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 --2- 172.21.15.110:0/2249673035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff99006e090 0x7ff99014bb90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 >> 172.21.15.110:0/2249673035 conn(0x7ff99006bc50 msgr2=0x7ff99014c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 shutdown_connections 2024-09-10T09:59:17.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.287+0000 7ff997a00640 1 -- 172.21.15.110:0/2249673035 wait complete. 2024-09-10T09:59:17.312 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f8a00640 1 Processor -- start 2024-09-10T09:59:17.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f8a00640 1 -- start start 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f8a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff8f415c4c0 0x7ff8f40afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f8a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff8f40cf840 0x7ff8f406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f8a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f415c8a0 con 0x7ff8f4072d50 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f8a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f406f270 con 0x7ff8f40cf840 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f8a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f40cea80 con 0x7ff8f415c4c0 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff8f40cf840 0x7ff8f406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff8f40cf840 0x7ff8f406ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46432/0 (socket says 172.21.15.110:46432) 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1400640 1 -- 172.21.15.110:0/3514625636 learned_addr learned my addr 172.21.15.110:0/3514625636 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1e00640 1 --2- 172.21.15.110:0/3514625636 >> v2:172.21.15.145:3300/0 conn(0x7ff8f415c4c0 0x7ff8f40afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1400640 1 -- 172.21.15.110:0/3514625636 >> v2:172.21.15.145:3300/0 conn(0x7ff8f415c4c0 msgr2=0x7ff8f40afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1400640 1 --2- 172.21.15.110:0/3514625636 >> v2:172.21.15.145:3300/0 conn(0x7ff8f415c4c0 0x7ff8f40afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1400640 1 -- 172.21.15.110:0/3514625636 >> v1:172.21.15.145:6789/0 conn(0x7ff8f4072d50 legacy=0x7ff8f415ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1400640 1 -- 172.21.15.110:0/3514625636 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f40ced60 con 0x7ff8f40cf840 2024-09-10T09:59:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.447+0000 7ff8f1e00640 1 --2- 172.21.15.110:0/3514625636 >> v2:172.21.15.145:3300/0 conn(0x7ff8f415c4c0 0x7ff8f40afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f1400640 1 --2- 172.21.15.110:0/3514625636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f406ed30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40b3580 tx=0x7ff8e001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8810b72be3ccf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f0a00640 1 -- 172.21.15.110:0/3514625636 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00b0070 con 0x7ff8f40cf840 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f0a00640 1 -- 172.21.15.110:0/3514625636 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e00ab600 con 0x7ff8f40cf840 2024-09-10T09:59:17.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f0a00640 1 -- 172.21.15.110:0/3514625636 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00ab9d0 con 0x7ff8f40cf840 2024-09-10T09:59:17.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 -- 172.21.15.110:0/3514625636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 msgr2=0x7ff8f406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/3514625636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f406ed30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40b3580 tx=0x7ff8e001c450 comp rx=0 tx=0).stop 2024-09-10T09:59:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 -- 172.21.15.110:0/3514625636 shutdown_connections 2024-09-10T09:59:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/3514625636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f406ed30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/3514625636 >> v2:172.21.15.145:3300/0 conn(0x7ff8f415c4c0 0x7ff8f40afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 -- 172.21.15.110:0/3514625636 >> 172.21.15.110:0/3514625636 conn(0x7ff8f40d07a0 msgr2=0x7ff8f40d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 -- 172.21.15.110:0/3514625636 shutdown_connections 2024-09-10T09:59:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 -- 172.21.15.110:0/3514625636 wait complete. 2024-09-10T09:59:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 Processor -- start 2024-09-10T09:59:17.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.451+0000 7ff8f8a00640 1 -- start start 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4072d50 0x7ff8f41585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f4160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8f415c4c0 0x7ff8f4158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41692d0 con 0x7ff8f40cf840 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4169150 con 0x7ff8f4072d50 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f4169450 con 0x7ff8f415c4c0 2024-09-10T09:59:17.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f4160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f4160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46438/0 (socket says 172.21.15.110:46438) 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 -- 172.21.15.110:0/133750195 learned_addr learned my addr 172.21.15.110:0/133750195 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1e00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4072d50 0x7ff8f41585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f2800640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8f415c4c0 0x7ff8f4158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 -- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8f415c4c0 msgr2=0x7ff8f4158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8f415c4c0 0x7ff8f4158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 -- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4072d50 msgr2=0x7ff8f41585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4072d50 0x7ff8f41585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 -- 172.21.15.110:0/133750195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f4158b30 con 0x7ff8f40cf840 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f2800640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8f415c4c0 0x7ff8f4158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f1400640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f4160ff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8e001ab00 tx=0x7ff8e001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8ea000640 1 -- 172.21.15.110:0/133750195 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00ba020 con 0x7ff8f40cf840 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8ea000640 1 -- 172.21.15.110:0/133750195 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e00b6640 con 0x7ff8f40cf840 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8ea000640 1 -- 172.21.15.110:0/133750195 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00b0040 con 0x7ff8f40cf840 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f4165160 con 0x7ff8f40cf840 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f40b5620 con 0x7ff8f40cf840 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.455+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f4159a30 con 0x7ff8f40cf840 2024-09-10T09:59:17.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.459+0000 7ff8ea000640 1 -- 172.21.15.110:0/133750195 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff8e00bd020 con 0x7ff8f40cf840 2024-09-10T09:59:17.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.459+0000 7ff8ea000640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff8d40678c0 0x7ff8d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.459+0000 7ff8ea000640 1 -- 172.21.15.110:0/133750195 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff8e011ff60 con 0x7ff8f40cf840 2024-09-10T09:59:17.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.459+0000 7ff8f1e00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff8d40678c0 0x7ff8d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.459+0000 7ff8f1e00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff8d40678c0 0x7ff8d4069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40af570 tx=0x7ff8e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.463+0000 7ff8ea000640 1 -- 172.21.15.110:0/133750195 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff8e00c4030 con 0x7ff8f40cf840 2024-09-10T09:59:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.643+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff8f415e070 con 0x7ff8d40678c0 2024-09-10T09:59:17.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.643+0000 7ff8ea000640 1 -- 172.21.15.110:0/133750195 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff8f415e070 con 0x7ff8d40678c0 2024-09-10T09:59:17.647 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T09:59:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T09:59:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:17.647 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T09:59:17.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.643+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff8d40678c0 msgr2=0x7ff8d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.643+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff8d40678c0 0x7ff8d4069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40af570 tx=0x7ff8e4008040 comp rx=0 tx=0).stop 2024-09-10T09:59:17.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.643+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 msgr2=0x7ff8f4160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.643+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f4160ff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8e001ab00 tx=0x7ff8e001ac90 comp rx=0 tx=0).stop 2024-09-10T09:59:17.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.643+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 shutdown_connections 2024-09-10T09:59:17.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.643+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff8d40678c0 0x7ff8d4069d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.647+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8f415c4c0 0x7ff8f4158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.647+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8f40cf840 0x7ff8f4160ff0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.647+0000 7ff8f8a00640 1 --2- 172.21.15.110:0/133750195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8f4072d50 0x7ff8f41585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.647+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 >> 172.21.15.110:0/133750195 conn(0x7ff8f40d07a0 msgr2=0x7ff8f415e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.647+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 shutdown_connections 2024-09-10T09:59:17.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.647+0000 7ff8f8a00640 1 -- 172.21.15.110:0/133750195 wait complete. 2024-09-10T09:59:17.668 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T09:59:17.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.803+0000 7f636c400640 1 Processor -- start 2024-09-10T09:59:17.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.803+0000 7f636c400640 1 -- start start 2024-09-10T09:59:17.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f636c400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6364162bf0 0x7f636406e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f636c400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636406eaa0 con 0x7f636406e680 2024-09-10T09:59:17.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f636c400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63640cd420 con 0x7f6364162810 2024-09-10T09:59:17.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f636c400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f63640b35b0 con 0x7f6364162bf0 2024-09-10T09:59:17.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f636a400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f636406e680 0x7f63640b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52688/0 (socket says 172.21.15.110:52688) 2024-09-10T09:59:17.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f636a400640 1 -- 172.21.15.110:0/1356467676 learned_addr learned my addr 172.21.15.110:0/1356467676 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:17.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f6369000640 1 --2- 172.21.15.110:0/1356467676 >> v2:172.21.15.145:3300/0 conn(0x7f6364162bf0 0x7f636406e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f6363e00640 1 -- 172.21.15.110:0/1356467676 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099626615 0 0) 0x7f636406eaa0 con 0x7f636406e680 2024-09-10T09:59:17.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f6363e00640 1 -- 172.21.15.110:0/1356467676 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f634c003530 con 0x7f636406e680 2024-09-10T09:59:17.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f6369000640 1 -- 172.21.15.110:0/1356467676 >> v1:172.21.15.145:6789/0 conn(0x7f6364162810 legacy=0x7f63640b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f6363e00640 1 -- 172.21.15.110:0/1356467676 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327450243 0 0) 0x7f63580014a0 con 0x7f6364162810 2024-09-10T09:59:17.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f6369000640 1 -- 172.21.15.110:0/1356467676 >> v1:172.21.15.110:6789/0 conn(0x7f636406e680 legacy=0x7f63640b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.807+0000 7f6369000640 1 -- 172.21.15.110:0/1356467676 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63640b3800 con 0x7f6364162bf0 2024-09-10T09:59:17.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f6369000640 1 --2- 172.21.15.110:0/1356467676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6364162bf0 0x7f636406e140 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f63500097c0 tx=0x7f635001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad15f94046e88b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f6363e00640 1 -- 172.21.15.110:0/1356467676 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500aa600 con 0x7f6364162bf0 2024-09-10T09:59:17.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f6363e00640 1 -- 172.21.15.110:0/1356467676 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63500af070 con 0x7f6364162bf0 2024-09-10T09:59:17.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f6363e00640 1 -- 172.21.15.110:0/1356467676 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500aa9c0 con 0x7f6364162bf0 2024-09-10T09:59:17.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- 172.21.15.110:0/1356467676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6364162bf0 msgr2=0x7f636406e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 --2- 172.21.15.110:0/1356467676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6364162bf0 0x7f636406e140 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f63500097c0 tx=0x7f635001c4a0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- 172.21.15.110:0/1356467676 shutdown_connections 2024-09-10T09:59:17.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 --2- 172.21.15.110:0/1356467676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6364162bf0 0x7f636406e140 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- 172.21.15.110:0/1356467676 >> 172.21.15.110:0/1356467676 conn(0x7f6364062e40 msgr2=0x7f636406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- 172.21.15.110:0/1356467676 shutdown_connections 2024-09-10T09:59:17.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- 172.21.15.110:0/1356467676 wait complete. 2024-09-10T09:59:17.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 Processor -- start 2024-09-10T09:59:17.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- start start 2024-09-10T09:59:17.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636406e680 0x7f636415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6364162810 0x7f6364158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6364162bf0 0x7f6364158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63640b1870 con 0x7f6364162bf0 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63640b4380 con 0x7f636406e680 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.811+0000 7f636c400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f63640b57c0 con 0x7f6364162810 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6364162bf0 0x7f6364158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6364162bf0 0x7f6364158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46450/0 (socket says 172.21.15.110:46450) 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 -- 172.21.15.110:0/1846658334 learned_addr learned my addr 172.21.15.110:0/1846658334 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f6369a00640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636406e680 0x7f636415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f6369000640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6364162810 0x7f6364158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 -- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6364162810 msgr2=0x7f6364158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6364162810 0x7f6364158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 -- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636406e680 msgr2=0x7f636415ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636406e680 0x7f636415ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 -- 172.21.15.110:0/1846658334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6364158d70 con 0x7f6364162bf0 2024-09-10T09:59:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f6369000640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6364162810 0x7f6364158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:17.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f6369a00640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636406e680 0x7f636415ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:17.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636a400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6364162bf0 0x7f6364158060 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c480 tx=0x7f635c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f6361600640 1 -- 172.21.15.110:0/1846658334 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c017070 con 0x7f6364162bf0 2024-09-10T09:59:17.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f6361600640 1 -- 172.21.15.110:0/1846658334 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635c005cf0 con 0x7f6364162bf0 2024-09-10T09:59:17.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f6361600640 1 -- 172.21.15.110:0/1846658334 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c0072b0 con 0x7f6364162bf0 2024-09-10T09:59:17.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6364163380 con 0x7f6364162bf0 2024-09-10T09:59:17.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6364163900 con 0x7f6364162bf0 2024-09-10T09:59:17.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636406e140 con 0x7f6364162bf0 2024-09-10T09:59:17.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.815+0000 7f6361600640 1 -- 172.21.15.110:0/1846658334 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f635c007450 con 0x7f6364162bf0 2024-09-10T09:59:17.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.819+0000 7f6361600640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f63440678c0 0x7f6344069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:17.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.819+0000 7f6361600640 1 -- 172.21.15.110:0/1846658334 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f635c084ff0 con 0x7f6364162bf0 2024-09-10T09:59:17.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.819+0000 7f6369a00640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f63440678c0 0x7f6344069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:17.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.819+0000 7f6369a00640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f63440678c0 0x7f6344069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6358001740 tx=0x7f6358009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:17.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.823+0000 7f6361600640 1 -- 172.21.15.110:0/1846658334 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f635c007740 con 0x7f6364162bf0 2024-09-10T09:59:17.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.979+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6364071b60 con 0x7f6364162bf0 2024-09-10T09:59:17.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.979+0000 7f6361600640 1 -- 172.21.15.110:0/1846658334 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f635c055b20 con 0x7f6364162bf0 2024-09-10T09:59:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f63440678c0 msgr2=0x7f6344069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f63440678c0 0x7f6344069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6358001740 tx=0x7f6358009040 comp rx=0 tx=0).stop 2024-09-10T09:59:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6364162bf0 msgr2=0x7f6364158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6364162bf0 0x7f6364158060 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c480 tx=0x7f635c00c940 comp rx=0 tx=0).stop 2024-09-10T09:59:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 shutdown_connections 2024-09-10T09:59:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f63440678c0 0x7f6344069d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6364162bf0 0x7f6364158060 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6364162810 0x7f6364158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 --2- 172.21.15.110:0/1846658334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636406e680 0x7f636415ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:17.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 >> 172.21.15.110:0/1846658334 conn(0x7f6364062e40 msgr2=0x7f63640ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:17.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 shutdown_connections 2024-09-10T09:59:17.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:17.983+0000 7f636c400640 1 -- 172.21.15.110:0/1846658334 wait complete. 2024-09-10T09:59:18.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:18.005 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp75220/subvol20260) 2024-09-10T09:59:18.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:18.032 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T09:59:18.086 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T09:59:18.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:18.087 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp75220/subvol20260) 2024-09-10T09:59:18.159 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:18.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp75220/subvol20260/.meta 2024-09-10T09:59:18.229 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol20260 subvol_grp75220 2024-09-10T09:59:18.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.407+0000 7fa21fe00640 1 Processor -- start 2024-09-10T09:59:18.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21fe00640 1 -- start start 2024-09-10T09:59:18.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa22015c080 0x7fa22006c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2200720e0 0x7fa2200b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21fe00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2200b0c10 con 0x7fa22006e290 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21fe00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa2200b09c0 con 0x7fa2200720e0 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21fe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa22006cb70 con 0x7fa22015c080 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21e000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2200720e0 0x7fa2200b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa22006e290 0x7fa2200b32b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44176/0 (socket says 172.21.15.110:44176) 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21e000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2200720e0 0x7fa2200b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54028/0 (socket says 172.21.15.110:54028) 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21f400640 1 -- 172.21.15.110:0/3179722777 learned_addr learned my addr 172.21.15.110:0/3179722777 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:18.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21ea00640 1 --2- 172.21.15.110:0/3179722777 >> v2:172.21.15.145:3300/0 conn(0x7fa22015c080 0x7fa22006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21e000640 1 -- 172.21.15.110:0/3179722777 >> v2:172.21.15.145:3300/0 conn(0x7fa22015c080 msgr2=0x7fa22006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21e000640 1 --2- 172.21.15.110:0/3179722777 >> v2:172.21.15.145:3300/0 conn(0x7fa22015c080 0x7fa22006c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21e000640 1 -- 172.21.15.110:0/3179722777 >> v1:172.21.15.110:6789/0 conn(0x7fa22006e290 legacy=0x7fa2200b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21e000640 1 -- 172.21.15.110:0/3179722777 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2200b0ef0 con 0x7fa2200720e0 2024-09-10T09:59:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.411+0000 7fa21d600640 1 -- 172.21.15.110:0/3179722777 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054970032 0 0) 0x7fa2200b0c10 con 0x7fa22006e290 2024-09-10T09:59:18.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21e000640 1 --2- 172.21.15.110:0/3179722777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2200720e0 0x7fa2200b68d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa22006cd30 tx=0x7fa20c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3d774eddd4b81d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21d600640 1 -- 172.21.15.110:0/3179722777 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0b0070 con 0x7fa2200720e0 2024-09-10T09:59:18.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21d600640 1 -- 172.21.15.110:0/3179722777 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa20c0ab600 con 0x7fa2200720e0 2024-09-10T09:59:18.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21d600640 1 -- 172.21.15.110:0/3179722777 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0ab9d0 con 0x7fa2200720e0 2024-09-10T09:59:18.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 -- 172.21.15.110:0/3179722777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2200720e0 msgr2=0x7fa2200b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 --2- 172.21.15.110:0/3179722777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2200720e0 0x7fa2200b68d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa22006cd30 tx=0x7fa20c01c480 comp rx=0 tx=0).stop 2024-09-10T09:59:18.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 -- 172.21.15.110:0/3179722777 shutdown_connections 2024-09-10T09:59:18.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 --2- 172.21.15.110:0/3179722777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2200720e0 0x7fa2200b68d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 --2- 172.21.15.110:0/3179722777 >> v2:172.21.15.145:3300/0 conn(0x7fa22015c080 0x7fa22006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 -- 172.21.15.110:0/3179722777 >> 172.21.15.110:0/3179722777 conn(0x7fa22006cd60 msgr2=0x7fa22006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 -- 172.21.15.110:0/3179722777 shutdown_connections 2024-09-10T09:59:18.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 -- 172.21.15.110:0/3179722777 wait complete. 2024-09-10T09:59:18.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 Processor -- start 2024-09-10T09:59:18.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.415+0000 7fa21fe00640 1 -- start start 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa22006e290 0x7fa220155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2200720e0 0x7fa22014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa22015c080 0x7fa2201539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa220168ee0 con 0x7fa22006e290 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa22015c670 con 0x7fa2200720e0 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa220169060 con 0x7fa22015c080 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa22006e290 0x7fa220155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa22006e290 0x7fa220155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54036/0 (socket says 172.21.15.110:54036) 2024-09-10T09:59:18.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 -- 172.21.15.110:0/2301868306 learned_addr learned my addr 172.21.15.110:0/2301868306 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21e000640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2200720e0 0x7fa22014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21f400640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa22015c080 0x7fa2201539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 -- 172.21.15.110:0/2301868306 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa22015c080 msgr2=0x7fa2201539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa22015c080 0x7fa2201539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 -- 172.21.15.110:0/2301868306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2200720e0 msgr2=0x7fa22014eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2200720e0 0x7fa22014eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 -- 172.21.15.110:0/2301868306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2201511c0 con 0x7fa22006e290 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21ea00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa22006e290 0x7fa220155ad0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e8a0 tx=0x7fa21000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa217400640 1 -- 172.21.15.110:0/2301868306 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210017070 con 0x7fa22006e290 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa217400640 1 -- 172.21.15.110:0/2301868306 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21000c940 con 0x7fa22006e290 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa217400640 1 -- 172.21.15.110:0/2301868306 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21000cd10 con 0x7fa22006e290 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2201514a0 con 0x7fa22006e290 2024-09-10T09:59:18.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa220151a60 con 0x7fa22006e290 2024-09-10T09:59:18.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.419+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2200b68d0 con 0x7fa22006e290 2024-09-10T09:59:18.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.423+0000 7fa217400640 1 -- 172.21.15.110:0/2301868306 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa21001a070 con 0x7fa22006e290 2024-09-10T09:59:18.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.423+0000 7fa217400640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2000678c0 0x7fa200069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.423+0000 7fa217400640 1 -- 172.21.15.110:0/2301868306 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa210085ee0 con 0x7fa22006e290 2024-09-10T09:59:18.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.423+0000 7fa21e000640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2000678c0 0x7fa200069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.423+0000 7fa21e000640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2000678c0 0x7fa200069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0ad000 tx=0x7fa20c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.427+0000 7fa217400640 1 -- 172.21.15.110:0/2301868306 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa2100563d0 con 0x7fa22006e290 2024-09-10T09:59:18.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.607+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol20260", "group_name": "subvol_grp75220", "target": ["mon-mgr", ""]}) -- 0x7fa22006d480 con 0x7fa2000678c0 2024-09-10T09:59:18.616 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:18.611+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:18.616 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:18.611+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:18.616 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:18.611+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:18.616 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:18.611+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:18.616 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:18.611+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:18.665 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:59:18.668 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:59:18.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa217400640 1 -- 172.21.15.110:0/2301868306 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa22006d480 con 0x7fa2000678c0 2024-09-10T09:59:18.757 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/subvol_grp75220/subvol20260 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2000678c0 msgr2=0x7fa200069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2000678c0 0x7fa200069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0ad000 tx=0x7fa20c008040 comp rx=0 tx=0).stop 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa22006e290 msgr2=0x7fa220155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa22006e290 0x7fa220155ad0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e8a0 tx=0x7fa21000ed60 comp rx=0 tx=0).stop 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 shutdown_connections 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa2000678c0 0x7fa200069d80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa22015c080 0x7fa2201539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2200720e0 0x7fa22014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 --2- 172.21.15.110:0/2301868306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa22006e290 0x7fa220155ad0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 >> 172.21.15.110:0/2301868306 conn(0x7fa22006cd60 msgr2=0x7fa22015edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 shutdown_connections 2024-09-10T09:59:18.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.755+0000 7fa21fe00640 1 -- 172.21.15.110:0/2301868306 wait complete. 2024-09-10T09:59:18.778 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol20260 subvol_grp75220 2024-09-10T09:59:18.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0eee00640 1 Processor -- start 2024-09-10T09:59:18.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0eee00640 1 -- start start 2024-09-10T09:59:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0eee00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd0e8155970 0x7fd0e80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0eee00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd0e806f1e0 0x7fd0e80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0eee00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e8155d50 con 0x7fd0e815c560 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0eee00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e80b4570 con 0x7fd0e8155970 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0eee00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e80b34d0 con 0x7fd0e806f1e0 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0e7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd0e8155970 0x7fd0e80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0e7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd0e8155970 0x7fd0e80cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54062/0 (socket says 172.21.15.110:54062) 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0e7e00640 1 -- 172.21.15.110:0/2967020802 learned_addr learned my addr 172.21.15.110:0/2967020802 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0ece00640 1 --2- 172.21.15.110:0/2967020802 >> v2:172.21.15.145:3300/0 conn(0x7fd0e806f1e0 0x7fd0e80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0e7e00640 1 -- 172.21.15.110:0/2967020802 >> v2:172.21.15.145:3300/0 conn(0x7fd0e806f1e0 msgr2=0x7fd0e80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0e7e00640 1 --2- 172.21.15.110:0/2967020802 >> v2:172.21.15.145:3300/0 conn(0x7fd0e806f1e0 0x7fd0e80ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0e7e00640 1 -- 172.21.15.110:0/2967020802 >> v1:172.21.15.145:6789/0 conn(0x7fd0e815c560 legacy=0x7fd0e806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0e7e00640 1 -- 172.21.15.110:0/2967020802 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e80b37b0 con 0x7fd0e8155970 2024-09-10T09:59:18.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.915+0000 7fd0ece00640 1 --2- 172.21.15.110:0/2967020802 >> v2:172.21.15.145:3300/0 conn(0x7fd0e806f1e0 0x7fd0e80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:18.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0e7e00640 1 --2- 172.21.15.110:0/2967020802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e80cc8c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80735e0 tx=0x7fd0d801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d95ccb675105951 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0e6a00640 1 -- 172.21.15.110:0/2967020802 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80aa610 con 0x7fd0e8155970 2024-09-10T09:59:18.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0e6a00640 1 -- 172.21.15.110:0/2967020802 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d80af070 con 0x7fd0e8155970 2024-09-10T09:59:18.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0e6a00640 1 -- 172.21.15.110:0/2967020802 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80aa9d0 con 0x7fd0e8155970 2024-09-10T09:59:18.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 -- 172.21.15.110:0/2967020802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 msgr2=0x7fd0e80cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 --2- 172.21.15.110:0/2967020802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e80cc8c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80735e0 tx=0x7fd0d801c4b0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 -- 172.21.15.110:0/2967020802 shutdown_connections 2024-09-10T09:59:18.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 --2- 172.21.15.110:0/2967020802 >> v2:172.21.15.145:3300/0 conn(0x7fd0e806f1e0 0x7fd0e80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 --2- 172.21.15.110:0/2967020802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e80cc8c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 -- 172.21.15.110:0/2967020802 >> 172.21.15.110:0/2967020802 conn(0x7fd0e806bc50 msgr2=0x7fd0e806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 -- 172.21.15.110:0/2967020802 shutdown_connections 2024-09-10T09:59:18.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 -- 172.21.15.110:0/2967020802 wait complete. 2024-09-10T09:59:18.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.919+0000 7fd0eee00640 1 Processor -- start 2024-09-10T09:59:18.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 -- start start 2024-09-10T09:59:18.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e806f1e0 0x7fd0e814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e81515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0e815c560 0x7fd0e80b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e80cf170 con 0x7fd0e8155970 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e80cce00 con 0x7fd0e806f1e0 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e80cf2f0 con 0x7fd0e815c560 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e81515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e81515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54076/0 (socket says 172.21.15.110:54076) 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 -- 172.21.15.110:0/4267720076 learned_addr learned my addr 172.21.15.110:0/4267720076 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7e00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e806f1e0 0x7fd0e814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0ece00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0e815c560 0x7fd0e80b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 -- 172.21.15.110:0/4267720076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0e815c560 msgr2=0x7fd0e80b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0e815c560 0x7fd0e80b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 -- 172.21.15.110:0/4267720076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e806f1e0 msgr2=0x7fd0e814ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e806f1e0 0x7fd0e814ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 -- 172.21.15.110:0/4267720076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e814ce80 con 0x7fd0e8155970 2024-09-10T09:59:18.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0e7400640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e81515e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400b410 tx=0x7fd0d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0dfe00640 1 -- 172.21.15.110:0/4267720076 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4015030 con 0x7fd0e8155970 2024-09-10T09:59:18.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0dfe00640 1 -- 172.21.15.110:0/4267720076 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d4004020 con 0x7fd0e8155970 2024-09-10T09:59:18.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0dfe00640 1 -- 172.21.15.110:0/4267720076 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4004450 con 0x7fd0e8155970 2024-09-10T09:59:18.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e814d160 con 0x7fd0e8155970 2024-09-10T09:59:18.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e814d720 con 0x7fd0e8155970 2024-09-10T09:59:18.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.923+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e80ce420 con 0x7fd0e8155970 2024-09-10T09:59:18.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.927+0000 7fd0dfe00640 1 -- 172.21.15.110:0/4267720076 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd0d4018070 con 0x7fd0e8155970 2024-09-10T09:59:18.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.927+0000 7fd0dfe00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd0c80678c0 0x7fd0c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.927+0000 7fd0dfe00640 1 -- 172.21.15.110:0/4267720076 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd0d4085660 con 0x7fd0e8155970 2024-09-10T09:59:18.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.927+0000 7fd0e7e00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd0c80678c0 0x7fd0c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.927+0000 7fd0e7e00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd0c80678c0 0x7fd0c8069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80ae000 tx=0x7fd0d80a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:18.931+0000 7fd0dfe00640 1 -- 172.21.15.110:0/4267720076 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd0d4055be0 con 0x7fd0e8155970 2024-09-10T09:59:19.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.111+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol20260", "group_name": "subvol_grp75220", "target": ["mon-mgr", ""]}) -- 0x7fd0e806ed80 con 0x7fd0c80678c0 2024-09-10T09:59:19.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.115+0000 7fd0dfe00640 1 -- 172.21.15.110:0/4267720076 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd0e806ed80 con 0x7fd0c80678c0 2024-09-10T09:59:19.120 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/subvol_grp75220/subvol20260 2024-09-10T09:59:19.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd0c80678c0 msgr2=0x7fd0c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd0c80678c0 0x7fd0c8069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80ae000 tx=0x7fd0d80a7000 comp rx=0 tx=0).stop 2024-09-10T09:59:19.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 msgr2=0x7fd0e81515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e81515e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400b410 tx=0x7fd0d400b8d0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 shutdown_connections 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd0c80678c0 0x7fd0c8069d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0e815c560 0x7fd0e80b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0e8155970 0x7fd0e81515e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 --2- 172.21.15.110:0/4267720076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0e806f1e0 0x7fd0e814ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 >> 172.21.15.110:0/4267720076 conn(0x7fd0e806bc50 msgr2=0x7fd0e80b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 shutdown_connections 2024-09-10T09:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.119+0000 7fd0eee00640 1 -- 172.21.15.110:0/4267720076 wait complete. 2024-09-10T09:59:19.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:19.141 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/2b99b9c02d38daef9a7da2ad1a241db1.meta) 2024-09-10T09:59:19.165 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T09:59:19.165 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T09:59:19.165 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T09:59:19.165 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/subvol_grp75220/subvol20260 2024-09-10T09:59:19.165 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T09:59:19.165 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:19.168 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol20260 subvol_grp75220 2024-09-10T09:59:19.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f82a0800640 1 Processor -- start 2024-09-10T09:59:19.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f82a0800640 1 -- start start 2024-09-10T09:59:19.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f82a0800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f82980cfe30 0x7f8298071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f82a0800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f82980cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f82a0800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f829806d2d0 con 0x7f82980cd510 2024-09-10T09:59:19.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f82a0800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f829815d9a0 con 0x7f82980cfe30 2024-09-10T09:59:19.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f82a0800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f829806d5b0 con 0x7f829806e7a0 2024-09-10T09:59:19.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f82980cfe30 0x7f8298071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f82980cfe30 0x7f8298071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54080/0 (socket says 172.21.15.110:54080) 2024-09-10T09:59:19.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829de00640 1 -- 172.21.15.110:0/3371804742 learned_addr learned my addr 172.21.15.110:0/3371804742 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:19.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829e800640 1 --2- 172.21.15.110:0/3371804742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f82980cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829de00640 1 -- 172.21.15.110:0/3371804742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 msgr2=0x7f82980cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829de00640 1 --2- 172.21.15.110:0/3371804742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f82980cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829de00640 1 -- 172.21.15.110:0/3371804742 >> v1:172.21.15.145:6789/0 conn(0x7f82980cd510 legacy=0x7f829806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829de00640 1 -- 172.21.15.110:0/3371804742 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829806d890 con 0x7f82980cfe30 2024-09-10T09:59:19.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.347+0000 7f829e800640 1 --2- 172.21.15.110:0/3371804742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f82980cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:19.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f829de00640 1 --2- 172.21.15.110:0/3371804742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298071a40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f828c009870 tx=0x7f828c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5ac6a5efe582c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f829ca00640 1 -- 172.21.15.110:0/3371804742 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0b1070 con 0x7f82980cfe30 2024-09-10T09:59:19.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f829ca00640 1 -- 172.21.15.110:0/3371804742 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c0a8050 con 0x7f82980cfe30 2024-09-10T09:59:19.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f829ca00640 1 -- 172.21.15.110:0/3371804742 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0ac860 con 0x7f82980cfe30 2024-09-10T09:59:19.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f82a0800640 1 -- 172.21.15.110:0/3371804742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 msgr2=0x7f8298071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f82a0800640 1 --2- 172.21.15.110:0/3371804742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298071a40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f828c009870 tx=0x7f828c01c4d0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f82a0800640 1 -- 172.21.15.110:0/3371804742 shutdown_connections 2024-09-10T09:59:19.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f82a0800640 1 --2- 172.21.15.110:0/3371804742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f82980cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f82a0800640 1 --2- 172.21.15.110:0/3371804742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298071a40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f82a0800640 1 -- 172.21.15.110:0/3371804742 >> 172.21.15.110:0/3371804742 conn(0x7f829806bc50 msgr2=0x7f829806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f82a0800640 1 -- 172.21.15.110:0/3371804742 shutdown_connections 2024-09-10T09:59:19.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.351+0000 7f82a0800640 1 -- 172.21.15.110:0/3371804742 wait complete. 2024-09-10T09:59:19.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 Processor -- start 2024-09-10T09:59:19.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 -- start start 2024-09-10T09:59:19.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f8298147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82980cd510 0x7f8298156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8298072190 con 0x7f82980cfe30 2024-09-10T09:59:19.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f829815d6c0 con 0x7f82980cd510 2024-09-10T09:59:19.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8298169180 con 0x7f829806e7a0 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54092/0 (socket says 172.21.15.110:54092) 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 -- 172.21.15.110:0/2509149173 learned_addr learned my addr 172.21.15.110:0/2509149173 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829de00640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f8298147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829d400640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82980cd510 0x7f8298156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 -- 172.21.15.110:0/2509149173 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 msgr2=0x7f8298147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f8298147930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 -- 172.21.15.110:0/2509149173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82980cd510 msgr2=0x7f8298156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82980cd510 0x7f8298156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 -- 172.21.15.110:0/2509149173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82980af740 con 0x7f82980cfe30 2024-09-10T09:59:19.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f829e800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298153e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8294007df0 tx=0x7f829400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f8292000640 1 -- 172.21.15.110:0/2509149173 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829401a070 con 0x7f82980cfe30 2024-09-10T09:59:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f8292000640 1 -- 172.21.15.110:0/2509149173 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82940039a0 con 0x7f82980cfe30 2024-09-10T09:59:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82980af9e0 con 0x7f82980cfe30 2024-09-10T09:59:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f8292000640 1 -- 172.21.15.110:0/2509149173 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294003d90 con 0x7f82980cfe30 2024-09-10T09:59:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82980b1e00 con 0x7f82980cfe30 2024-09-10T09:59:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.355+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82980cc760 con 0x7f82980cfe30 2024-09-10T09:59:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.359+0000 7f8292000640 1 -- 172.21.15.110:0/2509149173 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f829401d070 con 0x7f82980cfe30 2024-09-10T09:59:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.359+0000 7f8292000640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f827c0678c0 0x7f827c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.359+0000 7f8292000640 1 -- 172.21.15.110:0/2509149173 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8294085f20 con 0x7f82980cfe30 2024-09-10T09:59:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.359+0000 7f829de00640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f827c0678c0 0x7f827c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.359+0000 7f829de00640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f827c0678c0 0x7f827c069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f828c0ae000 tx=0x7f828c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.363+0000 7f8292000640 1 -- 172.21.15.110:0/2509149173 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8294010070 con 0x7f82980cfe30 2024-09-10T09:59:19.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.547+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol20260", "group_name": "subvol_grp75220", "target": ["mon-mgr", ""]}) -- 0x7f8298148060 con 0x7f827c0678c0 2024-09-10T09:59:19.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.571+0000 7f8292000640 1 -- 172.21.15.110:0/2509149173 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8298148060 con 0x7f827c0678c0 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f827c0678c0 msgr2=0x7f827c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f827c0678c0 0x7f827c069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f828c0ae000 tx=0x7f828c0a7000 comp rx=0 tx=0).stop 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 msgr2=0x7f8298153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298153e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8294007df0 tx=0x7f829400d4a0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 shutdown_connections 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f827c0678c0 0x7f827c069d80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82980cfe30 0x7f8298153e70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82980cd510 0x7f8298156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 --2- 172.21.15.110:0/2509149173 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f829806e7a0 0x7f8298147930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 >> 172.21.15.110:0/2509149173 conn(0x7f829806bc50 msgr2=0x7f829815bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 shutdown_connections 2024-09-10T09:59:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f82a0800640 1 -- 172.21.15.110:0/2509149173 wait complete. 2024-09-10T09:59:19.581 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.581 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.581 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.581 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.581 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.575+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.598 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.595+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.598 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.595+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.598 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.595+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.598 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.595+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.598 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:19.595+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:19.604 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T09:59:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:59:19.670 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-10T09:59:19.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:19.678 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp75220 2024-09-10T09:59:19.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.811+0000 7f8de4c00640 1 Processor -- start 2024-09-10T09:59:19.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.811+0000 7f8de4c00640 1 -- start start 2024-09-10T09:59:19.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8de4c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8de00b2bb0 0x7f8de00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8de4c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 0x7f8de00b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8de4c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de00b37c0 con 0x7f8de0145f30 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8de4c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8de00b3aa0 con 0x7f8de00b2bb0 2024-09-10T09:59:19.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8de4c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8de00b3d80 con 0x7f8de00cfbc0 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8ddd400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8de00b2bb0 0x7f8de00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8ddde00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f8de0145f30 0x7f8de00b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44186/0 (socket says 172.21.15.110:44186) 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8ddde00640 1 -- 172.21.15.110:0/4260303842 learned_addr learned my addr 172.21.15.110:0/4260303842 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8dde800640 1 --2- 172.21.15.110:0/4260303842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 0x7f8de00b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8ddd400640 1 -- 172.21.15.110:0/4260303842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 msgr2=0x7f8de00b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8ddd400640 1 --2- 172.21.15.110:0/4260303842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 0x7f8de00b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8ddd400640 1 -- 172.21.15.110:0/4260303842 >> v1:172.21.15.110:6789/0 conn(0x7f8de0145f30 legacy=0x7f8de00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8ddd400640 1 -- 172.21.15.110:0/4260303842 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0146310 con 0x7f8de00b2bb0 2024-09-10T09:59:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.815+0000 7f8ddca00640 1 -- 172.21.15.110:0/4260303842 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2530337262 0 0) 0x7f8de00b37c0 con 0x7f8de0145f30 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8ddd400640 1 --2- 172.21.15.110:0/4260303842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de00cf680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0097c0 tx=0x7f8dcc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fe1c99ec186503 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8ddca00640 1 -- 172.21.15.110:0/4260303842 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0b0070 con 0x7f8de00b2bb0 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8ddca00640 1 -- 172.21.15.110:0/4260303842 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dcc0ab600 con 0x7f8de00b2bb0 2024-09-10T09:59:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8ddca00640 1 -- 172.21.15.110:0/4260303842 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0ab9d0 con 0x7f8de00b2bb0 2024-09-10T09:59:19.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- 172.21.15.110:0/4260303842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 msgr2=0x7f8de00cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 --2- 172.21.15.110:0/4260303842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de00cf680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0097c0 tx=0x7f8dcc01c4a0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- 172.21.15.110:0/4260303842 shutdown_connections 2024-09-10T09:59:19.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 --2- 172.21.15.110:0/4260303842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 0x7f8de00b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 --2- 172.21.15.110:0/4260303842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de00cf680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- 172.21.15.110:0/4260303842 >> 172.21.15.110:0/4260303842 conn(0x7f8de0062e40 msgr2=0x7f8de006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:19.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- 172.21.15.110:0/4260303842 shutdown_connections 2024-09-10T09:59:19.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- 172.21.15.110:0/4260303842 wait complete. 2024-09-10T09:59:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 Processor -- start 2024-09-10T09:59:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- start start 2024-09-10T09:59:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de015f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 0x7f8de013aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8de0145f30 0x7f8de0138c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8de00b5660 con 0x7f8de00b2bb0 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8de00b54e0 con 0x7f8de0145f30 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.819+0000 7f8de4c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8de00b57e0 con 0x7f8de00cfbc0 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de015f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54154/0 (socket says 172.21.15.110:54154) 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 -- 172.21.15.110:0/297008486 learned_addr learned my addr 172.21.15.110:0/297008486 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddd400640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 0x7f8de013aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8dde800640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8de0145f30 0x7f8de0138c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 -- 172.21.15.110:0/297008486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 msgr2=0x7f8de013aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 0x7f8de013aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 -- 172.21.15.110:0/297008486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8de0145f30 msgr2=0x7f8de0138c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:19.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8de0145f30 0x7f8de0138c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:19.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 -- 172.21.15.110:0/297008486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0139d20 con 0x7f8de00b2bb0 2024-09-10T09:59:19.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8ddde00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de015f030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000cb10 tx=0x7f8dd000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8dd6000640 1 -- 172.21.15.110:0/297008486 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0017070 con 0x7f8de00b2bb0 2024-09-10T09:59:19.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8dd6000640 1 -- 172.21.15.110:0/297008486 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd000abb0 con 0x7f8de00b2bb0 2024-09-10T09:59:19.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8dd6000640 1 -- 172.21.15.110:0/297008486 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0012400 con 0x7f8de00b2bb0 2024-09-10T09:59:19.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de013a000 con 0x7f8de00b2bb0 2024-09-10T09:59:19.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de013a5c0 con 0x7f8de00b2bb0 2024-09-10T09:59:19.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de00cf680 con 0x7f8de00b2bb0 2024-09-10T09:59:19.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.823+0000 7f8dd6000640 1 -- 172.21.15.110:0/297008486 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8dd001a070 con 0x7f8de00b2bb0 2024-09-10T09:59:19.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.827+0000 7f8dd6000640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8dc00678c0 0x7f8dc0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:19.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.827+0000 7f8dd6000640 1 -- 172.21.15.110:0/297008486 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8dd0085930 con 0x7f8de00b2bb0 2024-09-10T09:59:19.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.827+0000 7f8ddd400640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8dc00678c0 0x7f8dc0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:19.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.827+0000 7f8ddd400640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8dc00678c0 0x7f8dc0069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0ad000 tx=0x7f8dcc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:19.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:19.831+0000 7f8dd6000640 1 -- 172.21.15.110:0/297008486 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8dd0055e10 con 0x7f8de00b2bb0 2024-09-10T09:59:20.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.023+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp75220", "target": ["mon-mgr", ""]}) -- 0x7f8de013a800 con 0x7f8dc00678c0 2024-09-10T09:59:20.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8dd6000640 1 -- 172.21.15.110:0/297008486 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8de013a800 con 0x7f8dc00678c0 2024-09-10T09:59:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8dc00678c0 msgr2=0x7f8dc0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8dc00678c0 0x7f8dc0069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0ad000 tx=0x7f8dcc008040 comp rx=0 tx=0).stop 2024-09-10T09:59:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 msgr2=0x7f8de015f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de015f030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000cb10 tx=0x7f8dd000cf10 comp rx=0 tx=0).stop 2024-09-10T09:59:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 shutdown_connections 2024-09-10T09:59:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8dc00678c0 0x7f8dc0069d80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8de0145f30 0x7f8de0138c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8de00cfbc0 0x7f8de013aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 --2- 172.21.15.110:0/297008486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8de00b2bb0 0x7f8de015f030 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 >> 172.21.15.110:0/297008486 conn(0x7f8de0062e40 msgr2=0x7f8de0144e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 shutdown_connections 2024-09-10T09:59:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.027+0000 7f8de4c00640 1 -- 172.21.15.110:0/297008486 wait complete. 2024-09-10T09:59:20.051 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T09:59:20.052 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T09:59:20.147 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T09:59:20.237 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T09:59:20.237 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T09:59:20.237 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:20.237 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:20.246 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:20.246 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:20.297 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T09:59:20.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.475+0000 7f05d5a00640 1 Processor -- start 2024-09-10T09:59:20.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.475+0000 7f05d5a00640 1 -- start start 2024-09-10T09:59:20.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05d5a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4ce0 0x7f05d0165690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05d5a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d015ae90 con 0x7f05d015aab0 2024-09-10T09:59:20.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05d5a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d015b170 con 0x7f05d00b4170 2024-09-10T09:59:20.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05d5a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f05d015b450 con 0x7f05d00b4ce0 2024-09-10T09:59:20.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cea00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f05d015aab0 0x7f05d0072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44188/0 (socket says 172.21.15.110:44188) 2024-09-10T09:59:20.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cea00640 1 -- 172.21.15.110:0/2017324755 learned_addr learned my addr 172.21.15.110:0/2017324755 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:20.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cf400640 1 --2- 172.21.15.110:0/2017324755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4ce0 0x7f05d0165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cd600640 1 -- 172.21.15.110:0/2017324755 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316944496 0 0) 0x7f05d015ae90 con 0x7f05d015aab0 2024-09-10T09:59:20.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cd600640 1 -- 172.21.15.110:0/2017324755 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b4003530 con 0x7f05d015aab0 2024-09-10T09:59:20.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cd600640 1 -- 172.21.15.110:0/2017324755 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709367872 0 0) 0x7f05d015b170 con 0x7f05d00b4170 2024-09-10T09:59:20.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cd600640 1 -- 172.21.15.110:0/2017324755 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05d015ae90 con 0x7f05d00b4170 2024-09-10T09:59:20.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cf400640 1 -- 172.21.15.110:0/2017324755 >> v1:172.21.15.145:6789/0 conn(0x7f05d00b4170 legacy=0x7f05d00b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cf400640 1 -- 172.21.15.110:0/2017324755 >> v1:172.21.15.110:6789/0 conn(0x7f05d015aab0 legacy=0x7f05d0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.479+0000 7f05cf400640 1 -- 172.21.15.110:0/2017324755 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d015b730 con 0x7f05d00b4ce0 2024-09-10T09:59:20.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05cf400640 1 --2- 172.21.15.110:0/2017324755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4ce0 0x7f05d0165690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05c80097f0 tx=0x7f05c801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b1490cc5a33e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:20.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05cd600640 1 -- 172.21.15.110:0/2017324755 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c80b0070 con 0x7f05d00b4ce0 2024-09-10T09:59:20.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05cd600640 1 -- 172.21.15.110:0/2017324755 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c8004230 con 0x7f05d00b4ce0 2024-09-10T09:59:20.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05cd600640 1 -- 172.21.15.110:0/2017324755 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8003620 con 0x7f05d00b4ce0 2024-09-10T09:59:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 -- 172.21.15.110:0/2017324755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4ce0 msgr2=0x7f05d0165690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 --2- 172.21.15.110:0/2017324755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4ce0 0x7f05d0165690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05c80097f0 tx=0x7f05c801c4e0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 -- 172.21.15.110:0/2017324755 shutdown_connections 2024-09-10T09:59:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 --2- 172.21.15.110:0/2017324755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4ce0 0x7f05d0165690 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 -- 172.21.15.110:0/2017324755 >> 172.21.15.110:0/2017324755 conn(0x7f05d0062d10 msgr2=0x7f05d006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 -- 172.21.15.110:0/2017324755 shutdown_connections 2024-09-10T09:59:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 -- 172.21.15.110:0/2017324755 wait complete. 2024-09-10T09:59:20.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 Processor -- start 2024-09-10T09:59:20.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.483+0000 7f05d5a00640 1 -- start start 2024-09-10T09:59:20.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4170 0x7f05d00c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d00b4ce0 0x7f05d0158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d015aab0 0x7f05d014a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0169090 con 0x7f05d015aab0 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0168f10 con 0x7f05d00b4ce0 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f05d0169210 con 0x7f05d00b4170 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d015aab0 0x7f05d014a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4170 0x7f05d00c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05ce000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d00b4ce0 0x7f05d0158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05ce000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d00b4ce0 0x7f05d0158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48710/0 (socket says 172.21.15.110:48710) 2024-09-10T09:59:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05ce000640 1 -- 172.21.15.110:0/2018977236 learned_addr learned my addr 172.21.15.110:0/2018977236 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4170 0x7f05d00c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:40686/0 (socket says 172.21.15.110:40686) 2024-09-10T09:59:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d015aab0 0x7f05d014a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54166/0 (socket says 172.21.15.110:54166) 2024-09-10T09:59:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cf400640 1 -- 172.21.15.110:0/2018977236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4170 msgr2=0x7f05d00c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cf400640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4170 0x7f05d00c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cf400640 1 -- 172.21.15.110:0/2018977236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d00b4ce0 msgr2=0x7f05d0158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cf400640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d00b4ce0 0x7f05d0158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cf400640 1 -- 172.21.15.110:0/2018977236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d00c5bb0 con 0x7f05d015aab0 2024-09-10T09:59:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05cf400640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d015aab0 0x7f05d014a980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f05c8004000 tx=0x7f05c8019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:20.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05c6a00640 1 -- 172.21.15.110:0/2018977236 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c80b0040 con 0x7f05d015aab0 2024-09-10T09:59:20.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05c6a00640 1 -- 172.21.15.110:0/2018977236 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c80bb070 con 0x7f05d015aab0 2024-09-10T09:59:20.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05c6a00640 1 -- 172.21.15.110:0/2018977236 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8003d20 con 0x7f05d015aab0 2024-09-10T09:59:20.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d00c5e90 con 0x7f05d015aab0 2024-09-10T09:59:20.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d00c6450 con 0x7f05d015aab0 2024-09-10T09:59:20.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d00b45d0 con 0x7f05d015aab0 2024-09-10T09:59:20.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.487+0000 7f05c6a00640 1 -- 172.21.15.110:0/2018977236 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f05c80b3070 con 0x7f05d015aab0 2024-09-10T09:59:20.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.491+0000 7f05c6a00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f05b0067950 0x7f05b0069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.491+0000 7f05c6a00640 1 -- 172.21.15.110:0/2018977236 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f05c811f9a0 con 0x7f05d015aab0 2024-09-10T09:59:20.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.491+0000 7f05cea00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f05b0067950 0x7f05b0069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.491+0000 7f05cea00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f05b0067950 0x7f05b0069e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f05c00012f0 tx=0x7f05c0001a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:20.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.495+0000 7f05c6a00640 1 -- 172.21.15.110:0/2018977236 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f05c80160d0 con 0x7f05d015aab0 2024-09-10T09:59:20.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f05d01685b0 con 0x7f05d015aab0 2024-09-10T09:59:20.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05c6a00640 1 -- 172.21.15.110:0/2018977236 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f05c80efc90 con 0x7f05d015aab0 2024-09-10T09:59:20.671 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f05b0067950 msgr2=0x7f05b0069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f05b0067950 0x7f05b0069e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f05c00012f0 tx=0x7f05c0001a20 comp rx=0 tx=0).stop 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d015aab0 msgr2=0x7f05d014a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d015aab0 0x7f05d014a980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f05c8004000 tx=0x7f05c8019280 comp rx=0 tx=0).stop 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 shutdown_connections 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f05b0067950 0x7f05b0069e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05d015aab0 0x7f05d014a980 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05d00b4ce0 0x7f05d0158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 --2- 172.21.15.110:0/2018977236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05d00b4170 0x7f05d00c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 >> 172.21.15.110:0/2018977236 conn(0x7f05d0062d10 msgr2=0x7f05d00b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:20.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 shutdown_connections 2024-09-10T09:59:20.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.667+0000 7f05d5a00640 1 -- 172.21.15.110:0/2018977236 wait complete. 2024-09-10T09:59:20.692 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T09:59:20.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:20.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:20.747 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:59:20.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:20.748 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T09:59:20.766 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T09:59:20.766 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:20.766 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:20.777 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:20.777 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:20.830 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T09:59:20.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.963+0000 7f546d600640 1 Processor -- start 2024-09-10T09:59:20.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.963+0000 7f546d600640 1 -- start start 2024-09-10T09:59:20.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.963+0000 7f546d600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f546815e540 0x7f54680b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.963+0000 7f546d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54680626e0 0x7f54680af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.963+0000 7f546d600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54680ae9e0 con 0x7f54680b5880 2024-09-10T09:59:20.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.963+0000 7f546d600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f54680aecc0 con 0x7f546815e540 2024-09-10T09:59:20.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.963+0000 7f546d600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f54680aefa0 con 0x7f54680626e0 2024-09-10T09:59:20.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5467400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f54680b5880 0x7f546806db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44200/0 (socket says 172.21.15.110:44200) 2024-09-10T09:59:20.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5467400640 1 -- 172.21.15.110:0/1674194936 learned_addr learned my addr 172.21.15.110:0/1674194936 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:20.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466a00640 1 --2- 172.21.15.110:0/1674194936 >> v2:172.21.15.145:3300/0 conn(0x7f546815e540 0x7f54680b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2580063148 0 0) 0x7f54680ae9e0 con 0x7f54680b5880 2024-09-10T09:59:20.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f544c003530 con 0x7f54680b5880 2024-09-10T09:59:20.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f545e000640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54680626e0 0x7f54680af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 468891372 0 0) 0x7f544c003530 con 0x7f54680b5880 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54680626e0 msgr2=0x7f54680af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54680626e0 0x7f54680af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 >> v2:172.21.15.145:3300/0 conn(0x7f546815e540 msgr2=0x7f54680b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 --2- 172.21.15.110:0/1674194936 >> v2:172.21.15.145:3300/0 conn(0x7f546815e540 0x7f54680b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54681621f0 con 0x7f54680b5880 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f5460001a00 con 0x7f54680b5880 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 >> v1:172.21.15.110:6789/0 conn(0x7f54680b5880 legacy=0x7f546806db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c0039c0 0x7f544c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54680626e0 0x7f544c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544c004d50 0x7f544c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54680aecc0 con 0x7f544c0039c0 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54680aefa0 con 0x7f544c004d50 2024-09-10T09:59:20.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f544c00ba00 con 0x7f54680626e0 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466a00640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c0039c0 0x7f544c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f545e000640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54680626e0 0x7f544c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5467400640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544c004d50 0x7f544c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466a00640 1 -- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54680626e0 msgr2=0x7f544c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466a00640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54680626e0 0x7f544c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466a00640 1 -- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544c004d50 msgr2=0x7f544c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466a00640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544c004d50 0x7f544c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5466a00640 1 -- 172.21.15.110:0/1674194936 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c003530 con 0x7f544c0039c0 2024-09-10T09:59:20.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.967+0000 7f5467400640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544c004d50 0x7f544c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:20.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f5466a00640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c0039c0 0x7f544c003de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5458009ae0 tx=0x7f545801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3c88fa59992c18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:20.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580096b0 con 0x7f544c0039c0 2024-09-10T09:59:20.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54580a8410 con 0x7f544c0039c0 2024-09-10T09:59:20.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f5466000640 1 -- 172.21.15.110:0/1674194936 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580a8800 con 0x7f544c0039c0 2024-09-10T09:59:20.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f546d600640 1 -- 172.21.15.110:0/1674194936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c0039c0 msgr2=0x7f544c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f546d600640 1 --2- 172.21.15.110:0/1674194936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f544c0039c0 0x7f544c003de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5458009ae0 tx=0x7f545801cab0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f5467400640 1 -- 172.21.15.110:0/1674194936 reap_dead start 2024-09-10T09:59:20.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f546d600640 1 -- 172.21.15.110:0/1674194936 shutdown_connections 2024-09-10T09:59:20.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f546d600640 1 -- 172.21.15.110:0/1674194936 >> 172.21.15.110:0/1674194936 conn(0x7f54680b0970 msgr2=0x7f54680b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:20.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f546d600640 1 -- 172.21.15.110:0/1674194936 shutdown_connections 2024-09-10T09:59:20.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f546d600640 1 -- 172.21.15.110:0/1674194936 wait complete. 2024-09-10T09:59:20.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f546d600640 1 Processor -- start 2024-09-10T09:59:20.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.971+0000 7f546d600640 1 -- start start 2024-09-10T09:59:20.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f544c0039c0 0x7f546815c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5468144530 0x7f54681533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5468153920 0x7f546815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5468169370 con 0x7f5468144530 2024-09-10T09:59:20.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54680b01c0 con 0x7f5468153920 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f54681694f0 con 0x7f544c0039c0 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5468144530 0x7f54681533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5468144530 0x7f54681533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54180/0 (socket says 172.21.15.110:54180) 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 -- 172.21.15.110:0/3172949639 learned_addr learned my addr 172.21.15.110:0/3172949639 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f5466a00640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f544c0039c0 0x7f546815c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f5467400640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5468153920 0x7f546815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 -- 172.21.15.110:0/3172949639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f544c0039c0 msgr2=0x7f546815c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f544c0039c0 0x7f546815c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 -- 172.21.15.110:0/3172949639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5468153920 msgr2=0x7f546815a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5468153920 0x7f546815a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 -- 172.21.15.110:0/3172949639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5468158910 con 0x7f5468144530 2024-09-10T09:59:20.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545e000640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5468144530 0x7f54681533e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5450011930 tx=0x7f5450011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:20.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545f400640 1 -- 172.21.15.110:0/3172949639 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545000fa30 con 0x7f5468144530 2024-09-10T09:59:20.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545f400640 1 -- 172.21.15.110:0/3172949639 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f545000fbd0 con 0x7f5468144530 2024-09-10T09:59:20.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545f400640 1 -- 172.21.15.110:0/3172949639 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450014440 con 0x7f5468144530 2024-09-10T09:59:20.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5468158bf0 con 0x7f5468144530 2024-09-10T09:59:20.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54681591b0 con 0x7f5468144530 2024-09-10T09:59:20.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54680b5340 con 0x7f5468144530 2024-09-10T09:59:20.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.975+0000 7f545f400640 1 -- 172.21.15.110:0/3172949639 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f5450012040 con 0x7f5468144530 2024-09-10T09:59:20.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.979+0000 7f545f400640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5444067980 0x7f5444069e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:20.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.979+0000 7f545f400640 1 -- 172.21.15.110:0/3172949639 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5450086340 con 0x7f5468144530 2024-09-10T09:59:20.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.979+0000 7f5466a00640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5444067980 0x7f5444069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:20.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.979+0000 7f5466a00640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5444067980 0x7f5444069e40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54580af000 tx=0x7f54580b3040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:20.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:20.983+0000 7f545f400640 1 -- 172.21.15.110:0/3172949639 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5450019030 con 0x7f5468144530 2024-09-10T09:59:21.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5468153d00 con 0x7f5468144530 2024-09-10T09:59:21.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f545f400640 1 -- 172.21.15.110:0/3172949639 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f5450056d90 con 0x7f5468144530 2024-09-10T09:59:21.154 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5444067980 msgr2=0x7f5444069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5444067980 0x7f5444069e40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54580af000 tx=0x7f54580b3040 comp rx=0 tx=0).stop 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5468144530 msgr2=0x7f54681533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5468144530 0x7f54681533e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5450011930 tx=0x7f5450011d30 comp rx=0 tx=0).stop 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 shutdown_connections 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5444067980 0x7f5444069e40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5468153920 0x7f546815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5468144530 0x7f54681533e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 --2- 172.21.15.110:0/3172949639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f544c0039c0 0x7f546815c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 >> 172.21.15.110:0/3172949639 conn(0x7f54680b0970 msgr2=0x7f54681632e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 shutdown_connections 2024-09-10T09:59:21.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.151+0000 7f546d600640 1 -- 172.21.15.110:0/3172949639 wait complete. 2024-09-10T09:59:21.174 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T09:59:21.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:21.174 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:21.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:59:21.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:21.203 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T09:59:21.259 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T09:59:21.259 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:21.259 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:21.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T09:59:21.268 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T09:59:21.268 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:21.268 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:21.311 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T09:59:21.312 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:21.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.443+0000 7f026c800640 1 Processor -- start 2024-09-10T09:59:21.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.443+0000 7f026c800640 1 -- start start 2024-09-10T09:59:21.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.443+0000 7f026c800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f026406e7a0 0x7f02640cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.443+0000 7f026c800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0264072150 con 0x7f02640cd510 2024-09-10T09:59:21.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.443+0000 7f026c800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026406d2d0 con 0x7f02640cfe30 2024-09-10T09:59:21.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f026c800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f026415d9a0 con 0x7f026406e7a0 2024-09-10T09:59:21.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f0269400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f02640cd510 0x7f026406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44214/0 (socket says 172.21.15.110:44214) 2024-09-10T09:59:21.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f0269400640 1 -- 172.21.15.110:0/3490732646 learned_addr learned my addr 172.21.15.110:0/3490732646 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:21.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f026a800640 1 --2- 172.21.15.110:0/3490732646 >> v2:172.21.15.145:3300/0 conn(0x7f026406e7a0 0x7f02640cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f0268a00640 1 -- 172.21.15.110:0/3490732646 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 523074479 0 0) 0x7f0264072150 con 0x7f02640cd510 2024-09-10T09:59:21.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f0268a00640 1 -- 172.21.15.110:0/3490732646 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f024c003530 con 0x7f02640cd510 2024-09-10T09:59:21.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f0268a00640 1 -- 172.21.15.110:0/3490732646 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025158619 0 0) 0x7f026406d2d0 con 0x7f02640cfe30 2024-09-10T09:59:21.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f0268a00640 1 -- 172.21.15.110:0/3490732646 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0264072150 con 0x7f02640cfe30 2024-09-10T09:59:21.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f026a800640 1 -- 172.21.15.110:0/3490732646 >> v1:172.21.15.145:6789/0 conn(0x7f02640cfe30 legacy=0x7f0264071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f026a800640 1 -- 172.21.15.110:0/3490732646 >> v1:172.21.15.110:6789/0 conn(0x7f02640cd510 legacy=0x7f026406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f026a800640 1 -- 172.21.15.110:0/3490732646 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026406d5b0 con 0x7f026406e7a0 2024-09-10T09:59:21.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.447+0000 7f026a800640 1 --2- 172.21.15.110:0/3490732646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f026406e7a0 0x7f02640cd9a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0260009d90 tx=0x7f026001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd9520053c8133cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f0268a00640 1 -- 172.21.15.110:0/3490732646 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600b2070 con 0x7f026406e7a0 2024-09-10T09:59:21.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f0268a00640 1 -- 172.21.15.110:0/3490732646 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02600a58b0 con 0x7f026406e7a0 2024-09-10T09:59:21.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f0268a00640 1 -- 172.21.15.110:0/3490732646 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600a5c80 con 0x7f026406e7a0 2024-09-10T09:59:21.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- 172.21.15.110:0/3490732646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f026406e7a0 msgr2=0x7f02640cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 --2- 172.21.15.110:0/3490732646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f026406e7a0 0x7f02640cd9a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0260009d90 tx=0x7f026001c8e0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- 172.21.15.110:0/3490732646 shutdown_connections 2024-09-10T09:59:21.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 --2- 172.21.15.110:0/3490732646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f026406e7a0 0x7f02640cd9a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- 172.21.15.110:0/3490732646 >> 172.21.15.110:0/3490732646 conn(0x7f026406bc50 msgr2=0x7f026406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- 172.21.15.110:0/3490732646 shutdown_connections 2024-09-10T09:59:21.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- 172.21.15.110:0/3490732646 wait complete. 2024-09-10T09:59:21.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 Processor -- start 2024-09-10T09:59:21.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- start start 2024-09-10T09:59:21.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f026406e7a0 0x7f0264148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02640cd510 0x7f02640b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02640cfe30 0x7f0264145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f026416a190 con 0x7f026406e7a0 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f026415d700 con 0x7f02640cd510 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026c800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f026416a310 con 0x7f02640cfe30 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f0269e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f026406e7a0 0x7f0264148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f0269e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f026406e7a0 0x7f0264148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54188/0 (socket says 172.21.15.110:54188) 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f0269e00640 1 -- 172.21.15.110:0/1262432013 learned_addr learned my addr 172.21.15.110:0/1262432013 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f0269400640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02640cd510 0x7f02640b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.451+0000 7f026a800640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02640cfe30 0x7f0264145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f0269e00640 1 -- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02640cfe30 msgr2=0x7f0264145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f0269e00640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02640cfe30 0x7f0264145ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f0269e00640 1 -- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02640cd510 msgr2=0x7f02640b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f0269e00640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02640cd510 0x7f02640b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f0269e00640 1 -- 172.21.15.110:0/1262432013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0264149040 con 0x7f026406e7a0 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f0269400640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02640cd510 0x7f02640b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f026a800640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02640cfe30 0x7f0264145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f0269e00640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f026406e7a0 0x7f0264148860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f025800c430 tx=0x7f025800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f025e000640 1 -- 172.21.15.110:0/1262432013 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258017070 con 0x7f026406e7a0 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f025e000640 1 -- 172.21.15.110:0/1262432013 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02580052f0 con 0x7f026406e7a0 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f025e000640 1 -- 172.21.15.110:0/1262432013 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258003840 con 0x7f026406e7a0 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02640b2a60 con 0x7f026406e7a0 2024-09-10T09:59:21.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02640b26c0 con 0x7f026406e7a0 2024-09-10T09:59:21.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026406e090 con 0x7f026406e7a0 2024-09-10T09:59:21.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f025e000640 1 -- 172.21.15.110:0/1262432013 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f025801a070 con 0x7f026406e7a0 2024-09-10T09:59:21.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.455+0000 7f025e000640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0248067950 0x7f0248069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.459+0000 7f025e000640 1 -- 172.21.15.110:0/1262432013 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0258085050 con 0x7f026406e7a0 2024-09-10T09:59:21.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.459+0000 7f0269400640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0248067950 0x7f0248069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.459+0000 7f0269400640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0248067950 0x7f0248069e10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02640cf620 tx=0x7f0254009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.463+0000 7f025e000640 1 -- 172.21.15.110:0/1262432013 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0258055bb0 con 0x7f026406e7a0 2024-09-10T09:59:21.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.651+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0264075560 con 0x7f026406e7a0 2024-09-10T09:59:21.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.651+0000 7f025e000640 1 -- 172.21.15.110:0/1262432013 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f02580554b0 con 0x7f026406e7a0 2024-09-10T09:59:21.656 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:21.656 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":17,"btime":"2024-09-10T09:59:17:286112+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4556,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1141773053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1141773053},{"type":"v1","addr":"172.21.15.145:6834","nonce":1141773053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/656217845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":656217845},{"type":"v1","addr":"172.21.15.145:6835","nonce":656217845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2244539155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2244539155},{"type":"v1","addr":"172.21.15.110:6835","nonce":2244539155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:03.573094+0000","modified":"2024-09-10T09:59:06.627577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1972132715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1972132715},{"type":"v1","addr":"172.21.15.110:6837","nonce":1972132715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-09-10T09:59:21.657 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 17 2024-09-10T09:59:21.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0248067950 msgr2=0x7f0248069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0248067950 0x7f0248069e10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02640cf620 tx=0x7f0254009040 comp rx=0 tx=0).stop 2024-09-10T09:59:21.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f026406e7a0 msgr2=0x7f0264148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f026406e7a0 0x7f0264148860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f025800c430 tx=0x7f025800c8f0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 shutdown_connections 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0248067950 0x7f0248069e10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f02640cfe30 0x7f0264145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f02640cd510 0x7f02640b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 --2- 172.21.15.110:0/1262432013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f026406e7a0 0x7f0264148860 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 >> 172.21.15.110:0/1262432013 conn(0x7f026406bc50 msgr2=0x7f0264071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 shutdown_connections 2024-09-10T09:59:21.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.655+0000 7f026c800640 1 -- 172.21.15.110:0/1262432013 wait complete. 2024-09-10T09:59:21.677 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:21.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.807+0000 7f164ae00640 1 Processor -- start 2024-09-10T09:59:21.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.807+0000 7f164ae00640 1 -- start start 2024-09-10T09:59:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f164ae00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f164415c560 0x7f164406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f164ae00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f164406f010 0x7f16440ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f164ae00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f164406f430 con 0x7f1644155970 2024-09-10T09:59:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f164ae00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f16440b4570 con 0x7f164406f010 2024-09-10T09:59:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f164ae00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1644155d50 con 0x7f164415c560 2024-09-10T09:59:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1643e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1644155970 0x7f16440cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44222/0 (socket says 172.21.15.110:44222) 2024-09-10T09:59:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1643e00640 1 -- 172.21.15.110:0/1043955282 learned_addr learned my addr 172.21.15.110:0/1043955282 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1648e00640 1 --2- 172.21.15.110:0/1043955282 >> v2:172.21.15.110:3300/0 conn(0x7f164406f010 0x7f16440ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1643400640 1 --2- 172.21.15.110:0/1043955282 >> v2:172.21.15.145:3300/0 conn(0x7f164415c560 0x7f164406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1648e00640 1 -- 172.21.15.110:0/1043955282 >> v2:172.21.15.145:3300/0 conn(0x7f164415c560 msgr2=0x7f164406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1648e00640 1 --2- 172.21.15.110:0/1043955282 >> v2:172.21.15.145:3300/0 conn(0x7f164415c560 0x7f164406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1648e00640 1 -- 172.21.15.110:0/1043955282 >> v1:172.21.15.110:6789/0 conn(0x7f1644155970 legacy=0x7f16440cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1648e00640 1 -- 172.21.15.110:0/1043955282 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16440b34d0 con 0x7f164406f010 2024-09-10T09:59:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.811+0000 7f1642a00640 1 -- 172.21.15.110:0/1043955282 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3105957965 0 0) 0x7f164406f430 con 0x7f1644155970 2024-09-10T09:59:21.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f1648e00640 1 --2- 172.21.15.110:0/1043955282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164406f010 0x7f16440ce420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f164406d930 tx=0x7f163c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab31e1c582f61062 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f1642a00640 1 -- 172.21.15.110:0/1043955282 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c0b2070 con 0x7f164406f010 2024-09-10T09:59:21.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f1642a00640 1 -- 172.21.15.110:0/1043955282 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c0a59c0 con 0x7f164406f010 2024-09-10T09:59:21.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f1642a00640 1 -- 172.21.15.110:0/1043955282 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c0a5d90 con 0x7f164406f010 2024-09-10T09:59:21.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- 172.21.15.110:0/1043955282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164406f010 msgr2=0x7f16440ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 --2- 172.21.15.110:0/1043955282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164406f010 0x7f16440ce420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f164406d930 tx=0x7f163c01c900 comp rx=0 tx=0).stop 2024-09-10T09:59:21.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- 172.21.15.110:0/1043955282 shutdown_connections 2024-09-10T09:59:21.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 --2- 172.21.15.110:0/1043955282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164406f010 0x7f16440ce420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 --2- 172.21.15.110:0/1043955282 >> v2:172.21.15.145:3300/0 conn(0x7f164415c560 0x7f164406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- 172.21.15.110:0/1043955282 >> 172.21.15.110:0/1043955282 conn(0x7f164406bc50 msgr2=0x7f164406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- 172.21.15.110:0/1043955282 shutdown_connections 2024-09-10T09:59:21.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- 172.21.15.110:0/1043955282 wait complete. 2024-09-10T09:59:21.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 Processor -- start 2024-09-10T09:59:21.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- start start 2024-09-10T09:59:21.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164406f010 0x7f16441452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1644155970 0x7f1644072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164415c560 0x7f1644145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16440cf170 con 0x7f164415c560 2024-09-10T09:59:21.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f16440b4050 con 0x7f164406f010 2024-09-10T09:59:21.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.815+0000 7f164ae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f16440cf2f0 con 0x7f1644155970 2024-09-10T09:59:21.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164415c560 0x7f1644145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164415c560 0x7f1644145cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54208/0 (socket says 172.21.15.110:54208) 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 -- 172.21.15.110:0/3654171336 learned_addr learned my addr 172.21.15.110:0/3654171336 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1643e00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164406f010 0x7f16441452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1643400640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1644155970 0x7f1644072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 -- 172.21.15.110:0/3654171336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1644155970 msgr2=0x7f1644072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1644155970 0x7f1644072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 -- 172.21.15.110:0/3654171336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164406f010 msgr2=0x7f16441452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164406f010 0x7f16441452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 -- 172.21.15.110:0/3654171336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16441457e0 con 0x7f164415c560 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f1648e00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164415c560 0x7f1644145cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f163c0b0040 tx=0x7f163c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f163be00640 1 -- 172.21.15.110:0/3654171336 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c0b2050 con 0x7f164415c560 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f163be00640 1 -- 172.21.15.110:0/3654171336 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c0b3040 con 0x7f164415c560 2024-09-10T09:59:21.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f163be00640 1 -- 172.21.15.110:0/3654171336 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c0ad850 con 0x7f164415c560 2024-09-10T09:59:21.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1644142ba0 con 0x7f164415c560 2024-09-10T09:59:21.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1644143160 con 0x7f164415c560 2024-09-10T09:59:21.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16440cf650 con 0x7f164415c560 2024-09-10T09:59:21.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.819+0000 7f163be00640 1 -- 172.21.15.110:0/3654171336 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f163c0c0020 con 0x7f164415c560 2024-09-10T09:59:21.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.823+0000 7f163be00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1624067950 0x7f1624069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.823+0000 7f163be00640 1 -- 172.21.15.110:0/3654171336 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f163c1207c0 con 0x7f164415c560 2024-09-10T09:59:21.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.823+0000 7f1643e00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1624067950 0x7f1624069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.823+0000 7f1643e00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1624067950 0x7f1624069e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1634007fd0 tx=0x7f1634007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:21.827+0000 7f163be00640 1 -- 172.21.15.110:0/3654171336 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f16441457e0 con 0x7f164415c560 2024-09-10T09:59:22.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.015+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f164406eb00 con 0x7f164415c560 2024-09-10T09:59:22.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.015+0000 7f163be00640 1 -- 172.21.15.110:0/3654171336 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f163c016520 con 0x7f164415c560 2024-09-10T09:59:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:22.020 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":17,"btime":"2024-09-10T09:59:17:286112+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4556,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1141773053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1141773053},{"type":"v1","addr":"172.21.15.145:6834","nonce":1141773053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/656217845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":656217845},{"type":"v1","addr":"172.21.15.145:6835","nonce":656217845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2244539155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2244539155},{"type":"v1","addr":"172.21.15.110:6835","nonce":2244539155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:03.573094+0000","modified":"2024-09-10T09:59:06.627577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1972132715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1972132715},{"type":"v1","addr":"172.21.15.110:6837","nonce":1972132715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-09-10T09:59:22.021 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 17 2024-09-10T09:59:22.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1624067950 msgr2=0x7f1624069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1624067950 0x7f1624069e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1634007fd0 tx=0x7f1634007d60 comp rx=0 tx=0).stop 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164415c560 msgr2=0x7f1644145cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164415c560 0x7f1644145cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f163c0b0040 tx=0x7f163c0196a0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 shutdown_connections 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1624067950 0x7f1624069e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f164415c560 0x7f1644145cf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1644155970 0x7f1644072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 --2- 172.21.15.110:0/3654171336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f164406f010 0x7f16441452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 >> 172.21.15.110:0/3654171336 conn(0x7f164406bc50 msgr2=0x7f16440cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 shutdown_connections 2024-09-10T09:59:22.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.019+0000 7f164ae00640 1 -- 172.21.15.110:0/3654171336 wait complete. 2024-09-10T09:59:22.041 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:22.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.171+0000 7fee8fc00640 1 Processor -- start 2024-09-10T09:59:22.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.171+0000 7fee8fc00640 1 -- start start 2024-09-10T09:59:22.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.171+0000 7fee8fc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fee88155970 0x7fee880cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.171+0000 7fee8fc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee8806f1e0 0x7fee880ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.171+0000 7fee8fc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee88155d50 con 0x7fee8815c560 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.171+0000 7fee8fc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fee880b4570 con 0x7fee88155970 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.171+0000 7fee8fc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fee880b34d0 con 0x7fee8806f1e0 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee87e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fee8815c560 0x7fee8806ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44236/0 (socket says 172.21.15.110:44236) 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee87e00640 1 -- 172.21.15.110:0/779009624 learned_addr learned my addr 172.21.15.110:0/779009624 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee8dc00640 1 --2- 172.21.15.110:0/779009624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee8806f1e0 0x7fee880ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee8d200640 1 --2- 172.21.15.110:0/779009624 >> v2:172.21.15.145:3300/0 conn(0x7fee88155970 0x7fee880cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee87400640 1 -- 172.21.15.110:0/779009624 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1168026088 0 0) 0x7fee88155d50 con 0x7fee8815c560 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee87400640 1 -- 172.21.15.110:0/779009624 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee6c003530 con 0x7fee8815c560 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee8dc00640 1 -- 172.21.15.110:0/779009624 >> v2:172.21.15.145:3300/0 conn(0x7fee88155970 msgr2=0x7fee880cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee8dc00640 1 --2- 172.21.15.110:0/779009624 >> v2:172.21.15.145:3300/0 conn(0x7fee88155970 0x7fee880cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee8dc00640 1 -- 172.21.15.110:0/779009624 >> v1:172.21.15.110:6789/0 conn(0x7fee8815c560 legacy=0x7fee8806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee8dc00640 1 -- 172.21.15.110:0/779009624 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee880b37b0 con 0x7fee8806f1e0 2024-09-10T09:59:22.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.175+0000 7fee8dc00640 1 --2- 172.21.15.110:0/779009624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee8806f1e0 0x7fee880ce420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee80009d90 tx=0x7fee8001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b10a46f57210fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee87400640 1 -- 172.21.15.110:0/779009624 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee800b2070 con 0x7fee8806f1e0 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee87400640 1 -- 172.21.15.110:0/779009624 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee800a5a00 con 0x7fee8806f1e0 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee87400640 1 -- 172.21.15.110:0/779009624 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee800a5dd0 con 0x7fee8806f1e0 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- 172.21.15.110:0/779009624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee8806f1e0 msgr2=0x7fee880ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 --2- 172.21.15.110:0/779009624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee8806f1e0 0x7fee880ce420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee80009d90 tx=0x7fee8001c940 comp rx=0 tx=0).stop 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- 172.21.15.110:0/779009624 shutdown_connections 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 --2- 172.21.15.110:0/779009624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee8806f1e0 0x7fee880ce420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 --2- 172.21.15.110:0/779009624 >> v2:172.21.15.145:3300/0 conn(0x7fee88155970 0x7fee880cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- 172.21.15.110:0/779009624 >> 172.21.15.110:0/779009624 conn(0x7fee8806bc50 msgr2=0x7fee8806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- 172.21.15.110:0/779009624 shutdown_connections 2024-09-10T09:59:22.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- 172.21.15.110:0/779009624 wait complete. 2024-09-10T09:59:22.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 Processor -- start 2024-09-10T09:59:22.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- start start 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee8806f1e0 0x7fee88148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee88155970 0x7fee8815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee8815c560 0x7fee8815f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee880cf170 con 0x7fee8806f1e0 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fee880cce00 con 0x7fee8815c560 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8fc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fee880cf2f0 con 0x7fee88155970 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8d200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee8806f1e0 0x7fee88148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8d200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee8806f1e0 0x7fee88148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54222/0 (socket says 172.21.15.110:54222) 2024-09-10T09:59:22.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8d200640 1 -- 172.21.15.110:0/386431890 learned_addr learned my addr 172.21.15.110:0/386431890 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee87e00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee88155970 0x7fee8815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.179+0000 7fee8dc00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee8815c560 0x7fee8815f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8d200640 1 -- 172.21.15.110:0/386431890 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee88155970 msgr2=0x7fee8815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8d200640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee88155970 0x7fee8815ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8d200640 1 -- 172.21.15.110:0/386431890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee8815c560 msgr2=0x7fee8815f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8d200640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee8815c560 0x7fee8815f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8d200640 1 -- 172.21.15.110:0/386431890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8815d000 con 0x7fee8806f1e0 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8d200640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee8806f1e0 0x7fee88148e80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e960 tx=0x7fee7800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee84c00640 1 -- 172.21.15.110:0/386431890 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78017070 con 0x7fee8806f1e0 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee84c00640 1 -- 172.21.15.110:0/386431890 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7800ca00 con 0x7fee8806f1e0 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8815d2e0 con 0x7fee8806f1e0 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee84c00640 1 -- 172.21.15.110:0/386431890 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7800cdb0 con 0x7fee8806f1e0 2024-09-10T09:59:22.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8815d8a0 con 0x7fee8806f1e0 2024-09-10T09:59:22.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee880ce420 con 0x7fee8806f1e0 2024-09-10T09:59:22.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee84c00640 1 -- 172.21.15.110:0/386431890 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7fee7801a070 con 0x7fee8806f1e0 2024-09-10T09:59:22.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee84c00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fee68067980 0x7fee68069e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee84c00640 1 -- 172.21.15.110:0/386431890 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fee78086040 con 0x7fee8806f1e0 2024-09-10T09:59:22.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.183+0000 7fee87e00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fee68067980 0x7fee68069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.187+0000 7fee87e00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fee68067980 0x7fee68069e40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee88150110 tx=0x7fee74000bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.191+0000 7fee84c00640 1 -- 172.21.15.110:0/386431890 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fee780564c0 con 0x7fee8806f1e0 2024-09-10T09:59:22.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.315+0000 7fee84c00640 1 -- 172.21.15.110:0/386431890 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fee78052b30 con 0x7fee8806f1e0 2024-09-10T09:59:22.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.355+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee8815bc60 con 0x7fee8806f1e0 2024-09-10T09:59:22.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.355+0000 7fee84c00640 1 -- 172.21.15.110:0/386431890 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16765 (secure 0 0 0) 0x7fee7806f1d0 con 0x7fee8806f1e0 2024-09-10T09:59:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:22.360 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":34,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:05.220702+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:01.639334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:02.583881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:22.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fee68067980 msgr2=0x7fee68069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fee68067980 0x7fee68069e40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee88150110 tx=0x7fee74000bd0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee8806f1e0 msgr2=0x7fee88148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee8806f1e0 0x7fee88148e80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e960 tx=0x7fee7800ee20 comp rx=0 tx=0).stop 2024-09-10T09:59:22.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 shutdown_connections 2024-09-10T09:59:22.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fee68067980 0x7fee68069e40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee8815c560 0x7fee8815f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee88155970 0x7fee8815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 --2- 172.21.15.110:0/386431890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee8806f1e0 0x7fee88148e80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 >> 172.21.15.110:0/386431890 conn(0x7fee8806bc50 msgr2=0x7fee8806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 shutdown_connections 2024-09-10T09:59:22.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.359+0000 7fee8fc00640 1 -- 172.21.15.110:0/386431890 wait complete. 2024-09-10T09:59:22.381 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T09:59:22.381 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:22.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.511+0000 7f60d5c00640 1 Processor -- start 2024-09-10T09:59:22.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.511+0000 7f60d5c00640 1 -- start start 2024-09-10T09:59:22.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60d5c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f60d00cfe30 0x7f60d0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60d5c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d006e7a0 0x7f60d00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60d5c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d006d2d0 con 0x7f60d00cd510 2024-09-10T09:59:22.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60d5c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f60d015d9a0 con 0x7f60d00cfe30 2024-09-10T09:59:22.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60d5c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f60d006d5b0 con 0x7f60d006e7a0 2024-09-10T09:59:22.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f60d00cfe30 0x7f60d0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f60d00cfe30 0x7f60d0071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54224/0 (socket says 172.21.15.110:54224) 2024-09-10T09:59:22.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cea00640 1 -- 172.21.15.110:0/814879106 learned_addr learned my addr 172.21.15.110:0/814879106 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:22.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cf400640 1 --2- 172.21.15.110:0/814879106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d006e7a0 0x7f60d00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cea00640 1 -- 172.21.15.110:0/814879106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d006e7a0 msgr2=0x7f60d00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:22.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cea00640 1 --2- 172.21.15.110:0/814879106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d006e7a0 0x7f60d00cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cea00640 1 -- 172.21.15.110:0/814879106 >> v1:172.21.15.145:6789/0 conn(0x7f60d00cd510 legacy=0x7f60d006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:22.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cea00640 1 -- 172.21.15.110:0/814879106 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d006d890 con 0x7f60d00cfe30 2024-09-10T09:59:22.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.515+0000 7f60cea00640 1 --2- 172.21.15.110:0/814879106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d00cfe30 0x7f60d0071a40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009e50 tx=0x7f60c001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3367e25c04dd3a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60cd600640 1 -- 172.21.15.110:0/814879106 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00a56f0 con 0x7f60d00cfe30 2024-09-10T09:59:22.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60cd600640 1 -- 172.21.15.110:0/814879106 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c00a5890 con 0x7f60d00cfe30 2024-09-10T09:59:22.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60cd600640 1 -- 172.21.15.110:0/814879106 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00a5c60 con 0x7f60d00cfe30 2024-09-10T09:59:22.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- 172.21.15.110:0/814879106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d00cfe30 msgr2=0x7f60d0071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 --2- 172.21.15.110:0/814879106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d00cfe30 0x7f60d0071a40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009e50 tx=0x7f60c001c960 comp rx=0 tx=0).stop 2024-09-10T09:59:22.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- 172.21.15.110:0/814879106 shutdown_connections 2024-09-10T09:59:22.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 --2- 172.21.15.110:0/814879106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d006e7a0 0x7f60d00cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 --2- 172.21.15.110:0/814879106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d00cfe30 0x7f60d0071a40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- 172.21.15.110:0/814879106 >> 172.21.15.110:0/814879106 conn(0x7f60d006bc50 msgr2=0x7f60d006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:22.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- 172.21.15.110:0/814879106 shutdown_connections 2024-09-10T09:59:22.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- 172.21.15.110:0/814879106 wait complete. 2024-09-10T09:59:22.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 Processor -- start 2024-09-10T09:59:22.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- start start 2024-09-10T09:59:22.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d006e7a0 0x7f60d013ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d00cd510 0x7f60d00af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60d00cfe30 0x7f60d00ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0072190 con 0x7f60d006e7a0 2024-09-10T09:59:22.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f60d015d6c0 con 0x7f60d00cfe30 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60d5c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f60d0169180 con 0x7f60d00cd510 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60cea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d006e7a0 0x7f60d013ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60cea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d006e7a0 0x7f60d013ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54226/0 (socket says 172.21.15.110:54226) 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60cea00640 1 -- 172.21.15.110:0/3615343088 learned_addr learned my addr 172.21.15.110:0/3615343088 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60ce000640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d00cd510 0x7f60d00af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.519+0000 7f60cf400640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60d00cfe30 0x7f60d00ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60cea00640 1 -- 172.21.15.110:0/3615343088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d00cd510 msgr2=0x7f60d00af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60cea00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d00cd510 0x7f60d00af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60cea00640 1 -- 172.21.15.110:0/3615343088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60d00cfe30 msgr2=0x7f60d00ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60cea00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60d00cfe30 0x7f60d00ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60cea00640 1 -- 172.21.15.110:0/3615343088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d00b1440 con 0x7f60d006e7a0 2024-09-10T09:59:22.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60cea00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d006e7a0 0x7f60d013ccc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f60c001b010 tx=0x7f60c0019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60c6a00640 1 -- 172.21.15.110:0/3615343088 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00bc020 con 0x7f60d006e7a0 2024-09-10T09:59:22.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60c6a00640 1 -- 172.21.15.110:0/3615343088 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c00b8880 con 0x7f60d006e7a0 2024-09-10T09:59:22.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d00b1720 con 0x7f60d006e7a0 2024-09-10T09:59:22.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60c6a00640 1 -- 172.21.15.110:0/3615343088 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0009510 con 0x7f60d006e7a0 2024-09-10T09:59:22.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d00b1ce0 con 0x7f60d006e7a0 2024-09-10T09:59:22.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d006c240 con 0x7f60d006e7a0 2024-09-10T09:59:22.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60c6a00640 1 -- 172.21.15.110:0/3615343088 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f60c00bf020 con 0x7f60d006e7a0 2024-09-10T09:59:22.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60c6a00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f60b0067a70 0x7f60b0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60c6a00640 1 -- 172.21.15.110:0/3615343088 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f60c01210e0 con 0x7f60d006e7a0 2024-09-10T09:59:22.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.523+0000 7f60ce000640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f60b0067a70 0x7f60b0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.527+0000 7f60ce000640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f60b0067a70 0x7f60b0069f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f60d00b46a0 tx=0x7f60bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.531+0000 7f60c6a00640 1 -- 172.21.15.110:0/3615343088 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f60c00c6030 con 0x7f60d006e7a0 2024-09-10T09:59:22.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60d015f670 con 0x7f60d006e7a0 2024-09-10T09:59:22.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60c6a00640 1 -- 172.21.15.110:0/3615343088 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f60c00165e0 con 0x7f60d006e7a0 2024-09-10T09:59:22.722 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:22.723 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":17,"btime":"2024-09-10T09:59:17:286112+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4556,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1141773053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1141773053},{"type":"v1","addr":"172.21.15.145:6834","nonce":1141773053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/656217845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":656217845},{"type":"v1","addr":"172.21.15.145:6835","nonce":656217845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2244539155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2244539155},{"type":"v1","addr":"172.21.15.110:6835","nonce":2244539155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:03.573094+0000","modified":"2024-09-10T09:59:06.627577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1972132715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1972132715},{"type":"v1","addr":"172.21.15.110:6837","nonce":1972132715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-09-10T09:59:22.723 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 17 2024-09-10T09:59:22.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f60b0067a70 msgr2=0x7f60b0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f60b0067a70 0x7f60b0069f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f60d00b46a0 tx=0x7f60bc008040 comp rx=0 tx=0).stop 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d006e7a0 msgr2=0x7f60d013ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d006e7a0 0x7f60d013ccc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f60c001b010 tx=0x7f60c0019700 comp rx=0 tx=0).stop 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 shutdown_connections 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f60b0067a70 0x7f60b0069f30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60d00cfe30 0x7f60d00ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f60d00cd510 0x7f60d00af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 --2- 172.21.15.110:0/3615343088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d006e7a0 0x7f60d013ccc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 >> 172.21.15.110:0/3615343088 conn(0x7f60d006bc50 msgr2=0x7f60d01513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 shutdown_connections 2024-09-10T09:59:22.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.719+0000 7f60d5c00640 1 -- 172.21.15.110:0/3615343088 wait complete. 2024-09-10T09:59:22.743 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53e6400640 1 Processor -- start 2024-09-10T09:59:22.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53e6400640 1 -- start start 2024-09-10T09:59:22.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53e6400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e015c560 0x7f53e006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53e6400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f53e006f010 0x7f53e00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53e6400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e006f430 con 0x7f53e0155970 2024-09-10T09:59:22.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53e6400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f53e00b4570 con 0x7f53e006f010 2024-09-10T09:59:22.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53e6400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f53e0155d50 con 0x7f53e015c560 2024-09-10T09:59:22.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53df400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f53e0155970 0x7f53e00cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44246/0 (socket says 172.21.15.110:44246) 2024-09-10T09:59:22.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53df400640 1 -- 172.21.15.110:0/889444088 learned_addr learned my addr 172.21.15.110:0/889444088 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:22.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53dfe00640 1 --2- 172.21.15.110:0/889444088 >> v2:172.21.15.110:3300/0 conn(0x7f53e006f010 0x7f53e00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53dea00640 1 --2- 172.21.15.110:0/889444088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e015c560 0x7f53e006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53de000640 1 -- 172.21.15.110:0/889444088 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2875924373 0 0) 0x7f53e006f430 con 0x7f53e0155970 2024-09-10T09:59:22.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53de000640 1 -- 172.21.15.110:0/889444088 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53c4003530 con 0x7f53e0155970 2024-09-10T09:59:22.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53dfe00640 1 -- 172.21.15.110:0/889444088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e015c560 msgr2=0x7f53e006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:22.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53dfe00640 1 --2- 172.21.15.110:0/889444088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e015c560 0x7f53e006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53dfe00640 1 -- 172.21.15.110:0/889444088 >> v1:172.21.15.110:6789/0 conn(0x7f53e0155970 legacy=0x7f53e00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.875+0000 7f53dfe00640 1 -- 172.21.15.110:0/889444088 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e00b34d0 con 0x7f53e006f010 2024-09-10T09:59:22.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53dfe00640 1 --2- 172.21.15.110:0/889444088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e006f010 0x7f53e00ce420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009d90 tx=0x7f53d801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4faf9eb9e737d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53de000640 1 -- 172.21.15.110:0/889444088 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80b2070 con 0x7f53e006f010 2024-09-10T09:59:22.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53de000640 1 -- 172.21.15.110:0/889444088 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d80a5970 con 0x7f53e006f010 2024-09-10T09:59:22.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53de000640 1 -- 172.21.15.110:0/889444088 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80a5d40 con 0x7f53e006f010 2024-09-10T09:59:22.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53e6400640 1 -- 172.21.15.110:0/889444088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e006f010 msgr2=0x7f53e00ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53e6400640 1 --2- 172.21.15.110:0/889444088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e006f010 0x7f53e00ce420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009d90 tx=0x7f53d801c8b0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53e6400640 1 -- 172.21.15.110:0/889444088 shutdown_connections 2024-09-10T09:59:22.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53e6400640 1 --2- 172.21.15.110:0/889444088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e006f010 0x7f53e00ce420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53e6400640 1 --2- 172.21.15.110:0/889444088 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e015c560 0x7f53e006ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53e6400640 1 -- 172.21.15.110:0/889444088 >> 172.21.15.110:0/889444088 conn(0x7f53e006bc50 msgr2=0x7f53e006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:22.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53e6400640 1 -- 172.21.15.110:0/889444088 shutdown_connections 2024-09-10T09:59:22.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.879+0000 7f53e6400640 1 -- 172.21.15.110:0/889444088 wait complete. 2024-09-10T09:59:22.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 Processor -- start 2024-09-10T09:59:22.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 -- start start 2024-09-10T09:59:22.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e006f010 0x7f53e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e0155970 0x7f53e006df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e015c560 0x7f53e015e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53e00cf170 con 0x7f53e015c560 2024-09-10T09:59:22.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f53e00b4050 con 0x7f53e0155970 2024-09-10T09:59:22.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f53e00cf2f0 con 0x7f53e006f010 2024-09-10T09:59:22.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e015c560 0x7f53e015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e015c560 0x7f53e015e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54252/0 (socket says 172.21.15.110:54252) 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 -- 172.21.15.110:0/2430209042 learned_addr learned my addr 172.21.15.110:0/2430209042 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53df400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e006f010 0x7f53e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dea00640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e0155970 0x7f53e006df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 -- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e006f010 msgr2=0x7f53e006d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e006f010 0x7f53e006d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 -- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e0155970 msgr2=0x7f53e006df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e0155970 0x7f53e006df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 -- 172.21.15.110:0/2430209042 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e015cbf0 con 0x7f53e015c560 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53df400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e006f010 0x7f53e006d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:22.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53dfe00640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e015c560 0x7f53e015e990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f53d801af60 tx=0x7f53d8019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53d7400640 1 -- 172.21.15.110:0/2430209042 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80b2050 con 0x7f53e015c560 2024-09-10T09:59:22.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53d7400640 1 -- 172.21.15.110:0/2430209042 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d80be070 con 0x7f53e015c560 2024-09-10T09:59:22.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53d7400640 1 -- 172.21.15.110:0/2430209042 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80ada50 con 0x7f53e015c560 2024-09-10T09:59:22.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e015ced0 con 0x7f53e015c560 2024-09-10T09:59:22.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e015d490 con 0x7f53e015c560 2024-09-10T09:59:22.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.883+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e00ce420 con 0x7f53e015c560 2024-09-10T09:59:22.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.887+0000 7f53d7400640 1 -- 172.21.15.110:0/2430209042 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f53d80a57f0 con 0x7f53e015c560 2024-09-10T09:59:22.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.887+0000 7f53d7400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53c0067a70 0x7f53c0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:22.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.887+0000 7f53d7400640 1 -- 172.21.15.110:0/2430209042 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f53d81211a0 con 0x7f53e015c560 2024-09-10T09:59:22.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.887+0000 7f53df400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53c0067a70 0x7f53c0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:22.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.887+0000 7f53df400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53c0067a70 0x7f53c0069f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f53d0005890 tx=0x7f53d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:22.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:22.891+0000 7f53d7400640 1 -- 172.21.15.110:0/2430209042 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f53d8016560 con 0x7f53e015c560 2024-09-10T09:59:23.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.079+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53e0071290 con 0x7f53e015c560 2024-09-10T09:59:23.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.079+0000 7f53d7400640 1 -- 172.21.15.110:0/2430209042 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f53d80f19c0 con 0x7f53e015c560 2024-09-10T09:59:23.085 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:23.085 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":17,"btime":"2024-09-10T09:59:17:286112+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4556,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1141773053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1141773053},{"type":"v1","addr":"172.21.15.145:6834","nonce":1141773053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/656217845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":656217845},{"type":"v1","addr":"172.21.15.145:6835","nonce":656217845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2244539155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2244539155},{"type":"v1","addr":"172.21.15.110:6835","nonce":2244539155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:03.573094+0000","modified":"2024-09-10T09:59:06.627577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1972132715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1972132715},{"type":"v1","addr":"172.21.15.110:6837","nonce":1972132715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-09-10T09:59:23.085 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 17 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53c0067a70 msgr2=0x7f53c0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53c0067a70 0x7f53c0069f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f53d0005890 tx=0x7f53d0009040 comp rx=0 tx=0).stop 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e015c560 msgr2=0x7f53e015e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e015c560 0x7f53e015e990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f53d801af60 tx=0x7f53d8019650 comp rx=0 tx=0).stop 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 shutdown_connections 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53c0067a70 0x7f53c0069f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e015c560 0x7f53e015e990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e0155970 0x7f53e006df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 --2- 172.21.15.110:0/2430209042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e006f010 0x7f53e006d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 >> 172.21.15.110:0/2430209042 conn(0x7f53e006bc50 msgr2=0x7f53e00b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 shutdown_connections 2024-09-10T09:59:23.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.083+0000 7f53e6400640 1 -- 172.21.15.110:0/2430209042 wait complete. 2024-09-10T09:59:23.106 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:23.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61cde00640 1 Processor -- start 2024-09-10T09:59:23.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61cde00640 1 -- start start 2024-09-10T09:59:23.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61cde00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 0x7f61c80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61cde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f61c815c560 0x7f61c806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61cde00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c8155d50 con 0x7f61c806f010 2024-09-10T09:59:23.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61cde00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f61c80b4570 con 0x7f61c815c560 2024-09-10T09:59:23.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61cde00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f61c806f450 con 0x7f61c8155970 2024-09-10T09:59:23.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61be000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f61c815c560 0x7f61c806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61be000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f61c815c560 0x7f61c806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54264/0 (socket says 172.21.15.110:54264) 2024-09-10T09:59:23.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.239+0000 7f61be000640 1 -- 172.21.15.110:0/868410242 learned_addr learned my addr 172.21.15.110:0/868410242 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:23.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61be000640 1 -- 172.21.15.110:0/868410242 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 msgr2=0x7f61c80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61be000640 1 --2- 172.21.15.110:0/868410242 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 0x7f61c80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61be000640 1 -- 172.21.15.110:0/868410242 >> v1:172.21.15.145:6789/0 conn(0x7f61c806f010 legacy=0x7f61c815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61be000640 1 -- 172.21.15.110:0/868410242 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c80b34d0 con 0x7f61c815c560 2024-09-10T09:59:23.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61be000640 1 --2- 172.21.15.110:0/868410242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c815c560 0x7f61c806ead0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f61c806f5d0 tx=0x7f61b001e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82359e8b13a34649 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61c6000640 1 -- 172.21.15.110:0/868410242 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00aa640 con 0x7f61c815c560 2024-09-10T09:59:23.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61c6000640 1 -- 172.21.15.110:0/868410242 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b00aa7e0 con 0x7f61c815c560 2024-09-10T09:59:23.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61c6000640 1 -- 172.21.15.110:0/868410242 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00aabb0 con 0x7f61c815c560 2024-09-10T09:59:23.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61cde00640 1 -- 172.21.15.110:0/868410242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c815c560 msgr2=0x7f61c806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61cde00640 1 --2- 172.21.15.110:0/868410242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c815c560 0x7f61c806ead0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f61c806f5d0 tx=0x7f61b001e650 comp rx=0 tx=0).stop 2024-09-10T09:59:23.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61cde00640 1 -- 172.21.15.110:0/868410242 shutdown_connections 2024-09-10T09:59:23.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61cde00640 1 --2- 172.21.15.110:0/868410242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c815c560 0x7f61c806ead0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61cde00640 1 --2- 172.21.15.110:0/868410242 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 0x7f61c80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.243+0000 7f61cde00640 1 -- 172.21.15.110:0/868410242 >> 172.21.15.110:0/868410242 conn(0x7f61c806bc50 msgr2=0x7f61c806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 -- 172.21.15.110:0/868410242 shutdown_connections 2024-09-10T09:59:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 -- 172.21.15.110:0/868410242 wait complete. 2024-09-10T09:59:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 Processor -- start 2024-09-10T09:59:23.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 -- start start 2024-09-10T09:59:23.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c806f010 0x7f61c8164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 0x7f61c814e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c815c560 0x7f61c814ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80cce00 con 0x7f61c806f010 2024-09-10T09:59:23.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80b4050 con 0x7f61c815c560 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f61c806d9c0 con 0x7f61c8155970 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c806f010 0x7f61c8164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c806f010 0x7f61c8164640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54270/0 (socket says 172.21.15.110:54270) 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 -- 172.21.15.110:0/317448679 learned_addr learned my addr 172.21.15.110:0/317448679 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c7400640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c815c560 0x7f61c814ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61be000640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 0x7f61c814e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 -- 172.21.15.110:0/317448679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 msgr2=0x7f61c814e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 0x7f61c814e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 -- 172.21.15.110:0/317448679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c815c560 msgr2=0x7f61c814ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c815c560 0x7f61c814ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 -- 172.21.15.110:0/317448679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c8150b90 con 0x7f61c806f010 2024-09-10T09:59:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61c6a00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c806f010 0x7f61c8164640 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b580 tx=0x7f61b800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61bfe00640 1 -- 172.21.15.110:0/317448679 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8014070 con 0x7f61c806f010 2024-09-10T09:59:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61bfe00640 1 -- 172.21.15.110:0/317448679 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8010890 con 0x7f61c806f010 2024-09-10T09:59:23.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.247+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c8150e70 con 0x7f61c806f010 2024-09-10T09:59:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.251+0000 7f61bfe00640 1 -- 172.21.15.110:0/317448679 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8010c80 con 0x7f61c806f010 2024-09-10T09:59:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.251+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c8151430 con 0x7f61c806f010 2024-09-10T09:59:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.251+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c80cc8c0 con 0x7f61c806f010 2024-09-10T09:59:23.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.251+0000 7f61bfe00640 1 -- 172.21.15.110:0/317448679 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f61b8017070 con 0x7f61c806f010 2024-09-10T09:59:23.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.251+0000 7f61bfe00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f61a4067a70 0x7f61a4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.251+0000 7f61bfe00640 1 -- 172.21.15.110:0/317448679 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f61b8085cf0 con 0x7f61c806f010 2024-09-10T09:59:23.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.251+0000 7f61be000640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f61a4067a70 0x7f61a4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.251+0000 7f61be000640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f61a4067a70 0x7f61a4069f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61b00b1000 tx=0x7f61b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.255+0000 7f61bfe00640 1 -- 172.21.15.110:0/317448679 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f61b8055f80 con 0x7f61c806f010 2024-09-10T09:59:23.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.415+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61c80710f0 con 0x7f61c806f010 2024-09-10T09:59:23.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61bfe00640 1 -- 172.21.15.110:0/317448679 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16765 (secure 0 0 0) 0x7f61b8010e20 con 0x7f61c806f010 2024-09-10T09:59:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:23.422 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":34,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:05.220702+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:01.639334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:02.583881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f61a4067a70 msgr2=0x7f61a4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f61a4067a70 0x7f61a4069f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61b00b1000 tx=0x7f61b0009040 comp rx=0 tx=0).stop 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c806f010 msgr2=0x7f61c8164640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c806f010 0x7f61c8164640 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b580 tx=0x7f61b800ba40 comp rx=0 tx=0).stop 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 shutdown_connections 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f61a4067a70 0x7f61a4069f30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61c815c560 0x7f61c814ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61c8155970 0x7f61c814e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 --2- 172.21.15.110:0/317448679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61c806f010 0x7f61c8164640 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 >> 172.21.15.110:0/317448679 conn(0x7f61c806bc50 msgr2=0x7f61c8165720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 shutdown_connections 2024-09-10T09:59:23.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.419+0000 7f61cde00640 1 -- 172.21.15.110:0/317448679 wait complete. 2024-09-10T09:59:23.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T09:59:23.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f9086c00640 1 Processor -- start 2024-09-10T09:59:23.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f9086c00640 1 -- start start 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f9086c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f908015e540 0x7f90800b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f9086c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f90800626e0 0x7f90800af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f9086c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800b5880 0x7f90800afce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f9086c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f90800ae9e0 con 0x7f908015e540 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f9086c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f90800aecc0 con 0x7f90800626e0 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f9086c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f90800aefa0 con 0x7f90800b5880 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f907fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f908015e540 0x7f90800b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f907fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f908015e540 0x7f90800b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54282/0 (socket says 172.21.15.110:54282) 2024-09-10T09:59:23.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.575+0000 7f907fe00640 1 -- 172.21.15.110:0/1073849326 learned_addr learned my addr 172.21.15.110:0/1073849326 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:23.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9084c00640 1 --2- 172.21.15.110:0/1073849326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800b5880 0x7f90800afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907f400640 1 --2- 172.21.15.110:0/1073849326 >> v2:172.21.15.145:3300/0 conn(0x7f90800626e0 0x7f90800af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907fe00640 1 -- 172.21.15.110:0/1073849326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800b5880 msgr2=0x7f90800afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907fe00640 1 --2- 172.21.15.110:0/1073849326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800b5880 0x7f90800afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907fe00640 1 -- 172.21.15.110:0/1073849326 >> v2:172.21.15.145:3300/0 conn(0x7f90800626e0 msgr2=0x7f90800af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907fe00640 1 --2- 172.21.15.110:0/1073849326 >> v2:172.21.15.145:3300/0 conn(0x7f90800626e0 0x7f90800af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907fe00640 1 -- 172.21.15.110:0/1073849326 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90801621f0 con 0x7f908015e540 2024-09-10T09:59:23.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9084c00640 1 --2- 172.21.15.110:0/1073849326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800b5880 0x7f90800afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:23.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907f400640 1 --2- 172.21.15.110:0/1073849326 >> v2:172.21.15.145:3300/0 conn(0x7f90800626e0 0x7f90800af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:23.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907fe00640 1 --2- 172.21.15.110:0/1073849326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f908015e540 0x7f90800b5340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9070009870 tx=0x7f907001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68ebc9b4fc9e67e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907ea00640 1 -- 172.21.15.110:0/1073849326 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90700b0070 con 0x7f908015e540 2024-09-10T09:59:23.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907ea00640 1 -- 172.21.15.110:0/1073849326 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90700ab600 con 0x7f908015e540 2024-09-10T09:59:23.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f907ea00640 1 -- 172.21.15.110:0/1073849326 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90700ab9d0 con 0x7f908015e540 2024-09-10T09:59:23.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9086c00640 1 -- 172.21.15.110:0/1073849326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f908015e540 msgr2=0x7f90800b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9086c00640 1 --2- 172.21.15.110:0/1073849326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f908015e540 0x7f90800b5340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9070009870 tx=0x7f907001c490 comp rx=0 tx=0).stop 2024-09-10T09:59:23.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9086c00640 1 -- 172.21.15.110:0/1073849326 shutdown_connections 2024-09-10T09:59:23.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9086c00640 1 --2- 172.21.15.110:0/1073849326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800b5880 0x7f90800afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9086c00640 1 --2- 172.21.15.110:0/1073849326 >> v2:172.21.15.145:3300/0 conn(0x7f90800626e0 0x7f90800af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9086c00640 1 --2- 172.21.15.110:0/1073849326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f908015e540 0x7f90800b5340 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.579+0000 7f9086c00640 1 -- 172.21.15.110:0/1073849326 >> 172.21.15.110:0/1073849326 conn(0x7f90800b0970 msgr2=0x7f90800b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 -- 172.21.15.110:0/1073849326 shutdown_connections 2024-09-10T09:59:23.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 -- 172.21.15.110:0/1073849326 wait complete. 2024-09-10T09:59:23.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 Processor -- start 2024-09-10T09:59:23.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 -- start start 2024-09-10T09:59:23.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800626e0 0x7f908014b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90800b5880 0x7f908014a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f908015e540 0x7f908015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f908006e6f0 con 0x7f90800b5880 2024-09-10T09:59:23.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f908006e570 con 0x7f908015e540 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f908006e870 con 0x7f90800626e0 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90800b5880 0x7f908014a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90800b5880 0x7f908014a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54290/0 (socket says 172.21.15.110:54290) 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 -- 172.21.15.110:0/869937145 learned_addr learned my addr 172.21.15.110:0/869937145 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907fe00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800626e0 0x7f908014b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9084c00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f908015e540 0x7f908015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 -- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800626e0 msgr2=0x7f908014b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800626e0 0x7f908014b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 -- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f908015e540 msgr2=0x7f908015fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f908015e540 0x7f908015fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 -- 172.21.15.110:0/869937145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908014b9b0 con 0x7f90800b5880 2024-09-10T09:59:23.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9084c00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f908015e540 0x7f908015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:23.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f907f400640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90800b5880 0x7f908014a850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c7c0 tx=0x7f906c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9077e00640 1 -- 172.21.15.110:0/869937145 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c015070 con 0x7f90800b5880 2024-09-10T09:59:23.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9077e00640 1 -- 172.21.15.110:0/869937145 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906c007a30 con 0x7f90800b5880 2024-09-10T09:59:23.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9077e00640 1 -- 172.21.15.110:0/869937145 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c007e00 con 0x7f90800b5880 2024-09-10T09:59:23.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908014bc90 con 0x7f90800b5880 2024-09-10T09:59:23.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.583+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908014c250 con 0x7f90800b5880 2024-09-10T09:59:23.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.587+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90800b5340 con 0x7f90800b5880 2024-09-10T09:59:23.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.587+0000 7f9077e00640 1 -- 172.21.15.110:0/869937145 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f906c018070 con 0x7f90800b5880 2024-09-10T09:59:23.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.587+0000 7f9077e00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9060067a70 0x7f9060069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.587+0000 7f9077e00640 1 -- 172.21.15.110:0/869937145 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f906c085f60 con 0x7f90800b5880 2024-09-10T09:59:23.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.587+0000 7f907fe00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9060067a70 0x7f9060069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.587+0000 7f907fe00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9060067a70 0x7f9060069f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90700ad000 tx=0x7f9070008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.591+0000 7f9077e00640 1 -- 172.21.15.110:0/869937145 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f906c0560c0 con 0x7f90800b5880 2024-09-10T09:59:23.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.779+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f908015d890 con 0x7f90800b5880 2024-09-10T09:59:23.784 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T09:59:23.779+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T09:59:23.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.783+0000 7f9077e00640 1 -- 172.21.15.110:0/869937145 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f906c05a3c0 con 0x7f90800b5880 2024-09-10T09:59:23.786 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T09:59:23.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.783+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9060067a70 msgr2=0x7f9060069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.783+0000 7f9086c00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9060067a70 0x7f9060069f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90700ad000 tx=0x7f9070008040 comp rx=0 tx=0).stop 2024-09-10T09:59:23.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.783+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90800b5880 msgr2=0x7f908014a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.783+0000 7f9086c00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90800b5880 0x7f908014a850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c7c0 tx=0x7f906c00cc80 comp rx=0 tx=0).stop 2024-09-10T09:59:23.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.787+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 shutdown_connections 2024-09-10T09:59:23.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.787+0000 7f9086c00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9060067a70 0x7f9060069f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.787+0000 7f9086c00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f908015e540 0x7f908015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.787+0000 7f9086c00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90800b5880 0x7f908014a850 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.787+0000 7f9086c00640 1 --2- 172.21.15.110:0/869937145 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90800626e0 0x7f908014b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.787+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 >> 172.21.15.110:0/869937145 conn(0x7f90800b0970 msgr2=0x7f908006f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.787+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 shutdown_connections 2024-09-10T09:59:23.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.787+0000 7f9086c00640 1 -- 172.21.15.110:0/869937145 wait complete. 2024-09-10T09:59:23.810 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T09:59:23.857 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T09:59:23.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41b6000640 1 Processor -- start 2024-09-10T09:59:23.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41b6000640 1 -- start start 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41b6000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f41b0145f30 0x7f41b00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41b6000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f41b00b29e0 0x7f41b00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41b6000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b00b37c0 con 0x7f41b00cfbc0 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41b6000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f41b00b2e50 con 0x7f41b00b29e0 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41b6000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f41b00b3aa0 con 0x7f41b0145f30 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41aea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f41b00b29e0 0x7f41b00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41aea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f41b00b29e0 0x7f41b00cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54336/0 (socket says 172.21.15.110:54336) 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41aea00640 1 -- 172.21.15.110:0/3834549143 learned_addr learned my addr 172.21.15.110:0/3834549143 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:23.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41af400640 1 --2- 172.21.15.110:0/3834549143 >> v2:172.21.15.145:3300/0 conn(0x7f41b0145f30 0x7f41b00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41aea00640 1 -- 172.21.15.110:0/3834549143 >> v2:172.21.15.145:3300/0 conn(0x7f41b0145f30 msgr2=0x7f41b00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41aea00640 1 --2- 172.21.15.110:0/3834549143 >> v2:172.21.15.145:3300/0 conn(0x7f41b0145f30 0x7f41b00b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41aea00640 1 -- 172.21.15.110:0/3834549143 >> v1:172.21.15.145:6789/0 conn(0x7f41b00cfbc0 legacy=0x7f41b0160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.951+0000 7f41aea00640 1 -- 172.21.15.110:0/3834549143 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b00b3d80 con 0x7f41b00b29e0 2024-09-10T09:59:23.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41aea00640 1 --2- 172.21.15.110:0/3834549143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00b29e0 0x7f41b00cf680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f41b00b3000 tx=0x7f419c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b60b79007e46f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41ae000640 1 -- 172.21.15.110:0/3834549143 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0b0070 con 0x7f41b00b29e0 2024-09-10T09:59:23.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41ae000640 1 -- 172.21.15.110:0/3834549143 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419c0ab600 con 0x7f41b00b29e0 2024-09-10T09:59:23.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41ae000640 1 -- 172.21.15.110:0/3834549143 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0ab9d0 con 0x7f41b00b29e0 2024-09-10T09:59:23.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41b6000640 1 -- 172.21.15.110:0/3834549143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00b29e0 msgr2=0x7f41b00cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41b6000640 1 --2- 172.21.15.110:0/3834549143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00b29e0 0x7f41b00cf680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f41b00b3000 tx=0x7f419c01c520 comp rx=0 tx=0).stop 2024-09-10T09:59:23.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41b6000640 1 -- 172.21.15.110:0/3834549143 shutdown_connections 2024-09-10T09:59:23.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41b6000640 1 --2- 172.21.15.110:0/3834549143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00b29e0 0x7f41b00cf680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41b6000640 1 --2- 172.21.15.110:0/3834549143 >> v2:172.21.15.145:3300/0 conn(0x7f41b0145f30 0x7f41b00b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41b6000640 1 -- 172.21.15.110:0/3834549143 >> 172.21.15.110:0/3834549143 conn(0x7f41b0062e40 msgr2=0x7f41b006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41b6000640 1 -- 172.21.15.110:0/3834549143 shutdown_connections 2024-09-10T09:59:23.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.955+0000 7f41b6000640 1 -- 172.21.15.110:0/3834549143 wait complete. 2024-09-10T09:59:23.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 Processor -- start 2024-09-10T09:59:23.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 -- start start 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41b00b29e0 0x7f41b013f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00cfbc0 0x7f41b015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41b0145f30 0x7f41b01447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f41b0169530 con 0x7f41b00cfbc0 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f41b01693b0 con 0x7f41b0145f30 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f41b01696b0 con 0x7f41b00b29e0 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00cfbc0 0x7f41b015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00cfbc0 0x7f41b015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54340/0 (socket says 172.21.15.110:54340) 2024-09-10T09:59:23.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 -- 172.21.15.110:0/3536437971 learned_addr learned my addr 172.21.15.110:0/3536437971 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:23.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41afe00640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41b0145f30 0x7f41b01447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41af400640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41b00b29e0 0x7f41b013f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 -- 172.21.15.110:0/3536437971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41b00b29e0 msgr2=0x7f41b013f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41b00b29e0 0x7f41b013f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 -- 172.21.15.110:0/3536437971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41b0145f30 msgr2=0x7f41b01447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41b0145f30 0x7f41b01447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 -- 172.21.15.110:0/3536437971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b013e850 con 0x7f41b00cfbc0 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41aea00640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00cfbc0 0x7f41b015c370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f419c0ae040 tx=0x7f419c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41a7400640 1 -- 172.21.15.110:0/3536437971 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0b0040 con 0x7f41b00cfbc0 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41a7400640 1 -- 172.21.15.110:0/3536437971 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419c0ba070 con 0x7f41b00cfbc0 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b013eb30 con 0x7f41b00cfbc0 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41a7400640 1 -- 172.21.15.110:0/3536437971 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0b4670 con 0x7f41b00cfbc0 2024-09-10T09:59:23.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b013f0f0 con 0x7f41b00cfbc0 2024-09-10T09:59:23.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.959+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b00cf680 con 0x7f41b00cfbc0 2024-09-10T09:59:23.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.963+0000 7f41a7400640 1 -- 172.21.15.110:0/3536437971 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f419c0b4810 con 0x7f41b00cfbc0 2024-09-10T09:59:23.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.963+0000 7f41a7400640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4190067aa0 0x7f4190069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.963+0000 7f41a7400640 1 -- 172.21.15.110:0/3536437971 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f419c11fb70 con 0x7f41b00cfbc0 2024-09-10T09:59:23.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.963+0000 7f41af400640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4190067aa0 0x7f4190069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.963+0000 7f41af400640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4190067aa0 0x7f4190069f60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41a0011340 tx=0x7f41a000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:23.967+0000 7f41a7400640 1 -- 172.21.15.110:0/3536437971 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f419c016110 con 0x7f41b00cfbc0 2024-09-10T09:59:24.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.159+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f41b0071c00 con 0x7f41b00cfbc0 2024-09-10T09:59:24.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.783+0000 7f41a7400640 1 -- 172.21.15.110:0/3536437971 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f419c0efb40 con 0x7f41b00cfbc0 2024-09-10T09:59:24.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4190067aa0 msgr2=0x7f4190069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:24.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4190067aa0 0x7f4190069f60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41a0011340 tx=0x7f41a000b040 comp rx=0 tx=0).stop 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00cfbc0 msgr2=0x7f41b015c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00cfbc0 0x7f41b015c370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f419c0ae040 tx=0x7f419c0192c0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 shutdown_connections 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4190067aa0 0x7f4190069f60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41b0145f30 0x7f41b01447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41b00cfbc0 0x7f41b015c370 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 --2- 172.21.15.110:0/3536437971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41b00b29e0 0x7f41b013f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 >> 172.21.15.110:0/3536437971 conn(0x7f41b0062e40 msgr2=0x7f41b01429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 shutdown_connections 2024-09-10T09:59:24.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.787+0000 7f41b6000640 1 -- 172.21.15.110:0/3536437971 wait complete. 2024-09-10T09:59:24.814 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T09:59:24.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e39400640 1 Processor -- start 2024-09-10T09:59:24.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e39400640 1 -- start start 2024-09-10T09:59:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e39400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2e3406eb30 0x7f2e340d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e39400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 0x7f2e34071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e39400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340706a0 con 0x7f2e34073290 2024-09-10T09:59:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e39400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3415ee90 con 0x7f2e3406eb30 2024-09-10T09:59:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e39400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2e340710d0 con 0x7f2e34163000 2024-09-10T09:59:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e32800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2e3406eb30 0x7f2e340d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e33200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2e34073290 0x7f2e3406d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44264/0 (socket says 172.21.15.110:44264) 2024-09-10T09:59:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.951+0000 7f2e33200640 1 -- 172.21.15.110:0/2167541826 learned_addr learned my addr 172.21.15.110:0/2167541826 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e31e00640 1 --2- 172.21.15.110:0/2167541826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 0x7f2e34071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e32800640 1 -- 172.21.15.110:0/2167541826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 msgr2=0x7f2e34071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e32800640 1 --2- 172.21.15.110:0/2167541826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 0x7f2e34071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e32800640 1 -- 172.21.15.110:0/2167541826 >> v1:172.21.15.110:6789/0 conn(0x7f2e34073290 legacy=0x7f2e3406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e32800640 1 -- 172.21.15.110:0/2167541826 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e340d1370 con 0x7f2e3406eb30 2024-09-10T09:59:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e31400640 1 -- 172.21.15.110:0/2167541826 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 707916313 0 0) 0x7f2e340706a0 con 0x7f2e34073290 2024-09-10T09:59:24.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e32800640 1 --2- 172.21.15.110:0/2167541826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3406eb30 0x7f2e340d0cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009d90 tx=0x7f2e2401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c167ef36179146a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:24.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e31400640 1 -- 172.21.15.110:0/2167541826 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e240b2070 con 0x7f2e3406eb30 2024-09-10T09:59:24.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e31400640 1 -- 172.21.15.110:0/2167541826 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e240a5880 con 0x7f2e3406eb30 2024-09-10T09:59:24.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e31400640 1 -- 172.21.15.110:0/2167541826 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e240a5c50 con 0x7f2e3406eb30 2024-09-10T09:59:24.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e39400640 1 -- 172.21.15.110:0/2167541826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3406eb30 msgr2=0x7f2e340d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:24.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e39400640 1 --2- 172.21.15.110:0/2167541826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3406eb30 0x7f2e340d0cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009d90 tx=0x7f2e2401c8b0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.955+0000 7f2e39400640 1 -- 172.21.15.110:0/2167541826 shutdown_connections 2024-09-10T09:59:24.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 --2- 172.21.15.110:0/2167541826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 0x7f2e34071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 --2- 172.21.15.110:0/2167541826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3406eb30 0x7f2e340d0cb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 -- 172.21.15.110:0/2167541826 >> 172.21.15.110:0/2167541826 conn(0x7f2e340d25c0 msgr2=0x7f2e340d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 -- 172.21.15.110:0/2167541826 shutdown_connections 2024-09-10T09:59:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 -- 172.21.15.110:0/2167541826 wait complete. 2024-09-10T09:59:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 Processor -- start 2024-09-10T09:59:24.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 -- start start 2024-09-10T09:59:24.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e3406eb30 0x7f2e34157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e34073290 0x7f2e340ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 0x7f2e340ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341633e0 con 0x7f2e34073290 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34073b10 con 0x7f2e3406eb30 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e39400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2e34071aa0 con 0x7f2e34163000 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e34073290 0x7f2e340ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e34073290 0x7f2e340ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54346/0 (socket says 172.21.15.110:54346) 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 -- 172.21.15.110:0/488479826 learned_addr learned my addr 172.21.15.110:0/488479826 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e32800640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e3406eb30 0x7f2e34157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e33200640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 0x7f2e340ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 -- 172.21.15.110:0/488479826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 msgr2=0x7f2e340ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 0x7f2e340ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 -- 172.21.15.110:0/488479826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e3406eb30 msgr2=0x7f2e34157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e3406eb30 0x7f2e34157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 -- 172.21.15.110:0/488479826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e34160210 con 0x7f2e34073290 2024-09-10T09:59:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e31e00640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e34073290 0x7f2e340ce310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000d600 tx=0x7f2e2000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.959+0000 7f2e2aa00640 1 -- 172.21.15.110:0/488479826 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20004240 con 0x7f2e34073290 2024-09-10T09:59:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e2aa00640 1 -- 172.21.15.110:0/488479826 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e200043e0 con 0x7f2e34073290 2024-09-10T09:59:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e2aa00640 1 -- 172.21.15.110:0/488479826 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20003840 con 0x7f2e34073290 2024-09-10T09:59:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e341604f0 con 0x7f2e34073290 2024-09-10T09:59:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e341581e0 con 0x7f2e34073290 2024-09-10T09:59:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3406fc10 con 0x7f2e34073290 2024-09-10T09:59:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e2aa00640 1 -- 172.21.15.110:0/488479826 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f2e200039e0 con 0x7f2e34073290 2024-09-10T09:59:24.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e2aa00640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e14067a70 0x7f2e14069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:24.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e2aa00640 1 -- 172.21.15.110:0/488479826 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f2e200861a0 con 0x7f2e34073290 2024-09-10T09:59:24.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e32800640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e14067a70 0x7f2e14069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:24.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.963+0000 7f2e32800640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e14067a70 0x7f2e14069f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e240af000 tx=0x7f2e240a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:24.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:24.967+0000 7f2e2aa00640 1 -- 172.21.15.110:0/488479826 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2e200563a0 con 0x7f2e34073290 2024-09-10T09:59:25.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.127+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e3406f930 con 0x7f2e34073290 2024-09-10T09:59:25.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.787+0000 7f2e2aa00640 1 -- 172.21.15.110:0/488479826 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f2e2005a6a0 con 0x7f2e34073290 2024-09-10T09:59:25.791 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T09:59:25.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e14067a70 msgr2=0x7f2e14069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e14067a70 0x7f2e14069f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e240af000 tx=0x7f2e240a7070 comp rx=0 tx=0).stop 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e34073290 msgr2=0x7f2e340ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e34073290 0x7f2e340ce310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000d600 tx=0x7f2e2000dac0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 shutdown_connections 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2e14067a70 0x7f2e14069f30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e34163000 0x7f2e340ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e34073290 0x7f2e340ce310 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 --2- 172.21.15.110:0/488479826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e3406eb30 0x7f2e34157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 >> 172.21.15.110:0/488479826 conn(0x7f2e340d25c0 msgr2=0x7f2e340b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 shutdown_connections 2024-09-10T09:59:25.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.791+0000 7f2e39400640 1 -- 172.21.15.110:0/488479826 wait complete. 2024-09-10T09:59:25.815 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T09:59:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93eb000640 1 Processor -- start 2024-09-10T09:59:25.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93eb000640 1 -- start start 2024-09-10T09:59:25.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93eb000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e415e690 0x7f93e406e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93eb000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f93e40aef50 0x7f93e406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93eb000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e406c710 con 0x7f93e406bc50 2024-09-10T09:59:25.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93eb000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f93e40b4680 con 0x7f93e40aef50 2024-09-10T09:59:25.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93eb000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f93e40b4960 con 0x7f93e415e690 2024-09-10T09:59:25.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e415e690 0x7f93e406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93e9000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f93e40aef50 0x7f93e406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93e9000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f93e40aef50 0x7f93e406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48886/0 (socket says 172.21.15.110:48886) 2024-09-10T09:59:25.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93e9000640 1 -- 172.21.15.110:0/68436121 learned_addr learned my addr 172.21.15.110:0/68436121 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:25.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93e9000640 1 -- 172.21.15.110:0/68436121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e415e690 msgr2=0x7f93e406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:25.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93e9000640 1 --2- 172.21.15.110:0/68436121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e415e690 0x7f93e406e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93e9000640 1 -- 172.21.15.110:0/68436121 >> v1:172.21.15.145:6789/0 conn(0x7f93e406bc50 legacy=0x7f93e40ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.951+0000 7f93e9000640 1 -- 172.21.15.110:0/68436121 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e40b4c40 con 0x7f93e40aef50 2024-09-10T09:59:25.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93e9000640 1 --2- 172.21.15.110:0/68436121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e40aef50 0x7f93e406c090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009870 tx=0x7f93dc01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee0a8b5a349be30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93e2a00640 1 -- 172.21.15.110:0/68436121 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0a95f0 con 0x7f93e40aef50 2024-09-10T09:59:25.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93e2a00640 1 -- 172.21.15.110:0/68436121 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc0a9790 con 0x7f93e40aef50 2024-09-10T09:59:25.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93e2a00640 1 -- 172.21.15.110:0/68436121 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0a9b60 con 0x7f93e40aef50 2024-09-10T09:59:25.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93eb000640 1 -- 172.21.15.110:0/68436121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e40aef50 msgr2=0x7f93e406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93eb000640 1 --2- 172.21.15.110:0/68436121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e40aef50 0x7f93e406c090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009870 tx=0x7f93dc01c530 comp rx=0 tx=0).stop 2024-09-10T09:59:25.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93eb000640 1 -- 172.21.15.110:0/68436121 shutdown_connections 2024-09-10T09:59:25.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93eb000640 1 --2- 172.21.15.110:0/68436121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e40aef50 0x7f93e406c090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93eb000640 1 --2- 172.21.15.110:0/68436121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e415e690 0x7f93e406e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93eb000640 1 -- 172.21.15.110:0/68436121 >> 172.21.15.110:0/68436121 conn(0x7f93e40d1530 msgr2=0x7f93e40d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:25.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93eb000640 1 -- 172.21.15.110:0/68436121 shutdown_connections 2024-09-10T09:59:25.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.955+0000 7f93eb000640 1 -- 172.21.15.110:0/68436121 wait complete. 2024-09-10T09:59:25.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 Processor -- start 2024-09-10T09:59:25.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 -- start start 2024-09-10T09:59:25.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e406bc50 0x7f93e4155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e40aef50 0x7f93e41519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e415e690 0x7f93e415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93e415ebf0 con 0x7f93e415e690 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f93e415ea70 con 0x7f93e406bc50 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f93e40b5c50 con 0x7f93e40aef50 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e406bc50 0x7f93e4155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e415e690 0x7f93e415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e415e690 0x7f93e415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54348/0 (socket says 172.21.15.110:54348) 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e406bc50 0x7f93e4155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48894/0 (socket says 172.21.15.110:48894) 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 -- 172.21.15.110:0/2855014673 learned_addr learned my addr 172.21.15.110:0/2855014673 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:25.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e3400640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e40aef50 0x7f93e41519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 -- 172.21.15.110:0/2855014673 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e40aef50 msgr2=0x7f93e41519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e40aef50 0x7f93e41519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 -- 172.21.15.110:0/2855014673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e406bc50 msgr2=0x7f93e4155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e406bc50 0x7f93e4155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 -- 172.21.15.110:0/2855014673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e415b5f0 con 0x7f93e415e690 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93e9000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e415e690 0x7f93e415ad10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f93dc01a160 tx=0x7f93dc01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93dbe00640 1 -- 172.21.15.110:0/2855014673 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0b1600 con 0x7f93e415e690 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93dbe00640 1 -- 172.21.15.110:0/2855014673 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc0b17a0 con 0x7f93e415e690 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e415b8d0 con 0x7f93e415e690 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93dbe00640 1 -- 172.21.15.110:0/2855014673 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0b1b50 con 0x7f93e415e690 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.959+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e415be90 con 0x7f93e415e690 2024-09-10T09:59:25.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.963+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e406c090 con 0x7f93e415e690 2024-09-10T09:59:25.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.963+0000 7f93dbe00640 1 -- 172.21.15.110:0/2855014673 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f93dc0b1cf0 con 0x7f93e415e690 2024-09-10T09:59:25.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.963+0000 7f93dbe00640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93c4067a70 0x7f93c4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:25.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.963+0000 7f93dbe00640 1 -- 172.21.15.110:0/2855014673 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7f93dc120870 con 0x7f93e415e690 2024-09-10T09:59:25.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.963+0000 7f93e3e00640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93c4067a70 0x7f93c4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:25.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.963+0000 7f93e3e00640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93c4067a70 0x7f93c4069f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93d4011340 tx=0x7f93d400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:25.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:25.967+0000 7f93dbe00640 1 -- 172.21.15.110:0/2855014673 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f93dc0ad020 con 0x7f93e415e690 2024-09-10T09:59:26.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.123+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f93e40b3fe0 con 0x7f93e415e690 2024-09-10T09:59:26.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.791+0000 7f93dbe00640 1 -- 172.21.15.110:0/2855014673 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7f93dc016150 con 0x7f93e415e690 2024-09-10T09:59:26.797 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T09:59:26.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93c4067a70 msgr2=0x7f93c4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93c4067a70 0x7f93c4069f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93d4011340 tx=0x7f93d400b040 comp rx=0 tx=0).stop 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e415e690 msgr2=0x7f93e415ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e415e690 0x7f93e415ad10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f93dc01a160 tx=0x7f93dc01ad70 comp rx=0 tx=0).stop 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 shutdown_connections 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f93c4067a70 0x7f93c4069f30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e415e690 0x7f93e415ad10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93e40aef50 0x7f93e41519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 --2- 172.21.15.110:0/2855014673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93e406bc50 0x7f93e4155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 >> 172.21.15.110:0/2855014673 conn(0x7f93e40d1530 msgr2=0x7f93e40ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:26.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 shutdown_connections 2024-09-10T09:59:26.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.795+0000 7f93eb000640 1 -- 172.21.15.110:0/2855014673 wait complete. 2024-09-10T09:59:26.824 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T09:59:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.955+0000 7f9971200640 1 Processor -- start 2024-09-10T09:59:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.955+0000 7f9971200640 1 -- start start 2024-09-10T09:59:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9971200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:26.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9971200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996c0b08c0 con 0x7f996c06c7a0 2024-09-10T09:59:26.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9971200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996c0b35b0 con 0x7f996c0b5000 2024-09-10T09:59:26.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9971200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f996c0b41c0 con 0x7f996c06c360 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f996a800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f996c06c7a0 0x7f996c06e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44268/0 (socket says 172.21.15.110:44268) 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f996a800640 1 -- 172.21.15.110:0/3764343063 learned_addr learned my addr 172.21.15.110:0/3764343063 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9963e00640 1 --2- 172.21.15.110:0/3764343063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9969400640 1 -- 172.21.15.110:0/3764343063 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935907573 0 0) 0x7f996c0b08c0 con 0x7f996c06c7a0 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9969400640 1 -- 172.21.15.110:0/3764343063 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9950003530 con 0x7f996c06c7a0 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9969400640 1 -- 172.21.15.110:0/3764343063 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2023691926 0 0) 0x7f996c0b35b0 con 0x7f996c0b5000 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9969400640 1 -- 172.21.15.110:0/3764343063 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f996c0b08c0 con 0x7f996c0b5000 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9963e00640 1 -- 172.21.15.110:0/3764343063 >> v1:172.21.15.145:6789/0 conn(0x7f996c0b5000 legacy=0x7f996c158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9963e00640 1 -- 172.21.15.110:0/3764343063 >> v1:172.21.15.110:6789/0 conn(0x7f996c06c7a0 legacy=0x7f996c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.959+0000 7f9963e00640 1 -- 172.21.15.110:0/3764343063 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c0b3890 con 0x7f996c06c360 2024-09-10T09:59:26.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9963e00640 1 --2- 172.21.15.110:0/3764343063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c0b22d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f996c0b4340 tx=0x7f995401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e8f3e5e2254d101 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:26.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9969400640 1 -- 172.21.15.110:0/3764343063 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540b0070 con 0x7f996c06c360 2024-09-10T09:59:26.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9969400640 1 -- 172.21.15.110:0/3764343063 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99540ab600 con 0x7f996c06c360 2024-09-10T09:59:26.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9969400640 1 -- 172.21.15.110:0/3764343063 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540ab9d0 con 0x7f996c06c360 2024-09-10T09:59:26.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 -- 172.21.15.110:0/3764343063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 msgr2=0x7f996c0b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 --2- 172.21.15.110:0/3764343063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c0b22d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f996c0b4340 tx=0x7f995401c4c0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 -- 172.21.15.110:0/3764343063 shutdown_connections 2024-09-10T09:59:26.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 --2- 172.21.15.110:0/3764343063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c0b22d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 -- 172.21.15.110:0/3764343063 >> 172.21.15.110:0/3764343063 conn(0x7f996c062e40 msgr2=0x7f996c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:26.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 -- 172.21.15.110:0/3764343063 shutdown_connections 2024-09-10T09:59:26.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 -- 172.21.15.110:0/3764343063 wait complete. 2024-09-10T09:59:26.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 Processor -- start 2024-09-10T09:59:26.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.963+0000 7f9971200640 1 -- start start 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c1f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f996c06c7a0 0x7f996c1f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f996c14e8b0 0x7f996c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f996c06e850 con 0x7f996c14e8b0 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f996c0b2a20 con 0x7f996c06c7a0 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f996c06e9d0 con 0x7f996c06c360 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f996c14e8b0 0x7f996c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9969e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c1f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f996c14e8b0 0x7f996c0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54364/0 (socket says 172.21.15.110:54364) 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9963e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f996c06c7a0 0x7f996c1f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:26.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 -- 172.21.15.110:0/3562878779 learned_addr learned my addr 172.21.15.110:0/3562878779 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:26.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 -- 172.21.15.110:0/3562878779 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 msgr2=0x7f996c1f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c1f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 -- 172.21.15.110:0/3562878779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f996c06c7a0 msgr2=0x7f996c1f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f996c06c7a0 0x7f996c1f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 -- 172.21.15.110:0/3562878779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1f4430 con 0x7f996c14e8b0 2024-09-10T09:59:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f996a800640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f996c14e8b0 0x7f996c0ced10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f996400cb40 tx=0x7f996400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9962000640 1 -- 172.21.15.110:0/3562878779 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964017070 con 0x7f996c14e8b0 2024-09-10T09:59:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9962000640 1 -- 172.21.15.110:0/3562878779 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996400acc0 con 0x7f996c14e8b0 2024-09-10T09:59:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9962000640 1 -- 172.21.15.110:0/3562878779 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964012640 con 0x7f996c14e8b0 2024-09-10T09:59:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1f4710 con 0x7f996c14e8b0 2024-09-10T09:59:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1f4cd0 con 0x7f996c14e8b0 2024-09-10T09:59:26.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996c0b22d0 con 0x7f996c14e8b0 2024-09-10T09:59:26.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.967+0000 7f9962000640 1 -- 172.21.15.110:0/3562878779 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f996401a070 con 0x7f996c14e8b0 2024-09-10T09:59:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.971+0000 7f9962000640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9948067aa0 0x7f9948069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.971+0000 7f9962000640 1 -- 172.21.15.110:0/3562878779 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f99640859b0 con 0x7f996c14e8b0 2024-09-10T09:59:26.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.971+0000 7f9969e00640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9948067aa0 0x7f9948069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:26.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.971+0000 7f9969e00640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9948067aa0 0x7f9948069f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f995c001720 tx=0x7f995c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:26.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:26.975+0000 7f9962000640 1 -- 172.21.15.110:0/3562878779 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9964055fb0 con 0x7f996c14e8b0 2024-09-10T09:59:27.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.131+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f996c06df50 con 0x7f996c14e8b0 2024-09-10T09:59:27.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.135+0000 7f9962000640 1 -- 172.21.15.110:0/3562878779 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f996405a2b0 con 0x7f996c14e8b0 2024-09-10T09:59:27.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.135+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9948067aa0 msgr2=0x7f9948069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.135+0000 7f9971200640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9948067aa0 0x7f9948069f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f995c001720 tx=0x7f995c009040 comp rx=0 tx=0).stop 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.135+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f996c14e8b0 msgr2=0x7f996c0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.135+0000 7f9971200640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f996c14e8b0 0x7f996c0ced10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f996400cb40 tx=0x7f996400a060 comp rx=0 tx=0).stop 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.139+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 shutdown_connections 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.139+0000 7f9971200640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9948067aa0 0x7f9948069f60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.139+0000 7f9971200640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f996c14e8b0 0x7f996c0ced10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.139+0000 7f9971200640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f996c06c7a0 0x7f996c1f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.139+0000 7f9971200640 1 --2- 172.21.15.110:0/3562878779 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f996c06c360 0x7f996c1f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.139+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 >> 172.21.15.110:0/3562878779 conn(0x7f996c062e40 msgr2=0x7f996c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.139+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 shutdown_connections 2024-09-10T09:59:27.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.139+0000 7f9971200640 1 -- 172.21.15.110:0/3562878779 wait complete. 2024-09-10T09:59:27.160 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-10T09:59:27.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.295+0000 7fc651400640 1 Processor -- start 2024-09-10T09:59:27.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.295+0000 7fc651400640 1 -- start start 2024-09-10T09:59:27.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc651400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc64c0b1c60 0x7fc64c142320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc651400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c15fd80 0x7fc64c13f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc651400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc64c15c790 con 0x7fc64c0731e0 2024-09-10T09:59:27.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc651400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc64c0b2040 con 0x7fc64c0b1c60 2024-09-10T09:59:27.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc651400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc64c142860 con 0x7fc64c15fd80 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc64c0b1c60 0x7fc64c142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c15fd80 0x7fc64c13f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc64c0b1c60 0x7fc64c142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48904/0 (socket says 172.21.15.110:48904) 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c15fd80 0x7fc64c13f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:40952/0 (socket says 172.21.15.110:40952) 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64be00640 1 -- 172.21.15.110:0/1871975489 learned_addr learned my addr 172.21.15.110:0/1871975489 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64be00640 1 -- 172.21.15.110:0/1871975489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c15fd80 msgr2=0x7fc64c13f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64be00640 1 --2- 172.21.15.110:0/1871975489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c15fd80 0x7fc64c13f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64be00640 1 -- 172.21.15.110:0/1871975489 >> v1:172.21.15.145:6789/0 conn(0x7fc64c0731e0 legacy=0x7fc64c073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64be00640 1 -- 172.21.15.110:0/1871975489 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64c1602b0 con 0x7fc64c0b1c60 2024-09-10T09:59:27.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.299+0000 7fc64b400640 1 --2- 172.21.15.110:0/1871975489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c15fd80 0x7fc64c13f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:27.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc64be00640 1 --2- 172.21.15.110:0/1871975489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0b1c60 0x7fc64c142320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009e50 tx=0x7fc63c01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8df001fed8a2a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:27.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc64aa00640 1 -- 172.21.15.110:0/1871975489 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0a5700 con 0x7fc64c0b1c60 2024-09-10T09:59:27.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc64aa00640 1 -- 172.21.15.110:0/1871975489 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c0a58a0 con 0x7fc64c0b1c60 2024-09-10T09:59:27.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc64aa00640 1 -- 172.21.15.110:0/1871975489 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0a5c70 con 0x7fc64c0b1c60 2024-09-10T09:59:27.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- 172.21.15.110:0/1871975489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0b1c60 msgr2=0x7fc64c142320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 --2- 172.21.15.110:0/1871975489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0b1c60 0x7fc64c142320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009e50 tx=0x7fc63c01ca30 comp rx=0 tx=0).stop 2024-09-10T09:59:27.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- 172.21.15.110:0/1871975489 shutdown_connections 2024-09-10T09:59:27.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 --2- 172.21.15.110:0/1871975489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c15fd80 0x7fc64c13f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 --2- 172.21.15.110:0/1871975489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0b1c60 0x7fc64c142320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- 172.21.15.110:0/1871975489 >> 172.21.15.110:0/1871975489 conn(0x7fc64c0b42d0 msgr2=0x7fc64c0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:27.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- 172.21.15.110:0/1871975489 shutdown_connections 2024-09-10T09:59:27.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- 172.21.15.110:0/1871975489 wait complete. 2024-09-10T09:59:27.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 Processor -- start 2024-09-10T09:59:27.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- start start 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0731e0 0x7fc64c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c0b1c60 0x7fc64c155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc64c15fd80 0x7fc64c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c15d460 con 0x7fc64c15fd80 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c13fe20 con 0x7fc64c0731e0 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.303+0000 7fc651400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc64c15d5e0 con 0x7fc64c0b1c60 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0731e0 0x7fc64c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c0b1c60 0x7fc64c155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc650a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc64c15fd80 0x7fc64c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0731e0 0x7fc64c0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48912/0 (socket says 172.21.15.110:48912) 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c0b1c60 0x7fc64c155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:40954/0 (socket says 172.21.15.110:40954) 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 -- 172.21.15.110:0/419599709 learned_addr learned my addr 172.21.15.110:0/419599709 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 -- 172.21.15.110:0/419599709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c0b1c60 msgr2=0x7fc64c155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c0b1c60 0x7fc64c155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 -- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc64c15fd80 msgr2=0x7fc64c15d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc64c15fd80 0x7fc64c15d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 -- 172.21.15.110:0/419599709 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64c0cfd10 con 0x7fc64c0731e0 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64b400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c0b1c60 0x7fc64c155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:27.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc64be00640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0731e0 0x7fc64c0cf7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc63c01a660 tx=0x7fc63c0197d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:27.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc643e00640 1 -- 172.21.15.110:0/419599709 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c009510 con 0x7fc64c0731e0 2024-09-10T09:59:27.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc643e00640 1 -- 172.21.15.110:0/419599709 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c0b8470 con 0x7fc64c0731e0 2024-09-10T09:59:27.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc643e00640 1 -- 172.21.15.110:0/419599709 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0b8860 con 0x7fc64c0731e0 2024-09-10T09:59:27.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64c0b6ac0 con 0x7fc64c0731e0 2024-09-10T09:59:27.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc64c134b70 con 0x7fc64c0731e0 2024-09-10T09:59:27.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc64c142320 con 0x7fc64c0731e0 2024-09-10T09:59:27.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.307+0000 7fc643e00640 1 -- 172.21.15.110:0/419599709 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fc63c0c0030 con 0x7fc64c0731e0 2024-09-10T09:59:27.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.311+0000 7fc643e00640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc62c067aa0 0x7fc62c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.311+0000 7fc643e00640 1 -- 172.21.15.110:0/419599709 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fc63c11fff0 con 0x7fc64c0731e0 2024-09-10T09:59:27.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.311+0000 7fc64b400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc62c067aa0 0x7fc62c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.311+0000 7fc64b400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc62c067aa0 0x7fc62c069f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc64c166ed0 tx=0x7fc638009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:27.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.315+0000 7fc643e00640 1 -- 172.21.15.110:0/419599709 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc63c016620 con 0x7fc64c0731e0 2024-09-10T09:59:27.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.527+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fc64c137950 con 0x7fc64c0731e0 2024-09-10T09:59:27.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc643e00640 1 -- 172.21.15.110:0/419599709 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v80) ==== 158+0+0 (secure 0 0 0) 0x7fc63c0f05f0 con 0x7fc64c0731e0 2024-09-10T09:59:27.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc62c067aa0 msgr2=0x7fc62c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc651400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc62c067aa0 0x7fc62c069f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc64c166ed0 tx=0x7fc638009190 comp rx=0 tx=0).stop 2024-09-10T09:59:27.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0731e0 msgr2=0x7fc64c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc651400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0731e0 0x7fc64c0cf7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc63c01a660 tx=0x7fc63c0197d0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 shutdown_connections 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc651400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc62c067aa0 0x7fc62c069f60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc651400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc64c15fd80 0x7fc64c15d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.799+0000 7fc651400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc64c0b1c60 0x7fc64c155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.803+0000 7fc651400640 1 --2- 172.21.15.110:0/419599709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc64c0731e0 0x7fc64c0cf7d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.803+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 >> 172.21.15.110:0/419599709 conn(0x7fc64c0b42d0 msgr2=0x7fc64c0b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.803+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 shutdown_connections 2024-09-10T09:59:27.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.803+0000 7fc651400640 1 -- 172.21.15.110:0/419599709 wait complete. 2024-09-10T09:59:27.824 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-10T09:59:27.824 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T09:59:27.825 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:27.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.955+0000 7f64dc400640 1 Processor -- start 2024-09-10T09:59:27.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.955+0000 7f64dc400640 1 -- start start 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64dc400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f64d40cfe30 0x7f64d4071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64dc400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f64d406e7a0 0x7f64d40cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64dc400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64d406d2d0 con 0x7f64d40cd510 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64dc400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f64d415d9a0 con 0x7f64d406e7a0 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64dc400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f64d406d5b0 con 0x7f64d40cfe30 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64da400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f64d406e7a0 0x7f64d40cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64d9000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f64d40cd510 0x7f64d406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:44272/0 (socket says 172.21.15.110:44272) 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64d9000640 1 -- 172.21.15.110:0/2989710868 learned_addr learned my addr 172.21.15.110:0/2989710868 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:27.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64d9a00640 1 --2- 172.21.15.110:0/2989710868 >> v2:172.21.15.145:3300/0 conn(0x7f64d40cfe30 0x7f64d4071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64da400640 1 -- 172.21.15.110:0/2989710868 >> v2:172.21.15.145:3300/0 conn(0x7f64d40cfe30 msgr2=0x7f64d4071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:27.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64da400640 1 --2- 172.21.15.110:0/2989710868 >> v2:172.21.15.145:3300/0 conn(0x7f64d40cfe30 0x7f64d4071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64da400640 1 -- 172.21.15.110:0/2989710868 >> v1:172.21.15.110:6789/0 conn(0x7f64d40cd510 legacy=0x7f64d406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64da400640 1 -- 172.21.15.110:0/2989710868 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d406d890 con 0x7f64d406e7a0 2024-09-10T09:59:27.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.959+0000 7f64d3e00640 1 -- 172.21.15.110:0/2989710868 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 487125187 0 0) 0x7f64d406d2d0 con 0x7f64d40cd510 2024-09-10T09:59:27.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64da400640 1 --2- 172.21.15.110:0/2989710868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40cd9a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f64d40ce120 tx=0x7f64cc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dced99c6f262bba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:27.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64d3e00640 1 -- 172.21.15.110:0/2989710868 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0b2070 con 0x7f64d406e7a0 2024-09-10T09:59:27.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64d3e00640 1 -- 172.21.15.110:0/2989710868 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64cc0a5870 con 0x7f64d406e7a0 2024-09-10T09:59:27.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64d3e00640 1 -- 172.21.15.110:0/2989710868 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0a5c40 con 0x7f64d406e7a0 2024-09-10T09:59:27.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- 172.21.15.110:0/2989710868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 msgr2=0x7f64d40cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 --2- 172.21.15.110:0/2989710868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40cd9a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f64d40ce120 tx=0x7f64cc01c8a0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- 172.21.15.110:0/2989710868 shutdown_connections 2024-09-10T09:59:27.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 --2- 172.21.15.110:0/2989710868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40cd9a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 --2- 172.21.15.110:0/2989710868 >> v2:172.21.15.145:3300/0 conn(0x7f64d40cfe30 0x7f64d4071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- 172.21.15.110:0/2989710868 >> 172.21.15.110:0/2989710868 conn(0x7f64d406bc50 msgr2=0x7f64d406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:27.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- 172.21.15.110:0/2989710868 shutdown_connections 2024-09-10T09:59:27.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- 172.21.15.110:0/2989710868 wait complete. 2024-09-10T09:59:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 Processor -- start 2024-09-10T09:59:27.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- start start 2024-09-10T09:59:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64d40cd510 0x7f64d414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64d40cfe30 0x7f64d414be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4072190 con 0x7f64d406e7a0 2024-09-10T09:59:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f64d415d6c0 con 0x7f64d40cd510 2024-09-10T09:59:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.963+0000 7f64dc400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f64d4169180 con 0x7f64d40cfe30 2024-09-10T09:59:27.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54402/0 (socket says 172.21.15.110:54402) 2024-09-10T09:59:27.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 -- 172.21.15.110:0/1748030123 learned_addr learned my addr 172.21.15.110:0/1748030123 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 -- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64d40cfe30 msgr2=0x7f64d414be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64da400640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64d40cfe30 0x7f64d414be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9000640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64d40cd510 0x7f64d414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64d40cfe30 0x7f64d414be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 -- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64d40cd510 msgr2=0x7f64d414b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64d40cd510 0x7f64d414b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 -- 172.21.15.110:0/1748030123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d40b2f40 con 0x7f64d406e7a0 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64da400640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64d40cfe30 0x7f64d414be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9000640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64d40cd510 0x7f64d414b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9a00640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40b0cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f64c800e870 tx=0x7f64c800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:27.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d1600640 1 -- 172.21.15.110:0/1748030123 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8017070 con 0x7f64d406e7a0 2024-09-10T09:59:27.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d1600640 1 -- 172.21.15.110:0/1748030123 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c800c940 con 0x7f64d406e7a0 2024-09-10T09:59:27.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d40b0730 con 0x7f64d406e7a0 2024-09-10T09:59:27.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d1600640 1 -- 172.21.15.110:0/1748030123 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c800ccf0 con 0x7f64d406e7a0 2024-09-10T09:59:27.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d40b28c0 con 0x7f64d406e7a0 2024-09-10T09:59:27.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d415f780 con 0x7f64d406e7a0 2024-09-10T09:59:27.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d1600640 1 -- 172.21.15.110:0/1748030123 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f64c801a070 con 0x7f64d406e7a0 2024-09-10T09:59:27.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d1600640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64b4067a70 0x7f64b4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d1600640 1 -- 172.21.15.110:0/1748030123 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f64c8085fe0 con 0x7f64d406e7a0 2024-09-10T09:59:27.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.967+0000 7f64d9000640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64b4067a70 0x7f64b4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.971+0000 7f64d9000640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64b4067a70 0x7f64b4069f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f64c0007fd0 tx=0x7f64c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:27.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:27.971+0000 7f64d1600640 1 -- 172.21.15.110:0/1748030123 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f64c8056b20 con 0x7f64d406e7a0 2024-09-10T09:59:28.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.135+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64d40719c0 con 0x7f64d406e7a0 2024-09-10T09:59:28.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64d1600640 1 -- 172.21.15.110:0/1748030123 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13070 (secure 0 0 0) 0x7f64c8056420 con 0x7f64d406e7a0 2024-09-10T09:59:28.142 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:28.142 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":37,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:26.791179+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6837/1972132715":"2024-09-11T09:59:23.784840+0000","172.21.15.110:6836/1972132715":"2024-09-11T09:59:23.784840+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:28.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64b4067a70 msgr2=0x7f64b4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64b4067a70 0x7f64b4069f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f64c0007fd0 tx=0x7f64c0009040 comp rx=0 tx=0).stop 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 msgr2=0x7f64d40b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40b0cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f64c800e870 tx=0x7f64c800ec70 comp rx=0 tx=0).stop 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 shutdown_connections 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64b4067a70 0x7f64b4069f30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64d40cfe30 0x7f64d414be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64d40cd510 0x7f64d414b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 --2- 172.21.15.110:0/1748030123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64d406e7a0 0x7f64d40b0cb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 >> 172.21.15.110:0/1748030123 conn(0x7f64d406bc50 msgr2=0x7f64d4150a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 shutdown_connections 2024-09-10T09:59:28.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.139+0000 7f64dc400640 1 -- 172.21.15.110:0/1748030123 wait complete. 2024-09-10T09:59:28.162 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T09:59:28.163 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T09:59:28.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe4a200640 1 Processor -- start 2024-09-10T09:59:28.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe4a200640 1 -- start start 2024-09-10T09:59:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe4a200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbe44145f30 0x7fbe440b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe4a200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe440b37c0 con 0x7fbe440b29e0 2024-09-10T09:59:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe4a200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe440b3aa0 con 0x7fbe440cfd90 2024-09-10T09:59:28.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe4a200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbe440b2e50 con 0x7fbe44145f30 2024-09-10T09:59:28.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe43400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbe44145f30 0x7fbe440b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe42a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fbe440b29e0 0x7fbe440cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35332/0 (socket says 172.21.15.110:35332) 2024-09-10T09:59:28.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe42a00640 1 -- 172.21.15.110:0/3726758757 learned_addr learned my addr 172.21.15.110:0/3726758757 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:28.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe43400640 1 -- 172.21.15.110:0/3726758757 >> v1:172.21.15.145:6789/0 conn(0x7fbe440cfd90 legacy=0x7fbe440b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe43400640 1 -- 172.21.15.110:0/3726758757 >> v1:172.21.15.110:6789/0 conn(0x7fbe440b29e0 legacy=0x7fbe440cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.299+0000 7fbe43400640 1 -- 172.21.15.110:0/3726758757 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe440b3d80 con 0x7fbe44145f30 2024-09-10T09:59:28.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe43400640 1 --2- 172.21.15.110:0/3726758757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe44145f30 0x7fbe440b24a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe340097c0 tx=0x7fbe3401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820363b2d1017a00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe42000640 1 -- 172.21.15.110:0/3726758757 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe340b0070 con 0x7fbe44145f30 2024-09-10T09:59:28.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe42000640 1 -- 172.21.15.110:0/3726758757 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe340ab600 con 0x7fbe44145f30 2024-09-10T09:59:28.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe42000640 1 -- 172.21.15.110:0/3726758757 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe340ab9d0 con 0x7fbe44145f30 2024-09-10T09:59:28.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe4a200640 1 -- 172.21.15.110:0/3726758757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe44145f30 msgr2=0x7fbe440b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe4a200640 1 --2- 172.21.15.110:0/3726758757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe44145f30 0x7fbe440b24a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe340097c0 tx=0x7fbe3401c460 comp rx=0 tx=0).stop 2024-09-10T09:59:28.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe4a200640 1 -- 172.21.15.110:0/3726758757 shutdown_connections 2024-09-10T09:59:28.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe4a200640 1 --2- 172.21.15.110:0/3726758757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe44145f30 0x7fbe440b24a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe4a200640 1 -- 172.21.15.110:0/3726758757 >> 172.21.15.110:0/3726758757 conn(0x7fbe44062e40 msgr2=0x7fbe4406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe4a200640 1 -- 172.21.15.110:0/3726758757 shutdown_connections 2024-09-10T09:59:28.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe4a200640 1 -- 172.21.15.110:0/3726758757 wait complete. 2024-09-10T09:59:28.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.303+0000 7fbe4a200640 1 Processor -- start 2024-09-10T09:59:28.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 -- start start 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe440b29e0 0x7fbe4415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe440cfd90 0x7fbe4415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe44145f30 0x7fbe44141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe440b56f0 con 0x7fbe440b29e0 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe440d0170 con 0x7fbe440cfd90 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbe440b5870 con 0x7fbe44145f30 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe440b29e0 0x7fbe4415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe440b29e0 0x7fbe4415bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56782/0 (socket says 172.21.15.110:56782) 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 -- 172.21.15.110:0/2274895096 learned_addr learned my addr 172.21.15.110:0/2274895096 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe42a00640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe440cfd90 0x7fbe4415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43e00640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe44145f30 0x7fbe44141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 -- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe44145f30 msgr2=0x7fbe44141540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe44145f30 0x7fbe44141540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 -- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe440cfd90 msgr2=0x7fbe4415fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe440cfd90 0x7fbe4415fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 -- 172.21.15.110:0/2274895096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4415c3e0 con 0x7fbe440b29e0 2024-09-10T09:59:28.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43e00640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe44145f30 0x7fbe44141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:28.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe43400640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe440b29e0 0x7fbe4415bea0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbe34019070 tx=0x7fbe3401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe3b400640 1 -- 172.21.15.110:0/2274895096 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe340b0050 con 0x7fbe440b29e0 2024-09-10T09:59:28.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe3b400640 1 -- 172.21.15.110:0/2274895096 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe340bc070 con 0x7fbe440b29e0 2024-09-10T09:59:28.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe3b400640 1 -- 172.21.15.110:0/2274895096 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe340aac00 con 0x7fbe440b29e0 2024-09-10T09:59:28.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe441f75f0 con 0x7fbe440b29e0 2024-09-10T09:59:28.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe441f7bb0 con 0x7fbe440b29e0 2024-09-10T09:59:28.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.307+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe440b24a0 con 0x7fbe440b29e0 2024-09-10T09:59:28.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.311+0000 7fbe3b400640 1 -- 172.21.15.110:0/2274895096 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fbe340c1460 con 0x7fbe440b29e0 2024-09-10T09:59:28.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.311+0000 7fbe3b400640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe24067aa0 0x7fbe24069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.311+0000 7fbe3b400640 1 -- 172.21.15.110:0/2274895096 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fbe341206d0 con 0x7fbe440b29e0 2024-09-10T09:59:28.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.311+0000 7fbe42a00640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe24067aa0 0x7fbe24069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.311+0000 7fbe42a00640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe24067aa0 0x7fbe24069f60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe4415fa80 tx=0x7fbe30007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.315+0000 7fbe3b400640 1 -- 172.21.15.110:0/2274895096 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbe34016050 con 0x7fbe440b29e0 2024-09-10T09:59:28.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.471+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe44071e80 con 0x7fbe440b29e0 2024-09-10T09:59:28.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.471+0000 7fbe3b400640 1 -- 172.21.15.110:0/2274895096 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7fbe340f1320 con 0x7fbe440b29e0 2024-09-10T09:59:28.476 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T09:59:28.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe24067aa0 msgr2=0x7fbe24069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe24067aa0 0x7fbe24069f60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe4415fa80 tx=0x7fbe30007e90 comp rx=0 tx=0).stop 2024-09-10T09:59:28.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe440b29e0 msgr2=0x7fbe4415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe440b29e0 0x7fbe4415bea0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbe34019070 tx=0x7fbe3401aca0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 shutdown_connections 2024-09-10T09:59:28.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbe24067aa0 0x7fbe24069f60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe44145f30 0x7fbe44141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe440cfd90 0x7fbe4415fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 --2- 172.21.15.110:0/2274895096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe440b29e0 0x7fbe4415bea0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 >> 172.21.15.110:0/2274895096 conn(0x7fbe44062e40 msgr2=0x7fbe440cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 shutdown_connections 2024-09-10T09:59:28.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.475+0000 7fbe4a200640 1 -- 172.21.15.110:0/2274895096 wait complete. 2024-09-10T09:59:28.498 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-10T09:59:28.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.627+0000 7f9e0cc00640 1 Processor -- start 2024-09-10T09:59:28.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e0cc00640 1 -- start start 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e0cc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e08155970 0x7f9e080cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e0cc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e0806f1e0 0x7f9e080ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e0cc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e08155d50 con 0x7f9e0815c560 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e0cc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9e080b4570 con 0x7f9e0806f1e0 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e0cc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e080b34d0 con 0x7f9e08155970 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e06800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e0806f1e0 0x7f9e080ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e05400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9e0815c560 0x7f9e0806ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35338/0 (socket says 172.21.15.110:35338) 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e05400640 1 -- 172.21.15.110:0/2578131981 learned_addr learned my addr 172.21.15.110:0/2578131981 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:28.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e05e00640 1 --2- 172.21.15.110:0/2578131981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e08155970 0x7f9e080cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e06800640 1 -- 172.21.15.110:0/2578131981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e08155970 msgr2=0x7f9e080cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:28.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e06800640 1 --2- 172.21.15.110:0/2578131981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e08155970 0x7f9e080cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e06800640 1 -- 172.21.15.110:0/2578131981 >> v1:172.21.15.110:6789/0 conn(0x7f9e0815c560 legacy=0x7f9e0806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e06800640 1 -- 172.21.15.110:0/2578131981 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e080b37b0 con 0x7f9e0806f1e0 2024-09-10T09:59:28.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.631+0000 7f9e04a00640 1 -- 172.21.15.110:0/2578131981 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 503753073 0 0) 0x7f9e08155d50 con 0x7f9e0815c560 2024-09-10T09:59:28.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e06800640 1 --2- 172.21.15.110:0/2578131981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0806f1e0 0x7f9e080ce420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097c0 tx=0x7f9e0001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582eb2d86305dd22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e04a00640 1 -- 172.21.15.110:0/2578131981 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000b1070 con 0x7f9e0806f1e0 2024-09-10T09:59:28.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e04a00640 1 -- 172.21.15.110:0/2578131981 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e000a8050 con 0x7f9e0806f1e0 2024-09-10T09:59:28.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e04a00640 1 -- 172.21.15.110:0/2578131981 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000ac860 con 0x7f9e0806f1e0 2024-09-10T09:59:28.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 -- 172.21.15.110:0/2578131981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0806f1e0 msgr2=0x7f9e080ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/2578131981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0806f1e0 0x7f9e080ce420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097c0 tx=0x7f9e0001c3e0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 -- 172.21.15.110:0/2578131981 shutdown_connections 2024-09-10T09:59:28.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/2578131981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0806f1e0 0x7f9e080ce420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/2578131981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e08155970 0x7f9e080cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 -- 172.21.15.110:0/2578131981 >> 172.21.15.110:0/2578131981 conn(0x7f9e0806bc50 msgr2=0x7f9e0806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:28.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 -- 172.21.15.110:0/2578131981 shutdown_connections 2024-09-10T09:59:28.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 -- 172.21.15.110:0/2578131981 wait complete. 2024-09-10T09:59:28.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 Processor -- start 2024-09-10T09:59:28.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.635+0000 7f9e0cc00640 1 -- start start 2024-09-10T09:59:28.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e0806f1e0 0x7f9e0806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e08155970 0x7f9e0815f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0815c560 0x7f9e0815f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e080cf170 con 0x7f9e0815c560 2024-09-10T09:59:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e080cce00 con 0x7f9e08155970 2024-09-10T09:59:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e080cf2f0 con 0x7f9e0806f1e0 2024-09-10T09:59:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0815c560 0x7f9e0815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0815c560 0x7f9e0815f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56806/0 (socket says 172.21.15.110:56806) 2024-09-10T09:59:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 -- 172.21.15.110:0/1880308825 learned_addr learned my addr 172.21.15.110:0/1880308825 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e05e00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e0806f1e0 0x7f9e0806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e05400640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e08155970 0x7f9e0815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 -- 172.21.15.110:0/1880308825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e0806f1e0 msgr2=0x7f9e0806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e0806f1e0 0x7f9e0806dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 -- 172.21.15.110:0/1880308825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e08155970 msgr2=0x7f9e0815f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e08155970 0x7f9e0815f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:28.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 -- 172.21.15.110:0/1880308825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0815fab0 con 0x7f9e0815c560 2024-09-10T09:59:28.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e06800640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0815c560 0x7f9e0815f570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097c0 tx=0x7f9e00019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9dfe000640 1 -- 172.21.15.110:0/1880308825 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000bb070 con 0x7f9e0815c560 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9dfe000640 1 -- 172.21.15.110:0/1880308825 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e000accf0 con 0x7f9e0815c560 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9dfe000640 1 -- 172.21.15.110:0/1880308825 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000b1040 con 0x7f9e0815c560 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0815ca50 con 0x7f9e0815c560 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0815d2d0 con 0x7f9e0815c560 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e080ce420 con 0x7f9e0815c560 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.639+0000 7f9dfe000640 1 -- 172.21.15.110:0/1880308825 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f9e000be070 con 0x7f9e0815c560 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.643+0000 7f9dfe000640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9de8067a70 0x7f9de8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.643+0000 7f9dfe000640 1 -- 172.21.15.110:0/1880308825 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f9e0011ee30 con 0x7f9e0815c560 2024-09-10T09:59:28.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.643+0000 7f9e05e00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9de8067a70 0x7f9de8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:28.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.643+0000 7f9e05e00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9de8067a70 0x7f9de8069f30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9df80074a0 tx=0x7f9df800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:28.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.647+0000 7f9dfe000640 1 -- 172.21.15.110:0/1880308825 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9e00016090 con 0x7f9e0815c560 2024-09-10T09:59:28.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:28.859+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f9e080b1630 con 0x7f9e0815c560 2024-09-10T09:59:29.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9dfe000640 1 -- 172.21.15.110:0/1880308825 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v82) ==== 143+0+0 (secure 0 0 0) 0x7f9e000efa80 con 0x7f9e0815c560 2024-09-10T09:59:29.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9de8067a70 msgr2=0x7f9de8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9de8067a70 0x7f9de8069f30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9df80074a0 tx=0x7f9df800b040 comp rx=0 tx=0).stop 2024-09-10T09:59:29.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0815c560 msgr2=0x7f9e0815f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0815c560 0x7f9e0815f570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097c0 tx=0x7f9e00019180 comp rx=0 tx=0).stop 2024-09-10T09:59:29.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 shutdown_connections 2024-09-10T09:59:29.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9de8067a70 0x7f9de8069f30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e0815c560 0x7f9e0815f570 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e08155970 0x7f9e0815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 --2- 172.21.15.110:0/1880308825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e0806f1e0 0x7f9e0806dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 >> 172.21.15.110:0/1880308825 conn(0x7f9e0806bc50 msgr2=0x7f9e0806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:29.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 shutdown_connections 2024-09-10T09:59:29.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.803+0000 7f9e0cc00640 1 -- 172.21.15.110:0/1880308825 wait complete. 2024-09-10T09:59:29.831 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T09:59:29.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.967+0000 7ff15a400640 1 Processor -- start 2024-09-10T09:59:29.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.967+0000 7ff15a400640 1 -- start start 2024-09-10T09:59:29.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.967+0000 7ff15a400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff1540ce4d0 0x7ff15406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.967+0000 7ff15a400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1540b4960 con 0x7ff15406e850 2024-09-10T09:59:29.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.967+0000 7ff15a400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1540b4c40 con 0x7ff15415b330 2024-09-10T09:59:29.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.967+0000 7ff15a400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff1540b5570 con 0x7ff1540ce4d0 2024-09-10T09:59:29.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.967+0000 7ff152a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff1540ce4d0 0x7ff15406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff153e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff15406e850 0x7ff1540b0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35348/0 (socket says 172.21.15.110:35348) 2024-09-10T09:59:29.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff153e00640 1 -- 172.21.15.110:0/199635832 learned_addr learned my addr 172.21.15.110:0/199635832 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:29.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff152a00640 1 -- 172.21.15.110:0/199635832 >> v1:172.21.15.145:6789/0 conn(0x7ff15415b330 legacy=0x7ff15406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff152a00640 1 -- 172.21.15.110:0/199635832 >> v1:172.21.15.110:6789/0 conn(0x7ff15406e850 legacy=0x7ff1540b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff152a00640 1 -- 172.21.15.110:0/199635832 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1540b4f20 con 0x7ff1540ce4d0 2024-09-10T09:59:29.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff152a00640 1 --2- 172.21.15.110:0/199635832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540ce4d0 0x7ff15406c6a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff1400097c0 tx=0x7ff14001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5e8b45a09a87f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff152000640 1 -- 172.21.15.110:0/199635832 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400b1070 con 0x7ff1540ce4d0 2024-09-10T09:59:29.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff152000640 1 -- 172.21.15.110:0/199635832 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400ac410 con 0x7ff1540ce4d0 2024-09-10T09:59:29.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff152000640 1 -- 172.21.15.110:0/199635832 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400ac7e0 con 0x7ff1540ce4d0 2024-09-10T09:59:29.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff15a400640 1 -- 172.21.15.110:0/199635832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540ce4d0 msgr2=0x7ff15406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.971+0000 7ff15a400640 1 --2- 172.21.15.110:0/199635832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540ce4d0 0x7ff15406c6a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff1400097c0 tx=0x7ff14001c4a0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- 172.21.15.110:0/199635832 shutdown_connections 2024-09-10T09:59:29.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 --2- 172.21.15.110:0/199635832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540ce4d0 0x7ff15406c6a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- 172.21.15.110:0/199635832 >> 172.21.15.110:0/199635832 conn(0x7ff154062e40 msgr2=0x7ff15406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:29.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- 172.21.15.110:0/199635832 shutdown_connections 2024-09-10T09:59:29.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- 172.21.15.110:0/199635832 wait complete. 2024-09-10T09:59:29.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 Processor -- start 2024-09-10T09:59:29.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- start start 2024-09-10T09:59:29.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff15406e850 0x7ff15414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff15415b330 0x7ff15406db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff15415dad0 0x7ff15406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff154169450 con 0x7ff15415dad0 2024-09-10T09:59:29.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541692d0 con 0x7ff15406e850 2024-09-10T09:59:29.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff1541695d0 con 0x7ff15415b330 2024-09-10T09:59:29.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff15415dad0 0x7ff15406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff15415dad0 0x7ff15406e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56818/0 (socket says 172.21.15.110:56818) 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 -- 172.21.15.110:0/3396372134 learned_addr learned my addr 172.21.15.110:0/3396372134 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff15406e850 0x7ff15414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff152a00640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff15415b330 0x7ff15406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 -- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff15415b330 msgr2=0x7ff15406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff15415b330 0x7ff15406db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 -- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff15406e850 msgr2=0x7ff15414fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff15406e850 0x7ff15414fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 -- 172.21.15.110:0/3396372134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff154156b80 con 0x7ff15415dad0 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff152a00640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff15415b330 0x7ff15406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff15406e850 0x7ff15414fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff153e00640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff15415dad0 0x7ff15406e0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c3f0 tx=0x7ff14c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff14b400640 1 -- 172.21.15.110:0/3396372134 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c017070 con 0x7ff15415dad0 2024-09-10T09:59:29.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff14b400640 1 -- 172.21.15.110:0/3396372134 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14c005cf0 con 0x7ff15415dad0 2024-09-10T09:59:29.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff154156e60 con 0x7ff15415dad0 2024-09-10T09:59:29.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.975+0000 7ff14b400640 1 -- 172.21.15.110:0/3396372134 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0072d0 con 0x7ff15415dad0 2024-09-10T09:59:29.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.979+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff154157420 con 0x7ff15415dad0 2024-09-10T09:59:29.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.979+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15406c6a0 con 0x7ff15415dad0 2024-09-10T09:59:29.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.979+0000 7ff14b400640 1 -- 172.21.15.110:0/3396372134 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff14c01a070 con 0x7ff15415dad0 2024-09-10T09:59:29.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.979+0000 7ff14b400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff134067a70 0x7ff134069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.979+0000 7ff14b400640 1 -- 172.21.15.110:0/3396372134 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7ff14c084f50 con 0x7ff15415dad0 2024-09-10T09:59:29.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.979+0000 7ff153400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff134067a70 0x7ff134069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.979+0000 7ff153400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff134067a70 0x7ff134069f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff144001d70 tx=0x7ff144009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:29.983+0000 7ff14b400640 1 -- 172.21.15.110:0/3396372134 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff14c055420 con 0x7ff15415dad0 2024-09-10T09:59:30.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.139+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff1540b0a00 con 0x7ff15415dad0 2024-09-10T09:59:30.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.143+0000 7ff14b400640 1 -- 172.21.15.110:0/3396372134 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7ff14c059720 con 0x7ff15415dad0 2024-09-10T09:59:30.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff134067a70 msgr2=0x7ff134069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff134067a70 0x7ff134069f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff144001d70 tx=0x7ff144009040 comp rx=0 tx=0).stop 2024-09-10T09:59:30.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff15415dad0 msgr2=0x7ff15406e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff15415dad0 0x7ff15406e0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c3f0 tx=0x7ff14c00c8b0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 shutdown_connections 2024-09-10T09:59:30.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff134067a70 0x7ff134069f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff15415dad0 0x7ff15406e0a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff15415b330 0x7ff15406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 --2- 172.21.15.110:0/3396372134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff15406e850 0x7ff15414fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 >> 172.21.15.110:0/3396372134 conn(0x7ff154062e40 msgr2=0x7ff1540aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 shutdown_connections 2024-09-10T09:59:30.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.147+0000 7ff15a400640 1 -- 172.21.15.110:0/3396372134 wait complete. 2024-09-10T09:59:30.170 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T09:59:30.170 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T09:59:30.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T09:59:30.357 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T09:59:30.357 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:30.357 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:30.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T09:59:30.365 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:30.366 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:30.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T09:59:30.375 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:30.375 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:30.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T09:59:30.417 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:30.417 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:30.430 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T09:59:30.430 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T09:59:30.431 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:30.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc911000640 1 Processor -- start 2024-09-10T09:59:30.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc911000640 1 -- start start 2024-09-10T09:59:30.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc911000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc90c0d13e0 0x7fc90c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc911000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc90c1603c0 0x7fc90c070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc911000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c0706a0 con 0x7fc90c15c960 2024-09-10T09:59:30.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc911000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c15cd40 con 0x7fc90c0d13e0 2024-09-10T09:59:30.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc911000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c0b6370 con 0x7fc90c1603c0 2024-09-10T09:59:30.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc909400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc90c0d13e0 0x7fc90c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc909400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc90c0d13e0 0x7fc90c0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56834/0 (socket says 172.21.15.110:56834) 2024-09-10T09:59:30.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc909400640 1 -- 172.21.15.110:0/794428696 learned_addr learned my addr 172.21.15.110:0/794428696 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:30.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc90a800640 1 --2- 172.21.15.110:0/794428696 >> v2:172.21.15.145:3300/0 conn(0x7fc90c1603c0 0x7fc90c070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc909400640 1 -- 172.21.15.110:0/794428696 >> v2:172.21.15.145:3300/0 conn(0x7fc90c1603c0 msgr2=0x7fc90c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:30.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc909400640 1 --2- 172.21.15.110:0/794428696 >> v2:172.21.15.145:3300/0 conn(0x7fc90c1603c0 0x7fc90c070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc909400640 1 -- 172.21.15.110:0/794428696 >> v1:172.21.15.145:6789/0 conn(0x7fc90c15c960 legacy=0x7fc90c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.587+0000 7fc909400640 1 -- 172.21.15.110:0/794428696 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c070a80 con 0x7fc90c0d13e0 2024-09-10T09:59:30.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc909400640 1 --2- 172.21.15.110:0/794428696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c0d13e0 0x7fc90c0b5e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc90c075ef0 tx=0x7fc8f801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e691e47d51e5de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc908a00640 1 -- 172.21.15.110:0/794428696 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80ab660 con 0x7fc90c0d13e0 2024-09-10T09:59:30.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc908a00640 1 -- 172.21.15.110:0/794428696 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f80b0070 con 0x7fc90c0d13e0 2024-09-10T09:59:30.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc908a00640 1 -- 172.21.15.110:0/794428696 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80aba20 con 0x7fc90c0d13e0 2024-09-10T09:59:30.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc911000640 1 -- 172.21.15.110:0/794428696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c0d13e0 msgr2=0x7fc90c0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc911000640 1 --2- 172.21.15.110:0/794428696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c0d13e0 0x7fc90c0b5e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc90c075ef0 tx=0x7fc8f801c460 comp rx=0 tx=0).stop 2024-09-10T09:59:30.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc911000640 1 -- 172.21.15.110:0/794428696 shutdown_connections 2024-09-10T09:59:30.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc911000640 1 --2- 172.21.15.110:0/794428696 >> v2:172.21.15.145:3300/0 conn(0x7fc90c1603c0 0x7fc90c070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc911000640 1 --2- 172.21.15.110:0/794428696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c0d13e0 0x7fc90c0b5e30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc911000640 1 -- 172.21.15.110:0/794428696 >> 172.21.15.110:0/794428696 conn(0x7fc90c0d25c0 msgr2=0x7fc90c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc911000640 1 -- 172.21.15.110:0/794428696 shutdown_connections 2024-09-10T09:59:30.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.591+0000 7fc911000640 1 -- 172.21.15.110:0/794428696 wait complete. 2024-09-10T09:59:30.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 Processor -- start 2024-09-10T09:59:30.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 -- start start 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc90c0d13e0 0x7fc90c14be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc90c15c960 0x7fc90c157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c1603c0 0x7fc90c14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c0d3150 con 0x7fc90c1603c0 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1611b0 con 0x7fc90c0d13e0 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c0d32d0 con 0x7fc90c15c960 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c1603c0 0x7fc90c14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c1603c0 0x7fc90c14abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56850/0 (socket says 172.21.15.110:56850) 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 -- 172.21.15.110:0/2150945836 learned_addr learned my addr 172.21.15.110:0/2150945836 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc909e00640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc90c0d13e0 0x7fc90c14be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc909400640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc90c15c960 0x7fc90c157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 -- 172.21.15.110:0/2150945836 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc90c15c960 msgr2=0x7fc90c157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc90c15c960 0x7fc90c157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 -- 172.21.15.110:0/2150945836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc90c0d13e0 msgr2=0x7fc90c14be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc90c0d13e0 0x7fc90c14be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 -- 172.21.15.110:0/2150945836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c14c7b0 con 0x7fc90c1603c0 2024-09-10T09:59:30.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc90a800640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c1603c0 0x7fc90c14abf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc90400e7f0 tx=0x7fc90400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc902000640 1 -- 172.21.15.110:0/2150945836 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904017070 con 0x7fc90c1603c0 2024-09-10T09:59:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc902000640 1 -- 172.21.15.110:0/2150945836 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90400c940 con 0x7fc90c1603c0 2024-09-10T09:59:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c15e7b0 con 0x7fc90c1603c0 2024-09-10T09:59:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc902000640 1 -- 172.21.15.110:0/2150945836 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90400eea0 con 0x7fc90c1603c0 2024-09-10T09:59:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.595+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c15ed70 con 0x7fc90c1603c0 2024-09-10T09:59:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.599+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c0b5e30 con 0x7fc90c1603c0 2024-09-10T09:59:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.599+0000 7fc902000640 1 -- 172.21.15.110:0/2150945836 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fc90401a070 con 0x7fc90c1603c0 2024-09-10T09:59:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.599+0000 7fc902000640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8ec067a70 0x7fc8ec069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.599+0000 7fc902000640 1 -- 172.21.15.110:0/2150945836 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fc904085af0 con 0x7fc90c1603c0 2024-09-10T09:59:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.599+0000 7fc909e00640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8ec067a70 0x7fc8ec069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.599+0000 7fc909e00640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8ec067a70 0x7fc8ec069f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc001ea0 tx=0x7fc8fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.603+0000 7fc902000640 1 -- 172.21.15.110:0/2150945836 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc904056070 con 0x7fc90c1603c0 2024-09-10T09:59:30.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.791+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc90c0d34d0 con 0x7fc90c1603c0 2024-09-10T09:59:30.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.791+0000 7fc902000640 1 -- 172.21.15.110:0/2150945836 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3920 (secure 0 0 0) 0x7fc90405a370 con 0x7fc90c1603c0 2024-09-10T09:59:30.796 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:30.796 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":19,"btime":"2024-09-10T09:59:24:787104+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4556,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1141773053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1141773053},{"type":"v1","addr":"172.21.15.145:6834","nonce":1141773053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/656217845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":656217845},{"type":"v1","addr":"172.21.15.145:6835","nonce":656217845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2244539155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2244539155},{"type":"v1","addr":"172.21.15.110:6835","nonce":2244539155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/3015670975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":3015670975},{"type":"v1","addr":"172.21.15.110:6837","nonce":3015670975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-09-10T09:59:30.797 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 19 2024-09-10T09:59:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8ec067a70 msgr2=0x7fc8ec069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8ec067a70 0x7fc8ec069f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc001ea0 tx=0x7fc8fc009040 comp rx=0 tx=0).stop 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c1603c0 msgr2=0x7fc90c14abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c1603c0 0x7fc90c14abf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc90400e7f0 tx=0x7fc90400ecb0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 shutdown_connections 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc8ec067a70 0x7fc8ec069f30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc90c1603c0 0x7fc90c14abf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc90c15c960 0x7fc90c157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 --2- 172.21.15.110:0/2150945836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc90c0d13e0 0x7fc90c14be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 >> 172.21.15.110:0/2150945836 conn(0x7fc90c0d25c0 msgr2=0x7fc90c0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 shutdown_connections 2024-09-10T09:59:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:30.795+0000 7fc911000640 1 -- 172.21.15.110:0/2150945836 wait complete. 2024-09-10T09:59:30.818 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T09:59:30.818 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T09:59:30.819 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T09:59:30.819 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:59:30.819 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T09:59:30.819 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T09:59:30.819 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T09:59:30.819 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:59:30.819 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T09:59:30.819 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T09:59:30.820 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T09:59:30.820 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:59:30.820 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T09:59:30.820 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T09:59:30.820 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T09:59:30.820 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:59:30.835 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T09:59:30.835 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T09:59:30.836 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T09:59:30.836 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T09:59:30.837 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T09:59:30.837 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T09:59:30.838 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T09:59:30.838 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T09:59:30.881 INFO:tasks.ceph.mds.b:Started 2024-09-10T09:59:30.881 DEBUG:teuthology.parallel:result is None 2024-09-10T09:59:30.883 INFO:tasks.ceph.mds.d:Started 2024-09-10T09:59:30.883 DEBUG:teuthology.parallel:result is None 2024-09-10T09:59:30.884 INFO:tasks.ceph.mds.c:Started 2024-09-10T09:59:30.884 DEBUG:teuthology.parallel:result is None 2024-09-10T09:59:30.885 INFO:tasks.ceph.mds.a:Started 2024-09-10T09:59:30.886 DEBUG:teuthology.parallel:result is None 2024-09-10T09:59:30.886 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T09:59:30.972 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T09:59:30.972 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T09:59:30.973 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T09:59:30.973 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T09:59:31.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f9396200640 1 Processor -- start 2024-09-10T09:59:31.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f9396200640 1 -- start start 2024-09-10T09:59:31.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f9396200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900cdcb0 0x7f9390167160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f9396200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f939015d4d0 con 0x7f93900b3f40 2024-09-10T09:59:31.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f9396200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f939015d7b0 con 0x7f93900cfb90 2024-09-10T09:59:31.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f9396200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f939006d130 con 0x7f93900cdcb0 2024-09-10T09:59:31.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938ea00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f93900b3f40 0x7f93900cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35364/0 (socket says 172.21.15.110:35364) 2024-09-10T09:59:31.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938ea00640 1 -- 172.21.15.110:0/2370847304 learned_addr learned my addr 172.21.15.110:0/2370847304 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:31.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938fe00640 1 --2- 172.21.15.110:0/2370847304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900cdcb0 0x7f9390167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938e000640 1 -- 172.21.15.110:0/2370847304 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3612772085 0 0) 0x7f939015d4d0 con 0x7f93900b3f40 2024-09-10T09:59:31.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938e000640 1 -- 172.21.15.110:0/2370847304 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9374003530 con 0x7f93900b3f40 2024-09-10T09:59:31.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938e000640 1 -- 172.21.15.110:0/2370847304 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232889947 0 0) 0x7f939015d7b0 con 0x7f93900cfb90 2024-09-10T09:59:31.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938e000640 1 -- 172.21.15.110:0/2370847304 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f939015d4d0 con 0x7f93900cfb90 2024-09-10T09:59:31.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938fe00640 1 -- 172.21.15.110:0/2370847304 >> v1:172.21.15.145:6789/0 conn(0x7f93900cfb90 legacy=0x7f939015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938fe00640 1 -- 172.21.15.110:0/2370847304 >> v1:172.21.15.110:6789/0 conn(0x7f93900b3f40 legacy=0x7f93900cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.027+0000 7f938fe00640 1 -- 172.21.15.110:0/2370847304 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939006d380 con 0x7f93900cdcb0 2024-09-10T09:59:31.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f938fe00640 1 --2- 172.21.15.110:0/2370847304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900cdcb0 0x7f9390167160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f939006d2b0 tx=0x7f938801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10384248fa2ed664 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:31.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f938e000640 1 -- 172.21.15.110:0/2370847304 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880b2070 con 0x7f93900cdcb0 2024-09-10T09:59:31.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f938e000640 1 -- 172.21.15.110:0/2370847304 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93880a57c0 con 0x7f93900cdcb0 2024-09-10T09:59:31.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f938e000640 1 -- 172.21.15.110:0/2370847304 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880a5b90 con 0x7f93900cdcb0 2024-09-10T09:59:31.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f9396200640 1 -- 172.21.15.110:0/2370847304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900cdcb0 msgr2=0x7f9390167160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f9396200640 1 --2- 172.21.15.110:0/2370847304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900cdcb0 0x7f9390167160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f939006d2b0 tx=0x7f938801c900 comp rx=0 tx=0).stop 2024-09-10T09:59:31.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f9396200640 1 -- 172.21.15.110:0/2370847304 shutdown_connections 2024-09-10T09:59:31.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f9396200640 1 --2- 172.21.15.110:0/2370847304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900cdcb0 0x7f9390167160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f9396200640 1 -- 172.21.15.110:0/2370847304 >> 172.21.15.110:0/2370847304 conn(0x7f9390062d10 msgr2=0x7f939006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:31.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f9396200640 1 -- 172.21.15.110:0/2370847304 shutdown_connections 2024-09-10T09:59:31.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.031+0000 7f9396200640 1 -- 172.21.15.110:0/2370847304 wait complete. 2024-09-10T09:59:31.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 Processor -- start 2024-09-10T09:59:31.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 -- start start 2024-09-10T09:59:31.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900b3f40 0x7f93900b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93900cdcb0 0x7f939014be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93900cfb90 0x7f9390154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f939016a250 con 0x7f93900cfb90 2024-09-10T09:59:31.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f939016a0d0 con 0x7f93900cdcb0 2024-09-10T09:59:31.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f939016a3d0 con 0x7f93900b3f40 2024-09-10T09:59:31.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93900cfb90 0x7f9390154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93900cfb90 0x7f9390154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56872/0 (socket says 172.21.15.110:56872) 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 -- 172.21.15.110:0/3083080821 learned_addr learned my addr 172.21.15.110:0/3083080821 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938f400640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900b3f40 0x7f93900b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938ea00640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93900cdcb0 0x7f939014be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 -- 172.21.15.110:0/3083080821 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900b3f40 msgr2=0x7f93900b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900b3f40 0x7f93900b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 -- 172.21.15.110:0/3083080821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93900cdcb0 msgr2=0x7f939014be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93900cdcb0 0x7f939014be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 -- 172.21.15.110:0/3083080821 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93900b1ef0 con 0x7f93900cfb90 2024-09-10T09:59:31.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f938fe00640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93900cfb90 0x7f9390154140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f93880b0040 tx=0x7f93880196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:31.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9387400640 1 -- 172.21.15.110:0/3083080821 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880b2050 con 0x7f93900cfb90 2024-09-10T09:59:31.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9387400640 1 -- 172.21.15.110:0/3083080821 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93880be070 con 0x7f93900cfb90 2024-09-10T09:59:31.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9390071250 con 0x7f93900cfb90 2024-09-10T09:59:31.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9387400640 1 -- 172.21.15.110:0/3083080821 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880ad890 con 0x7f93900cfb90 2024-09-10T09:59:31.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.035+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9390071810 con 0x7f93900cfb90 2024-09-10T09:59:31.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.039+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93900cd5a0 con 0x7f93900cfb90 2024-09-10T09:59:31.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.039+0000 7f9387400640 1 -- 172.21.15.110:0/3083080821 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f93880ada30 con 0x7f93900cfb90 2024-09-10T09:59:31.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.039+0000 7f9387400640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9370067a70 0x7f9370069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.039+0000 7f9387400640 1 -- 172.21.15.110:0/3083080821 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f9388120cb0 con 0x7f93900cfb90 2024-09-10T09:59:31.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.039+0000 7f938f400640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9370067a70 0x7f9370069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.039+0000 7f938f400640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9370067a70 0x7f9370069f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9380007fd0 tx=0x7f9380009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:31.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.043+0000 7f9387400640 1 -- 172.21.15.110:0/3083080821 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9388016580 con 0x7f93900cfb90 2024-09-10T09:59:31.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.211+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f93900b05a0 con 0x7f93900cfb90 2024-09-10T09:59:31.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.815+0000 7f9387400640 1 -- 172.21.15.110:0/3083080821 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) ==== 97+0+0 (secure 0 0 0) 0x7f93880f1920 con 0x7f93900cfb90 2024-09-10T09:59:31.823 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T09:59:31.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9370067a70 msgr2=0x7f9370069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9370067a70 0x7f9370069f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9380007fd0 tx=0x7f9380009040 comp rx=0 tx=0).stop 2024-09-10T09:59:31.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93900cfb90 msgr2=0x7f9390154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93900cfb90 0x7f9390154140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f93880b0040 tx=0x7f93880196a0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 shutdown_connections 2024-09-10T09:59:31.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9370067a70 0x7f9370069f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93900cfb90 0x7f9390154140 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93900cdcb0 0x7f939014be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 --2- 172.21.15.110:0/3083080821 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93900b3f40 0x7f93900b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 >> 172.21.15.110:0/3083080821 conn(0x7f9390062d10 msgr2=0x7f939006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:31.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 shutdown_connections 2024-09-10T09:59:31.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.819+0000 7f9396200640 1 -- 172.21.15.110:0/3083080821 wait complete. 2024-09-10T09:59:31.847 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T09:59:31.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.979+0000 7fcc0be00640 1 Processor -- start 2024-09-10T09:59:31.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.979+0000 7fcc0be00640 1 -- start start 2024-09-10T09:59:31.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc0c15e620 0x7fcc0c0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc0c0b6710 0x7fcc0c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0be00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc0c06d060 con 0x7fcc0c0b54f0 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0be00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0c0d1e90 con 0x7fcc0c0b6710 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0be00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0c06ce10 con 0x7fcc0c15e620 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc0c0b6710 0x7fcc0c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc0c0b6710 0x7fcc0c06e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56880/0 (socket says 172.21.15.110:56880) 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0b400640 1 -- 172.21.15.110:0/2982221692 learned_addr learned my addr 172.21.15.110:0/2982221692 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0aa00640 1 --2- 172.21.15.110:0/2982221692 >> v2:172.21.15.145:3300/0 conn(0x7fcc0c15e620 0x7fcc0c0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0b400640 1 -- 172.21.15.110:0/2982221692 >> v2:172.21.15.145:3300/0 conn(0x7fcc0c15e620 msgr2=0x7fcc0c0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0b400640 1 --2- 172.21.15.110:0/2982221692 >> v2:172.21.15.145:3300/0 conn(0x7fcc0c15e620 0x7fcc0c0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0b400640 1 -- 172.21.15.110:0/2982221692 >> v1:172.21.15.110:6789/0 conn(0x7fcc0c0b54f0 legacy=0x7fcc0c0b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.983+0000 7fcc0b400640 1 -- 172.21.15.110:0/2982221692 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c06d340 con 0x7fcc0c0b6710 2024-09-10T09:59:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0b400640 1 --2- 172.21.15.110:0/2982221692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c06e100 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c071a10 tx=0x7fcc0401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de341594e220045 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc09600640 1 -- 172.21.15.110:0/2982221692 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc040a5620 con 0x7fcc0c0b6710 2024-09-10T09:59:31.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc09600640 1 -- 172.21.15.110:0/2982221692 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc040a57c0 con 0x7fcc0c0b6710 2024-09-10T09:59:31.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc09600640 1 -- 172.21.15.110:0/2982221692 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc040a5b90 con 0x7fcc0c0b6710 2024-09-10T09:59:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- 172.21.15.110:0/2982221692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 msgr2=0x7fcc0c06e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2982221692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c06e100 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c071a10 tx=0x7fcc0401c960 comp rx=0 tx=0).stop 2024-09-10T09:59:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- 172.21.15.110:0/2982221692 shutdown_connections 2024-09-10T09:59:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2982221692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c06e100 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2982221692 >> v2:172.21.15.145:3300/0 conn(0x7fcc0c15e620 0x7fcc0c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- 172.21.15.110:0/2982221692 >> 172.21.15.110:0/2982221692 conn(0x7fcc0c06f440 msgr2=0x7fcc0c06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- 172.21.15.110:0/2982221692 shutdown_connections 2024-09-10T09:59:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- 172.21.15.110:0/2982221692 wait complete. 2024-09-10T09:59:31.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 Processor -- start 2024-09-10T09:59:31.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- start start 2024-09-10T09:59:31.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc0c0b54f0 0x7fcc0c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c15e620 0x7fcc0c0d1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c16b470 con 0x7fcc0c0b6710 2024-09-10T09:59:31.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c16b2f0 con 0x7fcc0c15e620 2024-09-10T09:59:31.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0be00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c16b5f0 con 0x7fcc0c0b54f0 2024-09-10T09:59:31.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.987+0000 7fcc0a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c0cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56890/0 (socket says 172.21.15.110:56890) 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0a000640 1 -- 172.21.15.110:0/2424548996 learned_addr learned my addr 172.21.15.110:0/2424548996 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0aa00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc0c0b54f0 0x7fcc0c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0b400640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c15e620 0x7fcc0c0d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0a000640 1 -- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc0c0b54f0 msgr2=0x7fcc0c159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0a000640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc0c0b54f0 0x7fcc0c159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0a000640 1 -- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c15e620 msgr2=0x7fcc0c0d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0a000640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c15e620 0x7fcc0c0d1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0a000640 1 -- 172.21.15.110:0/2424548996 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c15c6c0 con 0x7fcc0c0b6710 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0b400640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c15e620 0x7fcc0c0d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0a000640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c0cb870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c420 tx=0x7fcbf800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc03400640 1 -- 172.21.15.110:0/2424548996 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80052d0 con 0x7fcc0c0b6710 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc03400640 1 -- 172.21.15.110:0/2424548996 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf8005470 con 0x7fcc0c0b6710 2024-09-10T09:59:31.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c15c3e0 con 0x7fcc0c0b6710 2024-09-10T09:59:31.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc03400640 1 -- 172.21.15.110:0/2424548996 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8007260 con 0x7fcc0c0b6710 2024-09-10T09:59:31.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c14dc00 con 0x7fcc0c0b6710 2024-09-10T09:59:31.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c14df30 con 0x7fcc0c0b6710 2024-09-10T09:59:31.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc03400640 1 -- 172.21.15.110:0/2424548996 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fcbf8007400 con 0x7fcc0c0b6710 2024-09-10T09:59:31.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc03400640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcbec067a70 0x7fcbec069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:31.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.991+0000 7fcc03400640 1 -- 172.21.15.110:0/2424548996 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fcbf80850f0 con 0x7fcc0c0b6710 2024-09-10T09:59:31.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.995+0000 7fcc0aa00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcbec067a70 0x7fcbec069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:31.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.995+0000 7fcc0aa00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcbec067a70 0x7fcbec069f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0079c0 tx=0x7fcbfc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:32.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:31.999+0000 7fcc03400640 1 -- 172.21.15.110:0/2424548996 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcbf80556a0 con 0x7fcc0c0b6710 2024-09-10T09:59:32.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.207+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcc0c06e100 con 0x7fcc0c0b6710 2024-09-10T09:59:32.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc03400640 1 -- 172.21.15.110:0/2424548996 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7fcbf80599a0 con 0x7fcc0c0b6710 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:59:32.214 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.216 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.217 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:59:32.218 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.220 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.221 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:59:32.223 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T09:59:32.225 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T09:59:32.226 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.227 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T09:59:32.228 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcbec067a70 msgr2=0x7fcbec069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcbec067a70 0x7fcbec069f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0079c0 tx=0x7fcbfc008040 comp rx=0 tx=0).stop 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 msgr2=0x7fcc0c0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c0cb870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c420 tx=0x7fcbf800c8e0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 shutdown_connections 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcbec067a70 0x7fcbec069f30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc0c15e620 0x7fcc0c0d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc0c0b6710 0x7fcc0c0cb870 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 --2- 172.21.15.110:0/2424548996 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc0c0b54f0 0x7fcc0c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 >> 172.21.15.110:0/2424548996 conn(0x7fcc0c06f440 msgr2=0x7fcc0c0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:32.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 shutdown_connections 2024-09-10T09:59:32.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.211+0000 7fcc0be00640 1 -- 172.21.15.110:0/2424548996 wait complete. 2024-09-10T09:59:32.235 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T09:59:32.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.367+0000 7f1f63400640 1 Processor -- start 2024-09-10T09:59:32.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.367+0000 7f1f63400640 1 -- start start 2024-09-10T09:59:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f63400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1f5c15c960 0x7f1f5c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f63400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1f5c1603c0 0x7f1f5c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f63400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f5c0706a0 con 0x7f1f5c0d04c0 2024-09-10T09:59:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f63400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1f5c15cd40 con 0x7f1f5c15c960 2024-09-10T09:59:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f63400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1f5c0b6370 con 0x7f1f5c1603c0 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f60a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1f5c15c960 0x7f1f5c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f5be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1f5c1603c0 0x7f1f5c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f61400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1f5c0d04c0 0x7f1f5c070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35390/0 (socket says 172.21.15.110:35390) 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f60a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1f5c15c960 0x7f1f5c0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56906/0 (socket says 172.21.15.110:56906) 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f61400640 1 -- 172.21.15.110:0/2207852655 learned_addr learned my addr 172.21.15.110:0/2207852655 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f60a00640 1 -- 172.21.15.110:0/2207852655 >> v2:172.21.15.145:3300/0 conn(0x7f1f5c1603c0 msgr2=0x7f1f5c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f60a00640 1 --2- 172.21.15.110:0/2207852655 >> v2:172.21.15.145:3300/0 conn(0x7f1f5c1603c0 0x7f1f5c0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f60a00640 1 -- 172.21.15.110:0/2207852655 >> v1:172.21.15.110:6789/0 conn(0x7f1f5c0d04c0 legacy=0x7f1f5c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f60a00640 1 -- 172.21.15.110:0/2207852655 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c070a80 con 0x7f1f5c15c960 2024-09-10T09:59:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f5be00640 1 --2- 172.21.15.110:0/2207852655 >> v2:172.21.15.145:3300/0 conn(0x7f1f5c1603c0 0x7f1f5c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.371+0000 7f1f60a00640 1 --2- 172.21.15.110:0/2207852655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c0b5e30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0b64f0 tx=0x7f1f5001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8bc33e6f52cee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f5b400640 1 -- 172.21.15.110:0/2207852655 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500b0070 con 0x7f1f5c15c960 2024-09-10T09:59:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f5b400640 1 -- 172.21.15.110:0/2207852655 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f500ab600 con 0x7f1f5c15c960 2024-09-10T09:59:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f5b400640 1 -- 172.21.15.110:0/2207852655 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500ab9d0 con 0x7f1f5c15c960 2024-09-10T09:59:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- 172.21.15.110:0/2207852655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 msgr2=0x7f1f5c0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 --2- 172.21.15.110:0/2207852655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c0b5e30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0b64f0 tx=0x7f1f5001c310 comp rx=0 tx=0).stop 2024-09-10T09:59:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- 172.21.15.110:0/2207852655 shutdown_connections 2024-09-10T09:59:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 --2- 172.21.15.110:0/2207852655 >> v2:172.21.15.145:3300/0 conn(0x7f1f5c1603c0 0x7f1f5c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 --2- 172.21.15.110:0/2207852655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c0b5e30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- 172.21.15.110:0/2207852655 >> 172.21.15.110:0/2207852655 conn(0x7f1f5c0d25c0 msgr2=0x7f1f5c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- 172.21.15.110:0/2207852655 shutdown_connections 2024-09-10T09:59:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- 172.21.15.110:0/2207852655 wait complete. 2024-09-10T09:59:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 Processor -- start 2024-09-10T09:59:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- start start 2024-09-10T09:59:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1f5c0d04c0 0x7f1f5c14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f5c1603c0 0x7f1f5c06e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c0d3150 con 0x7f1f5c15c960 2024-09-10T09:59:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c1611b0 con 0x7f1f5c1603c0 2024-09-10T09:59:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f63400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5c0d32d0 con 0x7f1f5c0d04c0 2024-09-10T09:59:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f5be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f5be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c06cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56922/0 (socket says 172.21.15.110:56922) 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f5be00640 1 -- 172.21.15.110:0/347806524 learned_addr learned my addr 172.21.15.110:0/347806524 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f60a00640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1f5c0d04c0 0x7f1f5c14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.375+0000 7f1f61400640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f5c1603c0 0x7f1f5c06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f5be00640 1 -- 172.21.15.110:0/347806524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1f5c0d04c0 msgr2=0x7f1f5c14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f5be00640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1f5c0d04c0 0x7f1f5c14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f5be00640 1 -- 172.21.15.110:0/347806524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f5c1603c0 msgr2=0x7f1f5c06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f5be00640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f5c1603c0 0x7f1f5c06e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f5be00640 1 -- 172.21.15.110:0/347806524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c06d660 con 0x7f1f5c15c960 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f5be00640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c06cd60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800ebb0 tx=0x7f1f4800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f58c00640 1 -- 172.21.15.110:0/347806524 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48017070 con 0x7f1f5c15c960 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f58c00640 1 -- 172.21.15.110:0/347806524 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4800cc30 con 0x7f1f5c15c960 2024-09-10T09:59:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f58c00640 1 -- 172.21.15.110:0/347806524 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48013470 con 0x7f1f5c15c960 2024-09-10T09:59:32.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5c06d940 con 0x7f1f5c15c960 2024-09-10T09:59:32.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5c06df00 con 0x7f1f5c15c960 2024-09-10T09:59:32.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5c071a80 con 0x7f1f5c15c960 2024-09-10T09:59:32.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f58c00640 1 -- 172.21.15.110:0/347806524 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f1f4801a070 con 0x7f1f5c15c960 2024-09-10T09:59:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f58c00640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1f3c067a70 0x7f1f3c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f58c00640 1 -- 172.21.15.110:0/347806524 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f1f48085e50 con 0x7f1f5c15c960 2024-09-10T09:59:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.379+0000 7f1f60a00640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1f3c067a70 0x7f1f3c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.383+0000 7f1f60a00640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1f3c067a70 0x7f1f3c069f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ad000 tx=0x7f1f500a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:32.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.387+0000 7f1f58c00640 1 -- 172.21.15.110:0/347806524 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1f480564b0 con 0x7f1f5c15c960 2024-09-10T09:59:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.543+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1f5c0d34d0 con 0x7f1f5c15c960 2024-09-10T09:59:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.543+0000 7f1f58c00640 1 -- 172.21.15.110:0/347806524 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7f1f4805a7b0 con 0x7f1f5c15c960 2024-09-10T09:59:32.549 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:32.549 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T09:59:32.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:32.549 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T09:59:32.549 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T09:59:32.549 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:32.549 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1f3c067a70 msgr2=0x7f1f3c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1f3c067a70 0x7f1f3c069f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ad000 tx=0x7f1f500a6000 comp rx=0 tx=0).stop 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 msgr2=0x7f1f5c06cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c06cd60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800ebb0 tx=0x7f1f4800c520 comp rx=0 tx=0).stop 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 shutdown_connections 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1f3c067a70 0x7f1f3c069f30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f5c1603c0 0x7f1f5c06e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f5c15c960 0x7f1f5c06cd60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 --2- 172.21.15.110:0/347806524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1f5c0d04c0 0x7f1f5c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 >> 172.21.15.110:0/347806524 conn(0x7f1f5c0d25c0 msgr2=0x7f1f5c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 shutdown_connections 2024-09-10T09:59:32.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.547+0000 7f1f63400640 1 -- 172.21.15.110:0/347806524 wait complete. 2024-09-10T09:59:32.569 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T09:59:32.569 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T09:59:32.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.703+0000 7f244b200640 1 Processor -- start 2024-09-10T09:59:32.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.703+0000 7f244b200640 1 -- start start 2024-09-10T09:59:32.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f244b200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2444151870 0x7f24440b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f244b200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f244415ebd0 0x7f24440b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f244b200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24440b47d0 con 0x7f24440b5ad0 2024-09-10T09:59:32.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f244b200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f24440b42b0 con 0x7f2444151870 2024-09-10T09:59:32.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f244b200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f24440b4ab0 con 0x7f244415ebd0 2024-09-10T09:59:32.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f2449e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2444151870 0x7f24440b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f2449e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2444151870 0x7f24440b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56924/0 (socket says 172.21.15.110:56924) 2024-09-10T09:59:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f2449e00640 1 -- 172.21.15.110:0/2306931525 learned_addr learned my addr 172.21.15.110:0/2306931525 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f244a800640 1 --2- 172.21.15.110:0/2306931525 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f244415ebd0 0x7f24440b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f2449e00640 1 -- 172.21.15.110:0/2306931525 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f244415ebd0 msgr2=0x7f24440b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f2449e00640 1 --2- 172.21.15.110:0/2306931525 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f244415ebd0 0x7f24440b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f2449e00640 1 -- 172.21.15.110:0/2306931525 >> v1:172.21.15.145:6789/0 conn(0x7f24440b5ad0 legacy=0x7f244415e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f2449e00640 1 -- 172.21.15.110:0/2306931525 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24440b4d90 con 0x7f2444151870 2024-09-10T09:59:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.707+0000 7f244a800640 1 --2- 172.21.15.110:0/2306931525 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f244415ebd0 0x7f24440b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f2449e00640 1 --2- 172.21.15.110:0/2306931525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2444151870 0x7f24440b5590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2438009870 tx=0x7f243801d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6789735f4402690 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f2448a00640 1 -- 172.21.15.110:0/2306931525 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380a76d0 con 0x7f2444151870 2024-09-10T09:59:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f2448a00640 1 -- 172.21.15.110:0/2306931525 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24380a7870 con 0x7f2444151870 2024-09-10T09:59:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f2448a00640 1 -- 172.21.15.110:0/2306931525 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380a7c40 con 0x7f2444151870 2024-09-10T09:59:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 -- 172.21.15.110:0/2306931525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2444151870 msgr2=0x7f24440b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 --2- 172.21.15.110:0/2306931525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2444151870 0x7f24440b5590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2438009870 tx=0x7f243801d6d0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 -- 172.21.15.110:0/2306931525 shutdown_connections 2024-09-10T09:59:32.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 --2- 172.21.15.110:0/2306931525 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f244415ebd0 0x7f24440b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 --2- 172.21.15.110:0/2306931525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2444151870 0x7f24440b5590 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 -- 172.21.15.110:0/2306931525 >> 172.21.15.110:0/2306931525 conn(0x7f244406d0f0 msgr2=0x7f244406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:32.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 -- 172.21.15.110:0/2306931525 shutdown_connections 2024-09-10T09:59:32.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 -- 172.21.15.110:0/2306931525 wait complete. 2024-09-10T09:59:32.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 Processor -- start 2024-09-10T09:59:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.711+0000 7f244b200640 1 -- start start 2024-09-10T09:59:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24440b5ad0 0x7f24440b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2444151870 0x7f24441f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f244415ebd0 0x7f24441487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24440ce350 con 0x7f244415ebd0 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24440ce1d0 con 0x7f2444151870 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f24440ce4d0 con 0x7f24440b5ad0 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f244415ebd0 0x7f24441487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f244415ebd0 0x7f24441487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56934/0 (socket says 172.21.15.110:56934) 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 -- 172.21.15.110:0/2067357567 learned_addr learned my addr 172.21.15.110:0/2067357567 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f2449e00640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24440b5ad0 0x7f24440b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f2449400640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2444151870 0x7f24441f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 -- 172.21.15.110:0/2067357567 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24440b5ad0 msgr2=0x7f24440b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24440b5ad0 0x7f24440b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 -- 172.21.15.110:0/2067357567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2444151870 msgr2=0x7f24441f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2444151870 0x7f24441f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:32.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 -- 172.21.15.110:0/2067357567 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24440b2880 con 0x7f244415ebd0 2024-09-10T09:59:32.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244a800640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f244415ebd0 0x7f24441487e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2440007e70 tx=0x7f244000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:32.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f243e000640 1 -- 172.21.15.110:0/2067357567 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244001a070 con 0x7f244415ebd0 2024-09-10T09:59:32.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f243e000640 1 -- 172.21.15.110:0/2067357567 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2440003bc0 con 0x7f244415ebd0 2024-09-10T09:59:32.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f243e000640 1 -- 172.21.15.110:0/2067357567 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440004030 con 0x7f244415ebd0 2024-09-10T09:59:32.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24440b2b60 con 0x7f244415ebd0 2024-09-10T09:59:32.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24441533e0 con 0x7f244415ebd0 2024-09-10T09:59:32.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24440b5590 con 0x7f244415ebd0 2024-09-10T09:59:32.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.715+0000 7f243e000640 1 -- 172.21.15.110:0/2067357567 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f244001d070 con 0x7f244415ebd0 2024-09-10T09:59:32.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.719+0000 7f243e000640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2428067a70 0x7f2428069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:32.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.719+0000 7f243e000640 1 -- 172.21.15.110:0/2067357567 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f2440085f90 con 0x7f244415ebd0 2024-09-10T09:59:32.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.719+0000 7f2449e00640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2428067a70 0x7f2428069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:32.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.719+0000 7f2449e00640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2428067a70 0x7f2428069f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f24380b0000 tx=0x7f2438008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:32.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.723+0000 7f243e000640 1 -- 172.21.15.110:0/2067357567 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2440010070 con 0x7f244415ebd0 2024-09-10T09:59:32.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:32.891+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2444073fa0 con 0x7f244415ebd0 2024-09-10T09:59:33.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f243e000640 1 -- 172.21.15.110:0/2067357567 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7f2440056450 con 0x7f244415ebd0 2024-09-10T09:59:33.826 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T09:59:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2428067a70 msgr2=0x7f2428069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2428067a70 0x7f2428069f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f24380b0000 tx=0x7f2438008040 comp rx=0 tx=0).stop 2024-09-10T09:59:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f244415ebd0 msgr2=0x7f24441487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:33.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f244415ebd0 0x7f24441487e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2440007e70 tx=0x7f244000d4a0 comp rx=0 tx=0).stop 2024-09-10T09:59:33.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 shutdown_connections 2024-09-10T09:59:33.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2428067a70 0x7f2428069f30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:33.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f244415ebd0 0x7f24441487e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:33.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2444151870 0x7f24441f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:33.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 --2- 172.21.15.110:0/2067357567 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24440b5ad0 0x7f24440b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:33.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 >> 172.21.15.110:0/2067357567 conn(0x7f244406d0f0 msgr2=0x7f244415c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:33.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 shutdown_connections 2024-09-10T09:59:33.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.822+0000 7f244b200640 1 -- 172.21.15.110:0/2067357567 wait complete. 2024-09-10T09:59:33.854 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T09:59:33.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d9e00640 1 Processor -- start 2024-09-10T09:59:33.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d9e00640 1 -- start start 2024-09-10T09:59:33.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d9e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 0x7fe4d4071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:33.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d9e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe4d40cd510 0x7fe4d40cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:33.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d9e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d406d2d0 con 0x7fe4d406e090 2024-09-10T09:59:33.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d9e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d40cdeb0 con 0x7fe4d40cd510 2024-09-10T09:59:33.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d9e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d415d9a0 con 0x7fe4d40cfe30 2024-09-10T09:59:33.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4cbe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe4d40cd510 0x7fe4d40cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:33.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 0x7fe4d4071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:34.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4cbe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe4d40cd510 0x7fe4d40cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:32906/0 (socket says 172.21.15.110:32906) 2024-09-10T09:59:34.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4cbe00640 1 -- 172.21.15.110:0/2044943707 learned_addr learned my addr 172.21.15.110:0/2044943707 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:34.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4d3400640 1 --1- 172.21.15.110:0/2044943707 >> v1:172.21.15.145:6789/0 conn(0x7fe4d406e090 0x7fe4d406e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:51786/0 (socket says 172.21.15.110:51786) 2024-09-10T09:59:34.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4cbe00640 1 -- 172.21.15.110:0/2044943707 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 msgr2=0x7fe4d4071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:34.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4cbe00640 1 --2- 172.21.15.110:0/2044943707 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 0x7fe4d4071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4cbe00640 1 -- 172.21.15.110:0/2044943707 >> v1:172.21.15.145:6789/0 conn(0x7fe4d406e090 legacy=0x7fe4d406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:34.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.994+0000 7fe4cbe00640 1 -- 172.21.15.110:0/2044943707 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d406d5b0 con 0x7fe4d40cd510 2024-09-10T09:59:34.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4cbe00640 1 --2- 172.21.15.110:0/2044943707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d40cd510 0x7fe4d40cd970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009870 tx=0x7fe4bc01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6779effd4f36c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:34.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d2000640 1 -- 172.21.15.110:0/2044943707 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0b1070 con 0x7fe4d40cd510 2024-09-10T09:59:34.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d2000640 1 -- 172.21.15.110:0/2044943707 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4bc0a8050 con 0x7fe4d40cd510 2024-09-10T09:59:34.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d2000640 1 -- 172.21.15.110:0/2044943707 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0ac860 con 0x7fe4d40cd510 2024-09-10T09:59:34.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 -- 172.21.15.110:0/2044943707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d40cd510 msgr2=0x7fe4d40cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:34.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/2044943707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d40cd510 0x7fe4d40cd970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009870 tx=0x7fe4bc01c4d0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 -- 172.21.15.110:0/2044943707 shutdown_connections 2024-09-10T09:59:34.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/2044943707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d40cd510 0x7fe4d40cd970 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/2044943707 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 0x7fe4d4071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 -- 172.21.15.110:0/2044943707 >> 172.21.15.110:0/2044943707 conn(0x7fe4d406bc50 msgr2=0x7fe4d406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:34.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 -- 172.21.15.110:0/2044943707 shutdown_connections 2024-09-10T09:59:34.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 -- 172.21.15.110:0/2044943707 wait complete. 2024-09-10T09:59:34.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 Processor -- start 2024-09-10T09:59:34.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:33.998+0000 7fe4d9e00640 1 -- start start 2024-09-10T09:59:34.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d406e090 0x7fe4d414d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d40cd510 0x7fe4d4152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 0x7fe4d4157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4072190 con 0x7fe4d40cd510 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d415d640 con 0x7fe4d406e090 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d4169180 con 0x7fe4d40cfe30 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d40cd510 0x7fe4d4152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d40cd510 0x7fe4d4152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56948/0 (socket says 172.21.15.110:56948) 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 -- 172.21.15.110:0/712158540 learned_addr learned my addr 172.21.15.110:0/712158540 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d3400640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 0x7fe4d4157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d2a00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d406e090 0x7fe4d414d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 -- 172.21.15.110:0/712158540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 msgr2=0x7fe4d4157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:34.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 0x7fe4d4157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 -- 172.21.15.110:0/712158540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d406e090 msgr2=0x7fe4d414d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d406e090 0x7fe4d414d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 -- 172.21.15.110:0/712158540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d415faa0 con 0x7fe4d40cd510 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cbe00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d40cd510 0x7fe4d4152de0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc01a100 tx=0x7fe4bc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cb400640 1 -- 172.21.15.110:0/712158540 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0b1040 con 0x7fe4d40cd510 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cb400640 1 -- 172.21.15.110:0/712158540 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4bc0acbf0 con 0x7fe4d40cd510 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d40b3670 con 0x7fe4d40cd510 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4cb400640 1 -- 172.21.15.110:0/712158540 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0b7450 con 0x7fe4d40cd510 2024-09-10T09:59:34.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d414ffe0 con 0x7fe4d40cd510 2024-09-10T09:59:34.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.002+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d40cd970 con 0x7fe4d40cd510 2024-09-10T09:59:34.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.006+0000 7fe4cb400640 1 -- 172.21.15.110:0/712158540 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fe4bc0be030 con 0x7fe4d40cd510 2024-09-10T09:59:34.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.006+0000 7fe4cb400640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4b0067a70 0x7fe4b0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:34.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.006+0000 7fe4cb400640 1 -- 172.21.15.110:0/712158540 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7fe4bc11f550 con 0x7fe4d40cd510 2024-09-10T09:59:34.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.006+0000 7fe4d2a00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4b0067a70 0x7fe4b0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:34.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.006+0000 7fe4d2a00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4b0067a70 0x7fe4b0069f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009b60 tx=0x7fe4c4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:34.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.010+0000 7fe4cb400640 1 -- 172.21.15.110:0/712158540 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe4bc0c5020 con 0x7fe4d40cd510 2024-09-10T09:59:34.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.170+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe4d4157590 con 0x7fe4d40cd510 2024-09-10T09:59:34.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.822+0000 7fe4cb400640 1 -- 172.21.15.110:0/712158540 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7fe4bc016150 con 0x7fe4d40cd510 2024-09-10T09:59:34.829 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T09:59:34.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4b0067a70 msgr2=0x7fe4b0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:34.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4b0067a70 0x7fe4b0069f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009b60 tx=0x7fe4c4009190 comp rx=0 tx=0).stop 2024-09-10T09:59:34.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d40cd510 msgr2=0x7fe4d4152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:34.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d40cd510 0x7fe4d4152de0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc01a100 tx=0x7fe4bc01ad10 comp rx=0 tx=0).stop 2024-09-10T09:59:34.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 shutdown_connections 2024-09-10T09:59:34.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4b0067a70 0x7fe4b0069f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4d40cfe30 0x7fe4d4157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d40cd510 0x7fe4d4152de0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 --2- 172.21.15.110:0/712158540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d406e090 0x7fe4d414d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:34.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 >> 172.21.15.110:0/712158540 conn(0x7fe4d406bc50 msgr2=0x7fe4d406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:34.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 shutdown_connections 2024-09-10T09:59:34.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.826+0000 7fe4d9e00640 1 -- 172.21.15.110:0/712158540 wait complete. 2024-09-10T09:59:34.865 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T09:59:35.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.998+0000 7fa286600640 1 Processor -- start 2024-09-10T09:59:35.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:34.998+0000 7fa286600640 1 -- start start 2024-09-10T09:59:35.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa286600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa280145f30 0x7fa2800b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa286600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa2800b29e0 0x7fa2800cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa286600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2800b37c0 con 0x7fa2800cfbc0 2024-09-10T09:59:35.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa286600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa2800b2e20 con 0x7fa2800b29e0 2024-09-10T09:59:35.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa286600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa2800b3aa0 con 0x7fa280145f30 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27fe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa2800cfbc0 0x7fa280160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35402/0 (socket says 172.21.15.110:35402) 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27fe00640 1 -- 172.21.15.110:0/2799033420 learned_addr learned my addr 172.21.15.110:0/2799033420 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27ea00640 1 --2- 172.21.15.110:0/2799033420 >> v2:172.21.15.145:3300/0 conn(0x7fa2800b29e0 0x7fa2800cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27f400640 1 --2- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa280145f30 0x7fa2800b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27e000640 1 -- 172.21.15.110:0/2799033420 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 257016490 0 0) 0x7fa2800b37c0 con 0x7fa2800cfbc0 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27e000640 1 -- 172.21.15.110:0/2799033420 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa264003530 con 0x7fa2800cfbc0 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27ea00640 1 -- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa280145f30 msgr2=0x7fa2800b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27ea00640 1 --2- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa280145f30 0x7fa2800b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27ea00640 1 -- 172.21.15.110:0/2799033420 >> v1:172.21.15.110:6789/0 conn(0x7fa2800cfbc0 legacy=0x7fa280160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27ea00640 1 -- 172.21.15.110:0/2799033420 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2800b3d80 con 0x7fa2800b29e0 2024-09-10T09:59:35.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.002+0000 7fa27f400640 1 --2- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa280145f30 0x7fa2800b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:35.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa27ea00640 1 --2- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2800b29e0 0x7fa2800cf680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0097c0 tx=0x7fa26c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47221e018538c07f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:35.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa27e000640 1 -- 172.21.15.110:0/2799033420 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0b0070 con 0x7fa2800b29e0 2024-09-10T09:59:35.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa27e000640 1 -- 172.21.15.110:0/2799033420 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26c0ab600 con 0x7fa2800b29e0 2024-09-10T09:59:35.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa27e000640 1 -- 172.21.15.110:0/2799033420 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0ab9d0 con 0x7fa2800b29e0 2024-09-10T09:59:35.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 -- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2800b29e0 msgr2=0x7fa2800cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 --2- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2800b29e0 0x7fa2800cf680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0097c0 tx=0x7fa26c01c4a0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 -- 172.21.15.110:0/2799033420 shutdown_connections 2024-09-10T09:59:35.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 --2- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2800b29e0 0x7fa2800cf680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 --2- 172.21.15.110:0/2799033420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa280145f30 0x7fa2800b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 -- 172.21.15.110:0/2799033420 >> 172.21.15.110:0/2799033420 conn(0x7fa280062e40 msgr2=0x7fa28006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:35.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 -- 172.21.15.110:0/2799033420 shutdown_connections 2024-09-10T09:59:35.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 -- 172.21.15.110:0/2799033420 wait complete. 2024-09-10T09:59:35.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 Processor -- start 2024-09-10T09:59:35.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.006+0000 7fa286600640 1 -- start start 2024-09-10T09:59:35.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2800b29e0 0x7fa28013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2800cfbc0 0x7fa280143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa280145f30 0x7fa28013be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa280169530 con 0x7fa2800cfbc0 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801693b0 con 0x7fa280145f30 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa2801696b0 con 0x7fa2800b29e0 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2800cfbc0 0x7fa280143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2800cfbc0 0x7fa280143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56960/0 (socket says 172.21.15.110:56960) 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 -- 172.21.15.110:0/298488716 learned_addr learned my addr 172.21.15.110:0/298488716 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27f400640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2800b29e0 0x7fa28013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27fe00640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa280145f30 0x7fa28013be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 -- 172.21.15.110:0/298488716 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2800b29e0 msgr2=0x7fa28013f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2800b29e0 0x7fa28013f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 -- 172.21.15.110:0/298488716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa280145f30 msgr2=0x7fa28013be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa280145f30 0x7fa28013be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 -- 172.21.15.110:0/298488716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2800b1160 con 0x7fa2800cfbc0 2024-09-10T09:59:35.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa27ea00640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2800cfbc0 0x7fa280143cb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0ae040 tx=0x7fa26c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:35.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa277400640 1 -- 172.21.15.110:0/298488716 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0b0040 con 0x7fa2800cfbc0 2024-09-10T09:59:35.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa277400640 1 -- 172.21.15.110:0/298488716 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26c0b9070 con 0x7fa2800cfbc0 2024-09-10T09:59:35.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2800b1440 con 0x7fa2800cfbc0 2024-09-10T09:59:35.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa277400640 1 -- 172.21.15.110:0/298488716 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0b4650 con 0x7fa2800cfbc0 2024-09-10T09:59:35.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2800b1a00 con 0x7fa2800cfbc0 2024-09-10T09:59:35.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.010+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2800cf680 con 0x7fa2800cfbc0 2024-09-10T09:59:35.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.014+0000 7fa277400640 1 -- 172.21.15.110:0/298488716 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fa26c0b47f0 con 0x7fa2800cfbc0 2024-09-10T09:59:35.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.014+0000 7fa277400640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa260067a70 0x7fa260069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:35.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.014+0000 7fa277400640 1 -- 172.21.15.110:0/298488716 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7fa26c120730 con 0x7fa2800cfbc0 2024-09-10T09:59:35.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.014+0000 7fa27f400640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa260067a70 0x7fa260069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:35.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.014+0000 7fa27f400640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa260067a70 0x7fa260069f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa270009bd0 tx=0x7fa27000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:35.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.018+0000 7fa277400640 1 -- 172.21.15.110:0/298488716 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa26c016090 con 0x7fa2800cfbc0 2024-09-10T09:59:35.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.206+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa2800723b0 con 0x7fa2800cfbc0 2024-09-10T09:59:35.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.850+0000 7fa277400640 1 -- 172.21.15.110:0/298488716 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7fa26c0f08d0 con 0x7fa2800cfbc0 2024-09-10T09:59:35.862 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 6 and data pool 7 2024-09-10T09:59:35.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.862+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa260067a70 msgr2=0x7fa260069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa260067a70 0x7fa260069f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa270009bd0 tx=0x7fa27000b040 comp rx=0 tx=0).stop 2024-09-10T09:59:35.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2800cfbc0 msgr2=0x7fa280143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:35.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2800cfbc0 0x7fa280143cb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0ae040 tx=0x7fa26c019240 comp rx=0 tx=0).stop 2024-09-10T09:59:35.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 shutdown_connections 2024-09-10T09:59:35.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa260067a70 0x7fa260069f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa280145f30 0x7fa28013be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2800cfbc0 0x7fa280143cb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 --2- 172.21.15.110:0/298488716 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2800b29e0 0x7fa28013f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:35.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 >> 172.21.15.110:0/298488716 conn(0x7fa280062e40 msgr2=0x7fa280071ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:35.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 shutdown_connections 2024-09-10T09:59:35.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:35.866+0000 7fa286600640 1 -- 172.21.15.110:0/298488716 wait complete. 2024-09-10T09:59:35.898 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:36.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.026+0000 7f6f0e800640 1 Processor -- start 2024-09-10T09:59:36.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.026+0000 7f6f0e800640 1 -- start start 2024-09-10T09:59:36.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f0e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6f080cd510 0x7f6f080cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f0e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6f0806e090 0x7f6f0806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f0e800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f08072150 con 0x7f6f080cfe30 2024-09-10T09:59:36.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f0e800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0815d9a0 con 0x7f6f080cd510 2024-09-10T09:59:36.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f0e800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0806d2d0 con 0x7f6f0806e090 2024-09-10T09:59:36.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f06a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6f080cd510 0x7f6f080cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f06a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6f080cd510 0x7f6f080cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56974/0 (socket says 172.21.15.110:56974) 2024-09-10T09:59:36.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f06a00640 1 -- 172.21.15.110:0/4062859007 learned_addr learned my addr 172.21.15.110:0/4062859007 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:36.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f07e00640 1 --2- 172.21.15.110:0/4062859007 >> v2:172.21.15.145:3300/0 conn(0x7f6f0806e090 0x7f6f0806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f06a00640 1 -- 172.21.15.110:0/4062859007 >> v2:172.21.15.145:3300/0 conn(0x7f6f0806e090 msgr2=0x7f6f0806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:36.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f06a00640 1 --2- 172.21.15.110:0/4062859007 >> v2:172.21.15.145:3300/0 conn(0x7f6f0806e090 0x7f6f0806e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f06a00640 1 -- 172.21.15.110:0/4062859007 >> v1:172.21.15.145:6789/0 conn(0x7f6f080cfe30 legacy=0x7f6f08071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:36.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.030+0000 7f6f06a00640 1 -- 172.21.15.110:0/4062859007 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0806d5b0 con 0x7f6f080cd510 2024-09-10T09:59:36.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f06a00640 1 --2- 172.21.15.110:0/4062859007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f080cd9a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806cb00 tx=0x7f6ef401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54aa5d80a7192616 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f06000640 1 -- 172.21.15.110:0/4062859007 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40b1070 con 0x7f6f080cd510 2024-09-10T09:59:36.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f06000640 1 -- 172.21.15.110:0/4062859007 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef40ac600 con 0x7f6f080cd510 2024-09-10T09:59:36.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f06000640 1 -- 172.21.15.110:0/4062859007 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40ac9d0 con 0x7f6f080cd510 2024-09-10T09:59:36.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 -- 172.21.15.110:0/4062859007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 msgr2=0x7f6f080cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 --2- 172.21.15.110:0/4062859007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f080cd9a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806cb00 tx=0x7f6ef401c490 comp rx=0 tx=0).stop 2024-09-10T09:59:36.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 -- 172.21.15.110:0/4062859007 shutdown_connections 2024-09-10T09:59:36.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 --2- 172.21.15.110:0/4062859007 >> v2:172.21.15.145:3300/0 conn(0x7f6f0806e090 0x7f6f0806e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 --2- 172.21.15.110:0/4062859007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f080cd9a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 -- 172.21.15.110:0/4062859007 >> 172.21.15.110:0/4062859007 conn(0x7f6f0806bc50 msgr2=0x7f6f0806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:36.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 -- 172.21.15.110:0/4062859007 shutdown_connections 2024-09-10T09:59:36.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 -- 172.21.15.110:0/4062859007 wait complete. 2024-09-10T09:59:36.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 Processor -- start 2024-09-10T09:59:36.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.034+0000 7f6f0e800640 1 -- start start 2024-09-10T09:59:36.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f0806e090 0x7f6f0814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f081504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f080cfe30 0x7f6f08150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08169180 con 0x7f6f080cd510 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0815d680 con 0x7f6f080cfe30 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6f08169300 con 0x7f6f0806e090 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f081504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f081504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56984/0 (socket says 172.21.15.110:56984) 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 -- 172.21.15.110:0/3758980700 learned_addr learned my addr 172.21.15.110:0/3758980700 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f07e00640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f080cfe30 0x7f6f08150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f07400640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f0806e090 0x7f6f0814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 -- 172.21.15.110:0/3758980700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f0806e090 msgr2=0x7f6f0814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f0806e090 0x7f6f0814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 -- 172.21.15.110:0/3758980700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f080cfe30 msgr2=0x7f6f08150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f080cfe30 0x7f6f08150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 -- 172.21.15.110:0/3758980700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f080b0de0 con 0x7f6f080cd510 2024-09-10T09:59:36.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f06a00640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f081504f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6ef401a0c0 tx=0x7f6ef401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6eff400640 1 -- 172.21.15.110:0/3758980700 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40b1040 con 0x7f6f080cd510 2024-09-10T09:59:36.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6eff400640 1 -- 172.21.15.110:0/3758980700 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef40b7450 con 0x7f6f080cd510 2024-09-10T09:59:36.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6eff400640 1 -- 172.21.15.110:0/3758980700 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40b7820 con 0x7f6f080cd510 2024-09-10T09:59:36.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f080b10c0 con 0x7f6f080cd510 2024-09-10T09:59:36.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f080b1680 con 0x7f6f080cd510 2024-09-10T09:59:36.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f080cc620 con 0x7f6f080cd510 2024-09-10T09:59:36.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.038+0000 7f6eff400640 1 -- 172.21.15.110:0/3758980700 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f6ef40be030 con 0x7f6f080cd510 2024-09-10T09:59:36.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.042+0000 7f6eff400640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ee8067a70 0x7f6ee8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.042+0000 7f6eff400640 1 -- 172.21.15.110:0/3758980700 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f6ef411f6f0 con 0x7f6f080cd510 2024-09-10T09:59:36.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.042+0000 7f6f07400640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ee8067a70 0x7f6ee8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.042+0000 7f6f07400640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ee8067a70 0x7f6ee8069f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8001cb0 tx=0x7f6ef8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.046+0000 7f6eff400640 1 -- 172.21.15.110:0/3758980700 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6ef4016110 con 0x7f6f080cd510 2024-09-10T09:59:36.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.202+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f0815ec00 con 0x7f6f080cd510 2024-09-10T09:59:36.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6eff400640 1 -- 172.21.15.110:0/3758980700 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16722 (secure 0 0 0) 0x7f6ef40f0050 con 0x7f6f080cd510 2024-09-10T09:59:36.210 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:36.210 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":44,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:35.213724+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:32.939641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:34.219832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:36.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ee8067a70 msgr2=0x7f6ee8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ee8067a70 0x7f6ee8069f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8001cb0 tx=0x7f6ef8009040 comp rx=0 tx=0).stop 2024-09-10T09:59:36.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 msgr2=0x7f6f081504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f081504f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6ef401a0c0 tx=0x7f6ef401acd0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 shutdown_connections 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ee8067a70 0x7f6ee8069f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f080cfe30 0x7f6f08150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f080cd510 0x7f6f081504f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 --2- 172.21.15.110:0/3758980700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f0806e090 0x7f6f0814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 >> 172.21.15.110:0/3758980700 conn(0x7f6f0806bc50 msgr2=0x7f6f08071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 shutdown_connections 2024-09-10T09:59:36.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.206+0000 7f6f0e800640 1 -- 172.21.15.110:0/3758980700 wait complete. 2024-09-10T09:59:36.232 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:36.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f4825200640 1 Processor -- start 2024-09-10T09:59:36.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f4825200640 1 -- start start 2024-09-10T09:59:36.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f4825200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f482015b4c0 0x7f48200b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f4825200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4820072ee0 0x7f4820072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f4825200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4820072b50 con 0x7f4820072740 2024-09-10T09:59:36.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f4825200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f48200b63a0 con 0x7f482015b4c0 2024-09-10T09:59:36.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f4825200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f48200b6100 con 0x7f4820072ee0 2024-09-10T09:59:36.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f481de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f482015b4c0 0x7f48200b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f481de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f482015b4c0 0x7f48200b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56986/0 (socket says 172.21.15.110:56986) 2024-09-10T09:59:36.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f481de00640 1 -- 172.21.15.110:0/733529376 learned_addr learned my addr 172.21.15.110:0/733529376 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:36.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f481d400640 1 --2- 172.21.15.110:0/733529376 >> v2:172.21.15.145:3300/0 conn(0x7f4820072ee0 0x7f4820072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f481de00640 1 -- 172.21.15.110:0/733529376 >> v2:172.21.15.145:3300/0 conn(0x7f4820072ee0 msgr2=0x7f4820072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:36.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f481de00640 1 --2- 172.21.15.110:0/733529376 >> v2:172.21.15.145:3300/0 conn(0x7f4820072ee0 0x7f4820072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f481de00640 1 -- 172.21.15.110:0/733529376 >> v1:172.21.15.110:6789/0 conn(0x7f4820072740 legacy=0x7f48200b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.362+0000 7f481de00640 1 -- 172.21.15.110:0/733529376 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482006f0c0 con 0x7f482015b4c0 2024-09-10T09:59:36.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f481de00640 1 --2- 172.21.15.110:0/733529376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f482015b4c0 0x7f48200b5bc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4820070d10 tx=0x7f481001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e1e39bc2c70047 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f481ca00640 1 -- 172.21.15.110:0/733529376 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48100a5700 con 0x7f482015b4c0 2024-09-10T09:59:36.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f481ca00640 1 -- 172.21.15.110:0/733529376 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48100a58a0 con 0x7f482015b4c0 2024-09-10T09:59:36.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f481ca00640 1 -- 172.21.15.110:0/733529376 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48100a5c70 con 0x7f482015b4c0 2024-09-10T09:59:36.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f4825200640 1 -- 172.21.15.110:0/733529376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f482015b4c0 msgr2=0x7f48200b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f4825200640 1 --2- 172.21.15.110:0/733529376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f482015b4c0 0x7f48200b5bc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4820070d10 tx=0x7f481001c930 comp rx=0 tx=0).stop 2024-09-10T09:59:36.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f4825200640 1 -- 172.21.15.110:0/733529376 shutdown_connections 2024-09-10T09:59:36.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f4825200640 1 --2- 172.21.15.110:0/733529376 >> v2:172.21.15.145:3300/0 conn(0x7f4820072ee0 0x7f4820072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f4825200640 1 --2- 172.21.15.110:0/733529376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f482015b4c0 0x7f48200b5bc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f4825200640 1 -- 172.21.15.110:0/733529376 >> 172.21.15.110:0/733529376 conn(0x7f48200d1530 msgr2=0x7f48200d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:36.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f4825200640 1 -- 172.21.15.110:0/733529376 shutdown_connections 2024-09-10T09:59:36.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.366+0000 7f4825200640 1 -- 172.21.15.110:0/733529376 wait complete. 2024-09-10T09:59:36.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 Processor -- start 2024-09-10T09:59:36.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 -- start start 2024-09-10T09:59:36.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4820072740 0x7f4820157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4820072ee0 0x7f4820152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f482015b4c0 0x7f4820151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f482015ba20 con 0x7f4820072740 2024-09-10T09:59:36.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f482015b8a0 con 0x7f4820072ee0 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4820168110 con 0x7f482015b4c0 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4820072740 0x7f4820157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4820072740 0x7f4820157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56998/0 (socket says 172.21.15.110:56998) 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 -- 172.21.15.110:0/4157417645 learned_addr learned my addr 172.21.15.110:0/4157417645 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481d400640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4820072ee0 0x7f4820152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481e800640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f482015b4c0 0x7f4820151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 -- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f482015b4c0 msgr2=0x7f4820151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f482015b4c0 0x7f4820151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 -- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4820072ee0 msgr2=0x7f4820152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4820072ee0 0x7f4820152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 -- 172.21.15.110:0/4157417645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482015a480 con 0x7f4820072740 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481e800640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f482015b4c0 0x7f4820151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:36.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f481de00640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4820072740 0x7f4820157e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4810019540 tx=0x7f48100196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4816000640 1 -- 172.21.15.110:0/4157417645 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48100bc020 con 0x7f4820072740 2024-09-10T09:59:36.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4816000640 1 -- 172.21.15.110:0/4157417645 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48100b86b0 con 0x7f4820072740 2024-09-10T09:59:36.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482015a760 con 0x7f4820072740 2024-09-10T09:59:36.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4816000640 1 -- 172.21.15.110:0/4157417645 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48100b8aa0 con 0x7f4820072740 2024-09-10T09:59:36.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48200aec30 con 0x7f4820072740 2024-09-10T09:59:36.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.370+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4820072200 con 0x7f4820072740 2024-09-10T09:59:36.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.374+0000 7f4816000640 1 -- 172.21.15.110:0/4157417645 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f48100bf020 con 0x7f4820072740 2024-09-10T09:59:36.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.374+0000 7f4816000640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4800067aa0 0x7f4800069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.374+0000 7f4816000640 1 -- 172.21.15.110:0/4157417645 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f48101212d0 con 0x7f4820072740 2024-09-10T09:59:36.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.374+0000 7f481d400640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4800067aa0 0x7f4800069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.374+0000 7f481d400640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4800067aa0 0x7f4800069f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f48200b3270 tx=0x7f480c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.378+0000 7f4816000640 1 -- 172.21.15.110:0/4157417645 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f48100165e0 con 0x7f4820072740 2024-09-10T09:59:36.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.534+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48200b5dd0 con 0x7f4820072740 2024-09-10T09:59:36.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4816000640 1 -- 172.21.15.110:0/4157417645 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16722 (secure 0 0 0) 0x7f48100f1ca0 con 0x7f4820072740 2024-09-10T09:59:36.542 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:36.542 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":44,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:35.213724+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:32.939641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:34.219832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:36.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4800067aa0 msgr2=0x7f4800069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4800067aa0 0x7f4800069f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f48200b3270 tx=0x7f480c008040 comp rx=0 tx=0).stop 2024-09-10T09:59:36.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4820072740 msgr2=0x7f4820157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4820072740 0x7f4820157e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4810019540 tx=0x7f48100196d0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 shutdown_connections 2024-09-10T09:59:36.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4800067aa0 0x7f4800069f60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f482015b4c0 0x7f4820151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4820072ee0 0x7f4820152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 --2- 172.21.15.110:0/4157417645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4820072740 0x7f4820157e50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 >> 172.21.15.110:0/4157417645 conn(0x7f48200d1530 msgr2=0x7f48200b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:36.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 shutdown_connections 2024-09-10T09:59:36.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.538+0000 7f4825200640 1 -- 172.21.15.110:0/4157417645 wait complete. 2024-09-10T09:59:36.564 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T09:59:36.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efed2000640 1 Processor -- start 2024-09-10T09:59:36.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efed2000640 1 -- start start 2024-09-10T09:59:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efed2000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efecc15c820 0x7efecc071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efed2000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efecc071980 0x7efecc073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efed2000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efecc0b5730 con 0x7efecc0737d0 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efed2000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7efecc0b5a10 con 0x7efecc071980 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efed2000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7efecc0b5cf0 con 0x7efecc15c820 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efecaa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efecc071980 0x7efecc073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efecaa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efecc071980 0x7efecc073290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57002/0 (socket says 172.21.15.110:57002) 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efecaa00640 1 -- 172.21.15.110:0/2087004692 learned_addr learned my addr 172.21.15.110:0/2087004692 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efecb400640 1 --2- 172.21.15.110:0/2087004692 >> v2:172.21.15.145:3300/0 conn(0x7efecc15c820 0x7efecc071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efecaa00640 1 -- 172.21.15.110:0/2087004692 >> v2:172.21.15.145:3300/0 conn(0x7efecc15c820 msgr2=0x7efecc071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efecaa00640 1 --2- 172.21.15.110:0/2087004692 >> v2:172.21.15.145:3300/0 conn(0x7efecc15c820 0x7efecc071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efecaa00640 1 -- 172.21.15.110:0/2087004692 >> v1:172.21.15.110:6789/0 conn(0x7efecc0737d0 legacy=0x7efecc0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.698+0000 7efecaa00640 1 -- 172.21.15.110:0/2087004692 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc0b5fd0 con 0x7efecc071980 2024-09-10T09:59:36.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efecaa00640 1 --2- 172.21.15.110:0/2087004692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc073290 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efecc069500 tx=0x7efeb801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0667de4f00f5554 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efeca000640 1 -- 172.21.15.110:0/2087004692 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80aa5f0 con 0x7efecc071980 2024-09-10T09:59:36.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efeca000640 1 -- 172.21.15.110:0/2087004692 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb80af070 con 0x7efecc071980 2024-09-10T09:59:36.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efeca000640 1 -- 172.21.15.110:0/2087004692 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80aa9b0 con 0x7efecc071980 2024-09-10T09:59:36.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efed2000640 1 -- 172.21.15.110:0/2087004692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 msgr2=0x7efecc073290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efed2000640 1 --2- 172.21.15.110:0/2087004692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc073290 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efecc069500 tx=0x7efeb801c4f0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efed2000640 1 -- 172.21.15.110:0/2087004692 shutdown_connections 2024-09-10T09:59:36.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efed2000640 1 --2- 172.21.15.110:0/2087004692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc073290 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efed2000640 1 --2- 172.21.15.110:0/2087004692 >> v2:172.21.15.145:3300/0 conn(0x7efecc15c820 0x7efecc071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efed2000640 1 -- 172.21.15.110:0/2087004692 >> 172.21.15.110:0/2087004692 conn(0x7efecc0d25a0 msgr2=0x7efecc0d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:36.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efed2000640 1 -- 172.21.15.110:0/2087004692 shutdown_connections 2024-09-10T09:59:36.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.702+0000 7efed2000640 1 -- 172.21.15.110:0/2087004692 wait complete. 2024-09-10T09:59:36.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 Processor -- start 2024-09-10T09:59:36.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 -- start start 2024-09-10T09:59:36.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efecc0737d0 0x7efecc154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efecc15c820 0x7efecc147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efecc169340 con 0x7efecc071980 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efecc1691c0 con 0x7efecc15c820 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efecc1694c0 con 0x7efecc0737d0 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57008/0 (socket says 172.21.15.110:57008) 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 -- 172.21.15.110:0/402812427 learned_addr learned my addr 172.21.15.110:0/402812427 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecaa00640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efecc0737d0 0x7efecc154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecbe00640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efecc15c820 0x7efecc147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 -- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efecc0737d0 msgr2=0x7efecc154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efecc0737d0 0x7efecc154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 -- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efecc15c820 msgr2=0x7efecc147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:36.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efecc15c820 0x7efecc147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:36.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 -- 172.21.15.110:0/402812427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc15db20 con 0x7efecc071980 2024-09-10T09:59:36.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecaa00640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efecc0737d0 0x7efecc154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:36.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efecb400640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc153d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efebc00efc0 tx=0x7efebc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efec3400640 1 -- 172.21.15.110:0/402812427 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc018020 con 0x7efecc071980 2024-09-10T09:59:36.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efec3400640 1 -- 172.21.15.110:0/402812427 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efebc014480 con 0x7efecc071980 2024-09-10T09:59:36.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efec3400640 1 -- 172.21.15.110:0/402812427 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc014850 con 0x7efecc071980 2024-09-10T09:59:36.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc15d840 con 0x7efecc071980 2024-09-10T09:59:36.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.706+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc152550 con 0x7efecc071980 2024-09-10T09:59:36.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.710+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efecc073290 con 0x7efecc071980 2024-09-10T09:59:36.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.710+0000 7efec3400640 1 -- 172.21.15.110:0/402812427 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7efebc01b020 con 0x7efecc071980 2024-09-10T09:59:36.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.710+0000 7efec3400640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efeac067a70 0x7efeac069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:36.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.710+0000 7efec3400640 1 -- 172.21.15.110:0/402812427 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7efebc086bd0 con 0x7efecc071980 2024-09-10T09:59:36.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.710+0000 7efecaa00640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efeac067a70 0x7efeac069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:36.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.710+0000 7efecaa00640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efeac067a70 0x7efeac069f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efeb80ae000 tx=0x7efeb80a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:36.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.714+0000 7efec3400640 1 -- 172.21.15.110:0/402812427 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efebc022070 con 0x7efecc071980 2024-09-10T09:59:36.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:36.910+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efecc0b4070 con 0x7efecc071980 2024-09-10T09:59:37.827 INFO:tasks.ceph.mds.a.smithi110.stderr:2024-09-10T09:59:37.822+0000 7ff99be00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T09:59:37.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.866+0000 7efec3400640 1 -- 172.21.15.110:0/402812427 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7efebc05d020 con 0x7efecc071980 2024-09-10T09:59:37.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efeac067a70 msgr2=0x7efeac069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efeac067a70 0x7efeac069f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efeb80ae000 tx=0x7efeb80a7000 comp rx=0 tx=0).stop 2024-09-10T09:59:37.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 msgr2=0x7efecc153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:37.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc153d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efebc00efc0 tx=0x7efebc00c490 comp rx=0 tx=0).stop 2024-09-10T09:59:37.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 shutdown_connections 2024-09-10T09:59:37.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efeac067a70 0x7efeac069f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efecc15c820 0x7efecc147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efecc0737d0 0x7efecc154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 --2- 172.21.15.110:0/402812427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efecc071980 0x7efecc153d50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:37.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 >> 172.21.15.110:0/402812427 conn(0x7efecc0d25a0 msgr2=0x7efecc15ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:37.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 shutdown_connections 2024-09-10T09:59:37.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:37.870+0000 7efed2000640 1 -- 172.21.15.110:0/402812427 wait complete. 2024-09-10T09:59:37.895 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T09:59:37.895 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T09:59:38.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4fb800640 1 Processor -- start 2024-09-10T09:59:38.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4fb800640 1 -- start start 2024-09-10T09:59:38.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4fb800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe4f40b2bb0 0x7fe4f40cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4fb800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f40cfbc0 0x7fe4f40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4fb800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4f40b37c0 con 0x7fe4f4145f30 2024-09-10T09:59:38.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4fb800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe4f40b3aa0 con 0x7fe4f40b2bb0 2024-09-10T09:59:38.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4fb800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f40b3d80 con 0x7fe4f40cfbc0 2024-09-10T09:59:38.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4f8e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe4f4145f30 0x7fe4f40b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35424/0 (socket says 172.21.15.110:35424) 2024-09-10T09:59:38.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.030+0000 7fe4f8e00640 1 -- 172.21.15.110:0/830636015 learned_addr learned my addr 172.21.15.110:0/830636015 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:38.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f9800640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f40cfbc0 0x7fe4f40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3e00640 1 --2- 172.21.15.110:0/830636015 >> v2:172.21.15.145:3300/0 conn(0x7fe4f40b2bb0 0x7fe4f40cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225050674 0 0) 0x7fe4f40b37c0 con 0x7fe4f4145f30 2024-09-10T09:59:38.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4d8003530 con 0x7fe4f4145f30 2024-09-10T09:59:38.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2865058888 0 0) 0x7fe4d8003530 con 0x7fe4f4145f30 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f40cfbc0 msgr2=0x7fe4f40b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f40cfbc0 0x7fe4f40b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 >> v2:172.21.15.145:3300/0 conn(0x7fe4f40b2bb0 msgr2=0x7fe4f40cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 --2- 172.21.15.110:0/830636015 >> v2:172.21.15.145:3300/0 conn(0x7fe4f40b2bb0 0x7fe4f40cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f4146310 con 0x7fe4f4145f30 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fe4e4001a20 con 0x7fe4f4145f30 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 >> v1:172.21.15.110:6789/0 conn(0x7fe4f4145f30 legacy=0x7fe4f40b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8004810 0x7fe4d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f40cfbc0 0x7fe4d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f40b3aa0 con 0x7fe4d80039c0 2024-09-10T09:59:38.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f40b3d80 con 0x7fe4d8004810 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d800b9d0 con 0x7fe4f40cfbc0 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f8e00640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f3e00640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8004810 0x7fe4d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f9800640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f40cfbc0 0x7fe4d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f8e00640 1 -- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f40cfbc0 msgr2=0x7fe4d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f8e00640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f40cfbc0 0x7fe4d80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f8e00640 1 -- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8004810 msgr2=0x7fe4d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f8e00640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8004810 0x7fe4d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.034+0000 7fe4f8e00640 1 -- 172.21.15.110:0/830636015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d8003530 con 0x7fe4d80039c0 2024-09-10T09:59:38.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4f8e00640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4d8003de0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400a890 tx=0x7fe4e401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58b3f401ba0bf7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:38.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400a4e0 con 0x7fe4d80039c0 2024-09-10T09:59:38.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e40ad630 con 0x7fe4d80039c0 2024-09-10T09:59:38.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4f3400640 1 -- 172.21.15.110:0/830636015 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e40ada20 con 0x7fe4d80039c0 2024-09-10T09:59:38.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 -- 172.21.15.110:0/830636015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 msgr2=0x7fe4d8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:38.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 --2- 172.21.15.110:0/830636015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4d8003de0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400a890 tx=0x7fe4e401d3a0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4f9800640 1 -- 172.21.15.110:0/830636015 reap_dead start 2024-09-10T09:59:38.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 -- 172.21.15.110:0/830636015 shutdown_connections 2024-09-10T09:59:38.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 -- 172.21.15.110:0/830636015 >> 172.21.15.110:0/830636015 conn(0x7fe4f4062e40 msgr2=0x7fe4f406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:38.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 -- 172.21.15.110:0/830636015 shutdown_connections 2024-09-10T09:59:38.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 -- 172.21.15.110:0/830636015 wait complete. 2024-09-10T09:59:38.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 Processor -- start 2024-09-10T09:59:38.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 -- start start 2024-09-10T09:59:38.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4f4138c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.038+0000 7fe4fb800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f415bdb0 0x7fe4f4143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4fb800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f4071c50 0x7fe4f40720d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4fb800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f40b5100 con 0x7fe4d80039c0 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4fb800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f40b4f80 con 0x7fe4f4071c50 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4fb800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f40b5280 con 0x7fe4f415bdb0 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4f4138c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4f4138c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57030/0 (socket says 172.21.15.110:57030) 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 -- 172.21.15.110:0/1971399156 learned_addr learned my addr 172.21.15.110:0/1971399156 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f3e00640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f415bdb0 0x7fe4f4143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f9800640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f4071c50 0x7fe4f40720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 -- 172.21.15.110:0/1971399156 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f415bdb0 msgr2=0x7fe4f4143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:38.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f415bdb0 0x7fe4f4143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 -- 172.21.15.110:0/1971399156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f4071c50 msgr2=0x7fe4f40720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f4071c50 0x7fe4f40720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 -- 172.21.15.110:0/1971399156 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f415c230 con 0x7fe4d80039c0 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f8e00640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4f4138c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe4e401ce10 tx=0x7fe4d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f0c00640 1 -- 172.21.15.110:0/1971399156 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e40ad7e0 con 0x7fe4d80039c0 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f0c00640 1 -- 172.21.15.110:0/1971399156 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e40ad430 con 0x7fe4d80039c0 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f4141eb0 con 0x7fe4d80039c0 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f0c00640 1 -- 172.21.15.110:0/1971399156 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4001a20 con 0x7fe4d80039c0 2024-09-10T09:59:38.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f4142470 con 0x7fe4d80039c0 2024-09-10T09:59:38.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f415deb0 con 0x7fe4d80039c0 2024-09-10T09:59:38.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.042+0000 7fe4f0c00640 1 -- 172.21.15.110:0/1971399156 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fe4e40ada00 con 0x7fe4d80039c0 2024-09-10T09:59:38.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.046+0000 7fe4f0c00640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4d4067aa0 0x7fe4d4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:38.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.046+0000 7fe4f0c00640 1 -- 172.21.15.110:0/1971399156 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe4e4120310 con 0x7fe4d80039c0 2024-09-10T09:59:38.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.046+0000 7fe4f3e00640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4d4067aa0 0x7fe4d4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:38.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.046+0000 7fe4f3e00640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4d4067aa0 0x7fe4d4069f60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4072af0 tx=0x7fe4e0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:38.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.050+0000 7fe4f0c00640 1 -- 172.21.15.110:0/1971399156 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe4e40170b0 con 0x7fe4d80039c0 2024-09-10T09:59:38.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.238+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe4f4143bd0 con 0x7fe4d80039c0 2024-09-10T09:59:38.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.866+0000 7fe4f0c00640 1 -- 172.21.15.110:0/1971399156 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fe4e40f0460 con 0x7fe4d80039c0 2024-09-10T09:59:38.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4d4067aa0 msgr2=0x7fe4d4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4d4067aa0 0x7fe4d4069f60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4072af0 tx=0x7fe4e0009190 comp rx=0 tx=0).stop 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 msgr2=0x7fe4f4138c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4f4138c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe4e401ce10 tx=0x7fe4d8005840 comp rx=0 tx=0).stop 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 shutdown_connections 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe4d4067aa0 0x7fe4d4069f60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f4071c50 0x7fe4f40720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4f415bdb0 0x7fe4f4143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 --2- 172.21.15.110:0/1971399156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d80039c0 0x7fe4f4138c40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 >> 172.21.15.110:0/1971399156 conn(0x7fe4f4062e40 msgr2=0x7fe4f40af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 shutdown_connections 2024-09-10T09:59:38.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:38.870+0000 7fe4fb800640 1 -- 172.21.15.110:0/1971399156 wait complete. 2024-09-10T09:59:38.896 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T09:59:38.896 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:39.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.026+0000 7f9c09c00640 1 Processor -- start 2024-09-10T09:59:39.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.026+0000 7f9c09c00640 1 -- start start 2024-09-10T09:59:39.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c09c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c04155680 0x7f9c040ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c09c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040cd7e0 con 0x7f9c040cd260 2024-09-10T09:59:39.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c09c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040cdac0 con 0x7f9c040cf2e0 2024-09-10T09:59:39.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c09c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9c04072770 con 0x7f9c04155680 2024-09-10T09:59:39.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9c040cd260 0x7f9c04072f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51732/0 (socket says 172.21.15.110:51732) 2024-09-10T09:59:39.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03400640 1 -- 172.21.15.110:0/3081292862 learned_addr learned my addr 172.21.15.110:0/3081292862 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03e00640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c04155680 0x7f9c040ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158330698 0 0) 0x7f9c040cd7e0 con 0x7f9c040cd260 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be8003530 con 0x7f9c040cd260 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 109379454 0 0) 0x7f9be8003530 con 0x7f9c040cd260 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c04155680 msgr2=0x7f9c040ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c04155680 0x7f9c040ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 >> v1:172.21.15.145:6789/0 conn(0x7f9c040cf2e0 legacy=0x7f9c0415c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c04072a50 con 0x7f9c040cd260 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f9bf0001680 con 0x7f9c040cd260 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 >> v1:172.21.15.110:6789/0 conn(0x7f9c040cd260 legacy=0x7f9c04072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c04155680 0x7f9be80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be8003f70 0x7f9be8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9be8004d50 0x7f9be80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04072770 con 0x7f9be8003f70 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9be800bab0 con 0x7f9be8004d50 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9be800dec0 con 0x7f9c04155680 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03400640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be8003f70 0x7f9be8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03e00640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c04155680 0x7f9be80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c09200640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9be8004d50 0x7f9be80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03400640 1 -- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c04155680 msgr2=0x7f9be80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03400640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9c04155680 0x7f9be80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03400640 1 -- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9be8004d50 msgr2=0x7f9be80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03400640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9be8004d50 0x7f9be80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.030+0000 7f9c03400640 1 -- 172.21.15.110:0/3081292862 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be8003530 con 0x7f9be8003f70 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c03400640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be8003f70 0x7f9be8004810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000c190 tx=0x7f9bf001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e1e27b7e320af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0007d00 con 0x7f9be8003f70 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf0005b00 con 0x7f9be8003f70 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c02a00640 1 -- 172.21.15.110:0/3081292862 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0007070 con 0x7f9be8003f70 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c09c00640 1 -- 172.21.15.110:0/3081292862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be8003f70 msgr2=0x7f9be8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c09c00640 1 --2- 172.21.15.110:0/3081292862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9be8003f70 0x7f9be8004810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000c190 tx=0x7f9bf001f390 comp rx=0 tx=0).stop 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c09200640 1 -- 172.21.15.110:0/3081292862 reap_dead start 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c09c00640 1 -- 172.21.15.110:0/3081292862 shutdown_connections 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c09c00640 1 -- 172.21.15.110:0/3081292862 >> 172.21.15.110:0/3081292862 conn(0x7f9c0406d0f0 msgr2=0x7f9c0406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c09c00640 1 -- 172.21.15.110:0/3081292862 shutdown_connections 2024-09-10T09:59:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.034+0000 7f9c09c00640 1 -- 172.21.15.110:0/3081292862 wait complete. 2024-09-10T09:59:39.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 Processor -- start 2024-09-10T09:59:39.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 -- start start 2024-09-10T09:59:39.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9be8003f70 0x7f9c04157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c04155d60 0x7f9c041561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c041497b0 0x7f9c04149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04155a60 con 0x7f9c041497b0 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9c040723f0 con 0x7f9c04155d60 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9c04155be0 con 0x7f9be8003f70 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c041497b0 0x7f9c04149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9be8003f70 0x7f9c04157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c041497b0 0x7f9c04149bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46850/0 (socket says 172.21.15.110:46850) 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c03400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c04155d60 0x7f9c041561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9be8003f70 0x7f9c04157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:43654/0 (socket says 172.21.15.110:43654) 2024-09-10T09:59:39.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 -- 172.21.15.110:0/1734980502 learned_addr learned my addr 172.21.15.110:0/1734980502 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 -- 172.21.15.110:0/1734980502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9be8003f70 msgr2=0x7f9c04157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9be8003f70 0x7f9c04157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 -- 172.21.15.110:0/1734980502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c04155d60 msgr2=0x7f9c041561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c04155d60 0x7f9c041561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 -- 172.21.15.110:0/1734980502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c04156970 con 0x7f9c041497b0 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09200640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c041497b0 0x7f9c04149bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9c040cd6f0 tx=0x7f9bfc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9bfbe00640 1 -- 172.21.15.110:0/1734980502 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc003980 con 0x7f9c041497b0 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9bfbe00640 1 -- 172.21.15.110:0/1734980502 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bfc018070 con 0x7f9c041497b0 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9bfbe00640 1 -- 172.21.15.110:0/1734980502 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc003d60 con 0x7f9c041497b0 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c04156c50 con 0x7f9c041497b0 2024-09-10T09:59:39.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0414a130 con 0x7f9c041497b0 2024-09-10T09:59:39.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.038+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c04157170 con 0x7f9c041497b0 2024-09-10T09:59:39.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.042+0000 7f9bfbe00640 1 -- 172.21.15.110:0/1734980502 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f9bfc0050d0 con 0x7f9c041497b0 2024-09-10T09:59:39.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.042+0000 7f9bfbe00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9be4067a70 0x7f9be4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.042+0000 7f9bfbe00640 1 -- 172.21.15.110:0/1734980502 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f9bfc085b60 con 0x7f9c041497b0 2024-09-10T09:59:39.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.042+0000 7f9c03e00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9be4067a70 0x7f9be4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.042+0000 7f9c03e00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9be4067a70 0x7f9be4069f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009ae0 tx=0x7f9bf4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.046+0000 7f9bfbe00640 1 -- 172.21.15.110:0/1734980502 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9c04157170 con 0x7f9c041497b0 2024-09-10T09:59:39.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.234+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c041596e0 con 0x7f9c041497b0 2024-09-10T09:59:39.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.234+0000 7f9bfbe00640 1 -- 172.21.15.110:0/1734980502 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5232 (secure 0 0 0) 0x7f9bfc055dd0 con 0x7f9c041497b0 2024-09-10T09:59:39.240 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:39.240 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":26,"btime":"2024-09-10T09:59:38:872051+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T09:59:39.240 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 26 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9be4067a70 msgr2=0x7f9be4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9be4067a70 0x7f9be4069f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009ae0 tx=0x7f9bf4008040 comp rx=0 tx=0).stop 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c041497b0 msgr2=0x7f9c04149bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c041497b0 0x7f9c04149bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9c040cd6f0 tx=0x7f9bfc00ec10 comp rx=0 tx=0).stop 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 shutdown_connections 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9be4067a70 0x7f9be4069f30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c041497b0 0x7f9c04149bf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c04155d60 0x7f9c041561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 --2- 172.21.15.110:0/1734980502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9be8003f70 0x7f9c04157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 >> 172.21.15.110:0/1734980502 conn(0x7f9c0406d0f0 msgr2=0x7f9c041e4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 shutdown_connections 2024-09-10T09:59:39.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.238+0000 7f9c09c00640 1 -- 172.21.15.110:0/1734980502 wait complete. 2024-09-10T09:59:39.261 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:59:39.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f61a6800640 1 Processor -- start 2024-09-10T09:59:39.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f61a6800640 1 -- start start 2024-09-10T09:59:39.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f61a6800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61a01603c0 0x7f61a00b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f61a6800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61a00706a0 con 0x7f61a00d13e0 2024-09-10T09:59:39.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f61a6800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61a0070a80 con 0x7f61a015c960 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f61a6800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f61a015cd40 con 0x7f61a01603c0 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f619ea00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f61a00d13e0 0x7f61a0070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51744/0 (socket says 172.21.15.110:51744) 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f619ea00640 1 -- 172.21.15.110:0/2918796666 learned_addr learned my addr 172.21.15.110:0/2918796666 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f619fe00640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61a01603c0 0x7f61a00b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098978422 0 0) 0x7f61a00706a0 con 0x7f61a00d13e0 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6184003530 con 0x7f61a00d13e0 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.394+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3538577576 0 0) 0x7f61a0070a80 con 0x7f61a015c960 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61a00706a0 con 0x7f61a015c960 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4046482017 0 0) 0x7f6184003530 con 0x7f61a00d13e0 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61a01603c0 msgr2=0x7f61a00b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61a01603c0 0x7f61a00b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 >> v1:172.21.15.145:6789/0 conn(0x7f61a015c960 legacy=0x7f61a00d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a0160910 con 0x7f61a00d13e0 2024-09-10T09:59:39.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f618c001680 con 0x7f61a00d13e0 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 >> v1:172.21.15.110:6789/0 conn(0x7f61a00d13e0 legacy=0x7f61a0070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61a01603c0 0x7f6184003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6184004130 0x7f6184004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184004d50 0x7f61840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61a015cd40 con 0x7f6184004130 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f618400bb90 con 0x7f6184004d50 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f618400bd90 con 0x7f61a01603c0 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619ea00640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6184004130 0x7f6184004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619fe00640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184004d50 0x7f61840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619f400640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61a01603c0 0x7f6184003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619ea00640 1 -- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61a01603c0 msgr2=0x7f6184003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:39.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619ea00640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f61a01603c0 0x7f6184003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619ea00640 1 -- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184004d50 msgr2=0x7f61840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:39.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619ea00640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6184004d50 0x7f61840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619ea00640 1 -- 172.21.15.110:0/2918796666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6184003530 con 0x7f6184004130 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619ea00640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6184004130 0x7f6184004810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f618c007fc0 tx=0x7f618c01caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d43ca0f65ff191f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c007c10 con 0x7f6184004130 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c0a9070 con 0x7f6184004130 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.398+0000 7f619e000640 1 -- 172.21.15.110:0/2918796666 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c005d20 con 0x7f6184004130 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- 172.21.15.110:0/2918796666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6184004130 msgr2=0x7f6184004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 --2- 172.21.15.110:0/2918796666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6184004130 0x7f6184004810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f618c007fc0 tx=0x7f618c01caf0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f619fe00640 1 -- 172.21.15.110:0/2918796666 reap_dead start 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- 172.21.15.110:0/2918796666 shutdown_connections 2024-09-10T09:59:39.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- 172.21.15.110:0/2918796666 >> 172.21.15.110:0/2918796666 conn(0x7f61a00d25c0 msgr2=0x7f61a00d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:39.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- 172.21.15.110:0/2918796666 shutdown_connections 2024-09-10T09:59:39.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- 172.21.15.110:0/2918796666 wait complete. 2024-09-10T09:59:39.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 Processor -- start 2024-09-10T09:59:39.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- start start 2024-09-10T09:59:39.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6184004130 0x7f61a015e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a0149380 0x7f61a0155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61a01487e0 0x7f61a0148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61a00d3150 con 0x7f61a0149380 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f61a00b6240 con 0x7f61a01487e0 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f61a6800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f61a00d32d0 con 0x7f6184004130 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f619ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a0149380 0x7f61a0155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f619ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a0149380 0x7f61a0155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46856/0 (socket says 172.21.15.110:46856) 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f619ea00640 1 -- 172.21.15.110:0/2078555484 learned_addr learned my addr 172.21.15.110:0/2078555484 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f619f400640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6184004130 0x7f61a015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.402+0000 7f619fe00640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61a01487e0 0x7f61a0148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f619ea00640 1 -- 172.21.15.110:0/2078555484 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6184004130 msgr2=0x7f61a015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f619ea00640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6184004130 0x7f61a015e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f619ea00640 1 -- 172.21.15.110:0/2078555484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61a01487e0 msgr2=0x7f61a0148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f619ea00640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61a01487e0 0x7f61a0148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f619ea00640 1 -- 172.21.15.110:0/2078555484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a0149bf0 con 0x7f61a0149380 2024-09-10T09:59:39.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f619ea00640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a0149380 0x7f61a0155880 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f618c01c560 tx=0x7f6184005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f6197400640 1 -- 172.21.15.110:0/2078555484 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c007c10 con 0x7f61a0149380 2024-09-10T09:59:39.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f6197400640 1 -- 172.21.15.110:0/2078555484 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c005d00 con 0x7f61a0149380 2024-09-10T09:59:39.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f6197400640 1 -- 172.21.15.110:0/2078555484 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c0a57d0 con 0x7f61a0149380 2024-09-10T09:59:39.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a015ec60 con 0x7f61a0149380 2024-09-10T09:59:39.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a0158d10 con 0x7f61a0149380 2024-09-10T09:59:39.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a00d0cd0 con 0x7f61a0149380 2024-09-10T09:59:39.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f6197400640 1 -- 172.21.15.110:0/2078555484 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f618c0a5970 con 0x7f61a0149380 2024-09-10T09:59:39.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f6197400640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6180067aa0 0x7f6180069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f6197400640 1 -- 172.21.15.110:0/2078555484 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f618c11f780 con 0x7f61a0149380 2024-09-10T09:59:39.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.406+0000 7f619f400640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6180067aa0 0x7f6180069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.410+0000 7f619f400640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6180067aa0 0x7f6180069f60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6190007e70 tx=0x7f6190009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.410+0000 7f6197400640 1 -- 172.21.15.110:0/2078555484 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f618c016830 con 0x7f61a0149380 2024-09-10T09:59:39.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.570+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61a00d34d0 con 0x7f61a0149380 2024-09-10T09:59:39.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f6197400640 1 -- 172.21.15.110:0/2078555484 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16709 (secure 0 0 0) 0x7f618c0effa0 con 0x7f61a0149380 2024-09-10T09:59:39.578 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:39.578 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":45,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T09:59:36.841239+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:32.939641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:34.219832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:59:39.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6180067aa0 msgr2=0x7f6180069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6180067aa0 0x7f6180069f60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6190007e70 tx=0x7f6190009040 comp rx=0 tx=0).stop 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a0149380 msgr2=0x7f61a0155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a0149380 0x7f61a0155880 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f618c01c560 tx=0x7f6184005840 comp rx=0 tx=0).stop 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 shutdown_connections 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6180067aa0 0x7f6180069f60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61a01487e0 0x7f61a0148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a0149380 0x7f61a0155880 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 --2- 172.21.15.110:0/2078555484 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6184004130 0x7f61a015e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 >> 172.21.15.110:0/2078555484 conn(0x7f61a00d25c0 msgr2=0x7f61a0073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 shutdown_connections 2024-09-10T09:59:39.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.574+0000 7f61a6800640 1 -- 172.21.15.110:0/2078555484 wait complete. 2024-09-10T09:59:39.599 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T09:59:39.600 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:59:39.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2688400640 1 Processor -- start 2024-09-10T09:59:39.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2688400640 1 -- start start 2024-09-10T09:59:39.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2688400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 0x7f26800b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2688400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f26801603c0 0x7f2680070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2688400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26800706a0 con 0x7f268015c960 2024-09-10T09:59:39.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2688400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f268015cd40 con 0x7f26801603c0 2024-09-10T09:59:39.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2688400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f26800b6370 con 0x7f26800d13e0 2024-09-10T09:59:39.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2685a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f268015c960 0x7f26800d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:54194/0 (socket says 172.21.15.110:54194) 2024-09-10T09:59:39.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2685a00640 1 -- 172.21.15.110:0/3737838224 learned_addr learned my addr 172.21.15.110:0/3737838224 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:39.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2686400640 1 --2- 172.21.15.110:0/3737838224 >> v2:172.21.15.145:3300/0 conn(0x7f26801603c0 0x7f2680070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2685000640 1 --2- 172.21.15.110:0/3737838224 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 0x7f26800b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2686400640 1 -- 172.21.15.110:0/3737838224 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 msgr2=0x7f26800b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:39.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2686400640 1 --2- 172.21.15.110:0/3737838224 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 0x7f26800b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2686400640 1 -- 172.21.15.110:0/3737838224 >> v1:172.21.15.145:6789/0 conn(0x7f268015c960 legacy=0x7f26800d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f267fe00640 1 -- 172.21.15.110:0/3737838224 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4289469616 0 0) 0x7f26800706a0 con 0x7f268015c960 2024-09-10T09:59:39.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.730+0000 7f2686400640 1 -- 172.21.15.110:0/3737838224 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2680070a80 con 0x7f26801603c0 2024-09-10T09:59:39.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2686400640 1 --2- 172.21.15.110:0/3737838224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26801603c0 0x7f2680070f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26780097c0 tx=0x7f267801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa95aeb3e7ad5833 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f267fe00640 1 -- 172.21.15.110:0/3737838224 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780b1070 con 0x7f26801603c0 2024-09-10T09:59:39.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f267fe00640 1 -- 172.21.15.110:0/3737838224 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26780a8050 con 0x7f26801603c0 2024-09-10T09:59:39.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f267fe00640 1 -- 172.21.15.110:0/3737838224 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780ac860 con 0x7f26801603c0 2024-09-10T09:59:39.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2688400640 1 -- 172.21.15.110:0/3737838224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26801603c0 msgr2=0x7f2680070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2688400640 1 --2- 172.21.15.110:0/3737838224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26801603c0 0x7f2680070f60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26780097c0 tx=0x7f267801c3e0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2688400640 1 -- 172.21.15.110:0/3737838224 shutdown_connections 2024-09-10T09:59:39.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2688400640 1 --2- 172.21.15.110:0/3737838224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26801603c0 0x7f2680070f60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2688400640 1 --2- 172.21.15.110:0/3737838224 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 0x7f26800b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2688400640 1 -- 172.21.15.110:0/3737838224 >> 172.21.15.110:0/3737838224 conn(0x7f26800d25c0 msgr2=0x7f26800d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:39.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2688400640 1 -- 172.21.15.110:0/3737838224 shutdown_connections 2024-09-10T09:59:39.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.734+0000 7f2688400640 1 -- 172.21.15.110:0/3737838224 wait complete. 2024-09-10T09:59:39.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 Processor -- start 2024-09-10T09:59:39.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 -- start start 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 0x7f268015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f268015c960 0x7f268006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26801603c0 0x7f26801550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26800d3150 con 0x7f26801603c0 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f26801611b0 con 0x7f268015c960 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f26800d32d0 con 0x7f26800d13e0 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26801603c0 0x7f26801550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26801603c0 0x7f26801550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46858/0 (socket says 172.21.15.110:46858) 2024-09-10T09:59:39.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 -- 172.21.15.110:0/1151071622 learned_addr learned my addr 172.21.15.110:0/1151071622 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:39.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2685000640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f268015c960 0x7f268006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2685a00640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 0x7f268015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 -- 172.21.15.110:0/1151071622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 msgr2=0x7f268015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 0x7f268015faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 -- 172.21.15.110:0/1151071622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f268015c960 msgr2=0x7f268006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f268015c960 0x7f268006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 -- 172.21.15.110:0/1151071622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268015e690 con 0x7f26801603c0 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2686400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26801603c0 0x7f26801550c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f267801a010 tx=0x7f267801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f267d600640 1 -- 172.21.15.110:0/1151071622 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780b1040 con 0x7f26801603c0 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f267d600640 1 -- 172.21.15.110:0/1151071622 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26780bc070 con 0x7f26801603c0 2024-09-10T09:59:39.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f267d600640 1 -- 172.21.15.110:0/1151071622 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780b7450 con 0x7f26801603c0 2024-09-10T09:59:39.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268015b250 con 0x7f26801603c0 2024-09-10T09:59:39.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268006de10 con 0x7f26801603c0 2024-09-10T09:59:39.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.738+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26800b5e30 con 0x7f26801603c0 2024-09-10T09:59:39.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.742+0000 7f267d600640 1 -- 172.21.15.110:0/1151071622 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f26780b75f0 con 0x7f26801603c0 2024-09-10T09:59:39.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.742+0000 7f267d600640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2660067a70 0x7f2660069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:39.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.742+0000 7f267d600640 1 -- 172.21.15.110:0/1151071622 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2678120360 con 0x7f26801603c0 2024-09-10T09:59:39.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.742+0000 7f2685a00640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2660067a70 0x7f2660069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:39.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.742+0000 7f2685a00640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2660067a70 0x7f2660069f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2674007fd0 tx=0x7f2674007f40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:39.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.746+0000 7f267d600640 1 -- 172.21.15.110:0/1151071622 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f268015e690 con 0x7f26801603c0 2024-09-10T09:59:39.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.902+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26800d34d0 con 0x7f2660067a70 2024-09-10T09:59:39.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.910+0000 7f267d600640 1 -- 172.21.15.110:0/1151071622 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f26800d34d0 con 0x7f2660067a70 2024-09-10T09:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:39.919 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:59:39.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2660067a70 msgr2=0x7f2660069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2660067a70 0x7f2660069f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2674007fd0 tx=0x7f2674007f40 comp rx=0 tx=0).stop 2024-09-10T09:59:39.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26801603c0 msgr2=0x7f26801550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:39.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26801603c0 0x7f26801550c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f267801a010 tx=0x7f267801ac20 comp rx=0 tx=0).stop 2024-09-10T09:59:39.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 shutdown_connections 2024-09-10T09:59:39.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2660067a70 0x7f2660069f30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26801603c0 0x7f26801550c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f268015c960 0x7f268006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 --2- 172.21.15.110:0/1151071622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26800d13e0 0x7f268015faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:39.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 >> 172.21.15.110:0/1151071622 conn(0x7f26800d25c0 msgr2=0x7f26800ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:39.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 shutdown_connections 2024-09-10T09:59:39.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:39.914+0000 7f2688400640 1 -- 172.21.15.110:0/1151071622 wait complete. 2024-09-10T09:59:39.939 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-10T09:59:39.370449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238144,"kb_used_data":5252,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736576,"statfs":{"total":773094113280,"available":772850253824,"internally_reserved":0,"allocated":5378048,"data_stored":2830555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.543981"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850002+0000","last_change":"2024-09-10T09:59:35.876141+0000","last_active":"2024-09-10T09:59:36.850002+0000","last_peered":"2024-09-10T09:59:36.850002+0000","last_clean":"2024-09-10T09:59:36.850002+0000","last_became_active":"2024-09-10T09:59:35.875635+0000","last_became_peered":"2024-09-10T09:59:35.875635+0000","last_unstale":"2024-09-10T09:59:36.850002+0000","last_undegraded":"2024-09-10T09:59:36.850002+0000","last_fullsized":"2024-09-10T09:59:36.850002+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:27:12.613140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972549+0000","last_change":"2024-09-10T09:59:35.862505+0000","last_active":"2024-09-10T09:59:36.972549+0000","last_peered":"2024-09-10T09:59:36.972549+0000","last_clean":"2024-09-10T09:59:36.972549+0000","last_became_active":"2024-09-10T09:59:35.862216+0000","last_became_peered":"2024-09-10T09:59:35.862216+0000","last_unstale":"2024-09-10T09:59:36.972549+0000","last_undegraded":"2024-09-10T09:59:36.972549+0000","last_fullsized":"2024-09-10T09:59:36.972549+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.150900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882544+0000","last_change":"2024-09-10T09:59:35.859409+0000","last_active":"2024-09-10T09:59:36.882544+0000","last_peered":"2024-09-10T09:59:36.882544+0000","last_clean":"2024-09-10T09:59:36.882544+0000","last_became_active":"2024-09-10T09:59:35.859109+0000","last_became_peered":"2024-09-10T09:59:35.859109+0000","last_unstale":"2024-09-10T09:59:36.882544+0000","last_undegraded":"2024-09-10T09:59:36.882544+0000","last_fullsized":"2024-09-10T09:59:36.882544+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:04.752794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972206+0000","last_change":"2024-09-10T09:59:35.878310+0000","last_active":"2024-09-10T09:59:36.972206+0000","last_peered":"2024-09-10T09:59:36.972206+0000","last_clean":"2024-09-10T09:59:36.972206+0000","last_became_active":"2024-09-10T09:59:35.878020+0000","last_became_peered":"2024-09-10T09:59:35.878020+0000","last_unstale":"2024-09-10T09:59:36.972206+0000","last_undegraded":"2024-09-10T09:59:36.972206+0000","last_fullsized":"2024-09-10T09:59:36.972206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:43.863724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972413+0000","last_change":"2024-09-10T09:59:35.863783+0000","last_active":"2024-09-10T09:59:36.972413+0000","last_peered":"2024-09-10T09:59:36.972413+0000","last_clean":"2024-09-10T09:59:36.972413+0000","last_became_active":"2024-09-10T09:59:35.863444+0000","last_became_peered":"2024-09-10T09:59:35.863444+0000","last_unstale":"2024-09-10T09:59:36.972413+0000","last_undegraded":"2024-09-10T09:59:36.972413+0000","last_fullsized":"2024-09-10T09:59:36.972413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:00.415525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.870205+0000","last_change":"2024-09-10T09:59:35.860245+0000","last_active":"2024-09-10T09:59:35.870205+0000","last_peered":"2024-09-10T09:59:35.870205+0000","last_clean":"2024-09-10T09:59:35.870205+0000","last_became_active":"2024-09-10T09:59:35.859982+0000","last_became_peered":"2024-09-10T09:59:35.859982+0000","last_unstale":"2024-09-10T09:59:35.870205+0000","last_undegraded":"2024-09-10T09:59:35.870205+0000","last_fullsized":"2024-09-10T09:59:35.870205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:07.106814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850187+0000","last_change":"2024-09-10T09:59:35.867580+0000","last_active":"2024-09-10T09:59:36.850187+0000","last_peered":"2024-09-10T09:59:36.850187+0000","last_clean":"2024-09-10T09:59:36.850187+0000","last_became_active":"2024-09-10T09:59:35.867298+0000","last_became_peered":"2024-09-10T09:59:35.867298+0000","last_unstale":"2024-09-10T09:59:36.850187+0000","last_undegraded":"2024-09-10T09:59:36.850187+0000","last_fullsized":"2024-09-10T09:59:36.850187+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:21.294113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971551+0000","last_change":"2024-09-10T09:59:35.869302+0000","last_active":"2024-09-10T09:59:36.971551+0000","last_peered":"2024-09-10T09:59:36.971551+0000","last_clean":"2024-09-10T09:59:36.971551+0000","last_became_active":"2024-09-10T09:59:35.867049+0000","last_became_peered":"2024-09-10T09:59:35.867049+0000","last_unstale":"2024-09-10T09:59:36.971551+0000","last_undegraded":"2024-09-10T09:59:36.971551+0000","last_fullsized":"2024-09-10T09:59:36.971551+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:51.625342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972439+0000","last_change":"2024-09-10T09:59:35.865933+0000","last_active":"2024-09-10T09:59:36.972439+0000","last_peered":"2024-09-10T09:59:36.972439+0000","last_clean":"2024-09-10T09:59:36.972439+0000","last_became_active":"2024-09-10T09:59:35.865199+0000","last_became_peered":"2024-09-10T09:59:35.865199+0000","last_unstale":"2024-09-10T09:59:36.972439+0000","last_undegraded":"2024-09-10T09:59:36.972439+0000","last_fullsized":"2024-09-10T09:59:36.972439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:56.698002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972420+0000","last_change":"2024-09-10T09:59:35.863397+0000","last_active":"2024-09-10T09:59:36.972420+0000","last_peered":"2024-09-10T09:59:36.972420+0000","last_clean":"2024-09-10T09:59:36.972420+0000","last_became_active":"2024-09-10T09:59:35.863049+0000","last_became_peered":"2024-09-10T09:59:35.863049+0000","last_unstale":"2024-09-10T09:59:36.972420+0000","last_undegraded":"2024-09-10T09:59:36.972420+0000","last_fullsized":"2024-09-10T09:59:36.972420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:01.266810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972290+0000","last_change":"2024-09-10T09:59:35.866372+0000","last_active":"2024-09-10T09:59:36.972290+0000","last_peered":"2024-09-10T09:59:36.972290+0000","last_clean":"2024-09-10T09:59:36.972290+0000","last_became_active":"2024-09-10T09:59:35.866140+0000","last_became_peered":"2024-09-10T09:59:35.866140+0000","last_unstale":"2024-09-10T09:59:36.972290+0000","last_undegraded":"2024-09-10T09:59:36.972290+0000","last_fullsized":"2024-09-10T09:59:36.972290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:47.550206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972422+0000","last_change":"2024-09-10T09:59:35.867821+0000","last_active":"2024-09-10T09:59:36.972422+0000","last_peered":"2024-09-10T09:59:36.972422+0000","last_clean":"2024-09-10T09:59:36.972422+0000","last_became_active":"2024-09-10T09:59:35.867530+0000","last_became_peered":"2024-09-10T09:59:35.867530+0000","last_unstale":"2024-09-10T09:59:36.972422+0000","last_undegraded":"2024-09-10T09:59:36.972422+0000","last_fullsized":"2024-09-10T09:59:36.972422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:48.114927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845787+0000","last_change":"2024-09-10T09:59:35.856914+0000","last_active":"2024-09-10T09:59:36.845787+0000","last_peered":"2024-09-10T09:59:36.845787+0000","last_clean":"2024-09-10T09:59:36.845787+0000","last_became_active":"2024-09-10T09:59:35.856671+0000","last_became_peered":"2024-09-10T09:59:35.856671+0000","last_unstale":"2024-09-10T09:59:36.845787+0000","last_undegraded":"2024-09-10T09:59:36.845787+0000","last_fullsized":"2024-09-10T09:59:36.845787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:43.973652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972071+0000","last_change":"2024-09-10T09:59:35.860679+0000","last_active":"2024-09-10T09:59:36.972071+0000","last_peered":"2024-09-10T09:59:36.972071+0000","last_clean":"2024-09-10T09:59:36.972071+0000","last_became_active":"2024-09-10T09:59:35.860358+0000","last_became_peered":"2024-09-10T09:59:35.860358+0000","last_unstale":"2024-09-10T09:59:36.972071+0000","last_undegraded":"2024-09-10T09:59:36.972071+0000","last_fullsized":"2024-09-10T09:59:36.972071+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:44.448528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.868445+0000","last_change":"2024-09-10T09:59:35.867073+0000","last_active":"2024-09-10T09:59:35.868445+0000","last_peered":"2024-09-10T09:59:35.868445+0000","last_clean":"2024-09-10T09:59:35.868445+0000","last_became_active":"2024-09-10T09:59:35.866800+0000","last_became_peered":"2024-09-10T09:59:35.866800+0000","last_unstale":"2024-09-10T09:59:35.868445+0000","last_undegraded":"2024-09-10T09:59:35.868445+0000","last_fullsized":"2024-09-10T09:59:35.868445+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:10:19.862442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849034+0000","last_change":"2024-09-10T09:59:35.863102+0000","last_active":"2024-09-10T09:59:36.849034+0000","last_peered":"2024-09-10T09:59:36.849034+0000","last_clean":"2024-09-10T09:59:36.849034+0000","last_became_active":"2024-09-10T09:59:35.862642+0000","last_became_peered":"2024-09-10T09:59:35.862642+0000","last_unstale":"2024-09-10T09:59:36.849034+0000","last_undegraded":"2024-09-10T09:59:36.849034+0000","last_fullsized":"2024-09-10T09:59:36.849034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:41.280886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882613+0000","last_change":"2024-09-10T09:59:35.867101+0000","last_active":"2024-09-10T09:59:36.882613+0000","last_peered":"2024-09-10T09:59:36.882613+0000","last_clean":"2024-09-10T09:59:36.882613+0000","last_became_active":"2024-09-10T09:59:35.866416+0000","last_became_peered":"2024-09-10T09:59:35.866416+0000","last_unstale":"2024-09-10T09:59:36.882613+0000","last_undegraded":"2024-09-10T09:59:36.882613+0000","last_fullsized":"2024-09-10T09:59:36.882613+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:09.990262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882775+0000","last_change":"2024-09-10T09:59:35.861444+0000","last_active":"2024-09-10T09:59:36.882775+0000","last_peered":"2024-09-10T09:59:36.882775+0000","last_clean":"2024-09-10T09:59:36.882775+0000","last_became_active":"2024-09-10T09:59:35.860859+0000","last_became_peered":"2024-09-10T09:59:35.860859+0000","last_unstale":"2024-09-10T09:59:36.882775+0000","last_undegraded":"2024-09-10T09:59:36.882775+0000","last_fullsized":"2024-09-10T09:59:36.882775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:56.488732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850140+0000","last_change":"2024-09-10T09:59:35.869726+0000","last_active":"2024-09-10T09:59:36.850140+0000","last_peered":"2024-09-10T09:59:36.850140+0000","last_clean":"2024-09-10T09:59:36.850140+0000","last_became_active":"2024-09-10T09:59:35.869441+0000","last_became_peered":"2024-09-10T09:59:35.869441+0000","last_unstale":"2024-09-10T09:59:36.850140+0000","last_undegraded":"2024-09-10T09:59:36.850140+0000","last_fullsized":"2024-09-10T09:59:36.850140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:22.255424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845537+0000","last_change":"2024-09-10T09:59:35.866032+0000","last_active":"2024-09-10T09:59:36.845537+0000","last_peered":"2024-09-10T09:59:36.845537+0000","last_clean":"2024-09-10T09:59:36.845537+0000","last_became_active":"2024-09-10T09:59:35.865773+0000","last_became_peered":"2024-09-10T09:59:35.865773+0000","last_unstale":"2024-09-10T09:59:36.845537+0000","last_undegraded":"2024-09-10T09:59:36.845537+0000","last_fullsized":"2024-09-10T09:59:36.845537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.941766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.868879+0000","last_change":"2024-09-10T09:59:35.867639+0000","last_active":"2024-09-10T09:59:35.868879+0000","last_peered":"2024-09-10T09:59:35.868879+0000","last_clean":"2024-09-10T09:59:35.868879+0000","last_became_active":"2024-09-10T09:59:35.867125+0000","last_became_peered":"2024-09-10T09:59:35.867125+0000","last_unstale":"2024-09-10T09:59:35.868879+0000","last_undegraded":"2024-09-10T09:59:35.868879+0000","last_fullsized":"2024-09-10T09:59:35.868879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:31.314231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972131+0000","last_change":"2024-09-10T09:59:35.874448+0000","last_active":"2024-09-10T09:59:36.972131+0000","last_peered":"2024-09-10T09:59:36.972131+0000","last_clean":"2024-09-10T09:59:36.972131+0000","last_became_active":"2024-09-10T09:59:35.874179+0000","last_became_peered":"2024-09-10T09:59:35.874179+0000","last_unstale":"2024-09-10T09:59:36.972131+0000","last_undegraded":"2024-09-10T09:59:36.972131+0000","last_fullsized":"2024-09-10T09:59:36.972131+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:00.415525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849433+0000","last_change":"2024-09-10T09:59:35.872953+0000","last_active":"2024-09-10T09:59:36.849433+0000","last_peered":"2024-09-10T09:59:36.849433+0000","last_clean":"2024-09-10T09:59:36.849433+0000","last_became_active":"2024-09-10T09:59:35.872695+0000","last_became_peered":"2024-09-10T09:59:35.872695+0000","last_unstale":"2024-09-10T09:59:36.849433+0000","last_undegraded":"2024-09-10T09:59:36.849433+0000","last_fullsized":"2024-09-10T09:59:36.849433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:00.472914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849202+0000","last_change":"2024-09-10T09:59:35.869862+0000","last_active":"2024-09-10T09:59:36.849202+0000","last_peered":"2024-09-10T09:59:36.849202+0000","last_clean":"2024-09-10T09:59:36.849202+0000","last_became_active":"2024-09-10T09:59:35.869239+0000","last_became_peered":"2024-09-10T09:59:35.869239+0000","last_unstale":"2024-09-10T09:59:36.849202+0000","last_undegraded":"2024-09-10T09:59:36.849202+0000","last_fullsized":"2024-09-10T09:59:36.849202+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:08.249486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849409+0000","last_change":"2024-09-10T09:59:35.859953+0000","last_active":"2024-09-10T09:59:36.849409+0000","last_peered":"2024-09-10T09:59:36.849409+0000","last_clean":"2024-09-10T09:59:36.849409+0000","last_became_active":"2024-09-10T09:59:35.859464+0000","last_became_peered":"2024-09-10T09:59:35.859464+0000","last_unstale":"2024-09-10T09:59:36.849409+0000","last_undegraded":"2024-09-10T09:59:36.849409+0000","last_fullsized":"2024-09-10T09:59:36.849409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:17.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845353+0000","last_change":"2024-09-10T09:59:35.853723+0000","last_active":"2024-09-10T09:59:36.845353+0000","last_peered":"2024-09-10T09:59:36.845353+0000","last_clean":"2024-09-10T09:59:36.845353+0000","last_became_active":"2024-09-10T09:59:35.853472+0000","last_became_peered":"2024-09-10T09:59:35.853472+0000","last_unstale":"2024-09-10T09:59:36.845353+0000","last_undegraded":"2024-09-10T09:59:36.845353+0000","last_fullsized":"2024-09-10T09:59:36.845353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:44.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882456+0000","last_change":"2024-09-10T09:59:35.874083+0000","last_active":"2024-09-10T09:59:36.882456+0000","last_peered":"2024-09-10T09:59:36.882456+0000","last_clean":"2024-09-10T09:59:36.882456+0000","last_became_active":"2024-09-10T09:59:35.873850+0000","last_became_peered":"2024-09-10T09:59:35.873850+0000","last_unstale":"2024-09-10T09:59:36.882456+0000","last_undegraded":"2024-09-10T09:59:36.882456+0000","last_fullsized":"2024-09-10T09:59:36.882456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:51.625342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972418+0000","last_change":"2024-09-10T09:59:35.867957+0000","last_active":"2024-09-10T09:59:36.972418+0000","last_peered":"2024-09-10T09:59:36.972418+0000","last_clean":"2024-09-10T09:59:36.972418+0000","last_became_active":"2024-09-10T09:59:35.867605+0000","last_became_peered":"2024-09-10T09:59:35.867605+0000","last_unstale":"2024-09-10T09:59:36.972418+0000","last_undegraded":"2024-09-10T09:59:36.972418+0000","last_fullsized":"2024-09-10T09:59:36.972418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:43.863724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849736+0000","last_change":"2024-09-10T09:59:35.860589+0000","last_active":"2024-09-10T09:59:36.849736+0000","last_peered":"2024-09-10T09:59:36.849736+0000","last_clean":"2024-09-10T09:59:36.849736+0000","last_became_active":"2024-09-10T09:59:35.860322+0000","last_became_peered":"2024-09-10T09:59:35.860322+0000","last_unstale":"2024-09-10T09:59:36.849736+0000","last_undegraded":"2024-09-10T09:59:36.849736+0000","last_fullsized":"2024-09-10T09:59:36.849736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:58.654843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882321+0000","last_change":"2024-09-10T09:59:35.866732+0000","last_active":"2024-09-10T09:59:36.882321+0000","last_peered":"2024-09-10T09:59:36.882321+0000","last_clean":"2024-09-10T09:59:36.882321+0000","last_became_active":"2024-09-10T09:59:35.866451+0000","last_became_peered":"2024-09-10T09:59:35.866451+0000","last_unstale":"2024-09-10T09:59:36.882321+0000","last_undegraded":"2024-09-10T09:59:36.882321+0000","last_fullsized":"2024-09-10T09:59:36.882321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:18.251454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.874305+0000","last_change":"2024-09-10T09:59:35.871652+0000","last_active":"2024-09-10T09:59:35.874305+0000","last_peered":"2024-09-10T09:59:35.874305+0000","last_clean":"2024-09-10T09:59:35.874305+0000","last_became_active":"2024-09-10T09:59:35.871394+0000","last_became_peered":"2024-09-10T09:59:35.871394+0000","last_unstale":"2024-09-10T09:59:35.874305+0000","last_undegraded":"2024-09-10T09:59:35.874305+0000","last_fullsized":"2024-09-10T09:59:35.874305+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:15.916085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845299+0000","last_change":"2024-09-10T09:59:35.848708+0000","last_active":"2024-09-10T09:59:36.845299+0000","last_peered":"2024-09-10T09:59:36.845299+0000","last_clean":"2024-09-10T09:59:36.845299+0000","last_became_active":"2024-09-10T09:59:35.848440+0000","last_became_peered":"2024-09-10T09:59:35.848440+0000","last_unstale":"2024-09-10T09:59:36.845299+0000","last_undegraded":"2024-09-10T09:59:36.845299+0000","last_fullsized":"2024-09-10T09:59:36.845299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:22.200828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971601+0000","last_change":"2024-09-10T09:59:35.852650+0000","last_active":"2024-09-10T09:59:36.971601+0000","last_peered":"2024-09-10T09:59:36.971601+0000","last_clean":"2024-09-10T09:59:36.971601+0000","last_became_active":"2024-09-10T09:59:35.852363+0000","last_became_peered":"2024-09-10T09:59:35.852363+0000","last_unstale":"2024-09-10T09:59:36.971601+0000","last_undegraded":"2024-09-10T09:59:36.971601+0000","last_fullsized":"2024-09-10T09:59:36.971601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:00.472914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.886471+0000","last_change":"2024-09-10T09:59:34.855180+0000","last_active":"2024-09-10T09:59:35.886471+0000","last_peered":"2024-09-10T09:59:35.886471+0000","last_clean":"2024-09-10T09:59:35.886471+0000","last_became_active":"2024-09-10T09:59:34.854770+0000","last_became_peered":"2024-09-10T09:59:34.854770+0000","last_unstale":"2024-09-10T09:59:35.886471+0000","last_undegraded":"2024-09-10T09:59:35.886471+0000","last_fullsized":"2024-09-10T09:59:35.886471+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:41.886061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850282+0000","last_change":"2024-09-10T09:59:35.859718+0000","last_active":"2024-09-10T09:59:36.850282+0000","last_peered":"2024-09-10T09:59:36.850282+0000","last_clean":"2024-09-10T09:59:36.850282+0000","last_became_active":"2024-09-10T09:59:35.859108+0000","last_became_peered":"2024-09-10T09:59:35.859108+0000","last_unstale":"2024-09-10T09:59:36.850282+0000","last_undegraded":"2024-09-10T09:59:36.850282+0000","last_fullsized":"2024-09-10T09:59:36.850282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:01.478298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882726+0000","last_change":"2024-09-10T09:59:34.845608+0000","last_active":"2024-09-10T09:59:36.882726+0000","last_peered":"2024-09-10T09:59:36.882726+0000","last_clean":"2024-09-10T09:59:36.882726+0000","last_became_active":"2024-09-10T09:59:34.844174+0000","last_became_peered":"2024-09-10T09:59:34.844174+0000","last_unstale":"2024-09-10T09:59:36.882726+0000","last_undegraded":"2024-09-10T09:59:36.882726+0000","last_fullsized":"2024-09-10T09:59:36.882726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:58.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972578+0000","last_change":"2024-09-10T09:59:35.865285+0000","last_active":"2024-09-10T09:59:36.972578+0000","last_peered":"2024-09-10T09:59:36.972578+0000","last_clean":"2024-09-10T09:59:36.972578+0000","last_became_active":"2024-09-10T09:59:35.864955+0000","last_became_peered":"2024-09-10T09:59:35.864955+0000","last_unstale":"2024-09-10T09:59:36.972578+0000","last_undegraded":"2024-09-10T09:59:36.972578+0000","last_fullsized":"2024-09-10T09:59:36.972578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:47.550206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971843+0000","last_change":"2024-09-10T09:59:34.849471+0000","last_active":"2024-09-10T09:59:36.971843+0000","last_peered":"2024-09-10T09:59:36.971843+0000","last_clean":"2024-09-10T09:59:36.971843+0000","last_became_active":"2024-09-10T09:59:34.849154+0000","last_became_peered":"2024-09-10T09:59:34.849154+0000","last_unstale":"2024-09-10T09:59:36.971843+0000","last_undegraded":"2024-09-10T09:59:36.971843+0000","last_fullsized":"2024-09-10T09:59:36.971843+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:19:44.067649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971809+0000","last_change":"2024-09-10T09:59:35.862681+0000","last_active":"2024-09-10T09:59:36.971809+0000","last_peered":"2024-09-10T09:59:36.971809+0000","last_clean":"2024-09-10T09:59:36.971809+0000","last_became_active":"2024-09-10T09:59:35.861184+0000","last_became_peered":"2024-09-10T09:59:35.861184+0000","last_unstale":"2024-09-10T09:59:36.971809+0000","last_undegraded":"2024-09-10T09:59:36.971809+0000","last_fullsized":"2024-09-10T09:59:36.971809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:07.603507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849289+0000","last_change":"2024-09-10T09:59:34.851653+0000","last_active":"2024-09-10T09:59:36.849289+0000","last_peered":"2024-09-10T09:59:36.849289+0000","last_clean":"2024-09-10T09:59:36.849289+0000","last_became_active":"2024-09-10T09:59:34.851343+0000","last_became_peered":"2024-09-10T09:59:34.851343+0000","last_unstale":"2024-09-10T09:59:36.849289+0000","last_undegraded":"2024-09-10T09:59:36.849289+0000","last_fullsized":"2024-09-10T09:59:36.849289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:50.619585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882314+0000","last_change":"2024-09-10T09:59:35.875108+0000","last_active":"2024-09-10T09:59:36.882314+0000","last_peered":"2024-09-10T09:59:36.882314+0000","last_clean":"2024-09-10T09:59:36.882314+0000","last_became_active":"2024-09-10T09:59:35.874752+0000","last_became_peered":"2024-09-10T09:59:35.874752+0000","last_unstale":"2024-09-10T09:59:36.882314+0000","last_undegraded":"2024-09-10T09:59:36.882314+0000","last_fullsized":"2024-09-10T09:59:36.882314+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:04.018787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.882624+0000","last_change":"2024-09-10T09:59:34.852867+0000","last_active":"2024-09-10T09:59:35.882624+0000","last_peered":"2024-09-10T09:59:35.882624+0000","last_clean":"2024-09-10T09:59:35.882624+0000","last_became_active":"2024-09-10T09:59:34.852590+0000","last_became_peered":"2024-09-10T09:59:34.852590+0000","last_unstale":"2024-09-10T09:59:35.882624+0000","last_undegraded":"2024-09-10T09:59:35.882624+0000","last_fullsized":"2024-09-10T09:59:35.882624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:12.173456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972093+0000","last_change":"2024-09-10T09:59:35.871042+0000","last_active":"2024-09-10T09:59:36.972093+0000","last_peered":"2024-09-10T09:59:36.972093+0000","last_clean":"2024-09-10T09:59:36.972093+0000","last_became_active":"2024-09-10T09:59:35.870773+0000","last_became_peered":"2024-09-10T09:59:35.870773+0000","last_unstale":"2024-09-10T09:59:36.972093+0000","last_undegraded":"2024-09-10T09:59:36.972093+0000","last_fullsized":"2024-09-10T09:59:36.972093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:56.488732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882542+0000","last_change":"2024-09-10T09:59:34.850529+0000","last_active":"2024-09-10T09:59:36.882542+0000","last_peered":"2024-09-10T09:59:36.882542+0000","last_clean":"2024-09-10T09:59:36.882542+0000","last_became_active":"2024-09-10T09:59:34.850253+0000","last_became_peered":"2024-09-10T09:59:34.850253+0000","last_unstale":"2024-09-10T09:59:36.882542+0000","last_undegraded":"2024-09-10T09:59:36.882542+0000","last_fullsized":"2024-09-10T09:59:36.882542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:30.925120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971627+0000","last_change":"2024-09-10T09:59:35.871988+0000","last_active":"2024-09-10T09:59:36.971627+0000","last_peered":"2024-09-10T09:59:36.971627+0000","last_clean":"2024-09-10T09:59:36.971627+0000","last_became_active":"2024-09-10T09:59:35.866803+0000","last_became_peered":"2024-09-10T09:59:35.866803+0000","last_unstale":"2024-09-10T09:59:36.971627+0000","last_undegraded":"2024-09-10T09:59:36.971627+0000","last_fullsized":"2024-09-10T09:59:36.971627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:04.752794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972297+0000","last_change":"2024-09-10T09:59:34.847706+0000","last_active":"2024-09-10T09:59:36.972297+0000","last_peered":"2024-09-10T09:59:36.972297+0000","last_clean":"2024-09-10T09:59:36.972297+0000","last_became_active":"2024-09-10T09:59:34.847399+0000","last_became_peered":"2024-09-10T09:59:34.847399+0000","last_unstale":"2024-09-10T09:59:36.972297+0000","last_undegraded":"2024-09-10T09:59:36.972297+0000","last_fullsized":"2024-09-10T09:59:36.972297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:21.249667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850310+0000","last_change":"2024-09-10T09:59:35.869630+0000","last_active":"2024-09-10T09:59:36.850310+0000","last_peered":"2024-09-10T09:59:36.850310+0000","last_clean":"2024-09-10T09:59:36.850310+0000","last_became_active":"2024-09-10T09:59:35.869278+0000","last_became_peered":"2024-09-10T09:59:35.869278+0000","last_unstale":"2024-09-10T09:59:36.850310+0000","last_undegraded":"2024-09-10T09:59:36.850310+0000","last_fullsized":"2024-09-10T09:59:36.850310+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:04.018787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971721+0000","last_change":"2024-09-10T09:59:34.858338+0000","last_active":"2024-09-10T09:59:36.971721+0000","last_peered":"2024-09-10T09:59:36.971721+0000","last_clean":"2024-09-10T09:59:36.971721+0000","last_became_active":"2024-09-10T09:59:34.858063+0000","last_became_peered":"2024-09-10T09:59:34.858063+0000","last_unstale":"2024-09-10T09:59:36.971721+0000","last_undegraded":"2024-09-10T09:59:36.971721+0000","last_fullsized":"2024-09-10T09:59:36.971721+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:24.936010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882627+0000","last_change":"2024-09-10T09:59:35.861660+0000","last_active":"2024-09-10T09:59:36.882627+0000","last_peered":"2024-09-10T09:59:36.882627+0000","last_clean":"2024-09-10T09:59:36.882627+0000","last_became_active":"2024-09-10T09:59:35.861189+0000","last_became_peered":"2024-09-10T09:59:35.861189+0000","last_unstale":"2024-09-10T09:59:36.882627+0000","last_undegraded":"2024-09-10T09:59:36.882627+0000","last_fullsized":"2024-09-10T09:59:36.882627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:44.448528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882632+0000","last_change":"2024-09-10T09:59:34.842965+0000","last_active":"2024-09-10T09:59:36.882632+0000","last_peered":"2024-09-10T09:59:36.882632+0000","last_clean":"2024-09-10T09:59:36.882632+0000","last_became_active":"2024-09-10T09:59:34.842704+0000","last_became_peered":"2024-09-10T09:59:34.842704+0000","last_unstale":"2024-09-10T09:59:36.882632+0000","last_undegraded":"2024-09-10T09:59:36.882632+0000","last_fullsized":"2024-09-10T09:59:36.882632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:07.243729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849930+0000","last_change":"2024-09-10T09:59:35.876201+0000","last_active":"2024-09-10T09:59:36.849930+0000","last_peered":"2024-09-10T09:59:36.849930+0000","last_clean":"2024-09-10T09:59:36.849930+0000","last_became_active":"2024-09-10T09:59:35.875909+0000","last_became_peered":"2024-09-10T09:59:35.875909+0000","last_unstale":"2024-09-10T09:59:36.849930+0000","last_undegraded":"2024-09-10T09:59:36.849930+0000","last_fullsized":"2024-09-10T09:59:36.849930+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:56.698002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972500+0000","last_change":"2024-09-10T09:59:34.857534+0000","last_active":"2024-09-10T09:59:36.972500+0000","last_peered":"2024-09-10T09:59:36.972500+0000","last_clean":"2024-09-10T09:59:36.972500+0000","last_became_active":"2024-09-10T09:59:34.857328+0000","last_became_peered":"2024-09-10T09:59:34.857328+0000","last_unstale":"2024-09-10T09:59:36.972500+0000","last_undegraded":"2024-09-10T09:59:36.972500+0000","last_fullsized":"2024-09-10T09:59:36.972500+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:06.597750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.859867+0000","last_change":"2024-09-10T09:59:35.855283+0000","last_active":"2024-09-10T09:59:35.859867+0000","last_peered":"2024-09-10T09:59:35.859867+0000","last_clean":"2024-09-10T09:59:35.859867+0000","last_became_active":"2024-09-10T09:59:35.851687+0000","last_became_peered":"2024-09-10T09:59:35.851687+0000","last_unstale":"2024-09-10T09:59:35.859867+0000","last_undegraded":"2024-09-10T09:59:35.859867+0000","last_fullsized":"2024-09-10T09:59:35.859867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849977+0000","last_change":"2024-09-10T09:59:34.852584+0000","last_active":"2024-09-10T09:59:36.849977+0000","last_peered":"2024-09-10T09:59:36.849977+0000","last_clean":"2024-09-10T09:59:36.849977+0000","last_became_active":"2024-09-10T09:59:34.852035+0000","last_became_peered":"2024-09-10T09:59:34.852035+0000","last_unstale":"2024-09-10T09:59:36.849977+0000","last_undegraded":"2024-09-10T09:59:36.849977+0000","last_fullsized":"2024-09-10T09:59:36.849977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:08.984505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850074+0000","last_change":"2024-09-10T09:59:35.868293+0000","last_active":"2024-09-10T09:59:36.850074+0000","last_peered":"2024-09-10T09:59:36.850074+0000","last_clean":"2024-09-10T09:59:36.850074+0000","last_became_active":"2024-09-10T09:59:35.867259+0000","last_became_peered":"2024-09-10T09:59:35.867259+0000","last_unstale":"2024-09-10T09:59:36.850074+0000","last_undegraded":"2024-09-10T09:59:36.850074+0000","last_fullsized":"2024-09-10T09:59:36.850074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:11.514360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845855+0000","last_change":"2024-09-10T09:59:34.847644+0000","last_active":"2024-09-10T09:59:36.845855+0000","last_peered":"2024-09-10T09:59:36.845855+0000","last_clean":"2024-09-10T09:59:36.845855+0000","last_became_active":"2024-09-10T09:59:34.847430+0000","last_became_peered":"2024-09-10T09:59:34.847430+0000","last_unstale":"2024-09-10T09:59:36.845855+0000","last_undegraded":"2024-09-10T09:59:36.845855+0000","last_fullsized":"2024-09-10T09:59:36.845855+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:10.100603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849660+0000","last_change":"2024-09-10T09:59:35.871745+0000","last_active":"2024-09-10T09:59:36.849660+0000","last_peered":"2024-09-10T09:59:36.849660+0000","last_clean":"2024-09-10T09:59:36.849660+0000","last_became_active":"2024-09-10T09:59:35.871406+0000","last_became_peered":"2024-09-10T09:59:35.871406+0000","last_unstale":"2024-09-10T09:59:36.849660+0000","last_undegraded":"2024-09-10T09:59:36.849660+0000","last_fullsized":"2024-09-10T09:59:36.849660+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:07.603507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.885731+0000","last_change":"2024-09-10T09:59:34.860564+0000","last_active":"2024-09-10T09:59:35.885731+0000","last_peered":"2024-09-10T09:59:35.885731+0000","last_clean":"2024-09-10T09:59:35.885731+0000","last_became_active":"2024-09-10T09:59:34.860265+0000","last_became_peered":"2024-09-10T09:59:34.860265+0000","last_unstale":"2024-09-10T09:59:35.885731+0000","last_undegraded":"2024-09-10T09:59:35.885731+0000","last_fullsized":"2024-09-10T09:59:35.885731+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:49.424166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972130+0000","last_change":"2024-09-10T09:59:35.860785+0000","last_active":"2024-09-10T09:59:36.972130+0000","last_peered":"2024-09-10T09:59:36.972130+0000","last_clean":"2024-09-10T09:59:36.972130+0000","last_became_active":"2024-09-10T09:59:35.860316+0000","last_became_peered":"2024-09-10T09:59:35.860316+0000","last_unstale":"2024-09-10T09:59:36.972130+0000","last_undegraded":"2024-09-10T09:59:36.972130+0000","last_fullsized":"2024-09-10T09:59:36.972130+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:48.114927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849146+0000","last_change":"2024-09-10T09:59:34.853615+0000","last_active":"2024-09-10T09:59:36.849146+0000","last_peered":"2024-09-10T09:59:36.849146+0000","last_clean":"2024-09-10T09:59:36.849146+0000","last_became_active":"2024-09-10T09:59:34.853350+0000","last_became_peered":"2024-09-10T09:59:34.853350+0000","last_unstale":"2024-09-10T09:59:36.849146+0000","last_undegraded":"2024-09-10T09:59:36.849146+0000","last_fullsized":"2024-09-10T09:59:36.849146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:36.753045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.856443+0000","last_change":"2024-09-10T09:59:35.851051+0000","last_active":"2024-09-10T09:59:35.856443+0000","last_peered":"2024-09-10T09:59:35.856443+0000","last_clean":"2024-09-10T09:59:35.856443+0000","last_became_active":"2024-09-10T09:59:35.850794+0000","last_became_peered":"2024-09-10T09:59:35.850794+0000","last_unstale":"2024-09-10T09:59:35.856443+0000","last_undegraded":"2024-09-10T09:59:35.856443+0000","last_fullsized":"2024-09-10T09:59:35.856443+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:36:20.488994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.853885+0000","last_change":"2024-09-10T09:59:34.860525+0000","last_active":"2024-09-10T09:59:35.853885+0000","last_peered":"2024-09-10T09:59:35.853885+0000","last_clean":"2024-09-10T09:59:35.853885+0000","last_became_active":"2024-09-10T09:59:34.860201+0000","last_became_peered":"2024-09-10T09:59:34.860201+0000","last_unstale":"2024-09-10T09:59:35.853885+0000","last_undegraded":"2024-09-10T09:59:35.853885+0000","last_fullsized":"2024-09-10T09:59:35.853885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:36.130009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849885+0000","last_change":"2024-09-10T09:59:35.879632+0000","last_active":"2024-09-10T09:59:36.849885+0000","last_peered":"2024-09-10T09:59:36.849885+0000","last_clean":"2024-09-10T09:59:36.849885+0000","last_became_active":"2024-09-10T09:59:35.879271+0000","last_became_peered":"2024-09-10T09:59:35.879271+0000","last_unstale":"2024-09-10T09:59:36.849885+0000","last_undegraded":"2024-09-10T09:59:36.849885+0000","last_fullsized":"2024-09-10T09:59:36.849885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:01.266810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971782+0000","last_change":"2024-09-10T09:59:34.858632+0000","last_active":"2024-09-10T09:59:36.971782+0000","last_peered":"2024-09-10T09:59:36.971782+0000","last_clean":"2024-09-10T09:59:36.971782+0000","last_became_active":"2024-09-10T09:59:34.858404+0000","last_became_peered":"2024-09-10T09:59:34.858404+0000","last_unstale":"2024-09-10T09:59:36.971782+0000","last_undegraded":"2024-09-10T09:59:36.971782+0000","last_fullsized":"2024-09-10T09:59:36.971782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:42.967896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845511+0000","last_change":"2024-09-10T09:59:35.862497+0000","last_active":"2024-09-10T09:59:36.845511+0000","last_peered":"2024-09-10T09:59:36.845511+0000","last_clean":"2024-09-10T09:59:36.845511+0000","last_became_active":"2024-09-10T09:59:35.862249+0000","last_became_peered":"2024-09-10T09:59:35.862249+0000","last_unstale":"2024-09-10T09:59:36.845511+0000","last_undegraded":"2024-09-10T09:59:36.845511+0000","last_fullsized":"2024-09-10T09:59:36.845511+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:57.138953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845638+0000","last_change":"2024-09-10T09:59:34.842045+0000","last_active":"2024-09-10T09:59:36.845638+0000","last_peered":"2024-09-10T09:59:36.845638+0000","last_clean":"2024-09-10T09:59:36.845638+0000","last_became_active":"2024-09-10T09:59:34.841784+0000","last_became_peered":"2024-09-10T09:59:34.841784+0000","last_unstale":"2024-09-10T09:59:36.845638+0000","last_undegraded":"2024-09-10T09:59:36.845638+0000","last_fullsized":"2024-09-10T09:59:36.845638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:21.426674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972409+0000","last_change":"2024-09-10T09:59:35.867689+0000","last_active":"2024-09-10T09:59:36.972409+0000","last_peered":"2024-09-10T09:59:36.972409+0000","last_clean":"2024-09-10T09:59:36.972409+0000","last_became_active":"2024-09-10T09:59:35.867344+0000","last_became_peered":"2024-09-10T09:59:35.867344+0000","last_unstale":"2024-09-10T09:59:36.972409+0000","last_undegraded":"2024-09-10T09:59:36.972409+0000","last_fullsized":"2024-09-10T09:59:36.972409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:06.176821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882819+0000","last_change":"2024-09-10T09:59:34.839165+0000","last_active":"2024-09-10T09:59:36.882819+0000","last_peered":"2024-09-10T09:59:36.882819+0000","last_clean":"2024-09-10T09:59:36.882819+0000","last_became_active":"2024-09-10T09:59:34.838801+0000","last_became_peered":"2024-09-10T09:59:34.838801+0000","last_unstale":"2024-09-10T09:59:36.882819+0000","last_undegraded":"2024-09-10T09:59:36.882819+0000","last_fullsized":"2024-09-10T09:59:36.882819+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:40.275130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850105+0000","last_change":"2024-09-10T09:59:35.872064+0000","last_active":"2024-09-10T09:59:36.850105+0000","last_peered":"2024-09-10T09:59:36.850105+0000","last_clean":"2024-09-10T09:59:36.850105+0000","last_became_active":"2024-09-10T09:59:35.871677+0000","last_became_peered":"2024-09-10T09:59:35.871677+0000","last_unstale":"2024-09-10T09:59:36.850105+0000","last_undegraded":"2024-09-10T09:59:36.850105+0000","last_fullsized":"2024-09-10T09:59:36.850105+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:30.781399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972409+0000","last_change":"2024-09-10T09:59:34.844856+0000","last_active":"2024-09-10T09:59:36.972409+0000","last_peered":"2024-09-10T09:59:36.972409+0000","last_clean":"2024-09-10T09:59:36.972409+0000","last_became_active":"2024-09-10T09:59:34.844565+0000","last_became_peered":"2024-09-10T09:59:34.844565+0000","last_unstale":"2024-09-10T09:59:36.972409+0000","last_undegraded":"2024-09-10T09:59:36.972409+0000","last_fullsized":"2024-09-10T09:59:36.972409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:00.472541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849923+0000","last_change":"2024-09-10T09:59:35.872607+0000","last_active":"2024-09-10T09:59:36.849923+0000","last_peered":"2024-09-10T09:59:36.849923+0000","last_clean":"2024-09-10T09:59:36.849923+0000","last_became_active":"2024-09-10T09:59:35.872236+0000","last_became_peered":"2024-09-10T09:59:35.872236+0000","last_unstale":"2024-09-10T09:59:36.849923+0000","last_undegraded":"2024-09-10T09:59:36.849923+0000","last_fullsized":"2024-09-10T09:59:36.849923+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:28.168031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.855037+0000","last_change":"2024-09-10T09:59:34.856202+0000","last_active":"2024-09-10T09:59:35.855037+0000","last_peered":"2024-09-10T09:59:35.855037+0000","last_clean":"2024-09-10T09:59:35.855037+0000","last_became_active":"2024-09-10T09:59:34.854659+0000","last_became_peered":"2024-09-10T09:59:34.854659+0000","last_unstale":"2024-09-10T09:59:35.855037+0000","last_undegraded":"2024-09-10T09:59:35.855037+0000","last_fullsized":"2024-09-10T09:59:35.855037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:16.311597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.854752+0000","last_change":"2024-09-10T09:59:35.852713+0000","last_active":"2024-09-10T09:59:35.854752+0000","last_peered":"2024-09-10T09:59:35.854752+0000","last_clean":"2024-09-10T09:59:35.854752+0000","last_became_active":"2024-09-10T09:59:35.852459+0000","last_became_peered":"2024-09-10T09:59:35.852459+0000","last_unstale":"2024-09-10T09:59:35.854752+0000","last_undegraded":"2024-09-10T09:59:35.854752+0000","last_fullsized":"2024-09-10T09:59:35.854752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:11.110896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.882660+0000","last_change":"2024-09-10T09:59:34.852783+0000","last_active":"2024-09-10T09:59:35.882660+0000","last_peered":"2024-09-10T09:59:35.882660+0000","last_clean":"2024-09-10T09:59:35.882660+0000","last_became_active":"2024-09-10T09:59:34.852565+0000","last_became_peered":"2024-09-10T09:59:34.852565+0000","last_unstale":"2024-09-10T09:59:35.882660+0000","last_undegraded":"2024-09-10T09:59:35.882660+0000","last_fullsized":"2024-09-10T09:59:35.882660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:48.099183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971448+0000","last_change":"2024-09-10T09:59:35.858605+0000","last_active":"2024-09-10T09:59:36.971448+0000","last_peered":"2024-09-10T09:59:36.971448+0000","last_clean":"2024-09-10T09:59:36.971448+0000","last_became_active":"2024-09-10T09:59:35.858297+0000","last_became_peered":"2024-09-10T09:59:35.858297+0000","last_unstale":"2024-09-10T09:59:36.971448+0000","last_undegraded":"2024-09-10T09:59:36.971448+0000","last_fullsized":"2024-09-10T09:59:36.971448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:17.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972174+0000","last_change":"2024-09-10T09:59:34.850629+0000","last_active":"2024-09-10T09:59:36.972174+0000","last_peered":"2024-09-10T09:59:36.972174+0000","last_clean":"2024-09-10T09:59:36.972174+0000","last_became_active":"2024-09-10T09:59:34.850364+0000","last_became_peered":"2024-09-10T09:59:34.850364+0000","last_unstale":"2024-09-10T09:59:36.972174+0000","last_undegraded":"2024-09-10T09:59:36.972174+0000","last_fullsized":"2024-09-10T09:59:36.972174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:29.775642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845442+0000","last_change":"2024-09-10T09:59:35.864933+0000","last_active":"2024-09-10T09:59:36.845442+0000","last_peered":"2024-09-10T09:59:36.845442+0000","last_clean":"2024-09-10T09:59:36.845442+0000","last_became_active":"2024-09-10T09:59:35.864355+0000","last_became_peered":"2024-09-10T09:59:35.864355+0000","last_unstale":"2024-09-10T09:59:36.845442+0000","last_undegraded":"2024-09-10T09:59:36.845442+0000","last_fullsized":"2024-09-10T09:59:36.845442+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:34.261115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971979+0000","last_change":"2024-09-10T09:59:34.858361+0000","last_active":"2024-09-10T09:59:36.971979+0000","last_peered":"2024-09-10T09:59:36.971979+0000","last_clean":"2024-09-10T09:59:36.971979+0000","last_became_active":"2024-09-10T09:59:34.858101+0000","last_became_peered":"2024-09-10T09:59:34.858101+0000","last_unstale":"2024-09-10T09:59:36.971979+0000","last_undegraded":"2024-09-10T09:59:36.971979+0000","last_fullsized":"2024-09-10T09:59:36.971979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:58.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971826+0000","last_change":"2024-09-10T09:59:35.858731+0000","last_active":"2024-09-10T09:59:36.971826+0000","last_peered":"2024-09-10T09:59:36.971826+0000","last_clean":"2024-09-10T09:59:36.971826+0000","last_became_active":"2024-09-10T09:59:35.858491+0000","last_became_peered":"2024-09-10T09:59:35.858491+0000","last_unstale":"2024-09-10T09:59:36.971826+0000","last_undegraded":"2024-09-10T09:59:36.971826+0000","last_fullsized":"2024-09-10T09:59:36.971826+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:09.990262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849117+0000","last_change":"2024-09-10T09:59:34.853830+0000","last_active":"2024-09-10T09:59:36.849117+0000","last_peered":"2024-09-10T09:59:36.849117+0000","last_clean":"2024-09-10T09:59:36.849117+0000","last_became_active":"2024-09-10T09:59:34.853095+0000","last_became_peered":"2024-09-10T09:59:34.853095+0000","last_unstale":"2024-09-10T09:59:36.849117+0000","last_undegraded":"2024-09-10T09:59:36.849117+0000","last_fullsized":"2024-09-10T09:59:36.849117+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850405+0000","last_change":"2024-09-10T09:59:35.872663+0000","last_active":"2024-09-10T09:59:36.850405+0000","last_peered":"2024-09-10T09:59:36.850405+0000","last_clean":"2024-09-10T09:59:36.850405+0000","last_became_active":"2024-09-10T09:59:35.871352+0000","last_became_peered":"2024-09-10T09:59:35.871352+0000","last_unstale":"2024-09-10T09:59:36.850405+0000","last_undegraded":"2024-09-10T09:59:36.850405+0000","last_fullsized":"2024-09-10T09:59:36.850405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:18.251454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.852029+0000","last_change":"2024-09-10T09:59:34.851432+0000","last_active":"2024-09-10T09:59:35.852029+0000","last_peered":"2024-09-10T09:59:35.852029+0000","last_clean":"2024-09-10T09:59:35.852029+0000","last_became_active":"2024-09-10T09:59:34.851159+0000","last_became_peered":"2024-09-10T09:59:34.851159+0000","last_unstale":"2024-09-10T09:59:35.852029+0000","last_undegraded":"2024-09-10T09:59:35.852029+0000","last_fullsized":"2024-09-10T09:59:35.852029+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:18.546980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972273+0000","last_change":"2024-09-10T09:59:35.858022+0000","last_active":"2024-09-10T09:59:36.972273+0000","last_peered":"2024-09-10T09:59:36.972273+0000","last_clean":"2024-09-10T09:59:36.972273+0000","last_became_active":"2024-09-10T09:59:35.857760+0000","last_became_peered":"2024-09-10T09:59:35.857760+0000","last_unstale":"2024-09-10T09:59:36.972273+0000","last_undegraded":"2024-09-10T09:59:36.972273+0000","last_fullsized":"2024-09-10T09:59:36.972273+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.150900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850082+0000","last_change":"2024-09-10T09:59:34.852727+0000","last_active":"2024-09-10T09:59:36.850082+0000","last_peered":"2024-09-10T09:59:36.850082+0000","last_clean":"2024-09-10T09:59:36.850082+0000","last_became_active":"2024-09-10T09:59:34.852404+0000","last_became_peered":"2024-09-10T09:59:34.852404+0000","last_unstale":"2024-09-10T09:59:36.850082+0000","last_undegraded":"2024-09-10T09:59:36.850082+0000","last_fullsized":"2024-09-10T09:59:36.850082+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:43.442772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849854+0000","last_change":"2024-09-10T09:59:35.868037+0000","last_active":"2024-09-10T09:59:36.849854+0000","last_peered":"2024-09-10T09:59:36.849854+0000","last_clean":"2024-09-10T09:59:36.849854+0000","last_became_active":"2024-09-10T09:59:35.867685+0000","last_became_peered":"2024-09-10T09:59:35.867685+0000","last_unstale":"2024-09-10T09:59:36.849854+0000","last_undegraded":"2024-09-10T09:59:36.849854+0000","last_fullsized":"2024-09-10T09:59:36.849854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:06.176821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971420+0000","last_change":"2024-09-10T09:59:34.843590+0000","last_active":"2024-09-10T09:59:36.971420+0000","last_peered":"2024-09-10T09:59:36.971420+0000","last_clean":"2024-09-10T09:59:36.971420+0000","last_became_active":"2024-09-10T09:59:34.843324+0000","last_became_peered":"2024-09-10T09:59:34.843324+0000","last_unstale":"2024-09-10T09:59:36.971420+0000","last_undegraded":"2024-09-10T09:59:36.971420+0000","last_fullsized":"2024-09-10T09:59:36.971420+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:58.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.874223+0000","last_change":"2024-09-10T09:59:35.869955+0000","last_active":"2024-09-10T09:59:35.874223+0000","last_peered":"2024-09-10T09:59:35.874223+0000","last_clean":"2024-09-10T09:59:35.874223+0000","last_became_active":"2024-09-10T09:59:35.869103+0000","last_became_peered":"2024-09-10T09:59:35.869103+0000","last_unstale":"2024-09-10T09:59:35.874223+0000","last_undegraded":"2024-09-10T09:59:35.874223+0000","last_fullsized":"2024-09-10T09:59:35.874223+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:30.276894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849671+0000","last_change":"2024-09-10T09:59:34.851811+0000","last_active":"2024-09-10T09:59:36.849671+0000","last_peered":"2024-09-10T09:59:36.849671+0000","last_clean":"2024-09-10T09:59:36.849671+0000","last_became_active":"2024-09-10T09:59:34.851406+0000","last_became_peered":"2024-09-10T09:59:34.851406+0000","last_unstale":"2024-09-10T09:59:36.849671+0000","last_undegraded":"2024-09-10T09:59:36.849671+0000","last_fullsized":"2024-09-10T09:59:36.849671+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:03.747037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.866411+0000","last_change":"2024-09-10T09:59:35.865294+0000","last_active":"2024-09-10T09:59:35.866411+0000","last_peered":"2024-09-10T09:59:35.866411+0000","last_clean":"2024-09-10T09:59:35.866411+0000","last_became_active":"2024-09-10T09:59:35.865015+0000","last_became_peered":"2024-09-10T09:59:35.865015+0000","last_unstale":"2024-09-10T09:59:35.866411+0000","last_undegraded":"2024-09-10T09:59:35.866411+0000","last_fullsized":"2024-09-10T09:59:35.866411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:16.906350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882501+0000","last_change":"2024-09-10T09:59:34.843607+0000","last_active":"2024-09-10T09:59:36.882501+0000","last_peered":"2024-09-10T09:59:36.882501+0000","last_clean":"2024-09-10T09:59:36.882501+0000","last_became_active":"2024-09-10T09:59:34.843335+0000","last_became_peered":"2024-09-10T09:59:34.843335+0000","last_unstale":"2024-09-10T09:59:36.882501+0000","last_undegraded":"2024-09-10T09:59:36.882501+0000","last_fullsized":"2024-09-10T09:59:36.882501+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:20.784501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972217+0000","last_change":"2024-09-10T09:59:35.862583+0000","last_active":"2024-09-10T09:59:36.972217+0000","last_peered":"2024-09-10T09:59:36.972217+0000","last_clean":"2024-09-10T09:59:36.972217+0000","last_became_active":"2024-09-10T09:59:35.862275+0000","last_became_peered":"2024-09-10T09:59:35.862275+0000","last_unstale":"2024-09-10T09:59:36.972217+0000","last_undegraded":"2024-09-10T09:59:36.972217+0000","last_fullsized":"2024-09-10T09:59:36.972217+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:56.488732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971395+0000","last_change":"2024-09-10T09:59:34.851351+0000","last_active":"2024-09-10T09:59:36.971395+0000","last_peered":"2024-09-10T09:59:36.971395+0000","last_clean":"2024-09-10T09:59:36.971395+0000","last_became_active":"2024-09-10T09:59:34.851070+0000","last_became_peered":"2024-09-10T09:59:34.851070+0000","last_unstale":"2024-09-10T09:59:36.971395+0000","last_undegraded":"2024-09-10T09:59:36.971395+0000","last_fullsized":"2024-09-10T09:59:36.971395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:20.784501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882364+0000","last_change":"2024-09-10T09:59:35.872635+0000","last_active":"2024-09-10T09:59:36.882364+0000","last_peered":"2024-09-10T09:59:36.882364+0000","last_clean":"2024-09-10T09:59:36.882364+0000","last_became_active":"2024-09-10T09:59:35.872274+0000","last_became_peered":"2024-09-10T09:59:35.872274+0000","last_unstale":"2024-09-10T09:59:36.882364+0000","last_undegraded":"2024-09-10T09:59:36.882364+0000","last_fullsized":"2024-09-10T09:59:36.882364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:37.758802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971797+0000","last_change":"2024-09-10T09:59:34.859646+0000","last_active":"2024-09-10T09:59:36.971797+0000","last_peered":"2024-09-10T09:59:36.971797+0000","last_clean":"2024-09-10T09:59:36.971797+0000","last_became_active":"2024-09-10T09:59:34.859344+0000","last_became_peered":"2024-09-10T09:59:34.859344+0000","last_unstale":"2024-09-10T09:59:36.971797+0000","last_undegraded":"2024-09-10T09:59:36.971797+0000","last_fullsized":"2024-09-10T09:59:36.971797+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:40.256438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972457+0000","last_change":"2024-09-10T09:59:35.868872+0000","last_active":"2024-09-10T09:59:36.972457+0000","last_peered":"2024-09-10T09:59:36.972457+0000","last_clean":"2024-09-10T09:59:36.972457+0000","last_became_active":"2024-09-10T09:59:35.868629+0000","last_became_peered":"2024-09-10T09:59:35.868629+0000","last_unstale":"2024-09-10T09:59:36.972457+0000","last_undegraded":"2024-09-10T09:59:36.972457+0000","last_fullsized":"2024-09-10T09:59:36.972457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:27:12.613140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972695+0000","last_change":"2024-09-10T09:59:34.855715+0000","last_active":"2024-09-10T09:59:36.972695+0000","last_peered":"2024-09-10T09:59:36.972695+0000","last_clean":"2024-09-10T09:59:36.972695+0000","last_became_active":"2024-09-10T09:59:34.855430+0000","last_became_peered":"2024-09-10T09:59:34.855430+0000","last_unstale":"2024-09-10T09:59:36.972695+0000","last_undegraded":"2024-09-10T09:59:36.972695+0000","last_fullsized":"2024-09-10T09:59:36.972695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:50.619585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":210,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849179+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:59:36.849179+0000","last_peered":"2024-09-10T09:59:36.849179+0000","last_clean":"2024-09-10T09:59:36.849179+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:59:36.849179+0000","last_undegraded":"2024-09-10T09:59:36.849179+0000","last_fullsized":"2024-09-10T09:59:36.849179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1118208,"data_stored":797797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":208011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":524288,"data_stored":207409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":532480,"data_stored":208951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":536576,"data_stored":208973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":524288,"data_stored":207409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":797741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":194264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:59:39.943 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:59:40.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.110+0000 7fe0dfc00640 1 Processor -- start 2024-09-10T09:59:40.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.110+0000 7fe0dfc00640 1 -- start start 2024-09-10T09:59:40.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.110+0000 7fe0dfc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe0d8155970 0x7fe0d80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.110+0000 7fe0dfc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d8155d50 con 0x7fe0d806f1e0 2024-09-10T09:59:40.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.110+0000 7fe0dfc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d80b34d0 con 0x7fe0d815c560 2024-09-10T09:59:40.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.110+0000 7fe0dfc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d80b4570 con 0x7fe0d8155970 2024-09-10T09:59:40.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0dd200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe0d8155970 0x7fe0d80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0dd200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe0d8155970 0x7fe0d80cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46864/0 (socket says 172.21.15.110:46864) 2024-09-10T09:59:40.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0dd200640 1 -- 172.21.15.110:0/3445042170 learned_addr learned my addr 172.21.15.110:0/3445042170 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:40.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0dd200640 1 -- 172.21.15.110:0/3445042170 >> v1:172.21.15.145:6789/0 conn(0x7fe0d815c560 legacy=0x7fe0d806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:40.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0dd200640 1 -- 172.21.15.110:0/3445042170 >> v1:172.21.15.110:6789/0 conn(0x7fe0d806f1e0 legacy=0x7fe0d80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0dd200640 1 -- 172.21.15.110:0/3445042170 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d80b37b0 con 0x7fe0d8155970 2024-09-10T09:59:40.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0dd200640 1 --2- 172.21.15.110:0/3445042170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d8155970 0x7fe0d80cc8c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009870 tx=0x7fe0c801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538f877e99d44260 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0d7400640 1 -- 172.21.15.110:0/3445042170 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80a95f0 con 0x7fe0d8155970 2024-09-10T09:59:40.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0d7400640 1 -- 172.21.15.110:0/3445042170 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c80a9790 con 0x7fe0d8155970 2024-09-10T09:59:40.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.114+0000 7fe0d7400640 1 -- 172.21.15.110:0/3445042170 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80a9b60 con 0x7fe0d8155970 2024-09-10T09:59:40.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3445042170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d8155970 msgr2=0x7fe0d80cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 --2- 172.21.15.110:0/3445042170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d8155970 0x7fe0d80cc8c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009870 tx=0x7fe0c801c530 comp rx=0 tx=0).stop 2024-09-10T09:59:40.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3445042170 shutdown_connections 2024-09-10T09:59:40.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 --2- 172.21.15.110:0/3445042170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d8155970 0x7fe0d80cc8c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3445042170 >> 172.21.15.110:0/3445042170 conn(0x7fe0d806bc50 msgr2=0x7fe0d806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:40.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3445042170 shutdown_connections 2024-09-10T09:59:40.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3445042170 wait complete. 2024-09-10T09:59:40.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 Processor -- start 2024-09-10T09:59:40.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- start start 2024-09-10T09:59:40.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d806f1e0 0x7fe0d8157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe0d8155970 0x7fe0d8159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d815c560 0x7fe0d814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80cf340 con 0x7fe0d806f1e0 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80cce00 con 0x7fe0d815c560 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dfc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d80cf4c0 con 0x7fe0d8155970 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d806f1e0 0x7fe0d8157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d806f1e0 0x7fe0d8157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46868/0 (socket says 172.21.15.110:46868) 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 -- 172.21.15.110:0/3278494338 learned_addr learned my addr 172.21.15.110:0/3278494338 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0ddc00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d815c560 0x7fe0d814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0d7e00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe0d8155970 0x7fe0d8159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 -- 172.21.15.110:0/3278494338 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe0d8155970 msgr2=0x7fe0d8159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe0d8155970 0x7fe0d8159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 -- 172.21.15.110:0/3278494338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d815c560 msgr2=0x7fe0d814deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d815c560 0x7fe0d814deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 -- 172.21.15.110:0/3278494338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d81576d0 con 0x7fe0d806f1e0 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.118+0000 7fe0dd200640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d806f1e0 0x7fe0d8157190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80ab040 tx=0x7fe0c801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0d4c00640 1 -- 172.21.15.110:0/3278494338 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80b1800 con 0x7fe0d806f1e0 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0d4c00640 1 -- 172.21.15.110:0/3278494338 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c80b19c0 con 0x7fe0d806f1e0 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d81560f0 con 0x7fe0d806f1e0 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0d4c00640 1 -- 172.21.15.110:0/3278494338 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80b1db0 con 0x7fe0d806f1e0 2024-09-10T09:59:40.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d81566b0 con 0x7fe0d806f1e0 2024-09-10T09:59:40.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d80cc8c0 con 0x7fe0d806f1e0 2024-09-10T09:59:40.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0d4c00640 1 -- 172.21.15.110:0/3278494338 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fe0c80c53f0 con 0x7fe0d806f1e0 2024-09-10T09:59:40.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0d4c00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe0b8067a70 0x7fe0b8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0d4c00640 1 -- 172.21.15.110:0/3278494338 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe0c8120c20 con 0x7fe0d806f1e0 2024-09-10T09:59:40.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0d7e00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe0b8067a70 0x7fe0b8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.122+0000 7fe0d7e00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe0b8067a70 0x7fe0b8069f30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe0d815d630 tx=0x7fe0c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.126+0000 7fe0d4c00640 1 -- 172.21.15.110:0/3278494338 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe0c80ad020 con 0x7fe0d806f1e0 2024-09-10T09:59:40.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.286+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0d80af690 con 0x7fe0b8067a70 2024-09-10T09:59:40.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.290+0000 7fe0d4c00640 1 -- 172.21.15.110:0/3278494338 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fe0d80af690 con 0x7fe0b8067a70 2024-09-10T09:59:40.298 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:40.299 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T09:59:40.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe0b8067a70 msgr2=0x7fe0b8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe0b8067a70 0x7fe0b8069f30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe0d815d630 tx=0x7fe0c4009040 comp rx=0 tx=0).stop 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d806f1e0 msgr2=0x7fe0d8157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d806f1e0 0x7fe0d8157190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80ab040 tx=0x7fe0c801ad70 comp rx=0 tx=0).stop 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 shutdown_connections 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe0b8067a70 0x7fe0b8069f30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0d815c560 0x7fe0d814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe0d8155970 0x7fe0d8159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 --2- 172.21.15.110:0/3278494338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0d806f1e0 0x7fe0d8157190 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 >> 172.21.15.110:0/3278494338 conn(0x7fe0d806bc50 msgr2=0x7fe0d806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 shutdown_connections 2024-09-10T09:59:40.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.294+0000 7fe0dfc00640 1 -- 172.21.15.110:0/3278494338 wait complete. 2024-09-10T09:59:40.319 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-10T09:59:39.370449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238144,"kb_used_data":5252,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736576,"statfs":{"total":773094113280,"available":772850253824,"internally_reserved":0,"allocated":5378048,"data_stored":2830555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":238276140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.543981"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850002+0000","last_change":"2024-09-10T09:59:35.876141+0000","last_active":"2024-09-10T09:59:36.850002+0000","last_peered":"2024-09-10T09:59:36.850002+0000","last_clean":"2024-09-10T09:59:36.850002+0000","last_became_active":"2024-09-10T09:59:35.875635+0000","last_became_peered":"2024-09-10T09:59:35.875635+0000","last_unstale":"2024-09-10T09:59:36.850002+0000","last_undegraded":"2024-09-10T09:59:36.850002+0000","last_fullsized":"2024-09-10T09:59:36.850002+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:27:12.613140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972549+0000","last_change":"2024-09-10T09:59:35.862505+0000","last_active":"2024-09-10T09:59:36.972549+0000","last_peered":"2024-09-10T09:59:36.972549+0000","last_clean":"2024-09-10T09:59:36.972549+0000","last_became_active":"2024-09-10T09:59:35.862216+0000","last_became_peered":"2024-09-10T09:59:35.862216+0000","last_unstale":"2024-09-10T09:59:36.972549+0000","last_undegraded":"2024-09-10T09:59:36.972549+0000","last_fullsized":"2024-09-10T09:59:36.972549+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.150900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882544+0000","last_change":"2024-09-10T09:59:35.859409+0000","last_active":"2024-09-10T09:59:36.882544+0000","last_peered":"2024-09-10T09:59:36.882544+0000","last_clean":"2024-09-10T09:59:36.882544+0000","last_became_active":"2024-09-10T09:59:35.859109+0000","last_became_peered":"2024-09-10T09:59:35.859109+0000","last_unstale":"2024-09-10T09:59:36.882544+0000","last_undegraded":"2024-09-10T09:59:36.882544+0000","last_fullsized":"2024-09-10T09:59:36.882544+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:04.752794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972206+0000","last_change":"2024-09-10T09:59:35.878310+0000","last_active":"2024-09-10T09:59:36.972206+0000","last_peered":"2024-09-10T09:59:36.972206+0000","last_clean":"2024-09-10T09:59:36.972206+0000","last_became_active":"2024-09-10T09:59:35.878020+0000","last_became_peered":"2024-09-10T09:59:35.878020+0000","last_unstale":"2024-09-10T09:59:36.972206+0000","last_undegraded":"2024-09-10T09:59:36.972206+0000","last_fullsized":"2024-09-10T09:59:36.972206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:43.863724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972413+0000","last_change":"2024-09-10T09:59:35.863783+0000","last_active":"2024-09-10T09:59:36.972413+0000","last_peered":"2024-09-10T09:59:36.972413+0000","last_clean":"2024-09-10T09:59:36.972413+0000","last_became_active":"2024-09-10T09:59:35.863444+0000","last_became_peered":"2024-09-10T09:59:35.863444+0000","last_unstale":"2024-09-10T09:59:36.972413+0000","last_undegraded":"2024-09-10T09:59:36.972413+0000","last_fullsized":"2024-09-10T09:59:36.972413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:00.415525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.870205+0000","last_change":"2024-09-10T09:59:35.860245+0000","last_active":"2024-09-10T09:59:35.870205+0000","last_peered":"2024-09-10T09:59:35.870205+0000","last_clean":"2024-09-10T09:59:35.870205+0000","last_became_active":"2024-09-10T09:59:35.859982+0000","last_became_peered":"2024-09-10T09:59:35.859982+0000","last_unstale":"2024-09-10T09:59:35.870205+0000","last_undegraded":"2024-09-10T09:59:35.870205+0000","last_fullsized":"2024-09-10T09:59:35.870205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:07.106814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850187+0000","last_change":"2024-09-10T09:59:35.867580+0000","last_active":"2024-09-10T09:59:36.850187+0000","last_peered":"2024-09-10T09:59:36.850187+0000","last_clean":"2024-09-10T09:59:36.850187+0000","last_became_active":"2024-09-10T09:59:35.867298+0000","last_became_peered":"2024-09-10T09:59:35.867298+0000","last_unstale":"2024-09-10T09:59:36.850187+0000","last_undegraded":"2024-09-10T09:59:36.850187+0000","last_fullsized":"2024-09-10T09:59:36.850187+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:21.294113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971551+0000","last_change":"2024-09-10T09:59:35.869302+0000","last_active":"2024-09-10T09:59:36.971551+0000","last_peered":"2024-09-10T09:59:36.971551+0000","last_clean":"2024-09-10T09:59:36.971551+0000","last_became_active":"2024-09-10T09:59:35.867049+0000","last_became_peered":"2024-09-10T09:59:35.867049+0000","last_unstale":"2024-09-10T09:59:36.971551+0000","last_undegraded":"2024-09-10T09:59:36.971551+0000","last_fullsized":"2024-09-10T09:59:36.971551+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:51.625342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972439+0000","last_change":"2024-09-10T09:59:35.865933+0000","last_active":"2024-09-10T09:59:36.972439+0000","last_peered":"2024-09-10T09:59:36.972439+0000","last_clean":"2024-09-10T09:59:36.972439+0000","last_became_active":"2024-09-10T09:59:35.865199+0000","last_became_peered":"2024-09-10T09:59:35.865199+0000","last_unstale":"2024-09-10T09:59:36.972439+0000","last_undegraded":"2024-09-10T09:59:36.972439+0000","last_fullsized":"2024-09-10T09:59:36.972439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:56.698002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972420+0000","last_change":"2024-09-10T09:59:35.863397+0000","last_active":"2024-09-10T09:59:36.972420+0000","last_peered":"2024-09-10T09:59:36.972420+0000","last_clean":"2024-09-10T09:59:36.972420+0000","last_became_active":"2024-09-10T09:59:35.863049+0000","last_became_peered":"2024-09-10T09:59:35.863049+0000","last_unstale":"2024-09-10T09:59:36.972420+0000","last_undegraded":"2024-09-10T09:59:36.972420+0000","last_fullsized":"2024-09-10T09:59:36.972420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:01.266810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972290+0000","last_change":"2024-09-10T09:59:35.866372+0000","last_active":"2024-09-10T09:59:36.972290+0000","last_peered":"2024-09-10T09:59:36.972290+0000","last_clean":"2024-09-10T09:59:36.972290+0000","last_became_active":"2024-09-10T09:59:35.866140+0000","last_became_peered":"2024-09-10T09:59:35.866140+0000","last_unstale":"2024-09-10T09:59:36.972290+0000","last_undegraded":"2024-09-10T09:59:36.972290+0000","last_fullsized":"2024-09-10T09:59:36.972290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:47.550206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972422+0000","last_change":"2024-09-10T09:59:35.867821+0000","last_active":"2024-09-10T09:59:36.972422+0000","last_peered":"2024-09-10T09:59:36.972422+0000","last_clean":"2024-09-10T09:59:36.972422+0000","last_became_active":"2024-09-10T09:59:35.867530+0000","last_became_peered":"2024-09-10T09:59:35.867530+0000","last_unstale":"2024-09-10T09:59:36.972422+0000","last_undegraded":"2024-09-10T09:59:36.972422+0000","last_fullsized":"2024-09-10T09:59:36.972422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:48.114927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845787+0000","last_change":"2024-09-10T09:59:35.856914+0000","last_active":"2024-09-10T09:59:36.845787+0000","last_peered":"2024-09-10T09:59:36.845787+0000","last_clean":"2024-09-10T09:59:36.845787+0000","last_became_active":"2024-09-10T09:59:35.856671+0000","last_became_peered":"2024-09-10T09:59:35.856671+0000","last_unstale":"2024-09-10T09:59:36.845787+0000","last_undegraded":"2024-09-10T09:59:36.845787+0000","last_fullsized":"2024-09-10T09:59:36.845787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:43.973652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972071+0000","last_change":"2024-09-10T09:59:35.860679+0000","last_active":"2024-09-10T09:59:36.972071+0000","last_peered":"2024-09-10T09:59:36.972071+0000","last_clean":"2024-09-10T09:59:36.972071+0000","last_became_active":"2024-09-10T09:59:35.860358+0000","last_became_peered":"2024-09-10T09:59:35.860358+0000","last_unstale":"2024-09-10T09:59:36.972071+0000","last_undegraded":"2024-09-10T09:59:36.972071+0000","last_fullsized":"2024-09-10T09:59:36.972071+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:44.448528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.868445+0000","last_change":"2024-09-10T09:59:35.867073+0000","last_active":"2024-09-10T09:59:35.868445+0000","last_peered":"2024-09-10T09:59:35.868445+0000","last_clean":"2024-09-10T09:59:35.868445+0000","last_became_active":"2024-09-10T09:59:35.866800+0000","last_became_peered":"2024-09-10T09:59:35.866800+0000","last_unstale":"2024-09-10T09:59:35.868445+0000","last_undegraded":"2024-09-10T09:59:35.868445+0000","last_fullsized":"2024-09-10T09:59:35.868445+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:10:19.862442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849034+0000","last_change":"2024-09-10T09:59:35.863102+0000","last_active":"2024-09-10T09:59:36.849034+0000","last_peered":"2024-09-10T09:59:36.849034+0000","last_clean":"2024-09-10T09:59:36.849034+0000","last_became_active":"2024-09-10T09:59:35.862642+0000","last_became_peered":"2024-09-10T09:59:35.862642+0000","last_unstale":"2024-09-10T09:59:36.849034+0000","last_undegraded":"2024-09-10T09:59:36.849034+0000","last_fullsized":"2024-09-10T09:59:36.849034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:41.280886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882613+0000","last_change":"2024-09-10T09:59:35.867101+0000","last_active":"2024-09-10T09:59:36.882613+0000","last_peered":"2024-09-10T09:59:36.882613+0000","last_clean":"2024-09-10T09:59:36.882613+0000","last_became_active":"2024-09-10T09:59:35.866416+0000","last_became_peered":"2024-09-10T09:59:35.866416+0000","last_unstale":"2024-09-10T09:59:36.882613+0000","last_undegraded":"2024-09-10T09:59:36.882613+0000","last_fullsized":"2024-09-10T09:59:36.882613+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:09.990262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882775+0000","last_change":"2024-09-10T09:59:35.861444+0000","last_active":"2024-09-10T09:59:36.882775+0000","last_peered":"2024-09-10T09:59:36.882775+0000","last_clean":"2024-09-10T09:59:36.882775+0000","last_became_active":"2024-09-10T09:59:35.860859+0000","last_became_peered":"2024-09-10T09:59:35.860859+0000","last_unstale":"2024-09-10T09:59:36.882775+0000","last_undegraded":"2024-09-10T09:59:36.882775+0000","last_fullsized":"2024-09-10T09:59:36.882775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:56.488732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850140+0000","last_change":"2024-09-10T09:59:35.869726+0000","last_active":"2024-09-10T09:59:36.850140+0000","last_peered":"2024-09-10T09:59:36.850140+0000","last_clean":"2024-09-10T09:59:36.850140+0000","last_became_active":"2024-09-10T09:59:35.869441+0000","last_became_peered":"2024-09-10T09:59:35.869441+0000","last_unstale":"2024-09-10T09:59:36.850140+0000","last_undegraded":"2024-09-10T09:59:36.850140+0000","last_fullsized":"2024-09-10T09:59:36.850140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:22.255424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845537+0000","last_change":"2024-09-10T09:59:35.866032+0000","last_active":"2024-09-10T09:59:36.845537+0000","last_peered":"2024-09-10T09:59:36.845537+0000","last_clean":"2024-09-10T09:59:36.845537+0000","last_became_active":"2024-09-10T09:59:35.865773+0000","last_became_peered":"2024-09-10T09:59:35.865773+0000","last_unstale":"2024-09-10T09:59:36.845537+0000","last_undegraded":"2024-09-10T09:59:36.845537+0000","last_fullsized":"2024-09-10T09:59:36.845537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.941766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.868879+0000","last_change":"2024-09-10T09:59:35.867639+0000","last_active":"2024-09-10T09:59:35.868879+0000","last_peered":"2024-09-10T09:59:35.868879+0000","last_clean":"2024-09-10T09:59:35.868879+0000","last_became_active":"2024-09-10T09:59:35.867125+0000","last_became_peered":"2024-09-10T09:59:35.867125+0000","last_unstale":"2024-09-10T09:59:35.868879+0000","last_undegraded":"2024-09-10T09:59:35.868879+0000","last_fullsized":"2024-09-10T09:59:35.868879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:31.314231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972131+0000","last_change":"2024-09-10T09:59:35.874448+0000","last_active":"2024-09-10T09:59:36.972131+0000","last_peered":"2024-09-10T09:59:36.972131+0000","last_clean":"2024-09-10T09:59:36.972131+0000","last_became_active":"2024-09-10T09:59:35.874179+0000","last_became_peered":"2024-09-10T09:59:35.874179+0000","last_unstale":"2024-09-10T09:59:36.972131+0000","last_undegraded":"2024-09-10T09:59:36.972131+0000","last_fullsized":"2024-09-10T09:59:36.972131+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:00.415525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849433+0000","last_change":"2024-09-10T09:59:35.872953+0000","last_active":"2024-09-10T09:59:36.849433+0000","last_peered":"2024-09-10T09:59:36.849433+0000","last_clean":"2024-09-10T09:59:36.849433+0000","last_became_active":"2024-09-10T09:59:35.872695+0000","last_became_peered":"2024-09-10T09:59:35.872695+0000","last_unstale":"2024-09-10T09:59:36.849433+0000","last_undegraded":"2024-09-10T09:59:36.849433+0000","last_fullsized":"2024-09-10T09:59:36.849433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:00.472914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849202+0000","last_change":"2024-09-10T09:59:35.869862+0000","last_active":"2024-09-10T09:59:36.849202+0000","last_peered":"2024-09-10T09:59:36.849202+0000","last_clean":"2024-09-10T09:59:36.849202+0000","last_became_active":"2024-09-10T09:59:35.869239+0000","last_became_peered":"2024-09-10T09:59:35.869239+0000","last_unstale":"2024-09-10T09:59:36.849202+0000","last_undegraded":"2024-09-10T09:59:36.849202+0000","last_fullsized":"2024-09-10T09:59:36.849202+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:08.249486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849409+0000","last_change":"2024-09-10T09:59:35.859953+0000","last_active":"2024-09-10T09:59:36.849409+0000","last_peered":"2024-09-10T09:59:36.849409+0000","last_clean":"2024-09-10T09:59:36.849409+0000","last_became_active":"2024-09-10T09:59:35.859464+0000","last_became_peered":"2024-09-10T09:59:35.859464+0000","last_unstale":"2024-09-10T09:59:36.849409+0000","last_undegraded":"2024-09-10T09:59:36.849409+0000","last_fullsized":"2024-09-10T09:59:36.849409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:17.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845353+0000","last_change":"2024-09-10T09:59:35.853723+0000","last_active":"2024-09-10T09:59:36.845353+0000","last_peered":"2024-09-10T09:59:36.845353+0000","last_clean":"2024-09-10T09:59:36.845353+0000","last_became_active":"2024-09-10T09:59:35.853472+0000","last_became_peered":"2024-09-10T09:59:35.853472+0000","last_unstale":"2024-09-10T09:59:36.845353+0000","last_undegraded":"2024-09-10T09:59:36.845353+0000","last_fullsized":"2024-09-10T09:59:36.845353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:44.561251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882456+0000","last_change":"2024-09-10T09:59:35.874083+0000","last_active":"2024-09-10T09:59:36.882456+0000","last_peered":"2024-09-10T09:59:36.882456+0000","last_clean":"2024-09-10T09:59:36.882456+0000","last_became_active":"2024-09-10T09:59:35.873850+0000","last_became_peered":"2024-09-10T09:59:35.873850+0000","last_unstale":"2024-09-10T09:59:36.882456+0000","last_undegraded":"2024-09-10T09:59:36.882456+0000","last_fullsized":"2024-09-10T09:59:36.882456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:51.625342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972418+0000","last_change":"2024-09-10T09:59:35.867957+0000","last_active":"2024-09-10T09:59:36.972418+0000","last_peered":"2024-09-10T09:59:36.972418+0000","last_clean":"2024-09-10T09:59:36.972418+0000","last_became_active":"2024-09-10T09:59:35.867605+0000","last_became_peered":"2024-09-10T09:59:35.867605+0000","last_unstale":"2024-09-10T09:59:36.972418+0000","last_undegraded":"2024-09-10T09:59:36.972418+0000","last_fullsized":"2024-09-10T09:59:36.972418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:43.863724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849736+0000","last_change":"2024-09-10T09:59:35.860589+0000","last_active":"2024-09-10T09:59:36.849736+0000","last_peered":"2024-09-10T09:59:36.849736+0000","last_clean":"2024-09-10T09:59:36.849736+0000","last_became_active":"2024-09-10T09:59:35.860322+0000","last_became_peered":"2024-09-10T09:59:35.860322+0000","last_unstale":"2024-09-10T09:59:36.849736+0000","last_undegraded":"2024-09-10T09:59:36.849736+0000","last_fullsized":"2024-09-10T09:59:36.849736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:58.654843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882321+0000","last_change":"2024-09-10T09:59:35.866732+0000","last_active":"2024-09-10T09:59:36.882321+0000","last_peered":"2024-09-10T09:59:36.882321+0000","last_clean":"2024-09-10T09:59:36.882321+0000","last_became_active":"2024-09-10T09:59:35.866451+0000","last_became_peered":"2024-09-10T09:59:35.866451+0000","last_unstale":"2024-09-10T09:59:36.882321+0000","last_undegraded":"2024-09-10T09:59:36.882321+0000","last_fullsized":"2024-09-10T09:59:36.882321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:18.251454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.874305+0000","last_change":"2024-09-10T09:59:35.871652+0000","last_active":"2024-09-10T09:59:35.874305+0000","last_peered":"2024-09-10T09:59:35.874305+0000","last_clean":"2024-09-10T09:59:35.874305+0000","last_became_active":"2024-09-10T09:59:35.871394+0000","last_became_peered":"2024-09-10T09:59:35.871394+0000","last_unstale":"2024-09-10T09:59:35.874305+0000","last_undegraded":"2024-09-10T09:59:35.874305+0000","last_fullsized":"2024-09-10T09:59:35.874305+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:15.916085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845299+0000","last_change":"2024-09-10T09:59:35.848708+0000","last_active":"2024-09-10T09:59:36.845299+0000","last_peered":"2024-09-10T09:59:36.845299+0000","last_clean":"2024-09-10T09:59:36.845299+0000","last_became_active":"2024-09-10T09:59:35.848440+0000","last_became_peered":"2024-09-10T09:59:35.848440+0000","last_unstale":"2024-09-10T09:59:36.845299+0000","last_undegraded":"2024-09-10T09:59:36.845299+0000","last_fullsized":"2024-09-10T09:59:36.845299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:05:22.200828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971601+0000","last_change":"2024-09-10T09:59:35.852650+0000","last_active":"2024-09-10T09:59:36.971601+0000","last_peered":"2024-09-10T09:59:36.971601+0000","last_clean":"2024-09-10T09:59:36.971601+0000","last_became_active":"2024-09-10T09:59:35.852363+0000","last_became_peered":"2024-09-10T09:59:35.852363+0000","last_unstale":"2024-09-10T09:59:36.971601+0000","last_undegraded":"2024-09-10T09:59:36.971601+0000","last_fullsized":"2024-09-10T09:59:36.971601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:00.472914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.886471+0000","last_change":"2024-09-10T09:59:34.855180+0000","last_active":"2024-09-10T09:59:35.886471+0000","last_peered":"2024-09-10T09:59:35.886471+0000","last_clean":"2024-09-10T09:59:35.886471+0000","last_became_active":"2024-09-10T09:59:34.854770+0000","last_became_peered":"2024-09-10T09:59:34.854770+0000","last_unstale":"2024-09-10T09:59:35.886471+0000","last_undegraded":"2024-09-10T09:59:35.886471+0000","last_fullsized":"2024-09-10T09:59:35.886471+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:41.886061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850282+0000","last_change":"2024-09-10T09:59:35.859718+0000","last_active":"2024-09-10T09:59:36.850282+0000","last_peered":"2024-09-10T09:59:36.850282+0000","last_clean":"2024-09-10T09:59:36.850282+0000","last_became_active":"2024-09-10T09:59:35.859108+0000","last_became_peered":"2024-09-10T09:59:35.859108+0000","last_unstale":"2024-09-10T09:59:36.850282+0000","last_undegraded":"2024-09-10T09:59:36.850282+0000","last_fullsized":"2024-09-10T09:59:36.850282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:01.478298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882726+0000","last_change":"2024-09-10T09:59:34.845608+0000","last_active":"2024-09-10T09:59:36.882726+0000","last_peered":"2024-09-10T09:59:36.882726+0000","last_clean":"2024-09-10T09:59:36.882726+0000","last_became_active":"2024-09-10T09:59:34.844174+0000","last_became_peered":"2024-09-10T09:59:34.844174+0000","last_unstale":"2024-09-10T09:59:36.882726+0000","last_undegraded":"2024-09-10T09:59:36.882726+0000","last_fullsized":"2024-09-10T09:59:36.882726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:58.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972578+0000","last_change":"2024-09-10T09:59:35.865285+0000","last_active":"2024-09-10T09:59:36.972578+0000","last_peered":"2024-09-10T09:59:36.972578+0000","last_clean":"2024-09-10T09:59:36.972578+0000","last_became_active":"2024-09-10T09:59:35.864955+0000","last_became_peered":"2024-09-10T09:59:35.864955+0000","last_unstale":"2024-09-10T09:59:36.972578+0000","last_undegraded":"2024-09-10T09:59:36.972578+0000","last_fullsized":"2024-09-10T09:59:36.972578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:47.550206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971843+0000","last_change":"2024-09-10T09:59:34.849471+0000","last_active":"2024-09-10T09:59:36.971843+0000","last_peered":"2024-09-10T09:59:36.971843+0000","last_clean":"2024-09-10T09:59:36.971843+0000","last_became_active":"2024-09-10T09:59:34.849154+0000","last_became_peered":"2024-09-10T09:59:34.849154+0000","last_unstale":"2024-09-10T09:59:36.971843+0000","last_undegraded":"2024-09-10T09:59:36.971843+0000","last_fullsized":"2024-09-10T09:59:36.971843+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:19:44.067649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971809+0000","last_change":"2024-09-10T09:59:35.862681+0000","last_active":"2024-09-10T09:59:36.971809+0000","last_peered":"2024-09-10T09:59:36.971809+0000","last_clean":"2024-09-10T09:59:36.971809+0000","last_became_active":"2024-09-10T09:59:35.861184+0000","last_became_peered":"2024-09-10T09:59:35.861184+0000","last_unstale":"2024-09-10T09:59:36.971809+0000","last_undegraded":"2024-09-10T09:59:36.971809+0000","last_fullsized":"2024-09-10T09:59:36.971809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:07.603507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849289+0000","last_change":"2024-09-10T09:59:34.851653+0000","last_active":"2024-09-10T09:59:36.849289+0000","last_peered":"2024-09-10T09:59:36.849289+0000","last_clean":"2024-09-10T09:59:36.849289+0000","last_became_active":"2024-09-10T09:59:34.851343+0000","last_became_peered":"2024-09-10T09:59:34.851343+0000","last_unstale":"2024-09-10T09:59:36.849289+0000","last_undegraded":"2024-09-10T09:59:36.849289+0000","last_fullsized":"2024-09-10T09:59:36.849289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:50.619585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882314+0000","last_change":"2024-09-10T09:59:35.875108+0000","last_active":"2024-09-10T09:59:36.882314+0000","last_peered":"2024-09-10T09:59:36.882314+0000","last_clean":"2024-09-10T09:59:36.882314+0000","last_became_active":"2024-09-10T09:59:35.874752+0000","last_became_peered":"2024-09-10T09:59:35.874752+0000","last_unstale":"2024-09-10T09:59:36.882314+0000","last_undegraded":"2024-09-10T09:59:36.882314+0000","last_fullsized":"2024-09-10T09:59:36.882314+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:04.018787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.882624+0000","last_change":"2024-09-10T09:59:34.852867+0000","last_active":"2024-09-10T09:59:35.882624+0000","last_peered":"2024-09-10T09:59:35.882624+0000","last_clean":"2024-09-10T09:59:35.882624+0000","last_became_active":"2024-09-10T09:59:34.852590+0000","last_became_peered":"2024-09-10T09:59:34.852590+0000","last_unstale":"2024-09-10T09:59:35.882624+0000","last_undegraded":"2024-09-10T09:59:35.882624+0000","last_fullsized":"2024-09-10T09:59:35.882624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:12.173456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972093+0000","last_change":"2024-09-10T09:59:35.871042+0000","last_active":"2024-09-10T09:59:36.972093+0000","last_peered":"2024-09-10T09:59:36.972093+0000","last_clean":"2024-09-10T09:59:36.972093+0000","last_became_active":"2024-09-10T09:59:35.870773+0000","last_became_peered":"2024-09-10T09:59:35.870773+0000","last_unstale":"2024-09-10T09:59:36.972093+0000","last_undegraded":"2024-09-10T09:59:36.972093+0000","last_fullsized":"2024-09-10T09:59:36.972093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:56.488732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882542+0000","last_change":"2024-09-10T09:59:34.850529+0000","last_active":"2024-09-10T09:59:36.882542+0000","last_peered":"2024-09-10T09:59:36.882542+0000","last_clean":"2024-09-10T09:59:36.882542+0000","last_became_active":"2024-09-10T09:59:34.850253+0000","last_became_peered":"2024-09-10T09:59:34.850253+0000","last_unstale":"2024-09-10T09:59:36.882542+0000","last_undegraded":"2024-09-10T09:59:36.882542+0000","last_fullsized":"2024-09-10T09:59:36.882542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:30.925120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971627+0000","last_change":"2024-09-10T09:59:35.871988+0000","last_active":"2024-09-10T09:59:36.971627+0000","last_peered":"2024-09-10T09:59:36.971627+0000","last_clean":"2024-09-10T09:59:36.971627+0000","last_became_active":"2024-09-10T09:59:35.866803+0000","last_became_peered":"2024-09-10T09:59:35.866803+0000","last_unstale":"2024-09-10T09:59:36.971627+0000","last_undegraded":"2024-09-10T09:59:36.971627+0000","last_fullsized":"2024-09-10T09:59:36.971627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:04.752794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972297+0000","last_change":"2024-09-10T09:59:34.847706+0000","last_active":"2024-09-10T09:59:36.972297+0000","last_peered":"2024-09-10T09:59:36.972297+0000","last_clean":"2024-09-10T09:59:36.972297+0000","last_became_active":"2024-09-10T09:59:34.847399+0000","last_became_peered":"2024-09-10T09:59:34.847399+0000","last_unstale":"2024-09-10T09:59:36.972297+0000","last_undegraded":"2024-09-10T09:59:36.972297+0000","last_fullsized":"2024-09-10T09:59:36.972297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:21.249667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850310+0000","last_change":"2024-09-10T09:59:35.869630+0000","last_active":"2024-09-10T09:59:36.850310+0000","last_peered":"2024-09-10T09:59:36.850310+0000","last_clean":"2024-09-10T09:59:36.850310+0000","last_became_active":"2024-09-10T09:59:35.869278+0000","last_became_peered":"2024-09-10T09:59:35.869278+0000","last_unstale":"2024-09-10T09:59:36.850310+0000","last_undegraded":"2024-09-10T09:59:36.850310+0000","last_fullsized":"2024-09-10T09:59:36.850310+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:04.018787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971721+0000","last_change":"2024-09-10T09:59:34.858338+0000","last_active":"2024-09-10T09:59:36.971721+0000","last_peered":"2024-09-10T09:59:36.971721+0000","last_clean":"2024-09-10T09:59:36.971721+0000","last_became_active":"2024-09-10T09:59:34.858063+0000","last_became_peered":"2024-09-10T09:59:34.858063+0000","last_unstale":"2024-09-10T09:59:36.971721+0000","last_undegraded":"2024-09-10T09:59:36.971721+0000","last_fullsized":"2024-09-10T09:59:36.971721+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:24.936010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882627+0000","last_change":"2024-09-10T09:59:35.861660+0000","last_active":"2024-09-10T09:59:36.882627+0000","last_peered":"2024-09-10T09:59:36.882627+0000","last_clean":"2024-09-10T09:59:36.882627+0000","last_became_active":"2024-09-10T09:59:35.861189+0000","last_became_peered":"2024-09-10T09:59:35.861189+0000","last_unstale":"2024-09-10T09:59:36.882627+0000","last_undegraded":"2024-09-10T09:59:36.882627+0000","last_fullsized":"2024-09-10T09:59:36.882627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:44.448528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882632+0000","last_change":"2024-09-10T09:59:34.842965+0000","last_active":"2024-09-10T09:59:36.882632+0000","last_peered":"2024-09-10T09:59:36.882632+0000","last_clean":"2024-09-10T09:59:36.882632+0000","last_became_active":"2024-09-10T09:59:34.842704+0000","last_became_peered":"2024-09-10T09:59:34.842704+0000","last_unstale":"2024-09-10T09:59:36.882632+0000","last_undegraded":"2024-09-10T09:59:36.882632+0000","last_fullsized":"2024-09-10T09:59:36.882632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:07.243729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849930+0000","last_change":"2024-09-10T09:59:35.876201+0000","last_active":"2024-09-10T09:59:36.849930+0000","last_peered":"2024-09-10T09:59:36.849930+0000","last_clean":"2024-09-10T09:59:36.849930+0000","last_became_active":"2024-09-10T09:59:35.875909+0000","last_became_peered":"2024-09-10T09:59:35.875909+0000","last_unstale":"2024-09-10T09:59:36.849930+0000","last_undegraded":"2024-09-10T09:59:36.849930+0000","last_fullsized":"2024-09-10T09:59:36.849930+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:56.698002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972500+0000","last_change":"2024-09-10T09:59:34.857534+0000","last_active":"2024-09-10T09:59:36.972500+0000","last_peered":"2024-09-10T09:59:36.972500+0000","last_clean":"2024-09-10T09:59:36.972500+0000","last_became_active":"2024-09-10T09:59:34.857328+0000","last_became_peered":"2024-09-10T09:59:34.857328+0000","last_unstale":"2024-09-10T09:59:36.972500+0000","last_undegraded":"2024-09-10T09:59:36.972500+0000","last_fullsized":"2024-09-10T09:59:36.972500+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:06.597750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.859867+0000","last_change":"2024-09-10T09:59:35.855283+0000","last_active":"2024-09-10T09:59:35.859867+0000","last_peered":"2024-09-10T09:59:35.859867+0000","last_clean":"2024-09-10T09:59:35.859867+0000","last_became_active":"2024-09-10T09:59:35.851687+0000","last_became_peered":"2024-09-10T09:59:35.851687+0000","last_unstale":"2024-09-10T09:59:35.859867+0000","last_undegraded":"2024-09-10T09:59:35.859867+0000","last_fullsized":"2024-09-10T09:59:35.859867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:25.390402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849977+0000","last_change":"2024-09-10T09:59:34.852584+0000","last_active":"2024-09-10T09:59:36.849977+0000","last_peered":"2024-09-10T09:59:36.849977+0000","last_clean":"2024-09-10T09:59:36.849977+0000","last_became_active":"2024-09-10T09:59:34.852035+0000","last_became_peered":"2024-09-10T09:59:34.852035+0000","last_unstale":"2024-09-10T09:59:36.849977+0000","last_undegraded":"2024-09-10T09:59:36.849977+0000","last_fullsized":"2024-09-10T09:59:36.849977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:08.984505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850074+0000","last_change":"2024-09-10T09:59:35.868293+0000","last_active":"2024-09-10T09:59:36.850074+0000","last_peered":"2024-09-10T09:59:36.850074+0000","last_clean":"2024-09-10T09:59:36.850074+0000","last_became_active":"2024-09-10T09:59:35.867259+0000","last_became_peered":"2024-09-10T09:59:35.867259+0000","last_unstale":"2024-09-10T09:59:36.850074+0000","last_undegraded":"2024-09-10T09:59:36.850074+0000","last_fullsized":"2024-09-10T09:59:36.850074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:11.514360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845855+0000","last_change":"2024-09-10T09:59:34.847644+0000","last_active":"2024-09-10T09:59:36.845855+0000","last_peered":"2024-09-10T09:59:36.845855+0000","last_clean":"2024-09-10T09:59:36.845855+0000","last_became_active":"2024-09-10T09:59:34.847430+0000","last_became_peered":"2024-09-10T09:59:34.847430+0000","last_unstale":"2024-09-10T09:59:36.845855+0000","last_undegraded":"2024-09-10T09:59:36.845855+0000","last_fullsized":"2024-09-10T09:59:36.845855+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:10.100603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849660+0000","last_change":"2024-09-10T09:59:35.871745+0000","last_active":"2024-09-10T09:59:36.849660+0000","last_peered":"2024-09-10T09:59:36.849660+0000","last_clean":"2024-09-10T09:59:36.849660+0000","last_became_active":"2024-09-10T09:59:35.871406+0000","last_became_peered":"2024-09-10T09:59:35.871406+0000","last_unstale":"2024-09-10T09:59:36.849660+0000","last_undegraded":"2024-09-10T09:59:36.849660+0000","last_fullsized":"2024-09-10T09:59:36.849660+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:07.603507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.885731+0000","last_change":"2024-09-10T09:59:34.860564+0000","last_active":"2024-09-10T09:59:35.885731+0000","last_peered":"2024-09-10T09:59:35.885731+0000","last_clean":"2024-09-10T09:59:35.885731+0000","last_became_active":"2024-09-10T09:59:34.860265+0000","last_became_peered":"2024-09-10T09:59:34.860265+0000","last_unstale":"2024-09-10T09:59:35.885731+0000","last_undegraded":"2024-09-10T09:59:35.885731+0000","last_fullsized":"2024-09-10T09:59:35.885731+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:49.424166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972130+0000","last_change":"2024-09-10T09:59:35.860785+0000","last_active":"2024-09-10T09:59:36.972130+0000","last_peered":"2024-09-10T09:59:36.972130+0000","last_clean":"2024-09-10T09:59:36.972130+0000","last_became_active":"2024-09-10T09:59:35.860316+0000","last_became_peered":"2024-09-10T09:59:35.860316+0000","last_unstale":"2024-09-10T09:59:36.972130+0000","last_undegraded":"2024-09-10T09:59:36.972130+0000","last_fullsized":"2024-09-10T09:59:36.972130+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:48.114927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849146+0000","last_change":"2024-09-10T09:59:34.853615+0000","last_active":"2024-09-10T09:59:36.849146+0000","last_peered":"2024-09-10T09:59:36.849146+0000","last_clean":"2024-09-10T09:59:36.849146+0000","last_became_active":"2024-09-10T09:59:34.853350+0000","last_became_peered":"2024-09-10T09:59:34.853350+0000","last_unstale":"2024-09-10T09:59:36.849146+0000","last_undegraded":"2024-09-10T09:59:36.849146+0000","last_fullsized":"2024-09-10T09:59:36.849146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:36.753045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.856443+0000","last_change":"2024-09-10T09:59:35.851051+0000","last_active":"2024-09-10T09:59:35.856443+0000","last_peered":"2024-09-10T09:59:35.856443+0000","last_clean":"2024-09-10T09:59:35.856443+0000","last_became_active":"2024-09-10T09:59:35.850794+0000","last_became_peered":"2024-09-10T09:59:35.850794+0000","last_unstale":"2024-09-10T09:59:35.856443+0000","last_undegraded":"2024-09-10T09:59:35.856443+0000","last_fullsized":"2024-09-10T09:59:35.856443+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:36:20.488994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.853885+0000","last_change":"2024-09-10T09:59:34.860525+0000","last_active":"2024-09-10T09:59:35.853885+0000","last_peered":"2024-09-10T09:59:35.853885+0000","last_clean":"2024-09-10T09:59:35.853885+0000","last_became_active":"2024-09-10T09:59:34.860201+0000","last_became_peered":"2024-09-10T09:59:34.860201+0000","last_unstale":"2024-09-10T09:59:35.853885+0000","last_undegraded":"2024-09-10T09:59:35.853885+0000","last_fullsized":"2024-09-10T09:59:35.853885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:36.130009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849885+0000","last_change":"2024-09-10T09:59:35.879632+0000","last_active":"2024-09-10T09:59:36.849885+0000","last_peered":"2024-09-10T09:59:36.849885+0000","last_clean":"2024-09-10T09:59:36.849885+0000","last_became_active":"2024-09-10T09:59:35.879271+0000","last_became_peered":"2024-09-10T09:59:35.879271+0000","last_unstale":"2024-09-10T09:59:36.849885+0000","last_undegraded":"2024-09-10T09:59:36.849885+0000","last_fullsized":"2024-09-10T09:59:36.849885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:01.266810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971782+0000","last_change":"2024-09-10T09:59:34.858632+0000","last_active":"2024-09-10T09:59:36.971782+0000","last_peered":"2024-09-10T09:59:36.971782+0000","last_clean":"2024-09-10T09:59:36.971782+0000","last_became_active":"2024-09-10T09:59:34.858404+0000","last_became_peered":"2024-09-10T09:59:34.858404+0000","last_unstale":"2024-09-10T09:59:36.971782+0000","last_undegraded":"2024-09-10T09:59:36.971782+0000","last_fullsized":"2024-09-10T09:59:36.971782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:42.967896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845511+0000","last_change":"2024-09-10T09:59:35.862497+0000","last_active":"2024-09-10T09:59:36.845511+0000","last_peered":"2024-09-10T09:59:36.845511+0000","last_clean":"2024-09-10T09:59:36.845511+0000","last_became_active":"2024-09-10T09:59:35.862249+0000","last_became_peered":"2024-09-10T09:59:35.862249+0000","last_unstale":"2024-09-10T09:59:36.845511+0000","last_undegraded":"2024-09-10T09:59:36.845511+0000","last_fullsized":"2024-09-10T09:59:36.845511+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:57.138953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845638+0000","last_change":"2024-09-10T09:59:34.842045+0000","last_active":"2024-09-10T09:59:36.845638+0000","last_peered":"2024-09-10T09:59:36.845638+0000","last_clean":"2024-09-10T09:59:36.845638+0000","last_became_active":"2024-09-10T09:59:34.841784+0000","last_became_peered":"2024-09-10T09:59:34.841784+0000","last_unstale":"2024-09-10T09:59:36.845638+0000","last_undegraded":"2024-09-10T09:59:36.845638+0000","last_fullsized":"2024-09-10T09:59:36.845638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:21.426674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972409+0000","last_change":"2024-09-10T09:59:35.867689+0000","last_active":"2024-09-10T09:59:36.972409+0000","last_peered":"2024-09-10T09:59:36.972409+0000","last_clean":"2024-09-10T09:59:36.972409+0000","last_became_active":"2024-09-10T09:59:35.867344+0000","last_became_peered":"2024-09-10T09:59:35.867344+0000","last_unstale":"2024-09-10T09:59:36.972409+0000","last_undegraded":"2024-09-10T09:59:36.972409+0000","last_fullsized":"2024-09-10T09:59:36.972409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:06.176821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882819+0000","last_change":"2024-09-10T09:59:34.839165+0000","last_active":"2024-09-10T09:59:36.882819+0000","last_peered":"2024-09-10T09:59:36.882819+0000","last_clean":"2024-09-10T09:59:36.882819+0000","last_became_active":"2024-09-10T09:59:34.838801+0000","last_became_peered":"2024-09-10T09:59:34.838801+0000","last_unstale":"2024-09-10T09:59:36.882819+0000","last_undegraded":"2024-09-10T09:59:36.882819+0000","last_fullsized":"2024-09-10T09:59:36.882819+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:40.275130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850105+0000","last_change":"2024-09-10T09:59:35.872064+0000","last_active":"2024-09-10T09:59:36.850105+0000","last_peered":"2024-09-10T09:59:36.850105+0000","last_clean":"2024-09-10T09:59:36.850105+0000","last_became_active":"2024-09-10T09:59:35.871677+0000","last_became_peered":"2024-09-10T09:59:35.871677+0000","last_unstale":"2024-09-10T09:59:36.850105+0000","last_undegraded":"2024-09-10T09:59:36.850105+0000","last_fullsized":"2024-09-10T09:59:36.850105+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:30.781399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972409+0000","last_change":"2024-09-10T09:59:34.844856+0000","last_active":"2024-09-10T09:59:36.972409+0000","last_peered":"2024-09-10T09:59:36.972409+0000","last_clean":"2024-09-10T09:59:36.972409+0000","last_became_active":"2024-09-10T09:59:34.844565+0000","last_became_peered":"2024-09-10T09:59:34.844565+0000","last_unstale":"2024-09-10T09:59:36.972409+0000","last_undegraded":"2024-09-10T09:59:36.972409+0000","last_fullsized":"2024-09-10T09:59:36.972409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:00.472541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849923+0000","last_change":"2024-09-10T09:59:35.872607+0000","last_active":"2024-09-10T09:59:36.849923+0000","last_peered":"2024-09-10T09:59:36.849923+0000","last_clean":"2024-09-10T09:59:36.849923+0000","last_became_active":"2024-09-10T09:59:35.872236+0000","last_became_peered":"2024-09-10T09:59:35.872236+0000","last_unstale":"2024-09-10T09:59:36.849923+0000","last_undegraded":"2024-09-10T09:59:36.849923+0000","last_fullsized":"2024-09-10T09:59:36.849923+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:28.168031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.855037+0000","last_change":"2024-09-10T09:59:34.856202+0000","last_active":"2024-09-10T09:59:35.855037+0000","last_peered":"2024-09-10T09:59:35.855037+0000","last_clean":"2024-09-10T09:59:35.855037+0000","last_became_active":"2024-09-10T09:59:34.854659+0000","last_became_peered":"2024-09-10T09:59:34.854659+0000","last_unstale":"2024-09-10T09:59:35.855037+0000","last_undegraded":"2024-09-10T09:59:35.855037+0000","last_fullsized":"2024-09-10T09:59:35.855037+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:16.311597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.854752+0000","last_change":"2024-09-10T09:59:35.852713+0000","last_active":"2024-09-10T09:59:35.854752+0000","last_peered":"2024-09-10T09:59:35.854752+0000","last_clean":"2024-09-10T09:59:35.854752+0000","last_became_active":"2024-09-10T09:59:35.852459+0000","last_became_peered":"2024-09-10T09:59:35.852459+0000","last_unstale":"2024-09-10T09:59:35.854752+0000","last_undegraded":"2024-09-10T09:59:35.854752+0000","last_fullsized":"2024-09-10T09:59:35.854752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:11.110896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.882660+0000","last_change":"2024-09-10T09:59:34.852783+0000","last_active":"2024-09-10T09:59:35.882660+0000","last_peered":"2024-09-10T09:59:35.882660+0000","last_clean":"2024-09-10T09:59:35.882660+0000","last_became_active":"2024-09-10T09:59:34.852565+0000","last_became_peered":"2024-09-10T09:59:34.852565+0000","last_unstale":"2024-09-10T09:59:35.882660+0000","last_undegraded":"2024-09-10T09:59:35.882660+0000","last_fullsized":"2024-09-10T09:59:35.882660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:48.099183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971448+0000","last_change":"2024-09-10T09:59:35.858605+0000","last_active":"2024-09-10T09:59:36.971448+0000","last_peered":"2024-09-10T09:59:36.971448+0000","last_clean":"2024-09-10T09:59:36.971448+0000","last_became_active":"2024-09-10T09:59:35.858297+0000","last_became_peered":"2024-09-10T09:59:35.858297+0000","last_unstale":"2024-09-10T09:59:36.971448+0000","last_undegraded":"2024-09-10T09:59:36.971448+0000","last_fullsized":"2024-09-10T09:59:36.971448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:17.113345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972174+0000","last_change":"2024-09-10T09:59:34.850629+0000","last_active":"2024-09-10T09:59:36.972174+0000","last_peered":"2024-09-10T09:59:36.972174+0000","last_clean":"2024-09-10T09:59:36.972174+0000","last_became_active":"2024-09-10T09:59:34.850364+0000","last_became_peered":"2024-09-10T09:59:34.850364+0000","last_unstale":"2024-09-10T09:59:36.972174+0000","last_undegraded":"2024-09-10T09:59:36.972174+0000","last_fullsized":"2024-09-10T09:59:36.972174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:29.775642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.845442+0000","last_change":"2024-09-10T09:59:35.864933+0000","last_active":"2024-09-10T09:59:36.845442+0000","last_peered":"2024-09-10T09:59:36.845442+0000","last_clean":"2024-09-10T09:59:36.845442+0000","last_became_active":"2024-09-10T09:59:35.864355+0000","last_became_peered":"2024-09-10T09:59:35.864355+0000","last_unstale":"2024-09-10T09:59:36.845442+0000","last_undegraded":"2024-09-10T09:59:36.845442+0000","last_fullsized":"2024-09-10T09:59:36.845442+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:34.261115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971979+0000","last_change":"2024-09-10T09:59:34.858361+0000","last_active":"2024-09-10T09:59:36.971979+0000","last_peered":"2024-09-10T09:59:36.971979+0000","last_clean":"2024-09-10T09:59:36.971979+0000","last_became_active":"2024-09-10T09:59:34.858101+0000","last_became_peered":"2024-09-10T09:59:34.858101+0000","last_unstale":"2024-09-10T09:59:36.971979+0000","last_undegraded":"2024-09-10T09:59:36.971979+0000","last_fullsized":"2024-09-10T09:59:36.971979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:58.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971826+0000","last_change":"2024-09-10T09:59:35.858731+0000","last_active":"2024-09-10T09:59:36.971826+0000","last_peered":"2024-09-10T09:59:36.971826+0000","last_clean":"2024-09-10T09:59:36.971826+0000","last_became_active":"2024-09-10T09:59:35.858491+0000","last_became_peered":"2024-09-10T09:59:35.858491+0000","last_unstale":"2024-09-10T09:59:36.971826+0000","last_undegraded":"2024-09-10T09:59:36.971826+0000","last_fullsized":"2024-09-10T09:59:36.971826+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:09.990262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849117+0000","last_change":"2024-09-10T09:59:34.853830+0000","last_active":"2024-09-10T09:59:36.849117+0000","last_peered":"2024-09-10T09:59:36.849117+0000","last_clean":"2024-09-10T09:59:36.849117+0000","last_became_active":"2024-09-10T09:59:34.853095+0000","last_became_peered":"2024-09-10T09:59:34.853095+0000","last_unstale":"2024-09-10T09:59:36.849117+0000","last_undegraded":"2024-09-10T09:59:36.849117+0000","last_fullsized":"2024-09-10T09:59:36.849117+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:17.245698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850405+0000","last_change":"2024-09-10T09:59:35.872663+0000","last_active":"2024-09-10T09:59:36.850405+0000","last_peered":"2024-09-10T09:59:36.850405+0000","last_clean":"2024-09-10T09:59:36.850405+0000","last_became_active":"2024-09-10T09:59:35.871352+0000","last_became_peered":"2024-09-10T09:59:35.871352+0000","last_unstale":"2024-09-10T09:59:36.850405+0000","last_undegraded":"2024-09-10T09:59:36.850405+0000","last_fullsized":"2024-09-10T09:59:36.850405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:18.251454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.852029+0000","last_change":"2024-09-10T09:59:34.851432+0000","last_active":"2024-09-10T09:59:35.852029+0000","last_peered":"2024-09-10T09:59:35.852029+0000","last_clean":"2024-09-10T09:59:35.852029+0000","last_became_active":"2024-09-10T09:59:34.851159+0000","last_became_peered":"2024-09-10T09:59:34.851159+0000","last_unstale":"2024-09-10T09:59:35.852029+0000","last_undegraded":"2024-09-10T09:59:35.852029+0000","last_fullsized":"2024-09-10T09:59:35.852029+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:18.546980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972273+0000","last_change":"2024-09-10T09:59:35.858022+0000","last_active":"2024-09-10T09:59:36.972273+0000","last_peered":"2024-09-10T09:59:36.972273+0000","last_clean":"2024-09-10T09:59:36.972273+0000","last_became_active":"2024-09-10T09:59:35.857760+0000","last_became_peered":"2024-09-10T09:59:35.857760+0000","last_unstale":"2024-09-10T09:59:36.972273+0000","last_undegraded":"2024-09-10T09:59:36.972273+0000","last_fullsized":"2024-09-10T09:59:36.972273+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:10.150900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.850082+0000","last_change":"2024-09-10T09:59:34.852727+0000","last_active":"2024-09-10T09:59:36.850082+0000","last_peered":"2024-09-10T09:59:36.850082+0000","last_clean":"2024-09-10T09:59:36.850082+0000","last_became_active":"2024-09-10T09:59:34.852404+0000","last_became_peered":"2024-09-10T09:59:34.852404+0000","last_unstale":"2024-09-10T09:59:36.850082+0000","last_undegraded":"2024-09-10T09:59:36.850082+0000","last_fullsized":"2024-09-10T09:59:36.850082+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:43.442772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849854+0000","last_change":"2024-09-10T09:59:35.868037+0000","last_active":"2024-09-10T09:59:36.849854+0000","last_peered":"2024-09-10T09:59:36.849854+0000","last_clean":"2024-09-10T09:59:36.849854+0000","last_became_active":"2024-09-10T09:59:35.867685+0000","last_became_peered":"2024-09-10T09:59:35.867685+0000","last_unstale":"2024-09-10T09:59:36.849854+0000","last_undegraded":"2024-09-10T09:59:36.849854+0000","last_fullsized":"2024-09-10T09:59:36.849854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:06.176821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971420+0000","last_change":"2024-09-10T09:59:34.843590+0000","last_active":"2024-09-10T09:59:36.971420+0000","last_peered":"2024-09-10T09:59:36.971420+0000","last_clean":"2024-09-10T09:59:36.971420+0000","last_became_active":"2024-09-10T09:59:34.843324+0000","last_became_peered":"2024-09-10T09:59:34.843324+0000","last_unstale":"2024-09-10T09:59:36.971420+0000","last_undegraded":"2024-09-10T09:59:36.971420+0000","last_fullsized":"2024-09-10T09:59:36.971420+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:58.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.874223+0000","last_change":"2024-09-10T09:59:35.869955+0000","last_active":"2024-09-10T09:59:35.874223+0000","last_peered":"2024-09-10T09:59:35.874223+0000","last_clean":"2024-09-10T09:59:35.874223+0000","last_became_active":"2024-09-10T09:59:35.869103+0000","last_became_peered":"2024-09-10T09:59:35.869103+0000","last_unstale":"2024-09-10T09:59:35.874223+0000","last_undegraded":"2024-09-10T09:59:35.874223+0000","last_fullsized":"2024-09-10T09:59:35.874223+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:30.276894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849671+0000","last_change":"2024-09-10T09:59:34.851811+0000","last_active":"2024-09-10T09:59:36.849671+0000","last_peered":"2024-09-10T09:59:36.849671+0000","last_clean":"2024-09-10T09:59:36.849671+0000","last_became_active":"2024-09-10T09:59:34.851406+0000","last_became_peered":"2024-09-10T09:59:34.851406+0000","last_unstale":"2024-09-10T09:59:36.849671+0000","last_undegraded":"2024-09-10T09:59:36.849671+0000","last_fullsized":"2024-09-10T09:59:36.849671+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:03.747037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T09:59:35.866411+0000","last_change":"2024-09-10T09:59:35.865294+0000","last_active":"2024-09-10T09:59:35.866411+0000","last_peered":"2024-09-10T09:59:35.866411+0000","last_clean":"2024-09-10T09:59:35.866411+0000","last_became_active":"2024-09-10T09:59:35.865015+0000","last_became_peered":"2024-09-10T09:59:35.865015+0000","last_unstale":"2024-09-10T09:59:35.866411+0000","last_undegraded":"2024-09-10T09:59:35.866411+0000","last_fullsized":"2024-09-10T09:59:35.866411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:16.906350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882501+0000","last_change":"2024-09-10T09:59:34.843607+0000","last_active":"2024-09-10T09:59:36.882501+0000","last_peered":"2024-09-10T09:59:36.882501+0000","last_clean":"2024-09-10T09:59:36.882501+0000","last_became_active":"2024-09-10T09:59:34.843335+0000","last_became_peered":"2024-09-10T09:59:34.843335+0000","last_unstale":"2024-09-10T09:59:36.882501+0000","last_undegraded":"2024-09-10T09:59:36.882501+0000","last_fullsized":"2024-09-10T09:59:36.882501+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:20.784501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972217+0000","last_change":"2024-09-10T09:59:35.862583+0000","last_active":"2024-09-10T09:59:36.972217+0000","last_peered":"2024-09-10T09:59:36.972217+0000","last_clean":"2024-09-10T09:59:36.972217+0000","last_became_active":"2024-09-10T09:59:35.862275+0000","last_became_peered":"2024-09-10T09:59:35.862275+0000","last_unstale":"2024-09-10T09:59:36.972217+0000","last_undegraded":"2024-09-10T09:59:36.972217+0000","last_fullsized":"2024-09-10T09:59:36.972217+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:56.488732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971395+0000","last_change":"2024-09-10T09:59:34.851351+0000","last_active":"2024-09-10T09:59:36.971395+0000","last_peered":"2024-09-10T09:59:36.971395+0000","last_clean":"2024-09-10T09:59:36.971395+0000","last_became_active":"2024-09-10T09:59:34.851070+0000","last_became_peered":"2024-09-10T09:59:34.851070+0000","last_unstale":"2024-09-10T09:59:36.971395+0000","last_undegraded":"2024-09-10T09:59:36.971395+0000","last_fullsized":"2024-09-10T09:59:36.971395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:20.784501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.882364+0000","last_change":"2024-09-10T09:59:35.872635+0000","last_active":"2024-09-10T09:59:36.882364+0000","last_peered":"2024-09-10T09:59:36.882364+0000","last_clean":"2024-09-10T09:59:36.882364+0000","last_became_active":"2024-09-10T09:59:35.872274+0000","last_became_peered":"2024-09-10T09:59:35.872274+0000","last_unstale":"2024-09-10T09:59:36.882364+0000","last_undegraded":"2024-09-10T09:59:36.882364+0000","last_fullsized":"2024-09-10T09:59:36.882364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:37.758802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.971797+0000","last_change":"2024-09-10T09:59:34.859646+0000","last_active":"2024-09-10T09:59:36.971797+0000","last_peered":"2024-09-10T09:59:36.971797+0000","last_clean":"2024-09-10T09:59:36.971797+0000","last_became_active":"2024-09-10T09:59:34.859344+0000","last_became_peered":"2024-09-10T09:59:34.859344+0000","last_unstale":"2024-09-10T09:59:36.971797+0000","last_undegraded":"2024-09-10T09:59:36.971797+0000","last_fullsized":"2024-09-10T09:59:36.971797+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:40.256438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972457+0000","last_change":"2024-09-10T09:59:35.868872+0000","last_active":"2024-09-10T09:59:36.972457+0000","last_peered":"2024-09-10T09:59:36.972457+0000","last_clean":"2024-09-10T09:59:36.972457+0000","last_became_active":"2024-09-10T09:59:35.868629+0000","last_became_peered":"2024-09-10T09:59:35.868629+0000","last_unstale":"2024-09-10T09:59:36.972457+0000","last_undegraded":"2024-09-10T09:59:36.972457+0000","last_fullsized":"2024-09-10T09:59:36.972457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:34.825856+0000","last_clean_scrub_stamp":"2024-09-10T09:59:34.825856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:27:12.613140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.972695+0000","last_change":"2024-09-10T09:59:34.855715+0000","last_active":"2024-09-10T09:59:36.972695+0000","last_peered":"2024-09-10T09:59:36.972695+0000","last_clean":"2024-09-10T09:59:36.972695+0000","last_became_active":"2024-09-10T09:59:34.855430+0000","last_became_peered":"2024-09-10T09:59:34.855430+0000","last_unstale":"2024-09-10T09:59:36.972695+0000","last_undegraded":"2024-09-10T09:59:36.972695+0000","last_fullsized":"2024-09-10T09:59:36.972695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:59:33.820099+0000","last_clean_scrub_stamp":"2024-09-10T09:59:33.820099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:50.619585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":210,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T09:59:36.849179+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T09:59:36.849179+0000","last_peered":"2024-09-10T09:59:36.849179+0000","last_clean":"2024-09-10T09:59:36.849179+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T09:59:36.849179+0000","last_undegraded":"2024-09-10T09:59:36.849179+0000","last_fullsized":"2024-09-10T09:59:36.849179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1118208,"data_stored":797797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":208011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":524288,"data_stored":207409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":532480,"data_stored":208951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":536576,"data_stored":208973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":524288,"data_stored":207409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":797741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":194264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:59:40.323 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T09:59:40.324 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T09:59:40.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8323200640 1 Processor -- start 2024-09-10T09:59:40.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8323200640 1 -- start start 2024-09-10T09:59:40.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8323200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f831c15c560 0x7f831c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8323200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f831c06f010 0x7f831c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8323200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831c06f430 con 0x7f831c155970 2024-09-10T09:59:40.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8323200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f831c0b4570 con 0x7f831c06f010 2024-09-10T09:59:40.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8323200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f831c155d50 con 0x7f831c15c560 2024-09-10T09:59:40.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8321200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f831c06f010 0x7f831c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8321200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f831c06f010 0x7f831c0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46876/0 (socket says 172.21.15.110:46876) 2024-09-10T09:59:40.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8321200640 1 -- 172.21.15.110:0/764668505 learned_addr learned my addr 172.21.15.110:0/764668505 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:40.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f831b400640 1 --2- 172.21.15.110:0/764668505 >> v2:172.21.15.145:3300/0 conn(0x7f831c15c560 0x7f831c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8321200640 1 -- 172.21.15.110:0/764668505 >> v2:172.21.15.145:3300/0 conn(0x7f831c15c560 msgr2=0x7f831c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:40.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8321200640 1 --2- 172.21.15.110:0/764668505 >> v2:172.21.15.145:3300/0 conn(0x7f831c15c560 0x7f831c06ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8321200640 1 -- 172.21.15.110:0/764668505 >> v1:172.21.15.145:6789/0 conn(0x7f831c155970 legacy=0x7f831c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.498+0000 7f8321200640 1 -- 172.21.15.110:0/764668505 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c0b34d0 con 0x7f831c06f010 2024-09-10T09:59:40.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8321200640 1 --2- 172.21.15.110:0/764668505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c06f010 0x7f831c0ce420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f831c06d930 tx=0x7f831401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1230451d75526756 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f831aa00640 1 -- 172.21.15.110:0/764668505 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140a5880 con 0x7f831c06f010 2024-09-10T09:59:40.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f831aa00640 1 -- 172.21.15.110:0/764668505 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83140a5a20 con 0x7f831c06f010 2024-09-10T09:59:40.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f831aa00640 1 -- 172.21.15.110:0/764668505 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140a5df0 con 0x7f831c06f010 2024-09-10T09:59:40.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8323200640 1 -- 172.21.15.110:0/764668505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c06f010 msgr2=0x7f831c0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8323200640 1 --2- 172.21.15.110:0/764668505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c06f010 0x7f831c0ce420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f831c06d930 tx=0x7f831401c9c0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8323200640 1 -- 172.21.15.110:0/764668505 shutdown_connections 2024-09-10T09:59:40.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8323200640 1 --2- 172.21.15.110:0/764668505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c06f010 0x7f831c0ce420 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8323200640 1 --2- 172.21.15.110:0/764668505 >> v2:172.21.15.145:3300/0 conn(0x7f831c15c560 0x7f831c06ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8323200640 1 -- 172.21.15.110:0/764668505 >> 172.21.15.110:0/764668505 conn(0x7f831c06bc50 msgr2=0x7f831c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:40.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8323200640 1 -- 172.21.15.110:0/764668505 shutdown_connections 2024-09-10T09:59:40.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.502+0000 7f8323200640 1 -- 172.21.15.110:0/764668505 wait complete. 2024-09-10T09:59:40.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 Processor -- start 2024-09-10T09:59:40.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 -- start start 2024-09-10T09:59:40.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f831c06f010 0x7f831c0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f831c155970 0x7f831c143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c15c560 0x7f831c1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0cf170 con 0x7f831c15c560 2024-09-10T09:59:40.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0b4050 con 0x7f831c155970 2024-09-10T09:59:40.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0cf2f0 con 0x7f831c06f010 2024-09-10T09:59:40.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c15c560 0x7f831c1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c15c560 0x7f831c1441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46890/0 (socket says 172.21.15.110:46890) 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 -- 172.21.15.110:0/1152911290 learned_addr learned my addr 172.21.15.110:0/1152911290 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f831b400640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f831c155970 0x7f831c143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f831be00640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f831c06f010 0x7f831c0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 -- 172.21.15.110:0/1152911290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f831c06f010 msgr2=0x7f831c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f831c06f010 0x7f831c0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 -- 172.21.15.110:0/1152911290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f831c155970 msgr2=0x7f831c143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f831c155970 0x7f831c143c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 -- 172.21.15.110:0/1152911290 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c143960 con 0x7f831c15c560 2024-09-10T09:59:40.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8321200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c15c560 0x7f831c1441a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f83140af040 tx=0x7f8314019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8313e00640 1 -- 172.21.15.110:0/1152911290 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140b8670 con 0x7f831c15c560 2024-09-10T09:59:40.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8313e00640 1 -- 172.21.15.110:0/1152911290 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83140be070 con 0x7f831c15c560 2024-09-10T09:59:40.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1447d0 con 0x7f831c15c560 2024-09-10T09:59:40.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8313e00640 1 -- 172.21.15.110:0/1152911290 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140b8a70 con 0x7f831c15c560 2024-09-10T09:59:40.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c141bd0 con 0x7f831c15c560 2024-09-10T09:59:40.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.506+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c0cf650 con 0x7f831c15c560 2024-09-10T09:59:40.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.510+0000 7f8313e00640 1 -- 172.21.15.110:0/1152911290 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f83140b8c10 con 0x7f831c15c560 2024-09-10T09:59:40.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.510+0000 7f8313e00640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f82fc067a70 0x7f82fc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.510+0000 7f8313e00640 1 -- 172.21.15.110:0/1152911290 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f83141211b0 con 0x7f831c15c560 2024-09-10T09:59:40.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.510+0000 7f831be00640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f82fc067a70 0x7f82fc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.510+0000 7f831be00640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f82fc067a70 0x7f82fc069f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f831c15c3c0 tx=0x7f830c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.514+0000 7f8313e00640 1 -- 172.21.15.110:0/1152911290 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f83140165e0 con 0x7f831c15c560 2024-09-10T09:59:40.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.726+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f831c15f970 con 0x7f831c15c560 2024-09-10T09:59:40.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.726+0000 7f8313e00640 1 -- 172.21.15.110:0/1152911290 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f83140f1450 con 0x7f831c15c560 2024-09-10T09:59:40.733 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T09:59:40.733 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T09:59:40.733 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T09:59:40.733 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T09:59:40.733 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T09:59:40.733 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f82fc067a70 msgr2=0x7f82fc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f82fc067a70 0x7f82fc069f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f831c15c3c0 tx=0x7f830c009040 comp rx=0 tx=0).stop 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c15c560 msgr2=0x7f831c1441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c15c560 0x7f831c1441a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f83140af040 tx=0x7f8314019760 comp rx=0 tx=0).stop 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 shutdown_connections 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f82fc067a70 0x7f82fc069f30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c15c560 0x7f831c1441a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f831c155970 0x7f831c143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 --2- 172.21.15.110:0/1152911290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f831c06f010 0x7f831c0ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 >> 172.21.15.110:0/1152911290 conn(0x7f831c06bc50 msgr2=0x7f831c15a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:40.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 shutdown_connections 2024-09-10T09:59:40.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.730+0000 7f8323200640 1 -- 172.21.15.110:0/1152911290 wait complete. 2024-09-10T09:59:40.753 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:40.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.882+0000 7f4381400640 1 Processor -- start 2024-09-10T09:59:40.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4381400640 1 -- start start 2024-09-10T09:59:40.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4381400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f437c06eb30 0x7f437c0d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4381400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c073290 0x7f437c073720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4381400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437c0706a0 con 0x7f437c163000 2024-09-10T09:59:40.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4381400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f437c15ee90 con 0x7f437c06eb30 2024-09-10T09:59:40.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4381400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f437c0710d0 con 0x7f437c073290 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f437c06eb30 0x7f437c0d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f437c06eb30 0x7f437c0d0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:39324/0 (socket says 172.21.15.110:39324) 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4379e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f437c163000 0x7f437c071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:54224/0 (socket says 172.21.15.110:54224) 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437a800640 1 -- 172.21.15.110:0/76874052 learned_addr learned my addr 172.21.15.110:0/76874052 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437b200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c073290 0x7f437c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4379400640 1 -- 172.21.15.110:0/76874052 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1438735615 0 0) 0x7f437c0706a0 con 0x7f437c163000 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f4379400640 1 -- 172.21.15.110:0/76874052 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4360003530 con 0x7f437c163000 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437a800640 1 -- 172.21.15.110:0/76874052 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c073290 msgr2=0x7f437c073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:40.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437a800640 1 --2- 172.21.15.110:0/76874052 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c073290 0x7f437c073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437a800640 1 -- 172.21.15.110:0/76874052 >> v1:172.21.15.145:6789/0 conn(0x7f437c163000 legacy=0x7f437c071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437a800640 1 -- 172.21.15.110:0/76874052 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c0d1370 con 0x7f437c06eb30 2024-09-10T09:59:40.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.886+0000 7f437b200640 1 --2- 172.21.15.110:0/76874052 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c073290 0x7f437c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:40.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f437a800640 1 --2- 172.21.15.110:0/76874052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 0x7f437c0d0cb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f436c009e50 tx=0x7f436c01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe452fea67b48a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4379400640 1 -- 172.21.15.110:0/76874052 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0a5660 con 0x7f437c06eb30 2024-09-10T09:59:40.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4379400640 1 -- 172.21.15.110:0/76874052 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436c0a5800 con 0x7f437c06eb30 2024-09-10T09:59:40.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4379400640 1 -- 172.21.15.110:0/76874052 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0a5bd0 con 0x7f437c06eb30 2024-09-10T09:59:40.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 -- 172.21.15.110:0/76874052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 msgr2=0x7f437c0d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 --2- 172.21.15.110:0/76874052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 0x7f437c0d0cb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f436c009e50 tx=0x7f436c01c9a0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 -- 172.21.15.110:0/76874052 shutdown_connections 2024-09-10T09:59:40.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 --2- 172.21.15.110:0/76874052 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c073290 0x7f437c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 --2- 172.21.15.110:0/76874052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 0x7f437c0d0cb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 -- 172.21.15.110:0/76874052 >> 172.21.15.110:0/76874052 conn(0x7f437c0d25c0 msgr2=0x7f437c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:40.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 -- 172.21.15.110:0/76874052 shutdown_connections 2024-09-10T09:59:40.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 -- 172.21.15.110:0/76874052 wait complete. 2024-09-10T09:59:40.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 Processor -- start 2024-09-10T09:59:40.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.890+0000 7f4381400640 1 -- start start 2024-09-10T09:59:40.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 0x7f437c14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f437c073290 0x7f437c151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c163000 0x7f437c0b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f437c06dd60 con 0x7f437c073290 2024-09-10T09:59:40.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1633e0 con 0x7f437c06eb30 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f437c06dee0 con 0x7f437c163000 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f437c073290 0x7f437c151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f437a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 0x7f437c14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f437c073290 0x7f437c151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46898/0 (socket says 172.21.15.110:46898) 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 -- 172.21.15.110:0/1674934608 learned_addr learned my addr 172.21.15.110:0/1674934608 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f437b200640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c163000 0x7f437c0b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 -- 172.21.15.110:0/1674934608 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c163000 msgr2=0x7f437c0b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c163000 0x7f437c0b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 -- 172.21.15.110:0/1674934608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 msgr2=0x7f437c14fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:40.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 0x7f437c14fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:40.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 -- 172.21.15.110:0/1674934608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c0ce5b0 con 0x7f437c073290 2024-09-10T09:59:40.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4379e00640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f437c073290 0x7f437c151300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f436800c430 tx=0x7f436800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4372a00640 1 -- 172.21.15.110:0/1674934608 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680074e0 con 0x7f437c073290 2024-09-10T09:59:40.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4372a00640 1 -- 172.21.15.110:0/1674934608 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43680047f0 con 0x7f437c073290 2024-09-10T09:59:40.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c150820 con 0x7f437c073290 2024-09-10T09:59:40.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4372a00640 1 -- 172.21.15.110:0/1674934608 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680076a0 con 0x7f437c073290 2024-09-10T09:59:40.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c1503b0 con 0x7f437c073290 2024-09-10T09:59:40.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.894+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c150610 con 0x7f437c073290 2024-09-10T09:59:40.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.898+0000 7f4372a00640 1 -- 172.21.15.110:0/1674934608 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f436801a020 con 0x7f437c073290 2024-09-10T09:59:40.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.898+0000 7f4372a00640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f435c067a70 0x7f435c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:40.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.898+0000 7f4372a00640 1 -- 172.21.15.110:0/1674934608 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f43680857d0 con 0x7f437c073290 2024-09-10T09:59:40.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.898+0000 7f437a800640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f435c067a70 0x7f435c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:40.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.898+0000 7f437a800640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f435c067a70 0x7f435c069f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f436c0ae000 tx=0x7f436c0a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:40.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:40.902+0000 7f4372a00640 1 -- 172.21.15.110:0/1674934608 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4368055ff0 con 0x7f437c073290 2024-09-10T09:59:41.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437c0736a0 con 0x7f437c073290 2024-09-10T09:59:41.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4372a00640 1 -- 172.21.15.110:0/1674934608 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5232 (secure 0 0 0) 0x7f43680558f0 con 0x7f437c073290 2024-09-10T09:59:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:41.099 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":26,"btime":"2024-09-10T09:59:38:872051+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T09:59:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 26 2024-09-10T09:59:41.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f435c067a70 msgr2=0x7f435c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f435c067a70 0x7f435c069f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f436c0ae000 tx=0x7f436c0a7070 comp rx=0 tx=0).stop 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f437c073290 msgr2=0x7f437c151300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f437c073290 0x7f437c151300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f436800c430 tx=0x7f436800c830 comp rx=0 tx=0).stop 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 shutdown_connections 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f435c067a70 0x7f435c069f30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f437c163000 0x7f437c0b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f437c073290 0x7f437c151300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 --2- 172.21.15.110:0/1674934608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437c06eb30 0x7f437c14fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 >> 172.21.15.110:0/1674934608 conn(0x7f437c0d25c0 msgr2=0x7f437c0b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 shutdown_connections 2024-09-10T09:59:41.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.094+0000 7f4381400640 1 -- 172.21.15.110:0/1674934608 wait complete. 2024-09-10T09:59:41.524 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T09:59:35.214092+0000', 'modified': '2024-09-10T09:59:38.244548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5214}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5214': {'gid': 5214, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6837/2457861782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6835', 'nonce': 2457861782}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 2457861782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5214, 'qdb_cluster': [5214]} 2024-09-10T09:59:41.524 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T09:59:41.524 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T09:59:41.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.654+0000 7f5431200640 1 Processor -- start 2024-09-10T09:59:41.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.654+0000 7f5431200640 1 -- start start 2024-09-10T09:59:41.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5431200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c0726e0 0x7f542c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5431200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f542c06f3e0 con 0x7f542c0b5600 2024-09-10T09:59:41.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5431200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f542c15fda0 con 0x7f542c06efa0 2024-09-10T09:59:41.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5431200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f542c0cee50 con 0x7f542c0726e0 2024-09-10T09:59:41.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f542a800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f542c0b5600 0x7f542c0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51758/0 (socket says 172.21.15.110:51758) 2024-09-10T09:59:41.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f542a800640 1 -- 172.21.15.110:0/4034236407 learned_addr learned my addr 172.21.15.110:0/4034236407 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:41.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5429e00640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c0726e0 0x7f542c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4184088529 0 0) 0x7f542c06f3e0 con 0x7f542c0b5600 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5410005740 con 0x7f542c0b5600 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358426601 0 0) 0x7f542c15fda0 con 0x7f542c06efa0 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f542c06f3e0 con 0x7f542c06efa0 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 273418628 0 0) 0x7f5410005740 con 0x7f542c0b5600 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c0726e0 msgr2=0x7f542c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c0726e0 0x7f542c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 >> v1:172.21.15.145:6789/0 conn(0x7f542c06efa0 legacy=0x7f542c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c0ce520 con 0x7f542c0b5600 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f5424001a00 con 0x7f542c0b5600 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 >> v1:172.21.15.110:6789/0 conn(0x7f542c0b5600 legacy=0x7f542c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c0726e0 0x7f5410005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54100062f0 0x7f5410006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f54100073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f542c0cee50 con 0x7f5410006f60 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f541000dcc0 con 0x7f54100062f0 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f541000dec0 con 0x7f542c0726e0 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f542a800640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f54100073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5429e00640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c0726e0 0x7f5410005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f5429400640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54100062f0 0x7f5410006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f542a800640 1 -- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c0726e0 msgr2=0x7f5410005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f542a800640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c0726e0 0x7f5410005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f542a800640 1 -- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54100062f0 msgr2=0x7f5410006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f542a800640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54100062f0 0x7f5410006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.658+0000 7f542a800640 1 -- 172.21.15.110:0/4034236407 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5410005740 con 0x7f5410006f60 2024-09-10T09:59:41.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f542a800640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f54100073e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f542400a7c0 tx=0x7f542401fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac7a39e26733ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542400a410 con 0x7f5410006f60 2024-09-10T09:59:41.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54240aa690 con 0x7f5410006f60 2024-09-10T09:59:41.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5428a00640 1 -- 172.21.15.110:0/4034236407 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54240aaac0 con 0x7f5410006f60 2024-09-10T09:59:41.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5431200640 1 -- 172.21.15.110:0/4034236407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 msgr2=0x7f54100073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5431200640 1 --2- 172.21.15.110:0/4034236407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f54100073e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f542400a7c0 tx=0x7f542401fa40 comp rx=0 tx=0).stop 2024-09-10T09:59:41.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f542a800640 1 -- 172.21.15.110:0/4034236407 reap_dead start 2024-09-10T09:59:41.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5431200640 1 -- 172.21.15.110:0/4034236407 shutdown_connections 2024-09-10T09:59:41.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5431200640 1 -- 172.21.15.110:0/4034236407 >> 172.21.15.110:0/4034236407 conn(0x7f542c062d10 msgr2=0x7f542c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5431200640 1 -- 172.21.15.110:0/4034236407 shutdown_connections 2024-09-10T09:59:41.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5431200640 1 -- 172.21.15.110:0/4034236407 wait complete. 2024-09-10T09:59:41.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5431200640 1 Processor -- start 2024-09-10T09:59:41.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.662+0000 7f5431200640 1 -- start start 2024-09-10T09:59:41.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5431200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f542c14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5431200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f542c0b06b0 0x7f542c151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5431200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c152040 0x7f542c0b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5431200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f542c073270 con 0x7f5410006f60 2024-09-10T09:59:41.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5431200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f542c0ceae0 con 0x7f542c0b06b0 2024-09-10T09:59:41.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5431200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f542c072ac0 con 0x7f542c152040 2024-09-10T09:59:41.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f542c14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f542c14ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46928/0 (socket says 172.21.15.110:46928) 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 -- 172.21.15.110:0/214204866 learned_addr learned my addr 172.21.15.110:0/214204866 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f542a800640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c152040 0x7f542c0b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429400640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f542c0b06b0 0x7f542c151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 -- 172.21.15.110:0/214204866 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c152040 msgr2=0x7f542c0b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c152040 0x7f542c0b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 -- 172.21.15.110:0/214204866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f542c0b06b0 msgr2=0x7f542c151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f542c0b06b0 0x7f542c151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 -- 172.21.15.110:0/214204866 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c0b0b90 con 0x7f5410006f60 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5429e00640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f542c14ed50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f541c011a00 tx=0x7f541c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5422000640 1 -- 172.21.15.110:0/214204866 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c00fb50 con 0x7f5410006f60 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5422000640 1 -- 172.21.15.110:0/214204866 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f541c00fcf0 con 0x7f5410006f60 2024-09-10T09:59:41.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5422000640 1 -- 172.21.15.110:0/214204866 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c00fea0 con 0x7f5410006f60 2024-09-10T09:59:41.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5431200640 1 -- 172.21.15.110:0/214204866 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542c0b0e70 con 0x7f5410006f60 2024-09-10T09:59:41.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5431200640 1 -- 172.21.15.110:0/214204866 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542c0b1430 con 0x7f5410006f60 2024-09-10T09:59:41.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.666+0000 7f5422000640 1 -- 172.21.15.110:0/214204866 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f541c012040 con 0x7f5410006f60 2024-09-10T09:59:41.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5422000640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f540c067aa0 0x7f540c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5422000640 1 -- 172.21.15.110:0/214204866 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f541c0866c0 con 0x7f5410006f60 2024-09-10T09:59:41.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f540c067aa0 0x7f540c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5434602000 1 Processor -- start 2024-09-10T09:59:41.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5434602000 1 -- start start 2024-09-10T09:59:41.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f540c067aa0 0x7f540c069f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f542c0b31e0 tx=0x7f5418009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5434602000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 0x5575bcf96150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5434602000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf96690 0x5575bcf4c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5434602000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf4c6a0 0x5575bcf4ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5434602000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5575bce2fb60 con 0x5575bcf96690 2024-09-10T09:59:41.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5434602000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5575bcd96c70 con 0x5575bcf4c6a0 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5434602000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5575bcd3ced0 con 0x5575bcf95d70 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf96690 0x5575bcf4c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf96690 0x5575bcf4c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46932/0 (socket says 172.21.15.110:46932) 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 -- 172.21.15.110:0/3288606565 learned_addr learned my addr 172.21.15.110:0/3288606565 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f542a800640 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf4c6a0 0x5575bcf4ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429e00640 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 0x5575bcf96150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 -- 172.21.15.110:0/3288606565 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 msgr2=0x5575bcf96150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 0x5575bcf96150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 -- 172.21.15.110:0/3288606565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf4c6a0 msgr2=0x5575bcf4ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf4c6a0 0x5575bcf4ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 -- 172.21.15.110:0/3288606565 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575bcd4ee50 con 0x5575bcf96690 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5429400640 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf96690 0x5575bcf4c160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5418007fd0 tx=0x7f5418014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611e464590b78c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5416a00640 1 -- 172.21.15.110:0/3288606565 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418028070 con 0x5575bcf96690 2024-09-10T09:59:41.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5416a00640 1 -- 172.21.15.110:0/3288606565 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54180159c0 con 0x5575bcf96690 2024-09-10T09:59:41.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.670+0000 7f5416a00640 1 -- 172.21.15.110:0/3288606565 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418015d70 con 0x5575bcf96690 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- 172.21.15.110:0/3288606565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf96690 msgr2=0x5575bcf4c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf96690 0x5575bcf4c160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5418007fd0 tx=0x7f5418014000 comp rx=0 tx=0).stop 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- 172.21.15.110:0/3288606565 shutdown_connections 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf4c6a0 0x5575bcf4ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf96690 0x5575bcf4c160 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 --2- 172.21.15.110:0/3288606565 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 0x5575bcf96150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- 172.21.15.110:0/3288606565 >> 172.21.15.110:0/3288606565 conn(0x5575bcf4e600 msgr2=0x5575bcf46630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- 172.21.15.110:0/3288606565 shutdown_connections 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- 172.21.15.110:0/3288606565 wait complete. 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- start start 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 Processor -- start 2024-09-10T09:59:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf96690 0x5575bcf9c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf82490 0x5575bcf82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5575bcdf8b00 con 0x5575bcf95d70 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5575bcd71360 con 0x5575bcf96690 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5575bcd6e2d0 con 0x5575bcf82490 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 10 client.0 _ll_register_callbacks cb 0x5575bcf33820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46942/0 (socket says 172.21.15.110:46942) 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 -- 172.21.15.110:0/2356015475 learned_addr learned my addr 172.21.15.110:0/2356015475 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429400640 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf96690 0x5575bcf9c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f542a800640 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf82490 0x5575bcf82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 -- 172.21.15.110:0/2356015475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf82490 msgr2=0x5575bcf82870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf82490 0x5575bcf82870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 -- 172.21.15.110:0/2356015475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf96690 msgr2=0x5575bcf9c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf96690 0x5575bcf9c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 -- 172.21.15.110:0/2356015475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575bcd3d120 con 0x5575bcf95d70 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5429e00640 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9beb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f541c00f610 tx=0x7f541c05ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b731203117af2a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f540b400640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f540b400640 1 -- 172.21.15.110:0/2356015475 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c022040 con 0x5575bcf95d70 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f540b400640 1 -- 172.21.15.110:0/2356015475 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f541c056bb0 con 0x5575bcf95d70 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- 172.21.15.110:0/2356015475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54180159c0 con 0x5575bcf95d70 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f540b400640 1 -- 172.21.15.110:0/2356015475 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f541c084cb0 con 0x5575bcf95d70 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 10 client.5325 fetch_fsmap learned FSMap version 26 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f5434602000 1 -- 172.21.15.110:0/2356015475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5575bcd36450 con 0x5575bcf95d70 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.674+0000 7f540b400640 1 -- 172.21.15.110:0/2356015475 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54180159c0 con 0x5575bcf95d70 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f540b400640 1 -- 172.21.15.110:0/2356015475 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f541c057bd0 con 0x5575bcf95d70 2024-09-10T09:59:41.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 10 client.5325 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T09:59:41.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 10 client.5325 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T09:59:41.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 20 client.5325 populate_metadata read hostname 'smithi110' 2024-09-10T09:59:41.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] conn(0x5575bcf83280 0x5575bcf83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 4 client.5325 mds_command: new command op to 5214 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:41.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 -- 172.21.15.110:0/2356015475 --> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575bcd30c00 con 0x5575bcf83280 2024-09-10T09:59:41.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5429400640 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] conn(0x5575bcf83280 0x5575bcf83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5429400640 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] conn(0x5575bcf83280 0x5575bcf83660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bc0428cc7beabc server_cookie=d48a7c1d4d8e6388 in_seq=0 out_seq=0 2024-09-10T09:59:41.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f540b400640 10 client.5325 ms_handle_connect on v2:172.21.15.110:6835/2457861782 2024-09-10T09:59:41.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f540b400640 1 -- 172.21.15.110:0/2356015475 <== mds.0 v2:172.21.15.110:6835/2457861782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575bcd30c00 con 0x5575bcf83280 2024-09-10T09:59:41.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f540b400640 10 client.5325 handle_command_reply: tid=0 2024-09-10T09:59:41.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 client.5325 shutdown 2024-09-10T09:59:41.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 -- 172.21.15.110:0/2356015475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 msgr2=0x5575bcf9beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9beb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f541c00f610 tx=0x7f541c05ac50 comp rx=0 tx=0).stop 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 -- 172.21.15.110:0/2356015475 shutdown_connections 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] conn(0x5575bcf83280 0x5575bcf83660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf82490 0x5575bcf82870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf96690 0x5575bcf9c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f540b400640 0 client.5325 ms_handle_reset on v2:172.21.15.110:6835/2457861782 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 --2- 172.21.15.110:0/2356015475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9beb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 -- 172.21.15.110:0/2356015475 >> 172.21.15.110:0/2356015475 conn(0x5575bcf4c6a0 msgr2=0x5575bcf4d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 -- 172.21.15.110:0/2356015475 shutdown_connections 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 1 -- 172.21.15.110:0/2356015475 wait complete. 2024-09-10T09:59:41.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.678+0000 7f5434602000 20 client.5325 trim_cache size 0 max 16384 2024-09-10T09:59:41.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 Processor -- start 2024-09-10T09:59:41.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- start start 2024-09-10T09:59:41.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf4c6a0 0x5575bcfac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 0x5575bcfac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 0x5575bcf9aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1fc060 con 0x5575bcf4c6a0 2024-09-10T09:59:41.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1547c0 con 0x5575bcf82490 2024-09-10T09:59:41.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f542c1fba40 con 0x5575bcf95d70 2024-09-10T09:59:41.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf4c6a0 0x5575bcfac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf4c6a0 0x5575bcfac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46946/0 (socket says 172.21.15.110:46946) 2024-09-10T09:59:41.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 -- 172.21.15.110:0/1610139042 learned_addr learned my addr 172.21.15.110:0/1610139042 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:41.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 -- 172.21.15.110:0/1610139042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 msgr2=0x5575bcf9aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 --2- 172.21.15.110:0/1610139042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 0x5575bcf9aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 -- 172.21.15.110:0/1610139042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 msgr2=0x5575bcfac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 --2- 172.21.15.110:0/1610139042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 0x5575bcfac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 -- 172.21.15.110:0/1610139042 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575bcd1dfd0 con 0x5575bcf4c6a0 2024-09-10T09:59:41.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5429e00640 1 --2- 172.21.15.110:0/1610139042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf4c6a0 0x5575bcfac2d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f541c0532d0 tx=0x7f541c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2928d9de4d4c41b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5416000640 1 -- 172.21.15.110:0/1610139042 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c022040 con 0x5575bcf4c6a0 2024-09-10T09:59:41.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5416000640 1 -- 172.21.15.110:0/1610139042 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f541c056bb0 con 0x5575bcf4c6a0 2024-09-10T09:59:41.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5416000640 1 -- 172.21.15.110:0/1610139042 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c077030 con 0x5575bcf4c6a0 2024-09-10T09:59:41.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- 172.21.15.110:0/1610139042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf4c6a0 msgr2=0x5575bcfac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 --2- 172.21.15.110:0/1610139042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf4c6a0 0x5575bcfac2d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f541c0532d0 tx=0x7f541c01f000 comp rx=0 tx=0).stop 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- 172.21.15.110:0/1610139042 shutdown_connections 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 --2- 172.21.15.110:0/1610139042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf95d70 0x5575bcf9aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 --2- 172.21.15.110:0/1610139042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 0x5575bcfac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 --2- 172.21.15.110:0/1610139042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf4c6a0 0x5575bcfac2d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- 172.21.15.110:0/1610139042 >> 172.21.15.110:0/1610139042 conn(0x5575bcf81ce0 msgr2=0x5575bcf46ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- 172.21.15.110:0/1610139042 shutdown_connections 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- 172.21.15.110:0/1610139042 wait complete. 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 -- start start 2024-09-10T09:59:41.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.698+0000 7f5434602000 1 Processor -- start 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf4c6a0 0x5575bd009a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 0x5575bd009f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5575bcd95f80 con 0x5575bcf95d70 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5575bcd4eca0 con 0x5575bcf82490 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5575bcd9b2a0 con 0x5575bcf4c6a0 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 10 client.0 _ll_register_callbacks cb 0x7f541c0847a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46950/0 (socket says 172.21.15.110:46950) 2024-09-10T09:59:41.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 -- 172.21.15.110:0/2357940253 learned_addr learned my addr 172.21.15.110:0/2357940253 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5429400640 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 0x5575bd009f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5429e00640 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf4c6a0 0x5575bd009a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 -- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf4c6a0 msgr2=0x5575bd009a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf4c6a0 0x5575bd009a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 -- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 msgr2=0x5575bd009f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 0x5575bd009f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 -- 172.21.15.110:0/2357940253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575bce19750 con 0x5575bcf95d70 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5429e00640 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf4c6a0 0x5575bd009a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f542a800640 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9fc60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f542400a7c0 tx=0x7f5410007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac7ff6255830d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 1 -- 172.21.15.110:0/2357940253 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5424001f80 con 0x5575bcf95d70 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 -- 172.21.15.110:0/2357940253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f541c056bb0 con 0x5575bcf95d70 2024-09-10T09:59:41.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 1 -- 172.21.15.110:0/2357940253 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f542400a3f0 con 0x5575bcf95d70 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 1 -- 172.21.15.110:0/2357940253 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54240bc450 con 0x5575bcf95d70 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 10 client.5331 fetch_fsmap learned FSMap version 26 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 -- 172.21.15.110:0/2357940253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5575bcd24a30 con 0x5575bcf95d70 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 1 -- 172.21.15.110:0/2357940253 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f541c056bb0 con 0x5575bcf95d70 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 1 -- 172.21.15.110:0/2357940253 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f54240bc5f0 con 0x5575bcf95d70 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 10 client.5331 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 10 client.5331 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 20 client.5331 populate_metadata read hostname 'smithi110' 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] conn(0x5575bcf96690 0x5575bcfa01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 4 client.5331 mds_command: new command op to 5214 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:59:41.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 -- 172.21.15.110:0/2357940253 --> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] -- command(tid 0: {"prefix": "status"}) -- 0x5575bcda43a0 con 0x5575bcf96690 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5429e00640 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] conn(0x5575bcf96690 0x5575bcfa01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 10 client.5331 ms_handle_connect on v2:172.21.15.110:6835/2457861782 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5429e00640 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] conn(0x5575bcf96690 0x5575bcfa01a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53853567ced70455 server_cookie=4ae652af02e4c703 in_seq=0 out_seq=0 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 1 -- 172.21.15.110:0/2357940253 <== mds.0 v2:172.21.15.110:6835/2457861782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5575bcda43a0 con 0x5575bcf96690 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f540b400640 10 client.5331 handle_command_reply: tid=0 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 client.5331 shutdown 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 -- 172.21.15.110:0/2357940253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 msgr2=0x5575bcf9fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.702+0000 7f5434602000 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9fc60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f542400a7c0 tx=0x7f5410007a50 comp rx=0 tx=0).stop 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 1 -- 172.21.15.110:0/2357940253 shutdown_connections 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.110:6835/2457861782,v1:172.21.15.110:6837/2457861782] conn(0x5575bcf96690 0x5575bcfa01a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575bcf95d70 0x5575bcf9fc60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575bcf82490 0x5575bd009f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 1 --2- 172.21.15.110:0/2357940253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5575bcf4c6a0 0x5575bd009a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 1 -- 172.21.15.110:0/2357940253 >> 172.21.15.110:0/2357940253 conn(0x5575bcf81ce0 msgr2=0x5575bcfab2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f540b400640 0 client.5331 ms_handle_reset on v2:172.21.15.110:6835/2457861782 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 1 -- 172.21.15.110:0/2357940253 shutdown_connections 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 1 -- 172.21.15.110:0/2357940253 wait complete. 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5434602000 20 client.5331 trim_cache size 0 max 16384 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T09:59:41.714 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 5214, 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 5.8478843500000002, 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 26, 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 45, 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 45, 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 10.731741270000001 2024-09-10T09:59:41.715 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T09:59:41.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 -- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f540c067aa0 msgr2=0x7f540c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f540c067aa0 0x7f540c069f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f542c0b31e0 tx=0x7f5418009040 comp rx=0 tx=0).stop 2024-09-10T09:59:41.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 -- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 msgr2=0x7f542c14ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f542c14ed50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f541c011a00 tx=0x7f541c011ec0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 -- 172.21.15.110:0/214204866 shutdown_connections 2024-09-10T09:59:41.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f540c067aa0 0x7f540c069f60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f542c152040 0x7f542c0b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f542c0b06b0 0x7f542c151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 --2- 172.21.15.110:0/214204866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5410006f60 0x7f542c14ed50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 -- 172.21.15.110:0/214204866 >> 172.21.15.110:0/214204866 conn(0x7f542c062d10 msgr2=0x7f542c0b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 -- 172.21.15.110:0/214204866 shutdown_connections 2024-09-10T09:59:41.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.706+0000 7f5431200640 1 -- 172.21.15.110:0/214204866 wait complete. 2024-09-10T09:59:41.729 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T09:59:41.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.858+0000 7f2227600640 1 Processor -- start 2024-09-10T09:59:41.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.858+0000 7f2227600640 1 -- start start 2024-09-10T09:59:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f2227600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f222015e690 0x7f222006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f2227600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222006c710 con 0x7f222006bc50 2024-09-10T09:59:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f2227600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22200b4680 con 0x7f22200aef50 2024-09-10T09:59:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f2227600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f22200b4960 con 0x7f222015e690 2024-09-10T09:59:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f221fe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f222006bc50 0x7f22200ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51772/0 (socket says 172.21.15.110:51772) 2024-09-10T09:59:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f221fe00640 1 -- 172.21.15.110:0/2060361699 learned_addr learned my addr 172.21.15.110:0/2060361699 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f2224c00640 1 --2- 172.21.15.110:0/2060361699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f222015e690 0x7f222006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f221f400640 1 -- 172.21.15.110:0/2060361699 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3008693225 0 0) 0x7f222006c710 con 0x7f222006bc50 2024-09-10T09:59:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f221f400640 1 -- 172.21.15.110:0/2060361699 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2204003530 con 0x7f222006bc50 2024-09-10T09:59:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f221f400640 1 -- 172.21.15.110:0/2060361699 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423529170 0 0) 0x7f22200b4680 con 0x7f22200aef50 2024-09-10T09:59:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f221f400640 1 -- 172.21.15.110:0/2060361699 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f222006c710 con 0x7f22200aef50 2024-09-10T09:59:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f2224c00640 1 -- 172.21.15.110:0/2060361699 >> v1:172.21.15.145:6789/0 conn(0x7f22200aef50 legacy=0x7f22200b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f2224c00640 1 -- 172.21.15.110:0/2060361699 >> v1:172.21.15.110:6789/0 conn(0x7f222006bc50 legacy=0x7f22200ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.862+0000 7f2224c00640 1 -- 172.21.15.110:0/2060361699 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22200b4c40 con 0x7f222015e690 2024-09-10T09:59:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2224c00640 1 --2- 172.21.15.110:0/2060361699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f222015e690 0x7f222006e240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22100097f0 tx=0x7f221001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f89ef0b13320bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f221f400640 1 -- 172.21.15.110:0/2060361699 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22100aa610 con 0x7f222015e690 2024-09-10T09:59:41.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f221f400640 1 -- 172.21.15.110:0/2060361699 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22100af070 con 0x7f222015e690 2024-09-10T09:59:41.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f221f400640 1 -- 172.21.15.110:0/2060361699 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22100aa9b0 con 0x7f222015e690 2024-09-10T09:59:41.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 -- 172.21.15.110:0/2060361699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f222015e690 msgr2=0x7f222006e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 --2- 172.21.15.110:0/2060361699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f222015e690 0x7f222006e240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22100097f0 tx=0x7f221001c4b0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 -- 172.21.15.110:0/2060361699 shutdown_connections 2024-09-10T09:59:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 --2- 172.21.15.110:0/2060361699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f222015e690 0x7f222006e240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 -- 172.21.15.110:0/2060361699 >> 172.21.15.110:0/2060361699 conn(0x7f22200d1530 msgr2=0x7f22200d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 -- 172.21.15.110:0/2060361699 shutdown_connections 2024-09-10T09:59:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 -- 172.21.15.110:0/2060361699 wait complete. 2024-09-10T09:59:41.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 Processor -- start 2024-09-10T09:59:41.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.866+0000 7f2227600640 1 -- start start 2024-09-10T09:59:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f222006bc50 0x7f222014a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f22200aef50 0x7f22200d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f222015e690 0x7f2220147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22200b0aa0 con 0x7f222015e690 2024-09-10T09:59:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f222006e780 con 0x7f222006bc50 2024-09-10T09:59:41.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f222015ea70 con 0x7f22200aef50 2024-09-10T09:59:41.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2225600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f222015e690 0x7f2220147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2224c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f222006bc50 0x7f222014a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2224c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f222006bc50 0x7f222014a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:39410/0 (socket says 172.21.15.110:39410) 2024-09-10T09:59:41.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2224c00640 1 -- 172.21.15.110:0/4032049023 learned_addr learned my addr 172.21.15.110:0/4032049023 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:41.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f221fe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f22200aef50 0x7f22200d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2225600640 1 -- 172.21.15.110:0/4032049023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f22200aef50 msgr2=0x7f22200d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2225600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f22200aef50 0x7f22200d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2225600640 1 -- 172.21.15.110:0/4032049023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f222006bc50 msgr2=0x7f222014a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:41.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2225600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f222006bc50 0x7f222014a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:41.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2225600640 1 -- 172.21.15.110:0/4032049023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2220150740 con 0x7f222015e690 2024-09-10T09:59:41.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2225600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f222015e690 0x7f2220147b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2220075c90 tx=0x7f221800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f221cc00640 1 -- 172.21.15.110:0/4032049023 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218017070 con 0x7f222015e690 2024-09-10T09:59:41.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f221cc00640 1 -- 172.21.15.110:0/4032049023 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22180052d0 con 0x7f222015e690 2024-09-10T09:59:41.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f221cc00640 1 -- 172.21.15.110:0/4032049023 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218003840 con 0x7f222015e690 2024-09-10T09:59:41.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22201509e0 con 0x7f222015e690 2024-09-10T09:59:41.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2220150fa0 con 0x7f222015e690 2024-09-10T09:59:41.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.870+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2220157c00 con 0x7f222015e690 2024-09-10T09:59:41.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.874+0000 7f221cc00640 1 -- 172.21.15.110:0/4032049023 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f221801a070 con 0x7f222015e690 2024-09-10T09:59:41.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.874+0000 7f221cc00640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21fc067aa0 0x7f21fc069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:41.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.874+0000 7f221cc00640 1 -- 172.21.15.110:0/4032049023 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2218085170 con 0x7f222015e690 2024-09-10T09:59:41.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.874+0000 7f2224c00640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21fc067aa0 0x7f21fc069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:41.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.874+0000 7f2224c00640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21fc067aa0 0x7f21fc069f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f221001a0e0 tx=0x7f22100a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:41.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:41.878+0000 7f221cc00640 1 -- 172.21.15.110:0/4032049023 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2218005480 con 0x7f222015e690 2024-09-10T09:59:42.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.070+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2220157920 con 0x7f222015e690 2024-09-10T09:59:42.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.070+0000 7f221cc00640 1 -- 172.21.15.110:0/4032049023 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f2218055480 con 0x7f222015e690 2024-09-10T09:59:42.076 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T09:59:42.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21fc067aa0 msgr2=0x7f21fc069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:42.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21fc067aa0 0x7f21fc069f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f221001a0e0 tx=0x7f22100a7000 comp rx=0 tx=0).stop 2024-09-10T09:59:42.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f222015e690 msgr2=0x7f2220147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:42.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f222015e690 0x7f2220147b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2220075c90 tx=0x7f221800c820 comp rx=0 tx=0).stop 2024-09-10T09:59:42.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 shutdown_connections 2024-09-10T09:59:42.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f21fc067aa0 0x7f21fc069f60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f222015e690 0x7f2220147b90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f22200aef50 0x7f22200d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 --2- 172.21.15.110:0/4032049023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f222006bc50 0x7f222014a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:42.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 >> 172.21.15.110:0/4032049023 conn(0x7f22200d1530 msgr2=0x7f22200ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:42.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 shutdown_connections 2024-09-10T09:59:42.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.074+0000 7f2227600640 1 -- 172.21.15.110:0/4032049023 wait complete. 2024-09-10T09:59:42.097 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T09:59:42.097 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:59:42.097 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T09:59:42.098 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T09:59:42.098 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:42.098 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T09:59:42.098 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T09:59:42.098 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:59:42.098 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:59:42.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:42.098 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T09:59:42.107 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:59:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:59:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:59:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T09:59:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:42.107 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42392sec preferred_lft 42392sec 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T09:59:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T09:59:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:42.110 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:59:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T09:59:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T09:59:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:42.110 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:59:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T09:59:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T09:59:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:42.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:42.111 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:59:42.174 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T09:59:42.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:42.176 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T09:59:42.227 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T09:59:42.227 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:59:42.227 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T09:59:42.228 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:59:42.228 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T09:59:42.228 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T09:59:42.228 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:59:42.228 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:42.228 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:42.228 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:42.228 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:59:42.228 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:59:42.228 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T09:59:42.228 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:59:42.228 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:59:42.229 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:59:42.229 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T09:59:42.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:59:42.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:42.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:42.366 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T09:59:42.422 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T09:59:42.423 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T09:59:42.423 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:59:42.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:42.423 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T09:59:42.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:59:42.484 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:42.532 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:59:42.533 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:59:42.533 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:42.584 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T09:59:42.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:42.584 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:42.672 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:42.672 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:59:42.685 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:42.685 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:59:42.685 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T09:59:42.685 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T09:59:42.685 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T09:59:42.689 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:59:42.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:42.689 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:42.708 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:59:42.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:59:42.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:42.813 INFO:teuthology.orchestra.run.smithi110.stdout:client5340 2024-09-10T09:59:42.814 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:42.814 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:42.823 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T09:59:43.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:42.998+0000 7f8c15000640 1 Processor -- start 2024-09-10T09:59:43.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c15000640 1 -- start start 2024-09-10T09:59:43.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c15000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8c10072270 0x7f8c1006bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:43.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c15000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c100cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:43.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c15000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c1006ea70 con 0x7f8c1015c570 2024-09-10T09:59:43.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c15000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1006c600 con 0x7f8c10072270 2024-09-10T09:59:43.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c15000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1015c950 con 0x7f8c1006c190 2024-09-10T09:59:43.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0de00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f8c1015c570 0x7f8c1006e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51784/0 (socket says 172.21.15.110:51784) 2024-09-10T09:59:43.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0de00640 1 -- 172.21.15.110:0/1734740026 learned_addr learned my addr 172.21.15.110:0/1734740026 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:43.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c07e00640 1 --2- 172.21.15.110:0/1734740026 >> v2:172.21.15.145:3300/0 conn(0x7f8c10072270 0x7f8c1006bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:43.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0e800640 1 --2- 172.21.15.110:0/1734740026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c100cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:43.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0d400640 1 -- 172.21.15.110:0/1734740026 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121355850 0 0) 0x7f8c1006ea70 con 0x7f8c1015c570 2024-09-10T09:59:43.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0d400640 1 -- 172.21.15.110:0/1734740026 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bf4003530 con 0x7f8c1015c570 2024-09-10T09:59:43.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0e800640 1 -- 172.21.15.110:0/1734740026 >> v2:172.21.15.145:3300/0 conn(0x7f8c10072270 msgr2=0x7f8c1006bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:43.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0e800640 1 --2- 172.21.15.110:0/1734740026 >> v2:172.21.15.145:3300/0 conn(0x7f8c10072270 0x7f8c1006bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0e800640 1 -- 172.21.15.110:0/1734740026 >> v1:172.21.15.110:6789/0 conn(0x7f8c1015c570 legacy=0x7f8c1006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:43.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c0e800640 1 -- 172.21.15.110:0/1734740026 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1006ed50 con 0x7f8c1006c190 2024-09-10T09:59:43.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.002+0000 7f8c07e00640 1 --2- 172.21.15.110:0/1734740026 >> v2:172.21.15.145:3300/0 conn(0x7f8c10072270 0x7f8c1006bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:43.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c0e800640 1 --2- 172.21.15.110:0/1734740026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c100cf610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009d90 tx=0x7f8c0801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63da4cad9258a152 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:43.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c0d400640 1 -- 172.21.15.110:0/1734740026 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c080b2070 con 0x7f8c1006c190 2024-09-10T09:59:43.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c0d400640 1 -- 172.21.15.110:0/1734740026 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c080a5890 con 0x7f8c1006c190 2024-09-10T09:59:43.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c0d400640 1 -- 172.21.15.110:0/1734740026 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c080a5c60 con 0x7f8c1006c190 2024-09-10T09:59:43.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c15000640 1 -- 172.21.15.110:0/1734740026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 msgr2=0x7f8c100cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:43.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c15000640 1 --2- 172.21.15.110:0/1734740026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c100cf610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009d90 tx=0x7f8c0801c8e0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c15000640 1 -- 172.21.15.110:0/1734740026 shutdown_connections 2024-09-10T09:59:43.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c15000640 1 --2- 172.21.15.110:0/1734740026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c100cf610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c15000640 1 --2- 172.21.15.110:0/1734740026 >> v2:172.21.15.145:3300/0 conn(0x7f8c10072270 0x7f8c1006bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c15000640 1 -- 172.21.15.110:0/1734740026 >> 172.21.15.110:0/1734740026 conn(0x7f8c100d1530 msgr2=0x7f8c100d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:43.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c15000640 1 -- 172.21.15.110:0/1734740026 shutdown_connections 2024-09-10T09:59:43.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.006+0000 7f8c15000640 1 -- 172.21.15.110:0/1734740026 wait complete. 2024-09-10T09:59:43.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 Processor -- start 2024-09-10T09:59:43.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 -- start start 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c10147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c10072270 0x7f8c100ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c1015c570 0x7f8c1014e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c100d0360 con 0x7f8c10072270 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10070b20 con 0x7f8c1015c570 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8c100d04e0 con 0x7f8c1006c190 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c10072270 0x7f8c100ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c10072270 0x7f8c100ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46974/0 (socket says 172.21.15.110:46974) 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 -- 172.21.15.110:0/3681424990 learned_addr learned my addr 172.21.15.110:0/3681424990 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c0de00640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c10147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:43.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c0e800640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c1015c570 0x7f8c1014e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 -- 172.21.15.110:0/3681424990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 msgr2=0x7f8c10147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c10147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 -- 172.21.15.110:0/3681424990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c1015c570 msgr2=0x7f8c1014e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c1015c570 0x7f8c1014e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 -- 172.21.15.110:0/3681424990 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c100af140 con 0x7f8c10072270 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c07e00640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c10072270 0x7f8c100ae840 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800ebb0 tx=0x7f8bf800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c06000640 1 -- 172.21.15.110:0/3681424990 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8017070 con 0x7f8c10072270 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c06000640 1 -- 172.21.15.110:0/3681424990 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf800cc30 con 0x7f8c10072270 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c100af420 con 0x7f8c10072270 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c06000640 1 -- 172.21.15.110:0/3681424990 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8013470 con 0x7f8c10072270 2024-09-10T09:59:43.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c100af9e0 con 0x7f8c10072270 2024-09-10T09:59:43.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.010+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c100697d0 con 0x7f8c10072270 2024-09-10T09:59:43.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.014+0000 7f8c06000640 1 -- 172.21.15.110:0/3681424990 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f8bf801a070 con 0x7f8c10072270 2024-09-10T09:59:43.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.014+0000 7f8c06000640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8bec067aa0 0x7f8bec069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:43.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.014+0000 7f8c06000640 1 -- 172.21.15.110:0/3681424990 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8bf80862b0 con 0x7f8c10072270 2024-09-10T09:59:43.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.014+0000 7f8c0de00640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8bec067aa0 0x7f8bec069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:43.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.014+0000 7f8c0de00640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8bec067aa0 0x7f8bec069f60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005890 tx=0x7f8c00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:43.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.018+0000 7f8c06000640 1 -- 172.21.15.110:0/3681424990 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8bf8056b60 con 0x7f8c10072270 2024-09-10T09:59:43.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8c100cf610 con 0x7f8c10072270 2024-09-10T09:59:43.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c06000640 1 -- 172.21.15.110:0/3681424990 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f8bf8056460 con 0x7f8c10072270 2024-09-10T09:59:43.215 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8bec067aa0 msgr2=0x7f8bec069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8bec067aa0 0x7f8bec069f60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005890 tx=0x7f8c00009040 comp rx=0 tx=0).stop 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c10072270 msgr2=0x7f8c100ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c10072270 0x7f8c100ae840 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800ebb0 tx=0x7f8bf800c520 comp rx=0 tx=0).stop 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 shutdown_connections 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8bec067aa0 0x7f8bec069f60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c1015c570 0x7f8c1014e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c10072270 0x7f8c100ae840 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 --2- 172.21.15.110:0/3681424990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c1006c190 0x7f8c10147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:43.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 >> 172.21.15.110:0/3681424990 conn(0x7f8c100d1530 msgr2=0x7f8c10072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:43.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 shutdown_connections 2024-09-10T09:59:43.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:43.210+0000 7f8c15000640 1 -- 172.21.15.110:0/3681424990 wait complete. 2024-09-10T09:59:43.236 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T09:59:43.237 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:59:43.237 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T09:59:43.237 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T09:59:43.237 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:43.237 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T09:59:43.237 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T09:59:43.237 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:59:43.237 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T09:59:43.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:43.237 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T09:59:43.247 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:43.248 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42388sec preferred_lft 42388sec 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:43.249 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T09:59:43.250 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:59:43.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:43.251 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:59:43.313 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T09:59:43.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:43.316 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T09:59:43.375 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:59:43.376 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:59:43.376 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T09:59:43.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T09:59:43.376 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T09:59:43.377 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T09:59:43.377 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T09:59:43.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:59:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:59:43.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:59:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:43.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:43.520 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T09:59:43.529 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T09:59:43.529 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T09:59:43.529 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T09:59:43.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:43.529 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T09:59:43.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:59:43.593 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:43.642 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T09:59:43.642 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:59:43.642 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:43.693 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T09:59:43.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:43.694 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:43.786 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:43.786 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:59:43.800 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T09:59:43.801 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T09:59:43.801 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T09:59:43.801 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T09:59:43.801 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T09:59:43.805 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:59:43.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:43.805 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:43.870 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:59:43.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:59:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:59:43.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T09:59:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:59:43.971 INFO:teuthology.orchestra.run.smithi145.stdout:client4946 2024-09-10T09:59:43.971 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:43.971 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:43.980 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T09:59:44.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10dac00640 1 Processor -- start 2024-09-10T09:59:44.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10dac00640 1 -- start start 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10dac00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 0x7f10d4070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10dac00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f10d415e880 0x7f10d40b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10dac00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10d40d1f40 0x7f10d406e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10dac00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f10d40d1d10 con 0x7f10d415e880 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10dac00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f10d4071510 con 0x7f10d40d1f40 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10dac00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f10d40717f0 con 0x7f10d415e4a0 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f10d415e880 0x7f10d40b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f10d415e880 0x7f10d40b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46990/0 (socket says 172.21.15.110:46990) 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3400640 1 -- 172.21.15.110:0/3220113935 learned_addr learned my addr 172.21.15.110:0/3220113935 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d8c00640 1 --2- 172.21.15.110:0/3220113935 >> v2:172.21.15.145:3300/0 conn(0x7f10d40d1f40 0x7f10d406e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3e00640 1 --2- 172.21.15.110:0/3220113935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 0x7f10d4070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:44.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3400640 1 -- 172.21.15.110:0/3220113935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 msgr2=0x7f10d4070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:44.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3400640 1 --2- 172.21.15.110:0/3220113935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 0x7f10d4070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3400640 1 -- 172.21.15.110:0/3220113935 >> v2:172.21.15.145:3300/0 conn(0x7f10d40d1f40 msgr2=0x7f10d406e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:44.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3400640 1 --2- 172.21.15.110:0/3220113935 >> v2:172.21.15.145:3300/0 conn(0x7f10d40d1f40 0x7f10d406e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.118+0000 7f10d3400640 1 -- 172.21.15.110:0/3220113935 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d4071ad0 con 0x7f10d415e880 2024-09-10T09:59:44.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10d3400640 1 --2- 172.21.15.110:0/3220113935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d415e880 0x7f10d40b75c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f10d40b5a30 tx=0x7f10c001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d0baf60603b24e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10d2a00640 1 -- 172.21.15.110:0/3220113935 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c00b1070 con 0x7f10d415e880 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10d2a00640 1 -- 172.21.15.110:0/3220113935 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c00ac620 con 0x7f10d415e880 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10d2a00640 1 -- 172.21.15.110:0/3220113935 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c00ac9f0 con 0x7f10d415e880 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 -- 172.21.15.110:0/3220113935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d415e880 msgr2=0x7f10d40b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 --2- 172.21.15.110:0/3220113935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d415e880 0x7f10d40b75c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f10d40b5a30 tx=0x7f10c001c4f0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 -- 172.21.15.110:0/3220113935 shutdown_connections 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 --2- 172.21.15.110:0/3220113935 >> v2:172.21.15.145:3300/0 conn(0x7f10d40d1f40 0x7f10d406e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 --2- 172.21.15.110:0/3220113935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d415e880 0x7f10d40b75c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 --2- 172.21.15.110:0/3220113935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 0x7f10d4070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 -- 172.21.15.110:0/3220113935 >> 172.21.15.110:0/3220113935 conn(0x7f10d406f400 msgr2=0x7f10d406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 -- 172.21.15.110:0/3220113935 shutdown_connections 2024-09-10T09:59:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.122+0000 7f10dac00640 1 -- 172.21.15.110:0/3220113935 wait complete. 2024-09-10T09:59:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 Processor -- start 2024-09-10T09:59:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 -- start start 2024-09-10T09:59:44.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d40d1f40 0x7f10d414f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:44.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 0x7f10d415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:44.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d415e880 0x7f10d4156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:44.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10d406d6f0 con 0x7f10d40d1f40 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10d406d570 con 0x7f10d415e880 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f10d406d870 con 0x7f10d415e4a0 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d40d1f40 0x7f10d414f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d40d1f40 0x7f10d414f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46996/0 (socket says 172.21.15.110:46996) 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 -- 172.21.15.110:0/629486339 learned_addr learned my addr 172.21.15.110:0/629486339 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3400640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 0x7f10d415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d8c00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d415e880 0x7f10d4156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 -- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 msgr2=0x7f10d415d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 0x7f10d415d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 -- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d415e880 msgr2=0x7f10d4156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d415e880 0x7f10d4156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 -- 172.21.15.110:0/629486339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d414fc10 con 0x7f10d40d1f40 2024-09-10T09:59:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d8c00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d415e880 0x7f10d4156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10d3e00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d40d1f40 0x7f10d414f6d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f10c400e9b0 tx=0x7f10c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10cbe00640 1 -- 172.21.15.110:0/629486339 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4017070 con 0x7f10d40d1f40 2024-09-10T09:59:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10cbe00640 1 -- 172.21.15.110:0/629486339 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c400c9a0 con 0x7f10d40d1f40 2024-09-10T09:59:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10cbe00640 1 -- 172.21.15.110:0/629486339 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c400cd70 con 0x7f10d40d1f40 2024-09-10T09:59:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d415bea0 con 0x7f10d40d1f40 2024-09-10T09:59:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.126+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d415c6e0 con 0x7f10d40d1f40 2024-09-10T09:59:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.130+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d415c180 con 0x7f10d40d1f40 2024-09-10T09:59:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.130+0000 7f10cbe00640 1 -- 172.21.15.110:0/629486339 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f10c401a070 con 0x7f10d40d1f40 2024-09-10T09:59:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.130+0000 7f10cbe00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f10b4067a70 0x7f10b4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.130+0000 7f10cbe00640 1 -- 172.21.15.110:0/629486339 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f10c40861f0 con 0x7f10d40d1f40 2024-09-10T09:59:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.130+0000 7f10d3400640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f10b4067a70 0x7f10b4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.130+0000 7f10d3400640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f10b4067a70 0x7f10b4069f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f10c00ae000 tx=0x7f10c00a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.134+0000 7f10cbe00640 1 -- 172.21.15.110:0/629486339 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f10d415c180 con 0x7f10d40d1f40 2024-09-10T09:59:44.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.326+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f10d40b1900 con 0x7f10d40d1f40 2024-09-10T09:59:44.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.898+0000 7f10cbe00640 1 -- 172.21.15.110:0/629486339 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7f10c4021030 con 0x7f10d40d1f40 2024-09-10T09:59:44.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f10b4067a70 msgr2=0x7f10b4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:44.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f10b4067a70 0x7f10b4069f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f10c00ae000 tx=0x7f10c00a7000 comp rx=0 tx=0).stop 2024-09-10T09:59:44.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d40d1f40 msgr2=0x7f10d414f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:44.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d40d1f40 0x7f10d414f6d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f10c400e9b0 tx=0x7f10c400ee70 comp rx=0 tx=0).stop 2024-09-10T09:59:44.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 shutdown_connections 2024-09-10T09:59:44.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f10b4067a70 0x7f10b4069f30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d415e880 0x7f10d4156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10d415e4a0 0x7f10d415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 --2- 172.21.15.110:0/629486339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10d40d1f40 0x7f10d414f6d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 >> 172.21.15.110:0/629486339 conn(0x7f10d406f400 msgr2=0x7f10d4157790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 shutdown_connections 2024-09-10T09:59:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:44.902+0000 7f10dac00640 1 -- 172.21.15.110:0/629486339 wait complete. 2024-09-10T09:59:44.928 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T09:59:45.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.062+0000 7fc722400640 1 Processor -- start 2024-09-10T09:59:45.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.062+0000 7fc722400640 1 -- start start 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.062+0000 7fc722400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc71c0cfb90 0x7fc71c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc722400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc722400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc71c0cff70 0x7fc71c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc722400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc71c06d130 con 0x7fc71c0cff70 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc722400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc71c15d4d0 con 0x7fc71c0cfb90 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc722400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c15d7b0 con 0x7fc71c0b3f40 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc71c0cff70 0x7fc71c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc71c0cff70 0x7fc71c0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46998/0 (socket says 172.21.15.110:46998) 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 -- 172.21.15.110:0/1269311712 learned_addr learned my addr 172.21.15.110:0/1269311712 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71b400640 1 --2- 172.21.15.110:0/1269311712 >> v2:172.21.15.145:3300/0 conn(0x7fc71c0cfb90 0x7fc71c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71aa00640 1 --2- 172.21.15.110:0/1269311712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 -- 172.21.15.110:0/1269311712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 msgr2=0x7fc71c0cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:45.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 --2- 172.21.15.110:0/1269311712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c0cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 -- 172.21.15.110:0/1269311712 >> v2:172.21.15.145:3300/0 conn(0x7fc71c0cfb90 msgr2=0x7fc71c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:45.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 --2- 172.21.15.110:0/1269311712 >> v2:172.21.15.145:3300/0 conn(0x7fc71c0cfb90 0x7fc71c15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 -- 172.21.15.110:0/1269311712 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c06d380 con 0x7fc71c0cff70 2024-09-10T09:59:45.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.066+0000 7fc71be00640 1 --2- 172.21.15.110:0/1269311712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cff70 0x7fc71c0cdae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc71c073450 tx=0x7fc71401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e91864c2dcb0d28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71a000640 1 -- 172.21.15.110:0/1269311712 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140b0070 con 0x7fc71c0cff70 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71a000640 1 -- 172.21.15.110:0/1269311712 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7140ab600 con 0x7fc71c0cff70 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71a000640 1 -- 172.21.15.110:0/1269311712 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140ab9d0 con 0x7fc71c0cff70 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- 172.21.15.110:0/1269311712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cff70 msgr2=0x7fc71c0cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 --2- 172.21.15.110:0/1269311712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cff70 0x7fc71c0cdae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc71c073450 tx=0x7fc71401c490 comp rx=0 tx=0).stop 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- 172.21.15.110:0/1269311712 shutdown_connections 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 --2- 172.21.15.110:0/1269311712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cff70 0x7fc71c0cdae0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 --2- 172.21.15.110:0/1269311712 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 --2- 172.21.15.110:0/1269311712 >> v2:172.21.15.145:3300/0 conn(0x7fc71c0cfb90 0x7fc71c15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- 172.21.15.110:0/1269311712 >> 172.21.15.110:0/1269311712 conn(0x7fc71c062d10 msgr2=0x7fc71c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:45.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- 172.21.15.110:0/1269311712 shutdown_connections 2024-09-10T09:59:45.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- 172.21.15.110:0/1269311712 wait complete. 2024-09-10T09:59:45.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 Processor -- start 2024-09-10T09:59:45.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- start start 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c1649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cfb90 0x7fc71c150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c0cff70 0x7fc71c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c160ba0 con 0x7fc71c0cfb90 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c160a20 con 0x7fc71c0cff70 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc722400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c160d20 con 0x7fc71c0b3f40 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cfb90 0x7fc71c150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cfb90 0x7fc71c150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47010/0 (socket says 172.21.15.110:47010) 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71aa00640 1 -- 172.21.15.110:0/3388852240 learned_addr learned my addr 172.21.15.110:0/3388852240 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:45.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71be00640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c0cff70 0x7fc71c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71b400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c1649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71aa00640 1 -- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 msgr2=0x7fc71c1649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.070+0000 7fc71aa00640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c1649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc71aa00640 1 -- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c0cff70 msgr2=0x7fc71c15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc71aa00640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c0cff70 0x7fc71c15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc71aa00640 1 -- 172.21.15.110:0/3388852240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c151140 con 0x7fc71c0cfb90 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc71b400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c1649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc71aa00640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cfb90 0x7fc71c150c00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc70800e870 tx=0x7fc70800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc713400640 1 -- 172.21.15.110:0/3388852240 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708017070 con 0x7fc71c0cfb90 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc713400640 1 -- 172.21.15.110:0/3388852240 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70800ee60 con 0x7fc71c0cfb90 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc713400640 1 -- 172.21.15.110:0/3388852240 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70800cb00 con 0x7fc71c0cfb90 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c151410 con 0x7fc71c0cfb90 2024-09-10T09:59:45.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c161ef0 con 0x7fc71c0cfb90 2024-09-10T09:59:45.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c15ca10 con 0x7fc71c0cfb90 2024-09-10T09:59:45.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc713400640 1 -- 172.21.15.110:0/3388852240 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fc70801a070 con 0x7fc71c0cfb90 2024-09-10T09:59:45.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc713400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc6fc067a70 0x7fc6fc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc713400640 1 -- 172.21.15.110:0/3388852240 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc708085d40 con 0x7fc71c0cfb90 2024-09-10T09:59:45.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.074+0000 7fc71b400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc6fc067a70 0x7fc6fc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.078+0000 7fc71b400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc6fc067a70 0x7fc6fc069f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc71c1642f0 tx=0x7fc70c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.078+0000 7fc713400640 1 -- 172.21.15.110:0/3388852240 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc708055fa0 con 0x7fc71c0cfb90 2024-09-10T09:59:45.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.258+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc71c0b3b00 con 0x7fc6fc067a70 2024-09-10T09:59:45.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.258+0000 7fc713400640 1 -- 172.21.15.110:0/3388852240 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc71c0b3b00 con 0x7fc6fc067a70 2024-09-10T09:59:45.265 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T09:59:45.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T09:59:45.265 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T09:59:45.265 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T09:59:45.265 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T09:59:45.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc6fc067a70 msgr2=0x7fc6fc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc6fc067a70 0x7fc6fc069f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc71c1642f0 tx=0x7fc70c00b040 comp rx=0 tx=0).stop 2024-09-10T09:59:45.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cfb90 msgr2=0x7fc71c150c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cfb90 0x7fc71c150c00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc70800e870 tx=0x7fc70800ed30 comp rx=0 tx=0).stop 2024-09-10T09:59:45.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 shutdown_connections 2024-09-10T09:59:45.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc6fc067a70 0x7fc6fc069f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c0cff70 0x7fc71c15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71c0cfb90 0x7fc71c150c00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 --2- 172.21.15.110:0/3388852240 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc71c0b3f40 0x7fc71c1649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 >> 172.21.15.110:0/3388852240 conn(0x7fc71c062d10 msgr2=0x7fc71c15f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:45.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 shutdown_connections 2024-09-10T09:59:45.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.262+0000 7fc722400640 1 -- 172.21.15.110:0/3388852240 wait complete. 2024-09-10T09:59:45.286 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T09:59:45.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045f400640 1 Processor -- start 2024-09-10T09:59:45.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045f400640 1 -- start start 2024-09-10T09:59:45.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f04580cfce0 0x7f045815f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 0x7f045806e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045f400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04580b1e90 con 0x7f04580ae840 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045f400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f045806ead0 con 0x7f04580cfce0 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045f400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04580d00c0 con 0x7f045806e140 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f0457e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f04580ae840 0x7f04580b3b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51800/0 (socket says 172.21.15.110:51800) 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f0457e00640 1 -- 172.21.15.110:0/4161716928 learned_addr learned my addr 172.21.15.110:0/4161716928 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045ca00640 1 --2- 172.21.15.110:0/4161716928 >> v2:172.21.15.145:3300/0 conn(0x7f04580cfce0 0x7f045815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045d400640 1 --2- 172.21.15.110:0/4161716928 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 0x7f045806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f0457400640 1 -- 172.21.15.110:0/4161716928 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3324236485 0 0) 0x7f04580b1e90 con 0x7f04580ae840 2024-09-10T09:59:45.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f0457400640 1 -- 172.21.15.110:0/4161716928 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0440003530 con 0x7f04580ae840 2024-09-10T09:59:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045ca00640 1 -- 172.21.15.110:0/4161716928 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 msgr2=0x7f045806e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045ca00640 1 --2- 172.21.15.110:0/4161716928 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 0x7f045806e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f0457400640 1 -- 172.21.15.110:0/4161716928 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2300271257 0 0) 0x7f0440003530 con 0x7f04580ae840 2024-09-10T09:59:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045ca00640 1 -- 172.21.15.110:0/4161716928 >> v1:172.21.15.110:6789/0 conn(0x7f04580ae840 legacy=0x7f04580b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.422+0000 7f045ca00640 1 -- 172.21.15.110:0/4161716928 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04580b43d0 con 0x7f04580cfce0 2024-09-10T09:59:45.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045ca00640 1 --2- 172.21.15.110:0/4161716928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 0x7f045815f870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f044c0097c0 tx=0x7f044c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3471cc236489ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f0457400640 1 -- 172.21.15.110:0/4161716928 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c0b0070 con 0x7f04580cfce0 2024-09-10T09:59:45.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f0457400640 1 -- 172.21.15.110:0/4161716928 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c0ab400 con 0x7f04580cfce0 2024-09-10T09:59:45.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f0457400640 1 -- 172.21.15.110:0/4161716928 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444001680 con 0x7f04580cfce0 2024-09-10T09:59:45.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045f400640 1 -- 172.21.15.110:0/4161716928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 msgr2=0x7f045815f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045f400640 1 --2- 172.21.15.110:0/4161716928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 0x7f045815f870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f044c0097c0 tx=0x7f044c01c3c0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045f400640 1 -- 172.21.15.110:0/4161716928 shutdown_connections 2024-09-10T09:59:45.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045f400640 1 --2- 172.21.15.110:0/4161716928 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 0x7f045806e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045f400640 1 --2- 172.21.15.110:0/4161716928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 0x7f045815f870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045f400640 1 -- 172.21.15.110:0/4161716928 >> 172.21.15.110:0/4161716928 conn(0x7f0458062e40 msgr2=0x7f045806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:45.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045f400640 1 -- 172.21.15.110:0/4161716928 shutdown_connections 2024-09-10T09:59:45.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.426+0000 7f045f400640 1 -- 172.21.15.110:0/4161716928 wait complete. 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 Processor -- start 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 -- start start 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 0x7f0458158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04580ae840 0x7f0458157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 0x7f04580af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04580b2fe0 con 0x7f04580ae840 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04580b2e60 con 0x7f04580cfce0 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04580b3160 con 0x7f045806e140 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04580ae840 0x7f0458157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04580ae840 0x7f0458157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47012/0 (socket says 172.21.15.110:47012) 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 -- 172.21.15.110:0/378555730 learned_addr learned my addr 172.21.15.110:0/378555730 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:45.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045d400640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 0x7f04580af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045ca00640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 0x7f0458158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 -- 172.21.15.110:0/378555730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 msgr2=0x7f0458158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 0x7f0458158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 -- 172.21.15.110:0/378555730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 msgr2=0x7f04580af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 0x7f04580af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 -- 172.21.15.110:0/378555730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04581582b0 con 0x7f04580ae840 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0457e00640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04580ae840 0x7f0458157d70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f04440017f0 tx=0x7f044400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0454c00640 1 -- 172.21.15.110:0/378555730 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444005ae0 con 0x7f04580ae840 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0454c00640 1 -- 172.21.15.110:0/378555730 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0444005ca0 con 0x7f04580ae840 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0458159690 con 0x7f04580ae840 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f0454c00640 1 -- 172.21.15.110:0/378555730 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04440072b0 con 0x7f04580ae840 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0458157850 con 0x7f04580ae840 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.430+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04580b3960 con 0x7f04580ae840 2024-09-10T09:59:45.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.434+0000 7f0454c00640 1 -- 172.21.15.110:0/378555730 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f0444007450 con 0x7f04580ae840 2024-09-10T09:59:45.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.434+0000 7f0454c00640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0438067aa0 0x7f0438069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.434+0000 7f0454c00640 1 -- 172.21.15.110:0/378555730 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f0444085c10 con 0x7f04580ae840 2024-09-10T09:59:45.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.434+0000 7f045ca00640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0438067aa0 0x7f0438069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.434+0000 7f045ca00640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0438067aa0 0x7f0438069f60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f044c0ad000 tx=0x7f044c0a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.438+0000 7f0454c00640 1 -- 172.21.15.110:0/378555730 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0444055df0 con 0x7f04580ae840 2024-09-10T09:59:45.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.602+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f04580d0270 con 0x7f04580ae840 2024-09-10T09:59:45.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.602+0000 7f0454c00640 1 -- 172.21.15.110:0/378555730 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f044405a0f0 con 0x7f04580ae840 2024-09-10T09:59:45.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0438067aa0 msgr2=0x7f0438069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0438067aa0 0x7f0438069f60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f044c0ad000 tx=0x7f044c0a6000 comp rx=0 tx=0).stop 2024-09-10T09:59:45.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04580ae840 msgr2=0x7f0458157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04580ae840 0x7f0458157d70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f04440017f0 tx=0x7f044400cbe0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 shutdown_connections 2024-09-10T09:59:45.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0438067aa0 0x7f0438069f60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04580cfce0 0x7f04580af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04580ae840 0x7f0458157d70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 --2- 172.21.15.110:0/378555730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045806e140 0x7f0458158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 >> 172.21.15.110:0/378555730 conn(0x7f0458062e40 msgr2=0x7f0458071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 shutdown_connections 2024-09-10T09:59:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.606+0000 7f045f400640 1 -- 172.21.15.110:0/378555730 wait complete. 2024-09-10T09:59:45.629 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:45.629 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:45.638 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T09:59:45.638 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:45.689 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T09:59:45.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.862+0000 7f4656c00640 1 Processor -- start 2024-09-10T09:59:45.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.862+0000 7f4656c00640 1 -- start start 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4656c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f46500b5600 0x7f46500cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4656c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f465006efa0 0x7f46500b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4656c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f465006f3e0 con 0x7f46500726e0 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4656c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f46500cee50 con 0x7f46500b5600 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4656c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f465015fda0 con 0x7f465006efa0 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4647e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f46500b5600 0x7f46500cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4647e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f46500b5600 0x7f46500cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47022/0 (socket says 172.21.15.110:47022) 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4647e00640 1 -- 172.21.15.110:0/2154261822 learned_addr learned my addr 172.21.15.110:0/2154261822 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:45.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4654c00640 1 --2- 172.21.15.110:0/2154261822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f465006efa0 0x7f46500b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4647e00640 1 -- 172.21.15.110:0/2154261822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f465006efa0 msgr2=0x7f46500b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:45.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4647e00640 1 --2- 172.21.15.110:0/2154261822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f465006efa0 0x7f46500b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4647e00640 1 -- 172.21.15.110:0/2154261822 >> v1:172.21.15.145:6789/0 conn(0x7f46500726e0 legacy=0x7f46500b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:45.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4647e00640 1 -- 172.21.15.110:0/2154261822 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46500ce520 con 0x7f46500b5600 2024-09-10T09:59:45.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.866+0000 7f4654c00640 1 --2- 172.21.15.110:0/2154261822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f465006efa0 0x7f46500b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4647e00640 1 --2- 172.21.15.110:0/2154261822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46500b5600 0x7f46500cf2e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4638009870 tx=0x7f463801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41f81bf308345cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f464f400640 1 -- 172.21.15.110:0/2154261822 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380aa610 con 0x7f46500b5600 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f464f400640 1 -- 172.21.15.110:0/2154261822 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46380af070 con 0x7f46500b5600 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f464f400640 1 -- 172.21.15.110:0/2154261822 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380aa9d0 con 0x7f46500b5600 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 -- 172.21.15.110:0/2154261822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46500b5600 msgr2=0x7f46500cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 --2- 172.21.15.110:0/2154261822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46500b5600 0x7f46500cf2e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4638009870 tx=0x7f463801c4f0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 -- 172.21.15.110:0/2154261822 shutdown_connections 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 --2- 172.21.15.110:0/2154261822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f465006efa0 0x7f46500b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 --2- 172.21.15.110:0/2154261822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46500b5600 0x7f46500cf2e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 -- 172.21.15.110:0/2154261822 >> 172.21.15.110:0/2154261822 conn(0x7f4650062d10 msgr2=0x7f465006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 -- 172.21.15.110:0/2154261822 shutdown_connections 2024-09-10T09:59:45.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 -- 172.21.15.110:0/2154261822 wait complete. 2024-09-10T09:59:45.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 Processor -- start 2024-09-10T09:59:45.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 -- start start 2024-09-10T09:59:45.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.870+0000 7f4656c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f465006efa0 0x7f46501548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4656c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46500726e0 0x7f465015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4656c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46500b5600 0x7f465015d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4656c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4650072c40 con 0x7f465006efa0 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4656c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4650072ac0 con 0x7f46500b5600 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4656c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f465015f950 con 0x7f46500726e0 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f465006efa0 0x7f46501548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f465006efa0 0x7f46501548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47024/0 (socket says 172.21.15.110:47024) 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 -- 172.21.15.110:0/1655043071 learned_addr learned my addr 172.21.15.110:0/1655043071 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4647e00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46500726e0 0x7f465015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4654c00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46500b5600 0x7f465015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 -- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46500726e0 msgr2=0x7f465015d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46500726e0 0x7f465015d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 -- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46500b5600 msgr2=0x7f465015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46500b5600 0x7f465015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 -- 172.21.15.110:0/1655043071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4650154e30 con 0x7f465006efa0 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4647e00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46500726e0 0x7f465015d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464fe00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f465006efa0 0x7f46501548f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f464000c450 tx=0x7f464000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464cc00640 1 -- 172.21.15.110:0/1655043071 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640012830 con 0x7f465006efa0 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464cc00640 1 -- 172.21.15.110:0/1655043071 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4640017070 con 0x7f465006efa0 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464cc00640 1 -- 172.21.15.110:0/1655043071 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640012c30 con 0x7f465006efa0 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4650153870 con 0x7f465006efa0 2024-09-10T09:59:45.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46501540f0 con 0x7f465006efa0 2024-09-10T09:59:45.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46500b34d0 con 0x7f465006efa0 2024-09-10T09:59:45.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.874+0000 7f464cc00640 1 -- 172.21.15.110:0/1655043071 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f464001c460 con 0x7f465006efa0 2024-09-10T09:59:45.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.878+0000 7f464cc00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f462c067a70 0x7f462c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:45.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.878+0000 7f464cc00640 1 -- 172.21.15.110:0/1655043071 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f4640085e70 con 0x7f465006efa0 2024-09-10T09:59:45.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.878+0000 7f4647e00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f462c067a70 0x7f462c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:45.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.878+0000 7f4647e00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f462c067a70 0x7f462c069f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f46380ae000 tx=0x7f46380a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:45.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:45.882+0000 7f464cc00640 1 -- 172.21.15.110:0/1655043071 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4640056690 con 0x7f465006efa0 2024-09-10T09:59:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.050+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f46500b4f90 con 0x7f465006efa0 2024-09-10T09:59:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.050+0000 7f464cc00640 1 -- 172.21.15.110:0/1655043071 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f4640055f90 con 0x7f465006efa0 2024-09-10T09:59:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T09:59:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f462c067a70 msgr2=0x7f462c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f462c067a70 0x7f462c069f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f46380ae000 tx=0x7f46380a7000 comp rx=0 tx=0).stop 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f465006efa0 msgr2=0x7f46501548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f465006efa0 0x7f46501548f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f464000c450 tx=0x7f464000c910 comp rx=0 tx=0).stop 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 shutdown_connections 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f462c067a70 0x7f462c069f30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46500b5600 0x7f465015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46500726e0 0x7f465015d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 --2- 172.21.15.110:0/1655043071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f465006efa0 0x7f46501548f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 >> 172.21.15.110:0/1655043071 conn(0x7f4650062d10 msgr2=0x7f465015c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 shutdown_connections 2024-09-10T09:59:46.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.054+0000 7f4656c00640 1 -- 172.21.15.110:0/1655043071 wait complete. 2024-09-10T09:59:46.078 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T09:59:46.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:46.078 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T09:59:46.103 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T09:59:46.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:46.103 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T09:59:46.154 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:46.154 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:46.163 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T09:59:46.163 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:59:46.214 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T09:59:46.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbce400640 1 Processor -- start 2024-09-10T09:59:46.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbce400640 1 -- start start 2024-09-10T09:59:46.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbce400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcbc80b6f90 0x7fcbc8162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbce400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbc80b55d0 0x7fcbc80b7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbce400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc80b78b0 con 0x7fcbc80715f0 2024-09-10T09:59:46.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbce400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc81633d0 con 0x7fcbc80b6f90 2024-09-10T09:59:46.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbce400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc806e7b0 con 0x7fcbc80b55d0 2024-09-10T09:59:46.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcbc80715f0 0x7fcbc815ef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51812/0 (socket says 172.21.15.110:51812) 2024-09-10T09:59:46.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7e00640 1 -- 172.21.15.110:0/1341906745 learned_addr learned my addr 172.21.15.110:0/1341906745 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:46.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7400640 1 --2- 172.21.15.110:0/1341906745 >> v2:172.21.15.145:3300/0 conn(0x7fcbc80b6f90 0x7fcbc8162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbbfe00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbc80b55d0 0x7fcbc80b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 822008637 0 0) 0x7fcbc80b78b0 con 0x7fcbc80715f0 2024-09-10T09:59:46.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbac003530 con 0x7fcbc80715f0 2024-09-10T09:59:46.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2381459276 0 0) 0x7fcbac003530 con 0x7fcbc80715f0 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbc80b55d0 msgr2=0x7fcbc80b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbc80b55d0 0x7fcbc80b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 >> v2:172.21.15.145:3300/0 conn(0x7fcbc80b6f90 msgr2=0x7fcbc8162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 --2- 172.21.15.110:0/1341906745 >> v2:172.21.15.145:3300/0 conn(0x7fcbc80b6f90 0x7fcbc8162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc806ea90 con 0x7fcbc80715f0 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fcbc0001a00 con 0x7fcbc80715f0 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 >> v1:172.21.15.110:6789/0 conn(0x7fcbc80715f0 legacy=0x7fcbc815ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbac0039c0 0x7fcbac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbc80b55d0 0x7fcbac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbac004d50 0x7fcbac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81633d0 con 0x7fcbac004d50 2024-09-10T09:59:46.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc806e7b0 con 0x7fcbac0039c0 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcbac00b9d0 con 0x7fcbc80b55d0 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7e00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbac004d50 0x7fcbac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7400640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbac0039c0 0x7fcbac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbbfe00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbc80b55d0 0x7fcbac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7e00640 1 -- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbc80b55d0 msgr2=0x7fcbac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7e00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbc80b55d0 0x7fcbac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7e00640 1 -- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbac0039c0 msgr2=0x7fcbac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7e00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbac0039c0 0x7fcbac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.346+0000 7fcbc7e00640 1 -- 172.21.15.110:0/1341906745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac003530 con 0x7fcbac004d50 2024-09-10T09:59:46.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbc7e00640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbac004d50 0x7fcbac0051d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00013e0 tx=0x7fcbc001d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912eb774df644749 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:46.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000a410 con 0x7fcbac004d50 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc00ad630 con 0x7fcbac004d50 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbc6a00640 1 -- 172.21.15.110:0/1341906745 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc00ada20 con 0x7fcbac004d50 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbce400640 1 -- 172.21.15.110:0/1341906745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbac004d50 msgr2=0x7fcbac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbce400640 1 --2- 172.21.15.110:0/1341906745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbac004d50 0x7fcbac0051d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00013e0 tx=0x7fcbc001d2e0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbc7e00640 1 -- 172.21.15.110:0/1341906745 reap_dead start 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbce400640 1 -- 172.21.15.110:0/1341906745 shutdown_connections 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbce400640 1 -- 172.21.15.110:0/1341906745 >> 172.21.15.110:0/1341906745 conn(0x7fcbc806f440 msgr2=0x7fcbc806f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbce400640 1 -- 172.21.15.110:0/1341906745 shutdown_connections 2024-09-10T09:59:46.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.350+0000 7fcbce400640 1 -- 172.21.15.110:0/1341906745 wait complete. 2024-09-10T09:59:46.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 Processor -- start 2024-09-10T09:59:46.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 -- start start 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbac004d50 0x7fcbc81f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbc815a460 0x7fcbc815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbc8154b60 0x7fcbc815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8071aa0 con 0x7fcbc815a460 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8162970 con 0x7fcbc8154b60 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc81592b0 con 0x7fcbac004d50 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbc815a460 0x7fcbc815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbc815a460 0x7fcbc815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47036/0 (socket says 172.21.15.110:47036) 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 -- 172.21.15.110:0/3693813764 learned_addr learned my addr 172.21.15.110:0/3693813764 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 -- 172.21.15.110:0/3693813764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbac004d50 msgr2=0x7fcbc81f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbac004d50 0x7fcbc81f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 -- 172.21.15.110:0/3693813764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbc8154b60 msgr2=0x7fcbc815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbc8154b60 0x7fcbc815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 -- 172.21.15.110:0/3693813764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc815a930 con 0x7fcbc815a460 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbfe00640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbc815a460 0x7fcbc815b4d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0011930 tx=0x7fcbb0011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbf400640 1 -- 172.21.15.110:0/3693813764 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000fa70 con 0x7fcbc815a460 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbf400640 1 -- 172.21.15.110:0/3693813764 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb000fc10 con 0x7fcbc815a460 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbbf400640 1 -- 172.21.15.110:0/3693813764 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0014440 con 0x7fcbc815a460 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc815ac10 con 0x7fcbc815a460 2024-09-10T09:59:46.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.354+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc80b2320 con 0x7fcbc815a460 2024-09-10T09:59:46.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.358+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc8162e90 con 0x7fcbc815a460 2024-09-10T09:59:46.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.358+0000 7fcbbf400640 1 -- 172.21.15.110:0/3693813764 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fcbb0012040 con 0x7fcbc815a460 2024-09-10T09:59:46.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.358+0000 7fcbbf400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcba4067a70 0x7fcba4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.358+0000 7fcbbf400640 1 -- 172.21.15.110:0/3693813764 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fcbb0086640 con 0x7fcbc815a460 2024-09-10T09:59:46.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.358+0000 7fcbc7400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcba4067a70 0x7fcba4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.358+0000 7fcbc7400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcba4067a70 0x7fcba4069f30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8006b20 tx=0x7fcbb8006ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:46.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.362+0000 7fcbbf400640 1 -- 172.21.15.110:0/3693813764 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcbb0019030 con 0x7fcbc815a460 2024-09-10T09:59:46.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcbc806fbf0 con 0x7fcbc815a460 2024-09-10T09:59:46.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbbf400640 1 -- 172.21.15.110:0/3693813764 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7fcbb0056f00 con 0x7fcbc815a460 2024-09-10T09:59:46.534 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T09:59:46.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcba4067a70 msgr2=0x7fcba4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcba4067a70 0x7fcba4069f30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8006b20 tx=0x7fcbb8006ab0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbc815a460 msgr2=0x7fcbc815b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbc815a460 0x7fcbc815b4d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0011930 tx=0x7fcbb0011df0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 shutdown_connections 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fcba4067a70 0x7fcba4069f30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcbc8154b60 0x7fcbc815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbc815a460 0x7fcbc815b4d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 --2- 172.21.15.110:0/3693813764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcbac004d50 0x7fcbc81f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 >> 172.21.15.110:0/3693813764 conn(0x7fcbc806f440 msgr2=0x7fcbc815d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 shutdown_connections 2024-09-10T09:59:46.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.530+0000 7fcbce400640 1 -- 172.21.15.110:0/3693813764 wait complete. 2024-09-10T09:59:46.555 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T09:59:46.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:46.556 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T09:59:46.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T09:59:46.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:59:46.584 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T09:59:46.635 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:46.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b7200640 1 Processor -- start 2024-09-10T09:59:46.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b7200640 1 -- start start 2024-09-10T09:59:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b7200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe1b815e8f0 0x7fe1b80b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b7200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe1b806f160 0x7fe1b806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b7200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b80b5f50 con 0x7fe1b80b59d0 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b7200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b80b6230 con 0x7fe1b806f160 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b7200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b80b6510 con 0x7fe1b815e8f0 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b6800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe1b806f160 0x7fe1b806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b6800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe1b806f160 0x7fe1b806bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47040/0 (socket says 172.21.15.110:47040) 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b6800640 1 -- 172.21.15.110:0/2130490582 learned_addr learned my addr 172.21.15.110:0/2130490582 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b5e00640 1 --2- 172.21.15.110:0/2130490582 >> v2:172.21.15.145:3300/0 conn(0x7fe1b815e8f0 0x7fe1b80b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b6800640 1 -- 172.21.15.110:0/2130490582 >> v2:172.21.15.145:3300/0 conn(0x7fe1b815e8f0 msgr2=0x7fe1b80b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b6800640 1 --2- 172.21.15.110:0/2130490582 >> v2:172.21.15.145:3300/0 conn(0x7fe1b815e8f0 0x7fe1b80b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b6800640 1 -- 172.21.15.110:0/2130490582 >> v1:172.21.15.145:6789/0 conn(0x7fe1b80b59d0 legacy=0x7fe1b806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.766+0000 7fe1b6800640 1 -- 172.21.15.110:0/2130490582 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b80b67f0 con 0x7fe1b806f160 2024-09-10T09:59:46.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b6800640 1 --2- 172.21.15.110:0/2130490582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b806bd30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80d22c0 tx=0x7fe1b001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fe8c2beb51cb76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:46.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b4a00640 1 -- 172.21.15.110:0/2130490582 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00a56f0 con 0x7fe1b806f160 2024-09-10T09:59:46.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b4a00640 1 -- 172.21.15.110:0/2130490582 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b00a5890 con 0x7fe1b806f160 2024-09-10T09:59:46.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b4a00640 1 -- 172.21.15.110:0/2130490582 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00a5c60 con 0x7fe1b806f160 2024-09-10T09:59:46.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 -- 172.21.15.110:0/2130490582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 msgr2=0x7fe1b806bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2130490582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b806bd30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80d22c0 tx=0x7fe1b001c920 comp rx=0 tx=0).stop 2024-09-10T09:59:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 -- 172.21.15.110:0/2130490582 shutdown_connections 2024-09-10T09:59:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2130490582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b806bd30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2130490582 >> v2:172.21.15.145:3300/0 conn(0x7fe1b815e8f0 0x7fe1b80b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 -- 172.21.15.110:0/2130490582 >> 172.21.15.110:0/2130490582 conn(0x7fe1b80d1480 msgr2=0x7fe1b80d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 -- 172.21.15.110:0/2130490582 shutdown_connections 2024-09-10T09:59:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 -- 172.21.15.110:0/2130490582 wait complete. 2024-09-10T09:59:46.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 Processor -- start 2024-09-10T09:59:46.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.770+0000 7fe1b7200640 1 -- start start 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1b80b59d0 0x7fe1b814c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1b815e8f0 0x7fe1b814eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b806ce10 con 0x7fe1b806f160 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b806cc90 con 0x7fe1b815e8f0 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b806cf90 con 0x7fe1b80b59d0 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b814e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47048/0 (socket says 172.21.15.110:47048) 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 -- 172.21.15.110:0/2816454933 learned_addr learned my addr 172.21.15.110:0/2816454933 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b6800640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1b815e8f0 0x7fe1b814eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5400640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1b80b59d0 0x7fe1b814c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 -- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1b80b59d0 msgr2=0x7fe1b814c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1b80b59d0 0x7fe1b814c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 -- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1b815e8f0 msgr2=0x7fe1b814eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1b815e8f0 0x7fe1b814eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 -- 172.21.15.110:0/2816454933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b8151ad0 con 0x7fe1b806f160 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b6800640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1b815e8f0 0x7fe1b814eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5400640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1b80b59d0 0x7fe1b814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b5e00640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b814e160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800d800 tx=0x7fe1a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:46.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1ae000640 1 -- 172.21.15.110:0/2816454933 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800b6c0 con 0x7fe1b806f160 2024-09-10T09:59:46.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1ae000640 1 -- 172.21.15.110:0/2816454933 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a8019070 con 0x7fe1b806f160 2024-09-10T09:59:46.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1ae000640 1 -- 172.21.15.110:0/2816454933 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800bae0 con 0x7fe1b806f160 2024-09-10T09:59:46.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b8151db0 con 0x7fe1b806f160 2024-09-10T09:59:46.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b8154dc0 con 0x7fe1b806f160 2024-09-10T09:59:46.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b8154bb0 con 0x7fe1b806f160 2024-09-10T09:59:46.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.774+0000 7fe1ae000640 1 -- 172.21.15.110:0/2816454933 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fe1a800bc80 con 0x7fe1b806f160 2024-09-10T09:59:46.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.778+0000 7fe1ae000640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe198067a70 0x7fe198069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:46.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.778+0000 7fe1ae000640 1 -- 172.21.15.110:0/2816454933 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe1a8086550 con 0x7fe1b806f160 2024-09-10T09:59:46.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.778+0000 7fe1b5400640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe198067a70 0x7fe198069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:46.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.778+0000 7fe1b5400640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe198067a70 0x7fe198069f30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80d2030 tx=0x7fe1a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:46.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.782+0000 7fe1ae000640 1 -- 172.21.15.110:0/2816454933 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe1a8056d70 con 0x7fe1b806f160 2024-09-10T09:59:46.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1b806bd30 con 0x7fe1b806f160 2024-09-10T09:59:46.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1ae000640 1 -- 172.21.15.110:0/2816454933 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fe1a8056670 con 0x7fe1b806f160 2024-09-10T09:59:46.978 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:46.979 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T09:59:46.979 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T09:59:46.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe198067a70 msgr2=0x7fe198069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe198067a70 0x7fe198069f30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80d2030 tx=0x7fe1a4009040 comp rx=0 tx=0).stop 2024-09-10T09:59:46.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 msgr2=0x7fe1b814e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:46.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b814e160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800d800 tx=0x7fe1a800dcc0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 shutdown_connections 2024-09-10T09:59:46.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe198067a70 0x7fe198069f30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1b815e8f0 0x7fe1b814eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1b80b59d0 0x7fe1b814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 --2- 172.21.15.110:0/2816454933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b806f160 0x7fe1b814e160 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:46.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 >> 172.21.15.110:0/2816454933 conn(0x7fe1b80d1480 msgr2=0x7fe1b80d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:46.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 shutdown_connections 2024-09-10T09:59:46.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:46.974+0000 7fe1b7200640 1 -- 172.21.15.110:0/2816454933 wait complete. 2024-09-10T09:59:47.000 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T09:59:47.159 INFO:teuthology.orchestra.run.smithi110.stdout:[] 2024-09-10T09:59:47.160 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-10T09:59:47.160 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol46090 2024-09-10T09:59:47.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03b1a00640 1 Processor -- start 2024-09-10T09:59:47.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03b1a00640 1 -- start start 2024-09-10T09:59:47.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03b1a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f03ac15b330 0x7f03ac06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03b1a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03ac06e850 0x7f03ac0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03b1a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03ac0b4960 con 0x7f03ac0ce4d0 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03b1a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f03ac0b5570 con 0x7f03ac06e850 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03b1a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f03ac0b4c40 con 0x7f03ac15b330 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03ab400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03ac06e850 0x7f03ac0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03ab400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03ac06e850 0x7f03ac0b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47056/0 (socket says 172.21.15.110:47056) 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03ab400640 1 -- 172.21.15.110:0/2664517410 learned_addr learned my addr 172.21.15.110:0/2664517410 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03aaa00640 1 --2- 172.21.15.110:0/2664517410 >> v2:172.21.15.145:3300/0 conn(0x7f03ac15b330 0x7f03ac06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03ab400640 1 -- 172.21.15.110:0/2664517410 >> v2:172.21.15.145:3300/0 conn(0x7f03ac15b330 msgr2=0x7f03ac06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03ab400640 1 --2- 172.21.15.110:0/2664517410 >> v2:172.21.15.145:3300/0 conn(0x7f03ac15b330 0x7f03ac06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03ab400640 1 -- 172.21.15.110:0/2664517410 >> v1:172.21.15.110:6789/0 conn(0x7f03ac0ce4d0 legacy=0x7f03ac06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03ab400640 1 -- 172.21.15.110:0/2664517410 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac0b4f20 con 0x7f03ac06e850 2024-09-10T09:59:47.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.310+0000 7f03aaa00640 1 --2- 172.21.15.110:0/2664517410 >> v2:172.21.15.145:3300/0 conn(0x7f03ac15b330 0x7f03ac06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:47.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03ab400640 1 --2- 172.21.15.110:0/2664517410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac06e850 0x7f03ac0b0860 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f03ac06eca0 tx=0x7f03a401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313f169f84d0de6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:47.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03aa000640 1 -- 172.21.15.110:0/2664517410 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40a57d0 con 0x7f03ac06e850 2024-09-10T09:59:47.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03aa000640 1 -- 172.21.15.110:0/2664517410 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a40a5970 con 0x7f03ac06e850 2024-09-10T09:59:47.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03aa000640 1 -- 172.21.15.110:0/2664517410 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40a5d40 con 0x7f03ac06e850 2024-09-10T09:59:47.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 -- 172.21.15.110:0/2664517410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac06e850 msgr2=0x7f03ac0b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 --2- 172.21.15.110:0/2664517410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac06e850 0x7f03ac0b0860 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f03ac06eca0 tx=0x7f03a401c9f0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 -- 172.21.15.110:0/2664517410 shutdown_connections 2024-09-10T09:59:47.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 --2- 172.21.15.110:0/2664517410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac06e850 0x7f03ac0b0860 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 --2- 172.21.15.110:0/2664517410 >> v2:172.21.15.145:3300/0 conn(0x7f03ac15b330 0x7f03ac06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 -- 172.21.15.110:0/2664517410 >> 172.21.15.110:0/2664517410 conn(0x7f03ac062e40 msgr2=0x7f03ac06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:47.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 -- 172.21.15.110:0/2664517410 shutdown_connections 2024-09-10T09:59:47.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 -- 172.21.15.110:0/2664517410 wait complete. 2024-09-10T09:59:47.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.314+0000 7f03b1a00640 1 Processor -- start 2024-09-10T09:59:47.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 -- start start 2024-09-10T09:59:47.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03ac06e850 0x7f03ac146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac15b330 0x7f03ac071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03ac146aa0 0x7f03ac071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1680b0 con 0x7f03ac15b330 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac0b1800 con 0x7f03ac06e850 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f03ac168230 con 0x7f03ac146aa0 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac15b330 0x7f03ac071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac15b330 0x7f03ac071540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47060/0 (socket says 172.21.15.110:47060) 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 -- 172.21.15.110:0/3888039510 learned_addr learned my addr 172.21.15.110:0/3888039510 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03aaa00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03ac06e850 0x7f03ac146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03ab400640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03ac146aa0 0x7f03ac071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 -- 172.21.15.110:0/3888039510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03ac146aa0 msgr2=0x7f03ac071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03ac146aa0 0x7f03ac071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 -- 172.21.15.110:0/3888039510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03ac06e850 msgr2=0x7f03ac146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03ac06e850 0x7f03ac146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 -- 172.21.15.110:0/3888039510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac147150 con 0x7f03ac15b330 2024-09-10T09:59:47.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a3e00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac15b330 0x7f03ac071540 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f039400c420 tx=0x7f039400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:47.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a2a00640 1 -- 172.21.15.110:0/3888039510 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940052d0 con 0x7f03ac15b330 2024-09-10T09:59:47.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a2a00640 1 -- 172.21.15.110:0/3888039510 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0394005470 con 0x7f03ac15b330 2024-09-10T09:59:47.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac147430 con 0x7f03ac15b330 2024-09-10T09:59:47.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03a2a00640 1 -- 172.21.15.110:0/3888039510 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394007260 con 0x7f03ac15b330 2024-09-10T09:59:47.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1479f0 con 0x7f03ac15b330 2024-09-10T09:59:47.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.318+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac147bf0 con 0x7f03ac15b330 2024-09-10T09:59:47.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.322+0000 7f03a2a00640 1 -- 172.21.15.110:0/3888039510 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f0394007400 con 0x7f03ac15b330 2024-09-10T09:59:47.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.322+0000 7f03a2a00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0388067a70 0x7f0388069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.322+0000 7f03a2a00640 1 -- 172.21.15.110:0/3888039510 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f03940852f0 con 0x7f03ac15b330 2024-09-10T09:59:47.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.322+0000 7f03aaa00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0388067a70 0x7f0388069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.322+0000 7f03aaa00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0388067a70 0x7f0388069f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f03ac072bf0 tx=0x7f039c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:47.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.326+0000 7f03a2a00640 1 -- 172.21.15.110:0/3888039510 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f03ac147bf0 con 0x7f03ac15b330 2024-09-10T09:59:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.510+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol46090", "target": ["mon-mgr", ""]}) -- 0x7f03ac0b0860 con 0x7f0388067a70 2024-09-10T09:59:47.520 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:47.514+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:47.520 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:47.514+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:47.520 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:47.514+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:47.520 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:47.514+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:47.520 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T09:59:47.514+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:59:47.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03a2a00640 1 -- 172.21.15.110:0/3888039510 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f03ac0b0860 con 0x7f0388067a70 2024-09-10T09:59:47.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0388067a70 msgr2=0x7f0388069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0388067a70 0x7f0388069f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f03ac072bf0 tx=0x7f039c008040 comp rx=0 tx=0).stop 2024-09-10T09:59:47.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac15b330 msgr2=0x7f03ac071540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac15b330 0x7f03ac071540 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f039400c420 tx=0x7f039400c8e0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 shutdown_connections 2024-09-10T09:59:47.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0388067a70 0x7f0388069f30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03ac146aa0 0x7f03ac071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03ac15b330 0x7f03ac071540 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 --2- 172.21.15.110:0/3888039510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03ac06e850 0x7f03ac146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 >> 172.21.15.110:0/3888039510 conn(0x7f03ac062e40 msgr2=0x7f03ac0ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:47.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 shutdown_connections 2024-09-10T09:59:47.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.546+0000 7f03b1a00640 1 -- 172.21.15.110:0/3888039510 wait complete. 2024-09-10T09:59:47.572 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:47.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f3552200640 1 Processor -- start 2024-09-10T09:59:47.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f3552200640 1 -- start start 2024-09-10T09:59:47.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f3552200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cd510 0x7f354c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f3552200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f354c06e090 0x7f354c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f3552200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f354c072150 con 0x7f354c0cfe30 2024-09-10T09:59:47.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f3552200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f354c0cdeb0 con 0x7f354c06e090 2024-09-10T09:59:47.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f3552200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f354c15d9a0 con 0x7f354c0cd510 2024-09-10T09:59:47.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354b400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f354c0cfe30 0x7f354c071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51824/0 (socket says 172.21.15.110:51824) 2024-09-10T09:59:47.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354b400640 1 -- 172.21.15.110:0/3833579909 learned_addr learned my addr 172.21.15.110:0/3833579909 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:47.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354be00640 1 --2- 172.21.15.110:0/3833579909 >> v2:172.21.15.145:3300/0 conn(0x7f354c06e090 0x7f354c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354aa00640 1 --2- 172.21.15.110:0/3833579909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cd510 0x7f354c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354a000640 1 -- 172.21.15.110:0/3833579909 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2204241904 0 0) 0x7f354c072150 con 0x7f354c0cfe30 2024-09-10T09:59:47.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354a000640 1 -- 172.21.15.110:0/3833579909 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3530003530 con 0x7f354c0cfe30 2024-09-10T09:59:47.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354be00640 1 -- 172.21.15.110:0/3833579909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cd510 msgr2=0x7f354c0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:47.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354be00640 1 --2- 172.21.15.110:0/3833579909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cd510 0x7f354c0cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354be00640 1 -- 172.21.15.110:0/3833579909 >> v1:172.21.15.110:6789/0 conn(0x7f354c0cfe30 legacy=0x7f354c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.702+0000 7f354be00640 1 -- 172.21.15.110:0/3833579909 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c06ea60 con 0x7f354c06e090 2024-09-10T09:59:47.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f354be00640 1 --2- 172.21.15.110:0/3833579909 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 0x7f354c06e520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3544009d90 tx=0x7f354401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09efe3d4a116694 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:47.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f354a000640 1 -- 172.21.15.110:0/3833579909 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35440b2070 con 0x7f354c06e090 2024-09-10T09:59:47.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f354a000640 1 -- 172.21.15.110:0/3833579909 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35440a58b0 con 0x7f354c06e090 2024-09-10T09:59:47.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f354a000640 1 -- 172.21.15.110:0/3833579909 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35440a5c80 con 0x7f354c06e090 2024-09-10T09:59:47.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f3552200640 1 -- 172.21.15.110:0/3833579909 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 msgr2=0x7f354c06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f3552200640 1 --2- 172.21.15.110:0/3833579909 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 0x7f354c06e520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3544009d90 tx=0x7f354401c8e0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f3552200640 1 -- 172.21.15.110:0/3833579909 shutdown_connections 2024-09-10T09:59:47.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f3552200640 1 --2- 172.21.15.110:0/3833579909 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 0x7f354c06e520 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f3552200640 1 --2- 172.21.15.110:0/3833579909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cd510 0x7f354c0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f3552200640 1 -- 172.21.15.110:0/3833579909 >> 172.21.15.110:0/3833579909 conn(0x7f354c06bc50 msgr2=0x7f354c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:47.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f3552200640 1 -- 172.21.15.110:0/3833579909 shutdown_connections 2024-09-10T09:59:47.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.706+0000 7f3552200640 1 -- 172.21.15.110:0/3833579909 wait complete. 2024-09-10T09:59:47.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 Processor -- start 2024-09-10T09:59:47.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 -- start start 2024-09-10T09:59:47.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 0x7f354c151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f354c0cd510 0x7f354c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cfe30 0x7f354c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f354c15d5b0 con 0x7f354c0cd510 2024-09-10T09:59:47.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f354c15d430 con 0x7f354c06e090 2024-09-10T09:59:47.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f354c15d730 con 0x7f354c0cfe30 2024-09-10T09:59:47.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f354c0cd510 0x7f354c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f354c0cd510 0x7f354c156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47088/0 (socket says 172.21.15.110:47088) 2024-09-10T09:59:47.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 -- 172.21.15.110:0/1690830742 learned_addr learned my addr 172.21.15.110:0/1690830742 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:47.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354b400640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 0x7f354c151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354be00640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cfe30 0x7f354c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 -- 172.21.15.110:0/1690830742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cfe30 msgr2=0x7f354c151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cfe30 0x7f354c151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 -- 172.21.15.110:0/1690830742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 msgr2=0x7f354c151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 0x7f354c151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 -- 172.21.15.110:0/1690830742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c0b2790 con 0x7f354c0cd510 2024-09-10T09:59:47.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f354aa00640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f354c0cd510 0x7f354c156160 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f353800e8a0 tx=0x7f353800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:47.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3543400640 1 -- 172.21.15.110:0/1690830742 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538017070 con 0x7f354c0cd510 2024-09-10T09:59:47.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3543400640 1 -- 172.21.15.110:0/1690830742 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353800c940 con 0x7f354c0cd510 2024-09-10T09:59:47.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c0b24b0 con 0x7f354c0cd510 2024-09-10T09:59:47.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3543400640 1 -- 172.21.15.110:0/1690830742 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353800ccf0 con 0x7f354c0cd510 2024-09-10T09:59:47.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.710+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c155c40 con 0x7f354c0cd510 2024-09-10T09:59:47.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.714+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c06e520 con 0x7f354c0cd510 2024-09-10T09:59:47.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.714+0000 7f3543400640 1 -- 172.21.15.110:0/1690830742 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f353801a070 con 0x7f354c0cd510 2024-09-10T09:59:47.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.714+0000 7f3543400640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f352c067a70 0x7f352c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:47.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.714+0000 7f3543400640 1 -- 172.21.15.110:0/1690830742 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f3538085d60 con 0x7f354c0cd510 2024-09-10T09:59:47.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.714+0000 7f354b400640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f352c067a70 0x7f352c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:47.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.714+0000 7f354b400640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f352c067a70 0x7f352c069f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f353c007fd0 tx=0x7f353c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:47.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.718+0000 7f3543400640 1 -- 172.21.15.110:0/1690830742 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3538056070 con 0x7f354c0cd510 2024-09-10T09:59:47.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f354c0b2960 con 0x7f354c0cd510 2024-09-10T09:59:47.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3543400640 1 -- 172.21.15.110:0/1690830742 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f353805a370 con 0x7f354c0cd510 2024-09-10T09:59:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:47.915 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T09:59:47.915 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T09:59:47.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f352c067a70 msgr2=0x7f352c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f352c067a70 0x7f352c069f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f353c007fd0 tx=0x7f353c009040 comp rx=0 tx=0).stop 2024-09-10T09:59:47.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f354c0cd510 msgr2=0x7f354c156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:47.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f354c0cd510 0x7f354c156160 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f353800e8a0 tx=0x7f353800eca0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 shutdown_connections 2024-09-10T09:59:47.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f352c067a70 0x7f352c069f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f354c0cfe30 0x7f354c151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f354c0cd510 0x7f354c156160 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 --2- 172.21.15.110:0/1690830742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f354c06e090 0x7f354c151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:47.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 >> 172.21.15.110:0/1690830742 conn(0x7f354c06bc50 msgr2=0x7f354c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:47.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 shutdown_connections 2024-09-10T09:59:47.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:47.910+0000 7f3552200640 1 -- 172.21.15.110:0/1690830742 wait complete. 2024-09-10T09:59:47.937 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T09:59:48.095 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.55598842999999998,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.972301385797493,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T09:59:48.096 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.972301385797493 }, "state": "open", "uptime": 0.55598843 } ] 2024-09-10T09:59:48.096 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:48.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.226+0000 7efe41a00640 1 Processor -- start 2024-09-10T09:59:48.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.226+0000 7efe41a00640 1 -- start start 2024-09-10T09:59:48.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe41a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 0x7efe3c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:48.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe41a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efe3c06f1e0 0x7efe3c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:48.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe41a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3c155d50 con 0x7efe3c15c560 2024-09-10T09:59:48.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe41a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7efe3c0b4570 con 0x7efe3c06f1e0 2024-09-10T09:59:48.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe41a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c0b34d0 con 0x7efe3c155970 2024-09-10T09:59:48.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efe3c06f1e0 0x7efe3c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:48.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 0x7efe3c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:48.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efe3c06f1e0 0x7efe3c0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:33858/0 (socket says 172.21.15.110:33858) 2024-09-10T09:59:48.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3a000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7efe3c15c560 0x7efe3c06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:58102/0 (socket says 172.21.15.110:58102) 2024-09-10T09:59:48.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3b400640 1 -- 172.21.15.110:0/1194240367 learned_addr learned my addr 172.21.15.110:0/1194240367 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:48.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3b400640 1 -- 172.21.15.110:0/1194240367 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 msgr2=0x7efe3c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:48.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe39600640 1 -- 172.21.15.110:0/1194240367 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3520851721 0 0) 0x7efe3c155d50 con 0x7efe3c15c560 2024-09-10T09:59:48.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3b400640 1 --2- 172.21.15.110:0/1194240367 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 0x7efe3c0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3b400640 1 -- 172.21.15.110:0/1194240367 >> v1:172.21.15.145:6789/0 conn(0x7efe3c15c560 legacy=0x7efe3c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:48.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.230+0000 7efe3b400640 1 -- 172.21.15.110:0/1194240367 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c0b37b0 con 0x7efe3c06f1e0 2024-09-10T09:59:48.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe3b400640 1 --2- 172.21.15.110:0/1194240367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 0x7efe3c0ce420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe34009e50 tx=0x7efe3401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e04bcec520bc4c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:48.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe39600640 1 -- 172.21.15.110:0/1194240367 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340a5830 con 0x7efe3c06f1e0 2024-09-10T09:59:48.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe39600640 1 -- 172.21.15.110:0/1194240367 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe340a59d0 con 0x7efe3c06f1e0 2024-09-10T09:59:48.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe39600640 1 -- 172.21.15.110:0/1194240367 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340a5da0 con 0x7efe3c06f1e0 2024-09-10T09:59:48.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 -- 172.21.15.110:0/1194240367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 msgr2=0x7efe3c0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:48.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 --2- 172.21.15.110:0/1194240367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 0x7efe3c0ce420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe34009e50 tx=0x7efe3401c970 comp rx=0 tx=0).stop 2024-09-10T09:59:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 -- 172.21.15.110:0/1194240367 shutdown_connections 2024-09-10T09:59:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 --2- 172.21.15.110:0/1194240367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 0x7efe3c0ce420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 --2- 172.21.15.110:0/1194240367 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 0x7efe3c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 -- 172.21.15.110:0/1194240367 >> 172.21.15.110:0/1194240367 conn(0x7efe3c06bc50 msgr2=0x7efe3c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 -- 172.21.15.110:0/1194240367 shutdown_connections 2024-09-10T09:59:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 -- 172.21.15.110:0/1194240367 wait complete. 2024-09-10T09:59:48.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 Processor -- start 2024-09-10T09:59:48.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.234+0000 7efe41a00640 1 -- start start 2024-09-10T09:59:48.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 0x7efe3c158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:48.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 0x7efe3c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:48.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe3c15c560 0x7efe3c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:48.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c0cf170 con 0x7efe3c15c560 2024-09-10T09:59:48.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c0cce00 con 0x7efe3c06f1e0 2024-09-10T09:59:48.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c0cf2f0 con 0x7efe3c155970 2024-09-10T09:59:48.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe3c15c560 0x7efe3c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:48.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3a000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 0x7efe3c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:48.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 0x7efe3c158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe3c15c560 0x7efe3c15ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36180/0 (socket says 172.21.15.110:36180) 2024-09-10T09:59:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 -- 172.21.15.110:0/2139135250 learned_addr learned my addr 172.21.15.110:0/2139135250 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 -- 172.21.15.110:0/2139135250 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 msgr2=0x7efe3c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 0x7efe3c15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 -- 172.21.15.110:0/2139135250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 msgr2=0x7efe3c158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 0x7efe3c158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 -- 172.21.15.110:0/2139135250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c156f20 con 0x7efe3c15c560 2024-09-10T09:59:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe3b400640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe3c15c560 0x7efe3c15ca50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efe3401a5a0 tx=0x7efe34019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe32a00640 1 -- 172.21.15.110:0/2139135250 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340bc020 con 0x7efe3c15c560 2024-09-10T09:59:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe32a00640 1 -- 172.21.15.110:0/2139135250 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe340b8670 con 0x7efe3c15c560 2024-09-10T09:59:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe32a00640 1 -- 172.21.15.110:0/2139135250 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340b8a40 con 0x7efe3c15c560 2024-09-10T09:59:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c157200 con 0x7efe3c15c560 2024-09-10T09:59:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c1577c0 con 0x7efe3c15c560 2024-09-10T09:59:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c0ce420 con 0x7efe3c15c560 2024-09-10T09:59:48.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.238+0000 7efe32a00640 1 -- 172.21.15.110:0/2139135250 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7efe340bf020 con 0x7efe3c15c560 2024-09-10T09:59:48.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.242+0000 7efe32a00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe1c067a70 0x7efe1c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:48.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.242+0000 7efe32a00640 1 -- 172.21.15.110:0/2139135250 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7efe34120f10 con 0x7efe3c15c560 2024-09-10T09:59:48.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.242+0000 7efe3aa00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe1c067a70 0x7efe1c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:48.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.242+0000 7efe3aa00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe1c067a70 0x7efe1c069f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe3c15b060 tx=0x7efe2c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:48.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.246+0000 7efe32a00640 1 -- 172.21.15.110:0/2139135250 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efe340c6030 con 0x7efe3c15c560 2024-09-10T09:59:48.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.434+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe3c160a90 con 0x7efe3c15c560 2024-09-10T09:59:48.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe32a00640 1 -- 172.21.15.110:0/2139135250 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7efe34016620 con 0x7efe3c15c560 2024-09-10T09:59:48.442 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:48.443 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T09:59:48.443 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe1c067a70 msgr2=0x7efe1c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe1c067a70 0x7efe1c069f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe3c15b060 tx=0x7efe2c009190 comp rx=0 tx=0).stop 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe3c15c560 msgr2=0x7efe3c15ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe3c15c560 0x7efe3c15ca50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efe3401a5a0 tx=0x7efe34019710 comp rx=0 tx=0).stop 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 shutdown_connections 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe1c067a70 0x7efe1c069f30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe3c15c560 0x7efe3c15ca50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efe3c155970 0x7efe3c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 --2- 172.21.15.110:0/2139135250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe3c06f1e0 0x7efe3c158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:48.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 >> 172.21.15.110:0/2139135250 conn(0x7efe3c06bc50 msgr2=0x7efe3c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:48.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 shutdown_connections 2024-09-10T09:59:48.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:48.438+0000 7efe41a00640 1 -- 172.21.15.110:0/2139135250 wait complete. 2024-09-10T09:59:48.463 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T09:59:48.622 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.083977905,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.945513394761925,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T09:59:48.623 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.945513394761925 }, "state": "open", "uptime": 1.083977905 } ] 2024-09-10T09:59:48.623 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T09:59:48.721 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:59:48.724 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T09:59:53.624 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:53.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861be00640 1 Processor -- start 2024-09-10T09:59:53.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861be00640 1 -- start start 2024-09-10T09:59:53.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f861c0b1c60 0x7f861c142320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:53.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f861c0731e0 0x7f861c073650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861be00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861c15c790 con 0x7f861c15fd80 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861be00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f861c073b90 con 0x7f861c0b1c60 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861be00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f861c0b2040 con 0x7f861c0731e0 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f861c0b1c60 0x7f861c142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f861c0b1c60 0x7f861c142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36194/0 (socket says 172.21.15.110:36194) 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861aa00640 1 -- 172.21.15.110:0/105811008 learned_addr learned my addr 172.21.15.110:0/105811008 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861aa00640 1 -- 172.21.15.110:0/105811008 >> v2:172.21.15.145:3300/0 conn(0x7f861c0731e0 msgr2=0x7f861c073650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861b400640 1 --2- 172.21.15.110:0/105811008 >> v2:172.21.15.145:3300/0 conn(0x7f861c0731e0 0x7f861c073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861aa00640 1 --2- 172.21.15.110:0/105811008 >> v2:172.21.15.145:3300/0 conn(0x7f861c0731e0 0x7f861c073650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861aa00640 1 -- 172.21.15.110:0/105811008 >> v1:172.21.15.110:6789/0 conn(0x7f861c15fd80 legacy=0x7f861c13f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:53.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861aa00640 1 -- 172.21.15.110:0/105811008 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c142860 con 0x7f861c0b1c60 2024-09-10T09:59:53.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.766+0000 7f861b400640 1 --2- 172.21.15.110:0/105811008 >> v2:172.21.15.145:3300/0 conn(0x7f861c0731e0 0x7f861c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861aa00640 1 --2- 172.21.15.110:0/105811008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c142320 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f861c160e60 tx=0x7f860c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a091126ca77024 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861a000640 1 -- 172.21.15.110:0/105811008 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0b0070 con 0x7f861c0b1c60 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861a000640 1 -- 172.21.15.110:0/105811008 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c0ab600 con 0x7f861c0b1c60 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861a000640 1 -- 172.21.15.110:0/105811008 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0ab9d0 con 0x7f861c0b1c60 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 -- 172.21.15.110:0/105811008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 msgr2=0x7f861c142320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 --2- 172.21.15.110:0/105811008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c142320 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f861c160e60 tx=0x7f860c01c520 comp rx=0 tx=0).stop 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 -- 172.21.15.110:0/105811008 shutdown_connections 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 --2- 172.21.15.110:0/105811008 >> v2:172.21.15.145:3300/0 conn(0x7f861c0731e0 0x7f861c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 --2- 172.21.15.110:0/105811008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c142320 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 -- 172.21.15.110:0/105811008 >> 172.21.15.110:0/105811008 conn(0x7f861c0b42d0 msgr2=0x7f861c0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 -- 172.21.15.110:0/105811008 shutdown_connections 2024-09-10T09:59:53.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 -- 172.21.15.110:0/105811008 wait complete. 2024-09-10T09:59:53.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.770+0000 7f861be00640 1 Processor -- start 2024-09-10T09:59:53.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 -- start start 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f861c0731e0 0x7f861c165c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f861c15fd80 0x7f861c163db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f861c15d460 con 0x7f861c0b1c60 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f861c160b50 con 0x7f861c15fd80 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f861c15d5e0 con 0x7f861c0731e0 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c1644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36198/0 (socket says 172.21.15.110:36198) 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 -- 172.21.15.110:0/554027825 learned_addr learned my addr 172.21.15.110:0/554027825 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861aa00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f861c0731e0 0x7f861c165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861b400640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f861c15fd80 0x7f861c163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 -- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f861c0731e0 msgr2=0x7f861c165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f861c0731e0 0x7f861c165c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 -- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f861c15fd80 msgr2=0x7f861c163db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f861c15fd80 0x7f861c163db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 -- 172.21.15.110:0/554027825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c164bb0 con 0x7f861c0b1c60 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861b400640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f861c15fd80 0x7f861c163db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613e00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c1644b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f860400c450 tx=0x7f860400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613400640 1 -- 172.21.15.110:0/554027825 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604016020 con 0x7f861c0b1c60 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613400640 1 -- 172.21.15.110:0/554027825 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86040052d0 con 0x7f861c0b1c60 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c164e90 con 0x7f861c0b1c60 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f8613400640 1 -- 172.21.15.110:0/554027825 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604007260 con 0x7f861c0b1c60 2024-09-10T09:59:53.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c165450 con 0x7f861c0b1c60 2024-09-10T09:59:53.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.774+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c073650 con 0x7f861c0b1c60 2024-09-10T09:59:53.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.778+0000 7f8613400640 1 -- 172.21.15.110:0/554027825 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f8604007400 con 0x7f861c0b1c60 2024-09-10T09:59:53.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.778+0000 7f8613400640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f85f8067b00 0x7f85f8069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:53.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.778+0000 7f8613400640 1 -- 172.21.15.110:0/554027825 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8604085950 con 0x7f861c0b1c60 2024-09-10T09:59:53.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.778+0000 7f861aa00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f85f8067b00 0x7f85f8069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:53.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.778+0000 7f861aa00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f85f8067b00 0x7f85f8069fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f860c0ad000 tx=0x7f860c0a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:53.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.782+0000 7f8613400640 1 -- 172.21.15.110:0/554027825 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8604020020 con 0x7f861c0b1c60 2024-09-10T09:59:53.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.974+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f861c0b2260 con 0x7f861c0b1c60 2024-09-10T09:59:53.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.974+0000 7f8613400640 1 -- 172.21.15.110:0/554027825 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f861c15c790 con 0x7f861c0b1c60 2024-09-10T09:59:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:53.981 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T09:59:53.981 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T09:59:53.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f85f8067b00 msgr2=0x7f85f8069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:53.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f85f8067b00 0x7f85f8069fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f860c0ad000 tx=0x7f860c0a6000 comp rx=0 tx=0).stop 2024-09-10T09:59:53.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 msgr2=0x7f861c1644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:53.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c1644b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f860400c450 tx=0x7f860400c910 comp rx=0 tx=0).stop 2024-09-10T09:59:53.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 shutdown_connections 2024-09-10T09:59:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f85f8067b00 0x7f85f8069fc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f861c15fd80 0x7f861c163db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f861c0b1c60 0x7f861c1644b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 --2- 172.21.15.110:0/554027825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f861c0731e0 0x7f861c165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 >> 172.21.15.110:0/554027825 conn(0x7f861c0b42d0 msgr2=0x7f861c0cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 shutdown_connections 2024-09-10T09:59:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:53.978+0000 7f861be00640 1 -- 172.21.15.110:0/554027825 wait complete. 2024-09-10T09:59:54.001 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T09:59:54.160 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.6198706840000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.666604007023178,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T09:59:54.160 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.666604007023178 }, "state": "open", "uptime": 6.619870684 } ] 2024-09-10T09:59:54.160 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T09:59:59.161 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T09:59:59.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.302+0000 7f7fc1600640 1 Processor -- start 2024-09-10T09:59:59.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.302+0000 7f7fc1600640 1 -- start start 2024-09-10T09:59:59.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fc1600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7fbc145b00 0x7f7fbc0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:59.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fc1600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc15d050 0x7f7fbc0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:59.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fc1600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fbc071db0 con 0x7f7fbc0b5690 2024-09-10T09:59:59.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fc1600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc0b6010 con 0x7f7fbc145b00 2024-09-10T09:59:59.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fc1600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc0ce5f0 con 0x7f7fbc15d050 2024-09-10T09:59:59.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbb400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f7fbc0b5690 0x7f7fbc15c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59328/0 (socket says 172.21.15.110:59328) 2024-09-10T09:59:59.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbb400640 1 -- 172.21.15.110:0/3095927751 learned_addr learned my addr 172.21.15.110:0/3095927751 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fc0c00640 1 --2- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc15d050 0x7f7fbc0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbbe00640 1 --2- 172.21.15.110:0/3095927751 >> v2:172.21.15.145:3300/0 conn(0x7f7fbc145b00 0x7f7fbc0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbaa00640 1 -- 172.21.15.110:0/3095927751 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262947330 0 0) 0x7f7fbc071db0 con 0x7f7fbc0b5690 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbaa00640 1 -- 172.21.15.110:0/3095927751 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fa0003530 con 0x7f7fbc0b5690 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbbe00640 1 -- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc15d050 msgr2=0x7f7fbc0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbbe00640 1 --2- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc15d050 0x7f7fbc0b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbbe00640 1 -- 172.21.15.110:0/3095927751 >> v1:172.21.15.110:6789/0 conn(0x7f7fbc0b5690 legacy=0x7f7fbc15c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fbbe00640 1 -- 172.21.15.110:0/3095927751 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc0ce8d0 con 0x7f7fbc145b00 2024-09-10T09:59:59.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.306+0000 7f7fc0c00640 1 --2- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc15d050 0x7f7fbc0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:59.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fbbe00640 1 --2- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc145b00 0x7f7fbc0cf0d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0b6190 tx=0x7f7fac01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643da27483f4423e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:59.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fbaa00640 1 -- 172.21.15.110:0/3095927751 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0b0070 con 0x7f7fbc145b00 2024-09-10T09:59:59.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fbaa00640 1 -- 172.21.15.110:0/3095927751 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac0ab600 con 0x7f7fbc145b00 2024-09-10T09:59:59.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fbaa00640 1 -- 172.21.15.110:0/3095927751 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0ab9b0 con 0x7f7fbc145b00 2024-09-10T09:59:59.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc145b00 msgr2=0x7f7fbc0cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:59.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc145b00 0x7f7fbc0cf0d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0b6190 tx=0x7f7fac01c4a0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- 172.21.15.110:0/3095927751 shutdown_connections 2024-09-10T09:59:59.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc15d050 0x7f7fbc0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3095927751 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc145b00 0x7f7fbc0cf0d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- 172.21.15.110:0/3095927751 >> 172.21.15.110:0/3095927751 conn(0x7f7fbc06d0f0 msgr2=0x7f7fbc06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:59.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- 172.21.15.110:0/3095927751 shutdown_connections 2024-09-10T09:59:59.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- 172.21.15.110:0/3095927751 wait complete. 2024-09-10T09:59:59.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 Processor -- start 2024-09-10T09:59:59.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- start start 2024-09-10T09:59:59.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fbc0b5690 0x7f7fbc13a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:59.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc145b00 0x7f7fbc15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc15d050 0x7f7fbc0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0719d0 con 0x7f7fbc0b5690 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc071850 con 0x7f7fbc15d050 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.310+0000 7f7fc1600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc071b50 con 0x7f7fbc145b00 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fbc0b5690 0x7f7fbc13a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fbc0b5690 0x7f7fbc13a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56622/0 (socket says 172.21.15.110:56622) 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 -- 172.21.15.110:0/3284873471 learned_addr learned my addr 172.21.15.110:0/3284873471 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbb400640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc145b00 0x7f7fbc15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fc0c00640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc15d050 0x7f7fbc0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 -- 172.21.15.110:0/3284873471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc145b00 msgr2=0x7f7fbc15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc145b00 0x7f7fbc15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 -- 172.21.15.110:0/3284873471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc15d050 msgr2=0x7f7fbc0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc15d050 0x7f7fbc0b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 -- 172.21.15.110:0/3284873471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc13cc90 con 0x7f7fbc0b5690 2024-09-10T09:59:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbbe00640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fbc0b5690 0x7f7fbc13a300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0ae040 tx=0x7f7fac01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fb3e00640 1 -- 172.21.15.110:0/3284873471 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0b0050 con 0x7f7fbc0b5690 2024-09-10T09:59:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fb3e00640 1 -- 172.21.15.110:0/3284873471 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac0bc070 con 0x7f7fbc0b5690 2024-09-10T09:59:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc13cf70 con 0x7f7fbc0b5690 2024-09-10T09:59:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fb3e00640 1 -- 172.21.15.110:0/3284873471 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac006e20 con 0x7f7fbc0b5690 2024-09-10T09:59:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc13a840 con 0x7f7fbc0b5690 2024-09-10T09:59:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc0cf0d0 con 0x7f7fbc0b5690 2024-09-10T09:59:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fb3e00640 1 -- 172.21.15.110:0/3284873471 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f7fac0b5810 con 0x7f7fbc0b5690 2024-09-10T09:59:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fb3e00640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7f9c067b00 0x7f7f9c069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fb3e00640 1 -- 172.21.15.110:0/3284873471 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7fac1207c0 con 0x7f7fbc0b5690 2024-09-10T09:59:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.314+0000 7f7fbb400640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7f9c067b00 0x7f7f9c069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.318+0000 7f7fbb400640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7f9c067b00 0x7f7f9c069fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc166f50 tx=0x7f7fa8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:59.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.322+0000 7f7fb3e00640 1 -- 172.21.15.110:0/3284873471 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7fac016090 con 0x7f7fbc0b5690 2024-09-10T09:59:59.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.510+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fbc13ea40 con 0x7f7fbc0b5690 2024-09-10T09:59:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fb3e00640 1 -- 172.21.15.110:0/3284873471 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f7fac0f1070 con 0x7f7fbc0b5690 2024-09-10T09:59:59.518 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T09:59:59.518 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T09:59:59.518 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T09:59:59.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7f9c067b00 msgr2=0x7f7f9c069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:59.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7f9c067b00 0x7f7f9c069fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc166f50 tx=0x7f7fa8009040 comp rx=0 tx=0).stop 2024-09-10T09:59:59.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fbc0b5690 msgr2=0x7f7fbc13a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:59.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fbc0b5690 0x7f7fbc13a300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0ae040 tx=0x7f7fac01ace0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 shutdown_connections 2024-09-10T09:59:59.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7f9c067b00 0x7f7f9c069fc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fbc15d050 0x7f7fbc0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7fbc145b00 0x7f7fbc15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 --2- 172.21.15.110:0/3284873471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fbc0b5690 0x7f7fbc13a300 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 >> 172.21.15.110:0/3284873471 conn(0x7f7fbc06d0f0 msgr2=0x7f7fbc15eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 shutdown_connections 2024-09-10T09:59:59.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T09:59:59.514+0000 7f7fc1600640 1 -- 172.21.15.110:0/3284873471 wait complete. 2024-09-10T09:59:59.539 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T09:59:59.696 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.155772218999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.391238894966467,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T09:59:59.697 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.391238894966467 }, "state": "open", "uptime": 12.155772219 } ] 2024-09-10T09:59:59.697 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:04.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:04.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.838+0000 7f7634a00640 1 Processor -- start 2024-09-10T10:00:04.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.838+0000 7f7634a00640 1 -- start start 2024-09-10T10:00:04.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f7634a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f76300738f0 0x7f76300b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:04.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f7634a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f763006d020 con 0x7f763015e470 2024-09-10T10:00:04.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f7634a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76300b5c60 con 0x7f7630162020 2024-09-10T10:00:04.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f7634a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f76300b6870 con 0x7f76300738f0 2024-09-10T10:00:04.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f762f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f763015e470 0x7f76300731e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59334/0 (socket says 172.21.15.110:59334) 2024-09-10T10:00:04.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f762f400640 1 -- 172.21.15.110:0/1082694482 learned_addr learned my addr 172.21.15.110:0/1082694482 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:00:04.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f7627e00640 1 --2- 172.21.15.110:0/1082694482 >> v2:172.21.15.145:3300/0 conn(0x7f76300738f0 0x7f76300b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f762ea00640 1 -- 172.21.15.110:0/1082694482 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 488363592 0 0) 0x7f763006d020 con 0x7f763015e470 2024-09-10T10:00:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f762ea00640 1 -- 172.21.15.110:0/1082694482 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7614003530 con 0x7f763015e470 2024-09-10T10:00:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f762ea00640 1 -- 172.21.15.110:0/1082694482 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 845231338 0 0) 0x7f76300b5c60 con 0x7f7630162020 2024-09-10T10:00:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f762ea00640 1 -- 172.21.15.110:0/1082694482 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f763006d020 con 0x7f7630162020 2024-09-10T10:00:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f762ea00640 1 -- 172.21.15.110:0/1082694482 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1084908599 0 0) 0x7f7614003530 con 0x7f763015e470 2024-09-10T10:00:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f7627e00640 1 -- 172.21.15.110:0/1082694482 >> v1:172.21.15.145:6789/0 conn(0x7f7630162020 legacy=0x7f763006d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f7627e00640 1 -- 172.21.15.110:0/1082694482 >> v1:172.21.15.110:6789/0 conn(0x7f763015e470 legacy=0x7f76300731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.842+0000 7f7627e00640 1 -- 172.21.15.110:0/1082694482 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76300b5f40 con 0x7f76300738f0 2024-09-10T10:00:04.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7627e00640 1 --2- 172.21.15.110:0/1082694482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 0x7f76300b55d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f76180097c0 tx=0x7f761801ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea8ba2aec0a014a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f762ea00640 1 -- 172.21.15.110:0/1082694482 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180b5070 con 0x7f76300738f0 2024-09-10T10:00:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f762ea00640 1 -- 172.21.15.110:0/1082694482 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76180a8630 con 0x7f76300738f0 2024-09-10T10:00:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f762ea00640 1 -- 172.21.15.110:0/1082694482 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76200016a0 con 0x7f76300738f0 2024-09-10T10:00:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7634a00640 1 -- 172.21.15.110:0/1082694482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 msgr2=0x7f76300b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:04.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7634a00640 1 --2- 172.21.15.110:0/1082694482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 0x7f76300b55d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f76180097c0 tx=0x7f761801ebb0 comp rx=0 tx=0).stop 2024-09-10T10:00:04.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7634a00640 1 -- 172.21.15.110:0/1082694482 shutdown_connections 2024-09-10T10:00:04.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7634a00640 1 --2- 172.21.15.110:0/1082694482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 0x7f76300b55d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:04.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7634a00640 1 -- 172.21.15.110:0/1082694482 >> 172.21.15.110:0/1082694482 conn(0x7f76300b42d0 msgr2=0x7f76300b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:04.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7634a00640 1 -- 172.21.15.110:0/1082694482 shutdown_connections 2024-09-10T10:00:04.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7634a00640 1 -- 172.21.15.110:0/1082694482 wait complete. 2024-09-10T10:00:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.846+0000 7f7634a00640 1 Processor -- start 2024-09-10T10:00:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 -- start start 2024-09-10T10:00:04.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 0x7f7630152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f763015e470 0x7f763015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7630162020 0x7f763015cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f763015e850 con 0x7f763015e470 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76300b6500 con 0x7f76300738f0 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f763006de30 con 0x7f7630162020 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f763015e470 0x7f763015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f763015e470 0x7f763015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56636/0 (socket says 172.21.15.110:56636) 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 -- 172.21.15.110:0/2061219908 learned_addr learned my addr 172.21.15.110:0/2061219908 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f762f400640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 0x7f7630152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f762fe00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7630162020 0x7f763015cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 -- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7630162020 msgr2=0x7f763015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7630162020 0x7f763015cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 -- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 msgr2=0x7f7630152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 0x7f7630152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 -- 172.21.15.110:0/2061219908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763015ec20 con 0x7f763015e470 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f762fe00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7630162020 0x7f763015cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627e00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f763015e470 0x7f763015c7c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f76180097c0 tx=0x7f761801b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627400640 1 -- 172.21.15.110:0/2061219908 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180bf020 con 0x7f763015e470 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627400640 1 -- 172.21.15.110:0/2061219908 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7618004220 con 0x7f763015e470 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7627400640 1 -- 172.21.15.110:0/2061219908 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180b5040 con 0x7f763015e470 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763015ef00 con 0x7f763015e470 2024-09-10T10:00:04.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763015f4c0 con 0x7f763015e470 2024-09-10T10:00:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.850+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76300b5c60 con 0x7f763015e470 2024-09-10T10:00:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.854+0000 7f7627400640 1 -- 172.21.15.110:0/2061219908 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f76180c2070 con 0x7f763015e470 2024-09-10T10:00:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.854+0000 7f7627400640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f760c067b00 0x7f760c069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.854+0000 7f7627400640 1 -- 172.21.15.110:0/2061219908 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f7618122420 con 0x7f763015e470 2024-09-10T10:00:04.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.854+0000 7f762f400640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f760c067b00 0x7f760c069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:04.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.854+0000 7f762f400640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f760c067b00 0x7f760c069fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7620000c10 tx=0x7f7620001050 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:04.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:04.858+0000 7f7627400640 1 -- 172.21.15.110:0/2061219908 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7618018710 con 0x7f763015e470 2024-09-10T10:00:05.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76300b35b0 con 0x7f763015e470 2024-09-10T10:00:05.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7627400640 1 -- 172.21.15.110:0/2061219908 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f76180f2b20 con 0x7f763015e470 2024-09-10T10:00:05.058 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:05.059 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:05.059 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f760c067b00 msgr2=0x7f760c069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f760c067b00 0x7f760c069fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7620000c10 tx=0x7f7620001050 comp rx=0 tx=0).stop 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f763015e470 msgr2=0x7f763015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f763015e470 0x7f763015c7c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f76180097c0 tx=0x7f761801b950 comp rx=0 tx=0).stop 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 shutdown_connections 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f760c067b00 0x7f760c069fc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7630162020 0x7f763015cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f763015e470 0x7f763015c7c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 --2- 172.21.15.110:0/2061219908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76300738f0 0x7f7630152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:05.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 >> 172.21.15.110:0/2061219908 conn(0x7f76300b42d0 msgr2=0x7f763006d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:05.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 shutdown_connections 2024-09-10T10:00:05.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:05.054+0000 7f7634a00640 1 -- 172.21.15.110:0/2061219908 wait complete. 2024-09-10T10:00:05.080 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:05.240 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.699677945000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.118982882223868,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:05.240 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.118982882223868 }, "state": "open", "uptime": 17.699677945 } ] 2024-09-10T10:00:05.240 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:10.241 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:10.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc16a200640 1 Processor -- start 2024-09-10T10:00:10.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc16a200640 1 -- start start 2024-09-10T10:00:10.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc16a200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc1640b14a0 0x7fc1640b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:10.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc16a200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc164142350 con 0x7fc164073420 2024-09-10T10:00:10.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc16a200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc164142630 con 0x7fc164073800 2024-09-10T10:00:10.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc16a200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc1641602f0 con 0x7fc1640b14a0 2024-09-10T10:00:10.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc169800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc1640b14a0 0x7fc1640b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:10.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc169800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc1640b14a0 0x7fc1640b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38120/0 (socket says 172.21.15.110:38120) 2024-09-10T10:00:10.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.382+0000 7fc169800640 1 -- 172.21.15.110:0/1639586996 learned_addr learned my addr 172.21.15.110:0/1639586996 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:10.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc169800640 1 -- 172.21.15.110:0/1639586996 >> v1:172.21.15.145:6789/0 conn(0x7fc164073800 legacy=0x7fc1641419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:10.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc169800640 1 -- 172.21.15.110:0/1639586996 >> v1:172.21.15.110:6789/0 conn(0x7fc164073420 legacy=0x7fc164160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:10.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc169800640 1 -- 172.21.15.110:0/1639586996 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16415ff80 con 0x7fc1640b14a0 2024-09-10T10:00:10.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc169800640 1 --2- 172.21.15.110:0/1639586996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1640b14a0 0x7fc1640b1930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc164160470 tx=0x7fc15c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1e127c5a27bd04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:10.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc163400640 1 -- 172.21.15.110:0/1639586996 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0b0070 con 0x7fc1640b14a0 2024-09-10T10:00:10.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc163400640 1 -- 172.21.15.110:0/1639586996 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15c0ab600 con 0x7fc1640b14a0 2024-09-10T10:00:10.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc163400640 1 -- 172.21.15.110:0/1639586996 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0ab9d0 con 0x7fc1640b14a0 2024-09-10T10:00:10.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc16a200640 1 -- 172.21.15.110:0/1639586996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1640b14a0 msgr2=0x7fc1640b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:10.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc16a200640 1 --2- 172.21.15.110:0/1639586996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1640b14a0 0x7fc1640b1930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc164160470 tx=0x7fc15c01c470 comp rx=0 tx=0).stop 2024-09-10T10:00:10.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc16a200640 1 -- 172.21.15.110:0/1639586996 shutdown_connections 2024-09-10T10:00:10.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc16a200640 1 --2- 172.21.15.110:0/1639586996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1640b14a0 0x7fc1640b1930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:10.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.386+0000 7fc16a200640 1 -- 172.21.15.110:0/1639586996 >> 172.21.15.110:0/1639586996 conn(0x7fc16406f440 msgr2=0x7fc16406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:10.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 -- 172.21.15.110:0/1639586996 shutdown_connections 2024-09-10T10:00:10.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 -- 172.21.15.110:0/1639586996 wait complete. 2024-09-10T10:00:10.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 Processor -- start 2024-09-10T10:00:10.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 -- start start 2024-09-10T10:00:10.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc164073420 0x7fc16415b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:10.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc164073800 0x7fc164138a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:10.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1640b14a0 0x7fc1641404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:10.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc16415d480 con 0x7fc164073420 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc16415d300 con 0x7fc164073800 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc164169ff0 con 0x7fc1640b14a0 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc164073420 0x7fc16415b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc164073420 0x7fc16415b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38122/0 (socket says 172.21.15.110:38122) 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 -- 172.21.15.110:0/1767529602 learned_addr learned my addr 172.21.15.110:0/1767529602 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 -- 172.21.15.110:0/1767529602 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1640b14a0 msgr2=0x7fc1641404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1640b14a0 0x7fc1641404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 -- 172.21.15.110:0/1767529602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc164073800 msgr2=0x7fc164138a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc163e00640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc164073800 0x7fc164138a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc164073800 0x7fc164138a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 -- 172.21.15.110:0/1767529602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc164137380 con 0x7fc164073420 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc163e00640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc164073800 0x7fc164138a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:10.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc168e00640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc164073420 0x7fc16415b590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c730 tx=0x7fc15400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:10.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc160c00640 1 -- 172.21.15.110:0/1767529602 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154016020 con 0x7fc164073420 2024-09-10T10:00:10.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc160c00640 1 -- 172.21.15.110:0/1767529602 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc154006d60 con 0x7fc164073420 2024-09-10T10:00:10.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.390+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16415e670 con 0x7fc164073420 2024-09-10T10:00:10.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.394+0000 7fc160c00640 1 -- 172.21.15.110:0/1767529602 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154003bc0 con 0x7fc164073420 2024-09-10T10:00:10.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.394+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16415bad0 con 0x7fc164073420 2024-09-10T10:00:10.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.394+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1640b1930 con 0x7fc164073420 2024-09-10T10:00:10.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.394+0000 7fc160c00640 1 -- 172.21.15.110:0/1767529602 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fc15400a7f0 con 0x7fc164073420 2024-09-10T10:00:10.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.394+0000 7fc160c00640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc144067b00 0x7fc144069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:10.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.394+0000 7fc160c00640 1 -- 172.21.15.110:0/1767529602 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fc154085d50 con 0x7fc164073420 2024-09-10T10:00:10.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.394+0000 7fc163e00640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc144067b00 0x7fc144069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:10.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.394+0000 7fc163e00640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc144067b00 0x7fc144069fc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1641355b0 tx=0x7fc150008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.398+0000 7fc160c00640 1 -- 172.21.15.110:0/1767529602 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc154020020 con 0x7fc164073420 2024-09-10T10:00:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.586+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc16415c390 con 0x7fc164073420 2024-09-10T10:00:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc160c00640 1 -- 172.21.15.110:0/1767529602 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fc1540079f0 con 0x7fc164073420 2024-09-10T10:00:10.597 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:10.597 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:10.598 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:10.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc144067b00 msgr2=0x7fc144069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:10.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc144067b00 0x7fc144069fc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1641355b0 tx=0x7fc150008040 comp rx=0 tx=0).stop 2024-09-10T10:00:10.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc164073420 msgr2=0x7fc16415b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:10.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc164073420 0x7fc16415b590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c730 tx=0x7fc15400cbf0 comp rx=0 tx=0).stop 2024-09-10T10:00:10.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 shutdown_connections 2024-09-10T10:00:10.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc144067b00 0x7fc144069fc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:10.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1640b14a0 0x7fc1641404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:10.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc164073800 0x7fc164138a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:10.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 --2- 172.21.15.110:0/1767529602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc164073420 0x7fc16415b590 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:10.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 >> 172.21.15.110:0/1767529602 conn(0x7fc16406f440 msgr2=0x7fc1640b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:10.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 shutdown_connections 2024-09-10T10:00:10.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:10.590+0000 7fc16a200640 1 -- 172.21.15.110:0/1767529602 wait complete. 2024-09-10T10:00:10.615 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:10.773 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":23.235506618999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.850581104335145,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:10.773 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.850581104335145 }, "state": "open", "uptime": 23.235506619 } ] 2024-09-10T10:00:10.774 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:15.775 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:15.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.918+0000 7fb753200640 1 Processor -- start 2024-09-10T10:00:15.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.918+0000 7fb753200640 1 -- start start 2024-09-10T10:00:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb753200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c0b5690 0x7fb74c0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb753200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb74c0cf7e0 con 0x7fb74c15c940 2024-09-10T10:00:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb753200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb74c071db0 con 0x7fb74c145b00 2024-09-10T10:00:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb753200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb74c0b6010 con 0x7fb74c0b5690 2024-09-10T10:00:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb752800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb74c15c940 0x7fb74c15cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51272/0 (socket says 172.21.15.110:51272) 2024-09-10T10:00:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb752800640 1 -- 172.21.15.110:0/1888768887 learned_addr learned my addr 172.21.15.110:0/1888768887 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:00:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb751400640 1 --2- 172.21.15.110:0/1888768887 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c0b5690 0x7fb74c0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:15.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb750a00640 1 -- 172.21.15.110:0/1888768887 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042200414 0 0) 0x7fb74c0cf7e0 con 0x7fb74c15c940 2024-09-10T10:00:15.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb750a00640 1 -- 172.21.15.110:0/1888768887 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb734003530 con 0x7fb74c15c940 2024-09-10T10:00:15.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb750a00640 1 -- 172.21.15.110:0/1888768887 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695467810 0 0) 0x7fb74c071db0 con 0x7fb74c145b00 2024-09-10T10:00:15.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb751400640 1 -- 172.21.15.110:0/1888768887 >> v1:172.21.15.145:6789/0 conn(0x7fb74c145b00 legacy=0x7fb74c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:15.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb751400640 1 -- 172.21.15.110:0/1888768887 >> v1:172.21.15.110:6789/0 conn(0x7fb74c15c940 legacy=0x7fb74c15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:15.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.922+0000 7fb751400640 1 -- 172.21.15.110:0/1888768887 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c0ce5f0 con 0x7fb74c0b5690 2024-09-10T10:00:15.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb751400640 1 --2- 172.21.15.110:0/1888768887 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c0b5690 0x7fb74c0b5ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0097f0 tx=0x7fb73c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ae5ad594e23879 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:15.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb750a00640 1 -- 172.21.15.110:0/1888768887 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c0b0070 con 0x7fb74c0b5690 2024-09-10T10:00:15.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb750a00640 1 -- 172.21.15.110:0/1888768887 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73c0ab600 con 0x7fb74c0b5690 2024-09-10T10:00:15.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb750a00640 1 -- 172.21.15.110:0/1888768887 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c0ab9d0 con 0x7fb74c0b5690 2024-09-10T10:00:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 -- 172.21.15.110:0/1888768887 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c0b5690 msgr2=0x7fb74c0b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 --2- 172.21.15.110:0/1888768887 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c0b5690 0x7fb74c0b5ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0097f0 tx=0x7fb73c01c4e0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 -- 172.21.15.110:0/1888768887 shutdown_connections 2024-09-10T10:00:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 --2- 172.21.15.110:0/1888768887 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c0b5690 0x7fb74c0b5ad0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 -- 172.21.15.110:0/1888768887 >> 172.21.15.110:0/1888768887 conn(0x7fb74c06d0f0 msgr2=0x7fb74c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 -- 172.21.15.110:0/1888768887 shutdown_connections 2024-09-10T10:00:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 -- 172.21.15.110:0/1888768887 wait complete. 2024-09-10T10:00:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 Processor -- start 2024-09-10T10:00:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.926+0000 7fb753200640 1 -- start start 2024-09-10T10:00:15.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb74c0b5690 0x7fb74c15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:15.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb74c145b00 0x7fb74c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:15.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c15c940 0x7fb74c0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:15.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c0719d0 con 0x7fb74c145b00 2024-09-10T10:00:15.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c071850 con 0x7fb74c0b5690 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb74c071b50 con 0x7fb74c15c940 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb74c145b00 0x7fb74c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb74c145b00 0x7fb74c15b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38124/0 (socket says 172.21.15.110:38124) 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 -- 172.21.15.110:0/1841544723 learned_addr learned my addr 172.21.15.110:0/1841544723 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb752800640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c15c940 0x7fb74c0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751e00640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb74c0b5690 0x7fb74c15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 -- 172.21.15.110:0/1841544723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c15c940 msgr2=0x7fb74c0c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c15c940 0x7fb74c0c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 -- 172.21.15.110:0/1841544723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb74c0b5690 msgr2=0x7fb74c15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb74c0b5690 0x7fb74c15bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 -- 172.21.15.110:0/1841544723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c140f70 con 0x7fb74c145b00 2024-09-10T10:00:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb751400640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb74c145b00 0x7fb74c15b190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0097f0 tx=0x7fb73c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb746000640 1 -- 172.21.15.110:0/1841544723 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c0bb020 con 0x7fb74c145b00 2024-09-10T10:00:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb746000640 1 -- 172.21.15.110:0/1841544723 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73c0b1040 con 0x7fb74c145b00 2024-09-10T10:00:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74c141250 con 0x7fb74c145b00 2024-09-10T10:00:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb746000640 1 -- 172.21.15.110:0/1841544723 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c0b0050 con 0x7fb74c145b00 2024-09-10T10:00:15.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74c15b6d0 con 0x7fb74c145b00 2024-09-10T10:00:15.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74c0b5ad0 con 0x7fb74c145b00 2024-09-10T10:00:15.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.930+0000 7fb746000640 1 -- 172.21.15.110:0/1841544723 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fb73c0be070 con 0x7fb74c145b00 2024-09-10T10:00:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.934+0000 7fb746000640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb730067b00 0x7fb730069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.934+0000 7fb746000640 1 -- 172.21.15.110:0/1841544723 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fb73c11f740 con 0x7fb74c145b00 2024-09-10T10:00:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.934+0000 7fb751e00640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb730067b00 0x7fb730069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.934+0000 7fb751e00640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb730067b00 0x7fb730069fc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb740000f70 tx=0x7fb740009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:15.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:15.938+0000 7fb746000640 1 -- 172.21.15.110:0/1841544723 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb73c0160d0 con 0x7fb74c145b00 2024-09-10T10:00:16.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.126+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb74c0b1bb0 con 0x7fb74c145b00 2024-09-10T10:00:16.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.126+0000 7fb746000640 1 -- 172.21.15.110:0/1841544723 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fb73c0efe00 con 0x7fb74c145b00 2024-09-10T10:00:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:16.133 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:16.133 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb730067b00 msgr2=0x7fb730069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb730067b00 0x7fb730069fc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb740000f70 tx=0x7fb740009040 comp rx=0 tx=0).stop 2024-09-10T10:00:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb74c145b00 msgr2=0x7fb74c15b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb74c145b00 0x7fb74c15b190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0097f0 tx=0x7fb73c01ad20 comp rx=0 tx=0).stop 2024-09-10T10:00:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 shutdown_connections 2024-09-10T10:00:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb730067b00 0x7fb730069fc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb74c15c940 0x7fb74c0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb74c145b00 0x7fb74c15b190 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 --2- 172.21.15.110:0/1841544723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb74c0b5690 0x7fb74c15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 >> 172.21.15.110:0/1841544723 conn(0x7fb74c06d0f0 msgr2=0x7fb74c15cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 shutdown_connections 2024-09-10T10:00:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:16.130+0000 7fb753200640 1 -- 172.21.15.110:0/1841544723 wait complete. 2024-09-10T10:00:16.155 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:16.315 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.775355219000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.585399254475828,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:16.315 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.585399254475828 }, "state": "open", "uptime": 28.775355219 } ] 2024-09-10T10:00:16.315 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:18.780 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:00:18.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:00:21.316 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:21.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 Processor -- start 2024-09-10T10:00:21.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- start start 2024-09-10T10:00:21.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6948155680 0x7f69480ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:21.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69480cd260 0x7f6948072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:21.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69480cd7e0 con 0x7f69480734b0 2024-09-10T10:00:21.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f69480cdac0 con 0x7f69480cd260 2024-09-10T10:00:21.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6948072770 con 0x7f6948155680 2024-09-10T10:00:21.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6947400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6948155680 0x7f69480ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:21.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6946a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69480cd260 0x7f6948072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:21.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6947400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6948155680 0x7f69480ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:59400/0 (socket says 172.21.15.110:59400) 2024-09-10T10:00:21.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6947400640 1 -- 172.21.15.110:0/912641262 learned_addr learned my addr 172.21.15.110:0/912641262 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:21.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6947400640 1 -- 172.21.15.110:0/912641262 >> v2:172.21.15.145:3300/0 conn(0x7f69480cd260 msgr2=0x7f6948072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:21.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6947400640 1 --2- 172.21.15.110:0/912641262 >> v2:172.21.15.145:3300/0 conn(0x7f69480cd260 0x7f6948072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6947400640 1 -- 172.21.15.110:0/912641262 >> v1:172.21.15.145:6789/0 conn(0x7f69480734b0 legacy=0x7f69480cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:21.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6947400640 1 -- 172.21.15.110:0/912641262 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6948072a50 con 0x7f6948155680 2024-09-10T10:00:21.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6947400640 1 --2- 172.21.15.110:0/912641262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6948155680 0x7f69480ccd20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6938009e50 tx=0x7f693801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e405e354ed0baff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:21.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6946000640 1 -- 172.21.15.110:0/912641262 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380a5920 con 0x7f6948155680 2024-09-10T10:00:21.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6946000640 1 -- 172.21.15.110:0/912641262 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69380a5ac0 con 0x7f6948155680 2024-09-10T10:00:21.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f6946000640 1 -- 172.21.15.110:0/912641262 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380ad3f0 con 0x7f6948155680 2024-09-10T10:00:21.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- 172.21.15.110:0/912641262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6948155680 msgr2=0x7f69480ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:21.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 --2- 172.21.15.110:0/912641262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6948155680 0x7f69480ccd20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6938009e50 tx=0x7f693801ca30 comp rx=0 tx=0).stop 2024-09-10T10:00:21.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- 172.21.15.110:0/912641262 shutdown_connections 2024-09-10T10:00:21.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 --2- 172.21.15.110:0/912641262 >> v2:172.21.15.145:3300/0 conn(0x7f69480cd260 0x7f6948072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 --2- 172.21.15.110:0/912641262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6948155680 0x7f69480ccd20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- 172.21.15.110:0/912641262 >> 172.21.15.110:0/912641262 conn(0x7f694806d0f0 msgr2=0x7f694806d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:21.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- 172.21.15.110:0/912641262 shutdown_connections 2024-09-10T10:00:21.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.461+0000 7f694cc00640 1 -- 172.21.15.110:0/912641262 wait complete. 2024-09-10T10:00:21.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 Processor -- start 2024-09-10T10:00:21.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 -- start start 2024-09-10T10:00:21.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480734b0 0x7f69481f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:21.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69480cd260 0x7f69480b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:21.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6948155680 0x7f694814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:21.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f694815d0e0 con 0x7f69480734b0 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f694815cf60 con 0x7f6948155680 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f69480cf9f0 con 0x7f69480cd260 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480734b0 0x7f69481f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480734b0 0x7f69481f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59270/0 (socket says 172.21.15.110:59270) 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 -- 172.21.15.110:0/142563745 learned_addr learned my addr 172.21.15.110:0/142563745 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6946a00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69480cd260 0x7f69480b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947e00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6948155680 0x7f694814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 -- 172.21.15.110:0/142563745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69480cd260 msgr2=0x7f69480b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69480cd260 0x7f69480b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 -- 172.21.15.110:0/142563745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6948155680 msgr2=0x7f694814f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:21.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6948155680 0x7f694814f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 -- 172.21.15.110:0/142563745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69481f7620 con 0x7f69480734b0 2024-09-10T10:00:21.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f6947400640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480734b0 0x7f69481f70e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f69380af040 tx=0x7f69380197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:21.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f693f400640 1 -- 172.21.15.110:0/142563745 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380be020 con 0x7f69480734b0 2024-09-10T10:00:21.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f693f400640 1 -- 172.21.15.110:0/142563745 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69380b8440 con 0x7f69480734b0 2024-09-10T10:00:21.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69480b6450 con 0x7f69480734b0 2024-09-10T10:00:21.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f693f400640 1 -- 172.21.15.110:0/142563745 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380a5920 con 0x7f69480734b0 2024-09-10T10:00:21.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.465+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6948168520 con 0x7f69480734b0 2024-09-10T10:00:21.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.469+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6948072f70 con 0x7f69480734b0 2024-09-10T10:00:21.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.469+0000 7f693f400640 1 -- 172.21.15.110:0/142563745 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f69380c1070 con 0x7f69480734b0 2024-09-10T10:00:21.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.469+0000 7f693f400640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6928067b00 0x7f6928069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:21.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.469+0000 7f693f400640 1 -- 172.21.15.110:0/142563745 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f69381206d0 con 0x7f69480734b0 2024-09-10T10:00:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.469+0000 7f6946a00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6928067b00 0x7f6928069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.469+0000 7f6946a00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6928067b00 0x7f6928069fc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6948167060 tx=0x7f6934009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:21.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.473+0000 7f693f400640 1 -- 172.21.15.110:0/142563745 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6938016620 con 0x7f69480734b0 2024-09-10T10:00:21.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6948159050 con 0x7f69480734b0 2024-09-10T10:00:21.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f693f400640 1 -- 172.21.15.110:0/142563745 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f69380f08e0 con 0x7f69480734b0 2024-09-10T10:00:21.672 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:21.672 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:21.672 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:21.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6928067b00 msgr2=0x7f6928069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:21.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6928067b00 0x7f6928069fc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6948167060 tx=0x7f6934009190 comp rx=0 tx=0).stop 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480734b0 msgr2=0x7f69481f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480734b0 0x7f69481f70e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f69380af040 tx=0x7f69380197d0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 shutdown_connections 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6928067b00 0x7f6928069fc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6948155680 0x7f694814f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69480cd260 0x7f69480b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 --2- 172.21.15.110:0/142563745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480734b0 0x7f69481f70e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 >> 172.21.15.110:0/142563745 conn(0x7f694806d0f0 msgr2=0x7f694815ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.665+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 shutdown_connections 2024-09-10T10:00:21.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:21.669+0000 7f694cc00640 1 -- 172.21.15.110:0/142563745 wait complete. 2024-09-10T10:00:21.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:21.853 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.315220029999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.323589279055341,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:21.854 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.32358927905534 }, "state": "open", "uptime": 34.31522003 } ] 2024-09-10T10:00:21.854 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:26.855 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:27.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ffc800640 1 Processor -- start 2024-09-10T10:00:27.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ffc800640 1 -- start start 2024-09-10T10:00:27.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ffc800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3ff406c360 0x7f3ff40d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:27.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ffc800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff40d06a0 0x7f3ff406f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:27.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ffc800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff40b3dd0 con 0x7f3ff415c6d0 2024-09-10T10:00:27.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ffc800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff406e6f0 con 0x7f3ff406c360 2024-09-10T10:00:27.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ffc800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40d0af0 con 0x7f3ff40d06a0 2024-09-10T10:00:27.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ff9400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3ff406c360 0x7f3ff40d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:27.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ff9400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3ff406c360 0x7f3ff40d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59284/0 (socket says 172.21.15.110:59284) 2024-09-10T10:00:27.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ff9400640 1 -- 172.21.15.110:0/1939955641 learned_addr learned my addr 172.21.15.110:0/1939955641 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:27.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ff9400640 1 -- 172.21.15.110:0/1939955641 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff40d06a0 msgr2=0x7f3ff406f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:27.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ff9400640 1 --2- 172.21.15.110:0/1939955641 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff40d06a0 0x7f3ff406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ff9400640 1 -- 172.21.15.110:0/1939955641 >> v1:172.21.15.110:6789/0 conn(0x7f3ff415c6d0 legacy=0x7f3ff406bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:27.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.001+0000 7f3ff9400640 1 -- 172.21.15.110:0/1939955641 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40d0d40 con 0x7f3ff406c360 2024-09-10T10:00:27.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ff9400640 1 --2- 172.21.15.110:0/1939955641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff406c360 0x7f3ff40d0160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009870 tx=0x7f3fe401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0d98624ee9cba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:27.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ff8a00640 1 -- 172.21.15.110:0/1939955641 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40b0070 con 0x7f3ff406c360 2024-09-10T10:00:27.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ff8a00640 1 -- 172.21.15.110:0/1939955641 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe40ab600 con 0x7f3ff406c360 2024-09-10T10:00:27.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ff8a00640 1 -- 172.21.15.110:0/1939955641 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40ab9d0 con 0x7f3ff406c360 2024-09-10T10:00:27.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 -- 172.21.15.110:0/1939955641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff406c360 msgr2=0x7f3ff40d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:27.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1939955641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff406c360 0x7f3ff40d0160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009870 tx=0x7f3fe401c440 comp rx=0 tx=0).stop 2024-09-10T10:00:27.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 -- 172.21.15.110:0/1939955641 shutdown_connections 2024-09-10T10:00:27.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1939955641 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff40d06a0 0x7f3ff406f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1939955641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff406c360 0x7f3ff40d0160 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 -- 172.21.15.110:0/1939955641 >> 172.21.15.110:0/1939955641 conn(0x7f3ff406d360 msgr2=0x7f3ff40b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:27.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 -- 172.21.15.110:0/1939955641 shutdown_connections 2024-09-10T10:00:27.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 -- 172.21.15.110:0/1939955641 wait complete. 2024-09-10T10:00:27.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.005+0000 7f3ffc800640 1 Processor -- start 2024-09-10T10:00:27.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 -- start start 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff406c360 0x7f3ff41427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff40d06a0 0x7f3ff40b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff415c6d0 0x7f3ff40b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4070010 con 0x7f3ff40d06a0 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff406fe90 con 0x7f3ff415c6d0 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff4070190 con 0x7f3ff406c360 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff40d06a0 0x7f3ff40b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff40d06a0 0x7f3ff40b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59288/0 (socket says 172.21.15.110:59288) 2024-09-10T10:00:27.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 -- 172.21.15.110:0/1873980045 learned_addr learned my addr 172.21.15.110:0/1873980045 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9e00640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff406c360 0x7f3ff41427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffa800640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff415c6d0 0x7f3ff40b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 -- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff406c360 msgr2=0x7f3ff41427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff406c360 0x7f3ff41427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 -- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff415c6d0 msgr2=0x7f3ff40b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff415c6d0 0x7f3ff40b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 -- 172.21.15.110:0/1873980045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff4143a30 con 0x7f3ff40d06a0 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9e00640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff406c360 0x7f3ff41427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ff9400640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff40d06a0 0x7f3ff40b5250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40ae040 tx=0x7f3fe40191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3fee000640 1 -- 172.21.15.110:0/1873980045 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40bb070 con 0x7f3ff40d06a0 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3fee000640 1 -- 172.21.15.110:0/1873980045 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe40b6630 con 0x7f3ff40d06a0 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3fee000640 1 -- 172.21.15.110:0/1873980045 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40b0040 con 0x7f3ff40d06a0 2024-09-10T10:00:27.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff4143d10 con 0x7f3ff40d06a0 2024-09-10T10:00:27.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41442d0 con 0x7f3ff40d06a0 2024-09-10T10:00:27.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.009+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff40d0160 con 0x7f3ff40d06a0 2024-09-10T10:00:27.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.013+0000 7f3fee000640 1 -- 172.21.15.110:0/1873980045 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f3fe40be070 con 0x7f3ff40d06a0 2024-09-10T10:00:27.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.013+0000 7f3fee000640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3fd8067b00 0x7f3fd8069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:27.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.013+0000 7f3fee000640 1 -- 172.21.15.110:0/1873980045 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f3fe411fa40 con 0x7f3ff40d06a0 2024-09-10T10:00:27.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.013+0000 7f3ff9e00640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3fd8067b00 0x7f3fd8069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:27.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.013+0000 7f3ff9e00640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3fd8067b00 0x7f3fd8069fc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8001670 tx=0x7f3fe8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:27.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.017+0000 7f3fee000640 1 -- 172.21.15.110:0/1873980045 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3fe4016150 con 0x7f3ff40d06a0 2024-09-10T10:00:27.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.205+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ff40cdd60 con 0x7f3ff40d06a0 2024-09-10T10:00:27.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.205+0000 7f3fee000640 1 -- 172.21.15.110:0/1873980045 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f3fe40f0100 con 0x7f3ff40d06a0 2024-09-10T10:00:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:27.209 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:27.210 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:27.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3fd8067b00 msgr2=0x7f3fd8069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:27.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3fd8067b00 0x7f3fd8069fc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8001670 tx=0x7f3fe8009040 comp rx=0 tx=0).stop 2024-09-10T10:00:27.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff40d06a0 msgr2=0x7f3ff40b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff40d06a0 0x7f3ff40b5250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40ae040 tx=0x7f3fe40191e0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 shutdown_connections 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3fd8067b00 0x7f3fd8069fc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff415c6d0 0x7f3ff40b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ff40d06a0 0x7f3ff40b5250 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 --2- 172.21.15.110:0/1873980045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ff406c360 0x7f3ff41427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 >> 172.21.15.110:0/1873980045 conn(0x7f3ff406d360 msgr2=0x7f3ff4156950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 shutdown_connections 2024-09-10T10:00:27.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:27.209+0000 7f3ffc800640 1 -- 172.21.15.110:0/1873980045 wait complete. 2024-09-10T10:00:27.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:27.389 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.855098726000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.065108421098842,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:27.390 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.06510842109884 }, "state": "open", "uptime": 39.855098726 } ] 2024-09-10T10:00:27.390 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:32.391 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:32.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.533+0000 7f2019600640 1 Processor -- start 2024-09-10T10:00:32.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.533+0000 7f2019600640 1 -- start start 2024-09-10T10:00:32.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2019600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2014072f70 0x7f2014073400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2019600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f20140cf2e0 0x7f20140cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2019600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20140cd7e0 con 0x7f2014155680 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2019600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f20140cfc90 con 0x7f20140cf2e0 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2019600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f20140cdac0 con 0x7f2014072f70 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2013e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2014155680 0x7f20140ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:56804/0 (socket says 172.21.15.110:56804) 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2013e00640 1 -- 172.21.15.110:0/3426009351 learned_addr learned my addr 172.21.15.110:0/3426009351 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2018c00640 1 --2- 172.21.15.110:0/3426009351 >> v2:172.21.15.110:3300/0 conn(0x7f20140cf2e0 0x7f20140cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2013400640 1 --2- 172.21.15.110:0/3426009351 >> v2:172.21.15.145:3300/0 conn(0x7f2014072f70 0x7f2014073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2018c00640 1 -- 172.21.15.110:0/3426009351 >> v2:172.21.15.145:3300/0 conn(0x7f2014072f70 msgr2=0x7f2014073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2018c00640 1 --2- 172.21.15.110:0/3426009351 >> v2:172.21.15.145:3300/0 conn(0x7f2014072f70 0x7f2014073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2018c00640 1 -- 172.21.15.110:0/3426009351 >> v1:172.21.15.110:6789/0 conn(0x7f2014155680 legacy=0x7f20140ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.537+0000 7f2018c00640 1 -- 172.21.15.110:0/3426009351 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2014072770 con 0x7f20140cf2e0 2024-09-10T10:00:32.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2018c00640 1 --2- 172.21.15.110:0/3426009351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f20140cf750 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f201415b760 tx=0x7f200c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eaed2985457096 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2012a00640 1 -- 172.21.15.110:0/3426009351 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0b2070 con 0x7f20140cf2e0 2024-09-10T10:00:32.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2012a00640 1 -- 172.21.15.110:0/3426009351 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c0a59e0 con 0x7f20140cf2e0 2024-09-10T10:00:32.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2012a00640 1 -- 172.21.15.110:0/3426009351 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0a5db0 con 0x7f20140cf2e0 2024-09-10T10:00:32.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 -- 172.21.15.110:0/3426009351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 msgr2=0x7f20140cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 --2- 172.21.15.110:0/3426009351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f20140cf750 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f201415b760 tx=0x7f200c01c900 comp rx=0 tx=0).stop 2024-09-10T10:00:32.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 -- 172.21.15.110:0/3426009351 shutdown_connections 2024-09-10T10:00:32.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 --2- 172.21.15.110:0/3426009351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f20140cf750 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 --2- 172.21.15.110:0/3426009351 >> v2:172.21.15.145:3300/0 conn(0x7f2014072f70 0x7f2014073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 -- 172.21.15.110:0/3426009351 >> 172.21.15.110:0/3426009351 conn(0x7f201406d0f0 msgr2=0x7f201406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:32.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 -- 172.21.15.110:0/3426009351 shutdown_connections 2024-09-10T10:00:32.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 -- 172.21.15.110:0/3426009351 wait complete. 2024-09-10T10:00:32.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 Processor -- start 2024-09-10T10:00:32.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 -- start start 2024-09-10T10:00:32.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2014072f70 0x7f2014155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.541+0000 7f2019600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f201414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2019600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2014155680 0x7f20140b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2019600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20140cd430 con 0x7f20140cf2e0 2024-09-10T10:00:32.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2019600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2014155a60 con 0x7f2014072f70 2024-09-10T10:00:32.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2019600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f201414efd0 con 0x7f2014155680 2024-09-10T10:00:32.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f201414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f201414cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39250/0 (socket says 172.21.15.110:39250) 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 -- 172.21.15.110:0/941215927 learned_addr learned my addr 172.21.15.110:0/941215927 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013e00640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2014072f70 0x7f2014155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2018c00640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2014155680 0x7f20140b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 -- 172.21.15.110:0/941215927 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2014155680 msgr2=0x7f20140b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2014155680 0x7f20140b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 -- 172.21.15.110:0/941215927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2014072f70 msgr2=0x7f2014155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2014072f70 0x7f2014155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 -- 172.21.15.110:0/941215927 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201414f2b0 con 0x7f20140cf2e0 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2013400640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f201414cc50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f200000d800 tx=0x7f200000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f200be00640 1 -- 172.21.15.110:0/941215927 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000b6c0 con 0x7f20140cf2e0 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f200be00640 1 -- 172.21.15.110:0/941215927 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2000019070 con 0x7f20140cf2e0 2024-09-10T10:00:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f200be00640 1 -- 172.21.15.110:0/941215927 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000ba60 con 0x7f20140cf2e0 2024-09-10T10:00:32.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201414f590 con 0x7f20140cf2e0 2024-09-10T10:00:32.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201414d190 con 0x7f20140cf2e0 2024-09-10T10:00:32.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2014073400 con 0x7f20140cf2e0 2024-09-10T10:00:32.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.545+0000 7f200be00640 1 -- 172.21.15.110:0/941215927 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f200000bc00 con 0x7f20140cf2e0 2024-09-10T10:00:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.549+0000 7f200be00640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1ff4067b00 0x7f1ff4069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.549+0000 7f200be00640 1 -- 172.21.15.110:0/941215927 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f20000864f0 con 0x7f20140cf2e0 2024-09-10T10:00:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.549+0000 7f2013e00640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1ff4067b00 0x7f1ff4069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.549+0000 7f2013e00640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1ff4067b00 0x7f1ff4069fc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2004000f70 tx=0x7f2004007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.553+0000 7f200be00640 1 -- 172.21.15.110:0/941215927 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f20000565a0 con 0x7f20140cf2e0 2024-09-10T10:00:32.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.741+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f201415b540 con 0x7f20140cf2e0 2024-09-10T10:00:32.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.741+0000 7f200be00640 1 -- 172.21.15.110:0/941215927 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f200005a8a0 con 0x7f20140cf2e0 2024-09-10T10:00:32.744 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:32.744 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:32.745 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:32.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1ff4067b00 msgr2=0x7f1ff4069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1ff4067b00 0x7f1ff4069fc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2004000f70 tx=0x7f2004007e90 comp rx=0 tx=0).stop 2024-09-10T10:00:32.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 msgr2=0x7f201414cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f201414cc50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f200000d800 tx=0x7f200000dcc0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 shutdown_connections 2024-09-10T10:00:32.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1ff4067b00 0x7f1ff4069fc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2014155680 0x7f20140b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20140cf2e0 0x7f201414cc50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 --2- 172.21.15.110:0/941215927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2014072f70 0x7f2014155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 >> 172.21.15.110:0/941215927 conn(0x7f201406d0f0 msgr2=0x7f20140b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:32.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 shutdown_connections 2024-09-10T10:00:32.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:32.745+0000 7f2019600640 1 -- 172.21.15.110:0/941215927 wait complete. 2024-09-10T10:00:32.766 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:32.926 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":45.390988659000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.810097561008547,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:32.927 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.810097561008547 }, "state": "open", "uptime": 45.390988659 } ] 2024-09-10T10:00:32.927 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:37.928 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:38.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9337000640 1 Processor -- start 2024-09-10T10:00:38.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9337000640 1 -- start start 2024-09-10T10:00:38.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9337000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f93300739a0 0x7f9330163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9337000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93300706a0 con 0x7f93300d0cb0 2024-09-10T10:00:38.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9337000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93300d10f0 con 0x7f933006eb30 2024-09-10T10:00:38.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9337000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f933015ee90 con 0x7f93300739a0 2024-09-10T10:00:38.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9335000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f93300739a0 0x7f9330163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9327e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f93300d0cb0 0x7f9330073290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:56818/0 (socket says 172.21.15.110:56818) 2024-09-10T10:00:38.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9327e00640 1 -- 172.21.15.110:0/2995501703 learned_addr learned my addr 172.21.15.110:0/2995501703 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:00:38.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9335000640 1 -- 172.21.15.110:0/2995501703 >> v1:172.21.15.145:6789/0 conn(0x7f933006eb30 legacy=0x7f9330071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:38.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9335000640 1 -- 172.21.15.110:0/2995501703 >> v1:172.21.15.110:6789/0 conn(0x7f93300d0cb0 legacy=0x7f9330073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.073+0000 7f9335000640 1 -- 172.21.15.110:0/2995501703 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93300d1370 con 0x7f93300739a0 2024-09-10T10:00:38.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f9335000640 1 --2- 172.21.15.110:0/2995501703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300739a0 0x7f9330163000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9328009d90 tx=0x7f932801c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1e3b4aa3185a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f932f400640 1 -- 172.21.15.110:0/2995501703 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280b0600 con 0x7f93300739a0 2024-09-10T10:00:38.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f932f400640 1 -- 172.21.15.110:0/2995501703 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93280b07a0 con 0x7f93300739a0 2024-09-10T10:00:38.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f932f400640 1 -- 172.21.15.110:0/2995501703 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280b0b70 con 0x7f93300739a0 2024-09-10T10:00:38.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f9337000640 1 -- 172.21.15.110:0/2995501703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300739a0 msgr2=0x7f9330163000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f9337000640 1 --2- 172.21.15.110:0/2995501703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300739a0 0x7f9330163000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9328009d90 tx=0x7f932801c6c0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f9337000640 1 -- 172.21.15.110:0/2995501703 shutdown_connections 2024-09-10T10:00:38.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f9337000640 1 --2- 172.21.15.110:0/2995501703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300739a0 0x7f9330163000 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f9337000640 1 -- 172.21.15.110:0/2995501703 >> 172.21.15.110:0/2995501703 conn(0x7f93300d25c0 msgr2=0x7f93300d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:38.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f9337000640 1 -- 172.21.15.110:0/2995501703 shutdown_connections 2024-09-10T10:00:38.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.077+0000 7f9337000640 1 -- 172.21.15.110:0/2995501703 wait complete. 2024-09-10T10:00:38.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 Processor -- start 2024-09-10T10:00:38.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 -- start start 2024-09-10T10:00:38.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f933006eb30 0x7f9330152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93300739a0 0x7f9330158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300d0cb0 0x7f9330158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f933006dd60 con 0x7f93300d0cb0 2024-09-10T10:00:38.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9330071c70 con 0x7f933006eb30 2024-09-10T10:00:38.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f933006dee0 con 0x7f93300739a0 2024-09-10T10:00:38.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300d0cb0 0x7f9330158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300d0cb0 0x7f9330158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39266/0 (socket says 172.21.15.110:39266) 2024-09-10T10:00:38.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 -- 172.21.15.110:0/256467930 learned_addr learned my addr 172.21.15.110:0/256467930 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:38.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f932fe00640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f933006eb30 0x7f9330152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9327e00640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93300739a0 0x7f9330158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 -- 172.21.15.110:0/256467930 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93300739a0 msgr2=0x7f9330158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93300739a0 0x7f9330158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 -- 172.21.15.110:0/256467930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f933006eb30 msgr2=0x7f9330152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f933006eb30 0x7f9330152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 -- 172.21.15.110:0/256467930 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9330155ad0 con 0x7f93300d0cb0 2024-09-10T10:00:38.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9335000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300d0cb0 0x7f9330158b80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f93280192d0 tx=0x7f9328019460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f932cc00640 1 -- 172.21.15.110:0/256467930 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280ba5f0 con 0x7f93300d0cb0 2024-09-10T10:00:38.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f932cc00640 1 -- 172.21.15.110:0/256467930 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93280bf070 con 0x7f93300d0cb0 2024-09-10T10:00:38.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f932cc00640 1 -- 172.21.15.110:0/256467930 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280ba9b0 con 0x7f93300d0cb0 2024-09-10T10:00:38.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933015f1a0 con 0x7f93300d0cb0 2024-09-10T10:00:38.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933015f760 con 0x7f93300d0cb0 2024-09-10T10:00:38.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.081+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9330163000 con 0x7f93300d0cb0 2024-09-10T10:00:38.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.085+0000 7f932cc00640 1 -- 172.21.15.110:0/256467930 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f93280bab50 con 0x7f93300d0cb0 2024-09-10T10:00:38.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.085+0000 7f932cc00640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f930c067b00 0x7f930c069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.085+0000 7f932cc00640 1 -- 172.21.15.110:0/256467930 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f9328121470 con 0x7f93300d0cb0 2024-09-10T10:00:38.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.085+0000 7f932fe00640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f930c067b00 0x7f930c069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.085+0000 7f932fe00640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f930c067b00 0x7f930c069fc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93301fa880 tx=0x7f9320009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.089+0000 7f932cc00640 1 -- 172.21.15.110:0/256467930 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f93280b7070 con 0x7f93300d0cb0 2024-09-10T10:00:38.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.277+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93300b74e0 con 0x7f93300d0cb0 2024-09-10T10:00:38.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.277+0000 7f932cc00640 1 -- 172.21.15.110:0/256467930 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f93280ac030 con 0x7f93300d0cb0 2024-09-10T10:00:38.280 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:38.280 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:38.280 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:38.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f930c067b00 msgr2=0x7f930c069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f930c067b00 0x7f930c069fc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93301fa880 tx=0x7f9320009060 comp rx=0 tx=0).stop 2024-09-10T10:00:38.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300d0cb0 msgr2=0x7f9330158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300d0cb0 0x7f9330158b80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f93280192d0 tx=0x7f9328019460 comp rx=0 tx=0).stop 2024-09-10T10:00:38.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 shutdown_connections 2024-09-10T10:00:38.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f930c067b00 0x7f930c069fc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93300d0cb0 0x7f9330158b80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93300739a0 0x7f9330158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 --2- 172.21.15.110:0/256467930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f933006eb30 0x7f9330152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 >> 172.21.15.110:0/256467930 conn(0x7f93300d25c0 msgr2=0x7f93300b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:38.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 shutdown_connections 2024-09-10T10:00:38.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:38.281+0000 7f9337000640 1 -- 172.21.15.110:0/256467930 wait complete. 2024-09-10T10:00:38.302 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:38.460 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":50.922888806000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.558507973710626,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:38.461 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.558507973710626 }, "state": "open", "uptime": 50.922888806 } ] 2024-09-10T10:00:38.461 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:43.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:43.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e85600640 1 Processor -- start 2024-09-10T10:00:43.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e85600640 1 -- start start 2024-09-10T10:00:43.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e85600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e800b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e85600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1e80162020 0x7f1e8006d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e85600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8006d020 con 0x7f1e8015e470 2024-09-10T10:00:43.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e85600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1e800b6870 con 0x7f1e80162020 2024-09-10T10:00:43.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e85600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1e800b5c60 con 0x7f1e800738f0 2024-09-10T10:00:43.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e84c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1e80162020 0x7f1e8006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e84c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1e80162020 0x7f1e8006d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41326/0 (socket says 172.21.15.110:41326) 2024-09-10T10:00:43.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e84c00640 1 -- 172.21.15.110:0/1069452302 learned_addr learned my addr 172.21.15.110:0/1069452302 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:43.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e84c00640 1 -- 172.21.15.110:0/1069452302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 msgr2=0x7f1e800b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e7f400640 1 --2- 172.21.15.110:0/1069452302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e800b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e84c00640 1 --2- 172.21.15.110:0/1069452302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e800b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e84c00640 1 -- 172.21.15.110:0/1069452302 >> v1:172.21.15.145:6789/0 conn(0x7f1e8015e470 legacy=0x7f1e800731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e84c00640 1 -- 172.21.15.110:0/1069452302 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e800b5f40 con 0x7f1e80162020 2024-09-10T10:00:43.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.605+0000 7f1e7f400640 1 --2- 172.21.15.110:0/1069452302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e800b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e84c00640 1 --2- 172.21.15.110:0/1069452302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e80162020 0x7f1e8006d720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009e50 tx=0x7f1e7801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2298bd7629cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e7ea00640 1 -- 172.21.15.110:0/1069452302 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e780b4070 con 0x7f1e80162020 2024-09-10T10:00:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e7ea00640 1 -- 172.21.15.110:0/1069452302 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e780af400 con 0x7f1e80162020 2024-09-10T10:00:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e7ea00640 1 -- 172.21.15.110:0/1069452302 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e780af7d0 con 0x7f1e80162020 2024-09-10T10:00:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 -- 172.21.15.110:0/1069452302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e80162020 msgr2=0x7f1e8006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 --2- 172.21.15.110:0/1069452302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e80162020 0x7f1e8006d720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009e50 tx=0x7f1e7801f1e0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 -- 172.21.15.110:0/1069452302 shutdown_connections 2024-09-10T10:00:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 --2- 172.21.15.110:0/1069452302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e80162020 0x7f1e8006d720 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 --2- 172.21.15.110:0/1069452302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e800b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 -- 172.21.15.110:0/1069452302 >> 172.21.15.110:0/1069452302 conn(0x7f1e800b42d0 msgr2=0x7f1e800b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:43.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 -- 172.21.15.110:0/1069452302 shutdown_connections 2024-09-10T10:00:43.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 -- 172.21.15.110:0/1069452302 wait complete. 2024-09-10T10:00:43.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 Processor -- start 2024-09-10T10:00:43.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.609+0000 7f1e85600640 1 -- start start 2024-09-10T10:00:43.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e80145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8015e470 0x7f1e80146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e80162020 0x7f1e80155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8015e850 con 0x7f1e8015e470 2024-09-10T10:00:43.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1e800b6500 con 0x7f1e80162020 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8006dc60 con 0x7f1e800738f0 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8015e470 0x7f1e80146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8015e470 0x7f1e80146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41340/0 (socket says 172.21.15.110:41340) 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 -- 172.21.15.110:0/2757415123 learned_addr learned my addr 172.21.15.110:0/2757415123 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7fe00640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e80145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e84c00640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e80162020 0x7f1e80155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 -- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 msgr2=0x7f1e80145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e80145ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 -- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e80162020 msgr2=0x7f1e80155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e80162020 0x7f1e80155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 -- 172.21.15.110:0/2757415123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e80156100 con 0x7f1e8015e470 2024-09-10T10:00:43.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7fe00640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e80145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:43.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e84c00640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e80162020 0x7f1e80155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:43.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e7f400640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8015e470 0x7f1e80146770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c7f0 tx=0x7f1e6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e77e00640 1 -- 172.21.15.110:0/2757415123 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c015070 con 0x7f1e8015e470 2024-09-10T10:00:43.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e77e00640 1 -- 172.21.15.110:0/2757415123 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c007820 con 0x7f1e8015e470 2024-09-10T10:00:43.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e77e00640 1 -- 172.21.15.110:0/2757415123 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c007bd0 con 0x7f1e8015e470 2024-09-10T10:00:43.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e801538f0 con 0x7f1e8015e470 2024-09-10T10:00:43.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e80153eb0 con 0x7f1e8015e470 2024-09-10T10:00:43.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.613+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8006d700 con 0x7f1e8015e470 2024-09-10T10:00:43.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.617+0000 7f1e77e00640 1 -- 172.21.15.110:0/2757415123 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f1e6c018070 con 0x7f1e8015e470 2024-09-10T10:00:43.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.617+0000 7f1e77e00640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1e60067b00 0x7f1e60069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.617+0000 7f1e77e00640 1 -- 172.21.15.110:0/2757415123 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f1e6c086380 con 0x7f1e8015e470 2024-09-10T10:00:43.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.617+0000 7f1e7fe00640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1e60067b00 0x7f1e60069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.617+0000 7f1e7fe00640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1e60067b00 0x7f1e60069fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e70003ec0 tx=0x7f1e70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.621+0000 7f1e77e00640 1 -- 172.21.15.110:0/2757415123 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1e6c056a00 con 0x7f1e8015e470 2024-09-10T10:00:43.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.809+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e8015b030 con 0x7f1e8015e470 2024-09-10T10:00:43.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.809+0000 7f1e77e00640 1 -- 172.21.15.110:0/2757415123 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f1e6c056300 con 0x7f1e8015e470 2024-09-10T10:00:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:43.813 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:43.813 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:43.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1e60067b00 msgr2=0x7f1e60069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1e60067b00 0x7f1e60069fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e70003ec0 tx=0x7f1e70009040 comp rx=0 tx=0).stop 2024-09-10T10:00:43.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8015e470 msgr2=0x7f1e80146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8015e470 0x7f1e80146770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c7f0 tx=0x7f1e6c00ccb0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 shutdown_connections 2024-09-10T10:00:43.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1e60067b00 0x7f1e60069fc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1e80162020 0x7f1e80155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e8015e470 0x7f1e80146770 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 --2- 172.21.15.110:0/2757415123 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1e800738f0 0x7f1e80145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 >> 172.21.15.110:0/2757415123 conn(0x7f1e800b42d0 msgr2=0x7f1e80162510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:43.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 shutdown_connections 2024-09-10T10:00:43.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:43.813+0000 7f1e85600640 1 -- 172.21.15.110:0/2757415123 wait complete. 2024-09-10T10:00:43.834 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:43.992 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":56.454797108000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.310113227452408,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:43.992 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.31011322745241 }, "state": "open", "uptime": 56.454797108 } ] 2024-09-10T10:00:43.993 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:48.836 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:00:48.840 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:00:48.994 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:49.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.137+0000 7ff56f600640 1 Processor -- start 2024-09-10T10:00:49.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.137+0000 7ff56f600640 1 -- start start 2024-09-10T10:00:49.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.137+0000 7ff56f600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff5680cfe30 0x7ff568071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.137+0000 7ff56f600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff56806d2d0 con 0x7ff5680cd510 2024-09-10T10:00:49.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.137+0000 7ff56f600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff56806d5b0 con 0x7ff56806e7a0 2024-09-10T10:00:49.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.137+0000 7ff56f600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff56815d9a0 con 0x7ff5680cfe30 2024-09-10T10:00:49.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.137+0000 7ff567e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff5680cd510 0x7ff56806e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41498/0 (socket says 172.21.15.110:41498) 2024-09-10T10:00:49.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.137+0000 7ff567e00640 1 -- 172.21.15.110:0/2426971422 learned_addr learned my addr 172.21.15.110:0/2426971422 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:00:49.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff56cc00640 1 --2- 172.21.15.110:0/2426971422 >> v2:172.21.15.145:3300/0 conn(0x7ff5680cfe30 0x7ff568071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1304334284 0 0) 0x7ff56806d2d0 con 0x7ff5680cd510 2024-09-10T10:00:49.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff54c003530 con 0x7ff5680cd510 2024-09-10T10:00:49.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3492251478 0 0) 0x7ff54c003530 con 0x7ff5680cd510 2024-09-10T10:00:49.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 >> v2:172.21.15.145:3300/0 conn(0x7ff5680cfe30 msgr2=0x7ff568071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:49.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 --2- 172.21.15.110:0/2426971422 >> v2:172.21.15.145:3300/0 conn(0x7ff5680cfe30 0x7ff568071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 >> v1:172.21.15.145:6789/0 conn(0x7ff56806e7a0 legacy=0x7ff56815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56806d890 con 0x7ff5680cd510 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7ff554001680 con 0x7ff5680cd510 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 >> v1:172.21.15.110:6789/0 conn(0x7ff5680cd510 legacy=0x7ff56806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c003a10 0x7ff54c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff54c004810 0x7ff54c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54c005220 0x7ff54c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff56815d9a0 con 0x7ff54c004810 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c00c170 con 0x7ff54c003a10 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff54c00c370 con 0x7ff54c005220 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567e00640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff54c004810 0x7ff54c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff56cc00640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c003a10 0x7ff54c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff56d600640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54c005220 0x7ff54c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567e00640 1 -- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54c005220 msgr2=0x7ff54c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:49.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567e00640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54c005220 0x7ff54c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567e00640 1 -- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c003a10 msgr2=0x7ff54c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:49.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567e00640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c003a10 0x7ff54c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff567e00640 1 -- 172.21.15.110:0/2426971422 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c003530 con 0x7ff54c004810 2024-09-10T10:00:49.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff56d600640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54c005220 0x7ff54c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:49.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.141+0000 7ff56d600640 1 -- 172.21.15.110:0/2426971422 reap_dead start 2024-09-10T10:00:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff567e00640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff54c004810 0x7ff54c004c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c160 tx=0x7ff55401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794f59deabf231a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554007c70 con 0x7ff54c004810 2024-09-10T10:00:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5540a9070 con 0x7ff54c004810 2024-09-10T10:00:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff567400640 1 -- 172.21.15.110:0/2426971422 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554005dc0 con 0x7ff54c004810 2024-09-10T10:00:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- 172.21.15.110:0/2426971422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff54c004810 msgr2=0x7ff54c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff54c004810 0x7ff54c004c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c160 tx=0x7ff55401cba0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- 172.21.15.110:0/2426971422 shutdown_connections 2024-09-10T10:00:49.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 --2- 172.21.15.110:0/2426971422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff54c004810 0x7ff54c004c90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- 172.21.15.110:0/2426971422 >> 172.21.15.110:0/2426971422 conn(0x7ff56806bc50 msgr2=0x7ff56806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:49.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- 172.21.15.110:0/2426971422 shutdown_connections 2024-09-10T10:00:49.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- 172.21.15.110:0/2426971422 wait complete. 2024-09-10T10:00:49.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 Processor -- start 2024-09-10T10:00:49.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- start start 2024-09-10T10:00:49.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c004810 0x7ff568149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff568154bc0 0x7ff568155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5680c9040 0x7ff5680c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff56815cfb0 con 0x7ff568154bc0 2024-09-10T10:00:49.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff56815ce30 con 0x7ff54c004810 2024-09-10T10:00:49.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff56f600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff56815d130 con 0x7ff5680c9040 2024-09-10T10:00:49.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.145+0000 7ff567e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff568154bc0 0x7ff568155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff56d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5680c9040 0x7ff5680c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff56cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c004810 0x7ff568149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff567e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff568154bc0 0x7ff568155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38444/0 (socket says 172.21.15.110:38444) 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff567e00640 1 -- 172.21.15.110:0/1503181534 learned_addr learned my addr 172.21.15.110:0/1503181534 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff56d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5680c9040 0x7ff5680c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:48530/0 (socket says 172.21.15.110:48530) 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff567e00640 1 -- 172.21.15.110:0/1503181534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5680c9040 msgr2=0x7ff5680c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff567e00640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5680c9040 0x7ff5680c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff567e00640 1 -- 172.21.15.110:0/1503181534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c004810 msgr2=0x7ff568149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff567e00640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c004810 0x7ff568149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff567e00640 1 -- 172.21.15.110:0/1503181534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5680cb230 con 0x7ff568154bc0 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff56cc00640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c004810 0x7ff568149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff567e00640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff568154bc0 0x7ff568155040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c160 tx=0x7ff54c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:49.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff564c00640 1 -- 172.21.15.110:0/1503181534 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540bc070 con 0x7ff568154bc0 2024-09-10T10:00:49.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff564c00640 1 -- 172.21.15.110:0/1503181534 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff554000ee0 con 0x7ff568154bc0 2024-09-10T10:00:49.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff564c00640 1 -- 172.21.15.110:0/1503181534 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554007140 con 0x7ff568154bc0 2024-09-10T10:00:49.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5680c9d70 con 0x7ff568154bc0 2024-09-10T10:00:49.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5680cb7b0 con 0x7ff568154bc0 2024-09-10T10:00:49.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56815d4c0 con 0x7ff568154bc0 2024-09-10T10:00:49.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff564c00640 1 -- 172.21.15.110:0/1503181534 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7ff5540072e0 con 0x7ff568154bc0 2024-09-10T10:00:49.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff564c00640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff548067b30 0x7ff548069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.149+0000 7ff564c00640 1 -- 172.21.15.110:0/1503181534 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7ff55411f6e0 con 0x7ff568154bc0 2024-09-10T10:00:49.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.153+0000 7ff56cc00640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff548067b30 0x7ff548069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.153+0000 7ff56cc00640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff548067b30 0x7ff548069ff0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff558014400 tx=0x7ff558009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:49.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.157+0000 7ff564c00640 1 -- 172.21.15.110:0/1503181534 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff5540168b0 con 0x7ff568154bc0 2024-09-10T10:00:49.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff568073130 con 0x7ff568154bc0 2024-09-10T10:00:49.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff564c00640 1 -- 172.21.15.110:0/1503181534 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7ff5540eff70 con 0x7ff568154bc0 2024-09-10T10:00:49.348 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:49.348 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:49.348 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff548067b30 msgr2=0x7ff548069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff548067b30 0x7ff548069ff0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff558014400 tx=0x7ff558009190 comp rx=0 tx=0).stop 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff568154bc0 msgr2=0x7ff568155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff568154bc0 0x7ff568155040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c160 tx=0x7ff54c005840 comp rx=0 tx=0).stop 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 shutdown_connections 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff548067b30 0x7ff548069ff0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5680c9040 0x7ff5680c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff568154bc0 0x7ff568155040 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 --2- 172.21.15.110:0/1503181534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff54c004810 0x7ff568149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.345+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 >> 172.21.15.110:0/1503181534 conn(0x7ff56806bc50 msgr2=0x7ff56815f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.349+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 shutdown_connections 2024-09-10T10:00:49.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:49.349+0000 7ff56f600640 1 -- 172.21.15.110:0/1503181534 wait complete. 2024-09-10T10:00:49.368 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:49.526 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":61.990712422000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.064696611101571,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:49.527 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.06469661110157 }, "state": "open", "uptime": 61.990712422 } ] 2024-09-10T10:00:49.527 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:00:54.528 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:00:54.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1077a00640 1 Processor -- start 2024-09-10T10:00:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1077a00640 1 -- start start 2024-09-10T10:00:54.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1077a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 0x7f107006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1077a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1070072270 0x7f107006bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1077a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f107006c190 0x7f107006fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1077a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f107006c600 con 0x7f1070072270 2024-09-10T10:00:54.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1077a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f107015c950 con 0x7f107006c190 2024-09-10T10:00:54.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1077a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f107006ea70 con 0x7f107015c570 2024-09-10T10:00:54.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f106fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1070072270 0x7f107006bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f106fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1070072270 0x7f107006bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38446/0 (socket says 172.21.15.110:38446) 2024-09-10T10:00:54.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f106fe00640 1 -- 172.21.15.110:0/4109648106 learned_addr learned my addr 172.21.15.110:0/4109648106 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:54.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1075000640 1 --2- 172.21.15.110:0/4109648106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 0x7f107006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f1075a00640 1 --2- 172.21.15.110:0/4109648106 >> v2:172.21.15.145:3300/0 conn(0x7f107006c190 0x7f107006fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f106fe00640 1 -- 172.21.15.110:0/4109648106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 msgr2=0x7f107006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:54.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f106fe00640 1 --2- 172.21.15.110:0/4109648106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 0x7f107006e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f106fe00640 1 -- 172.21.15.110:0/4109648106 >> v2:172.21.15.145:3300/0 conn(0x7f107006c190 msgr2=0x7f107006fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:54.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f106fe00640 1 --2- 172.21.15.110:0/4109648106 >> v2:172.21.15.145:3300/0 conn(0x7f107006c190 0x7f107006fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.673+0000 7f106fe00640 1 -- 172.21.15.110:0/4109648106 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107006ed50 con 0x7f1070072270 2024-09-10T10:00:54.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f106fe00640 1 --2- 172.21.15.110:0/4109648106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f107006bc50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f10700b0850 tx=0x7f105c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306a68347215b1a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:54.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f106f400640 1 -- 172.21.15.110:0/4109648106 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0b0070 con 0x7f1070072270 2024-09-10T10:00:54.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f106f400640 1 -- 172.21.15.110:0/4109648106 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105c0ab600 con 0x7f1070072270 2024-09-10T10:00:54.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f106f400640 1 -- 172.21.15.110:0/4109648106 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0ab9d0 con 0x7f1070072270 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 -- 172.21.15.110:0/4109648106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 msgr2=0x7f107006bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 --2- 172.21.15.110:0/4109648106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f107006bc50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f10700b0850 tx=0x7f105c01c490 comp rx=0 tx=0).stop 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 -- 172.21.15.110:0/4109648106 shutdown_connections 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 --2- 172.21.15.110:0/4109648106 >> v2:172.21.15.145:3300/0 conn(0x7f107006c190 0x7f107006fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 --2- 172.21.15.110:0/4109648106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f107006bc50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 --2- 172.21.15.110:0/4109648106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 0x7f107006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 -- 172.21.15.110:0/4109648106 >> 172.21.15.110:0/4109648106 conn(0x7f10700d1530 msgr2=0x7f10700d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 -- 172.21.15.110:0/4109648106 shutdown_connections 2024-09-10T10:00:54.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.677+0000 7f1077a00640 1 -- 172.21.15.110:0/4109648106 wait complete. 2024-09-10T10:00:54.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 Processor -- start 2024-09-10T10:00:54.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 -- start start 2024-09-10T10:00:54.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107006c190 0x7f107014d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f10701595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 0x7f1070159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1070070860 con 0x7f1070072270 2024-09-10T10:00:54.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f107006e780 con 0x7f107006c190 2024-09-10T10:00:54.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f10700709e0 con 0x7f107015c570 2024-09-10T10:00:54.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f10701595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f10701595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38462/0 (socket says 172.21.15.110:38462) 2024-09-10T10:00:54.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 -- 172.21.15.110:0/4194252064 learned_addr learned my addr 172.21.15.110:0/4194252064 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:00:54.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1075a00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 0x7f1070159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1075000640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107006c190 0x7f107014d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 -- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 msgr2=0x7f1070159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 0x7f1070159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 -- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107006c190 msgr2=0x7f107014d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107006c190 0x7f107014d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 -- 172.21.15.110:0/4194252064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1070072f30 con 0x7f1070072270 2024-09-10T10:00:54.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1075000640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107006c190 0x7f107014d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:54.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106fe00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f10701595e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f105c01cd30 tx=0x7f105c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:54.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106cc00640 1 -- 172.21.15.110:0/4194252064 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0b9070 con 0x7f1070072270 2024-09-10T10:00:54.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106cc00640 1 -- 172.21.15.110:0/4194252064 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105c0b4420 con 0x7f1070072270 2024-09-10T10:00:54.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f106cc00640 1 -- 172.21.15.110:0/4194252064 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0b0040 con 0x7f1070072270 2024-09-10T10:00:54.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107015d520 con 0x7f1070072270 2024-09-10T10:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1070157fa0 con 0x7f1070072270 2024-09-10T10:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.681+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10700697d0 con 0x7f1070072270 2024-09-10T10:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.685+0000 7f106cc00640 1 -- 172.21.15.110:0/4194252064 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f105c0bc070 con 0x7f1070072270 2024-09-10T10:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.685+0000 7f106cc00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1050067b00 0x7f1050069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.685+0000 7f106cc00640 1 -- 172.21.15.110:0/4194252064 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f105c11f3c0 con 0x7f1070072270 2024-09-10T10:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.685+0000 7f1075000640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1050067b00 0x7f1050069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.685+0000 7f1075000640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1050067b00 0x7f1050069fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1060007a70 tx=0x7f1060008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:54.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.689+0000 7f106cc00640 1 -- 172.21.15.110:0/4194252064 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f105c016110 con 0x7f1070072270 2024-09-10T10:00:54.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.881+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1070158170 con 0x7f1070072270 2024-09-10T10:00:54.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f106cc00640 1 -- 172.21.15.110:0/4194252064 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f105c0ef4c0 con 0x7f1070072270 2024-09-10T10:00:54.886 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:00:54.886 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:00:54.886 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:00:54.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1050067b00 msgr2=0x7f1050069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1050067b00 0x7f1050069fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1060007a70 tx=0x7f1060008040 comp rx=0 tx=0).stop 2024-09-10T10:00:54.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 msgr2=0x7f10701595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f10701595e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f105c01cd30 tx=0x7f105c019230 comp rx=0 tx=0).stop 2024-09-10T10:00:54.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 shutdown_connections 2024-09-10T10:00:54.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1050067b00 0x7f1050069fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f107015c570 0x7f1070159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1070072270 0x7f10701595e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 --2- 172.21.15.110:0/4194252064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f107006c190 0x7f107014d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 >> 172.21.15.110:0/4194252064 conn(0x7f10700d1530 msgr2=0x7f10700b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:54.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 shutdown_connections 2024-09-10T10:00:54.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:00:54.885+0000 7f1077a00640 1 -- 172.21.15.110:0/4194252064 wait complete. 2024-09-10T10:00:54.907 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:00:55.066 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":67.530632746999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.822224877265835,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:00:55.067 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.822224877265835 }, "state": "open", "uptime": 67.530632747 } ] 2024-09-10T10:00:55.067 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:01:00.068 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:00.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.205+0000 7f46ce200640 1 Processor -- start 2024-09-10T10:01:00.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.205+0000 7f46ce200640 1 -- start start 2024-09-10T10:01:00.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46ce200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f46c80cfe30 0x7f46c8071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46ce200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80cd510 0x7f46c80cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46ce200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c806d2d0 con 0x7f46c806e090 2024-09-10T10:01:00.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46ce200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f46c80cdeb0 con 0x7f46c80cfe30 2024-09-10T10:01:00.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46ce200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f46c815d9a0 con 0x7f46c80cd510 2024-09-10T10:01:00.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f46c806e090 0x7f46c806e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51362/0 (socket says 172.21.15.110:51362) 2024-09-10T10:01:00.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7e00640 1 -- 172.21.15.110:0/1696619562 learned_addr learned my addr 172.21.15.110:0/1696619562 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7400640 1 --2- 172.21.15.110:0/1696619562 >> v2:172.21.15.145:3300/0 conn(0x7f46c80cfe30 0x7f46c8071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46bfe00640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80cd510 0x7f46c80cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3987017200 0 0) 0x7f46c806d2d0 con 0x7f46c806e090 2024-09-10T10:01:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46ac003530 con 0x7f46c806e090 2024-09-10T10:01:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 280951658 0 0) 0x7f46ac003530 con 0x7f46c806e090 2024-09-10T10:01:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80cd510 msgr2=0x7f46c80cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80cd510 0x7f46c80cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 >> v2:172.21.15.145:3300/0 conn(0x7f46c80cfe30 msgr2=0x7f46c8071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 --2- 172.21.15.110:0/1696619562 >> v2:172.21.15.145:3300/0 conn(0x7f46c80cfe30 0x7f46c8071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c806d5b0 con 0x7f46c806e090 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f46c0001a00 con 0x7f46c806e090 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7400640 1 --2- 172.21.15.110:0/1696619562 >> v2:172.21.15.145:3300/0 conn(0x7f46c80cfe30 0x7f46c8071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 >> v1:172.21.15.110:6789/0 conn(0x7f46c806e090 legacy=0x7f46c806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac004810 0x7f46ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80cd510 0x7f46ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f46c815d9a0 con 0x7f46ac0039c0 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac00b9d0 con 0x7f46ac004810 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f46ac00bba0 con 0x7f46c80cd510 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7400640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7e00640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80cd510 0x7f46ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46bfe00640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac004810 0x7f46ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7400640 1 -- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80cd510 msgr2=0x7f46ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7400640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80cd510 0x7f46ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7400640 1 -- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac004810 msgr2=0x7f46ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7400640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46ac004810 0x7f46ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.209+0000 7f46c7400640 1 -- 172.21.15.110:0/1696619562 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac003530 con 0x7f46ac0039c0 2024-09-10T10:01:00.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46c7400640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46ac003de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009ce0 tx=0x7f46b801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198684b036c9085b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:00.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b80b2070 con 0x7f46ac0039c0 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b80a5740 con 0x7f46ac0039c0 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46c6a00640 1 -- 172.21.15.110:0/1696619562 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8009880 con 0x7f46ac0039c0 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46ce200640 1 -- 172.21.15.110:0/1696619562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 msgr2=0x7f46ac003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46ce200640 1 --2- 172.21.15.110:0/1696619562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46ac003de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009ce0 tx=0x7f46b801cc70 comp rx=0 tx=0).stop 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46c7e00640 1 -- 172.21.15.110:0/1696619562 reap_dead start 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46ce200640 1 -- 172.21.15.110:0/1696619562 shutdown_connections 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46ce200640 1 -- 172.21.15.110:0/1696619562 >> 172.21.15.110:0/1696619562 conn(0x7f46c806bc50 msgr2=0x7f46c806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46ce200640 1 -- 172.21.15.110:0/1696619562 shutdown_connections 2024-09-10T10:01:00.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.213+0000 7f46ce200640 1 -- 172.21.15.110:0/1696619562 wait complete. 2024-09-10T10:01:00.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 Processor -- start 2024-09-10T10:01:00.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 -- start start 2024-09-10T10:01:00.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46c81f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80b35d0 0x7f46c81510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46c80cbaa0 0x7f46c80cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f46c80cdd00 con 0x7f46ac0039c0 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46c815d640 con 0x7f46c80cbaa0 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f46c8072190 con 0x7f46c80b35d0 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46c81f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46c81f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36850/0 (socket says 172.21.15.110:36850) 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 -- 172.21.15.110:0/496826642 learned_addr learned my addr 172.21.15.110:0/496826642 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46bfe00640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80b35d0 0x7f46c81510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7e00640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46c80cbaa0 0x7f46c80cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 -- 172.21.15.110:0/496826642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80b35d0 msgr2=0x7f46c81510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80b35d0 0x7f46c81510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 -- 172.21.15.110:0/496826642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46c80cbaa0 msgr2=0x7f46c80cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46c80cbaa0 0x7f46c80cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 -- 172.21.15.110:0/496826642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c80cc470 con 0x7f46ac0039c0 2024-09-10T10:01:00.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46c7400640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46c81f7a90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46b801caa0 tx=0x7f46ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:00.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46bf400640 1 -- 172.21.15.110:0/496826642 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b80b2040 con 0x7f46ac0039c0 2024-09-10T10:01:00.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46bf400640 1 -- 172.21.15.110:0/496826642 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b800f000 con 0x7f46ac0039c0 2024-09-10T10:01:00.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46bf400640 1 -- 172.21.15.110:0/496826642 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8009880 con 0x7f46ac0039c0 2024-09-10T10:01:00.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c81f7fd0 con 0x7f46ac0039c0 2024-09-10T10:01:00.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.217+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c80b3de0 con 0x7f46ac0039c0 2024-09-10T10:01:00.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.221+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c80cc790 con 0x7f46ac0039c0 2024-09-10T10:01:00.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.221+0000 7f46bf400640 1 -- 172.21.15.110:0/496826642 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f46b80c0020 con 0x7f46ac0039c0 2024-09-10T10:01:00.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.221+0000 7f46bf400640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46a4067b00 0x7f46a4069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:00.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.221+0000 7f46bf400640 1 -- 172.21.15.110:0/496826642 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f46b811fef0 con 0x7f46ac0039c0 2024-09-10T10:01:00.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.221+0000 7f46bfe00640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46a4067b00 0x7f46a4069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:00.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.221+0000 7f46bfe00640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46a4067b00 0x7f46a4069fc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46c80b3a30 tx=0x7f46b0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:00.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.225+0000 7f46bf400640 1 -- 172.21.15.110:0/496826642 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f46c81f7fd0 con 0x7f46ac0039c0 2024-09-10T10:01:00.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.413+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46c80719c0 con 0x7f46ac0039c0 2024-09-10T10:01:00.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.413+0000 7f46bf400640 1 -- 172.21.15.110:0/496826642 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f46b80169c0 con 0x7f46ac0039c0 2024-09-10T10:01:00.416 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:00.416 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:01:00.416 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.413+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46a4067b00 msgr2=0x7f46a4069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46a4067b00 0x7f46a4069fc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46c80b3a30 tx=0x7f46b0009190 comp rx=0 tx=0).stop 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 msgr2=0x7f46c81f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46c81f7a90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46b801caa0 tx=0x7f46ac005840 comp rx=0 tx=0).stop 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 shutdown_connections 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f46a4067b00 0x7f46a4069fc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46c80cbaa0 0x7f46c80cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46c80b35d0 0x7f46c81510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 --2- 172.21.15.110:0/496826642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46ac0039c0 0x7f46c81f7a90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:00.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 >> 172.21.15.110:0/496826642 conn(0x7f46c806bc50 msgr2=0x7f46c806e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:00.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 shutdown_connections 2024-09-10T10:01:00.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:00.417+0000 7f46ce200640 1 -- 172.21.15.110:0/496826642 wait complete. 2024-09-10T10:01:00.437 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:01:00.594 INFO:teuthology.orchestra.run.smithi110.stdout:[{"id":5397,"entity":{"name":{"type":"client","num":5397},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":53864042}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":73.058558302999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.583351995165323,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5397 172.21.15.110:0/53864042","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:01:00.595 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 53864042, "type": "any" }, "name": { "num": 5397, "type": "client" } }, "id": 5397, "inst": "client.5397 172.21.15.110:0/53864042", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.583351995165323 }, "state": "open", "uptime": 73.058558303 } ] 2024-09-10T10:01:00.595 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T10:01:05.596 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:05.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.741+0000 7ff42a200640 1 Processor -- start 2024-09-10T10:01:05.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.741+0000 7ff42a200640 1 -- start start 2024-09-10T10:01:05.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.741+0000 7ff42a200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff42415e620 0x7ff4240cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.741+0000 7ff42a200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42406d060 con 0x7ff4240b54f0 2024-09-10T10:01:05.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.741+0000 7ff42a200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42406d340 con 0x7ff4240b6710 2024-09-10T10:01:05.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.741+0000 7ff42a200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff4240d1e90 con 0x7ff42415e620 2024-09-10T10:01:05.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff4240b54f0 0x7ff4240b6000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51366/0 (socket says 172.21.15.110:51366) 2024-09-10T10:01:05.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423e00640 1 -- 172.21.15.110:0/2573274076 learned_addr learned my addr 172.21.15.110:0/2573274076 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:05.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff428e00640 1 --2- 172.21.15.110:0/2573274076 >> v2:172.21.15.145:3300/0 conn(0x7ff42415e620 0x7ff4240cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1121187777 0 0) 0x7ff42406d060 con 0x7ff4240b54f0 2024-09-10T10:01:05.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff408003530 con 0x7ff4240b54f0 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776470496 0 0) 0x7ff42406d340 con 0x7ff4240b6710 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff42406d060 con 0x7ff4240b6710 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1306367336 0 0) 0x7ff408003530 con 0x7ff4240b54f0 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 >> v2:172.21.15.145:3300/0 conn(0x7ff42415e620 msgr2=0x7ff4240cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 --2- 172.21.15.110:0/2573274076 >> v2:172.21.15.145:3300/0 conn(0x7ff42415e620 0x7ff4240cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 >> v1:172.21.15.145:6789/0 conn(0x7ff4240b6710 legacy=0x7ff42406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42406d620 con 0x7ff4240b54f0 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7ff410001680 con 0x7ff4240b54f0 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 >> v1:172.21.15.110:6789/0 conn(0x7ff4240b54f0 legacy=0x7ff4240b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff408003b50 0x7ff408003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408004810 0x7ff408004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4080051d0 0x7ff40800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff40800c0d0 con 0x7ff408003b50 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff40800c280 con 0x7ff4080051d0 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff40800c480 con 0x7ff408004810 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff428e00640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff408003b50 0x7ff408003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff423e00640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408004810 0x7ff408004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff429800640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4080051d0 0x7ff40800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff428e00640 1 -- 172.21.15.110:0/2573274076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408004810 msgr2=0x7ff408004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff428e00640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408004810 0x7ff408004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff428e00640 1 -- 172.21.15.110:0/2573274076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4080051d0 msgr2=0x7ff40800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff428e00640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4080051d0 0x7ff40800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff428e00640 1 -- 172.21.15.110:0/2573274076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff408003530 con 0x7ff408003b50 2024-09-10T10:01:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.745+0000 7ff429800640 1 -- 172.21.15.110:0/2573274076 reap_dead start 2024-09-10T10:01:05.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff428e00640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff408003b50 0x7ff408003f50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ae0 tx=0x7ff41401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed821eaa3d51a0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4140096b0 con 0x7ff408003b50 2024-09-10T10:01:05.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4140a6450 con 0x7ff408003b50 2024-09-10T10:01:05.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff423400640 1 -- 172.21.15.110:0/2573274076 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4140a6840 con 0x7ff408003b50 2024-09-10T10:01:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 -- 172.21.15.110:0/2573274076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff408003b50 msgr2=0x7ff408003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff408003b50 0x7ff408003f50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ae0 tx=0x7ff41401ca70 comp rx=0 tx=0).stop 2024-09-10T10:01:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 -- 172.21.15.110:0/2573274076 shutdown_connections 2024-09-10T10:01:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 --2- 172.21.15.110:0/2573274076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff408003b50 0x7ff408003f50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 -- 172.21.15.110:0/2573274076 >> 172.21.15.110:0/2573274076 conn(0x7ff42406f440 msgr2=0x7ff42406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 -- 172.21.15.110:0/2573274076 shutdown_connections 2024-09-10T10:01:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 -- 172.21.15.110:0/2573274076 wait complete. 2024-09-10T10:01:05.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 Processor -- start 2024-09-10T10:01:05.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 -- start start 2024-09-10T10:01:05.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408003b50 0x7ff424150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff42415c1f0 0x7ff424147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.749+0000 7ff42a200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424160340 0x7ff4241607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff42a200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240b6210 con 0x7ff424160340 2024-09-10T10:01:05.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff42a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240b5ce0 con 0x7ff42415c1f0 2024-09-10T10:01:05.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff42a200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff4240b6390 con 0x7ff408003b50 2024-09-10T10:01:05.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424160340 0x7ff4241607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424160340 0x7ff4241607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36866/0 (socket says 172.21.15.110:36866) 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 -- 172.21.15.110:0/1628811559 learned_addr learned my addr 172.21.15.110:0/1628811559 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff428e00640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408003b50 0x7ff424150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff423e00640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff42415c1f0 0x7ff424147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 -- 172.21.15.110:0/1628811559 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408003b50 msgr2=0x7ff424150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408003b50 0x7ff424150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 -- 172.21.15.110:0/1628811559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff42415c1f0 msgr2=0x7ff424147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff42415c1f0 0x7ff424147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 -- 172.21.15.110:0/1628811559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42415f5c0 con 0x7ff424160340 2024-09-10T10:01:05.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff429800640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424160340 0x7ff4241607c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff4240b6b80 tx=0x7ff41c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff420c00640 1 -- 172.21.15.110:0/1628811559 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c019070 con 0x7ff424160340 2024-09-10T10:01:05.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff420c00640 1 -- 172.21.15.110:0/1628811559 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c006b60 con 0x7ff424160340 2024-09-10T10:01:05.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff420c00640 1 -- 172.21.15.110:0/1628811559 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c003980 con 0x7ff424160340 2024-09-10T10:01:05.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42415f860 con 0x7ff424160340 2024-09-10T10:01:05.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42415c650 con 0x7ff424160340 2024-09-10T10:01:05.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42406e910 con 0x7ff424160340 2024-09-10T10:01:05.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.753+0000 7ff420c00640 1 -- 172.21.15.110:0/1628811559 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7ff41c01c070 con 0x7ff424160340 2024-09-10T10:01:05.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.757+0000 7ff420c00640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff404067b00 0x7ff404069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:05.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.757+0000 7ff420c00640 1 -- 172.21.15.110:0/1628811559 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7ff41c085780 con 0x7ff424160340 2024-09-10T10:01:05.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.757+0000 7ff428e00640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff404067b00 0x7ff404069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:05.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.757+0000 7ff428e00640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff404067b00 0x7ff404069fc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4140ad000 tx=0x7ff4140b2040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:05.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.761+0000 7ff420c00640 1 -- 172.21.15.110:0/1628811559 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff41c0558d0 con 0x7ff424160340 2024-09-10T10:01:05.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.949+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4240d0460 con 0x7ff424160340 2024-09-10T10:01:05.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff420c00640 1 -- 172.21.15.110:0/1628811559 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7ff41c059bd0 con 0x7ff424160340 2024-09-10T10:01:05.954 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:05.954 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:01:05.954 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:01:05.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff404067b00 msgr2=0x7ff404069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff404067b00 0x7ff404069fc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4140ad000 tx=0x7ff4140b2040 comp rx=0 tx=0).stop 2024-09-10T10:01:05.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424160340 msgr2=0x7ff4241607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:05.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424160340 0x7ff4241607c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff4240b6b80 tx=0x7ff41c00eb90 comp rx=0 tx=0).stop 2024-09-10T10:01:05.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 shutdown_connections 2024-09-10T10:01:05.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff404067b00 0x7ff404069fc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424160340 0x7ff4241607c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff42415c1f0 0x7ff424147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 --2- 172.21.15.110:0/1628811559 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff408003b50 0x7ff424150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:05.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 >> 172.21.15.110:0/1628811559 conn(0x7ff42406f440 msgr2=0x7ff4240cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:05.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 shutdown_connections 2024-09-10T10:01:05.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:05.953+0000 7ff42a200640 1 -- 172.21.15.110:0/1628811559 wait complete. 2024-09-10T10:01:05.975 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-10T10:01:06.143 INFO:teuthology.orchestra.run.smithi110.stdout:[] 2024-09-10T10:01:06.143 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-10T10:01:06.143 DEBUG:tasks.ceph_test_case:wait_until_true: success in 70s and 0 retries 2024-09-10T10:01:06.143 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:01:06.144 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:01:06.252 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:01:06.349 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:01:06.349 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:01:06.349 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:06.349 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:06.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:01:06.358 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:01:06.358 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:06.358 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:06.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:01:06.367 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:01:06.367 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:06.367 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:06.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:01:06.409 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:01:06.409 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:06.409 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:06.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:01:06.422 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:06.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.589+0000 7fd924000640 1 Processor -- start 2024-09-10T10:01:06.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.589+0000 7fd924000640 1 -- start start 2024-09-10T10:01:06.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd924000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd91c0ccb60 0x7fd91c072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd924000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd91c140c30 0x7fd91c15b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd924000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd91c0ccf40 con 0x7fd91c072670 2024-09-10T10:01:06.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd924000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd91c141080 con 0x7fd91c140c30 2024-09-10T10:01:06.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd924000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd91c072ad0 con 0x7fd91c0ccb60 2024-09-10T10:01:06.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd922000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd91c140c30 0x7fd91c15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd922000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd91c140c30 0x7fd91c15b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36880/0 (socket says 172.21.15.110:36880) 2024-09-10T10:01:06.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd922000640 1 -- 172.21.15.110:0/1993429675 learned_addr learned my addr 172.21.15.110:0/1993429675 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:06.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd921600640 1 --2- 172.21.15.110:0/1993429675 >> v2:172.21.15.145:3300/0 conn(0x7fd91c0ccb60 0x7fd91c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd922000640 1 -- 172.21.15.110:0/1993429675 >> v2:172.21.15.145:3300/0 conn(0x7fd91c0ccb60 msgr2=0x7fd91c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:06.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd922000640 1 --2- 172.21.15.110:0/1993429675 >> v2:172.21.15.145:3300/0 conn(0x7fd91c0ccb60 0x7fd91c072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd922000640 1 -- 172.21.15.110:0/1993429675 >> v1:172.21.15.145:6789/0 conn(0x7fd91c072670 legacy=0x7fd91c140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.593+0000 7fd922000640 1 -- 172.21.15.110:0/1993429675 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c1412a0 con 0x7fd91c140c30 2024-09-10T10:01:06.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd922000640 1 --2- 172.21.15.110:0/1993429675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c140c30 0x7fd91c15b120 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0704e0 tx=0x7fd91401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfa77ecebc64711 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:06.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd91be00640 1 -- 172.21.15.110:0/1993429675 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140a5880 con 0x7fd91c140c30 2024-09-10T10:01:06.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd91be00640 1 -- 172.21.15.110:0/1993429675 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9140a5a20 con 0x7fd91c140c30 2024-09-10T10:01:06.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd91be00640 1 -- 172.21.15.110:0/1993429675 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140a5df0 con 0x7fd91c140c30 2024-09-10T10:01:06.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 -- 172.21.15.110:0/1993429675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c140c30 msgr2=0x7fd91c15b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 --2- 172.21.15.110:0/1993429675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c140c30 0x7fd91c15b120 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0704e0 tx=0x7fd91401c9c0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 -- 172.21.15.110:0/1993429675 shutdown_connections 2024-09-10T10:01:06.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 --2- 172.21.15.110:0/1993429675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c140c30 0x7fd91c15b120 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 --2- 172.21.15.110:0/1993429675 >> v2:172.21.15.145:3300/0 conn(0x7fd91c0ccb60 0x7fd91c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 -- 172.21.15.110:0/1993429675 >> 172.21.15.110:0/1993429675 conn(0x7fd91c06bc50 msgr2=0x7fd91c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:06.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 -- 172.21.15.110:0/1993429675 shutdown_connections 2024-09-10T10:01:06.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 -- 172.21.15.110:0/1993429675 wait complete. 2024-09-10T10:01:06.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 Processor -- start 2024-09-10T10:01:06.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.597+0000 7fd924000640 1 -- start start 2024-09-10T10:01:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd91c072670 0x7fd91c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c0ccb60 0x7fd91c159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd91c140c30 0x7fd91c157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c15f520 con 0x7fd91c0ccb60 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c15f000 con 0x7fd91c072670 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd91c15a530 con 0x7fd91c140c30 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c0ccb60 0x7fd91c159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c0ccb60 0x7fd91c159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36896/0 (socket says 172.21.15.110:36896) 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 -- 172.21.15.110:0/139046824 learned_addr learned my addr 172.21.15.110:0/139046824 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd921600640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd91c072670 0x7fd91c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd922000640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd91c140c30 0x7fd91c157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 -- 172.21.15.110:0/139046824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd91c140c30 msgr2=0x7fd91c157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd91c140c30 0x7fd91c157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 -- 172.21.15.110:0/139046824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd91c072670 msgr2=0x7fd91c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd91c072670 0x7fd91c1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 -- 172.21.15.110:0/139046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c15a810 con 0x7fd91c0ccb60 2024-09-10T10:01:06.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd920c00640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c0ccb60 0x7fd91c159a40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c420 tx=0x7fd90800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:06.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd919600640 1 -- 172.21.15.110:0/139046824 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9080052d0 con 0x7fd91c0ccb60 2024-09-10T10:01:06.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd919600640 1 -- 172.21.15.110:0/139046824 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd908005470 con 0x7fd91c0ccb60 2024-09-10T10:01:06.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c1552d0 con 0x7fd91c0ccb60 2024-09-10T10:01:06.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd919600640 1 -- 172.21.15.110:0/139046824 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908007260 con 0x7fd91c0ccb60 2024-09-10T10:01:06.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c153d00 con 0x7fd91c0ccb60 2024-09-10T10:01:06.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91c153ff0 con 0x7fd91c0ccb60 2024-09-10T10:01:06.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.601+0000 7fd919600640 1 -- 172.21.15.110:0/139046824 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fd908007400 con 0x7fd91c0ccb60 2024-09-10T10:01:06.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.605+0000 7fd919600640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd8fc067a70 0x7fd8fc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.605+0000 7fd919600640 1 -- 172.21.15.110:0/139046824 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd9080854f0 con 0x7fd91c0ccb60 2024-09-10T10:01:06.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.605+0000 7fd921600640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd8fc067a70 0x7fd8fc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.605+0000 7fd921600640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd8fc067a70 0x7fd8fc069f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd910009b60 tx=0x7fd910009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:06.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.609+0000 7fd919600640 1 -- 172.21.15.110:0/139046824 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd9080556a0 con 0x7fd91c0ccb60 2024-09-10T10:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd91c1550c0 con 0x7fd91c0ccb60 2024-09-10T10:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd919600640 1 -- 172.21.15.110:0/139046824 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fd9080599a0 con 0x7fd91c0ccb60 2024-09-10T10:01:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:06.803 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:01:06.803 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:01:06.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd8fc067a70 msgr2=0x7fd8fc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd8fc067a70 0x7fd8fc069f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd910009b60 tx=0x7fd910009190 comp rx=0 tx=0).stop 2024-09-10T10:01:06.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c0ccb60 msgr2=0x7fd91c159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c0ccb60 0x7fd91c159a40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c420 tx=0x7fd90800c8e0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 shutdown_connections 2024-09-10T10:01:06.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd8fc067a70 0x7fd8fc069f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd91c140c30 0x7fd91c157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c0ccb60 0x7fd91c159a40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 --2- 172.21.15.110:0/139046824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd91c072670 0x7fd91c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 >> 172.21.15.110:0/139046824 conn(0x7fd91c06bc50 msgr2=0x7fd91c074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:06.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 shutdown_connections 2024-09-10T10:01:06.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.801+0000 7fd924000640 1 -- 172.21.15.110:0/139046824 wait complete. 2024-09-10T10:01:06.825 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:06.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb463800640 1 Processor -- start 2024-09-10T10:01:06.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb463800640 1 -- start start 2024-09-10T10:01:06.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb463800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c0d0cb0 0x7fb45c163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb463800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45c0706a0 con 0x7fb45c073290 2024-09-10T10:01:06.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb463800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45c0d10f0 con 0x7fb45c06eb30 2024-09-10T10:01:06.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb463800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c15ee90 con 0x7fb45c0d0cb0 2024-09-10T10:01:06.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb461800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb45c073290 0x7fb45c06d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51382/0 (socket says 172.21.15.110:51382) 2024-09-10T10:01:06.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb461800640 1 -- 172.21.15.110:0/3514401045 learned_addr learned my addr 172.21.15.110:0/3514401045 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:06.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb45be00640 1 --2- 172.21.15.110:0/3514401045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c0d0cb0 0x7fb45c163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb45b400640 1 -- 172.21.15.110:0/3514401045 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678798808 0 0) 0x7fb45c0706a0 con 0x7fb45c073290 2024-09-10T10:01:06.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb45b400640 1 -- 172.21.15.110:0/3514401045 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb440003530 con 0x7fb45c073290 2024-09-10T10:01:06.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb45b400640 1 -- 172.21.15.110:0/3514401045 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251990860 0 0) 0x7fb44c0014a0 con 0x7fb45c06eb30 2024-09-10T10:01:06.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb45b400640 1 -- 172.21.15.110:0/3514401045 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb45c0706a0 con 0x7fb45c06eb30 2024-09-10T10:01:06.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb45be00640 1 -- 172.21.15.110:0/3514401045 >> v1:172.21.15.145:6789/0 conn(0x7fb45c06eb30 legacy=0x7fb45c071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb45be00640 1 -- 172.21.15.110:0/3514401045 >> v1:172.21.15.110:6789/0 conn(0x7fb45c073290 legacy=0x7fb45c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.961+0000 7fb45be00640 1 -- 172.21.15.110:0/3514401045 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c0d1370 con 0x7fb45c0d0cb0 2024-09-10T10:01:06.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb45be00640 1 --2- 172.21.15.110:0/3514401045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c0d0cb0 0x7fb45c163000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097f0 tx=0x7fb44801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6286017e8d81d75f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:06.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb45b400640 1 -- 172.21.15.110:0/3514401045 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480b0070 con 0x7fb45c0d0cb0 2024-09-10T10:01:06.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb45b400640 1 -- 172.21.15.110:0/3514401045 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4480ab600 con 0x7fb45c0d0cb0 2024-09-10T10:01:06.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb45b400640 1 -- 172.21.15.110:0/3514401045 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480ab9d0 con 0x7fb45c0d0cb0 2024-09-10T10:01:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb463800640 1 -- 172.21.15.110:0/3514401045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c0d0cb0 msgr2=0x7fb45c163000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb463800640 1 --2- 172.21.15.110:0/3514401045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c0d0cb0 0x7fb45c163000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097f0 tx=0x7fb44801c450 comp rx=0 tx=0).stop 2024-09-10T10:01:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb463800640 1 -- 172.21.15.110:0/3514401045 shutdown_connections 2024-09-10T10:01:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb463800640 1 --2- 172.21.15.110:0/3514401045 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c0d0cb0 0x7fb45c163000 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb463800640 1 -- 172.21.15.110:0/3514401045 >> 172.21.15.110:0/3514401045 conn(0x7fb45c0d25c0 msgr2=0x7fb45c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb463800640 1 -- 172.21.15.110:0/3514401045 shutdown_connections 2024-09-10T10:01:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.965+0000 7fb463800640 1 -- 172.21.15.110:0/3514401045 wait complete. 2024-09-10T10:01:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 Processor -- start 2024-09-10T10:01:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 -- start start 2024-09-10T10:01:06.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c06eb30 0x7fb45c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c073290 0x7fb45c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb45c0d0cb0 0x7fb45c161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c06e070 con 0x7fb45c073290 2024-09-10T10:01:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c071c70 con 0x7fb45c0d0cb0 2024-09-10T10:01:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c073ad0 con 0x7fb45c06eb30 2024-09-10T10:01:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c073290 0x7fb45c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c073290 0x7fb45c159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36912/0 (socket says 172.21.15.110:36912) 2024-09-10T10:01:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 -- 172.21.15.110:0/3242450680 learned_addr learned my addr 172.21.15.110:0/3242450680 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb460e00640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c06eb30 0x7fb45c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb461800640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb45c0d0cb0 0x7fb45c161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 -- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c06eb30 msgr2=0x7fb45c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c06eb30 0x7fb45c1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 -- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb45c0d0cb0 msgr2=0x7fb45c161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb45c0d0cb0 0x7fb45c161830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 -- 172.21.15.110:0/3242450680 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c15af60 con 0x7fb45c073290 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb460e00640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c06eb30 0x7fb45c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb461800640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb45c0d0cb0 0x7fb45c161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:06.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb45be00640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c073290 0x7fb45c159ed0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097f0 tx=0x7fb4480191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:06.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb458c00640 1 -- 172.21.15.110:0/3242450680 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480b9020 con 0x7fb45c073290 2024-09-10T10:01:06.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb458c00640 1 -- 172.21.15.110:0/3242450680 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4480b5480 con 0x7fb45c073290 2024-09-10T10:01:06.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c15a9a0 con 0x7fb45c073290 2024-09-10T10:01:06.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb458c00640 1 -- 172.21.15.110:0/3242450680 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480b0040 con 0x7fb45c073290 2024-09-10T10:01:06.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.969+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c156290 con 0x7fb45c073290 2024-09-10T10:01:06.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.973+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c163000 con 0x7fb45c073290 2024-09-10T10:01:06.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.973+0000 7fb458c00640 1 -- 172.21.15.110:0/3242450680 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fb4480bc020 con 0x7fb45c073290 2024-09-10T10:01:06.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.973+0000 7fb458c00640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb43c067aa0 0x7fb43c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.973+0000 7fb458c00640 1 -- 172.21.15.110:0/3242450680 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fb44811ff60 con 0x7fb45c073290 2024-09-10T10:01:06.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.973+0000 7fb460e00640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb43c067aa0 0x7fb43c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.973+0000 7fb460e00640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb43c067aa0 0x7fb43c069f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb44c001dd0 tx=0x7fb44c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:06.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:06.977+0000 7fb458c00640 1 -- 172.21.15.110:0/3242450680 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb4480160d0 con 0x7fb45c073290 2024-09-10T10:01:07.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.169+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb45c0b1f90 con 0x7fb45c073290 2024-09-10T10:01:07.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.169+0000 7fb458c00640 1 -- 172.21.15.110:0/3242450680 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fb4480f06c0 con 0x7fb45c073290 2024-09-10T10:01:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:07.173 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:01:07.173 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb43c067aa0 msgr2=0x7fb43c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb43c067aa0 0x7fb43c069f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb44c001dd0 tx=0x7fb44c009040 comp rx=0 tx=0).stop 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c073290 msgr2=0x7fb45c159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c073290 0x7fb45c159ed0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097f0 tx=0x7fb4480191f0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 shutdown_connections 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb43c067aa0 0x7fb43c069f60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb45c0d0cb0 0x7fb45c161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb45c073290 0x7fb45c159ed0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 --2- 172.21.15.110:0/3242450680 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb45c06eb30 0x7fb45c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 >> 172.21.15.110:0/3242450680 conn(0x7fb45c0d25c0 msgr2=0x7fb45c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:07.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 shutdown_connections 2024-09-10T10:01:07.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.173+0000 7fb463800640 1 -- 172.21.15.110:0/3242450680 wait complete. 2024-09-10T10:01:07.194 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:07.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.401+0000 7f1001600640 1 Processor -- start 2024-09-10T10:01:07.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.401+0000 7f1001600640 1 -- start start 2024-09-10T10:01:07.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f1001600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f1001600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ffc155d50 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f1001600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ffc0b34d0 con 0x7f0ffc15c560 2024-09-10T10:01:07.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f1001600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc0b4570 con 0x7f0ffc155970 2024-09-10T10:01:07.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ffb400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0ffc06f1e0 0x7f0ffc0ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51396/0 (socket says 172.21.15.110:51396) 2024-09-10T10:01:07.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ffb400640 1 -- 172.21.15.110:0/1741412165 learned_addr learned my addr 172.21.15.110:0/1741412165 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:07.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ffaa00640 1 --2- 172.21.15.110:0/1741412165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ff9600640 1 -- 172.21.15.110:0/1741412165 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2516583751 0 0) 0x7f0ffc155d50 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ff9600640 1 -- 172.21.15.110:0/1741412165 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fe0003530 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ff9600640 1 -- 172.21.15.110:0/1741412165 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61713014 0 0) 0x7f0ffc0b34d0 con 0x7f0ffc15c560 2024-09-10T10:01:07.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ff9600640 1 -- 172.21.15.110:0/1741412165 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ffc155d50 con 0x7f0ffc15c560 2024-09-10T10:01:07.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ff9600640 1 -- 172.21.15.110:0/1741412165 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2931965528 0 0) 0x7f0fe0003530 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ffaa00640 1 -- 172.21.15.110:0/1741412165 >> v1:172.21.15.145:6789/0 conn(0x7f0ffc15c560 legacy=0x7f0ffc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ffaa00640 1 -- 172.21.15.110:0/1741412165 >> v1:172.21.15.110:6789/0 conn(0x7f0ffc06f1e0 legacy=0x7f0ffc0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.405+0000 7f0ffaa00640 1 -- 172.21.15.110:0/1741412165 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc0b37b0 con 0x7f0ffc155970 2024-09-10T10:01:07.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f0ffaa00640 1 --2- 172.21.15.110:0/1741412165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc0cc8c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009dc0 tx=0x7f0fec01c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d12129ba6c1a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:07.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f0ff9600640 1 -- 172.21.15.110:0/1741412165 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0b2070 con 0x7f0ffc155970 2024-09-10T10:01:07.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f0ff9600640 1 -- 172.21.15.110:0/1741412165 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fec0a5630 con 0x7f0ffc155970 2024-09-10T10:01:07.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f0ff9600640 1 -- 172.21.15.110:0/1741412165 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4001680 con 0x7f0ffc155970 2024-09-10T10:01:07.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- 172.21.15.110:0/1741412165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 msgr2=0x7f0ffc0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 --2- 172.21.15.110:0/1741412165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc0cc8c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009dc0 tx=0x7f0fec01c7f0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- 172.21.15.110:0/1741412165 shutdown_connections 2024-09-10T10:01:07.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 --2- 172.21.15.110:0/1741412165 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc0cc8c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- 172.21.15.110:0/1741412165 >> 172.21.15.110:0/1741412165 conn(0x7f0ffc06bc50 msgr2=0x7f0ffc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:07.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- 172.21.15.110:0/1741412165 shutdown_connections 2024-09-10T10:01:07.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- 172.21.15.110:0/1741412165 wait complete. 2024-09-10T10:01:07.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 Processor -- start 2024-09-10T10:01:07.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- start start 2024-09-10T10:01:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ffc06f1e0 0x7f0ffc156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ffc15c560 0x7f0ffc0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc0cf340 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc0cce00 con 0x7f0ffc15c560 2024-09-10T10:01:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.409+0000 7f1001600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc0cf4c0 con 0x7f0ffc155970 2024-09-10T10:01:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ffc06f1e0 0x7f0ffc156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ffc06f1e0 0x7f0ffc156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36920/0 (socket says 172.21.15.110:36920) 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 -- 172.21.15.110:0/3741688326 learned_addr learned my addr 172.21.15.110:0/3741688326 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffa000640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffb400640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ffc15c560 0x7f0ffc0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 -- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 msgr2=0x7f0ffc1512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc1512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 -- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ffc15c560 msgr2=0x7f0ffc0ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ffc15c560 0x7f0ffc0ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 -- 172.21.15.110:0/3741688326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc151820 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffb400640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ffc15c560 0x7f0ffc0ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ffaa00640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ffc06f1e0 0x7f0ffc156ed0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0fec01aea0 tx=0x7f0fec019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:07.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ff2a00640 1 -- 172.21.15.110:0/3741688326 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0bd070 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ff2a00640 1 -- 172.21.15.110:0/3741688326 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fec003ba0 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ff2a00640 1 -- 172.21.15.110:0/3741688326 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0b2040 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc166630 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc166bf0 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc0cc8c0 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.413+0000 7f0ff2a00640 1 -- 172.21.15.110:0/3741688326 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f0fec003fd0 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.417+0000 7f0ff2a00640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0fdc067a70 0x7f0fdc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.417+0000 7f0ff2a00640 1 -- 172.21.15.110:0/3741688326 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f0fec120bd0 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.417+0000 7f0ffa000640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0fdc067a70 0x7f0fdc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.417+0000 7f0ffa000640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0fdc067a70 0x7f0fdc069f30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc1fa810 tx=0x7f0fe8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:07.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.421+0000 7f0ff2a00640 1 -- 172.21.15.110:0/3741688326 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0fec0164a0 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.577+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ffc0ce5a0 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.577+0000 7f0ff2a00640 1 -- 172.21.15.110:0/3741688326 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16805 (secure 0 0 0) 0x7f0fec0f14e0 con 0x7f0ffc06f1e0 2024-09-10T10:01:07.581 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:07.582 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":47,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:00:04.960849+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:32.939641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:34.219832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.12","mappings":[{"from":0,"to":5}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:07.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0fdc067a70 msgr2=0x7f0fdc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0fdc067a70 0x7f0fdc069f30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc1fa810 tx=0x7f0fe8009040 comp rx=0 tx=0).stop 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ffc06f1e0 msgr2=0x7f0ffc156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ffc06f1e0 0x7f0ffc156ed0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0fec01aea0 tx=0x7f0fec019590 comp rx=0 tx=0).stop 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 shutdown_connections 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0fdc067a70 0x7f0fdc069f30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ffc15c560 0x7f0ffc0ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ffc155970 0x7f0ffc1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 --2- 172.21.15.110:0/3741688326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ffc06f1e0 0x7f0ffc156ed0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 >> 172.21.15.110:0/3741688326 conn(0x7f0ffc06bc50 msgr2=0x7f0ffc06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:07.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 shutdown_connections 2024-09-10T10:01:07.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.581+0000 7f1001600640 1 -- 172.21.15.110:0/3741688326 wait complete. 2024-09-10T10:01:07.603 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:01:07.603 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:07.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b8600640 1 Processor -- start 2024-09-10T10:01:07.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b8600640 1 -- start start 2024-09-10T10:01:07.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b8600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f00b006f1e0 0x7f00b0166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b8600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b0155d50 con 0x7f00b0155970 2024-09-10T10:01:07.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b8600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b00b34d0 con 0x7f00b015c560 2024-09-10T10:01:07.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b8600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f00b00b4570 con 0x7f00b006f1e0 2024-09-10T10:01:07.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b5c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f00b0155970 0x7f00b00cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:51408/0 (socket says 172.21.15.110:51408) 2024-09-10T10:01:07.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b5c00640 1 -- 172.21.15.110:0/134016291 learned_addr learned my addr 172.21.15.110:0/134016291 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:07.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b6600640 1 --2- 172.21.15.110:0/134016291 >> v2:172.21.15.145:3300/0 conn(0x7f00b006f1e0 0x7f00b0166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00afe00640 1 -- 172.21.15.110:0/134016291 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090577345 0 0) 0x7f00b0155d50 con 0x7f00b0155970 2024-09-10T10:01:07.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00afe00640 1 -- 172.21.15.110:0/134016291 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0098003530 con 0x7f00b0155970 2024-09-10T10:01:07.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b6600640 1 -- 172.21.15.110:0/134016291 >> v1:172.21.15.145:6789/0 conn(0x7f00b015c560 legacy=0x7f00b006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b6600640 1 -- 172.21.15.110:0/134016291 >> v1:172.21.15.110:6789/0 conn(0x7f00b0155970 legacy=0x7f00b00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.737+0000 7f00b6600640 1 -- 172.21.15.110:0/134016291 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b00b37b0 con 0x7f00b006f1e0 2024-09-10T10:01:07.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00b6600640 1 --2- 172.21.15.110:0/134016291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00b006f1e0 0x7f00b0166020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f00a80097c0 tx=0x7f00a801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77eca9dd455735c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:07.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00afe00640 1 -- 172.21.15.110:0/134016291 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a80a7070 con 0x7f00b006f1e0 2024-09-10T10:01:07.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00afe00640 1 -- 172.21.15.110:0/134016291 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a80b0070 con 0x7f00b006f1e0 2024-09-10T10:01:07.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00afe00640 1 -- 172.21.15.110:0/134016291 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a80ab890 con 0x7f00b006f1e0 2024-09-10T10:01:07.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00b8600640 1 -- 172.21.15.110:0/134016291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00b006f1e0 msgr2=0x7f00b0166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00b8600640 1 --2- 172.21.15.110:0/134016291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00b006f1e0 0x7f00b0166020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f00a80097c0 tx=0x7f00a801c470 comp rx=0 tx=0).stop 2024-09-10T10:01:07.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00b8600640 1 -- 172.21.15.110:0/134016291 shutdown_connections 2024-09-10T10:01:07.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00b8600640 1 --2- 172.21.15.110:0/134016291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00b006f1e0 0x7f00b0166020 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00b8600640 1 -- 172.21.15.110:0/134016291 >> 172.21.15.110:0/134016291 conn(0x7f00b006bc50 msgr2=0x7f00b006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:07.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00b8600640 1 -- 172.21.15.110:0/134016291 shutdown_connections 2024-09-10T10:01:07.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.741+0000 7f00b8600640 1 -- 172.21.15.110:0/134016291 wait complete. 2024-09-10T10:01:07.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 Processor -- start 2024-09-10T10:01:07.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 -- start start 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00b006f1e0 0x7f00b0148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00b0155970 0x7f00b01f9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00b015c560 0x7f00b015f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0168f00 con 0x7f00b006f1e0 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0168d80 con 0x7f00b0155970 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f00b0169080 con 0x7f00b015c560 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00b006f1e0 0x7f00b0148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00b006f1e0 0x7f00b0148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36936/0 (socket says 172.21.15.110:36936) 2024-09-10T10:01:07.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 -- 172.21.15.110:0/347814667 learned_addr learned my addr 172.21.15.110:0/347814667 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 -- 172.21.15.110:0/347814667 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00b015c560 msgr2=0x7f00b015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00b015c560 0x7f00b015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 -- 172.21.15.110:0/347814667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00b0155970 msgr2=0x7f00b01f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00b0155970 0x7f00b01f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 -- 172.21.15.110:0/347814667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b0152720 con 0x7f00b006f1e0 2024-09-10T10:01:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b5c00640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00b006f1e0 0x7f00b0148e80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00a400cbf0 tx=0x7f00a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00ad600640 1 -- 172.21.15.110:0/347814667 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4017070 con 0x7f00b006f1e0 2024-09-10T10:01:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00ad600640 1 -- 172.21.15.110:0/347814667 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a400ad40 con 0x7f00b006f1e0 2024-09-10T10:01:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b0152a00 con 0x7f00b006f1e0 2024-09-10T10:01:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00ad600640 1 -- 172.21.15.110:0/347814667 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4012620 con 0x7f00b006f1e0 2024-09-10T10:01:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b01f9940 con 0x7f00b006f1e0 2024-09-10T10:01:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.745+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b00cc8c0 con 0x7f00b006f1e0 2024-09-10T10:01:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.749+0000 7f00ad600640 1 -- 172.21.15.110:0/347814667 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f00a401a070 con 0x7f00b006f1e0 2024-09-10T10:01:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.749+0000 7f00ad600640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0090067aa0 0x7f0090069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.749+0000 7f00ad600640 1 -- 172.21.15.110:0/347814667 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f00a4085df0 con 0x7f00b006f1e0 2024-09-10T10:01:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.749+0000 7f00b5200640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0090067aa0 0x7f0090069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.749+0000 7f00b5200640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0090067aa0 0x7f0090069f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f009c000f70 tx=0x7f009c0079e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:07.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.753+0000 7f00ad600640 1 -- 172.21.15.110:0/347814667 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f00a4056660 con 0x7f00b006f1e0 2024-09-10T10:01:07.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.941+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00b014f490 con 0x7f00b006f1e0 2024-09-10T10:01:07.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.941+0000 7f00ad600640 1 -- 172.21.15.110:0/347814667 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f00a4055f60 con 0x7f00b006f1e0 2024-09-10T10:01:07.945 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:07.945 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:01:07.945 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:01:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0090067aa0 msgr2=0x7f0090069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0090067aa0 0x7f0090069f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f009c000f70 tx=0x7f009c0079e0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00b006f1e0 msgr2=0x7f00b0148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00b006f1e0 0x7f00b0148e80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00a400cbf0 tx=0x7f00a400a430 comp rx=0 tx=0).stop 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 shutdown_connections 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0090067aa0 0x7f0090069f60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00b015c560 0x7f00b015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00b0155970 0x7f00b01f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 --2- 172.21.15.110:0/347814667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00b006f1e0 0x7f00b0148e80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 >> 172.21.15.110:0/347814667 conn(0x7f00b006bc50 msgr2=0x7f00b006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 shutdown_connections 2024-09-10T10:01:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:07.945+0000 7f00b8600640 1 -- 172.21.15.110:0/347814667 wait complete. 2024-09-10T10:01:07.966 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:08.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.097+0000 7ff3ebe00640 1 Processor -- start 2024-09-10T10:01:08.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.097+0000 7ff3ebe00640 1 -- start start 2024-09-10T10:01:08.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3ebe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3e415c560 0x7ff3e406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3ebe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff3e406f010 0x7ff3e40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3ebe00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e406f430 con 0x7ff3e4155970 2024-09-10T10:01:08.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3ebe00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e40b4570 con 0x7ff3e406f010 2024-09-10T10:01:08.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3ebe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e4155d50 con 0x7ff3e415c560 2024-09-10T10:01:08.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3e9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff3e406f010 0x7ff3e40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3e9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff3e406f010 0x7ff3e40ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36950/0 (socket says 172.21.15.110:36950) 2024-09-10T10:01:08.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3e9e00640 1 -- 172.21.15.110:0/3768044652 learned_addr learned my addr 172.21.15.110:0/3768044652 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:08.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3e8a00640 1 --2- 172.21.15.110:0/3768044652 >> v2:172.21.15.145:3300/0 conn(0x7ff3e415c560 0x7ff3e406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3e9e00640 1 -- 172.21.15.110:0/3768044652 >> v2:172.21.15.145:3300/0 conn(0x7ff3e415c560 msgr2=0x7ff3e406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3e9e00640 1 --2- 172.21.15.110:0/3768044652 >> v2:172.21.15.145:3300/0 conn(0x7ff3e415c560 0x7ff3e406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3e9e00640 1 -- 172.21.15.110:0/3768044652 >> v1:172.21.15.110:6789/0 conn(0x7ff3e4155970 legacy=0x7ff3e40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.101+0000 7ff3e9e00640 1 -- 172.21.15.110:0/3768044652 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e40b34d0 con 0x7ff3e406f010 2024-09-10T10:01:08.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3e9e00640 1 --2- 172.21.15.110:0/3768044652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e40ce420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff3e406d930 tx=0x7ff3dc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a189aedea64b1ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3e3e00640 1 -- 172.21.15.110:0/3768044652 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0a5880 con 0x7ff3e406f010 2024-09-10T10:01:08.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3e3e00640 1 -- 172.21.15.110:0/3768044652 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc0a5a20 con 0x7ff3e406f010 2024-09-10T10:01:08.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3e3e00640 1 -- 172.21.15.110:0/3768044652 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0a5df0 con 0x7ff3e406f010 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- 172.21.15.110:0/3768044652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 msgr2=0x7ff3e40ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/3768044652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e40ce420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff3e406d930 tx=0x7ff3dc01c9c0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- 172.21.15.110:0/3768044652 shutdown_connections 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/3768044652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e40ce420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/3768044652 >> v2:172.21.15.145:3300/0 conn(0x7ff3e415c560 0x7ff3e406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- 172.21.15.110:0/3768044652 >> 172.21.15.110:0/3768044652 conn(0x7ff3e406bc50 msgr2=0x7ff3e406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- 172.21.15.110:0/3768044652 shutdown_connections 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- 172.21.15.110:0/3768044652 wait complete. 2024-09-10T10:01:08.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 Processor -- start 2024-09-10T10:01:08.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- start start 2024-09-10T10:01:08.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e41648b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e4155970 0x7ff3e41514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e415c560 0x7ff3e40b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40cf170 con 0x7ff3e406f010 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40b4050 con 0x7ff3e415c560 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.105+0000 7ff3ebe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e40cf2f0 con 0x7ff3e4155970 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e41648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e41648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36960/0 (socket says 172.21.15.110:36960) 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 -- 172.21.15.110:0/2164820959 learned_addr learned my addr 172.21.15.110:0/2164820959 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e8a00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e4155970 0x7ff3e41514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9e00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e415c560 0x7ff3e40b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 -- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e4155970 msgr2=0x7ff3e41514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e4155970 0x7ff3e41514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 -- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e415c560 msgr2=0x7ff3e40b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e415c560 0x7ff3e40b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 -- 172.21.15.110:0/2164820959 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e4164df0 con 0x7ff3e406f010 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e8a00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e4155970 0x7ff3e41514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9e00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e415c560 0x7ff3e40b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e9400640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e41648b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c420 tx=0x7ff3d800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e1600640 1 -- 172.21.15.110:0/2164820959 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d80052d0 con 0x7ff3e406f010 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e1600640 1 -- 172.21.15.110:0/2164820959 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d8005470 con 0x7ff3e406f010 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e1600640 1 -- 172.21.15.110:0/2164820959 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8007260 con 0x7ff3e406f010 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e40aa930 con 0x7ff3e406f010 2024-09-10T10:01:08.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e4162fa0 con 0x7ff3e406f010 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e40cf650 con 0x7ff3e406f010 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e1600640 1 -- 172.21.15.110:0/2164820959 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7ff3d8007400 con 0x7ff3e406f010 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.109+0000 7ff3e1600640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff3c4067aa0 0x7ff3c4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.113+0000 7ff3e1600640 1 -- 172.21.15.110:0/2164820959 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7ff3d8085550 con 0x7ff3e406f010 2024-09-10T10:01:08.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.113+0000 7ff3e8a00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff3c4067aa0 0x7ff3c4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.113+0000 7ff3e8a00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff3c4067aa0 0x7ff3c4069f60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40ccfa0 tx=0x7ff3d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.117+0000 7ff3e1600640 1 -- 172.21.15.110:0/2164820959 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff3d8055d30 con 0x7ff3e406f010 2024-09-10T10:01:08.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.305+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3e4075560 con 0x7ff3e406f010 2024-09-10T10:01:08.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.305+0000 7ff3e1600640 1 -- 172.21.15.110:0/2164820959 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7ff3d8055630 con 0x7ff3e406f010 2024-09-10T10:01:08.308 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:08.309 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"btime":"2024-09-10T09:59:44:895823+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:59:35.214092+0000","modified":"2024-09-10T09:59:38.244548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5214},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5214":{"gid":5214,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/2457861782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2457861782},{"type":"v1","addr":"172.21.15.110:6837","nonce":2457861782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5214,"qdb_cluster":[5214]},"id":3}]} 2024-09-10T10:01:08.309 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-09-10T10:01:08.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff3c4067aa0 msgr2=0x7ff3c4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff3c4067aa0 0x7ff3c4069f60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40ccfa0 tx=0x7ff3d0008040 comp rx=0 tx=0).stop 2024-09-10T10:01:08.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 msgr2=0x7ff3e41648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e41648b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c420 tx=0x7ff3d800c8e0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 shutdown_connections 2024-09-10T10:01:08.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff3c4067aa0 0x7ff3c4069f60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3e415c560 0x7ff3e40b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3e4155970 0x7ff3e41514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 --2- 172.21.15.110:0/2164820959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3e406f010 0x7ff3e41648b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 >> 172.21.15.110:0/2164820959 conn(0x7ff3e406bc50 msgr2=0x7ff3e415b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 shutdown_connections 2024-09-10T10:01:08.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.309+0000 7ff3ebe00640 1 -- 172.21.15.110:0/2164820959 wait complete. 2024-09-10T10:01:08.330 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:08.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f8201e00640 1 Processor -- start 2024-09-10T10:01:08.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f8201e00640 1 -- start start 2024-09-10T10:01:08.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f8201e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81fc0b6540 0x7f81fc0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f8201e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc0ce730 con 0x7f81fc06f200 2024-09-10T10:01:08.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f8201e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc0ce9d0 con 0x7f81fc15c340 2024-09-10T10:01:08.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f8201e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f81fc0cecb0 con 0x7f81fc0b6540 2024-09-10T10:01:08.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f81fbe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81fc0b6540 0x7f81fc0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f81fbe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81fc0b6540 0x7f81fc0cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39786/0 (socket says 172.21.15.110:39786) 2024-09-10T10:01:08.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f81fbe00640 1 -- 172.21.15.110:0/480235495 learned_addr learned my addr 172.21.15.110:0/480235495 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:08.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f8200a00640 1 --1- 172.21.15.110:0/480235495 >> v1:172.21.15.110:6789/0 conn(0x7f81fc06f200 0x7f81fc0b5e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49028/0 (socket says 172.21.15.110:49028) 2024-09-10T10:01:08.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f81fbe00640 1 -- 172.21.15.110:0/480235495 >> v1:172.21.15.145:6789/0 conn(0x7f81fc15c340 legacy=0x7f81fc0b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:08.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f81fbe00640 1 -- 172.21.15.110:0/480235495 >> v1:172.21.15.110:6789/0 conn(0x7f81fc06f200 legacy=0x7f81fc0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.465+0000 7f81fbe00640 1 -- 172.21.15.110:0/480235495 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0cef90 con 0x7f81fc0b6540 2024-09-10T10:01:08.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f81fbe00640 1 --2- 172.21.15.110:0/480235495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc0b6540 0x7f81fc0cf790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009870 tx=0x7f81fc0b5050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe2a4ea91eb111 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f81fb400640 1 -- 172.21.15.110:0/480235495 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80b0070 con 0x7f81fc0b6540 2024-09-10T10:01:08.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f81fb400640 1 -- 172.21.15.110:0/480235495 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e80ab600 con 0x7f81fc0b6540 2024-09-10T10:01:08.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f81fb400640 1 -- 172.21.15.110:0/480235495 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80ab9d0 con 0x7f81fc0b6540 2024-09-10T10:01:08.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 -- 172.21.15.110:0/480235495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc0b6540 msgr2=0x7f81fc0cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 --2- 172.21.15.110:0/480235495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc0b6540 0x7f81fc0cf790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009870 tx=0x7f81fc0b5050 comp rx=0 tx=0).stop 2024-09-10T10:01:08.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 -- 172.21.15.110:0/480235495 shutdown_connections 2024-09-10T10:01:08.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 --2- 172.21.15.110:0/480235495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc0b6540 0x7f81fc0cf790 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 -- 172.21.15.110:0/480235495 >> 172.21.15.110:0/480235495 conn(0x7f81fc06caa0 msgr2=0x7f81fc06ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 -- 172.21.15.110:0/480235495 shutdown_connections 2024-09-10T10:01:08.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 -- 172.21.15.110:0/480235495 wait complete. 2024-09-10T10:01:08.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 Processor -- start 2024-09-10T10:01:08.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.469+0000 7f8201e00640 1 -- start start 2024-09-10T10:01:08.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc06f200 0x7f81fc151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81fc0b6540 0x7f81fc1f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81fc15c340 0x7f81fc0732b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc072860 con 0x7f81fc06f200 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0d0640 con 0x7f81fc0b6540 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0729e0 con 0x7f81fc15c340 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc06f200 0x7f81fc151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc06f200 0x7f81fc151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39800/0 (socket says 172.21.15.110:39800) 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 -- 172.21.15.110:0/372289802 learned_addr learned my addr 172.21.15.110:0/372289802 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 -- 172.21.15.110:0/372289802 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81fc15c340 msgr2=0x7f81fc0732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81fc15c340 0x7f81fc0732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f81fbe00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81fc0b6540 0x7f81fc1f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 -- 172.21.15.110:0/372289802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81fc0b6540 msgr2=0x7f81fc1f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81fc0b6540 0x7f81fc1f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 -- 172.21.15.110:0/372289802 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0737f0 con 0x7f81fc06f200 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f81fbe00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81fc0b6540 0x7f81fc1f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8200a00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc06f200 0x7f81fc151640 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c420 tx=0x7f81f000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f81f8c00640 1 -- 172.21.15.110:0/372289802 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0017070 con 0x7f81fc06f200 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f81f8c00640 1 -- 172.21.15.110:0/372289802 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f00064e0 con 0x7f81fc06f200 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc154e90 con 0x7f81fc06f200 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f81f8c00640 1 -- 172.21.15.110:0/372289802 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f00049f0 con 0x7f81fc06f200 2024-09-10T10:01:08.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc154bb0 con 0x7f81fc06f200 2024-09-10T10:01:08.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc0cf790 con 0x7f81fc06f200 2024-09-10T10:01:08.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.473+0000 7f81f8c00640 1 -- 172.21.15.110:0/372289802 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f81f001a070 con 0x7f81fc06f200 2024-09-10T10:01:08.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.477+0000 7f81f8c00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81dc067a70 0x7f81dc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.477+0000 7f81f8c00640 1 -- 172.21.15.110:0/372289802 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f81f0085450 con 0x7f81fc06f200 2024-09-10T10:01:08.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.477+0000 7f81fbe00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81dc067a70 0x7f81dc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.477+0000 7f81fbe00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81dc067a70 0x7f81dc069f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f81e80ad000 tx=0x7f81e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.481+0000 7f81f8c00640 1 -- 172.21.15.110:0/372289802 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f81f0055c30 con 0x7f81fc06f200 2024-09-10T10:01:08.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.637+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81fc06d200 con 0x7f81fc06f200 2024-09-10T10:01:08.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f81f8c00640 1 -- 172.21.15.110:0/372289802 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16805 (secure 0 0 0) 0x7f81f0055530 con 0x7f81fc06f200 2024-09-10T10:01:08.642 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:08.642 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":47,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:00:04.960849+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:59:32.939641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T09:59:34.219832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.12","mappings":[{"from":0,"to":5}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:08.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81dc067a70 msgr2=0x7f81dc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81dc067a70 0x7f81dc069f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f81e80ad000 tx=0x7f81e8008040 comp rx=0 tx=0).stop 2024-09-10T10:01:08.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc06f200 msgr2=0x7f81fc151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc06f200 0x7f81fc151640 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c420 tx=0x7f81f000c8e0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 shutdown_connections 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81dc067a70 0x7f81dc069f30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81fc15c340 0x7f81fc0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81fc0b6540 0x7f81fc1f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 --2- 172.21.15.110:0/372289802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81fc06f200 0x7f81fc151640 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 >> 172.21.15.110:0/372289802 conn(0x7f81fc06caa0 msgr2=0x7f81fc06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 shutdown_connections 2024-09-10T10:01:08.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.641+0000 7f8201e00640 1 -- 172.21.15.110:0/372289802 wait complete. 2024-09-10T10:01:08.663 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:01:08.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.797+0000 7fde84800640 1 Processor -- start 2024-09-10T10:01:08.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.797+0000 7fde84800640 1 -- start start 2024-09-10T10:01:08.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.797+0000 7fde84800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0626e0 0x7fde7c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.797+0000 7fde84800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c0b0190 con 0x7fde7c0afce0 2024-09-10T10:01:08.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.797+0000 7fde84800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c0ae9e0 con 0x7fde7c15e540 2024-09-10T10:01:08.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.797+0000 7fde84800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0aecc0 con 0x7fde7c0626e0 2024-09-10T10:01:08.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde82800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fde7c0afce0 0x7fde7c06f450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49034/0 (socket says 172.21.15.110:49034) 2024-09-10T10:01:08.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde82800640 1 -- 172.21.15.110:0/1825219416 learned_addr learned my addr 172.21.15.110:0/1825219416 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:08.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde81400640 1 --2- 172.21.15.110:0/1825219416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0626e0 0x7fde7c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde80a00640 1 -- 172.21.15.110:0/1825219416 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521420432 0 0) 0x7fde7c0b0190 con 0x7fde7c0afce0 2024-09-10T10:01:08.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde80a00640 1 -- 172.21.15.110:0/1825219416 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde64003530 con 0x7fde7c0afce0 2024-09-10T10:01:08.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde80a00640 1 -- 172.21.15.110:0/1825219416 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389578776 0 0) 0x7fde7c0ae9e0 con 0x7fde7c15e540 2024-09-10T10:01:08.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde80a00640 1 -- 172.21.15.110:0/1825219416 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde7c0b0190 con 0x7fde7c15e540 2024-09-10T10:01:08.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde81400640 1 -- 172.21.15.110:0/1825219416 >> v1:172.21.15.145:6789/0 conn(0x7fde7c15e540 legacy=0x7fde7c0b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde81400640 1 -- 172.21.15.110:0/1825219416 >> v1:172.21.15.110:6789/0 conn(0x7fde7c0afce0 legacy=0x7fde7c06f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde81400640 1 -- 172.21.15.110:0/1825219416 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c0aefa0 con 0x7fde7c0626e0 2024-09-10T10:01:08.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde81400640 1 --2- 172.21.15.110:0/1825219416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0626e0 0x7fde7c0af7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0097f0 tx=0x7fde6c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1638fa4b148d1320 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.801+0000 7fde80a00640 1 -- 172.21.15.110:0/1825219416 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0b0070 con 0x7fde7c0626e0 2024-09-10T10:01:08.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde80a00640 1 -- 172.21.15.110:0/1825219416 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c0ab400 con 0x7fde7c0626e0 2024-09-10T10:01:08.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde80a00640 1 -- 172.21.15.110:0/1825219416 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0ab7b0 con 0x7fde7c0626e0 2024-09-10T10:01:08.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- 172.21.15.110:0/1825219416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0626e0 msgr2=0x7fde7c0af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 --2- 172.21.15.110:0/1825219416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0626e0 0x7fde7c0af7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0097f0 tx=0x7fde6c01c450 comp rx=0 tx=0).stop 2024-09-10T10:01:08.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- 172.21.15.110:0/1825219416 shutdown_connections 2024-09-10T10:01:08.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 --2- 172.21.15.110:0/1825219416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0626e0 0x7fde7c0af7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- 172.21.15.110:0/1825219416 >> 172.21.15.110:0/1825219416 conn(0x7fde7c0b0970 msgr2=0x7fde7c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- 172.21.15.110:0/1825219416 shutdown_connections 2024-09-10T10:01:08.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- 172.21.15.110:0/1825219416 wait complete. 2024-09-10T10:01:08.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 Processor -- start 2024-09-10T10:01:08.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- start start 2024-09-10T10:01:08.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde7c0626e0 0x7fde7c06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0afce0 0x7fde7c153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde7c15e540 0x7fde7c155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c16c510 con 0x7fde7c15e540 2024-09-10T10:01:08.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c16c390 con 0x7fde7c0626e0 2024-09-10T10:01:08.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde84800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c16c690 con 0x7fde7c0afce0 2024-09-10T10:01:08.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde82800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde7c15e540 0x7fde7c155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde82800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde7c15e540 0x7fde7c155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39802/0 (socket says 172.21.15.110:39802) 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde82800640 1 -- 172.21.15.110:0/172746163 learned_addr learned my addr 172.21.15.110:0/172746163 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde81400640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0afce0 0x7fde7c153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.805+0000 7fde81e00640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde7c0626e0 0x7fde7c06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde82800640 1 -- 172.21.15.110:0/172746163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0afce0 msgr2=0x7fde7c153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde82800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0afce0 0x7fde7c153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde82800640 1 -- 172.21.15.110:0/172746163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde7c0626e0 msgr2=0x7fde7c06ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde82800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde7c0626e0 0x7fde7c06ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde82800640 1 -- 172.21.15.110:0/172746163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c152b00 con 0x7fde7c15e540 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde82800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde7c15e540 0x7fde7c155590 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fde7800cba0 tx=0x7fde7800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde76000640 1 -- 172.21.15.110:0/172746163 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78017070 con 0x7fde7c15e540 2024-09-10T10:01:08.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde76000640 1 -- 172.21.15.110:0/172746163 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde7800ace0 con 0x7fde7c15e540 2024-09-10T10:01:08.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde76000640 1 -- 172.21.15.110:0/172746163 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78012620 con 0x7fde7c15e540 2024-09-10T10:01:08.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c152de0 con 0x7fde7c15e540 2024-09-10T10:01:08.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1533a0 con 0x7fde7c15e540 2024-09-10T10:01:08.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c0af7a0 con 0x7fde7c15e540 2024-09-10T10:01:08.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde76000640 1 -- 172.21.15.110:0/172746163 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fde7801a070 con 0x7fde7c15e540 2024-09-10T10:01:08.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde76000640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fde60067a70 0x7fde60069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:08.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde76000640 1 -- 172.21.15.110:0/172746163 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fde78085cd0 con 0x7fde7c15e540 2024-09-10T10:01:08.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.809+0000 7fde81e00640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fde60067a70 0x7fde60069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:08.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.813+0000 7fde81e00640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fde60067a70 0x7fde60069f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde7c06e740 tx=0x7fde70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:08.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:08.813+0000 7fde76000640 1 -- 172.21.15.110:0/172746163 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fde78055f90 con 0x7fde7c15e540 2024-09-10T10:01:09.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.005+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fde7c0d04f0 con 0x7fde7c15e540 2024-09-10T10:01:09.008 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:01:09.005+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:01:09.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde76000640 1 -- 172.21.15.110:0/172746163 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fde7805a290 con 0x7fde7c15e540 2024-09-10T10:01:09.010 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:01:09.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fde60067a70 msgr2=0x7fde60069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:09.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fde60067a70 0x7fde60069f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde7c06e740 tx=0x7fde70009040 comp rx=0 tx=0).stop 2024-09-10T10:01:09.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde7c15e540 msgr2=0x7fde7c155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:09.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde7c15e540 0x7fde7c155590 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fde7800cba0 tx=0x7fde7800a430 comp rx=0 tx=0).stop 2024-09-10T10:01:09.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 shutdown_connections 2024-09-10T10:01:09.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fde60067a70 0x7fde60069f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde7c15e540 0x7fde7c155590 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde7c0afce0 0x7fde7c153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 --2- 172.21.15.110:0/172746163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde7c0626e0 0x7fde7c06ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 >> 172.21.15.110:0/172746163 conn(0x7fde7c0b0970 msgr2=0x7fde7c14da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:09.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 shutdown_connections 2024-09-10T10:01:09.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.009+0000 7fde84800640 1 -- 172.21.15.110:0/172746163 wait complete. 2024-09-10T10:01:09.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:01:09.077 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:01:09.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.173+0000 7eff65400640 1 Processor -- start 2024-09-10T10:01:09.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.173+0000 7eff65400640 1 -- start start 2024-09-10T10:01:09.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff65400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7eff6015c500 0x7eff600d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:09.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff65400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7eff600b6720 0x7eff6006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff65400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff6006dc80 con 0x7eff600b0160 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff65400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7eff6006df60 con 0x7eff6015c500 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff65400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7eff6006e240 con 0x7eff600b6720 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff5fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7eff6015c500 0x7eff600d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff5fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7eff6015c500 0x7eff600d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39820/0 (socket says 172.21.15.110:39820) 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff5fe00640 1 -- 172.21.15.110:0/2138526615 learned_addr learned my addr 172.21.15.110:0/2138526615 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff64a00640 1 --2- 172.21.15.110:0/2138526615 >> v2:172.21.15.145:3300/0 conn(0x7eff600b6720 0x7eff6006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff5fe00640 1 -- 172.21.15.110:0/2138526615 >> v2:172.21.15.145:3300/0 conn(0x7eff600b6720 msgr2=0x7eff6006ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff5fe00640 1 --2- 172.21.15.110:0/2138526615 >> v2:172.21.15.145:3300/0 conn(0x7eff600b6720 0x7eff6006ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff5fe00640 1 -- 172.21.15.110:0/2138526615 >> v1:172.21.15.110:6789/0 conn(0x7eff600b0160 legacy=0x7eff600b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff5ea00640 1 -- 172.21.15.110:0/2138526615 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837761414 0 0) 0x7eff6006dc80 con 0x7eff600b0160 2024-09-10T10:01:09.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.177+0000 7eff5fe00640 1 -- 172.21.15.110:0/2138526615 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6006e520 con 0x7eff6015c500 2024-09-10T10:01:09.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff5fe00640 1 --2- 172.21.15.110:0/2138526615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff600d0670 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7eff600b3d90 tx=0x7eff5001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b230253c25005c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:09.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff5ea00640 1 -- 172.21.15.110:0/2138526615 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff500a5620 con 0x7eff6015c500 2024-09-10T10:01:09.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff5ea00640 1 -- 172.21.15.110:0/2138526615 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff500a57c0 con 0x7eff6015c500 2024-09-10T10:01:09.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff5ea00640 1 -- 172.21.15.110:0/2138526615 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff500a5b90 con 0x7eff6015c500 2024-09-10T10:01:09.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- 172.21.15.110:0/2138526615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 msgr2=0x7eff600d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:09.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 --2- 172.21.15.110:0/2138526615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff600d0670 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7eff600b3d90 tx=0x7eff5001c960 comp rx=0 tx=0).stop 2024-09-10T10:01:09.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- 172.21.15.110:0/2138526615 shutdown_connections 2024-09-10T10:01:09.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 --2- 172.21.15.110:0/2138526615 >> v2:172.21.15.145:3300/0 conn(0x7eff600b6720 0x7eff6006ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 --2- 172.21.15.110:0/2138526615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff600d0670 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- 172.21.15.110:0/2138526615 >> 172.21.15.110:0/2138526615 conn(0x7eff600b0970 msgr2=0x7eff600b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:09.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- 172.21.15.110:0/2138526615 shutdown_connections 2024-09-10T10:01:09.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- 172.21.15.110:0/2138526615 wait complete. 2024-09-10T10:01:09.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 Processor -- start 2024-09-10T10:01:09.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- start start 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff600b0160 0x7eff60149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff600b6720 0x7eff60148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff60072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7eff6006fe80 con 0x7eff6015c500 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7eff6006fd00 con 0x7eff600b6720 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff65400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7eff60070000 con 0x7eff600b0160 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff64a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff60072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff64a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff60072bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39826/0 (socket says 172.21.15.110:39826) 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.181+0000 7eff64a00640 1 -- 172.21.15.110:0/941710529 learned_addr learned my addr 172.21.15.110:0/941710529 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:09.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff5fe00640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff600b0160 0x7eff60149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:09.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff5f400640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff600b6720 0x7eff60148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:09.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff64a00640 1 -- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff600b0160 msgr2=0x7eff60149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:09.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff64a00640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff600b0160 0x7eff60149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff64a00640 1 -- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff600b6720 msgr2=0x7eff60148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:09.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff64a00640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff600b6720 0x7eff60148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:09.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff64a00640 1 -- 172.21.15.110:0/941710529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6014a610 con 0x7eff6015c500 2024-09-10T10:01:09.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff5f400640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff600b6720 0x7eff60148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:09.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff64a00640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff60072bd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7eff5800e7f0 tx=0x7eff5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:09.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff57e00640 1 -- 172.21.15.110:0/941710529 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58017070 con 0x7eff6015c500 2024-09-10T10:01:09.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff57e00640 1 -- 172.21.15.110:0/941710529 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5800c940 con 0x7eff6015c500 2024-09-10T10:01:09.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff60151cd0 con 0x7eff6015c500 2024-09-10T10:01:09.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff57e00640 1 -- 172.21.15.110:0/941710529 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5800eea0 con 0x7eff6015c500 2024-09-10T10:01:09.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff60152290 con 0x7eff6015c500 2024-09-10T10:01:09.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6006ed20 con 0x7eff6015c500 2024-09-10T10:01:09.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff57e00640 1 -- 172.21.15.110:0/941710529 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7eff5801a070 con 0x7eff6015c500 2024-09-10T10:01:09.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff57e00640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7eff40067a70 0x7eff40069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:09.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff57e00640 1 -- 172.21.15.110:0/941710529 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7eff58086310 con 0x7eff6015c500 2024-09-10T10:01:09.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.185+0000 7eff5fe00640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7eff40067a70 0x7eff40069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:09.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.189+0000 7eff5fe00640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7eff40067a70 0x7eff40069f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff500ae000 tx=0x7eff500a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:09.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.193+0000 7eff57e00640 1 -- 172.21.15.110:0/941710529 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7eff580563c0 con 0x7eff6015c500 2024-09-10T10:01:09.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:09.381+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7eff60151890 con 0x7eff6015c500 2024-09-10T10:01:10.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.009+0000 7eff57e00640 1 -- 172.21.15.110:0/941710529 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7eff5805a6c0 con 0x7eff6015c500 2024-09-10T10:01:10.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7eff40067a70 msgr2=0x7eff40069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7eff40067a70 0x7eff40069f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff500ae000 tx=0x7eff500a7070 comp rx=0 tx=0).stop 2024-09-10T10:01:10.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 msgr2=0x7eff60072bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff60072bd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7eff5800e7f0 tx=0x7eff5800ecb0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 shutdown_connections 2024-09-10T10:01:10.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7eff40067a70 0x7eff40069f30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff6015c500 0x7eff60072bd0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff600b6720 0x7eff60148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 --2- 172.21.15.110:0/941710529 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7eff600b0160 0x7eff60149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 >> 172.21.15.110:0/941710529 conn(0x7eff600b0970 msgr2=0x7eff601595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 shutdown_connections 2024-09-10T10:01:10.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.013+0000 7eff65400640 1 -- 172.21.15.110:0/941710529 wait complete. 2024-09-10T10:01:10.037 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:01:10.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4460c00640 1 Processor -- start 2024-09-10T10:01:10.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4460c00640 1 -- start start 2024-09-10T10:01:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4460c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c0b4bd0 0x7f445c06e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4460c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f445c163a60 0x7f445c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4460c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f445c06d770 con 0x7f445c071ff0 2024-09-10T10:01:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4460c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f445c06da50 con 0x7f445c163a60 2024-09-10T10:01:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4460c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f445c06dd30 con 0x7f445c0b4bd0 2024-09-10T10:01:10.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c0b4bd0 0x7f445c06e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f445c163a60 0x7f445c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c0b4bd0 0x7f445c06e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:45644/0 (socket says 172.21.15.110:45644) 2024-09-10T10:01:10.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f445c163a60 0x7f445c071ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:59410/0 (socket says 172.21.15.110:59410) 2024-09-10T10:01:10.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459e00640 1 -- 172.21.15.110:0/1807042241 learned_addr learned my addr 172.21.15.110:0/1807042241 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:10.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459400640 1 -- 172.21.15.110:0/1807042241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c0b4bd0 msgr2=0x7f445c06e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459400640 1 --2- 172.21.15.110:0/1807042241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c0b4bd0 0x7f445c06e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459400640 1 -- 172.21.15.110:0/1807042241 >> v1:172.21.15.145:6789/0 conn(0x7f445c071ff0 legacy=0x7f445c15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459400640 1 -- 172.21.15.110:0/1807042241 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c06e010 con 0x7f445c163a60 2024-09-10T10:01:10.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4458a00640 1 -- 172.21.15.110:0/1807042241 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245323539 0 0) 0x7f445c06d770 con 0x7f445c071ff0 2024-09-10T10:01:10.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.225+0000 7f4459e00640 1 --2- 172.21.15.110:0/1807042241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c0b4bd0 0x7f445c06e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:10.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4459400640 1 --2- 172.21.15.110:0/1807042241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c163a60 0x7f445c071ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f44480097c0 tx=0x7f444801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cb20eee691dc21f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4458a00640 1 -- 172.21.15.110:0/1807042241 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44480b1070 con 0x7f445c163a60 2024-09-10T10:01:10.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4458a00640 1 -- 172.21.15.110:0/1807042241 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44480ac410 con 0x7f445c163a60 2024-09-10T10:01:10.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4458a00640 1 -- 172.21.15.110:0/1807042241 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44480ac7e0 con 0x7f445c163a60 2024-09-10T10:01:10.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4460c00640 1 -- 172.21.15.110:0/1807042241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c163a60 msgr2=0x7f445c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4460c00640 1 --2- 172.21.15.110:0/1807042241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c163a60 0x7f445c071ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f44480097c0 tx=0x7f444801c4a0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4460c00640 1 -- 172.21.15.110:0/1807042241 shutdown_connections 2024-09-10T10:01:10.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4460c00640 1 --2- 172.21.15.110:0/1807042241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c163a60 0x7f445c071ab0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4460c00640 1 --2- 172.21.15.110:0/1807042241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c0b4bd0 0x7f445c06e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4460c00640 1 -- 172.21.15.110:0/1807042241 >> 172.21.15.110:0/1807042241 conn(0x7f445c06bc50 msgr2=0x7f445c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4460c00640 1 -- 172.21.15.110:0/1807042241 shutdown_connections 2024-09-10T10:01:10.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.229+0000 7f4460c00640 1 -- 172.21.15.110:0/1807042241 wait complete. 2024-09-10T10:01:10.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 Processor -- start 2024-09-10T10:01:10.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 -- start start 2024-09-10T10:01:10.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c071ff0 0x7f445c158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f445c0b4bd0 0x7f445c15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c163a60 0x7f445c1fb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f445c06d530 con 0x7f445c0b4bd0 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f445c06d3b0 con 0x7f445c071ff0 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f445c0b4fb0 con 0x7f445c163a60 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f445c0b4bd0 0x7f445c15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f445c0b4bd0 0x7f445c15eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39836/0 (socket says 172.21.15.110:39836) 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 -- 172.21.15.110:0/996483784 learned_addr learned my addr 172.21.15.110:0/996483784 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459e00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c071ff0 0x7f445c158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f445a800640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c163a60 0x7f445c1fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 -- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c163a60 msgr2=0x7f445c1fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c163a60 0x7f445c1fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 -- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c071ff0 msgr2=0x7f445c158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c071ff0 0x7f445c158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 -- 172.21.15.110:0/996483784 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c0b0830 con 0x7f445c0b4bd0 2024-09-10T10:01:10.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f445a800640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c163a60 0x7f445c1fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:10.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4459400640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f445c0b4bd0 0x7f445c15eed0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f444801ab50 tx=0x7f4448019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4452000640 1 -- 172.21.15.110:0/996483784 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44480bb020 con 0x7f445c0b4bd0 2024-09-10T10:01:10.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4452000640 1 -- 172.21.15.110:0/996483784 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44480acce0 con 0x7f445c0b4bd0 2024-09-10T10:01:10.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4452000640 1 -- 172.21.15.110:0/996483784 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44480b1040 con 0x7f445c0b4bd0 2024-09-10T10:01:10.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445c0b0ad0 con 0x7f445c0b4bd0 2024-09-10T10:01:10.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.233+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445c0b1090 con 0x7f445c0b4bd0 2024-09-10T10:01:10.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.237+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445c161080 con 0x7f445c0b4bd0 2024-09-10T10:01:10.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.237+0000 7f4452000640 1 -- 172.21.15.110:0/996483784 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f44480be020 con 0x7f445c0b4bd0 2024-09-10T10:01:10.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.237+0000 7f4452000640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f443c067a70 0x7f443c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.237+0000 7f4452000640 1 -- 172.21.15.110:0/996483784 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f4448120090 con 0x7f445c0b4bd0 2024-09-10T10:01:10.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.237+0000 7f4459e00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f443c067a70 0x7f443c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.237+0000 7f4459e00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f443c067a70 0x7f443c069f30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f444c0072c0 tx=0x7f444c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.241+0000 7f4452000640 1 -- 172.21.15.110:0/996483784 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f44480c5030 con 0x7f445c0b4bd0 2024-09-10T10:01:10.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:10.401+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f445c0afab0 con 0x7f445c0b4bd0 2024-09-10T10:01:11.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.129+0000 7f4452000640 1 -- 172.21.15.110:0/996483784 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7f4448016090 con 0x7f445c0b4bd0 2024-09-10T10:01:11.132 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:01:11.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f443c067a70 msgr2=0x7f443c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f443c067a70 0x7f443c069f30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f444c0072c0 tx=0x7f444c00b040 comp rx=0 tx=0).stop 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f445c0b4bd0 msgr2=0x7f445c15eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f445c0b4bd0 0x7f445c15eed0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f444801ab50 tx=0x7f4448019240 comp rx=0 tx=0).stop 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 shutdown_connections 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f443c067a70 0x7f443c069f30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f445c163a60 0x7f445c1fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f445c0b4bd0 0x7f445c15eed0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 --2- 172.21.15.110:0/996483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c071ff0 0x7f445c158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 >> 172.21.15.110:0/996483784 conn(0x7f445c06bc50 msgr2=0x7f445c0b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 shutdown_connections 2024-09-10T10:01:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.133+0000 7f4460c00640 1 -- 172.21.15.110:0/996483784 wait complete. 2024-09-10T10:01:11.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:01:11.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f041e800640 1 Processor -- start 2024-09-10T10:01:11.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f041e800640 1 -- start start 2024-09-10T10:01:11.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f041e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f04180cfe30 0x7f0418071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:11.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f041e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04180cd510 0x7f04180cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:11.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f041e800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041806d2d0 con 0x7f041806e090 2024-09-10T10:01:11.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f041e800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f041815d9a0 con 0x7f04180cd510 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f041e800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f041806d5b0 con 0x7f04180cfe30 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0416a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04180cd510 0x7f04180cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0417e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f041806e090 0x7f041806e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49068/0 (socket says 172.21.15.110:49068) 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0417e00640 1 -- 172.21.15.110:0/266767359 learned_addr learned my addr 172.21.15.110:0/266767359 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0417400640 1 --2- 172.21.15.110:0/266767359 >> v2:172.21.15.145:3300/0 conn(0x7f04180cfe30 0x7f0418071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0416a00640 1 -- 172.21.15.110:0/266767359 >> v2:172.21.15.145:3300/0 conn(0x7f04180cfe30 msgr2=0x7f0418071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0416a00640 1 --2- 172.21.15.110:0/266767359 >> v2:172.21.15.145:3300/0 conn(0x7f04180cfe30 0x7f0418071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0416a00640 1 -- 172.21.15.110:0/266767359 >> v1:172.21.15.110:6789/0 conn(0x7f041806e090 legacy=0x7f041806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0416a00640 1 -- 172.21.15.110:0/266767359 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041806d890 con 0x7f04180cd510 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0416000640 1 -- 172.21.15.110:0/266767359 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785804622 0 0) 0x7f041806d2d0 con 0x7f041806e090 2024-09-10T10:01:11.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.301+0000 7f0417400640 1 --2- 172.21.15.110:0/266767359 >> v2:172.21.15.145:3300/0 conn(0x7f04180cfe30 0x7f0418071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:11.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f0416a00640 1 --2- 172.21.15.110:0/266767359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f04180cd9a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f04180ce120 tx=0x7f040401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fb75c505f69478 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:11.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f0416000640 1 -- 172.21.15.110:0/266767359 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04040b0070 con 0x7f04180cd510 2024-09-10T10:01:11.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f0416000640 1 -- 172.21.15.110:0/266767359 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04040ab600 con 0x7f04180cd510 2024-09-10T10:01:11.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f0416000640 1 -- 172.21.15.110:0/266767359 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04040ab9d0 con 0x7f04180cd510 2024-09-10T10:01:11.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f041e800640 1 -- 172.21.15.110:0/266767359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 msgr2=0x7f04180cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:11.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f041e800640 1 --2- 172.21.15.110:0/266767359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f04180cd9a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f04180ce120 tx=0x7f040401c3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f041e800640 1 -- 172.21.15.110:0/266767359 shutdown_connections 2024-09-10T10:01:11.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f041e800640 1 --2- 172.21.15.110:0/266767359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f04180cd9a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f041e800640 1 --2- 172.21.15.110:0/266767359 >> v2:172.21.15.145:3300/0 conn(0x7f04180cfe30 0x7f0418071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f041e800640 1 -- 172.21.15.110:0/266767359 >> 172.21.15.110:0/266767359 conn(0x7f041806bc50 msgr2=0x7f041806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:11.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f041e800640 1 -- 172.21.15.110:0/266767359 shutdown_connections 2024-09-10T10:01:11.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.305+0000 7f041e800640 1 -- 172.21.15.110:0/266767359 wait complete. 2024-09-10T10:01:11.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 Processor -- start 2024-09-10T10:01:11.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 -- start start 2024-09-10T10:01:11.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f041806e090 0x7f04180b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:11.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f0418155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:11.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04180cfe30 0x7f04181499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:11.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0418072190 con 0x7f04180cd510 2024-09-10T10:01:11.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f041815d6c0 con 0x7f04180cfe30 2024-09-10T10:01:11.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0418169180 con 0x7f041806e090 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f0418155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f0418155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39846/0 (socket says 172.21.15.110:39846) 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 -- 172.21.15.110:0/1438467420 learned_addr learned my addr 172.21.15.110:0/1438467420 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0417400640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f041806e090 0x7f04180b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0417e00640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04180cfe30 0x7f04181499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 -- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f041806e090 msgr2=0x7f04180b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f041806e090 0x7f04180b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 -- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04180cfe30 msgr2=0x7f04181499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04180cfe30 0x7f04181499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 -- 172.21.15.110:0/1438467420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418159200 con 0x7f04180cd510 2024-09-10T10:01:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0417e00640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04180cfe30 0x7f04181499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:11.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0417400640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f041806e090 0x7f04180b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:11.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f0416a00640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f0418155b20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f040401a020 tx=0x7f040401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:11.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f040f400640 1 -- 172.21.15.110:0/1438467420 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04040b0050 con 0x7f04180cd510 2024-09-10T10:01:11.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f040f400640 1 -- 172.21.15.110:0/1438467420 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04040b1040 con 0x7f04180cd510 2024-09-10T10:01:11.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0418153c30 con 0x7f04180cd510 2024-09-10T10:01:11.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f040f400640 1 -- 172.21.15.110:0/1438467420 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404004420 con 0x7f04180cd510 2024-09-10T10:01:11.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181541f0 con 0x7f04180cd510 2024-09-10T10:01:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.309+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04181543c0 con 0x7f04180cd510 2024-09-10T10:01:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.313+0000 7f040f400640 1 -- 172.21.15.110:0/1438467420 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f04040be060 con 0x7f04180cd510 2024-09-10T10:01:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.313+0000 7f040f400640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03f8067a70 0x7f03f8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.313+0000 7f040f400640 1 -- 172.21.15.110:0/1438467420 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5415+0+0 (secure 0 0 0) 0x7f040411f950 con 0x7f04180cd510 2024-09-10T10:01:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.313+0000 7f0417400640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03f8067a70 0x7f03f8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:11.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.313+0000 7f0417400640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03f8067a70 0x7f03f8069f30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0408009540 tx=0x7f040800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.317+0000 7f040f400640 1 -- 172.21.15.110:0/1438467420 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0404016070 con 0x7f04180cd510 2024-09-10T10:01:11.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:11.477+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0418076350 con 0x7f04180cd510 2024-09-10T10:01:12.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.141+0000 7f040f400640 1 -- 172.21.15.110:0/1438467420 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7f04040f02b0 con 0x7f04180cd510 2024-09-10T10:01:12.146 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:01:12.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03f8067a70 msgr2=0x7f03f8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03f8067a70 0x7f03f8069f30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0408009540 tx=0x7f040800b040 comp rx=0 tx=0).stop 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 msgr2=0x7f0418155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f0418155b20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f040401a020 tx=0x7f040401ac30 comp rx=0 tx=0).stop 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 shutdown_connections 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03f8067a70 0x7f03f8069f30 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0408009540 tx=0x7f040800b040 comp rx=0 tx=0).stop 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04180cfe30 0x7f04181499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04180cd510 0x7f0418155b20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 --2- 172.21.15.110:0/1438467420 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f041806e090 0x7f04180b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 >> 172.21.15.110:0/1438467420 conn(0x7f041806bc50 msgr2=0x7f041806e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 shutdown_connections 2024-09-10T10:01:12.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.149+0000 7f041e800640 1 -- 172.21.15.110:0/1438467420 wait complete. 2024-09-10T10:01:12.183 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:01:12.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7f2400640 1 Processor -- start 2024-09-10T10:01:12.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7f2400640 1 -- start start 2024-09-10T10:01:12.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7f2400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa7ec0cfce0 0x7fa7ec15f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7f2400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa7ec06e140 0x7fa7ec06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7f2400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ec0b1e90 con 0x7fa7ec0ae840 2024-09-10T10:01:12.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7f2400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa7ec06ead0 con 0x7fa7ec06e140 2024-09-10T10:01:12.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7f2400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa7ec0d00c0 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7ebe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa7ec06e140 0x7fa7ec06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7ebe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa7ec06e140 0x7fa7ec06e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39856/0 (socket says 172.21.15.110:39856) 2024-09-10T10:01:12.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7ebe00640 1 -- 172.21.15.110:0/2681744984 learned_addr learned my addr 172.21.15.110:0/2681744984 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:12.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7eaa00640 1 --1- 172.21.15.110:0/2681744984 >> v1:172.21.15.110:6789/0 conn(0x7fa7ec0ae840 0x7fa7ec0b3b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49080/0 (socket says 172.21.15.110:49080) 2024-09-10T10:01:12.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7eb400640 1 --2- 172.21.15.110:0/2681744984 >> v2:172.21.15.145:3300/0 conn(0x7fa7ec0cfce0 0x7fa7ec15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7ebe00640 1 -- 172.21.15.110:0/2681744984 >> v2:172.21.15.145:3300/0 conn(0x7fa7ec0cfce0 msgr2=0x7fa7ec15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7ebe00640 1 --2- 172.21.15.110:0/2681744984 >> v2:172.21.15.145:3300/0 conn(0x7fa7ec0cfce0 0x7fa7ec15f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7ebe00640 1 -- 172.21.15.110:0/2681744984 >> v1:172.21.15.110:6789/0 conn(0x7fa7ec0ae840 legacy=0x7fa7ec0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7ebe00640 1 -- 172.21.15.110:0/2681744984 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec0b43d0 con 0x7fa7ec06e140 2024-09-10T10:01:12.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.321+0000 7fa7eb400640 1 --2- 172.21.15.110:0/2681744984 >> v2:172.21.15.145:3300/0 conn(0x7fa7ec0cfce0 0x7fa7ec15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:12.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7ebe00640 1 --2- 172.21.15.110:0/2681744984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec06e140 0x7fa7ec06e590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0734f0 tx=0x7fa7e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc5e1e1d3cab2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7ea000640 1 -- 172.21.15.110:0/2681744984 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40b0070 con 0x7fa7ec06e140 2024-09-10T10:01:12.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7ea000640 1 -- 172.21.15.110:0/2681744984 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e40ab600 con 0x7fa7ec06e140 2024-09-10T10:01:12.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7ea000640 1 -- 172.21.15.110:0/2681744984 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40ab9d0 con 0x7fa7ec06e140 2024-09-10T10:01:12.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 -- 172.21.15.110:0/2681744984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec06e140 msgr2=0x7fa7ec06e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 --2- 172.21.15.110:0/2681744984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec06e140 0x7fa7ec06e590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0734f0 tx=0x7fa7e401c520 comp rx=0 tx=0).stop 2024-09-10T10:01:12.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 -- 172.21.15.110:0/2681744984 shutdown_connections 2024-09-10T10:01:12.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 --2- 172.21.15.110:0/2681744984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec06e140 0x7fa7ec06e590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 --2- 172.21.15.110:0/2681744984 >> v2:172.21.15.145:3300/0 conn(0x7fa7ec0cfce0 0x7fa7ec15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 -- 172.21.15.110:0/2681744984 >> 172.21.15.110:0/2681744984 conn(0x7fa7ec062e40 msgr2=0x7fa7ec06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 -- 172.21.15.110:0/2681744984 shutdown_connections 2024-09-10T10:01:12.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 -- 172.21.15.110:0/2681744984 wait complete. 2024-09-10T10:01:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.325+0000 7fa7f2400640 1 Processor -- start 2024-09-10T10:01:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 -- start start 2024-09-10T10:01:12.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7ec06e140 0x7fa7ec136d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7ec0ae840 0x7fa7ec1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec0cfce0 0x7fa7ec148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec0b2fc0 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec0b2e40 con 0x7fa7ec06e140 2024-09-10T10:01:12.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ec0b3140 con 0x7fa7ec0ae840 2024-09-10T10:01:12.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec0cfce0 0x7fa7ec148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7eaa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7ec0ae840 0x7fa7ec1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7eb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7ec06e140 0x7fa7ec136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec0cfce0 0x7fa7ec148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39860/0 (socket says 172.21.15.110:39860) 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7eaa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7ec0ae840 0x7fa7ec1592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:45672/0 (socket says 172.21.15.110:45672) 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 -- 172.21.15.110:0/743360905 learned_addr learned my addr 172.21.15.110:0/743360905 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 -- 172.21.15.110:0/743360905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7ec0ae840 msgr2=0x7fa7ec1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7ec0ae840 0x7fa7ec1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 -- 172.21.15.110:0/743360905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7ec06e140 msgr2=0x7fa7ec136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7ec06e140 0x7fa7ec136d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 -- 172.21.15.110:0/743360905 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec137280 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7ebe00640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec0cfce0 0x7fa7ec148eb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40ae040 tx=0x7fa7e40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7e3400640 1 -- 172.21.15.110:0/743360905 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40b8070 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7e3400640 1 -- 172.21.15.110:0/743360905 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e40b4410 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7e3400640 1 -- 172.21.15.110:0/743360905 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40b0040 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ec159b20 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ec1483d0 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.329+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ec06e590 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.333+0000 7fa7e3400640 1 -- 172.21.15.110:0/743360905 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fa7e40bb070 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.333+0000 7fa7e3400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa7cc067a70 0x7fa7cc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.333+0000 7fa7e3400640 1 -- 172.21.15.110:0/743360905 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fa7e411f7f0 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.333+0000 7fa7eb400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa7cc067a70 0x7fa7cc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.333+0000 7fa7eb400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa7cc067a70 0x7fa7cc069f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc003b90 tx=0x7fa7dc009890 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.337+0000 7fa7e3400640 1 -- 172.21.15.110:0/743360905 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa7e4016110 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.493+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa7ec0d0270 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7e3400640 1 -- 172.21.15.110:0/743360905 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fa7e40efcf0 con 0x7fa7ec0cfce0 2024-09-10T10:01:12.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa7cc067a70 msgr2=0x7fa7cc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa7cc067a70 0x7fa7cc069f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc003b90 tx=0x7fa7dc009890 comp rx=0 tx=0).stop 2024-09-10T10:01:12.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec0cfce0 msgr2=0x7fa7ec148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec0cfce0 0x7fa7ec148eb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40ae040 tx=0x7fa7e40192c0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 shutdown_connections 2024-09-10T10:01:12.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa7cc067a70 0x7fa7cc069f30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7ec0cfce0 0x7fa7ec148eb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7ec0ae840 0x7fa7ec1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 --2- 172.21.15.110:0/743360905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7ec06e140 0x7fa7ec136d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 >> 172.21.15.110:0/743360905 conn(0x7fa7ec062e40 msgr2=0x7fa7ec15e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 shutdown_connections 2024-09-10T10:01:12.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.497+0000 7fa7f2400640 1 -- 172.21.15.110:0/743360905 wait complete. 2024-09-10T10:01:12.520 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-10T10:01:12.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.653+0000 7fe4b7e00640 1 Processor -- start 2024-09-10T10:01:12.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.653+0000 7fe4b7e00640 1 -- start start 2024-09-10T10:01:12.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.653+0000 7fe4b7e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe4b0155970 0x7fe4b00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.653+0000 7fe4b7e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b0155d50 con 0x7fe4b006f1e0 2024-09-10T10:01:12.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.653+0000 7fe4b7e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b00b34d0 con 0x7fe4b015c560 2024-09-10T10:01:12.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.653+0000 7fe4b7e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b00b4570 con 0x7fe4b0155970 2024-09-10T10:01:12.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.653+0000 7fe4b5e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe4b006f1e0 0x7fe4b00ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49090/0 (socket says 172.21.15.110:49090) 2024-09-10T10:01:12.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.653+0000 7fe4b5e00640 1 -- 172.21.15.110:0/974058690 learned_addr learned my addr 172.21.15.110:0/974058690 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:12.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5400640 1 --2- 172.21.15.110:0/974058690 >> v2:172.21.15.145:3300/0 conn(0x7fe4b0155970 0x7fe4b00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686013958 0 0) 0x7fe4b0155d50 con 0x7fe4b006f1e0 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe498003530 con 0x7fe4b006f1e0 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2551398196 0 0) 0x7fe498003530 con 0x7fe4b006f1e0 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 >> v2:172.21.15.145:3300/0 conn(0x7fe4b0155970 msgr2=0x7fe4b00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 --2- 172.21.15.110:0/974058690 >> v2:172.21.15.145:3300/0 conn(0x7fe4b0155970 0x7fe4b00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 >> v1:172.21.15.145:6789/0 conn(0x7fe4b015c560 legacy=0x7fe4b006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b00b37b0 con 0x7fe4b006f1e0 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fe4a8001680 con 0x7fe4b006f1e0 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 >> v1:172.21.15.110:6789/0 conn(0x7fe4b006f1e0 legacy=0x7fe4b00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4980039c0 0x7fe498003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980043c0 0x7fe498004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe498005090 0x7fe49800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe49800e1d0 con 0x7fe498005090 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe49800e380 con 0x7fe4980039c0 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe49800e580 con 0x7fe4980043c0 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5400640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4980039c0 0x7fe498003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5e00640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe498005090 0x7fe49800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b4a00640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980043c0 0x7fe498004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5400640 1 -- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980043c0 msgr2=0x7fe498004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5400640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980043c0 0x7fe498004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5400640 1 -- 172.21.15.110:0/974058690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe498005090 msgr2=0x7fe49800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5400640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe498005090 0x7fe49800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5400640 1 -- 172.21.15.110:0/974058690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe498003530 con 0x7fe4980039c0 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5e00640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe498005090 0x7fe49800dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:12.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.657+0000 7fe4b5e00640 1 -- 172.21.15.110:0/974058690 reap_dead start 2024-09-10T10:01:12.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b5400640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4980039c0 0x7fe498003da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009fc0 tx=0x7fe4a401cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4417060a76c06afa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40b2070 con 0x7fe4980039c0 2024-09-10T10:01:12.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a40a5b80 con 0x7fe4980039c0 2024-09-10T10:01:12.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4afe00640 1 -- 172.21.15.110:0/974058690 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4009c10 con 0x7fe4980039c0 2024-09-10T10:01:12.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 -- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4980039c0 msgr2=0x7fe498003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4980039c0 0x7fe498003da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009fc0 tx=0x7fe4a401cc80 comp rx=0 tx=0).stop 2024-09-10T10:01:12.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 -- 172.21.15.110:0/974058690 shutdown_connections 2024-09-10T10:01:12.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 --2- 172.21.15.110:0/974058690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4980039c0 0x7fe498003da0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 -- 172.21.15.110:0/974058690 >> 172.21.15.110:0/974058690 conn(0x7fe4b006bc50 msgr2=0x7fe4b006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:12.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 -- 172.21.15.110:0/974058690 shutdown_connections 2024-09-10T10:01:12.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 -- 172.21.15.110:0/974058690 wait complete. 2024-09-10T10:01:12.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 Processor -- start 2024-09-10T10:01:12.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.661+0000 7fe4b7e00640 1 -- start start 2024-09-10T10:01:12.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980039c0 0x7fe4b0157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4b01509b0 0x7fe4b0156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4b0150180 0x7fe4b01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b00b4050 con 0x7fe4b0150180 2024-09-10T10:01:12.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b00b3d70 con 0x7fe4b01509b0 2024-09-10T10:01:12.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b00cc8c0 con 0x7fe4980039c0 2024-09-10T10:01:12.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4b0150180 0x7fe4b01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4b0150180 0x7fe4b01516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39870/0 (socket says 172.21.15.110:39870) 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 -- 172.21.15.110:0/1156885239 learned_addr learned my addr 172.21.15.110:0/1156885239 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b4a00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4b01509b0 0x7fe4b0156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5400640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980039c0 0x7fe4b0157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 -- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980039c0 msgr2=0x7fe4b0157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980039c0 0x7fe4b0157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 -- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4b01509b0 msgr2=0x7fe4b0156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4b01509b0 0x7fe4b0156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 -- 172.21.15.110:0/1156885239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b01568d0 con 0x7fe4b0150180 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b4a00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4b01509b0 0x7fe4b0156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5400640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980039c0 0x7fe4b0157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:12.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b5e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4b0150180 0x7fe4b01516e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00b5890 tx=0x7fe4a8012860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4ad600640 1 -- 172.21.15.110:0/1156885239 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a801c070 con 0x7fe4b0150180 2024-09-10T10:01:12.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4ad600640 1 -- 172.21.15.110:0/1156885239 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a80155e0 con 0x7fe4b0150180 2024-09-10T10:01:12.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4ad600640 1 -- 172.21.15.110:0/1156885239 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8007d40 con 0x7fe4b0150180 2024-09-10T10:01:12.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b0156bb0 con 0x7fe4b0150180 2024-09-10T10:01:12.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b0157170 con 0x7fe4b0150180 2024-09-10T10:01:12.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b00ccc50 con 0x7fe4b0150180 2024-09-10T10:01:12.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.665+0000 7fe4ad600640 1 -- 172.21.15.110:0/1156885239 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fe4a80159b0 con 0x7fe4b0150180 2024-09-10T10:01:12.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.669+0000 7fe4ad600640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe490067a70 0x7fe490069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:12.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.669+0000 7fe4ad600640 1 -- 172.21.15.110:0/1156885239 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fe4a80881d0 con 0x7fe4b0150180 2024-09-10T10:01:12.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.669+0000 7fe4b5400640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe490067a70 0x7fe490069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:12.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.669+0000 7fe4b5400640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe490067a70 0x7fe490069f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40af000 tx=0x7fe4a40a7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:12.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.673+0000 7fe4ad600640 1 -- 172.21.15.110:0/1156885239 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe4a8058730 con 0x7fe4b0150180 2024-09-10T10:01:12.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:12.885+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7fe4b0073210 con 0x7fe4b0150180 2024-09-10T10:01:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4ad600640 1 -- 172.21.15.110:0/1156885239 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v154) ==== 140+0+0 (secure 0 0 0) 0x7fe4a805ca30 con 0x7fe4b0150180 2024-09-10T10:01:13.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe490067a70 msgr2=0x7fe490069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe490067a70 0x7fe490069f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40af000 tx=0x7fe4a40a7040 comp rx=0 tx=0).stop 2024-09-10T10:01:13.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4b0150180 msgr2=0x7fe4b01516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4b0150180 0x7fe4b01516e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00b5890 tx=0x7fe4a8012860 comp rx=0 tx=0).stop 2024-09-10T10:01:13.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 shutdown_connections 2024-09-10T10:01:13.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe490067a70 0x7fe490069f30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4b0150180 0x7fe4b01516e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4b01509b0 0x7fe4b0156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 --2- 172.21.15.110:0/1156885239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe4980039c0 0x7fe4b0157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 >> 172.21.15.110:0/1156885239 conn(0x7fe4b006bc50 msgr2=0x7fe4b00b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:13.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 shutdown_connections 2024-09-10T10:01:13.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.153+0000 7fe4b7e00640 1 -- 172.21.15.110:0/1156885239 wait complete. 2024-09-10T10:01:13.175 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-10T10:01:13.176 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-10T10:01:13.176 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:01:13.176 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:13.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.309+0000 7f07c2800640 1 Processor -- start 2024-09-10T10:01:13.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.309+0000 7f07c2800640 1 -- start start 2024-09-10T10:01:13.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.309+0000 7f07c2800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc155970 0x7f07bc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.309+0000 7f07c2800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f07bc15c560 0x7f07bc06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.309+0000 7f07c2800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07bc155d50 con 0x7f07bc06f010 2024-09-10T10:01:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.309+0000 7f07c2800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f07bc0b4570 con 0x7f07bc15c560 2024-09-10T10:01:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.309+0000 7f07c2800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc06f450 con 0x7f07bc155970 2024-09-10T10:01:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07b2a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f07bc15c560 0x7f07bc06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07b2a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f07bc15c560 0x7f07bc06ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39874/0 (socket says 172.21.15.110:39874) 2024-09-10T10:01:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07b2a00640 1 -- 172.21.15.110:0/2204584166 learned_addr learned my addr 172.21.15.110:0/2204584166 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07bb400640 1 --2- 172.21.15.110:0/2204584166 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc155970 0x7f07bc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07b2a00640 1 -- 172.21.15.110:0/2204584166 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc155970 msgr2=0x7f07bc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07b2a00640 1 --2- 172.21.15.110:0/2204584166 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc155970 0x7f07bc0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07b2a00640 1 -- 172.21.15.110:0/2204584166 >> v1:172.21.15.145:6789/0 conn(0x7f07bc06f010 legacy=0x7f07bc15eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07b2a00640 1 -- 172.21.15.110:0/2204584166 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc0b34d0 con 0x7f07bc15c560 2024-09-10T10:01:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07bb400640 1 --2- 172.21.15.110:0/2204584166 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc155970 0x7f07bc0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:13.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07b2a00640 1 --2- 172.21.15.110:0/2204584166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc06ead0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f07bc06f5d0 tx=0x7f07a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5395320801bbeaa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:13.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07baa00640 1 -- 172.21.15.110:0/2204584166 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40ab680 con 0x7f07bc15c560 2024-09-10T10:01:13.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07baa00640 1 -- 172.21.15.110:0/2204584166 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a40b0070 con 0x7f07bc15c560 2024-09-10T10:01:13.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.313+0000 7f07baa00640 1 -- 172.21.15.110:0/2204584166 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40aba40 con 0x7f07bc15c560 2024-09-10T10:01:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- 172.21.15.110:0/2204584166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 msgr2=0x7f07bc06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 --2- 172.21.15.110:0/2204584166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc06ead0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f07bc06f5d0 tx=0x7f07a401c480 comp rx=0 tx=0).stop 2024-09-10T10:01:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- 172.21.15.110:0/2204584166 shutdown_connections 2024-09-10T10:01:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 --2- 172.21.15.110:0/2204584166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc06ead0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 --2- 172.21.15.110:0/2204584166 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc155970 0x7f07bc0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- 172.21.15.110:0/2204584166 >> 172.21.15.110:0/2204584166 conn(0x7f07bc06bc50 msgr2=0x7f07bc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- 172.21.15.110:0/2204584166 shutdown_connections 2024-09-10T10:01:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- 172.21.15.110:0/2204584166 wait complete. 2024-09-10T10:01:13.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 Processor -- start 2024-09-10T10:01:13.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- start start 2024-09-10T10:01:13.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc06f010 0x7f07bc15e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07bc155970 0x7f07bc15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc0cce00 con 0x7f07bc15c560 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc0b4050 con 0x7f07bc155970 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07c2800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc06d9c0 con 0x7f07bc06f010 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07bbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07bbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39880/0 (socket says 172.21.15.110:39880) 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07bbe00640 1 -- 172.21.15.110:0/3318480657 learned_addr learned my addr 172.21.15.110:0/3318480657 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07bb400640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc06f010 0x7f07bc15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.317+0000 7f07b2a00640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07bc155970 0x7f07bc15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07bbe00640 1 -- 172.21.15.110:0/3318480657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc06f010 msgr2=0x7f07bc15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07bbe00640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc06f010 0x7f07bc15e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07bbe00640 1 -- 172.21.15.110:0/3318480657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07bc155970 msgr2=0x7f07bc15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07bbe00640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07bc155970 0x7f07bc15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07bbe00640 1 -- 172.21.15.110:0/3318480657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc146420 con 0x7f07bc15c560 2024-09-10T10:01:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07bbe00640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc145ee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f07b400c7a0 tx=0x7f07b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07b3e00640 1 -- 172.21.15.110:0/3318480657 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4003b80 con 0x7f07bc15c560 2024-09-10T10:01:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07b3e00640 1 -- 172.21.15.110:0/3318480657 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b4003d20 con 0x7f07bc15c560 2024-09-10T10:01:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc15d1c0 con 0x7f07bc15c560 2024-09-10T10:01:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07b3e00640 1 -- 172.21.15.110:0/3318480657 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b400aa90 con 0x7f07bc15c560 2024-09-10T10:01:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc14d490 con 0x7f07bc15c560 2024-09-10T10:01:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07bc0cc8c0 con 0x7f07bc15c560 2024-09-10T10:01:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07b3e00640 1 -- 172.21.15.110:0/3318480657 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f07b400ac30 con 0x7f07bc15c560 2024-09-10T10:01:13.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.321+0000 7f07b3e00640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0798067a70 0x7f0798069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.325+0000 7f07b3e00640 1 -- 172.21.15.110:0/3318480657 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f07b4085a60 con 0x7f07bc15c560 2024-09-10T10:01:13.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.325+0000 7f07bb400640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0798067a70 0x7f0798069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.325+0000 7f07bb400640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0798067a70 0x7f0798069f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07bc15b170 tx=0x7f07ac00b4f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:13.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.329+0000 7f07b3e00640 1 -- 172.21.15.110:0/3318480657 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f07b4055fc0 con 0x7f07bc15c560 2024-09-10T10:01:13.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.489+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07bc15ee80 con 0x7f07bc15c560 2024-09-10T10:01:13.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.489+0000 7f07b3e00640 1 -- 172.21.15.110:0/3318480657 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13070 (secure 0 0 0) 0x7f07b405a2c0 con 0x7f07bc15c560 2024-09-10T10:01:13.493 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:13.494 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":50,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:12.141141+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6837/2457861782":"2024-09-11T10:01:09.008012+0000","172.21.15.110:6835/2457861782":"2024-09-11T10:01:09.008012+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:13.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0798067a70 msgr2=0x7f0798069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0798067a70 0x7f0798069f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07bc15b170 tx=0x7f07ac00b4f0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 msgr2=0x7f07bc145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc145ee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f07b400c7a0 tx=0x7f07b400cc60 comp rx=0 tx=0).stop 2024-09-10T10:01:13.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 shutdown_connections 2024-09-10T10:01:13.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0798067a70 0x7f0798069f30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07bc15c560 0x7f07bc145ee0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07bc155970 0x7f07bc15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 --2- 172.21.15.110:0/3318480657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f07bc06f010 0x7f07bc15e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 >> 172.21.15.110:0/3318480657 conn(0x7f07bc06bc50 msgr2=0x7f07bc0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:13.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 shutdown_connections 2024-09-10T10:01:13.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.493+0000 7f07c2800640 1 -- 172.21.15.110:0/3318480657 wait complete. 2024-09-10T10:01:13.515 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:01:13.517 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:01:13.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.653+0000 7fb004e00640 1 Processor -- start 2024-09-10T10:01:13.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.653+0000 7fb004e00640 1 -- start start 2024-09-10T10:01:13.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.653+0000 7fb004e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb00015b330 0x7fb00006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.653+0000 7fb004e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0000ce4d0 0x7fb00006e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.653+0000 7fb004e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb0000b1c60 0x7fb00006e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.653+0000 7fb004e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb0000b5570 con 0x7fb0000b1c60 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.653+0000 7fb004e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb0000b4960 con 0x7fb00015b330 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.653+0000 7fb004e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0000b4c40 con 0x7fb0000ce4d0 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb0000b1c60 0x7fb00006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb0000b1c60 0x7fb00006e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39894/0 (socket says 172.21.15.110:39894) 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffd400640 1 --2- 172.21.15.110:0/4118033443 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0000ce4d0 0x7fb00006e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffde00640 1 --2- 172.21.15.110:0/4118033443 >> v2:172.21.15.145:3300/0 conn(0x7fb00015b330 0x7fb00006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 -- 172.21.15.110:0/4118033443 learned_addr learned my addr 172.21.15.110:0/4118033443 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:13.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 -- 172.21.15.110:0/4118033443 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0000ce4d0 msgr2=0x7fb00006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:13.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 --2- 172.21.15.110:0/4118033443 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0000ce4d0 0x7fb00006e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 -- 172.21.15.110:0/4118033443 >> v2:172.21.15.145:3300/0 conn(0x7fb00015b330 msgr2=0x7fb00006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:13.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 --2- 172.21.15.110:0/4118033443 >> v2:172.21.15.145:3300/0 conn(0x7fb00015b330 0x7fb00006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 -- 172.21.15.110:0/4118033443 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0000b4f20 con 0x7fb0000b1c60 2024-09-10T10:01:13.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffe800640 1 --2- 172.21.15.110:0/4118033443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb00006e680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb00006eca0 tx=0x7faff801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fab4cb04ff7f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:13.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffca00640 1 -- 172.21.15.110:0/4118033443 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80b4070 con 0x7fb0000b1c60 2024-09-10T10:01:13.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffca00640 1 -- 172.21.15.110:0/4118033443 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff80a7b90 con 0x7fb0000b1c60 2024-09-10T10:01:13.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.657+0000 7faffca00640 1 -- 172.21.15.110:0/4118033443 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80af440 con 0x7fb0000b1c60 2024-09-10T10:01:13.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- 172.21.15.110:0/4118033443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 msgr2=0x7fb00006e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 --2- 172.21.15.110:0/4118033443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb00006e680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb00006eca0 tx=0x7faff801ebf0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- 172.21.15.110:0/4118033443 shutdown_connections 2024-09-10T10:01:13.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 --2- 172.21.15.110:0/4118033443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb00006e680 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 --2- 172.21.15.110:0/4118033443 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0000ce4d0 0x7fb00006e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 --2- 172.21.15.110:0/4118033443 >> v2:172.21.15.145:3300/0 conn(0x7fb00015b330 0x7fb00006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- 172.21.15.110:0/4118033443 >> 172.21.15.110:0/4118033443 conn(0x7fb000062e40 msgr2=0x7fb00006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:13.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- 172.21.15.110:0/4118033443 shutdown_connections 2024-09-10T10:01:13.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- 172.21.15.110:0/4118033443 wait complete. 2024-09-10T10:01:13.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 Processor -- start 2024-09-10T10:01:13.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- start start 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb000141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb00015b330 0x7fb0000ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0000cee00 0x7fb000157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb00015fae0 con 0x7fb0000b1c60 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb00015f960 con 0x7fb0000cee00 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7fb004e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb00015fc60 con 0x7fb00015b330 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb000141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb000141740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39900/0 (socket says 172.21.15.110:39900) 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffde00640 1 -- 172.21.15.110:0/1580324053 learned_addr learned my addr 172.21.15.110:0/1580324053 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffd400640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb00015b330 0x7fb0000ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffe800640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0000cee00 0x7fb000157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffde00640 1 -- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb00015b330 msgr2=0x7fb0000ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffde00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb00015b330 0x7fb0000ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffde00640 1 -- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0000cee00 msgr2=0x7fb000157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffde00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0000cee00 0x7fb000157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffde00640 1 -- 172.21.15.110:0/1580324053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0001594c0 con 0x7fb0000b1c60 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffd400640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb00015b330 0x7fb0000ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.661+0000 7faffe800640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0000cee00 0x7fb000157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faffde00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb000141740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faff000efc0 tx=0x7faff000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faff6000640 1 -- 172.21.15.110:0/1580324053 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0018020 con 0x7fb0000b1c60 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faff6000640 1 -- 172.21.15.110:0/1580324053 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff0014480 con 0x7fb0000b1c60 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faff6000640 1 -- 172.21.15.110:0/1580324053 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0014850 con 0x7fb0000b1c60 2024-09-10T10:01:13.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb000158000 con 0x7fb0000b1c60 2024-09-10T10:01:13.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb000159a40 con 0x7fb0000b1c60 2024-09-10T10:01:13.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0000cf210 con 0x7fb0000b1c60 2024-09-10T10:01:13.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faff6000640 1 -- 172.21.15.110:0/1580324053 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7faff001b020 con 0x7fb0000b1c60 2024-09-10T10:01:13.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faff6000640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fafe0067aa0 0x7fafe0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faff6000640 1 -- 172.21.15.110:0/1580324053 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7faff0086780 con 0x7fb0000b1c60 2024-09-10T10:01:13.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faffd400640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fafe0067aa0 0x7fafe0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.665+0000 7faffd400640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fafe0067aa0 0x7fafe0069f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0001fa790 tx=0x7fafec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:13.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.669+0000 7faff6000640 1 -- 172.21.15.110:0/1580324053 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb0000cf210 con 0x7fb0000b1c60 2024-09-10T10:01:13.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.825+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb00006c6a0 con 0x7fb0000b1c60 2024-09-10T10:01:13.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.825+0000 7faff6000640 1 -- 172.21.15.110:0/1580324053 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7faff0022070 con 0x7fb0000b1c60 2024-09-10T10:01:13.829 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:01:13.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fafe0067aa0 msgr2=0x7fafe0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fafe0067aa0 0x7fafe0069f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0001fa790 tx=0x7fafec008040 comp rx=0 tx=0).stop 2024-09-10T10:01:13.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 msgr2=0x7fb000141740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb000141740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faff000efc0 tx=0x7faff000c490 comp rx=0 tx=0).stop 2024-09-10T10:01:13.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 shutdown_connections 2024-09-10T10:01:13.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fafe0067aa0 0x7fafe0069f60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0000cee00 0x7fb000157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb00015b330 0x7fb0000ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 --2- 172.21.15.110:0/1580324053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0000b1c60 0x7fb000141740 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 >> 172.21.15.110:0/1580324053 conn(0x7fb000062e40 msgr2=0x7fb0000ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 shutdown_connections 2024-09-10T10:01:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.829+0000 7fb004e00640 1 -- 172.21.15.110:0/1580324053 wait complete. 2024-09-10T10:01:13.851 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-10T10:01:13.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cf5400640 1 Processor -- start 2024-09-10T10:01:13.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cf5400640 1 -- start start 2024-09-10T10:01:13.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cf5400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4cf015c6f0 0x7f4cf00d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cf5400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4cf00b15a0 0x7f4cf00d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cf5400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf00d2300 con 0x7f4cf015fff0 2024-09-10T10:01:13.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cf5400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf0160ee0 con 0x7f4cf00b15a0 2024-09-10T10:01:13.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cf5400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf00d2020 con 0x7f4cf015c6f0 2024-09-10T10:01:13.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cef200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4cf00b15a0 0x7f4cf00d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cef200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4cf00b15a0 0x7f4cf00d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39904/0 (socket says 172.21.15.110:39904) 2024-09-10T10:01:13.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cef200640 1 -- 172.21.15.110:0/1703229929 learned_addr learned my addr 172.21.15.110:0/1703229929 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:13.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cee800640 1 --1- 172.21.15.110:0/1703229929 >> v1:172.21.15.110:6789/0 conn(0x7f4cf015fff0 0x7f4cf01603d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49092/0 (socket says 172.21.15.110:49092) 2024-09-10T10:01:13.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cede00640 1 --2- 172.21.15.110:0/1703229929 >> v2:172.21.15.145:3300/0 conn(0x7f4cf015c6f0 0x7f4cf00d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cef200640 1 -- 172.21.15.110:0/1703229929 >> v2:172.21.15.145:3300/0 conn(0x7f4cf015c6f0 msgr2=0x7f4cf00d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:13.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cef200640 1 --2- 172.21.15.110:0/1703229929 >> v2:172.21.15.145:3300/0 conn(0x7f4cf015c6f0 0x7f4cf00d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cef200640 1 -- 172.21.15.110:0/1703229929 >> v1:172.21.15.110:6789/0 conn(0x7f4cf015fff0 legacy=0x7f4cf01603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.985+0000 7f4cef200640 1 -- 172.21.15.110:0/1703229929 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf00716a0 con 0x7f4cf00b15a0 2024-09-10T10:01:13.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cef200640 1 --2- 172.21.15.110:0/1703229929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf00b15a0 0x7f4cf00d0ab0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4cf015ff50 tx=0x7f4ce801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9e07155750cd05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:13.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4ced400640 1 -- 172.21.15.110:0/1703229929 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80aa670 con 0x7f4cf00b15a0 2024-09-10T10:01:13.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4ced400640 1 -- 172.21.15.110:0/1703229929 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce80af070 con 0x7f4cf00b15a0 2024-09-10T10:01:13.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4ced400640 1 -- 172.21.15.110:0/1703229929 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80aaa30 con 0x7f4cf00b15a0 2024-09-10T10:01:13.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cf5400640 1 -- 172.21.15.110:0/1703229929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf00b15a0 msgr2=0x7f4cf00d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cf5400640 1 --2- 172.21.15.110:0/1703229929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf00b15a0 0x7f4cf00d0ab0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4cf015ff50 tx=0x7f4ce801c4f0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cf5400640 1 -- 172.21.15.110:0/1703229929 shutdown_connections 2024-09-10T10:01:13.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cf5400640 1 --2- 172.21.15.110:0/1703229929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf00b15a0 0x7f4cf00d0ab0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cf5400640 1 --2- 172.21.15.110:0/1703229929 >> v2:172.21.15.145:3300/0 conn(0x7f4cf015c6f0 0x7f4cf00d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cf5400640 1 -- 172.21.15.110:0/1703229929 >> 172.21.15.110:0/1703229929 conn(0x7f4cf00b4370 msgr2=0x7f4cf00b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:13.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cf5400640 1 -- 172.21.15.110:0/1703229929 shutdown_connections 2024-09-10T10:01:13.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.989+0000 7f4cf5400640 1 -- 172.21.15.110:0/1703229929 wait complete. 2024-09-10T10:01:13.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 Processor -- start 2024-09-10T10:01:13.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 -- start start 2024-09-10T10:01:13.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cf00b15a0 0x7f4cf01f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cf015c6f0 0x7f4cf01516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf015fff0 0x7f4cf015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01694f0 con 0x7f4cf015fff0 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0169370 con 0x7f4cf015c6f0 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf0169670 con 0x7f4cf00b15a0 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf015fff0 0x7f4cf015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf015fff0 0x7f4cf015cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39916/0 (socket says 172.21.15.110:39916) 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 -- 172.21.15.110:0/3737988723 learned_addr learned my addr 172.21.15.110:0/3737988723 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cee800640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cf00b15a0 0x7f4cf01f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cede00640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cf015c6f0 0x7f4cf01516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 -- 172.21.15.110:0/3737988723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cf00b15a0 msgr2=0x7f4cf01f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cf00b15a0 0x7f4cf01f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 -- 172.21.15.110:0/3737988723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cf015c6f0 msgr2=0x7f4cf01516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:13.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cf015c6f0 0x7f4cf01516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:13.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 -- 172.21.15.110:0/3737988723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf0151e20 con 0x7f4cf015fff0 2024-09-10T10:01:13.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cef200640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf015fff0 0x7f4cf015cad0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80ac040 tx=0x7f4ce801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:13.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4ce6a00640 1 -- 172.21.15.110:0/3737988723 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80bd5f0 con 0x7f4cf015fff0 2024-09-10T10:01:13.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4ce6a00640 1 -- 172.21.15.110:0/3737988723 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf00d1800 con 0x7f4cf015fff0 2024-09-10T10:01:13.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf0152100 con 0x7f4cf015fff0 2024-09-10T10:01:13.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4ce6a00640 1 -- 172.21.15.110:0/3737988723 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80bd9f0 con 0x7f4cf015fff0 2024-09-10T10:01:13.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf01526c0 con 0x7f4cf015fff0 2024-09-10T10:01:13.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.993+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf01593f0 con 0x7f4cf015fff0 2024-09-10T10:01:13.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.997+0000 7f4ce6a00640 1 -- 172.21.15.110:0/3737988723 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f4ce80b33f0 con 0x7f4cf015fff0 2024-09-10T10:01:13.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.997+0000 7f4ce6a00640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4cd0067aa0 0x7f4cd0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:14.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.997+0000 7f4ce6a00640 1 -- 172.21.15.110:0/3737988723 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f4ce81202d0 con 0x7f4cf015fff0 2024-09-10T10:01:14.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.997+0000 7f4cee800640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4cd0067aa0 0x7f4cd0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:14.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:13.997+0000 7f4cee800640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4cd0067aa0 0x7f4cd0069f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00012f0 tx=0x7f4ce0001640 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:14.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:14.001+0000 7f4ce6a00640 1 -- 172.21.15.110:0/3737988723 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4ce80c3060 con 0x7f4cf015fff0 2024-09-10T10:01:14.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:14.217+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f4cf01603d0 con 0x7f4cf015fff0 2024-09-10T10:01:15.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.157+0000 7f4ce6a00640 1 -- 172.21.15.110:0/3737988723 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v156) ==== 164+0+0 (secure 0 0 0) 0x7f4ce8016110 con 0x7f4cf015fff0 2024-09-10T10:01:15.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4cd0067aa0 msgr2=0x7f4cd0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4cd0067aa0 0x7f4cd0069f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00012f0 tx=0x7f4ce0001640 comp rx=0 tx=0).stop 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf015fff0 msgr2=0x7f4cf015cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf015fff0 0x7f4cf015cad0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80ac040 tx=0x7f4ce801ad30 comp rx=0 tx=0).stop 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 shutdown_connections 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4cd0067aa0 0x7f4cd0069f60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cf015fff0 0x7f4cf015cad0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cf015c6f0 0x7f4cf01516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 --2- 172.21.15.110:0/3737988723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cf00b15a0 0x7f4cf01f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 >> 172.21.15.110:0/3737988723 conn(0x7f4cf00b4370 msgr2=0x7f4cf0150460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 shutdown_connections 2024-09-10T10:01:15.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.161+0000 7f4cf5400640 1 -- 172.21.15.110:0/3737988723 wait complete. 2024-09-10T10:01:15.189 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:01:15.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d8400640 1 Processor -- start 2024-09-10T10:01:15.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d8400640 1 -- start start 2024-09-10T10:01:15.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d8400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa3d015b330 0x7fa3d006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d8400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d00b4960 con 0x7fa3d00ce4d0 2024-09-10T10:01:15.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d8400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d00b4c40 con 0x7fa3d006e850 2024-09-10T10:01:15.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d8400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d00b5570 con 0x7fa3d015b330 2024-09-10T10:01:15.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d5a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa3d015b330 0x7fa3d006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d5000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa3d00ce4d0 0x7fa3d006e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49106/0 (socket says 172.21.15.110:49106) 2024-09-10T10:01:15.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d5a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa3d015b330 0x7fa3d006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39930/0 (socket says 172.21.15.110:39930) 2024-09-10T10:01:15.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d5000640 1 -- 172.21.15.110:0/3354430892 learned_addr learned my addr 172.21.15.110:0/3354430892 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:15.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.329+0000 7fa3d5a00640 1 -- 172.21.15.110:0/3354430892 >> v1:172.21.15.145:6789/0 conn(0x7fa3d006e850 legacy=0x7fa3d00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d5a00640 1 -- 172.21.15.110:0/3354430892 >> v1:172.21.15.110:6789/0 conn(0x7fa3d00ce4d0 legacy=0x7fa3d006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d5a00640 1 -- 172.21.15.110:0/3354430892 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d00b4f20 con 0x7fa3d015b330 2024-09-10T10:01:15.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3cfe00640 1 -- 172.21.15.110:0/3354430892 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508201803 0 0) 0x7fa3d00b4960 con 0x7fa3d00ce4d0 2024-09-10T10:01:15.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d5a00640 1 --2- 172.21.15.110:0/3354430892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d006c6a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009d90 tx=0x7fa3c401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965678cf34d3803c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3cfe00640 1 -- 172.21.15.110:0/3354430892 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40b2070 con 0x7fa3d015b330 2024-09-10T10:01:15.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3cfe00640 1 -- 172.21.15.110:0/3354430892 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c40a5900 con 0x7fa3d015b330 2024-09-10T10:01:15.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3cfe00640 1 -- 172.21.15.110:0/3354430892 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40a5cd0 con 0x7fa3d015b330 2024-09-10T10:01:15.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d8400640 1 -- 172.21.15.110:0/3354430892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 msgr2=0x7fa3d006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d8400640 1 --2- 172.21.15.110:0/3354430892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d006c6a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009d90 tx=0x7fa3c401c930 comp rx=0 tx=0).stop 2024-09-10T10:01:15.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d8400640 1 -- 172.21.15.110:0/3354430892 shutdown_connections 2024-09-10T10:01:15.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d8400640 1 --2- 172.21.15.110:0/3354430892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d006c6a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d8400640 1 -- 172.21.15.110:0/3354430892 >> 172.21.15.110:0/3354430892 conn(0x7fa3d0062e40 msgr2=0x7fa3d006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d8400640 1 -- 172.21.15.110:0/3354430892 shutdown_connections 2024-09-10T10:01:15.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.333+0000 7fa3d8400640 1 -- 172.21.15.110:0/3354430892 wait complete. 2024-09-10T10:01:15.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 Processor -- start 2024-09-10T10:01:15.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 -- start start 2024-09-10T10:01:15.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3d006e850 0x7fa3d014ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d0151da0 0x7fa3d01553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0169450 con 0x7fa3d015b330 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01692d0 con 0x7fa3d0151da0 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d01695d0 con 0x7fa3d006e850 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d014f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39940/0 (socket says 172.21.15.110:39940) 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 -- 172.21.15.110:0/3573019244 learned_addr learned my addr 172.21.15.110:0/3573019244 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d6400640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d0151da0 0x7fa3d01553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5a00640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3d006e850 0x7fa3d014ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 -- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3d006e850 msgr2=0x7fa3d014ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3d006e850 0x7fa3d014ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 -- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d0151da0 msgr2=0x7fa3d01553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d0151da0 0x7fa3d01553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 -- 172.21.15.110:0/3573019244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d014efb0 con 0x7fa3d015b330 2024-09-10T10:01:15.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5a00640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3d006e850 0x7fa3d014ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:15.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d6400640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d0151da0 0x7fa3d01553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:15.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d5000640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d014f200 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c520 tx=0x7fa3bc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3cd600640 1 -- 172.21.15.110:0/3573019244 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc018070 con 0x7fa3d015b330 2024-09-10T10:01:15.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3cd600640 1 -- 172.21.15.110:0/3573019244 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc0074e0 con 0x7fa3d015b330 2024-09-10T10:01:15.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3cd600640 1 -- 172.21.15.110:0/3573019244 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0049f0 con 0x7fa3d015b330 2024-09-10T10:01:15.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d0152240 con 0x7fa3d015b330 2024-09-10T10:01:15.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.337+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d0152800 con 0x7fa3d015b330 2024-09-10T10:01:15.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.341+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d006c6a0 con 0x7fa3d015b330 2024-09-10T10:01:15.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.341+0000 7fa3cd600640 1 -- 172.21.15.110:0/3573019244 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fa3bc01b070 con 0x7fa3d015b330 2024-09-10T10:01:15.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.341+0000 7fa3cd600640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3b0067a70 0x7fa3b0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.341+0000 7fa3cd600640 1 -- 172.21.15.110:0/3573019244 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fa3bc085220 con 0x7fa3d015b330 2024-09-10T10:01:15.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.341+0000 7fa3d5a00640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3b0067a70 0x7fa3b0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.341+0000 7fa3d5a00640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3b0067a70 0x7fa3b0069f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40af000 tx=0x7fa3c40a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.345+0000 7fa3cd600640 1 -- 172.21.15.110:0/3573019244 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa3bc055cb0 con 0x7fa3d015b330 2024-09-10T10:01:15.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.505+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa3d006e210 con 0x7fa3d015b330 2024-09-10T10:01:15.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.505+0000 7fa3cd600640 1 -- 172.21.15.110:0/3573019244 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fa3bc0555b0 con 0x7fa3d015b330 2024-09-10T10:01:15.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3b0067a70 msgr2=0x7fa3b0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3b0067a70 0x7fa3b0069f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40af000 tx=0x7fa3c40a7070 comp rx=0 tx=0).stop 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 msgr2=0x7fa3d014f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d014f200 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c520 tx=0x7fa3bc00c9e0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 shutdown_connections 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa3b0067a70 0x7fa3b0069f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d0151da0 0x7fa3d01553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3d015b330 0x7fa3d014f200 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 --2- 172.21.15.110:0/3573019244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3d006e850 0x7fa3d014ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 >> 172.21.15.110:0/3573019244 conn(0x7fa3d0062e40 msgr2=0x7fa3d00b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 shutdown_connections 2024-09-10T10:01:15.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.509+0000 7fa3d8400640 1 -- 172.21.15.110:0/3573019244 wait complete. 2024-09-10T10:01:15.530 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:01:15.530 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:01:15.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:01:15.689 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:01:15.689 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:15.689 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:15.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:01:15.698 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:15.698 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:15.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:01:15.707 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:15.707 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:15.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:01:15.749 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:15.749 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:15.758 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:01:15.759 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:01:15.759 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:15.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd846e00640 1 Processor -- start 2024-09-10T10:01:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd846e00640 1 -- start start 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd846e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd84015c560 0x7fd84006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd846e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd84006f430 con 0x7fd84006f010 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd846e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd840155d50 con 0x7fd840155970 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd846e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd8400b4570 con 0x7fd84015c560 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd844e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fd84006f010 0x7fd8400ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49110/0 (socket says 172.21.15.110:49110) 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd844e00640 1 -- 172.21.15.110:0/465049262 learned_addr learned my addr 172.21.15.110:0/465049262 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83f400640 1 --2- 172.21.15.110:0/465049262 >> v2:172.21.15.145:3300/0 conn(0x7fd84015c560 0x7fd84006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2651178439 0 0) 0x7fd84006f430 con 0x7fd84006f010 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd824005740 con 0x7fd84006f010 2024-09-10T10:01:15.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 520357414 0 0) 0x7fd840155d50 con 0x7fd840155970 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd84006f430 con 0x7fd840155970 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 337997513 0 0) 0x7fd824005740 con 0x7fd84006f010 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 >> v2:172.21.15.145:3300/0 conn(0x7fd84015c560 msgr2=0x7fd84006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 --2- 172.21.15.110:0/465049262 >> v2:172.21.15.145:3300/0 conn(0x7fd84015c560 0x7fd84006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 >> v1:172.21.15.145:6789/0 conn(0x7fd840155970 legacy=0x7fd8400cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8400b34d0 con 0x7fd84006f010 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fd838001a00 con 0x7fd84006f010 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 >> v1:172.21.15.110:6789/0 conn(0x7fd84006f010 legacy=0x7fd8400ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd824005db0 0x7fd8240061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd824006a20 0x7fd824006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd82400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd82400e2b0 con 0x7fd8240073e0 2024-09-10T10:01:15.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd82400e430 con 0x7fd824005db0 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd82400e630 con 0x7fd824006a20 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd844e00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd82400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83fe00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd824005db0 0x7fd8240061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.925+0000 7fd83f400640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd824006a20 0x7fd824006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd844e00640 1 -- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd824006a20 msgr2=0x7fd824006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd844e00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd824006a20 0x7fd824006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd844e00640 1 -- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd824005db0 msgr2=0x7fd8240061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd844e00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd824005db0 0x7fd8240061b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd844e00640 1 -- 172.21.15.110:0/465049262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd824005740 con 0x7fd8240073e0 2024-09-10T10:01:15.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd83f400640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd824006a20 0x7fd824006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd844e00640 1 -- 172.21.15.110:0/465049262 reap_dead start 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd844e00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd82400dbe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd83800a730 tx=0x7fd83801f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d700b67d860199 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83800a350 con 0x7fd8240073e0 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8380ac450 con 0x7fd8240073e0 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd83ea00640 1 -- 172.21.15.110:0/465049262 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8380ac860 con 0x7fd8240073e0 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd846e00640 1 -- 172.21.15.110:0/465049262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 msgr2=0x7fd82400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd846e00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd82400dbe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd83800a730 tx=0x7fd83801f9a0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd846e00640 1 -- 172.21.15.110:0/465049262 shutdown_connections 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd846e00640 1 --2- 172.21.15.110:0/465049262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd82400dbe0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd846e00640 1 -- 172.21.15.110:0/465049262 >> 172.21.15.110:0/465049262 conn(0x7fd84006bc50 msgr2=0x7fd84006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd846e00640 1 -- 172.21.15.110:0/465049262 shutdown_connections 2024-09-10T10:01:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.929+0000 7fd846e00640 1 -- 172.21.15.110:0/465049262 wait complete. 2024-09-10T10:01:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 Processor -- start 2024-09-10T10:01:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 -- start start 2024-09-10T10:01:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd840149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd840156d60 0x7fd8400ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8401571e0 0x7fd8400aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8400b3d70 con 0x7fd8240073e0 2024-09-10T10:01:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd8400b3a90 con 0x7fd840156d60 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd8400b4050 con 0x7fd8401571e0 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd840149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd840149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39960/0 (socket says 172.21.15.110:39960) 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 -- 172.21.15.110:0/4163673969 learned_addr learned my addr 172.21.15.110:0/4163673969 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83f400640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd840156d60 0x7fd8400ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 -- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8401571e0 msgr2=0x7fd8400aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd844e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8401571e0 0x7fd8400aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8401571e0 0x7fd8400aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 -- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd840156d60 msgr2=0x7fd8400ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd840156d60 0x7fd8400ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 -- 172.21.15.110:0/4163673969 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd840151720 con 0x7fd8240073e0 2024-09-10T10:01:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83f400640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd840156d60 0x7fd8400ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:15.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd844e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8401571e0 0x7fd8400aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:15.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd83fe00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd840149e00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd83000f7d0 tx=0x7fd83000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd837e00640 1 -- 172.21.15.110:0/4163673969 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83001a070 con 0x7fd8240073e0 2024-09-10T10:01:15.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd837e00640 1 -- 172.21.15.110:0/4163673969 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd830005320 con 0x7fd8240073e0 2024-09-10T10:01:15.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd837e00640 1 -- 172.21.15.110:0/4163673969 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830006a20 con 0x7fd8240073e0 2024-09-10T10:01:15.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd840155fe0 con 0x7fd8240073e0 2024-09-10T10:01:15.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.933+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd840156280 con 0x7fd8240073e0 2024-09-10T10:01:15.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.937+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8400cc950 con 0x7fd8240073e0 2024-09-10T10:01:15.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.937+0000 7fd837e00640 1 -- 172.21.15.110:0/4163673969 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fd830006bc0 con 0x7fd8240073e0 2024-09-10T10:01:15.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.937+0000 7fd837e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd820067aa0 0x7fd820069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.937+0000 7fd837e00640 1 -- 172.21.15.110:0/4163673969 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fd830085480 con 0x7fd8240073e0 2024-09-10T10:01:15.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.937+0000 7fd83f400640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd820067aa0 0x7fd820069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.937+0000 7fd83f400640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd820067aa0 0x7fd820069f60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd82c006f90 tx=0x7fd82c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:15.941+0000 7fd837e00640 1 -- 172.21.15.110:0/4163673969 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd830056030 con 0x7fd8240073e0 2024-09-10T10:01:16.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.129+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd840072d80 con 0x7fd8240073e0 2024-09-10T10:01:16.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.129+0000 7fd837e00640 1 -- 172.21.15.110:0/4163673969 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3920 (secure 0 0 0) 0x7fd830055930 con 0x7fd8240073e0 2024-09-10T10:01:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:16.134 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":29,"btime":"2024-09-10T10:01:10:010048+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3066005763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3066005763},{"type":"v1","addr":"172.21.15.145:6834","nonce":3066005763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4143427839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4143427839},{"type":"v1","addr":"172.21.15.145:6835","nonce":4143427839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/956496493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":956496493},{"type":"v1","addr":"172.21.15.110:6836","nonce":956496493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5550,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1219113126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":1219113126},{"type":"v1","addr":"172.21.15.110:6837","nonce":1219113126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-10T10:01:16.134 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 29 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd820067aa0 msgr2=0x7fd820069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd820067aa0 0x7fd820069f60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd82c006f90 tx=0x7fd82c008040 comp rx=0 tx=0).stop 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 msgr2=0x7fd840149e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd840149e00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd83000f7d0 tx=0x7fd83000fc90 comp rx=0 tx=0).stop 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 shutdown_connections 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd820067aa0 0x7fd820069f60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8401571e0 0x7fd8400aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd840156d60 0x7fd8400ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 --2- 172.21.15.110:0/4163673969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8240073e0 0x7fd840149e00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 >> 172.21.15.110:0/4163673969 conn(0x7fd84006bc50 msgr2=0x7fd8400b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:16.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 shutdown_connections 2024-09-10T10:01:16.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.133+0000 7fd846e00640 1 -- 172.21.15.110:0/4163673969 wait complete. 2024-09-10T10:01:16.155 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:01:16.155 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:01:16.155 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:01:16.155 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:01:16.156 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:01:16.156 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:01:16.156 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:01:16.156 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:01:16.156 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:01:16.156 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:01:16.157 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:01:16.157 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:01:16.157 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:01:16.157 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:01:16.157 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:01:16.157 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:01:16.168 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:01:16.168 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:01:16.169 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:01:16.169 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:01:16.172 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:01:16.172 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:01:16.174 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:01:16.174 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:01:16.212 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:01:16.212 DEBUG:teuthology.parallel:result is None 2024-09-10T10:01:16.213 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:01:16.213 DEBUG:teuthology.parallel:result is None 2024-09-10T10:01:16.220 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:01:16.220 DEBUG:teuthology.parallel:result is None 2024-09-10T10:01:16.222 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:01:16.222 DEBUG:teuthology.parallel:result is None 2024-09-10T10:01:16.223 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:01:16.298 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:01:16.299 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:01:16.308 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:01:16.309 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:01:16.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b9800640 1 Processor -- start 2024-09-10T10:01:16.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b9800640 1 -- start start 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b9800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f80b40cfb90 0x7f80b415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b9800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40b3f40 0x7f80b40cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b9800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b415d4d0 con 0x7f80b40cff70 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b9800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f80b406d130 con 0x7f80b40cfb90 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b9800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f80b415d7b0 con 0x7f80b40b3f40 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b2a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f80b40cfb90 0x7f80b415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b2a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f80b40cfb90 0x7f80b415ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39984/0 (socket says 172.21.15.110:39984) 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b2a00640 1 -- 172.21.15.110:0/1384298585 learned_addr learned my addr 172.21.15.110:0/1384298585 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.365+0000 7f80b2000640 1 --2- 172.21.15.110:0/1384298585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40b3f40 0x7f80b40cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:16.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b2a00640 1 -- 172.21.15.110:0/1384298585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40b3f40 msgr2=0x7f80b40cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:16.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b2a00640 1 --2- 172.21.15.110:0/1384298585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40b3f40 0x7f80b40cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b2a00640 1 -- 172.21.15.110:0/1384298585 >> v1:172.21.15.145:6789/0 conn(0x7f80b40cff70 legacy=0x7f80b4160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:16.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b2a00640 1 -- 172.21.15.110:0/1384298585 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b406d380 con 0x7f80b40cfb90 2024-09-10T10:01:16.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b2a00640 1 --2- 172.21.15.110:0/1384298585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40cfb90 0x7f80b415ca10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009e50 tx=0x7f80a401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f274281186185464 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:16.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b1600640 1 -- 172.21.15.110:0/1384298585 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40a5660 con 0x7f80b40cfb90 2024-09-10T10:01:16.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b1600640 1 -- 172.21.15.110:0/1384298585 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a40a5800 con 0x7f80b40cfb90 2024-09-10T10:01:16.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b1600640 1 -- 172.21.15.110:0/1384298585 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40a5bd0 con 0x7f80b40cfb90 2024-09-10T10:01:16.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b9800640 1 -- 172.21.15.110:0/1384298585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40cfb90 msgr2=0x7f80b415ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:16.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.369+0000 7f80b9800640 1 --2- 172.21.15.110:0/1384298585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40cfb90 0x7f80b415ca10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009e50 tx=0x7f80a401c9a0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- 172.21.15.110:0/1384298585 shutdown_connections 2024-09-10T10:01:16.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 --2- 172.21.15.110:0/1384298585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40b3f40 0x7f80b40cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 --2- 172.21.15.110:0/1384298585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40cfb90 0x7f80b415ca10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- 172.21.15.110:0/1384298585 >> 172.21.15.110:0/1384298585 conn(0x7f80b4062d10 msgr2=0x7f80b406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:16.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- 172.21.15.110:0/1384298585 shutdown_connections 2024-09-10T10:01:16.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- 172.21.15.110:0/1384298585 wait complete. 2024-09-10T10:01:16.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 Processor -- start 2024-09-10T10:01:16.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- start start 2024-09-10T10:01:16.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40b3f40 0x7f80b4161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:16.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40cfb90 0x7f80b4150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b40cff70 0x7f80b4163410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4169500 con 0x7f80b40b3f40 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4169380 con 0x7f80b40cff70 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f80b4169680 con 0x7f80b40cfb90 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40b3f40 0x7f80b4161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40b3f40 0x7f80b4161b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39990/0 (socket says 172.21.15.110:39990) 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 -- 172.21.15.110:0/94094877 learned_addr learned my addr 172.21.15.110:0/94094877 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2000640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40cfb90 0x7f80b4150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b3400640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b40cff70 0x7f80b4163410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 -- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40cfb90 msgr2=0x7f80b4150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40cfb90 0x7f80b4150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 -- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b40cff70 msgr2=0x7f80b4163410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:16.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b40cff70 0x7f80b4163410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 -- 172.21.15.110:0/94094877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b4071620 con 0x7f80b40b3f40 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2000640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40cfb90 0x7f80b4150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b3400640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b40cff70 0x7f80b4163410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b2a00640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40b3f40 0x7f80b4161b20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f80a401a5d0 tx=0x7f80a4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80aaa00640 1 -- 172.21.15.110:0/94094877 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40bc030 con 0x7f80b40b3f40 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80aaa00640 1 -- 172.21.15.110:0/94094877 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a40b8680 con 0x7f80b40b3f40 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b4071900 con 0x7f80b40b3f40 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.373+0000 7f80aaa00640 1 -- 172.21.15.110:0/94094877 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4009510 con 0x7f80b40b3f40 2024-09-10T10:01:16.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.377+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b4071ec0 con 0x7f80b40b3f40 2024-09-10T10:01:16.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.377+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b40cd5a0 con 0x7f80b40b3f40 2024-09-10T10:01:16.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.377+0000 7f80aaa00640 1 -- 172.21.15.110:0/94094877 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f80a40bf070 con 0x7f80b40b3f40 2024-09-10T10:01:16.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.377+0000 7f80aaa00640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8094067a70 0x7f8094069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:16.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.377+0000 7f80aaa00640 1 -- 172.21.15.110:0/94094877 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f80a4120ee0 con 0x7f80b40b3f40 2024-09-10T10:01:16.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.377+0000 7f80b2000640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8094067a70 0x7f8094069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:16.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.377+0000 7f80b2000640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8094067a70 0x7f8094069f30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f80b40b08a0 tx=0x7f80a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:16.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.381+0000 7f80aaa00640 1 -- 172.21.15.110:0/94094877 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f80b40cdef0 con 0x7f80b40b3f40 2024-09-10T10:01:16.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:16.549+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f80b4071360 con 0x7f80b40b3f40 2024-09-10T10:01:17.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.181+0000 7f80aaa00640 1 -- 172.21.15.110:0/94094877 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) ==== 97+0+0 (secure 0 0 0) 0x7f80a4016620 con 0x7f80b40b3f40 2024-09-10T10:01:17.182 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.181+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8094067a70 msgr2=0x7f8094069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.181+0000 7f80b9800640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8094067a70 0x7f8094069f30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f80b40b08a0 tx=0x7f80a0008040 comp rx=0 tx=0).stop 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.181+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40b3f40 msgr2=0x7f80b4161b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.181+0000 7f80b9800640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40b3f40 0x7f80b4161b20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f80a401a5d0 tx=0x7f80a4019740 comp rx=0 tx=0).stop 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.185+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 shutdown_connections 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.185+0000 7f80b9800640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8094067a70 0x7f8094069f30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.185+0000 7f80b9800640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b40cff70 0x7f80b4163410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.185+0000 7f80b9800640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f80b40cfb90 0x7f80b4150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.185+0000 7f80b9800640 1 --2- 172.21.15.110:0/94094877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b40b3f40 0x7f80b4161b20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.185+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 >> 172.21.15.110:0/94094877 conn(0x7f80b4062d10 msgr2=0x7f80b40b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:17.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.185+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 shutdown_connections 2024-09-10T10:01:17.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.185+0000 7f80b9800640 1 -- 172.21.15.110:0/94094877 wait complete. 2024-09-10T10:01:17.210 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:01:17.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9b0200640 1 Processor -- start 2024-09-10T10:01:17.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9b0200640 1 -- start start 2024-09-10T10:01:17.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9b0200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb9a8155970 0x7fb9a80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9b0200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a8155d50 con 0x7fb9a815c560 2024-09-10T10:01:17.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9b0200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a80b34d0 con 0x7fb9a806f1e0 2024-09-10T10:01:17.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9b0200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a80b4570 con 0x7fb9a8155970 2024-09-10T10:01:17.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9ace00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb9a815c560 0x7fb9a806ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49132/0 (socket says 172.21.15.110:49132) 2024-09-10T10:01:17.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9ace00640 1 -- 172.21.15.110:0/395888557 learned_addr learned my addr 172.21.15.110:0/395888557 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:17.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.345+0000 7fb9ad800640 1 --2- 172.21.15.110:0/395888557 >> v2:172.21.15.145:3300/0 conn(0x7fb9a8155970 0x7fb9a80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9a7e00640 1 -- 172.21.15.110:0/395888557 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207965972 0 0) 0x7fb9a8155d50 con 0x7fb9a815c560 2024-09-10T10:01:17.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9a7e00640 1 -- 172.21.15.110:0/395888557 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb990003530 con 0x7fb9a815c560 2024-09-10T10:01:17.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9a7e00640 1 -- 172.21.15.110:0/395888557 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384382582 0 0) 0x7fb9a80b34d0 con 0x7fb9a806f1e0 2024-09-10T10:01:17.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9a7e00640 1 -- 172.21.15.110:0/395888557 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9a8155d50 con 0x7fb9a806f1e0 2024-09-10T10:01:17.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9ad800640 1 -- 172.21.15.110:0/395888557 >> v1:172.21.15.145:6789/0 conn(0x7fb9a806f1e0 legacy=0x7fb9a80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9ad800640 1 -- 172.21.15.110:0/395888557 >> v1:172.21.15.110:6789/0 conn(0x7fb9a815c560 legacy=0x7fb9a806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9ad800640 1 -- 172.21.15.110:0/395888557 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80b37b0 con 0x7fb9a8155970 2024-09-10T10:01:17.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9ad800640 1 --2- 172.21.15.110:0/395888557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 0x7fb9a80cc8c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0097c0 tx=0x7fb99c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d06993fc458c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:17.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9a7e00640 1 -- 172.21.15.110:0/395888557 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0a7070 con 0x7fb9a8155970 2024-09-10T10:01:17.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9a7e00640 1 -- 172.21.15.110:0/395888557 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c0b0070 con 0x7fb9a8155970 2024-09-10T10:01:17.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.349+0000 7fb9a7e00640 1 -- 172.21.15.110:0/395888557 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0ab890 con 0x7fb9a8155970 2024-09-10T10:01:17.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- 172.21.15.110:0/395888557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 msgr2=0x7fb9a80cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 --2- 172.21.15.110:0/395888557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 0x7fb9a80cc8c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0097c0 tx=0x7fb99c01c470 comp rx=0 tx=0).stop 2024-09-10T10:01:17.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- 172.21.15.110:0/395888557 shutdown_connections 2024-09-10T10:01:17.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 --2- 172.21.15.110:0/395888557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 0x7fb9a80cc8c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- 172.21.15.110:0/395888557 >> 172.21.15.110:0/395888557 conn(0x7fb9a806bc50 msgr2=0x7fb9a806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:17.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- 172.21.15.110:0/395888557 shutdown_connections 2024-09-10T10:01:17.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- 172.21.15.110:0/395888557 wait complete. 2024-09-10T10:01:17.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 Processor -- start 2024-09-10T10:01:17.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- start start 2024-09-10T10:01:17.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9a806f1e0 0x7fb9a813c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 0x7fb9a8151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9a815c560 0x7fb9a8155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80cf340 con 0x7fb9a806f1e0 2024-09-10T10:01:17.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80cce00 con 0x7fb9a8155970 2024-09-10T10:01:17.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9b0200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a80cf4c0 con 0x7fb9a815c560 2024-09-10T10:01:17.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ad800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9a806f1e0 0x7fb9a813c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ad800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9a806f1e0 0x7fb9a813c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40006/0 (socket says 172.21.15.110:40006) 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ad800640 1 -- 172.21.15.110:0/330929668 learned_addr learned my addr 172.21.15.110:0/330929668 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ace00640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 0x7fb9a8151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ae200640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9a815c560 0x7fb9a8155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ad800640 1 -- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9a815c560 msgr2=0x7fb9a8155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ad800640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9a815c560 0x7fb9a8155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ad800640 1 -- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 msgr2=0x7fb9a8151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ad800640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 0x7fb9a8151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ad800640 1 -- 172.21.15.110:0/330929668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a81527c0 con 0x7fb9a806f1e0 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ae200640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9a815c560 0x7fb9a8155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.353+0000 7fb9ace00640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 0x7fb9a8151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9ad800640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9a806f1e0 0x7fb9a813c610 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb99c019080 tx=0x7fb99c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9a5600640 1 -- 172.21.15.110:0/330929668 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0bf610 con 0x7fb9a806f1e0 2024-09-10T10:01:17.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9a5600640 1 -- 172.21.15.110:0/330929668 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c0abe50 con 0x7fb9a806f1e0 2024-09-10T10:01:17.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a8152aa0 con 0x7fb9a806f1e0 2024-09-10T10:01:17.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9a5600640 1 -- 172.21.15.110:0/330929668 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0bfa10 con 0x7fb9a806f1e0 2024-09-10T10:01:17.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a813d420 con 0x7fb9a806f1e0 2024-09-10T10:01:17.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a80cc8c0 con 0x7fb9a806f1e0 2024-09-10T10:01:17.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9a5600640 1 -- 172.21.15.110:0/330929668 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fb99c0bd020 con 0x7fb9a806f1e0 2024-09-10T10:01:17.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9a5600640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb988067a70 0x7fb988069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9a5600640 1 -- 172.21.15.110:0/330929668 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fb99c11fbb0 con 0x7fb9a806f1e0 2024-09-10T10:01:17.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.357+0000 7fb9ace00640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb988067a70 0x7fb988069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.361+0000 7fb9ace00640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb988067a70 0x7fb988069f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8157a30 tx=0x7fb994009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:17.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.361+0000 7fb9a5600640 1 -- 172.21.15.110:0/330929668 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb99c0c6020 con 0x7fb9a806f1e0 2024-09-10T10:01:17.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb9a813d660 con 0x7fb9a806f1e0 2024-09-10T10:01:17.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9a5600640 1 -- 172.21.15.110:0/330929668 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7fb99c016090 con 0x7fb9a806f1e0 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:01:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:01:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:01:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:01:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:01:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:01:17.597 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:01:17.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb988067a70 msgr2=0x7fb988069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb988067a70 0x7fb988069f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8157a30 tx=0x7fb994009040 comp rx=0 tx=0).stop 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9a806f1e0 msgr2=0x7fb9a813c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9a806f1e0 0x7fb9a813c610 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb99c019080 tx=0x7fb99c01acb0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 shutdown_connections 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb988067a70 0x7fb988069f30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb9a815c560 0x7fb9a8155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9a8155970 0x7fb9a8151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 --2- 172.21.15.110:0/330929668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9a806f1e0 0x7fb9a813c610 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.581+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 >> 172.21.15.110:0/330929668 conn(0x7fb9a806bc50 msgr2=0x7fb9a80ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.585+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 shutdown_connections 2024-09-10T10:01:17.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.585+0000 7fb9b0200640 1 -- 172.21.15.110:0/330929668 wait complete. 2024-09-10T10:01:17.604 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:01:17.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.737+0000 7f3d59c00640 1 Processor -- start 2024-09-10T10:01:17.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.737+0000 7f3d59c00640 1 -- start start 2024-09-10T10:01:17.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.737+0000 7f3d59c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3d540cf1b0 0x7f3d540b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.737+0000 7f3d59c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5406d060 con 0x7f3d5415e620 2024-09-10T10:01:17.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.737+0000 7f3d59c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5406d340 con 0x7f3d540b6540 2024-09-10T10:01:17.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.737+0000 7f3d59c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3d540d1e90 con 0x7f3d540cf1b0 2024-09-10T10:01:17.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.737+0000 7f3d53e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3d5415e620 0x7f3d540b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49136/0 (socket says 172.21.15.110:49136) 2024-09-10T10:01:17.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.737+0000 7f3d53e00640 1 -- 172.21.15.110:0/3384188631 learned_addr learned my addr 172.21.15.110:0/3384188631 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:17.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d4be00640 1 --2- 172.21.15.110:0/3384188631 >> v2:172.21.15.145:3300/0 conn(0x7f3d540cf1b0 0x7f3d540b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545762218 0 0) 0x7f3d5406d060 con 0x7f3d5415e620 2024-09-10T10:01:17.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d38003530 con 0x7f3d5415e620 2024-09-10T10:01:17.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1858342337 0 0) 0x7f3d38003530 con 0x7f3d5415e620 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 >> v2:172.21.15.145:3300/0 conn(0x7f3d540cf1b0 msgr2=0x7f3d540b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 --2- 172.21.15.110:0/3384188631 >> v2:172.21.15.145:3300/0 conn(0x7f3d540cf1b0 0x7f3d540b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 >> v1:172.21.15.145:6789/0 conn(0x7f3d540b6540 legacy=0x7f3d5406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5406d620 con 0x7f3d5415e620 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f3d44001680 con 0x7f3d5415e620 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 >> v1:172.21.15.110:6789/0 conn(0x7f3d5415e620 legacy=0x7f3d540b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d38003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d38004810 0x7f3d38004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d380051d0 0x7f3d3800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3800c140 con 0x7f3d380039c0 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3800c2f0 con 0x7f3d38004810 2024-09-10T10:01:17.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3800c4f0 con 0x7f3d380051d0 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53e00640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d38003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d4be00640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d38004810 0x7f3d38004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d59200640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d380051d0 0x7f3d3800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53e00640 1 -- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d380051d0 msgr2=0x7f3d3800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53e00640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d380051d0 0x7f3d3800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53e00640 1 -- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d38004810 msgr2=0x7f3d38004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53e00640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d38004810 0x7f3d38004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d53e00640 1 -- 172.21.15.110:0/3384188631 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d38003530 con 0x7f3d380039c0 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d4be00640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d38004810 0x7f3d38004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:17.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.741+0000 7f3d59200640 1 -- 172.21.15.110:0/3384188631 reap_dead start 2024-09-10T10:01:17.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d53e00640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d38003de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400c240 tx=0x7f3d4401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0832d2cc610866 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:17.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44007d20 con 0x7f3d380039c0 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d440a9070 con 0x7f3d380039c0 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d53400640 1 -- 172.21.15.110:0/3384188631 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44005d80 con 0x7f3d380039c0 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- 172.21.15.110:0/3384188631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 msgr2=0x7f3d38003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d38003de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400c240 tx=0x7f3d4401cc80 comp rx=0 tx=0).stop 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- 172.21.15.110:0/3384188631 shutdown_connections 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 --2- 172.21.15.110:0/3384188631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d38003de0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- 172.21.15.110:0/3384188631 >> 172.21.15.110:0/3384188631 conn(0x7f3d5406f440 msgr2=0x7f3d5406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- 172.21.15.110:0/3384188631 shutdown_connections 2024-09-10T10:01:17.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- 172.21.15.110:0/3384188631 wait complete. 2024-09-10T10:01:17.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 Processor -- start 2024-09-10T10:01:17.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- start start 2024-09-10T10:01:17.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d54161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d54154ba0 0x7f3d54157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d54155020 0x7f3d54158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5406e490 con 0x7f3d380039c0 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5406e310 con 0x7f3d54155020 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d59c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5406e610 con 0x7f3d54154ba0 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d53e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d54161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d53e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d54161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40014/0 (socket says 172.21.15.110:40014) 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.745+0000 7f3d53e00640 1 -- 172.21.15.110:0/1301989609 learned_addr learned my addr 172.21.15.110:0/1301989609 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d4be00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d54154ba0 0x7f3d54157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d59200640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d54155020 0x7f3d54158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d53e00640 1 -- 172.21.15.110:0/1301989609 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d54154ba0 msgr2=0x7f3d54157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d53e00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d54154ba0 0x7f3d54157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d53e00640 1 -- 172.21.15.110:0/1301989609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d54155020 msgr2=0x7f3d54158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d53e00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d54155020 0x7f3d54158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d53e00640 1 -- 172.21.15.110:0/1301989609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d541590e0 con 0x7f3d380039c0 2024-09-10T10:01:17.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d53e00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d54161690 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3d4401cab0 tx=0x7f3d38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:17.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d50c00640 1 -- 172.21.15.110:0/1301989609 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44007d20 con 0x7f3d380039c0 2024-09-10T10:01:17.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d50c00640 1 -- 172.21.15.110:0/1301989609 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d44007180 con 0x7f3d380039c0 2024-09-10T10:01:17.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5415b270 con 0x7f3d380039c0 2024-09-10T10:01:17.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d50c00640 1 -- 172.21.15.110:0/1301989609 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44007530 con 0x7f3d380039c0 2024-09-10T10:01:17.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5415b830 con 0x7f3d380039c0 2024-09-10T10:01:17.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5406e9a0 con 0x7f3d380039c0 2024-09-10T10:01:17.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d50c00640 1 -- 172.21.15.110:0/1301989609 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f3d440047f0 con 0x7f3d380039c0 2024-09-10T10:01:17.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d50c00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3d30067a70 0x7f3d30069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d50c00640 1 -- 172.21.15.110:0/1301989609 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f3d4411f840 con 0x7f3d380039c0 2024-09-10T10:01:17.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.749+0000 7f3d4be00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3d30067a70 0x7f3d30069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.753+0000 7f3d4be00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3d30067a70 0x7f3d30069f30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d54155570 tx=0x7f3d3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:17.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.753+0000 7f3d50c00640 1 -- 172.21.15.110:0/1301989609 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3d44016990 con 0x7f3d380039c0 2024-09-10T10:01:17.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.913+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3d540d0b20 con 0x7f3d380039c0 2024-09-10T10:01:17.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.913+0000 7f3d50c00640 1 -- 172.21.15.110:0/1301989609 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7f3d440f0440 con 0x7f3d380039c0 2024-09-10T10:01:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:17.917 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:01:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:01:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:01:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:17.917 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3d30067a70 msgr2=0x7f3d30069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3d30067a70 0x7f3d30069f30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d54155570 tx=0x7f3d3c008040 comp rx=0 tx=0).stop 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 msgr2=0x7f3d54161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d54161690 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3d4401cab0 tx=0x7f3d38005840 comp rx=0 tx=0).stop 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 shutdown_connections 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3d30067a70 0x7f3d30069f30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d54155020 0x7f3d54158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d54154ba0 0x7f3d54157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 --2- 172.21.15.110:0/1301989609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d380039c0 0x7f3d54161690 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 >> 172.21.15.110:0/1301989609 conn(0x7f3d5406f440 msgr2=0x7f3d5406f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:17.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 shutdown_connections 2024-09-10T10:01:17.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:17.917+0000 7f3d59c00640 1 -- 172.21.15.110:0/1301989609 wait complete. 2024-09-10T10:01:17.938 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:01:17.938 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:01:18.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.073+0000 7f0dbc200640 1 Processor -- start 2024-09-10T10:01:18.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.073+0000 7f0dbc200640 1 -- start start 2024-09-10T10:01:18.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dbc200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db40cfe30 0x7f0db4071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dbc200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0db406e7a0 0x7f0db40cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dbc200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db406d2d0 con 0x7f0db40cd510 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dbc200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0db40cdeb0 con 0x7f0db406e7a0 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dbc200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0db415d9a0 con 0x7f0db40cfe30 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0db8e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0db40cd510 0x7f0db406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49138/0 (socket says 172.21.15.110:49138) 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0db8e00640 1 -- 172.21.15.110:0/2926287218 learned_addr learned my addr 172.21.15.110:0/2926287218 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dba200640 1 --2- 172.21.15.110:0/2926287218 >> v2:172.21.15.110:3300/0 conn(0x7f0db406e7a0 0x7f0db40cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0db9800640 1 --2- 172.21.15.110:0/2926287218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db40cfe30 0x7f0db4071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0db3e00640 1 -- 172.21.15.110:0/2926287218 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759671460 0 0) 0x7f0db406d2d0 con 0x7f0db40cd510 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dba200640 1 -- 172.21.15.110:0/2926287218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db40cfe30 msgr2=0x7f0db4071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dba200640 1 --2- 172.21.15.110:0/2926287218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db40cfe30 0x7f0db4071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dba200640 1 -- 172.21.15.110:0/2926287218 >> v1:172.21.15.110:6789/0 conn(0x7f0db40cd510 legacy=0x7f0db406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0dba200640 1 -- 172.21.15.110:0/2926287218 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db406d5b0 con 0x7f0db406e7a0 2024-09-10T10:01:18.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.077+0000 7f0db9800640 1 --2- 172.21.15.110:0/2926287218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db40cfe30 0x7f0db4071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:18.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dba200640 1 --2- 172.21.15.110:0/2926287218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db406e7a0 0x7f0db40cd970 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009d90 tx=0x7f0dac01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0731cb88935bfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:18.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0db3e00640 1 -- 172.21.15.110:0/2926287218 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0b2070 con 0x7f0db406e7a0 2024-09-10T10:01:18.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0db3e00640 1 -- 172.21.15.110:0/2926287218 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dac0a58b0 con 0x7f0db406e7a0 2024-09-10T10:01:18.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0db3e00640 1 -- 172.21.15.110:0/2926287218 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0a5c80 con 0x7f0db406e7a0 2024-09-10T10:01:18.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 -- 172.21.15.110:0/2926287218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db406e7a0 msgr2=0x7f0db40cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 --2- 172.21.15.110:0/2926287218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db406e7a0 0x7f0db40cd970 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009d90 tx=0x7f0dac01c8e0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 -- 172.21.15.110:0/2926287218 shutdown_connections 2024-09-10T10:01:18.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 --2- 172.21.15.110:0/2926287218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db406e7a0 0x7f0db40cd970 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 --2- 172.21.15.110:0/2926287218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db40cfe30 0x7f0db4071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 -- 172.21.15.110:0/2926287218 >> 172.21.15.110:0/2926287218 conn(0x7f0db406bc50 msgr2=0x7f0db406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:18.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 -- 172.21.15.110:0/2926287218 shutdown_connections 2024-09-10T10:01:18.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 -- 172.21.15.110:0/2926287218 wait complete. 2024-09-10T10:01:18.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 Processor -- start 2024-09-10T10:01:18.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.081+0000 7f0dbc200640 1 -- start start 2024-09-10T10:01:18.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db406e7a0 0x7f0db40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0db40cd510 0x7f0db4157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db40cfe30 0x7f0db4150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0db4072190 con 0x7f0db40cfe30 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0db415d640 con 0x7f0db40cd510 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0db4169180 con 0x7f0db406e7a0 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db40cfe30 0x7f0db4150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db40cfe30 0x7f0db4150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40038/0 (socket says 172.21.15.110:40038) 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 -- 172.21.15.110:0/1938877475 learned_addr learned my addr 172.21.15.110:0/1938877475 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0db9800640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db406e7a0 0x7f0db40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0db8e00640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0db40cd510 0x7f0db4157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 -- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db406e7a0 msgr2=0x7f0db40b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db406e7a0 0x7f0db40b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 -- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0db40cd510 msgr2=0x7f0db4157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0db40cd510 0x7f0db4157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 -- 172.21.15.110:0/1938877475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db4157570 con 0x7f0db40cfe30 2024-09-10T10:01:18.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0db8e00640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0db40cd510 0x7f0db4157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:18.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dba200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db40cfe30 0x7f0db4150d90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0b0040 tx=0x7f0dac019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:18.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0db1600640 1 -- 172.21.15.110:0/1938877475 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0b2050 con 0x7f0db40cfe30 2024-09-10T10:01:18.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0db1600640 1 -- 172.21.15.110:0/1938877475 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dac0be070 con 0x7f0db40cfe30 2024-09-10T10:01:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0db1600640 1 -- 172.21.15.110:0/1938877475 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0ada90 con 0x7f0db40cfe30 2024-09-10T10:01:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db40b2620 con 0x7f0db40cfe30 2024-09-10T10:01:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db40b5100 con 0x7f0db40cfe30 2024-09-10T10:01:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db40cd970 con 0x7f0db40cfe30 2024-09-10T10:01:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.085+0000 7f0db1600640 1 -- 172.21.15.110:0/1938877475 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f0dac0adc30 con 0x7f0db40cfe30 2024-09-10T10:01:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.089+0000 7f0db1600640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d94067a70 0x7f0d94069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.089+0000 7f0db1600640 1 -- 172.21.15.110:0/1938877475 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f0dac120cd0 con 0x7f0db40cfe30 2024-09-10T10:01:18.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.089+0000 7f0db9800640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d94067a70 0x7f0d94069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.089+0000 7f0db9800640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d94067a70 0x7f0d94069f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0da800c900 tx=0x7f0da8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:18.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.093+0000 7f0db1600640 1 -- 172.21.15.110:0/1938877475 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0dac016560 con 0x7f0db40cfe30 2024-09-10T10:01:18.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:18.253+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0db4153b00 con 0x7f0db40cfe30 2024-09-10T10:01:18.890 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:01:18.893 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:01:19.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0db1600640 1 -- 172.21.15.110:0/1938877475 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7f0dac0ab340 con 0x7f0db40cfe30 2024-09-10T10:01:19.198 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:01:19.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d94067a70 msgr2=0x7f0d94069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d94067a70 0x7f0d94069f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0da800c900 tx=0x7f0da8008040 comp rx=0 tx=0).stop 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db40cfe30 msgr2=0x7f0db4150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db40cfe30 0x7f0db4150d90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0b0040 tx=0x7f0dac019680 comp rx=0 tx=0).stop 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 shutdown_connections 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d94067a70 0x7f0d94069f30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db40cfe30 0x7f0db4150d90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0db40cd510 0x7f0db4157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 --2- 172.21.15.110:0/1938877475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0db406e7a0 0x7f0db40b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 >> 172.21.15.110:0/1938877475 conn(0x7f0db406bc50 msgr2=0x7f0db415a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 shutdown_connections 2024-09-10T10:01:19.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.197+0000 7f0dbc200640 1 -- 172.21.15.110:0/1938877475 wait complete. 2024-09-10T10:01:19.226 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:01:19.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.365+0000 7f9eca200640 1 Processor -- start 2024-09-10T10:01:19.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.365+0000 7f9eca200640 1 -- start start 2024-09-10T10:01:19.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9eca200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9ec4155970 0x7f9ec40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:19.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9eca200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec415c560 0x7f9ec406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9eca200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9ec406f010 0x7f9ec415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9eca200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec40b4570 con 0x7f9ec406f010 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9eca200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec406f430 con 0x7f9ec4155970 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9eca200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec4155d50 con 0x7f9ec415c560 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9ec406f010 0x7f9ec415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9ec406f010 0x7f9ec415eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35714/0 (socket says 172.21.15.110:35714) 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3e00640 1 -- 172.21.15.110:0/1122513904 learned_addr learned my addr 172.21.15.110:0/1122513904 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec2a00640 1 --2- 172.21.15.110:0/1122513904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec415c560 0x7f9ec406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:19.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3400640 1 --2- 172.21.15.110:0/1122513904 >> v2:172.21.15.145:3300/0 conn(0x7f9ec4155970 0x7f9ec40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:19.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3e00640 1 -- 172.21.15.110:0/1122513904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec415c560 msgr2=0x7f9ec406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:19.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3e00640 1 --2- 172.21.15.110:0/1122513904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec415c560 0x7f9ec406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3e00640 1 -- 172.21.15.110:0/1122513904 >> v2:172.21.15.145:3300/0 conn(0x7f9ec4155970 msgr2=0x7f9ec40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:19.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3e00640 1 --2- 172.21.15.110:0/1122513904 >> v2:172.21.15.145:3300/0 conn(0x7f9ec4155970 0x7f9ec40cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3e00640 1 -- 172.21.15.110:0/1122513904 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec40b34d0 con 0x7f9ec406f010 2024-09-10T10:01:19.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec2a00640 1 --2- 172.21.15.110:0/1122513904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec415c560 0x7f9ec406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:19.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.369+0000 7f9ec3400640 1 --2- 172.21.15.110:0/1122513904 >> v2:172.21.15.145:3300/0 conn(0x7f9ec4155970 0x7f9ec40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:19.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9ec3e00640 1 --2- 172.21.15.110:0/1122513904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec406f010 0x7f9ec415eea0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9ec406cc30 tx=0x7f9ebc020a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e2a8fcce01cf81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:19.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9ec2000640 1 -- 172.21.15.110:0/1122513904 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0a9920 con 0x7f9ec406f010 2024-09-10T10:01:19.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9ec2000640 1 -- 172.21.15.110:0/1122513904 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc0a9ac0 con 0x7f9ec406f010 2024-09-10T10:01:19.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9ec2000640 1 -- 172.21.15.110:0/1122513904 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0b13f0 con 0x7f9ec406f010 2024-09-10T10:01:19.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 -- 172.21.15.110:0/1122513904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec406f010 msgr2=0x7f9ec415eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:19.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 --2- 172.21.15.110:0/1122513904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec406f010 0x7f9ec415eea0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9ec406cc30 tx=0x7f9ebc020a90 comp rx=0 tx=0).stop 2024-09-10T10:01:19.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 -- 172.21.15.110:0/1122513904 shutdown_connections 2024-09-10T10:01:19.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 --2- 172.21.15.110:0/1122513904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec406f010 0x7f9ec415eea0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 --2- 172.21.15.110:0/1122513904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec415c560 0x7f9ec406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 --2- 172.21.15.110:0/1122513904 >> v2:172.21.15.145:3300/0 conn(0x7f9ec4155970 0x7f9ec40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 -- 172.21.15.110:0/1122513904 >> 172.21.15.110:0/1122513904 conn(0x7f9ec406bc50 msgr2=0x7f9ec406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:19.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 -- 172.21.15.110:0/1122513904 shutdown_connections 2024-09-10T10:01:19.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 -- 172.21.15.110:0/1122513904 wait complete. 2024-09-10T10:01:19.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 Processor -- start 2024-09-10T10:01:19.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.373+0000 7f9eca200640 1 -- start start 2024-09-10T10:01:19.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ec406f010 0x7f9ec4151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:19.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec4155970 0x7f9ec4155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec415c560 0x7f9ec415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec40cce00 con 0x7f9ec415c560 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec40b4050 con 0x7f9ec406f010 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec406d9c0 con 0x7f9ec4155970 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec415c560 0x7f9ec415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec2a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec4155970 0x7f9ec4155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec415c560 0x7f9ec415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35720/0 (socket says 172.21.15.110:35720) 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 -- 172.21.15.110:0/1120664529 learned_addr learned my addr 172.21.15.110:0/1120664529 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3400640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ec406f010 0x7f9ec4151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 -- 172.21.15.110:0/1120664529 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec4155970 msgr2=0x7f9ec4155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:19.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec4155970 0x7f9ec4155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 -- 172.21.15.110:0/1120664529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ec406f010 msgr2=0x7f9ec4151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:19.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ec406f010 0x7f9ec4151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:19.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 -- 172.21.15.110:0/1120664529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec41568b0 con 0x7f9ec415c560 2024-09-10T10:01:19.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ec3e00640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec415c560 0x7f9ec415aaa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0b3040 tx=0x7f9ebc01d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:19.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ebb400640 1 -- 172.21.15.110:0/1120664529 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0c0020 con 0x7f9ec415c560 2024-09-10T10:01:19.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ebb400640 1 -- 172.21.15.110:0/1120664529 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc0bc470 con 0x7f9ec415c560 2024-09-10T10:01:19.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec4156b90 con 0x7f9ec415c560 2024-09-10T10:01:19.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9ebb400640 1 -- 172.21.15.110:0/1120664529 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0a9920 con 0x7f9ec415c560 2024-09-10T10:01:19.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec4157150 con 0x7f9ec415c560 2024-09-10T10:01:19.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.377+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec415eea0 con 0x7f9ec415c560 2024-09-10T10:01:19.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.381+0000 7f9ebb400640 1 -- 172.21.15.110:0/1120664529 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f9ebc0c3020 con 0x7f9ec415c560 2024-09-10T10:01:19.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.381+0000 7f9ebb400640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ea4067a70 0x7f9ea4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:19.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.381+0000 7f9ebb400640 1 -- 172.21.15.110:0/1120664529 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7f9ebc1257e0 con 0x7f9ec415c560 2024-09-10T10:01:19.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.381+0000 7f9ec3400640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ea4067a70 0x7f9ea4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:19.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.381+0000 7f9ec3400640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ea4067a70 0x7f9ea4069f30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4165140 tx=0x7f9eb4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:19.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.385+0000 7f9ebb400640 1 -- 172.21.15.110:0/1120664529 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9ebc0ca030 con 0x7f9ec415c560 2024-09-10T10:01:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:19.545+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9ec40b0830 con 0x7f9ec415c560 2024-09-10T10:01:20.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.201+0000 7f9ebb400640 1 -- 172.21.15.110:0/1120664529 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7f9ebc01aa00 con 0x7f9ec415c560 2024-09-10T10:01:20.205 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:01:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ea4067a70 msgr2=0x7f9ea4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ea4067a70 0x7f9ea4069f30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4165140 tx=0x7f9eb4008040 comp rx=0 tx=0).stop 2024-09-10T10:01:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec415c560 msgr2=0x7f9ec415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec415c560 0x7f9ec415aaa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0b3040 tx=0x7f9ebc01d830 comp rx=0 tx=0).stop 2024-09-10T10:01:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 shutdown_connections 2024-09-10T10:01:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9ea4067a70 0x7f9ea4069f30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ec415c560 0x7f9ec415aaa0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ec4155970 0x7f9ec4155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 --2- 172.21.15.110:0/1120664529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ec406f010 0x7f9ec4151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 >> 172.21.15.110:0/1120664529 conn(0x7f9ec406bc50 msgr2=0x7f9ec406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:20.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 shutdown_connections 2024-09-10T10:01:20.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.205+0000 7f9eca200640 1 -- 172.21.15.110:0/1120664529 wait complete. 2024-09-10T10:01:20.240 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:01:20.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5206c00640 1 Processor -- start 2024-09-10T10:01:20.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5206c00640 1 -- start start 2024-09-10T10:01:20.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5206c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5200145f30 0x7f52000b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5206c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52000b29e0 0x7f52000cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5206c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52000b37c0 con 0x7f52000cfbc0 2024-09-10T10:01:20.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5206c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f52000b2e20 con 0x7f5200145f30 2024-09-10T10:01:20.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5206c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52000b3aa0 con 0x7f52000b29e0 2024-09-10T10:01:20.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5204c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f52000cfbc0 0x7f5200160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45886/0 (socket says 172.21.15.110:45886) 2024-09-10T10:01:20.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f51ffe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5200145f30 0x7f52000b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f5204c00640 1 -- 172.21.15.110:0/1217239737 learned_addr learned my addr 172.21.15.110:0/1217239737 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:20.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f51ff400640 1 --2- 172.21.15.110:0/1217239737 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52000b29e0 0x7f52000cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f51ffe00640 1 -- 172.21.15.110:0/1217239737 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52000b29e0 msgr2=0x7f52000cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:20.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f51ffe00640 1 --2- 172.21.15.110:0/1217239737 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52000b29e0 0x7f52000cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f51ffe00640 1 -- 172.21.15.110:0/1217239737 >> v1:172.21.15.110:6789/0 conn(0x7f52000cfbc0 legacy=0x7f5200160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f51ffe00640 1 -- 172.21.15.110:0/1217239737 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52000b3d80 con 0x7f5200145f30 2024-09-10T10:01:20.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.377+0000 7f51fea00640 1 -- 172.21.15.110:0/1217239737 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523321524 0 0) 0x7f52000b37c0 con 0x7f52000cfbc0 2024-09-10T10:01:20.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f51ffe00640 1 --2- 172.21.15.110:0/1217239737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5200145f30 0x7f52000b24a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009d90 tx=0x7f51f001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a212576c3c0c3ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f51fea00640 1 -- 172.21.15.110:0/1217239737 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00b2070 con 0x7f5200145f30 2024-09-10T10:01:20.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f51fea00640 1 -- 172.21.15.110:0/1217239737 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f00a5830 con 0x7f5200145f30 2024-09-10T10:01:20.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f51fea00640 1 -- 172.21.15.110:0/1217239737 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00a5c00 con 0x7f5200145f30 2024-09-10T10:01:20.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f5206c00640 1 -- 172.21.15.110:0/1217239737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5200145f30 msgr2=0x7f52000b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f5206c00640 1 --2- 172.21.15.110:0/1217239737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5200145f30 0x7f52000b24a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009d90 tx=0x7f51f001c970 comp rx=0 tx=0).stop 2024-09-10T10:01:20.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f5206c00640 1 -- 172.21.15.110:0/1217239737 shutdown_connections 2024-09-10T10:01:20.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f5206c00640 1 --2- 172.21.15.110:0/1217239737 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52000b29e0 0x7f52000cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f5206c00640 1 --2- 172.21.15.110:0/1217239737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5200145f30 0x7f52000b24a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f5206c00640 1 -- 172.21.15.110:0/1217239737 >> 172.21.15.110:0/1217239737 conn(0x7f5200062e40 msgr2=0x7f520006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:20.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f5206c00640 1 -- 172.21.15.110:0/1217239737 shutdown_connections 2024-09-10T10:01:20.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.381+0000 7f5206c00640 1 -- 172.21.15.110:0/1217239737 wait complete. 2024-09-10T10:01:20.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 Processor -- start 2024-09-10T10:01:20.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 -- start start 2024-09-10T10:01:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52000b29e0 0x7f520015e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52000cfbc0 0x7f520013c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5200145f30 0x7f5200140470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5200169530 con 0x7f52000b29e0 2024-09-10T10:01:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52001693b0 con 0x7f52000cfbc0 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52001696b0 con 0x7f5200145f30 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52000b29e0 0x7f520015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52000b29e0 0x7f520015e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35746/0 (socket says 172.21.15.110:35746) 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 -- 172.21.15.110:0/2857018000 learned_addr learned my addr 172.21.15.110:0/2857018000 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 -- 172.21.15.110:0/2857018000 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5200145f30 msgr2=0x7f5200140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5200145f30 0x7f5200140470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 -- 172.21.15.110:0/2857018000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52000cfbc0 msgr2=0x7f520013c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52000cfbc0 0x7f520013c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 -- 172.21.15.110:0/2857018000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520013b790 con 0x7f52000b29e0 2024-09-10T10:01:20.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51ffe00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52000b29e0 0x7f520015e570 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f51f00b0040 tx=0x7f51f0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51f7e00640 1 -- 172.21.15.110:0/2857018000 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00bd020 con 0x7f52000b29e0 2024-09-10T10:01:20.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51f7e00640 1 -- 172.21.15.110:0/2857018000 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f00b8470 con 0x7f52000b29e0 2024-09-10T10:01:20.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f51f7e00640 1 -- 172.21.15.110:0/2857018000 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00b2040 con 0x7f52000b29e0 2024-09-10T10:01:20.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520013ba70 con 0x7f52000b29e0 2024-09-10T10:01:20.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520013c030 con 0x7f52000b29e0 2024-09-10T10:01:20.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.385+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52000cf680 con 0x7f52000b29e0 2024-09-10T10:01:20.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.389+0000 7f51f7e00640 1 -- 172.21.15.110:0/2857018000 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f51f00c0020 con 0x7f52000b29e0 2024-09-10T10:01:20.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.389+0000 7f51f7e00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f51e0067a70 0x7f51e0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.389+0000 7f51f7e00640 1 -- 172.21.15.110:0/2857018000 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7f51f0121350 con 0x7f52000b29e0 2024-09-10T10:01:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.389+0000 7f51ff400640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f51e0067a70 0x7f51e0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.389+0000 7f51ff400640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f51e0067a70 0x7f51e0069f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f520013ea70 tx=0x7f51ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.393+0000 7f51f7e00640 1 -- 172.21.15.110:0/2857018000 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f51f00c7030 con 0x7f52000b29e0 2024-09-10T10:01:20.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:20.581+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f52000ccc90 con 0x7f52000b29e0 2024-09-10T10:01:21.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.204+0000 7f51f7e00640 1 -- 172.21.15.110:0/2857018000 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f51f0016560 con 0x7f52000b29e0 2024-09-10T10:01:21.207 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 8 and data pool 9 2024-09-10T10:01:21.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f51e0067a70 msgr2=0x7f51e0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f51e0067a70 0x7f51e0069f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f520013ea70 tx=0x7f51ec008040 comp rx=0 tx=0).stop 2024-09-10T10:01:21.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52000b29e0 msgr2=0x7f520015e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52000b29e0 0x7f520015e570 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f51f00b0040 tx=0x7f51f0019710 comp rx=0 tx=0).stop 2024-09-10T10:01:21.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 shutdown_connections 2024-09-10T10:01:21.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f51e0067a70 0x7f51e0069f30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5200145f30 0x7f5200140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52000cfbc0 0x7f520013c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 --2- 172.21.15.110:0/2857018000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52000b29e0 0x7f520015e570 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 >> 172.21.15.110:0/2857018000 conn(0x7f5200062e40 msgr2=0x7f520006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:21.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 shutdown_connections 2024-09-10T10:01:21.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.208+0000 7f5206c00640 1 -- 172.21.15.110:0/2857018000 wait complete. 2024-09-10T10:01:21.262 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:21.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6ccc800640 1 Processor -- start 2024-09-10T10:01:21.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6ccc800640 1 -- start start 2024-09-10T10:01:21.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6ccc800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc4155970 0x7f6cc40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6ccc800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6cc406f1e0 0x7f6cc40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6ccc800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cc4155d50 con 0x7f6cc415c560 2024-09-10T10:01:21.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6ccc800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc40b4570 con 0x7f6cc406f1e0 2024-09-10T10:01:21.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6ccc800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc40b34d0 con 0x7f6cc4155970 2024-09-10T10:01:21.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6cc9e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc4155970 0x7f6cc40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6cca800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6cc406f1e0 0x7f6cc40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6cc9400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f6cc415c560 0x7f6cc406ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:37964/0 (socket says 172.21.15.110:37964) 2024-09-10T10:01:21.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.396+0000 7f6cc9400640 1 -- 172.21.15.110:0/3789209771 learned_addr learned my addr 172.21.15.110:0/3789209771 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:21.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc8a00640 1 -- 172.21.15.110:0/3789209771 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4271011443 0 0) 0x7f6cc4155d50 con 0x7f6cc415c560 2024-09-10T10:01:21.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc8a00640 1 -- 172.21.15.110:0/3789209771 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cac003530 con 0x7f6cc415c560 2024-09-10T10:01:21.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc9e00640 1 -- 172.21.15.110:0/3789209771 >> v2:172.21.15.145:3300/0 conn(0x7f6cc406f1e0 msgr2=0x7f6cc40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:21.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc9e00640 1 --2- 172.21.15.110:0/3789209771 >> v2:172.21.15.145:3300/0 conn(0x7f6cc406f1e0 0x7f6cc40ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc9e00640 1 -- 172.21.15.110:0/3789209771 >> v1:172.21.15.145:6789/0 conn(0x7f6cc415c560 legacy=0x7f6cc406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc9e00640 1 -- 172.21.15.110:0/3789209771 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc40b37b0 con 0x7f6cc4155970 2024-09-10T10:01:21.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cca800640 1 --2- 172.21.15.110:0/3789209771 >> v2:172.21.15.145:3300/0 conn(0x7f6cc406f1e0 0x7f6cc40ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:21.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc9e00640 1 --2- 172.21.15.110:0/3789209771 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc4155970 0x7f6cc40cc8c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80097c0 tx=0x7f6cb801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584c2344fc0fe21d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:21.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc8a00640 1 -- 172.21.15.110:0/3789209771 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80b1070 con 0x7f6cc4155970 2024-09-10T10:01:21.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc8a00640 1 -- 172.21.15.110:0/3789209771 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb80a8050 con 0x7f6cc4155970 2024-09-10T10:01:21.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.400+0000 7f6cc8a00640 1 -- 172.21.15.110:0/3789209771 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ac840 con 0x7f6cc4155970 2024-09-10T10:01:21.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- 172.21.15.110:0/3789209771 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc4155970 msgr2=0x7f6cc40cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3789209771 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc4155970 0x7f6cc40cc8c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80097c0 tx=0x7f6cb801c3e0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- 172.21.15.110:0/3789209771 shutdown_connections 2024-09-10T10:01:21.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3789209771 >> v2:172.21.15.145:3300/0 conn(0x7f6cc406f1e0 0x7f6cc40ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3789209771 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc4155970 0x7f6cc40cc8c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- 172.21.15.110:0/3789209771 >> 172.21.15.110:0/3789209771 conn(0x7f6cc406bc50 msgr2=0x7f6cc406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:21.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- 172.21.15.110:0/3789209771 shutdown_connections 2024-09-10T10:01:21.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- 172.21.15.110:0/3789209771 wait complete. 2024-09-10T10:01:21.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 Processor -- start 2024-09-10T10:01:21.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- start start 2024-09-10T10:01:21.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc406f1e0 0x7f6cc414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cc4155970 0x7f6cc4151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc415c560 0x7f6cc414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40cf170 con 0x7f6cc4155970 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40cce00 con 0x7f6cc415c560 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6ccc800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc40cf2f0 con 0x7f6cc406f1e0 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cc4155970 0x7f6cc4151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cc4155970 0x7f6cc4151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35750/0 (socket says 172.21.15.110:35750) 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 -- 172.21.15.110:0/3370184671 learned_addr learned my addr 172.21.15.110:0/3370184671 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9e00640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc406f1e0 0x7f6cc414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cca800640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc415c560 0x7f6cc414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 -- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc406f1e0 msgr2=0x7f6cc414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc406f1e0 0x7f6cc414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 -- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc415c560 msgr2=0x7f6cc414ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc415c560 0x7f6cc414ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 -- 172.21.15.110:0/3370184671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc4159990 con 0x7f6cc4155970 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cca800640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc415c560 0x7f6cc414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.404+0000 7f6cc9400640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cc4155970 0x7f6cc4151930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400c400 tx=0x7f6cb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6cbe000640 1 -- 172.21.15.110:0/3370184671 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4017070 con 0x7f6cc4155970 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6cbe000640 1 -- 172.21.15.110:0/3370184671 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4005cf0 con 0x7f6cc4155970 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6cbe000640 1 -- 172.21.15.110:0/3370184671 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40072d0 con 0x7f6cc4155970 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc4159c70 con 0x7f6cc4155970 2024-09-10T10:01:21.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc415a230 con 0x7f6cc4155970 2024-09-10T10:01:21.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc40ce420 con 0x7f6cc4155970 2024-09-10T10:01:21.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6cbe000640 1 -- 172.21.15.110:0/3370184671 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f6cb4007470 con 0x7f6cc4155970 2024-09-10T10:01:21.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6cbe000640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ca8067a70 0x7f6ca8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6cbe000640 1 -- 172.21.15.110:0/3370184671 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6cb4085270 con 0x7f6cc4155970 2024-09-10T10:01:21.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6cc9e00640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ca8067a70 0x7f6ca8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.408+0000 7f6cc9e00640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ca8067a70 0x7f6ca8069f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80ae000 tx=0x7f6cb8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:21.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.412+0000 7f6cbe000640 1 -- 172.21.15.110:0/3370184671 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6cb4055590 con 0x7f6cc4155970 2024-09-10T10:01:21.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.572+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cc406c1b0 con 0x7f6cc4155970 2024-09-10T10:01:21.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.572+0000 7f6cbe000640 1 -- 172.21.15.110:0/3370184671 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16774 (secure 0 0 0) 0x7f6cb4059890 con 0x7f6cc4155970 2024-09-10T10:01:21.576 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:21.576 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":57,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:20.586057+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:18.299917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:19.590875+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:21.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ca8067a70 msgr2=0x7f6ca8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ca8067a70 0x7f6ca8069f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80ae000 tx=0x7f6cb8008040 comp rx=0 tx=0).stop 2024-09-10T10:01:21.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cc4155970 msgr2=0x7f6cc4151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cc4155970 0x7f6cc4151930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400c400 tx=0x7f6cb400c8c0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 shutdown_connections 2024-09-10T10:01:21.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6ca8067a70 0x7f6ca8069f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cc415c560 0x7f6cc414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cc4155970 0x7f6cc4151930 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 --2- 172.21.15.110:0/3370184671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6cc406f1e0 0x7f6cc414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 >> 172.21.15.110:0/3370184671 conn(0x7f6cc406bc50 msgr2=0x7f6cc4071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:21.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 shutdown_connections 2024-09-10T10:01:21.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.576+0000 7f6ccc800640 1 -- 172.21.15.110:0/3370184671 wait complete. 2024-09-10T10:01:21.596 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:21.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9dcc00640 1 Processor -- start 2024-09-10T10:01:21.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9dcc00640 1 -- start start 2024-09-10T10:01:21.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9dcc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa9d815c730 0x7fa9d80d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9dcc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d80740b0 con 0x7fa9d80b6ca0 2024-09-10T10:01:21.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9dcc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d80717b0 con 0x7fa9d80cf380 2024-09-10T10:01:21.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9dcc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d80d0eb0 con 0x7fa9d815c730 2024-09-10T10:01:21.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9d5e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa9d815c730 0x7fa9d80d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9d6800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa9d80b6ca0 0x7fa9d80739a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45890/0 (socket says 172.21.15.110:45890) 2024-09-10T10:01:21.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9d5e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa9d815c730 0x7fa9d80d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35764/0 (socket says 172.21.15.110:35764) 2024-09-10T10:01:21.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9d6800640 1 -- 172.21.15.110:0/3895380767 learned_addr learned my addr 172.21.15.110:0/3895380767 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:21.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9d5e00640 1 -- 172.21.15.110:0/3895380767 >> v1:172.21.15.145:6789/0 conn(0x7fa9d80cf380 legacy=0x7fa9d8073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:21.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9d5e00640 1 -- 172.21.15.110:0/3895380767 >> v1:172.21.15.110:6789/0 conn(0x7fa9d80b6ca0 legacy=0x7fa9d80739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.728+0000 7fa9d5e00640 1 -- 172.21.15.110:0/3895380767 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d8071a90 con 0x7fa9d815c730 2024-09-10T10:01:21.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9d5e00640 1 --2- 172.21.15.110:0/3895380767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d815c730 0x7fa9d80d2530 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80097c0 tx=0x7fa9c801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98eec1a86253ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:21.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9d4a00640 1 -- 172.21.15.110:0/3895380767 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80b0070 con 0x7fa9d815c730 2024-09-10T10:01:21.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9d4a00640 1 -- 172.21.15.110:0/3895380767 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c80ab600 con 0x7fa9d815c730 2024-09-10T10:01:21.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9d4a00640 1 -- 172.21.15.110:0/3895380767 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80a4e60 con 0x7fa9d815c730 2024-09-10T10:01:21.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9dcc00640 1 -- 172.21.15.110:0/3895380767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d815c730 msgr2=0x7fa9d80d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9dcc00640 1 --2- 172.21.15.110:0/3895380767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d815c730 0x7fa9d80d2530 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80097c0 tx=0x7fa9c801c3d0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9dcc00640 1 -- 172.21.15.110:0/3895380767 shutdown_connections 2024-09-10T10:01:21.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9dcc00640 1 --2- 172.21.15.110:0/3895380767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d815c730 0x7fa9d80d2530 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9dcc00640 1 -- 172.21.15.110:0/3895380767 >> 172.21.15.110:0/3895380767 conn(0x7fa9d806f440 msgr2=0x7fa9d806f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:21.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9dcc00640 1 -- 172.21.15.110:0/3895380767 shutdown_connections 2024-09-10T10:01:21.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.732+0000 7fa9dcc00640 1 -- 172.21.15.110:0/3895380767 wait complete. 2024-09-10T10:01:21.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 Processor -- start 2024-09-10T10:01:21.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 -- start start 2024-09-10T10:01:21.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d80b6ca0 0x7fa9d8155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d80cf380 0x7fa9d814ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9d815c730 0x7fa9d8150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d80d2e80 con 0x7fa9d80cf380 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d815cd20 con 0x7fa9d80b6ca0 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d816ac30 con 0x7fa9d815c730 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d80cf380 0x7fa9d814ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d80cf380 0x7fa9d814ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35770/0 (socket says 172.21.15.110:35770) 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 -- 172.21.15.110:0/2549687639 learned_addr learned my addr 172.21.15.110:0/2549687639 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5e00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d80b6ca0 0x7fa9d8155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d6800640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9d815c730 0x7fa9d8150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 -- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9d815c730 msgr2=0x7fa9d8150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9d815c730 0x7fa9d8150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 -- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d80b6ca0 msgr2=0x7fa9d8155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d80b6ca0 0x7fa9d8155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 -- 172.21.15.110:0/2549687639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d80b0980 con 0x7fa9d80cf380 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d6800640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9d815c730 0x7fa9d8150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9d5400640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d80cf380 0x7fa9d814ad10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400c490 tx=0x7fa9c4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9ce000640 1 -- 172.21.15.110:0/2549687639 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4017070 con 0x7fa9d80cf380 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9ce000640 1 -- 172.21.15.110:0/2549687639 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4003600 con 0x7fa9d80cf380 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9ce000640 1 -- 172.21.15.110:0/2549687639 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4003a30 con 0x7fa9d80cf380 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d80b0c20 con 0x7fa9d80cf380 2024-09-10T10:01:21.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d80b11e0 con 0x7fa9d80cf380 2024-09-10T10:01:21.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.736+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d80b2380 con 0x7fa9d80cf380 2024-09-10T10:01:21.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.740+0000 7fa9ce000640 1 -- 172.21.15.110:0/2549687639 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fa9c4003bd0 con 0x7fa9d80cf380 2024-09-10T10:01:21.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.740+0000 7fa9ce000640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa9b8067aa0 0x7fa9b8069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:21.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.740+0000 7fa9ce000640 1 -- 172.21.15.110:0/2549687639 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fa9c40852b0 con 0x7fa9d80cf380 2024-09-10T10:01:21.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.740+0000 7fa9d5e00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa9b8067aa0 0x7fa9b8069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:21.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.740+0000 7fa9d5e00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa9b8067aa0 0x7fa9b8069f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80ad000 tx=0x7fa9c80a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:21.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.744+0000 7fa9ce000640 1 -- 172.21.15.110:0/2549687639 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa9c4055ad0 con 0x7fa9d80cf380 2024-09-10T10:01:21.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.900+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9d815f480 con 0x7fa9d80cf380 2024-09-10T10:01:21.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9ce000640 1 -- 172.21.15.110:0/2549687639 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16774 (secure 0 0 0) 0x7fa9c407ce20 con 0x7fa9d80cf380 2024-09-10T10:01:21.906 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:21.907 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":57,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:20.586057+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:18.299917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:19.590875+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa9b8067aa0 msgr2=0x7fa9b8069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa9b8067aa0 0x7fa9b8069f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80ad000 tx=0x7fa9c80a6000 comp rx=0 tx=0).stop 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d80cf380 msgr2=0x7fa9d814ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d80cf380 0x7fa9d814ad10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400c490 tx=0x7fa9c4001e80 comp rx=0 tx=0).stop 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 shutdown_connections 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa9b8067aa0 0x7fa9b8069f60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9d815c730 0x7fa9d8150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d80cf380 0x7fa9d814ad10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 --2- 172.21.15.110:0/2549687639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9d80b6ca0 0x7fa9d8155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:21.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 >> 172.21.15.110:0/2549687639 conn(0x7fa9d806f440 msgr2=0x7fa9d815e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:21.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 shutdown_connections 2024-09-10T10:01:21.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:21.904+0000 7fa9dcc00640 1 -- 172.21.15.110:0/2549687639 wait complete. 2024-09-10T10:01:21.927 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:01:22.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a27600640 1 Processor -- start 2024-09-10T10:01:22.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a27600640 1 -- start start 2024-09-10T10:01:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a27600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 0x7f5a2015f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a27600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5a200ae840 0x7f5a200b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a27600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a200b1e90 con 0x7f5a2006e140 2024-09-10T10:01:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a27600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5a200d00c0 con 0x7f5a200ae840 2024-09-10T10:01:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a27600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5a200b21c0 con 0x7f5a200cfce0 2024-09-10T10:01:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a1fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5a200ae840 0x7f5a200b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a24c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 0x7f5a2015f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a1fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5a200ae840 0x7f5a200b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:55536/0 (socket says 172.21.15.110:55536) 2024-09-10T10:01:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a1fe00640 1 -- 172.21.15.110:0/2542255475 learned_addr learned my addr 172.21.15.110:0/2542255475 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a25600640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f5a2006e140 0x7f5a200b2650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:37978/0 (socket says 172.21.15.110:37978) 2024-09-10T10:01:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a1fe00640 1 -- 172.21.15.110:0/2542255475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 msgr2=0x7f5a2015f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a1fe00640 1 --2- 172.21.15.110:0/2542255475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 0x7f5a2015f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a1fe00640 1 -- 172.21.15.110:0/2542255475 >> v1:172.21.15.145:6789/0 conn(0x7f5a2006e140 legacy=0x7f5a200b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a1fe00640 1 -- 172.21.15.110:0/2542255475 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a200b43d0 con 0x7f5a200ae840 2024-09-10T10:01:22.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.064+0000 7f5a1f400640 1 -- 172.21.15.110:0/2542255475 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3132669963 0 0) 0x7f5a200b1e90 con 0x7f5a2006e140 2024-09-10T10:01:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a1fe00640 1 --2- 172.21.15.110:0/2542255475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a200ae840 0x7f5a200b3b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a200d0240 tx=0x7f5a0c01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10cb8444ff83ddd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a1f400640 1 -- 172.21.15.110:0/2542255475 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0ab430 con 0x7f5a200ae840 2024-09-10T10:01:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a1f400640 1 -- 172.21.15.110:0/2542255475 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c0b0070 con 0x7f5a200ae840 2024-09-10T10:01:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a1f400640 1 -- 172.21.15.110:0/2542255475 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0ab7f0 con 0x7f5a200ae840 2024-09-10T10:01:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a27600640 1 -- 172.21.15.110:0/2542255475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a200ae840 msgr2=0x7f5a200b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a27600640 1 --2- 172.21.15.110:0/2542255475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a200ae840 0x7f5a200b3b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a200d0240 tx=0x7f5a0c01c540 comp rx=0 tx=0).stop 2024-09-10T10:01:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a27600640 1 -- 172.21.15.110:0/2542255475 shutdown_connections 2024-09-10T10:01:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a27600640 1 --2- 172.21.15.110:0/2542255475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a200ae840 0x7f5a200b3b80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a27600640 1 --2- 172.21.15.110:0/2542255475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 0x7f5a2015f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a27600640 1 -- 172.21.15.110:0/2542255475 >> 172.21.15.110:0/2542255475 conn(0x7f5a20062e40 msgr2=0x7f5a2006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a27600640 1 -- 172.21.15.110:0/2542255475 shutdown_connections 2024-09-10T10:01:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.068+0000 7f5a27600640 1 -- 172.21.15.110:0/2542255475 wait complete. 2024-09-10T10:01:22.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 Processor -- start 2024-09-10T10:01:22.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 -- start start 2024-09-10T10:01:22.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a2006e140 0x7f5a2014c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a200ae840 0x7f5a201592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 0x7f5a20157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2006e960 con 0x7f5a200ae840 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a200b40c0 con 0x7f5a2006e140 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5a20158210 con 0x7f5a200cfce0 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a200ae840 0x7f5a201592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a200ae840 0x7f5a201592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35772/0 (socket says 172.21.15.110:35772) 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 -- 172.21.15.110:0/1671282585 learned_addr learned my addr 172.21.15.110:0/1671282585 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a25600640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 0x7f5a20157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a24c00640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a2006e140 0x7f5a2014c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 -- 172.21.15.110:0/1671282585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 msgr2=0x7f5a20157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 0x7f5a20157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 -- 172.21.15.110:0/1671282585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a2006e140 msgr2=0x7f5a2014c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a2006e140 0x7f5a2014c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 -- 172.21.15.110:0/1671282585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a20159f90 con 0x7f5a200ae840 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1fe00640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a200ae840 0x7f5a201592d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0ad040 tx=0x7f5a0c0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1cc00640 1 -- 172.21.15.110:0/1671282585 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0abe20 con 0x7f5a200ae840 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1cc00640 1 -- 172.21.15.110:0/1671282585 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c0bd070 con 0x7f5a200ae840 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a201599d0 con 0x7f5a200ae840 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a1cc00640 1 -- 172.21.15.110:0/1671282585 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0b4640 con 0x7f5a200ae840 2024-09-10T10:01:22.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a20153d70 con 0x7f5a200ae840 2024-09-10T10:01:22.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.072+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a20153b90 con 0x7f5a200ae840 2024-09-10T10:01:22.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.076+0000 7f5a1cc00640 1 -- 172.21.15.110:0/1671282585 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f5a0c0b47e0 con 0x7f5a200ae840 2024-09-10T10:01:22.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.076+0000 7f5a1cc00640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5a00067a70 0x7f5a00069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.076+0000 7f5a1cc00640 1 -- 172.21.15.110:0/1671282585 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f5a0c120850 con 0x7f5a200ae840 2024-09-10T10:01:22.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.076+0000 7f5a24c00640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5a00067a70 0x7f5a00069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.076+0000 7f5a24c00640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5a00067a70 0x7f5a00069f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009ac0 tx=0x7f5a10009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.080+0000 7f5a1cc00640 1 -- 172.21.15.110:0/1671282585 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5a20153b90 con 0x7f5a200ae840 2024-09-10T10:01:22.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:22.272+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5a200aed10 con 0x7f5a200ae840 2024-09-10T10:01:23.183 INFO:tasks.ceph.mds.c.smithi110.stderr:2024-09-10T10:01:23.180+0000 7f7a8fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:01:23.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a1cc00640 1 -- 172.21.15.110:0/1671282585 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f5a0c0f5e20 con 0x7f5a200ae840 2024-09-10T10:01:23.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5a00067a70 msgr2=0x7f5a00069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:23.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5a00067a70 0x7f5a00069f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009ac0 tx=0x7f5a10009190 comp rx=0 tx=0).stop 2024-09-10T10:01:23.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a200ae840 msgr2=0x7f5a201592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a200ae840 0x7f5a201592d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0ad040 tx=0x7f5a0c0192e0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 shutdown_connections 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5a00067a70 0x7f5a00069f30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a200cfce0 0x7f5a20157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a200ae840 0x7f5a201592d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 --2- 172.21.15.110:0/1671282585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a2006e140 0x7f5a2014c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 >> 172.21.15.110:0/1671282585 conn(0x7f5a20062e40 msgr2=0x7f5a20152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 shutdown_connections 2024-09-10T10:01:23.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.232+0000 7f5a27600640 1 -- 172.21.15.110:0/1671282585 wait complete. 2024-09-10T10:01:23.257 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:01:23.258 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:01:23.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66dd400640 1 Processor -- start 2024-09-10T10:01:23.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66dd400640 1 -- start start 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66dd400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f66d80b2bb0 0x7f66d80cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66dd400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f66d80cfbc0 0x7f66d80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66dd400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d80b37c0 con 0x7f66d8145f30 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66dd400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f66d80b3aa0 con 0x7f66d80cfbc0 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66dd400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f66d80b3d80 con 0x7f66d80b2bb0 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d6800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f66d8145f30 0x7f66d80b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45900/0 (socket says 172.21.15.110:45900) 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d6800640 1 -- 172.21.15.110:0/3678330234 learned_addr learned my addr 172.21.15.110:0/3678330234 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d7200640 1 --2- 172.21.15.110:0/3678330234 >> v2:172.21.15.110:3300/0 conn(0x7f66d80cfbc0 0x7f66d80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d5400640 1 -- 172.21.15.110:0/3678330234 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3188479544 0 0) 0x7f66d80b37c0 con 0x7f66d8145f30 2024-09-10T10:01:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d5400640 1 -- 172.21.15.110:0/3678330234 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66bc003530 con 0x7f66d8145f30 2024-09-10T10:01:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d5e00640 1 --2- 172.21.15.110:0/3678330234 >> v2:172.21.15.145:3300/0 conn(0x7f66d80b2bb0 0x7f66d80cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d7200640 1 -- 172.21.15.110:0/3678330234 >> v2:172.21.15.145:3300/0 conn(0x7f66d80b2bb0 msgr2=0x7f66d80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d7200640 1 --2- 172.21.15.110:0/3678330234 >> v2:172.21.15.145:3300/0 conn(0x7f66d80b2bb0 0x7f66d80cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d7200640 1 -- 172.21.15.110:0/3678330234 >> v1:172.21.15.110:6789/0 conn(0x7f66d8145f30 legacy=0x7f66d80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d7200640 1 -- 172.21.15.110:0/3678330234 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d8146310 con 0x7f66d80cfbc0 2024-09-10T10:01:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.396+0000 7f66d5e00640 1 --2- 172.21.15.110:0/3678330234 >> v2:172.21.15.145:3300/0 conn(0x7f66d80b2bb0 0x7f66d80cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:23.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66d7200640 1 --2- 172.21.15.110:0/3678330234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d80cfbc0 0x7f66d80b4580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f66d806f4f0 tx=0x7f66d001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f9903010ee8fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:23.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66d5400640 1 -- 172.21.15.110:0/3678330234 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00b2070 con 0x7f66d80cfbc0 2024-09-10T10:01:23.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66d5400640 1 -- 172.21.15.110:0/3678330234 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d00a57f0 con 0x7f66d80cfbc0 2024-09-10T10:01:23.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66d5400640 1 -- 172.21.15.110:0/3678330234 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00a5ba0 con 0x7f66d80cfbc0 2024-09-10T10:01:23.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66dd400640 1 -- 172.21.15.110:0/3678330234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d80cfbc0 msgr2=0x7f66d80b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:23.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66dd400640 1 --2- 172.21.15.110:0/3678330234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d80cfbc0 0x7f66d80b4580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f66d806f4f0 tx=0x7f66d001c930 comp rx=0 tx=0).stop 2024-09-10T10:01:23.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66dd400640 1 -- 172.21.15.110:0/3678330234 shutdown_connections 2024-09-10T10:01:23.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66dd400640 1 --2- 172.21.15.110:0/3678330234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d80cfbc0 0x7f66d80b4580 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66dd400640 1 --2- 172.21.15.110:0/3678330234 >> v2:172.21.15.145:3300/0 conn(0x7f66d80b2bb0 0x7f66d80cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66dd400640 1 -- 172.21.15.110:0/3678330234 >> 172.21.15.110:0/3678330234 conn(0x7f66d8062e40 msgr2=0x7f66d806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:23.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66dd400640 1 -- 172.21.15.110:0/3678330234 shutdown_connections 2024-09-10T10:01:23.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.400+0000 7f66dd400640 1 -- 172.21.15.110:0/3678330234 wait complete. 2024-09-10T10:01:23.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 Processor -- start 2024-09-10T10:01:23.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 -- start start 2024-09-10T10:01:23.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66d80b2bb0 0x7f66d813d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:23.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66d80cfbc0 0x7f66d81669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:23.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d8145f30 0x7f66d8144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:23.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80b5660 con 0x7f66d8145f30 2024-09-10T10:01:23.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80b54e0 con 0x7f66d80cfbc0 2024-09-10T10:01:23.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f66d80b57e0 con 0x7f66d80b2bb0 2024-09-10T10:01:23.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d8145f30 0x7f66d8144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:23.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d8145f30 0x7f66d8144370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35794/0 (socket says 172.21.15.110:35794) 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 -- 172.21.15.110:0/3390620442 learned_addr learned my addr 172.21.15.110:0/3390620442 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d6800640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66d80b2bb0 0x7f66d813d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d5e00640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66d80cfbc0 0x7f66d81669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 -- 172.21.15.110:0/3390620442 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66d80b2bb0 msgr2=0x7f66d813d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66d80b2bb0 0x7f66d813d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 -- 172.21.15.110:0/3390620442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66d80cfbc0 msgr2=0x7f66d81669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66d80cfbc0 0x7f66d81669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 -- 172.21.15.110:0/3390620442 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d80b0290 con 0x7f66d8145f30 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66d7200640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d8145f30 0x7f66d8144370 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f66d0019540 tx=0x7f66d00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66cea00640 1 -- 172.21.15.110:0/3390620442 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00b2050 con 0x7f66d8145f30 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66cea00640 1 -- 172.21.15.110:0/3390620442 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d00be070 con 0x7f66d8145f30 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66cea00640 1 -- 172.21.15.110:0/3390620442 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0009450 con 0x7f66d8145f30 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d80b0550 con 0x7f66d8145f30 2024-09-10T10:01:23.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d80b0b10 con 0x7f66d8145f30 2024-09-10T10:01:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.404+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d80cf680 con 0x7f66d8145f30 2024-09-10T10:01:23.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.408+0000 7f66cea00640 1 -- 172.21.15.110:0/3390620442 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f66d00adcc0 con 0x7f66d8145f30 2024-09-10T10:01:23.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.408+0000 7f66cea00640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66b8067a70 0x7f66b8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:23.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.408+0000 7f66cea00640 1 -- 172.21.15.110:0/3390620442 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f66d01211f0 con 0x7f66d8145f30 2024-09-10T10:01:23.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.408+0000 7f66d6800640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66b8067a70 0x7f66b8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:23.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.408+0000 7f66d6800640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66b8067a70 0x7f66b8069f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f66c8007fd0 tx=0x7f66c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:23.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.412+0000 7f66cea00640 1 -- 172.21.15.110:0/3390620442 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f66d0016520 con 0x7f66d8145f30 2024-09-10T10:01:23.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:23.604+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f66d8140e60 con 0x7f66d8145f30 2024-09-10T10:01:24.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.240+0000 7f66cea00640 1 -- 172.21.15.110:0/3390620442 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f66d00f13c0 con 0x7f66d8145f30 2024-09-10T10:01:24.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66b8067a70 msgr2=0x7f66b8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66b8067a70 0x7f66b8069f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f66c8007fd0 tx=0x7f66c8009040 comp rx=0 tx=0).stop 2024-09-10T10:01:24.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d8145f30 msgr2=0x7f66d8144370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d8145f30 0x7f66d8144370 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f66d0019540 tx=0x7f66d00196d0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 shutdown_connections 2024-09-10T10:01:24.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f66b8067a70 0x7f66b8069f30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d8145f30 0x7f66d8144370 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66d80cfbc0 0x7f66d81669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 --2- 172.21.15.110:0/3390620442 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f66d80b2bb0 0x7f66d813d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 >> 172.21.15.110:0/3390620442 conn(0x7f66d8062e40 msgr2=0x7f66d8141380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:24.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 shutdown_connections 2024-09-10T10:01:24.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.244+0000 7f66dd400640 1 -- 172.21.15.110:0/3390620442 wait complete. 2024-09-10T10:01:24.268 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:01:24.268 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:24.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03eba00640 1 Processor -- start 2024-09-10T10:01:24.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03eba00640 1 -- start start 2024-09-10T10:01:24.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03eba00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f03e4155970 0x7f03e40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03eba00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03e415c560 0x7f03e406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03eba00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e406f430 con 0x7f03e406f010 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03eba00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f03e40b4570 con 0x7f03e415c560 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03eba00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f03e4155d50 con 0x7f03e4155970 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03e3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03e415c560 0x7f03e406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03e3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03e415c560 0x7f03e406ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35806/0 (socket says 172.21.15.110:35806) 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03e3e00640 1 -- 172.21.15.110:0/3614589157 learned_addr learned my addr 172.21.15.110:0/3614589157 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03e9000640 1 --2- 172.21.15.110:0/3614589157 >> v2:172.21.15.145:3300/0 conn(0x7f03e4155970 0x7f03e40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03e3e00640 1 -- 172.21.15.110:0/3614589157 >> v2:172.21.15.145:3300/0 conn(0x7f03e4155970 msgr2=0x7f03e40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03e3e00640 1 --2- 172.21.15.110:0/3614589157 >> v2:172.21.15.145:3300/0 conn(0x7f03e4155970 0x7f03e40cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03e3e00640 1 -- 172.21.15.110:0/3614589157 >> v1:172.21.15.145:6789/0 conn(0x7f03e406f010 legacy=0x7f03e415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:24.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.404+0000 7f03e3e00640 1 -- 172.21.15.110:0/3614589157 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e40b34d0 con 0x7f03e415c560 2024-09-10T10:01:24.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03e3e00640 1 --2- 172.21.15.110:0/3614589157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e406ead0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f03e406cc30 tx=0x7f03d001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0ecaa303fab8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:24.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03e3400640 1 -- 172.21.15.110:0/3614589157 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d00aa650 con 0x7f03e415c560 2024-09-10T10:01:24.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03e3400640 1 -- 172.21.15.110:0/3614589157 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d00af070 con 0x7f03e415c560 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03e3400640 1 -- 172.21.15.110:0/3614589157 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d00aaa10 con 0x7f03e415c560 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03eba00640 1 -- 172.21.15.110:0/3614589157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 msgr2=0x7f03e406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03eba00640 1 --2- 172.21.15.110:0/3614589157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e406ead0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f03e406cc30 tx=0x7f03d001c4f0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03eba00640 1 -- 172.21.15.110:0/3614589157 shutdown_connections 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03eba00640 1 --2- 172.21.15.110:0/3614589157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e406ead0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03eba00640 1 --2- 172.21.15.110:0/3614589157 >> v2:172.21.15.145:3300/0 conn(0x7f03e4155970 0x7f03e40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03eba00640 1 -- 172.21.15.110:0/3614589157 >> 172.21.15.110:0/3614589157 conn(0x7f03e406bc50 msgr2=0x7f03e406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03eba00640 1 -- 172.21.15.110:0/3614589157 shutdown_connections 2024-09-10T10:01:24.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.408+0000 7f03eba00640 1 -- 172.21.15.110:0/3614589157 wait complete. 2024-09-10T10:01:24.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 Processor -- start 2024-09-10T10:01:24.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 -- start start 2024-09-10T10:01:24.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03e406f010 0x7f03e41408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03e4155970 0x7f03e40b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e40b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40cce00 con 0x7f03e415c560 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40b4050 con 0x7f03e4155970 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f03e4168030 con 0x7f03e406f010 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e40b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e40b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35808/0 (socket says 172.21.15.110:35808) 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 -- 172.21.15.110:0/2718714296 learned_addr learned my addr 172.21.15.110:0/2718714296 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e3e00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03e4155970 0x7f03e40b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9000640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03e406f010 0x7f03e41408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 -- 172.21.15.110:0/2718714296 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03e406f010 msgr2=0x7f03e41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03e406f010 0x7f03e41408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 -- 172.21.15.110:0/2718714296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03e4155970 msgr2=0x7f03e40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03e4155970 0x7f03e40b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 -- 172.21.15.110:0/2718714296 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e4156d30 con 0x7f03e415c560 2024-09-10T10:01:24.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e9a00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e40b26f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c730 tx=0x7f03dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:24.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e0c00640 1 -- 172.21.15.110:0/2718714296 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc003b80 con 0x7f03e415c560 2024-09-10T10:01:24.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e0c00640 1 -- 172.21.15.110:0/2718714296 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc003d20 con 0x7f03e415c560 2024-09-10T10:01:24.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03e0c00640 1 -- 172.21.15.110:0/2718714296 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00aa30 con 0x7f03e415c560 2024-09-10T10:01:24.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e4140df0 con 0x7f03e415c560 2024-09-10T10:01:24.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e4143450 con 0x7f03e415c560 2024-09-10T10:01:24.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.412+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e40cc8c0 con 0x7f03e415c560 2024-09-10T10:01:24.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.416+0000 7f03e0c00640 1 -- 172.21.15.110:0/2718714296 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f03dc00abd0 con 0x7f03e415c560 2024-09-10T10:01:24.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.416+0000 7f03e0c00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03c4067a70 0x7f03c4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.416+0000 7f03e0c00640 1 -- 172.21.15.110:0/2718714296 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f03dc085da0 con 0x7f03e415c560 2024-09-10T10:01:24.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.416+0000 7f03e9000640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03c4067a70 0x7f03c4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.416+0000 7f03e9000640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03c4067a70 0x7f03c4069f30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006aa0 tx=0x7f03d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:24.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.420+0000 7f03e0c00640 1 -- 172.21.15.110:0/2718714296 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f03dc055f70 con 0x7f03e415c560 2024-09-10T10:01:24.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.608+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e41654c0 con 0x7f03e415c560 2024-09-10T10:01:24.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03e0c00640 1 -- 172.21.15.110:0/2718714296 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5235 (secure 0 0 0) 0x7f03dc05a270 con 0x7f03e415c560 2024-09-10T10:01:24.624 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:24.624 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":36,"btime":"2024-09-10T10:01:24:235432+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/2645162871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2645162871},{"type":"v1","addr":"172.21.15.145:6834","nonce":2645162871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1524995501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1524995501},{"type":"v1","addr":"172.21.15.145:6835","nonce":1524995501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/2642324353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2642324353},{"type":"v1","addr":"172.21.15.110:6836","nonce":2642324353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:20.586417+0000","modified":"2024-09-10T10:01:23.606983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1333637048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":1333637048},{"type":"v1","addr":"172.21.15.110:6837","nonce":1333637048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5640,"qdb_cluster":[5640]},"id":4}]} 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 36 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03c4067a70 msgr2=0x7f03c4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03c4067a70 0x7f03c4069f30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006aa0 tx=0x7f03d4008040 comp rx=0 tx=0).stop 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 msgr2=0x7f03e40b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e40b26f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c730 tx=0x7f03dc00cbf0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 shutdown_connections 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f03c4067a70 0x7f03c4069f30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03e415c560 0x7f03e40b26f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03e4155970 0x7f03e40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 --2- 172.21.15.110:0/2718714296 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03e406f010 0x7f03e41408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 >> 172.21.15.110:0/2718714296 conn(0x7f03e406bc50 msgr2=0x7f03e406e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:24.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 shutdown_connections 2024-09-10T10:01:24.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.612+0000 7f03eba00640 1 -- 172.21.15.110:0/2718714296 wait complete. 2024-09-10T10:01:24.635 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:24.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c28200640 1 Processor -- start 2024-09-10T10:01:24.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c28200640 1 -- start start 2024-09-10T10:01:24.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c28200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2c20162680 0x7f2c20073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c28200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162ac0 0x7f2c200737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c28200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c200706a0 con 0x7f2c20161f90 2024-09-10T10:01:24.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c28200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2c200d0040 con 0x7f2c20162680 2024-09-10T10:01:24.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c28200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2c20163060 con 0x7f2c20162ac0 2024-09-10T10:01:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c26200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162ac0 0x7f2c200737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c24e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2c20162680 0x7f2c20073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c25800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f2c20161f90 0x7f2c200cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:37986/0 (socket says 172.21.15.110:37986) 2024-09-10T10:01:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c25800640 1 -- 172.21.15.110:0/2245224665 learned_addr learned my addr 172.21.15.110:0/2245224665 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c26200640 1 -- 172.21.15.110:0/2245224665 >> v2:172.21.15.145:3300/0 conn(0x7f2c20162680 msgr2=0x7f2c20073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:24.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c26200640 1 --2- 172.21.15.110:0/2245224665 >> v2:172.21.15.145:3300/0 conn(0x7f2c20162680 0x7f2c20073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c26200640 1 -- 172.21.15.110:0/2245224665 >> v1:172.21.15.145:6789/0 conn(0x7f2c20161f90 legacy=0x7f2c200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c26200640 1 -- 172.21.15.110:0/2245224665 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c20163340 con 0x7f2c20162ac0 2024-09-10T10:01:24.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.768+0000 7f2c24e00640 1 --2- 172.21.15.110:0/2245224665 >> v2:172.21.15.145:3300/0 conn(0x7f2c20162680 0x7f2c20073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:24.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c26200640 1 --2- 172.21.15.110:0/2245224665 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162ac0 0x7f2c200737d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009d90 tx=0x7f2c1801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23baac705a33d052 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:24.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c1fe00640 1 -- 172.21.15.110:0/2245224665 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c180b2070 con 0x7f2c20162ac0 2024-09-10T10:01:24.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c1fe00640 1 -- 172.21.15.110:0/2245224665 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c180a5830 con 0x7f2c20162ac0 2024-09-10T10:01:24.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c1fe00640 1 -- 172.21.15.110:0/2245224665 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c180a5c00 con 0x7f2c20162ac0 2024-09-10T10:01:24.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c28200640 1 -- 172.21.15.110:0/2245224665 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162ac0 msgr2=0x7f2c200737d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c28200640 1 --2- 172.21.15.110:0/2245224665 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162ac0 0x7f2c200737d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009d90 tx=0x7f2c1801c970 comp rx=0 tx=0).stop 2024-09-10T10:01:24.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c28200640 1 -- 172.21.15.110:0/2245224665 shutdown_connections 2024-09-10T10:01:24.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c28200640 1 --2- 172.21.15.110:0/2245224665 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162ac0 0x7f2c200737d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c28200640 1 --2- 172.21.15.110:0/2245224665 >> v2:172.21.15.145:3300/0 conn(0x7f2c20162680 0x7f2c20073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c28200640 1 -- 172.21.15.110:0/2245224665 >> 172.21.15.110:0/2245224665 conn(0x7f2c200d25c0 msgr2=0x7f2c200d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:24.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c28200640 1 -- 172.21.15.110:0/2245224665 shutdown_connections 2024-09-10T10:01:24.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.772+0000 7f2c28200640 1 -- 172.21.15.110:0/2245224665 wait complete. 2024-09-10T10:01:24.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 Processor -- start 2024-09-10T10:01:24.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 -- start start 2024-09-10T10:01:24.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c20161f90 0x7f2c2015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162680 0x7f2c20154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c20162ac0 0x7f2c2015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2006dec0 con 0x7f2c20161f90 2024-09-10T10:01:24.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2006dd40 con 0x7f2c20162ac0 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2006e040 con 0x7f2c20162680 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c20161f90 0x7f2c2015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c20161f90 0x7f2c2015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35812/0 (socket says 172.21.15.110:35812) 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 -- 172.21.15.110:0/2283807447 learned_addr learned my addr 172.21.15.110:0/2283807447 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c26200640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c20162ac0 0x7f2c2015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c24e00640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162680 0x7f2c20154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 -- 172.21.15.110:0/2283807447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162680 msgr2=0x7f2c20154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162680 0x7f2c20154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 -- 172.21.15.110:0/2283807447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c20162ac0 msgr2=0x7f2c2015b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c20162ac0 0x7f2c2015b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 -- 172.21.15.110:0/2283807447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c20155ee0 con 0x7f2c20161f90 2024-09-10T10:01:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c25800640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c20161f90 0x7f2c2015cde0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c370 tx=0x7f2c1400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:24.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c1d600640 1 -- 172.21.15.110:0/2283807447 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14017070 con 0x7f2c20161f90 2024-09-10T10:01:24.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c1d600640 1 -- 172.21.15.110:0/2283807447 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c14005cf0 con 0x7f2c20161f90 2024-09-10T10:01:24.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c1d600640 1 -- 172.21.15.110:0/2283807447 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c140072f0 con 0x7f2c20161f90 2024-09-10T10:01:24.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2015c440 con 0x7f2c20161f90 2024-09-10T10:01:24.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2015ca00 con 0x7f2c20161f90 2024-09-10T10:01:24.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.776+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c200cf930 con 0x7f2c20161f90 2024-09-10T10:01:24.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.780+0000 7f2c1d600640 1 -- 172.21.15.110:0/2283807447 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f2c1401a070 con 0x7f2c20161f90 2024-09-10T10:01:24.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.780+0000 7f2c1d600640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2c00067aa0 0x7f2c00069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:24.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.780+0000 7f2c1d600640 1 -- 172.21.15.110:0/2283807447 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2c140852c0 con 0x7f2c20161f90 2024-09-10T10:01:24.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.780+0000 7f2c24e00640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2c00067aa0 0x7f2c00069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:24.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.780+0000 7f2c24e00640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2c00067aa0 0x7f2c00069f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009580 tx=0x7f2c0c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:24.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.784+0000 7f2c1d600640 1 -- 172.21.15.110:0/2283807447 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2c140554c0 con 0x7f2c20161f90 2024-09-10T10:01:24.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.944+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c200d0a10 con 0x7f2c20161f90 2024-09-10T10:01:24.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.944+0000 7f2c1d600640 1 -- 172.21.15.110:0/2283807447 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16761 (secure 0 0 0) 0x7f2c140597c0 con 0x7f2c20161f90 2024-09-10T10:01:24.949 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:24.949 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":58,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:22.219663+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:18.299917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:19.590875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2c00067aa0 msgr2=0x7f2c00069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2c00067aa0 0x7f2c00069f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009580 tx=0x7f2c0c009190 comp rx=0 tx=0).stop 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c20161f90 msgr2=0x7f2c2015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c20161f90 0x7f2c2015cde0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c370 tx=0x7f2c1400c830 comp rx=0 tx=0).stop 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 shutdown_connections 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2c00067aa0 0x7f2c00069f60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c20162ac0 0x7f2c2015b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c20162680 0x7f2c20154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 --2- 172.21.15.110:0/2283807447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c20161f90 0x7f2c2015cde0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:24.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 >> 172.21.15.110:0/2283807447 conn(0x7f2c200d25c0 msgr2=0x7f2c200b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:24.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 shutdown_connections 2024-09-10T10:01:24.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:24.948+0000 7f2c28200640 1 -- 172.21.15.110:0/2283807447 wait complete. 2024-09-10T10:01:24.970 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:01:24.970 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:01:25.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.100+0000 7f23ccc00640 1 Processor -- start 2024-09-10T10:01:25.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.100+0000 7f23ccc00640 1 -- start start 2024-09-10T10:01:25.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23ccc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f23c815c560 0x7f23c806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23ccc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 0x7f23c80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23ccc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c806f430 con 0x7f23c8155970 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23ccc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f23c80b4570 con 0x7f23c815c560 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23ccc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f23c8155d50 con 0x7f23c806f010 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23c5400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f23c815c560 0x7f23c806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23c5400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f23c815c560 0x7f23c806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35828/0 (socket says 172.21.15.110:35828) 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23c5400640 1 -- 172.21.15.110:0/549574950 learned_addr learned my addr 172.21.15.110:0/549574950 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23c6800640 1 --2- 172.21.15.110:0/549574950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 0x7f23c80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23c5400640 1 -- 172.21.15.110:0/549574950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 msgr2=0x7f23c80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23c5400640 1 --2- 172.21.15.110:0/549574950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 0x7f23c80ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23c5400640 1 -- 172.21.15.110:0/549574950 >> v1:172.21.15.145:6789/0 conn(0x7f23c8155970 legacy=0x7f23c80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.104+0000 7f23c5400640 1 -- 172.21.15.110:0/549574950 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c80b34d0 con 0x7f23c815c560 2024-09-10T10:01:25.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23c5400640 1 --2- 172.21.15.110:0/549574950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c815c560 0x7f23c806ead0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009870 tx=0x7f23b401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d315ba7322c9b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23c4a00640 1 -- 172.21.15.110:0/549574950 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40a95f0 con 0x7f23c815c560 2024-09-10T10:01:25.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23c4a00640 1 -- 172.21.15.110:0/549574950 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b40a9790 con 0x7f23c815c560 2024-09-10T10:01:25.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23c4a00640 1 -- 172.21.15.110:0/549574950 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40a9b60 con 0x7f23c815c560 2024-09-10T10:01:25.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- 172.21.15.110:0/549574950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c815c560 msgr2=0x7f23c806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 --2- 172.21.15.110:0/549574950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c815c560 0x7f23c806ead0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009870 tx=0x7f23b401c530 comp rx=0 tx=0).stop 2024-09-10T10:01:25.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- 172.21.15.110:0/549574950 shutdown_connections 2024-09-10T10:01:25.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 --2- 172.21.15.110:0/549574950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 0x7f23c80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 --2- 172.21.15.110:0/549574950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c815c560 0x7f23c806ead0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- 172.21.15.110:0/549574950 >> 172.21.15.110:0/549574950 conn(0x7f23c806bc50 msgr2=0x7f23c806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:25.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- 172.21.15.110:0/549574950 shutdown_connections 2024-09-10T10:01:25.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- 172.21.15.110:0/549574950 wait complete. 2024-09-10T10:01:25.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 Processor -- start 2024-09-10T10:01:25.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- start start 2024-09-10T10:01:25.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 0x7f23c8151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c8155970 0x7f23c81638b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c815c560 0x7f23c814a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23c80cf170 con 0x7f23c8155970 2024-09-10T10:01:25.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f23c80b4050 con 0x7f23c815c560 2024-09-10T10:01:25.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23ccc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f23c80cf2f0 con 0x7f23c806f010 2024-09-10T10:01:25.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23c5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c8155970 0x7f23c81638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23c5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c8155970 0x7f23c81638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35844/0 (socket says 172.21.15.110:35844) 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23c5400640 1 -- 172.21.15.110:0/1404150729 learned_addr learned my addr 172.21.15.110:0/1404150729 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.108+0000 7f23c5e00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 0x7f23c8151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23c6800640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c815c560 0x7f23c814a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23c5400640 1 -- 172.21.15.110:0/1404150729 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 msgr2=0x7f23c8151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23c5400640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 0x7f23c8151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23c5400640 1 -- 172.21.15.110:0/1404150729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c815c560 msgr2=0x7f23c814a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23c5400640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c815c560 0x7f23c814a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23c5400640 1 -- 172.21.15.110:0/1404150729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c8150660 con 0x7f23c8155970 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23c5400640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c8155970 0x7f23c81638b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f23b401abe0 tx=0x7f23b40192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23be000640 1 -- 172.21.15.110:0/1404150729 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40b1800 con 0x7f23c8155970 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23be000640 1 -- 172.21.15.110:0/1404150729 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b40b19a0 con 0x7f23c8155970 2024-09-10T10:01:25.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c8150940 con 0x7f23c8155970 2024-09-10T10:01:25.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23be000640 1 -- 172.21.15.110:0/1404150729 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40b1d70 con 0x7f23c8155970 2024-09-10T10:01:25.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c8160220 con 0x7f23c8155970 2024-09-10T10:01:25.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c8160420 con 0x7f23c8155970 2024-09-10T10:01:25.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23be000640 1 -- 172.21.15.110:0/1404150729 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f23b40b9420 con 0x7f23c8155970 2024-09-10T10:01:25.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23be000640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23a8067a70 0x7f23a8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.112+0000 7f23be000640 1 -- 172.21.15.110:0/1404150729 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f23b411fd20 con 0x7f23c8155970 2024-09-10T10:01:25.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.116+0000 7f23c5e00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23a8067a70 0x7f23a8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.116+0000 7f23c5e00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23a8067a70 0x7f23a8069f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23b8007fd0 tx=0x7f23b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.120+0000 7f23be000640 1 -- 172.21.15.110:0/1404150729 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f23c8160420 con 0x7f23c8155970 2024-09-10T10:01:25.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.276+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23c80ce420 con 0x7f23a8067a70 2024-09-10T10:01:25.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.284+0000 7f23be000640 1 -- 172.21.15.110:0/1404150729 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251939 (secure 0 0 0) 0x7f23c80ce420 con 0x7f23a8067a70 2024-09-10T10:01:25.288 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:25.290 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:01:25.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23a8067a70 msgr2=0x7f23a8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23a8067a70 0x7f23a8069f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23b8007fd0 tx=0x7f23b8009040 comp rx=0 tx=0).stop 2024-09-10T10:01:25.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c8155970 msgr2=0x7f23c81638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c8155970 0x7f23c81638b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f23b401abe0 tx=0x7f23b40192d0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 shutdown_connections 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f23a8067a70 0x7f23a8069f30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c815c560 0x7f23c814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23c8155970 0x7f23c81638b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 --2- 172.21.15.110:0/1404150729 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23c806f010 0x7f23c8151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 >> 172.21.15.110:0/1404150729 conn(0x7f23c806bc50 msgr2=0x7f23c806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 shutdown_connections 2024-09-10T10:01:25.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.288+0000 7f23ccc00640 1 -- 172.21.15.110:0/1404150729 wait complete. 2024-09-10T10:01:25.309 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T10:01:23.392888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243576,"kb_used_data":6528,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731144,"statfs":{"total":773094113280,"available":772844691456,"internally_reserved":0,"allocated":6684672,"data_stored":3409268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":242535980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.194262"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566855+0000","last_change":"2024-09-10T10:01:21.227763+0000","last_active":"2024-09-10T10:01:22.566855+0000","last_peered":"2024-09-10T10:01:22.566855+0000","last_clean":"2024-09-10T10:01:22.566855+0000","last_became_active":"2024-09-10T10:01:21.227445+0000","last_became_peered":"2024-09-10T10:01:21.227445+0000","last_unstale":"2024-09-10T10:01:22.566855+0000","last_undegraded":"2024-09-10T10:01:22.566855+0000","last_fullsized":"2024-09-10T10:01:22.566855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:00.108623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.229578+0000","last_change":"2024-09-10T10:01:21.226592+0000","last_active":"2024-09-10T10:01:21.229578+0000","last_peered":"2024-09-10T10:01:21.229578+0000","last_clean":"2024-09-10T10:01:21.229578+0000","last_became_active":"2024-09-10T10:01:21.226300+0000","last_became_peered":"2024-09-10T10:01:21.226300+0000","last_unstale":"2024-09-10T10:01:21.229578+0000","last_undegraded":"2024-09-10T10:01:21.229578+0000","last_fullsized":"2024-09-10T10:01:21.229578+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:26.797979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568213+0000","last_change":"2024-09-10T10:01:21.239647+0000","last_active":"2024-09-10T10:01:22.568213+0000","last_peered":"2024-09-10T10:01:22.568213+0000","last_clean":"2024-09-10T10:01:22.568213+0000","last_became_active":"2024-09-10T10:01:21.239370+0000","last_became_peered":"2024-09-10T10:01:21.239370+0000","last_unstale":"2024-09-10T10:01:22.568213+0000","last_undegraded":"2024-09-10T10:01:22.568213+0000","last_fullsized":"2024-09-10T10:01:22.568213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:28:31.791724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567762+0000","last_change":"2024-09-10T10:01:21.229770+0000","last_active":"2024-09-10T10:01:22.567762+0000","last_peered":"2024-09-10T10:01:22.567762+0000","last_clean":"2024-09-10T10:01:22.567762+0000","last_became_active":"2024-09-10T10:01:21.229499+0000","last_became_peered":"2024-09-10T10:01:21.229499+0000","last_unstale":"2024-09-10T10:01:22.567762+0000","last_undegraded":"2024-09-10T10:01:22.567762+0000","last_fullsized":"2024-09-10T10:01:22.567762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:43.234130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566611+0000","last_change":"2024-09-10T10:01:21.238238+0000","last_active":"2024-09-10T10:01:22.566611+0000","last_peered":"2024-09-10T10:01:22.566611+0000","last_clean":"2024-09-10T10:01:22.566611+0000","last_became_active":"2024-09-10T10:01:21.237676+0000","last_became_peered":"2024-09-10T10:01:21.237676+0000","last_unstale":"2024-09-10T10:01:22.566611+0000","last_undegraded":"2024-09-10T10:01:22.566611+0000","last_fullsized":"2024-09-10T10:01:22.566611+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:38.768949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567035+0000","last_change":"2024-09-10T10:01:21.236979+0000","last_active":"2024-09-10T10:01:22.567035+0000","last_peered":"2024-09-10T10:01:22.567035+0000","last_clean":"2024-09-10T10:01:22.567035+0000","last_became_active":"2024-09-10T10:01:21.236685+0000","last_became_peered":"2024-09-10T10:01:21.236685+0000","last_unstale":"2024-09-10T10:01:22.567035+0000","last_undegraded":"2024-09-10T10:01:22.567035+0000","last_fullsized":"2024-09-10T10:01:22.567035+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.508961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.231852+0000","last_change":"2024-09-10T10:01:21.237900+0000","last_active":"2024-09-10T10:01:22.231852+0000","last_peered":"2024-09-10T10:01:22.231852+0000","last_clean":"2024-09-10T10:01:22.231852+0000","last_became_active":"2024-09-10T10:01:21.237486+0000","last_became_peered":"2024-09-10T10:01:21.237486+0000","last_unstale":"2024-09-10T10:01:22.231852+0000","last_undegraded":"2024-09-10T10:01:22.231852+0000","last_fullsized":"2024-09-10T10:01:22.231852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:38.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235191+0000","last_change":"2024-09-10T10:01:21.227855+0000","last_active":"2024-09-10T10:01:21.235191+0000","last_peered":"2024-09-10T10:01:21.235191+0000","last_clean":"2024-09-10T10:01:21.235191+0000","last_became_active":"2024-09-10T10:01:21.225768+0000","last_became_peered":"2024-09-10T10:01:21.225768+0000","last_unstale":"2024-09-10T10:01:21.235191+0000","last_undegraded":"2024-09-10T10:01:21.235191+0000","last_fullsized":"2024-09-10T10:01:21.235191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:43.783217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232922+0000","last_change":"2024-09-10T10:01:21.230481+0000","last_active":"2024-09-10T10:01:22.232922+0000","last_peered":"2024-09-10T10:01:22.232922+0000","last_clean":"2024-09-10T10:01:22.232922+0000","last_became_active":"2024-09-10T10:01:21.230207+0000","last_became_peered":"2024-09-10T10:01:21.230207+0000","last_unstale":"2024-09-10T10:01:22.232922+0000","last_undegraded":"2024-09-10T10:01:22.232922+0000","last_fullsized":"2024-09-10T10:01:22.232922+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:00.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227356+0000","last_change":"2024-09-10T10:01:21.236084+0000","last_active":"2024-09-10T10:01:22.227356+0000","last_peered":"2024-09-10T10:01:22.227356+0000","last_clean":"2024-09-10T10:01:22.227356+0000","last_became_active":"2024-09-10T10:01:21.235775+0000","last_became_peered":"2024-09-10T10:01:21.235775+0000","last_unstale":"2024-09-10T10:01:22.227356+0000","last_undegraded":"2024-09-10T10:01:22.227356+0000","last_fullsized":"2024-09-10T10:01:22.227356+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:14.698590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227335+0000","last_change":"2024-09-10T10:01:21.227181+0000","last_active":"2024-09-10T10:01:22.227335+0000","last_peered":"2024-09-10T10:01:22.227335+0000","last_clean":"2024-09-10T10:01:22.227335+0000","last_became_active":"2024-09-10T10:01:21.226894+0000","last_became_peered":"2024-09-10T10:01:21.226894+0000","last_unstale":"2024-09-10T10:01:22.227335+0000","last_undegraded":"2024-09-10T10:01:22.227335+0000","last_fullsized":"2024-09-10T10:01:22.227335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:05.481366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567587+0000","last_change":"2024-09-10T10:01:21.229191+0000","last_active":"2024-09-10T10:01:22.567587+0000","last_peered":"2024-09-10T10:01:22.567587+0000","last_clean":"2024-09-10T10:01:22.567587+0000","last_became_active":"2024-09-10T10:01:21.228882+0000","last_became_peered":"2024-09-10T10:01:21.228882+0000","last_unstale":"2024-09-10T10:01:22.567587+0000","last_undegraded":"2024-09-10T10:01:22.567587+0000","last_fullsized":"2024-09-10T10:01:22.567587+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:08.856667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.233355+0000","last_change":"2024-09-10T10:01:21.227224+0000","last_active":"2024-09-10T10:01:22.233355+0000","last_peered":"2024-09-10T10:01:22.233355+0000","last_clean":"2024-09-10T10:01:22.233355+0000","last_became_active":"2024-09-10T10:01:21.226946+0000","last_became_peered":"2024-09-10T10:01:21.226946+0000","last_unstale":"2024-09-10T10:01:22.233355+0000","last_undegraded":"2024-09-10T10:01:22.233355+0000","last_fullsized":"2024-09-10T10:01:22.233355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:05.481366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232144+0000","last_change":"2024-09-10T10:01:21.239230+0000","last_active":"2024-09-10T10:01:22.232144+0000","last_peered":"2024-09-10T10:01:22.232144+0000","last_clean":"2024-09-10T10:01:22.232144+0000","last_became_active":"2024-09-10T10:01:21.238702+0000","last_became_peered":"2024-09-10T10:01:21.238702+0000","last_unstale":"2024-09-10T10:01:22.232144+0000","last_undegraded":"2024-09-10T10:01:22.232144+0000","last_fullsized":"2024-09-10T10:01:22.232144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:57.619556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232063+0000","last_change":"2024-09-10T10:01:21.226688+0000","last_active":"2024-09-10T10:01:22.232063+0000","last_peered":"2024-09-10T10:01:22.232063+0000","last_clean":"2024-09-10T10:01:22.232063+0000","last_became_active":"2024-09-10T10:01:21.226305+0000","last_became_peered":"2024-09-10T10:01:21.226305+0000","last_unstale":"2024-09-10T10:01:22.232063+0000","last_undegraded":"2024-09-10T10:01:22.232063+0000","last_fullsized":"2024-09-10T10:01:22.232063+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:19.047000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566969+0000","last_change":"2024-09-10T10:01:21.230703+0000","last_active":"2024-09-10T10:01:22.566969+0000","last_peered":"2024-09-10T10:01:22.566969+0000","last_clean":"2024-09-10T10:01:22.566969+0000","last_became_active":"2024-09-10T10:01:21.225406+0000","last_became_peered":"2024-09-10T10:01:21.225406+0000","last_unstale":"2024-09-10T10:01:22.566969+0000","last_undegraded":"2024-09-10T10:01:22.566969+0000","last_fullsized":"2024-09-10T10:01:22.566969+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:07.802751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235028+0000","last_change":"2024-09-10T10:01:21.229294+0000","last_active":"2024-09-10T10:01:21.235028+0000","last_peered":"2024-09-10T10:01:21.235028+0000","last_clean":"2024-09-10T10:01:21.235028+0000","last_became_active":"2024-09-10T10:01:21.228379+0000","last_became_peered":"2024-09-10T10:01:21.228379+0000","last_unstale":"2024-09-10T10:01:21.235028+0000","last_undegraded":"2024-09-10T10:01:21.235028+0000","last_fullsized":"2024-09-10T10:01:21.235028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:40.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568084+0000","last_change":"2024-09-10T10:01:21.244357+0000","last_active":"2024-09-10T10:01:22.568084+0000","last_peered":"2024-09-10T10:01:22.568084+0000","last_clean":"2024-09-10T10:01:22.568084+0000","last_became_active":"2024-09-10T10:01:21.244128+0000","last_became_peered":"2024-09-10T10:01:21.244128+0000","last_unstale":"2024-09-10T10:01:22.568084+0000","last_undegraded":"2024-09-10T10:01:22.568084+0000","last_fullsized":"2024-09-10T10:01:22.568084+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:04.680093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566785+0000","last_change":"2024-09-10T10:01:21.231413+0000","last_active":"2024-09-10T10:01:22.566785+0000","last_peered":"2024-09-10T10:01:22.566785+0000","last_clean":"2024-09-10T10:01:22.566785+0000","last_became_active":"2024-09-10T10:01:21.231156+0000","last_became_peered":"2024-09-10T10:01:21.231156+0000","last_unstale":"2024-09-10T10:01:22.566785+0000","last_undegraded":"2024-09-10T10:01:22.566785+0000","last_fullsized":"2024-09-10T10:01:22.566785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:51.151589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.243895+0000","last_change":"2024-09-10T10:01:21.242658+0000","last_active":"2024-09-10T10:01:21.243895+0000","last_peered":"2024-09-10T10:01:21.243895+0000","last_clean":"2024-09-10T10:01:21.243895+0000","last_became_active":"2024-09-10T10:01:21.242095+0000","last_became_peered":"2024-09-10T10:01:21.242095+0000","last_unstale":"2024-09-10T10:01:21.243895+0000","last_undegraded":"2024-09-10T10:01:21.243895+0000","last_fullsized":"2024-09-10T10:01:21.243895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:24.070238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567593+0000","last_change":"2024-09-10T10:01:21.247499+0000","last_active":"2024-09-10T10:01:22.567593+0000","last_peered":"2024-09-10T10:01:22.567593+0000","last_clean":"2024-09-10T10:01:22.567593+0000","last_became_active":"2024-09-10T10:01:21.246815+0000","last_became_peered":"2024-09-10T10:01:21.246815+0000","last_unstale":"2024-09-10T10:01:22.567593+0000","last_undegraded":"2024-09-10T10:01:22.567593+0000","last_fullsized":"2024-09-10T10:01:22.567593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:25.977903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.241962+0000","last_change":"2024-09-10T10:01:21.240088+0000","last_active":"2024-09-10T10:01:21.241962+0000","last_peered":"2024-09-10T10:01:21.241962+0000","last_clean":"2024-09-10T10:01:21.241962+0000","last_became_active":"2024-09-10T10:01:21.239856+0000","last_became_peered":"2024-09-10T10:01:21.239856+0000","last_unstale":"2024-09-10T10:01:21.241962+0000","last_undegraded":"2024-09-10T10:01:21.241962+0000","last_fullsized":"2024-09-10T10:01:21.241962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:28.623836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567239+0000","last_change":"2024-09-10T10:01:21.248432+0000","last_active":"2024-09-10T10:01:22.567239+0000","last_peered":"2024-09-10T10:01:22.567239+0000","last_clean":"2024-09-10T10:01:22.567239+0000","last_became_active":"2024-09-10T10:01:21.248123+0000","last_became_peered":"2024-09-10T10:01:21.248123+0000","last_unstale":"2024-09-10T10:01:22.567239+0000","last_undegraded":"2024-09-10T10:01:22.567239+0000","last_fullsized":"2024-09-10T10:01:22.567239+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:17.036026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232454+0000","last_change":"2024-09-10T10:01:21.227695+0000","last_active":"2024-09-10T10:01:22.232454+0000","last_peered":"2024-09-10T10:01:22.232454+0000","last_clean":"2024-09-10T10:01:22.232454+0000","last_became_active":"2024-09-10T10:01:21.227220+0000","last_became_peered":"2024-09-10T10:01:21.227220+0000","last_unstale":"2024-09-10T10:01:22.232454+0000","last_undegraded":"2024-09-10T10:01:22.232454+0000","last_fullsized":"2024-09-10T10:01:22.232454+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:40.905046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566986+0000","last_change":"2024-09-10T10:01:21.234111+0000","last_active":"2024-09-10T10:01:22.566986+0000","last_peered":"2024-09-10T10:01:22.566986+0000","last_clean":"2024-09-10T10:01:22.566986+0000","last_became_active":"2024-09-10T10:01:21.233822+0000","last_became_peered":"2024-09-10T10:01:21.233822+0000","last_unstale":"2024-09-10T10:01:22.566986+0000","last_undegraded":"2024-09-10T10:01:22.566986+0000","last_fullsized":"2024-09-10T10:01:22.566986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:32.111103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568085+0000","last_change":"2024-09-10T10:01:21.236963+0000","last_active":"2024-09-10T10:01:22.568085+0000","last_peered":"2024-09-10T10:01:22.568085+0000","last_clean":"2024-09-10T10:01:22.568085+0000","last_became_active":"2024-09-10T10:01:21.236409+0000","last_became_peered":"2024-09-10T10:01:21.236409+0000","last_unstale":"2024-09-10T10:01:22.568085+0000","last_undegraded":"2024-09-10T10:01:22.568085+0000","last_fullsized":"2024-09-10T10:01:22.568085+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:57.393646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567283+0000","last_change":"2024-09-10T10:01:21.230230+0000","last_active":"2024-09-10T10:01:22.567283+0000","last_peered":"2024-09-10T10:01:22.567283+0000","last_clean":"2024-09-10T10:01:22.567283+0000","last_became_active":"2024-09-10T10:01:21.229933+0000","last_became_peered":"2024-09-10T10:01:21.229933+0000","last_unstale":"2024-09-10T10:01:22.567283+0000","last_undegraded":"2024-09-10T10:01:22.567283+0000","last_fullsized":"2024-09-10T10:01:22.567283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:04.680093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567512+0000","last_change":"2024-09-10T10:01:21.236959+0000","last_active":"2024-09-10T10:01:22.567512+0000","last_peered":"2024-09-10T10:01:22.567512+0000","last_clean":"2024-09-10T10:01:22.567512+0000","last_became_active":"2024-09-10T10:01:21.236489+0000","last_became_peered":"2024-09-10T10:01:21.236489+0000","last_unstale":"2024-09-10T10:01:22.567512+0000","last_undegraded":"2024-09-10T10:01:22.567512+0000","last_fullsized":"2024-09-10T10:01:22.567512+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:47.538171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235221+0000","last_change":"2024-09-10T10:01:21.231102+0000","last_active":"2024-09-10T10:01:21.235221+0000","last_peered":"2024-09-10T10:01:21.235221+0000","last_clean":"2024-09-10T10:01:21.235221+0000","last_became_active":"2024-09-10T10:01:21.230805+0000","last_became_peered":"2024-09-10T10:01:21.230805+0000","last_unstale":"2024-09-10T10:01:21.235221+0000","last_undegraded":"2024-09-10T10:01:21.235221+0000","last_fullsized":"2024-09-10T10:01:21.235221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:45.805042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568025+0000","last_change":"2024-09-10T10:01:21.240643+0000","last_active":"2024-09-10T10:01:22.568025+0000","last_peered":"2024-09-10T10:01:22.568025+0000","last_clean":"2024-09-10T10:01:22.568025+0000","last_became_active":"2024-09-10T10:01:21.240361+0000","last_became_peered":"2024-09-10T10:01:21.240361+0000","last_unstale":"2024-09-10T10:01:22.568025+0000","last_undegraded":"2024-09-10T10:01:22.568025+0000","last_fullsized":"2024-09-10T10:01:22.568025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:21.760367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567364+0000","last_change":"2024-09-10T10:01:21.229612+0000","last_active":"2024-09-10T10:01:22.567364+0000","last_peered":"2024-09-10T10:01:22.567364+0000","last_clean":"2024-09-10T10:01:22.567364+0000","last_became_active":"2024-09-10T10:01:21.229354+0000","last_became_peered":"2024-09-10T10:01:21.229354+0000","last_unstale":"2024-09-10T10:01:22.567364+0000","last_undegraded":"2024-09-10T10:01:22.567364+0000","last_fullsized":"2024-09-10T10:01:22.567364+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:47.107132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566812+0000","last_change":"2024-09-10T10:01:21.231435+0000","last_active":"2024-09-10T10:01:22.566812+0000","last_peered":"2024-09-10T10:01:22.566812+0000","last_clean":"2024-09-10T10:01:22.566812+0000","last_became_active":"2024-09-10T10:01:21.230787+0000","last_became_peered":"2024-09-10T10:01:21.230787+0000","last_unstale":"2024-09-10T10:01:22.566812+0000","last_undegraded":"2024-09-10T10:01:22.566812+0000","last_fullsized":"2024-09-10T10:01:22.566812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:01.053965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567766+0000","last_change":"2024-09-10T10:01:21.236903+0000","last_active":"2024-09-10T10:01:22.567766+0000","last_peered":"2024-09-10T10:01:22.567766+0000","last_clean":"2024-09-10T10:01:22.567766+0000","last_became_active":"2024-09-10T10:01:21.236494+0000","last_became_peered":"2024-09-10T10:01:21.236494+0000","last_unstale":"2024-09-10T10:01:22.567766+0000","last_undegraded":"2024-09-10T10:01:22.567766+0000","last_fullsized":"2024-09-10T10:01:22.567766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:15.249762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567910+0000","last_change":"2024-09-10T10:01:20.227440+0000","last_active":"2024-09-10T10:01:22.567910+0000","last_peered":"2024-09-10T10:01:22.567910+0000","last_clean":"2024-09-10T10:01:22.567910+0000","last_became_active":"2024-09-10T10:01:20.227148+0000","last_became_peered":"2024-09-10T10:01:20.227148+0000","last_unstale":"2024-09-10T10:01:22.567910+0000","last_undegraded":"2024-09-10T10:01:22.567910+0000","last_fullsized":"2024-09-10T10:01:22.567910+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:59.541177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227537+0000","last_change":"2024-09-10T10:01:21.247156+0000","last_active":"2024-09-10T10:01:22.227537+0000","last_peered":"2024-09-10T10:01:22.227537+0000","last_clean":"2024-09-10T10:01:22.227537+0000","last_became_active":"2024-09-10T10:01:21.246848+0000","last_became_peered":"2024-09-10T10:01:21.246848+0000","last_unstale":"2024-09-10T10:01:22.227537+0000","last_undegraded":"2024-09-10T10:01:22.227537+0000","last_fullsized":"2024-09-10T10:01:22.227537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:09.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.253500+0000","last_change":"2024-09-10T10:01:20.217325+0000","last_active":"2024-09-10T10:01:21.253500+0000","last_peered":"2024-09-10T10:01:21.253500+0000","last_clean":"2024-09-10T10:01:21.253500+0000","last_became_active":"2024-09-10T10:01:20.217047+0000","last_became_peered":"2024-09-10T10:01:20.217047+0000","last_unstale":"2024-09-10T10:01:21.253500+0000","last_undegraded":"2024-09-10T10:01:21.253500+0000","last_fullsized":"2024-09-10T10:01:21.253500+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:18.504805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567415+0000","last_change":"2024-09-10T10:01:21.237965+0000","last_active":"2024-09-10T10:01:22.567415+0000","last_peered":"2024-09-10T10:01:22.567415+0000","last_clean":"2024-09-10T10:01:22.567415+0000","last_became_active":"2024-09-10T10:01:21.231838+0000","last_became_peered":"2024-09-10T10:01:21.231838+0000","last_unstale":"2024-09-10T10:01:22.567415+0000","last_undegraded":"2024-09-10T10:01:22.567415+0000","last_fullsized":"2024-09-10T10:01:22.567415+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:28:31.791724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567702+0000","last_change":"2024-09-10T10:01:20.229396+0000","last_active":"2024-09-10T10:01:22.567702+0000","last_peered":"2024-09-10T10:01:22.567702+0000","last_clean":"2024-09-10T10:01:22.567702+0000","last_became_active":"2024-09-10T10:01:20.229148+0000","last_became_peered":"2024-09-10T10:01:20.229148+0000","last_unstale":"2024-09-10T10:01:22.567702+0000","last_undegraded":"2024-09-10T10:01:22.567702+0000","last_fullsized":"2024-09-10T10:01:22.567702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:18.043310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567429+0000","last_change":"2024-09-10T10:01:21.235093+0000","last_active":"2024-09-10T10:01:22.567429+0000","last_peered":"2024-09-10T10:01:22.567429+0000","last_clean":"2024-09-10T10:01:22.567429+0000","last_became_active":"2024-09-10T10:01:21.234575+0000","last_became_peered":"2024-09-10T10:01:21.234575+0000","last_unstale":"2024-09-10T10:01:22.567429+0000","last_undegraded":"2024-09-10T10:01:22.567429+0000","last_fullsized":"2024-09-10T10:01:22.567429+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:16.260334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567799+0000","last_change":"2024-09-10T10:01:20.228296+0000","last_active":"2024-09-10T10:01:22.567799+0000","last_peered":"2024-09-10T10:01:22.567799+0000","last_clean":"2024-09-10T10:01:22.567799+0000","last_became_active":"2024-09-10T10:01:20.228008+0000","last_became_peered":"2024-09-10T10:01:20.228008+0000","last_unstale":"2024-09-10T10:01:22.567799+0000","last_undegraded":"2024-09-10T10:01:22.567799+0000","last_fullsized":"2024-09-10T10:01:22.567799+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:27.565888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.241941+0000","last_change":"2024-09-10T10:01:21.239574+0000","last_active":"2024-09-10T10:01:21.241941+0000","last_peered":"2024-09-10T10:01:21.241941+0000","last_clean":"2024-09-10T10:01:21.241941+0000","last_became_active":"2024-09-10T10:01:21.239160+0000","last_became_peered":"2024-09-10T10:01:21.239160+0000","last_unstale":"2024-09-10T10:01:21.241941+0000","last_undegraded":"2024-09-10T10:01:21.241941+0000","last_fullsized":"2024-09-10T10:01:21.241941+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:35.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232449+0000","last_change":"2024-09-10T10:01:20.224016+0000","last_active":"2024-09-10T10:01:22.232449+0000","last_peered":"2024-09-10T10:01:22.232449+0000","last_clean":"2024-09-10T10:01:22.232449+0000","last_became_active":"2024-09-10T10:01:20.223210+0000","last_became_peered":"2024-09-10T10:01:20.223210+0000","last_unstale":"2024-09-10T10:01:22.232449+0000","last_undegraded":"2024-09-10T10:01:22.232449+0000","last_fullsized":"2024-09-10T10:01:22.232449+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:08.995562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235240+0000","last_change":"2024-09-10T10:01:21.233432+0000","last_active":"2024-09-10T10:01:21.235240+0000","last_peered":"2024-09-10T10:01:21.235240+0000","last_clean":"2024-09-10T10:01:21.235240+0000","last_became_active":"2024-09-10T10:01:21.233101+0000","last_became_peered":"2024-09-10T10:01:21.233101+0000","last_unstale":"2024-09-10T10:01:21.235240+0000","last_undegraded":"2024-09-10T10:01:21.235240+0000","last_fullsized":"2024-09-10T10:01:21.235240+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:00.545368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567107+0000","last_change":"2024-09-10T10:01:20.223720+0000","last_active":"2024-09-10T10:01:22.567107+0000","last_peered":"2024-09-10T10:01:22.567107+0000","last_clean":"2024-09-10T10:01:22.567107+0000","last_became_active":"2024-09-10T10:01:20.223437+0000","last_became_peered":"2024-09-10T10:01:20.223437+0000","last_unstale":"2024-09-10T10:01:22.567107+0000","last_undegraded":"2024-09-10T10:01:22.567107+0000","last_fullsized":"2024-09-10T10:01:22.567107+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:00.514446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232605+0000","last_change":"2024-09-10T10:01:21.240686+0000","last_active":"2024-09-10T10:01:22.232605+0000","last_peered":"2024-09-10T10:01:22.232605+0000","last_clean":"2024-09-10T10:01:22.232605+0000","last_became_active":"2024-09-10T10:01:21.239623+0000","last_became_peered":"2024-09-10T10:01:21.239623+0000","last_unstale":"2024-09-10T10:01:22.232605+0000","last_undegraded":"2024-09-10T10:01:22.232605+0000","last_fullsized":"2024-09-10T10:01:22.232605+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:17.075965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567619+0000","last_change":"2024-09-10T10:01:20.229878+0000","last_active":"2024-09-10T10:01:22.567619+0000","last_peered":"2024-09-10T10:01:22.567619+0000","last_clean":"2024-09-10T10:01:22.567619+0000","last_became_active":"2024-09-10T10:01:20.229609+0000","last_became_peered":"2024-09-10T10:01:20.229609+0000","last_unstale":"2024-09-10T10:01:22.567619+0000","last_undegraded":"2024-09-10T10:01:22.567619+0000","last_fullsized":"2024-09-10T10:01:22.567619+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:33.605937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235006+0000","last_change":"2024-09-10T10:01:21.228892+0000","last_active":"2024-09-10T10:01:21.235006+0000","last_peered":"2024-09-10T10:01:21.235006+0000","last_clean":"2024-09-10T10:01:21.235006+0000","last_became_active":"2024-09-10T10:01:21.228635+0000","last_became_peered":"2024-09-10T10:01:21.228635+0000","last_unstale":"2024-09-10T10:01:21.235006+0000","last_undegraded":"2024-09-10T10:01:21.235006+0000","last_fullsized":"2024-09-10T10:01:21.235006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:53.942032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.231853+0000","last_change":"2024-09-10T10:01:20.231567+0000","last_active":"2024-09-10T10:01:22.231853+0000","last_peered":"2024-09-10T10:01:22.231853+0000","last_clean":"2024-09-10T10:01:22.231853+0000","last_became_active":"2024-09-10T10:01:20.231093+0000","last_became_peered":"2024-09-10T10:01:20.231093+0000","last_unstale":"2024-09-10T10:01:22.231853+0000","last_undegraded":"2024-09-10T10:01:22.231853+0000","last_fullsized":"2024-09-10T10:01:22.231853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:16.072275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232887+0000","last_change":"2024-09-10T10:01:21.230380+0000","last_active":"2024-09-10T10:01:22.232887+0000","last_peered":"2024-09-10T10:01:22.232887+0000","last_clean":"2024-09-10T10:01:22.232887+0000","last_became_active":"2024-09-10T10:01:21.230052+0000","last_became_peered":"2024-09-10T10:01:21.230052+0000","last_unstale":"2024-09-10T10:01:22.232887+0000","last_undegraded":"2024-09-10T10:01:22.232887+0000","last_fullsized":"2024-09-10T10:01:22.232887+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:48.794297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567794+0000","last_change":"2024-09-10T10:01:20.227521+0000","last_active":"2024-09-10T10:01:22.567794+0000","last_peered":"2024-09-10T10:01:22.567794+0000","last_clean":"2024-09-10T10:01:22.567794+0000","last_became_active":"2024-09-10T10:01:20.227209+0000","last_became_peered":"2024-09-10T10:01:20.227209+0000","last_unstale":"2024-09-10T10:01:22.567794+0000","last_undegraded":"2024-09-10T10:01:22.567794+0000","last_fullsized":"2024-09-10T10:01:22.567794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:37.025460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567688+0000","last_change":"2024-09-10T10:01:21.250138+0000","last_active":"2024-09-10T10:01:22.567688+0000","last_peered":"2024-09-10T10:01:22.567688+0000","last_clean":"2024-09-10T10:01:22.567688+0000","last_became_active":"2024-09-10T10:01:21.249900+0000","last_became_peered":"2024-09-10T10:01:21.249900+0000","last_unstale":"2024-09-10T10:01:22.567688+0000","last_undegraded":"2024-09-10T10:01:22.567688+0000","last_fullsized":"2024-09-10T10:01:22.567688+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:01.053965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.220473+0000","last_change":"2024-09-10T10:01:20.230620+0000","last_active":"2024-09-10T10:01:21.220473+0000","last_peered":"2024-09-10T10:01:21.220473+0000","last_clean":"2024-09-10T10:01:21.220473+0000","last_became_active":"2024-09-10T10:01:20.230403+0000","last_became_peered":"2024-09-10T10:01:20.230403+0000","last_unstale":"2024-09-10T10:01:21.220473+0000","last_undegraded":"2024-09-10T10:01:21.220473+0000","last_fullsized":"2024-09-10T10:01:21.220473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:31.610310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227130+0000","last_change":"2024-09-10T10:01:21.227977+0000","last_active":"2024-09-10T10:01:22.227130+0000","last_peered":"2024-09-10T10:01:22.227130+0000","last_clean":"2024-09-10T10:01:22.227130+0000","last_became_active":"2024-09-10T10:01:21.227662+0000","last_became_peered":"2024-09-10T10:01:21.227662+0000","last_unstale":"2024-09-10T10:01:22.227130+0000","last_undegraded":"2024-09-10T10:01:22.227130+0000","last_fullsized":"2024-09-10T10:01:22.227130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:40.905046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568525+0000","last_change":"2024-09-10T10:01:20.222444+0000","last_active":"2024-09-10T10:01:22.568525+0000","last_peered":"2024-09-10T10:01:22.568525+0000","last_clean":"2024-09-10T10:01:22.568525+0000","last_became_active":"2024-09-10T10:01:20.222195+0000","last_became_peered":"2024-09-10T10:01:20.222195+0000","last_unstale":"2024-09-10T10:01:22.568525+0000","last_undegraded":"2024-09-10T10:01:22.568525+0000","last_fullsized":"2024-09-10T10:01:22.568525+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:00.514446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568466+0000","last_change":"2024-09-10T10:01:21.242979+0000","last_active":"2024-09-10T10:01:22.568466+0000","last_peered":"2024-09-10T10:01:22.568466+0000","last_clean":"2024-09-10T10:01:22.568466+0000","last_became_active":"2024-09-10T10:01:21.242672+0000","last_became_peered":"2024-09-10T10:01:21.242672+0000","last_unstale":"2024-09-10T10:01:22.568466+0000","last_undegraded":"2024-09-10T10:01:22.568466+0000","last_fullsized":"2024-09-10T10:01:22.568466+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:15.249762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232680+0000","last_change":"2024-09-10T10:01:20.226378+0000","last_active":"2024-09-10T10:01:22.232680+0000","last_peered":"2024-09-10T10:01:22.232680+0000","last_clean":"2024-09-10T10:01:22.232680+0000","last_became_active":"2024-09-10T10:01:20.225673+0000","last_became_peered":"2024-09-10T10:01:20.225673+0000","last_unstale":"2024-09-10T10:01:22.232680+0000","last_undegraded":"2024-09-10T10:01:22.232680+0000","last_fullsized":"2024-09-10T10:01:22.232680+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:13.694899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567504+0000","last_change":"2024-09-10T10:01:21.233367+0000","last_active":"2024-09-10T10:01:22.567504+0000","last_peered":"2024-09-10T10:01:22.567504+0000","last_clean":"2024-09-10T10:01:22.567504+0000","last_became_active":"2024-09-10T10:01:21.233120+0000","last_became_peered":"2024-09-10T10:01:21.233120+0000","last_unstale":"2024-09-10T10:01:22.567504+0000","last_undegraded":"2024-09-10T10:01:22.567504+0000","last_fullsized":"2024-09-10T10:01:22.567504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:59.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.253337+0000","last_change":"2024-09-10T10:01:20.225354+0000","last_active":"2024-09-10T10:01:21.253337+0000","last_peered":"2024-09-10T10:01:21.253337+0000","last_clean":"2024-09-10T10:01:21.253337+0000","last_became_active":"2024-09-10T10:01:20.225103+0000","last_became_peered":"2024-09-10T10:01:20.225103+0000","last_unstale":"2024-09-10T10:01:21.253337+0000","last_undegraded":"2024-09-10T10:01:21.253337+0000","last_fullsized":"2024-09-10T10:01:21.253337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:59.373576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567752+0000","last_change":"2024-09-10T10:01:21.245475+0000","last_active":"2024-09-10T10:01:22.567752+0000","last_peered":"2024-09-10T10:01:22.567752+0000","last_clean":"2024-09-10T10:01:22.567752+0000","last_became_active":"2024-09-10T10:01:21.245162+0000","last_became_peered":"2024-09-10T10:01:21.245162+0000","last_unstale":"2024-09-10T10:01:22.567752+0000","last_undegraded":"2024-09-10T10:01:22.567752+0000","last_fullsized":"2024-09-10T10:01:22.567752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:07.802751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.233032+0000","last_change":"2024-09-10T10:01:20.218346+0000","last_active":"2024-09-10T10:01:22.233032+0000","last_peered":"2024-09-10T10:01:22.233032+0000","last_clean":"2024-09-10T10:01:22.233032+0000","last_became_active":"2024-09-10T10:01:20.217507+0000","last_became_peered":"2024-09-10T10:01:20.217507+0000","last_unstale":"2024-09-10T10:01:22.233032+0000","last_undegraded":"2024-09-10T10:01:22.233032+0000","last_fullsized":"2024-09-10T10:01:22.233032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:07.731912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.231833+0000","last_change":"2024-09-10T10:01:21.237219+0000","last_active":"2024-09-10T10:01:22.231833+0000","last_peered":"2024-09-10T10:01:22.231833+0000","last_clean":"2024-09-10T10:01:22.231833+0000","last_became_active":"2024-09-10T10:01:21.236918+0000","last_became_peered":"2024-09-10T10:01:21.236918+0000","last_unstale":"2024-09-10T10:01:22.231833+0000","last_undegraded":"2024-09-10T10:01:22.231833+0000","last_fullsized":"2024-09-10T10:01:22.231833+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:01.518136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567268+0000","last_change":"2024-09-10T10:01:20.230010+0000","last_active":"2024-09-10T10:01:22.567268+0000","last_peered":"2024-09-10T10:01:22.567268+0000","last_clean":"2024-09-10T10:01:22.567268+0000","last_became_active":"2024-09-10T10:01:20.229451+0000","last_became_peered":"2024-09-10T10:01:20.229451+0000","last_unstale":"2024-09-10T10:01:22.567268+0000","last_undegraded":"2024-09-10T10:01:22.567268+0000","last_fullsized":"2024-09-10T10:01:22.567268+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:33:39.478388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567449+0000","last_change":"2024-09-10T10:01:21.252000+0000","last_active":"2024-09-10T10:01:22.567449+0000","last_peered":"2024-09-10T10:01:22.567449+0000","last_clean":"2024-09-10T10:01:22.567449+0000","last_became_active":"2024-09-10T10:01:21.251788+0000","last_became_peered":"2024-09-10T10:01:21.251788+0000","last_unstale":"2024-09-10T10:01:22.567449+0000","last_undegraded":"2024-09-10T10:01:22.567449+0000","last_fullsized":"2024-09-10T10:01:22.567449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:57.393646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567271+0000","last_change":"2024-09-10T10:01:20.226645+0000","last_active":"2024-09-10T10:01:22.567271+0000","last_peered":"2024-09-10T10:01:22.567271+0000","last_clean":"2024-09-10T10:01:22.567271+0000","last_became_active":"2024-09-10T10:01:20.226312+0000","last_became_peered":"2024-09-10T10:01:20.226312+0000","last_unstale":"2024-09-10T10:01:22.567271+0000","last_undegraded":"2024-09-10T10:01:22.567271+0000","last_fullsized":"2024-09-10T10:01:22.567271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:33:39.478388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568266+0000","last_change":"2024-09-10T10:01:21.240697+0000","last_active":"2024-09-10T10:01:22.568266+0000","last_peered":"2024-09-10T10:01:22.568266+0000","last_clean":"2024-09-10T10:01:22.568266+0000","last_became_active":"2024-09-10T10:01:21.240429+0000","last_became_peered":"2024-09-10T10:01:21.240429+0000","last_unstale":"2024-09-10T10:01:22.568266+0000","last_undegraded":"2024-09-10T10:01:22.568266+0000","last_fullsized":"2024-09-10T10:01:22.568266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:16.260334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.253159+0000","last_change":"2024-09-10T10:01:20.227068+0000","last_active":"2024-09-10T10:01:21.253159+0000","last_peered":"2024-09-10T10:01:21.253159+0000","last_clean":"2024-09-10T10:01:21.253159+0000","last_became_active":"2024-09-10T10:01:20.226824+0000","last_became_peered":"2024-09-10T10:01:20.226824+0000","last_unstale":"2024-09-10T10:01:21.253159+0000","last_undegraded":"2024-09-10T10:01:21.253159+0000","last_fullsized":"2024-09-10T10:01:21.253159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:17.682399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568319+0000","last_change":"2024-09-10T10:01:21.234365+0000","last_active":"2024-09-10T10:01:22.568319+0000","last_peered":"2024-09-10T10:01:22.568319+0000","last_clean":"2024-09-10T10:01:22.568319+0000","last_became_active":"2024-09-10T10:01:21.234098+0000","last_became_peered":"2024-09-10T10:01:21.234098+0000","last_unstale":"2024-09-10T10:01:22.568319+0000","last_undegraded":"2024-09-10T10:01:22.568319+0000","last_fullsized":"2024-09-10T10:01:22.568319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:01.053965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567765+0000","last_change":"2024-09-10T10:01:20.231394+0000","last_active":"2024-09-10T10:01:22.567765+0000","last_peered":"2024-09-10T10:01:22.567765+0000","last_clean":"2024-09-10T10:01:22.567765+0000","last_became_active":"2024-09-10T10:01:20.230865+0000","last_became_peered":"2024-09-10T10:01:20.230865+0000","last_unstale":"2024-09-10T10:01:22.567765+0000","last_undegraded":"2024-09-10T10:01:22.567765+0000","last_fullsized":"2024-09-10T10:01:22.567765+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:56.615865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.234935+0000","last_change":"2024-09-10T10:01:21.226356+0000","last_active":"2024-09-10T10:01:21.234935+0000","last_peered":"2024-09-10T10:01:21.234935+0000","last_clean":"2024-09-10T10:01:21.234935+0000","last_became_active":"2024-09-10T10:01:21.225952+0000","last_became_peered":"2024-09-10T10:01:21.225952+0000","last_unstale":"2024-09-10T10:01:21.234935+0000","last_undegraded":"2024-09-10T10:01:21.234935+0000","last_fullsized":"2024-09-10T10:01:21.234935+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:15.527913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568014+0000","last_change":"2024-09-10T10:01:20.229567+0000","last_active":"2024-09-10T10:01:22.568014+0000","last_peered":"2024-09-10T10:01:22.568014+0000","last_clean":"2024-09-10T10:01:22.568014+0000","last_became_active":"2024-09-10T10:01:20.229235+0000","last_became_peered":"2024-09-10T10:01:20.229235+0000","last_unstale":"2024-09-10T10:01:22.568014+0000","last_undegraded":"2024-09-10T10:01:22.568014+0000","last_fullsized":"2024-09-10T10:01:22.568014+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:00.514446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568377+0000","last_change":"2024-09-10T10:01:21.242915+0000","last_active":"2024-09-10T10:01:22.568377+0000","last_peered":"2024-09-10T10:01:22.568377+0000","last_clean":"2024-09-10T10:01:22.568377+0000","last_became_active":"2024-09-10T10:01:21.241792+0000","last_became_peered":"2024-09-10T10:01:21.241792+0000","last_unstale":"2024-09-10T10:01:22.568377+0000","last_undegraded":"2024-09-10T10:01:22.568377+0000","last_fullsized":"2024-09-10T10:01:22.568377+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:54.480722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567422+0000","last_change":"2024-09-10T10:01:20.232468+0000","last_active":"2024-09-10T10:01:22.567422+0000","last_peered":"2024-09-10T10:01:22.567422+0000","last_clean":"2024-09-10T10:01:22.567422+0000","last_became_active":"2024-09-10T10:01:20.232222+0000","last_became_peered":"2024-09-10T10:01:20.232222+0000","last_unstale":"2024-09-10T10:01:22.567422+0000","last_undegraded":"2024-09-10T10:01:22.567422+0000","last_fullsized":"2024-09-10T10:01:22.567422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:37.025460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567900+0000","last_change":"2024-09-10T10:01:21.234080+0000","last_active":"2024-09-10T10:01:22.567900+0000","last_peered":"2024-09-10T10:01:22.567900+0000","last_clean":"2024-09-10T10:01:22.567900+0000","last_became_active":"2024-09-10T10:01:21.233812+0000","last_became_peered":"2024-09-10T10:01:21.233812+0000","last_unstale":"2024-09-10T10:01:22.567900+0000","last_undegraded":"2024-09-10T10:01:22.567900+0000","last_fullsized":"2024-09-10T10:01:22.567900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:17.036026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.253931+0000","last_change":"2024-09-10T10:01:20.222755+0000","last_active":"2024-09-10T10:01:21.253931+0000","last_peered":"2024-09-10T10:01:21.253931+0000","last_clean":"2024-09-10T10:01:21.253931+0000","last_became_active":"2024-09-10T10:01:20.222502+0000","last_became_peered":"2024-09-10T10:01:20.222502+0000","last_unstale":"2024-09-10T10:01:21.253931+0000","last_undegraded":"2024-09-10T10:01:21.253931+0000","last_fullsized":"2024-09-10T10:01:21.253931+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:31.821517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567233+0000","last_change":"2024-09-10T10:01:21.223608+0000","last_active":"2024-09-10T10:01:22.567233+0000","last_peered":"2024-09-10T10:01:22.567233+0000","last_clean":"2024-09-10T10:01:22.567233+0000","last_became_active":"2024-09-10T10:01:21.223353+0000","last_became_peered":"2024-09-10T10:01:21.223353+0000","last_unstale":"2024-09-10T10:01:22.567233+0000","last_undegraded":"2024-09-10T10:01:22.567233+0000","last_fullsized":"2024-09-10T10:01:22.567233+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:21.760367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227046+0000","last_change":"2024-09-10T10:01:20.227347+0000","last_active":"2024-09-10T10:01:22.227046+0000","last_peered":"2024-09-10T10:01:22.227046+0000","last_clean":"2024-09-10T10:01:22.227046+0000","last_became_active":"2024-09-10T10:01:20.227098+0000","last_became_peered":"2024-09-10T10:01:20.227098+0000","last_unstale":"2024-09-10T10:01:22.227046+0000","last_undegraded":"2024-09-10T10:01:22.227046+0000","last_fullsized":"2024-09-10T10:01:22.227046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:41.718511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566731+0000","last_change":"2024-09-10T10:01:21.239991+0000","last_active":"2024-09-10T10:01:22.566731+0000","last_peered":"2024-09-10T10:01:22.566731+0000","last_clean":"2024-09-10T10:01:22.566731+0000","last_became_active":"2024-09-10T10:01:21.239706+0000","last_became_peered":"2024-09-10T10:01:21.239706+0000","last_unstale":"2024-09-10T10:01:22.566731+0000","last_undegraded":"2024-09-10T10:01:22.566731+0000","last_fullsized":"2024-09-10T10:01:22.566731+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:25.977903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567111+0000","last_change":"2024-09-10T10:01:20.232430+0000","last_active":"2024-09-10T10:01:22.567111+0000","last_peered":"2024-09-10T10:01:22.567111+0000","last_clean":"2024-09-10T10:01:22.567111+0000","last_became_active":"2024-09-10T10:01:20.232190+0000","last_became_peered":"2024-09-10T10:01:20.232190+0000","last_unstale":"2024-09-10T10:01:22.567111+0000","last_undegraded":"2024-09-10T10:01:22.567111+0000","last_fullsized":"2024-09-10T10:01:22.567111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:37.025460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568107+0000","last_change":"2024-09-10T10:01:21.230457+0000","last_active":"2024-09-10T10:01:22.568107+0000","last_peered":"2024-09-10T10:01:22.568107+0000","last_clean":"2024-09-10T10:01:22.568107+0000","last_became_active":"2024-09-10T10:01:21.230179+0000","last_became_peered":"2024-09-10T10:01:21.230179+0000","last_unstale":"2024-09-10T10:01:22.568107+0000","last_undegraded":"2024-09-10T10:01:22.568107+0000","last_fullsized":"2024-09-10T10:01:22.568107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:25.977903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227500+0000","last_change":"2024-09-10T10:01:20.225516+0000","last_active":"2024-09-10T10:01:22.227500+0000","last_peered":"2024-09-10T10:01:22.227500+0000","last_clean":"2024-09-10T10:01:22.227500+0000","last_became_active":"2024-09-10T10:01:20.225249+0000","last_became_peered":"2024-09-10T10:01:20.225249+0000","last_unstale":"2024-09-10T10:01:22.227500+0000","last_undegraded":"2024-09-10T10:01:22.227500+0000","last_fullsized":"2024-09-10T10:01:22.227500+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.007582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568328+0000","last_change":"2024-09-10T10:01:21.234344+0000","last_active":"2024-09-10T10:01:22.568328+0000","last_peered":"2024-09-10T10:01:22.568328+0000","last_clean":"2024-09-10T10:01:22.568328+0000","last_became_active":"2024-09-10T10:01:21.234054+0000","last_became_peered":"2024-09-10T10:01:21.234054+0000","last_unstale":"2024-09-10T10:01:22.568328+0000","last_undegraded":"2024-09-10T10:01:22.568328+0000","last_fullsized":"2024-09-10T10:01:22.568328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:07.802751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567887+0000","last_change":"2024-09-10T10:01:20.229475+0000","last_active":"2024-09-10T10:01:22.567887+0000","last_peered":"2024-09-10T10:01:22.567887+0000","last_clean":"2024-09-10T10:01:22.567887+0000","last_became_active":"2024-09-10T10:01:20.229160+0000","last_became_peered":"2024-09-10T10:01:20.229160+0000","last_unstale":"2024-09-10T10:01:22.567887+0000","last_undegraded":"2024-09-10T10:01:22.567887+0000","last_fullsized":"2024-09-10T10:01:22.567887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:47.790606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227230+0000","last_change":"2024-09-10T10:01:21.227334+0000","last_active":"2024-09-10T10:01:22.227230+0000","last_peered":"2024-09-10T10:01:22.227230+0000","last_clean":"2024-09-10T10:01:22.227230+0000","last_became_active":"2024-09-10T10:01:21.227065+0000","last_became_peered":"2024-09-10T10:01:21.227065+0000","last_unstale":"2024-09-10T10:01:22.227230+0000","last_undegraded":"2024-09-10T10:01:22.227230+0000","last_fullsized":"2024-09-10T10:01:22.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:17.075965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.233471+0000","last_change":"2024-09-10T10:01:20.221240+0000","last_active":"2024-09-10T10:01:22.233471+0000","last_peered":"2024-09-10T10:01:22.233471+0000","last_clean":"2024-09-10T10:01:22.233471+0000","last_became_active":"2024-09-10T10:01:20.220953+0000","last_became_peered":"2024-09-10T10:01:20.220953+0000","last_unstale":"2024-09-10T10:01:22.233471+0000","last_undegraded":"2024-09-10T10:01:22.233471+0000","last_fullsized":"2024-09-10T10:01:22.233471+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:01.938706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567636+0000","last_change":"2024-09-10T10:01:21.234817+0000","last_active":"2024-09-10T10:01:22.567636+0000","last_peered":"2024-09-10T10:01:22.567636+0000","last_clean":"2024-09-10T10:01:22.567636+0000","last_became_active":"2024-09-10T10:01:21.233837+0000","last_became_peered":"2024-09-10T10:01:21.233837+0000","last_unstale":"2024-09-10T10:01:22.567636+0000","last_undegraded":"2024-09-10T10:01:22.567636+0000","last_fullsized":"2024-09-10T10:01:22.567636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:54.480722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567865+0000","last_change":"2024-09-10T10:01:20.228548+0000","last_active":"2024-09-10T10:01:22.567865+0000","last_peered":"2024-09-10T10:01:22.567865+0000","last_clean":"2024-09-10T10:01:22.567865+0000","last_became_active":"2024-09-10T10:01:20.228179+0000","last_became_peered":"2024-09-10T10:01:20.228179+0000","last_unstale":"2024-09-10T10:01:22.567865+0000","last_undegraded":"2024-09-10T10:01:22.567865+0000","last_fullsized":"2024-09-10T10:01:22.567865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:09.685892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.229641+0000","last_change":"2024-09-10T10:01:21.226666+0000","last_active":"2024-09-10T10:01:21.229641+0000","last_peered":"2024-09-10T10:01:21.229641+0000","last_clean":"2024-09-10T10:01:21.229641+0000","last_became_active":"2024-09-10T10:01:21.226335+0000","last_became_peered":"2024-09-10T10:01:21.226335+0000","last_unstale":"2024-09-10T10:01:21.229641+0000","last_undegraded":"2024-09-10T10:01:21.229641+0000","last_fullsized":"2024-09-10T10:01:21.229641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:20.173572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.226928+0000","last_change":"2024-09-10T10:01:20.226169+0000","last_active":"2024-09-10T10:01:22.226928+0000","last_peered":"2024-09-10T10:01:22.226928+0000","last_clean":"2024-09-10T10:01:22.226928+0000","last_became_active":"2024-09-10T10:01:20.225926+0000","last_became_peered":"2024-09-10T10:01:20.225926+0000","last_unstale":"2024-09-10T10:01:22.226928+0000","last_undegraded":"2024-09-10T10:01:22.226928+0000","last_fullsized":"2024-09-10T10:01:22.226928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:21.549893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567909+0000","last_change":"2024-09-10T10:01:21.248484+0000","last_active":"2024-09-10T10:01:22.567909+0000","last_peered":"2024-09-10T10:01:22.567909+0000","last_clean":"2024-09-10T10:01:22.567909+0000","last_became_active":"2024-09-10T10:01:21.248250+0000","last_became_peered":"2024-09-10T10:01:21.248250+0000","last_unstale":"2024-09-10T10:01:22.567909+0000","last_undegraded":"2024-09-10T10:01:22.567909+0000","last_fullsized":"2024-09-10T10:01:22.567909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.508961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232295+0000","last_change":"2024-09-10T10:01:20.225932+0000","last_active":"2024-09-10T10:01:22.232295+0000","last_peered":"2024-09-10T10:01:22.232295+0000","last_clean":"2024-09-10T10:01:22.232295+0000","last_became_active":"2024-09-10T10:01:20.225640+0000","last_became_peered":"2024-09-10T10:01:20.225640+0000","last_unstale":"2024-09-10T10:01:22.232295+0000","last_undegraded":"2024-09-10T10:01:22.232295+0000","last_fullsized":"2024-09-10T10:01:22.232295+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:04.477676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.226911+0000","last_change":"2024-09-10T10:01:21.237356+0000","last_active":"2024-09-10T10:01:22.226911+0000","last_peered":"2024-09-10T10:01:22.226911+0000","last_clean":"2024-09-10T10:01:22.226911+0000","last_became_active":"2024-09-10T10:01:21.236059+0000","last_became_peered":"2024-09-10T10:01:21.236059+0000","last_unstale":"2024-09-10T10:01:22.226911+0000","last_undegraded":"2024-09-10T10:01:22.226911+0000","last_fullsized":"2024-09-10T10:01:22.226911+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:40.729701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.226955+0000","last_change":"2024-09-10T10:01:20.226853+0000","last_active":"2024-09-10T10:01:22.226955+0000","last_peered":"2024-09-10T10:01:22.226955+0000","last_clean":"2024-09-10T10:01:22.226955+0000","last_became_active":"2024-09-10T10:01:20.223751+0000","last_became_peered":"2024-09-10T10:01:20.223751+0000","last_unstale":"2024-09-10T10:01:22.226955+0000","last_undegraded":"2024-09-10T10:01:22.226955+0000","last_fullsized":"2024-09-10T10:01:22.226955+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.446874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235049+0000","last_change":"2024-09-10T10:01:21.231732+0000","last_active":"2024-09-10T10:01:21.235049+0000","last_peered":"2024-09-10T10:01:21.235049+0000","last_clean":"2024-09-10T10:01:21.235049+0000","last_became_active":"2024-09-10T10:01:21.231438+0000","last_became_peered":"2024-09-10T10:01:21.231438+0000","last_unstale":"2024-09-10T10:01:21.235049+0000","last_undegraded":"2024-09-10T10:01:21.235049+0000","last_fullsized":"2024-09-10T10:01:21.235049+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:24.198868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.218540+0000","last_change":"2024-09-10T10:01:20.224508+0000","last_active":"2024-09-10T10:01:21.218540+0000","last_peered":"2024-09-10T10:01:21.218540+0000","last_clean":"2024-09-10T10:01:21.218540+0000","last_became_active":"2024-09-10T10:01:20.224244+0000","last_became_peered":"2024-09-10T10:01:20.224244+0000","last_unstale":"2024-09-10T10:01:21.218540+0000","last_undegraded":"2024-09-10T10:01:21.218540+0000","last_fullsized":"2024-09-10T10:01:21.218540+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:09.237904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235445+0000","last_change":"2024-09-10T10:01:21.234344+0000","last_active":"2024-09-10T10:01:21.235445+0000","last_peered":"2024-09-10T10:01:21.235445+0000","last_clean":"2024-09-10T10:01:21.235445+0000","last_became_active":"2024-09-10T10:01:21.233859+0000","last_became_peered":"2024-09-10T10:01:21.233859+0000","last_unstale":"2024-09-10T10:01:21.235445+0000","last_undegraded":"2024-09-10T10:01:21.235445+0000","last_fullsized":"2024-09-10T10:01:21.235445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:28:28.917095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.233182+0000","last_change":"2024-09-10T10:01:20.225688+0000","last_active":"2024-09-10T10:01:22.233182+0000","last_peered":"2024-09-10T10:01:22.233182+0000","last_clean":"2024-09-10T10:01:22.233182+0000","last_became_active":"2024-09-10T10:01:20.224529+0000","last_became_peered":"2024-09-10T10:01:20.224529+0000","last_unstale":"2024-09-10T10:01:22.233182+0000","last_undegraded":"2024-09-10T10:01:22.233182+0000","last_fullsized":"2024-09-10T10:01:22.233182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:36.771898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":234,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567213+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:01:22.567213+0000","last_peered":"2024-09-10T10:01:22.567213+0000","last_clean":"2024-09-10T10:01:22.567213+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:01:22.567213+0000","last_undegraded":"2024-09-10T10:01:22.567213+0000","last_fullsized":"2024-09-10T10:01:22.567213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":866917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":277041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":278001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":696320,"data_stored":282745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":679936,"data_stored":276479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1286144,"data_stored":868951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":684032,"data_stored":276445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:01:25.313 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:01:25.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.484+0000 7f0be4000640 1 Processor -- start 2024-09-10T10:01:25.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.484+0000 7f0be4000640 1 -- start start 2024-09-10T10:01:25.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be4000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0bdc0cfe30 0x7f0bdc071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be4000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0bdc0cd510 0x7f0bdc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be4000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bdc06d2d0 con 0x7f0bdc06e090 2024-09-10T10:01:25.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be4000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc15d9a0 con 0x7f0bdc0cfe30 2024-09-10T10:01:25.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be4000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc06d5b0 con 0x7f0bdc0cd510 2024-09-10T10:01:25.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be1600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0bdc0cfe30 0x7f0bdc071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be2000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0bdc06e090 0x7f0bdc06e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45902/0 (socket says 172.21.15.110:45902) 2024-09-10T10:01:25.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be2000640 1 -- 172.21.15.110:0/2364889977 learned_addr learned my addr 172.21.15.110:0/2364889977 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:25.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be0c00640 1 --2- 172.21.15.110:0/2364889977 >> v2:172.21.15.145:3300/0 conn(0x7f0bdc0cd510 0x7f0bdc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be1600640 1 -- 172.21.15.110:0/2364889977 >> v2:172.21.15.145:3300/0 conn(0x7f0bdc0cd510 msgr2=0x7f0bdc0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:25.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be1600640 1 --2- 172.21.15.110:0/2364889977 >> v2:172.21.15.145:3300/0 conn(0x7f0bdc0cd510 0x7f0bdc0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be1600640 1 -- 172.21.15.110:0/2364889977 >> v1:172.21.15.110:6789/0 conn(0x7f0bdc06e090 legacy=0x7f0bdc06e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be1600640 1 -- 172.21.15.110:0/2364889977 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc06d890 con 0x7f0bdc0cfe30 2024-09-10T10:01:25.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0bdbe00640 1 -- 172.21.15.110:0/2364889977 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3207714406 0 0) 0x7f0bdc06d2d0 con 0x7f0bdc06e090 2024-09-10T10:01:25.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.488+0000 7f0be0c00640 1 --2- 172.21.15.110:0/2364889977 >> v2:172.21.15.145:3300/0 conn(0x7f0bdc0cd510 0x7f0bdc0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:25.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be1600640 1 --2- 172.21.15.110:0/2364889977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cfe30 0x7f0bdc071a40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0ce0f0 tx=0x7f0bd001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3880673e0de738 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0bdbe00640 1 -- 172.21.15.110:0/2364889977 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00b0070 con 0x7f0bdc0cfe30 2024-09-10T10:01:25.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0bdbe00640 1 -- 172.21.15.110:0/2364889977 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd00ab600 con 0x7f0bdc0cfe30 2024-09-10T10:01:25.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0bdbe00640 1 -- 172.21.15.110:0/2364889977 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00ab9d0 con 0x7f0bdc0cfe30 2024-09-10T10:01:25.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 -- 172.21.15.110:0/2364889977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cfe30 msgr2=0x7f0bdc071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 --2- 172.21.15.110:0/2364889977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cfe30 0x7f0bdc071a40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0ce0f0 tx=0x7f0bd001c3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 -- 172.21.15.110:0/2364889977 shutdown_connections 2024-09-10T10:01:25.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 --2- 172.21.15.110:0/2364889977 >> v2:172.21.15.145:3300/0 conn(0x7f0bdc0cd510 0x7f0bdc0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 --2- 172.21.15.110:0/2364889977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cfe30 0x7f0bdc071a40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 -- 172.21.15.110:0/2364889977 >> 172.21.15.110:0/2364889977 conn(0x7f0bdc06bc50 msgr2=0x7f0bdc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:25.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 -- 172.21.15.110:0/2364889977 shutdown_connections 2024-09-10T10:01:25.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 -- 172.21.15.110:0/2364889977 wait complete. 2024-09-10T10:01:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 Processor -- start 2024-09-10T10:01:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.492+0000 7f0be4000640 1 -- start start 2024-09-10T10:01:25.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc06e090 0x7f0bdc150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cd510 0x7f0bdc154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0bdc0cfe30 0x7f0bdc14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc072190 con 0x7f0bdc0cd510 2024-09-10T10:01:25.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc15d6c0 con 0x7f0bdc06e090 2024-09-10T10:01:25.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0bdc169180 con 0x7f0bdc0cfe30 2024-09-10T10:01:25.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cd510 0x7f0bdc154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cd510 0x7f0bdc154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35866/0 (socket says 172.21.15.110:35866) 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 -- 172.21.15.110:0/3635281550 learned_addr learned my addr 172.21.15.110:0/3635281550 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be2000640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0bdc0cfe30 0x7f0bdc14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be1600640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc06e090 0x7f0bdc150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 -- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0bdc0cfe30 msgr2=0x7f0bdc14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0bdc0cfe30 0x7f0bdc14f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 -- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc06e090 msgr2=0x7f0bdc150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc06e090 0x7f0bdc150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 -- 172.21.15.110:0/3635281550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc150dc0 con 0x7f0bdc0cd510 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be2000640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0bdc0cfe30 0x7f0bdc14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be0c00640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cd510 0x7f0bdc154ff0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c7f0 tx=0x7f0bc800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0bd9600640 1 -- 172.21.15.110:0/3635281550 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8015070 con 0x7f0bdc0cd510 2024-09-10T10:01:25.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0bd9600640 1 -- 172.21.15.110:0/3635281550 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc8007820 con 0x7f0bdc0cd510 2024-09-10T10:01:25.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0bd9600640 1 -- 172.21.15.110:0/3635281550 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8007c10 con 0x7f0bdc0cd510 2024-09-10T10:01:25.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc14fd40 con 0x7f0bdc0cd510 2024-09-10T10:01:25.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc150300 con 0x7f0bdc0cd510 2024-09-10T10:01:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc0b3000 con 0x7f0bdc0cd510 2024-09-10T10:01:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.496+0000 7f0bd9600640 1 -- 172.21.15.110:0/3635281550 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f0bc8018070 con 0x7f0bdc0cd510 2024-09-10T10:01:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.500+0000 7f0bd9600640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0bbc067aa0 0x7f0bbc069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.500+0000 7f0bd9600640 1 -- 172.21.15.110:0/3635281550 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0bc8086110 con 0x7f0bdc0cd510 2024-09-10T10:01:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.500+0000 7f0be1600640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0bbc067aa0 0x7f0bbc069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.500+0000 7f0be1600640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0bbc067aa0 0x7f0bbc069f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00ad000 tx=0x7f0bd00a4f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.504+0000 7f0bd9600640 1 -- 172.21.15.110:0/3635281550 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0bc8056260 con 0x7f0bdc0cd510 2024-09-10T10:01:25.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.660+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bdc06e4a0 con 0x7f0bbc067aa0 2024-09-10T10:01:25.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.664+0000 7f0bd9600640 1 -- 172.21.15.110:0/3635281550 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251939 (secure 0 0 0) 0x7f0bdc06e4a0 con 0x7f0bbc067aa0 2024-09-10T10:01:25.671 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:25.672 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:01:25.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0bbc067aa0 msgr2=0x7f0bbc069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0bbc067aa0 0x7f0bbc069f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00ad000 tx=0x7f0bd00a4f90 comp rx=0 tx=0).stop 2024-09-10T10:01:25.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cd510 msgr2=0x7f0bdc154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cd510 0x7f0bdc154ff0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c7f0 tx=0x7f0bc800ccb0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 shutdown_connections 2024-09-10T10:01:25.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0bbc067aa0 0x7f0bbc069f60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0bdc0cfe30 0x7f0bdc14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bdc0cd510 0x7f0bdc154ff0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 --2- 172.21.15.110:0/3635281550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bdc06e090 0x7f0bdc150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 >> 172.21.15.110:0/3635281550 conn(0x7f0bdc06bc50 msgr2=0x7f0bdc15af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:25.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 shutdown_connections 2024-09-10T10:01:25.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.668+0000 7f0be4000640 1 -- 172.21.15.110:0/3635281550 wait complete. 2024-09-10T10:01:25.692 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-10T10:01:25.393329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243576,"kb_used_data":6528,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731144,"statfs":{"total":773094113280,"available":772844691456,"internally_reserved":0,"allocated":6684672,"data_stored":3409268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":242535980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001114"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566855+0000","last_change":"2024-09-10T10:01:21.227763+0000","last_active":"2024-09-10T10:01:22.566855+0000","last_peered":"2024-09-10T10:01:22.566855+0000","last_clean":"2024-09-10T10:01:22.566855+0000","last_became_active":"2024-09-10T10:01:21.227445+0000","last_became_peered":"2024-09-10T10:01:21.227445+0000","last_unstale":"2024-09-10T10:01:22.566855+0000","last_undegraded":"2024-09-10T10:01:22.566855+0000","last_fullsized":"2024-09-10T10:01:22.566855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:00.108623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.229578+0000","last_change":"2024-09-10T10:01:21.226592+0000","last_active":"2024-09-10T10:01:21.229578+0000","last_peered":"2024-09-10T10:01:21.229578+0000","last_clean":"2024-09-10T10:01:21.229578+0000","last_became_active":"2024-09-10T10:01:21.226300+0000","last_became_peered":"2024-09-10T10:01:21.226300+0000","last_unstale":"2024-09-10T10:01:21.229578+0000","last_undegraded":"2024-09-10T10:01:21.229578+0000","last_fullsized":"2024-09-10T10:01:21.229578+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:26.797979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568213+0000","last_change":"2024-09-10T10:01:21.239647+0000","last_active":"2024-09-10T10:01:22.568213+0000","last_peered":"2024-09-10T10:01:22.568213+0000","last_clean":"2024-09-10T10:01:22.568213+0000","last_became_active":"2024-09-10T10:01:21.239370+0000","last_became_peered":"2024-09-10T10:01:21.239370+0000","last_unstale":"2024-09-10T10:01:22.568213+0000","last_undegraded":"2024-09-10T10:01:22.568213+0000","last_fullsized":"2024-09-10T10:01:22.568213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:28:31.791724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567762+0000","last_change":"2024-09-10T10:01:21.229770+0000","last_active":"2024-09-10T10:01:22.567762+0000","last_peered":"2024-09-10T10:01:22.567762+0000","last_clean":"2024-09-10T10:01:22.567762+0000","last_became_active":"2024-09-10T10:01:21.229499+0000","last_became_peered":"2024-09-10T10:01:21.229499+0000","last_unstale":"2024-09-10T10:01:22.567762+0000","last_undegraded":"2024-09-10T10:01:22.567762+0000","last_fullsized":"2024-09-10T10:01:22.567762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:43.234130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566611+0000","last_change":"2024-09-10T10:01:21.238238+0000","last_active":"2024-09-10T10:01:22.566611+0000","last_peered":"2024-09-10T10:01:22.566611+0000","last_clean":"2024-09-10T10:01:22.566611+0000","last_became_active":"2024-09-10T10:01:21.237676+0000","last_became_peered":"2024-09-10T10:01:21.237676+0000","last_unstale":"2024-09-10T10:01:22.566611+0000","last_undegraded":"2024-09-10T10:01:22.566611+0000","last_fullsized":"2024-09-10T10:01:22.566611+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:38.768949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567035+0000","last_change":"2024-09-10T10:01:21.236979+0000","last_active":"2024-09-10T10:01:22.567035+0000","last_peered":"2024-09-10T10:01:22.567035+0000","last_clean":"2024-09-10T10:01:22.567035+0000","last_became_active":"2024-09-10T10:01:21.236685+0000","last_became_peered":"2024-09-10T10:01:21.236685+0000","last_unstale":"2024-09-10T10:01:22.567035+0000","last_undegraded":"2024-09-10T10:01:22.567035+0000","last_fullsized":"2024-09-10T10:01:22.567035+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.508961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.231852+0000","last_change":"2024-09-10T10:01:21.237900+0000","last_active":"2024-09-10T10:01:22.231852+0000","last_peered":"2024-09-10T10:01:22.231852+0000","last_clean":"2024-09-10T10:01:22.231852+0000","last_became_active":"2024-09-10T10:01:21.237486+0000","last_became_peered":"2024-09-10T10:01:21.237486+0000","last_unstale":"2024-09-10T10:01:22.231852+0000","last_undegraded":"2024-09-10T10:01:22.231852+0000","last_fullsized":"2024-09-10T10:01:22.231852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:38.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235191+0000","last_change":"2024-09-10T10:01:21.227855+0000","last_active":"2024-09-10T10:01:21.235191+0000","last_peered":"2024-09-10T10:01:21.235191+0000","last_clean":"2024-09-10T10:01:21.235191+0000","last_became_active":"2024-09-10T10:01:21.225768+0000","last_became_peered":"2024-09-10T10:01:21.225768+0000","last_unstale":"2024-09-10T10:01:21.235191+0000","last_undegraded":"2024-09-10T10:01:21.235191+0000","last_fullsized":"2024-09-10T10:01:21.235191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:43.783217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232922+0000","last_change":"2024-09-10T10:01:21.230481+0000","last_active":"2024-09-10T10:01:22.232922+0000","last_peered":"2024-09-10T10:01:22.232922+0000","last_clean":"2024-09-10T10:01:22.232922+0000","last_became_active":"2024-09-10T10:01:21.230207+0000","last_became_peered":"2024-09-10T10:01:21.230207+0000","last_unstale":"2024-09-10T10:01:22.232922+0000","last_undegraded":"2024-09-10T10:01:22.232922+0000","last_fullsized":"2024-09-10T10:01:22.232922+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:00.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227356+0000","last_change":"2024-09-10T10:01:21.236084+0000","last_active":"2024-09-10T10:01:22.227356+0000","last_peered":"2024-09-10T10:01:22.227356+0000","last_clean":"2024-09-10T10:01:22.227356+0000","last_became_active":"2024-09-10T10:01:21.235775+0000","last_became_peered":"2024-09-10T10:01:21.235775+0000","last_unstale":"2024-09-10T10:01:22.227356+0000","last_undegraded":"2024-09-10T10:01:22.227356+0000","last_fullsized":"2024-09-10T10:01:22.227356+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:14.698590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227335+0000","last_change":"2024-09-10T10:01:21.227181+0000","last_active":"2024-09-10T10:01:22.227335+0000","last_peered":"2024-09-10T10:01:22.227335+0000","last_clean":"2024-09-10T10:01:22.227335+0000","last_became_active":"2024-09-10T10:01:21.226894+0000","last_became_peered":"2024-09-10T10:01:21.226894+0000","last_unstale":"2024-09-10T10:01:22.227335+0000","last_undegraded":"2024-09-10T10:01:22.227335+0000","last_fullsized":"2024-09-10T10:01:22.227335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:05.481366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567587+0000","last_change":"2024-09-10T10:01:21.229191+0000","last_active":"2024-09-10T10:01:22.567587+0000","last_peered":"2024-09-10T10:01:22.567587+0000","last_clean":"2024-09-10T10:01:22.567587+0000","last_became_active":"2024-09-10T10:01:21.228882+0000","last_became_peered":"2024-09-10T10:01:21.228882+0000","last_unstale":"2024-09-10T10:01:22.567587+0000","last_undegraded":"2024-09-10T10:01:22.567587+0000","last_fullsized":"2024-09-10T10:01:22.567587+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:08.856667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.233355+0000","last_change":"2024-09-10T10:01:21.227224+0000","last_active":"2024-09-10T10:01:22.233355+0000","last_peered":"2024-09-10T10:01:22.233355+0000","last_clean":"2024-09-10T10:01:22.233355+0000","last_became_active":"2024-09-10T10:01:21.226946+0000","last_became_peered":"2024-09-10T10:01:21.226946+0000","last_unstale":"2024-09-10T10:01:22.233355+0000","last_undegraded":"2024-09-10T10:01:22.233355+0000","last_fullsized":"2024-09-10T10:01:22.233355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:05.481366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232144+0000","last_change":"2024-09-10T10:01:21.239230+0000","last_active":"2024-09-10T10:01:22.232144+0000","last_peered":"2024-09-10T10:01:22.232144+0000","last_clean":"2024-09-10T10:01:22.232144+0000","last_became_active":"2024-09-10T10:01:21.238702+0000","last_became_peered":"2024-09-10T10:01:21.238702+0000","last_unstale":"2024-09-10T10:01:22.232144+0000","last_undegraded":"2024-09-10T10:01:22.232144+0000","last_fullsized":"2024-09-10T10:01:22.232144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:57.619556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232063+0000","last_change":"2024-09-10T10:01:21.226688+0000","last_active":"2024-09-10T10:01:22.232063+0000","last_peered":"2024-09-10T10:01:22.232063+0000","last_clean":"2024-09-10T10:01:22.232063+0000","last_became_active":"2024-09-10T10:01:21.226305+0000","last_became_peered":"2024-09-10T10:01:21.226305+0000","last_unstale":"2024-09-10T10:01:22.232063+0000","last_undegraded":"2024-09-10T10:01:22.232063+0000","last_fullsized":"2024-09-10T10:01:22.232063+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:19.047000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566969+0000","last_change":"2024-09-10T10:01:21.230703+0000","last_active":"2024-09-10T10:01:22.566969+0000","last_peered":"2024-09-10T10:01:22.566969+0000","last_clean":"2024-09-10T10:01:22.566969+0000","last_became_active":"2024-09-10T10:01:21.225406+0000","last_became_peered":"2024-09-10T10:01:21.225406+0000","last_unstale":"2024-09-10T10:01:22.566969+0000","last_undegraded":"2024-09-10T10:01:22.566969+0000","last_fullsized":"2024-09-10T10:01:22.566969+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:07.802751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235028+0000","last_change":"2024-09-10T10:01:21.229294+0000","last_active":"2024-09-10T10:01:21.235028+0000","last_peered":"2024-09-10T10:01:21.235028+0000","last_clean":"2024-09-10T10:01:21.235028+0000","last_became_active":"2024-09-10T10:01:21.228379+0000","last_became_peered":"2024-09-10T10:01:21.228379+0000","last_unstale":"2024-09-10T10:01:21.235028+0000","last_undegraded":"2024-09-10T10:01:21.235028+0000","last_fullsized":"2024-09-10T10:01:21.235028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:40.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568084+0000","last_change":"2024-09-10T10:01:21.244357+0000","last_active":"2024-09-10T10:01:22.568084+0000","last_peered":"2024-09-10T10:01:22.568084+0000","last_clean":"2024-09-10T10:01:22.568084+0000","last_became_active":"2024-09-10T10:01:21.244128+0000","last_became_peered":"2024-09-10T10:01:21.244128+0000","last_unstale":"2024-09-10T10:01:22.568084+0000","last_undegraded":"2024-09-10T10:01:22.568084+0000","last_fullsized":"2024-09-10T10:01:22.568084+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:04.680093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566785+0000","last_change":"2024-09-10T10:01:21.231413+0000","last_active":"2024-09-10T10:01:22.566785+0000","last_peered":"2024-09-10T10:01:22.566785+0000","last_clean":"2024-09-10T10:01:22.566785+0000","last_became_active":"2024-09-10T10:01:21.231156+0000","last_became_peered":"2024-09-10T10:01:21.231156+0000","last_unstale":"2024-09-10T10:01:22.566785+0000","last_undegraded":"2024-09-10T10:01:22.566785+0000","last_fullsized":"2024-09-10T10:01:22.566785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:51.151589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.243895+0000","last_change":"2024-09-10T10:01:21.242658+0000","last_active":"2024-09-10T10:01:21.243895+0000","last_peered":"2024-09-10T10:01:21.243895+0000","last_clean":"2024-09-10T10:01:21.243895+0000","last_became_active":"2024-09-10T10:01:21.242095+0000","last_became_peered":"2024-09-10T10:01:21.242095+0000","last_unstale":"2024-09-10T10:01:21.243895+0000","last_undegraded":"2024-09-10T10:01:21.243895+0000","last_fullsized":"2024-09-10T10:01:21.243895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:24.070238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567593+0000","last_change":"2024-09-10T10:01:21.247499+0000","last_active":"2024-09-10T10:01:22.567593+0000","last_peered":"2024-09-10T10:01:22.567593+0000","last_clean":"2024-09-10T10:01:22.567593+0000","last_became_active":"2024-09-10T10:01:21.246815+0000","last_became_peered":"2024-09-10T10:01:21.246815+0000","last_unstale":"2024-09-10T10:01:22.567593+0000","last_undegraded":"2024-09-10T10:01:22.567593+0000","last_fullsized":"2024-09-10T10:01:22.567593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:25.977903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.241962+0000","last_change":"2024-09-10T10:01:21.240088+0000","last_active":"2024-09-10T10:01:21.241962+0000","last_peered":"2024-09-10T10:01:21.241962+0000","last_clean":"2024-09-10T10:01:21.241962+0000","last_became_active":"2024-09-10T10:01:21.239856+0000","last_became_peered":"2024-09-10T10:01:21.239856+0000","last_unstale":"2024-09-10T10:01:21.241962+0000","last_undegraded":"2024-09-10T10:01:21.241962+0000","last_fullsized":"2024-09-10T10:01:21.241962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:28.623836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567239+0000","last_change":"2024-09-10T10:01:21.248432+0000","last_active":"2024-09-10T10:01:22.567239+0000","last_peered":"2024-09-10T10:01:22.567239+0000","last_clean":"2024-09-10T10:01:22.567239+0000","last_became_active":"2024-09-10T10:01:21.248123+0000","last_became_peered":"2024-09-10T10:01:21.248123+0000","last_unstale":"2024-09-10T10:01:22.567239+0000","last_undegraded":"2024-09-10T10:01:22.567239+0000","last_fullsized":"2024-09-10T10:01:22.567239+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:17.036026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232454+0000","last_change":"2024-09-10T10:01:21.227695+0000","last_active":"2024-09-10T10:01:22.232454+0000","last_peered":"2024-09-10T10:01:22.232454+0000","last_clean":"2024-09-10T10:01:22.232454+0000","last_became_active":"2024-09-10T10:01:21.227220+0000","last_became_peered":"2024-09-10T10:01:21.227220+0000","last_unstale":"2024-09-10T10:01:22.232454+0000","last_undegraded":"2024-09-10T10:01:22.232454+0000","last_fullsized":"2024-09-10T10:01:22.232454+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:40.905046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566986+0000","last_change":"2024-09-10T10:01:21.234111+0000","last_active":"2024-09-10T10:01:22.566986+0000","last_peered":"2024-09-10T10:01:22.566986+0000","last_clean":"2024-09-10T10:01:22.566986+0000","last_became_active":"2024-09-10T10:01:21.233822+0000","last_became_peered":"2024-09-10T10:01:21.233822+0000","last_unstale":"2024-09-10T10:01:22.566986+0000","last_undegraded":"2024-09-10T10:01:22.566986+0000","last_fullsized":"2024-09-10T10:01:22.566986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:32.111103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568085+0000","last_change":"2024-09-10T10:01:21.236963+0000","last_active":"2024-09-10T10:01:22.568085+0000","last_peered":"2024-09-10T10:01:22.568085+0000","last_clean":"2024-09-10T10:01:22.568085+0000","last_became_active":"2024-09-10T10:01:21.236409+0000","last_became_peered":"2024-09-10T10:01:21.236409+0000","last_unstale":"2024-09-10T10:01:22.568085+0000","last_undegraded":"2024-09-10T10:01:22.568085+0000","last_fullsized":"2024-09-10T10:01:22.568085+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:57.393646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567283+0000","last_change":"2024-09-10T10:01:21.230230+0000","last_active":"2024-09-10T10:01:22.567283+0000","last_peered":"2024-09-10T10:01:22.567283+0000","last_clean":"2024-09-10T10:01:22.567283+0000","last_became_active":"2024-09-10T10:01:21.229933+0000","last_became_peered":"2024-09-10T10:01:21.229933+0000","last_unstale":"2024-09-10T10:01:22.567283+0000","last_undegraded":"2024-09-10T10:01:22.567283+0000","last_fullsized":"2024-09-10T10:01:22.567283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:04.680093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567512+0000","last_change":"2024-09-10T10:01:21.236959+0000","last_active":"2024-09-10T10:01:22.567512+0000","last_peered":"2024-09-10T10:01:22.567512+0000","last_clean":"2024-09-10T10:01:22.567512+0000","last_became_active":"2024-09-10T10:01:21.236489+0000","last_became_peered":"2024-09-10T10:01:21.236489+0000","last_unstale":"2024-09-10T10:01:22.567512+0000","last_undegraded":"2024-09-10T10:01:22.567512+0000","last_fullsized":"2024-09-10T10:01:22.567512+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:47.538171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235221+0000","last_change":"2024-09-10T10:01:21.231102+0000","last_active":"2024-09-10T10:01:21.235221+0000","last_peered":"2024-09-10T10:01:21.235221+0000","last_clean":"2024-09-10T10:01:21.235221+0000","last_became_active":"2024-09-10T10:01:21.230805+0000","last_became_peered":"2024-09-10T10:01:21.230805+0000","last_unstale":"2024-09-10T10:01:21.235221+0000","last_undegraded":"2024-09-10T10:01:21.235221+0000","last_fullsized":"2024-09-10T10:01:21.235221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:45.805042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568025+0000","last_change":"2024-09-10T10:01:21.240643+0000","last_active":"2024-09-10T10:01:22.568025+0000","last_peered":"2024-09-10T10:01:22.568025+0000","last_clean":"2024-09-10T10:01:22.568025+0000","last_became_active":"2024-09-10T10:01:21.240361+0000","last_became_peered":"2024-09-10T10:01:21.240361+0000","last_unstale":"2024-09-10T10:01:22.568025+0000","last_undegraded":"2024-09-10T10:01:22.568025+0000","last_fullsized":"2024-09-10T10:01:22.568025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:21.760367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567364+0000","last_change":"2024-09-10T10:01:21.229612+0000","last_active":"2024-09-10T10:01:22.567364+0000","last_peered":"2024-09-10T10:01:22.567364+0000","last_clean":"2024-09-10T10:01:22.567364+0000","last_became_active":"2024-09-10T10:01:21.229354+0000","last_became_peered":"2024-09-10T10:01:21.229354+0000","last_unstale":"2024-09-10T10:01:22.567364+0000","last_undegraded":"2024-09-10T10:01:22.567364+0000","last_fullsized":"2024-09-10T10:01:22.567364+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:47.107132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566812+0000","last_change":"2024-09-10T10:01:21.231435+0000","last_active":"2024-09-10T10:01:22.566812+0000","last_peered":"2024-09-10T10:01:22.566812+0000","last_clean":"2024-09-10T10:01:22.566812+0000","last_became_active":"2024-09-10T10:01:21.230787+0000","last_became_peered":"2024-09-10T10:01:21.230787+0000","last_unstale":"2024-09-10T10:01:22.566812+0000","last_undegraded":"2024-09-10T10:01:22.566812+0000","last_fullsized":"2024-09-10T10:01:22.566812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:01.053965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567766+0000","last_change":"2024-09-10T10:01:21.236903+0000","last_active":"2024-09-10T10:01:22.567766+0000","last_peered":"2024-09-10T10:01:22.567766+0000","last_clean":"2024-09-10T10:01:22.567766+0000","last_became_active":"2024-09-10T10:01:21.236494+0000","last_became_peered":"2024-09-10T10:01:21.236494+0000","last_unstale":"2024-09-10T10:01:22.567766+0000","last_undegraded":"2024-09-10T10:01:22.567766+0000","last_fullsized":"2024-09-10T10:01:22.567766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:15.249762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567910+0000","last_change":"2024-09-10T10:01:20.227440+0000","last_active":"2024-09-10T10:01:22.567910+0000","last_peered":"2024-09-10T10:01:22.567910+0000","last_clean":"2024-09-10T10:01:22.567910+0000","last_became_active":"2024-09-10T10:01:20.227148+0000","last_became_peered":"2024-09-10T10:01:20.227148+0000","last_unstale":"2024-09-10T10:01:22.567910+0000","last_undegraded":"2024-09-10T10:01:22.567910+0000","last_fullsized":"2024-09-10T10:01:22.567910+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:59.541177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227537+0000","last_change":"2024-09-10T10:01:21.247156+0000","last_active":"2024-09-10T10:01:22.227537+0000","last_peered":"2024-09-10T10:01:22.227537+0000","last_clean":"2024-09-10T10:01:22.227537+0000","last_became_active":"2024-09-10T10:01:21.246848+0000","last_became_peered":"2024-09-10T10:01:21.246848+0000","last_unstale":"2024-09-10T10:01:22.227537+0000","last_undegraded":"2024-09-10T10:01:22.227537+0000","last_fullsized":"2024-09-10T10:01:22.227537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:09.999253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.253500+0000","last_change":"2024-09-10T10:01:20.217325+0000","last_active":"2024-09-10T10:01:21.253500+0000","last_peered":"2024-09-10T10:01:21.253500+0000","last_clean":"2024-09-10T10:01:21.253500+0000","last_became_active":"2024-09-10T10:01:20.217047+0000","last_became_peered":"2024-09-10T10:01:20.217047+0000","last_unstale":"2024-09-10T10:01:21.253500+0000","last_undegraded":"2024-09-10T10:01:21.253500+0000","last_fullsized":"2024-09-10T10:01:21.253500+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:18.504805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567415+0000","last_change":"2024-09-10T10:01:21.237965+0000","last_active":"2024-09-10T10:01:22.567415+0000","last_peered":"2024-09-10T10:01:22.567415+0000","last_clean":"2024-09-10T10:01:22.567415+0000","last_became_active":"2024-09-10T10:01:21.231838+0000","last_became_peered":"2024-09-10T10:01:21.231838+0000","last_unstale":"2024-09-10T10:01:22.567415+0000","last_undegraded":"2024-09-10T10:01:22.567415+0000","last_fullsized":"2024-09-10T10:01:22.567415+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:28:31.791724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567702+0000","last_change":"2024-09-10T10:01:20.229396+0000","last_active":"2024-09-10T10:01:22.567702+0000","last_peered":"2024-09-10T10:01:22.567702+0000","last_clean":"2024-09-10T10:01:22.567702+0000","last_became_active":"2024-09-10T10:01:20.229148+0000","last_became_peered":"2024-09-10T10:01:20.229148+0000","last_unstale":"2024-09-10T10:01:22.567702+0000","last_undegraded":"2024-09-10T10:01:22.567702+0000","last_fullsized":"2024-09-10T10:01:22.567702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:18.043310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567429+0000","last_change":"2024-09-10T10:01:21.235093+0000","last_active":"2024-09-10T10:01:22.567429+0000","last_peered":"2024-09-10T10:01:22.567429+0000","last_clean":"2024-09-10T10:01:22.567429+0000","last_became_active":"2024-09-10T10:01:21.234575+0000","last_became_peered":"2024-09-10T10:01:21.234575+0000","last_unstale":"2024-09-10T10:01:22.567429+0000","last_undegraded":"2024-09-10T10:01:22.567429+0000","last_fullsized":"2024-09-10T10:01:22.567429+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:16.260334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567799+0000","last_change":"2024-09-10T10:01:20.228296+0000","last_active":"2024-09-10T10:01:22.567799+0000","last_peered":"2024-09-10T10:01:22.567799+0000","last_clean":"2024-09-10T10:01:22.567799+0000","last_became_active":"2024-09-10T10:01:20.228008+0000","last_became_peered":"2024-09-10T10:01:20.228008+0000","last_unstale":"2024-09-10T10:01:22.567799+0000","last_undegraded":"2024-09-10T10:01:22.567799+0000","last_fullsized":"2024-09-10T10:01:22.567799+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:27.565888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.241941+0000","last_change":"2024-09-10T10:01:21.239574+0000","last_active":"2024-09-10T10:01:21.241941+0000","last_peered":"2024-09-10T10:01:21.241941+0000","last_clean":"2024-09-10T10:01:21.241941+0000","last_became_active":"2024-09-10T10:01:21.239160+0000","last_became_peered":"2024-09-10T10:01:21.239160+0000","last_unstale":"2024-09-10T10:01:21.241941+0000","last_undegraded":"2024-09-10T10:01:21.241941+0000","last_fullsized":"2024-09-10T10:01:21.241941+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:35.101646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232449+0000","last_change":"2024-09-10T10:01:20.224016+0000","last_active":"2024-09-10T10:01:22.232449+0000","last_peered":"2024-09-10T10:01:22.232449+0000","last_clean":"2024-09-10T10:01:22.232449+0000","last_became_active":"2024-09-10T10:01:20.223210+0000","last_became_peered":"2024-09-10T10:01:20.223210+0000","last_unstale":"2024-09-10T10:01:22.232449+0000","last_undegraded":"2024-09-10T10:01:22.232449+0000","last_fullsized":"2024-09-10T10:01:22.232449+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:08.995562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235240+0000","last_change":"2024-09-10T10:01:21.233432+0000","last_active":"2024-09-10T10:01:21.235240+0000","last_peered":"2024-09-10T10:01:21.235240+0000","last_clean":"2024-09-10T10:01:21.235240+0000","last_became_active":"2024-09-10T10:01:21.233101+0000","last_became_peered":"2024-09-10T10:01:21.233101+0000","last_unstale":"2024-09-10T10:01:21.235240+0000","last_undegraded":"2024-09-10T10:01:21.235240+0000","last_fullsized":"2024-09-10T10:01:21.235240+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:00.545368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567107+0000","last_change":"2024-09-10T10:01:20.223720+0000","last_active":"2024-09-10T10:01:22.567107+0000","last_peered":"2024-09-10T10:01:22.567107+0000","last_clean":"2024-09-10T10:01:22.567107+0000","last_became_active":"2024-09-10T10:01:20.223437+0000","last_became_peered":"2024-09-10T10:01:20.223437+0000","last_unstale":"2024-09-10T10:01:22.567107+0000","last_undegraded":"2024-09-10T10:01:22.567107+0000","last_fullsized":"2024-09-10T10:01:22.567107+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:00.514446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232605+0000","last_change":"2024-09-10T10:01:21.240686+0000","last_active":"2024-09-10T10:01:22.232605+0000","last_peered":"2024-09-10T10:01:22.232605+0000","last_clean":"2024-09-10T10:01:22.232605+0000","last_became_active":"2024-09-10T10:01:21.239623+0000","last_became_peered":"2024-09-10T10:01:21.239623+0000","last_unstale":"2024-09-10T10:01:22.232605+0000","last_undegraded":"2024-09-10T10:01:22.232605+0000","last_fullsized":"2024-09-10T10:01:22.232605+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:17.075965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567619+0000","last_change":"2024-09-10T10:01:20.229878+0000","last_active":"2024-09-10T10:01:22.567619+0000","last_peered":"2024-09-10T10:01:22.567619+0000","last_clean":"2024-09-10T10:01:22.567619+0000","last_became_active":"2024-09-10T10:01:20.229609+0000","last_became_peered":"2024-09-10T10:01:20.229609+0000","last_unstale":"2024-09-10T10:01:22.567619+0000","last_undegraded":"2024-09-10T10:01:22.567619+0000","last_fullsized":"2024-09-10T10:01:22.567619+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:33.605937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235006+0000","last_change":"2024-09-10T10:01:21.228892+0000","last_active":"2024-09-10T10:01:21.235006+0000","last_peered":"2024-09-10T10:01:21.235006+0000","last_clean":"2024-09-10T10:01:21.235006+0000","last_became_active":"2024-09-10T10:01:21.228635+0000","last_became_peered":"2024-09-10T10:01:21.228635+0000","last_unstale":"2024-09-10T10:01:21.235006+0000","last_undegraded":"2024-09-10T10:01:21.235006+0000","last_fullsized":"2024-09-10T10:01:21.235006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:53.942032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.231853+0000","last_change":"2024-09-10T10:01:20.231567+0000","last_active":"2024-09-10T10:01:22.231853+0000","last_peered":"2024-09-10T10:01:22.231853+0000","last_clean":"2024-09-10T10:01:22.231853+0000","last_became_active":"2024-09-10T10:01:20.231093+0000","last_became_peered":"2024-09-10T10:01:20.231093+0000","last_unstale":"2024-09-10T10:01:22.231853+0000","last_undegraded":"2024-09-10T10:01:22.231853+0000","last_fullsized":"2024-09-10T10:01:22.231853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:16.072275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232887+0000","last_change":"2024-09-10T10:01:21.230380+0000","last_active":"2024-09-10T10:01:22.232887+0000","last_peered":"2024-09-10T10:01:22.232887+0000","last_clean":"2024-09-10T10:01:22.232887+0000","last_became_active":"2024-09-10T10:01:21.230052+0000","last_became_peered":"2024-09-10T10:01:21.230052+0000","last_unstale":"2024-09-10T10:01:22.232887+0000","last_undegraded":"2024-09-10T10:01:22.232887+0000","last_fullsized":"2024-09-10T10:01:22.232887+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:48.794297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567794+0000","last_change":"2024-09-10T10:01:20.227521+0000","last_active":"2024-09-10T10:01:22.567794+0000","last_peered":"2024-09-10T10:01:22.567794+0000","last_clean":"2024-09-10T10:01:22.567794+0000","last_became_active":"2024-09-10T10:01:20.227209+0000","last_became_peered":"2024-09-10T10:01:20.227209+0000","last_unstale":"2024-09-10T10:01:22.567794+0000","last_undegraded":"2024-09-10T10:01:22.567794+0000","last_fullsized":"2024-09-10T10:01:22.567794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:37.025460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567688+0000","last_change":"2024-09-10T10:01:21.250138+0000","last_active":"2024-09-10T10:01:22.567688+0000","last_peered":"2024-09-10T10:01:22.567688+0000","last_clean":"2024-09-10T10:01:22.567688+0000","last_became_active":"2024-09-10T10:01:21.249900+0000","last_became_peered":"2024-09-10T10:01:21.249900+0000","last_unstale":"2024-09-10T10:01:22.567688+0000","last_undegraded":"2024-09-10T10:01:22.567688+0000","last_fullsized":"2024-09-10T10:01:22.567688+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:01.053965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.220473+0000","last_change":"2024-09-10T10:01:20.230620+0000","last_active":"2024-09-10T10:01:21.220473+0000","last_peered":"2024-09-10T10:01:21.220473+0000","last_clean":"2024-09-10T10:01:21.220473+0000","last_became_active":"2024-09-10T10:01:20.230403+0000","last_became_peered":"2024-09-10T10:01:20.230403+0000","last_unstale":"2024-09-10T10:01:21.220473+0000","last_undegraded":"2024-09-10T10:01:21.220473+0000","last_fullsized":"2024-09-10T10:01:21.220473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:31.610310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227130+0000","last_change":"2024-09-10T10:01:21.227977+0000","last_active":"2024-09-10T10:01:22.227130+0000","last_peered":"2024-09-10T10:01:22.227130+0000","last_clean":"2024-09-10T10:01:22.227130+0000","last_became_active":"2024-09-10T10:01:21.227662+0000","last_became_peered":"2024-09-10T10:01:21.227662+0000","last_unstale":"2024-09-10T10:01:22.227130+0000","last_undegraded":"2024-09-10T10:01:22.227130+0000","last_fullsized":"2024-09-10T10:01:22.227130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:40.905046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568525+0000","last_change":"2024-09-10T10:01:20.222444+0000","last_active":"2024-09-10T10:01:22.568525+0000","last_peered":"2024-09-10T10:01:22.568525+0000","last_clean":"2024-09-10T10:01:22.568525+0000","last_became_active":"2024-09-10T10:01:20.222195+0000","last_became_peered":"2024-09-10T10:01:20.222195+0000","last_unstale":"2024-09-10T10:01:22.568525+0000","last_undegraded":"2024-09-10T10:01:22.568525+0000","last_fullsized":"2024-09-10T10:01:22.568525+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:00.514446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568466+0000","last_change":"2024-09-10T10:01:21.242979+0000","last_active":"2024-09-10T10:01:22.568466+0000","last_peered":"2024-09-10T10:01:22.568466+0000","last_clean":"2024-09-10T10:01:22.568466+0000","last_became_active":"2024-09-10T10:01:21.242672+0000","last_became_peered":"2024-09-10T10:01:21.242672+0000","last_unstale":"2024-09-10T10:01:22.568466+0000","last_undegraded":"2024-09-10T10:01:22.568466+0000","last_fullsized":"2024-09-10T10:01:22.568466+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:15.249762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232680+0000","last_change":"2024-09-10T10:01:20.226378+0000","last_active":"2024-09-10T10:01:22.232680+0000","last_peered":"2024-09-10T10:01:22.232680+0000","last_clean":"2024-09-10T10:01:22.232680+0000","last_became_active":"2024-09-10T10:01:20.225673+0000","last_became_peered":"2024-09-10T10:01:20.225673+0000","last_unstale":"2024-09-10T10:01:22.232680+0000","last_undegraded":"2024-09-10T10:01:22.232680+0000","last_fullsized":"2024-09-10T10:01:22.232680+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:13.694899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567504+0000","last_change":"2024-09-10T10:01:21.233367+0000","last_active":"2024-09-10T10:01:22.567504+0000","last_peered":"2024-09-10T10:01:22.567504+0000","last_clean":"2024-09-10T10:01:22.567504+0000","last_became_active":"2024-09-10T10:01:21.233120+0000","last_became_peered":"2024-09-10T10:01:21.233120+0000","last_unstale":"2024-09-10T10:01:22.567504+0000","last_undegraded":"2024-09-10T10:01:22.567504+0000","last_fullsized":"2024-09-10T10:01:22.567504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:59.459061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.253337+0000","last_change":"2024-09-10T10:01:20.225354+0000","last_active":"2024-09-10T10:01:21.253337+0000","last_peered":"2024-09-10T10:01:21.253337+0000","last_clean":"2024-09-10T10:01:21.253337+0000","last_became_active":"2024-09-10T10:01:20.225103+0000","last_became_peered":"2024-09-10T10:01:20.225103+0000","last_unstale":"2024-09-10T10:01:21.253337+0000","last_undegraded":"2024-09-10T10:01:21.253337+0000","last_fullsized":"2024-09-10T10:01:21.253337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:59.373576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567752+0000","last_change":"2024-09-10T10:01:21.245475+0000","last_active":"2024-09-10T10:01:22.567752+0000","last_peered":"2024-09-10T10:01:22.567752+0000","last_clean":"2024-09-10T10:01:22.567752+0000","last_became_active":"2024-09-10T10:01:21.245162+0000","last_became_peered":"2024-09-10T10:01:21.245162+0000","last_unstale":"2024-09-10T10:01:22.567752+0000","last_undegraded":"2024-09-10T10:01:22.567752+0000","last_fullsized":"2024-09-10T10:01:22.567752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:07.802751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.233032+0000","last_change":"2024-09-10T10:01:20.218346+0000","last_active":"2024-09-10T10:01:22.233032+0000","last_peered":"2024-09-10T10:01:22.233032+0000","last_clean":"2024-09-10T10:01:22.233032+0000","last_became_active":"2024-09-10T10:01:20.217507+0000","last_became_peered":"2024-09-10T10:01:20.217507+0000","last_unstale":"2024-09-10T10:01:22.233032+0000","last_undegraded":"2024-09-10T10:01:22.233032+0000","last_fullsized":"2024-09-10T10:01:22.233032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:07.731912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.231833+0000","last_change":"2024-09-10T10:01:21.237219+0000","last_active":"2024-09-10T10:01:22.231833+0000","last_peered":"2024-09-10T10:01:22.231833+0000","last_clean":"2024-09-10T10:01:22.231833+0000","last_became_active":"2024-09-10T10:01:21.236918+0000","last_became_peered":"2024-09-10T10:01:21.236918+0000","last_unstale":"2024-09-10T10:01:22.231833+0000","last_undegraded":"2024-09-10T10:01:22.231833+0000","last_fullsized":"2024-09-10T10:01:22.231833+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:01.518136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567268+0000","last_change":"2024-09-10T10:01:20.230010+0000","last_active":"2024-09-10T10:01:22.567268+0000","last_peered":"2024-09-10T10:01:22.567268+0000","last_clean":"2024-09-10T10:01:22.567268+0000","last_became_active":"2024-09-10T10:01:20.229451+0000","last_became_peered":"2024-09-10T10:01:20.229451+0000","last_unstale":"2024-09-10T10:01:22.567268+0000","last_undegraded":"2024-09-10T10:01:22.567268+0000","last_fullsized":"2024-09-10T10:01:22.567268+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:33:39.478388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567449+0000","last_change":"2024-09-10T10:01:21.252000+0000","last_active":"2024-09-10T10:01:22.567449+0000","last_peered":"2024-09-10T10:01:22.567449+0000","last_clean":"2024-09-10T10:01:22.567449+0000","last_became_active":"2024-09-10T10:01:21.251788+0000","last_became_peered":"2024-09-10T10:01:21.251788+0000","last_unstale":"2024-09-10T10:01:22.567449+0000","last_undegraded":"2024-09-10T10:01:22.567449+0000","last_fullsized":"2024-09-10T10:01:22.567449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:57.393646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567271+0000","last_change":"2024-09-10T10:01:20.226645+0000","last_active":"2024-09-10T10:01:22.567271+0000","last_peered":"2024-09-10T10:01:22.567271+0000","last_clean":"2024-09-10T10:01:22.567271+0000","last_became_active":"2024-09-10T10:01:20.226312+0000","last_became_peered":"2024-09-10T10:01:20.226312+0000","last_unstale":"2024-09-10T10:01:22.567271+0000","last_undegraded":"2024-09-10T10:01:22.567271+0000","last_fullsized":"2024-09-10T10:01:22.567271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:33:39.478388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568266+0000","last_change":"2024-09-10T10:01:21.240697+0000","last_active":"2024-09-10T10:01:22.568266+0000","last_peered":"2024-09-10T10:01:22.568266+0000","last_clean":"2024-09-10T10:01:22.568266+0000","last_became_active":"2024-09-10T10:01:21.240429+0000","last_became_peered":"2024-09-10T10:01:21.240429+0000","last_unstale":"2024-09-10T10:01:22.568266+0000","last_undegraded":"2024-09-10T10:01:22.568266+0000","last_fullsized":"2024-09-10T10:01:22.568266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:16.260334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.253159+0000","last_change":"2024-09-10T10:01:20.227068+0000","last_active":"2024-09-10T10:01:21.253159+0000","last_peered":"2024-09-10T10:01:21.253159+0000","last_clean":"2024-09-10T10:01:21.253159+0000","last_became_active":"2024-09-10T10:01:20.226824+0000","last_became_peered":"2024-09-10T10:01:20.226824+0000","last_unstale":"2024-09-10T10:01:21.253159+0000","last_undegraded":"2024-09-10T10:01:21.253159+0000","last_fullsized":"2024-09-10T10:01:21.253159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:17.682399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568319+0000","last_change":"2024-09-10T10:01:21.234365+0000","last_active":"2024-09-10T10:01:22.568319+0000","last_peered":"2024-09-10T10:01:22.568319+0000","last_clean":"2024-09-10T10:01:22.568319+0000","last_became_active":"2024-09-10T10:01:21.234098+0000","last_became_peered":"2024-09-10T10:01:21.234098+0000","last_unstale":"2024-09-10T10:01:22.568319+0000","last_undegraded":"2024-09-10T10:01:22.568319+0000","last_fullsized":"2024-09-10T10:01:22.568319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:01.053965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567765+0000","last_change":"2024-09-10T10:01:20.231394+0000","last_active":"2024-09-10T10:01:22.567765+0000","last_peered":"2024-09-10T10:01:22.567765+0000","last_clean":"2024-09-10T10:01:22.567765+0000","last_became_active":"2024-09-10T10:01:20.230865+0000","last_became_peered":"2024-09-10T10:01:20.230865+0000","last_unstale":"2024-09-10T10:01:22.567765+0000","last_undegraded":"2024-09-10T10:01:22.567765+0000","last_fullsized":"2024-09-10T10:01:22.567765+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:56.615865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.234935+0000","last_change":"2024-09-10T10:01:21.226356+0000","last_active":"2024-09-10T10:01:21.234935+0000","last_peered":"2024-09-10T10:01:21.234935+0000","last_clean":"2024-09-10T10:01:21.234935+0000","last_became_active":"2024-09-10T10:01:21.225952+0000","last_became_peered":"2024-09-10T10:01:21.225952+0000","last_unstale":"2024-09-10T10:01:21.234935+0000","last_undegraded":"2024-09-10T10:01:21.234935+0000","last_fullsized":"2024-09-10T10:01:21.234935+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:15.527913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568014+0000","last_change":"2024-09-10T10:01:20.229567+0000","last_active":"2024-09-10T10:01:22.568014+0000","last_peered":"2024-09-10T10:01:22.568014+0000","last_clean":"2024-09-10T10:01:22.568014+0000","last_became_active":"2024-09-10T10:01:20.229235+0000","last_became_peered":"2024-09-10T10:01:20.229235+0000","last_unstale":"2024-09-10T10:01:22.568014+0000","last_undegraded":"2024-09-10T10:01:22.568014+0000","last_fullsized":"2024-09-10T10:01:22.568014+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:00.514446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568377+0000","last_change":"2024-09-10T10:01:21.242915+0000","last_active":"2024-09-10T10:01:22.568377+0000","last_peered":"2024-09-10T10:01:22.568377+0000","last_clean":"2024-09-10T10:01:22.568377+0000","last_became_active":"2024-09-10T10:01:21.241792+0000","last_became_peered":"2024-09-10T10:01:21.241792+0000","last_unstale":"2024-09-10T10:01:22.568377+0000","last_undegraded":"2024-09-10T10:01:22.568377+0000","last_fullsized":"2024-09-10T10:01:22.568377+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:54.480722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567422+0000","last_change":"2024-09-10T10:01:20.232468+0000","last_active":"2024-09-10T10:01:22.567422+0000","last_peered":"2024-09-10T10:01:22.567422+0000","last_clean":"2024-09-10T10:01:22.567422+0000","last_became_active":"2024-09-10T10:01:20.232222+0000","last_became_peered":"2024-09-10T10:01:20.232222+0000","last_unstale":"2024-09-10T10:01:22.567422+0000","last_undegraded":"2024-09-10T10:01:22.567422+0000","last_fullsized":"2024-09-10T10:01:22.567422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:37.025460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567900+0000","last_change":"2024-09-10T10:01:21.234080+0000","last_active":"2024-09-10T10:01:22.567900+0000","last_peered":"2024-09-10T10:01:22.567900+0000","last_clean":"2024-09-10T10:01:22.567900+0000","last_became_active":"2024-09-10T10:01:21.233812+0000","last_became_peered":"2024-09-10T10:01:21.233812+0000","last_unstale":"2024-09-10T10:01:22.567900+0000","last_undegraded":"2024-09-10T10:01:22.567900+0000","last_fullsized":"2024-09-10T10:01:22.567900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:17.036026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.253931+0000","last_change":"2024-09-10T10:01:20.222755+0000","last_active":"2024-09-10T10:01:21.253931+0000","last_peered":"2024-09-10T10:01:21.253931+0000","last_clean":"2024-09-10T10:01:21.253931+0000","last_became_active":"2024-09-10T10:01:20.222502+0000","last_became_peered":"2024-09-10T10:01:20.222502+0000","last_unstale":"2024-09-10T10:01:21.253931+0000","last_undegraded":"2024-09-10T10:01:21.253931+0000","last_fullsized":"2024-09-10T10:01:21.253931+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:31.821517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567233+0000","last_change":"2024-09-10T10:01:21.223608+0000","last_active":"2024-09-10T10:01:22.567233+0000","last_peered":"2024-09-10T10:01:22.567233+0000","last_clean":"2024-09-10T10:01:22.567233+0000","last_became_active":"2024-09-10T10:01:21.223353+0000","last_became_peered":"2024-09-10T10:01:21.223353+0000","last_unstale":"2024-09-10T10:01:22.567233+0000","last_undegraded":"2024-09-10T10:01:22.567233+0000","last_fullsized":"2024-09-10T10:01:22.567233+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:21.760367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227046+0000","last_change":"2024-09-10T10:01:20.227347+0000","last_active":"2024-09-10T10:01:22.227046+0000","last_peered":"2024-09-10T10:01:22.227046+0000","last_clean":"2024-09-10T10:01:22.227046+0000","last_became_active":"2024-09-10T10:01:20.227098+0000","last_became_peered":"2024-09-10T10:01:20.227098+0000","last_unstale":"2024-09-10T10:01:22.227046+0000","last_undegraded":"2024-09-10T10:01:22.227046+0000","last_fullsized":"2024-09-10T10:01:22.227046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:41.718511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.566731+0000","last_change":"2024-09-10T10:01:21.239991+0000","last_active":"2024-09-10T10:01:22.566731+0000","last_peered":"2024-09-10T10:01:22.566731+0000","last_clean":"2024-09-10T10:01:22.566731+0000","last_became_active":"2024-09-10T10:01:21.239706+0000","last_became_peered":"2024-09-10T10:01:21.239706+0000","last_unstale":"2024-09-10T10:01:22.566731+0000","last_undegraded":"2024-09-10T10:01:22.566731+0000","last_fullsized":"2024-09-10T10:01:22.566731+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:25.977903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567111+0000","last_change":"2024-09-10T10:01:20.232430+0000","last_active":"2024-09-10T10:01:22.567111+0000","last_peered":"2024-09-10T10:01:22.567111+0000","last_clean":"2024-09-10T10:01:22.567111+0000","last_became_active":"2024-09-10T10:01:20.232190+0000","last_became_peered":"2024-09-10T10:01:20.232190+0000","last_unstale":"2024-09-10T10:01:22.567111+0000","last_undegraded":"2024-09-10T10:01:22.567111+0000","last_fullsized":"2024-09-10T10:01:22.567111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:37.025460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568107+0000","last_change":"2024-09-10T10:01:21.230457+0000","last_active":"2024-09-10T10:01:22.568107+0000","last_peered":"2024-09-10T10:01:22.568107+0000","last_clean":"2024-09-10T10:01:22.568107+0000","last_became_active":"2024-09-10T10:01:21.230179+0000","last_became_peered":"2024-09-10T10:01:21.230179+0000","last_unstale":"2024-09-10T10:01:22.568107+0000","last_undegraded":"2024-09-10T10:01:22.568107+0000","last_fullsized":"2024-09-10T10:01:22.568107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:25.977903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227500+0000","last_change":"2024-09-10T10:01:20.225516+0000","last_active":"2024-09-10T10:01:22.227500+0000","last_peered":"2024-09-10T10:01:22.227500+0000","last_clean":"2024-09-10T10:01:22.227500+0000","last_became_active":"2024-09-10T10:01:20.225249+0000","last_became_peered":"2024-09-10T10:01:20.225249+0000","last_unstale":"2024-09-10T10:01:22.227500+0000","last_undegraded":"2024-09-10T10:01:22.227500+0000","last_fullsized":"2024-09-10T10:01:22.227500+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.007582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.568328+0000","last_change":"2024-09-10T10:01:21.234344+0000","last_active":"2024-09-10T10:01:22.568328+0000","last_peered":"2024-09-10T10:01:22.568328+0000","last_clean":"2024-09-10T10:01:22.568328+0000","last_became_active":"2024-09-10T10:01:21.234054+0000","last_became_peered":"2024-09-10T10:01:21.234054+0000","last_unstale":"2024-09-10T10:01:22.568328+0000","last_undegraded":"2024-09-10T10:01:22.568328+0000","last_fullsized":"2024-09-10T10:01:22.568328+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:07.802751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567887+0000","last_change":"2024-09-10T10:01:20.229475+0000","last_active":"2024-09-10T10:01:22.567887+0000","last_peered":"2024-09-10T10:01:22.567887+0000","last_clean":"2024-09-10T10:01:22.567887+0000","last_became_active":"2024-09-10T10:01:20.229160+0000","last_became_peered":"2024-09-10T10:01:20.229160+0000","last_unstale":"2024-09-10T10:01:22.567887+0000","last_undegraded":"2024-09-10T10:01:22.567887+0000","last_fullsized":"2024-09-10T10:01:22.567887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:47.790606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.227230+0000","last_change":"2024-09-10T10:01:21.227334+0000","last_active":"2024-09-10T10:01:22.227230+0000","last_peered":"2024-09-10T10:01:22.227230+0000","last_clean":"2024-09-10T10:01:22.227230+0000","last_became_active":"2024-09-10T10:01:21.227065+0000","last_became_peered":"2024-09-10T10:01:21.227065+0000","last_unstale":"2024-09-10T10:01:22.227230+0000","last_undegraded":"2024-09-10T10:01:22.227230+0000","last_fullsized":"2024-09-10T10:01:22.227230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:17.075965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.233471+0000","last_change":"2024-09-10T10:01:20.221240+0000","last_active":"2024-09-10T10:01:22.233471+0000","last_peered":"2024-09-10T10:01:22.233471+0000","last_clean":"2024-09-10T10:01:22.233471+0000","last_became_active":"2024-09-10T10:01:20.220953+0000","last_became_peered":"2024-09-10T10:01:20.220953+0000","last_unstale":"2024-09-10T10:01:22.233471+0000","last_undegraded":"2024-09-10T10:01:22.233471+0000","last_fullsized":"2024-09-10T10:01:22.233471+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:01.938706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567636+0000","last_change":"2024-09-10T10:01:21.234817+0000","last_active":"2024-09-10T10:01:22.567636+0000","last_peered":"2024-09-10T10:01:22.567636+0000","last_clean":"2024-09-10T10:01:22.567636+0000","last_became_active":"2024-09-10T10:01:21.233837+0000","last_became_peered":"2024-09-10T10:01:21.233837+0000","last_unstale":"2024-09-10T10:01:22.567636+0000","last_undegraded":"2024-09-10T10:01:22.567636+0000","last_fullsized":"2024-09-10T10:01:22.567636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:54.480722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567865+0000","last_change":"2024-09-10T10:01:20.228548+0000","last_active":"2024-09-10T10:01:22.567865+0000","last_peered":"2024-09-10T10:01:22.567865+0000","last_clean":"2024-09-10T10:01:22.567865+0000","last_became_active":"2024-09-10T10:01:20.228179+0000","last_became_peered":"2024-09-10T10:01:20.228179+0000","last_unstale":"2024-09-10T10:01:22.567865+0000","last_undegraded":"2024-09-10T10:01:22.567865+0000","last_fullsized":"2024-09-10T10:01:22.567865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:09.685892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.229641+0000","last_change":"2024-09-10T10:01:21.226666+0000","last_active":"2024-09-10T10:01:21.229641+0000","last_peered":"2024-09-10T10:01:21.229641+0000","last_clean":"2024-09-10T10:01:21.229641+0000","last_became_active":"2024-09-10T10:01:21.226335+0000","last_became_peered":"2024-09-10T10:01:21.226335+0000","last_unstale":"2024-09-10T10:01:21.229641+0000","last_undegraded":"2024-09-10T10:01:21.229641+0000","last_fullsized":"2024-09-10T10:01:21.229641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:20.173572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.226928+0000","last_change":"2024-09-10T10:01:20.226169+0000","last_active":"2024-09-10T10:01:22.226928+0000","last_peered":"2024-09-10T10:01:22.226928+0000","last_clean":"2024-09-10T10:01:22.226928+0000","last_became_active":"2024-09-10T10:01:20.225926+0000","last_became_peered":"2024-09-10T10:01:20.225926+0000","last_unstale":"2024-09-10T10:01:22.226928+0000","last_undegraded":"2024-09-10T10:01:22.226928+0000","last_fullsized":"2024-09-10T10:01:22.226928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:21.549893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567909+0000","last_change":"2024-09-10T10:01:21.248484+0000","last_active":"2024-09-10T10:01:22.567909+0000","last_peered":"2024-09-10T10:01:22.567909+0000","last_clean":"2024-09-10T10:01:22.567909+0000","last_became_active":"2024-09-10T10:01:21.248250+0000","last_became_peered":"2024-09-10T10:01:21.248250+0000","last_unstale":"2024-09-10T10:01:22.567909+0000","last_undegraded":"2024-09-10T10:01:22.567909+0000","last_fullsized":"2024-09-10T10:01:22.567909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.508961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.232295+0000","last_change":"2024-09-10T10:01:20.225932+0000","last_active":"2024-09-10T10:01:22.232295+0000","last_peered":"2024-09-10T10:01:22.232295+0000","last_clean":"2024-09-10T10:01:22.232295+0000","last_became_active":"2024-09-10T10:01:20.225640+0000","last_became_peered":"2024-09-10T10:01:20.225640+0000","last_unstale":"2024-09-10T10:01:22.232295+0000","last_undegraded":"2024-09-10T10:01:22.232295+0000","last_fullsized":"2024-09-10T10:01:22.232295+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:04.477676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.226911+0000","last_change":"2024-09-10T10:01:21.237356+0000","last_active":"2024-09-10T10:01:22.226911+0000","last_peered":"2024-09-10T10:01:22.226911+0000","last_clean":"2024-09-10T10:01:22.226911+0000","last_became_active":"2024-09-10T10:01:21.236059+0000","last_became_peered":"2024-09-10T10:01:21.236059+0000","last_unstale":"2024-09-10T10:01:22.226911+0000","last_undegraded":"2024-09-10T10:01:22.226911+0000","last_fullsized":"2024-09-10T10:01:22.226911+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:40.729701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.226955+0000","last_change":"2024-09-10T10:01:20.226853+0000","last_active":"2024-09-10T10:01:22.226955+0000","last_peered":"2024-09-10T10:01:22.226955+0000","last_clean":"2024-09-10T10:01:22.226955+0000","last_became_active":"2024-09-10T10:01:20.223751+0000","last_became_peered":"2024-09-10T10:01:20.223751+0000","last_unstale":"2024-09-10T10:01:22.226955+0000","last_undegraded":"2024-09-10T10:01:22.226955+0000","last_fullsized":"2024-09-10T10:01:22.226955+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.446874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235049+0000","last_change":"2024-09-10T10:01:21.231732+0000","last_active":"2024-09-10T10:01:21.235049+0000","last_peered":"2024-09-10T10:01:21.235049+0000","last_clean":"2024-09-10T10:01:21.235049+0000","last_became_active":"2024-09-10T10:01:21.231438+0000","last_became_peered":"2024-09-10T10:01:21.231438+0000","last_unstale":"2024-09-10T10:01:21.235049+0000","last_undegraded":"2024-09-10T10:01:21.235049+0000","last_fullsized":"2024-09-10T10:01:21.235049+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:24.198868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.218540+0000","last_change":"2024-09-10T10:01:20.224508+0000","last_active":"2024-09-10T10:01:21.218540+0000","last_peered":"2024-09-10T10:01:21.218540+0000","last_clean":"2024-09-10T10:01:21.218540+0000","last_became_active":"2024-09-10T10:01:20.224244+0000","last_became_peered":"2024-09-10T10:01:20.224244+0000","last_unstale":"2024-09-10T10:01:21.218540+0000","last_undegraded":"2024-09-10T10:01:21.218540+0000","last_fullsized":"2024-09-10T10:01:21.218540+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:09.237904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T10:01:21.235445+0000","last_change":"2024-09-10T10:01:21.234344+0000","last_active":"2024-09-10T10:01:21.235445+0000","last_peered":"2024-09-10T10:01:21.235445+0000","last_clean":"2024-09-10T10:01:21.235445+0000","last_became_active":"2024-09-10T10:01:21.233859+0000","last_became_peered":"2024-09-10T10:01:21.233859+0000","last_unstale":"2024-09-10T10:01:21.235445+0000","last_undegraded":"2024-09-10T10:01:21.235445+0000","last_fullsized":"2024-09-10T10:01:21.235445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:20.198623+0000","last_clean_scrub_stamp":"2024-09-10T10:01:20.198623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:28:28.917095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.233182+0000","last_change":"2024-09-10T10:01:20.225688+0000","last_active":"2024-09-10T10:01:22.233182+0000","last_peered":"2024-09-10T10:01:22.233182+0000","last_clean":"2024-09-10T10:01:22.233182+0000","last_became_active":"2024-09-10T10:01:20.224529+0000","last_became_peered":"2024-09-10T10:01:20.224529+0000","last_unstale":"2024-09-10T10:01:22.233182+0000","last_undegraded":"2024-09-10T10:01:22.233182+0000","last_fullsized":"2024-09-10T10:01:22.233182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:19.194933+0000","last_clean_scrub_stamp":"2024-09-10T10:01:19.194933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:36.771898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":234,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T10:01:22.567213+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:01:22.567213+0000","last_peered":"2024-09-10T10:01:22.567213+0000","last_clean":"2024-09-10T10:01:22.567213+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:01:22.567213+0000","last_undegraded":"2024-09-10T10:01:22.567213+0000","last_fullsized":"2024-09-10T10:01:22.567213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":866917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":277041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":688128,"data_stored":278001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":696320,"data_stored":282745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":679936,"data_stored":276479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1286144,"data_stored":868951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":684032,"data_stored":276445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:01:25.696 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:01:25.696 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:01:25.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb7200640 1 Processor -- start 2024-09-10T10:01:25.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb7200640 1 -- start start 2024-09-10T10:01:25.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb7200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80b5da0 0x7f5fb8073f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb7200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5fb80712f0 0x7f5fb815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb7200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb815ccf0 con 0x7f5fb8145c70 2024-09-10T10:01:25.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb7200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb815cab0 con 0x7f5fb80712f0 2024-09-10T10:01:25.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb7200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb8072040 con 0x7f5fb80b5da0 2024-09-10T10:01:25.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb6800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5fb80712f0 0x7f5fb815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb5400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80b5da0 0x7f5fb8073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb6800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5fb80712f0 0x7f5fb815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:55656/0 (socket says 172.21.15.110:55656) 2024-09-10T10:01:25.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb6800640 1 -- 172.21.15.110:0/4174237635 learned_addr learned my addr 172.21.15.110:0/4174237635 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:25.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb5400640 1 -- 172.21.15.110:0/4174237635 >> v2:172.21.15.145:3300/0 conn(0x7f5fb80712f0 msgr2=0x7f5fb815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:25.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb5400640 1 --2- 172.21.15.110:0/4174237635 >> v2:172.21.15.145:3300/0 conn(0x7f5fb80712f0 0x7f5fb815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb5400640 1 -- 172.21.15.110:0/4174237635 >> v1:172.21.15.145:6789/0 conn(0x7f5fb8145c70 legacy=0x7f5fb80b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb4a00640 1 -- 172.21.15.110:0/4174237635 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 815336126 0 0) 0x7f5fb815ccf0 con 0x7f5fb8145c70 2024-09-10T10:01:25.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb5400640 1 -- 172.21.15.110:0/4174237635 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb8071a80 con 0x7f5fb80b5da0 2024-09-10T10:01:25.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.872+0000 7f5fb6800640 1 --2- 172.21.15.110:0/4174237635 >> v2:172.21.15.145:3300/0 conn(0x7f5fb80712f0 0x7f5fb815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:25.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb5400640 1 --2- 172.21.15.110:0/4174237635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80b5da0 0x7f5fb8073f10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097c0 tx=0x7f5fa401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a5571cc561d100 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb4a00640 1 -- 172.21.15.110:0/4174237635 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40b0070 con 0x7f5fb80b5da0 2024-09-10T10:01:25.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb4a00640 1 -- 172.21.15.110:0/4174237635 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa40ab600 con 0x7f5fb80b5da0 2024-09-10T10:01:25.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb4a00640 1 -- 172.21.15.110:0/4174237635 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40ab9d0 con 0x7f5fb80b5da0 2024-09-10T10:01:25.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb7200640 1 -- 172.21.15.110:0/4174237635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80b5da0 msgr2=0x7f5fb8073f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb7200640 1 --2- 172.21.15.110:0/4174237635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80b5da0 0x7f5fb8073f10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097c0 tx=0x7f5fa401c410 comp rx=0 tx=0).stop 2024-09-10T10:01:25.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb7200640 1 -- 172.21.15.110:0/4174237635 shutdown_connections 2024-09-10T10:01:25.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb7200640 1 --2- 172.21.15.110:0/4174237635 >> v2:172.21.15.145:3300/0 conn(0x7f5fb80712f0 0x7f5fb815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb7200640 1 --2- 172.21.15.110:0/4174237635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80b5da0 0x7f5fb8073f10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb7200640 1 -- 172.21.15.110:0/4174237635 >> 172.21.15.110:0/4174237635 conn(0x7f5fb806d0f0 msgr2=0x7f5fb806d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:25.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb7200640 1 -- 172.21.15.110:0/4174237635 shutdown_connections 2024-09-10T10:01:25.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.876+0000 7f5fb7200640 1 -- 172.21.15.110:0/4174237635 wait complete. 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 Processor -- start 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 -- start start 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80712f0 0x7f5fb813bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fb80b5da0 0x7f5fb8159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fb8145c70 0x7f5fb813f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb815e220 con 0x7f5fb8145c70 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8146260 con 0x7f5fb80b5da0 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb815e3a0 con 0x7f5fb80712f0 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fb8145c70 0x7f5fb813f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fb8145c70 0x7f5fb813f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35868/0 (socket says 172.21.15.110:35868) 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 -- 172.21.15.110:0/1672307099 learned_addr learned my addr 172.21.15.110:0/1672307099 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb5e00640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80712f0 0x7f5fb813bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb5400640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fb80b5da0 0x7f5fb8159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 -- 172.21.15.110:0/1672307099 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80712f0 msgr2=0x7f5fb813bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80712f0 0x7f5fb813bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 -- 172.21.15.110:0/1672307099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fb80b5da0 msgr2=0x7f5fb8159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fb80b5da0 0x7f5fb8159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 -- 172.21.15.110:0/1672307099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb813c200 con 0x7f5fb8145c70 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb6800640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fb8145c70 0x7f5fb813f1b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000efc0 tx=0x7f5fb000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fae000640 1 -- 172.21.15.110:0/1672307099 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000cbe0 con 0x7f5fb8145c70 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fae000640 1 -- 172.21.15.110:0/1672307099 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb000cd80 con 0x7f5fb8145c70 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb813c4e0 con 0x7f5fb8145c70 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fae000640 1 -- 172.21.15.110:0/1672307099 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0003bc0 con 0x7f5fb8145c70 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.880+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb80b3890 con 0x7f5fb8145c70 2024-09-10T10:01:25.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.884+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb8069bb0 con 0x7f5fb8145c70 2024-09-10T10:01:25.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.884+0000 7f5fae000640 1 -- 172.21.15.110:0/1672307099 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f5fb001b020 con 0x7f5fb8145c70 2024-09-10T10:01:25.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.884+0000 7f5fae000640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5f98067a70 0x7f5f98069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.884+0000 7f5fae000640 1 -- 172.21.15.110:0/1672307099 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5fb0086370 con 0x7f5fb8145c70 2024-09-10T10:01:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.884+0000 7f5fb5e00640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5f98067a70 0x7f5f98069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.884+0000 7f5fb5e00640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5f98067a70 0x7f5f98069f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8007fd0 tx=0x7f5fa8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:25.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:25.888+0000 7f5fae000640 1 -- 172.21.15.110:0/1672307099 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5fb0022020 con 0x7f5fb8145c70 2024-09-10T10:01:26.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.100+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5fb815e780 con 0x7f5fb8145c70 2024-09-10T10:01:26.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fae000640 1 -- 172.21.15.110:0/1672307099 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f5fb0010040 con 0x7f5fb8145c70 2024-09-10T10:01:26.107 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:01:26.107 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:01:26.107 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:01:26.107 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:01:26.107 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:01:26.109 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:01:26.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5f98067a70 msgr2=0x7f5f98069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5f98067a70 0x7f5f98069f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8007fd0 tx=0x7f5fa8009040 comp rx=0 tx=0).stop 2024-09-10T10:01:26.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fb8145c70 msgr2=0x7f5fb813f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fb8145c70 0x7f5fb813f1b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000efc0 tx=0x7f5fb000c490 comp rx=0 tx=0).stop 2024-09-10T10:01:26.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 shutdown_connections 2024-09-10T10:01:26.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5f98067a70 0x7f5f98069f30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fb8145c70 0x7f5fb813f1b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fb80b5da0 0x7f5fb8159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 --2- 172.21.15.110:0/1672307099 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fb80712f0 0x7f5fb813bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 >> 172.21.15.110:0/1672307099 conn(0x7f5fb806d0f0 msgr2=0x7f5fb80ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 shutdown_connections 2024-09-10T10:01:26.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.104+0000 7f5fb7200640 1 -- 172.21.15.110:0/1672307099 wait complete. 2024-09-10T10:01:26.127 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:26.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.260+0000 7fdb75e00640 1 Processor -- start 2024-09-10T10:01:26.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.260+0000 7fdb75e00640 1 -- start start 2024-09-10T10:01:26.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb75e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdb700cfe30 0x7fdb70071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb75e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdb700cd510 0x7fdb700cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb75e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb7006d2d0 con 0x7fdb7006e090 2024-09-10T10:01:26.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb75e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7015d9a0 con 0x7fdb700cd510 2024-09-10T10:01:26.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb75e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7006d5b0 con 0x7fdb700cfe30 2024-09-10T10:01:26.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fdb7006e090 0x7fdb7006e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45910/0 (socket says 172.21.15.110:45910) 2024-09-10T10:01:26.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6e000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdb700cd510 0x7fdb700cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6f400640 1 -- 172.21.15.110:0/3964344196 learned_addr learned my addr 172.21.15.110:0/3964344196 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:26.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6ea00640 1 --2- 172.21.15.110:0/3964344196 >> v2:172.21.15.145:3300/0 conn(0x7fdb700cfe30 0x7fdb70071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6e000640 1 -- 172.21.15.110:0/3964344196 >> v2:172.21.15.145:3300/0 conn(0x7fdb700cfe30 msgr2=0x7fdb70071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6e000640 1 --2- 172.21.15.110:0/3964344196 >> v2:172.21.15.145:3300/0 conn(0x7fdb700cfe30 0x7fdb70071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6e000640 1 -- 172.21.15.110:0/3964344196 >> v1:172.21.15.110:6789/0 conn(0x7fdb7006e090 legacy=0x7fdb7006e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6e000640 1 -- 172.21.15.110:0/3964344196 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7006d890 con 0x7fdb700cd510 2024-09-10T10:01:26.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.264+0000 7fdb6d600640 1 -- 172.21.15.110:0/3964344196 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584744314 0 0) 0x7fdb7006d2d0 con 0x7fdb7006e090 2024-09-10T10:01:26.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb6e000640 1 --2- 172.21.15.110:0/3964344196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb700cd510 0x7fdb700cd9a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdb700ce120 tx=0x7fdb5c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cffd4466b9527ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb6d600640 1 -- 172.21.15.110:0/3964344196 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0b0070 con 0x7fdb700cd510 2024-09-10T10:01:26.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb6d600640 1 -- 172.21.15.110:0/3964344196 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb5c0ab600 con 0x7fdb700cd510 2024-09-10T10:01:26.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb6d600640 1 -- 172.21.15.110:0/3964344196 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0ab9d0 con 0x7fdb700cd510 2024-09-10T10:01:26.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 -- 172.21.15.110:0/3964344196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb700cd510 msgr2=0x7fdb700cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 --2- 172.21.15.110:0/3964344196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb700cd510 0x7fdb700cd9a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdb700ce120 tx=0x7fdb5c01c3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 -- 172.21.15.110:0/3964344196 shutdown_connections 2024-09-10T10:01:26.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 --2- 172.21.15.110:0/3964344196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb700cd510 0x7fdb700cd9a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 --2- 172.21.15.110:0/3964344196 >> v2:172.21.15.145:3300/0 conn(0x7fdb700cfe30 0x7fdb70071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 -- 172.21.15.110:0/3964344196 >> 172.21.15.110:0/3964344196 conn(0x7fdb7006bc50 msgr2=0x7fdb7006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 -- 172.21.15.110:0/3964344196 shutdown_connections 2024-09-10T10:01:26.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 -- 172.21.15.110:0/3964344196 wait complete. 2024-09-10T10:01:26.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 Processor -- start 2024-09-10T10:01:26.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.268+0000 7fdb75e00640 1 -- start start 2024-09-10T10:01:26.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7006e090 0x7fdb70153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb700cd510 0x7fdb70148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb700cfe30 0x7fdb70151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70072190 con 0x7fdb7006e090 2024-09-10T10:01:26.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7015d6c0 con 0x7fdb700cfe30 2024-09-10T10:01:26.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdb70169180 con 0x7fdb700cd510 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7006e090 0x7fdb70153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7006e090 0x7fdb70153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35876/0 (socket says 172.21.15.110:35876) 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 -- 172.21.15.110:0/523500665 learned_addr learned my addr 172.21.15.110:0/523500665 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6f400640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb700cfe30 0x7fdb70151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6e000640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb700cd510 0x7fdb70148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 -- 172.21.15.110:0/523500665 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb700cd510 msgr2=0x7fdb70148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb700cd510 0x7fdb70148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 -- 172.21.15.110:0/523500665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb700cfe30 msgr2=0x7fdb70151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb700cfe30 0x7fdb70151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 -- 172.21.15.110:0/523500665 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb700b0040 con 0x7fdb7006e090 2024-09-10T10:01:26.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb6ea00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7006e090 0x7fdb70153f20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000efc0 tx=0x7fdb6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb67400640 1 -- 172.21.15.110:0/523500665 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60018020 con 0x7fdb7006e090 2024-09-10T10:01:26.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb67400640 1 -- 172.21.15.110:0/523500665 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb60014480 con 0x7fdb7006e090 2024-09-10T10:01:26.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb67400640 1 -- 172.21.15.110:0/523500665 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60014850 con 0x7fdb7006e090 2024-09-10T10:01:26.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb700b02e0 con 0x7fdb7006e090 2024-09-10T10:01:26.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb700b08a0 con 0x7fdb7006e090 2024-09-10T10:01:26.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7006c1d0 con 0x7fdb7006e090 2024-09-10T10:01:26.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.272+0000 7fdb67400640 1 -- 172.21.15.110:0/523500665 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fdb6001b020 con 0x7fdb7006e090 2024-09-10T10:01:26.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.276+0000 7fdb67400640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdb50067a70 0x7fdb50069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.276+0000 7fdb67400640 1 -- 172.21.15.110:0/523500665 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fdb60086d90 con 0x7fdb7006e090 2024-09-10T10:01:26.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.276+0000 7fdb6e000640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdb50067a70 0x7fdb50069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.276+0000 7fdb6e000640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdb50067a70 0x7fdb50069f30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0ad000 tx=0x7fdb5c0a4f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.280+0000 7fdb67400640 1 -- 172.21.15.110:0/523500665 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdb60022030 con 0x7fdb7006e090 2024-09-10T10:01:26.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.468+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb701532a0 con 0x7fdb7006e090 2024-09-10T10:01:26.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.468+0000 7fdb67400640 1 -- 172.21.15.110:0/523500665 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5235 (secure 0 0 0) 0x7fdb60010040 con 0x7fdb7006e090 2024-09-10T10:01:26.472 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:26.472 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":36,"btime":"2024-09-10T10:01:24:235432+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/2645162871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2645162871},{"type":"v1","addr":"172.21.15.145:6834","nonce":2645162871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1524995501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1524995501},{"type":"v1","addr":"172.21.15.145:6835","nonce":1524995501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/2642324353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2642324353},{"type":"v1","addr":"172.21.15.110:6836","nonce":2642324353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:20.586417+0000","modified":"2024-09-10T10:01:23.606983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1333637048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":1333637048},{"type":"v1","addr":"172.21.15.110:6837","nonce":1333637048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5640,"qdb_cluster":[5640]},"id":4}]} 2024-09-10T10:01:26.472 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 36 2024-09-10T10:01:26.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdb50067a70 msgr2=0x7fdb50069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdb50067a70 0x7fdb50069f30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0ad000 tx=0x7fdb5c0a4f90 comp rx=0 tx=0).stop 2024-09-10T10:01:26.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7006e090 msgr2=0x7fdb70153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7006e090 0x7fdb70153f20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000efc0 tx=0x7fdb6000c490 comp rx=0 tx=0).stop 2024-09-10T10:01:26.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 shutdown_connections 2024-09-10T10:01:26.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fdb50067a70 0x7fdb50069f30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb700cfe30 0x7fdb70151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb700cd510 0x7fdb70148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 --2- 172.21.15.110:0/523500665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb7006e090 0x7fdb70153f20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 >> 172.21.15.110:0/523500665 conn(0x7fdb7006bc50 msgr2=0x7fdb7006e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 shutdown_connections 2024-09-10T10:01:26.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:26.472+0000 7fdb75e00640 1 -- 172.21.15.110:0/523500665 wait complete. 2024-09-10T10:01:26.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:01:20.586417+0000', 'modified': '2024-09-10T10:01:23.606983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5640}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5640': {'gid': 5640, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6837/1333637048', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6835', 'nonce': 1333637048}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 1333637048}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5640, 'qdb_cluster': [5640]} 2024-09-10T10:01:26.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:01:26.897 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:01:27.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9ca5c00640 1 Processor -- start 2024-09-10T10:01:27.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9ca5c00640 1 -- start start 2024-09-10T10:01:27.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9ca5c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9ca015e540 0x7f9ca00b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9ca5c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00626e0 0x7f9ca00af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9ca5c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca00ae9e0 con 0x7f9ca00b5880 2024-09-10T10:01:27.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9ca5c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca00aecc0 con 0x7f9ca015e540 2024-09-10T10:01:27.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9ca5c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca00aefa0 con 0x7f9ca00626e0 2024-09-10T10:01:27.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00626e0 0x7f9ca00af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9f400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f9ca00b5880 0x7f9ca006db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:38008/0 (socket says 172.21.15.110:38008) 2024-09-10T10:01:27.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9f400640 1 -- 172.21.15.110:0/2119089803 learned_addr learned my addr 172.21.15.110:0/2119089803 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:27.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00626e0 0x7f9ca00af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:41334/0 (socket says 172.21.15.110:41334) 2024-09-10T10:01:27.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/2119089803 >> v2:172.21.15.145:3300/0 conn(0x7f9ca015e540 0x7f9ca00b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9e000640 1 -- 172.21.15.110:0/2119089803 >> v2:172.21.15.145:3300/0 conn(0x7f9ca015e540 msgr2=0x7f9ca00b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9e000640 1 --2- 172.21.15.110:0/2119089803 >> v2:172.21.15.145:3300/0 conn(0x7f9ca015e540 0x7f9ca00b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9e000640 1 -- 172.21.15.110:0/2119089803 >> v1:172.21.15.145:6789/0 conn(0x7f9ca00b5880 legacy=0x7f9ca006db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9e000640 1 -- 172.21.15.110:0/2119089803 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca01621f0 con 0x7f9ca00626e0 2024-09-10T10:01:27.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.032+0000 7f9c9d600640 1 -- 172.21.15.110:0/2119089803 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847198351 0 0) 0x7f9ca00ae9e0 con 0x7f9ca00b5880 2024-09-10T10:01:27.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9c9e000640 1 --2- 172.21.15.110:0/2119089803 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00626e0 0x7f9ca00af7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0097c0 tx=0x7f9c8c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd258f8be552f1ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9c9d600640 1 -- 172.21.15.110:0/2119089803 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0b0070 con 0x7f9ca00626e0 2024-09-10T10:01:27.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9c9d600640 1 -- 172.21.15.110:0/2119089803 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c0ab600 con 0x7f9ca00626e0 2024-09-10T10:01:27.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9c9d600640 1 -- 172.21.15.110:0/2119089803 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0ab9d0 con 0x7f9ca00626e0 2024-09-10T10:01:27.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 -- 172.21.15.110:0/2119089803 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00626e0 msgr2=0x7f9ca00af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/2119089803 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00626e0 0x7f9ca00af7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0097c0 tx=0x7f9c8c01c410 comp rx=0 tx=0).stop 2024-09-10T10:01:27.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 -- 172.21.15.110:0/2119089803 shutdown_connections 2024-09-10T10:01:27.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/2119089803 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00626e0 0x7f9ca00af7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/2119089803 >> v2:172.21.15.145:3300/0 conn(0x7f9ca015e540 0x7f9ca00b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 -- 172.21.15.110:0/2119089803 >> 172.21.15.110:0/2119089803 conn(0x7f9ca00b0970 msgr2=0x7f9ca00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 -- 172.21.15.110:0/2119089803 shutdown_connections 2024-09-10T10:01:27.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 -- 172.21.15.110:0/2119089803 wait complete. 2024-09-10T10:01:27.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.036+0000 7f9ca5c00640 1 Processor -- start 2024-09-10T10:01:27.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 -- start start 2024-09-10T10:01:27.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca00626e0 0x7f9ca01544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00b5880 0x7f9ca013ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca015e540 0x7f9ca015eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca016b710 con 0x7f9ca00626e0 2024-09-10T10:01:27.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca016b590 con 0x7f9ca015e540 2024-09-10T10:01:27.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca016b890 con 0x7f9ca00b5880 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca00626e0 0x7f9ca01544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca00626e0 0x7f9ca01544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35892/0 (socket says 172.21.15.110:35892) 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 -- 172.21.15.110:0/670376307 learned_addr learned my addr 172.21.15.110:0/670376307 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9e000640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00b5880 0x7f9ca013ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9f400640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca015e540 0x7f9ca015eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 -- 172.21.15.110:0/670376307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00b5880 msgr2=0x7f9ca013ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00b5880 0x7f9ca013ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 -- 172.21.15.110:0/670376307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca015e540 msgr2=0x7f9ca015eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca015e540 0x7f9ca015eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 -- 172.21.15.110:0/670376307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca0154c50 con 0x7f9ca00626e0 2024-09-10T10:01:27.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca00626e0 0x7f9ca01544b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000e8a0 tx=0x7f9c9000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c96a00640 1 -- 172.21.15.110:0/670376307 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90017070 con 0x7f9ca00626e0 2024-09-10T10:01:27.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c96a00640 1 -- 172.21.15.110:0/670376307 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9000c940 con 0x7f9ca00626e0 2024-09-10T10:01:27.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 -- 172.21.15.110:0/670376307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca015f280 con 0x7f9ca00626e0 2024-09-10T10:01:27.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9c96a00640 1 -- 172.21.15.110:0/670376307 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000ccf0 con 0x7f9ca00626e0 2024-09-10T10:01:27.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.040+0000 7f9ca5c00640 1 -- 172.21.15.110:0/670376307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca015f840 con 0x7f9ca00626e0 2024-09-10T10:01:27.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c96a00640 1 -- 172.21.15.110:0/670376307 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f9c9001a070 con 0x7f9ca00626e0 2024-09-10T10:01:27.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c96a00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9c80067a70 0x7f9c80069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c96a00640 1 -- 172.21.15.110:0/670376307 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9c90086170 con 0x7f9ca00626e0 2024-09-10T10:01:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9e000640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9c80067a70 0x7f9c80069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9ca8fab000 1 Processor -- start 2024-09-10T10:01:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9ca8fab000 1 -- start start 2024-09-10T10:01:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9e000640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9c80067a70 0x7f9c80069f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0ad000 tx=0x7f9c8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad435bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad4311d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4312290 0x564ad4312670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ad41f9b60 con 0x564ad435bb20 2024-09-10T10:01:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564ad4160c70 con 0x564ad435c440 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x564ad4106ed0 con 0x564ad4312290 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad435bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad435bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35902/0 (socket says 172.21.15.110:35902) 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 -- 172.21.15.110:0/4243762241 learned_addr learned my addr 172.21.15.110:0/4243762241 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9e000640 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad4311d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9f400640 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4312290 0x564ad4312670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 -- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4312290 msgr2=0x564ad4312670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4312290 0x564ad4312670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 -- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 msgr2=0x564ad4311d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad4311d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 -- 172.21.15.110:0/4243762241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad4118e50 con 0x564ad435bb20 2024-09-10T10:01:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9f400640 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4312290 0x564ad4312670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad435bf00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009310 tx=0x7f9c900562f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc622579ad8f11b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c8b400640 1 -- 172.21.15.110:0/4243762241 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9006e020 con 0x564ad435bb20 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c8b400640 1 -- 172.21.15.110:0/4243762241 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c90052a60 con 0x564ad435bb20 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.044+0000 7f9c8b400640 1 -- 172.21.15.110:0/4243762241 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90021040 con 0x564ad435bb20 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- 172.21.15.110:0/4243762241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 msgr2=0x564ad435bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad435bf00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009310 tx=0x7f9c900562f0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- 172.21.15.110:0/4243762241 shutdown_connections 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4312290 0x564ad4312670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad4311d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/4243762241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad435bf00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- 172.21.15.110:0/4243762241 >> 172.21.15.110:0/4243762241 conn(0x564ad4318600 msgr2=0x564ad4310630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- 172.21.15.110:0/4243762241 shutdown_connections 2024-09-10T10:01:27.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- 172.21.15.110:0/4243762241 wait complete. 2024-09-10T10:01:27.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- start start 2024-09-10T10:01:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 Processor -- start 2024-09-10T10:01:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad4316e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad431ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 0x564ad431d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ad41c2b00 con 0x564ad435bb20 2024-09-10T10:01:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564ad413b360 con 0x564ad435c440 2024-09-10T10:01:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x564ad41382d0 con 0x564ad4317390 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 10 client.0 _ll_register_callbacks cb 0x564ad42fd820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad4316e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad4316e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35918/0 (socket says 172.21.15.110:35918) 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 -- 172.21.15.110:0/350265929 learned_addr learned my addr 172.21.15.110:0/350265929 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9e000640 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad431ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9f400640 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 0x564ad431d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 -- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 msgr2=0x564ad431d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 0x564ad431d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 -- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 msgr2=0x564ad431ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad431ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 -- 172.21.15.110:0/350265929 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad4107120 con 0x564ad435bb20 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9e000640 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad431ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:27.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad4316e50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000c10 tx=0x7f9c900046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ce25366ca40762 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c7fe00640 1 -- 172.21.15.110:0/350265929 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90081030 con 0x564ad435bb20 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c7fe00640 1 -- 172.21.15.110:0/350265929 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9c9006fc50 con 0x564ad435bb20 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- 172.21.15.110:0/350265929 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c90052a60 con 0x564ad435bb20 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c7fe00640 1 -- 172.21.15.110:0/350265929 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9c9006e050 con 0x564ad435bb20 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 10 client.5760 fetch_fsmap learned FSMap version 36 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- 172.21.15.110:0/350265929 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x564ad4100450 con 0x564ad435bb20 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c7fe00640 1 -- 172.21.15.110:0/350265929 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c90052a60 con 0x564ad435bb20 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9c7fe00640 1 -- 172.21.15.110:0/350265929 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f9c90055840 con 0x564ad435bb20 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 10 client.5760 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:01:27.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 10 client.5760 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 20 client.5760 populate_metadata read hostname 'smithi110' 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] conn(0x564ad436a280 0x564ad436c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 4 client.5760 mds_command: new command op to 5640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.048+0000 7f9ca8fab000 1 -- 172.21.15.110:0/350265929 --> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ad40fac00 con 0x564ad436a280 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9c9e000640 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] conn(0x564ad436a280 0x564ad436c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9c9e000640 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] conn(0x564ad436a280 0x564ad436c670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86320f5a68571b8 server_cookie=dae6856b3cec049a in_seq=0 out_seq=0 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9c7fe00640 10 client.5760 ms_handle_connect on v2:172.21.15.110:6835/1333637048 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9c7fe00640 1 -- 172.21.15.110:0/350265929 <== mds.0 v2:172.21.15.110:6835/1333637048 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ad40fac00 con 0x564ad436a280 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9c7fe00640 10 client.5760 handle_command_reply: tid=0 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 client.5760 shutdown 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 -- 172.21.15.110:0/350265929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 msgr2=0x564ad4316e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad4316e50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000c10 tx=0x7f9c900046d0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 -- 172.21.15.110:0/350265929 shutdown_connections 2024-09-10T10:01:27.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] conn(0x564ad436a280 0x564ad436c670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 0x564ad431d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435c440 0x564ad431ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9c7fe00640 0 client.5760 ms_handle_reset on v2:172.21.15.110:6835/1333637048 2024-09-10T10:01:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/350265929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435bb20 0x564ad4316e50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 -- 172.21.15.110:0/350265929 >> 172.21.15.110:0/350265929 conn(0x564ad4312290 msgr2=0x564ad4310d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 -- 172.21.15.110:0/350265929 shutdown_connections 2024-09-10T10:01:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 1 -- 172.21.15.110:0/350265929 wait complete. 2024-09-10T10:01:27.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.052+0000 7f9ca8fab000 20 client.5760 trim_cache size 0 max 16384 2024-09-10T10:01:27.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 Processor -- start 2024-09-10T10:01:27.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- start start 2024-09-10T10:01:27.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 0x564ad435acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435bb20 0x564ad435b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad436c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0202e60 con 0x564ad435c440 2024-09-10T10:01:27.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0159a20 con 0x564ad435bb20 2024-09-10T10:01:27.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca015a070 con 0x564ad4317390 2024-09-10T10:01:27.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad436c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 0x564ad435acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435bb20 0x564ad435b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad436c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35922/0 (socket says 172.21.15.110:35922) 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 -- 172.21.15.110:0/335723827 learned_addr learned my addr 172.21.15.110:0/335723827 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 -- 172.21.15.110:0/335723827 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 msgr2=0x564ad435acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 --2- 172.21.15.110:0/335723827 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 0x564ad435acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 -- 172.21.15.110:0/335723827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435bb20 msgr2=0x564ad435b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 --2- 172.21.15.110:0/335723827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435bb20 0x564ad435b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 -- 172.21.15.110:0/335723827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad40e7fd0 con 0x564ad435c440 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9e000640 1 --2- 172.21.15.110:0/335723827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435bb20 0x564ad435b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c9f400640 1 --2- 172.21.15.110:0/335723827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad436c670 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9c98001a60 tx=0x7f9c98009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3aa824baffa1866 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c8aa00640 1 -- 172.21.15.110:0/335723827 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980075a0 con 0x564ad435c440 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c8aa00640 1 -- 172.21.15.110:0/335723827 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c98007760 con 0x564ad435c440 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9c8aa00640 1 -- 172.21.15.110:0/335723827 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98007950 con 0x564ad435c440 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- 172.21.15.110:0/335723827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 msgr2=0x564ad436c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/335723827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad436c670 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9c98001a60 tx=0x7f9c98009040 comp rx=0 tx=0).stop 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- 172.21.15.110:0/335723827 shutdown_connections 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/335723827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad436c670 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/335723827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad435bb20 0x564ad435b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/335723827 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad4317390 0x564ad435acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- 172.21.15.110:0/335723827 >> 172.21.15.110:0/335723827 conn(0x564ad4312290 msgr2=0x564ad4316650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- 172.21.15.110:0/335723827 shutdown_connections 2024-09-10T10:01:27.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- 172.21.15.110:0/335723827 wait complete. 2024-09-10T10:01:27.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 -- start start 2024-09-10T10:01:27.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.072+0000 7f9ca8fab000 1 Processor -- start 2024-09-10T10:01:27.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad4317390 0x564ad436f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad435bb20 0x564ad436f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad438a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ad415ff80 con 0x564ad435c440 2024-09-10T10:01:27.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564ad4118ca0 con 0x564ad4317390 2024-09-10T10:01:27.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x564ad41652a0 con 0x564ad435bb20 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 10 client.0 _ll_register_callbacks cb 0x7f9c90055d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad438a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad438a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35938/0 (socket says 172.21.15.110:35938) 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 -- 172.21.15.110:0/1387307531 learned_addr learned my addr 172.21.15.110:0/1387307531 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad4317390 0x564ad436f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9e000640 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad435bb20 0x564ad436f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 -- 172.21.15.110:0/1387307531 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad435bb20 msgr2=0x564ad436f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad435bb20 0x564ad436f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 -- 172.21.15.110:0/1387307531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad4317390 msgr2=0x564ad436f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad4317390 0x564ad436f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 -- 172.21.15.110:0/1387307531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad41e3750 con 0x564ad435c440 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9f400640 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad438a960 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9c98016000 tx=0x7f9c980016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6f224014a8610c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:01:27.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 1 -- 172.21.15.110:0/1387307531 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98025030 con 0x564ad435c440 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 1 -- 172.21.15.110:0/1387307531 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9c98017b50 con 0x564ad435c440 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- 172.21.15.110:0/1387307531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c98007760 con 0x564ad435c440 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 1 -- 172.21.15.110:0/1387307531 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9c98031050 con 0x564ad435c440 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 10 client.5766 fetch_fsmap learned FSMap version 36 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- 172.21.15.110:0/1387307531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x564ad40eea30 con 0x564ad435c440 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 1 -- 172.21.15.110:0/1387307531 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c98007760 con 0x564ad435c440 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 1 -- 172.21.15.110:0/1387307531 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f9c98034020 con 0x564ad435c440 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 10 client.5766 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 10 client.5766 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 20 client.5766 populate_metadata read hostname 'smithi110' 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] conn(0x564ad43cec00 0x564ad43cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 4 client.5766 mds_command: new command op to 5640 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- 172.21.15.110:0/1387307531 --> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] -- command(tid 0: {"prefix": "status"}) -- 0x564ad416e3a0 con 0x564ad43cec00 2024-09-10T10:01:27.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] conn(0x564ad43cec00 0x564ad43cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] conn(0x564ad43cec00 0x564ad43cefe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4487437f86a50e3f server_cookie=d39b2b921229b7bf in_seq=0 out_seq=0 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 10 client.5766 ms_handle_connect on v2:172.21.15.110:6835/1333637048 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 1 -- 172.21.15.110:0/1387307531 <== mds.0 v2:172.21.15.110:6835/1333637048 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564ad416e3a0 con 0x564ad43cec00 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 10 client.5766 handle_command_reply: tid=0 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 client.5766 shutdown 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- 172.21.15.110:0/1387307531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 msgr2=0x564ad438a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad438a960 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9c98016000 tx=0x7f9c980016c0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- 172.21.15.110:0/1387307531 shutdown_connections 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.110:6835/1333637048,v1:172.21.15.110:6837/1333637048] conn(0x564ad43cec00 0x564ad43cefe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ad435c440 0x564ad438a960 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564ad435bb20 0x564ad436f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9c7fe00640 0 client.5766 ms_handle_reset on v2:172.21.15.110:6835/1333637048 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 --2- 172.21.15.110:0/1387307531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564ad4317390 0x564ad436f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- 172.21.15.110:0/1387307531 >> 172.21.15.110:0/1387307531 conn(0x564ad4312290 msgr2=0x564ad4359cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.076+0000 7f9ca8fab000 1 -- 172.21.15.110:0/1387307531 shutdown_connections 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca8fab000 1 -- 172.21.15.110:0/1387307531 wait complete. 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca8fab000 20 client.5766 trim_cache size 0 max 16384 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 5640, 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:01:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 5.8732789309999998, 2024-09-10T10:01:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 36, 2024-09-10T10:01:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 58, 2024-09-10T10:01:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 58, 2024-09-10T10:01:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 10.778703674999999 2024-09-10T10:01:27.087 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 -- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9c80067a70 msgr2=0x7f9c80069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9c80067a70 0x7f9c80069f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0ad000 tx=0x7f9c8c008040 comp rx=0 tx=0).stop 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 -- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca00626e0 msgr2=0x7f9ca01544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca00626e0 0x7f9ca01544b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000e8a0 tx=0x7f9c9000eca0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 -- 172.21.15.110:0/670376307 shutdown_connections 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9c80067a70 0x7f9c80069f30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca015e540 0x7f9ca015eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca00b5880 0x7f9ca013ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 --2- 172.21.15.110:0/670376307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca00626e0 0x7f9ca01544b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 -- 172.21.15.110:0/670376307 >> 172.21.15.110:0/670376307 conn(0x7f9ca00b0970 msgr2=0x7f9ca015bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 -- 172.21.15.110:0/670376307 shutdown_connections 2024-09-10T10:01:27.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.080+0000 7f9ca5c00640 1 -- 172.21.15.110:0/670376307 wait complete. 2024-09-10T10:01:27.100 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:01:27.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.232+0000 7fb63fe00640 1 Processor -- start 2024-09-10T10:01:27.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.232+0000 7fb63fe00640 1 -- start start 2024-09-10T10:01:27.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb638155970 0x7fb6380cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63fe00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb638155d50 con 0x7fb63806f1e0 2024-09-10T10:01:27.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63fe00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6380b34d0 con 0x7fb63815c560 2024-09-10T10:01:27.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63fe00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb6380b4570 con 0x7fb638155970 2024-09-10T10:01:27.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63d400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb638155970 0x7fb6380cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63de00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb63806f1e0 0x7fb6380ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45916/0 (socket says 172.21.15.110:45916) 2024-09-10T10:01:27.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63de00640 1 -- 172.21.15.110:0/70481547 learned_addr learned my addr 172.21.15.110:0/70481547 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:27.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63d400640 1 -- 172.21.15.110:0/70481547 >> v1:172.21.15.145:6789/0 conn(0x7fb63815c560 legacy=0x7fb63806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63d400640 1 -- 172.21.15.110:0/70481547 >> v1:172.21.15.110:6789/0 conn(0x7fb63806f1e0 legacy=0x7fb6380ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb63d400640 1 -- 172.21.15.110:0/70481547 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6380b37b0 con 0x7fb638155970 2024-09-10T10:01:27.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.236+0000 7fb637e00640 1 -- 172.21.15.110:0/70481547 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1562287620 0 0) 0x7fb638155d50 con 0x7fb63806f1e0 2024-09-10T10:01:27.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63d400640 1 --2- 172.21.15.110:0/70481547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638155970 0x7fb6380cc8c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0097c0 tx=0x7fb62c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2906a81d43f41cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb637e00640 1 -- 172.21.15.110:0/70481547 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0a7070 con 0x7fb638155970 2024-09-10T10:01:27.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb637e00640 1 -- 172.21.15.110:0/70481547 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62c0b0070 con 0x7fb638155970 2024-09-10T10:01:27.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb637e00640 1 -- 172.21.15.110:0/70481547 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0ab890 con 0x7fb638155970 2024-09-10T10:01:27.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 -- 172.21.15.110:0/70481547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638155970 msgr2=0x7fb6380cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 --2- 172.21.15.110:0/70481547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638155970 0x7fb6380cc8c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0097c0 tx=0x7fb62c01c470 comp rx=0 tx=0).stop 2024-09-10T10:01:27.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 -- 172.21.15.110:0/70481547 shutdown_connections 2024-09-10T10:01:27.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 --2- 172.21.15.110:0/70481547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638155970 0x7fb6380cc8c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 -- 172.21.15.110:0/70481547 >> 172.21.15.110:0/70481547 conn(0x7fb63806bc50 msgr2=0x7fb63806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 -- 172.21.15.110:0/70481547 shutdown_connections 2024-09-10T10:01:27.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 -- 172.21.15.110:0/70481547 wait complete. 2024-09-10T10:01:27.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 Processor -- start 2024-09-10T10:01:27.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.240+0000 7fb63fe00640 1 -- start start 2024-09-10T10:01:27.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb63806f1e0 0x7fb63815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb638155970 0x7fb63814f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb63815c560 0x7fb6380aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb6380cf340 con 0x7fb63806f1e0 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6380cce00 con 0x7fb63815c560 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb6380cf4c0 con 0x7fb638155970 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb63806f1e0 0x7fb63815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb63806f1e0 0x7fb63815b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35954/0 (socket says 172.21.15.110:35954) 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 -- 172.21.15.110:0/2639883352 learned_addr learned my addr 172.21.15.110:0/2639883352 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63ca00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb638155970 0x7fb63814f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63de00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb63815c560 0x7fb6380aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 -- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb638155970 msgr2=0x7fb63814f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb638155970 0x7fb63814f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 -- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb63815c560 msgr2=0x7fb6380aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb63815c560 0x7fb6380aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 -- 172.21.15.110:0/2639883352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63814fe80 con 0x7fb63806f1e0 2024-09-10T10:01:27.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63de00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb63815c560 0x7fb6380aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:27.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63d400640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb63806f1e0 0x7fb63815b1a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0ad040 tx=0x7fb62c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb635600640 1 -- 172.21.15.110:0/2639883352 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c004220 con 0x7fb63806f1e0 2024-09-10T10:01:27.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb635600640 1 -- 172.21.15.110:0/2639883352 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62c0043c0 con 0x7fb63806f1e0 2024-09-10T10:01:27.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63815a080 con 0x7fb63806f1e0 2024-09-10T10:01:27.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb635600640 1 -- 172.21.15.110:0/2639883352 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c003820 con 0x7fb63806f1e0 2024-09-10T10:01:27.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63814d750 con 0x7fb63806f1e0 2024-09-10T10:01:27.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6380cc8c0 con 0x7fb63806f1e0 2024-09-10T10:01:27.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.244+0000 7fb635600640 1 -- 172.21.15.110:0/2639883352 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fb62c0039c0 con 0x7fb63806f1e0 2024-09-10T10:01:27.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.248+0000 7fb635600640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb618067a70 0x7fb618069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.248+0000 7fb635600640 1 -- 172.21.15.110:0/2639883352 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb62c120730 con 0x7fb63806f1e0 2024-09-10T10:01:27.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.248+0000 7fb63ca00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb618067a70 0x7fb618069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.248+0000 7fb63ca00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb618067a70 0x7fb618069f30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb6381fa420 tx=0x7fb624009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.252+0000 7fb635600640 1 -- 172.21.15.110:0/2639883352 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb62c016090 con 0x7fb63806f1e0 2024-09-10T10:01:27.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb6380b0760 con 0x7fb63806f1e0 2024-09-10T10:01:27.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb635600640 1 -- 172.21.15.110:0/2639883352 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7fb62c0f5e20 con 0x7fb63806f1e0 2024-09-10T10:01:27.450 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb618067a70 msgr2=0x7fb618069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb618067a70 0x7fb618069f30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb6381fa420 tx=0x7fb624009040 comp rx=0 tx=0).stop 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb63806f1e0 msgr2=0x7fb63815b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb63806f1e0 0x7fb63815b1a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0ad040 tx=0x7fb62c01acb0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 shutdown_connections 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb618067a70 0x7fb618069f30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb63815c560 0x7fb6380aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb638155970 0x7fb63814f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 --2- 172.21.15.110:0/2639883352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb63806f1e0 0x7fb63815b1a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 >> 172.21.15.110:0/2639883352 conn(0x7fb63806bc50 msgr2=0x7fb6380712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 shutdown_connections 2024-09-10T10:01:27.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27.448+0000 7fb63fe00640 1 -- 172.21.15.110:0/2639883352 wait complete. 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:01:27.472 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:01:27.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:27.473 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:01:27.482 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:01:27.482 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:01:27.482 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:01:27.482 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:27.482 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:01:27.482 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42287sec preferred_lft 42287sec 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:27.483 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:01:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:27.485 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:01:27.485 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:01:27.485 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:01:27.485 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:27.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:27.485 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:01:27.542 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:01:27.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:27.544 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:01:27.601 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:01:27.602 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:01:27.602 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:01:27.602 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:01:27.602 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:01:27.602 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:01:27.602 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:01:27.602 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:01:27.602 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:01:27.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:01:27.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:01:27.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:27.741 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:01:27.750 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:01:27.750 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:01:27.750 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:01:27.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:27.751 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:01:27.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:01:27.816 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:01:27.864 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:01:27.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:01:27.865 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:01:27.916 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:01:27.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:27.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:01:28.005 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:01:28.005 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:01:28.018 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:01:28.018 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:01:28.018 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:01:28.018 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:01:28.018 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:01:28.022 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:01:28.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:28.022 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:01:28.042 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:01:28.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:01:28.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:01:28.151 INFO:teuthology.orchestra.run.smithi110.stdout:client5518 2024-09-10T10:01:28.151 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:28.151 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:28.201 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:01:28.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.376+0000 7fc15ca00640 1 Processor -- start 2024-09-10T10:01:28.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc15ca00640 1 -- start start 2024-09-10T10:01:28.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc15ca00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1581568c0 0x7fc15806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc15ca00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc15815d500 con 0x7fc1580b6840 2024-09-10T10:01:28.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc15ca00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc15815d7e0 con 0x7fc158156ca0 2024-09-10T10:01:28.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc15ca00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc15806c9f0 con 0x7fc1581568c0 2024-09-10T10:01:28.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc157e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc1580b6840 0x7fc15815dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35600/0 (socket says 172.21.15.110:35600) 2024-09-10T10:01:28.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc157e00640 1 -- 172.21.15.110:0/2199296020 learned_addr learned my addr 172.21.15.110:0/2199296020 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:28.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc157400640 1 --2- 172.21.15.110:0/2199296020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1581568c0 0x7fc15806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc156000640 1 -- 172.21.15.110:0/2199296020 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084327724 0 0) 0x7fc15815d500 con 0x7fc1580b6840 2024-09-10T10:01:28.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc156000640 1 -- 172.21.15.110:0/2199296020 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc13c003530 con 0x7fc1580b6840 2024-09-10T10:01:28.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc156000640 1 -- 172.21.15.110:0/2199296020 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151745950 0 0) 0x7fc15815d7e0 con 0x7fc158156ca0 2024-09-10T10:01:28.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc156000640 1 -- 172.21.15.110:0/2199296020 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc15815d500 con 0x7fc158156ca0 2024-09-10T10:01:28.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc157400640 1 -- 172.21.15.110:0/2199296020 >> v1:172.21.15.145:6789/0 conn(0x7fc158156ca0 legacy=0x7fc15815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc157400640 1 -- 172.21.15.110:0/2199296020 >> v1:172.21.15.110:6789/0 conn(0x7fc1580b6840 legacy=0x7fc15815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.380+0000 7fc157400640 1 -- 172.21.15.110:0/2199296020 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15806e3e0 con 0x7fc1581568c0 2024-09-10T10:01:28.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc157400640 1 --2- 172.21.15.110:0/2199296020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1581568c0 0x7fc15806ebc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1480097f0 tx=0x7fc14801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea1afb537a929a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc156000640 1 -- 172.21.15.110:0/2199296020 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1480b0070 con 0x7fc1581568c0 2024-09-10T10:01:28.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc156000640 1 -- 172.21.15.110:0/2199296020 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1480ab600 con 0x7fc1581568c0 2024-09-10T10:01:28.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc156000640 1 -- 172.21.15.110:0/2199296020 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1480ab9d0 con 0x7fc1581568c0 2024-09-10T10:01:28.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 -- 172.21.15.110:0/2199296020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1581568c0 msgr2=0x7fc15806ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 --2- 172.21.15.110:0/2199296020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1581568c0 0x7fc15806ebc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1480097f0 tx=0x7fc14801c450 comp rx=0 tx=0).stop 2024-09-10T10:01:28.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 -- 172.21.15.110:0/2199296020 shutdown_connections 2024-09-10T10:01:28.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 --2- 172.21.15.110:0/2199296020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1581568c0 0x7fc15806ebc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 -- 172.21.15.110:0/2199296020 >> 172.21.15.110:0/2199296020 conn(0x7fc1580d1480 msgr2=0x7fc1580d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 -- 172.21.15.110:0/2199296020 shutdown_connections 2024-09-10T10:01:28.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 -- 172.21.15.110:0/2199296020 wait complete. 2024-09-10T10:01:28.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 Processor -- start 2024-09-10T10:01:28.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.384+0000 7fc15ca00640 1 -- start start 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1580b6840 0x7fc15814e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1581568c0 0x7fc15813ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc158156ca0 0x7fc158152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1580725d0 con 0x7fc1580b6840 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc158072450 con 0x7fc1581568c0 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc15816a8c0 con 0x7fc158156ca0 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1580b6840 0x7fc15814e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1580b6840 0x7fc15814e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39178/0 (socket says 172.21.15.110:39178) 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 -- 172.21.15.110:0/3219736177 learned_addr learned my addr 172.21.15.110:0/3219736177 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:28.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157e00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc158156ca0 0x7fc158152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc156a00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1581568c0 0x7fc15813ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 -- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc158156ca0 msgr2=0x7fc158152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc158156ca0 0x7fc158152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 -- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1581568c0 msgr2=0x7fc15813ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1581568c0 0x7fc15813ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 -- 172.21.15.110:0/3219736177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15813aad0 con 0x7fc1580b6840 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157e00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc158156ca0 0x7fc158152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc156a00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1581568c0 0x7fc15813ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:28.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc157400640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1580b6840 0x7fc15814e8e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc1480097f0 tx=0x7fc14801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc14f400640 1 -- 172.21.15.110:0/3219736177 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1480b0050 con 0x7fc1580b6840 2024-09-10T10:01:28.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc14f400640 1 -- 172.21.15.110:0/3219736177 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1480bc070 con 0x7fc1580b6840 2024-09-10T10:01:28.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15813d380 con 0x7fc1580b6840 2024-09-10T10:01:28.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc14f400640 1 -- 172.21.15.110:0/3219736177 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148006e20 con 0x7fc1580b6840 2024-09-10T10:01:28.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15813ad40 con 0x7fc1580b6840 2024-09-10T10:01:28.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1580b53c0 con 0x7fc1580b6840 2024-09-10T10:01:28.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.388+0000 7fc14f400640 1 -- 172.21.15.110:0/3219736177 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc1480b5830 con 0x7fc1580b6840 2024-09-10T10:01:28.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.392+0000 7fc14f400640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc138067a70 0x7fc138069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.392+0000 7fc14f400640 1 -- 172.21.15.110:0/3219736177 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc148120740 con 0x7fc1580b6840 2024-09-10T10:01:28.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.392+0000 7fc156a00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc138067a70 0x7fc138069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.392+0000 7fc156a00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc138067a70 0x7fc138069f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b3e60 tx=0x7fc144009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.396+0000 7fc14f400640 1 -- 172.21.15.110:0/3219736177 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc1480160d0 con 0x7fc1580b6840 2024-09-10T10:01:28.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.584+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc1580b50e0 con 0x7fc1580b6840 2024-09-10T10:01:28.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.584+0000 7fc14f400640 1 -- 172.21.15.110:0/3219736177 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7fc1480f0e50 con 0x7fc1580b6840 2024-09-10T10:01:28.589 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:01:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc138067a70 msgr2=0x7fc138069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc138067a70 0x7fc138069f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b3e60 tx=0x7fc144009040 comp rx=0 tx=0).stop 2024-09-10T10:01:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1580b6840 msgr2=0x7fc15814e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1580b6840 0x7fc15814e8e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc1480097f0 tx=0x7fc14801ac90 comp rx=0 tx=0).stop 2024-09-10T10:01:28.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 shutdown_connections 2024-09-10T10:01:28.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc138067a70 0x7fc138069f30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc158156ca0 0x7fc158152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1581568c0 0x7fc15813ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 --2- 172.21.15.110:0/3219736177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1580b6840 0x7fc15814e8e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 >> 172.21.15.110:0/3219736177 conn(0x7fc1580d1480 msgr2=0x7fc15815b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 shutdown_connections 2024-09-10T10:01:28.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:28.588+0000 7fc15ca00640 1 -- 172.21.15.110:0/3219736177 wait complete. 2024-09-10T10:01:28.610 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:01:28.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:01:28.610 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:01:28.610 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:01:28.610 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:01:28.611 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:01:28.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:01:28.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:01:28.611 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:28.611 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:01:28.621 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:01:28.621 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:28.622 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42282sec preferred_lft 42282sec 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:28.623 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:01:28.624 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:01:28.624 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:01:28.624 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:28.624 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:01:28.624 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:01:28.624 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:01:28.624 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:01:28.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:28.625 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:01:28.689 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:01:28.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:28.692 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:01:28.747 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:01:28.747 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:01:28.747 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:01:28.747 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:01:28.747 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:01:28.747 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:01:28.747 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:01:28.748 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:01:28.748 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:01:28.748 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:01:28.748 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:01:28.748 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:01:28.748 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:01:28.748 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:01:28.748 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:01:28.748 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:01:28.748 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:01:28.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:01:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:01:28.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:01:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:01:28.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:28.889 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:01:28.898 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:01:28.898 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:01:28.898 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:01:28.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:28.899 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:01:28.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:01:28.961 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:01:29.014 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:01:29.014 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:01:29.014 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:01:29.069 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:01:29.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:29.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:01:29.162 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:01:29.162 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:01:29.177 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:01:29.177 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:01:29.177 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:01:29.177 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:01:29.177 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:01:29.180 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:01:29.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:29.180 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:01:29.199 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:01:29.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:01:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:01:29.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:01:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:01:29.299 INFO:teuthology.orchestra.run.smithi145.stdout:client5784 2024-09-10T10:01:29.299 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:29.299 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:29.351 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:01:29.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.488+0000 7fe5cf400640 1 Processor -- start 2024-09-10T10:01:29.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.488+0000 7fe5cf400640 1 -- start start 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.488+0000 7fe5cf400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.488+0000 7fe5cf400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe5c806f1e0 0x7fe5c80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.488+0000 7fe5cf400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c8155d50 con 0x7fe5c815c560 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.488+0000 7fe5cf400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c80b4570 con 0x7fe5c806f1e0 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.488+0000 7fe5cf400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c80b34d0 con 0x7fe5c8155970 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.488+0000 7fe5cd400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe5c806f1e0 0x7fe5c80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cd400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe5c806f1e0 0x7fe5c80ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39184/0 (socket says 172.21.15.110:39184) 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cd400640 1 -- 172.21.15.110:0/2935906698 learned_addr learned my addr 172.21.15.110:0/2935906698 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cca00640 1 --2- 172.21.15.110:0/2935906698 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:29.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cd400640 1 -- 172.21.15.110:0/2935906698 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 msgr2=0x7fe5c80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:29.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cd400640 1 --2- 172.21.15.110:0/2935906698 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c80cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:29.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cd400640 1 -- 172.21.15.110:0/2935906698 >> v1:172.21.15.145:6789/0 conn(0x7fe5c815c560 legacy=0x7fe5c806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:29.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cd400640 1 -- 172.21.15.110:0/2935906698 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c80b37b0 con 0x7fe5c806f1e0 2024-09-10T10:01:29.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cca00640 1 --2- 172.21.15.110:0/2935906698 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:29.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cd400640 1 --2- 172.21.15.110:0/2935906698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c806f1e0 0x7fe5c80ce420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009e50 tx=0x7fe5c001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e120bcc6d4275b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:29.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5c7400640 1 -- 172.21.15.110:0/2935906698 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00a5830 con 0x7fe5c806f1e0 2024-09-10T10:01:29.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5c7400640 1 -- 172.21.15.110:0/2935906698 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c00a59d0 con 0x7fe5c806f1e0 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5c7400640 1 -- 172.21.15.110:0/2935906698 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00a5da0 con 0x7fe5c806f1e0 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cf400640 1 -- 172.21.15.110:0/2935906698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c806f1e0 msgr2=0x7fe5c80ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.492+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2935906698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c806f1e0 0x7fe5c80ce420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009e50 tx=0x7fe5c001c970 comp rx=0 tx=0).stop 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 -- 172.21.15.110:0/2935906698 shutdown_connections 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2935906698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c806f1e0 0x7fe5c80ce420 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2935906698 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 -- 172.21.15.110:0/2935906698 >> 172.21.15.110:0/2935906698 conn(0x7fe5c806bc50 msgr2=0x7fe5c806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 -- 172.21.15.110:0/2935906698 shutdown_connections 2024-09-10T10:01:29.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 -- 172.21.15.110:0/2935906698 wait complete. 2024-09-10T10:01:29.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 Processor -- start 2024-09-10T10:01:29.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 -- start start 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c806f1e0 0x7fe5c8151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c81650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c815c560 0x7fe5c81594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80cf170 con 0x7fe5c815c560 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80cce00 con 0x7fe5c806f1e0 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cf400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c80cf2f0 con 0x7fe5c8155970 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c815c560 0x7fe5c81594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c815c560 0x7fe5c81594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39190/0 (socket says 172.21.15.110:39190) 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 -- 172.21.15.110:0/2598519960 learned_addr learned my addr 172.21.15.110:0/2598519960 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cca00640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c806f1e0 0x7fe5c8151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:29.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5c7e00640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c81650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:29.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 -- 172.21.15.110:0/2598519960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 msgr2=0x7fe5c81650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:29.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c81650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:29.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 -- 172.21.15.110:0/2598519960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c806f1e0 msgr2=0x7fe5c8151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:29.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c806f1e0 0x7fe5c8151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:29.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 -- 172.21.15.110:0/2598519960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c815d400 con 0x7fe5c815c560 2024-09-10T10:01:29.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5cd400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c815c560 0x7fe5c81594d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe5c001a5a0 tx=0x7fe5c0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:29.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.496+0000 7fe5c4c00640 1 -- 172.21.15.110:0/2598519960 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00bc020 con 0x7fe5c815c560 2024-09-10T10:01:29.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5c4c00640 1 -- 172.21.15.110:0/2598519960 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c00b8670 con 0x7fe5c815c560 2024-09-10T10:01:29.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5c4c00640 1 -- 172.21.15.110:0/2598519960 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00b8a60 con 0x7fe5c815c560 2024-09-10T10:01:29.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c815d6e0 con 0x7fe5c815c560 2024-09-10T10:01:29.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c815dca0 con 0x7fe5c815c560 2024-09-10T10:01:29.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c80ce420 con 0x7fe5c815c560 2024-09-10T10:01:29.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5c4c00640 1 -- 172.21.15.110:0/2598519960 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fe5c00bf020 con 0x7fe5c815c560 2024-09-10T10:01:29.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5c4c00640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a8067a70 0x7fe5a8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:29.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5c4c00640 1 -- 172.21.15.110:0/2598519960 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe5c0120fe0 con 0x7fe5c815c560 2024-09-10T10:01:29.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5cca00640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a8067a70 0x7fe5a8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:29.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.500+0000 7fe5cca00640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a8067a70 0x7fe5a8069f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009950 tx=0x7fe5b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:29.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.504+0000 7fe5c4c00640 1 -- 172.21.15.110:0/2598519960 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe5c00c6030 con 0x7fe5c815c560 2024-09-10T10:01:29.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:29.696+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fe5c81542e0 con 0x7fe5c815c560 2024-09-10T10:01:30.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.264+0000 7fe5c4c00640 1 -- 172.21.15.110:0/2598519960 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7fe5c0016620 con 0x7fe5c815c560 2024-09-10T10:01:30.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a8067a70 msgr2=0x7fe5a8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a8067a70 0x7fe5a8069f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009950 tx=0x7fe5b8008040 comp rx=0 tx=0).stop 2024-09-10T10:01:30.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c815c560 msgr2=0x7fe5c81594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c815c560 0x7fe5c81594d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe5c001a5a0 tx=0x7fe5c0019710 comp rx=0 tx=0).stop 2024-09-10T10:01:30.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 shutdown_connections 2024-09-10T10:01:30.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a8067a70 0x7fe5a8069f30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c815c560 0x7fe5c81594d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c8155970 0x7fe5c81650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 --2- 172.21.15.110:0/2598519960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c806f1e0 0x7fe5c8151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 >> 172.21.15.110:0/2598519960 conn(0x7fe5c806bc50 msgr2=0x7fe5c80cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 shutdown_connections 2024-09-10T10:01:30.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.268+0000 7fe5cf400640 1 -- 172.21.15.110:0/2598519960 wait complete. 2024-09-10T10:01:30.292 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:01:30.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f5585800640 1 Processor -- start 2024-09-10T10:01:30.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f5585800640 1 -- start start 2024-09-10T10:01:30.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f5585800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f55800cf2e0 0x7f558006efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f5585800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f558015fda0 con 0x7f55800b5600 2024-09-10T10:01:30.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f5585800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55800ce520 con 0x7f55800726e0 2024-09-10T10:01:30.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f5585800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f55800cee50 con 0x7f55800cf2e0 2024-09-10T10:01:30.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f5577e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f55800b5600 0x7f55800b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35604/0 (socket says 172.21.15.110:35604) 2024-09-10T10:01:30.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f5577e00640 1 -- 172.21.15.110:0/4219334939 learned_addr learned my addr 172.21.15.110:0/4219334939 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f557f400640 1 --2- 172.21.15.110:0/4219334939 >> v2:172.21.15.110:3300/0 conn(0x7f55800cf2e0 0x7f558006efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f557f400640 1 -- 172.21.15.110:0/4219334939 >> v1:172.21.15.145:6789/0 conn(0x7f55800726e0 legacy=0x7f55800b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f557f400640 1 -- 172.21.15.110:0/4219334939 >> v1:172.21.15.110:6789/0 conn(0x7f55800b5600 legacy=0x7f55800b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.428+0000 7f557f400640 1 -- 172.21.15.110:0/4219334939 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55800ce800 con 0x7f55800cf2e0 2024-09-10T10:01:30.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f557f400640 1 --2- 172.21.15.110:0/4219334939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800cf2e0 0x7f558006efa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5578009d90 tx=0x7f557801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446815918920fd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f557e000640 1 -- 172.21.15.110:0/4219334939 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780b2070 con 0x7f55800cf2e0 2024-09-10T10:01:30.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f557e000640 1 -- 172.21.15.110:0/4219334939 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55780a5830 con 0x7f55800cf2e0 2024-09-10T10:01:30.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f557e000640 1 -- 172.21.15.110:0/4219334939 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780a5c00 con 0x7f55800cf2e0 2024-09-10T10:01:30.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 -- 172.21.15.110:0/4219334939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800cf2e0 msgr2=0x7f558006efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 --2- 172.21.15.110:0/4219334939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800cf2e0 0x7f558006efa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5578009d90 tx=0x7f557801c970 comp rx=0 tx=0).stop 2024-09-10T10:01:30.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 -- 172.21.15.110:0/4219334939 shutdown_connections 2024-09-10T10:01:30.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 --2- 172.21.15.110:0/4219334939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800cf2e0 0x7f558006efa0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 -- 172.21.15.110:0/4219334939 >> 172.21.15.110:0/4219334939 conn(0x7f5580062d10 msgr2=0x7f558006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 -- 172.21.15.110:0/4219334939 shutdown_connections 2024-09-10T10:01:30.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 -- 172.21.15.110:0/4219334939 wait complete. 2024-09-10T10:01:30.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 Processor -- start 2024-09-10T10:01:30.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.432+0000 7f5585800640 1 -- start start 2024-09-10T10:01:30.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55800726e0 0x7f5580144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800b5600 0x7f55800b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55800cf2e0 0x7f55801f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5580072c40 con 0x7f55800b5600 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5580072ac0 con 0x7f55800726e0 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f558015f950 con 0x7f55800cf2e0 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800b5600 0x7f55800b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800b5600 0x7f55800b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39208/0 (socket says 172.21.15.110:39208) 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 -- 172.21.15.110:0/3645946274 learned_addr learned my addr 172.21.15.110:0/3645946274 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f557ea00640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55800726e0 0x7f5580144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f557f400640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55800cf2e0 0x7f55801f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 -- 172.21.15.110:0/3645946274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55800cf2e0 msgr2=0x7f55801f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55800cf2e0 0x7f55801f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 -- 172.21.15.110:0/3645946274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55800726e0 msgr2=0x7f5580144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55800726e0 0x7f5580144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 -- 172.21.15.110:0/3645946274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5580155da0 con 0x7f55800b5600 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5577e00640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800b5600 0x7f55800b2640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f556800c4d0 tx=0x7f556800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5576a00640 1 -- 172.21.15.110:0/3645946274 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568017070 con 0x7f55800b5600 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5576a00640 1 -- 172.21.15.110:0/3645946274 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5568005cf0 con 0x7f55800b5600 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5576a00640 1 -- 172.21.15.110:0/3645946274 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568007270 con 0x7f55800b5600 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5580156080 con 0x7f55800b5600 2024-09-10T10:01:30.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5580156640 con 0x7f55800b5600 2024-09-10T10:01:30.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.436+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558006efa0 con 0x7f55800b5600 2024-09-10T10:01:30.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.440+0000 7f5576a00640 1 -- 172.21.15.110:0/3645946274 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f556801a070 con 0x7f55800b5600 2024-09-10T10:01:30.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.440+0000 7f5576a00640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f555c067a70 0x7f555c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.440+0000 7f5576a00640 1 -- 172.21.15.110:0/3645946274 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f55680850b0 con 0x7f55800b5600 2024-09-10T10:01:30.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.440+0000 7f557ea00640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f555c067a70 0x7f555c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.440+0000 7f557ea00640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f555c067a70 0x7f555c069f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f558006dbe0 tx=0x7f5570009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.444+0000 7f5576a00640 1 -- 172.21.15.110:0/3645946274 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f55680553c0 con 0x7f55800b5600 2024-09-10T10:01:30.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.620+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5580154700 con 0x7f555c067a70 2024-09-10T10:01:30.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.620+0000 7f5576a00640 1 -- 172.21.15.110:0/3645946274 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5580154700 con 0x7f555c067a70 2024-09-10T10:01:30.625 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:01:30.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:30.625 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:01:30.625 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:30.625 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f555c067a70 msgr2=0x7f555c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f555c067a70 0x7f555c069f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f558006dbe0 tx=0x7f5570009040 comp rx=0 tx=0).stop 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800b5600 msgr2=0x7f55800b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800b5600 0x7f55800b2640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f556800c4d0 tx=0x7f556800c990 comp rx=0 tx=0).stop 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 shutdown_connections 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f555c067a70 0x7f555c069f30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f55800cf2e0 0x7f55801f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55800b5600 0x7f55800b2640 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 --2- 172.21.15.110:0/3645946274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55800726e0 0x7f5580144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 >> 172.21.15.110:0/3645946274 conn(0x7f5580062d10 msgr2=0x7f5580071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 shutdown_connections 2024-09-10T10:01:30.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.624+0000 7f5585800640 1 -- 172.21.15.110:0/3645946274 wait complete. 2024-09-10T10:01:30.646 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:01:30.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f3520a00640 1 Processor -- start 2024-09-10T10:01:30.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f3520a00640 1 -- start start 2024-09-10T10:01:30.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f3520a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f351c06c290 0x7f351c147a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f3520a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f351c0b45c0 con 0x7f351c147f70 2024-09-10T10:01:30.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f3520a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f351c0b48a0 con 0x7f351c06beb0 2024-09-10T10:01:30.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f3520a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f351c148400 con 0x7f351c06c290 2024-09-10T10:01:30.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f351aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f351c06c290 0x7f351c147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f351be00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f351c147f70 0x7f351c0aeab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35616/0 (socket says 172.21.15.110:35616) 2024-09-10T10:01:30.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f351be00640 1 -- 172.21.15.110:0/2725686213 learned_addr learned my addr 172.21.15.110:0/2725686213 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:30.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f351aa00640 1 -- 172.21.15.110:0/2725686213 >> v1:172.21.15.145:6789/0 conn(0x7f351c06beb0 legacy=0x7f351c0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f351aa00640 1 -- 172.21.15.110:0/2725686213 >> v1:172.21.15.110:6789/0 conn(0x7f351c147f70 legacy=0x7f351c0aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.784+0000 7f351aa00640 1 -- 172.21.15.110:0/2725686213 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c0b4b80 con 0x7f351c06c290 2024-09-10T10:01:30.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f351aa00640 1 --2- 172.21.15.110:0/2725686213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06c290 0x7f351c147a30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f35080097c0 tx=0x7f350801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb14dc8dc0e83ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f351a000640 1 -- 172.21.15.110:0/2725686213 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35080ab430 con 0x7f351c06c290 2024-09-10T10:01:30.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f351a000640 1 -- 172.21.15.110:0/2725686213 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35080b0070 con 0x7f351c06c290 2024-09-10T10:01:30.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f351a000640 1 -- 172.21.15.110:0/2725686213 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35080ab7f0 con 0x7f351c06c290 2024-09-10T10:01:30.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f3520a00640 1 -- 172.21.15.110:0/2725686213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06c290 msgr2=0x7f351c147a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f3520a00640 1 --2- 172.21.15.110:0/2725686213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06c290 0x7f351c147a30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f35080097c0 tx=0x7f350801c430 comp rx=0 tx=0).stop 2024-09-10T10:01:30.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f3520a00640 1 -- 172.21.15.110:0/2725686213 shutdown_connections 2024-09-10T10:01:30.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f3520a00640 1 --2- 172.21.15.110:0/2725686213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06c290 0x7f351c147a30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f3520a00640 1 -- 172.21.15.110:0/2725686213 >> 172.21.15.110:0/2725686213 conn(0x7f351c06cd60 msgr2=0x7f351c06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f3520a00640 1 -- 172.21.15.110:0/2725686213 shutdown_connections 2024-09-10T10:01:30.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.788+0000 7f3520a00640 1 -- 172.21.15.110:0/2725686213 wait complete. 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 Processor -- start 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 -- start start 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06beb0 0x7f351c1421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f351c06c290 0x7f351c158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f351c147f70 0x7f351c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f351c0b6180 con 0x7f351c06beb0 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f351c0b6000 con 0x7f351c06c290 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f351c0af9d0 con 0x7f351c147f70 2024-09-10T10:01:30.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06beb0 0x7f351c1421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06beb0 0x7f351c1421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39220/0 (socket says 172.21.15.110:39220) 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 -- 172.21.15.110:0/4004555401 learned_addr learned my addr 172.21.15.110:0/4004555401 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 -- 172.21.15.110:0/4004555401 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f351c147f70 msgr2=0x7f351c15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f351c147f70 0x7f351c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 -- 172.21.15.110:0/4004555401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f351c06c290 msgr2=0x7f351c158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f351c06c290 0x7f351c158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 -- 172.21.15.110:0/4004555401 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c15b8e0 con 0x7f351c06beb0 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f351b400640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06beb0 0x7f351c1421a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c450 tx=0x7f350c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3513400640 1 -- 172.21.15.110:0/4004555401 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c012830 con 0x7f351c06beb0 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3513400640 1 -- 172.21.15.110:0/4004555401 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350c017070 con 0x7f351c06beb0 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3513400640 1 -- 172.21.15.110:0/4004555401 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c012c30 con 0x7f351c06beb0 2024-09-10T10:01:30.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c15bbc0 con 0x7f351c06beb0 2024-09-10T10:01:30.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c15c180 con 0x7f351c06beb0 2024-09-10T10:01:30.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.792+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c0afd00 con 0x7f351c06beb0 2024-09-10T10:01:30.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.796+0000 7f3513400640 1 -- 172.21.15.110:0/4004555401 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f350c01c460 con 0x7f351c06beb0 2024-09-10T10:01:30.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.796+0000 7f3513400640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34fc067a70 0x7f34fc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.796+0000 7f3513400640 1 -- 172.21.15.110:0/4004555401 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f350c085d40 con 0x7f351c06beb0 2024-09-10T10:01:30.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.796+0000 7f351aa00640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34fc067a70 0x7f34fc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.796+0000 7f351aa00640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34fc067a70 0x7f34fc069f30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35080af000 tx=0x7f3508004740 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.800+0000 7f3513400640 1 -- 172.21.15.110:0/4004555401 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f350c056690 con 0x7f351c06beb0 2024-09-10T10:01:30.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f351c073e80 con 0x7f351c06beb0 2024-09-10T10:01:30.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3513400640 1 -- 172.21.15.110:0/4004555401 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f350c055f90 con 0x7f351c06beb0 2024-09-10T10:01:30.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34fc067a70 msgr2=0x7f34fc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34fc067a70 0x7f34fc069f30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35080af000 tx=0x7f3508004740 comp rx=0 tx=0).stop 2024-09-10T10:01:30.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06beb0 msgr2=0x7f351c1421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06beb0 0x7f351c1421a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c450 tx=0x7f350c00c910 comp rx=0 tx=0).stop 2024-09-10T10:01:30.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 shutdown_connections 2024-09-10T10:01:30.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34fc067a70 0x7f34fc069f30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f351c147f70 0x7f351c15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f351c06c290 0x7f351c158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 --2- 172.21.15.110:0/4004555401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f351c06beb0 0x7f351c1421a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 >> 172.21.15.110:0/4004555401 conn(0x7f351c06cd60 msgr2=0x7f351c146550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 shutdown_connections 2024-09-10T10:01:30.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:30.960+0000 7f3520a00640 1 -- 172.21.15.110:0/4004555401 wait complete. 2024-09-10T10:01:30.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:30.983 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol57160) 2024-09-10T10:01:31.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:31.010 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T10:01:31.066 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:01:31.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:31.067 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol57160) 2024-09-10T10:01:31.137 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol13881 2024-09-10T10:01:31.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.320+0000 7f70d5a00640 1 Processor -- start 2024-09-10T10:01:31.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.320+0000 7f70d5a00640 1 -- start start 2024-09-10T10:01:31.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70d5a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:31.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70d5a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f70d006e850 0x7f70d00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:31.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70d5a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d00b4960 con 0x7f70d00ce4d0 2024-09-10T10:01:31.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70d5a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f70d00b5570 con 0x7f70d006e850 2024-09-10T10:01:31.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70d5a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f70d00b4c40 con 0x7f70d015b330 2024-09-10T10:01:31.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cf400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f70d006e850 0x7f70d00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:31.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cf400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f70d006e850 0x7f70d00b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39228/0 (socket says 172.21.15.110:39228) 2024-09-10T10:01:31.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cf400640 1 -- 172.21.15.110:0/2133079802 learned_addr learned my addr 172.21.15.110:0/2133079802 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:31.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cf400640 1 -- 172.21.15.110:0/2133079802 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 msgr2=0x7f70d006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:31.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cea00640 1 --2- 172.21.15.110:0/2133079802 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:31.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cf400640 1 --2- 172.21.15.110:0/2133079802 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cf400640 1 -- 172.21.15.110:0/2133079802 >> v1:172.21.15.145:6789/0 conn(0x7f70d00ce4d0 legacy=0x7f70d006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:31.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cf400640 1 -- 172.21.15.110:0/2133079802 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d00b4f20 con 0x7f70d006e850 2024-09-10T10:01:31.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.324+0000 7f70cea00640 1 --2- 172.21.15.110:0/2133079802 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:31.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70cf400640 1 --2- 172.21.15.110:0/2133079802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d006e850 0x7f70d00b0860 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009870 tx=0x7f70c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c146f4e92ddf84b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:31.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70cd600640 1 -- 172.21.15.110:0/2133079802 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c80ab5f0 con 0x7f70d006e850 2024-09-10T10:01:31.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70cd600640 1 -- 172.21.15.110:0/2133079802 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c80b0070 con 0x7f70d006e850 2024-09-10T10:01:31.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70cd600640 1 -- 172.21.15.110:0/2133079802 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c80ab9b0 con 0x7f70d006e850 2024-09-10T10:01:31.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 -- 172.21.15.110:0/2133079802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d006e850 msgr2=0x7f70d00b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:31.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 --2- 172.21.15.110:0/2133079802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d006e850 0x7f70d00b0860 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009870 tx=0x7f70c801c520 comp rx=0 tx=0).stop 2024-09-10T10:01:31.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 -- 172.21.15.110:0/2133079802 shutdown_connections 2024-09-10T10:01:31.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 --2- 172.21.15.110:0/2133079802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d006e850 0x7f70d00b0860 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 --2- 172.21.15.110:0/2133079802 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 -- 172.21.15.110:0/2133079802 >> 172.21.15.110:0/2133079802 conn(0x7f70d0062e40 msgr2=0x7f70d006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:31.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 -- 172.21.15.110:0/2133079802 shutdown_connections 2024-09-10T10:01:31.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 -- 172.21.15.110:0/2133079802 wait complete. 2024-09-10T10:01:31.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 Processor -- start 2024-09-10T10:01:31.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.328+0000 7f70d5a00640 1 -- start start 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d006e850 0x7f70d00ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d00ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d01591a0 0x7f70d0072310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01680b0 con 0x7f70d01591a0 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f70d00b17c0 con 0x7f70d006e850 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f70d0168230 con 0x7f70d015b330 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d01591a0 0x7f70d0072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d01591a0 0x7f70d0072310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39234/0 (socket says 172.21.15.110:39234) 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 -- 172.21.15.110:0/3846383130 learned_addr learned my addr 172.21.15.110:0/3846383130 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:31.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cea00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d006e850 0x7f70d00ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70ce000640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d00ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 -- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 msgr2=0x7f70d00ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d00ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 -- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d006e850 msgr2=0x7f70d00ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d006e850 0x7f70d00ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 -- 172.21.15.110:0/3846383130 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d00acdb0 con 0x7f70d01591a0 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70ce000640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d00ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70cf400640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d01591a0 0x7f70d0072310 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009870 tx=0x7f70c801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70c6a00640 1 -- 172.21.15.110:0/3846383130 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c80be5f0 con 0x7f70d01591a0 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70c6a00640 1 -- 172.21.15.110:0/3846383130 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c80be790 con 0x7f70d01591a0 2024-09-10T10:01:31.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70c6a00640 1 -- 172.21.15.110:0/3846383130 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c80beb80 con 0x7f70d01591a0 2024-09-10T10:01:31.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d00ad090 con 0x7f70d01591a0 2024-09-10T10:01:31.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d00ad650 con 0x7f70d01591a0 2024-09-10T10:01:31.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d006c6a0 con 0x7f70d01591a0 2024-09-10T10:01:31.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.332+0000 7f70c6a00640 1 -- 172.21.15.110:0/3846383130 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f70c80b43f0 con 0x7f70d01591a0 2024-09-10T10:01:31.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.336+0000 7f70c6a00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70b0067a70 0x7f70b0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:31.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.336+0000 7f70c6a00640 1 -- 172.21.15.110:0/3846383130 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f70c8120860 con 0x7f70d01591a0 2024-09-10T10:01:31.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.336+0000 7f70cea00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70b0067a70 0x7f70b0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:31.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.336+0000 7f70cea00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70b0067a70 0x7f70b0069f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f70d00cc640 tx=0x7f70c0007770 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:31.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.340+0000 7f70c6a00640 1 -- 172.21.15.110:0/3846383130 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f70c8016110 con 0x7f70d01591a0 2024-09-10T10:01:31.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.520+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol13881", "target": ["mon-mgr", ""]}) -- 0x7f70d0072e50 con 0x7f70b0067a70 2024-09-10T10:01:31.530 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:31.528+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:31.530 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:31.528+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:31.530 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:31.528+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:31.530 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:31.528+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:31.530 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:31.528+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:31.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70c6a00640 1 -- 172.21.15.110:0/3846383130 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f70d0072e50 con 0x7f70b0067a70 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70b0067a70 msgr2=0x7f70b0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70b0067a70 0x7f70b0069f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f70d00cc640 tx=0x7f70c0007770 comp rx=0 tx=0).stop 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d01591a0 msgr2=0x7f70d0072310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d01591a0 0x7f70d0072310 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009870 tx=0x7f70c801ad60 comp rx=0 tx=0).stop 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 shutdown_connections 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f70b0067a70 0x7f70b0069f30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d01591a0 0x7f70d0072310 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d015b330 0x7f70d00ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 --2- 172.21.15.110:0/3846383130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d006e850 0x7f70d00ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 >> 172.21.15.110:0/3846383130 conn(0x7f70d0062e40 msgr2=0x7f70d00b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:31.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 shutdown_connections 2024-09-10T10:01:31.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:31.560+0000 7f70d5a00640 1 -- 172.21.15.110:0/3846383130 wait complete. 2024-09-10T10:01:31.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:31.583 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol13881/.meta ./volumes/_nogroup/subvol57160) 2024-09-10T10:01:36.321 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57160 2024-09-10T10:01:36.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.464+0000 7f569f800640 1 Processor -- start 2024-09-10T10:01:36.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.464+0000 7f569f800640 1 -- start start 2024-09-10T10:01:36.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f569f800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f56980ce4d0 0x7f569806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f569f800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56980b4960 con 0x7f569806e850 2024-09-10T10:01:36.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f569f800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56980b4c40 con 0x7f569815b330 2024-09-10T10:01:36.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f569f800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f56980b5570 con 0x7f56980ce4d0 2024-09-10T10:01:36.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f5697e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f56980ce4d0 0x7f569806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f5697e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f56980ce4d0 0x7f569806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39242/0 (socket says 172.21.15.110:39242) 2024-09-10T10:01:36.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f5697e00640 1 -- 172.21.15.110:0/687859466 learned_addr learned my addr 172.21.15.110:0/687859466 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:36.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f5697e00640 1 -- 172.21.15.110:0/687859466 >> v1:172.21.15.145:6789/0 conn(0x7f569815b330 legacy=0x7f569806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f5697e00640 1 -- 172.21.15.110:0/687859466 >> v1:172.21.15.110:6789/0 conn(0x7f569806e850 legacy=0x7f56980b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f5697e00640 1 -- 172.21.15.110:0/687859466 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56980b4f20 con 0x7f56980ce4d0 2024-09-10T10:01:36.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.468+0000 7f5697400640 1 -- 172.21.15.110:0/687859466 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090123382 0 0) 0x7f56980b4960 con 0x7f569806e850 2024-09-10T10:01:36.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f5697e00640 1 --2- 172.21.15.110:0/687859466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56980ce4d0 0x7f569806c6a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5684009870 tx=0x7f568401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07ab6d9af0ddfc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:36.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f5697400640 1 -- 172.21.15.110:0/687859466 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840a7070 con 0x7f56980ce4d0 2024-09-10T10:01:36.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f5697400640 1 -- 172.21.15.110:0/687859466 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56840b0070 con 0x7f56980ce4d0 2024-09-10T10:01:36.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f5697400640 1 -- 172.21.15.110:0/687859466 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840ab890 con 0x7f56980ce4d0 2024-09-10T10:01:36.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 -- 172.21.15.110:0/687859466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56980ce4d0 msgr2=0x7f569806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 --2- 172.21.15.110:0/687859466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56980ce4d0 0x7f569806c6a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5684009870 tx=0x7f568401c560 comp rx=0 tx=0).stop 2024-09-10T10:01:36.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 -- 172.21.15.110:0/687859466 shutdown_connections 2024-09-10T10:01:36.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 --2- 172.21.15.110:0/687859466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56980ce4d0 0x7f569806c6a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 -- 172.21.15.110:0/687859466 >> 172.21.15.110:0/687859466 conn(0x7f5698062e40 msgr2=0x7f569806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:36.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 -- 172.21.15.110:0/687859466 shutdown_connections 2024-09-10T10:01:36.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 -- 172.21.15.110:0/687859466 wait complete. 2024-09-10T10:01:36.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 Processor -- start 2024-09-10T10:01:36.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.472+0000 7f569f800640 1 -- start start 2024-09-10T10:01:36.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f569806e850 0x7f569814e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f569815b330 0x7f5698072770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f569814f2a0 0x7f56980ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5698169450 con 0x7f569806e850 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f56981692d0 con 0x7f569814f2a0 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f56981695d0 con 0x7f569815b330 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f569806e850 0x7f569814e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f569806e850 0x7f569814e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39254/0 (socket says 172.21.15.110:39254) 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 -- 172.21.15.110:0/3980058136 learned_addr learned my addr 172.21.15.110:0/3980058136 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f5697e00640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f569815b330 0x7f5698072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569d800640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f569814f2a0 0x7f56980ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 -- 172.21.15.110:0/3980058136 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f569815b330 msgr2=0x7f5698072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f569815b330 0x7f5698072770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 -- 172.21.15.110:0/3980058136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f569814f2a0 msgr2=0x7f56980ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f569814f2a0 0x7f56980ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 -- 172.21.15.110:0/3980058136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5698154a10 con 0x7f569806e850 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569ce00640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f569806e850 0x7f569814e500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f568800c7a0 tx=0x7f568800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f5694c00640 1 -- 172.21.15.110:0/3980058136 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688003b80 con 0x7f569806e850 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f5694c00640 1 -- 172.21.15.110:0/3980058136 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5688003d20 con 0x7f569806e850 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5698154cf0 con 0x7f569806e850 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f5694c00640 1 -- 172.21.15.110:0/3980058136 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568800aa30 con 0x7f569806e850 2024-09-10T10:01:36.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569814ea40 con 0x7f569806e850 2024-09-10T10:01:36.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569806c6a0 con 0x7f569806e850 2024-09-10T10:01:36.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.476+0000 7f5694c00640 1 -- 172.21.15.110:0/3980058136 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f568800abd0 con 0x7f569806e850 2024-09-10T10:01:36.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.480+0000 7f5694c00640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5678067b30 0x7f5678069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.480+0000 7f5694c00640 1 -- 172.21.15.110:0/3980058136 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5688085e90 con 0x7f569806e850 2024-09-10T10:01:36.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.480+0000 7f5697e00640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5678067b30 0x7f5678069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.480+0000 7f5697e00640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5678067b30 0x7f5678069ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56840af000 tx=0x7f56840a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:36.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.484+0000 7f5694c00640 1 -- 172.21.15.110:0/3980058136 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5688055f80 con 0x7f569806e850 2024-09-10T10:01:36.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.664+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57160", "target": ["mon-mgr", ""]}) -- 0x7f569806e1a0 con 0x7f5678067b30 2024-09-10T10:01:36.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f5694c00640 1 -- 172.21.15.110:0/3980058136 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f569806e1a0 con 0x7f5678067b30 2024-09-10T10:01:36.701 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol57160 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5678067b30 msgr2=0x7f5678069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5678067b30 0x7f5678069ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56840af000 tx=0x7f56840a7000 comp rx=0 tx=0).stop 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f569806e850 msgr2=0x7f569814e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f569806e850 0x7f569814e500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f568800c7a0 tx=0x7f568800cc60 comp rx=0 tx=0).stop 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 shutdown_connections 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5678067b30 0x7f5678069ff0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f569814f2a0 0x7f56980ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f569815b330 0x7f5698072770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 --2- 172.21.15.110:0/3980058136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f569806e850 0x7f569814e500 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 >> 172.21.15.110:0/3980058136 conn(0x7f5698062e40 msgr2=0x7f56980b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 shutdown_connections 2024-09-10T10:01:36.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.700+0000 7f569f800640 1 -- 172.21.15.110:0/3980058136 wait complete. 2024-09-10T10:01:36.723 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57160 2024-09-10T10:01:36.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.856+0000 7f7edee00640 1 Processor -- start 2024-09-10T10:01:36.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.856+0000 7f7edee00640 1 -- start start 2024-09-10T10:01:36.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edee00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed815b330 0x7f7ed806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edee00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7ed806e850 0x7f7ed80b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edee00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed80b4960 con 0x7f7ed80ce4d0 2024-09-10T10:01:36.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edee00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed80b5570 con 0x7f7ed806e850 2024-09-10T10:01:36.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edee00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed80b4c40 con 0x7f7ed815b330 2024-09-10T10:01:36.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edce00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7ed806e850 0x7f7ed80b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edce00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7ed806e850 0x7f7ed80b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39268/0 (socket says 172.21.15.110:39268) 2024-09-10T10:01:36.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edce00640 1 -- 172.21.15.110:0/2736634681 learned_addr learned my addr 172.21.15.110:0/2736634681 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:36.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7ed7e00640 1 --2- 172.21.15.110:0/2736634681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed815b330 0x7f7ed806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edce00640 1 -- 172.21.15.110:0/2736634681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed815b330 msgr2=0x7f7ed806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:36.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edce00640 1 --2- 172.21.15.110:0/2736634681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed815b330 0x7f7ed806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edce00640 1 -- 172.21.15.110:0/2736634681 >> v1:172.21.15.145:6789/0 conn(0x7f7ed80ce4d0 legacy=0x7f7ed806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7edce00640 1 -- 172.21.15.110:0/2736634681 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed80b4f20 con 0x7f7ed806e850 2024-09-10T10:01:36.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.860+0000 7f7ed7e00640 1 --2- 172.21.15.110:0/2736634681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed815b330 0x7f7ed806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:36.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edce00640 1 --2- 172.21.15.110:0/2736634681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed806e850 0x7f7ed80b0860 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009e50 tx=0x7f7ed001ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9503034b9d917e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:36.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7ed6a00640 1 -- 172.21.15.110:0/2736634681 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00a7a20 con 0x7f7ed806e850 2024-09-10T10:01:36.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7ed6a00640 1 -- 172.21.15.110:0/2736634681 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed00a7bc0 con 0x7f7ed806e850 2024-09-10T10:01:36.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7ed6a00640 1 -- 172.21.15.110:0/2736634681 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00af3f0 con 0x7f7ed806e850 2024-09-10T10:01:36.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 -- 172.21.15.110:0/2736634681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed806e850 msgr2=0x7f7ed80b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 --2- 172.21.15.110:0/2736634681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed806e850 0x7f7ed80b0860 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009e50 tx=0x7f7ed001ec40 comp rx=0 tx=0).stop 2024-09-10T10:01:36.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 -- 172.21.15.110:0/2736634681 shutdown_connections 2024-09-10T10:01:36.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 --2- 172.21.15.110:0/2736634681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed806e850 0x7f7ed80b0860 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 --2- 172.21.15.110:0/2736634681 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed815b330 0x7f7ed806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 -- 172.21.15.110:0/2736634681 >> 172.21.15.110:0/2736634681 conn(0x7f7ed8062e40 msgr2=0x7f7ed806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:36.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 -- 172.21.15.110:0/2736634681 shutdown_connections 2024-09-10T10:01:36.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 -- 172.21.15.110:0/2736634681 wait complete. 2024-09-10T10:01:36.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 Processor -- start 2024-09-10T10:01:36.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.864+0000 7f7edee00640 1 -- start start 2024-09-10T10:01:36.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ed806e850 0x7f7ed8161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed815b330 0x7f7ed8071be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed81591a0 0x7f7ed81f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed80b36b0 con 0x7f7ed815b330 2024-09-10T10:01:36.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed80b17c0 con 0x7f7ed806e850 2024-09-10T10:01:36.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed80b3830 con 0x7f7ed81591a0 2024-09-10T10:01:36.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed815b330 0x7f7ed8071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed815b330 0x7f7ed8071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39278/0 (socket says 172.21.15.110:39278) 2024-09-10T10:01:36.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 -- 172.21.15.110:0/2815466510 learned_addr learned my addr 172.21.15.110:0/2815466510 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:36.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7e00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ed806e850 0x7f7ed8161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edce00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed81591a0 0x7f7ed81f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 -- 172.21.15.110:0/2815466510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed81591a0 msgr2=0x7f7ed81f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed81591a0 0x7f7ed81f6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 -- 172.21.15.110:0/2815466510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ed806e850 msgr2=0x7f7ed8161750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ed806e850 0x7f7ed8161750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 -- 172.21.15.110:0/2815466510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed8161c90 con 0x7f7ed815b330 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ed7400640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed815b330 0x7f7ed8071be0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c400 tx=0x7f7ec400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ecfe00640 1 -- 172.21.15.110:0/2815466510 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec40052d0 con 0x7f7ed815b330 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ecfe00640 1 -- 172.21.15.110:0/2815466510 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec4005470 con 0x7f7ed815b330 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ecfe00640 1 -- 172.21.15.110:0/2815466510 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4007260 con 0x7f7ed815b330 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed8161f10 con 0x7f7ed815b330 2024-09-10T10:01:36.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81624d0 con 0x7f7ed815b330 2024-09-10T10:01:36.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed806c6a0 con 0x7f7ed815b330 2024-09-10T10:01:36.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.868+0000 7f7ecfe00640 1 -- 172.21.15.110:0/2815466510 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f7ec4007400 con 0x7f7ed815b330 2024-09-10T10:01:36.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.872+0000 7f7ecfe00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7eb8067b00 0x7f7eb8069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.872+0000 7f7ecfe00640 1 -- 172.21.15.110:0/2815466510 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7ec4085450 con 0x7f7ed815b330 2024-09-10T10:01:36.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.872+0000 7f7ed7e00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7eb8067b00 0x7f7eb8069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.872+0000 7f7ed7e00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7eb8067b00 0x7f7eb8069fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8006e90 tx=0x7f7ec8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:36.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:36.876+0000 7f7ecfe00640 1 -- 172.21.15.110:0/2815466510 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7ed8161c90 con 0x7f7ed815b330 2024-09-10T10:01:37.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.056+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57160", "target": ["mon-mgr", ""]}) -- 0x7f7ed8063600 con 0x7f7eb8067b00 2024-09-10T10:01:37.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7ecfe00640 1 -- 172.21.15.110:0/2815466510 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f7ed8063600 con 0x7f7eb8067b00 2024-09-10T10:01:37.062 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol57160 2024-09-10T10:01:37.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7eb8067b00 msgr2=0x7f7eb8069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7eb8067b00 0x7f7eb8069fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8006e90 tx=0x7f7ec8008040 comp rx=0 tx=0).stop 2024-09-10T10:01:37.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed815b330 msgr2=0x7f7ed8071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed815b330 0x7f7ed8071be0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c400 tx=0x7f7ec400c8c0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 shutdown_connections 2024-09-10T10:01:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7eb8067b00 0x7f7eb8069fc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7ed81591a0 0x7f7ed81f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ed815b330 0x7f7ed8071be0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 --2- 172.21.15.110:0/2815466510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ed806e850 0x7f7ed8161750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 >> 172.21.15.110:0/2815466510 conn(0x7f7ed8062e40 msgr2=0x7f7ed80cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 shutdown_connections 2024-09-10T10:01:37.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.060+0000 7f7edee00640 1 -- 172.21.15.110:0/2815466510 wait complete. 2024-09-10T10:01:37.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:37.083 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/86fc187785147d7adf7f9cece1bd1c73.meta) 2024-09-10T10:01:37.109 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:01:37.109 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T10:01:37.109 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:01:37.110 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/_nogroup/subvol57160 2024-09-10T10:01:37.110 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:01:37.110 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:37.112 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol57160 alice 2024-09-10T10:01:37.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.296+0000 7f64c5c00640 1 Processor -- start 2024-09-10T10:01:37.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.296+0000 7f64c5c00640 1 -- start start 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64c5c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f64c015b170 0x7f64c0073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64c5c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f64c00cf2e0 0x7f64c00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64c5c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64c5c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f64c00cfc90 con 0x7f64c00cf2e0 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64c5c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f64c00b3910 con 0x7f64c015b170 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64c5c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f64c00b3b40 con 0x7f64c015ed40 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bf400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f64c00cf2e0 0x7f64c00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bf400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f64c00cf2e0 0x7f64c00cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39294/0 (socket says 172.21.15.110:39294) 2024-09-10T10:01:37.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bf400640 1 -- 172.21.15.110:0/3997282689 learned_addr learned my addr 172.21.15.110:0/3997282689 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64c5200640 1 --2- 172.21.15.110:0/3997282689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bfe00640 1 --2- 172.21.15.110:0/3997282689 >> v2:172.21.15.145:3300/0 conn(0x7f64c015b170 0x7f64c0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bf400640 1 -- 172.21.15.110:0/3997282689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 msgr2=0x7f64c015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bf400640 1 --2- 172.21.15.110:0/3997282689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bf400640 1 -- 172.21.15.110:0/3997282689 >> v2:172.21.15.145:3300/0 conn(0x7f64c015b170 msgr2=0x7f64c0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bf400640 1 --2- 172.21.15.110:0/3997282689 >> v2:172.21.15.145:3300/0 conn(0x7f64c015b170 0x7f64c0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64bf400640 1 -- 172.21.15.110:0/3997282689 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c00b3e20 con 0x7f64c00cf2e0 2024-09-10T10:01:37.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.300+0000 7f64c5200640 1 --2- 172.21.15.110:0/3997282689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:37.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64bf400640 1 --2- 172.21.15.110:0/3997282689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c00cf2e0 0x7f64c00cf750 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f64c006ffd0 tx=0x7f64ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e69d8867466d1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64bea00640 1 -- 172.21.15.110:0/3997282689 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0aa610 con 0x7f64c00cf2e0 2024-09-10T10:01:37.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64bea00640 1 -- 172.21.15.110:0/3997282689 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ac0af070 con 0x7f64c00cf2e0 2024-09-10T10:01:37.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64bea00640 1 -- 172.21.15.110:0/3997282689 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0aa9d0 con 0x7f64c00cf2e0 2024-09-10T10:01:37.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 -- 172.21.15.110:0/3997282689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c00cf2e0 msgr2=0x7f64c00cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3997282689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c00cf2e0 0x7f64c00cf750 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f64c006ffd0 tx=0x7f64ac01c4f0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 -- 172.21.15.110:0/3997282689 shutdown_connections 2024-09-10T10:01:37.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3997282689 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3997282689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c00cf2e0 0x7f64c00cf750 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3997282689 >> v2:172.21.15.145:3300/0 conn(0x7f64c015b170 0x7f64c0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 -- 172.21.15.110:0/3997282689 >> 172.21.15.110:0/3997282689 conn(0x7f64c006d0f0 msgr2=0x7f64c006d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 -- 172.21.15.110:0/3997282689 shutdown_connections 2024-09-10T10:01:37.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 -- 172.21.15.110:0/3997282689 wait complete. 2024-09-10T10:01:37.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 Processor -- start 2024-09-10T10:01:37.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 -- start start 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.304+0000 7f64c5c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64c00cf2e0 0x7f64c014cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c015b170 0x7f64c00b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f64c015b550 con 0x7f64c015b170 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f64c015fb00 con 0x7f64c00cf2e0 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f64c0072680 con 0x7f64c015ed40 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c015b170 0x7f64c00b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c015b170 0x7f64c00b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39298/0 (socket says 172.21.15.110:39298) 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 -- 172.21.15.110:0/3897286101 learned_addr learned my addr 172.21.15.110:0/3897286101 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bfe00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64c00cf2e0 0x7f64c014cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5200640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 -- 172.21.15.110:0/3897286101 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 msgr2=0x7f64c00ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c00ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 -- 172.21.15.110:0/3897286101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64c00cf2e0 msgr2=0x7f64c014cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64c00cf2e0 0x7f64c014cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 -- 172.21.15.110:0/3897286101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c00b19a0 con 0x7f64c015b170 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64bf400640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c015b170 0x7f64c00b0720 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f64ac01aba0 tx=0x7f64ac01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64b7e00640 1 -- 172.21.15.110:0/3897286101 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0bd5f0 con 0x7f64c015b170 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64b7e00640 1 -- 172.21.15.110:0/3897286101 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ac0bd790 con 0x7f64c015b170 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c00b1100 con 0x7f64c015b170 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64b7e00640 1 -- 172.21.15.110:0/3897286101 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0bdb40 con 0x7f64c015b170 2024-09-10T10:01:37.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c00b16c0 con 0x7f64c015b170 2024-09-10T10:01:37.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c00cf750 con 0x7f64c015b170 2024-09-10T10:01:37.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.308+0000 7f64b7e00640 1 -- 172.21.15.110:0/3897286101 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f64ac0b33f0 con 0x7f64c015b170 2024-09-10T10:01:37.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.312+0000 7f64b7e00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64a0067b00 0x7f64a0069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.312+0000 7f64b7e00640 1 -- 172.21.15.110:0/3897286101 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f64ac120f70 con 0x7f64c015b170 2024-09-10T10:01:37.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.312+0000 7f64bfe00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64a0067b00 0x7f64a0069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.312+0000 7f64bfe00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64a0067b00 0x7f64a0069fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f64c015c990 tx=0x7f64b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.316+0000 7f64b7e00640 1 -- 172.21.15.110:0/3897286101 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f64ac0c3060 con 0x7f64c015b170 2024-09-10T10:01:37.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.508+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol57160", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7f64c00b2ba0 con 0x7f64a0067b00 2024-09-10T10:01:37.520 INFO:tasks.ceph.mgr.y.smithi110.stderr:Creating meta for ID alice with tenant None 2024-09-10T10:01:37.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.548+0000 7f64b7e00640 1 -- 172.21.15.110:0/3897286101 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64c00b2ba0 con 0x7f64a0067b00 2024-09-10T10:01:37.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64a0067b00 msgr2=0x7f64a0069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64a0067b00 0x7f64a0069fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f64c015c990 tx=0x7f64b0008040 comp rx=0 tx=0).stop 2024-09-10T10:01:37.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c015b170 msgr2=0x7f64c00b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c015b170 0x7f64c00b0720 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f64ac01aba0 tx=0x7f64ac01ad30 comp rx=0 tx=0).stop 2024-09-10T10:01:37.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 shutdown_connections 2024-09-10T10:01:37.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f64a0067b00 0x7f64a0069fc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64c015ed40 0x7f64c00ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64c015b170 0x7f64c00b0720 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 --2- 172.21.15.110:0/3897286101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64c00cf2e0 0x7f64c014cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 >> 172.21.15.110:0/3897286101 conn(0x7f64c006d0f0 msgr2=0x7f64c00b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 shutdown_connections 2024-09-10T10:01:37.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.552+0000 7f64c5c00640 1 -- 172.21.15.110:0/3897286101 wait complete. 2024-09-10T10:01:37.574 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-10T10:01:37.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5ccc00640 1 Processor -- start 2024-09-10T10:01:37.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5ccc00640 1 -- start start 2024-09-10T10:01:37.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5ccc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cfb90 0x7fe5c815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5ccc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe5c80b3f40 0x7fe5c80cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5ccc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c815d4d0 con 0x7fe5c80cff70 2024-09-10T10:01:37.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5ccc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c806d130 con 0x7fe5c80b3f40 2024-09-10T10:01:37.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5ccc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c815d7b0 con 0x7fe5c80cfb90 2024-09-10T10:01:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5c6800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe5c80cff70 0x7fe5c8160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35638/0 (socket says 172.21.15.110:35638) 2024-09-10T10:01:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5c6800640 1 -- 172.21.15.110:0/2837555082 learned_addr learned my addr 172.21.15.110:0/2837555082 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5bfe00640 1 --2- 172.21.15.110:0/2837555082 >> v2:172.21.15.145:3300/0 conn(0x7fe5c80b3f40 0x7fe5c80cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5c5e00640 1 --2- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cfb90 0x7fe5c815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5c5400640 1 -- 172.21.15.110:0/2837555082 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2309650810 0 0) 0x7fe5c815d4d0 con 0x7fe5c80cff70 2024-09-10T10:01:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5c5400640 1 -- 172.21.15.110:0/2837555082 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5ac003530 con 0x7fe5c80cff70 2024-09-10T10:01:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5bfe00640 1 -- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cfb90 msgr2=0x7fe5c815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5bfe00640 1 --2- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cfb90 0x7fe5c815ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5bfe00640 1 -- 172.21.15.110:0/2837555082 >> v1:172.21.15.110:6789/0 conn(0x7fe5c80cff70 legacy=0x7fe5c8160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5bfe00640 1 -- 172.21.15.110:0/2837555082 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c806d380 con 0x7fe5c80b3f40 2024-09-10T10:01:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.708+0000 7fe5c5e00640 1 --2- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cfb90 0x7fe5c815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:37.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5bfe00640 1 --2- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 0x7fe5c80cd5a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00097c0 tx=0x7fe5b001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f71444f4cb43873 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5c5400640 1 -- 172.21.15.110:0/2837555082 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b00b0070 con 0x7fe5c80b3f40 2024-09-10T10:01:37.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5c5400640 1 -- 172.21.15.110:0/2837555082 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b00ab600 con 0x7fe5c80b3f40 2024-09-10T10:01:37.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5c5400640 1 -- 172.21.15.110:0/2837555082 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b00ab9b0 con 0x7fe5c80b3f40 2024-09-10T10:01:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5ccc00640 1 -- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 msgr2=0x7fe5c80cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 0x7fe5c80cd5a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00097c0 tx=0x7fe5b001c410 comp rx=0 tx=0).stop 2024-09-10T10:01:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5ccc00640 1 -- 172.21.15.110:0/2837555082 shutdown_connections 2024-09-10T10:01:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 0x7fe5c80cd5a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/2837555082 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cfb90 0x7fe5c815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5ccc00640 1 -- 172.21.15.110:0/2837555082 >> 172.21.15.110:0/2837555082 conn(0x7fe5c8062d10 msgr2=0x7fe5c806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5ccc00640 1 -- 172.21.15.110:0/2837555082 shutdown_connections 2024-09-10T10:01:37.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.712+0000 7fe5ccc00640 1 -- 172.21.15.110:0/2837555082 wait complete. 2024-09-10T10:01:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 Processor -- start 2024-09-10T10:01:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 -- start start 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 0x7fe5c8166040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c80cfb90 0x7fe5c8163e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cff70 0x7fe5c8145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8169500 con 0x7fe5c80cfb90 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8169380 con 0x7fe5c80b3f40 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c8169680 con 0x7fe5c80cff70 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c80cfb90 0x7fe5c8163e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c80cfb90 0x7fe5c8163e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39302/0 (socket says 172.21.15.110:39302) 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 -- 172.21.15.110:0/1485042230 learned_addr learned my addr 172.21.15.110:0/1485042230 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5c5e00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 0x7fe5c8166040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5c6800640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cff70 0x7fe5c8145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 -- 172.21.15.110:0/1485042230 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cff70 msgr2=0x7fe5c8145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cff70 0x7fe5c8145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 -- 172.21.15.110:0/1485042230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 msgr2=0x7fe5c8166040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 0x7fe5c8166040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 -- 172.21.15.110:0/1485042230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c81643b0 con 0x7fe5c80cfb90 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5bfe00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c80cfb90 0x7fe5c8163e70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe5b001a040 tx=0x7fe5b00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5be000640 1 -- 172.21.15.110:0/1485042230 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b00b0040 con 0x7fe5c80cfb90 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5be000640 1 -- 172.21.15.110:0/1485042230 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b00ab400 con 0x7fe5c80cfb90 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5be000640 1 -- 172.21.15.110:0/1485042230 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0006e20 con 0x7fe5c80cfb90 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c8164660 con 0x7fe5c80cfb90 2024-09-10T10:01:37.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c8164c20 con 0x7fe5c80cfb90 2024-09-10T10:01:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.716+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c80cd5a0 con 0x7fe5c80cfb90 2024-09-10T10:01:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.720+0000 7fe5be000640 1 -- 172.21.15.110:0/1485042230 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7fe5b00bc020 con 0x7fe5c80cfb90 2024-09-10T10:01:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.720+0000 7fe5be000640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a4067b00 0x7fe5a4069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.720+0000 7fe5be000640 1 -- 172.21.15.110:0/1485042230 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe5b011f890 con 0x7fe5c80cfb90 2024-09-10T10:01:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.720+0000 7fe5c5e00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a4067b00 0x7fe5a4069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.720+0000 7fe5c5e00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a4067b00 0x7fe5a4069fc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009580 tx=0x7fe5b8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.724+0000 7fe5be000640 1 -- 172.21.15.110:0/1485042230 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe5c80cdef0 con 0x7fe5c80cfb90 2024-09-10T10:01:37.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7fe5c80cc9c0 con 0x7fe5c80cfb90 2024-09-10T10:01:37.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5be000640 1 -- 172.21.15.110:0/1485042230 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v14) ==== 107+0+280 (secure 0 0 0) 0x7fe5b0016090 con 0x7fe5c80cfb90 2024-09-10T10:01:37.942 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:37.942 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:01:37.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:37.942 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.alice", 2024-09-10T10:01:37.942 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGeBmLkitHxAAEJDMPUZTmeNuGNgLJwWL8w==", 2024-09-10T10:01:37.942 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:37.943 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol57160", 2024-09-10T10:01:37.943 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:01:37.943 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-10T10:01:37.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:37.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:37.943 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:01:37.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a4067b00 msgr2=0x7fe5a4069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a4067b00 0x7fe5a4069fc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009580 tx=0x7fe5b8009190 comp rx=0 tx=0).stop 2024-09-10T10:01:37.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c80cfb90 msgr2=0x7fe5c8163e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c80cfb90 0x7fe5c8163e70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe5b001a040 tx=0x7fe5b00191b0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 shutdown_connections 2024-09-10T10:01:37.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe5a4067b00 0x7fe5a4069fc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5c80cff70 0x7fe5c8145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5c80cfb90 0x7fe5c8163e70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 --2- 172.21.15.110:0/1485042230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5c80b3f40 0x7fe5c8166040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 >> 172.21.15.110:0/1485042230 conn(0x7fe5c8062d10 msgr2=0x7fe5c80b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 shutdown_connections 2024-09-10T10:01:37.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:37.940+0000 7fe5ccc00640 1 -- 172.21.15.110:0/1485042230 wait complete. 2024-09-10T10:01:37.963 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol57160 2024-09-10T10:01:38.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.096+0000 7f4a47c00640 1 Processor -- start 2024-09-10T10:01:38.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a47c00640 1 -- start start 2024-09-10T10:01:38.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a47c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4a4006ec70 0x7f4a4006f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a47c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4a400b4c80 0x7f4a400b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a47c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a40072690 con 0x7f4a4015f010 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a47c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4a40072970 con 0x7f4a400b4c80 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a47c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4a40072c50 con 0x7f4a4006ec70 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a45c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4a400b4c80 0x7f4a400b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a45c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4a400b4c80 0x7f4a400b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39306/0 (socket says 172.21.15.110:39306) 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a45c00640 1 -- 172.21.15.110:0/3225059760 learned_addr learned my addr 172.21.15.110:0/3225059760 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a45200640 1 --1- 172.21.15.110:0/3225059760 >> v1:172.21.15.110:6789/0 conn(0x7f4a4015f010 0x7f4a4015f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35644/0 (socket says 172.21.15.110:35644) 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a3fe00640 1 --2- 172.21.15.110:0/3225059760 >> v2:172.21.15.145:3300/0 conn(0x7f4a4006ec70 0x7f4a4006f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a45c00640 1 -- 172.21.15.110:0/3225059760 >> v2:172.21.15.145:3300/0 conn(0x7f4a4006ec70 msgr2=0x7f4a4006f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a45c00640 1 --2- 172.21.15.110:0/3225059760 >> v2:172.21.15.145:3300/0 conn(0x7f4a4006ec70 0x7f4a4006f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a45c00640 1 -- 172.21.15.110:0/3225059760 >> v1:172.21.15.110:6789/0 conn(0x7f4a4015f010 legacy=0x7f4a4015f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.100+0000 7f4a45c00640 1 -- 172.21.15.110:0/3225059760 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a40072f30 con 0x7f4a400b4c80 2024-09-10T10:01:38.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a45c00640 1 --2- 172.21.15.110:0/3225059760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a400b4c80 0x7f4a400b50a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a400744a0 tx=0x7f4a3801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a847d96d6a6ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:38.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a3f400640 1 -- 172.21.15.110:0/3225059760 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a380b0070 con 0x7f4a400b4c80 2024-09-10T10:01:38.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a3f400640 1 -- 172.21.15.110:0/3225059760 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a380ab600 con 0x7f4a400b4c80 2024-09-10T10:01:38.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a3f400640 1 -- 172.21.15.110:0/3225059760 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a380ab9d0 con 0x7f4a400b4c80 2024-09-10T10:01:38.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 -- 172.21.15.110:0/3225059760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a400b4c80 msgr2=0x7f4a400b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3225059760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a400b4c80 0x7f4a400b50a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a400744a0 tx=0x7f4a3801c490 comp rx=0 tx=0).stop 2024-09-10T10:01:38.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 -- 172.21.15.110:0/3225059760 shutdown_connections 2024-09-10T10:01:38.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3225059760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a400b4c80 0x7f4a400b50a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3225059760 >> v2:172.21.15.145:3300/0 conn(0x7f4a4006ec70 0x7f4a4006f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 -- 172.21.15.110:0/3225059760 >> 172.21.15.110:0/3225059760 conn(0x7f4a4006ca70 msgr2=0x7f4a4006ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:38.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 -- 172.21.15.110:0/3225059760 shutdown_connections 2024-09-10T10:01:38.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 -- 172.21.15.110:0/3225059760 wait complete. 2024-09-10T10:01:38.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 Processor -- start 2024-09-10T10:01:38.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.104+0000 7f4a47c00640 1 -- start start 2024-09-10T10:01:38.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a4006ec70 0x7f4a4014b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4a400b4c80 0x7f4a40150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a4015f010 0x7f4a401517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40072450 con 0x7f4a4006ec70 2024-09-10T10:01:38.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4a400722d0 con 0x7f4a4015f010 2024-09-10T10:01:38.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4016b5f0 con 0x7f4a400b4c80 2024-09-10T10:01:38.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a4006ec70 0x7f4a4014b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a4006ec70 0x7f4a4014b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39316/0 (socket says 172.21.15.110:39316) 2024-09-10T10:01:38.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 -- 172.21.15.110:0/3787730111 learned_addr learned my addr 172.21.15.110:0/3787730111 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45c00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a4015f010 0x7f4a401517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 -- 172.21.15.110:0/3787730111 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4a400b4c80 msgr2=0x7f4a40150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4a400b4c80 0x7f4a40150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 -- 172.21.15.110:0/3787730111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a4015f010 msgr2=0x7f4a401517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a4015f010 0x7f4a401517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 -- 172.21.15.110:0/3787730111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a40151510 con 0x7f4a4006ec70 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a45200640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a4006ec70 0x7f4a4014b7b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c790 tx=0x7f4a3000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a3cc00640 1 -- 172.21.15.110:0/3787730111 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30017070 con 0x7f4a4006ec70 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a3cc00640 1 -- 172.21.15.110:0/3787730111 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3000a640 con 0x7f4a4006ec70 2024-09-10T10:01:38.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a40158a30 con 0x7f4a4006ec70 2024-09-10T10:01:38.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a3cc00640 1 -- 172.21.15.110:0/3787730111 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000a9f0 con 0x7f4a4006ec70 2024-09-10T10:01:38.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a400cfd90 con 0x7f4a4006ec70 2024-09-10T10:01:38.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4006f100 con 0x7f4a4006ec70 2024-09-10T10:01:38.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.108+0000 7f4a3cc00640 1 -- 172.21.15.110:0/3787730111 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f4a3001a070 con 0x7f4a4006ec70 2024-09-10T10:01:38.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.112+0000 7f4a3cc00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4a20067b30 0x7f4a20069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.112+0000 7f4a3cc00640 1 -- 172.21.15.110:0/3787730111 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4a30085650 con 0x7f4a4006ec70 2024-09-10T10:01:38.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.112+0000 7f4a3fe00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4a20067b30 0x7f4a20069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.112+0000 7f4a3fe00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4a20067b30 0x7f4a20069ff0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a40150ad0 tx=0x7f4a2c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:38.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.116+0000 7f4a3cc00640 1 -- 172.21.15.110:0/3787730111 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4a300557e0 con 0x7f4a4006ec70 2024-09-10T10:01:38.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.296+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol57160", "target": ["mon-mgr", ""]}) -- 0x7f4a40071ea0 con 0x7f4a20067b30 2024-09-10T10:01:38.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.324+0000 7f4a3cc00640 1 -- 172.21.15.110:0/3787730111 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4a40071ea0 con 0x7f4a20067b30 2024-09-10T10:01:38.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4a20067b30 msgr2=0x7f4a20069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4a20067b30 0x7f4a20069ff0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a40150ad0 tx=0x7f4a2c008040 comp rx=0 tx=0).stop 2024-09-10T10:01:38.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a4006ec70 msgr2=0x7f4a4014b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a4006ec70 0x7f4a4014b7b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c790 tx=0x7f4a3000cc50 comp rx=0 tx=0).stop 2024-09-10T10:01:38.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 shutdown_connections 2024-09-10T10:01:38.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4a20067b30 0x7f4a20069ff0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a4015f010 0x7f4a401517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4a400b4c80 0x7f4a40150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 --2- 172.21.15.110:0/3787730111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a4006ec70 0x7f4a4014b7b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 >> 172.21.15.110:0/3787730111 conn(0x7f4a4006ca70 msgr2=0x7f4a4015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:38.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 shutdown_connections 2024-09-10T10:01:38.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f4a47c00640 1 -- 172.21.15.110:0/3787730111 wait complete. 2024-09-10T10:01:38.332 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.332 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.332 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.332 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.333 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.328+0000 7f30dd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.351 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.348+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.351 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.348+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.351 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.348+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.352 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.348+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.352 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:01:38.348+0000 7f30dec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:38.355 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol13881 2024-09-10T10:01:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d5a400640 1 Processor -- start 2024-09-10T10:01:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d5a400640 1 -- start start 2024-09-10T10:01:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d5a400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d54145f30 0x7f0d540b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d5a400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0d540b29e0 0x7f0d540cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d5a400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0d540cfbc0 0x7f0d54160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d5a400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0d540b2e20 con 0x7f0d540cfbc0 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d5a400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0d540b37c0 con 0x7f0d540b29e0 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d5a400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0d540b3aa0 con 0x7f0d54145f30 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0d540cfbc0 0x7f0d54160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0d540cfbc0 0x7f0d54160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49584/0 (socket says 172.21.15.110:49584) 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53e00640 1 -- 172.21.15.110:0/2698457037 learned_addr learned my addr 172.21.15.110:0/2698457037 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53400640 1 --2- 172.21.15.110:0/2698457037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d54145f30 0x7f0d540b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d52a00640 1 --2- 172.21.15.110:0/2698457037 >> v2:172.21.15.145:3300/0 conn(0x7f0d540b29e0 0x7f0d540cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53e00640 1 -- 172.21.15.110:0/2698457037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d54145f30 msgr2=0x7f0d540b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:38.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53e00640 1 --2- 172.21.15.110:0/2698457037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d54145f30 0x7f0d540b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53e00640 1 -- 172.21.15.110:0/2698457037 >> v2:172.21.15.145:3300/0 conn(0x7f0d540b29e0 msgr2=0x7f0d540cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:38.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53e00640 1 --2- 172.21.15.110:0/2698457037 >> v2:172.21.15.145:3300/0 conn(0x7f0d540b29e0 0x7f0d540cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.492+0000 7f0d53e00640 1 -- 172.21.15.110:0/2698457037 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d540b3d80 con 0x7f0d540cfbc0 2024-09-10T10:01:38.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d53e00640 1 --2- 172.21.15.110:0/2698457037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d540cfbc0 0x7f0d54160030 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0d5406f1a0 tx=0x7f0d4c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a3a64a7d69f1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:38.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d52000640 1 -- 172.21.15.110:0/2698457037 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0b0070 con 0x7f0d540cfbc0 2024-09-10T10:01:38.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d52000640 1 -- 172.21.15.110:0/2698457037 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c0ab600 con 0x7f0d540cfbc0 2024-09-10T10:01:38.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d52000640 1 -- 172.21.15.110:0/2698457037 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0ab9d0 con 0x7f0d540cfbc0 2024-09-10T10:01:38.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 -- 172.21.15.110:0/2698457037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d540cfbc0 msgr2=0x7f0d54160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 --2- 172.21.15.110:0/2698457037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d540cfbc0 0x7f0d54160030 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0d5406f1a0 tx=0x7f0d4c01c260 comp rx=0 tx=0).stop 2024-09-10T10:01:38.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 -- 172.21.15.110:0/2698457037 shutdown_connections 2024-09-10T10:01:38.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 --2- 172.21.15.110:0/2698457037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d540cfbc0 0x7f0d54160030 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 --2- 172.21.15.110:0/2698457037 >> v2:172.21.15.145:3300/0 conn(0x7f0d540b29e0 0x7f0d540cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 --2- 172.21.15.110:0/2698457037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d54145f30 0x7f0d540b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 -- 172.21.15.110:0/2698457037 >> 172.21.15.110:0/2698457037 conn(0x7f0d54062e40 msgr2=0x7f0d5406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:38.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 -- 172.21.15.110:0/2698457037 shutdown_connections 2024-09-10T10:01:38.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.496+0000 7f0d5a400640 1 -- 172.21.15.110:0/2698457037 wait complete. 2024-09-10T10:01:38.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 Processor -- start 2024-09-10T10:01:38.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 -- start start 2024-09-10T10:01:38.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d540b29e0 0x7f0d541413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d540cfbc0 0x7f0d54140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d54145f30 0x7f0d54140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d540b48f0 con 0x7f0d54145f30 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d540b4770 con 0x7f0d540b29e0 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0d540b4a70 con 0x7f0d540cfbc0 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d540b29e0 0x7f0d541413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d540cfbc0 0x7f0d54140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d540b29e0 0x7f0d541413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:55810/0 (socket says 172.21.15.110:55810) 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53400640 1 -- 172.21.15.110:0/629996953 learned_addr learned my addr 172.21.15.110:0/629996953 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53e00640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d54145f30 0x7f0d54140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53e00640 1 -- 172.21.15.110:0/629996953 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d540cfbc0 msgr2=0x7f0d54140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53e00640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d540cfbc0 0x7f0d54140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53e00640 1 -- 172.21.15.110:0/629996953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d540b29e0 msgr2=0x7f0d541413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53e00640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d540b29e0 0x7f0d541413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53e00640 1 -- 172.21.15.110:0/629996953 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d54140ad0 con 0x7f0d54145f30 2024-09-10T10:01:38.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d540b29e0 0x7f0d541413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:38.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d53e00640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d54145f30 0x7f0d54140590 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c0ab000 tx=0x7f0d4c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:38.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d4b400640 1 -- 172.21.15.110:0/629996953 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0b0040 con 0x7f0d54145f30 2024-09-10T10:01:38.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d4b400640 1 -- 172.21.15.110:0/629996953 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c0b4410 con 0x7f0d54145f30 2024-09-10T10:01:38.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d4b400640 1 -- 172.21.15.110:0/629996953 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0b47e0 con 0x7f0d54145f30 2024-09-10T10:01:38.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5413f290 con 0x7f0d54145f30 2024-09-10T10:01:38.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5413f850 con 0x7f0d54145f30 2024-09-10T10:01:38.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.500+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d54160030 con 0x7f0d54145f30 2024-09-10T10:01:38.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.504+0000 7f0d4b400640 1 -- 172.21.15.110:0/629996953 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f0d4c0b4980 con 0x7f0d54145f30 2024-09-10T10:01:38.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.504+0000 7f0d4b400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d34067b30 0x7f0d34069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.504+0000 7f0d4b400640 1 -- 172.21.15.110:0/629996953 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0d4c11fb70 con 0x7f0d54145f30 2024-09-10T10:01:38.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.504+0000 7f0d53400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d34067b30 0x7f0d34069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.504+0000 7f0d53400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d34067b30 0x7f0d34069ff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0d440094c0 tx=0x7f0d4400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:38.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.508+0000 7f0d4b400640 1 -- 172.21.15.110:0/629996953 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0d4c016110 con 0x7f0d54145f30 2024-09-10T10:01:38.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:38.688+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol13881", "target": ["mon-mgr", ""]}) -- 0x7f0d5415c5b0 con 0x7f0d34067b30 2024-09-10T10:01:40.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:40.308+0000 7f0d4b400640 1 -- 172.21.15.110:0/629996953 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f0d4c0ec320 con 0x7f0d54145f30 2024-09-10T10:01:41.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.320+0000 7f0d4b400640 1 -- 172.21.15.110:0/629996953 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d5415c5b0 con 0x7f0d34067b30 2024-09-10T10:01:41.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d34067b30 msgr2=0x7f0d34069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d34067b30 0x7f0d34069ff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0d440094c0 tx=0x7f0d4400b040 comp rx=0 tx=0).stop 2024-09-10T10:01:41.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d54145f30 msgr2=0x7f0d54140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d54145f30 0x7f0d54140590 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c0ab000 tx=0x7f0d4c019000 comp rx=0 tx=0).stop 2024-09-10T10:01:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 shutdown_connections 2024-09-10T10:01:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0d34067b30 0x7f0d34069ff0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d54145f30 0x7f0d54140590 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d540cfbc0 0x7f0d54140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 --2- 172.21.15.110:0/629996953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d540b29e0 0x7f0d541413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 >> 172.21.15.110:0/629996953 conn(0x7f0d54062e40 msgr2=0x7f0d54072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 shutdown_connections 2024-09-10T10:01:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.324+0000 7f0d5a400640 1 -- 172.21.15.110:0/629996953 wait complete. 2024-09-10T10:01:41.355 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T10:01:41.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:01:41.412 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-10T10:01:41.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:01:41.419 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:01:41.420 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:01:41.528 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:01:41.621 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:01:41.621 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:01:41.621 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:41.621 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:41.630 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:41.630 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:41.681 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:01:41.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.856+0000 7fb04ba00640 1 Processor -- start 2024-09-10T10:01:41.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.856+0000 7fb04ba00640 1 -- start start 2024-09-10T10:01:41.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb04ba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 0x7fb0440cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb04ba00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb0440cff70 0x7fb0440cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb04ba00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04415d4d0 con 0x7fb0440cfb90 2024-09-10T10:01:41.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb04ba00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb04406d130 con 0x7fb0440cff70 2024-09-10T10:01:41.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb04ba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb04415d7b0 con 0x7fb0440b3f40 2024-09-10T10:01:41.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb049a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb0440cff70 0x7fb0440cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb049a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb0440cff70 0x7fb0440cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49610/0 (socket says 172.21.15.110:49610) 2024-09-10T10:01:41.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb049a00640 1 -- 172.21.15.110:0/605492289 learned_addr learned my addr 172.21.15.110:0/605492289 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:41.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb043e00640 1 --2- 172.21.15.110:0/605492289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 0x7fb0440cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb049a00640 1 -- 172.21.15.110:0/605492289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 msgr2=0x7fb0440cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:41.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb049a00640 1 --2- 172.21.15.110:0/605492289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 0x7fb0440cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb049a00640 1 -- 172.21.15.110:0/605492289 >> v1:172.21.15.145:6789/0 conn(0x7fb0440cfb90 legacy=0x7fb04415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.860+0000 7fb049a00640 1 -- 172.21.15.110:0/605492289 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04406d380 con 0x7fb0440cff70 2024-09-10T10:01:41.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb049a00640 1 --2- 172.21.15.110:0/605492289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb0440cdae0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009e50 tx=0x7fb03c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bced7f6aa4dd64ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:41.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb043400640 1 -- 172.21.15.110:0/605492289 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a5660 con 0x7fb0440cff70 2024-09-10T10:01:41.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb043400640 1 -- 172.21.15.110:0/605492289 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c0a5800 con 0x7fb0440cff70 2024-09-10T10:01:41.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb043400640 1 -- 172.21.15.110:0/605492289 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a5bd0 con 0x7fb0440cff70 2024-09-10T10:01:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 -- 172.21.15.110:0/605492289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 msgr2=0x7fb0440cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 --2- 172.21.15.110:0/605492289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb0440cdae0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009e50 tx=0x7fb03c01c9a0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 -- 172.21.15.110:0/605492289 shutdown_connections 2024-09-10T10:01:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 --2- 172.21.15.110:0/605492289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb0440cdae0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 --2- 172.21.15.110:0/605492289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 0x7fb0440cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 -- 172.21.15.110:0/605492289 >> 172.21.15.110:0/605492289 conn(0x7fb044062d10 msgr2=0x7fb04406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:41.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 -- 172.21.15.110:0/605492289 shutdown_connections 2024-09-10T10:01:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 -- 172.21.15.110:0/605492289 wait complete. 2024-09-10T10:01:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 Processor -- start 2024-09-10T10:01:41.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.864+0000 7fb04ba00640 1 -- start start 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 0x7fb044161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0440cfb90 0x7fb044163f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb044155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb04406e720 con 0x7fb0440cff70 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb04406e5a0 con 0x7fb0440cfb90 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb04406e8a0 con 0x7fb0440b3f40 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb044155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb044155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49614/0 (socket says 172.21.15.110:49614) 2024-09-10T10:01:41.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 -- 172.21.15.110:0/1185697759 learned_addr learned my addr 172.21.15.110:0/1185697759 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:41.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb043e00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0440cfb90 0x7fb044163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049000640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 0x7fb044161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 -- 172.21.15.110:0/1185697759 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 msgr2=0x7fb044161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 0x7fb044161110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 -- 172.21.15.110:0/1185697759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0440cfb90 msgr2=0x7fb044163f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0440cfb90 0x7fb044163f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 -- 172.21.15.110:0/1185697759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb044164440 con 0x7fb0440cff70 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb049a00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb044155da0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb03c01a5d0 tx=0x7fb03c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb040c00640 1 -- 172.21.15.110:0/1185697759 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0bc020 con 0x7fb0440cff70 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb040c00640 1 -- 172.21.15.110:0/1185697759 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c0b8690 con 0x7fb0440cff70 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb040c00640 1 -- 172.21.15.110:0/1185697759 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0b8a80 con 0x7fb0440cff70 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0441616f0 con 0x7fb0440cff70 2024-09-10T10:01:41.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb044161cb0 con 0x7fb0440cff70 2024-09-10T10:01:41.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04415ca10 con 0x7fb0440cff70 2024-09-10T10:01:41.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.868+0000 7fb040c00640 1 -- 172.21.15.110:0/1185697759 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb03c0bf020 con 0x7fb0440cff70 2024-09-10T10:01:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.872+0000 7fb040c00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.872+0000 7fb040c00640 1 -- 172.21.15.110:0/1185697759 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb03c121050 con 0x7fb0440cff70 2024-09-10T10:01:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.872+0000 7fb049000640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.872+0000 7fb049000640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb034001810 tx=0x7fb034009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:41.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:41.876+0000 7fb040c00640 1 -- 172.21.15.110:0/1185697759 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb03c0c6030 con 0x7fb0440cff70 2024-09-10T10:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb044000970 con 0x7fb0440cff70 2024-09-10T10:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb040c00640 1 -- 172.21.15.110:0/1185697759 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7fb03c016620 con 0x7fb0440cff70 2024-09-10T10:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:01:42.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 msgr2=0x7fb02406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb034001810 tx=0x7fb034009040 comp rx=0 tx=0).stop 2024-09-10T10:01:42.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 msgr2=0x7fb044155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb044155da0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb03c01a5d0 tx=0x7fb03c019740 comp rx=0 tx=0).stop 2024-09-10T10:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 shutdown_connections 2024-09-10T10:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0440cff70 0x7fb044155da0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0440cfb90 0x7fb044163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 --2- 172.21.15.110:0/1185697759 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0440b3f40 0x7fb044161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 >> 172.21.15.110:0/1185697759 conn(0x7fb044062d10 msgr2=0x7fb0440b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 shutdown_connections 2024-09-10T10:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.044+0000 7fb04ba00640 1 -- 172.21.15.110:0/1185697759 wait complete. 2024-09-10T10:01:42.067 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:01:42.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:42.067 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:01:42.112 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:01:42.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:42.113 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:01:42.166 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:01:42.166 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:42.166 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:42.175 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:42.175 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:42.226 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:01:42.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73b4a00640 1 Processor -- start 2024-09-10T10:01:42.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73b4a00640 1 -- start start 2024-09-10T10:01:42.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73b4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b00b1b80 0x7f73b0073200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73b4a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f73b0071900 0x7f73b0073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73b4a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b00d2d10 con 0x7f73b015c570 2024-09-10T10:01:42.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73b4a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f73b00d2ff0 con 0x7f73b0071900 2024-09-10T10:01:42.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73b4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73b00d0630 con 0x7f73b00b1b80 2024-09-10T10:01:42.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ae800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f73b0071900 0x7f73b0073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ad400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b00b1b80 0x7f73b0073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ade00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f73b015c570 0x7f73b00d1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:37840/0 (socket says 172.21.15.110:37840) 2024-09-10T10:01:42.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ae800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f73b0071900 0x7f73b0073740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:55836/0 (socket says 172.21.15.110:55836) 2024-09-10T10:01:42.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ad400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b00b1b80 0x7f73b0073200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:45712/0 (socket says 172.21.15.110:45712) 2024-09-10T10:01:42.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ade00640 1 -- 172.21.15.110:0/640898523 learned_addr learned my addr 172.21.15.110:0/640898523 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:42.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73aca00640 1 -- 172.21.15.110:0/640898523 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2462386245 0 0) 0x7f73b00d2d10 con 0x7f73b015c570 2024-09-10T10:01:42.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73aca00640 1 -- 172.21.15.110:0/640898523 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7394003530 con 0x7f73b015c570 2024-09-10T10:01:42.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ad400640 1 -- 172.21.15.110:0/640898523 >> v2:172.21.15.145:3300/0 conn(0x7f73b0071900 msgr2=0x7f73b0073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:42.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ad400640 1 --2- 172.21.15.110:0/640898523 >> v2:172.21.15.145:3300/0 conn(0x7f73b0071900 0x7f73b0073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ad400640 1 -- 172.21.15.110:0/640898523 >> v1:172.21.15.145:6789/0 conn(0x7f73b015c570 legacy=0x7f73b00d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ad400640 1 -- 172.21.15.110:0/640898523 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b00d0910 con 0x7f73b00b1b80 2024-09-10T10:01:42.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.364+0000 7f73ae800640 1 --2- 172.21.15.110:0/640898523 >> v2:172.21.15.145:3300/0 conn(0x7f73b0071900 0x7f73b0073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:42.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73ad400640 1 --2- 172.21.15.110:0/640898523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b00b1b80 0x7f73b0073200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7398009d90 tx=0x7f739801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a9a3f933f32c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:42.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73aca00640 1 -- 172.21.15.110:0/640898523 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980b2070 con 0x7f73b00b1b80 2024-09-10T10:01:42.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73aca00640 1 -- 172.21.15.110:0/640898523 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73980a5970 con 0x7f73b00b1b80 2024-09-10T10:01:42.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73aca00640 1 -- 172.21.15.110:0/640898523 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980a5d20 con 0x7f73b00b1b80 2024-09-10T10:01:42.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73b4a00640 1 -- 172.21.15.110:0/640898523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b00b1b80 msgr2=0x7f73b0073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73b4a00640 1 --2- 172.21.15.110:0/640898523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b00b1b80 0x7f73b0073200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7398009d90 tx=0x7f739801c8b0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73b4a00640 1 -- 172.21.15.110:0/640898523 shutdown_connections 2024-09-10T10:01:42.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73b4a00640 1 --2- 172.21.15.110:0/640898523 >> v2:172.21.15.145:3300/0 conn(0x7f73b0071900 0x7f73b0073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73b4a00640 1 --2- 172.21.15.110:0/640898523 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b00b1b80 0x7f73b0073200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73b4a00640 1 -- 172.21.15.110:0/640898523 >> 172.21.15.110:0/640898523 conn(0x7f73b006f440 msgr2=0x7f73b006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:42.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73b4a00640 1 -- 172.21.15.110:0/640898523 shutdown_connections 2024-09-10T10:01:42.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.368+0000 7f73b4a00640 1 -- 172.21.15.110:0/640898523 wait complete. 2024-09-10T10:01:42.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 Processor -- start 2024-09-10T10:01:42.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 -- start start 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b0071900 0x7f73b00c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73b00b1b80 0x7f73b01f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b015c570 0x7f73b00b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73b015cad0 con 0x7f73b00b1b80 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f73b015c950 con 0x7f73b0071900 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73b0169400 con 0x7f73b015c570 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73b00b1b80 0x7f73b01f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73b00b1b80 0x7f73b01f7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49624/0 (socket says 172.21.15.110:49624) 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 -- 172.21.15.110:0/879786886 learned_addr learned my addr 172.21.15.110:0/879786886 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ade00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b0071900 0x7f73b00c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ae800640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b015c570 0x7f73b00b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 -- 172.21.15.110:0/879786886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b015c570 msgr2=0x7f73b00b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b015c570 0x7f73b00b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 -- 172.21.15.110:0/879786886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b0071900 msgr2=0x7f73b00c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b0071900 0x7f73b00c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 -- 172.21.15.110:0/879786886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b015eee0 con 0x7f73b00b1b80 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73ad400640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73b00b1b80 0x7f73b01f7420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f739801a4e0 tx=0x7f7398019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73a6000640 1 -- 172.21.15.110:0/879786886 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980b2050 con 0x7f73b00b1b80 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73a6000640 1 -- 172.21.15.110:0/879786886 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73980b3040 con 0x7f73b00b1b80 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73a6000640 1 -- 172.21.15.110:0/879786886 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398003db0 con 0x7f73b00b1b80 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b01f7960 con 0x7f73b00b1b80 2024-09-10T10:01:42.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.372+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b015f460 con 0x7f73b00b1b80 2024-09-10T10:01:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.376+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b00d1110 con 0x7f73b00b1b80 2024-09-10T10:01:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.376+0000 7f73a6000640 1 -- 172.21.15.110:0/879786886 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f73980c0020 con 0x7f73b00b1b80 2024-09-10T10:01:42.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.376+0000 7f73a6000640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f738c067bc0 0x7f738c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.376+0000 7f73a6000640 1 -- 172.21.15.110:0/879786886 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f73981206f0 con 0x7f73b00b1b80 2024-09-10T10:01:42.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.376+0000 7f73ade00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f738c067bc0 0x7f738c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.376+0000 7f73ade00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f738c067bc0 0x7f738c06a080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f73a0007fd0 tx=0x7f73a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:42.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.380+0000 7f73a6000640 1 -- 172.21.15.110:0/879786886 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f73980c7020 con 0x7f73b00b1b80 2024-09-10T10:01:42.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.548+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f73b00b6db0 con 0x7f73b00b1b80 2024-09-10T10:01:42.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.548+0000 7f73a6000640 1 -- 172.21.15.110:0/879786886 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7f7398016560 con 0x7f73b00b1b80 2024-09-10T10:01:42.553 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:01:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f738c067bc0 msgr2=0x7f738c06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f738c067bc0 0x7f738c06a080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f73a0007fd0 tx=0x7f73a0009040 comp rx=0 tx=0).stop 2024-09-10T10:01:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73b00b1b80 msgr2=0x7f73b01f7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73b00b1b80 0x7f73b01f7420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f739801a4e0 tx=0x7f7398019650 comp rx=0 tx=0).stop 2024-09-10T10:01:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 shutdown_connections 2024-09-10T10:01:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f738c067bc0 0x7f738c06a080 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73b015c570 0x7f73b00b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73b00b1b80 0x7f73b01f7420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 --2- 172.21.15.110:0/879786886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b0071900 0x7f73b00c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 >> 172.21.15.110:0/879786886 conn(0x7f73b006f440 msgr2=0x7f73b00b60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:42.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 shutdown_connections 2024-09-10T10:01:42.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.552+0000 7f73b4a00640 1 -- 172.21.15.110:0/879786886 wait complete. 2024-09-10T10:01:42.574 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:01:42.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:42.574 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:01:42.600 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:01:42.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:01:42.601 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:01:42.651 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:01:42.651 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:42.651 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:42.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:01:42.660 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:01:42.660 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:42.660 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:42.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:01:42.702 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:42.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4ff600640 1 Processor -- start 2024-09-10T10:01:42.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4ff600640 1 -- start start 2024-09-10T10:01:42.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4ff600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4f80730c0 0x7fc4f806ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4ff600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f806eec0 con 0x7fc4f815e700 2024-09-10T10:01:42.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4ff600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f806f1a0 con 0x7fc4f806e890 2024-09-10T10:01:42.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4ff600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f815eae0 con 0x7fc4f80730c0 2024-09-10T10:01:42.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4fcc00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc4f815e700 0x7fc4f80af270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40430/0 (socket says 172.21.15.110:40430) 2024-09-10T10:01:42.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4fcc00640 1 -- 172.21.15.110:0/377611017 learned_addr learned my addr 172.21.15.110:0/377611017 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:42.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4fd600640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4f80730c0 0x7fc4f806ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974165744 0 0) 0x7fc4f806eec0 con 0x7fc4f815e700 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4dc003530 con 0x7fc4f815e700 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249187733 0 0) 0x7fc4f806f1a0 con 0x7fc4f806e890 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4f806eec0 con 0x7fc4f806e890 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1480154090 0 0) 0x7fc4dc003530 con 0x7fc4f815e700 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4f80730c0 msgr2=0x7fc4f806ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4f80730c0 0x7fc4f806ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 >> v1:172.21.15.145:6789/0 conn(0x7fc4f806e890 legacy=0x7fc4f80729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f806f480 con 0x7fc4f815e700 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fc4e8001a00 con 0x7fc4f815e700 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 >> v1:172.21.15.110:6789/0 conn(0x7fc4f815e700 legacy=0x7fc4f80af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4dc003b80 0x7fc4dc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4dc004810 0x7fc4dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4f80730c0 0x7fc4dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f815eae0 con 0x7fc4dc004810 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc00ba80 con 0x7fc4dc003b80 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc4dc00bc80 con 0x7fc4f80730c0 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4f7e00640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4dc004810 0x7fc4dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4fcc00640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4dc003b80 0x7fc4dc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.836+0000 7fc4fd600640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4f80730c0 0x7fc4dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7e00640 1 -- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4f80730c0 msgr2=0x7fc4dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7e00640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4f80730c0 0x7fc4dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7e00640 1 -- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4dc003b80 msgr2=0x7fc4dc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7e00640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4dc003b80 0x7fc4dc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7e00640 1 -- 172.21.15.110:0/377611017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc003530 con 0x7fc4dc004810 2024-09-10T10:01:42.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7e00640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4dc004810 0x7fc4dc004c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4007fd0 tx=0x7fc4e401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06013ac25d975e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:42.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4001520 con 0x7fc4dc004810 2024-09-10T10:01:42.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e40052f0 con 0x7fc4dc004810 2024-09-10T10:01:42.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4f7400640 1 -- 172.21.15.110:0/377611017 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40a8910 con 0x7fc4dc004810 2024-09-10T10:01:42.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4ff600640 1 -- 172.21.15.110:0/377611017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4dc004810 msgr2=0x7fc4dc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4ff600640 1 --2- 172.21.15.110:0/377611017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4dc004810 0x7fc4dc004c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4007fd0 tx=0x7fc4e401ecb0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4fd600640 1 -- 172.21.15.110:0/377611017 reap_dead start 2024-09-10T10:01:42.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4ff600640 1 -- 172.21.15.110:0/377611017 shutdown_connections 2024-09-10T10:01:42.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.840+0000 7fc4ff600640 1 -- 172.21.15.110:0/377611017 >> 172.21.15.110:0/377611017 conn(0x7fc4f806cb90 msgr2=0x7fc4f806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:42.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 -- 172.21.15.110:0/377611017 shutdown_connections 2024-09-10T10:01:42.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 -- 172.21.15.110:0/377611017 wait complete. 2024-09-10T10:01:42.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 Processor -- start 2024-09-10T10:01:42.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 -- start start 2024-09-10T10:01:42.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4dc004810 0x7fc4f81605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4f80cee30 0x7fc4f80cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4f814b190 0x7fc4f814b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f806fa40 con 0x7fc4f814b190 2024-09-10T10:01:42.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f80af980 con 0x7fc4f80cee30 2024-09-10T10:01:42.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f806ff60 con 0x7fc4dc004810 2024-09-10T10:01:42.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4f814b190 0x7fc4f814b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4f814b190 0x7fc4f814b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49638/0 (socket says 172.21.15.110:49638) 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 -- 172.21.15.110:0/3502647278 learned_addr learned my addr 172.21.15.110:0/3502647278 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4f7e00640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4f80cee30 0x7fc4f80cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fcc00640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4dc004810 0x7fc4f81605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 -- 172.21.15.110:0/3502647278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4dc004810 msgr2=0x7fc4f81605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4dc004810 0x7fc4f81605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 -- 172.21.15.110:0/3502647278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4f80cee30 msgr2=0x7fc4f80cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4f80cee30 0x7fc4f80cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 -- 172.21.15.110:0/3502647278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f80cf2c0 con 0x7fc4f814b190 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4fd600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4f814b190 0x7fc4f814b640 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80b0920 tx=0x7fc4f0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4f4c00640 1 -- 172.21.15.110:0/3502647278 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0004240 con 0x7fc4f814b190 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4f4c00640 1 -- 172.21.15.110:0/3502647278 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f001b070 con 0x7fc4f814b190 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4f4c00640 1 -- 172.21.15.110:0/3502647278 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000f7a0 con 0x7fc4f814b190 2024-09-10T10:01:42.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f815faf0 con 0x7fc4f814b190 2024-09-10T10:01:42.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.844+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f81600b0 con 0x7fc4f814b190 2024-09-10T10:01:42.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.848+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f8070320 con 0x7fc4f814b190 2024-09-10T10:01:42.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.848+0000 7fc4f4c00640 1 -- 172.21.15.110:0/3502647278 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fc4f0012040 con 0x7fc4f814b190 2024-09-10T10:01:42.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.848+0000 7fc4f4c00640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc4d8067c20 0x7fc4d806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:42.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.848+0000 7fc4f4c00640 1 -- 172.21.15.110:0/3502647278 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc4f0085f40 con 0x7fc4f814b190 2024-09-10T10:01:42.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.848+0000 7fc4fcc00640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc4d8067c20 0x7fc4d806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:42.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.848+0000 7fc4fcc00640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc4d8067c20 0x7fc4d806a0e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8002240 tx=0x7fc4e8006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:42.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:42.852+0000 7fc4f4c00640 1 -- 172.21.15.110:0/3502647278 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc4f0056630 con 0x7fc4f814b190 2024-09-10T10:01:43.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.040+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4f80afb00 con 0x7fc4f814b190 2024-09-10T10:01:43.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.040+0000 7fc4f4c00640 1 -- 172.21.15.110:0/3502647278 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5235 (secure 0 0 0) 0x7fc4f0055f30 con 0x7fc4f814b190 2024-09-10T10:01:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:43.045 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":37,"btime":"2024-09-10T10:01:30:262647+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/2645162871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2645162871},{"type":"v1","addr":"172.21.15.145:6834","nonce":2645162871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1524995501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1524995501},{"type":"v1","addr":"172.21.15.145:6835","nonce":1524995501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/2642324353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2642324353},{"type":"v1","addr":"172.21.15.110:6836","nonce":2642324353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:20.586417+0000","modified":"2024-09-10T10:01:23.606983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1333637048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":1333637048},{"type":"v1","addr":"172.21.15.110:6837","nonce":1333637048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5640,"qdb_cluster":[5640]},"id":4}]} 2024-09-10T10:01:43.045 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 37 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc4d8067c20 msgr2=0x7fc4d806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc4d8067c20 0x7fc4d806a0e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8002240 tx=0x7fc4e8006040 comp rx=0 tx=0).stop 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4f814b190 msgr2=0x7fc4f814b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4f814b190 0x7fc4f814b640 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80b0920 tx=0x7fc4f0011cf0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 shutdown_connections 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc4d8067c20 0x7fc4d806a0e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4f814b190 0x7fc4f814b640 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4f80cee30 0x7fc4f80cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 --2- 172.21.15.110:0/3502647278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc4dc004810 0x7fc4f81605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 >> 172.21.15.110:0/3502647278 conn(0x7fc4f806cb90 msgr2=0x7fc4f80b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:43.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 shutdown_connections 2024-09-10T10:01:43.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.044+0000 7fc4ff600640 1 -- 172.21.15.110:0/3502647278 wait complete. 2024-09-10T10:01:43.067 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:43.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.196+0000 7f7444800640 1 Processor -- start 2024-09-10T10:01:43.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.196+0000 7f7444800640 1 -- start start 2024-09-10T10:01:43.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7444800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f743c06f1e0 0x7f743c166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7444800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f743c155d50 con 0x7f743c15c560 2024-09-10T10:01:43.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7444800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f743c0b34d0 con 0x7f743c155970 2024-09-10T10:01:43.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7444800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f743c0b4570 con 0x7f743c06f1e0 2024-09-10T10:01:43.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7441400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f743c15c560 0x7f743c06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40440/0 (socket says 172.21.15.110:40440) 2024-09-10T10:01:43.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7441400640 1 -- 172.21.15.110:0/947688311 learned_addr learned my addr 172.21.15.110:0/947688311 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:43.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7442800640 1 --2- 172.21.15.110:0/947688311 >> v2:172.21.15.145:3300/0 conn(0x7f743c06f1e0 0x7f743c166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7440a00640 1 -- 172.21.15.110:0/947688311 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 815798444 0 0) 0x7f743c155d50 con 0x7f743c15c560 2024-09-10T10:01:43.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7440a00640 1 -- 172.21.15.110:0/947688311 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7424003530 con 0x7f743c15c560 2024-09-10T10:01:43.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7442800640 1 -- 172.21.15.110:0/947688311 >> v1:172.21.15.145:6789/0 conn(0x7f743c155970 legacy=0x7f743c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7442800640 1 -- 172.21.15.110:0/947688311 >> v1:172.21.15.110:6789/0 conn(0x7f743c15c560 legacy=0x7f743c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.200+0000 7f7442800640 1 -- 172.21.15.110:0/947688311 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c0b37b0 con 0x7f743c06f1e0 2024-09-10T10:01:43.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7442800640 1 --2- 172.21.15.110:0/947688311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 0x7f743c166020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7438009d90 tx=0x7f743801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f630e0442d0c9d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7440a00640 1 -- 172.21.15.110:0/947688311 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380b2070 con 0x7f743c06f1e0 2024-09-10T10:01:43.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7440a00640 1 -- 172.21.15.110:0/947688311 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74380a5a00 con 0x7f743c06f1e0 2024-09-10T10:01:43.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7440a00640 1 -- 172.21.15.110:0/947688311 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380a5dd0 con 0x7f743c06f1e0 2024-09-10T10:01:43.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 -- 172.21.15.110:0/947688311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 msgr2=0x7f743c166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 --2- 172.21.15.110:0/947688311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 0x7f743c166020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7438009d90 tx=0x7f743801c940 comp rx=0 tx=0).stop 2024-09-10T10:01:43.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 -- 172.21.15.110:0/947688311 shutdown_connections 2024-09-10T10:01:43.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 --2- 172.21.15.110:0/947688311 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 0x7f743c166020 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 -- 172.21.15.110:0/947688311 >> 172.21.15.110:0/947688311 conn(0x7f743c06bc50 msgr2=0x7f743c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:43.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 -- 172.21.15.110:0/947688311 shutdown_connections 2024-09-10T10:01:43.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 -- 172.21.15.110:0/947688311 wait complete. 2024-09-10T10:01:43.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 Processor -- start 2024-09-10T10:01:43.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.204+0000 7f7444800640 1 -- start start 2024-09-10T10:01:43.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 0x7f743c147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f743c155970 0x7f743c15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c15c560 0x7f743c0728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f743c168f00 con 0x7f743c15c560 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f743c168d80 con 0x7f743c06f1e0 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f743c169080 con 0x7f743c155970 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c15c560 0x7f743c0728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c15c560 0x7f743c0728f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49654/0 (socket says 172.21.15.110:49654) 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 -- 172.21.15.110:0/4284546304 learned_addr learned my addr 172.21.15.110:0/4284546304 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7441400640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f743c155970 0x7f743c15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7441e00640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 0x7f743c147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 -- 172.21.15.110:0/4284546304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f743c155970 msgr2=0x7f743c15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f743c155970 0x7f743c15ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 -- 172.21.15.110:0/4284546304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 msgr2=0x7f743c147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 0x7f743c147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 -- 172.21.15.110:0/4284546304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c1483b0 con 0x7f743c15c560 2024-09-10T10:01:43.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7442800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c15c560 0x7f743c0728f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f743801a570 tx=0x7f74380196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7436000640 1 -- 172.21.15.110:0/4284546304 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380b2050 con 0x7f743c15c560 2024-09-10T10:01:43.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7436000640 1 -- 172.21.15.110:0/4284546304 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74380be070 con 0x7f743c15c560 2024-09-10T10:01:43.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7436000640 1 -- 172.21.15.110:0/4284546304 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380ada50 con 0x7f743c15c560 2024-09-10T10:01:43.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c15dec0 con 0x7f743c15c560 2024-09-10T10:01:43.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c15e480 con 0x7f743c15c560 2024-09-10T10:01:43.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c0cc8c0 con 0x7f743c15c560 2024-09-10T10:01:43.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.208+0000 7f7436000640 1 -- 172.21.15.110:0/4284546304 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f74380adbf0 con 0x7f743c15c560 2024-09-10T10:01:43.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.212+0000 7f7436000640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7420067c20 0x7f742006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.212+0000 7f7436000640 1 -- 172.21.15.110:0/4284546304 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7438121400 con 0x7f743c15c560 2024-09-10T10:01:43.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.212+0000 7f7441e00640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7420067c20 0x7f742006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.212+0000 7f7441e00640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7420067c20 0x7f742006a0e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7430007fd0 tx=0x7f7430009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.216+0000 7f7436000640 1 -- 172.21.15.110:0/4284546304 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7438016560 con 0x7f743c15c560 2024-09-10T10:01:43.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.408+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f743c0b1840 con 0x7f743c15c560 2024-09-10T10:01:43.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.408+0000 7f7436000640 1 -- 172.21.15.110:0/4284546304 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5235 (secure 0 0 0) 0x7f74380f14a0 con 0x7f743c15c560 2024-09-10T10:01:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:43.412 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":37,"btime":"2024-09-10T10:01:30:262647+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/2645162871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2645162871},{"type":"v1","addr":"172.21.15.145:6834","nonce":2645162871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1524995501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1524995501},{"type":"v1","addr":"172.21.15.145:6835","nonce":1524995501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/2642324353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2642324353},{"type":"v1","addr":"172.21.15.110:6836","nonce":2642324353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:20.586417+0000","modified":"2024-09-10T10:01:23.606983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1333637048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":1333637048},{"type":"v1","addr":"172.21.15.110:6837","nonce":1333637048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5640,"qdb_cluster":[5640]},"id":4}]} 2024-09-10T10:01:43.412 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 37 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7420067c20 msgr2=0x7f742006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7420067c20 0x7f742006a0e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7430007fd0 tx=0x7f7430009040 comp rx=0 tx=0).stop 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c15c560 msgr2=0x7f743c0728f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c15c560 0x7f743c0728f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f743801a570 tx=0x7f74380196e0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 shutdown_connections 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7420067c20 0x7f742006a0e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c15c560 0x7f743c0728f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f743c155970 0x7f743c15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 --2- 172.21.15.110:0/4284546304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f743c06f1e0 0x7f743c147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 >> 172.21.15.110:0/4284546304 conn(0x7f743c06bc50 msgr2=0x7f743c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:43.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 shutdown_connections 2024-09-10T10:01:43.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.412+0000 7f7444800640 1 -- 172.21.15.110:0/4284546304 wait complete. 2024-09-10T10:01:43.434 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:43.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbedc200640 1 Processor -- start 2024-09-10T10:01:43.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbedc200640 1 -- start start 2024-09-10T10:01:43.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbedc200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed4155970 0x7fbed40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbedc200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbed406f1e0 0x7fbed40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbedc200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbed4155d50 con 0x7fbed415c560 2024-09-10T10:01:43.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbedc200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbed40b4570 con 0x7fbed406f1e0 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbedc200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbed40b34d0 con 0x7fbed4155970 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbeda200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbed406f1e0 0x7fbed40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbeda200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbed406f1e0 0x7fbed40ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49658/0 (socket says 172.21.15.110:49658) 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbeda200640 1 -- 172.21.15.110:0/1086738289 learned_addr learned my addr 172.21.15.110:0/1086738289 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.568+0000 7fbed9800640 1 --2- 172.21.15.110:0/1086738289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed4155970 0x7fbed40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbeda200640 1 -- 172.21.15.110:0/1086738289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed4155970 msgr2=0x7fbed40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbeda200640 1 --2- 172.21.15.110:0/1086738289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed4155970 0x7fbed40cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbeda200640 1 -- 172.21.15.110:0/1086738289 >> v1:172.21.15.145:6789/0 conn(0x7fbed415c560 legacy=0x7fbed406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbeda200640 1 -- 172.21.15.110:0/1086738289 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed40b37b0 con 0x7fbed406f1e0 2024-09-10T10:01:43.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbeda200640 1 --2- 172.21.15.110:0/1086738289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed406f1e0 0x7fbed40ce420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009e50 tx=0x7fbecc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da2bae9d6b232e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbed3e00640 1 -- 172.21.15.110:0/1086738289 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0a5830 con 0x7fbed406f1e0 2024-09-10T10:01:43.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbed3e00640 1 -- 172.21.15.110:0/1086738289 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbecc0a59d0 con 0x7fbed406f1e0 2024-09-10T10:01:43.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbed3e00640 1 -- 172.21.15.110:0/1086738289 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0a5da0 con 0x7fbed406f1e0 2024-09-10T10:01:43.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbedc200640 1 -- 172.21.15.110:0/1086738289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed406f1e0 msgr2=0x7fbed40ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbedc200640 1 --2- 172.21.15.110:0/1086738289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed406f1e0 0x7fbed40ce420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009e50 tx=0x7fbecc01c970 comp rx=0 tx=0).stop 2024-09-10T10:01:43.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbedc200640 1 -- 172.21.15.110:0/1086738289 shutdown_connections 2024-09-10T10:01:43.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbedc200640 1 --2- 172.21.15.110:0/1086738289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed406f1e0 0x7fbed40ce420 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbedc200640 1 --2- 172.21.15.110:0/1086738289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed4155970 0x7fbed40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbedc200640 1 -- 172.21.15.110:0/1086738289 >> 172.21.15.110:0/1086738289 conn(0x7fbed406bc50 msgr2=0x7fbed406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:43.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbedc200640 1 -- 172.21.15.110:0/1086738289 shutdown_connections 2024-09-10T10:01:43.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.572+0000 7fbedc200640 1 -- 172.21.15.110:0/1086738289 wait complete. 2024-09-10T10:01:43.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 Processor -- start 2024-09-10T10:01:43.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 -- start start 2024-09-10T10:01:43.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed406f1e0 0x7fbed40aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed4155970 0x7fbed406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed415c560 0x7fbed41523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbed40cf170 con 0x7fbed4155970 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbed40cce00 con 0x7fbed415c560 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbed40cf2f0 con 0x7fbed406f1e0 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed406f1e0 0x7fbed40aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed406f1e0 0x7fbed40aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:45784/0 (socket says 172.21.15.110:45784) 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 -- 172.21.15.110:0/1487959634 learned_addr learned my addr 172.21.15.110:0/1487959634 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed8e00640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed4155970 0x7fbed406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbeda200640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed415c560 0x7fbed41523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 -- 172.21.15.110:0/1487959634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed415c560 msgr2=0x7fbed41523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed415c560 0x7fbed41523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 -- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed4155970 msgr2=0x7fbed406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed4155970 0x7fbed406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 -- 172.21.15.110:0/1487959634 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed4157610 con 0x7fbed406f1e0 2024-09-10T10:01:43.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed8e00640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed4155970 0x7fbed406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:43.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed9800640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed406f1e0 0x7fbed40aae40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbec800d800 tx=0x7fbec800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed1600640 1 -- 172.21.15.110:0/1487959634 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec800b6c0 con 0x7fbed406f1e0 2024-09-10T10:01:43.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed1600640 1 -- 172.21.15.110:0/1487959634 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec8019070 con 0x7fbed406f1e0 2024-09-10T10:01:43.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbed1600640 1 -- 172.21.15.110:0/1487959634 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec800ba60 con 0x7fbed406f1e0 2024-09-10T10:01:43.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.576+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed41578f0 con 0x7fbed406f1e0 2024-09-10T10:01:43.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.580+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed4157eb0 con 0x7fbed406f1e0 2024-09-10T10:01:43.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.580+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed40ce420 con 0x7fbed406f1e0 2024-09-10T10:01:43.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.580+0000 7fbed1600640 1 -- 172.21.15.110:0/1487959634 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fbec800bc00 con 0x7fbed406f1e0 2024-09-10T10:01:43.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.580+0000 7fbed1600640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbeb4067c20 0x7fbeb406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.580+0000 7fbed1600640 1 -- 172.21.15.110:0/1487959634 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fbec80868a0 con 0x7fbed406f1e0 2024-09-10T10:01:43.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.580+0000 7fbed8e00640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbeb4067c20 0x7fbeb406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.580+0000 7fbed8e00640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbeb4067c20 0x7fbeb406a0e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbed4165140 tx=0x7fbec0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.584+0000 7fbed1600640 1 -- 172.21.15.110:0/1487959634 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbec8056940 con 0x7fbed406f1e0 2024-09-10T10:01:43.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.744+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbed4153940 con 0x7fbed406f1e0 2024-09-10T10:01:43.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.744+0000 7fbed1600640 1 -- 172.21.15.110:0/1487959634 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16761 (secure 0 0 0) 0x7fbec8086020 con 0x7fbed406f1e0 2024-09-10T10:01:43.748 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:43.748 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":58,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:22.219663+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:18.299917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:19.590875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbeb4067c20 msgr2=0x7fbeb406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbeb4067c20 0x7fbeb406a0e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbed4165140 tx=0x7fbec0009040 comp rx=0 tx=0).stop 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed406f1e0 msgr2=0x7fbed40aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed406f1e0 0x7fbed40aae40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbec800d800 tx=0x7fbec800dcc0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 shutdown_connections 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbeb4067c20 0x7fbeb406a0e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed415c560 0x7fbed41523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed4155970 0x7fbed406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 --2- 172.21.15.110:0/1487959634 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbed406f1e0 0x7fbed40aae40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 >> 172.21.15.110:0/1487959634 conn(0x7fbed406bc50 msgr2=0x7fbed4153e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:43.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 shutdown_connections 2024-09-10T10:01:43.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.748+0000 7fbedc200640 1 -- 172.21.15.110:0/1487959634 wait complete. 2024-09-10T10:01:43.768 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:01:43.769 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:43.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.900+0000 7fe676200640 1 Processor -- start 2024-09-10T10:01:43.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.900+0000 7fe676200640 1 -- start start 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe676200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe67015c560 0x7fe67006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe676200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe67006f010 0x7fe6700ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe676200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe67006f430 con 0x7fe670155970 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe676200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe6700b4570 con 0x7fe67006f010 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe676200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe670155d50 con 0x7fe67015c560 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe670155970 0x7fe6700cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40450/0 (socket says 172.21.15.110:40450) 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66f400640 1 -- 172.21.15.110:0/2834055318 learned_addr learned my addr 172.21.15.110:0/2834055318 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66fe00640 1 --2- 172.21.15.110:0/2834055318 >> v2:172.21.15.145:3300/0 conn(0x7fe67006f010 0x7fe6700ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66ea00640 1 --2- 172.21.15.110:0/2834055318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe67015c560 0x7fe67006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66e000640 1 -- 172.21.15.110:0/2834055318 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492553620 0 0) 0x7fe67006f430 con 0x7fe670155970 2024-09-10T10:01:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66e000640 1 -- 172.21.15.110:0/2834055318 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe654003530 con 0x7fe670155970 2024-09-10T10:01:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66fe00640 1 -- 172.21.15.110:0/2834055318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe67015c560 msgr2=0x7fe67006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66fe00640 1 --2- 172.21.15.110:0/2834055318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe67015c560 0x7fe67006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66fe00640 1 -- 172.21.15.110:0/2834055318 >> v1:172.21.15.110:6789/0 conn(0x7fe670155970 legacy=0x7fe6700cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.904+0000 7fe66fe00640 1 -- 172.21.15.110:0/2834055318 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6700b34d0 con 0x7fe67006f010 2024-09-10T10:01:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe66fe00640 1 --2- 172.21.15.110:0/2834055318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 0x7fe6700ce420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe668009d90 tx=0x7fe66801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752294028dbdd09e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe66e000640 1 -- 172.21.15.110:0/2834055318 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680b2070 con 0x7fe67006f010 2024-09-10T10:01:43.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe66e000640 1 -- 172.21.15.110:0/2834055318 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6680a5970 con 0x7fe67006f010 2024-09-10T10:01:43.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe66e000640 1 -- 172.21.15.110:0/2834055318 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680a5d40 con 0x7fe67006f010 2024-09-10T10:01:43.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- 172.21.15.110:0/2834055318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 msgr2=0x7fe6700ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 --2- 172.21.15.110:0/2834055318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 0x7fe6700ce420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe668009d90 tx=0x7fe66801c8b0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- 172.21.15.110:0/2834055318 shutdown_connections 2024-09-10T10:01:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 --2- 172.21.15.110:0/2834055318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 0x7fe6700ce420 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 --2- 172.21.15.110:0/2834055318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe67015c560 0x7fe67006ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- 172.21.15.110:0/2834055318 >> 172.21.15.110:0/2834055318 conn(0x7fe67006bc50 msgr2=0x7fe67006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- 172.21.15.110:0/2834055318 shutdown_connections 2024-09-10T10:01:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- 172.21.15.110:0/2834055318 wait complete. 2024-09-10T10:01:43.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 Processor -- start 2024-09-10T10:01:43.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- start start 2024-09-10T10:01:43.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 0x7fe670158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe670155970 0x7fe670147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe67015c560 0x7fe670148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700cf170 con 0x7fe67015c560 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700b4050 con 0x7fe67006f010 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe676200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe6700cf2f0 con 0x7fe670155970 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.908+0000 7fe66fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe67015c560 0x7fe670148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe67015c560 0x7fe670148180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49680/0 (socket says 172.21.15.110:49680) 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66fe00640 1 -- 172.21.15.110:0/2356783517 learned_addr learned my addr 172.21.15.110:0/2356783517 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66ea00640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe670155970 0x7fe670147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66f400640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 0x7fe670158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66fe00640 1 -- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe670155970 msgr2=0x7fe670147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66fe00640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe670155970 0x7fe670147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66fe00640 1 -- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 msgr2=0x7fe670158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66fe00640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 0x7fe670158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66fe00640 1 -- 172.21.15.110:0/2356783517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe670156b20 con 0x7fe67015c560 2024-09-10T10:01:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66ea00640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe670155970 0x7fe670147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe66fe00640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe67015c560 0x7fe670148180 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe66801af60 tx=0x7fe668019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe667400640 1 -- 172.21.15.110:0/2356783517 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680b2050 con 0x7fe67015c560 2024-09-10T10:01:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe667400640 1 -- 172.21.15.110:0/2356783517 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6680be070 con 0x7fe67015c560 2024-09-10T10:01:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe670156e00 con 0x7fe67015c560 2024-09-10T10:01:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe667400640 1 -- 172.21.15.110:0/2356783517 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680ada50 con 0x7fe67015c560 2024-09-10T10:01:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6701573c0 con 0x7fe67015c560 2024-09-10T10:01:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6700ce420 con 0x7fe67015c560 2024-09-10T10:01:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe667400640 1 -- 172.21.15.110:0/2356783517 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe6680a57f0 con 0x7fe67015c560 2024-09-10T10:01:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe667400640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe650067c50 0x7fe65006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.912+0000 7fe667400640 1 -- 172.21.15.110:0/2356783517 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe668121360 con 0x7fe67015c560 2024-09-10T10:01:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.916+0000 7fe66f400640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe650067c50 0x7fe65006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.916+0000 7fe66f400640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe650067c50 0x7fe65006a110 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe660005890 tx=0x7fe660009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:43.920+0000 7fe667400640 1 -- 172.21.15.110:0/2356783517 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe668016560 con 0x7fe67015c560 2024-09-10T10:01:44.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.108+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6700b0e30 con 0x7fe67015c560 2024-09-10T10:01:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.108+0000 7fe667400640 1 -- 172.21.15.110:0/2356783517 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5235 (secure 0 0 0) 0x7fe6680f1ae0 con 0x7fe67015c560 2024-09-10T10:01:44.112 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:44.112 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":37,"btime":"2024-09-10T10:01:30:262647+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/2645162871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2645162871},{"type":"v1","addr":"172.21.15.145:6834","nonce":2645162871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1524995501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1524995501},{"type":"v1","addr":"172.21.15.145:6835","nonce":1524995501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/2642324353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2642324353},{"type":"v1","addr":"172.21.15.110:6836","nonce":2642324353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:20.586417+0000","modified":"2024-09-10T10:01:23.606983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1333637048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":1333637048},{"type":"v1","addr":"172.21.15.110:6837","nonce":1333637048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5640,"qdb_cluster":[5640]},"id":4}]} 2024-09-10T10:01:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 37 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe650067c50 msgr2=0x7fe65006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe650067c50 0x7fe65006a110 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe660005890 tx=0x7fe660009040 comp rx=0 tx=0).stop 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe67015c560 msgr2=0x7fe670148180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe67015c560 0x7fe670148180 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe66801af60 tx=0x7fe668019650 comp rx=0 tx=0).stop 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 shutdown_connections 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe650067c50 0x7fe65006a110 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe67015c560 0x7fe670148180 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe670155970 0x7fe670147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 --2- 172.21.15.110:0/2356783517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe67006f010 0x7fe670158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 >> 172.21.15.110:0/2356783517 conn(0x7fe67006bc50 msgr2=0x7fe6700cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 shutdown_connections 2024-09-10T10:01:44.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.112+0000 7fe676200640 1 -- 172.21.15.110:0/2356783517 wait complete. 2024-09-10T10:01:44.132 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:44.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb889600640 1 Processor -- start 2024-09-10T10:01:44.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb889600640 1 -- start start 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb889600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb8840cd510 0x7fb8840cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb889600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb884072150 con 0x7fb8840cfe30 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb889600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb88406d2d0 con 0x7fb88406e090 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb889600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb88415d9a0 con 0x7fb8840cd510 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb882a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb8840cfe30 0x7fb884071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40456/0 (socket says 172.21.15.110:40456) 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb882a00640 1 -- 172.21.15.110:0/686151937 learned_addr learned my addr 172.21.15.110:0/686151937 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb882000640 1 --2- 172.21.15.110:0/686151937 >> v2:172.21.15.145:3300/0 conn(0x7fb8840cd510 0x7fb8840cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2831675376 0 0) 0x7fb884072150 con 0x7fb8840cfe30 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb868003530 con 0x7fb8840cfe30 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2608147864 0 0) 0x7fb868003530 con 0x7fb8840cfe30 2024-09-10T10:01:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 >> v2:172.21.15.145:3300/0 conn(0x7fb8840cd510 msgr2=0x7fb8840cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 --2- 172.21.15.110:0/686151937 >> v2:172.21.15.145:3300/0 conn(0x7fb8840cd510 0x7fb8840cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 >> v1:172.21.15.145:6789/0 conn(0x7fb88406e090 legacy=0x7fb88406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88406d5b0 con 0x7fb8840cfe30 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fb874001a00 con 0x7fb8840cfe30 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 >> v1:172.21.15.110:6789/0 conn(0x7fb8840cfe30 legacy=0x7fb884071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb868003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb868004810 0x7fb868004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb868005220 0x7fb86800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb88415d9a0 con 0x7fb868005220 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb86800c170 con 0x7fb868003a10 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.268+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb86800c370 con 0x7fb868004810 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb882a00640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb868003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb883400640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb868005220 0x7fb86800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb882000640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb868004810 0x7fb868004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb882a00640 1 -- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb868004810 msgr2=0x7fb868004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb882a00640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb868004810 0x7fb868004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb882a00640 1 -- 172.21.15.110:0/686151937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb868005220 msgr2=0x7fb86800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb882a00640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb868005220 0x7fb86800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb882a00640 1 -- 172.21.15.110:0/686151937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb868003530 con 0x7fb868003a10 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb883400640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb868005220 0x7fb86800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb883400640 1 -- 172.21.15.110:0/686151937 reap_dead start 2024-09-10T10:01:44.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb882a00640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb868003e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb87400a780 tx=0x7fb87401f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2241d41816903d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87400a3d0 con 0x7fb868003a10 2024-09-10T10:01:44.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8740aa630 con 0x7fb868003a10 2024-09-10T10:01:44.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.272+0000 7fb881600640 1 -- 172.21.15.110:0/686151937 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740aaaa0 con 0x7fb868003a10 2024-09-10T10:01:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 msgr2=0x7fb868003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb868003e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb87400a780 tx=0x7fb87401f9e0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- 172.21.15.110:0/686151937 shutdown_connections 2024-09-10T10:01:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 --2- 172.21.15.110:0/686151937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb868003e30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- 172.21.15.110:0/686151937 >> 172.21.15.110:0/686151937 conn(0x7fb88406bc50 msgr2=0x7fb88406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- 172.21.15.110:0/686151937 shutdown_connections 2024-09-10T10:01:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- 172.21.15.110:0/686151937 wait complete. 2024-09-10T10:01:44.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 Processor -- start 2024-09-10T10:01:44.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- start start 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb88414d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb8840b0040 0x7fb884141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8840ae870 0x7fb8840aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb8840cdb20 con 0x7fb8840ae870 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb8840cd9a0 con 0x7fb868003a10 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb889600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb8840cdca0 con 0x7fb8840b0040 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb883400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8840ae870 0x7fb8840aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb883400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8840ae870 0x7fb8840aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49694/0 (socket says 172.21.15.110:49694) 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb883400640 1 -- 172.21.15.110:0/3979623574 learned_addr learned my addr 172.21.15.110:0/3979623574 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb882a00640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb88414d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb882000640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb8840b0040 0x7fb884141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb883400640 1 -- 172.21.15.110:0/3979623574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb8840b0040 msgr2=0x7fb884141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb883400640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb8840b0040 0x7fb884141be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb883400640 1 -- 172.21.15.110:0/3979623574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 msgr2=0x7fb88414d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb883400640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb88414d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.276+0000 7fb883400640 1 -- 172.21.15.110:0/3979623574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8840b0520 con 0x7fb8840ae870 2024-09-10T10:01:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb883400640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8840ae870 0x7fb8840aed20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb8840715a0 tx=0x7fb87c00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb87aa00640 1 -- 172.21.15.110:0/3979623574 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c01a070 con 0x7fb8840ae870 2024-09-10T10:01:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb87aa00640 1 -- 172.21.15.110:0/3979623574 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb87c005110 con 0x7fb8840ae870 2024-09-10T10:01:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb87aa00640 1 -- 172.21.15.110:0/3979623574 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0013a0 con 0x7fb8840ae870 2024-09-10T10:01:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8840b0800 con 0x7fb8840ae870 2024-09-10T10:01:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8840b0dc0 con 0x7fb8840ae870 2024-09-10T10:01:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88406e4e0 con 0x7fb8840ae870 2024-09-10T10:01:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb87aa00640 1 -- 172.21.15.110:0/3979623574 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb87c01d070 con 0x7fb8840ae870 2024-09-10T10:01:44.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb87aa00640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb864067c20 0x7fb86406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb87aa00640 1 -- 172.21.15.110:0/3979623574 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb87c085420 con 0x7fb8840ae870 2024-09-10T10:01:44.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb882a00640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb864067c20 0x7fb86406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.280+0000 7fb882a00640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb864067c20 0x7fb86406a0e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb868004790 tx=0x7fb8740b5040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.284+0000 7fb87aa00640 1 -- 172.21.15.110:0/3979623574 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb87c055550 con 0x7fb8840ae870 2024-09-10T10:01:44.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.476+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb88415d650 con 0x7fb8840ae870 2024-09-10T10:01:44.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.476+0000 7fb87aa00640 1 -- 172.21.15.110:0/3979623574 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5235 (secure 0 0 0) 0x7fb87c059850 con 0x7fb8840ae870 2024-09-10T10:01:44.480 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:44.480 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":37,"btime":"2024-09-10T10:01:30:262647+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/2645162871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2645162871},{"type":"v1","addr":"172.21.15.145:6834","nonce":2645162871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1524995501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1524995501},{"type":"v1","addr":"172.21.15.145:6835","nonce":1524995501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/2642324353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2642324353},{"type":"v1","addr":"172.21.15.110:6836","nonce":2642324353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:20.586417+0000","modified":"2024-09-10T10:01:23.606983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5640":{"gid":5640,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6837/1333637048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":1333637048},{"type":"v1","addr":"172.21.15.110:6837","nonce":1333637048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5640,"qdb_cluster":[5640]},"id":4}]} 2024-09-10T10:01:44.480 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 37 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb864067c20 msgr2=0x7fb86406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb864067c20 0x7fb86406a0e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb868004790 tx=0x7fb8740b5040 comp rx=0 tx=0).stop 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8840ae870 msgr2=0x7fb8840aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8840ae870 0x7fb8840aed20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb8840715a0 tx=0x7fb87c00fb40 comp rx=0 tx=0).stop 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 shutdown_connections 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb864067c20 0x7fb86406a0e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8840ae870 0x7fb8840aed20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb8840b0040 0x7fb884141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 --2- 172.21.15.110:0/3979623574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb868003a10 0x7fb88414d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 >> 172.21.15.110:0/3979623574 conn(0x7fb88406bc50 msgr2=0x7fb884153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 shutdown_connections 2024-09-10T10:01:44.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.480+0000 7fb889600640 1 -- 172.21.15.110:0/3979623574 wait complete. 2024-09-10T10:01:44.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:44.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e28600640 1 Processor -- start 2024-09-10T10:01:44.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e28600640 1 -- start start 2024-09-10T10:01:44.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e28600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e200cfe30 0x7f7e20071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e28600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2006d2d0 con 0x7f7e200cd510 2024-09-10T10:01:44.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e28600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2006d5b0 con 0x7f7e2006e7a0 2024-09-10T10:01:44.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e28600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2015d9a0 con 0x7f7e200cfe30 2024-09-10T10:01:44.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e25200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f7e200cd510 0x7f7e2006e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40464/0 (socket says 172.21.15.110:40464) 2024-09-10T10:01:44.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e25200640 1 -- 172.21.15.110:0/1628126196 learned_addr learned my addr 172.21.15.110:0/1628126196 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:44.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.632+0000 7f7e25c00640 1 --2- 172.21.15.110:0/1628126196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e200cfe30 0x7f7e20071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e1fe00640 1 -- 172.21.15.110:0/1628126196 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1522889123 0 0) 0x7f7e2006d2d0 con 0x7f7e200cd510 2024-09-10T10:01:44.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e1fe00640 1 -- 172.21.15.110:0/1628126196 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e08003530 con 0x7f7e200cd510 2024-09-10T10:01:44.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e1fe00640 1 -- 172.21.15.110:0/1628126196 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291842064 0 0) 0x7f7e2006d5b0 con 0x7f7e2006e7a0 2024-09-10T10:01:44.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e1fe00640 1 -- 172.21.15.110:0/1628126196 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e2006d2d0 con 0x7f7e2006e7a0 2024-09-10T10:01:44.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e25c00640 1 -- 172.21.15.110:0/1628126196 >> v1:172.21.15.145:6789/0 conn(0x7f7e2006e7a0 legacy=0x7f7e2015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e25c00640 1 -- 172.21.15.110:0/1628126196 >> v1:172.21.15.110:6789/0 conn(0x7f7e200cd510 legacy=0x7f7e2006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e25c00640 1 -- 172.21.15.110:0/1628126196 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2006d890 con 0x7f7e200cfe30 2024-09-10T10:01:44.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e25c00640 1 --2- 172.21.15.110:0/1628126196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e200cfe30 0x7f7e20071a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009dc0 tx=0x7f7e1401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0657356c7bdc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e1fe00640 1 -- 172.21.15.110:0/1628126196 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140b2070 con 0x7f7e200cfe30 2024-09-10T10:01:44.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e1fe00640 1 -- 172.21.15.110:0/1628126196 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e140a58d0 con 0x7f7e200cfe30 2024-09-10T10:01:44.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.636+0000 7f7e1fe00640 1 -- 172.21.15.110:0/1628126196 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140a5ca0 con 0x7f7e200cfe30 2024-09-10T10:01:44.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- 172.21.15.110:0/1628126196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e200cfe30 msgr2=0x7f7e20071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 --2- 172.21.15.110:0/1628126196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e200cfe30 0x7f7e20071a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009dc0 tx=0x7f7e1401c900 comp rx=0 tx=0).stop 2024-09-10T10:01:44.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- 172.21.15.110:0/1628126196 shutdown_connections 2024-09-10T10:01:44.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 --2- 172.21.15.110:0/1628126196 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e200cfe30 0x7f7e20071a40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- 172.21.15.110:0/1628126196 >> 172.21.15.110:0/1628126196 conn(0x7f7e2006bc50 msgr2=0x7f7e2006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- 172.21.15.110:0/1628126196 shutdown_connections 2024-09-10T10:01:44.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- 172.21.15.110:0/1628126196 wait complete. 2024-09-10T10:01:44.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 Processor -- start 2024-09-10T10:01:44.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- start start 2024-09-10T10:01:44.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2006e7a0 0x7f7e201513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e200cd510 0x7f7e20155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e200cfe30 0x7f7e20156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2015d740 con 0x7f7e200cd510 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20072190 con 0x7f7e200cfe30 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e28600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2016abb0 con 0x7f7e2006e7a0 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e200cd510 0x7f7e20155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e200cd510 0x7f7e20155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49702/0 (socket says 172.21.15.110:49702) 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25200640 1 -- 172.21.15.110:0/2988316218 learned_addr learned my addr 172.21.15.110:0/2988316218 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25c00640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2006e7a0 0x7f7e201513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e26600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e200cfe30 0x7f7e20156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25200640 1 -- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2006e7a0 msgr2=0x7f7e201513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25200640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2006e7a0 0x7f7e201513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25200640 1 -- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e200cfe30 msgr2=0x7f7e20156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25200640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e200cfe30 0x7f7e20156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25200640 1 -- 172.21.15.110:0/2988316218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e200b2c20 con 0x7f7e200cd510 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e25c00640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2006e7a0 0x7f7e201513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.640+0000 7f7e26600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e200cfe30 0x7f7e20156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e25200640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e200cd510 0x7f7e20155cd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005890 tx=0x7f7e0c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e1d600640 1 -- 172.21.15.110:0/2988316218 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c017070 con 0x7f7e200cd510 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e1d600640 1 -- 172.21.15.110:0/2988316218 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0c005cf0 con 0x7f7e200cd510 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e200b2380 con 0x7f7e200cd510 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e1d600640 1 -- 172.21.15.110:0/2988316218 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c007310 con 0x7f7e200cd510 2024-09-10T10:01:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e200b2940 con 0x7f7e200cd510 2024-09-10T10:01:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2015f1c0 con 0x7f7e200cd510 2024-09-10T10:01:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e1d600640 1 -- 172.21.15.110:0/2988316218 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f7e0c01a070 con 0x7f7e200cd510 2024-09-10T10:01:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e1d600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7e00067c20 0x7f7e0006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e1d600640 1 -- 172.21.15.110:0/2988316218 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7e0c085630 con 0x7f7e200cd510 2024-09-10T10:01:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.644+0000 7f7e25c00640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7e00067c20 0x7f7e0006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.648+0000 7f7e25c00640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7e00067c20 0x7f7e0006a0e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e140af000 tx=0x7f7e140a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.648+0000 7f7e1d600640 1 -- 172.21.15.110:0/2988316218 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7e0c055d60 con 0x7f7e200cd510 2024-09-10T10:01:44.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.808+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e2015eee0 con 0x7f7e200cd510 2024-09-10T10:01:44.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.808+0000 7f7e1d600640 1 -- 172.21.15.110:0/2988316218 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16761 (secure 0 0 0) 0x7f7e0c055660 con 0x7f7e200cd510 2024-09-10T10:01:44.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:44.812 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":58,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:22.219663+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:18.299917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:19.590875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7e00067c20 msgr2=0x7f7e0006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7e00067c20 0x7f7e0006a0e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e140af000 tx=0x7f7e140a7070 comp rx=0 tx=0).stop 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e200cd510 msgr2=0x7f7e20155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e200cd510 0x7f7e20155cd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005890 tx=0x7f7e0c00c8b0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 shutdown_connections 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7e00067c20 0x7f7e0006a0e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e200cfe30 0x7f7e20156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e200cd510 0x7f7e20155cd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 --2- 172.21.15.110:0/2988316218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7e2006e7a0 0x7f7e201513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 >> 172.21.15.110:0/2988316218 conn(0x7f7e2006bc50 msgr2=0x7f7e20167960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 shutdown_connections 2024-09-10T10:01:44.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.812+0000 7f7e28600640 1 -- 172.21.15.110:0/2988316218 wait complete. 2024-09-10T10:01:44.833 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:01:44.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.964+0000 7f4b0d200640 1 Processor -- start 2024-09-10T10:01:44.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.964+0000 7f4b0d200640 1 -- start start 2024-09-10T10:01:44.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b0d200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4b080726e0 0x7f4b080cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b0d200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4b080b5600 0x7f4b080b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b0d200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0806f410 con 0x7f4b0806efa0 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b0d200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4b080cee50 con 0x7f4b080b5600 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b0d200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0815fda0 con 0x7f4b080726e0 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b06800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4b080b5600 0x7f4b080b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b05400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f4b0806efa0 0x7f4b080b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40472/0 (socket says 172.21.15.110:40472) 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b05400640 1 -- 172.21.15.110:0/1530176539 learned_addr learned my addr 172.21.15.110:0/1530176539 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b05e00640 1 --2- 172.21.15.110:0/1530176539 >> v2:172.21.15.145:3300/0 conn(0x7f4b080726e0 0x7f4b080cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b06800640 1 -- 172.21.15.110:0/1530176539 >> v2:172.21.15.145:3300/0 conn(0x7f4b080726e0 msgr2=0x7f4b080cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b06800640 1 --2- 172.21.15.110:0/1530176539 >> v2:172.21.15.145:3300/0 conn(0x7f4b080726e0 0x7f4b080cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b06800640 1 -- 172.21.15.110:0/1530176539 >> v1:172.21.15.110:6789/0 conn(0x7f4b0806efa0 legacy=0x7f4b080b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b06800640 1 -- 172.21.15.110:0/1530176539 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080ce520 con 0x7f4b080b5600 2024-09-10T10:01:44.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b04a00640 1 -- 172.21.15.110:0/1530176539 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2038691863 0 0) 0x7f4af4001120 con 0x7f4b0806efa0 2024-09-10T10:01:44.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.968+0000 7f4b05e00640 1 --2- 172.21.15.110:0/1530176539 >> v2:172.21.15.145:3300/0 conn(0x7f4b080726e0 0x7f4b080cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:44.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b06800640 1 --2- 172.21.15.110:0/1530176539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b080b34d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4b0806c8f0 tx=0x7f4b0001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2801c2bf772af631 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b04a00640 1 -- 172.21.15.110:0/1530176539 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000b2070 con 0x7f4b080b5600 2024-09-10T10:01:44.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b04a00640 1 -- 172.21.15.110:0/1530176539 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b000a59c0 con 0x7f4b080b5600 2024-09-10T10:01:44.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b04a00640 1 -- 172.21.15.110:0/1530176539 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000a5d90 con 0x7f4b080b5600 2024-09-10T10:01:44.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 -- 172.21.15.110:0/1530176539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 msgr2=0x7f4b080b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 --2- 172.21.15.110:0/1530176539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b080b34d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4b0806c8f0 tx=0x7f4b0001c900 comp rx=0 tx=0).stop 2024-09-10T10:01:44.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 -- 172.21.15.110:0/1530176539 shutdown_connections 2024-09-10T10:01:44.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 --2- 172.21.15.110:0/1530176539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b080b34d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 --2- 172.21.15.110:0/1530176539 >> v2:172.21.15.145:3300/0 conn(0x7f4b080726e0 0x7f4b080cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 -- 172.21.15.110:0/1530176539 >> 172.21.15.110:0/1530176539 conn(0x7f4b08062d10 msgr2=0x7f4b0806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 -- 172.21.15.110:0/1530176539 shutdown_connections 2024-09-10T10:01:44.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 -- 172.21.15.110:0/1530176539 wait complete. 2024-09-10T10:01:44.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 Processor -- start 2024-09-10T10:01:44.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.972+0000 7f4b0d200640 1 -- start start 2024-09-10T10:01:44.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b0806efa0 0x7f4b0815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4b080726e0 0x7f4b0815d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b0814ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08072c40 con 0x7f4b080b5600 2024-09-10T10:01:44.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08072ac0 con 0x7f4b0806efa0 2024-09-10T10:01:44.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0815f950 con 0x7f4b080726e0 2024-09-10T10:01:44.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b0814ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b0814ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49716/0 (socket says 172.21.15.110:49716) 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 -- 172.21.15.110:0/3075832230 learned_addr learned my addr 172.21.15.110:0/3075832230 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b05e00640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b0806efa0 0x7f4b0815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b05400640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4b080726e0 0x7f4b0815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 -- 172.21.15.110:0/3075832230 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4b080726e0 msgr2=0x7f4b0815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4b080726e0 0x7f4b0815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 -- 172.21.15.110:0/3075832230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b0806efa0 msgr2=0x7f4b0815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b0806efa0 0x7f4b0815cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 -- 172.21.15.110:0/3075832230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0815ad50 con 0x7f4b080b5600 2024-09-10T10:01:44.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b06800640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b0814ca70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4b000b0040 tx=0x7f4b000196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4afe000640 1 -- 172.21.15.110:0/3075832230 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000b2050 con 0x7f4b080b5600 2024-09-10T10:01:44.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4afe000640 1 -- 172.21.15.110:0/3075832230 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b000b3040 con 0x7f4b080b5600 2024-09-10T10:01:44.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4afe000640 1 -- 172.21.15.110:0/3075832230 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000ad850 con 0x7f4b080b5600 2024-09-10T10:01:44.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b08159cb0 con 0x7f4b080b5600 2024-09-10T10:01:44.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0815a270 con 0x7f4b080b5600 2024-09-10T10:01:44.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b080b34d0 con 0x7f4b080b5600 2024-09-10T10:01:44.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.976+0000 7f4afe000640 1 -- 172.21.15.110:0/3075832230 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f4b000c0020 con 0x7f4b080b5600 2024-09-10T10:01:44.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.980+0000 7f4afe000640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4ae8067c20 0x7f4ae806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:44.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.980+0000 7f4afe000640 1 -- 172.21.15.110:0/3075832230 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4b00120b90 con 0x7f4b080b5600 2024-09-10T10:01:44.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.980+0000 7f4b05e00640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4ae8067c20 0x7f4ae806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:44.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.980+0000 7f4b05e00640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4ae8067c20 0x7f4ae806a0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009950 tx=0x7f4af8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:44.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:44.984+0000 7f4afe000640 1 -- 172.21.15.110:0/3075832230 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4b00016520 con 0x7f4b080b5600 2024-09-10T10:01:45.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.172+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4b0815c5f0 con 0x7f4b080b5600 2024-09-10T10:01:45.175 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:01:45.172+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:01:45.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4afe000640 1 -- 172.21.15.110:0/3075832230 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f4b000f0cc0 con 0x7f4b080b5600 2024-09-10T10:01:45.178 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:01:45.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4ae8067c20 msgr2=0x7f4ae806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4ae8067c20 0x7f4ae806a0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009950 tx=0x7f4af8008040 comp rx=0 tx=0).stop 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 msgr2=0x7f4b0814ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b0814ca70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4b000b0040 tx=0x7f4b000196a0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 shutdown_connections 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4ae8067c20 0x7f4ae806a0e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b080b5600 0x7f4b0814ca70 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4b080726e0 0x7f4b0815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 --2- 172.21.15.110:0/3075832230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b0806efa0 0x7f4b0815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.176+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 >> 172.21.15.110:0/3075832230 conn(0x7f4b08062d10 msgr2=0x7f4b080714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.180+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 shutdown_connections 2024-09-10T10:01:45.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.180+0000 7f4b0d200640 1 -- 172.21.15.110:0/3075832230 wait complete. 2024-09-10T10:01:45.203 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:01:45.249 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:01:45.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.344+0000 7fd711000640 1 Processor -- start 2024-09-10T10:01:45.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.344+0000 7fd711000640 1 -- start start 2024-09-10T10:01:45.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd711000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c15b330 0x7fd70c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd711000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd70c06e850 0x7fd70c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd711000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd70c0b4960 con 0x7fd70c0ce4d0 2024-09-10T10:01:45.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd711000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd70c0b5570 con 0x7fd70c06e850 2024-09-10T10:01:45.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd711000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd70c0b4c40 con 0x7fd70c15b330 2024-09-10T10:01:45.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd70a800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd70c06e850 0x7fd70c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd70a800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd70c06e850 0x7fd70c0b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49750/0 (socket says 172.21.15.110:49750) 2024-09-10T10:01:45.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd709e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c15b330 0x7fd70c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd70a800640 1 -- 172.21.15.110:0/1047943788 learned_addr learned my addr 172.21.15.110:0/1047943788 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:45.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd70a800640 1 -- 172.21.15.110:0/1047943788 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c15b330 msgr2=0x7fd70c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:45.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd70a800640 1 --2- 172.21.15.110:0/1047943788 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c15b330 0x7fd70c06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd70a800640 1 -- 172.21.15.110:0/1047943788 >> v1:172.21.15.145:6789/0 conn(0x7fd70c0ce4d0 legacy=0x7fd70c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd70a800640 1 -- 172.21.15.110:0/1047943788 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c0b4f20 con 0x7fd70c06e850 2024-09-10T10:01:45.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.348+0000 7fd70a800640 1 --2- 172.21.15.110:0/1047943788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c06e850 0x7fd70c0b0860 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd704009e50 tx=0x7fd70401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c380099187eb9195 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:45.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd708a00640 1 -- 172.21.15.110:0/1047943788 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7040a5810 con 0x7fd70c06e850 2024-09-10T10:01:45.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd708a00640 1 -- 172.21.15.110:0/1047943788 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7040a59b0 con 0x7fd70c06e850 2024-09-10T10:01:45.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd708a00640 1 -- 172.21.15.110:0/1047943788 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7040a5d80 con 0x7fd70c06e850 2024-09-10T10:01:45.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- 172.21.15.110:0/1047943788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c06e850 msgr2=0x7fd70c0b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 --2- 172.21.15.110:0/1047943788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c06e850 0x7fd70c0b0860 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd704009e50 tx=0x7fd70401ca30 comp rx=0 tx=0).stop 2024-09-10T10:01:45.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- 172.21.15.110:0/1047943788 shutdown_connections 2024-09-10T10:01:45.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 --2- 172.21.15.110:0/1047943788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c06e850 0x7fd70c0b0860 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 --2- 172.21.15.110:0/1047943788 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c15b330 0x7fd70c06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- 172.21.15.110:0/1047943788 >> 172.21.15.110:0/1047943788 conn(0x7fd70c062e40 msgr2=0x7fd70c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:45.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- 172.21.15.110:0/1047943788 shutdown_connections 2024-09-10T10:01:45.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- 172.21.15.110:0/1047943788 wait complete. 2024-09-10T10:01:45.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 Processor -- start 2024-09-10T10:01:45.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- start start 2024-09-10T10:01:45.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c06e850 0x7fd70c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd70c15b330 0x7fd70c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c15d050 0x7fd70c152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1660a0 con 0x7fd70c15d050 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c0b17c0 con 0x7fd70c15b330 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd711000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd70c166220 con 0x7fd70c06e850 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd70a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c15d050 0x7fd70c152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd70a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c15d050 0x7fd70c152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49766/0 (socket says 172.21.15.110:49766) 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd70a800640 1 -- 172.21.15.110:0/611574013 learned_addr learned my addr 172.21.15.110:0/611574013 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd709e00640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c06e850 0x7fd70c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd709400640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd70c15b330 0x7fd70c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd70a800640 1 -- 172.21.15.110:0/611574013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c06e850 msgr2=0x7fd70c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd70a800640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c06e850 0x7fd70c15c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd70a800640 1 -- 172.21.15.110:0/611574013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd70c15b330 msgr2=0x7fd70c1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.352+0000 7fd70a800640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd70c15b330 0x7fd70c1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd70a800640 1 -- 172.21.15.110:0/611574013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c15d4e0 con 0x7fd70c15d050 2024-09-10T10:01:45.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd70a800640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c15d050 0x7fd70c152680 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd7040af040 tx=0x7fd7040197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:45.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd702000640 1 -- 172.21.15.110:0/611574013 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7040b8670 con 0x7fd70c15d050 2024-09-10T10:01:45.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd702000640 1 -- 172.21.15.110:0/611574013 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7040be070 con 0x7fd70c15d050 2024-09-10T10:01:45.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd702000640 1 -- 172.21.15.110:0/611574013 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7040b8a50 con 0x7fd70c15d050 2024-09-10T10:01:45.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c15d7c0 con 0x7fd70c15d050 2024-09-10T10:01:45.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c152160 con 0x7fd70c15d050 2024-09-10T10:01:45.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c06c6a0 con 0x7fd70c15d050 2024-09-10T10:01:45.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd702000640 1 -- 172.21.15.110:0/611574013 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fd7040b8bf0 con 0x7fd70c15d050 2024-09-10T10:01:45.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd702000640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd6ec067c50 0x7fd6ec06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd702000640 1 -- 172.21.15.110:0/611574013 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7fd704120cb0 con 0x7fd70c15d050 2024-09-10T10:01:45.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.356+0000 7fd709e00640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd6ec067c50 0x7fd6ec06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.360+0000 7fd709e00640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd6ec067c50 0x7fd6ec06a110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd70c167080 tx=0x7fd6fc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:45.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.364+0000 7fd702000640 1 -- 172.21.15.110:0/611574013 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd7040c7020 con 0x7fd70c15d050 2024-09-10T10:01:45.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:45.556+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd70c0cccd0 con 0x7fd70c15d050 2024-09-10T10:01:46.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.176+0000 7fd702000640 1 -- 172.21.15.110:0/611574013 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7fd704016620 con 0x7fd70c15d050 2024-09-10T10:01:46.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd6ec067c50 msgr2=0x7fd6ec06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:46.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd6ec067c50 0x7fd6ec06a110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd70c167080 tx=0x7fd6fc009190 comp rx=0 tx=0).stop 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c15d050 msgr2=0x7fd70c152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c15d050 0x7fd70c152680 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd7040af040 tx=0x7fd7040197d0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 shutdown_connections 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd6ec067c50 0x7fd6ec06a110 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd70c15d050 0x7fd70c152680 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd70c15b330 0x7fd70c1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 --2- 172.21.15.110:0/611574013 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd70c06e850 0x7fd70c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 >> 172.21.15.110:0/611574013 conn(0x7fd70c062e40 msgr2=0x7fd70c0cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 shutdown_connections 2024-09-10T10:01:46.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.180+0000 7fd711000640 1 -- 172.21.15.110:0/611574013 wait complete. 2024-09-10T10:01:46.205 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:01:46.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff5600640 1 Processor -- start 2024-09-10T10:01:46.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff5600640 1 -- start start 2024-09-10T10:01:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff5600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1ff015c080 0x7f1ff006c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff5600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff00720e0 0x7f1ff00b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff5600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1ff006e290 0x7f1ff00b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff5600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff00b09c0 con 0x7f1ff006e290 2024-09-10T10:01:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff5600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff006cb70 con 0x7f1ff015c080 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff5600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff00b0c10 con 0x7f1ff00720e0 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff4c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1ff006e290 0x7f1ff00b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff4c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1ff006e290 0x7f1ff00b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49782/0 (socket says 172.21.15.110:49782) 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff4c00640 1 -- 172.21.15.110:0/3282606922 learned_addr learned my addr 172.21.15.110:0/3282606922 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1fef400640 1 --2- 172.21.15.110:0/3282606922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff00720e0 0x7f1ff00b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1fefe00640 1 --2- 172.21.15.110:0/3282606922 >> v2:172.21.15.145:3300/0 conn(0x7f1ff015c080 0x7f1ff006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff4c00640 1 -- 172.21.15.110:0/3282606922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff00720e0 msgr2=0x7f1ff00b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff4c00640 1 --2- 172.21.15.110:0/3282606922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff00720e0 0x7f1ff00b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff4c00640 1 -- 172.21.15.110:0/3282606922 >> v2:172.21.15.145:3300/0 conn(0x7f1ff015c080 msgr2=0x7f1ff006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff4c00640 1 --2- 172.21.15.110:0/3282606922 >> v2:172.21.15.145:3300/0 conn(0x7f1ff015c080 0x7f1ff006c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1ff4c00640 1 -- 172.21.15.110:0/3282606922 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff00b0ef0 con 0x7f1ff006e290 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1fef400640 1 --2- 172.21.15.110:0/3282606922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff00720e0 0x7f1ff00b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.392+0000 7f1fefe00640 1 --2- 172.21.15.110:0/3282606922 >> v2:172.21.15.145:3300/0 conn(0x7f1ff015c080 0x7f1ff006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:46.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff4c00640 1 --2- 172.21.15.110:0/3282606922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff006e290 0x7f1ff00b6e10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009e50 tx=0x7f1fe801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b84d1bb6bb8f72c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:46.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1feea00640 1 -- 172.21.15.110:0/3282606922 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80a56d0 con 0x7f1ff006e290 2024-09-10T10:01:46.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1feea00640 1 -- 172.21.15.110:0/3282606922 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe80a5870 con 0x7f1ff006e290 2024-09-10T10:01:46.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1feea00640 1 -- 172.21.15.110:0/3282606922 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80a5c40 con 0x7f1ff006e290 2024-09-10T10:01:46.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 -- 172.21.15.110:0/3282606922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff006e290 msgr2=0x7f1ff00b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:46.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 --2- 172.21.15.110:0/3282606922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff006e290 0x7f1ff00b6e10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009e50 tx=0x7f1fe801c960 comp rx=0 tx=0).stop 2024-09-10T10:01:46.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 -- 172.21.15.110:0/3282606922 shutdown_connections 2024-09-10T10:01:46.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 --2- 172.21.15.110:0/3282606922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff006e290 0x7f1ff00b6e10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 --2- 172.21.15.110:0/3282606922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff00720e0 0x7f1ff00b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 --2- 172.21.15.110:0/3282606922 >> v2:172.21.15.145:3300/0 conn(0x7f1ff015c080 0x7f1ff006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 -- 172.21.15.110:0/3282606922 >> 172.21.15.110:0/3282606922 conn(0x7f1ff006cd60 msgr2=0x7f1ff006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:46.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 -- 172.21.15.110:0/3282606922 shutdown_connections 2024-09-10T10:01:46.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 -- 172.21.15.110:0/3282606922 wait complete. 2024-09-10T10:01:46.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 Processor -- start 2024-09-10T10:01:46.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.396+0000 7f1ff5600640 1 -- start start 2024-09-10T10:01:46.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ff006e290 0x7f1ff00b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff00720e0 0x7f1ff0144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff015c080 0x7f1ff0149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff00b3ac0 con 0x7f1ff00720e0 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff015c670 con 0x7f1ff006e290 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff00b3c40 con 0x7f1ff015c080 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fef400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff00720e0 0x7f1ff0144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fef400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff00720e0 0x7f1ff0144c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49796/0 (socket says 172.21.15.110:49796) 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fefe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ff006e290 0x7f1ff00b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fef400640 1 -- 172.21.15.110:0/1343890466 learned_addr learned my addr 172.21.15.110:0/1343890466 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff4c00640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff015c080 0x7f1ff0149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fefe00640 1 -- 172.21.15.110:0/1343890466 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff015c080 msgr2=0x7f1ff0149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:46.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fefe00640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff015c080 0x7f1ff0149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fefe00640 1 -- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff00720e0 msgr2=0x7f1ff0144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:46.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fefe00640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff00720e0 0x7f1ff0144c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:46.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fefe00640 1 -- 172.21.15.110:0/1343890466 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff00b6fa0 con 0x7f1ff006e290 2024-09-10T10:01:46.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fef400640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff00720e0 0x7f1ff0144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:46.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fefe00640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ff006e290 0x7f1ff00b69c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000d5c0 tx=0x7f1fe000da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:46.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fe7e00640 1 -- 172.21.15.110:0/1343890466 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00125f0 con 0x7f1ff006e290 2024-09-10T10:01:46.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fe7e00640 1 -- 172.21.15.110:0/1343890466 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe0019070 con 0x7f1ff006e290 2024-09-10T10:01:46.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1fe7e00640 1 -- 172.21.15.110:0/1343890466 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0012990 con 0x7f1ff006e290 2024-09-10T10:01:46.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff0159890 con 0x7f1ff006e290 2024-09-10T10:01:46.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff0159e50 con 0x7f1ff006e290 2024-09-10T10:01:46.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.400+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff00b3ff0 con 0x7f1ff006e290 2024-09-10T10:01:46.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.404+0000 7f1fe7e00640 1 -- 172.21.15.110:0/1343890466 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f1fe0012b30 con 0x7f1ff006e290 2024-09-10T10:01:46.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.404+0000 7f1fe7e00640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1fd0067c50 0x7f1fd006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:46.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.404+0000 7f1fe7e00640 1 -- 172.21.15.110:0/1343890466 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7f1fe0086150 con 0x7f1ff006e290 2024-09-10T10:01:46.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.404+0000 7f1fef400640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1fd0067c50 0x7f1fd006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:46.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.404+0000 7f1fef400640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1fd0067c50 0x7f1fd006a110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00d27e0 tx=0x7f1fdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:46.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.408+0000 7f1fe7e00640 1 -- 172.21.15.110:0/1343890466 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1fe0023020 con 0x7f1ff006e290 2024-09-10T10:01:46.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:46.568+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ff014b570 con 0x7f1ff006e290 2024-09-10T10:01:47.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.332+0000 7f1fe7e00640 1 -- 172.21.15.110:0/1343890466 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7f1fe0056190 con 0x7f1ff006e290 2024-09-10T10:01:47.337 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:01:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1fd0067c50 msgr2=0x7f1fd006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1fd0067c50 0x7f1fd006a110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00d27e0 tx=0x7f1fdc008040 comp rx=0 tx=0).stop 2024-09-10T10:01:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ff006e290 msgr2=0x7f1ff00b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ff006e290 0x7f1ff00b69c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000d5c0 tx=0x7f1fe000da80 comp rx=0 tx=0).stop 2024-09-10T10:01:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 shutdown_connections 2024-09-10T10:01:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f1fd0067c50 0x7f1fd006a110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1ff015c080 0x7f1ff0149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ff00720e0 0x7f1ff0144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 --2- 172.21.15.110:0/1343890466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ff006e290 0x7f1ff00b69c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 >> 172.21.15.110:0/1343890466 conn(0x7f1ff006cd60 msgr2=0x7f1ff00724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 shutdown_connections 2024-09-10T10:01:47.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.336+0000 7f1ff5600640 1 -- 172.21.15.110:0/1343890466 wait complete. 2024-09-10T10:01:47.363 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:01:47.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.504+0000 7ff928400640 1 Processor -- start 2024-09-10T10:01:47.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.504+0000 7ff928400640 1 -- start start 2024-09-10T10:01:47.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.504+0000 7ff928400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff9200cd510 0x7ff9200cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.504+0000 7ff928400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff92006e090 0x7ff92006e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.504+0000 7ff928400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff920072150 con 0x7ff9200cfe30 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.504+0000 7ff928400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff92015d9a0 con 0x7ff92006e090 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.504+0000 7ff928400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff92006d2d0 con 0x7ff9200cd510 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff926400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff92006e090 0x7ff92006e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff926400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff92006e090 0x7ff92006e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49804/0 (socket says 172.21.15.110:49804) 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff926400640 1 -- 172.21.15.110:0/926894162 learned_addr learned my addr 172.21.15.110:0/926894162 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff925000640 1 --2- 172.21.15.110:0/926894162 >> v2:172.21.15.145:3300/0 conn(0x7ff9200cd510 0x7ff9200cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff926400640 1 -- 172.21.15.110:0/926894162 >> v2:172.21.15.145:3300/0 conn(0x7ff9200cd510 msgr2=0x7ff9200cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff926400640 1 --2- 172.21.15.110:0/926894162 >> v2:172.21.15.145:3300/0 conn(0x7ff9200cd510 0x7ff9200cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff926400640 1 -- 172.21.15.110:0/926894162 >> v1:172.21.15.145:6789/0 conn(0x7ff9200cfe30 legacy=0x7ff920071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:47.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff926400640 1 -- 172.21.15.110:0/926894162 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92006d5b0 con 0x7ff92006e090 2024-09-10T10:01:47.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff926400640 1 --2- 172.21.15.110:0/926894162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff92006e4e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff9200ce0f0 tx=0x7ff91801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c7ed90725f04f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff91fe00640 1 -- 172.21.15.110:0/926894162 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9180b2070 con 0x7ff92006e090 2024-09-10T10:01:47.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff91fe00640 1 -- 172.21.15.110:0/926894162 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9180a9070 con 0x7ff92006e090 2024-09-10T10:01:47.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.508+0000 7ff91fe00640 1 -- 172.21.15.110:0/926894162 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9180ad880 con 0x7ff92006e090 2024-09-10T10:01:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- 172.21.15.110:0/926894162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 msgr2=0x7ff92006e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 --2- 172.21.15.110:0/926894162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff92006e4e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff9200ce0f0 tx=0x7ff91801c450 comp rx=0 tx=0).stop 2024-09-10T10:01:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- 172.21.15.110:0/926894162 shutdown_connections 2024-09-10T10:01:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 --2- 172.21.15.110:0/926894162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff92006e4e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 --2- 172.21.15.110:0/926894162 >> v2:172.21.15.145:3300/0 conn(0x7ff9200cd510 0x7ff9200cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- 172.21.15.110:0/926894162 >> 172.21.15.110:0/926894162 conn(0x7ff92006bc50 msgr2=0x7ff92006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- 172.21.15.110:0/926894162 shutdown_connections 2024-09-10T10:01:47.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- 172.21.15.110:0/926894162 wait complete. 2024-09-10T10:01:47.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 Processor -- start 2024-09-10T10:01:47.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- start start 2024-09-10T10:01:47.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff920147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9200cd510 0x7ff9200b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9200cfe30 0x7ff9200b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff920169180 con 0x7ff92006e090 2024-09-10T10:01:47.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff92015d6c0 con 0x7ff9200cfe30 2024-09-10T10:01:47.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff928400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff920169300 con 0x7ff9200cd510 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff920147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff920147b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49812/0 (socket says 172.21.15.110:49812) 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925a00640 1 -- 172.21.15.110:0/350254587 learned_addr learned my addr 172.21.15.110:0/350254587 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925000640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9200cd510 0x7ff9200b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff926400640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9200cfe30 0x7ff9200b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925a00640 1 -- 172.21.15.110:0/350254587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9200cd510 msgr2=0x7ff9200b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925a00640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9200cd510 0x7ff9200b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925a00640 1 -- 172.21.15.110:0/350254587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9200cfe30 msgr2=0x7ff9200b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925a00640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9200cfe30 0x7ff9200b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.512+0000 7ff925a00640 1 -- 172.21.15.110:0/350254587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9200b20d0 con 0x7ff92006e090 2024-09-10T10:01:47.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff925a00640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff920147b20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff91400ba50 tx=0x7ff91400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff91d600640 1 -- 172.21.15.110:0/350254587 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914019070 con 0x7ff92006e090 2024-09-10T10:01:47.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff91d600640 1 -- 172.21.15.110:0/350254587 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91400cc20 con 0x7ff92006e090 2024-09-10T10:01:47.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff91d600640 1 -- 172.21.15.110:0/350254587 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914014680 con 0x7ff92006e090 2024-09-10T10:01:47.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9200b0250 con 0x7ff92006e090 2024-09-10T10:01:47.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9200b0810 con 0x7ff92006e090 2024-09-10T10:01:47.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92006e4e0 con 0x7ff92006e090 2024-09-10T10:01:47.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff91d600640 1 -- 172.21.15.110:0/350254587 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff91401c070 con 0x7ff92006e090 2024-09-10T10:01:47.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff91d600640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff900067c20 0x7ff90006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff91d600640 1 -- 172.21.15.110:0/350254587 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5357+0+0 (secure 0 0 0) 0x7ff914085e00 con 0x7ff92006e090 2024-09-10T10:01:47.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.516+0000 7ff925000640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff900067c20 0x7ff90006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.520+0000 7ff925000640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff900067c20 0x7ff90006a0e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff92014f9b0 tx=0x7ff90c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.520+0000 7ff91d600640 1 -- 172.21.15.110:0/350254587 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff914055f00 con 0x7ff92006e090 2024-09-10T10:01:47.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:47.680+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff9200b0020 con 0x7ff92006e090 2024-09-10T10:01:48.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.336+0000 7ff91d600640 1 -- 172.21.15.110:0/350254587 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7ff91405a200 con 0x7ff92006e090 2024-09-10T10:01:48.338 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:01:48.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff900067c20 msgr2=0x7ff90006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff900067c20 0x7ff90006a0e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff92014f9b0 tx=0x7ff90c008040 comp rx=0 tx=0).stop 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 msgr2=0x7ff920147b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff920147b20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff91400ba50 tx=0x7ff91400bf10 comp rx=0 tx=0).stop 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 shutdown_connections 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff900067c20 0x7ff90006a0e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9200cfe30 0x7ff9200b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9200cd510 0x7ff9200b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 --2- 172.21.15.110:0/350254587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92006e090 0x7ff920147b20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 >> 172.21.15.110:0/350254587 conn(0x7ff92006bc50 msgr2=0x7ff920071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 shutdown_connections 2024-09-10T10:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.340+0000 7ff928400640 1 -- 172.21.15.110:0/350254587 wait complete. 2024-09-10T10:01:48.366 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:01:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.500+0000 7fc888200640 1 Processor -- start 2024-09-10T10:01:48.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.500+0000 7fc888200640 1 -- start start 2024-09-10T10:01:48.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc888200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc8800ce4d0 0x7fc88006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc888200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8800b4960 con 0x7fc88015b330 2024-09-10T10:01:48.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc888200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8800b4c40 con 0x7fc88006e850 2024-09-10T10:01:48.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc888200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc8800b5570 con 0x7fc8800ce4d0 2024-09-10T10:01:48.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc885800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc88015b330 0x7fc88006e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42770/0 (socket says 172.21.15.110:42770) 2024-09-10T10:01:48.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc885800640 1 -- 172.21.15.110:0/516974652 learned_addr learned my addr 172.21.15.110:0/516974652 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:48.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc884e00640 1 --2- 172.21.15.110:0/516974652 >> v2:172.21.15.145:3300/0 conn(0x7fc8800ce4d0 0x7fc88006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc87fe00640 1 -- 172.21.15.110:0/516974652 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990977494 0 0) 0x7fc8800b4960 con 0x7fc88015b330 2024-09-10T10:01:48.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc87fe00640 1 -- 172.21.15.110:0/516974652 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc868005740 con 0x7fc88015b330 2024-09-10T10:01:48.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc87fe00640 1 -- 172.21.15.110:0/516974652 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2960394980 0 0) 0x7fc868005740 con 0x7fc88015b330 2024-09-10T10:01:48.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc884e00640 1 -- 172.21.15.110:0/516974652 >> v1:172.21.15.145:6789/0 conn(0x7fc88006e850 legacy=0x7fc8800b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc884e00640 1 -- 172.21.15.110:0/516974652 >> v1:172.21.15.110:6789/0 conn(0x7fc88015b330 legacy=0x7fc88006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.504+0000 7fc884e00640 1 -- 172.21.15.110:0/516974652 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8800b4f20 con 0x7fc8800ce4d0 2024-09-10T10:01:48.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc884e00640 1 --2- 172.21.15.110:0/516974652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8800ce4d0 0x7fc88006c6a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0097c0 tx=0x7fc86c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=151315bfdba5e0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:48.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc87fe00640 1 -- 172.21.15.110:0/516974652 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0b0070 con 0x7fc8800ce4d0 2024-09-10T10:01:48.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc87fe00640 1 -- 172.21.15.110:0/516974652 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc86c0ab400 con 0x7fc8800ce4d0 2024-09-10T10:01:48.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc87fe00640 1 -- 172.21.15.110:0/516974652 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874001680 con 0x7fc8800ce4d0 2024-09-10T10:01:48.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 -- 172.21.15.110:0/516974652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8800ce4d0 msgr2=0x7fc88006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 --2- 172.21.15.110:0/516974652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8800ce4d0 0x7fc88006c6a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0097c0 tx=0x7fc86c01c3c0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 -- 172.21.15.110:0/516974652 shutdown_connections 2024-09-10T10:01:48.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 --2- 172.21.15.110:0/516974652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8800ce4d0 0x7fc88006c6a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 -- 172.21.15.110:0/516974652 >> 172.21.15.110:0/516974652 conn(0x7fc880062e40 msgr2=0x7fc88006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:48.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 -- 172.21.15.110:0/516974652 shutdown_connections 2024-09-10T10:01:48.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 -- 172.21.15.110:0/516974652 wait complete. 2024-09-10T10:01:48.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 Processor -- start 2024-09-10T10:01:48.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.508+0000 7fc888200640 1 -- start start 2024-09-10T10:01:48.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88006e850 0x7fc88014e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88015b330 0x7fc88006d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc88014e9d0 0x7fc88006d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800b3b00 con 0x7fc88006e850 2024-09-10T10:01:48.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800b3980 con 0x7fc88015b330 2024-09-10T10:01:48.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc8800b3c80 con 0x7fc88014e9d0 2024-09-10T10:01:48.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88006e850 0x7fc88014e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88006e850 0x7fc88014e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36884/0 (socket says 172.21.15.110:36884) 2024-09-10T10:01:48.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 -- 172.21.15.110:0/2056201093 learned_addr learned my addr 172.21.15.110:0/2056201093 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:48.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 -- 172.21.15.110:0/2056201093 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc88014e9d0 msgr2=0x7fc88006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc88014e9d0 0x7fc88006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 -- 172.21.15.110:0/2056201093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88015b330 msgr2=0x7fc88006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88015b330 0x7fc88006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 -- 172.21.15.110:0/2056201093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88014d950 con 0x7fc88006e850 2024-09-10T10:01:48.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc885800640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88006e850 0x7fc88014e1e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c700 tx=0x7fc87400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:48.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc87d600640 1 -- 172.21.15.110:0/2056201093 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874005ae0 con 0x7fc88006e850 2024-09-10T10:01:48.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc87d600640 1 -- 172.21.15.110:0/2056201093 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc874005c80 con 0x7fc88006e850 2024-09-10T10:01:48.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc87d600640 1 -- 172.21.15.110:0/2056201093 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874007290 con 0x7fc88006e850 2024-09-10T10:01:48.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88014dc30 con 0x7fc88006e850 2024-09-10T10:01:48.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc880149250 con 0x7fc88006e850 2024-09-10T10:01:48.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88015d560 con 0x7fc88006e850 2024-09-10T10:01:48.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.512+0000 7fc87d600640 1 -- 172.21.15.110:0/2056201093 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fc874007430 con 0x7fc88006e850 2024-09-10T10:01:48.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.516+0000 7fc87d600640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc860067c20 0x7fc86006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.516+0000 7fc87d600640 1 -- 172.21.15.110:0/2056201093 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fc874085d50 con 0x7fc88006e850 2024-09-10T10:01:48.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.516+0000 7fc884e00640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc860067c20 0x7fc86006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.516+0000 7fc884e00640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc860067c20 0x7fc86006a0e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0ad000 tx=0x7fc86c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:48.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.520+0000 7fc87d600640 1 -- 172.21.15.110:0/2056201093 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc88015d560 con 0x7fc88006e850 2024-09-10T10:01:48.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.684+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc88014ef70 con 0x7fc88006e850 2024-09-10T10:01:48.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.684+0000 7fc87d600640 1 -- 172.21.15.110:0/2056201093 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fc874019300 con 0x7fc88006e850 2024-09-10T10:01:48.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc860067c20 msgr2=0x7fc86006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc860067c20 0x7fc86006a0e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0ad000 tx=0x7fc86c008040 comp rx=0 tx=0).stop 2024-09-10T10:01:48.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88006e850 msgr2=0x7fc88014e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88006e850 0x7fc88014e1e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c700 tx=0x7fc87400cbc0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 shutdown_connections 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc860067c20 0x7fc86006a0e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc88014e9d0 0x7fc88006d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88015b330 0x7fc88006d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 --2- 172.21.15.110:0/2056201093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88006e850 0x7fc88014e1e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 >> 172.21.15.110:0/2056201093 conn(0x7fc880062e40 msgr2=0x7fc880072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 shutdown_connections 2024-09-10T10:01:48.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.688+0000 7fc888200640 1 -- 172.21.15.110:0/2056201093 wait complete. 2024-09-10T10:01:48.711 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-10T10:01:48.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2b4a00640 1 Processor -- start 2024-09-10T10:01:48.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2b4a00640 1 -- start start 2024-09-10T10:01:48.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2b4a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa2b006bc50 0x7fa2b006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2b4a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2b00ae840 0x7fa2b00aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2b4a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b006c710 con 0x7fa2b015e690 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2b4a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b00af1d0 con 0x7fa2b00ae840 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2b4a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b00b4680 con 0x7fa2b006bc50 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2ae800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa2b00ae840 0x7fa2b00aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2ade00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa2b015e690 0x7fa2b006e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42784/0 (socket says 172.21.15.110:42784) 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2ade00640 1 -- 172.21.15.110:0/1523392655 learned_addr learned my addr 172.21.15.110:0/1523392655 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2ad400640 1 --2- 172.21.15.110:0/1523392655 >> v2:172.21.15.145:3300/0 conn(0x7fa2b006bc50 0x7fa2b006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2ae800640 1 -- 172.21.15.110:0/1523392655 >> v2:172.21.15.145:3300/0 conn(0x7fa2b006bc50 msgr2=0x7fa2b006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2ae800640 1 --2- 172.21.15.110:0/1523392655 >> v2:172.21.15.145:3300/0 conn(0x7fa2b006bc50 0x7fa2b006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2ae800640 1 -- 172.21.15.110:0/1523392655 >> v1:172.21.15.110:6789/0 conn(0x7fa2b015e690 legacy=0x7fa2b006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.844+0000 7fa2ae800640 1 -- 172.21.15.110:0/1523392655 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b00b4960 con 0x7fa2b00ae840 2024-09-10T10:01:48.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2ae800640 1 --2- 172.21.15.110:0/1523392655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b00aec90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0069820 tx=0x7fa2a801c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e15441b140022f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:48.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2aca00640 1 -- 172.21.15.110:0/1523392655 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80b2070 con 0x7fa2b00ae840 2024-09-10T10:01:48.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2aca00640 1 -- 172.21.15.110:0/1523392655 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a80a57d0 con 0x7fa2b00ae840 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2aca00640 1 -- 172.21.15.110:0/1523392655 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80a5ba0 con 0x7fa2b00ae840 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1523392655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 msgr2=0x7fa2b00aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1523392655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b00aec90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0069820 tx=0x7fa2a801c790 comp rx=0 tx=0).stop 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1523392655 shutdown_connections 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1523392655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b00aec90 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1523392655 >> v2:172.21.15.145:3300/0 conn(0x7fa2b006bc50 0x7fa2b006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1523392655 >> 172.21.15.110:0/1523392655 conn(0x7fa2b00d1530 msgr2=0x7fa2b00d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1523392655 shutdown_connections 2024-09-10T10:01:48.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.848+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1523392655 wait complete. 2024-09-10T10:01:48.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 Processor -- start 2024-09-10T10:01:48.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 -- start start 2024-09-10T10:01:48.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2b006bc50 0x7fa2b014aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2b015e690 0x7fa2b015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b015ebf0 con 0x7fa2b00ae840 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b015ea70 con 0x7fa2b006bc50 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b00b5c50 con 0x7fa2b015e690 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b014b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36904/0 (socket says 172.21.15.110:36904) 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 -- 172.21.15.110:0/1382992637 learned_addr learned my addr 172.21.15.110:0/1382992637 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ade00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2b006bc50 0x7fa2b014aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ae800640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2b015e690 0x7fa2b015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 -- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2b015e690 msgr2=0x7fa2b015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2b015e690 0x7fa2b015ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 -- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2b006bc50 msgr2=0x7fa2b014aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2b006bc50 0x7fa2b014aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 -- 172.21.15.110:0/1382992637 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b01597a0 con 0x7fa2b00ae840 2024-09-10T10:01:48.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ae800640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2b015e690 0x7fa2b015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:48.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ade00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2b006bc50 0x7fa2b014aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:48.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2ad400640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b014b0e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00d800 tx=0x7fa29c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:48.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2a6000640 1 -- 172.21.15.110:0/1382992637 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00b6c0 con 0x7fa2b00ae840 2024-09-10T10:01:48.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2a6000640 1 -- 172.21.15.110:0/1382992637 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29c019070 con 0x7fa2b00ae840 2024-09-10T10:01:48.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b0159a80 con 0x7fa2b00ae840 2024-09-10T10:01:48.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2a6000640 1 -- 172.21.15.110:0/1382992637 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00ba60 con 0x7fa2b00ae840 2024-09-10T10:01:48.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.852+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b015a040 con 0x7fa2b00ae840 2024-09-10T10:01:48.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.856+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b006c090 con 0x7fa2b00ae840 2024-09-10T10:01:48.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.856+0000 7fa2a6000640 1 -- 172.21.15.110:0/1382992637 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fa29c00bc00 con 0x7fa2b00ae840 2024-09-10T10:01:48.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.856+0000 7fa2a6000640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa290067c20 0x7fa29006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.856+0000 7fa2a6000640 1 -- 172.21.15.110:0/1382992637 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fa29c0863a0 con 0x7fa2b00ae840 2024-09-10T10:01:48.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.856+0000 7fa2ade00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa290067c20 0x7fa29006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.856+0000 7fa2ade00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa290067c20 0x7fa29006a0e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0000f70 tx=0x7fa2a0007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:48.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:48.860+0000 7fa2a6000640 1 -- 172.21.15.110:0/1382992637 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa29c056dd0 con 0x7fa2b00ae840 2024-09-10T10:01:48.945 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:01:48.948 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:01:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.076+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fa2b00b22c0 con 0x7fa2b00ae840 2024-09-10T10:01:49.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.336+0000 7fa2a6000640 1 -- 172.21.15.110:0/1382992637 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v188) ==== 161+0+0 (secure 0 0 0) 0x7fa29c0566d0 con 0x7fa2b00ae840 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa290067c20 msgr2=0x7fa29006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa290067c20 0x7fa29006a0e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0000f70 tx=0x7fa2a0007d30 comp rx=0 tx=0).stop 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 msgr2=0x7fa2b014b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b014b0e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00d800 tx=0x7fa29c00dcc0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 shutdown_connections 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa290067c20 0x7fa29006a0e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa2b015e690 0x7fa2b015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2b00ae840 0x7fa2b014b0e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 --2- 172.21.15.110:0/1382992637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2b006bc50 0x7fa2b014aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 >> 172.21.15.110:0/1382992637 conn(0x7fa2b00d1530 msgr2=0x7fa2b006e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 shutdown_connections 2024-09-10T10:01:49.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.340+0000 7fa2b4a00640 1 -- 172.21.15.110:0/1382992637 wait complete. 2024-09-10T10:01:49.363 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-10T10:01:49.363 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:01:49.363 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:49.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57ac600640 1 Processor -- start 2024-09-10T10:01:49.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57ac600640 1 -- start start 2024-09-10T10:01:49.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57ac600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cfe30 0x7f57a4071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57ac600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f57a40cd510 0x7f57a40cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57ac600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f57a406e090 0x7f57a406e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57ac600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f57a40cdeb0 con 0x7f57a40cd510 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57ac600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f57a415d9a0 con 0x7f57a406e090 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57ac600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f57a406ea80 con 0x7f57a40cfe30 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57a9200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f57a40cd510 0x7f57a40cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.496+0000 7f57a9200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f57a40cd510 0x7f57a40cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36914/0 (socket says 172.21.15.110:36914) 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a9200640 1 -- 172.21.15.110:0/3612974290 learned_addr learned my addr 172.21.15.110:0/3612974290 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a9c00640 1 --2- 172.21.15.110:0/3612974290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cfe30 0x7f57a4071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57aa600640 1 --2- 172.21.15.110:0/3612974290 >> v2:172.21.15.145:3300/0 conn(0x7f57a406e090 0x7f57a406e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a9200640 1 -- 172.21.15.110:0/3612974290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cfe30 msgr2=0x7f57a4071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a9200640 1 --2- 172.21.15.110:0/3612974290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cfe30 0x7f57a4071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a9200640 1 -- 172.21.15.110:0/3612974290 >> v2:172.21.15.145:3300/0 conn(0x7f57a406e090 msgr2=0x7f57a406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a9200640 1 --2- 172.21.15.110:0/3612974290 >> v2:172.21.15.145:3300/0 conn(0x7f57a406e090 0x7f57a406e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a9200640 1 -- 172.21.15.110:0/3612974290 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a406d2d0 con 0x7f57a40cd510 2024-09-10T10:01:49.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a9200640 1 --2- 172.21.15.110:0/3612974290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a40cd510 0x7f57a40cd970 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f57a415a560 tx=0x7f579001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d652c4a6ce91b99b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a3e00640 1 -- 172.21.15.110:0/3612974290 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900b2070 con 0x7f57a40cd510 2024-09-10T10:01:49.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a3e00640 1 -- 172.21.15.110:0/3612974290 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57900a9070 con 0x7f57a40cd510 2024-09-10T10:01:49.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.500+0000 7f57a3e00640 1 -- 172.21.15.110:0/3612974290 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900ad880 con 0x7f57a40cd510 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- 172.21.15.110:0/3612974290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a40cd510 msgr2=0x7f57a40cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 --2- 172.21.15.110:0/3612974290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a40cd510 0x7f57a40cd970 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f57a415a560 tx=0x7f579001c490 comp rx=0 tx=0).stop 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- 172.21.15.110:0/3612974290 shutdown_connections 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 --2- 172.21.15.110:0/3612974290 >> v2:172.21.15.145:3300/0 conn(0x7f57a406e090 0x7f57a406e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 --2- 172.21.15.110:0/3612974290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a40cd510 0x7f57a40cd970 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 --2- 172.21.15.110:0/3612974290 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cfe30 0x7f57a4071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- 172.21.15.110:0/3612974290 >> 172.21.15.110:0/3612974290 conn(0x7f57a406bc50 msgr2=0x7f57a406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- 172.21.15.110:0/3612974290 shutdown_connections 2024-09-10T10:01:49.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- 172.21.15.110:0/3612974290 wait complete. 2024-09-10T10:01:49.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 Processor -- start 2024-09-10T10:01:49.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- start start 2024-09-10T10:01:49.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a406e090 0x7f57a415e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cd510 0x7f57a40b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57a40cfe30 0x7f57a4157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f57a415d430 con 0x7f57a406e090 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4072190 con 0x7f57a40cfe30 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57ac600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f57a415d5b0 con 0x7f57a40cd510 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a406e090 0x7f57a415e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a406e090 0x7f57a415e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36922/0 (socket says 172.21.15.110:36922) 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 -- 172.21.15.110:0/2552465054 learned_addr learned my addr 172.21.15.110:0/2552465054 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57aa600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57a40cfe30 0x7f57a4157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9200640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cd510 0x7f57a40b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 -- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cd510 msgr2=0x7f57a40b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cd510 0x7f57a40b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 -- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57a40cfe30 msgr2=0x7f57a4157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57a40cfe30 0x7f57a4157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 -- 172.21.15.110:0/2552465054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a40b3700 con 0x7f57a406e090 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57aa600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57a40cfe30 0x7f57a4157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.504+0000 7f57a9c00640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a406e090 0x7f57a415e8e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5798007e60 tx=0x7f579800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57a1600640 1 -- 172.21.15.110:0/2552465054 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798014620 con 0x7f57a406e090 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57a1600640 1 -- 172.21.15.110:0/2552465054 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5798019070 con 0x7f57a406e090 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57a1600640 1 -- 172.21.15.110:0/2552465054 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798014a20 con 0x7f57a406e090 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a40b39a0 con 0x7f57a406e090 2024-09-10T10:01:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a40b3f60 con 0x7f57a406e090 2024-09-10T10:01:49.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a40cd970 con 0x7f57a406e090 2024-09-10T10:01:49.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57a1600640 1 -- 172.21.15.110:0/2552465054 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f5798014bc0 con 0x7f57a406e090 2024-09-10T10:01:49.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57a1600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5784067c20 0x7f578406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57a1600640 1 -- 172.21.15.110:0/2552465054 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f5798086670 con 0x7f57a406e090 2024-09-10T10:01:49.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57a9200640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5784067c20 0x7f578406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.508+0000 7f57a9200640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5784067c20 0x7f578406a0e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57900af000 tx=0x7f5790008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.512+0000 7f57a1600640 1 -- 172.21.15.110:0/2552465054 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f57980570f0 con 0x7f57a406e090 2024-09-10T10:01:49.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.668+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57a406c240 con 0x7f57a406e090 2024-09-10T10:01:49.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57a1600640 1 -- 172.21.15.110:0/2552465054 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+13070 (secure 0 0 0) 0x7f57980569f0 con 0x7f57a406e090 2024-09-10T10:01:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:49.674 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":61,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:48.332826+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6837/1333637048":"2024-09-11T10:01:45.175833+0000","172.21.15.110:6835/1333637048":"2024-09-11T10:01:45.175833+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:49.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5784067c20 msgr2=0x7f578406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5784067c20 0x7f578406a0e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57900af000 tx=0x7f5790008040 comp rx=0 tx=0).stop 2024-09-10T10:01:49.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a406e090 msgr2=0x7f57a415e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a406e090 0x7f57a415e8e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5798007e60 tx=0x7f579800e4b0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 shutdown_connections 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5784067c20 0x7f578406a0e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57a40cfe30 0x7f57a4157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57a40cd510 0x7f57a40b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 --2- 172.21.15.110:0/2552465054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57a406e090 0x7f57a415e8e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 >> 172.21.15.110:0/2552465054 conn(0x7f57a406bc50 msgr2=0x7f57a4151850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 shutdown_connections 2024-09-10T10:01:49.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.672+0000 7f57ac600640 1 -- 172.21.15.110:0/2552465054 wait complete. 2024-09-10T10:01:49.697 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:01:49.697 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:01:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.832+0000 7f723bc00640 1 Processor -- start 2024-09-10T10:01:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.832+0000 7f723bc00640 1 -- start start 2024-09-10T10:01:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f723bc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7234145f30 0x7f72340b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f723bc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f72340cfd90 0x7f72340b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f723bc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72340b37c0 con 0x7f72340b29e0 2024-09-10T10:01:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f723bc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f72340b2e50 con 0x7f72340cfd90 2024-09-10T10:01:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f723bc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f72340d0170 con 0x7f7234145f30 2024-09-10T10:01:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f72340cfd90 0x7f72340b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f72340cfd90 0x7f72340b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36932/0 (socket says 172.21.15.110:36932) 2024-09-10T10:01:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239c00640 1 -- 172.21.15.110:0/3213070509 learned_addr learned my addr 172.21.15.110:0/3213070509 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239200640 1 --2- 172.21.15.110:0/3213070509 >> v2:172.21.15.145:3300/0 conn(0x7f7234145f30 0x7f72340b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239c00640 1 -- 172.21.15.110:0/3213070509 >> v2:172.21.15.145:3300/0 conn(0x7f7234145f30 msgr2=0x7f72340b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239c00640 1 --2- 172.21.15.110:0/3213070509 >> v2:172.21.15.145:3300/0 conn(0x7f7234145f30 0x7f72340b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239c00640 1 -- 172.21.15.110:0/3213070509 >> v1:172.21.15.145:6789/0 conn(0x7f72340b29e0 legacy=0x7f72340cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239c00640 1 -- 172.21.15.110:0/3213070509 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340b3aa0 con 0x7f72340cfd90 2024-09-10T10:01:49.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.836+0000 7f7239c00640 1 --2- 172.21.15.110:0/3213070509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f72340b4580 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f72340b3000 tx=0x7f722c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa94a0dbdd49cfc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f7233400640 1 -- 172.21.15.110:0/3213070509 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c0b0070 con 0x7f72340cfd90 2024-09-10T10:01:49.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f7233400640 1 -- 172.21.15.110:0/3213070509 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722c0ab600 con 0x7f72340cfd90 2024-09-10T10:01:49.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f7233400640 1 -- 172.21.15.110:0/3213070509 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c0ab9d0 con 0x7f72340cfd90 2024-09-10T10:01:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- 172.21.15.110:0/3213070509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 msgr2=0x7f72340b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 --2- 172.21.15.110:0/3213070509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f72340b4580 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f72340b3000 tx=0x7f722c01c520 comp rx=0 tx=0).stop 2024-09-10T10:01:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- 172.21.15.110:0/3213070509 shutdown_connections 2024-09-10T10:01:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 --2- 172.21.15.110:0/3213070509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f72340b4580 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 --2- 172.21.15.110:0/3213070509 >> v2:172.21.15.145:3300/0 conn(0x7f7234145f30 0x7f72340b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- 172.21.15.110:0/3213070509 >> 172.21.15.110:0/3213070509 conn(0x7f7234062e40 msgr2=0x7f723406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- 172.21.15.110:0/3213070509 shutdown_connections 2024-09-10T10:01:49.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- 172.21.15.110:0/3213070509 wait complete. 2024-09-10T10:01:49.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 Processor -- start 2024-09-10T10:01:49.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- start start 2024-09-10T10:01:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f72340b29e0 0x7f72341409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f723415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7234145f30 0x7f7234144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f72340b56a0 con 0x7f72340cfd90 2024-09-10T10:01:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f72340b5520 con 0x7f7234145f30 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f723bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f72340b5820 con 0x7f72340b29e0 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f7233e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f723415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f7233e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f723415be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36942/0 (socket says 172.21.15.110:36942) 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f7233e00640 1 -- 172.21.15.110:0/4126365023 learned_addr learned my addr 172.21.15.110:0/4126365023 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f7239200640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f72340b29e0 0x7f72341409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.840+0000 7f7239c00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7234145f30 0x7f7234144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7233e00640 1 -- 172.21.15.110:0/4126365023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f72340b29e0 msgr2=0x7f72341409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7233e00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f72340b29e0 0x7f72341409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7233e00640 1 -- 172.21.15.110:0/4126365023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7234145f30 msgr2=0x7f7234144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7233e00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7234145f30 0x7f7234144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7233e00640 1 -- 172.21.15.110:0/4126365023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72341396c0 con 0x7f72340cfd90 2024-09-10T10:01:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7233e00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f723415be90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f722000c390 tx=0x7f722000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7230c00640 1 -- 172.21.15.110:0/4126365023 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220017070 con 0x7f72340cfd90 2024-09-10T10:01:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7230c00640 1 -- 172.21.15.110:0/4126365023 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72200052f0 con 0x7f72340cfd90 2024-09-10T10:01:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7230c00640 1 -- 172.21.15.110:0/4126365023 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220003840 con 0x7f72340cfd90 2024-09-10T10:01:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72341399a0 con 0x7f72340cfd90 2024-09-10T10:01:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723413a240 con 0x7f72340cfd90 2024-09-10T10:01:49.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72340b4580 con 0x7f72340cfd90 2024-09-10T10:01:49.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7230c00640 1 -- 172.21.15.110:0/4126365023 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f722001a070 con 0x7f72340cfd90 2024-09-10T10:01:49.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7230c00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7214067c20 0x7f721406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:49.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7230c00640 1 -- 172.21.15.110:0/4126365023 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f7220085120 con 0x7f72340cfd90 2024-09-10T10:01:49.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.844+0000 7f7239200640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7214067c20 0x7f721406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:49.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.848+0000 7f7239200640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7214067c20 0x7f721406a0e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f72340b1ce0 tx=0x7f722400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:49.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:49.848+0000 7f7230c00640 1 -- 172.21.15.110:0/4126365023 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7220055550 con 0x7f72340cfd90 2024-09-10T10:01:50.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.008+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72340cdd30 con 0x7f72340cfd90 2024-09-10T10:01:50.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f7230c00640 1 -- 172.21.15.110:0/4126365023 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7f7220059850 con 0x7f72340cfd90 2024-09-10T10:01:50.014 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7214067c20 msgr2=0x7f721406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7214067c20 0x7f721406a0e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f72340b1ce0 tx=0x7f722400b040 comp rx=0 tx=0).stop 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 msgr2=0x7f723415be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f723415be90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f722000c390 tx=0x7f722000c850 comp rx=0 tx=0).stop 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 shutdown_connections 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f7214067c20 0x7f721406a0e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7234145f30 0x7f7234144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72340cfd90 0x7f723415be90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 --2- 172.21.15.110:0/4126365023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f72340b29e0 0x7f72341409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 >> 172.21.15.110:0/4126365023 conn(0x7f7234062e40 msgr2=0x7f72340cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:50.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 shutdown_connections 2024-09-10T10:01:50.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.012+0000 7f723bc00640 1 -- 172.21.15.110:0/4126365023 wait complete. 2024-09-10T10:01:50.034 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-10T10:01:50.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.164+0000 7fa69d600640 1 Processor -- start 2024-09-10T10:01:50.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.164+0000 7fa69d600640 1 -- start start 2024-09-10T10:01:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa69d600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa6980cd510 0x7fa6980cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa69d600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa69806e090 0x7fa69806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa69d600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa698072150 con 0x7fa6980cfe30 2024-09-10T10:01:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa69d600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa69815d9a0 con 0x7fa6980cd510 2024-09-10T10:01:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa69d600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa69806d2d0 con 0x7fa69806e090 2024-09-10T10:01:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa696000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa6980cd510 0x7fa6980cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa696a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa6980cfe30 0x7fa698071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42798/0 (socket says 172.21.15.110:42798) 2024-09-10T10:01:50.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa696a00640 1 -- 172.21.15.110:0/992919793 learned_addr learned my addr 172.21.15.110:0/992919793 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa697400640 1 --2- 172.21.15.110:0/992919793 >> v2:172.21.15.145:3300/0 conn(0x7fa69806e090 0x7fa69806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa696000640 1 -- 172.21.15.110:0/992919793 >> v2:172.21.15.145:3300/0 conn(0x7fa69806e090 msgr2=0x7fa69806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa696000640 1 --2- 172.21.15.110:0/992919793 >> v2:172.21.15.145:3300/0 conn(0x7fa69806e090 0x7fa69806e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa696000640 1 -- 172.21.15.110:0/992919793 >> v1:172.21.15.110:6789/0 conn(0x7fa6980cfe30 legacy=0x7fa698071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa696000640 1 -- 172.21.15.110:0/992919793 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69806d5b0 con 0x7fa6980cd510 2024-09-10T10:01:50.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.168+0000 7fa695600640 1 -- 172.21.15.110:0/992919793 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2674151963 0 0) 0x7fa698072150 con 0x7fa6980cfe30 2024-09-10T10:01:50.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa696000640 1 --2- 172.21.15.110:0/992919793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cd510 0x7fa6980cd9a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa69806cb00 tx=0x7fa68401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcfe1be2a9e8a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:50.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa695600640 1 -- 172.21.15.110:0/992919793 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6840b0070 con 0x7fa6980cd510 2024-09-10T10:01:50.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa695600640 1 -- 172.21.15.110:0/992919793 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6840ab600 con 0x7fa6980cd510 2024-09-10T10:01:50.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa695600640 1 -- 172.21.15.110:0/992919793 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6840ab9d0 con 0x7fa6980cd510 2024-09-10T10:01:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 -- 172.21.15.110:0/992919793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cd510 msgr2=0x7fa6980cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 --2- 172.21.15.110:0/992919793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cd510 0x7fa6980cd9a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa69806cb00 tx=0x7fa68401c3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 -- 172.21.15.110:0/992919793 shutdown_connections 2024-09-10T10:01:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 --2- 172.21.15.110:0/992919793 >> v2:172.21.15.145:3300/0 conn(0x7fa69806e090 0x7fa69806e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 --2- 172.21.15.110:0/992919793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cd510 0x7fa6980cd9a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 -- 172.21.15.110:0/992919793 >> 172.21.15.110:0/992919793 conn(0x7fa69806bc50 msgr2=0x7fa69806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 -- 172.21.15.110:0/992919793 shutdown_connections 2024-09-10T10:01:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 -- 172.21.15.110:0/992919793 wait complete. 2024-09-10T10:01:50.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 Processor -- start 2024-09-10T10:01:50.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.172+0000 7fa69d600640 1 -- start start 2024-09-10T10:01:50.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa69806e090 0x7fa6980aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6980cd510 0x7fa698152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cfe30 0x7fa69814e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa698169180 con 0x7fa6980cfe30 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa69815d680 con 0x7fa6980cd510 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa698169300 con 0x7fa69806e090 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa697400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cfe30 0x7fa69814e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa69806e090 0x7fa6980aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6980cd510 0x7fa698152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa69806e090 0x7fa6980aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:47156/0 (socket says 172.21.15.110:47156) 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa697400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cfe30 0x7fa69814e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36958/0 (socket says 172.21.15.110:36958) 2024-09-10T10:01:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696a00640 1 -- 172.21.15.110:0/1796960062 learned_addr learned my addr 172.21.15.110:0/1796960062 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696000640 1 -- 172.21.15.110:0/1796960062 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa69806e090 msgr2=0x7fa6980aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696000640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa69806e090 0x7fa6980aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696000640 1 -- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cfe30 msgr2=0x7fa69814e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696000640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cfe30 0x7fa69814e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696000640 1 -- 172.21.15.110:0/1796960062 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6981546b0 con 0x7fa6980cd510 2024-09-10T10:01:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa697400640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cfe30 0x7fa69814e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa696000640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6980cd510 0x7fa698152e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa6840ab000 tx=0x7fa684019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa68ea00640 1 -- 172.21.15.110:0/1796960062 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6840b0040 con 0x7fa6980cd510 2024-09-10T10:01:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa68ea00640 1 -- 172.21.15.110:0/1796960062 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6840ab400 con 0x7fa6980cd510 2024-09-10T10:01:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa698154950 con 0x7fa6980cd510 2024-09-10T10:01:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa68ea00640 1 -- 172.21.15.110:0/1796960062 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6840b4630 con 0x7fa6980cd510 2024-09-10T10:01:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69814eeb0 con 0x7fa6980cd510 2024-09-10T10:01:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.176+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69806e4a0 con 0x7fa6980cd510 2024-09-10T10:01:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.180+0000 7fa68ea00640 1 -- 172.21.15.110:0/1796960062 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fa6840bb020 con 0x7fa6980cd510 2024-09-10T10:01:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.180+0000 7fa68ea00640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa678067c20 0x7fa67806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.180+0000 7fa68ea00640 1 -- 172.21.15.110:0/1796960062 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fa68411f220 con 0x7fa6980cd510 2024-09-10T10:01:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.180+0000 7fa696a00640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa678067c20 0x7fa67806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.180+0000 7fa696a00640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa678067c20 0x7fa67806a0e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa6880012f0 tx=0x7fa688001710 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.184+0000 7fa68ea00640 1 -- 172.21.15.110:0/1796960062 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa6840c2020 con 0x7fa6980cd510 2024-09-10T10:01:50.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:50.396+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7fa698075560 con 0x7fa6980cd510 2024-09-10T10:01:51.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.348+0000 7fa68ea00640 1 -- 172.21.15.110:0/1796960062 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v190) ==== 134+0+0 (secure 0 0 0) 0x7fa684016070 con 0x7fa6980cd510 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa678067c20 msgr2=0x7fa67806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa678067c20 0x7fa67806a0e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa6880012f0 tx=0x7fa688001710 comp rx=0 tx=0).stop 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6980cd510 msgr2=0x7fa698152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6980cd510 0x7fa698152e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa6840ab000 tx=0x7fa684019190 comp rx=0 tx=0).stop 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 shutdown_connections 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa678067c20 0x7fa67806a0e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6980cfe30 0x7fa69814e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6980cd510 0x7fa698152e60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 --2- 172.21.15.110:0/1796960062 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa69806e090 0x7fa6980aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 >> 172.21.15.110:0/1796960062 conn(0x7fa69806bc50 msgr2=0x7fa69815f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 shutdown_connections 2024-09-10T10:01:51.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.352+0000 7fa69d600640 1 -- 172.21.15.110:0/1796960062 wait complete. 2024-09-10T10:01:51.379 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:01:51.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb04d800640 1 Processor -- start 2024-09-10T10:01:51.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb04d800640 1 -- start start 2024-09-10T10:01:51.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb04d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 0x7fb04806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb04d800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0480b4960 con 0x7fb04806e850 2024-09-10T10:01:51.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb04d800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0480b4c40 con 0x7fb0480ce4d0 2024-09-10T10:01:51.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb04d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0480b5570 con 0x7fb04815b330 2024-09-10T10:01:51.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb047400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb04806e850 0x7fb0480b0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42814/0 (socket says 172.21.15.110:42814) 2024-09-10T10:01:51.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb047400640 1 -- 172.21.15.110:0/2150545971 learned_addr learned my addr 172.21.15.110:0/2150545971 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:51.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046a00640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 0x7fb04806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463523174 0 0) 0x7fb0480b4960 con 0x7fb04806e850 2024-09-10T10:01:51.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb02c005740 con 0x7fb04806e850 2024-09-10T10:01:51.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2454490330 0 0) 0x7fb02c005740 con 0x7fb04806e850 2024-09-10T10:01:51.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 msgr2=0x7fb04806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 0x7fb04806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 >> v1:172.21.15.145:6789/0 conn(0x7fb0480ce4d0 legacy=0x7fb04806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0480b4f20 con 0x7fb04806e850 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fb040001a00 con 0x7fb04806e850 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 >> v1:172.21.15.110:6789/0 conn(0x7fb04806e850 legacy=0x7fb0480b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c005c00 0x7fb02c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 0x7fb02c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c006f60 0x7fb02c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0480b5570 con 0x7fb02c005c00 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00dc90 con 0x7fb02c006f60 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c00de90 con 0x7fb04815b330 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046a00640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c005c00 0x7fb02c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb047400640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c006f60 0x7fb02c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb03fe00640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 0x7fb02c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046a00640 1 -- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 msgr2=0x7fb02c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046a00640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 0x7fb02c006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046a00640 1 -- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c006f60 msgr2=0x7fb02c0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:51.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046a00640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c006f60 0x7fb02c0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb046a00640 1 -- 172.21.15.110:0/2150545971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c005740 con 0x7fb02c005c00 2024-09-10T10:01:51.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.520+0000 7fb03fe00640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb04815b330 0x7fb02c006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:51.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb046a00640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c005c00 0x7fb02c006020 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb038009ab0 tx=0x7fb03801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396bd87fc8f6d333 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380b2070 con 0x7fb02c005c00 2024-09-10T10:01:51.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0380a5530 con 0x7fb02c005c00 2024-09-10T10:01:51.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb046000640 1 -- 172.21.15.110:0/2150545971 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380096b0 con 0x7fb02c005c00 2024-09-10T10:01:51.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb04d800640 1 -- 172.21.15.110:0/2150545971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c005c00 msgr2=0x7fb02c006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb04d800640 1 --2- 172.21.15.110:0/2150545971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c005c00 0x7fb02c006020 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb038009ab0 tx=0x7fb03801ca50 comp rx=0 tx=0).stop 2024-09-10T10:01:51.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb047400640 1 -- 172.21.15.110:0/2150545971 reap_dead start 2024-09-10T10:01:51.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb04d800640 1 -- 172.21.15.110:0/2150545971 shutdown_connections 2024-09-10T10:01:51.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb04d800640 1 -- 172.21.15.110:0/2150545971 >> 172.21.15.110:0/2150545971 conn(0x7fb048062e40 msgr2=0x7fb04806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb04d800640 1 -- 172.21.15.110:0/2150545971 shutdown_connections 2024-09-10T10:01:51.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.524+0000 7fb04d800640 1 -- 172.21.15.110:0/2150545971 wait complete. 2024-09-10T10:01:51.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 Processor -- start 2024-09-10T10:01:51.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 -- start start 2024-09-10T10:01:51.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c005c00 0x7fb0481538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb048072590 0x7fb0481e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0481e92f0 0x7fb048157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0480b36b0 con 0x7fb0481e92f0 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb04806cab0 con 0x7fb02c005c00 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0480b3830 con 0x7fb048072590 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0481e92f0 0x7fb048157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0481e92f0 0x7fb048157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36970/0 (socket says 172.21.15.110:36970) 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 -- 172.21.15.110:0/98504257 learned_addr learned my addr 172.21.15.110:0/98504257 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb046a00640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c005c00 0x7fb0481538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb03fe00640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb048072590 0x7fb0481e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 -- 172.21.15.110:0/98504257 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb048072590 msgr2=0x7fb0481e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb048072590 0x7fb0481e8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 -- 172.21.15.110:0/98504257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c005c00 msgr2=0x7fb0481538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c005c00 0x7fb0481538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 -- 172.21.15.110:0/98504257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb048152850 con 0x7fb0481e92f0 2024-09-10T10:01:51.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb047400640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0481e92f0 0x7fb048157900 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb040000f90 tx=0x7fb0400079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb03ea00640 1 -- 172.21.15.110:0/98504257 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04000aaa0 con 0x7fb0481e92f0 2024-09-10T10:01:51.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb03ea00640 1 -- 172.21.15.110:0/98504257 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb040019070 con 0x7fb0481e92f0 2024-09-10T10:01:51.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb048152b30 con 0x7fb0481e92f0 2024-09-10T10:01:51.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb03ea00640 1 -- 172.21.15.110:0/98504257 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04000ae60 con 0x7fb0481e92f0 2024-09-10T10:01:51.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.528+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0481530f0 con 0x7fb0481e92f0 2024-09-10T10:01:51.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.532+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04806e140 con 0x7fb0481e92f0 2024-09-10T10:01:51.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.532+0000 7fb03ea00640 1 -- 172.21.15.110:0/98504257 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb04001e460 con 0x7fb0481e92f0 2024-09-10T10:01:51.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.532+0000 7fb03ea00640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.532+0000 7fb03ea00640 1 -- 172.21.15.110:0/98504257 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fb0400862b0 con 0x7fb0481e92f0 2024-09-10T10:01:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.532+0000 7fb046a00640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.532+0000 7fb046a00640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb0380af000 tx=0x7fb0380046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:51.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.536+0000 7fb03ea00640 1 -- 172.21.15.110:0/98504257 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb040056ca0 con 0x7fb0481e92f0 2024-09-10T10:01:51.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.696+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb0480cfac0 con 0x7fb0481e92f0 2024-09-10T10:01:51.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb03ea00640 1 -- 172.21.15.110:0/98504257 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fb0400565a0 con 0x7fb0481e92f0 2024-09-10T10:01:51.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 msgr2=0x7fb02406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb0380af000 tx=0x7fb0380046d0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0481e92f0 msgr2=0x7fb048157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:51.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0481e92f0 0x7fb048157900 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb040000f90 tx=0x7fb0400079e0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 shutdown_connections 2024-09-10T10:01:51.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb024067c20 0x7fb02406a0e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0481e92f0 0x7fb048157900 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb048072590 0x7fb0481e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 --2- 172.21.15.110:0/98504257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb02c005c00 0x7fb0481538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:51.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 >> 172.21.15.110:0/98504257 conn(0x7fb048062e40 msgr2=0x7fb04806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:51.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 shutdown_connections 2024-09-10T10:01:51.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:51.700+0000 7fb04d800640 1 -- 172.21.15.110:0/98504257 wait complete. 2024-09-10T10:01:51.724 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:01:51.724 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:01:51.819 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:01:51.905 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:01:51.905 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:51.905 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:51.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:01:51.914 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:51.914 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:51.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:01:51.923 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:01:51.923 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:51.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:01:51.965 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:01:51.965 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:01:51.974 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:01:51.974 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:01:51.974 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:01:52.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f6485e00640 1 Processor -- start 2024-09-10T10:01:52.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f6485e00640 1 -- start start 2024-09-10T10:01:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f6485e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f648006eb30 0x7f64800d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f6485e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64800706a0 con 0x7f6480163000 2024-09-10T10:01:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f6485e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64800d1370 con 0x7f6480073290 2024-09-10T10:01:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f6485e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f648015ee90 con 0x7f648006eb30 2024-09-10T10:01:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f647ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f648006eb30 0x7f64800d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f647e000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6480163000 0x7f6480071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42826/0 (socket says 172.21.15.110:42826) 2024-09-10T10:01:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f647ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f648006eb30 0x7f64800d0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36984/0 (socket says 172.21.15.110:36984) 2024-09-10T10:01:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f647e000640 1 -- 172.21.15.110:0/2693595326 learned_addr learned my addr 172.21.15.110:0/2693595326 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:52.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f647ea00640 1 -- 172.21.15.110:0/2693595326 >> v1:172.21.15.145:6789/0 conn(0x7f6480073290 legacy=0x7f648006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:52.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f647ea00640 1 -- 172.21.15.110:0/2693595326 >> v1:172.21.15.110:6789/0 conn(0x7f6480163000 legacy=0x7f6480071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.140+0000 7f647ea00640 1 -- 172.21.15.110:0/2693595326 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64800d1650 con 0x7f648006eb30 2024-09-10T10:01:52.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f647ea00640 1 --2- 172.21.15.110:0/2693595326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f648006eb30 0x7f64800d0cb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6470009d90 tx=0x7f647001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7885fb4be6544bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:52.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f647d600640 1 -- 172.21.15.110:0/2693595326 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700b2070 con 0x7f648006eb30 2024-09-10T10:01:52.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f647d600640 1 -- 172.21.15.110:0/2693595326 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64700a5760 con 0x7f648006eb30 2024-09-10T10:01:52.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f647d600640 1 -- 172.21.15.110:0/2693595326 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700a5b30 con 0x7f648006eb30 2024-09-10T10:01:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f6485e00640 1 -- 172.21.15.110:0/2693595326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f648006eb30 msgr2=0x7f64800d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f6485e00640 1 --2- 172.21.15.110:0/2693595326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f648006eb30 0x7f64800d0cb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6470009d90 tx=0x7f647001c8a0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f6485e00640 1 -- 172.21.15.110:0/2693595326 shutdown_connections 2024-09-10T10:01:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f6485e00640 1 --2- 172.21.15.110:0/2693595326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f648006eb30 0x7f64800d0cb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f6485e00640 1 -- 172.21.15.110:0/2693595326 >> 172.21.15.110:0/2693595326 conn(0x7f64800d25c0 msgr2=0x7f64800d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f6485e00640 1 -- 172.21.15.110:0/2693595326 shutdown_connections 2024-09-10T10:01:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.144+0000 7f6485e00640 1 -- 172.21.15.110:0/2693595326 wait complete. 2024-09-10T10:01:52.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 Processor -- start 2024-09-10T10:01:52.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 -- start start 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f648006eb30 0x7f648015f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6480073290 0x7f6480160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6480163000 0x7f6480153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6480073af0 con 0x7f6480073290 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f648006e070 con 0x7f648006eb30 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f648016cf30 con 0x7f6480163000 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6480073290 0x7f6480160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6480073290 0x7f6480160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36998/0 (socket says 172.21.15.110:36998) 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 -- 172.21.15.110:0/4168372355 learned_addr learned my addr 172.21.15.110:0/4168372355 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647ea00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f648006eb30 0x7f648015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647f400640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6480163000 0x7f6480153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 -- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6480163000 msgr2=0x7f6480153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6480163000 0x7f6480153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 -- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f648006eb30 msgr2=0x7f648015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f648006eb30 0x7f648015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 -- 172.21.15.110:0/4168372355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64801615e0 con 0x7f6480073290 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647ea00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f648006eb30 0x7f648015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f647e000640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6480073290 0x7f6480160d00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c420 tx=0x7f646c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6477400640 1 -- 172.21.15.110:0/4168372355 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c017070 con 0x7f6480073290 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6477400640 1 -- 172.21.15.110:0/4168372355 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f646c005cf0 con 0x7f6480073290 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6477400640 1 -- 172.21.15.110:0/4168372355 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c0072f0 con 0x7f6480073290 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64801618c0 con 0x7f6480073290 2024-09-10T10:01:52.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6480161e80 con 0x7f6480073290 2024-09-10T10:01:52.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.148+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64800d0cb0 con 0x7f6480073290 2024-09-10T10:01:52.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.152+0000 7f6477400640 1 -- 172.21.15.110:0/4168372355 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f646c01a070 con 0x7f6480073290 2024-09-10T10:01:52.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.152+0000 7f6477400640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6460067c20 0x7f646006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.152+0000 7f6477400640 1 -- 172.21.15.110:0/4168372355 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f646c085280 con 0x7f6480073290 2024-09-10T10:01:52.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.152+0000 7f647ea00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6460067c20 0x7f646006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.152+0000 7f647ea00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6460067c20 0x7f646006a0e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f64700af000 tx=0x7f64700a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:52.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.156+0000 7f6477400640 1 -- 172.21.15.110:0/4168372355 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f646c055d00 con 0x7f6480073290 2024-09-10T10:01:52.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.344+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64800b69b0 con 0x7f6480073290 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.344+0000 7f6477400640 1 -- 172.21.15.110:0/4168372355 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3923 (secure 0 0 0) 0x7f646c055600 con 0x7f6480073290 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":39,"btime":"2024-09-10T10:01:46:178359+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/2645162871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2645162871},{"type":"v1","addr":"172.21.15.145:6834","nonce":2645162871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1524995501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1524995501},{"type":"v1","addr":"172.21.15.145:6835","nonce":1524995501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/2642324353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2642324353},{"type":"v1","addr":"172.21.15.110:6836","nonce":2642324353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2599516777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2599516777},{"type":"v1","addr":"172.21.15.110:6837","nonce":2599516777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 39 2024-09-10T10:01:52.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6460067c20 msgr2=0x7f646006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6460067c20 0x7f646006a0e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f64700af000 tx=0x7f64700a7070 comp rx=0 tx=0).stop 2024-09-10T10:01:52.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6480073290 msgr2=0x7f6480160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6480073290 0x7f6480160d00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c420 tx=0x7f646c00c8e0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 shutdown_connections 2024-09-10T10:01:52.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6460067c20 0x7f646006a0e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6480163000 0x7f6480153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6480073290 0x7f6480160d00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 --2- 172.21.15.110:0/4168372355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f648006eb30 0x7f648015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 >> 172.21.15.110:0/4168372355 conn(0x7f64800d25c0 msgr2=0x7f6480157960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:52.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 shutdown_connections 2024-09-10T10:01:52.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.348+0000 7f6485e00640 1 -- 172.21.15.110:0/4168372355 wait complete. 2024-09-10T10:01:52.372 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:01:52.372 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:01:52.372 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:01:52.372 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:01:52.373 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:01:52.373 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:01:52.373 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:01:52.373 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:01:52.373 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:01:52.374 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:01:52.374 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:01:52.374 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:01:52.374 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:01:52.374 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:01:52.374 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:01:52.374 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:01:52.387 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:01:52.387 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:01:52.388 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:01:52.388 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:01:52.392 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:01:52.393 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:01:52.393 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:01:52.394 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:01:52.431 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:01:52.432 DEBUG:teuthology.parallel:result is None 2024-09-10T10:01:52.433 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:01:52.433 DEBUG:teuthology.parallel:result is None 2024-09-10T10:01:52.441 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:01:52.441 DEBUG:teuthology.parallel:result is None 2024-09-10T10:01:52.443 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:01:52.443 DEBUG:teuthology.parallel:result is None 2024-09-10T10:01:52.443 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:01:52.517 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:01:52.518 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:01:52.531 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:01:52.531 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:01:52.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.584+0000 7ff1c0800640 1 Processor -- start 2024-09-10T10:01:52.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.584+0000 7ff1c0800640 1 -- start start 2024-09-10T10:01:52.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.584+0000 7ff1c0800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cfb90 0x7ff1b815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.584+0000 7ff1c0800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1b80b3f40 0x7ff1b80cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.584+0000 7ff1c0800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b815d4d0 con 0x7ff1b80cff70 2024-09-10T10:01:52.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.584+0000 7ff1c0800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b806d130 con 0x7ff1b80b3f40 2024-09-10T10:01:52.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.584+0000 7ff1c0800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b815d7b0 con 0x7ff1b80cfb90 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bde00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cfb90 0x7ff1b815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bd400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1b80b3f40 0x7ff1b80cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bd400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1b80b3f40 0x7ff1b80cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:43034/0 (socket says 172.21.15.110:43034) 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bd400640 1 -- 172.21.15.110:0/3175079987 learned_addr learned my addr 172.21.15.110:0/3175079987 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bde00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cfb90 0x7ff1b815ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:47230/0 (socket says 172.21.15.110:47230) 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bd400640 1 -- 172.21.15.110:0/3175079987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cfb90 msgr2=0x7ff1b815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bd400640 1 --2- 172.21.15.110:0/3175079987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cfb90 0x7ff1b815ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bd400640 1 -- 172.21.15.110:0/3175079987 >> v1:172.21.15.145:6789/0 conn(0x7ff1b80cff70 legacy=0x7ff1b8160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bd400640 1 -- 172.21.15.110:0/3175079987 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b806d380 con 0x7ff1b80b3f40 2024-09-10T10:01:52.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bd400640 1 --2- 172.21.15.110:0/3175079987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80b3f40 0x7ff1b80cd5a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009870 tx=0x7ff1a801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9320b01644942b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:52.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bca00640 1 -- 172.21.15.110:0/3175079987 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80b0070 con 0x7ff1b80b3f40 2024-09-10T10:01:52.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bca00640 1 -- 172.21.15.110:0/3175079987 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a80ab600 con 0x7ff1b80b3f40 2024-09-10T10:01:52.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.588+0000 7ff1bca00640 1 -- 172.21.15.110:0/3175079987 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80ab9d0 con 0x7ff1b80b3f40 2024-09-10T10:01:52.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- 172.21.15.110:0/3175079987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80b3f40 msgr2=0x7ff1b80cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 --2- 172.21.15.110:0/3175079987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80b3f40 0x7ff1b80cd5a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009870 tx=0x7ff1a801c4d0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- 172.21.15.110:0/3175079987 shutdown_connections 2024-09-10T10:01:52.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 --2- 172.21.15.110:0/3175079987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80b3f40 0x7ff1b80cd5a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 --2- 172.21.15.110:0/3175079987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cfb90 0x7ff1b815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- 172.21.15.110:0/3175079987 >> 172.21.15.110:0/3175079987 conn(0x7ff1b8062d10 msgr2=0x7ff1b806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:52.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- 172.21.15.110:0/3175079987 shutdown_connections 2024-09-10T10:01:52.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- 172.21.15.110:0/3175079987 wait complete. 2024-09-10T10:01:52.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 Processor -- start 2024-09-10T10:01:52.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- start start 2024-09-10T10:01:52.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1b80b3f40 0x7ff1b813ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80cfb90 0x7ff1b8071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cff70 0x7ff1b80ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8169500 con 0x7ff1b80b3f40 2024-09-10T10:01:52.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8169380 con 0x7ff1b80cfb90 2024-09-10T10:01:52.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1c0800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b8169680 con 0x7ff1b80cff70 2024-09-10T10:01:52.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1b80b3f40 0x7ff1b813ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1b80b3f40 0x7ff1b813ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37046/0 (socket says 172.21.15.110:37046) 2024-09-10T10:01:52.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 -- 172.21.15.110:0/1574194329 learned_addr learned my addr 172.21.15.110:0/1574194329 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:52.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bd400640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80cfb90 0x7ff1b8071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1be800640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cff70 0x7ff1b80ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 -- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cff70 msgr2=0x7ff1b80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cff70 0x7ff1b80ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 -- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80cfb90 msgr2=0x7ff1b8071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80cfb90 0x7ff1b8071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 -- 172.21.15.110:0/1574194329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b813d200 con 0x7ff1b80b3f40 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bd400640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80cfb90 0x7ff1b8071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.592+0000 7ff1bde00640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1b80b3f40 0x7ff1b813ccc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00e960 tx=0x7ff1ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1b2000640 1 -- 172.21.15.110:0/1574194329 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac017070 con 0x7ff1b80b3f40 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1b2000640 1 -- 172.21.15.110:0/1574194329 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac00ca00 con 0x7ff1b80b3f40 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1b2000640 1 -- 172.21.15.110:0/1574194329 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac00cdb0 con 0x7ff1b80b3f40 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b80b02e0 con 0x7ff1b80b3f40 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b80b08a0 con 0x7ff1b80b3f40 2024-09-10T10:01:52.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b80cd5a0 con 0x7ff1b80b3f40 2024-09-10T10:01:52.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1b2000640 1 -- 172.21.15.110:0/1574194329 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff1ac01a070 con 0x7ff1b80b3f40 2024-09-10T10:01:52.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1b2000640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff19c067c20 0x7ff19c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1b2000640 1 -- 172.21.15.110:0/1574194329 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7ff1ac086280 con 0x7ff1b80b3f40 2024-09-10T10:01:52.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.596+0000 7ff1bd400640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff19c067c20 0x7ff19c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.600+0000 7ff1bd400640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff19c067c20 0x7ff19c06a0e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80ad000 tx=0x7ff1a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:52.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.600+0000 7ff1b2000640 1 -- 172.21.15.110:0/1574194329 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff1ac056620 con 0x7ff1b80b3f40 2024-09-10T10:01:52.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:52.772+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff1b80b54d0 con 0x7ff1b80b3f40 2024-09-10T10:01:53.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.360+0000 7ff1b2000640 1 -- 172.21.15.110:0/1574194329 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7ff1ac05a920 con 0x7ff1b80b3f40 2024-09-10T10:01:53.364 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:01:53.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff19c067c20 msgr2=0x7ff19c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff19c067c20 0x7ff19c06a0e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80ad000 tx=0x7ff1a8008040 comp rx=0 tx=0).stop 2024-09-10T10:01:53.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1b80b3f40 msgr2=0x7ff1b813ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1b80b3f40 0x7ff1b813ccc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00e960 tx=0x7ff1ac00ee20 comp rx=0 tx=0).stop 2024-09-10T10:01:53.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 shutdown_connections 2024-09-10T10:01:53.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7ff19c067c20 0x7ff19c06a0e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff1b80cff70 0x7ff1b80ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1b80cfb90 0x7ff1b8071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 --2- 172.21.15.110:0/1574194329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1b80b3f40 0x7ff1b813ccc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 >> 172.21.15.110:0/1574194329 conn(0x7ff1b8062d10 msgr2=0x7ff1b8166310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 shutdown_connections 2024-09-10T10:01:53.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.364+0000 7ff1c0800640 1 -- 172.21.15.110:0/1574194329 wait complete. 2024-09-10T10:01:53.390 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:01:53.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11da00640 1 Processor -- start 2024-09-10T10:01:53.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11da00640 1 -- start start 2024-09-10T10:01:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11da00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc1181568d0 0x7fc1180b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11da00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc1180ceeb0 0x7fc11806e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11da00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 0x7fc1180af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11da00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc1180cf620 con 0x7fc1180ceeb0 2024-09-10T10:01:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11da00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc11806d610 con 0x7fc1181568d0 2024-09-10T10:01:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11da00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc11806d8f0 con 0x7fc11806ec80 2024-09-10T10:01:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc1180ceeb0 0x7fc11806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc1180ceeb0 0x7fc11806e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37052/0 (socket says 172.21.15.110:37052) 2024-09-10T10:01:53.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117400640 1 -- 172.21.15.110:0/3069709127 learned_addr learned my addr 172.21.15.110:0/3069709127 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:53.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117e00640 1 --2- 172.21.15.110:0/3069709127 >> v2:172.21.15.145:3300/0 conn(0x7fc1181568d0 0x7fc1180b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc11d000640 1 --2- 172.21.15.110:0/3069709127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 0x7fc1180af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117400640 1 -- 172.21.15.110:0/3069709127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 msgr2=0x7fc1180af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117400640 1 --2- 172.21.15.110:0/3069709127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 0x7fc1180af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117400640 1 -- 172.21.15.110:0/3069709127 >> v2:172.21.15.145:3300/0 conn(0x7fc1181568d0 msgr2=0x7fc1180b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117400640 1 --2- 172.21.15.110:0/3069709127 >> v2:172.21.15.145:3300/0 conn(0x7fc1181568d0 0x7fc1180b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.524+0000 7fc117400640 1 -- 172.21.15.110:0/3069709127 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11806dbd0 con 0x7fc1180ceeb0 2024-09-10T10:01:53.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc117400640 1 --2- 172.21.15.110:0/3069709127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1180ceeb0 0x7fc11806e740 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b2d90 tx=0x7fc10401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6df952345f50f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc116a00640 1 -- 172.21.15.110:0/3069709127 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1040b0070 con 0x7fc1180ceeb0 2024-09-10T10:01:53.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc116a00640 1 -- 172.21.15.110:0/3069709127 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1040ab600 con 0x7fc1180ceeb0 2024-09-10T10:01:53.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc116a00640 1 -- 172.21.15.110:0/3069709127 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1040ab9d0 con 0x7fc1180ceeb0 2024-09-10T10:01:53.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 -- 172.21.15.110:0/3069709127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1180ceeb0 msgr2=0x7fc11806e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 --2- 172.21.15.110:0/3069709127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1180ceeb0 0x7fc11806e740 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b2d90 tx=0x7fc10401c520 comp rx=0 tx=0).stop 2024-09-10T10:01:53.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 -- 172.21.15.110:0/3069709127 shutdown_connections 2024-09-10T10:01:53.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 --2- 172.21.15.110:0/3069709127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 0x7fc1180af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 --2- 172.21.15.110:0/3069709127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1180ceeb0 0x7fc11806e740 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 --2- 172.21.15.110:0/3069709127 >> v2:172.21.15.145:3300/0 conn(0x7fc1181568d0 0x7fc1180b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 -- 172.21.15.110:0/3069709127 >> 172.21.15.110:0/3069709127 conn(0x7fc118062ce0 msgr2=0x7fc1180b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 -- 172.21.15.110:0/3069709127 shutdown_connections 2024-09-10T10:01:53.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.528+0000 7fc11da00640 1 -- 172.21.15.110:0/3069709127 wait complete. 2024-09-10T10:01:53.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 Processor -- start 2024-09-10T10:01:53.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 -- start start 2024-09-10T10:01:53.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 0x7fc1180acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1180ceeb0 0x7fc118159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1181568d0 0x7fc1180ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc11815ca00 con 0x7fc1181568d0 2024-09-10T10:01:53.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc11815c880 con 0x7fc1180ceeb0 2024-09-10T10:01:53.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc1180cd0c0 con 0x7fc11806ec80 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1181568d0 0x7fc1180ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc117e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 0x7fc1180acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1181568d0 0x7fc1180ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37064/0 (socket says 172.21.15.110:37064) 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc117400640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1180ceeb0 0x7fc118159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 -- 172.21.15.110:0/1471140557 learned_addr learned my addr 172.21.15.110:0/1471140557 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 -- 172.21.15.110:0/1471140557 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 msgr2=0x7fc1180acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 0x7fc1180acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 -- 172.21.15.110:0/1471140557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1180ceeb0 msgr2=0x7fc118159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1180ceeb0 0x7fc118159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 -- 172.21.15.110:0/1471140557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc118157dd0 con 0x7fc1181568d0 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11d000640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1181568d0 0x7fc1180ae630 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc11000d560 tx=0x7fc11000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc10fe00640 1 -- 172.21.15.110:0/1471140557 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100125f0 con 0x7fc1181568d0 2024-09-10T10:01:53.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc10fe00640 1 -- 172.21.15.110:0/1471140557 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc110019070 con 0x7fc1181568d0 2024-09-10T10:01:53.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1181580b0 con 0x7fc1181568d0 2024-09-10T10:01:53.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc10fe00640 1 -- 172.21.15.110:0/1471140557 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100129f0 con 0x7fc1181568d0 2024-09-10T10:01:53.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc118158670 con 0x7fc1181568d0 2024-09-10T10:01:53.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.532+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1180af480 con 0x7fc1181568d0 2024-09-10T10:01:53.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.536+0000 7fc10fe00640 1 -- 172.21.15.110:0/1471140557 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fc110012b90 con 0x7fc1181568d0 2024-09-10T10:01:53.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.536+0000 7fc10fe00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0f8067c20 0x7fc0f806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.536+0000 7fc10fe00640 1 -- 172.21.15.110:0/1471140557 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7fc1100861b0 con 0x7fc1181568d0 2024-09-10T10:01:53.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.536+0000 7fc117e00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0f8067c20 0x7fc0f806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.536+0000 7fc117e00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0f8067c20 0x7fc0f806a0e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc108003ab0 tx=0x7fc1080098d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.540+0000 7fc10fe00640 1 -- 172.21.15.110:0/1471140557 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc1100565a0 con 0x7fc1181568d0 2024-09-10T10:01:53.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc118158870 con 0x7fc1181568d0 2024-09-10T10:01:53.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc10fe00640 1 -- 172.21.15.110:0/1471140557 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5617 (secure 0 0 0) 0x7fc11005a8a0 con 0x7fc1181568d0 2024-09-10T10:01:53.758 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:53.758 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:01:53.758 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:01:53.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.758 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.759 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:53.760 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:01:53.762 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:01:53.764 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:01:53.766 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:01:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:01:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.alice", 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGeBmLkitHxAAEJDMPUZTmeNuGNgLJwWL8w==", 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol57160", 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:01:53.773 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:01:53.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0f8067c20 msgr2=0x7fc0f806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0f8067c20 0x7fc0f806a0e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc108003ab0 tx=0x7fc1080098d0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1181568d0 msgr2=0x7fc1180ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1181568d0 0x7fc1180ae630 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc11000d560 tx=0x7fc11000da20 comp rx=0 tx=0).stop 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 shutdown_connections 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc0f8067c20 0x7fc0f806a0e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1181568d0 0x7fc1180ae630 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1180ceeb0 0x7fc118159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 --2- 172.21.15.110:0/1471140557 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc11806ec80 0x7fc1180acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 >> 172.21.15.110:0/1471140557 conn(0x7fc118062ce0 msgr2=0x7fc1180b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 shutdown_connections 2024-09-10T10:01:53.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.756+0000 7fc11da00640 1 -- 172.21.15.110:0/1471140557 wait complete. 2024-09-10T10:01:53.778 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-10T10:01:53.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43de400640 1 Processor -- start 2024-09-10T10:01:53.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43de400640 1 -- start start 2024-09-10T10:01:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43de400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80b3f40 0x7f43d80cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43de400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f43d80cff70 0x7f43d80cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43de400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d815d4d0 con 0x7f43d80cfb90 2024-09-10T10:01:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43de400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f43d806d130 con 0x7f43d80cff70 2024-09-10T10:01:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43de400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f43d815d7b0 con 0x7f43d80b3f40 2024-09-10T10:01:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d7400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f43d80cfb90 0x7f43d815ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42864/0 (socket says 172.21.15.110:42864) 2024-09-10T10:01:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d7400640 1 -- 172.21.15.110:0/2815796587 learned_addr learned my addr 172.21.15.110:0/2815796587 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:53.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d7e00640 1 --2- 172.21.15.110:0/2815796587 >> v2:172.21.15.110:3300/0 conn(0x7f43d80cff70 0x7f43d80cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d6000640 1 -- 172.21.15.110:0/2815796587 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1757949812 0 0) 0x7f43d815d4d0 con 0x7f43d80cfb90 2024-09-10T10:01:53.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d6a00640 1 --2- 172.21.15.110:0/2815796587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80b3f40 0x7f43d80cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d6000640 1 -- 172.21.15.110:0/2815796587 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43bc003530 con 0x7f43d80cfb90 2024-09-10T10:01:53.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d7e00640 1 -- 172.21.15.110:0/2815796587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80b3f40 msgr2=0x7f43d80cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:53.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d7e00640 1 --2- 172.21.15.110:0/2815796587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80b3f40 0x7f43d80cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d7e00640 1 -- 172.21.15.110:0/2815796587 >> v1:172.21.15.110:6789/0 conn(0x7f43d80cfb90 legacy=0x7f43d815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.912+0000 7f43d7e00640 1 -- 172.21.15.110:0/2815796587 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d806d380 con 0x7f43d80cff70 2024-09-10T10:01:53.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43d7e00640 1 --2- 172.21.15.110:0/2815796587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cff70 0x7f43d80cdae0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f43d00097c0 tx=0x7f43d001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33155281ad62d5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43d6000640 1 -- 172.21.15.110:0/2815796587 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d00b0070 con 0x7f43d80cff70 2024-09-10T10:01:53.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43d6000640 1 -- 172.21.15.110:0/2815796587 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d00ab600 con 0x7f43d80cff70 2024-09-10T10:01:53.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43d6000640 1 -- 172.21.15.110:0/2815796587 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d00ab9b0 con 0x7f43d80cff70 2024-09-10T10:01:53.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43de400640 1 -- 172.21.15.110:0/2815796587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cff70 msgr2=0x7f43d80cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43de400640 1 --2- 172.21.15.110:0/2815796587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cff70 0x7f43d80cdae0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f43d00097c0 tx=0x7f43d001c410 comp rx=0 tx=0).stop 2024-09-10T10:01:53.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43de400640 1 -- 172.21.15.110:0/2815796587 shutdown_connections 2024-09-10T10:01:53.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43de400640 1 --2- 172.21.15.110:0/2815796587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cff70 0x7f43d80cdae0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43de400640 1 --2- 172.21.15.110:0/2815796587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80b3f40 0x7f43d80cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43de400640 1 -- 172.21.15.110:0/2815796587 >> 172.21.15.110:0/2815796587 conn(0x7f43d8062d10 msgr2=0x7f43d806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43de400640 1 -- 172.21.15.110:0/2815796587 shutdown_connections 2024-09-10T10:01:53.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.916+0000 7f43de400640 1 -- 172.21.15.110:0/2815796587 wait complete. 2024-09-10T10:01:53.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 Processor -- start 2024-09-10T10:01:53.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 -- start start 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43d80b3f40 0x7f43d81642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cfb90 0x7f43d8150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80cff70 0x7f43d8165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43d806e720 con 0x7f43d80cfb90 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43d806e5a0 con 0x7f43d80b3f40 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f43d806e8a0 con 0x7f43d80cff70 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cfb90 0x7f43d8150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cfb90 0x7f43d8150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37080/0 (socket says 172.21.15.110:37080) 2024-09-10T10:01:53.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 -- 172.21.15.110:0/2338596695 learned_addr learned my addr 172.21.15.110:0/2338596695 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:53.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d7400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43d80b3f40 0x7f43d81642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d7e00640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80cff70 0x7f43d8165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 -- 172.21.15.110:0/2338596695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80cff70 msgr2=0x7f43d8165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80cff70 0x7f43d8165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 -- 172.21.15.110:0/2338596695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43d80b3f40 msgr2=0x7f43d81642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43d80b3f40 0x7f43d81642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 -- 172.21.15.110:0/2338596695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d81612d0 con 0x7f43d80cfb90 2024-09-10T10:01:53.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43d6a00640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cfb90 0x7f43d8150660 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f43c400d5f0 tx=0x7f43c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43cf400640 1 -- 172.21.15.110:0/2338596695 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c40125f0 con 0x7f43d80cfb90 2024-09-10T10:01:53.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43cf400640 1 -- 172.21.15.110:0/2338596695 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c4019070 con 0x7f43d80cfb90 2024-09-10T10:01:53.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43cf400640 1 -- 172.21.15.110:0/2338596695 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c40129b0 con 0x7f43d80cfb90 2024-09-10T10:01:53.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d81615b0 con 0x7f43d80cfb90 2024-09-10T10:01:53.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d8161b70 con 0x7f43d80cfb90 2024-09-10T10:01:53.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.920+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d815ca10 con 0x7f43d80cfb90 2024-09-10T10:01:53.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.924+0000 7f43cf400640 1 -- 172.21.15.110:0/2338596695 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f43c4012b50 con 0x7f43d80cfb90 2024-09-10T10:01:53.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.924+0000 7f43cf400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f43b8067c20 0x7f43b806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.924+0000 7f43cf400640 1 -- 172.21.15.110:0/2338596695 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f43c40860c0 con 0x7f43d80cfb90 2024-09-10T10:01:53.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.924+0000 7f43d7400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f43b8067c20 0x7f43b806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.924+0000 7f43d7400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f43b8067c20 0x7f43b806a0e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005c10 tx=0x7f43c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:53.928+0000 7f43cf400640 1 -- 172.21.15.110:0/2338596695 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f43c4056540 con 0x7f43d80cfb90 2024-09-10T10:01:54.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.140+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7f43d815fbb0 con 0x7f43d80cfb90 2024-09-10T10:01:54.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.140+0000 7f43cf400640 1 -- 172.21.15.110:0/2338596695 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v15) ==== 82+0+0 (secure 0 0 0) 0x7f43c405a840 con 0x7f43d80cfb90 2024-09-10T10:01:54.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f43b8067c20 msgr2=0x7f43b806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f43b8067c20 0x7f43b806a0e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005c10 tx=0x7f43c8009040 comp rx=0 tx=0).stop 2024-09-10T10:01:54.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cfb90 msgr2=0x7f43d8150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cfb90 0x7f43d8150660 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f43c400d5f0 tx=0x7f43c400dab0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 shutdown_connections 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f43b8067c20 0x7f43b806a0e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f43d80cff70 0x7f43d8165bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43d80cfb90 0x7f43d8150660 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 --2- 172.21.15.110:0/2338596695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43d80b3f40 0x7f43d81642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 >> 172.21.15.110:0/2338596695 conn(0x7f43d8062d10 msgr2=0x7f43d80b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 shutdown_connections 2024-09-10T10:01:54.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.144+0000 7f43de400640 1 -- 172.21.15.110:0/2338596695 wait complete. 2024-09-10T10:01:54.165 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:01:54.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.296+0000 7f81e0200640 1 Processor -- start 2024-09-10T10:01:54.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.296+0000 7f81e0200640 1 -- start start 2024-09-10T10:01:54.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81e0200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81d80aef50 0x7f81d806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81e0200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d81482e0 con 0x7f81d8147f00 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81e0200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d815ee20 con 0x7f81d806ec20 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81e0200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f81d80b34d0 con 0x7f81d80aef50 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81dd800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f81d8147f00 0x7f81d80ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42878/0 (socket says 172.21.15.110:42878) 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81dce00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81d80aef50 0x7f81d806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81dd800640 1 -- 172.21.15.110:0/2331232504 learned_addr learned my addr 172.21.15.110:0/2331232504 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81dce00640 1 -- 172.21.15.110:0/2331232504 >> v1:172.21.15.145:6789/0 conn(0x7f81d806ec20 legacy=0x7f81d8072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81dce00640 1 -- 172.21.15.110:0/2331232504 >> v1:172.21.15.110:6789/0 conn(0x7f81d8147f00 legacy=0x7f81d80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81d7e00640 1 -- 172.21.15.110:0/2331232504 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 119045912 0 0) 0x7f81d81482e0 con 0x7f81d8147f00 2024-09-10T10:01:54.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81dce00640 1 -- 172.21.15.110:0/2331232504 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d815f100 con 0x7f81d80aef50 2024-09-10T10:01:54.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.300+0000 7f81dce00640 1 --2- 172.21.15.110:0/2331232504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d80aef50 0x7f81d806e6e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f81c40097c0 tx=0x7f81c401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b19b1293b1a0b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:54.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81d7e00640 1 -- 172.21.15.110:0/2331232504 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40b0070 con 0x7f81d80aef50 2024-09-10T10:01:54.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81d7e00640 1 -- 172.21.15.110:0/2331232504 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c40ab600 con 0x7f81d80aef50 2024-09-10T10:01:54.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81d7e00640 1 -- 172.21.15.110:0/2331232504 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40ab9d0 con 0x7f81d80aef50 2024-09-10T10:01:54.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- 172.21.15.110:0/2331232504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d80aef50 msgr2=0x7f81d806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 --2- 172.21.15.110:0/2331232504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d80aef50 0x7f81d806e6e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f81c40097c0 tx=0x7f81c401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- 172.21.15.110:0/2331232504 shutdown_connections 2024-09-10T10:01:54.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 --2- 172.21.15.110:0/2331232504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d80aef50 0x7f81d806e6e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- 172.21.15.110:0/2331232504 >> 172.21.15.110:0/2331232504 conn(0x7f81d806bc50 msgr2=0x7f81d806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:54.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- 172.21.15.110:0/2331232504 shutdown_connections 2024-09-10T10:01:54.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- 172.21.15.110:0/2331232504 wait complete. 2024-09-10T10:01:54.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 Processor -- start 2024-09-10T10:01:54.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- start start 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d806ec20 0x7f81d8147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81d80aef50 0x7f81d813d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81d8147f00 0x7f81d8140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81d816c590 con 0x7f81d806ec20 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81d816c410 con 0x7f81d8147f00 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81e0200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81d816c710 con 0x7f81d80aef50 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dd800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d806ec20 0x7f81d8147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dd800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d806ec20 0x7f81d8147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37088/0 (socket says 172.21.15.110:37088) 2024-09-10T10:01:54.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dd800640 1 -- 172.21.15.110:0/412667060 learned_addr learned my addr 172.21.15.110:0/412667060 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81de200640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81d8147f00 0x7f81d8140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dce00640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81d80aef50 0x7f81d813d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dd800640 1 -- 172.21.15.110:0/412667060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81d80aef50 msgr2=0x7f81d813d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dd800640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81d80aef50 0x7f81d813d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dd800640 1 -- 172.21.15.110:0/412667060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81d8147f00 msgr2=0x7f81d8140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dd800640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81d8147f00 0x7f81d8140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.304+0000 7f81dd800640 1 -- 172.21.15.110:0/412667060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d813d9e0 con 0x7f81d806ec20 2024-09-10T10:01:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81dd800640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d806ec20 0x7f81d8147250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c8c0 tx=0x7f81cc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81d5600640 1 -- 172.21.15.110:0/412667060 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc018070 con 0x7f81d806ec20 2024-09-10T10:01:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81d5600640 1 -- 172.21.15.110:0/412667060 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81cc00a640 con 0x7f81d806ec20 2024-09-10T10:01:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d81461b0 con 0x7f81d806ec20 2024-09-10T10:01:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81d5600640 1 -- 172.21.15.110:0/412667060 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00aa70 con 0x7f81d806ec20 2024-09-10T10:01:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d8146770 con 0x7f81d806ec20 2024-09-10T10:01:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d806e6e0 con 0x7f81d806ec20 2024-09-10T10:01:54.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81d5600640 1 -- 172.21.15.110:0/412667060 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f81cc01b070 con 0x7f81d806ec20 2024-09-10T10:01:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81d5600640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81b8067c20 0x7f81b806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81d5600640 1 -- 172.21.15.110:0/412667060 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f81cc0855a0 con 0x7f81d806ec20 2024-09-10T10:01:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.308+0000 7f81dce00640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81b8067c20 0x7f81b806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.312+0000 7f81dce00640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81b8067c20 0x7f81b806a0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f81c40ad000 tx=0x7f81c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:54.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.312+0000 7f81d5600640 1 -- 172.21.15.110:0/412667060 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f81cc055f70 con 0x7f81d806ec20 2024-09-10T10:01:54.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.480+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f81d806c310 con 0x7f81d806ec20 2024-09-10T10:01:54.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.480+0000 7f81d5600640 1 -- 172.21.15.110:0/412667060 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) ==== 84+0+65 (secure 0 0 0) 0x7f81cc055870 con 0x7f81d806ec20 2024-09-10T10:01:54.486 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:54.486 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:01:54.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:01:54.486 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:01:54.486 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:01:54.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:01:54.486 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:01:54.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81b8067c20 msgr2=0x7f81b806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81b8067c20 0x7f81b806a0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f81c40ad000 tx=0x7f81c4008040 comp rx=0 tx=0).stop 2024-09-10T10:01:54.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d806ec20 msgr2=0x7f81d8147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d806ec20 0x7f81d8147250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c8c0 tx=0x7f81cc00cd80 comp rx=0 tx=0).stop 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 shutdown_connections 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f81b8067c20 0x7f81b806a0e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81d8147f00 0x7f81d8140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81d80aef50 0x7f81d813d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 --2- 172.21.15.110:0/412667060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81d806ec20 0x7f81d8147250 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 >> 172.21.15.110:0/412667060 conn(0x7f81d806bc50 msgr2=0x7f81d80ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 shutdown_connections 2024-09-10T10:01:54.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.484+0000 7f81e0200640 1 -- 172.21.15.110:0/412667060 wait complete. 2024-09-10T10:01:54.506 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:01:54.506 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:01:54.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7e4a00640 1 Processor -- start 2024-09-10T10:01:54.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7e4a00640 1 -- start start 2024-09-10T10:01:54.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7e4a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7e00cfe30 0x7fe7e0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7e4a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7e00cd510 0x7fe7e00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7e4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e006e090 0x7fe7e006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7e4a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e015d9a0 con 0x7fe7e00cfe30 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7e4a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e006ea60 con 0x7fe7e00cd510 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7e4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e006d2d0 con 0x7fe7e006e090 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7e00cfe30 0x7fe7e0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7e00cfe30 0x7fe7e0071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37100/0 (socket says 172.21.15.110:37100) 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dde00640 1 -- 172.21.15.110:0/327888522 learned_addr learned my addr 172.21.15.110:0/327888522 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dd400640 1 --2- 172.21.15.110:0/327888522 >> v2:172.21.15.145:3300/0 conn(0x7fe7e00cd510 0x7fe7e00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7de800640 1 --2- 172.21.15.110:0/327888522 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e006e090 0x7fe7e006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dde00640 1 -- 172.21.15.110:0/327888522 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e006e090 msgr2=0x7fe7e006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:54.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dde00640 1 --2- 172.21.15.110:0/327888522 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e006e090 0x7fe7e006e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dde00640 1 -- 172.21.15.110:0/327888522 >> v2:172.21.15.145:3300/0 conn(0x7fe7e00cd510 msgr2=0x7fe7e00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:54.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dde00640 1 --2- 172.21.15.110:0/327888522 >> v2:172.21.15.145:3300/0 conn(0x7fe7e00cd510 0x7fe7e00cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dde00640 1 -- 172.21.15.110:0/327888522 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e006d5b0 con 0x7fe7e00cfe30 2024-09-10T10:01:54.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.644+0000 7fe7dd400640 1 --2- 172.21.15.110:0/327888522 >> v2:172.21.15.145:3300/0 conn(0x7fe7e00cd510 0x7fe7e00cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:54.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7dde00640 1 --2- 172.21.15.110:0/327888522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cfe30 0x7fe7e0071a40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00ce0f0 tx=0x7fe7d001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae161b33a279336 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:54.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7dca00640 1 -- 172.21.15.110:0/327888522 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00b2070 con 0x7fe7e00cfe30 2024-09-10T10:01:54.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7dca00640 1 -- 172.21.15.110:0/327888522 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d00a9070 con 0x7fe7e00cfe30 2024-09-10T10:01:54.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7dca00640 1 -- 172.21.15.110:0/327888522 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00ad880 con 0x7fe7e00cfe30 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 -- 172.21.15.110:0/327888522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cfe30 msgr2=0x7fe7e0071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/327888522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cfe30 0x7fe7e0071a40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00ce0f0 tx=0x7fe7d001c490 comp rx=0 tx=0).stop 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 -- 172.21.15.110:0/327888522 shutdown_connections 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/327888522 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e006e090 0x7fe7e006e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/327888522 >> v2:172.21.15.145:3300/0 conn(0x7fe7e00cd510 0x7fe7e00cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/327888522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cfe30 0x7fe7e0071a40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 -- 172.21.15.110:0/327888522 >> 172.21.15.110:0/327888522 conn(0x7fe7e006bc50 msgr2=0x7fe7e006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 -- 172.21.15.110:0/327888522 shutdown_connections 2024-09-10T10:01:54.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 -- 172.21.15.110:0/327888522 wait complete. 2024-09-10T10:01:54.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.648+0000 7fe7e4a00640 1 Processor -- start 2024-09-10T10:01:54.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 -- start start 2024-09-10T10:01:54.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7e006e090 0x7fe7e01f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cd510 0x7fe7e00aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e00cfe30 0x7fe7e01529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0072190 con 0x7fe7e00cd510 2024-09-10T10:01:54.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e015d680 con 0x7fe7e006e090 2024-09-10T10:01:54.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e0160010 con 0x7fe7e00cfe30 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cd510 0x7fe7e00aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cd510 0x7fe7e00aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37114/0 (socket says 172.21.15.110:37114) 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 -- 172.21.15.110:0/1540662542 learned_addr learned my addr 172.21.15.110:0/1540662542 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dde00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7e006e090 0x7fe7e01f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7de800640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e00cfe30 0x7fe7e01529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 -- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e00cfe30 msgr2=0x7fe7e01529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e00cfe30 0x7fe7e01529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 -- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7e006e090 msgr2=0x7fe7e01f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7e006e090 0x7fe7e01f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 -- 172.21.15.110:0/1540662542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e00af6d0 con 0x7fe7e00cd510 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dde00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7e006e090 0x7fe7e01f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7dd400640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cd510 0x7fe7e00aee80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00e810 tx=0x7fe7cc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7d6000640 1 -- 172.21.15.110:0/1540662542 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc017070 con 0x7fe7e00cd510 2024-09-10T10:01:54.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7d6000640 1 -- 172.21.15.110:0/1540662542 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7cc00c8d0 con 0x7fe7e00cd510 2024-09-10T10:01:54.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e00af9b0 con 0x7fe7e00cd510 2024-09-10T10:01:54.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7d6000640 1 -- 172.21.15.110:0/1540662542 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00f040 con 0x7fe7e00cd510 2024-09-10T10:01:54.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e00aff70 con 0x7fe7e00cd510 2024-09-10T10:01:54.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.652+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e0159e60 con 0x7fe7e00cd510 2024-09-10T10:01:54.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.656+0000 7fe7d6000640 1 -- 172.21.15.110:0/1540662542 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe7cc01a070 con 0x7fe7e00cd510 2024-09-10T10:01:54.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.656+0000 7fe7d6000640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe7c0067c20 0x7fe7c006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:54.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.656+0000 7fe7d6000640 1 -- 172.21.15.110:0/1540662542 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7fe7cc085b10 con 0x7fe7e00cd510 2024-09-10T10:01:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.656+0000 7fe7dde00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe7c0067c20 0x7fe7c006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.656+0000 7fe7dde00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe7c0067c20 0x7fe7c006a0e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00af000 tx=0x7fe7d00a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:54.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.660+0000 7fe7d6000640 1 -- 172.21.15.110:0/1540662542 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe7cc056510 con 0x7fe7e00cd510 2024-09-10T10:01:54.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:54.816+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe7e0156940 con 0x7fe7e00cd510 2024-09-10T10:01:55.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.372+0000 7fe7d6000640 1 -- 172.21.15.110:0/1540662542 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) ==== 138+0+0 (secure 0 0 0) 0x7fe7cc055e10 con 0x7fe7e00cd510 2024-09-10T10:01:55.377 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:01:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe7c0067c20 msgr2=0x7fe7c006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe7c0067c20 0x7fe7c006a0e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00af000 tx=0x7fe7d00a7000 comp rx=0 tx=0).stop 2024-09-10T10:01:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cd510 msgr2=0x7fe7e00aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cd510 0x7fe7e00aee80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00e810 tx=0x7fe7cc00ecd0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 shutdown_connections 2024-09-10T10:01:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fe7c0067c20 0x7fe7c006a0e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7e00cfe30 0x7fe7e01529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e00cd510 0x7fe7e00aee80 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 --2- 172.21.15.110:0/1540662542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7e006e090 0x7fe7e01f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 >> 172.21.15.110:0/1540662542 conn(0x7fe7e006bc50 msgr2=0x7fe7e015a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 shutdown_connections 2024-09-10T10:01:55.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.376+0000 7fe7e4a00640 1 -- 172.21.15.110:0/1540662542 wait complete. 2024-09-10T10:01:55.403 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:01:55.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2608e00640 1 Processor -- start 2024-09-10T10:01:55.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2608e00640 1 -- start start 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2608e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2604155970 0x7f26040cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2608e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f260415c560 0x7f260406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2608e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260406f430 con 0x7f260406f010 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2608e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f26040b4570 con 0x7f2604155970 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2608e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2604155d50 con 0x7f260415c560 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2601e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2604155970 0x7f26040cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2601e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2604155970 0x7f26040cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37118/0 (socket says 172.21.15.110:37118) 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2601e00640 1 -- 172.21.15.110:0/2931186014 learned_addr learned my addr 172.21.15.110:0/2931186014 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:55.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2601400640 1 --2- 172.21.15.110:0/2931186014 >> v2:172.21.15.145:3300/0 conn(0x7f260415c560 0x7f260406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2601e00640 1 -- 172.21.15.110:0/2931186014 >> v2:172.21.15.145:3300/0 conn(0x7f260415c560 msgr2=0x7f260406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2601e00640 1 --2- 172.21.15.110:0/2931186014 >> v2:172.21.15.145:3300/0 conn(0x7f260415c560 0x7f260406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2601e00640 1 -- 172.21.15.110:0/2931186014 >> v1:172.21.15.145:6789/0 conn(0x7f260406f010 legacy=0x7f260415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:55.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.544+0000 7f2601e00640 1 -- 172.21.15.110:0/2931186014 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26040b34d0 con 0x7f2604155970 2024-09-10T10:01:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2601e00640 1 --2- 172.21.15.110:0/2931186014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f26040cc8c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f260406d930 tx=0x7f25f401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79502c5497ddd29b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2600a00640 1 -- 172.21.15.110:0/2931186014 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f40aa610 con 0x7f2604155970 2024-09-10T10:01:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2600a00640 1 -- 172.21.15.110:0/2931186014 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f40af070 con 0x7f2604155970 2024-09-10T10:01:55.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2600a00640 1 -- 172.21.15.110:0/2931186014 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f40aa9d0 con 0x7f2604155970 2024-09-10T10:01:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2608e00640 1 -- 172.21.15.110:0/2931186014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 msgr2=0x7f26040cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2608e00640 1 --2- 172.21.15.110:0/2931186014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f26040cc8c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f260406d930 tx=0x7f25f401c4b0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2608e00640 1 -- 172.21.15.110:0/2931186014 shutdown_connections 2024-09-10T10:01:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2608e00640 1 --2- 172.21.15.110:0/2931186014 >> v2:172.21.15.145:3300/0 conn(0x7f260415c560 0x7f260406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2608e00640 1 --2- 172.21.15.110:0/2931186014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f26040cc8c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2608e00640 1 -- 172.21.15.110:0/2931186014 >> 172.21.15.110:0/2931186014 conn(0x7f260406bc50 msgr2=0x7f260406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2608e00640 1 -- 172.21.15.110:0/2931186014 shutdown_connections 2024-09-10T10:01:55.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.548+0000 7f2608e00640 1 -- 172.21.15.110:0/2931186014 wait complete. 2024-09-10T10:01:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 Processor -- start 2024-09-10T10:01:55.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 -- start start 2024-09-10T10:01:55.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f260406f010 0x7f26041458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f2604158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f260415c560 0x7f2604144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26040cce00 con 0x7f2604155970 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f26040b4050 con 0x7f260406f010 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2604168030 con 0x7f260415c560 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f2604158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f2604158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37120/0 (socket says 172.21.15.110:37120) 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 -- 172.21.15.110:0/3124119075 learned_addr learned my addr 172.21.15.110:0/3124119075 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f260406f010 0x7f26041458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2602800640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f260415c560 0x7f2604144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 -- 172.21.15.110:0/3124119075 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f260415c560 msgr2=0x7f2604144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f260415c560 0x7f2604144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 -- 172.21.15.110:0/3124119075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f260406f010 msgr2=0x7f26041458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f260406f010 0x7f26041458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 -- 172.21.15.110:0/3124119075 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2604145e20 con 0x7f2604155970 2024-09-10T10:01:55.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2601400640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f2604158460 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f25f000d830 tx=0x7f25f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f25fa000640 1 -- 172.21.15.110:0/3124119075 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000b6c0 con 0x7f2604155970 2024-09-10T10:01:55.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f25fa000640 1 -- 172.21.15.110:0/3124119075 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f0019070 con 0x7f2604155970 2024-09-10T10:01:55.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f25fa000640 1 -- 172.21.15.110:0/3124119075 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000bae0 con 0x7f2604155970 2024-09-10T10:01:55.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26041432e0 con 0x7f2604155970 2024-09-10T10:01:55.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041438a0 con 0x7f2604155970 2024-09-10T10:01:55.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.552+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26040cc8c0 con 0x7f2604155970 2024-09-10T10:01:55.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.556+0000 7f25fa000640 1 -- 172.21.15.110:0/3124119075 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f25f001e460 con 0x7f2604155970 2024-09-10T10:01:55.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.556+0000 7f25fa000640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25e4067c20 0x7f25e406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.556+0000 7f25fa000640 1 -- 172.21.15.110:0/3124119075 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5227+0+0 (secure 0 0 0) 0x7f25f0087140 con 0x7f2604155970 2024-09-10T10:01:55.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.556+0000 7f2601e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25e4067c20 0x7f25e406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.556+0000 7f2601e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25e4067c20 0x7f25e406a0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f25f40ae000 tx=0x7f25f40a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.560+0000 7f25fa000640 1 -- 172.21.15.110:0/3124119075 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f25f0057a90 con 0x7f2604155970 2024-09-10T10:01:55.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:55.716+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2604143e90 con 0x7f2604155970 2024-09-10T10:01:56.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f25fa000640 1 -- 172.21.15.110:0/3124119075 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) ==== 170+0+0 (secure 0 0 0) 0x7f25f0057390 con 0x7f2604155970 2024-09-10T10:01:56.386 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:01:56.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25e4067c20 msgr2=0x7f25e406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25e4067c20 0x7f25e406a0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f25f40ae000 tx=0x7f25f40a7000 comp rx=0 tx=0).stop 2024-09-10T10:01:56.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 msgr2=0x7f2604158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f2604158460 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f25f000d830 tx=0x7f25f000dcf0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 shutdown_connections 2024-09-10T10:01:56.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f25e4067c20 0x7f25e406a0e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f260415c560 0x7f2604144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2604155970 0x7f2604158460 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 --2- 172.21.15.110:0/3124119075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f260406f010 0x7f26041458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 >> 172.21.15.110:0/3124119075 conn(0x7f260406bc50 msgr2=0x7f260415df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:56.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 shutdown_connections 2024-09-10T10:01:56.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.384+0000 7f2608e00640 1 -- 172.21.15.110:0/3124119075 wait complete. 2024-09-10T10:01:56.424 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:01:56.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb2a5800640 1 Processor -- start 2024-09-10T10:01:56.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb2a5800640 1 -- start start 2024-09-10T10:01:56.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb2a5800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb2a006e850 0x7fb2a006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb2a5800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a00b4960 con 0x7fb2a015b330 2024-09-10T10:01:56.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb2a5800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a00b4c40 con 0x7fb2a00ce4d0 2024-09-10T10:01:56.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb2a5800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a00b5570 con 0x7fb2a006e850 2024-09-10T10:01:56.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb29ea00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb2a015b330 0x7fb2a006e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42886/0 (socket says 172.21.15.110:42886) 2024-09-10T10:01:56.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb29ea00640 1 -- 172.21.15.110:0/3172292350 learned_addr learned my addr 172.21.15.110:0/3172292350 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:56.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb29f400640 1 --2- 172.21.15.110:0/3172292350 >> v2:172.21.15.145:3300/0 conn(0x7fb2a006e850 0x7fb2a006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb29d600640 1 -- 172.21.15.110:0/3172292350 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188984938 0 0) 0x7fb2a00b4960 con 0x7fb2a015b330 2024-09-10T10:01:56.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb29d600640 1 -- 172.21.15.110:0/3172292350 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb284005740 con 0x7fb2a015b330 2024-09-10T10:01:56.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb29f400640 1 -- 172.21.15.110:0/3172292350 >> v1:172.21.15.145:6789/0 conn(0x7fb2a00ce4d0 legacy=0x7fb2a00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb29f400640 1 -- 172.21.15.110:0/3172292350 >> v1:172.21.15.110:6789/0 conn(0x7fb2a015b330 legacy=0x7fb2a006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.560+0000 7fb29f400640 1 -- 172.21.15.110:0/3172292350 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a00b4f20 con 0x7fb2a006e850 2024-09-10T10:01:56.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb29f400640 1 --2- 172.21.15.110:0/3172292350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a006e850 0x7fb2a006c6a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb298009d90 tx=0x7fb29801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7290796f0c13fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:56.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb29d600640 1 -- 172.21.15.110:0/3172292350 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2980b2070 con 0x7fb2a006e850 2024-09-10T10:01:56.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb29d600640 1 -- 172.21.15.110:0/3172292350 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2980a5940 con 0x7fb2a006e850 2024-09-10T10:01:56.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb29d600640 1 -- 172.21.15.110:0/3172292350 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2980a5d10 con 0x7fb2a006e850 2024-09-10T10:01:56.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb2a5800640 1 -- 172.21.15.110:0/3172292350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a006e850 msgr2=0x7fb2a006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb2a5800640 1 --2- 172.21.15.110:0/3172292350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a006e850 0x7fb2a006c6a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb298009d90 tx=0x7fb29801c970 comp rx=0 tx=0).stop 2024-09-10T10:01:56.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb2a5800640 1 -- 172.21.15.110:0/3172292350 shutdown_connections 2024-09-10T10:01:56.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb2a5800640 1 --2- 172.21.15.110:0/3172292350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a006e850 0x7fb2a006c6a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb2a5800640 1 -- 172.21.15.110:0/3172292350 >> 172.21.15.110:0/3172292350 conn(0x7fb2a0062e40 msgr2=0x7fb2a006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:56.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb2a5800640 1 -- 172.21.15.110:0/3172292350 shutdown_connections 2024-09-10T10:01:56.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.564+0000 7fb2a5800640 1 -- 172.21.15.110:0/3172292350 wait complete. 2024-09-10T10:01:56.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 Processor -- start 2024-09-10T10:01:56.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 -- start start 2024-09-10T10:01:56.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a006e850 0x7fb2a00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb2a015b330 0x7fb2a00aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a014e7c0 0x7fb2a0142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a00b3b00 con 0x7fb2a006e850 2024-09-10T10:01:56.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a00b3980 con 0x7fb2a014e7c0 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a00b3c80 con 0x7fb2a015b330 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a006e850 0x7fb2a00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a006e850 0x7fb2a00ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37124/0 (socket says 172.21.15.110:37124) 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 -- 172.21.15.110:0/1374635651 learned_addr learned my addr 172.21.15.110:0/1374635651 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29f400640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a014e7c0 0x7fb2a0142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29e000640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb2a015b330 0x7fb2a00aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 -- 172.21.15.110:0/1374635651 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb2a015b330 msgr2=0x7fb2a00aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb2a015b330 0x7fb2a00aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 -- 172.21.15.110:0/1374635651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a014e7c0 msgr2=0x7fb2a0142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a014e7c0 0x7fb2a0142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 -- 172.21.15.110:0/1374635651 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a01474f0 con 0x7fb2a006e850 2024-09-10T10:01:56.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb29ea00640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a006e850 0x7fb2a00ae840 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb29000c450 tx=0x7fb29000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:56.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb296a00640 1 -- 172.21.15.110:0/1374635651 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290017070 con 0x7fb2a006e850 2024-09-10T10:01:56.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb296a00640 1 -- 172.21.15.110:0/1374635651 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb290005cf0 con 0x7fb2a006e850 2024-09-10T10:01:56.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb296a00640 1 -- 172.21.15.110:0/1374635651 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290007270 con 0x7fb2a006e850 2024-09-10T10:01:56.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a01477d0 con 0x7fb2a006e850 2024-09-10T10:01:56.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.568+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a0147d90 con 0x7fb2a006e850 2024-09-10T10:01:56.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.572+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a006c6a0 con 0x7fb2a006e850 2024-09-10T10:01:56.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.572+0000 7fb296a00640 1 -- 172.21.15.110:0/1374635651 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb29001a070 con 0x7fb2a006e850 2024-09-10T10:01:56.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.572+0000 7fb296a00640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb280067c20 0x7fb28006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.572+0000 7fb296a00640 1 -- 172.21.15.110:0/1374635651 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5609+0+0 (secure 0 0 0) 0x7fb290085910 con 0x7fb2a006e850 2024-09-10T10:01:56.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.572+0000 7fb29e000640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb280067c20 0x7fb28006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.572+0000 7fb29e000640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb280067c20 0x7fb28006a0e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb2a015eb90 tx=0x7fb28c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:56.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.576+0000 7fb296a00640 1 -- 172.21.15.110:0/1374635651 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb290055a20 con 0x7fb2a006e850 2024-09-10T10:01:56.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:56.764+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb2a00cd630 con 0x7fb2a006e850 2024-09-10T10:01:57.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.384+0000 7fb296a00640 1 -- 172.21.15.110:0/1374635651 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7fb290059d20 con 0x7fb2a006e850 2024-09-10T10:01:57.388 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 10 and data pool 11 2024-09-10T10:01:57.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb280067c20 msgr2=0x7fb28006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb280067c20 0x7fb28006a0e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb2a015eb90 tx=0x7fb28c009040 comp rx=0 tx=0).stop 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a006e850 msgr2=0x7fb2a00ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a006e850 0x7fb2a00ae840 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb29000c450 tx=0x7fb29000c910 comp rx=0 tx=0).stop 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 shutdown_connections 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fb280067c20 0x7fb28006a0e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2a014e7c0 0x7fb2a0142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb2a015b330 0x7fb2a00aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 --2- 172.21.15.110:0/1374635651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a006e850 0x7fb2a00ae840 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 >> 172.21.15.110:0/1374635651 conn(0x7fb2a0062e40 msgr2=0x7fb2a006d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 shutdown_connections 2024-09-10T10:01:57.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.388+0000 7fb2a5800640 1 -- 172.21.15.110:0/1374635651 wait complete. 2024-09-10T10:01:57.433 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:57.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.564+0000 7fa1d0200640 1 Processor -- start 2024-09-10T10:01:57.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1d0200640 1 -- start start 2024-09-10T10:01:57.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1d0200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa1c815c960 0x7fa1c80b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1d0200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c80706a0 con 0x7fa1c80d13e0 2024-09-10T10:01:57.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1d0200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c8070a80 con 0x7fa1c81603c0 2024-09-10T10:01:57.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1d0200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c815cd40 con 0x7fa1c815c960 2024-09-10T10:01:57.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1ce200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa1c80d13e0 0x7fa1c8070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42894/0 (socket says 172.21.15.110:42894) 2024-09-10T10:01:57.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1ce200640 1 -- 172.21.15.110:0/1889586791 learned_addr learned my addr 172.21.15.110:0/1889586791 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:57.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1cd800640 1 --2- 172.21.15.110:0/1889586791 >> v2:172.21.15.145:3300/0 conn(0x7fa1c815c960 0x7fa1c80b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1c7e00640 1 -- 172.21.15.110:0/1889586791 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524530914 0 0) 0x7fa1c80706a0 con 0x7fa1c80d13e0 2024-09-10T10:01:57.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1c7e00640 1 -- 172.21.15.110:0/1889586791 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1b0003530 con 0x7fa1c80d13e0 2024-09-10T10:01:57.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1cd800640 1 -- 172.21.15.110:0/1889586791 >> v1:172.21.15.145:6789/0 conn(0x7fa1c81603c0 legacy=0x7fa1c80d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1cd800640 1 -- 172.21.15.110:0/1889586791 >> v1:172.21.15.110:6789/0 conn(0x7fa1c80d13e0 legacy=0x7fa1c8070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.568+0000 7fa1cd800640 1 -- 172.21.15.110:0/1889586791 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c8160910 con 0x7fa1c815c960 2024-09-10T10:01:57.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1cd800640 1 --2- 172.21.15.110:0/1889586791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c815c960 0x7fa1c80b5e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0097c0 tx=0x7fa1bc01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14dee53717b9bdf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1c7e00640 1 -- 172.21.15.110:0/1889586791 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0b0070 con 0x7fa1c815c960 2024-09-10T10:01:57.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1c7e00640 1 -- 172.21.15.110:0/1889586791 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1bc0ab600 con 0x7fa1c815c960 2024-09-10T10:01:57.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1c7e00640 1 -- 172.21.15.110:0/1889586791 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0ab9b0 con 0x7fa1c815c960 2024-09-10T10:01:57.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 -- 172.21.15.110:0/1889586791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c815c960 msgr2=0x7fa1c80b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 --2- 172.21.15.110:0/1889586791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c815c960 0x7fa1c80b5e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0097c0 tx=0x7fa1bc01c380 comp rx=0 tx=0).stop 2024-09-10T10:01:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 -- 172.21.15.110:0/1889586791 shutdown_connections 2024-09-10T10:01:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 --2- 172.21.15.110:0/1889586791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c815c960 0x7fa1c80b5e30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 -- 172.21.15.110:0/1889586791 >> 172.21.15.110:0/1889586791 conn(0x7fa1c80d25c0 msgr2=0x7fa1c80d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 -- 172.21.15.110:0/1889586791 shutdown_connections 2024-09-10T10:01:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 -- 172.21.15.110:0/1889586791 wait complete. 2024-09-10T10:01:57.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 Processor -- start 2024-09-10T10:01:57.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.572+0000 7fa1d0200640 1 -- start start 2024-09-10T10:01:57.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa1c80d13e0 0x7fa1c8157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1c815c960 0x7fa1c81662c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c81603c0 0x7fa1c8151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c80d32d0 con 0x7fa1c815c960 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c80d3150 con 0x7fa1c81603c0 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c8071a80 con 0x7fa1c80d13e0 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1c815c960 0x7fa1c81662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1c815c960 0x7fa1c81662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37132/0 (socket says 172.21.15.110:37132) 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 -- 172.21.15.110:0/3639886174 learned_addr learned my addr 172.21.15.110:0/3639886174 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cd800640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa1c80d13e0 0x7fa1c8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1ce200640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c81603c0 0x7fa1c8151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 -- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa1c80d13e0 msgr2=0x7fa1c8157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa1c80d13e0 0x7fa1c8157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 -- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c81603c0 msgr2=0x7fa1c8151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c81603c0 0x7fa1c8151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 -- 172.21.15.110:0/3639886174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c8166b40 con 0x7fa1c815c960 2024-09-10T10:01:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1ce200640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c81603c0 0x7fa1c8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1cce00640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1c815c960 0x7fa1c81662c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c340 tx=0x7fa1b400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1c5600640 1 -- 172.21.15.110:0/3639886174 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40052d0 con 0x7fa1c815c960 2024-09-10T10:01:57.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1c5600640 1 -- 172.21.15.110:0/3639886174 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b4005470 con 0x7fa1c815c960 2024-09-10T10:01:57.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1c5600640 1 -- 172.21.15.110:0/3639886174 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4007260 con 0x7fa1c815c960 2024-09-10T10:01:57.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c8166e20 con 0x7fa1c815c960 2024-09-10T10:01:57.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c81673e0 con 0x7fa1c815c960 2024-09-10T10:01:57.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.576+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c81677c0 con 0x7fa1c815c960 2024-09-10T10:01:57.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.580+0000 7fa1c5600640 1 -- 172.21.15.110:0/3639886174 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fa1b4007400 con 0x7fa1c815c960 2024-09-10T10:01:57.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.580+0000 7fa1c5600640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa1a8067c20 0x7fa1a806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.580+0000 7fa1c5600640 1 -- 172.21.15.110:0/3639886174 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fa1b4085510 con 0x7fa1c815c960 2024-09-10T10:01:57.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.580+0000 7fa1cd800640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa1a8067c20 0x7fa1a806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.580+0000 7fa1cd800640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa1a8067c20 0x7fa1a806a0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0ad000 tx=0x7fa1bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.584+0000 7fa1c5600640 1 -- 172.21.15.110:0/3639886174 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa1b4055680 con 0x7fa1c815c960 2024-09-10T10:01:57.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.740+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1c80b5e30 con 0x7fa1c815c960 2024-09-10T10:01:57.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.740+0000 7fa1c5600640 1 -- 172.21.15.110:0/3639886174 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16777 (secure 0 0 0) 0x7fa1b4059980 con 0x7fa1c815c960 2024-09-10T10:01:57.745 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:57.745 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":68,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:56.769695+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:54.865555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:55.765126+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:57.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa1a8067c20 msgr2=0x7fa1a806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa1a8067c20 0x7fa1a806a0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0ad000 tx=0x7fa1bc008040 comp rx=0 tx=0).stop 2024-09-10T10:01:57.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1c815c960 msgr2=0x7fa1c81662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1c815c960 0x7fa1c81662c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c340 tx=0x7fa1b400c800 comp rx=0 tx=0).stop 2024-09-10T10:01:57.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 shutdown_connections 2024-09-10T10:01:57.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa1a8067c20 0x7fa1a806a0e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1c81603c0 0x7fa1c8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1c815c960 0x7fa1c81662c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 --2- 172.21.15.110:0/3639886174 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa1c80d13e0 0x7fa1c8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 >> 172.21.15.110:0/3639886174 conn(0x7fa1c80d25c0 msgr2=0x7fa1c80b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 shutdown_connections 2024-09-10T10:01:57.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.744+0000 7fa1d0200640 1 -- 172.21.15.110:0/3639886174 wait complete. 2024-09-10T10:01:57.767 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:01:57.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36bae00640 1 Processor -- start 2024-09-10T10:01:57.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36bae00640 1 -- start start 2024-09-10T10:01:57.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36bae00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f36b40d0cb0 0x7f36b4163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36bae00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b40706a0 con 0x7f36b406eb30 2024-09-10T10:01:57.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36bae00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b40d10f0 con 0x7f36b4073290 2024-09-10T10:01:57.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36bae00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f36b415ee90 con 0x7f36b40d0cb0 2024-09-10T10:01:57.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36b3e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f36b406eb30 0x7f36b4071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:42904/0 (socket says 172.21.15.110:42904) 2024-09-10T10:01:57.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36b3e00640 1 -- 172.21.15.110:0/1160855868 learned_addr learned my addr 172.21.15.110:0/1160855868 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:57.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36b3400640 1 --2- 172.21.15.110:0/1160855868 >> v2:172.21.15.145:3300/0 conn(0x7f36b40d0cb0 0x7f36b4163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36b2a00640 1 -- 172.21.15.110:0/1160855868 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758541700 0 0) 0x7f36b40706a0 con 0x7f36b406eb30 2024-09-10T10:01:57.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36b2a00640 1 -- 172.21.15.110:0/1160855868 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3698003530 con 0x7f36b406eb30 2024-09-10T10:01:57.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36b3400640 1 -- 172.21.15.110:0/1160855868 >> v1:172.21.15.145:6789/0 conn(0x7f36b4073290 legacy=0x7f36b406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36b3400640 1 -- 172.21.15.110:0/1160855868 >> v1:172.21.15.110:6789/0 conn(0x7f36b406eb30 legacy=0x7f36b4071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.900+0000 7f36b3400640 1 -- 172.21.15.110:0/1160855868 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b40d1370 con 0x7f36b40d0cb0 2024-09-10T10:01:57.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36b3400640 1 --2- 172.21.15.110:0/1160855868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b40d0cb0 0x7f36b4163000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f369c0097c0 tx=0x7f369c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8d9ab4891950b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36b2a00640 1 -- 172.21.15.110:0/1160855868 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c0b0070 con 0x7f36b40d0cb0 2024-09-10T10:01:57.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36b2a00640 1 -- 172.21.15.110:0/1160855868 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369c0ab600 con 0x7f36b40d0cb0 2024-09-10T10:01:57.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36b2a00640 1 -- 172.21.15.110:0/1160855868 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c0ab9d0 con 0x7f36b40d0cb0 2024-09-10T10:01:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36bae00640 1 -- 172.21.15.110:0/1160855868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b40d0cb0 msgr2=0x7f36b4163000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36bae00640 1 --2- 172.21.15.110:0/1160855868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b40d0cb0 0x7f36b4163000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f369c0097c0 tx=0x7f369c01c410 comp rx=0 tx=0).stop 2024-09-10T10:01:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36bae00640 1 -- 172.21.15.110:0/1160855868 shutdown_connections 2024-09-10T10:01:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36bae00640 1 --2- 172.21.15.110:0/1160855868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b40d0cb0 0x7f36b4163000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36bae00640 1 -- 172.21.15.110:0/1160855868 >> 172.21.15.110:0/1160855868 conn(0x7f36b40d25c0 msgr2=0x7f36b40d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36bae00640 1 -- 172.21.15.110:0/1160855868 shutdown_connections 2024-09-10T10:01:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.904+0000 7f36bae00640 1 -- 172.21.15.110:0/1160855868 wait complete. 2024-09-10T10:01:57.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 Processor -- start 2024-09-10T10:01:57.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 -- start start 2024-09-10T10:01:57.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b406eb30 0x7f36b40cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4073290 0x7f36b4161c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36b40d0cb0 0x7f36b415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36b406e070 con 0x7f36b4073290 2024-09-10T10:01:57.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4071c70 con 0x7f36b406eb30 2024-09-10T10:01:57.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f36b4073ab0 con 0x7f36b40d0cb0 2024-09-10T10:01:57.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4073290 0x7f36b4161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4073290 0x7f36b4161c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37138/0 (socket says 172.21.15.110:37138) 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 -- 172.21.15.110:0/873015756 learned_addr learned my addr 172.21.15.110:0/873015756 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b8e00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36b40d0cb0 0x7f36b415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3e00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b406eb30 0x7f36b40cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 -- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36b40d0cb0 msgr2=0x7f36b415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36b40d0cb0 0x7f36b415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 -- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b406eb30 msgr2=0x7f36b40cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b406eb30 0x7f36b40cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 -- 172.21.15.110:0/873015756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b40cbe50 con 0x7f36b4073290 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b8e00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36b40d0cb0 0x7f36b415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:57.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36b3400640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4073290 0x7f36b4161c90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f369c01a040 tx=0x7f369c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36abe00640 1 -- 172.21.15.110:0/873015756 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c0b0040 con 0x7f36b4073290 2024-09-10T10:01:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36abe00640 1 -- 172.21.15.110:0/873015756 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369c0bb070 con 0x7f36b4073290 2024-09-10T10:01:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36abe00640 1 -- 172.21.15.110:0/873015756 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c0b6670 con 0x7f36b4073290 2024-09-10T10:01:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b4155f20 con 0x7f36b4073290 2024-09-10T10:01:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b415f970 con 0x7f36b4073290 2024-09-10T10:01:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.908+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b4163000 con 0x7f36b4073290 2024-09-10T10:01:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.912+0000 7f36abe00640 1 -- 172.21.15.110:0/873015756 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f369c0b3070 con 0x7f36b4073290 2024-09-10T10:01:57.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.912+0000 7f36abe00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3690067c50 0x7f369006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:57.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.912+0000 7f36abe00640 1 -- 172.21.15.110:0/873015756 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f369c1208e0 con 0x7f36b4073290 2024-09-10T10:01:57.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.912+0000 7f36b3e00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3690067c50 0x7f369006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:57.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.912+0000 7f36b3e00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3690067c50 0x7f369006a110 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36a4001570 tx=0x7f36a4001a00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:57.916+0000 7f36abe00640 1 -- 172.21.15.110:0/873015756 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f369c016090 con 0x7f36b4073290 2024-09-10T10:01:58.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.072+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36b40b7400 con 0x7f36b4073290 2024-09-10T10:01:58.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.072+0000 7f36abe00640 1 -- 172.21.15.110:0/873015756 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16777 (secure 0 0 0) 0x7f369c0f11b0 con 0x7f36b4073290 2024-09-10T10:01:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:01:58.077 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":68,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:56.769695+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:54.865555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:55.765126+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3690067c50 msgr2=0x7f369006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3690067c50 0x7f369006a110 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36a4001570 tx=0x7f36a4001a00 comp rx=0 tx=0).stop 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4073290 msgr2=0x7f36b4161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4073290 0x7f36b4161c90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f369c01a040 tx=0x7f369c0191b0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 shutdown_connections 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3690067c50 0x7f369006a110 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36b40d0cb0 0x7f36b415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4073290 0x7f36b4161c90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 --2- 172.21.15.110:0/873015756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b406eb30 0x7f36b40cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 >> 172.21.15.110:0/873015756 conn(0x7f36b40d25c0 msgr2=0x7f36b40b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:58.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 shutdown_connections 2024-09-10T10:01:58.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.076+0000 7f36bae00640 1 -- 172.21.15.110:0/873015756 wait complete. 2024-09-10T10:01:58.099 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:01:58.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06b0800640 1 Processor -- start 2024-09-10T10:01:58.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06b0800640 1 -- start start 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06b0800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f06a815b330 0x7f06a806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06b0800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80ce4d0 0x7f06a806e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06b0800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a80b1870 con 0x7f06a80b1c60 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06b0800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f06a80b5570 con 0x7f06a815b330 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06b0800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f06a80b4960 con 0x7f06a80ce4d0 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06ae800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f06a80b1c60 0x7f06a815ef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36850/0 (socket says 172.21.15.110:36850) 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06ae800640 1 -- 172.21.15.110:0/386970855 learned_addr learned my addr 172.21.15.110:0/386970855 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06ade00640 1 --2- 172.21.15.110:0/386970855 >> v2:172.21.15.145:3300/0 conn(0x7f06a815b330 0x7f06a806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06ad400640 1 --2- 172.21.15.110:0/386970855 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80ce4d0 0x7f06a806e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06aca00640 1 -- 172.21.15.110:0/386970855 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577982021 0 0) 0x7f06a80b1870 con 0x7f06a80b1c60 2024-09-10T10:01:58.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06aca00640 1 -- 172.21.15.110:0/386970855 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0690003530 con 0x7f06a80b1c60 2024-09-10T10:01:58.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06aca00640 1 -- 172.21.15.110:0/386970855 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2553044607 0 0) 0x7f0690003530 con 0x7f06a80b1c60 2024-09-10T10:01:58.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06ade00640 1 -- 172.21.15.110:0/386970855 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80ce4d0 msgr2=0x7f06a806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:58.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06ade00640 1 --2- 172.21.15.110:0/386970855 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80ce4d0 0x7f06a806e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06ade00640 1 -- 172.21.15.110:0/386970855 >> v1:172.21.15.110:6789/0 conn(0x7f06a80b1c60 legacy=0x7f06a815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.236+0000 7f06ade00640 1 -- 172.21.15.110:0/386970855 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a80b4c40 con 0x7f06a815b330 2024-09-10T10:01:58.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06ade00640 1 --2- 172.21.15.110:0/386970855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 0x7f06a806c6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f069c0097c0 tx=0x7f069c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57155e251b936d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:58.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06aca00640 1 -- 172.21.15.110:0/386970855 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0b0070 con 0x7f06a815b330 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06aca00640 1 -- 172.21.15.110:0/386970855 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c0ab400 con 0x7f06a815b330 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06aca00640 1 -- 172.21.15.110:0/386970855 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4001a00 con 0x7f06a815b330 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06b0800640 1 -- 172.21.15.110:0/386970855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 msgr2=0x7f06a806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06b0800640 1 --2- 172.21.15.110:0/386970855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 0x7f06a806c6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f069c0097c0 tx=0x7f069c01c3c0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06b0800640 1 -- 172.21.15.110:0/386970855 shutdown_connections 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06b0800640 1 --2- 172.21.15.110:0/386970855 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80ce4d0 0x7f06a806e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06b0800640 1 --2- 172.21.15.110:0/386970855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 0x7f06a806c6a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06b0800640 1 -- 172.21.15.110:0/386970855 >> 172.21.15.110:0/386970855 conn(0x7f06a8062e40 msgr2=0x7f06a806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06b0800640 1 -- 172.21.15.110:0/386970855 shutdown_connections 2024-09-10T10:01:58.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.240+0000 7f06b0800640 1 -- 172.21.15.110:0/386970855 wait complete. 2024-09-10T10:01:58.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 Processor -- start 2024-09-10T10:01:58.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 -- start start 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80b1c60 0x7f06a8150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 0x7f06a815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06a8164480 0x7f06a8164930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8168230 con 0x7f06a8164480 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81680b0 con 0x7f06a815b330 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f06a81683b0 con 0x7f06a80b1c60 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06a8164480 0x7f06a8164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06a8164480 0x7f06a8164930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43994/0 (socket says 172.21.15.110:43994) 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 -- 172.21.15.110:0/35023885 learned_addr learned my addr 172.21.15.110:0/35023885 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ad400640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 0x7f06a815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ade00640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80b1c60 0x7f06a8150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 -- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80b1c60 msgr2=0x7f06a8150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80b1c60 0x7f06a8150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 -- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 msgr2=0x7f06a815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 0x7f06a815d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 -- 172.21.15.110:0/35023885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a815bd20 con 0x7f06a8164480 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ade00640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80b1c60 0x7f06a8150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06ae800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06a8164480 0x7f06a8164930 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f06a4001570 tx=0x7f06a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06a2000640 1 -- 172.21.15.110:0/35023885 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4012400 con 0x7f06a8164480 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06a2000640 1 -- 172.21.15.110:0/35023885 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a40125c0 con 0x7f06a8164480 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06a2000640 1 -- 172.21.15.110:0/35023885 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a40129b0 con 0x7f06a8164480 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a815c000 con 0x7f06a8164480 2024-09-10T10:01:58.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.244+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a815c5c0 con 0x7f06a8164480 2024-09-10T10:01:58.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.248+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a806e140 con 0x7f06a8164480 2024-09-10T10:01:58.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.248+0000 7f06a2000640 1 -- 172.21.15.110:0/35023885 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f06a4012b50 con 0x7f06a8164480 2024-09-10T10:01:58.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.248+0000 7f06a2000640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f068c067c20 0x7f068c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.248+0000 7f06a2000640 1 -- 172.21.15.110:0/35023885 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f06a4086b50 con 0x7f06a8164480 2024-09-10T10:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.248+0000 7f06ade00640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f068c067c20 0x7f068c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.248+0000 7f06ade00640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f068c067c20 0x7f068c06a0e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f069c0ad000 tx=0x7f069c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:58.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.252+0000 7f06a2000640 1 -- 172.21.15.110:0/35023885 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f06a40571d0 con 0x7f06a8164480 2024-09-10T10:01:58.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:58.448+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f06a80b19f0 con 0x7f06a8164480 2024-09-10T10:01:59.375 INFO:tasks.ceph.mds.d.smithi145.stderr:2024-09-10T10:01:59.372+0000 7fa59a600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:01:59.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.408+0000 7f06a2000640 1 -- 172.21.15.110:0/35023885 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f06a4056ad0 con 0x7f06a8164480 2024-09-10T10:01:59.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f068c067c20 msgr2=0x7f068c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f068c067c20 0x7f068c06a0e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f069c0ad000 tx=0x7f069c008040 comp rx=0 tx=0).stop 2024-09-10T10:01:59.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06a8164480 msgr2=0x7f06a8164930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06a8164480 0x7f06a8164930 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f06a4001570 tx=0x7f06a400a430 comp rx=0 tx=0).stop 2024-09-10T10:01:59.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 shutdown_connections 2024-09-10T10:01:59.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f068c067c20 0x7f068c06a0e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06a8164480 0x7f06a8164930 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06a815b330 0x7f06a815d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 --2- 172.21.15.110:0/35023885 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06a80b1c60 0x7f06a8150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 >> 172.21.15.110:0/35023885 conn(0x7f06a8062e40 msgr2=0x7f06a815e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:59.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 shutdown_connections 2024-09-10T10:01:59.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.412+0000 7f06b0800640 1 -- 172.21.15.110:0/35023885 wait complete. 2024-09-10T10:01:59.438 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:01:59.439 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:01:59.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.576+0000 7f0944a00640 1 Processor -- start 2024-09-10T10:01:59.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f0944a00640 1 -- start start 2024-09-10T10:01:59.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f0944a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f094006e850 0x7f094006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f0944a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09400b4960 con 0x7f094015b330 2024-09-10T10:01:59.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f0944a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09400b4c40 con 0x7f09400ce4d0 2024-09-10T10:01:59.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f0944a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f09400b5570 con 0x7f094006e850 2024-09-10T10:01:59.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093de00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f094015b330 0x7f094006e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36858/0 (socket says 172.21.15.110:36858) 2024-09-10T10:01:59.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093de00640 1 -- 172.21.15.110:0/2367452305 learned_addr learned my addr 172.21.15.110:0/2367452305 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:01:59.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093e800640 1 --2- 172.21.15.110:0/2367452305 >> v2:172.21.15.145:3300/0 conn(0x7f094006e850 0x7f094006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2747531874 0 0) 0x7f09400b4960 con 0x7f094015b330 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0924005740 con 0x7f094015b330 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2758761481 0 0) 0x7f0924005740 con 0x7f094015b330 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 >> v2:172.21.15.145:3300/0 conn(0x7f094006e850 msgr2=0x7f094006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 --2- 172.21.15.110:0/2367452305 >> v2:172.21.15.145:3300/0 conn(0x7f094006e850 0x7f094006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 >> v1:172.21.15.145:6789/0 conn(0x7f09400ce4d0 legacy=0x7f09400b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09400b4f20 con 0x7f094015b330 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f0930001a00 con 0x7f094015b330 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 >> v1:172.21.15.110:6789/0 conn(0x7f094015b330 legacy=0x7f094006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0924005bd0 0x7f0924005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0924006a20 0x7f0924006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09240073e0 0x7f092400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f09400b5570 con 0x7f09240073e0 2024-09-10T10:01:59.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f092400e280 con 0x7f0924006a20 2024-09-10T10:01:59.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f092400e480 con 0x7f0924005bd0 2024-09-10T10:01:59.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093e800640 1 --2- 172.21.15.110:0/2367452305 >> v2:172.21.15.145:3300/0 conn(0x7f094006e850 0x7f094006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:59.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093e800640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09240073e0 0x7f092400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093de00640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0924005bd0 0x7f0924005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093d400640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0924006a20 0x7f0924006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093e800640 1 -- 172.21.15.110:0/2367452305 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0924005bd0 msgr2=0x7f0924005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093e800640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0924005bd0 0x7f0924005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093e800640 1 -- 172.21.15.110:0/2367452305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0924006a20 msgr2=0x7f0924006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093e800640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0924006a20 0x7f0924006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.580+0000 7f093e800640 1 -- 172.21.15.110:0/2367452305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0924005740 con 0x7f09240073e0 2024-09-10T10:01:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f093e800640 1 -- 172.21.15.110:0/2367452305 reap_dead start 2024-09-10T10:01:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f093e800640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09240073e0 0x7f092400dbe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f09400b0670 tx=0x7f093801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3d45c5386cb9dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938009880 con 0x7f09240073e0 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09380a58f0 con 0x7f09240073e0 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f093ca00640 1 -- 172.21.15.110:0/2367452305 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380a5cc0 con 0x7f09240073e0 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f0944a00640 1 -- 172.21.15.110:0/2367452305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09240073e0 msgr2=0x7f092400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f0944a00640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09240073e0 0x7f092400dbe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f09400b0670 tx=0x7f093801cc30 comp rx=0 tx=0).stop 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f0944a00640 1 -- 172.21.15.110:0/2367452305 shutdown_connections 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f0944a00640 1 --2- 172.21.15.110:0/2367452305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09240073e0 0x7f092400dbe0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f0944a00640 1 -- 172.21.15.110:0/2367452305 >> 172.21.15.110:0/2367452305 conn(0x7f0940062e40 msgr2=0x7f094006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f0944a00640 1 -- 172.21.15.110:0/2367452305 shutdown_connections 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.584+0000 7f0944a00640 1 -- 172.21.15.110:0/2367452305 wait complete. 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 Processor -- start 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 -- start start 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09240073e0 0x7f094014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09401510a0 0x7f0940140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f09400cf770 0x7f09400cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f09400b1c60 con 0x7f09401510a0 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f094015b710 con 0x7f09240073e0 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f09400b1de0 con 0x7f09400cf770 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09401510a0 0x7f0940140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09401510a0 0x7f0940140650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44014/0 (socket says 172.21.15.110:44014) 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 -- 172.21.15.110:0/2458537750 learned_addr learned my addr 172.21.15.110:0/2458537750 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093de00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09240073e0 0x7f094014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093e800640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f09400cf770 0x7f09400cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 -- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f09400cf770 msgr2=0x7f09400cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f09400cf770 0x7f09400cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 -- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09240073e0 msgr2=0x7f094014aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09240073e0 0x7f094014aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 -- 172.21.15.110:0/2458537750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0940140b90 con 0x7f09401510a0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093e800640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f09400cf770 0x7f09400cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f093d400640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09401510a0 0x7f0940140650 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f092c00f5f0 tx=0x7f092c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0936000640 1 -- 172.21.15.110:0/2458537750 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c0052e0 con 0x7f09401510a0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0936000640 1 -- 172.21.15.110:0/2458537750 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092c018070 con 0x7f09401510a0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0936000640 1 -- 172.21.15.110:0/2458537750 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c003820 con 0x7f09401510a0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09401525e0 con 0x7f09401510a0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0940151820 con 0x7f09401510a0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.588+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094006c6a0 con 0x7f09401510a0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.592+0000 7f0936000640 1 -- 172.21.15.110:0/2458537750 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f092c0039c0 con 0x7f09401510a0 2024-09-10T10:01:59.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.592+0000 7f0936000640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0920067c20 0x7f092006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.592+0000 7f0936000640 1 -- 172.21.15.110:0/2458537750 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f092c085fa0 con 0x7f09401510a0 2024-09-10T10:01:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.592+0000 7f093de00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0920067c20 0x7f092006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.592+0000 7f093de00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0920067c20 0x7f092006a0e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0930004b70 tx=0x7f0930006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.596+0000 7f0936000640 1 -- 172.21.15.110:0/2458537750 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0940140b90 con 0x7f09401510a0 2024-09-10T10:01:59.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:01:59.788+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f094006db00 con 0x7f09401510a0 2024-09-10T10:02:00.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0936000640 1 -- 172.21.15.110:0/2458537750 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f092c0566d0 con 0x7f09401510a0 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0920067c20 msgr2=0x7f092006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0920067c20 0x7f092006a0e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0930004b70 tx=0x7f0930006040 comp rx=0 tx=0).stop 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09401510a0 msgr2=0x7f0940140650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09401510a0 0x7f0940140650 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f092c00f5f0 tx=0x7f092c00fab0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 shutdown_connections 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f0920067c20 0x7f092006a0e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f09400cf770 0x7f09400cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f09401510a0 0x7f0940140650 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 --2- 172.21.15.110:0/2458537750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09240073e0 0x7f094014aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 >> 172.21.15.110:0/2458537750 conn(0x7f0940062e40 msgr2=0x7f09400cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 shutdown_connections 2024-09-10T10:02:00.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.412+0000 7f0944a00640 1 -- 172.21.15.110:0/2458537750 wait complete. 2024-09-10T10:02:00.438 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:02:00.438 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:00.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9fa0a00640 1 Processor -- start 2024-09-10T10:02:00.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9fa0a00640 1 -- start start 2024-09-10T10:02:00.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9fa0a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9f9c15e700 0x7f9f9c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9fa0a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9f9c0729b0 0x7f9f9c072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9fa0a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f9c0703a0 con 0x7f9f9c06ff60 2024-09-10T10:02:00.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9fa0a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c15eae0 con 0x7f9f9c15e700 2024-09-10T10:02:00.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9fa0a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c0af980 con 0x7f9f9c0729b0 2024-09-10T10:02:00.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f99e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9f9c15e700 0x7f9f9c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f99400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9f9c06ff60 0x7f9f9c0af270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36860/0 (socket says 172.21.15.110:36860) 2024-09-10T10:02:00.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f99400640 1 -- 172.21.15.110:0/4257454262 learned_addr learned my addr 172.21.15.110:0/4257454262 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:00.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f9a800640 1 --2- 172.21.15.110:0/4257454262 >> v2:172.21.15.145:3300/0 conn(0x7f9f9c0729b0 0x7f9f9c072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f99e00640 1 -- 172.21.15.110:0/4257454262 >> v2:172.21.15.145:3300/0 conn(0x7f9f9c0729b0 msgr2=0x7f9f9c072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:00.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f99e00640 1 --2- 172.21.15.110:0/4257454262 >> v2:172.21.15.145:3300/0 conn(0x7f9f9c0729b0 0x7f9f9c072dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f99e00640 1 -- 172.21.15.110:0/4257454262 >> v1:172.21.15.110:6789/0 conn(0x7f9f9c06ff60 legacy=0x7f9f9c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f98a00640 1 -- 172.21.15.110:0/4257454262 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704500797 0 0) 0x7f9f88001120 con 0x7f9f9c06ff60 2024-09-10T10:02:00.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f99e00640 1 -- 172.21.15.110:0/4257454262 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c073300 con 0x7f9f9c15e700 2024-09-10T10:02:00.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.572+0000 7f9f9a800640 1 --2- 172.21.15.110:0/4257454262 >> v2:172.21.15.145:3300/0 conn(0x7f9f9c0729b0 0x7f9f9c072dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:00.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9f99e00640 1 --2- 172.21.15.110:0/4257454262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c06e890 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c062890 tx=0x7f9f8c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318553098a089357 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:00.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9f98a00640 1 -- 172.21.15.110:0/4257454262 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0ab650 con 0x7f9f9c15e700 2024-09-10T10:02:00.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9f98a00640 1 -- 172.21.15.110:0/4257454262 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0b0070 con 0x7f9f9c15e700 2024-09-10T10:02:00.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9f98a00640 1 -- 172.21.15.110:0/4257454262 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0aba10 con 0x7f9f9c15e700 2024-09-10T10:02:00.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9fa0a00640 1 -- 172.21.15.110:0/4257454262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 msgr2=0x7f9f9c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/4257454262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c06e890 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c062890 tx=0x7f9f8c01c450 comp rx=0 tx=0).stop 2024-09-10T10:02:00.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9fa0a00640 1 -- 172.21.15.110:0/4257454262 shutdown_connections 2024-09-10T10:02:00.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/4257454262 >> v2:172.21.15.145:3300/0 conn(0x7f9f9c0729b0 0x7f9f9c072dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/4257454262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c06e890 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9fa0a00640 1 -- 172.21.15.110:0/4257454262 >> 172.21.15.110:0/4257454262 conn(0x7f9f9c06cb90 msgr2=0x7f9f9c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:00.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9fa0a00640 1 -- 172.21.15.110:0/4257454262 shutdown_connections 2024-09-10T10:02:00.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.576+0000 7f9fa0a00640 1 -- 172.21.15.110:0/4257454262 wait complete. 2024-09-10T10:02:00.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 Processor -- start 2024-09-10T10:02:00.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 -- start start 2024-09-10T10:02:00.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f9c06ff60 0x7f9f9c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9c0729b0 0x7f9f9c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c1385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c06fa40 con 0x7f9f9c15e700 2024-09-10T10:02:00.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c06f760 con 0x7f9f9c06ff60 2024-09-10T10:02:00.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9c0b4660 con 0x7f9f9c0729b0 2024-09-10T10:02:00.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c1385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c1385b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44034/0 (socket says 172.21.15.110:44034) 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 -- 172.21.15.110:0/648247034 learned_addr learned my addr 172.21.15.110:0/648247034 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f99e00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f9c06ff60 0x7f9f9c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f99400640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9c0729b0 0x7f9f9c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 -- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9c0729b0 msgr2=0x7f9f9c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9c0729b0 0x7f9f9c0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 -- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f9c06ff60 msgr2=0x7f9f9c0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f9c06ff60 0x7f9f9c0cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 -- 172.21.15.110:0/648247034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c0cf510 con 0x7f9f9c15e700 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f99400640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9c0729b0 0x7f9f9c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f9a800640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c1385b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400cef0 tx=0x7f9f9400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:00.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f92000640 1 -- 172.21.15.110:0/648247034 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9401a070 con 0x7f9f9c15e700 2024-09-10T10:02:00.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f92000640 1 -- 172.21.15.110:0/648247034 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9400d860 con 0x7f9f9c15e700 2024-09-10T10:02:00.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9f92000640 1 -- 172.21.15.110:0/648247034 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400dc50 con 0x7f9f9c15e700 2024-09-10T10:02:00.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9c0cf7f0 con 0x7f9f9c15e700 2024-09-10T10:02:00.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9c0cfdb0 con 0x7f9f9c15e700 2024-09-10T10:02:00.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.580+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f9c0b49f0 con 0x7f9f9c15e700 2024-09-10T10:02:00.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.584+0000 7f9f92000640 1 -- 172.21.15.110:0/648247034 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f9f9401d070 con 0x7f9f9c15e700 2024-09-10T10:02:00.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.584+0000 7f9f92000640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9f7c067c20 0x7f9f7c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.584+0000 7f9f92000640 1 -- 172.21.15.110:0/648247034 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f9f94086950 con 0x7f9f9c15e700 2024-09-10T10:02:00.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.584+0000 7f9f99e00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9f7c067c20 0x7f9f7c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.584+0000 7f9f99e00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9f7c067c20 0x7f9f7c06a0e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0af000 tx=0x7f9f8c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:00.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.588+0000 7f9f92000640 1 -- 172.21.15.110:0/648247034 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9f94056b80 con 0x7f9f9c15e700 2024-09-10T10:02:00.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.776+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f9c0afb00 con 0x7f9f9c15e700 2024-09-10T10:02:00.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.776+0000 7f9f92000640 1 -- 172.21.15.110:0/648247034 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5237 (secure 0 0 0) 0x7f9f9405ae80 con 0x7f9f9c15e700 2024-09-10T10:02:00.780 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:00.780 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":45,"btime":"2024-09-10T10:02:00:413261+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:00.781 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 45 2024-09-10T10:02:00.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9f7c067c20 msgr2=0x7f9f7c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9f7c067c20 0x7f9f7c06a0e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0af000 tx=0x7f9f8c0a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:00.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 msgr2=0x7f9f9c1385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c1385b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400cef0 tx=0x7f9f9400a710 comp rx=0 tx=0).stop 2024-09-10T10:02:00.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 shutdown_connections 2024-09-10T10:02:00.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9f7c067c20 0x7f9f7c06a0e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9c15e700 0x7f9f9c1385b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9c0729b0 0x7f9f9c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 --2- 172.21.15.110:0/648247034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f9c06ff60 0x7f9f9c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 >> 172.21.15.110:0/648247034 conn(0x7f9f9c06cb90 msgr2=0x7f9f9c15efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:00.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 shutdown_connections 2024-09-10T10:02:00.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.780+0000 7f9fa0a00640 1 -- 172.21.15.110:0/648247034 wait complete. 2024-09-10T10:02:00.802 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:00.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f4800640 1 Processor -- start 2024-09-10T10:02:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f4800640 1 -- start start 2024-09-10T10:02:00.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f4800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f4800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ec0ceab0 con 0x7f00ec15c330 2024-09-10T10:02:00.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f4800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ec0b0510 con 0x7f00ec0cff20 2024-09-10T10:02:00.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f4800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec0703d0 con 0x7f00ec15c710 2024-09-10T10:02:00.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f1e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f00ec15c330 0x7f00ec06dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36864/0 (socket says 172.21.15.110:36864) 2024-09-10T10:02:00.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f1e00640 1 -- 172.21.15.110:0/1552063213 learned_addr learned my addr 172.21.15.110:0/1552063213 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f1400640 1 --2- 172.21.15.110:0/1552063213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f0a00640 1 -- 172.21.15.110:0/1552063213 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066941135 0 0) 0x7f00ec0ceab0 con 0x7f00ec15c330 2024-09-10T10:02:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f0a00640 1 -- 172.21.15.110:0/1552063213 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00d4003530 con 0x7f00ec15c330 2024-09-10T10:02:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f0a00640 1 -- 172.21.15.110:0/1552063213 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317487570 0 0) 0x7f00ec0b0510 con 0x7f00ec0cff20 2024-09-10T10:02:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f0a00640 1 -- 172.21.15.110:0/1552063213 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00ec0ceab0 con 0x7f00ec0cff20 2024-09-10T10:02:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f1400640 1 -- 172.21.15.110:0/1552063213 >> v1:172.21.15.145:6789/0 conn(0x7f00ec0cff20 legacy=0x7f00ec0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f1400640 1 -- 172.21.15.110:0/1552063213 >> v1:172.21.15.110:6789/0 conn(0x7f00ec15c330 legacy=0x7f00ec06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.936+0000 7f00f1400640 1 -- 172.21.15.110:0/1552063213 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec15d930 con 0x7f00ec15c710 2024-09-10T10:02:00.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f1400640 1 --2- 172.21.15.110:0/1552063213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec06fe90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0097f0 tx=0x7f00dc01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d9611085818a5f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:00.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f0a00640 1 -- 172.21.15.110:0/1552063213 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0aa610 con 0x7f00ec15c710 2024-09-10T10:02:00.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f0a00640 1 -- 172.21.15.110:0/1552063213 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc0af070 con 0x7f00ec15c710 2024-09-10T10:02:00.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f0a00640 1 -- 172.21.15.110:0/1552063213 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0aa9b0 con 0x7f00ec15c710 2024-09-10T10:02:00.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f4800640 1 -- 172.21.15.110:0/1552063213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 msgr2=0x7f00ec06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f4800640 1 --2- 172.21.15.110:0/1552063213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec06fe90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0097f0 tx=0x7f00dc01c4b0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f4800640 1 -- 172.21.15.110:0/1552063213 shutdown_connections 2024-09-10T10:02:00.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f4800640 1 --2- 172.21.15.110:0/1552063213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec06fe90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f4800640 1 -- 172.21.15.110:0/1552063213 >> 172.21.15.110:0/1552063213 conn(0x7f00ec06cb90 msgr2=0x7f00ec06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:00.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f4800640 1 -- 172.21.15.110:0/1552063213 shutdown_connections 2024-09-10T10:02:00.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.940+0000 7f00f4800640 1 -- 172.21.15.110:0/1552063213 wait complete. 2024-09-10T10:02:00.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 Processor -- start 2024-09-10T10:02:00.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 -- start start 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00ec0cff20 0x7f00ec14fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00ec15c330 0x7f00ec157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec15d680 con 0x7f00ec15c330 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec15d500 con 0x7f00ec0cff20 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec0735a0 con 0x7f00ec15c710 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00ec15c330 0x7f00ec157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00ec15c330 0x7f00ec157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44038/0 (socket says 172.21.15.110:44038) 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 -- 172.21.15.110:0/1175637726 learned_addr learned my addr 172.21.15.110:0/1175637726 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:00.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1e00640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00ec0cff20 0x7f00ec14fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f2800640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 -- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 msgr2=0x7f00ec149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 -- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00ec0cff20 msgr2=0x7f00ec14fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00ec0cff20 0x7f00ec14fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 -- 172.21.15.110:0/1175637726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec0b5900 con 0x7f00ec15c330 2024-09-10T10:02:00.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f2800640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:00.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f1400640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00ec15c330 0x7f00ec157fb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0ac040 tx=0x7f00dc019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:00.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00e6000640 1 -- 172.21.15.110:0/1175637726 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc003820 con 0x7f00ec15c330 2024-09-10T10:02:00.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00e6000640 1 -- 172.21.15.110:0/1175637726 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc0039e0 con 0x7f00ec15c330 2024-09-10T10:02:00.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec1564e0 con 0x7f00ec15c330 2024-09-10T10:02:00.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00e6000640 1 -- 172.21.15.110:0/1175637726 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc003dd0 con 0x7f00ec15c330 2024-09-10T10:02:00.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec156aa0 con 0x7f00ec15c330 2024-09-10T10:02:00.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.944+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec0b5ad0 con 0x7f00ec15c330 2024-09-10T10:02:00.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.948+0000 7f00e6000640 1 -- 172.21.15.110:0/1175637726 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f00dc0c2060 con 0x7f00ec15c330 2024-09-10T10:02:00.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.948+0000 7f00e6000640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00d0067c20 0x7f00d006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.948+0000 7f00e6000640 1 -- 172.21.15.110:0/1175637726 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f00dc11fa50 con 0x7f00ec15c330 2024-09-10T10:02:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.948+0000 7f00f1e00640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00d0067c20 0x7f00d006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.948+0000 7f00f1e00640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00d0067c20 0x7f00d006a0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f00e0007fd0 tx=0x7f00e0007d80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:00.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:00.952+0000 7f00e6000640 1 -- 172.21.15.110:0/1175637726 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f00dc0a7070 con 0x7f00ec15c330 2024-09-10T10:02:01.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.108+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00ec06fe90 con 0x7f00ec15c330 2024-09-10T10:02:01.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00e6000640 1 -- 172.21.15.110:0/1175637726 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16764 (secure 0 0 0) 0x7f00dc0160d0 con 0x7f00ec15c330 2024-09-10T10:02:01.114 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:01.114 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":69,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:01:58.397429+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:54.865555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:55.765126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:01.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00d0067c20 msgr2=0x7f00d006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00d0067c20 0x7f00d006a0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f00e0007fd0 tx=0x7f00e0007d80 comp rx=0 tx=0).stop 2024-09-10T10:02:01.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00ec15c330 msgr2=0x7f00ec157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00ec15c330 0x7f00ec157fb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0ac040 tx=0x7f00dc019250 comp rx=0 tx=0).stop 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 shutdown_connections 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f00d0067c20 0x7f00d006a0e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00ec15c710 0x7f00ec149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00ec15c330 0x7f00ec157fb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 --2- 172.21.15.110:0/1175637726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00ec0cff20 0x7f00ec14fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 >> 172.21.15.110:0/1175637726 conn(0x7f00ec06cb90 msgr2=0x7f00ec06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 shutdown_connections 2024-09-10T10:02:01.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.112+0000 7f00f4800640 1 -- 172.21.15.110:0/1175637726 wait complete. 2024-09-10T10:02:01.135 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:02:01.135 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:02:01.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddd2000640 1 Processor -- start 2024-09-10T10:02:01.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddd2000640 1 -- start start 2024-09-10T10:02:01.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddd2000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fddcc155970 0x7fddcc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddd2000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddcc155d50 con 0x7fddcc15c560 2024-09-10T10:02:01.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddd2000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddcc0b34d0 con 0x7fddcc06f1e0 2024-09-10T10:02:01.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddd2000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fddcc0b4570 con 0x7fddcc155970 2024-09-10T10:02:01.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddcaa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fddcc15c560 0x7fddcc06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36868/0 (socket says 172.21.15.110:36868) 2024-09-10T10:02:01.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddcaa00640 1 -- 172.21.15.110:0/1203238042 learned_addr learned my addr 172.21.15.110:0/1203238042 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:01.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddcb400640 1 --2- 172.21.15.110:0/1203238042 >> v2:172.21.15.145:3300/0 conn(0x7fddcc155970 0x7fddcc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddca000640 1 -- 172.21.15.110:0/1203238042 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1120408278 0 0) 0x7fddcc155d50 con 0x7fddcc15c560 2024-09-10T10:02:01.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddca000640 1 -- 172.21.15.110:0/1203238042 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddb0003530 con 0x7fddcc15c560 2024-09-10T10:02:01.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddcb400640 1 -- 172.21.15.110:0/1203238042 >> v1:172.21.15.145:6789/0 conn(0x7fddcc06f1e0 legacy=0x7fddcc0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddcb400640 1 -- 172.21.15.110:0/1203238042 >> v1:172.21.15.110:6789/0 conn(0x7fddcc15c560 legacy=0x7fddcc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.268+0000 7fddcb400640 1 -- 172.21.15.110:0/1203238042 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc0b37b0 con 0x7fddcc155970 2024-09-10T10:02:01.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddcb400640 1 --2- 172.21.15.110:0/1203238042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 0x7fddcc0cc8c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0097c0 tx=0x7fddbc01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d8d6a6068528e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:01.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddca000640 1 -- 172.21.15.110:0/1203238042 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0a7070 con 0x7fddcc155970 2024-09-10T10:02:01.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddca000640 1 -- 172.21.15.110:0/1203238042 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddbc0b0070 con 0x7fddcc155970 2024-09-10T10:02:01.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddca000640 1 -- 172.21.15.110:0/1203238042 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0ab890 con 0x7fddcc155970 2024-09-10T10:02:01.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddd2000640 1 -- 172.21.15.110:0/1203238042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 msgr2=0x7fddcc0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddd2000640 1 --2- 172.21.15.110:0/1203238042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 0x7fddcc0cc8c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0097c0 tx=0x7fddbc01c470 comp rx=0 tx=0).stop 2024-09-10T10:02:01.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddd2000640 1 -- 172.21.15.110:0/1203238042 shutdown_connections 2024-09-10T10:02:01.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddd2000640 1 --2- 172.21.15.110:0/1203238042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 0x7fddcc0cc8c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddd2000640 1 -- 172.21.15.110:0/1203238042 >> 172.21.15.110:0/1203238042 conn(0x7fddcc06bc50 msgr2=0x7fddcc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:01.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddd2000640 1 -- 172.21.15.110:0/1203238042 shutdown_connections 2024-09-10T10:02:01.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.272+0000 7fddd2000640 1 -- 172.21.15.110:0/1203238042 wait complete. 2024-09-10T10:02:01.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 Processor -- start 2024-09-10T10:02:01.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 -- start start 2024-09-10T10:02:01.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc06f1e0 0x7fddcc151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 0x7fddcc0aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddcc15c560 0x7fddcc0aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc0cf340 con 0x7fddcc06f1e0 2024-09-10T10:02:01.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc0cce00 con 0x7fddcc155970 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fddcc0cf4c0 con 0x7fddcc15c560 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc06f1e0 0x7fddcc151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc06f1e0 0x7fddcc151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44046/0 (socket says 172.21.15.110:44046) 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 -- 172.21.15.110:0/291875159 learned_addr learned my addr 172.21.15.110:0/291875159 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcaa00640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 0x7fddcc0aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcbe00640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddcc15c560 0x7fddcc0aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 -- 172.21.15.110:0/291875159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddcc15c560 msgr2=0x7fddcc0aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddcc15c560 0x7fddcc0aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 -- 172.21.15.110:0/291875159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 msgr2=0x7fddcc0aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 0x7fddcc0aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 -- 172.21.15.110:0/291875159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc15cee0 con 0x7fddcc06f1e0 2024-09-10T10:02:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddcb400640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc06f1e0 0x7fddcc151c80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fddbc019080 tx=0x7fddbc01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:01.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddc3400640 1 -- 172.21.15.110:0/291875159 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0bf610 con 0x7fddcc06f1e0 2024-09-10T10:02:01.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddc3400640 1 -- 172.21.15.110:0/291875159 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddbc0bf7d0 con 0x7fddcc06f1e0 2024-09-10T10:02:01.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddc3400640 1 -- 172.21.15.110:0/291875159 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0bfbc0 con 0x7fddcc06f1e0 2024-09-10T10:02:01.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddcc15d220 con 0x7fddcc06f1e0 2024-09-10T10:02:01.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.276+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddcc15da60 con 0x7fddcc06f1e0 2024-09-10T10:02:01.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.280+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddcc0cc8c0 con 0x7fddcc06f1e0 2024-09-10T10:02:01.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.280+0000 7fddc3400640 1 -- 172.21.15.110:0/291875159 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fddbc0bd020 con 0x7fddcc06f1e0 2024-09-10T10:02:01.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.280+0000 7fddc3400640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fddac067c20 0x7fddac06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.280+0000 7fddc3400640 1 -- 172.21.15.110:0/291875159 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fddbc120c70 con 0x7fddcc06f1e0 2024-09-10T10:02:01.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.280+0000 7fddcaa00640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fddac067c20 0x7fddac06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.280+0000 7fddcaa00640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fddac067c20 0x7fddac06a0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fddcc150c40 tx=0x7fddb8007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:01.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.284+0000 7fddc3400640 1 -- 172.21.15.110:0/291875159 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fddcc15d220 con 0x7fddcc06f1e0 2024-09-10T10:02:01.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.444+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddcc15d400 con 0x7fddac067c20 2024-09-10T10:02:01.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.452+0000 7fddc3400640 1 -- 172.21.15.110:0/291875159 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251428 (secure 0 0 0) 0x7fddb8005860 con 0x7fddac067c20 2024-09-10T10:02:01.456 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:02:01.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.452+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fddac067c20 msgr2=0x7fddac06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.452+0000 7fddd2000640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fddac067c20 0x7fddac06a0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fddcc150c40 tx=0x7fddb8007d40 comp rx=0 tx=0).stop 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.452+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc06f1e0 msgr2=0x7fddcc151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.452+0000 7fddd2000640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc06f1e0 0x7fddcc151c80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fddbc019080 tx=0x7fddbc01acb0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.452+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 shutdown_connections 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.452+0000 7fddd2000640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fddac067c20 0x7fddac06a0e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.456+0000 7fddd2000640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddcc15c560 0x7fddcc0aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.456+0000 7fddd2000640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddcc155970 0x7fddcc0aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.456+0000 7fddd2000640 1 --2- 172.21.15.110:0/291875159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc06f1e0 0x7fddcc151c80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.456+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 >> 172.21.15.110:0/291875159 conn(0x7fddcc06bc50 msgr2=0x7fddcc0710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.456+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 shutdown_connections 2024-09-10T10:02:01.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.456+0000 7fddd2000640 1 -- 172.21.15.110:0/291875159 wait complete. 2024-09-10T10:02:01.477 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-10T10:02:01.402066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248728,"kb_used_data":7520,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725992,"statfs":{"total":773094113280,"available":772839415808,"internally_reserved":0,"allocated":7700480,"data_stored":3865801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":246795820},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002773"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431713+0000","last_change":"2024-09-10T10:01:57.425627+0000","last_active":"2024-09-10T10:01:57.431713+0000","last_peered":"2024-09-10T10:01:57.431713+0000","last_clean":"2024-09-10T10:01:57.431713+0000","last_became_active":"2024-09-10T10:01:57.425083+0000","last_became_peered":"2024-09-10T10:01:57.425083+0000","last_unstale":"2024-09-10T10:01:57.431713+0000","last_undegraded":"2024-09-10T10:01:57.431713+0000","last_fullsized":"2024-09-10T10:01:57.431713+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419573+0000","last_change":"2024-09-10T10:01:57.410364+0000","last_active":"2024-09-10T10:01:57.419573+0000","last_peered":"2024-09-10T10:01:57.419573+0000","last_clean":"2024-09-10T10:01:57.419573+0000","last_became_active":"2024-09-10T10:01:57.410086+0000","last_became_peered":"2024-09-10T10:01:57.410086+0000","last_unstale":"2024-09-10T10:01:57.419573+0000","last_undegraded":"2024-09-10T10:01:57.419573+0000","last_fullsized":"2024-09-10T10:01:57.419573+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.839704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.397555+0000","last_change":"2024-09-10T10:01:56.397555+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.397555+0000","last_undegraded":"2024-09-10T10:01:56.397555+0000","last_fullsized":"2024-09-10T10:01:56.397555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422592+0000","last_change":"2024-09-10T10:01:57.419086+0000","last_active":"2024-09-10T10:01:57.422592+0000","last_peered":"2024-09-10T10:01:57.422592+0000","last_clean":"2024-09-10T10:01:57.422592+0000","last_became_active":"2024-09-10T10:01:57.418640+0000","last_became_peered":"2024-09-10T10:01:57.418640+0000","last_unstale":"2024-09-10T10:01:57.422592+0000","last_undegraded":"2024-09-10T10:01:57.422592+0000","last_fullsized":"2024-09-10T10:01:57.422592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:37.232883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417356+0000","last_change":"2024-09-10T10:01:57.412199+0000","last_active":"2024-09-10T10:01:57.417356+0000","last_peered":"2024-09-10T10:01:57.417356+0000","last_clean":"2024-09-10T10:01:57.417356+0000","last_became_active":"2024-09-10T10:01:57.411926+0000","last_became_peered":"2024-09-10T10:01:57.411926+0000","last_unstale":"2024-09-10T10:01:57.417356+0000","last_undegraded":"2024-09-10T10:01:57.417356+0000","last_fullsized":"2024-09-10T10:01:57.417356+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:50.877508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421654+0000","last_change":"2024-09-10T10:01:57.411801+0000","last_active":"2024-09-10T10:01:57.421654+0000","last_peered":"2024-09-10T10:01:57.421654+0000","last_clean":"2024-09-10T10:01:57.421654+0000","last_became_active":"2024-09-10T10:01:57.411506+0000","last_became_peered":"2024-09-10T10:01:57.411506+0000","last_unstale":"2024-09-10T10:01:57.421654+0000","last_undegraded":"2024-09-10T10:01:57.421654+0000","last_fullsized":"2024-09-10T10:01:57.421654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:53.511190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431828+0000","last_change":"2024-09-10T10:01:57.427391+0000","last_active":"2024-09-10T10:01:57.431828+0000","last_peered":"2024-09-10T10:01:57.431828+0000","last_clean":"2024-09-10T10:01:57.431828+0000","last_became_active":"2024-09-10T10:01:57.427095+0000","last_became_peered":"2024-09-10T10:01:57.427095+0000","last_unstale":"2024-09-10T10:01:57.431828+0000","last_undegraded":"2024-09-10T10:01:57.431828+0000","last_fullsized":"2024-09-10T10:01:57.431828+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414913+0000","last_change":"2024-09-10T10:01:57.405094+0000","last_active":"2024-09-10T10:01:57.414913+0000","last_peered":"2024-09-10T10:01:57.414913+0000","last_clean":"2024-09-10T10:01:57.414913+0000","last_became_active":"2024-09-10T10:01:57.404803+0000","last_became_peered":"2024-09-10T10:01:57.404803+0000","last_unstale":"2024-09-10T10:01:57.414913+0000","last_undegraded":"2024-09-10T10:01:57.414913+0000","last_fullsized":"2024-09-10T10:01:57.414913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:51.428680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419139+0000","last_change":"2024-09-10T10:01:57.414814+0000","last_active":"2024-09-10T10:01:57.419139+0000","last_peered":"2024-09-10T10:01:57.419139+0000","last_clean":"2024-09-10T10:01:57.419139+0000","last_became_active":"2024-09-10T10:01:57.414440+0000","last_became_peered":"2024-09-10T10:01:57.414440+0000","last_unstale":"2024-09-10T10:01:57.419139+0000","last_undegraded":"2024-09-10T10:01:57.419139+0000","last_fullsized":"2024-09-10T10:01:57.419139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:04.748497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415584+0000","last_change":"2024-09-10T10:01:57.411661+0000","last_active":"2024-09-10T10:01:57.415584+0000","last_peered":"2024-09-10T10:01:57.415584+0000","last_clean":"2024-09-10T10:01:57.415584+0000","last_became_active":"2024-09-10T10:01:57.410577+0000","last_became_peered":"2024-09-10T10:01:57.410577+0000","last_unstale":"2024-09-10T10:01:57.415584+0000","last_undegraded":"2024-09-10T10:01:57.415584+0000","last_fullsized":"2024-09-10T10:01:57.415584+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:53.511190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423038+0000","last_change":"2024-09-10T10:01:57.420979+0000","last_active":"2024-09-10T10:01:57.423038+0000","last_peered":"2024-09-10T10:01:57.423038+0000","last_clean":"2024-09-10T10:01:57.423038+0000","last_became_active":"2024-09-10T10:01:57.420695+0000","last_became_peered":"2024-09-10T10:01:57.420695+0000","last_unstale":"2024-09-10T10:01:57.423038+0000","last_undegraded":"2024-09-10T10:01:57.423038+0000","last_fullsized":"2024-09-10T10:01:57.423038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413485+0000","last_change":"2024-09-10T10:01:57.403883+0000","last_active":"2024-09-10T10:01:57.413485+0000","last_peered":"2024-09-10T10:01:57.413485+0000","last_clean":"2024-09-10T10:01:57.413485+0000","last_became_active":"2024-09-10T10:01:57.403600+0000","last_became_peered":"2024-09-10T10:01:57.403600+0000","last_unstale":"2024-09-10T10:01:57.413485+0000","last_undegraded":"2024-09-10T10:01:57.413485+0000","last_fullsized":"2024-09-10T10:01:57.413485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:53.214945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415178+0000","last_change":"2024-09-10T10:01:57.409326+0000","last_active":"2024-09-10T10:01:57.415178+0000","last_peered":"2024-09-10T10:01:57.415178+0000","last_clean":"2024-09-10T10:01:57.415178+0000","last_became_active":"2024-09-10T10:01:57.409053+0000","last_became_peered":"2024-09-10T10:01:57.409053+0000","last_unstale":"2024-09-10T10:01:57.415178+0000","last_undegraded":"2024-09-10T10:01:57.415178+0000","last_fullsized":"2024-09-10T10:01:57.415178+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424665+0000","last_change":"2024-09-10T10:01:57.421822+0000","last_active":"2024-09-10T10:01:57.424665+0000","last_peered":"2024-09-10T10:01:57.424665+0000","last_clean":"2024-09-10T10:01:57.424665+0000","last_became_active":"2024-09-10T10:01:57.421559+0000","last_became_peered":"2024-09-10T10:01:57.421559+0000","last_unstale":"2024-09-10T10:01:57.424665+0000","last_undegraded":"2024-09-10T10:01:57.424665+0000","last_fullsized":"2024-09-10T10:01:57.424665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:10.788546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414952+0000","last_change":"2024-09-10T10:01:57.410832+0000","last_active":"2024-09-10T10:01:57.414952+0000","last_peered":"2024-09-10T10:01:57.414952+0000","last_clean":"2024-09-10T10:01:57.414952+0000","last_became_active":"2024-09-10T10:01:57.410575+0000","last_became_peered":"2024-09-10T10:01:57.410575+0000","last_unstale":"2024-09-10T10:01:57.414952+0000","last_undegraded":"2024-09-10T10:01:57.414952+0000","last_fullsized":"2024-09-10T10:01:57.414952+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421152+0000","last_change":"2024-09-10T10:01:57.419056+0000","last_active":"2024-09-10T10:01:57.421152+0000","last_peered":"2024-09-10T10:01:57.421152+0000","last_clean":"2024-09-10T10:01:57.421152+0000","last_became_active":"2024-09-10T10:01:57.418704+0000","last_became_peered":"2024-09-10T10:01:57.418704+0000","last_unstale":"2024-09-10T10:01:57.421152+0000","last_undegraded":"2024-09-10T10:01:57.421152+0000","last_fullsized":"2024-09-10T10:01:57.421152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:58.162428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419118+0000","last_change":"2024-09-10T10:01:57.414713+0000","last_active":"2024-09-10T10:01:57.419118+0000","last_peered":"2024-09-10T10:01:57.419118+0000","last_clean":"2024-09-10T10:01:57.419118+0000","last_became_active":"2024-09-10T10:01:57.414227+0000","last_became_peered":"2024-09-10T10:01:57.414227+0000","last_unstale":"2024-09-10T10:01:57.419118+0000","last_undegraded":"2024-09-10T10:01:57.419118+0000","last_fullsized":"2024-09-10T10:01:57.419118+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:06.958521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.418366+0000","last_change":"2024-09-10T10:01:57.407315+0000","last_active":"2024-09-10T10:01:57.418366+0000","last_peered":"2024-09-10T10:01:57.418366+0000","last_clean":"2024-09-10T10:01:57.418366+0000","last_became_active":"2024-09-10T10:01:57.406568+0000","last_became_peered":"2024-09-10T10:01:57.406568+0000","last_unstale":"2024-09-10T10:01:57.418366+0000","last_undegraded":"2024-09-10T10:01:57.418366+0000","last_fullsized":"2024-09-10T10:01:57.418366+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:02.156822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414529+0000","last_change":"2024-09-10T10:01:57.405757+0000","last_active":"2024-09-10T10:01:57.414529+0000","last_peered":"2024-09-10T10:01:57.414529+0000","last_clean":"2024-09-10T10:01:57.414529+0000","last_became_active":"2024-09-10T10:01:57.404501+0000","last_became_peered":"2024-09-10T10:01:57.404501+0000","last_unstale":"2024-09-10T10:01:57.414529+0000","last_undegraded":"2024-09-10T10:01:57.414529+0000","last_fullsized":"2024-09-10T10:01:57.414529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:03.020211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424741+0000","last_change":"2024-09-10T10:01:57.423165+0000","last_active":"2024-09-10T10:01:57.424741+0000","last_peered":"2024-09-10T10:01:57.424741+0000","last_clean":"2024-09-10T10:01:57.424741+0000","last_became_active":"2024-09-10T10:01:57.422527+0000","last_became_peered":"2024-09-10T10:01:57.422527+0000","last_unstale":"2024-09-10T10:01:57.424741+0000","last_undegraded":"2024-09-10T10:01:57.424741+0000","last_fullsized":"2024-09-10T10:01:57.424741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417175+0000","last_change":"2024-09-10T10:01:57.409584+0000","last_active":"2024-09-10T10:01:57.417175+0000","last_peered":"2024-09-10T10:01:57.417175+0000","last_clean":"2024-09-10T10:01:57.417175+0000","last_became_active":"2024-09-10T10:01:57.409327+0000","last_became_peered":"2024-09-10T10:01:57.409327+0000","last_unstale":"2024-09-10T10:01:57.417175+0000","last_undegraded":"2024-09-10T10:01:57.417175+0000","last_fullsized":"2024-09-10T10:01:57.417175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:39.030596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419510+0000","last_change":"2024-09-10T10:01:57.416054+0000","last_active":"2024-09-10T10:01:57.419510+0000","last_peered":"2024-09-10T10:01:57.419510+0000","last_clean":"2024-09-10T10:01:57.419510+0000","last_became_active":"2024-09-10T10:01:57.415813+0000","last_became_peered":"2024-09-10T10:01:57.415813+0000","last_unstale":"2024-09-10T10:01:57.419510+0000","last_undegraded":"2024-09-10T10:01:57.419510+0000","last_fullsized":"2024-09-10T10:01:57.419510+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:11.341717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431987+0000","last_change":"2024-09-10T10:01:57.430689+0000","last_active":"2024-09-10T10:01:57.431987+0000","last_peered":"2024-09-10T10:01:57.431987+0000","last_clean":"2024-09-10T10:01:57.431987+0000","last_became_active":"2024-09-10T10:01:57.430349+0000","last_became_peered":"2024-09-10T10:01:57.430349+0000","last_unstale":"2024-09-10T10:01:57.431987+0000","last_undegraded":"2024-09-10T10:01:57.431987+0000","last_fullsized":"2024-09-10T10:01:57.431987+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:41.204498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431524+0000","last_change":"2024-09-10T10:01:57.424818+0000","last_active":"2024-09-10T10:01:57.431524+0000","last_peered":"2024-09-10T10:01:57.431524+0000","last_clean":"2024-09-10T10:01:57.431524+0000","last_became_active":"2024-09-10T10:01:57.424469+0000","last_became_peered":"2024-09-10T10:01:57.424469+0000","last_unstale":"2024-09-10T10:01:57.431524+0000","last_undegraded":"2024-09-10T10:01:57.431524+0000","last_fullsized":"2024-09-10T10:01:57.431524+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415184+0000","last_change":"2024-09-10T10:01:57.412902+0000","last_active":"2024-09-10T10:01:57.415184+0000","last_peered":"2024-09-10T10:01:57.415184+0000","last_clean":"2024-09-10T10:01:57.415184+0000","last_became_active":"2024-09-10T10:01:57.412071+0000","last_became_peered":"2024-09-10T10:01:57.412071+0000","last_unstale":"2024-09-10T10:01:57.415184+0000","last_undegraded":"2024-09-10T10:01:57.415184+0000","last_fullsized":"2024-09-10T10:01:57.415184+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:47.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431901+0000","last_change":"2024-09-10T10:01:57.428212+0000","last_active":"2024-09-10T10:01:57.431901+0000","last_peered":"2024-09-10T10:01:57.431901+0000","last_clean":"2024-09-10T10:01:57.431901+0000","last_became_active":"2024-09-10T10:01:57.427898+0000","last_became_peered":"2024-09-10T10:01:57.427898+0000","last_unstale":"2024-09-10T10:01:57.431901+0000","last_undegraded":"2024-09-10T10:01:57.431901+0000","last_fullsized":"2024-09-10T10:01:57.431901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.395624+0000","last_change":"2024-09-10T10:01:56.395624+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.395624+0000","last_undegraded":"2024-09-10T10:01:56.395624+0000","last_fullsized":"2024-09-10T10:01:56.395624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415287+0000","last_change":"2024-09-10T10:01:57.411461+0000","last_active":"2024-09-10T10:01:57.415287+0000","last_peered":"2024-09-10T10:01:57.415287+0000","last_clean":"2024-09-10T10:01:57.415287+0000","last_became_active":"2024-09-10T10:01:57.410729+0000","last_became_peered":"2024-09-10T10:01:57.410729+0000","last_unstale":"2024-09-10T10:01:57.415287+0000","last_undegraded":"2024-09-10T10:01:57.415287+0000","last_fullsized":"2024-09-10T10:01:57.415287+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.839704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394962+0000","last_change":"2024-09-10T10:01:56.394962+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394962+0000","last_undegraded":"2024-09-10T10:01:56.394962+0000","last_fullsized":"2024-09-10T10:01:56.394962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431869+0000","last_change":"2024-09-10T10:01:57.428097+0000","last_active":"2024-09-10T10:01:57.431869+0000","last_peered":"2024-09-10T10:01:57.431869+0000","last_clean":"2024-09-10T10:01:57.431869+0000","last_became_active":"2024-09-10T10:01:57.427838+0000","last_became_peered":"2024-09-10T10:01:57.427838+0000","last_unstale":"2024-09-10T10:01:57.431869+0000","last_undegraded":"2024-09-10T10:01:57.431869+0000","last_fullsized":"2024-09-10T10:01:57.431869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.426209+0000","last_change":"2024-09-10T10:01:57.416701+0000","last_active":"2024-09-10T10:01:57.426209+0000","last_peered":"2024-09-10T10:01:57.426209+0000","last_clean":"2024-09-10T10:01:57.426209+0000","last_became_active":"2024-09-10T10:01:57.416416+0000","last_became_peered":"2024-09-10T10:01:57.416416+0000","last_unstale":"2024-09-10T10:01:57.426209+0000","last_undegraded":"2024-09-10T10:01:57.426209+0000","last_fullsized":"2024-09-10T10:01:57.426209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.508652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419651+0000","last_change":"2024-09-10T10:01:57.412058+0000","last_active":"2024-09-10T10:01:57.419651+0000","last_peered":"2024-09-10T10:01:57.419651+0000","last_clean":"2024-09-10T10:01:57.419651+0000","last_became_active":"2024-09-10T10:01:57.411787+0000","last_became_peered":"2024-09-10T10:01:57.411787+0000","last_unstale":"2024-09-10T10:01:57.419651+0000","last_undegraded":"2024-09-10T10:01:57.419651+0000","last_fullsized":"2024-09-10T10:01:57.419651+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:59.314051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431681+0000","last_change":"2024-09-10T10:01:57.426610+0000","last_active":"2024-09-10T10:01:57.431681+0000","last_peered":"2024-09-10T10:01:57.431681+0000","last_clean":"2024-09-10T10:01:57.431681+0000","last_became_active":"2024-09-10T10:01:57.423435+0000","last_became_peered":"2024-09-10T10:01:57.423435+0000","last_unstale":"2024-09-10T10:01:57.431681+0000","last_undegraded":"2024-09-10T10:01:57.431681+0000","last_fullsized":"2024-09-10T10:01:57.431681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.428911+0000","last_change":"2024-09-10T10:01:56.414321+0000","last_active":"2024-09-10T10:01:57.428911+0000","last_peered":"2024-09-10T10:01:57.428911+0000","last_clean":"2024-09-10T10:01:57.428911+0000","last_became_active":"2024-09-10T10:01:56.414030+0000","last_became_peered":"2024-09-10T10:01:56.414030+0000","last_unstale":"2024-09-10T10:01:57.428911+0000","last_undegraded":"2024-09-10T10:01:57.428911+0000","last_fullsized":"2024-09-10T10:01:57.428911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419094+0000","last_change":"2024-09-10T10:01:57.414555+0000","last_active":"2024-09-10T10:01:57.419094+0000","last_peered":"2024-09-10T10:01:57.419094+0000","last_clean":"2024-09-10T10:01:57.419094+0000","last_became_active":"2024-09-10T10:01:57.413819+0000","last_became_peered":"2024-09-10T10:01:57.413819+0000","last_unstale":"2024-09-10T10:01:57.419094+0000","last_undegraded":"2024-09-10T10:01:57.419094+0000","last_fullsized":"2024-09-10T10:01:57.419094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:39.835600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423916+0000","last_change":"2024-09-10T10:01:56.408386+0000","last_active":"2024-09-10T10:01:57.423916+0000","last_peered":"2024-09-10T10:01:57.423916+0000","last_clean":"2024-09-10T10:01:57.423916+0000","last_became_active":"2024-09-10T10:01:56.408127+0000","last_became_peered":"2024-09-10T10:01:56.408127+0000","last_unstale":"2024-09-10T10:01:57.423916+0000","last_undegraded":"2024-09-10T10:01:57.423916+0000","last_fullsized":"2024-09-10T10:01:57.423916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:59.865257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421159+0000","last_change":"2024-09-10T10:01:57.417292+0000","last_active":"2024-09-10T10:01:57.421159+0000","last_peered":"2024-09-10T10:01:57.421159+0000","last_clean":"2024-09-10T10:01:57.421159+0000","last_became_active":"2024-09-10T10:01:57.416917+0000","last_became_peered":"2024-09-10T10:01:57.416917+0000","last_unstale":"2024-09-10T10:01:57.421159+0000","last_undegraded":"2024-09-10T10:01:57.421159+0000","last_fullsized":"2024-09-10T10:01:57.421159+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:11.341717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422155+0000","last_change":"2024-09-10T10:01:56.409023+0000","last_active":"2024-09-10T10:01:57.422155+0000","last_peered":"2024-09-10T10:01:57.422155+0000","last_clean":"2024-09-10T10:01:57.422155+0000","last_became_active":"2024-09-10T10:01:56.408748+0000","last_became_peered":"2024-09-10T10:01:56.408748+0000","last_unstale":"2024-09-10T10:01:57.422155+0000","last_undegraded":"2024-09-10T10:01:57.422155+0000","last_fullsized":"2024-09-10T10:01:57.422155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:50.426131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413959+0000","last_change":"2024-09-10T10:01:57.408727+0000","last_active":"2024-09-10T10:01:57.413959+0000","last_peered":"2024-09-10T10:01:57.413959+0000","last_clean":"2024-09-10T10:01:57.413959+0000","last_became_active":"2024-09-10T10:01:57.408448+0000","last_became_peered":"2024-09-10T10:01:57.408448+0000","last_unstale":"2024-09-10T10:01:57.413959+0000","last_undegraded":"2024-09-10T10:01:57.413959+0000","last_fullsized":"2024-09-10T10:01:57.413959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427217+0000","last_change":"2024-09-10T10:01:56.409387+0000","last_active":"2024-09-10T10:01:57.427217+0000","last_peered":"2024-09-10T10:01:57.427217+0000","last_clean":"2024-09-10T10:01:57.427217+0000","last_became_active":"2024-09-10T10:01:56.407044+0000","last_became_peered":"2024-09-10T10:01:56.407044+0000","last_unstale":"2024-09-10T10:01:57.427217+0000","last_undegraded":"2024-09-10T10:01:57.427217+0000","last_fullsized":"2024-09-10T10:01:57.427217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423211+0000","last_change":"2024-09-10T10:01:57.420630+0000","last_active":"2024-09-10T10:01:57.423211+0000","last_peered":"2024-09-10T10:01:57.423211+0000","last_clean":"2024-09-10T10:01:57.423211+0000","last_became_active":"2024-09-10T10:01:57.416989+0000","last_became_peered":"2024-09-10T10:01:57.416989+0000","last_unstale":"2024-09-10T10:01:57.423211+0000","last_undegraded":"2024-09-10T10:01:57.423211+0000","last_fullsized":"2024-09-10T10:01:57.423211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:14.901947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.407559+0000","last_change":"2024-09-10T10:01:56.404458+0000","last_active":"2024-09-10T10:01:56.407559+0000","last_peered":"2024-09-10T10:01:56.407559+0000","last_clean":"2024-09-10T10:01:56.407559+0000","last_became_active":"2024-09-10T10:01:56.404205+0000","last_became_peered":"2024-09-10T10:01:56.404205+0000","last_unstale":"2024-09-10T10:01:56.407559+0000","last_undegraded":"2024-09-10T10:01:56.407559+0000","last_fullsized":"2024-09-10T10:01:56.407559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:14.034052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417285+0000","last_change":"2024-09-10T10:01:57.412841+0000","last_active":"2024-09-10T10:01:57.417285+0000","last_peered":"2024-09-10T10:01:57.417285+0000","last_clean":"2024-09-10T10:01:57.417285+0000","last_became_active":"2024-09-10T10:01:57.412551+0000","last_became_peered":"2024-09-10T10:01:57.412551+0000","last_unstale":"2024-09-10T10:01:57.417285+0000","last_undegraded":"2024-09-10T10:01:57.417285+0000","last_fullsized":"2024-09-10T10:01:57.417285+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:59.314051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421685+0000","last_change":"2024-09-10T10:01:56.398879+0000","last_active":"2024-09-10T10:01:57.421685+0000","last_peered":"2024-09-10T10:01:57.421685+0000","last_clean":"2024-09-10T10:01:57.421685+0000","last_became_active":"2024-09-10T10:01:56.398576+0000","last_became_peered":"2024-09-10T10:01:56.398576+0000","last_unstale":"2024-09-10T10:01:57.421685+0000","last_undegraded":"2024-09-10T10:01:57.421685+0000","last_fullsized":"2024-09-10T10:01:57.421685+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:01.154272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417679+0000","last_change":"2024-09-10T10:01:57.412438+0000","last_active":"2024-09-10T10:01:57.417679+0000","last_peered":"2024-09-10T10:01:57.417679+0000","last_clean":"2024-09-10T10:01:57.417679+0000","last_became_active":"2024-09-10T10:01:57.412174+0000","last_became_peered":"2024-09-10T10:01:57.412174+0000","last_unstale":"2024-09-10T10:01:57.417679+0000","last_undegraded":"2024-09-10T10:01:57.417679+0000","last_fullsized":"2024-09-10T10:01:57.417679+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:46.178171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.391826+0000","last_change":"2024-09-10T10:01:56.406654+0000","last_active":"2024-09-10T10:01:57.391826+0000","last_peered":"2024-09-10T10:01:57.391826+0000","last_clean":"2024-09-10T10:01:57.391826+0000","last_became_active":"2024-09-10T10:01:56.406375+0000","last_became_peered":"2024-09-10T10:01:56.406375+0000","last_unstale":"2024-09-10T10:01:57.391826+0000","last_undegraded":"2024-09-10T10:01:57.391826+0000","last_fullsized":"2024-09-10T10:01:57.391826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:07.017690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414349+0000","last_change":"2024-09-10T10:01:57.408804+0000","last_active":"2024-09-10T10:01:57.414349+0000","last_peered":"2024-09-10T10:01:57.414349+0000","last_clean":"2024-09-10T10:01:57.414349+0000","last_became_active":"2024-09-10T10:01:57.405976+0000","last_became_peered":"2024-09-10T10:01:57.405976+0000","last_unstale":"2024-09-10T10:01:57.414349+0000","last_undegraded":"2024-09-10T10:01:57.414349+0000","last_fullsized":"2024-09-10T10:01:57.414349+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398400+0000","last_change":"2024-09-10T10:01:56.411631+0000","last_active":"2024-09-10T10:01:57.398400+0000","last_peered":"2024-09-10T10:01:57.398400+0000","last_clean":"2024-09-10T10:01:57.398400+0000","last_became_active":"2024-09-10T10:01:56.411378+0000","last_became_peered":"2024-09-10T10:01:56.411378+0000","last_unstale":"2024-09-10T10:01:57.398400+0000","last_undegraded":"2024-09-10T10:01:57.398400+0000","last_fullsized":"2024-09-10T10:01:57.398400+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:38.118765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424610+0000","last_change":"2024-09-10T10:01:57.422782+0000","last_active":"2024-09-10T10:01:57.424610+0000","last_peered":"2024-09-10T10:01:57.424610+0000","last_clean":"2024-09-10T10:01:57.424610+0000","last_became_active":"2024-09-10T10:01:57.420706+0000","last_became_peered":"2024-09-10T10:01:57.420706+0000","last_unstale":"2024-09-10T10:01:57.424610+0000","last_undegraded":"2024-09-10T10:01:57.424610+0000","last_fullsized":"2024-09-10T10:01:57.424610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398697+0000","last_change":"2024-09-10T10:01:56.403288+0000","last_active":"2024-09-10T10:01:57.398697+0000","last_peered":"2024-09-10T10:01:57.398697+0000","last_clean":"2024-09-10T10:01:57.398697+0000","last_became_active":"2024-09-10T10:01:56.402990+0000","last_became_peered":"2024-09-10T10:01:56.402990+0000","last_unstale":"2024-09-10T10:01:57.398697+0000","last_undegraded":"2024-09-10T10:01:57.398697+0000","last_fullsized":"2024-09-10T10:01:57.398697+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431695+0000","last_change":"2024-09-10T10:01:57.425385+0000","last_active":"2024-09-10T10:01:57.431695+0000","last_peered":"2024-09-10T10:01:57.431695+0000","last_clean":"2024-09-10T10:01:57.431695+0000","last_became_active":"2024-09-10T10:01:57.425032+0000","last_became_peered":"2024-09-10T10:01:57.425032+0000","last_unstale":"2024-09-10T10:01:57.431695+0000","last_undegraded":"2024-09-10T10:01:57.431695+0000","last_fullsized":"2024-09-10T10:01:57.431695+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.395612+0000","last_change":"2024-09-10T10:01:56.407609+0000","last_active":"2024-09-10T10:01:57.395612+0000","last_peered":"2024-09-10T10:01:57.395612+0000","last_clean":"2024-09-10T10:01:57.395612+0000","last_became_active":"2024-09-10T10:01:56.405401+0000","last_became_peered":"2024-09-10T10:01:56.405401+0000","last_unstale":"2024-09-10T10:01:57.395612+0000","last_undegraded":"2024-09-10T10:01:57.395612+0000","last_fullsized":"2024-09-10T10:01:57.395612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419650+0000","last_change":"2024-09-10T10:01:57.412223+0000","last_active":"2024-09-10T10:01:57.419650+0000","last_peered":"2024-09-10T10:01:57.419650+0000","last_clean":"2024-09-10T10:01:57.419650+0000","last_became_active":"2024-09-10T10:01:57.411477+0000","last_became_peered":"2024-09-10T10:01:57.411477+0000","last_unstale":"2024-09-10T10:01:57.419650+0000","last_undegraded":"2024-09-10T10:01:57.419650+0000","last_fullsized":"2024-09-10T10:01:57.419650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:47.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.407993+0000","last_change":"2024-09-10T10:01:56.406998+0000","last_active":"2024-09-10T10:01:56.407993+0000","last_peered":"2024-09-10T10:01:56.407993+0000","last_clean":"2024-09-10T10:01:56.407993+0000","last_became_active":"2024-09-10T10:01:56.406722+0000","last_became_peered":"2024-09-10T10:01:56.406722+0000","last_unstale":"2024-09-10T10:01:56.407993+0000","last_undegraded":"2024-09-10T10:01:56.407993+0000","last_fullsized":"2024-09-10T10:01:56.407993+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:27.783209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414220+0000","last_change":"2024-09-10T10:01:57.409970+0000","last_active":"2024-09-10T10:01:57.414220+0000","last_peered":"2024-09-10T10:01:57.414220+0000","last_clean":"2024-09-10T10:01:57.414220+0000","last_became_active":"2024-09-10T10:01:57.406199+0000","last_became_peered":"2024-09-10T10:01:57.406199+0000","last_unstale":"2024-09-10T10:01:57.414220+0000","last_undegraded":"2024-09-10T10:01:57.414220+0000","last_fullsized":"2024-09-10T10:01:57.414220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:41.660285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.426946+0000","last_change":"2024-09-10T10:01:56.410726+0000","last_active":"2024-09-10T10:01:57.426946+0000","last_peered":"2024-09-10T10:01:57.426946+0000","last_clean":"2024-09-10T10:01:57.426946+0000","last_became_active":"2024-09-10T10:01:56.410239+0000","last_became_peered":"2024-09-10T10:01:56.410239+0000","last_unstale":"2024-09-10T10:01:57.426946+0000","last_undegraded":"2024-09-10T10:01:57.426946+0000","last_fullsized":"2024-09-10T10:01:57.426946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:44.033036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.429665+0000","last_change":"2024-09-10T10:01:57.423932+0000","last_active":"2024-09-10T10:01:57.429665+0000","last_peered":"2024-09-10T10:01:57.429665+0000","last_clean":"2024-09-10T10:01:57.429665+0000","last_became_active":"2024-09-10T10:01:57.423635+0000","last_became_peered":"2024-09-10T10:01:57.423635+0000","last_unstale":"2024-09-10T10:01:57.429665+0000","last_undegraded":"2024-09-10T10:01:57.429665+0000","last_fullsized":"2024-09-10T10:01:57.429665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431982+0000","last_change":"2024-09-10T10:01:56.411145+0000","last_active":"2024-09-10T10:01:57.431982+0000","last_peered":"2024-09-10T10:01:57.431982+0000","last_clean":"2024-09-10T10:01:57.431982+0000","last_became_active":"2024-09-10T10:01:56.410821+0000","last_became_peered":"2024-09-10T10:01:56.410821+0000","last_unstale":"2024-09-10T10:01:57.431982+0000","last_undegraded":"2024-09-10T10:01:57.431982+0000","last_fullsized":"2024-09-10T10:01:57.431982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:34.635430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431813+0000","last_change":"2024-09-10T10:01:57.427059+0000","last_active":"2024-09-10T10:01:57.431813+0000","last_peered":"2024-09-10T10:01:57.431813+0000","last_clean":"2024-09-10T10:01:57.431813+0000","last_became_active":"2024-09-10T10:01:57.423512+0000","last_became_peered":"2024-09-10T10:01:57.423512+0000","last_unstale":"2024-09-10T10:01:57.431813+0000","last_undegraded":"2024-09-10T10:01:57.431813+0000","last_fullsized":"2024-09-10T10:01:57.431813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394598+0000","last_change":"2024-09-10T10:01:56.405376+0000","last_active":"2024-09-10T10:01:57.394598+0000","last_peered":"2024-09-10T10:01:57.394598+0000","last_clean":"2024-09-10T10:01:57.394598+0000","last_became_active":"2024-09-10T10:01:56.405056+0000","last_became_peered":"2024-09-10T10:01:56.405056+0000","last_unstale":"2024-09-10T10:01:57.394598+0000","last_undegraded":"2024-09-10T10:01:57.394598+0000","last_fullsized":"2024-09-10T10:01:57.394598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:56.936736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415321+0000","last_change":"2024-09-10T10:01:57.411055+0000","last_active":"2024-09-10T10:01:57.415321+0000","last_peered":"2024-09-10T10:01:57.415321+0000","last_clean":"2024-09-10T10:01:57.415321+0000","last_became_active":"2024-09-10T10:01:57.410789+0000","last_became_peered":"2024-09-10T10:01:57.410789+0000","last_unstale":"2024-09-10T10:01:57.415321+0000","last_undegraded":"2024-09-10T10:01:57.415321+0000","last_fullsized":"2024-09-10T10:01:57.415321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.412528+0000","last_change":"2024-09-10T10:01:56.412704+0000","last_active":"2024-09-10T10:01:57.412528+0000","last_peered":"2024-09-10T10:01:57.412528+0000","last_clean":"2024-09-10T10:01:57.412528+0000","last_became_active":"2024-09-10T10:01:56.410023+0000","last_became_peered":"2024-09-10T10:01:56.410023+0000","last_unstale":"2024-09-10T10:01:57.412528+0000","last_undegraded":"2024-09-10T10:01:57.412528+0000","last_fullsized":"2024-09-10T10:01:57.412528+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:57.159878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415068+0000","last_change":"2024-09-10T10:01:57.407954+0000","last_active":"2024-09-10T10:01:57.415068+0000","last_peered":"2024-09-10T10:01:57.415068+0000","last_clean":"2024-09-10T10:01:57.415068+0000","last_became_active":"2024-09-10T10:01:57.407651+0000","last_became_peered":"2024-09-10T10:01:57.407651+0000","last_unstale":"2024-09-10T10:01:57.415068+0000","last_undegraded":"2024-09-10T10:01:57.415068+0000","last_fullsized":"2024-09-10T10:01:57.415068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:16.908619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.430713+0000","last_change":"2024-09-10T10:01:56.410104+0000","last_active":"2024-09-10T10:01:57.430713+0000","last_peered":"2024-09-10T10:01:57.430713+0000","last_clean":"2024-09-10T10:01:57.430713+0000","last_became_active":"2024-09-10T10:01:56.409756+0000","last_became_peered":"2024-09-10T10:01:56.409756+0000","last_unstale":"2024-09-10T10:01:57.430713+0000","last_undegraded":"2024-09-10T10:01:57.430713+0000","last_fullsized":"2024-09-10T10:01:57.430713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.409223+0000","last_change":"2024-09-10T10:01:57.405556+0000","last_active":"2024-09-10T10:01:57.409223+0000","last_peered":"2024-09-10T10:01:57.409223+0000","last_clean":"2024-09-10T10:01:57.409223+0000","last_became_active":"2024-09-10T10:01:57.405222+0000","last_became_peered":"2024-09-10T10:01:57.405222+0000","last_unstale":"2024-09-10T10:01:57.409223+0000","last_undegraded":"2024-09-10T10:01:57.409223+0000","last_fullsized":"2024-09-10T10:01:57.409223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:03.020211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.425896+0000","last_change":"2024-09-10T10:01:56.401752+0000","last_active":"2024-09-10T10:01:57.425896+0000","last_peered":"2024-09-10T10:01:57.425896+0000","last_clean":"2024-09-10T10:01:57.425896+0000","last_became_active":"2024-09-10T10:01:56.401508+0000","last_became_peered":"2024-09-10T10:01:56.401508+0000","last_unstale":"2024-09-10T10:01:57.425896+0000","last_undegraded":"2024-09-10T10:01:57.425896+0000","last_fullsized":"2024-09-10T10:01:57.425896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:52.212395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431672+0000","last_change":"2024-09-10T10:01:57.425326+0000","last_active":"2024-09-10T10:01:57.431672+0000","last_peered":"2024-09-10T10:01:57.431672+0000","last_clean":"2024-09-10T10:01:57.431672+0000","last_became_active":"2024-09-10T10:01:57.424606+0000","last_became_peered":"2024-09-10T10:01:57.424606+0000","last_unstale":"2024-09-10T10:01:57.431672+0000","last_undegraded":"2024-09-10T10:01:57.431672+0000","last_fullsized":"2024-09-10T10:01:57.431672+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:23.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422525+0000","last_change":"2024-09-10T10:01:56.409357+0000","last_active":"2024-09-10T10:01:57.422525+0000","last_peered":"2024-09-10T10:01:57.422525+0000","last_clean":"2024-09-10T10:01:57.422525+0000","last_became_active":"2024-09-10T10:01:56.409062+0000","last_became_peered":"2024-09-10T10:01:56.409062+0000","last_unstale":"2024-09-10T10:01:57.422525+0000","last_undegraded":"2024-09-10T10:01:57.422525+0000","last_fullsized":"2024-09-10T10:01:57.422525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:29.657091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413399+0000","last_change":"2024-09-10T10:01:57.401546+0000","last_active":"2024-09-10T10:01:57.413399+0000","last_peered":"2024-09-10T10:01:57.413399+0000","last_clean":"2024-09-10T10:01:57.413399+0000","last_became_active":"2024-09-10T10:01:57.401293+0000","last_became_peered":"2024-09-10T10:01:57.401293+0000","last_unstale":"2024-09-10T10:01:57.413399+0000","last_undegraded":"2024-09-10T10:01:57.413399+0000","last_fullsized":"2024-09-10T10:01:57.413399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:19.413048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427627+0000","last_change":"2024-09-10T10:01:56.399162+0000","last_active":"2024-09-10T10:01:57.427627+0000","last_peered":"2024-09-10T10:01:57.427627+0000","last_clean":"2024-09-10T10:01:57.427627+0000","last_became_active":"2024-09-10T10:01:56.398886+0000","last_became_peered":"2024-09-10T10:01:56.398886+0000","last_unstale":"2024-09-10T10:01:57.427627+0000","last_undegraded":"2024-09-10T10:01:57.427627+0000","last_fullsized":"2024-09-10T10:01:57.427627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414198+0000","last_change":"2024-09-10T10:01:57.409275+0000","last_active":"2024-09-10T10:01:57.414198+0000","last_peered":"2024-09-10T10:01:57.414198+0000","last_clean":"2024-09-10T10:01:57.414198+0000","last_became_active":"2024-09-10T10:01:57.409007+0000","last_became_peered":"2024-09-10T10:01:57.409007+0000","last_unstale":"2024-09-10T10:01:57.414198+0000","last_undegraded":"2024-09-10T10:01:57.414198+0000","last_fullsized":"2024-09-10T10:01:57.414198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:17.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.405547+0000","last_change":"2024-09-10T10:01:56.414600+0000","last_active":"2024-09-10T10:01:57.405547+0000","last_peered":"2024-09-10T10:01:57.405547+0000","last_clean":"2024-09-10T10:01:57.405547+0000","last_became_active":"2024-09-10T10:01:56.414312+0000","last_became_peered":"2024-09-10T10:01:56.414312+0000","last_unstale":"2024-09-10T10:01:57.405547+0000","last_undegraded":"2024-09-10T10:01:57.405547+0000","last_fullsized":"2024-09-10T10:01:57.405547+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:13.899397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394589+0000","last_change":"2024-09-10T10:01:56.394589+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394589+0000","last_undegraded":"2024-09-10T10:01:56.394589+0000","last_fullsized":"2024-09-10T10:01:56.394589+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394540+0000","last_change":"2024-09-10T10:01:56.399440+0000","last_active":"2024-09-10T10:01:57.394540+0000","last_peered":"2024-09-10T10:01:57.394540+0000","last_clean":"2024-09-10T10:01:57.394540+0000","last_became_active":"2024-09-10T10:01:56.399153+0000","last_became_peered":"2024-09-10T10:01:56.399153+0000","last_unstale":"2024-09-10T10:01:57.394540+0000","last_undegraded":"2024-09-10T10:01:57.394540+0000","last_fullsized":"2024-09-10T10:01:57.394540+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:21.685329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.393504+0000","last_change":"2024-09-10T10:01:56.393504+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.393504+0000","last_undegraded":"2024-09-10T10:01:56.393504+0000","last_fullsized":"2024-09-10T10:01:56.393504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.412691+0000","last_change":"2024-09-10T10:01:56.410145+0000","last_active":"2024-09-10T10:01:57.412691+0000","last_peered":"2024-09-10T10:01:57.412691+0000","last_clean":"2024-09-10T10:01:57.412691+0000","last_became_active":"2024-09-10T10:01:56.409854+0000","last_became_peered":"2024-09-10T10:01:56.409854+0000","last_unstale":"2024-09-10T10:01:57.412691+0000","last_undegraded":"2024-09-10T10:01:57.412691+0000","last_fullsized":"2024-09-10T10:01:57.412691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:23.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.393865+0000","last_change":"2024-09-10T10:01:56.393865+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.393865+0000","last_undegraded":"2024-09-10T10:01:56.393865+0000","last_fullsized":"2024-09-10T10:01:56.393865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.395889+0000","last_change":"2024-09-10T10:01:56.404958+0000","last_active":"2024-09-10T10:01:57.395889+0000","last_peered":"2024-09-10T10:01:57.395889+0000","last_clean":"2024-09-10T10:01:57.395889+0000","last_became_active":"2024-09-10T10:01:56.404619+0000","last_became_peered":"2024-09-10T10:01:56.404619+0000","last_unstale":"2024-09-10T10:01:57.395889+0000","last_undegraded":"2024-09-10T10:01:57.395889+0000","last_fullsized":"2024-09-10T10:01:57.395889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415096+0000","last_change":"2024-09-10T10:01:57.409907+0000","last_active":"2024-09-10T10:01:57.415096+0000","last_peered":"2024-09-10T10:01:57.415096+0000","last_clean":"2024-09-10T10:01:57.415096+0000","last_became_active":"2024-09-10T10:01:57.409375+0000","last_became_peered":"2024-09-10T10:01:57.409375+0000","last_unstale":"2024-09-10T10:01:57.415096+0000","last_undegraded":"2024-09-10T10:01:57.415096+0000","last_fullsized":"2024-09-10T10:01:57.415096+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:23.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.425893+0000","last_change":"2024-09-10T10:01:56.400929+0000","last_active":"2024-09-10T10:01:57.425893+0000","last_peered":"2024-09-10T10:01:57.425893+0000","last_clean":"2024-09-10T10:01:57.425893+0000","last_became_active":"2024-09-10T10:01:56.400665+0000","last_became_peered":"2024-09-10T10:01:56.400665+0000","last_unstale":"2024-09-10T10:01:57.425893+0000","last_undegraded":"2024-09-10T10:01:57.425893+0000","last_fullsized":"2024-09-10T10:01:57.425893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:32.570015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431796+0000","last_change":"2024-09-10T10:01:57.426884+0000","last_active":"2024-09-10T10:01:57.431796+0000","last_peered":"2024-09-10T10:01:57.431796+0000","last_clean":"2024-09-10T10:01:57.431796+0000","last_became_active":"2024-09-10T10:01:57.426621+0000","last_became_peered":"2024-09-10T10:01:57.426621+0000","last_unstale":"2024-09-10T10:01:57.431796+0000","last_undegraded":"2024-09-10T10:01:57.431796+0000","last_fullsized":"2024-09-10T10:01:57.431796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:35.517149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413728+0000","last_change":"2024-09-10T10:01:56.410855+0000","last_active":"2024-09-10T10:01:57.413728+0000","last_peered":"2024-09-10T10:01:57.413728+0000","last_clean":"2024-09-10T10:01:57.413728+0000","last_became_active":"2024-09-10T10:01:56.410597+0000","last_became_peered":"2024-09-10T10:01:56.410597+0000","last_unstale":"2024-09-10T10:01:57.413728+0000","last_undegraded":"2024-09-10T10:01:57.413728+0000","last_fullsized":"2024-09-10T10:01:57.413728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.934821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.429750+0000","last_change":"2024-09-10T10:01:57.423850+0000","last_active":"2024-09-10T10:01:57.429750+0000","last_peered":"2024-09-10T10:01:57.429750+0000","last_clean":"2024-09-10T10:01:57.429750+0000","last_became_active":"2024-09-10T10:01:57.423309+0000","last_became_peered":"2024-09-10T10:01:57.423309+0000","last_unstale":"2024-09-10T10:01:57.429750+0000","last_undegraded":"2024-09-10T10:01:57.429750+0000","last_fullsized":"2024-09-10T10:01:57.429750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.405952+0000","last_change":"2024-09-10T10:01:56.409831+0000","last_active":"2024-09-10T10:01:57.405952+0000","last_peered":"2024-09-10T10:01:57.405952+0000","last_clean":"2024-09-10T10:01:57.405952+0000","last_became_active":"2024-09-10T10:01:56.409512+0000","last_became_peered":"2024-09-10T10:01:56.409512+0000","last_unstale":"2024-09-10T10:01:57.405952+0000","last_undegraded":"2024-09-10T10:01:57.405952+0000","last_fullsized":"2024-09-10T10:01:57.405952+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:05.784097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431774+0000","last_change":"2024-09-10T10:01:57.427760+0000","last_active":"2024-09-10T10:01:57.431774+0000","last_peered":"2024-09-10T10:01:57.431774+0000","last_clean":"2024-09-10T10:01:57.431774+0000","last_became_active":"2024-09-10T10:01:57.427511+0000","last_became_peered":"2024-09-10T10:01:57.427511+0000","last_unstale":"2024-09-10T10:01:57.431774+0000","last_undegraded":"2024-09-10T10:01:57.431774+0000","last_fullsized":"2024-09-10T10:01:57.431774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:52.470311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.397237+0000","last_change":"2024-09-10T10:01:56.402970+0000","last_active":"2024-09-10T10:01:57.397237+0000","last_peered":"2024-09-10T10:01:57.397237+0000","last_clean":"2024-09-10T10:01:57.397237+0000","last_became_active":"2024-09-10T10:01:56.402500+0000","last_became_peered":"2024-09-10T10:01:56.402500+0000","last_unstale":"2024-09-10T10:01:57.397237+0000","last_undegraded":"2024-09-10T10:01:57.397237+0000","last_fullsized":"2024-09-10T10:01:57.397237+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:26.744793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415252+0000","last_change":"2024-09-10T10:01:57.409373+0000","last_active":"2024-09-10T10:01:57.415252+0000","last_peered":"2024-09-10T10:01:57.415252+0000","last_clean":"2024-09-10T10:01:57.415252+0000","last_became_active":"2024-09-10T10:01:57.409114+0000","last_became_peered":"2024-09-10T10:01:57.409114+0000","last_unstale":"2024-09-10T10:01:57.415252+0000","last_undegraded":"2024-09-10T10:01:57.415252+0000","last_fullsized":"2024-09-10T10:01:57.415252+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:53.254884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.400447+0000","last_change":"2024-09-10T10:01:56.411209+0000","last_active":"2024-09-10T10:01:57.400447+0000","last_peered":"2024-09-10T10:01:57.400447+0000","last_clean":"2024-09-10T10:01:57.400447+0000","last_became_active":"2024-09-10T10:01:56.410808+0000","last_became_peered":"2024-09-10T10:01:56.410808+0000","last_unstale":"2024-09-10T10:01:57.400447+0000","last_undegraded":"2024-09-10T10:01:57.400447+0000","last_fullsized":"2024-09-10T10:01:57.400447+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:39.392910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421360+0000","last_change":"2024-09-10T10:01:57.418730+0000","last_active":"2024-09-10T10:01:57.421360+0000","last_peered":"2024-09-10T10:01:57.421360+0000","last_clean":"2024-09-10T10:01:57.421360+0000","last_became_active":"2024-09-10T10:01:57.418395+0000","last_became_peered":"2024-09-10T10:01:57.418395+0000","last_unstale":"2024-09-10T10:01:57.421360+0000","last_undegraded":"2024-09-10T10:01:57.421360+0000","last_fullsized":"2024-09-10T10:01:57.421360+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:58.162428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.391574+0000","last_change":"2024-09-10T10:01:56.409314+0000","last_active":"2024-09-10T10:01:57.391574+0000","last_peered":"2024-09-10T10:01:57.391574+0000","last_clean":"2024-09-10T10:01:57.391574+0000","last_became_active":"2024-09-10T10:01:56.409077+0000","last_became_peered":"2024-09-10T10:01:56.409077+0000","last_unstale":"2024-09-10T10:01:57.391574+0000","last_undegraded":"2024-09-10T10:01:57.391574+0000","last_fullsized":"2024-09-10T10:01:57.391574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.220387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.416721+0000","last_change":"2024-09-10T10:01:57.407297+0000","last_active":"2024-09-10T10:01:57.416721+0000","last_peered":"2024-09-10T10:01:57.416721+0000","last_clean":"2024-09-10T10:01:57.416721+0000","last_became_active":"2024-09-10T10:01:57.407030+0000","last_became_peered":"2024-09-10T10:01:57.407030+0000","last_unstale":"2024-09-10T10:01:57.416721+0000","last_undegraded":"2024-09-10T10:01:57.416721+0000","last_fullsized":"2024-09-10T10:01:57.416721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:33.572565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398463+0000","last_change":"2024-09-10T10:01:56.411518+0000","last_active":"2024-09-10T10:01:57.398463+0000","last_peered":"2024-09-10T10:01:57.398463+0000","last_clean":"2024-09-10T10:01:57.398463+0000","last_became_active":"2024-09-10T10:01:56.410515+0000","last_became_peered":"2024-09-10T10:01:56.410515+0000","last_unstale":"2024-09-10T10:01:57.398463+0000","last_undegraded":"2024-09-10T10:01:57.398463+0000","last_fullsized":"2024-09-10T10:01:57.398463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421441+0000","last_change":"2024-09-10T10:01:57.408569+0000","last_active":"2024-09-10T10:01:57.421441+0000","last_peered":"2024-09-10T10:01:57.421441+0000","last_clean":"2024-09-10T10:01:57.421441+0000","last_became_active":"2024-09-10T10:01:57.408302+0000","last_became_peered":"2024-09-10T10:01:57.408302+0000","last_unstale":"2024-09-10T10:01:57.421441+0000","last_undegraded":"2024-09-10T10:01:57.421441+0000","last_fullsized":"2024-09-10T10:01:57.421441+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:39.030596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394334+0000","last_change":"2024-09-10T10:01:56.407472+0000","last_active":"2024-09-10T10:01:57.394334+0000","last_peered":"2024-09-10T10:01:57.394334+0000","last_clean":"2024-09-10T10:01:57.394334+0000","last_became_active":"2024-09-10T10:01:56.407112+0000","last_became_peered":"2024-09-10T10:01:56.407112+0000","last_unstale":"2024-09-10T10:01:57.394334+0000","last_undegraded":"2024-09-10T10:01:57.394334+0000","last_fullsized":"2024-09-10T10:01:57.394334+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:36.230334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394297+0000","last_change":"2024-09-10T10:01:56.394297+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394297+0000","last_undegraded":"2024-09-10T10:01:56.394297+0000","last_fullsized":"2024-09-10T10:01:56.394297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427116+0000","last_change":"2024-09-10T10:01:56.408027+0000","last_active":"2024-09-10T10:01:57.427116+0000","last_peered":"2024-09-10T10:01:57.427116+0000","last_clean":"2024-09-10T10:01:57.427116+0000","last_became_active":"2024-09-10T10:01:56.407749+0000","last_became_peered":"2024-09-10T10:01:56.407749+0000","last_unstale":"2024-09-10T10:01:57.427116+0000","last_undegraded":"2024-09-10T10:01:57.427116+0000","last_fullsized":"2024-09-10T10:01:57.427116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:42.979120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":252,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.403667+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:01:57.403667+0000","last_peered":"2024-09-10T10:01:57.403667+0000","last_clean":"2024-09-10T10:01:57.403667+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:01:57.403667+0000","last_undegraded":"2024-09-10T10:01:57.403667+0000","last_fullsized":"2024-09-10T10:01:57.403667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":930573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":341201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":929957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:01.482 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:02:01.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db8e00640 1 Processor -- start 2024-09-10T10:02:01.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db8e00640 1 -- start start 2024-09-10T10:02:01.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db8e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5db4147f00 0x7f5db40ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db8e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db406edf0 0x7f5db4072310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db8e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db40af1a0 con 0x7f5db40aed80 2024-09-10T10:02:01.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db8e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5db40b34d0 con 0x7f5db4147f00 2024-09-10T10:02:01.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db8e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5db41482e0 con 0x7f5db406edf0 2024-09-10T10:02:01.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db1e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5db4147f00 0x7f5db40ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db1e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5db4147f00 0x7f5db40ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44048/0 (socket says 172.21.15.110:44048) 2024-09-10T10:02:01.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db1e00640 1 -- 172.21.15.110:0/857584899 learned_addr learned my addr 172.21.15.110:0/857584899 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:01.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db2800640 1 --2- 172.21.15.110:0/857584899 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db406edf0 0x7f5db4072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db1e00640 1 -- 172.21.15.110:0/857584899 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db406edf0 msgr2=0x7f5db4072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:01.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db1e00640 1 --2- 172.21.15.110:0/857584899 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db406edf0 0x7f5db4072310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db1e00640 1 -- 172.21.15.110:0/857584899 >> v1:172.21.15.145:6789/0 conn(0x7f5db40aed80 legacy=0x7f5db406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.656+0000 7f5db1e00640 1 -- 172.21.15.110:0/857584899 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db415ee20 con 0x7f5db4147f00 2024-09-10T10:02:01.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db1e00640 1 --2- 172.21.15.110:0/857584899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db40ae840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009e50 tx=0x7f5da401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d156accddda32ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:01.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db0a00640 1 -- 172.21.15.110:0/857584899 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da40a5810 con 0x7f5db4147f00 2024-09-10T10:02:01.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db0a00640 1 -- 172.21.15.110:0/857584899 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da40a59b0 con 0x7f5db4147f00 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db0a00640 1 -- 172.21.15.110:0/857584899 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da40a5d80 con 0x7f5db4147f00 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db8e00640 1 -- 172.21.15.110:0/857584899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 msgr2=0x7f5db40ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db8e00640 1 --2- 172.21.15.110:0/857584899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db40ae840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009e50 tx=0x7f5da401ca30 comp rx=0 tx=0).stop 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db8e00640 1 -- 172.21.15.110:0/857584899 shutdown_connections 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db8e00640 1 --2- 172.21.15.110:0/857584899 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db406edf0 0x7f5db4072310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db8e00640 1 --2- 172.21.15.110:0/857584899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db40ae840 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db8e00640 1 -- 172.21.15.110:0/857584899 >> 172.21.15.110:0/857584899 conn(0x7f5db406bc50 msgr2=0x7f5db406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db8e00640 1 -- 172.21.15.110:0/857584899 shutdown_connections 2024-09-10T10:02:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.660+0000 7f5db8e00640 1 -- 172.21.15.110:0/857584899 wait complete. 2024-09-10T10:02:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 Processor -- start 2024-09-10T10:02:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 -- start start 2024-09-10T10:02:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5db406edf0 0x7f5db4146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db40aed80 0x7f5db41fa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db41faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40731e0 con 0x7f5db4147f00 2024-09-10T10:02:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4073060 con 0x7f5db406edf0 2024-09-10T10:02:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5db4073360 con 0x7f5db40aed80 2024-09-10T10:02:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db41faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db41faeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44064/0 (socket says 172.21.15.110:44064) 2024-09-10T10:02:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 -- 172.21.15.110:0/778327117 learned_addr learned my addr 172.21.15.110:0/778327117 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db1e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5db406edf0 0x7f5db4146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db1400640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db40aed80 0x7f5db41fa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 -- 172.21.15.110:0/778327117 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db40aed80 msgr2=0x7f5db41fa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db40aed80 0x7f5db41fa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 -- 172.21.15.110:0/778327117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5db406edf0 msgr2=0x7f5db4146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5db406edf0 0x7f5db4146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 -- 172.21.15.110:0/778327117 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db40b1970 con 0x7f5db4147f00 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db2800640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db41faeb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00d560 tx=0x7f5dac00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5daa000640 1 -- 172.21.15.110:0/778327117 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0125f0 con 0x7f5db4147f00 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5daa000640 1 -- 172.21.15.110:0/778327117 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac019070 con 0x7f5db4147f00 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db415be50 con 0x7f5db4147f00 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5daa000640 1 -- 172.21.15.110:0/778327117 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0129f0 con 0x7f5db4147f00 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.664+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db4147490 con 0x7f5db4147f00 2024-09-10T10:02:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.668+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db40722f0 con 0x7f5db4147f00 2024-09-10T10:02:01.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.668+0000 7f5daa000640 1 -- 172.21.15.110:0/778327117 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f5dac012b90 con 0x7f5db4147f00 2024-09-10T10:02:01.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.668+0000 7f5daa000640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5d94067c20 0x7f5d9406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:01.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.668+0000 7f5daa000640 1 -- 172.21.15.110:0/778327117 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f5dac086520 con 0x7f5db4147f00 2024-09-10T10:02:01.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.668+0000 7f5db1e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5d94067c20 0x7f5d9406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:01.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.668+0000 7f5db1e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5d94067c20 0x7f5d9406a0e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5da40ae000 tx=0x7f5da40a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.672+0000 7f5daa000640 1 -- 172.21.15.110:0/778327117 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5db415be50 con 0x7f5db4147f00 2024-09-10T10:02:01.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.828+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5db40b3700 con 0x7f5d94067c20 2024-09-10T10:02:01.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5daa000640 1 -- 172.21.15.110:0/778327117 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251428 (secure 0 0 0) 0x7f5db40b3700 con 0x7f5d94067c20 2024-09-10T10:02:01.839 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:01.841 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5d94067c20 msgr2=0x7f5d9406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5d94067c20 0x7f5d9406a0e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5da40ae000 tx=0x7f5da40a7070 comp rx=0 tx=0).stop 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 msgr2=0x7f5db41faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db41faeb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00d560 tx=0x7f5dac00da20 comp rx=0 tx=0).stop 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 shutdown_connections 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f5d94067c20 0x7f5d9406a0e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4147f00 0x7f5db41faeb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5db40aed80 0x7f5db41fa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 --2- 172.21.15.110:0/778327117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5db406edf0 0x7f5db4146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:01.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 >> 172.21.15.110:0/778327117 conn(0x7f5db406bc50 msgr2=0x7f5db40b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 shutdown_connections 2024-09-10T10:02:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:01.836+0000 7f5db8e00640 1 -- 172.21.15.110:0/778327117 wait complete. 2024-09-10T10:02:01.861 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-10T10:02:01.402066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248728,"kb_used_data":7520,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725992,"statfs":{"total":773094113280,"available":772839415808,"internally_reserved":0,"allocated":7700480,"data_stored":3865801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":246795820},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002773"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431713+0000","last_change":"2024-09-10T10:01:57.425627+0000","last_active":"2024-09-10T10:01:57.431713+0000","last_peered":"2024-09-10T10:01:57.431713+0000","last_clean":"2024-09-10T10:01:57.431713+0000","last_became_active":"2024-09-10T10:01:57.425083+0000","last_became_peered":"2024-09-10T10:01:57.425083+0000","last_unstale":"2024-09-10T10:01:57.431713+0000","last_undegraded":"2024-09-10T10:01:57.431713+0000","last_fullsized":"2024-09-10T10:01:57.431713+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419573+0000","last_change":"2024-09-10T10:01:57.410364+0000","last_active":"2024-09-10T10:01:57.419573+0000","last_peered":"2024-09-10T10:01:57.419573+0000","last_clean":"2024-09-10T10:01:57.419573+0000","last_became_active":"2024-09-10T10:01:57.410086+0000","last_became_peered":"2024-09-10T10:01:57.410086+0000","last_unstale":"2024-09-10T10:01:57.419573+0000","last_undegraded":"2024-09-10T10:01:57.419573+0000","last_fullsized":"2024-09-10T10:01:57.419573+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.839704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.397555+0000","last_change":"2024-09-10T10:01:56.397555+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.397555+0000","last_undegraded":"2024-09-10T10:01:56.397555+0000","last_fullsized":"2024-09-10T10:01:56.397555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422592+0000","last_change":"2024-09-10T10:01:57.419086+0000","last_active":"2024-09-10T10:01:57.422592+0000","last_peered":"2024-09-10T10:01:57.422592+0000","last_clean":"2024-09-10T10:01:57.422592+0000","last_became_active":"2024-09-10T10:01:57.418640+0000","last_became_peered":"2024-09-10T10:01:57.418640+0000","last_unstale":"2024-09-10T10:01:57.422592+0000","last_undegraded":"2024-09-10T10:01:57.422592+0000","last_fullsized":"2024-09-10T10:01:57.422592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:37.232883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417356+0000","last_change":"2024-09-10T10:01:57.412199+0000","last_active":"2024-09-10T10:01:57.417356+0000","last_peered":"2024-09-10T10:01:57.417356+0000","last_clean":"2024-09-10T10:01:57.417356+0000","last_became_active":"2024-09-10T10:01:57.411926+0000","last_became_peered":"2024-09-10T10:01:57.411926+0000","last_unstale":"2024-09-10T10:01:57.417356+0000","last_undegraded":"2024-09-10T10:01:57.417356+0000","last_fullsized":"2024-09-10T10:01:57.417356+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:50.877508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421654+0000","last_change":"2024-09-10T10:01:57.411801+0000","last_active":"2024-09-10T10:01:57.421654+0000","last_peered":"2024-09-10T10:01:57.421654+0000","last_clean":"2024-09-10T10:01:57.421654+0000","last_became_active":"2024-09-10T10:01:57.411506+0000","last_became_peered":"2024-09-10T10:01:57.411506+0000","last_unstale":"2024-09-10T10:01:57.421654+0000","last_undegraded":"2024-09-10T10:01:57.421654+0000","last_fullsized":"2024-09-10T10:01:57.421654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:53.511190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431828+0000","last_change":"2024-09-10T10:01:57.427391+0000","last_active":"2024-09-10T10:01:57.431828+0000","last_peered":"2024-09-10T10:01:57.431828+0000","last_clean":"2024-09-10T10:01:57.431828+0000","last_became_active":"2024-09-10T10:01:57.427095+0000","last_became_peered":"2024-09-10T10:01:57.427095+0000","last_unstale":"2024-09-10T10:01:57.431828+0000","last_undegraded":"2024-09-10T10:01:57.431828+0000","last_fullsized":"2024-09-10T10:01:57.431828+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414913+0000","last_change":"2024-09-10T10:01:57.405094+0000","last_active":"2024-09-10T10:01:57.414913+0000","last_peered":"2024-09-10T10:01:57.414913+0000","last_clean":"2024-09-10T10:01:57.414913+0000","last_became_active":"2024-09-10T10:01:57.404803+0000","last_became_peered":"2024-09-10T10:01:57.404803+0000","last_unstale":"2024-09-10T10:01:57.414913+0000","last_undegraded":"2024-09-10T10:01:57.414913+0000","last_fullsized":"2024-09-10T10:01:57.414913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:51.428680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419139+0000","last_change":"2024-09-10T10:01:57.414814+0000","last_active":"2024-09-10T10:01:57.419139+0000","last_peered":"2024-09-10T10:01:57.419139+0000","last_clean":"2024-09-10T10:01:57.419139+0000","last_became_active":"2024-09-10T10:01:57.414440+0000","last_became_peered":"2024-09-10T10:01:57.414440+0000","last_unstale":"2024-09-10T10:01:57.419139+0000","last_undegraded":"2024-09-10T10:01:57.419139+0000","last_fullsized":"2024-09-10T10:01:57.419139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:04.748497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415584+0000","last_change":"2024-09-10T10:01:57.411661+0000","last_active":"2024-09-10T10:01:57.415584+0000","last_peered":"2024-09-10T10:01:57.415584+0000","last_clean":"2024-09-10T10:01:57.415584+0000","last_became_active":"2024-09-10T10:01:57.410577+0000","last_became_peered":"2024-09-10T10:01:57.410577+0000","last_unstale":"2024-09-10T10:01:57.415584+0000","last_undegraded":"2024-09-10T10:01:57.415584+0000","last_fullsized":"2024-09-10T10:01:57.415584+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:53.511190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423038+0000","last_change":"2024-09-10T10:01:57.420979+0000","last_active":"2024-09-10T10:01:57.423038+0000","last_peered":"2024-09-10T10:01:57.423038+0000","last_clean":"2024-09-10T10:01:57.423038+0000","last_became_active":"2024-09-10T10:01:57.420695+0000","last_became_peered":"2024-09-10T10:01:57.420695+0000","last_unstale":"2024-09-10T10:01:57.423038+0000","last_undegraded":"2024-09-10T10:01:57.423038+0000","last_fullsized":"2024-09-10T10:01:57.423038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413485+0000","last_change":"2024-09-10T10:01:57.403883+0000","last_active":"2024-09-10T10:01:57.413485+0000","last_peered":"2024-09-10T10:01:57.413485+0000","last_clean":"2024-09-10T10:01:57.413485+0000","last_became_active":"2024-09-10T10:01:57.403600+0000","last_became_peered":"2024-09-10T10:01:57.403600+0000","last_unstale":"2024-09-10T10:01:57.413485+0000","last_undegraded":"2024-09-10T10:01:57.413485+0000","last_fullsized":"2024-09-10T10:01:57.413485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:53.214945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415178+0000","last_change":"2024-09-10T10:01:57.409326+0000","last_active":"2024-09-10T10:01:57.415178+0000","last_peered":"2024-09-10T10:01:57.415178+0000","last_clean":"2024-09-10T10:01:57.415178+0000","last_became_active":"2024-09-10T10:01:57.409053+0000","last_became_peered":"2024-09-10T10:01:57.409053+0000","last_unstale":"2024-09-10T10:01:57.415178+0000","last_undegraded":"2024-09-10T10:01:57.415178+0000","last_fullsized":"2024-09-10T10:01:57.415178+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424665+0000","last_change":"2024-09-10T10:01:57.421822+0000","last_active":"2024-09-10T10:01:57.424665+0000","last_peered":"2024-09-10T10:01:57.424665+0000","last_clean":"2024-09-10T10:01:57.424665+0000","last_became_active":"2024-09-10T10:01:57.421559+0000","last_became_peered":"2024-09-10T10:01:57.421559+0000","last_unstale":"2024-09-10T10:01:57.424665+0000","last_undegraded":"2024-09-10T10:01:57.424665+0000","last_fullsized":"2024-09-10T10:01:57.424665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:10.788546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414952+0000","last_change":"2024-09-10T10:01:57.410832+0000","last_active":"2024-09-10T10:01:57.414952+0000","last_peered":"2024-09-10T10:01:57.414952+0000","last_clean":"2024-09-10T10:01:57.414952+0000","last_became_active":"2024-09-10T10:01:57.410575+0000","last_became_peered":"2024-09-10T10:01:57.410575+0000","last_unstale":"2024-09-10T10:01:57.414952+0000","last_undegraded":"2024-09-10T10:01:57.414952+0000","last_fullsized":"2024-09-10T10:01:57.414952+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421152+0000","last_change":"2024-09-10T10:01:57.419056+0000","last_active":"2024-09-10T10:01:57.421152+0000","last_peered":"2024-09-10T10:01:57.421152+0000","last_clean":"2024-09-10T10:01:57.421152+0000","last_became_active":"2024-09-10T10:01:57.418704+0000","last_became_peered":"2024-09-10T10:01:57.418704+0000","last_unstale":"2024-09-10T10:01:57.421152+0000","last_undegraded":"2024-09-10T10:01:57.421152+0000","last_fullsized":"2024-09-10T10:01:57.421152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:58.162428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419118+0000","last_change":"2024-09-10T10:01:57.414713+0000","last_active":"2024-09-10T10:01:57.419118+0000","last_peered":"2024-09-10T10:01:57.419118+0000","last_clean":"2024-09-10T10:01:57.419118+0000","last_became_active":"2024-09-10T10:01:57.414227+0000","last_became_peered":"2024-09-10T10:01:57.414227+0000","last_unstale":"2024-09-10T10:01:57.419118+0000","last_undegraded":"2024-09-10T10:01:57.419118+0000","last_fullsized":"2024-09-10T10:01:57.419118+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:06.958521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.418366+0000","last_change":"2024-09-10T10:01:57.407315+0000","last_active":"2024-09-10T10:01:57.418366+0000","last_peered":"2024-09-10T10:01:57.418366+0000","last_clean":"2024-09-10T10:01:57.418366+0000","last_became_active":"2024-09-10T10:01:57.406568+0000","last_became_peered":"2024-09-10T10:01:57.406568+0000","last_unstale":"2024-09-10T10:01:57.418366+0000","last_undegraded":"2024-09-10T10:01:57.418366+0000","last_fullsized":"2024-09-10T10:01:57.418366+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:02.156822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414529+0000","last_change":"2024-09-10T10:01:57.405757+0000","last_active":"2024-09-10T10:01:57.414529+0000","last_peered":"2024-09-10T10:01:57.414529+0000","last_clean":"2024-09-10T10:01:57.414529+0000","last_became_active":"2024-09-10T10:01:57.404501+0000","last_became_peered":"2024-09-10T10:01:57.404501+0000","last_unstale":"2024-09-10T10:01:57.414529+0000","last_undegraded":"2024-09-10T10:01:57.414529+0000","last_fullsized":"2024-09-10T10:01:57.414529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:03.020211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424741+0000","last_change":"2024-09-10T10:01:57.423165+0000","last_active":"2024-09-10T10:01:57.424741+0000","last_peered":"2024-09-10T10:01:57.424741+0000","last_clean":"2024-09-10T10:01:57.424741+0000","last_became_active":"2024-09-10T10:01:57.422527+0000","last_became_peered":"2024-09-10T10:01:57.422527+0000","last_unstale":"2024-09-10T10:01:57.424741+0000","last_undegraded":"2024-09-10T10:01:57.424741+0000","last_fullsized":"2024-09-10T10:01:57.424741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417175+0000","last_change":"2024-09-10T10:01:57.409584+0000","last_active":"2024-09-10T10:01:57.417175+0000","last_peered":"2024-09-10T10:01:57.417175+0000","last_clean":"2024-09-10T10:01:57.417175+0000","last_became_active":"2024-09-10T10:01:57.409327+0000","last_became_peered":"2024-09-10T10:01:57.409327+0000","last_unstale":"2024-09-10T10:01:57.417175+0000","last_undegraded":"2024-09-10T10:01:57.417175+0000","last_fullsized":"2024-09-10T10:01:57.417175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:39.030596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419510+0000","last_change":"2024-09-10T10:01:57.416054+0000","last_active":"2024-09-10T10:01:57.419510+0000","last_peered":"2024-09-10T10:01:57.419510+0000","last_clean":"2024-09-10T10:01:57.419510+0000","last_became_active":"2024-09-10T10:01:57.415813+0000","last_became_peered":"2024-09-10T10:01:57.415813+0000","last_unstale":"2024-09-10T10:01:57.419510+0000","last_undegraded":"2024-09-10T10:01:57.419510+0000","last_fullsized":"2024-09-10T10:01:57.419510+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:11.341717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431987+0000","last_change":"2024-09-10T10:01:57.430689+0000","last_active":"2024-09-10T10:01:57.431987+0000","last_peered":"2024-09-10T10:01:57.431987+0000","last_clean":"2024-09-10T10:01:57.431987+0000","last_became_active":"2024-09-10T10:01:57.430349+0000","last_became_peered":"2024-09-10T10:01:57.430349+0000","last_unstale":"2024-09-10T10:01:57.431987+0000","last_undegraded":"2024-09-10T10:01:57.431987+0000","last_fullsized":"2024-09-10T10:01:57.431987+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:41.204498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431524+0000","last_change":"2024-09-10T10:01:57.424818+0000","last_active":"2024-09-10T10:01:57.431524+0000","last_peered":"2024-09-10T10:01:57.431524+0000","last_clean":"2024-09-10T10:01:57.431524+0000","last_became_active":"2024-09-10T10:01:57.424469+0000","last_became_peered":"2024-09-10T10:01:57.424469+0000","last_unstale":"2024-09-10T10:01:57.431524+0000","last_undegraded":"2024-09-10T10:01:57.431524+0000","last_fullsized":"2024-09-10T10:01:57.431524+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415184+0000","last_change":"2024-09-10T10:01:57.412902+0000","last_active":"2024-09-10T10:01:57.415184+0000","last_peered":"2024-09-10T10:01:57.415184+0000","last_clean":"2024-09-10T10:01:57.415184+0000","last_became_active":"2024-09-10T10:01:57.412071+0000","last_became_peered":"2024-09-10T10:01:57.412071+0000","last_unstale":"2024-09-10T10:01:57.415184+0000","last_undegraded":"2024-09-10T10:01:57.415184+0000","last_fullsized":"2024-09-10T10:01:57.415184+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:47.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431901+0000","last_change":"2024-09-10T10:01:57.428212+0000","last_active":"2024-09-10T10:01:57.431901+0000","last_peered":"2024-09-10T10:01:57.431901+0000","last_clean":"2024-09-10T10:01:57.431901+0000","last_became_active":"2024-09-10T10:01:57.427898+0000","last_became_peered":"2024-09-10T10:01:57.427898+0000","last_unstale":"2024-09-10T10:01:57.431901+0000","last_undegraded":"2024-09-10T10:01:57.431901+0000","last_fullsized":"2024-09-10T10:01:57.431901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.395624+0000","last_change":"2024-09-10T10:01:56.395624+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.395624+0000","last_undegraded":"2024-09-10T10:01:56.395624+0000","last_fullsized":"2024-09-10T10:01:56.395624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415287+0000","last_change":"2024-09-10T10:01:57.411461+0000","last_active":"2024-09-10T10:01:57.415287+0000","last_peered":"2024-09-10T10:01:57.415287+0000","last_clean":"2024-09-10T10:01:57.415287+0000","last_became_active":"2024-09-10T10:01:57.410729+0000","last_became_peered":"2024-09-10T10:01:57.410729+0000","last_unstale":"2024-09-10T10:01:57.415287+0000","last_undegraded":"2024-09-10T10:01:57.415287+0000","last_fullsized":"2024-09-10T10:01:57.415287+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.839704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394962+0000","last_change":"2024-09-10T10:01:56.394962+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394962+0000","last_undegraded":"2024-09-10T10:01:56.394962+0000","last_fullsized":"2024-09-10T10:01:56.394962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431869+0000","last_change":"2024-09-10T10:01:57.428097+0000","last_active":"2024-09-10T10:01:57.431869+0000","last_peered":"2024-09-10T10:01:57.431869+0000","last_clean":"2024-09-10T10:01:57.431869+0000","last_became_active":"2024-09-10T10:01:57.427838+0000","last_became_peered":"2024-09-10T10:01:57.427838+0000","last_unstale":"2024-09-10T10:01:57.431869+0000","last_undegraded":"2024-09-10T10:01:57.431869+0000","last_fullsized":"2024-09-10T10:01:57.431869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.426209+0000","last_change":"2024-09-10T10:01:57.416701+0000","last_active":"2024-09-10T10:01:57.426209+0000","last_peered":"2024-09-10T10:01:57.426209+0000","last_clean":"2024-09-10T10:01:57.426209+0000","last_became_active":"2024-09-10T10:01:57.416416+0000","last_became_peered":"2024-09-10T10:01:57.416416+0000","last_unstale":"2024-09-10T10:01:57.426209+0000","last_undegraded":"2024-09-10T10:01:57.426209+0000","last_fullsized":"2024-09-10T10:01:57.426209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.508652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419651+0000","last_change":"2024-09-10T10:01:57.412058+0000","last_active":"2024-09-10T10:01:57.419651+0000","last_peered":"2024-09-10T10:01:57.419651+0000","last_clean":"2024-09-10T10:01:57.419651+0000","last_became_active":"2024-09-10T10:01:57.411787+0000","last_became_peered":"2024-09-10T10:01:57.411787+0000","last_unstale":"2024-09-10T10:01:57.419651+0000","last_undegraded":"2024-09-10T10:01:57.419651+0000","last_fullsized":"2024-09-10T10:01:57.419651+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:59.314051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431681+0000","last_change":"2024-09-10T10:01:57.426610+0000","last_active":"2024-09-10T10:01:57.431681+0000","last_peered":"2024-09-10T10:01:57.431681+0000","last_clean":"2024-09-10T10:01:57.431681+0000","last_became_active":"2024-09-10T10:01:57.423435+0000","last_became_peered":"2024-09-10T10:01:57.423435+0000","last_unstale":"2024-09-10T10:01:57.431681+0000","last_undegraded":"2024-09-10T10:01:57.431681+0000","last_fullsized":"2024-09-10T10:01:57.431681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.428911+0000","last_change":"2024-09-10T10:01:56.414321+0000","last_active":"2024-09-10T10:01:57.428911+0000","last_peered":"2024-09-10T10:01:57.428911+0000","last_clean":"2024-09-10T10:01:57.428911+0000","last_became_active":"2024-09-10T10:01:56.414030+0000","last_became_peered":"2024-09-10T10:01:56.414030+0000","last_unstale":"2024-09-10T10:01:57.428911+0000","last_undegraded":"2024-09-10T10:01:57.428911+0000","last_fullsized":"2024-09-10T10:01:57.428911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419094+0000","last_change":"2024-09-10T10:01:57.414555+0000","last_active":"2024-09-10T10:01:57.419094+0000","last_peered":"2024-09-10T10:01:57.419094+0000","last_clean":"2024-09-10T10:01:57.419094+0000","last_became_active":"2024-09-10T10:01:57.413819+0000","last_became_peered":"2024-09-10T10:01:57.413819+0000","last_unstale":"2024-09-10T10:01:57.419094+0000","last_undegraded":"2024-09-10T10:01:57.419094+0000","last_fullsized":"2024-09-10T10:01:57.419094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:39.835600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423916+0000","last_change":"2024-09-10T10:01:56.408386+0000","last_active":"2024-09-10T10:01:57.423916+0000","last_peered":"2024-09-10T10:01:57.423916+0000","last_clean":"2024-09-10T10:01:57.423916+0000","last_became_active":"2024-09-10T10:01:56.408127+0000","last_became_peered":"2024-09-10T10:01:56.408127+0000","last_unstale":"2024-09-10T10:01:57.423916+0000","last_undegraded":"2024-09-10T10:01:57.423916+0000","last_fullsized":"2024-09-10T10:01:57.423916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:59.865257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421159+0000","last_change":"2024-09-10T10:01:57.417292+0000","last_active":"2024-09-10T10:01:57.421159+0000","last_peered":"2024-09-10T10:01:57.421159+0000","last_clean":"2024-09-10T10:01:57.421159+0000","last_became_active":"2024-09-10T10:01:57.416917+0000","last_became_peered":"2024-09-10T10:01:57.416917+0000","last_unstale":"2024-09-10T10:01:57.421159+0000","last_undegraded":"2024-09-10T10:01:57.421159+0000","last_fullsized":"2024-09-10T10:01:57.421159+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:11.341717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422155+0000","last_change":"2024-09-10T10:01:56.409023+0000","last_active":"2024-09-10T10:01:57.422155+0000","last_peered":"2024-09-10T10:01:57.422155+0000","last_clean":"2024-09-10T10:01:57.422155+0000","last_became_active":"2024-09-10T10:01:56.408748+0000","last_became_peered":"2024-09-10T10:01:56.408748+0000","last_unstale":"2024-09-10T10:01:57.422155+0000","last_undegraded":"2024-09-10T10:01:57.422155+0000","last_fullsized":"2024-09-10T10:01:57.422155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:50.426131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413959+0000","last_change":"2024-09-10T10:01:57.408727+0000","last_active":"2024-09-10T10:01:57.413959+0000","last_peered":"2024-09-10T10:01:57.413959+0000","last_clean":"2024-09-10T10:01:57.413959+0000","last_became_active":"2024-09-10T10:01:57.408448+0000","last_became_peered":"2024-09-10T10:01:57.408448+0000","last_unstale":"2024-09-10T10:01:57.413959+0000","last_undegraded":"2024-09-10T10:01:57.413959+0000","last_fullsized":"2024-09-10T10:01:57.413959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427217+0000","last_change":"2024-09-10T10:01:56.409387+0000","last_active":"2024-09-10T10:01:57.427217+0000","last_peered":"2024-09-10T10:01:57.427217+0000","last_clean":"2024-09-10T10:01:57.427217+0000","last_became_active":"2024-09-10T10:01:56.407044+0000","last_became_peered":"2024-09-10T10:01:56.407044+0000","last_unstale":"2024-09-10T10:01:57.427217+0000","last_undegraded":"2024-09-10T10:01:57.427217+0000","last_fullsized":"2024-09-10T10:01:57.427217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423211+0000","last_change":"2024-09-10T10:01:57.420630+0000","last_active":"2024-09-10T10:01:57.423211+0000","last_peered":"2024-09-10T10:01:57.423211+0000","last_clean":"2024-09-10T10:01:57.423211+0000","last_became_active":"2024-09-10T10:01:57.416989+0000","last_became_peered":"2024-09-10T10:01:57.416989+0000","last_unstale":"2024-09-10T10:01:57.423211+0000","last_undegraded":"2024-09-10T10:01:57.423211+0000","last_fullsized":"2024-09-10T10:01:57.423211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:14.901947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.407559+0000","last_change":"2024-09-10T10:01:56.404458+0000","last_active":"2024-09-10T10:01:56.407559+0000","last_peered":"2024-09-10T10:01:56.407559+0000","last_clean":"2024-09-10T10:01:56.407559+0000","last_became_active":"2024-09-10T10:01:56.404205+0000","last_became_peered":"2024-09-10T10:01:56.404205+0000","last_unstale":"2024-09-10T10:01:56.407559+0000","last_undegraded":"2024-09-10T10:01:56.407559+0000","last_fullsized":"2024-09-10T10:01:56.407559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:14.034052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417285+0000","last_change":"2024-09-10T10:01:57.412841+0000","last_active":"2024-09-10T10:01:57.417285+0000","last_peered":"2024-09-10T10:01:57.417285+0000","last_clean":"2024-09-10T10:01:57.417285+0000","last_became_active":"2024-09-10T10:01:57.412551+0000","last_became_peered":"2024-09-10T10:01:57.412551+0000","last_unstale":"2024-09-10T10:01:57.417285+0000","last_undegraded":"2024-09-10T10:01:57.417285+0000","last_fullsized":"2024-09-10T10:01:57.417285+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:59.314051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421685+0000","last_change":"2024-09-10T10:01:56.398879+0000","last_active":"2024-09-10T10:01:57.421685+0000","last_peered":"2024-09-10T10:01:57.421685+0000","last_clean":"2024-09-10T10:01:57.421685+0000","last_became_active":"2024-09-10T10:01:56.398576+0000","last_became_peered":"2024-09-10T10:01:56.398576+0000","last_unstale":"2024-09-10T10:01:57.421685+0000","last_undegraded":"2024-09-10T10:01:57.421685+0000","last_fullsized":"2024-09-10T10:01:57.421685+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:01.154272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417679+0000","last_change":"2024-09-10T10:01:57.412438+0000","last_active":"2024-09-10T10:01:57.417679+0000","last_peered":"2024-09-10T10:01:57.417679+0000","last_clean":"2024-09-10T10:01:57.417679+0000","last_became_active":"2024-09-10T10:01:57.412174+0000","last_became_peered":"2024-09-10T10:01:57.412174+0000","last_unstale":"2024-09-10T10:01:57.417679+0000","last_undegraded":"2024-09-10T10:01:57.417679+0000","last_fullsized":"2024-09-10T10:01:57.417679+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:46.178171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.391826+0000","last_change":"2024-09-10T10:01:56.406654+0000","last_active":"2024-09-10T10:01:57.391826+0000","last_peered":"2024-09-10T10:01:57.391826+0000","last_clean":"2024-09-10T10:01:57.391826+0000","last_became_active":"2024-09-10T10:01:56.406375+0000","last_became_peered":"2024-09-10T10:01:56.406375+0000","last_unstale":"2024-09-10T10:01:57.391826+0000","last_undegraded":"2024-09-10T10:01:57.391826+0000","last_fullsized":"2024-09-10T10:01:57.391826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:07.017690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414349+0000","last_change":"2024-09-10T10:01:57.408804+0000","last_active":"2024-09-10T10:01:57.414349+0000","last_peered":"2024-09-10T10:01:57.414349+0000","last_clean":"2024-09-10T10:01:57.414349+0000","last_became_active":"2024-09-10T10:01:57.405976+0000","last_became_peered":"2024-09-10T10:01:57.405976+0000","last_unstale":"2024-09-10T10:01:57.414349+0000","last_undegraded":"2024-09-10T10:01:57.414349+0000","last_fullsized":"2024-09-10T10:01:57.414349+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398400+0000","last_change":"2024-09-10T10:01:56.411631+0000","last_active":"2024-09-10T10:01:57.398400+0000","last_peered":"2024-09-10T10:01:57.398400+0000","last_clean":"2024-09-10T10:01:57.398400+0000","last_became_active":"2024-09-10T10:01:56.411378+0000","last_became_peered":"2024-09-10T10:01:56.411378+0000","last_unstale":"2024-09-10T10:01:57.398400+0000","last_undegraded":"2024-09-10T10:01:57.398400+0000","last_fullsized":"2024-09-10T10:01:57.398400+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:38.118765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424610+0000","last_change":"2024-09-10T10:01:57.422782+0000","last_active":"2024-09-10T10:01:57.424610+0000","last_peered":"2024-09-10T10:01:57.424610+0000","last_clean":"2024-09-10T10:01:57.424610+0000","last_became_active":"2024-09-10T10:01:57.420706+0000","last_became_peered":"2024-09-10T10:01:57.420706+0000","last_unstale":"2024-09-10T10:01:57.424610+0000","last_undegraded":"2024-09-10T10:01:57.424610+0000","last_fullsized":"2024-09-10T10:01:57.424610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398697+0000","last_change":"2024-09-10T10:01:56.403288+0000","last_active":"2024-09-10T10:01:57.398697+0000","last_peered":"2024-09-10T10:01:57.398697+0000","last_clean":"2024-09-10T10:01:57.398697+0000","last_became_active":"2024-09-10T10:01:56.402990+0000","last_became_peered":"2024-09-10T10:01:56.402990+0000","last_unstale":"2024-09-10T10:01:57.398697+0000","last_undegraded":"2024-09-10T10:01:57.398697+0000","last_fullsized":"2024-09-10T10:01:57.398697+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431695+0000","last_change":"2024-09-10T10:01:57.425385+0000","last_active":"2024-09-10T10:01:57.431695+0000","last_peered":"2024-09-10T10:01:57.431695+0000","last_clean":"2024-09-10T10:01:57.431695+0000","last_became_active":"2024-09-10T10:01:57.425032+0000","last_became_peered":"2024-09-10T10:01:57.425032+0000","last_unstale":"2024-09-10T10:01:57.431695+0000","last_undegraded":"2024-09-10T10:01:57.431695+0000","last_fullsized":"2024-09-10T10:01:57.431695+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.395612+0000","last_change":"2024-09-10T10:01:56.407609+0000","last_active":"2024-09-10T10:01:57.395612+0000","last_peered":"2024-09-10T10:01:57.395612+0000","last_clean":"2024-09-10T10:01:57.395612+0000","last_became_active":"2024-09-10T10:01:56.405401+0000","last_became_peered":"2024-09-10T10:01:56.405401+0000","last_unstale":"2024-09-10T10:01:57.395612+0000","last_undegraded":"2024-09-10T10:01:57.395612+0000","last_fullsized":"2024-09-10T10:01:57.395612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419650+0000","last_change":"2024-09-10T10:01:57.412223+0000","last_active":"2024-09-10T10:01:57.419650+0000","last_peered":"2024-09-10T10:01:57.419650+0000","last_clean":"2024-09-10T10:01:57.419650+0000","last_became_active":"2024-09-10T10:01:57.411477+0000","last_became_peered":"2024-09-10T10:01:57.411477+0000","last_unstale":"2024-09-10T10:01:57.419650+0000","last_undegraded":"2024-09-10T10:01:57.419650+0000","last_fullsized":"2024-09-10T10:01:57.419650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:47.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.407993+0000","last_change":"2024-09-10T10:01:56.406998+0000","last_active":"2024-09-10T10:01:56.407993+0000","last_peered":"2024-09-10T10:01:56.407993+0000","last_clean":"2024-09-10T10:01:56.407993+0000","last_became_active":"2024-09-10T10:01:56.406722+0000","last_became_peered":"2024-09-10T10:01:56.406722+0000","last_unstale":"2024-09-10T10:01:56.407993+0000","last_undegraded":"2024-09-10T10:01:56.407993+0000","last_fullsized":"2024-09-10T10:01:56.407993+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:27.783209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414220+0000","last_change":"2024-09-10T10:01:57.409970+0000","last_active":"2024-09-10T10:01:57.414220+0000","last_peered":"2024-09-10T10:01:57.414220+0000","last_clean":"2024-09-10T10:01:57.414220+0000","last_became_active":"2024-09-10T10:01:57.406199+0000","last_became_peered":"2024-09-10T10:01:57.406199+0000","last_unstale":"2024-09-10T10:01:57.414220+0000","last_undegraded":"2024-09-10T10:01:57.414220+0000","last_fullsized":"2024-09-10T10:01:57.414220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:41.660285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.426946+0000","last_change":"2024-09-10T10:01:56.410726+0000","last_active":"2024-09-10T10:01:57.426946+0000","last_peered":"2024-09-10T10:01:57.426946+0000","last_clean":"2024-09-10T10:01:57.426946+0000","last_became_active":"2024-09-10T10:01:56.410239+0000","last_became_peered":"2024-09-10T10:01:56.410239+0000","last_unstale":"2024-09-10T10:01:57.426946+0000","last_undegraded":"2024-09-10T10:01:57.426946+0000","last_fullsized":"2024-09-10T10:01:57.426946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:44.033036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.429665+0000","last_change":"2024-09-10T10:01:57.423932+0000","last_active":"2024-09-10T10:01:57.429665+0000","last_peered":"2024-09-10T10:01:57.429665+0000","last_clean":"2024-09-10T10:01:57.429665+0000","last_became_active":"2024-09-10T10:01:57.423635+0000","last_became_peered":"2024-09-10T10:01:57.423635+0000","last_unstale":"2024-09-10T10:01:57.429665+0000","last_undegraded":"2024-09-10T10:01:57.429665+0000","last_fullsized":"2024-09-10T10:01:57.429665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431982+0000","last_change":"2024-09-10T10:01:56.411145+0000","last_active":"2024-09-10T10:01:57.431982+0000","last_peered":"2024-09-10T10:01:57.431982+0000","last_clean":"2024-09-10T10:01:57.431982+0000","last_became_active":"2024-09-10T10:01:56.410821+0000","last_became_peered":"2024-09-10T10:01:56.410821+0000","last_unstale":"2024-09-10T10:01:57.431982+0000","last_undegraded":"2024-09-10T10:01:57.431982+0000","last_fullsized":"2024-09-10T10:01:57.431982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:34.635430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431813+0000","last_change":"2024-09-10T10:01:57.427059+0000","last_active":"2024-09-10T10:01:57.431813+0000","last_peered":"2024-09-10T10:01:57.431813+0000","last_clean":"2024-09-10T10:01:57.431813+0000","last_became_active":"2024-09-10T10:01:57.423512+0000","last_became_peered":"2024-09-10T10:01:57.423512+0000","last_unstale":"2024-09-10T10:01:57.431813+0000","last_undegraded":"2024-09-10T10:01:57.431813+0000","last_fullsized":"2024-09-10T10:01:57.431813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394598+0000","last_change":"2024-09-10T10:01:56.405376+0000","last_active":"2024-09-10T10:01:57.394598+0000","last_peered":"2024-09-10T10:01:57.394598+0000","last_clean":"2024-09-10T10:01:57.394598+0000","last_became_active":"2024-09-10T10:01:56.405056+0000","last_became_peered":"2024-09-10T10:01:56.405056+0000","last_unstale":"2024-09-10T10:01:57.394598+0000","last_undegraded":"2024-09-10T10:01:57.394598+0000","last_fullsized":"2024-09-10T10:01:57.394598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:56.936736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415321+0000","last_change":"2024-09-10T10:01:57.411055+0000","last_active":"2024-09-10T10:01:57.415321+0000","last_peered":"2024-09-10T10:01:57.415321+0000","last_clean":"2024-09-10T10:01:57.415321+0000","last_became_active":"2024-09-10T10:01:57.410789+0000","last_became_peered":"2024-09-10T10:01:57.410789+0000","last_unstale":"2024-09-10T10:01:57.415321+0000","last_undegraded":"2024-09-10T10:01:57.415321+0000","last_fullsized":"2024-09-10T10:01:57.415321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.412528+0000","last_change":"2024-09-10T10:01:56.412704+0000","last_active":"2024-09-10T10:01:57.412528+0000","last_peered":"2024-09-10T10:01:57.412528+0000","last_clean":"2024-09-10T10:01:57.412528+0000","last_became_active":"2024-09-10T10:01:56.410023+0000","last_became_peered":"2024-09-10T10:01:56.410023+0000","last_unstale":"2024-09-10T10:01:57.412528+0000","last_undegraded":"2024-09-10T10:01:57.412528+0000","last_fullsized":"2024-09-10T10:01:57.412528+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:57.159878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415068+0000","last_change":"2024-09-10T10:01:57.407954+0000","last_active":"2024-09-10T10:01:57.415068+0000","last_peered":"2024-09-10T10:01:57.415068+0000","last_clean":"2024-09-10T10:01:57.415068+0000","last_became_active":"2024-09-10T10:01:57.407651+0000","last_became_peered":"2024-09-10T10:01:57.407651+0000","last_unstale":"2024-09-10T10:01:57.415068+0000","last_undegraded":"2024-09-10T10:01:57.415068+0000","last_fullsized":"2024-09-10T10:01:57.415068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:16.908619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.430713+0000","last_change":"2024-09-10T10:01:56.410104+0000","last_active":"2024-09-10T10:01:57.430713+0000","last_peered":"2024-09-10T10:01:57.430713+0000","last_clean":"2024-09-10T10:01:57.430713+0000","last_became_active":"2024-09-10T10:01:56.409756+0000","last_became_peered":"2024-09-10T10:01:56.409756+0000","last_unstale":"2024-09-10T10:01:57.430713+0000","last_undegraded":"2024-09-10T10:01:57.430713+0000","last_fullsized":"2024-09-10T10:01:57.430713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.409223+0000","last_change":"2024-09-10T10:01:57.405556+0000","last_active":"2024-09-10T10:01:57.409223+0000","last_peered":"2024-09-10T10:01:57.409223+0000","last_clean":"2024-09-10T10:01:57.409223+0000","last_became_active":"2024-09-10T10:01:57.405222+0000","last_became_peered":"2024-09-10T10:01:57.405222+0000","last_unstale":"2024-09-10T10:01:57.409223+0000","last_undegraded":"2024-09-10T10:01:57.409223+0000","last_fullsized":"2024-09-10T10:01:57.409223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:03.020211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.425896+0000","last_change":"2024-09-10T10:01:56.401752+0000","last_active":"2024-09-10T10:01:57.425896+0000","last_peered":"2024-09-10T10:01:57.425896+0000","last_clean":"2024-09-10T10:01:57.425896+0000","last_became_active":"2024-09-10T10:01:56.401508+0000","last_became_peered":"2024-09-10T10:01:56.401508+0000","last_unstale":"2024-09-10T10:01:57.425896+0000","last_undegraded":"2024-09-10T10:01:57.425896+0000","last_fullsized":"2024-09-10T10:01:57.425896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:52.212395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431672+0000","last_change":"2024-09-10T10:01:57.425326+0000","last_active":"2024-09-10T10:01:57.431672+0000","last_peered":"2024-09-10T10:01:57.431672+0000","last_clean":"2024-09-10T10:01:57.431672+0000","last_became_active":"2024-09-10T10:01:57.424606+0000","last_became_peered":"2024-09-10T10:01:57.424606+0000","last_unstale":"2024-09-10T10:01:57.431672+0000","last_undegraded":"2024-09-10T10:01:57.431672+0000","last_fullsized":"2024-09-10T10:01:57.431672+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:23.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422525+0000","last_change":"2024-09-10T10:01:56.409357+0000","last_active":"2024-09-10T10:01:57.422525+0000","last_peered":"2024-09-10T10:01:57.422525+0000","last_clean":"2024-09-10T10:01:57.422525+0000","last_became_active":"2024-09-10T10:01:56.409062+0000","last_became_peered":"2024-09-10T10:01:56.409062+0000","last_unstale":"2024-09-10T10:01:57.422525+0000","last_undegraded":"2024-09-10T10:01:57.422525+0000","last_fullsized":"2024-09-10T10:01:57.422525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:29.657091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413399+0000","last_change":"2024-09-10T10:01:57.401546+0000","last_active":"2024-09-10T10:01:57.413399+0000","last_peered":"2024-09-10T10:01:57.413399+0000","last_clean":"2024-09-10T10:01:57.413399+0000","last_became_active":"2024-09-10T10:01:57.401293+0000","last_became_peered":"2024-09-10T10:01:57.401293+0000","last_unstale":"2024-09-10T10:01:57.413399+0000","last_undegraded":"2024-09-10T10:01:57.413399+0000","last_fullsized":"2024-09-10T10:01:57.413399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:19.413048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427627+0000","last_change":"2024-09-10T10:01:56.399162+0000","last_active":"2024-09-10T10:01:57.427627+0000","last_peered":"2024-09-10T10:01:57.427627+0000","last_clean":"2024-09-10T10:01:57.427627+0000","last_became_active":"2024-09-10T10:01:56.398886+0000","last_became_peered":"2024-09-10T10:01:56.398886+0000","last_unstale":"2024-09-10T10:01:57.427627+0000","last_undegraded":"2024-09-10T10:01:57.427627+0000","last_fullsized":"2024-09-10T10:01:57.427627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414198+0000","last_change":"2024-09-10T10:01:57.409275+0000","last_active":"2024-09-10T10:01:57.414198+0000","last_peered":"2024-09-10T10:01:57.414198+0000","last_clean":"2024-09-10T10:01:57.414198+0000","last_became_active":"2024-09-10T10:01:57.409007+0000","last_became_peered":"2024-09-10T10:01:57.409007+0000","last_unstale":"2024-09-10T10:01:57.414198+0000","last_undegraded":"2024-09-10T10:01:57.414198+0000","last_fullsized":"2024-09-10T10:01:57.414198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:17.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.405547+0000","last_change":"2024-09-10T10:01:56.414600+0000","last_active":"2024-09-10T10:01:57.405547+0000","last_peered":"2024-09-10T10:01:57.405547+0000","last_clean":"2024-09-10T10:01:57.405547+0000","last_became_active":"2024-09-10T10:01:56.414312+0000","last_became_peered":"2024-09-10T10:01:56.414312+0000","last_unstale":"2024-09-10T10:01:57.405547+0000","last_undegraded":"2024-09-10T10:01:57.405547+0000","last_fullsized":"2024-09-10T10:01:57.405547+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:13.899397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394589+0000","last_change":"2024-09-10T10:01:56.394589+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394589+0000","last_undegraded":"2024-09-10T10:01:56.394589+0000","last_fullsized":"2024-09-10T10:01:56.394589+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394540+0000","last_change":"2024-09-10T10:01:56.399440+0000","last_active":"2024-09-10T10:01:57.394540+0000","last_peered":"2024-09-10T10:01:57.394540+0000","last_clean":"2024-09-10T10:01:57.394540+0000","last_became_active":"2024-09-10T10:01:56.399153+0000","last_became_peered":"2024-09-10T10:01:56.399153+0000","last_unstale":"2024-09-10T10:01:57.394540+0000","last_undegraded":"2024-09-10T10:01:57.394540+0000","last_fullsized":"2024-09-10T10:01:57.394540+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:21.685329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.393504+0000","last_change":"2024-09-10T10:01:56.393504+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.393504+0000","last_undegraded":"2024-09-10T10:01:56.393504+0000","last_fullsized":"2024-09-10T10:01:56.393504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.412691+0000","last_change":"2024-09-10T10:01:56.410145+0000","last_active":"2024-09-10T10:01:57.412691+0000","last_peered":"2024-09-10T10:01:57.412691+0000","last_clean":"2024-09-10T10:01:57.412691+0000","last_became_active":"2024-09-10T10:01:56.409854+0000","last_became_peered":"2024-09-10T10:01:56.409854+0000","last_unstale":"2024-09-10T10:01:57.412691+0000","last_undegraded":"2024-09-10T10:01:57.412691+0000","last_fullsized":"2024-09-10T10:01:57.412691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:23.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.393865+0000","last_change":"2024-09-10T10:01:56.393865+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.393865+0000","last_undegraded":"2024-09-10T10:01:56.393865+0000","last_fullsized":"2024-09-10T10:01:56.393865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.395889+0000","last_change":"2024-09-10T10:01:56.404958+0000","last_active":"2024-09-10T10:01:57.395889+0000","last_peered":"2024-09-10T10:01:57.395889+0000","last_clean":"2024-09-10T10:01:57.395889+0000","last_became_active":"2024-09-10T10:01:56.404619+0000","last_became_peered":"2024-09-10T10:01:56.404619+0000","last_unstale":"2024-09-10T10:01:57.395889+0000","last_undegraded":"2024-09-10T10:01:57.395889+0000","last_fullsized":"2024-09-10T10:01:57.395889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415096+0000","last_change":"2024-09-10T10:01:57.409907+0000","last_active":"2024-09-10T10:01:57.415096+0000","last_peered":"2024-09-10T10:01:57.415096+0000","last_clean":"2024-09-10T10:01:57.415096+0000","last_became_active":"2024-09-10T10:01:57.409375+0000","last_became_peered":"2024-09-10T10:01:57.409375+0000","last_unstale":"2024-09-10T10:01:57.415096+0000","last_undegraded":"2024-09-10T10:01:57.415096+0000","last_fullsized":"2024-09-10T10:01:57.415096+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:23.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.425893+0000","last_change":"2024-09-10T10:01:56.400929+0000","last_active":"2024-09-10T10:01:57.425893+0000","last_peered":"2024-09-10T10:01:57.425893+0000","last_clean":"2024-09-10T10:01:57.425893+0000","last_became_active":"2024-09-10T10:01:56.400665+0000","last_became_peered":"2024-09-10T10:01:56.400665+0000","last_unstale":"2024-09-10T10:01:57.425893+0000","last_undegraded":"2024-09-10T10:01:57.425893+0000","last_fullsized":"2024-09-10T10:01:57.425893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:32.570015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431796+0000","last_change":"2024-09-10T10:01:57.426884+0000","last_active":"2024-09-10T10:01:57.431796+0000","last_peered":"2024-09-10T10:01:57.431796+0000","last_clean":"2024-09-10T10:01:57.431796+0000","last_became_active":"2024-09-10T10:01:57.426621+0000","last_became_peered":"2024-09-10T10:01:57.426621+0000","last_unstale":"2024-09-10T10:01:57.431796+0000","last_undegraded":"2024-09-10T10:01:57.431796+0000","last_fullsized":"2024-09-10T10:01:57.431796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:35.517149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413728+0000","last_change":"2024-09-10T10:01:56.410855+0000","last_active":"2024-09-10T10:01:57.413728+0000","last_peered":"2024-09-10T10:01:57.413728+0000","last_clean":"2024-09-10T10:01:57.413728+0000","last_became_active":"2024-09-10T10:01:56.410597+0000","last_became_peered":"2024-09-10T10:01:56.410597+0000","last_unstale":"2024-09-10T10:01:57.413728+0000","last_undegraded":"2024-09-10T10:01:57.413728+0000","last_fullsized":"2024-09-10T10:01:57.413728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.934821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.429750+0000","last_change":"2024-09-10T10:01:57.423850+0000","last_active":"2024-09-10T10:01:57.429750+0000","last_peered":"2024-09-10T10:01:57.429750+0000","last_clean":"2024-09-10T10:01:57.429750+0000","last_became_active":"2024-09-10T10:01:57.423309+0000","last_became_peered":"2024-09-10T10:01:57.423309+0000","last_unstale":"2024-09-10T10:01:57.429750+0000","last_undegraded":"2024-09-10T10:01:57.429750+0000","last_fullsized":"2024-09-10T10:01:57.429750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.405952+0000","last_change":"2024-09-10T10:01:56.409831+0000","last_active":"2024-09-10T10:01:57.405952+0000","last_peered":"2024-09-10T10:01:57.405952+0000","last_clean":"2024-09-10T10:01:57.405952+0000","last_became_active":"2024-09-10T10:01:56.409512+0000","last_became_peered":"2024-09-10T10:01:56.409512+0000","last_unstale":"2024-09-10T10:01:57.405952+0000","last_undegraded":"2024-09-10T10:01:57.405952+0000","last_fullsized":"2024-09-10T10:01:57.405952+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:05.784097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431774+0000","last_change":"2024-09-10T10:01:57.427760+0000","last_active":"2024-09-10T10:01:57.431774+0000","last_peered":"2024-09-10T10:01:57.431774+0000","last_clean":"2024-09-10T10:01:57.431774+0000","last_became_active":"2024-09-10T10:01:57.427511+0000","last_became_peered":"2024-09-10T10:01:57.427511+0000","last_unstale":"2024-09-10T10:01:57.431774+0000","last_undegraded":"2024-09-10T10:01:57.431774+0000","last_fullsized":"2024-09-10T10:01:57.431774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:52.470311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.397237+0000","last_change":"2024-09-10T10:01:56.402970+0000","last_active":"2024-09-10T10:01:57.397237+0000","last_peered":"2024-09-10T10:01:57.397237+0000","last_clean":"2024-09-10T10:01:57.397237+0000","last_became_active":"2024-09-10T10:01:56.402500+0000","last_became_peered":"2024-09-10T10:01:56.402500+0000","last_unstale":"2024-09-10T10:01:57.397237+0000","last_undegraded":"2024-09-10T10:01:57.397237+0000","last_fullsized":"2024-09-10T10:01:57.397237+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:26.744793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415252+0000","last_change":"2024-09-10T10:01:57.409373+0000","last_active":"2024-09-10T10:01:57.415252+0000","last_peered":"2024-09-10T10:01:57.415252+0000","last_clean":"2024-09-10T10:01:57.415252+0000","last_became_active":"2024-09-10T10:01:57.409114+0000","last_became_peered":"2024-09-10T10:01:57.409114+0000","last_unstale":"2024-09-10T10:01:57.415252+0000","last_undegraded":"2024-09-10T10:01:57.415252+0000","last_fullsized":"2024-09-10T10:01:57.415252+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:53.254884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.400447+0000","last_change":"2024-09-10T10:01:56.411209+0000","last_active":"2024-09-10T10:01:57.400447+0000","last_peered":"2024-09-10T10:01:57.400447+0000","last_clean":"2024-09-10T10:01:57.400447+0000","last_became_active":"2024-09-10T10:01:56.410808+0000","last_became_peered":"2024-09-10T10:01:56.410808+0000","last_unstale":"2024-09-10T10:01:57.400447+0000","last_undegraded":"2024-09-10T10:01:57.400447+0000","last_fullsized":"2024-09-10T10:01:57.400447+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:39.392910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421360+0000","last_change":"2024-09-10T10:01:57.418730+0000","last_active":"2024-09-10T10:01:57.421360+0000","last_peered":"2024-09-10T10:01:57.421360+0000","last_clean":"2024-09-10T10:01:57.421360+0000","last_became_active":"2024-09-10T10:01:57.418395+0000","last_became_peered":"2024-09-10T10:01:57.418395+0000","last_unstale":"2024-09-10T10:01:57.421360+0000","last_undegraded":"2024-09-10T10:01:57.421360+0000","last_fullsized":"2024-09-10T10:01:57.421360+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:58.162428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.391574+0000","last_change":"2024-09-10T10:01:56.409314+0000","last_active":"2024-09-10T10:01:57.391574+0000","last_peered":"2024-09-10T10:01:57.391574+0000","last_clean":"2024-09-10T10:01:57.391574+0000","last_became_active":"2024-09-10T10:01:56.409077+0000","last_became_peered":"2024-09-10T10:01:56.409077+0000","last_unstale":"2024-09-10T10:01:57.391574+0000","last_undegraded":"2024-09-10T10:01:57.391574+0000","last_fullsized":"2024-09-10T10:01:57.391574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.220387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.416721+0000","last_change":"2024-09-10T10:01:57.407297+0000","last_active":"2024-09-10T10:01:57.416721+0000","last_peered":"2024-09-10T10:01:57.416721+0000","last_clean":"2024-09-10T10:01:57.416721+0000","last_became_active":"2024-09-10T10:01:57.407030+0000","last_became_peered":"2024-09-10T10:01:57.407030+0000","last_unstale":"2024-09-10T10:01:57.416721+0000","last_undegraded":"2024-09-10T10:01:57.416721+0000","last_fullsized":"2024-09-10T10:01:57.416721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:33.572565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398463+0000","last_change":"2024-09-10T10:01:56.411518+0000","last_active":"2024-09-10T10:01:57.398463+0000","last_peered":"2024-09-10T10:01:57.398463+0000","last_clean":"2024-09-10T10:01:57.398463+0000","last_became_active":"2024-09-10T10:01:56.410515+0000","last_became_peered":"2024-09-10T10:01:56.410515+0000","last_unstale":"2024-09-10T10:01:57.398463+0000","last_undegraded":"2024-09-10T10:01:57.398463+0000","last_fullsized":"2024-09-10T10:01:57.398463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421441+0000","last_change":"2024-09-10T10:01:57.408569+0000","last_active":"2024-09-10T10:01:57.421441+0000","last_peered":"2024-09-10T10:01:57.421441+0000","last_clean":"2024-09-10T10:01:57.421441+0000","last_became_active":"2024-09-10T10:01:57.408302+0000","last_became_peered":"2024-09-10T10:01:57.408302+0000","last_unstale":"2024-09-10T10:01:57.421441+0000","last_undegraded":"2024-09-10T10:01:57.421441+0000","last_fullsized":"2024-09-10T10:01:57.421441+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:39.030596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394334+0000","last_change":"2024-09-10T10:01:56.407472+0000","last_active":"2024-09-10T10:01:57.394334+0000","last_peered":"2024-09-10T10:01:57.394334+0000","last_clean":"2024-09-10T10:01:57.394334+0000","last_became_active":"2024-09-10T10:01:56.407112+0000","last_became_peered":"2024-09-10T10:01:56.407112+0000","last_unstale":"2024-09-10T10:01:57.394334+0000","last_undegraded":"2024-09-10T10:01:57.394334+0000","last_fullsized":"2024-09-10T10:01:57.394334+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:36.230334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394297+0000","last_change":"2024-09-10T10:01:56.394297+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394297+0000","last_undegraded":"2024-09-10T10:01:56.394297+0000","last_fullsized":"2024-09-10T10:01:56.394297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427116+0000","last_change":"2024-09-10T10:01:56.408027+0000","last_active":"2024-09-10T10:01:57.427116+0000","last_peered":"2024-09-10T10:01:57.427116+0000","last_clean":"2024-09-10T10:01:57.427116+0000","last_became_active":"2024-09-10T10:01:56.407749+0000","last_became_peered":"2024-09-10T10:01:56.407749+0000","last_unstale":"2024-09-10T10:01:57.427116+0000","last_undegraded":"2024-09-10T10:01:57.427116+0000","last_fullsized":"2024-09-10T10:01:57.427116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:42.979120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":252,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.403667+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:01:57.403667+0000","last_peered":"2024-09-10T10:01:57.403667+0000","last_clean":"2024-09-10T10:01:57.403667+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:01:57.403667+0000","last_undegraded":"2024-09-10T10:01:57.403667+0000","last_fullsized":"2024-09-10T10:01:57.403667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":930573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":341201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":929957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:01.864 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:02:02.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.040+0000 7f3042200640 1 Processor -- start 2024-09-10T10:02:02.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.040+0000 7f3042200640 1 -- start start 2024-09-10T10:02:02.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.040+0000 7f3042200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f303c15e830 0x7f303c0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.040+0000 7f3042200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.040+0000 7f3042200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303c0af160 con 0x7f303c15e450 2024-09-10T10:02:02.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f3042200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f303c073290 con 0x7f303c15e830 2024-09-10T10:02:02.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f3042200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f303c06e5e0 con 0x7f303c0aed80 2024-09-10T10:02:02.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f303c15e830 0x7f303c0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f303c15e830 0x7f303c0ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44068/0 (socket says 172.21.15.110:44068) 2024-09-10T10:02:02.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303be00640 1 -- 172.21.15.110:0/2479764746 learned_addr learned my addr 172.21.15.110:0/2479764746 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:02.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f3041800640 1 --2- 172.21.15.110:0/2479764746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303be00640 1 -- 172.21.15.110:0/2479764746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 msgr2=0x7f303c0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:02.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303be00640 1 --2- 172.21.15.110:0/2479764746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c0b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303be00640 1 -- 172.21.15.110:0/2479764746 >> v1:172.21.15.145:6789/0 conn(0x7f303c15e450 legacy=0x7f303c072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:02.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303be00640 1 -- 172.21.15.110:0/2479764746 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c06e8c0 con 0x7f303c15e830 2024-09-10T10:02:02.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f3041800640 1 --2- 172.21.15.110:0/2479764746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c0b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:02.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303be00640 1 --2- 172.21.15.110:0/2479764746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c0ae840 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3028009870 tx=0x7f302801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbe3680c4e24106 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303b400640 1 -- 172.21.15.110:0/2479764746 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30280aa610 con 0x7f303c15e830 2024-09-10T10:02:02.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303b400640 1 -- 172.21.15.110:0/2479764746 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30280af070 con 0x7f303c15e830 2024-09-10T10:02:02.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.044+0000 7f303b400640 1 -- 172.21.15.110:0/2479764746 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30280aa9d0 con 0x7f303c15e830 2024-09-10T10:02:02.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- 172.21.15.110:0/2479764746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 msgr2=0x7f303c0ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 --2- 172.21.15.110:0/2479764746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c0ae840 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3028009870 tx=0x7f302801c4f0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- 172.21.15.110:0/2479764746 shutdown_connections 2024-09-10T10:02:02.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 --2- 172.21.15.110:0/2479764746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c0b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 --2- 172.21.15.110:0/2479764746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c0ae840 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- 172.21.15.110:0/2479764746 >> 172.21.15.110:0/2479764746 conn(0x7f303c06cd60 msgr2=0x7f303c06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:02.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- 172.21.15.110:0/2479764746 shutdown_connections 2024-09-10T10:02:02.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- 172.21.15.110:0/2479764746 wait complete. 2024-09-10T10:02:02.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 Processor -- start 2024-09-10T10:02:02.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- start start 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c15e450 0x7f303c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f303c06e3c0 con 0x7f303c15e830 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f303c06e240 con 0x7f303c15e450 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3042200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f303c0b52a0 con 0x7f303c0aed80 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3041800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3041800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44076/0 (socket says 172.21.15.110:44076) 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3041800640 1 -- 172.21.15.110:0/624359882 learned_addr learned my addr 172.21.15.110:0/624359882 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:02.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f303be00640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c15e450 0x7f303c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.048+0000 7f3040e00640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3041800640 1 -- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 msgr2=0x7f303c159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3041800640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3041800640 1 -- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c15e450 msgr2=0x7f303c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3041800640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c15e450 0x7f303c0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3041800640 1 -- 172.21.15.110:0/624359882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c1581a0 con 0x7f303c15e830 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f303be00640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c15e450 0x7f303c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3041800640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c157950 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f303400c7f0 tx=0x7f303400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3038c00640 1 -- 172.21.15.110:0/624359882 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034015070 con 0x7f303c15e830 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3038c00640 1 -- 172.21.15.110:0/624359882 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3034007860 con 0x7f303c15e830 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c158480 con 0x7f303c15e830 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3038c00640 1 -- 172.21.15.110:0/624359882 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034007c50 con 0x7f303c15e830 2024-09-10T10:02:02.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c158a40 con 0x7f303c15e830 2024-09-10T10:02:02.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303c0b5e00 con 0x7f303c15e830 2024-09-10T10:02:02.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3038c00640 1 -- 172.21.15.110:0/624359882 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f3034018070 con 0x7f303c15e830 2024-09-10T10:02:02.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3038c00640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f301c067c20 0x7f301c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3038c00640 1 -- 172.21.15.110:0/624359882 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f30340864a0 con 0x7f303c15e830 2024-09-10T10:02:02.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.052+0000 7f3040e00640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f301c067c20 0x7f301c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.056+0000 7f3040e00640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f301c067c20 0x7f301c06a0e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f302c005530 tx=0x7f302c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.056+0000 7f3038c00640 1 -- 172.21.15.110:0/624359882 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3034056ac0 con 0x7f303c15e830 2024-09-10T10:02:02.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.212+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f303c0b5b20 con 0x7f301c067c20 2024-09-10T10:02:02.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.216+0000 7f3038c00640 1 -- 172.21.15.110:0/624359882 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251428 (secure 0 0 0) 0x7f303c0b5b20 con 0x7f301c067c20 2024-09-10T10:02:02.223 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:02.225 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f301c067c20 msgr2=0x7f301c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f301c067c20 0x7f301c06a0e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f302c005530 tx=0x7f302c009040 comp rx=0 tx=0).stop 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 msgr2=0x7f303c157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c157950 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f303400c7f0 tx=0x7f303400ccb0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 shutdown_connections 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f301c067c20 0x7f301c06a0e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f303c15e830 0x7f303c157950 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c15e450 0x7f303c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 --2- 172.21.15.110:0/624359882 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f303c0aed80 0x7f303c159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 >> 172.21.15.110:0/624359882 conn(0x7f303c06cd60 msgr2=0x7f303c0b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:02.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 shutdown_connections 2024-09-10T10:02:02.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.220+0000 7f3042200640 1 -- 172.21.15.110:0/624359882 wait complete. 2024-09-10T10:02:02.244 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-10T10:02:01.402066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248728,"kb_used_data":7520,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725992,"statfs":{"total":773094113280,"available":772839415808,"internally_reserved":0,"allocated":7700480,"data_stored":3865801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":246795820},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002773"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431713+0000","last_change":"2024-09-10T10:01:57.425627+0000","last_active":"2024-09-10T10:01:57.431713+0000","last_peered":"2024-09-10T10:01:57.431713+0000","last_clean":"2024-09-10T10:01:57.431713+0000","last_became_active":"2024-09-10T10:01:57.425083+0000","last_became_peered":"2024-09-10T10:01:57.425083+0000","last_unstale":"2024-09-10T10:01:57.431713+0000","last_undegraded":"2024-09-10T10:01:57.431713+0000","last_fullsized":"2024-09-10T10:01:57.431713+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419573+0000","last_change":"2024-09-10T10:01:57.410364+0000","last_active":"2024-09-10T10:01:57.419573+0000","last_peered":"2024-09-10T10:01:57.419573+0000","last_clean":"2024-09-10T10:01:57.419573+0000","last_became_active":"2024-09-10T10:01:57.410086+0000","last_became_peered":"2024-09-10T10:01:57.410086+0000","last_unstale":"2024-09-10T10:01:57.419573+0000","last_undegraded":"2024-09-10T10:01:57.419573+0000","last_fullsized":"2024-09-10T10:01:57.419573+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.839704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.397555+0000","last_change":"2024-09-10T10:01:56.397555+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.397555+0000","last_undegraded":"2024-09-10T10:01:56.397555+0000","last_fullsized":"2024-09-10T10:01:56.397555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422592+0000","last_change":"2024-09-10T10:01:57.419086+0000","last_active":"2024-09-10T10:01:57.422592+0000","last_peered":"2024-09-10T10:01:57.422592+0000","last_clean":"2024-09-10T10:01:57.422592+0000","last_became_active":"2024-09-10T10:01:57.418640+0000","last_became_peered":"2024-09-10T10:01:57.418640+0000","last_unstale":"2024-09-10T10:01:57.422592+0000","last_undegraded":"2024-09-10T10:01:57.422592+0000","last_fullsized":"2024-09-10T10:01:57.422592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:37.232883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417356+0000","last_change":"2024-09-10T10:01:57.412199+0000","last_active":"2024-09-10T10:01:57.417356+0000","last_peered":"2024-09-10T10:01:57.417356+0000","last_clean":"2024-09-10T10:01:57.417356+0000","last_became_active":"2024-09-10T10:01:57.411926+0000","last_became_peered":"2024-09-10T10:01:57.411926+0000","last_unstale":"2024-09-10T10:01:57.417356+0000","last_undegraded":"2024-09-10T10:01:57.417356+0000","last_fullsized":"2024-09-10T10:01:57.417356+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:50.877508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421654+0000","last_change":"2024-09-10T10:01:57.411801+0000","last_active":"2024-09-10T10:01:57.421654+0000","last_peered":"2024-09-10T10:01:57.421654+0000","last_clean":"2024-09-10T10:01:57.421654+0000","last_became_active":"2024-09-10T10:01:57.411506+0000","last_became_peered":"2024-09-10T10:01:57.411506+0000","last_unstale":"2024-09-10T10:01:57.421654+0000","last_undegraded":"2024-09-10T10:01:57.421654+0000","last_fullsized":"2024-09-10T10:01:57.421654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:53.511190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431828+0000","last_change":"2024-09-10T10:01:57.427391+0000","last_active":"2024-09-10T10:01:57.431828+0000","last_peered":"2024-09-10T10:01:57.431828+0000","last_clean":"2024-09-10T10:01:57.431828+0000","last_became_active":"2024-09-10T10:01:57.427095+0000","last_became_peered":"2024-09-10T10:01:57.427095+0000","last_unstale":"2024-09-10T10:01:57.431828+0000","last_undegraded":"2024-09-10T10:01:57.431828+0000","last_fullsized":"2024-09-10T10:01:57.431828+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414913+0000","last_change":"2024-09-10T10:01:57.405094+0000","last_active":"2024-09-10T10:01:57.414913+0000","last_peered":"2024-09-10T10:01:57.414913+0000","last_clean":"2024-09-10T10:01:57.414913+0000","last_became_active":"2024-09-10T10:01:57.404803+0000","last_became_peered":"2024-09-10T10:01:57.404803+0000","last_unstale":"2024-09-10T10:01:57.414913+0000","last_undegraded":"2024-09-10T10:01:57.414913+0000","last_fullsized":"2024-09-10T10:01:57.414913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:51.428680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419139+0000","last_change":"2024-09-10T10:01:57.414814+0000","last_active":"2024-09-10T10:01:57.419139+0000","last_peered":"2024-09-10T10:01:57.419139+0000","last_clean":"2024-09-10T10:01:57.419139+0000","last_became_active":"2024-09-10T10:01:57.414440+0000","last_became_peered":"2024-09-10T10:01:57.414440+0000","last_unstale":"2024-09-10T10:01:57.419139+0000","last_undegraded":"2024-09-10T10:01:57.419139+0000","last_fullsized":"2024-09-10T10:01:57.419139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:04.748497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415584+0000","last_change":"2024-09-10T10:01:57.411661+0000","last_active":"2024-09-10T10:01:57.415584+0000","last_peered":"2024-09-10T10:01:57.415584+0000","last_clean":"2024-09-10T10:01:57.415584+0000","last_became_active":"2024-09-10T10:01:57.410577+0000","last_became_peered":"2024-09-10T10:01:57.410577+0000","last_unstale":"2024-09-10T10:01:57.415584+0000","last_undegraded":"2024-09-10T10:01:57.415584+0000","last_fullsized":"2024-09-10T10:01:57.415584+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:53.511190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423038+0000","last_change":"2024-09-10T10:01:57.420979+0000","last_active":"2024-09-10T10:01:57.423038+0000","last_peered":"2024-09-10T10:01:57.423038+0000","last_clean":"2024-09-10T10:01:57.423038+0000","last_became_active":"2024-09-10T10:01:57.420695+0000","last_became_peered":"2024-09-10T10:01:57.420695+0000","last_unstale":"2024-09-10T10:01:57.423038+0000","last_undegraded":"2024-09-10T10:01:57.423038+0000","last_fullsized":"2024-09-10T10:01:57.423038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413485+0000","last_change":"2024-09-10T10:01:57.403883+0000","last_active":"2024-09-10T10:01:57.413485+0000","last_peered":"2024-09-10T10:01:57.413485+0000","last_clean":"2024-09-10T10:01:57.413485+0000","last_became_active":"2024-09-10T10:01:57.403600+0000","last_became_peered":"2024-09-10T10:01:57.403600+0000","last_unstale":"2024-09-10T10:01:57.413485+0000","last_undegraded":"2024-09-10T10:01:57.413485+0000","last_fullsized":"2024-09-10T10:01:57.413485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:53.214945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415178+0000","last_change":"2024-09-10T10:01:57.409326+0000","last_active":"2024-09-10T10:01:57.415178+0000","last_peered":"2024-09-10T10:01:57.415178+0000","last_clean":"2024-09-10T10:01:57.415178+0000","last_became_active":"2024-09-10T10:01:57.409053+0000","last_became_peered":"2024-09-10T10:01:57.409053+0000","last_unstale":"2024-09-10T10:01:57.415178+0000","last_undegraded":"2024-09-10T10:01:57.415178+0000","last_fullsized":"2024-09-10T10:01:57.415178+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424665+0000","last_change":"2024-09-10T10:01:57.421822+0000","last_active":"2024-09-10T10:01:57.424665+0000","last_peered":"2024-09-10T10:01:57.424665+0000","last_clean":"2024-09-10T10:01:57.424665+0000","last_became_active":"2024-09-10T10:01:57.421559+0000","last_became_peered":"2024-09-10T10:01:57.421559+0000","last_unstale":"2024-09-10T10:01:57.424665+0000","last_undegraded":"2024-09-10T10:01:57.424665+0000","last_fullsized":"2024-09-10T10:01:57.424665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:10.788546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414952+0000","last_change":"2024-09-10T10:01:57.410832+0000","last_active":"2024-09-10T10:01:57.414952+0000","last_peered":"2024-09-10T10:01:57.414952+0000","last_clean":"2024-09-10T10:01:57.414952+0000","last_became_active":"2024-09-10T10:01:57.410575+0000","last_became_peered":"2024-09-10T10:01:57.410575+0000","last_unstale":"2024-09-10T10:01:57.414952+0000","last_undegraded":"2024-09-10T10:01:57.414952+0000","last_fullsized":"2024-09-10T10:01:57.414952+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421152+0000","last_change":"2024-09-10T10:01:57.419056+0000","last_active":"2024-09-10T10:01:57.421152+0000","last_peered":"2024-09-10T10:01:57.421152+0000","last_clean":"2024-09-10T10:01:57.421152+0000","last_became_active":"2024-09-10T10:01:57.418704+0000","last_became_peered":"2024-09-10T10:01:57.418704+0000","last_unstale":"2024-09-10T10:01:57.421152+0000","last_undegraded":"2024-09-10T10:01:57.421152+0000","last_fullsized":"2024-09-10T10:01:57.421152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:58.162428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419118+0000","last_change":"2024-09-10T10:01:57.414713+0000","last_active":"2024-09-10T10:01:57.419118+0000","last_peered":"2024-09-10T10:01:57.419118+0000","last_clean":"2024-09-10T10:01:57.419118+0000","last_became_active":"2024-09-10T10:01:57.414227+0000","last_became_peered":"2024-09-10T10:01:57.414227+0000","last_unstale":"2024-09-10T10:01:57.419118+0000","last_undegraded":"2024-09-10T10:01:57.419118+0000","last_fullsized":"2024-09-10T10:01:57.419118+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:06.958521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.418366+0000","last_change":"2024-09-10T10:01:57.407315+0000","last_active":"2024-09-10T10:01:57.418366+0000","last_peered":"2024-09-10T10:01:57.418366+0000","last_clean":"2024-09-10T10:01:57.418366+0000","last_became_active":"2024-09-10T10:01:57.406568+0000","last_became_peered":"2024-09-10T10:01:57.406568+0000","last_unstale":"2024-09-10T10:01:57.418366+0000","last_undegraded":"2024-09-10T10:01:57.418366+0000","last_fullsized":"2024-09-10T10:01:57.418366+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:02.156822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414529+0000","last_change":"2024-09-10T10:01:57.405757+0000","last_active":"2024-09-10T10:01:57.414529+0000","last_peered":"2024-09-10T10:01:57.414529+0000","last_clean":"2024-09-10T10:01:57.414529+0000","last_became_active":"2024-09-10T10:01:57.404501+0000","last_became_peered":"2024-09-10T10:01:57.404501+0000","last_unstale":"2024-09-10T10:01:57.414529+0000","last_undegraded":"2024-09-10T10:01:57.414529+0000","last_fullsized":"2024-09-10T10:01:57.414529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:03.020211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424741+0000","last_change":"2024-09-10T10:01:57.423165+0000","last_active":"2024-09-10T10:01:57.424741+0000","last_peered":"2024-09-10T10:01:57.424741+0000","last_clean":"2024-09-10T10:01:57.424741+0000","last_became_active":"2024-09-10T10:01:57.422527+0000","last_became_peered":"2024-09-10T10:01:57.422527+0000","last_unstale":"2024-09-10T10:01:57.424741+0000","last_undegraded":"2024-09-10T10:01:57.424741+0000","last_fullsized":"2024-09-10T10:01:57.424741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417175+0000","last_change":"2024-09-10T10:01:57.409584+0000","last_active":"2024-09-10T10:01:57.417175+0000","last_peered":"2024-09-10T10:01:57.417175+0000","last_clean":"2024-09-10T10:01:57.417175+0000","last_became_active":"2024-09-10T10:01:57.409327+0000","last_became_peered":"2024-09-10T10:01:57.409327+0000","last_unstale":"2024-09-10T10:01:57.417175+0000","last_undegraded":"2024-09-10T10:01:57.417175+0000","last_fullsized":"2024-09-10T10:01:57.417175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:39.030596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419510+0000","last_change":"2024-09-10T10:01:57.416054+0000","last_active":"2024-09-10T10:01:57.419510+0000","last_peered":"2024-09-10T10:01:57.419510+0000","last_clean":"2024-09-10T10:01:57.419510+0000","last_became_active":"2024-09-10T10:01:57.415813+0000","last_became_peered":"2024-09-10T10:01:57.415813+0000","last_unstale":"2024-09-10T10:01:57.419510+0000","last_undegraded":"2024-09-10T10:01:57.419510+0000","last_fullsized":"2024-09-10T10:01:57.419510+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:11.341717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431987+0000","last_change":"2024-09-10T10:01:57.430689+0000","last_active":"2024-09-10T10:01:57.431987+0000","last_peered":"2024-09-10T10:01:57.431987+0000","last_clean":"2024-09-10T10:01:57.431987+0000","last_became_active":"2024-09-10T10:01:57.430349+0000","last_became_peered":"2024-09-10T10:01:57.430349+0000","last_unstale":"2024-09-10T10:01:57.431987+0000","last_undegraded":"2024-09-10T10:01:57.431987+0000","last_fullsized":"2024-09-10T10:01:57.431987+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:41.204498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431524+0000","last_change":"2024-09-10T10:01:57.424818+0000","last_active":"2024-09-10T10:01:57.431524+0000","last_peered":"2024-09-10T10:01:57.431524+0000","last_clean":"2024-09-10T10:01:57.431524+0000","last_became_active":"2024-09-10T10:01:57.424469+0000","last_became_peered":"2024-09-10T10:01:57.424469+0000","last_unstale":"2024-09-10T10:01:57.431524+0000","last_undegraded":"2024-09-10T10:01:57.431524+0000","last_fullsized":"2024-09-10T10:01:57.431524+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415184+0000","last_change":"2024-09-10T10:01:57.412902+0000","last_active":"2024-09-10T10:01:57.415184+0000","last_peered":"2024-09-10T10:01:57.415184+0000","last_clean":"2024-09-10T10:01:57.415184+0000","last_became_active":"2024-09-10T10:01:57.412071+0000","last_became_peered":"2024-09-10T10:01:57.412071+0000","last_unstale":"2024-09-10T10:01:57.415184+0000","last_undegraded":"2024-09-10T10:01:57.415184+0000","last_fullsized":"2024-09-10T10:01:57.415184+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:47.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431901+0000","last_change":"2024-09-10T10:01:57.428212+0000","last_active":"2024-09-10T10:01:57.431901+0000","last_peered":"2024-09-10T10:01:57.431901+0000","last_clean":"2024-09-10T10:01:57.431901+0000","last_became_active":"2024-09-10T10:01:57.427898+0000","last_became_peered":"2024-09-10T10:01:57.427898+0000","last_unstale":"2024-09-10T10:01:57.431901+0000","last_undegraded":"2024-09-10T10:01:57.431901+0000","last_fullsized":"2024-09-10T10:01:57.431901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.395624+0000","last_change":"2024-09-10T10:01:56.395624+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.395624+0000","last_undegraded":"2024-09-10T10:01:56.395624+0000","last_fullsized":"2024-09-10T10:01:56.395624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415287+0000","last_change":"2024-09-10T10:01:57.411461+0000","last_active":"2024-09-10T10:01:57.415287+0000","last_peered":"2024-09-10T10:01:57.415287+0000","last_clean":"2024-09-10T10:01:57.415287+0000","last_became_active":"2024-09-10T10:01:57.410729+0000","last_became_peered":"2024-09-10T10:01:57.410729+0000","last_unstale":"2024-09-10T10:01:57.415287+0000","last_undegraded":"2024-09-10T10:01:57.415287+0000","last_fullsized":"2024-09-10T10:01:57.415287+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.839704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394962+0000","last_change":"2024-09-10T10:01:56.394962+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394962+0000","last_undegraded":"2024-09-10T10:01:56.394962+0000","last_fullsized":"2024-09-10T10:01:56.394962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431869+0000","last_change":"2024-09-10T10:01:57.428097+0000","last_active":"2024-09-10T10:01:57.431869+0000","last_peered":"2024-09-10T10:01:57.431869+0000","last_clean":"2024-09-10T10:01:57.431869+0000","last_became_active":"2024-09-10T10:01:57.427838+0000","last_became_peered":"2024-09-10T10:01:57.427838+0000","last_unstale":"2024-09-10T10:01:57.431869+0000","last_undegraded":"2024-09-10T10:01:57.431869+0000","last_fullsized":"2024-09-10T10:01:57.431869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.426209+0000","last_change":"2024-09-10T10:01:57.416701+0000","last_active":"2024-09-10T10:01:57.426209+0000","last_peered":"2024-09-10T10:01:57.426209+0000","last_clean":"2024-09-10T10:01:57.426209+0000","last_became_active":"2024-09-10T10:01:57.416416+0000","last_became_peered":"2024-09-10T10:01:57.416416+0000","last_unstale":"2024-09-10T10:01:57.426209+0000","last_undegraded":"2024-09-10T10:01:57.426209+0000","last_fullsized":"2024-09-10T10:01:57.426209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.508652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419651+0000","last_change":"2024-09-10T10:01:57.412058+0000","last_active":"2024-09-10T10:01:57.419651+0000","last_peered":"2024-09-10T10:01:57.419651+0000","last_clean":"2024-09-10T10:01:57.419651+0000","last_became_active":"2024-09-10T10:01:57.411787+0000","last_became_peered":"2024-09-10T10:01:57.411787+0000","last_unstale":"2024-09-10T10:01:57.419651+0000","last_undegraded":"2024-09-10T10:01:57.419651+0000","last_fullsized":"2024-09-10T10:01:57.419651+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:59.314051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431681+0000","last_change":"2024-09-10T10:01:57.426610+0000","last_active":"2024-09-10T10:01:57.431681+0000","last_peered":"2024-09-10T10:01:57.431681+0000","last_clean":"2024-09-10T10:01:57.431681+0000","last_became_active":"2024-09-10T10:01:57.423435+0000","last_became_peered":"2024-09-10T10:01:57.423435+0000","last_unstale":"2024-09-10T10:01:57.431681+0000","last_undegraded":"2024-09-10T10:01:57.431681+0000","last_fullsized":"2024-09-10T10:01:57.431681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.428911+0000","last_change":"2024-09-10T10:01:56.414321+0000","last_active":"2024-09-10T10:01:57.428911+0000","last_peered":"2024-09-10T10:01:57.428911+0000","last_clean":"2024-09-10T10:01:57.428911+0000","last_became_active":"2024-09-10T10:01:56.414030+0000","last_became_peered":"2024-09-10T10:01:56.414030+0000","last_unstale":"2024-09-10T10:01:57.428911+0000","last_undegraded":"2024-09-10T10:01:57.428911+0000","last_fullsized":"2024-09-10T10:01:57.428911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419094+0000","last_change":"2024-09-10T10:01:57.414555+0000","last_active":"2024-09-10T10:01:57.419094+0000","last_peered":"2024-09-10T10:01:57.419094+0000","last_clean":"2024-09-10T10:01:57.419094+0000","last_became_active":"2024-09-10T10:01:57.413819+0000","last_became_peered":"2024-09-10T10:01:57.413819+0000","last_unstale":"2024-09-10T10:01:57.419094+0000","last_undegraded":"2024-09-10T10:01:57.419094+0000","last_fullsized":"2024-09-10T10:01:57.419094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:39.835600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423916+0000","last_change":"2024-09-10T10:01:56.408386+0000","last_active":"2024-09-10T10:01:57.423916+0000","last_peered":"2024-09-10T10:01:57.423916+0000","last_clean":"2024-09-10T10:01:57.423916+0000","last_became_active":"2024-09-10T10:01:56.408127+0000","last_became_peered":"2024-09-10T10:01:56.408127+0000","last_unstale":"2024-09-10T10:01:57.423916+0000","last_undegraded":"2024-09-10T10:01:57.423916+0000","last_fullsized":"2024-09-10T10:01:57.423916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:59.865257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421159+0000","last_change":"2024-09-10T10:01:57.417292+0000","last_active":"2024-09-10T10:01:57.421159+0000","last_peered":"2024-09-10T10:01:57.421159+0000","last_clean":"2024-09-10T10:01:57.421159+0000","last_became_active":"2024-09-10T10:01:57.416917+0000","last_became_peered":"2024-09-10T10:01:57.416917+0000","last_unstale":"2024-09-10T10:01:57.421159+0000","last_undegraded":"2024-09-10T10:01:57.421159+0000","last_fullsized":"2024-09-10T10:01:57.421159+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:11.341717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422155+0000","last_change":"2024-09-10T10:01:56.409023+0000","last_active":"2024-09-10T10:01:57.422155+0000","last_peered":"2024-09-10T10:01:57.422155+0000","last_clean":"2024-09-10T10:01:57.422155+0000","last_became_active":"2024-09-10T10:01:56.408748+0000","last_became_peered":"2024-09-10T10:01:56.408748+0000","last_unstale":"2024-09-10T10:01:57.422155+0000","last_undegraded":"2024-09-10T10:01:57.422155+0000","last_fullsized":"2024-09-10T10:01:57.422155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:50.426131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413959+0000","last_change":"2024-09-10T10:01:57.408727+0000","last_active":"2024-09-10T10:01:57.413959+0000","last_peered":"2024-09-10T10:01:57.413959+0000","last_clean":"2024-09-10T10:01:57.413959+0000","last_became_active":"2024-09-10T10:01:57.408448+0000","last_became_peered":"2024-09-10T10:01:57.408448+0000","last_unstale":"2024-09-10T10:01:57.413959+0000","last_undegraded":"2024-09-10T10:01:57.413959+0000","last_fullsized":"2024-09-10T10:01:57.413959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427217+0000","last_change":"2024-09-10T10:01:56.409387+0000","last_active":"2024-09-10T10:01:57.427217+0000","last_peered":"2024-09-10T10:01:57.427217+0000","last_clean":"2024-09-10T10:01:57.427217+0000","last_became_active":"2024-09-10T10:01:56.407044+0000","last_became_peered":"2024-09-10T10:01:56.407044+0000","last_unstale":"2024-09-10T10:01:57.427217+0000","last_undegraded":"2024-09-10T10:01:57.427217+0000","last_fullsized":"2024-09-10T10:01:57.427217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423211+0000","last_change":"2024-09-10T10:01:57.420630+0000","last_active":"2024-09-10T10:01:57.423211+0000","last_peered":"2024-09-10T10:01:57.423211+0000","last_clean":"2024-09-10T10:01:57.423211+0000","last_became_active":"2024-09-10T10:01:57.416989+0000","last_became_peered":"2024-09-10T10:01:57.416989+0000","last_unstale":"2024-09-10T10:01:57.423211+0000","last_undegraded":"2024-09-10T10:01:57.423211+0000","last_fullsized":"2024-09-10T10:01:57.423211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:14.901947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.407559+0000","last_change":"2024-09-10T10:01:56.404458+0000","last_active":"2024-09-10T10:01:56.407559+0000","last_peered":"2024-09-10T10:01:56.407559+0000","last_clean":"2024-09-10T10:01:56.407559+0000","last_became_active":"2024-09-10T10:01:56.404205+0000","last_became_peered":"2024-09-10T10:01:56.404205+0000","last_unstale":"2024-09-10T10:01:56.407559+0000","last_undegraded":"2024-09-10T10:01:56.407559+0000","last_fullsized":"2024-09-10T10:01:56.407559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:14.034052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417285+0000","last_change":"2024-09-10T10:01:57.412841+0000","last_active":"2024-09-10T10:01:57.417285+0000","last_peered":"2024-09-10T10:01:57.417285+0000","last_clean":"2024-09-10T10:01:57.417285+0000","last_became_active":"2024-09-10T10:01:57.412551+0000","last_became_peered":"2024-09-10T10:01:57.412551+0000","last_unstale":"2024-09-10T10:01:57.417285+0000","last_undegraded":"2024-09-10T10:01:57.417285+0000","last_fullsized":"2024-09-10T10:01:57.417285+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:59.314051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421685+0000","last_change":"2024-09-10T10:01:56.398879+0000","last_active":"2024-09-10T10:01:57.421685+0000","last_peered":"2024-09-10T10:01:57.421685+0000","last_clean":"2024-09-10T10:01:57.421685+0000","last_became_active":"2024-09-10T10:01:56.398576+0000","last_became_peered":"2024-09-10T10:01:56.398576+0000","last_unstale":"2024-09-10T10:01:57.421685+0000","last_undegraded":"2024-09-10T10:01:57.421685+0000","last_fullsized":"2024-09-10T10:01:57.421685+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:01.154272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417679+0000","last_change":"2024-09-10T10:01:57.412438+0000","last_active":"2024-09-10T10:01:57.417679+0000","last_peered":"2024-09-10T10:01:57.417679+0000","last_clean":"2024-09-10T10:01:57.417679+0000","last_became_active":"2024-09-10T10:01:57.412174+0000","last_became_peered":"2024-09-10T10:01:57.412174+0000","last_unstale":"2024-09-10T10:01:57.417679+0000","last_undegraded":"2024-09-10T10:01:57.417679+0000","last_fullsized":"2024-09-10T10:01:57.417679+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:46.178171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.391826+0000","last_change":"2024-09-10T10:01:56.406654+0000","last_active":"2024-09-10T10:01:57.391826+0000","last_peered":"2024-09-10T10:01:57.391826+0000","last_clean":"2024-09-10T10:01:57.391826+0000","last_became_active":"2024-09-10T10:01:56.406375+0000","last_became_peered":"2024-09-10T10:01:56.406375+0000","last_unstale":"2024-09-10T10:01:57.391826+0000","last_undegraded":"2024-09-10T10:01:57.391826+0000","last_fullsized":"2024-09-10T10:01:57.391826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:07.017690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414349+0000","last_change":"2024-09-10T10:01:57.408804+0000","last_active":"2024-09-10T10:01:57.414349+0000","last_peered":"2024-09-10T10:01:57.414349+0000","last_clean":"2024-09-10T10:01:57.414349+0000","last_became_active":"2024-09-10T10:01:57.405976+0000","last_became_peered":"2024-09-10T10:01:57.405976+0000","last_unstale":"2024-09-10T10:01:57.414349+0000","last_undegraded":"2024-09-10T10:01:57.414349+0000","last_fullsized":"2024-09-10T10:01:57.414349+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398400+0000","last_change":"2024-09-10T10:01:56.411631+0000","last_active":"2024-09-10T10:01:57.398400+0000","last_peered":"2024-09-10T10:01:57.398400+0000","last_clean":"2024-09-10T10:01:57.398400+0000","last_became_active":"2024-09-10T10:01:56.411378+0000","last_became_peered":"2024-09-10T10:01:56.411378+0000","last_unstale":"2024-09-10T10:01:57.398400+0000","last_undegraded":"2024-09-10T10:01:57.398400+0000","last_fullsized":"2024-09-10T10:01:57.398400+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:38.118765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424610+0000","last_change":"2024-09-10T10:01:57.422782+0000","last_active":"2024-09-10T10:01:57.424610+0000","last_peered":"2024-09-10T10:01:57.424610+0000","last_clean":"2024-09-10T10:01:57.424610+0000","last_became_active":"2024-09-10T10:01:57.420706+0000","last_became_peered":"2024-09-10T10:01:57.420706+0000","last_unstale":"2024-09-10T10:01:57.424610+0000","last_undegraded":"2024-09-10T10:01:57.424610+0000","last_fullsized":"2024-09-10T10:01:57.424610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398697+0000","last_change":"2024-09-10T10:01:56.403288+0000","last_active":"2024-09-10T10:01:57.398697+0000","last_peered":"2024-09-10T10:01:57.398697+0000","last_clean":"2024-09-10T10:01:57.398697+0000","last_became_active":"2024-09-10T10:01:56.402990+0000","last_became_peered":"2024-09-10T10:01:56.402990+0000","last_unstale":"2024-09-10T10:01:57.398697+0000","last_undegraded":"2024-09-10T10:01:57.398697+0000","last_fullsized":"2024-09-10T10:01:57.398697+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431695+0000","last_change":"2024-09-10T10:01:57.425385+0000","last_active":"2024-09-10T10:01:57.431695+0000","last_peered":"2024-09-10T10:01:57.431695+0000","last_clean":"2024-09-10T10:01:57.431695+0000","last_became_active":"2024-09-10T10:01:57.425032+0000","last_became_peered":"2024-09-10T10:01:57.425032+0000","last_unstale":"2024-09-10T10:01:57.431695+0000","last_undegraded":"2024-09-10T10:01:57.431695+0000","last_fullsized":"2024-09-10T10:01:57.431695+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.395612+0000","last_change":"2024-09-10T10:01:56.407609+0000","last_active":"2024-09-10T10:01:57.395612+0000","last_peered":"2024-09-10T10:01:57.395612+0000","last_clean":"2024-09-10T10:01:57.395612+0000","last_became_active":"2024-09-10T10:01:56.405401+0000","last_became_peered":"2024-09-10T10:01:56.405401+0000","last_unstale":"2024-09-10T10:01:57.395612+0000","last_undegraded":"2024-09-10T10:01:57.395612+0000","last_fullsized":"2024-09-10T10:01:57.395612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419650+0000","last_change":"2024-09-10T10:01:57.412223+0000","last_active":"2024-09-10T10:01:57.419650+0000","last_peered":"2024-09-10T10:01:57.419650+0000","last_clean":"2024-09-10T10:01:57.419650+0000","last_became_active":"2024-09-10T10:01:57.411477+0000","last_became_peered":"2024-09-10T10:01:57.411477+0000","last_unstale":"2024-09-10T10:01:57.419650+0000","last_undegraded":"2024-09-10T10:01:57.419650+0000","last_fullsized":"2024-09-10T10:01:57.419650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:47.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.407993+0000","last_change":"2024-09-10T10:01:56.406998+0000","last_active":"2024-09-10T10:01:56.407993+0000","last_peered":"2024-09-10T10:01:56.407993+0000","last_clean":"2024-09-10T10:01:56.407993+0000","last_became_active":"2024-09-10T10:01:56.406722+0000","last_became_peered":"2024-09-10T10:01:56.406722+0000","last_unstale":"2024-09-10T10:01:56.407993+0000","last_undegraded":"2024-09-10T10:01:56.407993+0000","last_fullsized":"2024-09-10T10:01:56.407993+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:27.783209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414220+0000","last_change":"2024-09-10T10:01:57.409970+0000","last_active":"2024-09-10T10:01:57.414220+0000","last_peered":"2024-09-10T10:01:57.414220+0000","last_clean":"2024-09-10T10:01:57.414220+0000","last_became_active":"2024-09-10T10:01:57.406199+0000","last_became_peered":"2024-09-10T10:01:57.406199+0000","last_unstale":"2024-09-10T10:01:57.414220+0000","last_undegraded":"2024-09-10T10:01:57.414220+0000","last_fullsized":"2024-09-10T10:01:57.414220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:41.660285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.426946+0000","last_change":"2024-09-10T10:01:56.410726+0000","last_active":"2024-09-10T10:01:57.426946+0000","last_peered":"2024-09-10T10:01:57.426946+0000","last_clean":"2024-09-10T10:01:57.426946+0000","last_became_active":"2024-09-10T10:01:56.410239+0000","last_became_peered":"2024-09-10T10:01:56.410239+0000","last_unstale":"2024-09-10T10:01:57.426946+0000","last_undegraded":"2024-09-10T10:01:57.426946+0000","last_fullsized":"2024-09-10T10:01:57.426946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:44.033036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.429665+0000","last_change":"2024-09-10T10:01:57.423932+0000","last_active":"2024-09-10T10:01:57.429665+0000","last_peered":"2024-09-10T10:01:57.429665+0000","last_clean":"2024-09-10T10:01:57.429665+0000","last_became_active":"2024-09-10T10:01:57.423635+0000","last_became_peered":"2024-09-10T10:01:57.423635+0000","last_unstale":"2024-09-10T10:01:57.429665+0000","last_undegraded":"2024-09-10T10:01:57.429665+0000","last_fullsized":"2024-09-10T10:01:57.429665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431982+0000","last_change":"2024-09-10T10:01:56.411145+0000","last_active":"2024-09-10T10:01:57.431982+0000","last_peered":"2024-09-10T10:01:57.431982+0000","last_clean":"2024-09-10T10:01:57.431982+0000","last_became_active":"2024-09-10T10:01:56.410821+0000","last_became_peered":"2024-09-10T10:01:56.410821+0000","last_unstale":"2024-09-10T10:01:57.431982+0000","last_undegraded":"2024-09-10T10:01:57.431982+0000","last_fullsized":"2024-09-10T10:01:57.431982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:34.635430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431813+0000","last_change":"2024-09-10T10:01:57.427059+0000","last_active":"2024-09-10T10:01:57.431813+0000","last_peered":"2024-09-10T10:01:57.431813+0000","last_clean":"2024-09-10T10:01:57.431813+0000","last_became_active":"2024-09-10T10:01:57.423512+0000","last_became_peered":"2024-09-10T10:01:57.423512+0000","last_unstale":"2024-09-10T10:01:57.431813+0000","last_undegraded":"2024-09-10T10:01:57.431813+0000","last_fullsized":"2024-09-10T10:01:57.431813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394598+0000","last_change":"2024-09-10T10:01:56.405376+0000","last_active":"2024-09-10T10:01:57.394598+0000","last_peered":"2024-09-10T10:01:57.394598+0000","last_clean":"2024-09-10T10:01:57.394598+0000","last_became_active":"2024-09-10T10:01:56.405056+0000","last_became_peered":"2024-09-10T10:01:56.405056+0000","last_unstale":"2024-09-10T10:01:57.394598+0000","last_undegraded":"2024-09-10T10:01:57.394598+0000","last_fullsized":"2024-09-10T10:01:57.394598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:56.936736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415321+0000","last_change":"2024-09-10T10:01:57.411055+0000","last_active":"2024-09-10T10:01:57.415321+0000","last_peered":"2024-09-10T10:01:57.415321+0000","last_clean":"2024-09-10T10:01:57.415321+0000","last_became_active":"2024-09-10T10:01:57.410789+0000","last_became_peered":"2024-09-10T10:01:57.410789+0000","last_unstale":"2024-09-10T10:01:57.415321+0000","last_undegraded":"2024-09-10T10:01:57.415321+0000","last_fullsized":"2024-09-10T10:01:57.415321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.412528+0000","last_change":"2024-09-10T10:01:56.412704+0000","last_active":"2024-09-10T10:01:57.412528+0000","last_peered":"2024-09-10T10:01:57.412528+0000","last_clean":"2024-09-10T10:01:57.412528+0000","last_became_active":"2024-09-10T10:01:56.410023+0000","last_became_peered":"2024-09-10T10:01:56.410023+0000","last_unstale":"2024-09-10T10:01:57.412528+0000","last_undegraded":"2024-09-10T10:01:57.412528+0000","last_fullsized":"2024-09-10T10:01:57.412528+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:57.159878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415068+0000","last_change":"2024-09-10T10:01:57.407954+0000","last_active":"2024-09-10T10:01:57.415068+0000","last_peered":"2024-09-10T10:01:57.415068+0000","last_clean":"2024-09-10T10:01:57.415068+0000","last_became_active":"2024-09-10T10:01:57.407651+0000","last_became_peered":"2024-09-10T10:01:57.407651+0000","last_unstale":"2024-09-10T10:01:57.415068+0000","last_undegraded":"2024-09-10T10:01:57.415068+0000","last_fullsized":"2024-09-10T10:01:57.415068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:16.908619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.430713+0000","last_change":"2024-09-10T10:01:56.410104+0000","last_active":"2024-09-10T10:01:57.430713+0000","last_peered":"2024-09-10T10:01:57.430713+0000","last_clean":"2024-09-10T10:01:57.430713+0000","last_became_active":"2024-09-10T10:01:56.409756+0000","last_became_peered":"2024-09-10T10:01:56.409756+0000","last_unstale":"2024-09-10T10:01:57.430713+0000","last_undegraded":"2024-09-10T10:01:57.430713+0000","last_fullsized":"2024-09-10T10:01:57.430713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.409223+0000","last_change":"2024-09-10T10:01:57.405556+0000","last_active":"2024-09-10T10:01:57.409223+0000","last_peered":"2024-09-10T10:01:57.409223+0000","last_clean":"2024-09-10T10:01:57.409223+0000","last_became_active":"2024-09-10T10:01:57.405222+0000","last_became_peered":"2024-09-10T10:01:57.405222+0000","last_unstale":"2024-09-10T10:01:57.409223+0000","last_undegraded":"2024-09-10T10:01:57.409223+0000","last_fullsized":"2024-09-10T10:01:57.409223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:03.020211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.425896+0000","last_change":"2024-09-10T10:01:56.401752+0000","last_active":"2024-09-10T10:01:57.425896+0000","last_peered":"2024-09-10T10:01:57.425896+0000","last_clean":"2024-09-10T10:01:57.425896+0000","last_became_active":"2024-09-10T10:01:56.401508+0000","last_became_peered":"2024-09-10T10:01:56.401508+0000","last_unstale":"2024-09-10T10:01:57.425896+0000","last_undegraded":"2024-09-10T10:01:57.425896+0000","last_fullsized":"2024-09-10T10:01:57.425896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:52.212395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431672+0000","last_change":"2024-09-10T10:01:57.425326+0000","last_active":"2024-09-10T10:01:57.431672+0000","last_peered":"2024-09-10T10:01:57.431672+0000","last_clean":"2024-09-10T10:01:57.431672+0000","last_became_active":"2024-09-10T10:01:57.424606+0000","last_became_peered":"2024-09-10T10:01:57.424606+0000","last_unstale":"2024-09-10T10:01:57.431672+0000","last_undegraded":"2024-09-10T10:01:57.431672+0000","last_fullsized":"2024-09-10T10:01:57.431672+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:23.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422525+0000","last_change":"2024-09-10T10:01:56.409357+0000","last_active":"2024-09-10T10:01:57.422525+0000","last_peered":"2024-09-10T10:01:57.422525+0000","last_clean":"2024-09-10T10:01:57.422525+0000","last_became_active":"2024-09-10T10:01:56.409062+0000","last_became_peered":"2024-09-10T10:01:56.409062+0000","last_unstale":"2024-09-10T10:01:57.422525+0000","last_undegraded":"2024-09-10T10:01:57.422525+0000","last_fullsized":"2024-09-10T10:01:57.422525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:29.657091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413399+0000","last_change":"2024-09-10T10:01:57.401546+0000","last_active":"2024-09-10T10:01:57.413399+0000","last_peered":"2024-09-10T10:01:57.413399+0000","last_clean":"2024-09-10T10:01:57.413399+0000","last_became_active":"2024-09-10T10:01:57.401293+0000","last_became_peered":"2024-09-10T10:01:57.401293+0000","last_unstale":"2024-09-10T10:01:57.413399+0000","last_undegraded":"2024-09-10T10:01:57.413399+0000","last_fullsized":"2024-09-10T10:01:57.413399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:19.413048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427627+0000","last_change":"2024-09-10T10:01:56.399162+0000","last_active":"2024-09-10T10:01:57.427627+0000","last_peered":"2024-09-10T10:01:57.427627+0000","last_clean":"2024-09-10T10:01:57.427627+0000","last_became_active":"2024-09-10T10:01:56.398886+0000","last_became_peered":"2024-09-10T10:01:56.398886+0000","last_unstale":"2024-09-10T10:01:57.427627+0000","last_undegraded":"2024-09-10T10:01:57.427627+0000","last_fullsized":"2024-09-10T10:01:57.427627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414198+0000","last_change":"2024-09-10T10:01:57.409275+0000","last_active":"2024-09-10T10:01:57.414198+0000","last_peered":"2024-09-10T10:01:57.414198+0000","last_clean":"2024-09-10T10:01:57.414198+0000","last_became_active":"2024-09-10T10:01:57.409007+0000","last_became_peered":"2024-09-10T10:01:57.409007+0000","last_unstale":"2024-09-10T10:01:57.414198+0000","last_undegraded":"2024-09-10T10:01:57.414198+0000","last_fullsized":"2024-09-10T10:01:57.414198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:17.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.405547+0000","last_change":"2024-09-10T10:01:56.414600+0000","last_active":"2024-09-10T10:01:57.405547+0000","last_peered":"2024-09-10T10:01:57.405547+0000","last_clean":"2024-09-10T10:01:57.405547+0000","last_became_active":"2024-09-10T10:01:56.414312+0000","last_became_peered":"2024-09-10T10:01:56.414312+0000","last_unstale":"2024-09-10T10:01:57.405547+0000","last_undegraded":"2024-09-10T10:01:57.405547+0000","last_fullsized":"2024-09-10T10:01:57.405547+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:13.899397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394589+0000","last_change":"2024-09-10T10:01:56.394589+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394589+0000","last_undegraded":"2024-09-10T10:01:56.394589+0000","last_fullsized":"2024-09-10T10:01:56.394589+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394540+0000","last_change":"2024-09-10T10:01:56.399440+0000","last_active":"2024-09-10T10:01:57.394540+0000","last_peered":"2024-09-10T10:01:57.394540+0000","last_clean":"2024-09-10T10:01:57.394540+0000","last_became_active":"2024-09-10T10:01:56.399153+0000","last_became_peered":"2024-09-10T10:01:56.399153+0000","last_unstale":"2024-09-10T10:01:57.394540+0000","last_undegraded":"2024-09-10T10:01:57.394540+0000","last_fullsized":"2024-09-10T10:01:57.394540+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:21.685329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.393504+0000","last_change":"2024-09-10T10:01:56.393504+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.393504+0000","last_undegraded":"2024-09-10T10:01:56.393504+0000","last_fullsized":"2024-09-10T10:01:56.393504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.412691+0000","last_change":"2024-09-10T10:01:56.410145+0000","last_active":"2024-09-10T10:01:57.412691+0000","last_peered":"2024-09-10T10:01:57.412691+0000","last_clean":"2024-09-10T10:01:57.412691+0000","last_became_active":"2024-09-10T10:01:56.409854+0000","last_became_peered":"2024-09-10T10:01:56.409854+0000","last_unstale":"2024-09-10T10:01:57.412691+0000","last_undegraded":"2024-09-10T10:01:57.412691+0000","last_fullsized":"2024-09-10T10:01:57.412691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:23.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.393865+0000","last_change":"2024-09-10T10:01:56.393865+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.393865+0000","last_undegraded":"2024-09-10T10:01:56.393865+0000","last_fullsized":"2024-09-10T10:01:56.393865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.395889+0000","last_change":"2024-09-10T10:01:56.404958+0000","last_active":"2024-09-10T10:01:57.395889+0000","last_peered":"2024-09-10T10:01:57.395889+0000","last_clean":"2024-09-10T10:01:57.395889+0000","last_became_active":"2024-09-10T10:01:56.404619+0000","last_became_peered":"2024-09-10T10:01:56.404619+0000","last_unstale":"2024-09-10T10:01:57.395889+0000","last_undegraded":"2024-09-10T10:01:57.395889+0000","last_fullsized":"2024-09-10T10:01:57.395889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415096+0000","last_change":"2024-09-10T10:01:57.409907+0000","last_active":"2024-09-10T10:01:57.415096+0000","last_peered":"2024-09-10T10:01:57.415096+0000","last_clean":"2024-09-10T10:01:57.415096+0000","last_became_active":"2024-09-10T10:01:57.409375+0000","last_became_peered":"2024-09-10T10:01:57.409375+0000","last_unstale":"2024-09-10T10:01:57.415096+0000","last_undegraded":"2024-09-10T10:01:57.415096+0000","last_fullsized":"2024-09-10T10:01:57.415096+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:23.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.425893+0000","last_change":"2024-09-10T10:01:56.400929+0000","last_active":"2024-09-10T10:01:57.425893+0000","last_peered":"2024-09-10T10:01:57.425893+0000","last_clean":"2024-09-10T10:01:57.425893+0000","last_became_active":"2024-09-10T10:01:56.400665+0000","last_became_peered":"2024-09-10T10:01:56.400665+0000","last_unstale":"2024-09-10T10:01:57.425893+0000","last_undegraded":"2024-09-10T10:01:57.425893+0000","last_fullsized":"2024-09-10T10:01:57.425893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:32.570015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431796+0000","last_change":"2024-09-10T10:01:57.426884+0000","last_active":"2024-09-10T10:01:57.431796+0000","last_peered":"2024-09-10T10:01:57.431796+0000","last_clean":"2024-09-10T10:01:57.431796+0000","last_became_active":"2024-09-10T10:01:57.426621+0000","last_became_peered":"2024-09-10T10:01:57.426621+0000","last_unstale":"2024-09-10T10:01:57.431796+0000","last_undegraded":"2024-09-10T10:01:57.431796+0000","last_fullsized":"2024-09-10T10:01:57.431796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:35.517149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413728+0000","last_change":"2024-09-10T10:01:56.410855+0000","last_active":"2024-09-10T10:01:57.413728+0000","last_peered":"2024-09-10T10:01:57.413728+0000","last_clean":"2024-09-10T10:01:57.413728+0000","last_became_active":"2024-09-10T10:01:56.410597+0000","last_became_peered":"2024-09-10T10:01:56.410597+0000","last_unstale":"2024-09-10T10:01:57.413728+0000","last_undegraded":"2024-09-10T10:01:57.413728+0000","last_fullsized":"2024-09-10T10:01:57.413728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.934821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.429750+0000","last_change":"2024-09-10T10:01:57.423850+0000","last_active":"2024-09-10T10:01:57.429750+0000","last_peered":"2024-09-10T10:01:57.429750+0000","last_clean":"2024-09-10T10:01:57.429750+0000","last_became_active":"2024-09-10T10:01:57.423309+0000","last_became_peered":"2024-09-10T10:01:57.423309+0000","last_unstale":"2024-09-10T10:01:57.429750+0000","last_undegraded":"2024-09-10T10:01:57.429750+0000","last_fullsized":"2024-09-10T10:01:57.429750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.405952+0000","last_change":"2024-09-10T10:01:56.409831+0000","last_active":"2024-09-10T10:01:57.405952+0000","last_peered":"2024-09-10T10:01:57.405952+0000","last_clean":"2024-09-10T10:01:57.405952+0000","last_became_active":"2024-09-10T10:01:56.409512+0000","last_became_peered":"2024-09-10T10:01:56.409512+0000","last_unstale":"2024-09-10T10:01:57.405952+0000","last_undegraded":"2024-09-10T10:01:57.405952+0000","last_fullsized":"2024-09-10T10:01:57.405952+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:05.784097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431774+0000","last_change":"2024-09-10T10:01:57.427760+0000","last_active":"2024-09-10T10:01:57.431774+0000","last_peered":"2024-09-10T10:01:57.431774+0000","last_clean":"2024-09-10T10:01:57.431774+0000","last_became_active":"2024-09-10T10:01:57.427511+0000","last_became_peered":"2024-09-10T10:01:57.427511+0000","last_unstale":"2024-09-10T10:01:57.431774+0000","last_undegraded":"2024-09-10T10:01:57.431774+0000","last_fullsized":"2024-09-10T10:01:57.431774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:52.470311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.397237+0000","last_change":"2024-09-10T10:01:56.402970+0000","last_active":"2024-09-10T10:01:57.397237+0000","last_peered":"2024-09-10T10:01:57.397237+0000","last_clean":"2024-09-10T10:01:57.397237+0000","last_became_active":"2024-09-10T10:01:56.402500+0000","last_became_peered":"2024-09-10T10:01:56.402500+0000","last_unstale":"2024-09-10T10:01:57.397237+0000","last_undegraded":"2024-09-10T10:01:57.397237+0000","last_fullsized":"2024-09-10T10:01:57.397237+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:26.744793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415252+0000","last_change":"2024-09-10T10:01:57.409373+0000","last_active":"2024-09-10T10:01:57.415252+0000","last_peered":"2024-09-10T10:01:57.415252+0000","last_clean":"2024-09-10T10:01:57.415252+0000","last_became_active":"2024-09-10T10:01:57.409114+0000","last_became_peered":"2024-09-10T10:01:57.409114+0000","last_unstale":"2024-09-10T10:01:57.415252+0000","last_undegraded":"2024-09-10T10:01:57.415252+0000","last_fullsized":"2024-09-10T10:01:57.415252+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:53.254884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.400447+0000","last_change":"2024-09-10T10:01:56.411209+0000","last_active":"2024-09-10T10:01:57.400447+0000","last_peered":"2024-09-10T10:01:57.400447+0000","last_clean":"2024-09-10T10:01:57.400447+0000","last_became_active":"2024-09-10T10:01:56.410808+0000","last_became_peered":"2024-09-10T10:01:56.410808+0000","last_unstale":"2024-09-10T10:01:57.400447+0000","last_undegraded":"2024-09-10T10:01:57.400447+0000","last_fullsized":"2024-09-10T10:01:57.400447+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:39.392910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421360+0000","last_change":"2024-09-10T10:01:57.418730+0000","last_active":"2024-09-10T10:01:57.421360+0000","last_peered":"2024-09-10T10:01:57.421360+0000","last_clean":"2024-09-10T10:01:57.421360+0000","last_became_active":"2024-09-10T10:01:57.418395+0000","last_became_peered":"2024-09-10T10:01:57.418395+0000","last_unstale":"2024-09-10T10:01:57.421360+0000","last_undegraded":"2024-09-10T10:01:57.421360+0000","last_fullsized":"2024-09-10T10:01:57.421360+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:58.162428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.391574+0000","last_change":"2024-09-10T10:01:56.409314+0000","last_active":"2024-09-10T10:01:57.391574+0000","last_peered":"2024-09-10T10:01:57.391574+0000","last_clean":"2024-09-10T10:01:57.391574+0000","last_became_active":"2024-09-10T10:01:56.409077+0000","last_became_peered":"2024-09-10T10:01:56.409077+0000","last_unstale":"2024-09-10T10:01:57.391574+0000","last_undegraded":"2024-09-10T10:01:57.391574+0000","last_fullsized":"2024-09-10T10:01:57.391574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.220387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.416721+0000","last_change":"2024-09-10T10:01:57.407297+0000","last_active":"2024-09-10T10:01:57.416721+0000","last_peered":"2024-09-10T10:01:57.416721+0000","last_clean":"2024-09-10T10:01:57.416721+0000","last_became_active":"2024-09-10T10:01:57.407030+0000","last_became_peered":"2024-09-10T10:01:57.407030+0000","last_unstale":"2024-09-10T10:01:57.416721+0000","last_undegraded":"2024-09-10T10:01:57.416721+0000","last_fullsized":"2024-09-10T10:01:57.416721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:33.572565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398463+0000","last_change":"2024-09-10T10:01:56.411518+0000","last_active":"2024-09-10T10:01:57.398463+0000","last_peered":"2024-09-10T10:01:57.398463+0000","last_clean":"2024-09-10T10:01:57.398463+0000","last_became_active":"2024-09-10T10:01:56.410515+0000","last_became_peered":"2024-09-10T10:01:56.410515+0000","last_unstale":"2024-09-10T10:01:57.398463+0000","last_undegraded":"2024-09-10T10:01:57.398463+0000","last_fullsized":"2024-09-10T10:01:57.398463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421441+0000","last_change":"2024-09-10T10:01:57.408569+0000","last_active":"2024-09-10T10:01:57.421441+0000","last_peered":"2024-09-10T10:01:57.421441+0000","last_clean":"2024-09-10T10:01:57.421441+0000","last_became_active":"2024-09-10T10:01:57.408302+0000","last_became_peered":"2024-09-10T10:01:57.408302+0000","last_unstale":"2024-09-10T10:01:57.421441+0000","last_undegraded":"2024-09-10T10:01:57.421441+0000","last_fullsized":"2024-09-10T10:01:57.421441+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:39.030596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394334+0000","last_change":"2024-09-10T10:01:56.407472+0000","last_active":"2024-09-10T10:01:57.394334+0000","last_peered":"2024-09-10T10:01:57.394334+0000","last_clean":"2024-09-10T10:01:57.394334+0000","last_became_active":"2024-09-10T10:01:56.407112+0000","last_became_peered":"2024-09-10T10:01:56.407112+0000","last_unstale":"2024-09-10T10:01:57.394334+0000","last_undegraded":"2024-09-10T10:01:57.394334+0000","last_fullsized":"2024-09-10T10:01:57.394334+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:36.230334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394297+0000","last_change":"2024-09-10T10:01:56.394297+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394297+0000","last_undegraded":"2024-09-10T10:01:56.394297+0000","last_fullsized":"2024-09-10T10:01:56.394297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427116+0000","last_change":"2024-09-10T10:01:56.408027+0000","last_active":"2024-09-10T10:01:57.427116+0000","last_peered":"2024-09-10T10:01:57.427116+0000","last_clean":"2024-09-10T10:01:57.427116+0000","last_became_active":"2024-09-10T10:01:56.407749+0000","last_became_peered":"2024-09-10T10:01:56.407749+0000","last_unstale":"2024-09-10T10:01:57.427116+0000","last_undegraded":"2024-09-10T10:01:57.427116+0000","last_fullsized":"2024-09-10T10:01:57.427116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:42.979120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":252,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.403667+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:01:57.403667+0000","last_peered":"2024-09-10T10:01:57.403667+0000","last_clean":"2024-09-10T10:01:57.403667+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:01:57.403667+0000","last_undegraded":"2024-09-10T10:01:57.403667+0000","last_fullsized":"2024-09-10T10:01:57.403667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":930573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":341201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":929957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:02.248 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3d is not active+clean 2024-09-10T10:02:02.248 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:01:56.397555+0000', 'last_change': '2024-09-10T10:01:56.397555+0000', 'last_active': '2024-09-10T10:01:56.377542+0000', 'last_peered': '2024-09-10T10:01:56.377542+0000', 'last_clean': '2024-09-10T10:01:56.377542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:01:56.397555+0000', 'last_undegraded': '2024-09-10T10:01:56.397555+0000', 'last_fullsized': '2024-09-10T10:01:56.397555+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_clean_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:02:02.248 INFO:tasks.ceph.ceph_manager.ceph:PG 11.25 is not active+clean 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:01:56.395624+0000', 'last_change': '2024-09-10T10:01:56.395624+0000', 'last_active': '2024-09-10T10:01:56.377542+0000', 'last_peered': '2024-09-10T10:01:56.377542+0000', 'last_clean': '2024-09-10T10:01:56.377542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:01:56.395624+0000', 'last_undegraded': '2024-09-10T10:01:56.395624+0000', 'last_fullsized': '2024-09-10T10:01:56.395624+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_clean_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:PG 11.23 is not active+clean 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:01:56.394962+0000', 'last_change': '2024-09-10T10:01:56.394962+0000', 'last_active': '2024-09-10T10:01:56.377542+0000', 'last_peered': '2024-09-10T10:01:56.377542+0000', 'last_clean': '2024-09-10T10:01:56.377542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:01:56.394962+0000', 'last_undegraded': '2024-09-10T10:01:56.394962+0000', 'last_fullsized': '2024-09-10T10:01:56.394962+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_clean_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:PG 11.b is not active+clean 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:01:56.394589+0000', 'last_change': '2024-09-10T10:01:56.394589+0000', 'last_active': '2024-09-10T10:01:56.377542+0000', 'last_peered': '2024-09-10T10:01:56.377542+0000', 'last_clean': '2024-09-10T10:01:56.377542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:01:56.394589+0000', 'last_undegraded': '2024-09-10T10:01:56.394589+0000', 'last_fullsized': '2024-09-10T10:01:56.394589+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_clean_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:PG 11.8 is not active+clean 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:01:56.393504+0000', 'last_change': '2024-09-10T10:01:56.393504+0000', 'last_active': '2024-09-10T10:01:56.377542+0000', 'last_peered': '2024-09-10T10:01:56.377542+0000', 'last_clean': '2024-09-10T10:01:56.377542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:01:56.393504+0000', 'last_undegraded': '2024-09-10T10:01:56.393504+0000', 'last_fullsized': '2024-09-10T10:01:56.393504+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_clean_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:PG 11.9 is not active+clean 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:01:56.393865+0000', 'last_change': '2024-09-10T10:01:56.393865+0000', 'last_active': '2024-09-10T10:01:56.377542+0000', 'last_peered': '2024-09-10T10:01:56.377542+0000', 'last_clean': '2024-09-10T10:01:56.377542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:01:56.393865+0000', 'last_undegraded': '2024-09-10T10:01:56.393865+0000', 'last_fullsized': '2024-09-10T10:01:56.393865+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_clean_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:02:02.249 INFO:tasks.ceph.ceph_manager.ceph:PG 11.a is not active+clean 2024-09-10T10:02:02.250 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:01:56.394297+0000', 'last_change': '2024-09-10T10:01:56.394297+0000', 'last_active': '2024-09-10T10:01:56.377542+0000', 'last_peered': '2024-09-10T10:01:56.377542+0000', 'last_clean': '2024-09-10T10:01:56.377542+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:01:56.394297+0000', 'last_undegraded': '2024-09-10T10:01:56.394297+0000', 'last_fullsized': '2024-09-10T10:01:56.394297+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'last_clean_scrub_stamp': '2024-09-10T10:01:56.377542+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:02:02.250 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T10:02:02.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f283d800640 1 Processor -- start 2024-09-10T10:02:02.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f283d800640 1 -- start start 2024-09-10T10:02:02.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f283d800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f283815fd80 0x7f283813f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f283d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f28380731e0 0x7f2838073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f283d800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f283815c790 con 0x7f28380b1c60 2024-09-10T10:02:02.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f283d800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f28380b2040 con 0x7f283815fd80 2024-09-10T10:02:02.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f283d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f28381602b0 con 0x7f28380731e0 2024-09-10T10:02:02.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2837e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f28380b1c60 0x7f2838142320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36882/0 (socket says 172.21.15.110:36882) 2024-09-10T10:02:02.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2837e00640 1 -- 172.21.15.110:0/3028950177 learned_addr learned my addr 172.21.15.110:0/3028950177 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:02.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2837400640 1 --2- 172.21.15.110:0/3028950177 >> v2:172.21.15.145:3300/0 conn(0x7f283815fd80 0x7f283813f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f283ce00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f28380731e0 0x7f2838073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3844912264 0 0) 0x7f2828001120 con 0x7f28380b1c60 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f281c003530 con 0x7f28380b1c60 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1930503724 0 0) 0x7f281c003530 con 0x7f28380b1c60 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f28380731e0 msgr2=0x7f2838073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f28380731e0 0x7f2838073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 >> v2:172.21.15.145:3300/0 conn(0x7f283815fd80 msgr2=0x7f283813f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 --2- 172.21.15.110:0/3028950177 >> v2:172.21.15.145:3300/0 conn(0x7f283815fd80 0x7f283813f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2838160590 con 0x7f28380b1c60 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f2828001850 con 0x7f28380b1c60 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2837400640 1 --2- 172.21.15.110:0/3028950177 >> v2:172.21.15.145:3300/0 conn(0x7f283815fd80 0x7f283813f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 >> v1:172.21.15.110:6789/0 conn(0x7f28380b1c60 legacy=0x7f2838142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.420+0000 7f2836a00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f28380731e0 0x7f281c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2836a00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f281c003f40 0x7f281c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2836a00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f281c004d50 0x7f281c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28380b2040 con 0x7f281c004d50 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f28381602b0 con 0x7f281c003f40 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f281c00ba20 con 0x7f28380731e0 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f283ce00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f281c004d50 0x7f281c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2837400640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f281c003f40 0x7f281c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2837e00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f28380731e0 0x7f281c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f283ce00640 1 -- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f28380731e0 msgr2=0x7f281c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f283ce00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f28380731e0 0x7f281c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f283ce00640 1 -- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f281c003f40 msgr2=0x7f281c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f283ce00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f281c003f40 0x7f281c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f283ce00640 1 -- 172.21.15.110:0/3028950177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c003530 con 0x7f281c004d50 2024-09-10T10:02:02.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f283ce00640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f281c004d50 0x7f281c0051d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2830007360 tx=0x7f283001ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8c8c4fcc6ef511 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300b2070 con 0x7f281c004d50 2024-09-10T10:02:02.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28300a5d40 con 0x7f281c004d50 2024-09-10T10:02:02.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.424+0000 7f2836a00640 1 -- 172.21.15.110:0/3028950177 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830009be0 con 0x7f281c004d50 2024-09-10T10:02:02.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- 172.21.15.110:0/3028950177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f281c004d50 msgr2=0x7f281c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 --2- 172.21.15.110:0/3028950177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f281c004d50 0x7f281c0051d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2830007360 tx=0x7f283001ce40 comp rx=0 tx=0).stop 2024-09-10T10:02:02.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283ce00640 1 -- 172.21.15.110:0/3028950177 reap_dead start 2024-09-10T10:02:02.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- 172.21.15.110:0/3028950177 shutdown_connections 2024-09-10T10:02:02.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- 172.21.15.110:0/3028950177 >> 172.21.15.110:0/3028950177 conn(0x7f28380b42d0 msgr2=0x7f28380b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:02.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- 172.21.15.110:0/3028950177 shutdown_connections 2024-09-10T10:02:02.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- 172.21.15.110:0/3028950177 wait complete. 2024-09-10T10:02:02.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 Processor -- start 2024-09-10T10:02:02.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- start start 2024-09-10T10:02:02.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f281c004d50 0x7f283813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2838153e90 0x7f28381553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283815f480 0x7f28381564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f283813fe20 con 0x7f2838153e90 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2838160b50 con 0x7f283815f480 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2838073670 con 0x7f281c004d50 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f2837400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2838153e90 0x7f28381553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f2837400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2838153e90 0x7f28381553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44100/0 (socket says 172.21.15.110:44100) 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f2837400640 1 -- 172.21.15.110:0/248143180 learned_addr learned my addr 172.21.15.110:0/248143180 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f283ce00640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283815f480 0x7f28381564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.428+0000 7f2837e00640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f281c004d50 0x7f283813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f2837400640 1 -- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f281c004d50 msgr2=0x7f283813b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f2837400640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f281c004d50 0x7f283813b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f2837400640 1 -- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283815f480 msgr2=0x7f28381564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f2837400640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283815f480 0x7f28381564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f2837400640 1 -- 172.21.15.110:0/248143180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283813b6e0 con 0x7f2838153e90 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f283ce00640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283815f480 0x7f28381564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f2837400640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2838153e90 0x7f28381553f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2824011cd0 tx=0x7f282400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f282fe00640 1 -- 172.21.15.110:0/248143180 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282400fd20 con 0x7f2838153e90 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f282fe00640 1 -- 172.21.15.110:0/248143180 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2824004030 con 0x7f2838153e90 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f282fe00640 1 -- 172.21.15.110:0/248143180 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824004420 con 0x7f2838153e90 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2838154350 con 0x7f2838153e90 2024-09-10T10:02:02.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2838154910 con 0x7f2838153e90 2024-09-10T10:02:02.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2838073a00 con 0x7f2838153e90 2024-09-10T10:02:02.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f282fe00640 1 -- 172.21.15.110:0/248143180 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f2824012040 con 0x7f2838153e90 2024-09-10T10:02:02.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f282fe00640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2818067c50 0x7f281806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f282fe00640 1 -- 172.21.15.110:0/248143180 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f2824086790 con 0x7f2838153e90 2024-09-10T10:02:02.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.432+0000 7f2837e00640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2818067c50 0x7f281806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.436+0000 7f2837e00640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2818067c50 0x7f281806a110 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2828001810 tx=0x7f2828009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.440+0000 7f282fe00640 1 -- 172.21.15.110:0/248143180 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2824019070 con 0x7f2838153e90 2024-09-10T10:02:02.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.652+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f28380b3430 con 0x7f2838153e90 2024-09-10T10:02:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.652+0000 7f282fe00640 1 -- 172.21.15.110:0/248143180 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1560 (secure 0 0 0) 0x7f2824056ea0 con 0x7f2838153e90 2024-09-10T10:02:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:02.655 INFO:teuthology.orchestra.run.smithi110.stdout:{"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":244,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":69,"num_osds":8,"num_up_osds":8,"osd_up_since":1725962292,"num_in_osds":8,"osd_in_since":1725962280,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":90},{"state_name":"creating+peering","count":7}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592688,"bytes_used":254697472,"bytes_avail":772839415808,"bytes_total":773094113280,"inactive_pgs_ratio":0.072164945304393768,"write_bytes_sec":2217,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":45,"btime":"2024-09-10T10:02:00:413261+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":6033}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T10:01:45.397838+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T10:02:02.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.652+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2818067c50 msgr2=0x7f281806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.652+0000 7f283d800640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2818067c50 0x7f281806a110 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2828001810 tx=0x7f2828009040 comp rx=0 tx=0).stop 2024-09-10T10:02:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.652+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2838153e90 msgr2=0x7f28381553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.652+0000 7f283d800640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2838153e90 0x7f28381553f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2824011cd0 tx=0x7f282400f610 comp rx=0 tx=0).stop 2024-09-10T10:02:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.656+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 shutdown_connections 2024-09-10T10:02:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.656+0000 7f283d800640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f2818067c50 0x7f281806a110 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.656+0000 7f283d800640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283815f480 0x7f28381564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.656+0000 7f283d800640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2838153e90 0x7f28381553f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.656+0000 7f283d800640 1 --2- 172.21.15.110:0/248143180 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f281c004d50 0x7f283813b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.656+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 >> 172.21.15.110:0/248143180 conn(0x7f28380b42d0 msgr2=0x7f28380b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:02.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.656+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 shutdown_connections 2024-09-10T10:02:02.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.656+0000 7f283d800640 1 -- 172.21.15.110:0/248143180 wait complete. 2024-09-10T10:02:02.676 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T10:02:02.676 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:02:02.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.808+0000 7f3dd8400640 1 Processor -- start 2024-09-10T10:02:02.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.808+0000 7f3dd8400640 1 -- start start 2024-09-10T10:02:02.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.808+0000 7f3dd8400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3dd00cd510 0x7f3dd00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.808+0000 7f3dd8400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd006e090 0x7f3dd006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.808+0000 7f3dd8400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd0072150 con 0x7f3dd00cfe30 2024-09-10T10:02:02.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.808+0000 7f3dd8400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd015d9a0 con 0x7f3dd00cd510 2024-09-10T10:02:02.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.808+0000 7f3dd8400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd006ea60 con 0x7f3dd006e090 2024-09-10T10:02:02.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd6400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd006e090 0x7f3dd006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd5000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3dd00cd510 0x7f3dd00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd6400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd006e090 0x7f3dd006e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:36220/0 (socket says 172.21.15.110:36220) 2024-09-10T10:02:02.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd6400640 1 -- 172.21.15.110:0/3836142597 learned_addr learned my addr 172.21.15.110:0/3836142597 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:02.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd6400640 1 -- 172.21.15.110:0/3836142597 >> v2:172.21.15.145:3300/0 conn(0x7f3dd00cd510 msgr2=0x7f3dd00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:02.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd6400640 1 --2- 172.21.15.110:0/3836142597 >> v2:172.21.15.145:3300/0 conn(0x7f3dd00cd510 0x7f3dd00cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd6400640 1 -- 172.21.15.110:0/3836142597 >> v1:172.21.15.145:6789/0 conn(0x7f3dd00cfe30 legacy=0x7f3dd0071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd6400640 1 -- 172.21.15.110:0/3836142597 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd006d2d0 con 0x7f3dd006e090 2024-09-10T10:02:02.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dd6400640 1 --2- 172.21.15.110:0/3836142597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd006e090 0x7f3dd006e520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009870 tx=0x7f3dc801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732511a540c088b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.812+0000 7f3dcfe00640 1 -- 172.21.15.110:0/3836142597 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80b1070 con 0x7f3dd006e090 2024-09-10T10:02:02.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dcfe00640 1 -- 172.21.15.110:0/3836142597 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc80a8050 con 0x7f3dd006e090 2024-09-10T10:02:02.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dcfe00640 1 -- 172.21.15.110:0/3836142597 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80ac860 con 0x7f3dd006e090 2024-09-10T10:02:02.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- 172.21.15.110:0/3836142597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd006e090 msgr2=0x7f3dd006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3836142597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd006e090 0x7f3dd006e520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009870 tx=0x7f3dc801c4d0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- 172.21.15.110:0/3836142597 shutdown_connections 2024-09-10T10:02:02.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3836142597 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd006e090 0x7f3dd006e520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3836142597 >> v2:172.21.15.145:3300/0 conn(0x7f3dd00cd510 0x7f3dd00cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- 172.21.15.110:0/3836142597 >> 172.21.15.110:0/3836142597 conn(0x7f3dd006bc50 msgr2=0x7f3dd006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:02.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- 172.21.15.110:0/3836142597 shutdown_connections 2024-09-10T10:02:02.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- 172.21.15.110:0/3836142597 wait complete. 2024-09-10T10:02:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 Processor -- start 2024-09-10T10:02:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- start start 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dd006e090 0x7f3dd00b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd00cd510 0x7f3dd01573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd00cfe30 0x7f3dd014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00b35d0 con 0x7f3dd006e090 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd015d6c0 con 0x7f3dd00cd510 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd8400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd00b3750 con 0x7f3dd00cfe30 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dd006e090 0x7f3dd00b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dd006e090 0x7f3dd00b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44114/0 (socket says 172.21.15.110:44114) 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd00cd510 0x7f3dd01573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5a00640 1 -- 172.21.15.110:0/3705398304 learned_addr learned my addr 172.21.15.110:0/3705398304 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd6400640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd00cfe30 0x7f3dd014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5a00640 1 -- 172.21.15.110:0/3705398304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd00cfe30 msgr2=0x7f3dd014d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5a00640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd00cfe30 0x7f3dd014d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5a00640 1 -- 172.21.15.110:0/3705398304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd00cd510 msgr2=0x7f3dd01573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5a00640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd00cd510 0x7f3dd01573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.816+0000 7f3dd5a00640 1 -- 172.21.15.110:0/3705398304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd0157920 con 0x7f3dd006e090 2024-09-10T10:02:02.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dd5a00640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dd006e090 0x7f3dd00b22c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c730 tx=0x7f3dc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dcd600640 1 -- 172.21.15.110:0/3705398304 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4016020 con 0x7f3dd006e090 2024-09-10T10:02:02.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dcd600640 1 -- 172.21.15.110:0/3705398304 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4006d60 con 0x7f3dd006e090 2024-09-10T10:02:02.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dcd600640 1 -- 172.21.15.110:0/3705398304 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4003bc0 con 0x7f3dd006e090 2024-09-10T10:02:02.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd00b2090 con 0x7f3dd006e090 2024-09-10T10:02:02.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd00b2c30 con 0x7f3dd006e090 2024-09-10T10:02:02.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd006e520 con 0x7f3dd006e090 2024-09-10T10:02:02.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dcd600640 1 -- 172.21.15.110:0/3705398304 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f3dc400a850 con 0x7f3dd006e090 2024-09-10T10:02:02.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dcd600640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3db0067c20 0x7f3db006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dcd600640 1 -- 172.21.15.110:0/3705398304 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f3dc4085d00 con 0x7f3dd006e090 2024-09-10T10:02:02.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.820+0000 7f3dd5000640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3db0067c20 0x7f3db006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.824+0000 7f3dd5000640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3db0067c20 0x7f3db006a0e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3dd015fbe0 tx=0x7f3dbc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.824+0000 7f3dcd600640 1 -- 172.21.15.110:0/3705398304 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3dc4020020 con 0x7f3dd006e090 2024-09-10T10:02:02.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.980+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3dd0157ba0 con 0x7f3db0067c20 2024-09-10T10:02:02.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.984+0000 7f3dcd600640 1 -- 172.21.15.110:0/3705398304 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251428 (secure 0 0 0) 0x7f3dd0157ba0 con 0x7f3db0067c20 2024-09-10T10:02:02.990 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:02.991 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:02:02.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3db0067c20 msgr2=0x7f3db006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3db0067c20 0x7f3db006a0e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3dd015fbe0 tx=0x7f3dbc009190 comp rx=0 tx=0).stop 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dd006e090 msgr2=0x7f3dd00b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dd006e090 0x7f3dd00b22c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c730 tx=0x7f3dc400cbf0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 shutdown_connections 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3db0067c20 0x7f3db006a0e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dd00cfe30 0x7f3dd014d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dd00cd510 0x7f3dd01573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 --2- 172.21.15.110:0/3705398304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dd006e090 0x7f3dd00b22c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 >> 172.21.15.110:0/3705398304 conn(0x7f3dd006bc50 msgr2=0x7f3dd006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 shutdown_connections 2024-09-10T10:02:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:02.988+0000 7f3dd8400640 1 -- 172.21.15.110:0/3705398304 wait complete. 2024-09-10T10:02:03.011 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-10T10:02:01.402066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248728,"kb_used_data":7520,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725992,"statfs":{"total":773094113280,"available":772839415808,"internally_reserved":0,"allocated":7700480,"data_stored":3865801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":246795820},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002773"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431713+0000","last_change":"2024-09-10T10:01:57.425627+0000","last_active":"2024-09-10T10:01:57.431713+0000","last_peered":"2024-09-10T10:01:57.431713+0000","last_clean":"2024-09-10T10:01:57.431713+0000","last_became_active":"2024-09-10T10:01:57.425083+0000","last_became_peered":"2024-09-10T10:01:57.425083+0000","last_unstale":"2024-09-10T10:01:57.431713+0000","last_undegraded":"2024-09-10T10:01:57.431713+0000","last_fullsized":"2024-09-10T10:01:57.431713+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419573+0000","last_change":"2024-09-10T10:01:57.410364+0000","last_active":"2024-09-10T10:01:57.419573+0000","last_peered":"2024-09-10T10:01:57.419573+0000","last_clean":"2024-09-10T10:01:57.419573+0000","last_became_active":"2024-09-10T10:01:57.410086+0000","last_became_peered":"2024-09-10T10:01:57.410086+0000","last_unstale":"2024-09-10T10:01:57.419573+0000","last_undegraded":"2024-09-10T10:01:57.419573+0000","last_fullsized":"2024-09-10T10:01:57.419573+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.839704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.397555+0000","last_change":"2024-09-10T10:01:56.397555+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.397555+0000","last_undegraded":"2024-09-10T10:01:56.397555+0000","last_fullsized":"2024-09-10T10:01:56.397555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422592+0000","last_change":"2024-09-10T10:01:57.419086+0000","last_active":"2024-09-10T10:01:57.422592+0000","last_peered":"2024-09-10T10:01:57.422592+0000","last_clean":"2024-09-10T10:01:57.422592+0000","last_became_active":"2024-09-10T10:01:57.418640+0000","last_became_peered":"2024-09-10T10:01:57.418640+0000","last_unstale":"2024-09-10T10:01:57.422592+0000","last_undegraded":"2024-09-10T10:01:57.422592+0000","last_fullsized":"2024-09-10T10:01:57.422592+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:37.232883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417356+0000","last_change":"2024-09-10T10:01:57.412199+0000","last_active":"2024-09-10T10:01:57.417356+0000","last_peered":"2024-09-10T10:01:57.417356+0000","last_clean":"2024-09-10T10:01:57.417356+0000","last_became_active":"2024-09-10T10:01:57.411926+0000","last_became_peered":"2024-09-10T10:01:57.411926+0000","last_unstale":"2024-09-10T10:01:57.417356+0000","last_undegraded":"2024-09-10T10:01:57.417356+0000","last_fullsized":"2024-09-10T10:01:57.417356+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:50.877508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421654+0000","last_change":"2024-09-10T10:01:57.411801+0000","last_active":"2024-09-10T10:01:57.421654+0000","last_peered":"2024-09-10T10:01:57.421654+0000","last_clean":"2024-09-10T10:01:57.421654+0000","last_became_active":"2024-09-10T10:01:57.411506+0000","last_became_peered":"2024-09-10T10:01:57.411506+0000","last_unstale":"2024-09-10T10:01:57.421654+0000","last_undegraded":"2024-09-10T10:01:57.421654+0000","last_fullsized":"2024-09-10T10:01:57.421654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:53.511190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431828+0000","last_change":"2024-09-10T10:01:57.427391+0000","last_active":"2024-09-10T10:01:57.431828+0000","last_peered":"2024-09-10T10:01:57.431828+0000","last_clean":"2024-09-10T10:01:57.431828+0000","last_became_active":"2024-09-10T10:01:57.427095+0000","last_became_peered":"2024-09-10T10:01:57.427095+0000","last_unstale":"2024-09-10T10:01:57.431828+0000","last_undegraded":"2024-09-10T10:01:57.431828+0000","last_fullsized":"2024-09-10T10:01:57.431828+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414913+0000","last_change":"2024-09-10T10:01:57.405094+0000","last_active":"2024-09-10T10:01:57.414913+0000","last_peered":"2024-09-10T10:01:57.414913+0000","last_clean":"2024-09-10T10:01:57.414913+0000","last_became_active":"2024-09-10T10:01:57.404803+0000","last_became_peered":"2024-09-10T10:01:57.404803+0000","last_unstale":"2024-09-10T10:01:57.414913+0000","last_undegraded":"2024-09-10T10:01:57.414913+0000","last_fullsized":"2024-09-10T10:01:57.414913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:51.428680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419139+0000","last_change":"2024-09-10T10:01:57.414814+0000","last_active":"2024-09-10T10:01:57.419139+0000","last_peered":"2024-09-10T10:01:57.419139+0000","last_clean":"2024-09-10T10:01:57.419139+0000","last_became_active":"2024-09-10T10:01:57.414440+0000","last_became_peered":"2024-09-10T10:01:57.414440+0000","last_unstale":"2024-09-10T10:01:57.419139+0000","last_undegraded":"2024-09-10T10:01:57.419139+0000","last_fullsized":"2024-09-10T10:01:57.419139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:04.748497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415584+0000","last_change":"2024-09-10T10:01:57.411661+0000","last_active":"2024-09-10T10:01:57.415584+0000","last_peered":"2024-09-10T10:01:57.415584+0000","last_clean":"2024-09-10T10:01:57.415584+0000","last_became_active":"2024-09-10T10:01:57.410577+0000","last_became_peered":"2024-09-10T10:01:57.410577+0000","last_unstale":"2024-09-10T10:01:57.415584+0000","last_undegraded":"2024-09-10T10:01:57.415584+0000","last_fullsized":"2024-09-10T10:01:57.415584+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:53.511190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423038+0000","last_change":"2024-09-10T10:01:57.420979+0000","last_active":"2024-09-10T10:01:57.423038+0000","last_peered":"2024-09-10T10:01:57.423038+0000","last_clean":"2024-09-10T10:01:57.423038+0000","last_became_active":"2024-09-10T10:01:57.420695+0000","last_became_peered":"2024-09-10T10:01:57.420695+0000","last_unstale":"2024-09-10T10:01:57.423038+0000","last_undegraded":"2024-09-10T10:01:57.423038+0000","last_fullsized":"2024-09-10T10:01:57.423038+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413485+0000","last_change":"2024-09-10T10:01:57.403883+0000","last_active":"2024-09-10T10:01:57.413485+0000","last_peered":"2024-09-10T10:01:57.413485+0000","last_clean":"2024-09-10T10:01:57.413485+0000","last_became_active":"2024-09-10T10:01:57.403600+0000","last_became_peered":"2024-09-10T10:01:57.403600+0000","last_unstale":"2024-09-10T10:01:57.413485+0000","last_undegraded":"2024-09-10T10:01:57.413485+0000","last_fullsized":"2024-09-10T10:01:57.413485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:53.214945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415178+0000","last_change":"2024-09-10T10:01:57.409326+0000","last_active":"2024-09-10T10:01:57.415178+0000","last_peered":"2024-09-10T10:01:57.415178+0000","last_clean":"2024-09-10T10:01:57.415178+0000","last_became_active":"2024-09-10T10:01:57.409053+0000","last_became_peered":"2024-09-10T10:01:57.409053+0000","last_unstale":"2024-09-10T10:01:57.415178+0000","last_undegraded":"2024-09-10T10:01:57.415178+0000","last_fullsized":"2024-09-10T10:01:57.415178+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424665+0000","last_change":"2024-09-10T10:01:57.421822+0000","last_active":"2024-09-10T10:01:57.424665+0000","last_peered":"2024-09-10T10:01:57.424665+0000","last_clean":"2024-09-10T10:01:57.424665+0000","last_became_active":"2024-09-10T10:01:57.421559+0000","last_became_peered":"2024-09-10T10:01:57.421559+0000","last_unstale":"2024-09-10T10:01:57.424665+0000","last_undegraded":"2024-09-10T10:01:57.424665+0000","last_fullsized":"2024-09-10T10:01:57.424665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:10.788546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414952+0000","last_change":"2024-09-10T10:01:57.410832+0000","last_active":"2024-09-10T10:01:57.414952+0000","last_peered":"2024-09-10T10:01:57.414952+0000","last_clean":"2024-09-10T10:01:57.414952+0000","last_became_active":"2024-09-10T10:01:57.410575+0000","last_became_peered":"2024-09-10T10:01:57.410575+0000","last_unstale":"2024-09-10T10:01:57.414952+0000","last_undegraded":"2024-09-10T10:01:57.414952+0000","last_fullsized":"2024-09-10T10:01:57.414952+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421152+0000","last_change":"2024-09-10T10:01:57.419056+0000","last_active":"2024-09-10T10:01:57.421152+0000","last_peered":"2024-09-10T10:01:57.421152+0000","last_clean":"2024-09-10T10:01:57.421152+0000","last_became_active":"2024-09-10T10:01:57.418704+0000","last_became_peered":"2024-09-10T10:01:57.418704+0000","last_unstale":"2024-09-10T10:01:57.421152+0000","last_undegraded":"2024-09-10T10:01:57.421152+0000","last_fullsized":"2024-09-10T10:01:57.421152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:58.162428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419118+0000","last_change":"2024-09-10T10:01:57.414713+0000","last_active":"2024-09-10T10:01:57.419118+0000","last_peered":"2024-09-10T10:01:57.419118+0000","last_clean":"2024-09-10T10:01:57.419118+0000","last_became_active":"2024-09-10T10:01:57.414227+0000","last_became_peered":"2024-09-10T10:01:57.414227+0000","last_unstale":"2024-09-10T10:01:57.419118+0000","last_undegraded":"2024-09-10T10:01:57.419118+0000","last_fullsized":"2024-09-10T10:01:57.419118+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:06.958521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.418366+0000","last_change":"2024-09-10T10:01:57.407315+0000","last_active":"2024-09-10T10:01:57.418366+0000","last_peered":"2024-09-10T10:01:57.418366+0000","last_clean":"2024-09-10T10:01:57.418366+0000","last_became_active":"2024-09-10T10:01:57.406568+0000","last_became_peered":"2024-09-10T10:01:57.406568+0000","last_unstale":"2024-09-10T10:01:57.418366+0000","last_undegraded":"2024-09-10T10:01:57.418366+0000","last_fullsized":"2024-09-10T10:01:57.418366+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:02.156822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414529+0000","last_change":"2024-09-10T10:01:57.405757+0000","last_active":"2024-09-10T10:01:57.414529+0000","last_peered":"2024-09-10T10:01:57.414529+0000","last_clean":"2024-09-10T10:01:57.414529+0000","last_became_active":"2024-09-10T10:01:57.404501+0000","last_became_peered":"2024-09-10T10:01:57.404501+0000","last_unstale":"2024-09-10T10:01:57.414529+0000","last_undegraded":"2024-09-10T10:01:57.414529+0000","last_fullsized":"2024-09-10T10:01:57.414529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:03.020211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424741+0000","last_change":"2024-09-10T10:01:57.423165+0000","last_active":"2024-09-10T10:01:57.424741+0000","last_peered":"2024-09-10T10:01:57.424741+0000","last_clean":"2024-09-10T10:01:57.424741+0000","last_became_active":"2024-09-10T10:01:57.422527+0000","last_became_peered":"2024-09-10T10:01:57.422527+0000","last_unstale":"2024-09-10T10:01:57.424741+0000","last_undegraded":"2024-09-10T10:01:57.424741+0000","last_fullsized":"2024-09-10T10:01:57.424741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417175+0000","last_change":"2024-09-10T10:01:57.409584+0000","last_active":"2024-09-10T10:01:57.417175+0000","last_peered":"2024-09-10T10:01:57.417175+0000","last_clean":"2024-09-10T10:01:57.417175+0000","last_became_active":"2024-09-10T10:01:57.409327+0000","last_became_peered":"2024-09-10T10:01:57.409327+0000","last_unstale":"2024-09-10T10:01:57.417175+0000","last_undegraded":"2024-09-10T10:01:57.417175+0000","last_fullsized":"2024-09-10T10:01:57.417175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:39.030596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419510+0000","last_change":"2024-09-10T10:01:57.416054+0000","last_active":"2024-09-10T10:01:57.419510+0000","last_peered":"2024-09-10T10:01:57.419510+0000","last_clean":"2024-09-10T10:01:57.419510+0000","last_became_active":"2024-09-10T10:01:57.415813+0000","last_became_peered":"2024-09-10T10:01:57.415813+0000","last_unstale":"2024-09-10T10:01:57.419510+0000","last_undegraded":"2024-09-10T10:01:57.419510+0000","last_fullsized":"2024-09-10T10:01:57.419510+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:11.341717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431987+0000","last_change":"2024-09-10T10:01:57.430689+0000","last_active":"2024-09-10T10:01:57.431987+0000","last_peered":"2024-09-10T10:01:57.431987+0000","last_clean":"2024-09-10T10:01:57.431987+0000","last_became_active":"2024-09-10T10:01:57.430349+0000","last_became_peered":"2024-09-10T10:01:57.430349+0000","last_unstale":"2024-09-10T10:01:57.431987+0000","last_undegraded":"2024-09-10T10:01:57.431987+0000","last_fullsized":"2024-09-10T10:01:57.431987+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:41.204498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431524+0000","last_change":"2024-09-10T10:01:57.424818+0000","last_active":"2024-09-10T10:01:57.431524+0000","last_peered":"2024-09-10T10:01:57.431524+0000","last_clean":"2024-09-10T10:01:57.431524+0000","last_became_active":"2024-09-10T10:01:57.424469+0000","last_became_peered":"2024-09-10T10:01:57.424469+0000","last_unstale":"2024-09-10T10:01:57.431524+0000","last_undegraded":"2024-09-10T10:01:57.431524+0000","last_fullsized":"2024-09-10T10:01:57.431524+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415184+0000","last_change":"2024-09-10T10:01:57.412902+0000","last_active":"2024-09-10T10:01:57.415184+0000","last_peered":"2024-09-10T10:01:57.415184+0000","last_clean":"2024-09-10T10:01:57.415184+0000","last_became_active":"2024-09-10T10:01:57.412071+0000","last_became_peered":"2024-09-10T10:01:57.412071+0000","last_unstale":"2024-09-10T10:01:57.415184+0000","last_undegraded":"2024-09-10T10:01:57.415184+0000","last_fullsized":"2024-09-10T10:01:57.415184+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:47.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431901+0000","last_change":"2024-09-10T10:01:57.428212+0000","last_active":"2024-09-10T10:01:57.431901+0000","last_peered":"2024-09-10T10:01:57.431901+0000","last_clean":"2024-09-10T10:01:57.431901+0000","last_became_active":"2024-09-10T10:01:57.427898+0000","last_became_peered":"2024-09-10T10:01:57.427898+0000","last_unstale":"2024-09-10T10:01:57.431901+0000","last_undegraded":"2024-09-10T10:01:57.431901+0000","last_fullsized":"2024-09-10T10:01:57.431901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:16.405387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.395624+0000","last_change":"2024-09-10T10:01:56.395624+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.395624+0000","last_undegraded":"2024-09-10T10:01:56.395624+0000","last_fullsized":"2024-09-10T10:01:56.395624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415287+0000","last_change":"2024-09-10T10:01:57.411461+0000","last_active":"2024-09-10T10:01:57.415287+0000","last_peered":"2024-09-10T10:01:57.415287+0000","last_clean":"2024-09-10T10:01:57.415287+0000","last_became_active":"2024-09-10T10:01:57.410729+0000","last_became_peered":"2024-09-10T10:01:57.410729+0000","last_unstale":"2024-09-10T10:01:57.415287+0000","last_undegraded":"2024-09-10T10:01:57.415287+0000","last_fullsized":"2024-09-10T10:01:57.415287+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:05.839704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394962+0000","last_change":"2024-09-10T10:01:56.394962+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394962+0000","last_undegraded":"2024-09-10T10:01:56.394962+0000","last_fullsized":"2024-09-10T10:01:56.394962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431869+0000","last_change":"2024-09-10T10:01:57.428097+0000","last_active":"2024-09-10T10:01:57.431869+0000","last_peered":"2024-09-10T10:01:57.431869+0000","last_clean":"2024-09-10T10:01:57.431869+0000","last_became_active":"2024-09-10T10:01:57.427838+0000","last_became_peered":"2024-09-10T10:01:57.427838+0000","last_unstale":"2024-09-10T10:01:57.431869+0000","last_undegraded":"2024-09-10T10:01:57.431869+0000","last_fullsized":"2024-09-10T10:01:57.431869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.426209+0000","last_change":"2024-09-10T10:01:57.416701+0000","last_active":"2024-09-10T10:01:57.426209+0000","last_peered":"2024-09-10T10:01:57.426209+0000","last_clean":"2024-09-10T10:01:57.426209+0000","last_became_active":"2024-09-10T10:01:57.416416+0000","last_became_peered":"2024-09-10T10:01:57.416416+0000","last_unstale":"2024-09-10T10:01:57.426209+0000","last_undegraded":"2024-09-10T10:01:57.426209+0000","last_fullsized":"2024-09-10T10:01:57.426209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.508652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419651+0000","last_change":"2024-09-10T10:01:57.412058+0000","last_active":"2024-09-10T10:01:57.419651+0000","last_peered":"2024-09-10T10:01:57.419651+0000","last_clean":"2024-09-10T10:01:57.419651+0000","last_became_active":"2024-09-10T10:01:57.411787+0000","last_became_peered":"2024-09-10T10:01:57.411787+0000","last_unstale":"2024-09-10T10:01:57.419651+0000","last_undegraded":"2024-09-10T10:01:57.419651+0000","last_fullsized":"2024-09-10T10:01:57.419651+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:59.314051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431681+0000","last_change":"2024-09-10T10:01:57.426610+0000","last_active":"2024-09-10T10:01:57.431681+0000","last_peered":"2024-09-10T10:01:57.431681+0000","last_clean":"2024-09-10T10:01:57.431681+0000","last_became_active":"2024-09-10T10:01:57.423435+0000","last_became_peered":"2024-09-10T10:01:57.423435+0000","last_unstale":"2024-09-10T10:01:57.431681+0000","last_undegraded":"2024-09-10T10:01:57.431681+0000","last_fullsized":"2024-09-10T10:01:57.431681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.428911+0000","last_change":"2024-09-10T10:01:56.414321+0000","last_active":"2024-09-10T10:01:57.428911+0000","last_peered":"2024-09-10T10:01:57.428911+0000","last_clean":"2024-09-10T10:01:57.428911+0000","last_became_active":"2024-09-10T10:01:56.414030+0000","last_became_peered":"2024-09-10T10:01:56.414030+0000","last_unstale":"2024-09-10T10:01:57.428911+0000","last_undegraded":"2024-09-10T10:01:57.428911+0000","last_fullsized":"2024-09-10T10:01:57.428911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419094+0000","last_change":"2024-09-10T10:01:57.414555+0000","last_active":"2024-09-10T10:01:57.419094+0000","last_peered":"2024-09-10T10:01:57.419094+0000","last_clean":"2024-09-10T10:01:57.419094+0000","last_became_active":"2024-09-10T10:01:57.413819+0000","last_became_peered":"2024-09-10T10:01:57.413819+0000","last_unstale":"2024-09-10T10:01:57.419094+0000","last_undegraded":"2024-09-10T10:01:57.419094+0000","last_fullsized":"2024-09-10T10:01:57.419094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:39.835600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423916+0000","last_change":"2024-09-10T10:01:56.408386+0000","last_active":"2024-09-10T10:01:57.423916+0000","last_peered":"2024-09-10T10:01:57.423916+0000","last_clean":"2024-09-10T10:01:57.423916+0000","last_became_active":"2024-09-10T10:01:56.408127+0000","last_became_peered":"2024-09-10T10:01:56.408127+0000","last_unstale":"2024-09-10T10:01:57.423916+0000","last_undegraded":"2024-09-10T10:01:57.423916+0000","last_fullsized":"2024-09-10T10:01:57.423916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:59.865257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421159+0000","last_change":"2024-09-10T10:01:57.417292+0000","last_active":"2024-09-10T10:01:57.421159+0000","last_peered":"2024-09-10T10:01:57.421159+0000","last_clean":"2024-09-10T10:01:57.421159+0000","last_became_active":"2024-09-10T10:01:57.416917+0000","last_became_peered":"2024-09-10T10:01:57.416917+0000","last_unstale":"2024-09-10T10:01:57.421159+0000","last_undegraded":"2024-09-10T10:01:57.421159+0000","last_fullsized":"2024-09-10T10:01:57.421159+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:11.341717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422155+0000","last_change":"2024-09-10T10:01:56.409023+0000","last_active":"2024-09-10T10:01:57.422155+0000","last_peered":"2024-09-10T10:01:57.422155+0000","last_clean":"2024-09-10T10:01:57.422155+0000","last_became_active":"2024-09-10T10:01:56.408748+0000","last_became_peered":"2024-09-10T10:01:56.408748+0000","last_unstale":"2024-09-10T10:01:57.422155+0000","last_undegraded":"2024-09-10T10:01:57.422155+0000","last_fullsized":"2024-09-10T10:01:57.422155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:50.426131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413959+0000","last_change":"2024-09-10T10:01:57.408727+0000","last_active":"2024-09-10T10:01:57.413959+0000","last_peered":"2024-09-10T10:01:57.413959+0000","last_clean":"2024-09-10T10:01:57.413959+0000","last_became_active":"2024-09-10T10:01:57.408448+0000","last_became_peered":"2024-09-10T10:01:57.408448+0000","last_unstale":"2024-09-10T10:01:57.413959+0000","last_undegraded":"2024-09-10T10:01:57.413959+0000","last_fullsized":"2024-09-10T10:01:57.413959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427217+0000","last_change":"2024-09-10T10:01:56.409387+0000","last_active":"2024-09-10T10:01:57.427217+0000","last_peered":"2024-09-10T10:01:57.427217+0000","last_clean":"2024-09-10T10:01:57.427217+0000","last_became_active":"2024-09-10T10:01:56.407044+0000","last_became_peered":"2024-09-10T10:01:56.407044+0000","last_unstale":"2024-09-10T10:01:57.427217+0000","last_undegraded":"2024-09-10T10:01:57.427217+0000","last_fullsized":"2024-09-10T10:01:57.427217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.423211+0000","last_change":"2024-09-10T10:01:57.420630+0000","last_active":"2024-09-10T10:01:57.423211+0000","last_peered":"2024-09-10T10:01:57.423211+0000","last_clean":"2024-09-10T10:01:57.423211+0000","last_became_active":"2024-09-10T10:01:57.416989+0000","last_became_peered":"2024-09-10T10:01:57.416989+0000","last_unstale":"2024-09-10T10:01:57.423211+0000","last_undegraded":"2024-09-10T10:01:57.423211+0000","last_fullsized":"2024-09-10T10:01:57.423211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:14.901947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.407559+0000","last_change":"2024-09-10T10:01:56.404458+0000","last_active":"2024-09-10T10:01:56.407559+0000","last_peered":"2024-09-10T10:01:56.407559+0000","last_clean":"2024-09-10T10:01:56.407559+0000","last_became_active":"2024-09-10T10:01:56.404205+0000","last_became_peered":"2024-09-10T10:01:56.404205+0000","last_unstale":"2024-09-10T10:01:56.407559+0000","last_undegraded":"2024-09-10T10:01:56.407559+0000","last_fullsized":"2024-09-10T10:01:56.407559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:14.034052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417285+0000","last_change":"2024-09-10T10:01:57.412841+0000","last_active":"2024-09-10T10:01:57.417285+0000","last_peered":"2024-09-10T10:01:57.417285+0000","last_clean":"2024-09-10T10:01:57.417285+0000","last_became_active":"2024-09-10T10:01:57.412551+0000","last_became_peered":"2024-09-10T10:01:57.412551+0000","last_unstale":"2024-09-10T10:01:57.417285+0000","last_undegraded":"2024-09-10T10:01:57.417285+0000","last_fullsized":"2024-09-10T10:01:57.417285+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:59.314051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421685+0000","last_change":"2024-09-10T10:01:56.398879+0000","last_active":"2024-09-10T10:01:57.421685+0000","last_peered":"2024-09-10T10:01:57.421685+0000","last_clean":"2024-09-10T10:01:57.421685+0000","last_became_active":"2024-09-10T10:01:56.398576+0000","last_became_peered":"2024-09-10T10:01:56.398576+0000","last_unstale":"2024-09-10T10:01:57.421685+0000","last_undegraded":"2024-09-10T10:01:57.421685+0000","last_fullsized":"2024-09-10T10:01:57.421685+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:01.154272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.417679+0000","last_change":"2024-09-10T10:01:57.412438+0000","last_active":"2024-09-10T10:01:57.417679+0000","last_peered":"2024-09-10T10:01:57.417679+0000","last_clean":"2024-09-10T10:01:57.417679+0000","last_became_active":"2024-09-10T10:01:57.412174+0000","last_became_peered":"2024-09-10T10:01:57.412174+0000","last_unstale":"2024-09-10T10:01:57.417679+0000","last_undegraded":"2024-09-10T10:01:57.417679+0000","last_fullsized":"2024-09-10T10:01:57.417679+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:46.178171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.391826+0000","last_change":"2024-09-10T10:01:56.406654+0000","last_active":"2024-09-10T10:01:57.391826+0000","last_peered":"2024-09-10T10:01:57.391826+0000","last_clean":"2024-09-10T10:01:57.391826+0000","last_became_active":"2024-09-10T10:01:56.406375+0000","last_became_peered":"2024-09-10T10:01:56.406375+0000","last_unstale":"2024-09-10T10:01:57.391826+0000","last_undegraded":"2024-09-10T10:01:57.391826+0000","last_fullsized":"2024-09-10T10:01:57.391826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:07.017690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414349+0000","last_change":"2024-09-10T10:01:57.408804+0000","last_active":"2024-09-10T10:01:57.414349+0000","last_peered":"2024-09-10T10:01:57.414349+0000","last_clean":"2024-09-10T10:01:57.414349+0000","last_became_active":"2024-09-10T10:01:57.405976+0000","last_became_peered":"2024-09-10T10:01:57.405976+0000","last_unstale":"2024-09-10T10:01:57.414349+0000","last_undegraded":"2024-09-10T10:01:57.414349+0000","last_fullsized":"2024-09-10T10:01:57.414349+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398400+0000","last_change":"2024-09-10T10:01:56.411631+0000","last_active":"2024-09-10T10:01:57.398400+0000","last_peered":"2024-09-10T10:01:57.398400+0000","last_clean":"2024-09-10T10:01:57.398400+0000","last_became_active":"2024-09-10T10:01:56.411378+0000","last_became_peered":"2024-09-10T10:01:56.411378+0000","last_unstale":"2024-09-10T10:01:57.398400+0000","last_undegraded":"2024-09-10T10:01:57.398400+0000","last_fullsized":"2024-09-10T10:01:57.398400+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:38.118765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.424610+0000","last_change":"2024-09-10T10:01:57.422782+0000","last_active":"2024-09-10T10:01:57.424610+0000","last_peered":"2024-09-10T10:01:57.424610+0000","last_clean":"2024-09-10T10:01:57.424610+0000","last_became_active":"2024-09-10T10:01:57.420706+0000","last_became_peered":"2024-09-10T10:01:57.420706+0000","last_unstale":"2024-09-10T10:01:57.424610+0000","last_undegraded":"2024-09-10T10:01:57.424610+0000","last_fullsized":"2024-09-10T10:01:57.424610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398697+0000","last_change":"2024-09-10T10:01:56.403288+0000","last_active":"2024-09-10T10:01:57.398697+0000","last_peered":"2024-09-10T10:01:57.398697+0000","last_clean":"2024-09-10T10:01:57.398697+0000","last_became_active":"2024-09-10T10:01:56.402990+0000","last_became_peered":"2024-09-10T10:01:56.402990+0000","last_unstale":"2024-09-10T10:01:57.398697+0000","last_undegraded":"2024-09-10T10:01:57.398697+0000","last_fullsized":"2024-09-10T10:01:57.398697+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431695+0000","last_change":"2024-09-10T10:01:57.425385+0000","last_active":"2024-09-10T10:01:57.431695+0000","last_peered":"2024-09-10T10:01:57.431695+0000","last_clean":"2024-09-10T10:01:57.431695+0000","last_became_active":"2024-09-10T10:01:57.425032+0000","last_became_peered":"2024-09-10T10:01:57.425032+0000","last_unstale":"2024-09-10T10:01:57.431695+0000","last_undegraded":"2024-09-10T10:01:57.431695+0000","last_fullsized":"2024-09-10T10:01:57.431695+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.395612+0000","last_change":"2024-09-10T10:01:56.407609+0000","last_active":"2024-09-10T10:01:57.395612+0000","last_peered":"2024-09-10T10:01:57.395612+0000","last_clean":"2024-09-10T10:01:57.395612+0000","last_became_active":"2024-09-10T10:01:56.405401+0000","last_became_peered":"2024-09-10T10:01:56.405401+0000","last_unstale":"2024-09-10T10:01:57.395612+0000","last_undegraded":"2024-09-10T10:01:57.395612+0000","last_fullsized":"2024-09-10T10:01:57.395612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.419650+0000","last_change":"2024-09-10T10:01:57.412223+0000","last_active":"2024-09-10T10:01:57.419650+0000","last_peered":"2024-09-10T10:01:57.419650+0000","last_clean":"2024-09-10T10:01:57.419650+0000","last_became_active":"2024-09-10T10:01:57.411477+0000","last_became_peered":"2024-09-10T10:01:57.411477+0000","last_unstale":"2024-09-10T10:01:57.419650+0000","last_undegraded":"2024-09-10T10:01:57.419650+0000","last_fullsized":"2024-09-10T10:01:57.419650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:47.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.407993+0000","last_change":"2024-09-10T10:01:56.406998+0000","last_active":"2024-09-10T10:01:56.407993+0000","last_peered":"2024-09-10T10:01:56.407993+0000","last_clean":"2024-09-10T10:01:56.407993+0000","last_became_active":"2024-09-10T10:01:56.406722+0000","last_became_peered":"2024-09-10T10:01:56.406722+0000","last_unstale":"2024-09-10T10:01:56.407993+0000","last_undegraded":"2024-09-10T10:01:56.407993+0000","last_fullsized":"2024-09-10T10:01:56.407993+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:27.783209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414220+0000","last_change":"2024-09-10T10:01:57.409970+0000","last_active":"2024-09-10T10:01:57.414220+0000","last_peered":"2024-09-10T10:01:57.414220+0000","last_clean":"2024-09-10T10:01:57.414220+0000","last_became_active":"2024-09-10T10:01:57.406199+0000","last_became_peered":"2024-09-10T10:01:57.406199+0000","last_unstale":"2024-09-10T10:01:57.414220+0000","last_undegraded":"2024-09-10T10:01:57.414220+0000","last_fullsized":"2024-09-10T10:01:57.414220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:41.660285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.426946+0000","last_change":"2024-09-10T10:01:56.410726+0000","last_active":"2024-09-10T10:01:57.426946+0000","last_peered":"2024-09-10T10:01:57.426946+0000","last_clean":"2024-09-10T10:01:57.426946+0000","last_became_active":"2024-09-10T10:01:56.410239+0000","last_became_peered":"2024-09-10T10:01:56.410239+0000","last_unstale":"2024-09-10T10:01:57.426946+0000","last_undegraded":"2024-09-10T10:01:57.426946+0000","last_fullsized":"2024-09-10T10:01:57.426946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:44.033036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.429665+0000","last_change":"2024-09-10T10:01:57.423932+0000","last_active":"2024-09-10T10:01:57.429665+0000","last_peered":"2024-09-10T10:01:57.429665+0000","last_clean":"2024-09-10T10:01:57.429665+0000","last_became_active":"2024-09-10T10:01:57.423635+0000","last_became_peered":"2024-09-10T10:01:57.423635+0000","last_unstale":"2024-09-10T10:01:57.429665+0000","last_undegraded":"2024-09-10T10:01:57.429665+0000","last_fullsized":"2024-09-10T10:01:57.429665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431982+0000","last_change":"2024-09-10T10:01:56.411145+0000","last_active":"2024-09-10T10:01:57.431982+0000","last_peered":"2024-09-10T10:01:57.431982+0000","last_clean":"2024-09-10T10:01:57.431982+0000","last_became_active":"2024-09-10T10:01:56.410821+0000","last_became_peered":"2024-09-10T10:01:56.410821+0000","last_unstale":"2024-09-10T10:01:57.431982+0000","last_undegraded":"2024-09-10T10:01:57.431982+0000","last_fullsized":"2024-09-10T10:01:57.431982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:34.635430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431813+0000","last_change":"2024-09-10T10:01:57.427059+0000","last_active":"2024-09-10T10:01:57.431813+0000","last_peered":"2024-09-10T10:01:57.431813+0000","last_clean":"2024-09-10T10:01:57.431813+0000","last_became_active":"2024-09-10T10:01:57.423512+0000","last_became_peered":"2024-09-10T10:01:57.423512+0000","last_unstale":"2024-09-10T10:01:57.431813+0000","last_undegraded":"2024-09-10T10:01:57.431813+0000","last_fullsized":"2024-09-10T10:01:57.431813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394598+0000","last_change":"2024-09-10T10:01:56.405376+0000","last_active":"2024-09-10T10:01:57.394598+0000","last_peered":"2024-09-10T10:01:57.394598+0000","last_clean":"2024-09-10T10:01:57.394598+0000","last_became_active":"2024-09-10T10:01:56.405056+0000","last_became_peered":"2024-09-10T10:01:56.405056+0000","last_unstale":"2024-09-10T10:01:57.394598+0000","last_undegraded":"2024-09-10T10:01:57.394598+0000","last_fullsized":"2024-09-10T10:01:57.394598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:56.936736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415321+0000","last_change":"2024-09-10T10:01:57.411055+0000","last_active":"2024-09-10T10:01:57.415321+0000","last_peered":"2024-09-10T10:01:57.415321+0000","last_clean":"2024-09-10T10:01:57.415321+0000","last_became_active":"2024-09-10T10:01:57.410789+0000","last_became_peered":"2024-09-10T10:01:57.410789+0000","last_unstale":"2024-09-10T10:01:57.415321+0000","last_undegraded":"2024-09-10T10:01:57.415321+0000","last_fullsized":"2024-09-10T10:01:57.415321+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.412528+0000","last_change":"2024-09-10T10:01:56.412704+0000","last_active":"2024-09-10T10:01:57.412528+0000","last_peered":"2024-09-10T10:01:57.412528+0000","last_clean":"2024-09-10T10:01:57.412528+0000","last_became_active":"2024-09-10T10:01:56.410023+0000","last_became_peered":"2024-09-10T10:01:56.410023+0000","last_unstale":"2024-09-10T10:01:57.412528+0000","last_undegraded":"2024-09-10T10:01:57.412528+0000","last_fullsized":"2024-09-10T10:01:57.412528+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:57.159878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415068+0000","last_change":"2024-09-10T10:01:57.407954+0000","last_active":"2024-09-10T10:01:57.415068+0000","last_peered":"2024-09-10T10:01:57.415068+0000","last_clean":"2024-09-10T10:01:57.415068+0000","last_became_active":"2024-09-10T10:01:57.407651+0000","last_became_peered":"2024-09-10T10:01:57.407651+0000","last_unstale":"2024-09-10T10:01:57.415068+0000","last_undegraded":"2024-09-10T10:01:57.415068+0000","last_fullsized":"2024-09-10T10:01:57.415068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:16.908619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.430713+0000","last_change":"2024-09-10T10:01:56.410104+0000","last_active":"2024-09-10T10:01:57.430713+0000","last_peered":"2024-09-10T10:01:57.430713+0000","last_clean":"2024-09-10T10:01:57.430713+0000","last_became_active":"2024-09-10T10:01:56.409756+0000","last_became_peered":"2024-09-10T10:01:56.409756+0000","last_unstale":"2024-09-10T10:01:57.430713+0000","last_undegraded":"2024-09-10T10:01:57.430713+0000","last_fullsized":"2024-09-10T10:01:57.430713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.409223+0000","last_change":"2024-09-10T10:01:57.405556+0000","last_active":"2024-09-10T10:01:57.409223+0000","last_peered":"2024-09-10T10:01:57.409223+0000","last_clean":"2024-09-10T10:01:57.409223+0000","last_became_active":"2024-09-10T10:01:57.405222+0000","last_became_peered":"2024-09-10T10:01:57.405222+0000","last_unstale":"2024-09-10T10:01:57.409223+0000","last_undegraded":"2024-09-10T10:01:57.409223+0000","last_fullsized":"2024-09-10T10:01:57.409223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:03.020211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.425896+0000","last_change":"2024-09-10T10:01:56.401752+0000","last_active":"2024-09-10T10:01:57.425896+0000","last_peered":"2024-09-10T10:01:57.425896+0000","last_clean":"2024-09-10T10:01:57.425896+0000","last_became_active":"2024-09-10T10:01:56.401508+0000","last_became_peered":"2024-09-10T10:01:56.401508+0000","last_unstale":"2024-09-10T10:01:57.425896+0000","last_undegraded":"2024-09-10T10:01:57.425896+0000","last_fullsized":"2024-09-10T10:01:57.425896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:52.212395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431672+0000","last_change":"2024-09-10T10:01:57.425326+0000","last_active":"2024-09-10T10:01:57.431672+0000","last_peered":"2024-09-10T10:01:57.431672+0000","last_clean":"2024-09-10T10:01:57.431672+0000","last_became_active":"2024-09-10T10:01:57.424606+0000","last_became_peered":"2024-09-10T10:01:57.424606+0000","last_unstale":"2024-09-10T10:01:57.431672+0000","last_undegraded":"2024-09-10T10:01:57.431672+0000","last_fullsized":"2024-09-10T10:01:57.431672+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:23.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.422525+0000","last_change":"2024-09-10T10:01:56.409357+0000","last_active":"2024-09-10T10:01:57.422525+0000","last_peered":"2024-09-10T10:01:57.422525+0000","last_clean":"2024-09-10T10:01:57.422525+0000","last_became_active":"2024-09-10T10:01:56.409062+0000","last_became_peered":"2024-09-10T10:01:56.409062+0000","last_unstale":"2024-09-10T10:01:57.422525+0000","last_undegraded":"2024-09-10T10:01:57.422525+0000","last_fullsized":"2024-09-10T10:01:57.422525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:29.657091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413399+0000","last_change":"2024-09-10T10:01:57.401546+0000","last_active":"2024-09-10T10:01:57.413399+0000","last_peered":"2024-09-10T10:01:57.413399+0000","last_clean":"2024-09-10T10:01:57.413399+0000","last_became_active":"2024-09-10T10:01:57.401293+0000","last_became_peered":"2024-09-10T10:01:57.401293+0000","last_unstale":"2024-09-10T10:01:57.413399+0000","last_undegraded":"2024-09-10T10:01:57.413399+0000","last_fullsized":"2024-09-10T10:01:57.413399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:19.413048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427627+0000","last_change":"2024-09-10T10:01:56.399162+0000","last_active":"2024-09-10T10:01:57.427627+0000","last_peered":"2024-09-10T10:01:57.427627+0000","last_clean":"2024-09-10T10:01:57.427627+0000","last_became_active":"2024-09-10T10:01:56.398886+0000","last_became_peered":"2024-09-10T10:01:56.398886+0000","last_unstale":"2024-09-10T10:01:57.427627+0000","last_undegraded":"2024-09-10T10:01:57.427627+0000","last_fullsized":"2024-09-10T10:01:57.427627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.414198+0000","last_change":"2024-09-10T10:01:57.409275+0000","last_active":"2024-09-10T10:01:57.414198+0000","last_peered":"2024-09-10T10:01:57.414198+0000","last_clean":"2024-09-10T10:01:57.414198+0000","last_became_active":"2024-09-10T10:01:57.409007+0000","last_became_peered":"2024-09-10T10:01:57.409007+0000","last_unstale":"2024-09-10T10:01:57.414198+0000","last_undegraded":"2024-09-10T10:01:57.414198+0000","last_fullsized":"2024-09-10T10:01:57.414198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:17.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.405547+0000","last_change":"2024-09-10T10:01:56.414600+0000","last_active":"2024-09-10T10:01:57.405547+0000","last_peered":"2024-09-10T10:01:57.405547+0000","last_clean":"2024-09-10T10:01:57.405547+0000","last_became_active":"2024-09-10T10:01:56.414312+0000","last_became_peered":"2024-09-10T10:01:56.414312+0000","last_unstale":"2024-09-10T10:01:57.405547+0000","last_undegraded":"2024-09-10T10:01:57.405547+0000","last_fullsized":"2024-09-10T10:01:57.405547+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:13.899397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394589+0000","last_change":"2024-09-10T10:01:56.394589+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394589+0000","last_undegraded":"2024-09-10T10:01:56.394589+0000","last_fullsized":"2024-09-10T10:01:56.394589+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394540+0000","last_change":"2024-09-10T10:01:56.399440+0000","last_active":"2024-09-10T10:01:57.394540+0000","last_peered":"2024-09-10T10:01:57.394540+0000","last_clean":"2024-09-10T10:01:57.394540+0000","last_became_active":"2024-09-10T10:01:56.399153+0000","last_became_peered":"2024-09-10T10:01:56.399153+0000","last_unstale":"2024-09-10T10:01:57.394540+0000","last_undegraded":"2024-09-10T10:01:57.394540+0000","last_fullsized":"2024-09-10T10:01:57.394540+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:21.685329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.393504+0000","last_change":"2024-09-10T10:01:56.393504+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.393504+0000","last_undegraded":"2024-09-10T10:01:56.393504+0000","last_fullsized":"2024-09-10T10:01:56.393504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.412691+0000","last_change":"2024-09-10T10:01:56.410145+0000","last_active":"2024-09-10T10:01:57.412691+0000","last_peered":"2024-09-10T10:01:57.412691+0000","last_clean":"2024-09-10T10:01:57.412691+0000","last_became_active":"2024-09-10T10:01:56.409854+0000","last_became_peered":"2024-09-10T10:01:56.409854+0000","last_unstale":"2024-09-10T10:01:57.412691+0000","last_undegraded":"2024-09-10T10:01:57.412691+0000","last_fullsized":"2024-09-10T10:01:57.412691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:23.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.393865+0000","last_change":"2024-09-10T10:01:56.393865+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.393865+0000","last_undegraded":"2024-09-10T10:01:56.393865+0000","last_fullsized":"2024-09-10T10:01:56.393865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.395889+0000","last_change":"2024-09-10T10:01:56.404958+0000","last_active":"2024-09-10T10:01:57.395889+0000","last_peered":"2024-09-10T10:01:57.395889+0000","last_clean":"2024-09-10T10:01:57.395889+0000","last_became_active":"2024-09-10T10:01:56.404619+0000","last_became_peered":"2024-09-10T10:01:56.404619+0000","last_unstale":"2024-09-10T10:01:57.395889+0000","last_undegraded":"2024-09-10T10:01:57.395889+0000","last_fullsized":"2024-09-10T10:01:57.395889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415096+0000","last_change":"2024-09-10T10:01:57.409907+0000","last_active":"2024-09-10T10:01:57.415096+0000","last_peered":"2024-09-10T10:01:57.415096+0000","last_clean":"2024-09-10T10:01:57.415096+0000","last_became_active":"2024-09-10T10:01:57.409375+0000","last_became_peered":"2024-09-10T10:01:57.409375+0000","last_unstale":"2024-09-10T10:01:57.415096+0000","last_undegraded":"2024-09-10T10:01:57.415096+0000","last_fullsized":"2024-09-10T10:01:57.415096+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:23.903293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.425893+0000","last_change":"2024-09-10T10:01:56.400929+0000","last_active":"2024-09-10T10:01:57.425893+0000","last_peered":"2024-09-10T10:01:57.425893+0000","last_clean":"2024-09-10T10:01:57.425893+0000","last_became_active":"2024-09-10T10:01:56.400665+0000","last_became_peered":"2024-09-10T10:01:56.400665+0000","last_unstale":"2024-09-10T10:01:57.425893+0000","last_undegraded":"2024-09-10T10:01:57.425893+0000","last_fullsized":"2024-09-10T10:01:57.425893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:32.570015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431796+0000","last_change":"2024-09-10T10:01:57.426884+0000","last_active":"2024-09-10T10:01:57.431796+0000","last_peered":"2024-09-10T10:01:57.431796+0000","last_clean":"2024-09-10T10:01:57.431796+0000","last_became_active":"2024-09-10T10:01:57.426621+0000","last_became_peered":"2024-09-10T10:01:57.426621+0000","last_unstale":"2024-09-10T10:01:57.431796+0000","last_undegraded":"2024-09-10T10:01:57.431796+0000","last_fullsized":"2024-09-10T10:01:57.431796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:35.517149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.413728+0000","last_change":"2024-09-10T10:01:56.410855+0000","last_active":"2024-09-10T10:01:57.413728+0000","last_peered":"2024-09-10T10:01:57.413728+0000","last_clean":"2024-09-10T10:01:57.413728+0000","last_became_active":"2024-09-10T10:01:56.410597+0000","last_became_peered":"2024-09-10T10:01:56.410597+0000","last_unstale":"2024-09-10T10:01:57.413728+0000","last_undegraded":"2024-09-10T10:01:57.413728+0000","last_fullsized":"2024-09-10T10:01:57.413728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.934821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.429750+0000","last_change":"2024-09-10T10:01:57.423850+0000","last_active":"2024-09-10T10:01:57.429750+0000","last_peered":"2024-09-10T10:01:57.429750+0000","last_clean":"2024-09-10T10:01:57.429750+0000","last_became_active":"2024-09-10T10:01:57.423309+0000","last_became_peered":"2024-09-10T10:01:57.423309+0000","last_unstale":"2024-09-10T10:01:57.429750+0000","last_undegraded":"2024-09-10T10:01:57.429750+0000","last_fullsized":"2024-09-10T10:01:57.429750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.405952+0000","last_change":"2024-09-10T10:01:56.409831+0000","last_active":"2024-09-10T10:01:57.405952+0000","last_peered":"2024-09-10T10:01:57.405952+0000","last_clean":"2024-09-10T10:01:57.405952+0000","last_became_active":"2024-09-10T10:01:56.409512+0000","last_became_peered":"2024-09-10T10:01:56.409512+0000","last_unstale":"2024-09-10T10:01:57.405952+0000","last_undegraded":"2024-09-10T10:01:57.405952+0000","last_fullsized":"2024-09-10T10:01:57.405952+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:05.784097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.431774+0000","last_change":"2024-09-10T10:01:57.427760+0000","last_active":"2024-09-10T10:01:57.431774+0000","last_peered":"2024-09-10T10:01:57.431774+0000","last_clean":"2024-09-10T10:01:57.431774+0000","last_became_active":"2024-09-10T10:01:57.427511+0000","last_became_peered":"2024-09-10T10:01:57.427511+0000","last_unstale":"2024-09-10T10:01:57.431774+0000","last_undegraded":"2024-09-10T10:01:57.431774+0000","last_fullsized":"2024-09-10T10:01:57.431774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:52.470311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.397237+0000","last_change":"2024-09-10T10:01:56.402970+0000","last_active":"2024-09-10T10:01:57.397237+0000","last_peered":"2024-09-10T10:01:57.397237+0000","last_clean":"2024-09-10T10:01:57.397237+0000","last_became_active":"2024-09-10T10:01:56.402500+0000","last_became_peered":"2024-09-10T10:01:56.402500+0000","last_unstale":"2024-09-10T10:01:57.397237+0000","last_undegraded":"2024-09-10T10:01:57.397237+0000","last_fullsized":"2024-09-10T10:01:57.397237+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:26.744793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.415252+0000","last_change":"2024-09-10T10:01:57.409373+0000","last_active":"2024-09-10T10:01:57.415252+0000","last_peered":"2024-09-10T10:01:57.415252+0000","last_clean":"2024-09-10T10:01:57.415252+0000","last_became_active":"2024-09-10T10:01:57.409114+0000","last_became_peered":"2024-09-10T10:01:57.409114+0000","last_unstale":"2024-09-10T10:01:57.415252+0000","last_undegraded":"2024-09-10T10:01:57.415252+0000","last_fullsized":"2024-09-10T10:01:57.415252+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:02:53.254884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.400447+0000","last_change":"2024-09-10T10:01:56.411209+0000","last_active":"2024-09-10T10:01:57.400447+0000","last_peered":"2024-09-10T10:01:57.400447+0000","last_clean":"2024-09-10T10:01:57.400447+0000","last_became_active":"2024-09-10T10:01:56.410808+0000","last_became_peered":"2024-09-10T10:01:56.410808+0000","last_unstale":"2024-09-10T10:01:57.400447+0000","last_undegraded":"2024-09-10T10:01:57.400447+0000","last_fullsized":"2024-09-10T10:01:57.400447+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:39.392910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421360+0000","last_change":"2024-09-10T10:01:57.418730+0000","last_active":"2024-09-10T10:01:57.421360+0000","last_peered":"2024-09-10T10:01:57.421360+0000","last_clean":"2024-09-10T10:01:57.421360+0000","last_became_active":"2024-09-10T10:01:57.418395+0000","last_became_peered":"2024-09-10T10:01:57.418395+0000","last_unstale":"2024-09-10T10:01:57.421360+0000","last_undegraded":"2024-09-10T10:01:57.421360+0000","last_fullsized":"2024-09-10T10:01:57.421360+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:58.162428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.391574+0000","last_change":"2024-09-10T10:01:56.409314+0000","last_active":"2024-09-10T10:01:57.391574+0000","last_peered":"2024-09-10T10:01:57.391574+0000","last_clean":"2024-09-10T10:01:57.391574+0000","last_became_active":"2024-09-10T10:01:56.409077+0000","last_became_peered":"2024-09-10T10:01:56.409077+0000","last_unstale":"2024-09-10T10:01:57.391574+0000","last_undegraded":"2024-09-10T10:01:57.391574+0000","last_fullsized":"2024-09-10T10:01:57.391574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.220387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.416721+0000","last_change":"2024-09-10T10:01:57.407297+0000","last_active":"2024-09-10T10:01:57.416721+0000","last_peered":"2024-09-10T10:01:57.416721+0000","last_clean":"2024-09-10T10:01:57.416721+0000","last_became_active":"2024-09-10T10:01:57.407030+0000","last_became_peered":"2024-09-10T10:01:57.407030+0000","last_unstale":"2024-09-10T10:01:57.416721+0000","last_undegraded":"2024-09-10T10:01:57.416721+0000","last_fullsized":"2024-09-10T10:01:57.416721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:33.572565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.398463+0000","last_change":"2024-09-10T10:01:56.411518+0000","last_active":"2024-09-10T10:01:57.398463+0000","last_peered":"2024-09-10T10:01:57.398463+0000","last_clean":"2024-09-10T10:01:57.398463+0000","last_became_active":"2024-09-10T10:01:56.410515+0000","last_became_peered":"2024-09-10T10:01:56.410515+0000","last_unstale":"2024-09-10T10:01:57.398463+0000","last_undegraded":"2024-09-10T10:01:57.398463+0000","last_fullsized":"2024-09-10T10:01:57.398463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.421441+0000","last_change":"2024-09-10T10:01:57.408569+0000","last_active":"2024-09-10T10:01:57.421441+0000","last_peered":"2024-09-10T10:01:57.421441+0000","last_clean":"2024-09-10T10:01:57.421441+0000","last_became_active":"2024-09-10T10:01:57.408302+0000","last_became_peered":"2024-09-10T10:01:57.408302+0000","last_unstale":"2024-09-10T10:01:57.421441+0000","last_undegraded":"2024-09-10T10:01:57.421441+0000","last_fullsized":"2024-09-10T10:01:57.421441+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:39.030596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.394334+0000","last_change":"2024-09-10T10:01:56.407472+0000","last_active":"2024-09-10T10:01:57.394334+0000","last_peered":"2024-09-10T10:01:57.394334+0000","last_clean":"2024-09-10T10:01:57.394334+0000","last_became_active":"2024-09-10T10:01:56.407112+0000","last_became_peered":"2024-09-10T10:01:56.407112+0000","last_unstale":"2024-09-10T10:01:57.394334+0000","last_undegraded":"2024-09-10T10:01:57.394334+0000","last_fullsized":"2024-09-10T10:01:57.394334+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:36.230334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:01:56.394297+0000","last_change":"2024-09-10T10:01:56.394297+0000","last_active":"2024-09-10T10:01:56.377542+0000","last_peered":"2024-09-10T10:01:56.377542+0000","last_clean":"2024-09-10T10:01:56.377542+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:01:56.394297+0000","last_undegraded":"2024-09-10T10:01:56.394297+0000","last_fullsized":"2024-09-10T10:01:56.394297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.427116+0000","last_change":"2024-09-10T10:01:56.408027+0000","last_active":"2024-09-10T10:01:57.427116+0000","last_peered":"2024-09-10T10:01:57.427116+0000","last_clean":"2024-09-10T10:01:57.427116+0000","last_became_active":"2024-09-10T10:01:56.407749+0000","last_became_peered":"2024-09-10T10:01:56.407749+0000","last_unstale":"2024-09-10T10:01:57.427116+0000","last_undegraded":"2024-09-10T10:01:57.427116+0000","last_fullsized":"2024-09-10T10:01:57.427116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:42.979120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":252,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T10:01:57.403667+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:01:57.403667+0000","last_peered":"2024-09-10T10:01:57.403667+0000","last_clean":"2024-09-10T10:01:57.403667+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:01:57.403667+0000","last_undegraded":"2024-09-10T10:01:57.403667+0000","last_fullsized":"2024-09-10T10:01:57.403667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":930573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":341201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":929957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":806912,"data_stored":332814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:06.015 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:02:06.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1e1000640 1 Processor -- start 2024-09-10T10:02:06.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1e1000640 1 -- start start 2024-09-10T10:02:06.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1e1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc06e7a0 0x7fc1dc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1e1000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1dc072150 con 0x7fc1dc0cd510 2024-09-10T10:02:06.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1e1000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1dc06d2d0 con 0x7fc1dc0cfe30 2024-09-10T10:02:06.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1e1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc1dc15d9a0 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1d9400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc1dc0cd510 0x7fc1dc06e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36888/0 (socket says 172.21.15.110:36888) 2024-09-10T10:02:06.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1d9400640 1 -- 172.21.15.110:0/752400932 learned_addr learned my addr 172.21.15.110:0/752400932 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:06.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1da800640 1 --2- 172.21.15.110:0/752400932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc06e7a0 0x7fc1dc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1d8a00640 1 -- 172.21.15.110:0/752400932 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 75502953 0 0) 0x7fc1dc072150 con 0x7fc1dc0cd510 2024-09-10T10:02:06.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1d8a00640 1 -- 172.21.15.110:0/752400932 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1c0003530 con 0x7fc1dc0cd510 2024-09-10T10:02:06.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1d8a00640 1 -- 172.21.15.110:0/752400932 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553170596 0 0) 0x7fc1dc06d2d0 con 0x7fc1dc0cfe30 2024-09-10T10:02:06.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1d8a00640 1 -- 172.21.15.110:0/752400932 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1dc072150 con 0x7fc1dc0cfe30 2024-09-10T10:02:06.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1da800640 1 -- 172.21.15.110:0/752400932 >> v1:172.21.15.145:6789/0 conn(0x7fc1dc0cfe30 legacy=0x7fc1dc071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1da800640 1 -- 172.21.15.110:0/752400932 >> v1:172.21.15.110:6789/0 conn(0x7fc1dc0cd510 legacy=0x7fc1dc06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.164+0000 7fc1da800640 1 -- 172.21.15.110:0/752400932 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc06d5b0 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1da800640 1 --2- 172.21.15.110:0/752400932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc06e7a0 0x7fc1dc0cd9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009dc0 tx=0x7fc1d401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd3eb8b5ccbcd34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1d8a00640 1 -- 172.21.15.110:0/752400932 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d40b2070 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1d8a00640 1 -- 172.21.15.110:0/752400932 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d40a58d0 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1d8a00640 1 -- 172.21.15.110:0/752400932 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d40a5c80 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1e1000640 1 -- 172.21.15.110:0/752400932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc06e7a0 msgr2=0x7fc1dc0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1e1000640 1 --2- 172.21.15.110:0/752400932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc06e7a0 0x7fc1dc0cd9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009dc0 tx=0x7fc1d401c900 comp rx=0 tx=0).stop 2024-09-10T10:02:06.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1e1000640 1 -- 172.21.15.110:0/752400932 shutdown_connections 2024-09-10T10:02:06.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1e1000640 1 --2- 172.21.15.110:0/752400932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc06e7a0 0x7fc1dc0cd9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1e1000640 1 -- 172.21.15.110:0/752400932 >> 172.21.15.110:0/752400932 conn(0x7fc1dc06bc50 msgr2=0x7fc1dc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:06.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1e1000640 1 -- 172.21.15.110:0/752400932 shutdown_connections 2024-09-10T10:02:06.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.168+0000 7fc1e1000640 1 -- 172.21.15.110:0/752400932 wait complete. 2024-09-10T10:02:06.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 Processor -- start 2024-09-10T10:02:06.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 -- start start 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1dc06e7a0 0x7fc1dc0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1dc0cd510 0x7fc1dc0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc0cfe30 0x7fc1dc0b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc16a190 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc15d700 con 0x7fc1dc0cd510 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc1dc16a310 con 0x7fc1dc0cfe30 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1dc06e7a0 0x7fc1dc0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1dc06e7a0 0x7fc1dc0b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44124/0 (socket says 172.21.15.110:44124) 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 -- 172.21.15.110:0/2956843016 learned_addr learned my addr 172.21.15.110:0/2956843016 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:06.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9400640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1dc0cd510 0x7fc1dc0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1da800640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc0cfe30 0x7fc1dc0b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 -- 172.21.15.110:0/2956843016 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc0cfe30 msgr2=0x7fc1dc0b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc0cfe30 0x7fc1dc0b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 -- 172.21.15.110:0/2956843016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1dc0cd510 msgr2=0x7fc1dc0b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1dc0cd510 0x7fc1dc0b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 -- 172.21.15.110:0/2956843016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc159f20 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d9e00640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1dc06e7a0 0x7fc1dc0b35d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00c460 tx=0x7fc1cc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d2000640 1 -- 172.21.15.110:0/2956843016 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc017070 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d2000640 1 -- 172.21.15.110:0/2956843016 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc0052f0 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1d2000640 1 -- 172.21.15.110:0/2956843016 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc003840 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc0b0170 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc0b0730 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.172+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc06e090 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.176+0000 7fc1d2000640 1 -- 172.21.15.110:0/2956843016 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fc1cc01a070 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.176+0000 7fc1d2000640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1bc067c20 0x7fc1bc06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.176+0000 7fc1d2000640 1 -- 172.21.15.110:0/2956843016 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7fc1cc0854b0 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.176+0000 7fc1d9400640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1bc067c20 0x7fc1bc06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.176+0000 7fc1d9400640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1bc067c20 0x7fc1bc06a0e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15c9a0 tx=0x7fc1c8007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.180+0000 7fc1d2000640 1 -- 172.21.15.110:0/2956843016 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc1cc0555c0 con 0x7fc1dc06e7a0 2024-09-10T10:02:06.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.332+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1dc15a200 con 0x7fc1bc067c20 2024-09-10T10:02:06.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.340+0000 7fc1d2000640 1 -- 172.21.15.110:0/2956843016 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252061 (secure 0 0 0) 0x7fc1dc15a200 con 0x7fc1bc067c20 2024-09-10T10:02:06.346 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:06.347 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:02:06.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1bc067c20 msgr2=0x7fc1bc06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1bc067c20 0x7fc1bc06a0e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15c9a0 tx=0x7fc1c8007d40 comp rx=0 tx=0).stop 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1dc06e7a0 msgr2=0x7fc1dc0b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1dc06e7a0 0x7fc1dc0b35d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00c460 tx=0x7fc1cc00c920 comp rx=0 tx=0).stop 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 shutdown_connections 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fc1bc067c20 0x7fc1bc06a0e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc1dc0cfe30 0x7fc1dc0b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1dc0cd510 0x7fc1dc0b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 --2- 172.21.15.110:0/2956843016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1dc06e7a0 0x7fc1dc0b35d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 >> 172.21.15.110:0/2956843016 conn(0x7fc1dc06bc50 msgr2=0x7fc1dc15f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 shutdown_connections 2024-09-10T10:02:06.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.344+0000 7fc1e1000640 1 -- 172.21.15.110:0/2956843016 wait complete. 2024-09-10T10:02:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-10T10:02:05.448249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253076,"kb_used_data":7720,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721644,"statfs":{"total":773094113280,"available":772834963456,"internally_reserved":0,"allocated":7905280,"data_stored":3952668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":251055660},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.048725"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404545+0000","last_change":"2024-09-10T10:01:57.425627+0000","last_active":"2024-09-10T10:01:58.404545+0000","last_peered":"2024-09-10T10:01:58.404545+0000","last_clean":"2024-09-10T10:01:58.404545+0000","last_became_active":"2024-09-10T10:01:57.425083+0000","last_became_peered":"2024-09-10T10:01:57.425083+0000","last_unstale":"2024-09-10T10:01:58.404545+0000","last_undegraded":"2024-09-10T10:01:58.404545+0000","last_fullsized":"2024-09-10T10:01:58.404545+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:25.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404163+0000","last_change":"2024-09-10T10:01:57.410364+0000","last_active":"2024-09-10T10:01:58.404163+0000","last_peered":"2024-09-10T10:01:58.404163+0000","last_clean":"2024-09-10T10:01:58.404163+0000","last_became_active":"2024-09-10T10:01:57.410086+0000","last_became_peered":"2024-09-10T10:01:57.410086+0000","last_unstale":"2024-09-10T10:01:58.404163+0000","last_undegraded":"2024-09-10T10:01:58.404163+0000","last_fullsized":"2024-09-10T10:01:58.404163+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:35.517149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199275+0000","last_change":"2024-09-10T10:01:57.421015+0000","last_active":"2024-09-10T10:02:00.199275+0000","last_peered":"2024-09-10T10:02:00.199275+0000","last_clean":"2024-09-10T10:02:00.199275+0000","last_became_active":"2024-09-10T10:01:57.420717+0000","last_became_peered":"2024-09-10T10:01:57.420717+0000","last_unstale":"2024-09-10T10:02:00.199275+0000","last_undegraded":"2024-09-10T10:02:00.199275+0000","last_fullsized":"2024-09-10T10:02:00.199275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:05:25.035793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.743153+0000","last_change":"2024-09-10T10:01:57.419086+0000","last_active":"2024-09-10T10:01:59.743153+0000","last_peered":"2024-09-10T10:01:59.743153+0000","last_clean":"2024-09-10T10:01:59.743153+0000","last_became_active":"2024-09-10T10:01:57.418640+0000","last_became_peered":"2024-09-10T10:01:57.418640+0000","last_unstale":"2024-09-10T10:01:59.743153+0000","last_undegraded":"2024-09-10T10:01:59.743153+0000","last_fullsized":"2024-09-10T10:01:59.743153+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:57.939286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401689+0000","last_change":"2024-09-10T10:01:57.412199+0000","last_active":"2024-09-10T10:01:58.401689+0000","last_peered":"2024-09-10T10:01:58.401689+0000","last_clean":"2024-09-10T10:01:58.401689+0000","last_became_active":"2024-09-10T10:01:57.411926+0000","last_became_peered":"2024-09-10T10:01:57.411926+0000","last_unstale":"2024-09-10T10:01:58.401689+0000","last_undegraded":"2024-09-10T10:01:58.401689+0000","last_fullsized":"2024-09-10T10:01:58.401689+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:55.225918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403401+0000","last_change":"2024-09-10T10:01:57.411801+0000","last_active":"2024-09-10T10:01:58.403401+0000","last_peered":"2024-09-10T10:01:58.403401+0000","last_clean":"2024-09-10T10:01:58.403401+0000","last_became_active":"2024-09-10T10:01:57.411506+0000","last_became_peered":"2024-09-10T10:01:57.411506+0000","last_unstale":"2024-09-10T10:01:58.403401+0000","last_undegraded":"2024-09-10T10:01:58.403401+0000","last_fullsized":"2024-09-10T10:01:58.403401+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:39.937371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198780+0000","last_change":"2024-09-10T10:01:57.427391+0000","last_active":"2024-09-10T10:02:00.198780+0000","last_peered":"2024-09-10T10:02:00.198780+0000","last_clean":"2024-09-10T10:02:00.198780+0000","last_became_active":"2024-09-10T10:01:57.427095+0000","last_became_peered":"2024-09-10T10:01:57.427095+0000","last_unstale":"2024-09-10T10:02:00.198780+0000","last_undegraded":"2024-09-10T10:02:00.198780+0000","last_fullsized":"2024-09-10T10:02:00.198780+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.614990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403728+0000","last_change":"2024-09-10T10:01:57.405094+0000","last_active":"2024-09-10T10:01:58.403728+0000","last_peered":"2024-09-10T10:01:58.403728+0000","last_clean":"2024-09-10T10:01:58.403728+0000","last_became_active":"2024-09-10T10:01:57.404803+0000","last_became_peered":"2024-09-10T10:01:57.404803+0000","last_unstale":"2024-09-10T10:01:58.403728+0000","last_undegraded":"2024-09-10T10:01:58.403728+0000","last_fullsized":"2024-09-10T10:01:58.403728+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.402033+0000","last_change":"2024-09-10T10:01:57.414814+0000","last_active":"2024-09-10T10:01:58.402033+0000","last_peered":"2024-09-10T10:01:58.402033+0000","last_clean":"2024-09-10T10:01:58.402033+0000","last_became_active":"2024-09-10T10:01:57.414440+0000","last_became_peered":"2024-09-10T10:01:57.414440+0000","last_unstale":"2024-09-10T10:01:58.402033+0000","last_undegraded":"2024-09-10T10:01:58.402033+0000","last_fullsized":"2024-09-10T10:01:58.402033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:36.723786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.200119+0000","last_change":"2024-09-10T10:01:57.411661+0000","last_active":"2024-09-10T10:02:00.200119+0000","last_peered":"2024-09-10T10:02:00.200119+0000","last_clean":"2024-09-10T10:02:00.200119+0000","last_became_active":"2024-09-10T10:01:57.410577+0000","last_became_peered":"2024-09-10T10:01:57.410577+0000","last_unstale":"2024-09-10T10:02:00.200119+0000","last_undegraded":"2024-09-10T10:02:00.200119+0000","last_fullsized":"2024-09-10T10:02:00.200119+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:35.517149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404591+0000","last_change":"2024-09-10T10:01:57.420979+0000","last_active":"2024-09-10T10:01:58.404591+0000","last_peered":"2024-09-10T10:01:58.404591+0000","last_clean":"2024-09-10T10:01:58.404591+0000","last_became_active":"2024-09-10T10:01:57.420695+0000","last_became_peered":"2024-09-10T10:01:57.420695+0000","last_unstale":"2024-09-10T10:01:58.404591+0000","last_undegraded":"2024-09-10T10:01:58.404591+0000","last_fullsized":"2024-09-10T10:01:58.404591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:07.585992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.743156+0000","last_change":"2024-09-10T10:01:57.403883+0000","last_active":"2024-09-10T10:01:59.743156+0000","last_peered":"2024-09-10T10:01:59.743156+0000","last_clean":"2024-09-10T10:01:59.743156+0000","last_became_active":"2024-09-10T10:01:57.403600+0000","last_became_peered":"2024-09-10T10:01:57.403600+0000","last_unstale":"2024-09-10T10:01:59.743156+0000","last_undegraded":"2024-09-10T10:01:59.743156+0000","last_fullsized":"2024-09-10T10:01:59.743156+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:40.859011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198465+0000","last_change":"2024-09-10T10:01:57.409326+0000","last_active":"2024-09-10T10:02:00.198465+0000","last_peered":"2024-09-10T10:02:00.198465+0000","last_clean":"2024-09-10T10:02:00.198465+0000","last_became_active":"2024-09-10T10:01:57.409053+0000","last_became_peered":"2024-09-10T10:01:57.409053+0000","last_unstale":"2024-09-10T10:02:00.198465+0000","last_undegraded":"2024-09-10T10:02:00.198465+0000","last_fullsized":"2024-09-10T10:02:00.198465+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:07.585992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401826+0000","last_change":"2024-09-10T10:01:57.421822+0000","last_active":"2024-09-10T10:01:58.401826+0000","last_peered":"2024-09-10T10:01:58.401826+0000","last_clean":"2024-09-10T10:01:58.401826+0000","last_became_active":"2024-09-10T10:01:57.421559+0000","last_became_peered":"2024-09-10T10:01:57.421559+0000","last_unstale":"2024-09-10T10:01:58.401826+0000","last_undegraded":"2024-09-10T10:01:58.401826+0000","last_fullsized":"2024-09-10T10:01:58.401826+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:33.798474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198119+0000","last_change":"2024-09-10T10:01:57.410832+0000","last_active":"2024-09-10T10:02:00.198119+0000","last_peered":"2024-09-10T10:02:00.198119+0000","last_clean":"2024-09-10T10:02:00.198119+0000","last_became_active":"2024-09-10T10:01:57.410575+0000","last_became_peered":"2024-09-10T10:01:57.410575+0000","last_unstale":"2024-09-10T10:02:00.198119+0000","last_undegraded":"2024-09-10T10:02:00.198119+0000","last_fullsized":"2024-09-10T10:02:00.198119+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:54.032055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199295+0000","last_change":"2024-09-10T10:01:57.419056+0000","last_active":"2024-09-10T10:02:00.199295+0000","last_peered":"2024-09-10T10:02:00.199295+0000","last_clean":"2024-09-10T10:02:00.199295+0000","last_became_active":"2024-09-10T10:01:57.418704+0000","last_became_peered":"2024-09-10T10:01:57.418704+0000","last_unstale":"2024-09-10T10:02:00.199295+0000","last_undegraded":"2024-09-10T10:02:00.199295+0000","last_fullsized":"2024-09-10T10:02:00.199295+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:24.580129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.402101+0000","last_change":"2024-09-10T10:01:57.414713+0000","last_active":"2024-09-10T10:01:58.402101+0000","last_peered":"2024-09-10T10:01:58.402101+0000","last_clean":"2024-09-10T10:01:58.402101+0000","last_became_active":"2024-09-10T10:01:57.414227+0000","last_became_peered":"2024-09-10T10:01:57.414227+0000","last_unstale":"2024-09-10T10:01:58.402101+0000","last_undegraded":"2024-09-10T10:01:58.402101+0000","last_fullsized":"2024-09-10T10:01:58.402101+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:46.699359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.743466+0000","last_change":"2024-09-10T10:01:57.407315+0000","last_active":"2024-09-10T10:01:59.743466+0000","last_peered":"2024-09-10T10:01:59.743466+0000","last_clean":"2024-09-10T10:01:59.743466+0000","last_became_active":"2024-09-10T10:01:57.406568+0000","last_became_peered":"2024-09-10T10:01:57.406568+0000","last_unstale":"2024-09-10T10:01:59.743466+0000","last_undegraded":"2024-09-10T10:01:59.743466+0000","last_fullsized":"2024-09-10T10:01:59.743466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:52.439252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404683+0000","last_change":"2024-09-10T10:01:57.405757+0000","last_active":"2024-09-10T10:01:58.404683+0000","last_peered":"2024-09-10T10:01:58.404683+0000","last_clean":"2024-09-10T10:01:58.404683+0000","last_became_active":"2024-09-10T10:01:57.404501+0000","last_became_peered":"2024-09-10T10:01:57.404501+0000","last_unstale":"2024-09-10T10:01:58.404683+0000","last_undegraded":"2024-09-10T10:01:58.404683+0000","last_fullsized":"2024-09-10T10:01:58.404683+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:14.777587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198268+0000","last_change":"2024-09-10T10:01:57.423165+0000","last_active":"2024-09-10T10:02:00.198268+0000","last_peered":"2024-09-10T10:02:00.198268+0000","last_clean":"2024-09-10T10:02:00.198268+0000","last_became_active":"2024-09-10T10:01:57.422527+0000","last_became_peered":"2024-09-10T10:01:57.422527+0000","last_unstale":"2024-09-10T10:02:00.198268+0000","last_undegraded":"2024-09-10T10:02:00.198268+0000","last_fullsized":"2024-09-10T10:02:00.198268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:55.992911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199575+0000","last_change":"2024-09-10T10:01:57.409584+0000","last_active":"2024-09-10T10:02:00.199575+0000","last_peered":"2024-09-10T10:02:00.199575+0000","last_clean":"2024-09-10T10:02:00.199575+0000","last_became_active":"2024-09-10T10:01:57.409327+0000","last_became_peered":"2024-09-10T10:01:57.409327+0000","last_unstale":"2024-09-10T10:02:00.199575+0000","last_undegraded":"2024-09-10T10:02:00.199575+0000","last_fullsized":"2024-09-10T10:02:00.199575+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.508652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199556+0000","last_change":"2024-09-10T10:01:57.416054+0000","last_active":"2024-09-10T10:02:00.199556+0000","last_peered":"2024-09-10T10:02:00.199556+0000","last_clean":"2024-09-10T10:02:00.199556+0000","last_became_active":"2024-09-10T10:01:57.415813+0000","last_became_peered":"2024-09-10T10:01:57.415813+0000","last_unstale":"2024-09-10T10:02:00.199556+0000","last_undegraded":"2024-09-10T10:02:00.199556+0000","last_fullsized":"2024-09-10T10:02:00.199556+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:39.937371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404312+0000","last_change":"2024-09-10T10:01:57.430689+0000","last_active":"2024-09-10T10:01:58.404312+0000","last_peered":"2024-09-10T10:01:58.404312+0000","last_clean":"2024-09-10T10:01:58.404312+0000","last_became_active":"2024-09-10T10:01:57.430349+0000","last_became_peered":"2024-09-10T10:01:57.430349+0000","last_unstale":"2024-09-10T10:01:58.404312+0000","last_undegraded":"2024-09-10T10:01:58.404312+0000","last_fullsized":"2024-09-10T10:01:58.404312+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:54.032055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404960+0000","last_change":"2024-09-10T10:01:57.424818+0000","last_active":"2024-09-10T10:01:58.404960+0000","last_peered":"2024-09-10T10:01:58.404960+0000","last_clean":"2024-09-10T10:01:58.404960+0000","last_became_active":"2024-09-10T10:01:57.424469+0000","last_became_peered":"2024-09-10T10:01:57.424469+0000","last_unstale":"2024-09-10T10:01:58.404960+0000","last_undegraded":"2024-09-10T10:01:58.404960+0000","last_fullsized":"2024-09-10T10:01:58.404960+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:42.621121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.200118+0000","last_change":"2024-09-10T10:01:57.412902+0000","last_active":"2024-09-10T10:02:00.200118+0000","last_peered":"2024-09-10T10:02:00.200118+0000","last_clean":"2024-09-10T10:02:00.200118+0000","last_became_active":"2024-09-10T10:01:57.412071+0000","last_became_peered":"2024-09-10T10:01:57.412071+0000","last_unstale":"2024-09-10T10:02:00.200118+0000","last_undegraded":"2024-09-10T10:02:00.200118+0000","last_fullsized":"2024-09-10T10:02:00.200118+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199102+0000","last_change":"2024-09-10T10:01:57.428212+0000","last_active":"2024-09-10T10:02:00.199102+0000","last_peered":"2024-09-10T10:02:00.199102+0000","last_clean":"2024-09-10T10:02:00.199102+0000","last_became_active":"2024-09-10T10:01:57.427898+0000","last_became_peered":"2024-09-10T10:01:57.427898+0000","last_unstale":"2024-09-10T10:02:00.199102+0000","last_undegraded":"2024-09-10T10:02:00.199102+0000","last_fullsized":"2024-09-10T10:02:00.199102+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:52.470311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199189+0000","last_change":"2024-09-10T10:01:57.418092+0000","last_active":"2024-09-10T10:02:00.199189+0000","last_peered":"2024-09-10T10:02:00.199189+0000","last_clean":"2024-09-10T10:02:00.199189+0000","last_became_active":"2024-09-10T10:01:57.415065+0000","last_became_peered":"2024-09-10T10:01:57.415065+0000","last_unstale":"2024-09-10T10:02:00.199189+0000","last_undegraded":"2024-09-10T10:02:00.199189+0000","last_fullsized":"2024-09-10T10:02:00.199189+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:09.564706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199787+0000","last_change":"2024-09-10T10:01:57.411461+0000","last_active":"2024-09-10T10:02:00.199787+0000","last_peered":"2024-09-10T10:02:00.199787+0000","last_clean":"2024-09-10T10:02:00.199787+0000","last_became_active":"2024-09-10T10:01:57.410729+0000","last_became_peered":"2024-09-10T10:01:57.410729+0000","last_unstale":"2024-09-10T10:02:00.199787+0000","last_undegraded":"2024-09-10T10:02:00.199787+0000","last_fullsized":"2024-09-10T10:02:00.199787+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:49.225109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199032+0000","last_change":"2024-09-10T10:01:57.427510+0000","last_active":"2024-09-10T10:02:00.199032+0000","last_peered":"2024-09-10T10:02:00.199032+0000","last_clean":"2024-09-10T10:02:00.199032+0000","last_became_active":"2024-09-10T10:01:57.427174+0000","last_became_peered":"2024-09-10T10:01:57.427174+0000","last_unstale":"2024-09-10T10:02:00.199032+0000","last_undegraded":"2024-09-10T10:02:00.199032+0000","last_fullsized":"2024-09-10T10:02:00.199032+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:22.040841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198856+0000","last_change":"2024-09-10T10:01:57.428097+0000","last_active":"2024-09-10T10:02:00.198856+0000","last_peered":"2024-09-10T10:02:00.198856+0000","last_clean":"2024-09-10T10:02:00.198856+0000","last_became_active":"2024-09-10T10:01:57.427838+0000","last_became_peered":"2024-09-10T10:01:57.427838+0000","last_unstale":"2024-09-10T10:02:00.198856+0000","last_undegraded":"2024-09-10T10:02:00.198856+0000","last_fullsized":"2024-09-10T10:02:00.198856+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:53.600390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198965+0000","last_change":"2024-09-10T10:01:57.416701+0000","last_active":"2024-09-10T10:02:00.198965+0000","last_peered":"2024-09-10T10:02:00.198965+0000","last_clean":"2024-09-10T10:02:00.198965+0000","last_became_active":"2024-09-10T10:01:57.416416+0000","last_became_peered":"2024-09-10T10:01:57.416416+0000","last_unstale":"2024-09-10T10:02:00.198965+0000","last_undegraded":"2024-09-10T10:02:00.198965+0000","last_fullsized":"2024-09-10T10:02:00.198965+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:25.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403518+0000","last_change":"2024-09-10T10:01:57.412058+0000","last_active":"2024-09-10T10:01:58.403518+0000","last_peered":"2024-09-10T10:01:58.403518+0000","last_clean":"2024-09-10T10:01:58.403518+0000","last_became_active":"2024-09-10T10:01:57.411787+0000","last_became_peered":"2024-09-10T10:01:57.411787+0000","last_unstale":"2024-09-10T10:01:58.403518+0000","last_undegraded":"2024-09-10T10:01:58.403518+0000","last_fullsized":"2024-09-10T10:01:58.403518+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.508652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404804+0000","last_change":"2024-09-10T10:01:57.426610+0000","last_active":"2024-09-10T10:01:58.404804+0000","last_peered":"2024-09-10T10:01:58.404804+0000","last_clean":"2024-09-10T10:01:58.404804+0000","last_became_active":"2024-09-10T10:01:57.423435+0000","last_became_peered":"2024-09-10T10:01:57.423435+0000","last_unstale":"2024-09-10T10:01:58.404804+0000","last_undegraded":"2024-09-10T10:01:58.404804+0000","last_fullsized":"2024-09-10T10:01:58.404804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:02.637150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404815+0000","last_change":"2024-09-10T10:01:56.414321+0000","last_active":"2024-09-10T10:01:58.404815+0000","last_peered":"2024-09-10T10:01:58.404815+0000","last_clean":"2024-09-10T10:01:58.404815+0000","last_became_active":"2024-09-10T10:01:56.414030+0000","last_became_peered":"2024-09-10T10:01:56.414030+0000","last_unstale":"2024-09-10T10:01:58.404815+0000","last_undegraded":"2024-09-10T10:01:58.404815+0000","last_fullsized":"2024-09-10T10:01:58.404815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.402185+0000","last_change":"2024-09-10T10:01:57.414555+0000","last_active":"2024-09-10T10:01:58.402185+0000","last_peered":"2024-09-10T10:01:58.402185+0000","last_clean":"2024-09-10T10:01:58.402185+0000","last_became_active":"2024-09-10T10:01:57.413819+0000","last_became_peered":"2024-09-10T10:01:57.413819+0000","last_unstale":"2024-09-10T10:01:58.402185+0000","last_undegraded":"2024-09-10T10:01:58.402185+0000","last_fullsized":"2024-09-10T10:01:58.402185+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:05.665279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198397+0000","last_change":"2024-09-10T10:01:56.408386+0000","last_active":"2024-09-10T10:02:00.198397+0000","last_peered":"2024-09-10T10:02:00.198397+0000","last_clean":"2024-09-10T10:02:00.198397+0000","last_became_active":"2024-09-10T10:01:56.408127+0000","last_became_peered":"2024-09-10T10:01:56.408127+0000","last_unstale":"2024-09-10T10:02:00.198397+0000","last_undegraded":"2024-09-10T10:02:00.198397+0000","last_fullsized":"2024-09-10T10:02:00.198397+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:59.865257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404287+0000","last_change":"2024-09-10T10:01:57.417292+0000","last_active":"2024-09-10T10:01:58.404287+0000","last_peered":"2024-09-10T10:01:58.404287+0000","last_clean":"2024-09-10T10:01:58.404287+0000","last_became_active":"2024-09-10T10:01:57.416917+0000","last_became_peered":"2024-09-10T10:01:57.416917+0000","last_unstale":"2024-09-10T10:01:58.404287+0000","last_undegraded":"2024-09-10T10:01:58.404287+0000","last_fullsized":"2024-09-10T10:01:58.404287+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:27.747342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199976+0000","last_change":"2024-09-10T10:01:56.409023+0000","last_active":"2024-09-10T10:02:00.199976+0000","last_peered":"2024-09-10T10:02:00.199976+0000","last_clean":"2024-09-10T10:02:00.199976+0000","last_became_active":"2024-09-10T10:01:56.408748+0000","last_became_peered":"2024-09-10T10:01:56.408748+0000","last_unstale":"2024-09-10T10:02:00.199976+0000","last_undegraded":"2024-09-10T10:02:00.199976+0000","last_fullsized":"2024-09-10T10:02:00.199976+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:50.426131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404591+0000","last_change":"2024-09-10T10:01:57.408727+0000","last_active":"2024-09-10T10:01:58.404591+0000","last_peered":"2024-09-10T10:01:58.404591+0000","last_clean":"2024-09-10T10:01:58.404591+0000","last_became_active":"2024-09-10T10:01:57.408448+0000","last_became_peered":"2024-09-10T10:01:57.408448+0000","last_unstale":"2024-09-10T10:01:58.404591+0000","last_undegraded":"2024-09-10T10:01:58.404591+0000","last_fullsized":"2024-09-10T10:01:58.404591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:12.099313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404537+0000","last_change":"2024-09-10T10:01:56.409387+0000","last_active":"2024-09-10T10:01:58.404537+0000","last_peered":"2024-09-10T10:01:58.404537+0000","last_clean":"2024-09-10T10:01:58.404537+0000","last_became_active":"2024-09-10T10:01:56.407044+0000","last_became_peered":"2024-09-10T10:01:56.407044+0000","last_unstale":"2024-09-10T10:01:58.404537+0000","last_undegraded":"2024-09-10T10:01:58.404537+0000","last_fullsized":"2024-09-10T10:01:58.404537+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199062+0000","last_change":"2024-09-10T10:01:57.420630+0000","last_active":"2024-09-10T10:02:00.199062+0000","last_peered":"2024-09-10T10:02:00.199062+0000","last_clean":"2024-09-10T10:02:00.199062+0000","last_became_active":"2024-09-10T10:01:57.416989+0000","last_became_peered":"2024-09-10T10:01:57.416989+0000","last_unstale":"2024-09-10T10:02:00.199062+0000","last_undegraded":"2024-09-10T10:02:00.199062+0000","last_fullsized":"2024-09-10T10:02:00.199062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:29.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199136+0000","last_change":"2024-09-10T10:01:56.404458+0000","last_active":"2024-09-10T10:02:00.199136+0000","last_peered":"2024-09-10T10:02:00.199136+0000","last_clean":"2024-09-10T10:02:00.199136+0000","last_became_active":"2024-09-10T10:01:56.404205+0000","last_became_peered":"2024-09-10T10:01:56.404205+0000","last_unstale":"2024-09-10T10:02:00.199136+0000","last_undegraded":"2024-09-10T10:02:00.199136+0000","last_fullsized":"2024-09-10T10:02:00.199136+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:15.210244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199613+0000","last_change":"2024-09-10T10:01:57.412841+0000","last_active":"2024-09-10T10:02:00.199613+0000","last_peered":"2024-09-10T10:02:00.199613+0000","last_clean":"2024-09-10T10:02:00.199613+0000","last_became_active":"2024-09-10T10:01:57.412551+0000","last_became_peered":"2024-09-10T10:01:57.412551+0000","last_unstale":"2024-09-10T10:02:00.199613+0000","last_undegraded":"2024-09-10T10:02:00.199613+0000","last_fullsized":"2024-09-10T10:02:00.199613+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:14.901947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403503+0000","last_change":"2024-09-10T10:01:56.398879+0000","last_active":"2024-09-10T10:01:58.403503+0000","last_peered":"2024-09-10T10:01:58.403503+0000","last_clean":"2024-09-10T10:01:58.403503+0000","last_became_active":"2024-09-10T10:01:56.398576+0000","last_became_peered":"2024-09-10T10:01:56.398576+0000","last_unstale":"2024-09-10T10:01:58.403503+0000","last_undegraded":"2024-09-10T10:01:58.403503+0000","last_fullsized":"2024-09-10T10:01:58.403503+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:01.154272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401436+0000","last_change":"2024-09-10T10:01:57.412438+0000","last_active":"2024-09-10T10:01:58.401436+0000","last_peered":"2024-09-10T10:01:58.401436+0000","last_clean":"2024-09-10T10:01:58.401436+0000","last_became_active":"2024-09-10T10:01:57.412174+0000","last_became_peered":"2024-09-10T10:01:57.412174+0000","last_unstale":"2024-09-10T10:01:58.401436+0000","last_undegraded":"2024-09-10T10:01:58.401436+0000","last_fullsized":"2024-09-10T10:01:58.401436+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:46.868501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.743024+0000","last_change":"2024-09-10T10:01:56.406654+0000","last_active":"2024-09-10T10:01:59.743024+0000","last_peered":"2024-09-10T10:01:59.743024+0000","last_clean":"2024-09-10T10:01:59.743024+0000","last_became_active":"2024-09-10T10:01:56.406375+0000","last_became_peered":"2024-09-10T10:01:56.406375+0000","last_unstale":"2024-09-10T10:01:59.743024+0000","last_undegraded":"2024-09-10T10:01:59.743024+0000","last_fullsized":"2024-09-10T10:01:59.743024+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:07.017690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198179+0000","last_change":"2024-09-10T10:01:57.408804+0000","last_active":"2024-09-10T10:02:00.198179+0000","last_peered":"2024-09-10T10:02:00.198179+0000","last_clean":"2024-09-10T10:02:00.198179+0000","last_became_active":"2024-09-10T10:01:57.405976+0000","last_became_peered":"2024-09-10T10:01:57.405976+0000","last_unstale":"2024-09-10T10:02:00.198179+0000","last_undegraded":"2024-09-10T10:02:00.198179+0000","last_fullsized":"2024-09-10T10:02:00.198179+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:41.204498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401441+0000","last_change":"2024-09-10T10:01:56.411631+0000","last_active":"2024-09-10T10:01:58.401441+0000","last_peered":"2024-09-10T10:01:58.401441+0000","last_clean":"2024-09-10T10:01:58.401441+0000","last_became_active":"2024-09-10T10:01:56.411378+0000","last_became_peered":"2024-09-10T10:01:56.411378+0000","last_unstale":"2024-09-10T10:01:58.401441+0000","last_undegraded":"2024-09-10T10:01:58.401441+0000","last_fullsized":"2024-09-10T10:01:58.401441+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:38.118765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199284+0000","last_change":"2024-09-10T10:01:57.422782+0000","last_active":"2024-09-10T10:02:00.199284+0000","last_peered":"2024-09-10T10:02:00.199284+0000","last_clean":"2024-09-10T10:02:00.199284+0000","last_became_active":"2024-09-10T10:01:57.420706+0000","last_became_peered":"2024-09-10T10:01:57.420706+0000","last_unstale":"2024-09-10T10:02:00.199284+0000","last_undegraded":"2024-09-10T10:02:00.199284+0000","last_fullsized":"2024-09-10T10:02:00.199284+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:54.032055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.200042+0000","last_change":"2024-09-10T10:01:56.403288+0000","last_active":"2024-09-10T10:02:00.200042+0000","last_peered":"2024-09-10T10:02:00.200042+0000","last_clean":"2024-09-10T10:02:00.200042+0000","last_became_active":"2024-09-10T10:01:56.402990+0000","last_became_peered":"2024-09-10T10:01:56.402990+0000","last_unstale":"2024-09-10T10:02:00.200042+0000","last_undegraded":"2024-09-10T10:02:00.200042+0000","last_fullsized":"2024-09-10T10:02:00.200042+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404710+0000","last_change":"2024-09-10T10:01:57.425385+0000","last_active":"2024-09-10T10:01:58.404710+0000","last_peered":"2024-09-10T10:01:58.404710+0000","last_clean":"2024-09-10T10:01:58.404710+0000","last_became_active":"2024-09-10T10:01:57.425032+0000","last_became_peered":"2024-09-10T10:01:57.425032+0000","last_unstale":"2024-09-10T10:01:58.404710+0000","last_undegraded":"2024-09-10T10:01:58.404710+0000","last_fullsized":"2024-09-10T10:01:58.404710+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:50.093786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404275+0000","last_change":"2024-09-10T10:01:56.407609+0000","last_active":"2024-09-10T10:01:58.404275+0000","last_peered":"2024-09-10T10:01:58.404275+0000","last_clean":"2024-09-10T10:01:58.404275+0000","last_became_active":"2024-09-10T10:01:56.405401+0000","last_became_peered":"2024-09-10T10:01:56.405401+0000","last_unstale":"2024-09-10T10:01:58.404275+0000","last_undegraded":"2024-09-10T10:01:58.404275+0000","last_fullsized":"2024-09-10T10:01:58.404275+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404436+0000","last_change":"2024-09-10T10:01:57.412223+0000","last_active":"2024-09-10T10:01:58.404436+0000","last_peered":"2024-09-10T10:01:58.404436+0000","last_clean":"2024-09-10T10:01:58.404436+0000","last_became_active":"2024-09-10T10:01:57.411477+0000","last_became_peered":"2024-09-10T10:01:57.411477+0000","last_unstale":"2024-09-10T10:01:58.404436+0000","last_undegraded":"2024-09-10T10:01:58.404436+0000","last_fullsized":"2024-09-10T10:01:58.404436+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199304+0000","last_change":"2024-09-10T10:01:56.406998+0000","last_active":"2024-09-10T10:02:00.199304+0000","last_peered":"2024-09-10T10:02:00.199304+0000","last_clean":"2024-09-10T10:02:00.199304+0000","last_became_active":"2024-09-10T10:01:56.406722+0000","last_became_peered":"2024-09-10T10:01:56.406722+0000","last_unstale":"2024-09-10T10:02:00.199304+0000","last_undegraded":"2024-09-10T10:02:00.199304+0000","last_fullsized":"2024-09-10T10:02:00.199304+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:17.905647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401574+0000","last_change":"2024-09-10T10:01:57.409970+0000","last_active":"2024-09-10T10:01:58.401574+0000","last_peered":"2024-09-10T10:01:58.401574+0000","last_clean":"2024-09-10T10:01:58.401574+0000","last_became_active":"2024-09-10T10:01:57.406199+0000","last_became_peered":"2024-09-10T10:01:57.406199+0000","last_unstale":"2024-09-10T10:01:58.401574+0000","last_undegraded":"2024-09-10T10:01:58.401574+0000","last_fullsized":"2024-09-10T10:01:58.401574+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:16.660997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.743331+0000","last_change":"2024-09-10T10:01:56.410726+0000","last_active":"2024-09-10T10:01:59.743331+0000","last_peered":"2024-09-10T10:01:59.743331+0000","last_clean":"2024-09-10T10:01:59.743331+0000","last_became_active":"2024-09-10T10:01:56.410239+0000","last_became_peered":"2024-09-10T10:01:56.410239+0000","last_unstale":"2024-09-10T10:01:59.743331+0000","last_undegraded":"2024-09-10T10:01:59.743331+0000","last_fullsized":"2024-09-10T10:01:59.743331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:44.033036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404002+0000","last_change":"2024-09-10T10:01:57.423932+0000","last_active":"2024-09-10T10:01:58.404002+0000","last_peered":"2024-09-10T10:01:58.404002+0000","last_clean":"2024-09-10T10:01:58.404002+0000","last_became_active":"2024-09-10T10:01:57.423635+0000","last_became_peered":"2024-09-10T10:01:57.423635+0000","last_unstale":"2024-09-10T10:01:58.404002+0000","last_undegraded":"2024-09-10T10:01:58.404002+0000","last_fullsized":"2024-09-10T10:01:58.404002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:55.992911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.743080+0000","last_change":"2024-09-10T10:01:56.411145+0000","last_active":"2024-09-10T10:01:59.743080+0000","last_peered":"2024-09-10T10:01:59.743080+0000","last_clean":"2024-09-10T10:01:59.743080+0000","last_became_active":"2024-09-10T10:01:56.410821+0000","last_became_peered":"2024-09-10T10:01:56.410821+0000","last_unstale":"2024-09-10T10:01:59.743080+0000","last_undegraded":"2024-09-10T10:01:59.743080+0000","last_fullsized":"2024-09-10T10:01:59.743080+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:34.635430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199152+0000","last_change":"2024-09-10T10:01:57.427059+0000","last_active":"2024-09-10T10:02:00.199152+0000","last_peered":"2024-09-10T10:02:00.199152+0000","last_clean":"2024-09-10T10:02:00.199152+0000","last_became_active":"2024-09-10T10:01:57.423512+0000","last_became_peered":"2024-09-10T10:01:57.423512+0000","last_unstale":"2024-09-10T10:02:00.199152+0000","last_undegraded":"2024-09-10T10:02:00.199152+0000","last_fullsized":"2024-09-10T10:02:00.199152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:41.204498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403628+0000","last_change":"2024-09-10T10:01:56.405376+0000","last_active":"2024-09-10T10:01:58.403628+0000","last_peered":"2024-09-10T10:01:58.403628+0000","last_clean":"2024-09-10T10:01:58.403628+0000","last_became_active":"2024-09-10T10:01:56.405056+0000","last_became_peered":"2024-09-10T10:01:56.405056+0000","last_unstale":"2024-09-10T10:01:58.403628+0000","last_undegraded":"2024-09-10T10:01:58.403628+0000","last_fullsized":"2024-09-10T10:01:58.403628+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:56.936736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198177+0000","last_change":"2024-09-10T10:01:57.411055+0000","last_active":"2024-09-10T10:02:00.198177+0000","last_peered":"2024-09-10T10:02:00.198177+0000","last_clean":"2024-09-10T10:02:00.198177+0000","last_became_active":"2024-09-10T10:01:57.410789+0000","last_became_peered":"2024-09-10T10:01:57.410789+0000","last_unstale":"2024-09-10T10:02:00.198177+0000","last_undegraded":"2024-09-10T10:02:00.198177+0000","last_fullsized":"2024-09-10T10:02:00.198177+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:52.470311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198797+0000","last_change":"2024-09-10T10:01:56.412704+0000","last_active":"2024-09-10T10:02:00.198797+0000","last_peered":"2024-09-10T10:02:00.198797+0000","last_clean":"2024-09-10T10:02:00.198797+0000","last_became_active":"2024-09-10T10:01:56.410023+0000","last_became_peered":"2024-09-10T10:01:56.410023+0000","last_unstale":"2024-09-10T10:02:00.198797+0000","last_undegraded":"2024-09-10T10:02:00.198797+0000","last_fullsized":"2024-09-10T10:02:00.198797+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:57.159878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401596+0000","last_change":"2024-09-10T10:01:57.407954+0000","last_active":"2024-09-10T10:01:58.401596+0000","last_peered":"2024-09-10T10:01:58.401596+0000","last_clean":"2024-09-10T10:01:58.401596+0000","last_became_active":"2024-09-10T10:01:57.407651+0000","last_became_peered":"2024-09-10T10:01:57.407651+0000","last_unstale":"2024-09-10T10:01:58.401596+0000","last_undegraded":"2024-09-10T10:01:58.401596+0000","last_fullsized":"2024-09-10T10:01:58.401596+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:37.697054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199444+0000","last_change":"2024-09-10T10:01:56.410104+0000","last_active":"2024-09-10T10:02:00.199444+0000","last_peered":"2024-09-10T10:02:00.199444+0000","last_clean":"2024-09-10T10:02:00.199444+0000","last_became_active":"2024-09-10T10:01:56.409756+0000","last_became_peered":"2024-09-10T10:01:56.409756+0000","last_unstale":"2024-09-10T10:02:00.199444+0000","last_undegraded":"2024-09-10T10:02:00.199444+0000","last_fullsized":"2024-09-10T10:02:00.199444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:06.968092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.200179+0000","last_change":"2024-09-10T10:01:57.405556+0000","last_active":"2024-09-10T10:02:00.200179+0000","last_peered":"2024-09-10T10:02:00.200179+0000","last_clean":"2024-09-10T10:02:00.200179+0000","last_became_active":"2024-09-10T10:01:57.405222+0000","last_became_peered":"2024-09-10T10:01:57.405222+0000","last_unstale":"2024-09-10T10:02:00.200179+0000","last_undegraded":"2024-09-10T10:02:00.200179+0000","last_fullsized":"2024-09-10T10:02:00.200179+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.867807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404012+0000","last_change":"2024-09-10T10:01:56.401752+0000","last_active":"2024-09-10T10:01:58.404012+0000","last_peered":"2024-09-10T10:01:58.404012+0000","last_clean":"2024-09-10T10:01:58.404012+0000","last_became_active":"2024-09-10T10:01:56.401508+0000","last_became_peered":"2024-09-10T10:01:56.401508+0000","last_unstale":"2024-09-10T10:01:58.404012+0000","last_undegraded":"2024-09-10T10:01:58.404012+0000","last_fullsized":"2024-09-10T10:01:58.404012+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:52.212395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404894+0000","last_change":"2024-09-10T10:01:57.425326+0000","last_active":"2024-09-10T10:01:58.404894+0000","last_peered":"2024-09-10T10:01:58.404894+0000","last_clean":"2024-09-10T10:01:58.404894+0000","last_became_active":"2024-09-10T10:01:57.424606+0000","last_became_peered":"2024-09-10T10:01:57.424606+0000","last_unstale":"2024-09-10T10:01:58.404894+0000","last_undegraded":"2024-09-10T10:01:58.404894+0000","last_fullsized":"2024-09-10T10:01:58.404894+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:56.736455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.200223+0000","last_change":"2024-09-10T10:01:56.409357+0000","last_active":"2024-09-10T10:02:00.200223+0000","last_peered":"2024-09-10T10:02:00.200223+0000","last_clean":"2024-09-10T10:02:00.200223+0000","last_became_active":"2024-09-10T10:01:56.409062+0000","last_became_peered":"2024-09-10T10:01:56.409062+0000","last_unstale":"2024-09-10T10:02:00.200223+0000","last_undegraded":"2024-09-10T10:02:00.200223+0000","last_fullsized":"2024-09-10T10:02:00.200223+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:29.657091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.743292+0000","last_change":"2024-09-10T10:01:57.401546+0000","last_active":"2024-09-10T10:01:59.743292+0000","last_peered":"2024-09-10T10:01:59.743292+0000","last_clean":"2024-09-10T10:01:59.743292+0000","last_became_active":"2024-09-10T10:01:57.401293+0000","last_became_peered":"2024-09-10T10:01:57.401293+0000","last_unstale":"2024-09-10T10:01:59.743292+0000","last_undegraded":"2024-09-10T10:01:59.743292+0000","last_fullsized":"2024-09-10T10:01:59.743292+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:07.970642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404816+0000","last_change":"2024-09-10T10:01:56.399162+0000","last_active":"2024-09-10T10:01:58.404816+0000","last_peered":"2024-09-10T10:01:58.404816+0000","last_clean":"2024-09-10T10:01:58.404816+0000","last_became_active":"2024-09-10T10:01:56.398886+0000","last_became_peered":"2024-09-10T10:01:56.398886+0000","last_unstale":"2024-09-10T10:01:58.404816+0000","last_undegraded":"2024-09-10T10:01:58.404816+0000","last_fullsized":"2024-09-10T10:01:58.404816+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:51.436702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401954+0000","last_change":"2024-09-10T10:01:57.409275+0000","last_active":"2024-09-10T10:01:58.401954+0000","last_peered":"2024-09-10T10:01:58.401954+0000","last_clean":"2024-09-10T10:01:58.401954+0000","last_became_active":"2024-09-10T10:01:57.409007+0000","last_became_peered":"2024-09-10T10:01:57.409007+0000","last_unstale":"2024-09-10T10:01:58.401954+0000","last_undegraded":"2024-09-10T10:01:58.401954+0000","last_fullsized":"2024-09-10T10:01:58.401954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:24.973215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.405203+0000","last_change":"2024-09-10T10:01:56.414600+0000","last_active":"2024-09-10T10:01:58.405203+0000","last_peered":"2024-09-10T10:01:58.405203+0000","last_clean":"2024-09-10T10:01:58.405203+0000","last_became_active":"2024-09-10T10:01:56.414312+0000","last_became_peered":"2024-09-10T10:01:56.414312+0000","last_unstale":"2024-09-10T10:01:58.405203+0000","last_undegraded":"2024-09-10T10:01:58.405203+0000","last_fullsized":"2024-09-10T10:01:58.405203+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:13.899397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199056+0000","last_change":"2024-09-10T10:01:57.410362+0000","last_active":"2024-09-10T10:02:00.199056+0000","last_peered":"2024-09-10T10:02:00.199056+0000","last_clean":"2024-09-10T10:02:00.199056+0000","last_became_active":"2024-09-10T10:01:57.408060+0000","last_became_peered":"2024-09-10T10:01:57.408060+0000","last_unstale":"2024-09-10T10:02:00.199056+0000","last_undegraded":"2024-09-10T10:02:00.199056+0000","last_fullsized":"2024-09-10T10:02:00.199056+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:38.275338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403644+0000","last_change":"2024-09-10T10:01:56.399440+0000","last_active":"2024-09-10T10:01:58.403644+0000","last_peered":"2024-09-10T10:01:58.403644+0000","last_clean":"2024-09-10T10:01:58.403644+0000","last_became_active":"2024-09-10T10:01:56.399153+0000","last_became_peered":"2024-09-10T10:01:56.399153+0000","last_unstale":"2024-09-10T10:01:58.403644+0000","last_undegraded":"2024-09-10T10:01:58.403644+0000","last_fullsized":"2024-09-10T10:01:58.403644+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:21.685329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198806+0000","last_change":"2024-09-10T10:01:57.412799+0000","last_active":"2024-09-10T10:02:00.198806+0000","last_peered":"2024-09-10T10:02:00.198806+0000","last_clean":"2024-09-10T10:02:00.198806+0000","last_became_active":"2024-09-10T10:01:57.412545+0000","last_became_peered":"2024-09-10T10:01:57.412545+0000","last_unstale":"2024-09-10T10:02:00.198806+0000","last_undegraded":"2024-09-10T10:02:00.198806+0000","last_fullsized":"2024-09-10T10:02:00.198806+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:31.461265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198909+0000","last_change":"2024-09-10T10:01:56.410145+0000","last_active":"2024-09-10T10:02:00.198909+0000","last_peered":"2024-09-10T10:02:00.198909+0000","last_clean":"2024-09-10T10:02:00.198909+0000","last_became_active":"2024-09-10T10:01:56.409854+0000","last_became_peered":"2024-09-10T10:01:56.409854+0000","last_unstale":"2024-09-10T10:02:00.198909+0000","last_undegraded":"2024-09-10T10:02:00.198909+0000","last_fullsized":"2024-09-10T10:02:00.198909+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:23.577580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198854+0000","last_change":"2024-09-10T10:01:57.415831+0000","last_active":"2024-09-10T10:02:00.198854+0000","last_peered":"2024-09-10T10:02:00.198854+0000","last_clean":"2024-09-10T10:02:00.198854+0000","last_became_active":"2024-09-10T10:01:57.415536+0000","last_became_peered":"2024-09-10T10:01:57.415536+0000","last_unstale":"2024-09-10T10:02:00.198854+0000","last_undegraded":"2024-09-10T10:02:00.198854+0000","last_fullsized":"2024-09-10T10:02:00.198854+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:04.303781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198013+0000","last_change":"2024-09-10T10:01:56.404958+0000","last_active":"2024-09-10T10:02:00.198013+0000","last_peered":"2024-09-10T10:02:00.198013+0000","last_clean":"2024-09-10T10:02:00.198013+0000","last_became_active":"2024-09-10T10:01:56.404619+0000","last_became_peered":"2024-09-10T10:01:56.404619+0000","last_unstale":"2024-09-10T10:02:00.198013+0000","last_undegraded":"2024-09-10T10:02:00.198013+0000","last_fullsized":"2024-09-10T10:02:00.198013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:34.514599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198457+0000","last_change":"2024-09-10T10:01:57.409907+0000","last_active":"2024-09-10T10:02:00.198457+0000","last_peered":"2024-09-10T10:02:00.198457+0000","last_clean":"2024-09-10T10:02:00.198457+0000","last_became_active":"2024-09-10T10:01:57.409375+0000","last_became_peered":"2024-09-10T10:01:57.409375+0000","last_unstale":"2024-09-10T10:02:00.198457+0000","last_undegraded":"2024-09-10T10:02:00.198457+0000","last_fullsized":"2024-09-10T10:02:00.198457+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:25.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.405024+0000","last_change":"2024-09-10T10:01:56.400929+0000","last_active":"2024-09-10T10:01:58.405024+0000","last_peered":"2024-09-10T10:01:58.405024+0000","last_clean":"2024-09-10T10:01:58.405024+0000","last_became_active":"2024-09-10T10:01:56.400665+0000","last_became_peered":"2024-09-10T10:01:56.400665+0000","last_unstale":"2024-09-10T10:01:58.405024+0000","last_undegraded":"2024-09-10T10:01:58.405024+0000","last_fullsized":"2024-09-10T10:01:58.405024+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:32.570015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199225+0000","last_change":"2024-09-10T10:01:57.426884+0000","last_active":"2024-09-10T10:02:00.199225+0000","last_peered":"2024-09-10T10:02:00.199225+0000","last_clean":"2024-09-10T10:02:00.199225+0000","last_became_active":"2024-09-10T10:01:57.426621+0000","last_became_peered":"2024-09-10T10:01:57.426621+0000","last_unstale":"2024-09-10T10:02:00.199225+0000","last_undegraded":"2024-09-10T10:02:00.199225+0000","last_fullsized":"2024-09-10T10:02:00.199225+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:55.992911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199478+0000","last_change":"2024-09-10T10:01:56.410855+0000","last_active":"2024-09-10T10:02:00.199478+0000","last_peered":"2024-09-10T10:02:00.199478+0000","last_clean":"2024-09-10T10:02:00.199478+0000","last_became_active":"2024-09-10T10:01:56.410597+0000","last_became_peered":"2024-09-10T10:01:56.410597+0000","last_unstale":"2024-09-10T10:02:00.199478+0000","last_undegraded":"2024-09-10T10:02:00.199478+0000","last_fullsized":"2024-09-10T10:02:00.199478+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:38.934821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.405120+0000","last_change":"2024-09-10T10:01:57.423850+0000","last_active":"2024-09-10T10:01:58.405120+0000","last_peered":"2024-09-10T10:01:58.405120+0000","last_clean":"2024-09-10T10:01:58.405120+0000","last_became_active":"2024-09-10T10:01:57.423309+0000","last_became_peered":"2024-09-10T10:01:57.423309+0000","last_unstale":"2024-09-10T10:01:58.405120+0000","last_undegraded":"2024-09-10T10:01:58.405120+0000","last_fullsized":"2024-09-10T10:01:58.405120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:53.870084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401913+0000","last_change":"2024-09-10T10:01:56.409831+0000","last_active":"2024-09-10T10:01:58.401913+0000","last_peered":"2024-09-10T10:01:58.401913+0000","last_clean":"2024-09-10T10:01:58.401913+0000","last_became_active":"2024-09-10T10:01:56.409512+0000","last_became_peered":"2024-09-10T10:01:56.409512+0000","last_unstale":"2024-09-10T10:01:58.401913+0000","last_undegraded":"2024-09-10T10:01:58.401913+0000","last_fullsized":"2024-09-10T10:01:58.401913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:11:05.784097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404789+0000","last_change":"2024-09-10T10:01:57.427760+0000","last_active":"2024-09-10T10:01:58.404789+0000","last_peered":"2024-09-10T10:01:58.404789+0000","last_clean":"2024-09-10T10:01:58.404789+0000","last_became_active":"2024-09-10T10:01:57.427511+0000","last_became_peered":"2024-09-10T10:01:57.427511+0000","last_unstale":"2024-09-10T10:01:58.404789+0000","last_undegraded":"2024-09-10T10:01:58.404789+0000","last_fullsized":"2024-09-10T10:01:58.404789+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:59.968067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198230+0000","last_change":"2024-09-10T10:01:56.402970+0000","last_active":"2024-09-10T10:02:00.198230+0000","last_peered":"2024-09-10T10:02:00.198230+0000","last_clean":"2024-09-10T10:02:00.198230+0000","last_became_active":"2024-09-10T10:01:56.402500+0000","last_became_peered":"2024-09-10T10:01:56.402500+0000","last_unstale":"2024-09-10T10:02:00.198230+0000","last_undegraded":"2024-09-10T10:02:00.198230+0000","last_fullsized":"2024-09-10T10:02:00.198230+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:26.744793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401680+0000","last_change":"2024-09-10T10:01:57.409373+0000","last_active":"2024-09-10T10:01:58.401680+0000","last_peered":"2024-09-10T10:01:58.401680+0000","last_clean":"2024-09-10T10:01:58.401680+0000","last_became_active":"2024-09-10T10:01:57.409114+0000","last_became_peered":"2024-09-10T10:01:57.409114+0000","last_unstale":"2024-09-10T10:01:58.401680+0000","last_undegraded":"2024-09-10T10:01:58.401680+0000","last_fullsized":"2024-09-10T10:01:58.401680+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:14.208069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.401717+0000","last_change":"2024-09-10T10:01:56.411209+0000","last_active":"2024-09-10T10:01:58.401717+0000","last_peered":"2024-09-10T10:01:58.401717+0000","last_clean":"2024-09-10T10:01:58.401717+0000","last_became_active":"2024-09-10T10:01:56.410808+0000","last_became_peered":"2024-09-10T10:01:56.410808+0000","last_unstale":"2024-09-10T10:01:58.401717+0000","last_undegraded":"2024-09-10T10:01:58.401717+0000","last_fullsized":"2024-09-10T10:01:58.401717+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:39.392910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403558+0000","last_change":"2024-09-10T10:01:57.418730+0000","last_active":"2024-09-10T10:01:58.403558+0000","last_peered":"2024-09-10T10:01:58.403558+0000","last_clean":"2024-09-10T10:01:58.403558+0000","last_became_active":"2024-09-10T10:01:57.418395+0000","last_became_peered":"2024-09-10T10:01:57.418395+0000","last_unstale":"2024-09-10T10:01:58.403558+0000","last_undegraded":"2024-09-10T10:01:58.403558+0000","last_fullsized":"2024-09-10T10:01:58.403558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:14.901947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.742969+0000","last_change":"2024-09-10T10:01:56.409314+0000","last_active":"2024-09-10T10:01:59.742969+0000","last_peered":"2024-09-10T10:01:59.742969+0000","last_clean":"2024-09-10T10:01:59.742969+0000","last_became_active":"2024-09-10T10:01:56.409077+0000","last_became_peered":"2024-09-10T10:01:56.409077+0000","last_unstale":"2024-09-10T10:01:59.742969+0000","last_undegraded":"2024-09-10T10:01:59.742969+0000","last_fullsized":"2024-09-10T10:01:59.742969+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:08.220387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:59.743104+0000","last_change":"2024-09-10T10:01:57.407297+0000","last_active":"2024-09-10T10:01:59.743104+0000","last_peered":"2024-09-10T10:01:59.743104+0000","last_clean":"2024-09-10T10:01:59.743104+0000","last_became_active":"2024-09-10T10:01:57.407030+0000","last_became_peered":"2024-09-10T10:01:57.407030+0000","last_unstale":"2024-09-10T10:01:59.743104+0000","last_undegraded":"2024-09-10T10:01:59.743104+0000","last_fullsized":"2024-09-10T10:01:59.743104+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:22.687879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.199229+0000","last_change":"2024-09-10T10:01:56.411518+0000","last_active":"2024-09-10T10:02:00.199229+0000","last_peered":"2024-09-10T10:02:00.199229+0000","last_clean":"2024-09-10T10:02:00.199229+0000","last_became_active":"2024-09-10T10:01:56.410515+0000","last_became_peered":"2024-09-10T10:01:56.410515+0000","last_unstale":"2024-09-10T10:02:00.199229+0000","last_undegraded":"2024-09-10T10:02:00.199229+0000","last_fullsized":"2024-09-10T10:02:00.199229+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:39.856462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403226+0000","last_change":"2024-09-10T10:01:57.408569+0000","last_active":"2024-09-10T10:01:58.403226+0000","last_peered":"2024-09-10T10:01:58.403226+0000","last_clean":"2024-09-10T10:01:58.403226+0000","last_became_active":"2024-09-10T10:01:57.408302+0000","last_became_peered":"2024-09-10T10:01:57.408302+0000","last_unstale":"2024-09-10T10:01:58.403226+0000","last_undegraded":"2024-09-10T10:01:58.403226+0000","last_fullsized":"2024-09-10T10:01:58.403226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:24.580129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403634+0000","last_change":"2024-09-10T10:01:56.407472+0000","last_active":"2024-09-10T10:01:58.403634+0000","last_peered":"2024-09-10T10:01:58.403634+0000","last_clean":"2024-09-10T10:01:58.403634+0000","last_became_active":"2024-09-10T10:01:56.407112+0000","last_became_peered":"2024-09-10T10:01:56.407112+0000","last_unstale":"2024-09-10T10:01:58.403634+0000","last_undegraded":"2024-09-10T10:01:58.403634+0000","last_fullsized":"2024-09-10T10:01:58.403634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:36.230334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:02:00.198989+0000","last_change":"2024-09-10T10:01:57.410765+0000","last_active":"2024-09-10T10:02:00.198989+0000","last_peered":"2024-09-10T10:02:00.198989+0000","last_clean":"2024-09-10T10:02:00.198989+0000","last_became_active":"2024-09-10T10:01:57.410504+0000","last_became_peered":"2024-09-10T10:01:57.410504+0000","last_unstale":"2024-09-10T10:02:00.198989+0000","last_undegraded":"2024-09-10T10:02:00.198989+0000","last_fullsized":"2024-09-10T10:02:00.198989+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:56.377542+0000","last_clean_scrub_stamp":"2024-09-10T10:01:56.377542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:28.265579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.403717+0000","last_change":"2024-09-10T10:01:56.408027+0000","last_active":"2024-09-10T10:01:58.403717+0000","last_peered":"2024-09-10T10:01:58.403717+0000","last_clean":"2024-09-10T10:01:58.403717+0000","last_became_active":"2024-09-10T10:01:56.407749+0000","last_became_peered":"2024-09-10T10:01:56.407749+0000","last_unstale":"2024-09-10T10:01:58.403717+0000","last_undegraded":"2024-09-10T10:01:58.403717+0000","last_fullsized":"2024-09-10T10:01:58.403717+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:55.374992+0000","last_clean_scrub_stamp":"2024-09-10T10:01:55.374992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:42.979120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":254,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:01:58.404347+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:01:58.404347+0000","last_peered":"2024-09-10T10:01:58.404347+0000","last_clean":"2024-09-10T10:01:58.404347+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:01:58.404347+0000","last_undegraded":"2024-09-10T10:01:58.404347+0000","last_fullsized":"2024-09-10T10:01:58.404347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":936873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":839680,"data_stored":345981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":345979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":345959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":843776,"data_stored":347501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":348117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":936257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542194,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":346001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:06.371 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:02:06.371 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:02:06.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.548+0000 7fae0e200640 1 Processor -- start 2024-09-10T10:02:06.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.548+0000 7fae0e200640 1 -- start start 2024-09-10T10:02:06.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae0e200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fae0806e7a0 0x7fae080cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae0e200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae08072150 con 0x7fae080cfe30 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae0e200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0806d2d0 con 0x7fae080cd510 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae0e200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fae0815d9a0 con 0x7fae0806e7a0 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae07e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fae0806e7a0 0x7fae080cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae07400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fae080cfe30 0x7fae08071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36902/0 (socket says 172.21.15.110:36902) 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae07400640 1 -- 172.21.15.110:0/2520725780 learned_addr learned my addr 172.21.15.110:0/2520725780 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae07e00640 1 -- 172.21.15.110:0/2520725780 >> v1:172.21.15.145:6789/0 conn(0x7fae080cd510 legacy=0x7fae0806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae07e00640 1 -- 172.21.15.110:0/2520725780 >> v1:172.21.15.110:6789/0 conn(0x7fae080cfe30 legacy=0x7fae08071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae07e00640 1 -- 172.21.15.110:0/2520725780 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0806d5b0 con 0x7fae0806e7a0 2024-09-10T10:02:06.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae07e00640 1 --2- 172.21.15.110:0/2520725780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0806e7a0 0x7fae080cd9a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fae00009d90 tx=0x7fae0001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5f47d6155003dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae06000640 1 -- 172.21.15.110:0/2520725780 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae000b2070 con 0x7fae0806e7a0 2024-09-10T10:02:06.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae06000640 1 -- 172.21.15.110:0/2520725780 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae000a58b0 con 0x7fae0806e7a0 2024-09-10T10:02:06.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.552+0000 7fae06000640 1 -- 172.21.15.110:0/2520725780 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae000a5c80 con 0x7fae0806e7a0 2024-09-10T10:02:06.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- 172.21.15.110:0/2520725780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0806e7a0 msgr2=0x7fae080cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 --2- 172.21.15.110:0/2520725780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0806e7a0 0x7fae080cd9a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fae00009d90 tx=0x7fae0001c8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- 172.21.15.110:0/2520725780 shutdown_connections 2024-09-10T10:02:06.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 --2- 172.21.15.110:0/2520725780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0806e7a0 0x7fae080cd9a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- 172.21.15.110:0/2520725780 >> 172.21.15.110:0/2520725780 conn(0x7fae0806bc50 msgr2=0x7fae0806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:06.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- 172.21.15.110:0/2520725780 shutdown_connections 2024-09-10T10:02:06.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- 172.21.15.110:0/2520725780 wait complete. 2024-09-10T10:02:06.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 Processor -- start 2024-09-10T10:02:06.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- start start 2024-09-10T10:02:06.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0806e7a0 0x7fae080b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae080cd510 0x7fae08146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080cfe30 0x7fae080ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae0816a190 con 0x7fae080cd510 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae0815d700 con 0x7fae0806e7a0 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae0e200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fae0816a310 con 0x7fae080cfe30 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae06a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae080cd510 0x7fae08146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae06a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae080cd510 0x7fae08146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44154/0 (socket says 172.21.15.110:44154) 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae06a00640 1 -- 172.21.15.110:0/1906175249 learned_addr learned my addr 172.21.15.110:0/1906175249 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae07400640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0806e7a0 0x7fae080b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae07e00640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080cfe30 0x7fae080ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae06a00640 1 -- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080cfe30 msgr2=0x7fae080ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae06a00640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080cfe30 0x7fae080ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae06a00640 1 -- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0806e7a0 msgr2=0x7fae080b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae06a00640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0806e7a0 0x7fae080b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae06a00640 1 -- 172.21.15.110:0/1906175249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0814fa60 con 0x7fae080cd510 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.556+0000 7fae07400640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0806e7a0 0x7fae080b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fae06a00640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae080cd510 0x7fae08146b30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c3c0 tx=0x7fadf400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fadff400640 1 -- 172.21.15.110:0/1906175249 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4017070 con 0x7fae080cd510 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fadff400640 1 -- 172.21.15.110:0/1906175249 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf40052d0 con 0x7fae080cd510 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fadff400640 1 -- 172.21.15.110:0/1906175249 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4003840 con 0x7fae080cd510 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0814ebc0 con 0x7fae080cd510 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae08153c80 con 0x7fae080cd510 2024-09-10T10:02:06.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0806e090 con 0x7fae080cd510 2024-09-10T10:02:06.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fadff400640 1 -- 172.21.15.110:0/1906175249 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fadf401a070 con 0x7fae080cd510 2024-09-10T10:02:06.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fadff400640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fade8067c50 0x7fade806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fadff400640 1 -- 172.21.15.110:0/1906175249 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7fadf4085580 con 0x7fae080cd510 2024-09-10T10:02:06.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.560+0000 7fae07400640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fade8067c50 0x7fade806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.564+0000 7fae07400640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fade8067c50 0x7fade806a110 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005890 tx=0x7fadf8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.564+0000 7fadff400640 1 -- 172.21.15.110:0/1906175249 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fadf4005480 con 0x7fae080cd510 2024-09-10T10:02:06.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.776+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fae080ce5b0 con 0x7fae080cd510 2024-09-10T10:02:06.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fadff400640 1 -- 172.21.15.110:0/1906175249 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fadf4055c10 con 0x7fae080cd510 2024-09-10T10:02:06.782 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:02:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:02:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:02:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:02:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:02:06.782 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:02:06.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fade8067c50 msgr2=0x7fade806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fade8067c50 0x7fade806a110 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005890 tx=0x7fadf8009040 comp rx=0 tx=0).stop 2024-09-10T10:02:06.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae080cd510 msgr2=0x7fae08146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae080cd510 0x7fae08146b30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c3c0 tx=0x7fadf400c880 comp rx=0 tx=0).stop 2024-09-10T10:02:06.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 shutdown_connections 2024-09-10T10:02:06.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fade8067c50 0x7fade806a110 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080cfe30 0x7fae080ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae080cd510 0x7fae08146b30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 --2- 172.21.15.110:0/1906175249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0806e7a0 0x7fae080b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 >> 172.21.15.110:0/1906175249 conn(0x7fae0806bc50 msgr2=0x7fae0806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:06.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 shutdown_connections 2024-09-10T10:02:06.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.780+0000 7fae0e200640 1 -- 172.21.15.110:0/1906175249 wait complete. 2024-09-10T10:02:06.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.932+0000 7fa4db000640 1 Processor -- start 2024-09-10T10:02:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4db000640 1 -- start start 2024-09-10T10:02:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4db000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40d05c0 0x7fa4d40d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4db000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa4d40b59c0 0x7fa4d406ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4db000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4d40cf540 0x7fa4d40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4db000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa4d415b970 con 0x7fa4d40cf540 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4db000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa4d40d0ee0 con 0x7fa4d40b59c0 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4db000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d415bc10 con 0x7fa4d40d05c0 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4da600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4d40cf540 0x7fa4d40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4da600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4d40cf540 0x7fa4d40cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44162/0 (socket says 172.21.15.110:44162) 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4d9c00640 1 --2- 172.21.15.110:0/3790621686 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40d05c0 0x7fa4d40d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4da600640 1 -- 172.21.15.110:0/3790621686 learned_addr learned my addr 172.21.15.110:0/3790621686 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4d9200640 1 --2- 172.21.15.110:0/3790621686 >> v2:172.21.15.145:3300/0 conn(0x7fa4d40b59c0 0x7fa4d406ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4da600640 1 -- 172.21.15.110:0/3790621686 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40d05c0 msgr2=0x7fa4d40d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4da600640 1 --2- 172.21.15.110:0/3790621686 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40d05c0 0x7fa4d40d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4da600640 1 -- 172.21.15.110:0/3790621686 >> v2:172.21.15.145:3300/0 conn(0x7fa4d40b59c0 msgr2=0x7fa4d406ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4da600640 1 --2- 172.21.15.110:0/3790621686 >> v2:172.21.15.145:3300/0 conn(0x7fa4d40b59c0 0x7fa4d406ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.936+0000 7fa4da600640 1 -- 172.21.15.110:0/3790621686 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d415bef0 con 0x7fa4d40cf540 2024-09-10T10:02:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4da600640 1 --2- 172.21.15.110:0/3790621686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40cf540 0x7fa4d40cf9f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4069120 tx=0x7fa4cc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ece80e273a78700 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4d3e00640 1 -- 172.21.15.110:0/3790621686 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0a5750 con 0x7fa4d40cf540 2024-09-10T10:02:06.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4d3e00640 1 -- 172.21.15.110:0/3790621686 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc0a58f0 con 0x7fa4d40cf540 2024-09-10T10:02:06.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4d3e00640 1 -- 172.21.15.110:0/3790621686 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0a5cc0 con 0x7fa4d40cf540 2024-09-10T10:02:06.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 -- 172.21.15.110:0/3790621686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40cf540 msgr2=0x7fa4d40cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 --2- 172.21.15.110:0/3790621686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40cf540 0x7fa4d40cf9f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4069120 tx=0x7fa4cc01c9c0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 -- 172.21.15.110:0/3790621686 shutdown_connections 2024-09-10T10:02:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 --2- 172.21.15.110:0/3790621686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40cf540 0x7fa4d40cf9f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 --2- 172.21.15.110:0/3790621686 >> v2:172.21.15.145:3300/0 conn(0x7fa4d40b59c0 0x7fa4d406ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 --2- 172.21.15.110:0/3790621686 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40d05c0 0x7fa4d40d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 -- 172.21.15.110:0/3790621686 >> 172.21.15.110:0/3790621686 conn(0x7fa4d4062ce0 msgr2=0x7fa4d40b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 -- 172.21.15.110:0/3790621686 shutdown_connections 2024-09-10T10:02:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.940+0000 7fa4db000640 1 -- 172.21.15.110:0/3790621686 wait complete. 2024-09-10T10:02:06.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 Processor -- start 2024-09-10T10:02:06.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 -- start start 2024-09-10T10:02:06.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4d40b59c0 0x7fa4d415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40cf540 0x7fa4d4157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40d05c0 0x7fa4d4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4155ca0 con 0x7fa4d40d05c0 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4155b20 con 0x7fa4d40b59c0 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d415f5d0 con 0x7fa4d40cf540 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40d05c0 0x7fa4d4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4d9c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4d40b59c0 0x7fa4d415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40d05c0 0x7fa4d4157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44168/0 (socket says 172.21.15.110:44168) 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 -- 172.21.15.110:0/2368017571 learned_addr learned my addr 172.21.15.110:0/2368017571 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4d9200640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40cf540 0x7fa4d4157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 -- 172.21.15.110:0/2368017571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40cf540 msgr2=0x7fa4d4157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40cf540 0x7fa4d4157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 -- 172.21.15.110:0/2368017571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4d40b59c0 msgr2=0x7fa4d415d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4d40b59c0 0x7fa4d415d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 -- 172.21.15.110:0/2368017571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d415dd60 con 0x7fa4d40d05c0 2024-09-10T10:02:06.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4d9c00640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4d40b59c0 0x7fa4d415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:06.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4da600640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40d05c0 0x7fa4d4157550 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc01b070 tx=0x7fa4cc019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4d1600640 1 -- 172.21.15.110:0/2368017571 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc009510 con 0x7fa4d40d05c0 2024-09-10T10:02:06.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4d1600640 1 -- 172.21.15.110:0/2368017571 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc0be070 con 0x7fa4d40d05c0 2024-09-10T10:02:06.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4d1600640 1 -- 172.21.15.110:0/2368017571 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0b88d0 con 0x7fa4d40d05c0 2024-09-10T10:02:06.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d415e040 con 0x7fa4d40d05c0 2024-09-10T10:02:06.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d415e600 con 0x7fa4d40d05c0 2024-09-10T10:02:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.944+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d415f990 con 0x7fa4d40d05c0 2024-09-10T10:02:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.948+0000 7fa4d1600640 1 -- 172.21.15.110:0/2368017571 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fa4cc0b8a70 con 0x7fa4d40d05c0 2024-09-10T10:02:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.948+0000 7fa4d1600640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4b4067c20 0x7fa4b406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.948+0000 7fa4d1600640 1 -- 172.21.15.110:0/2368017571 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7fa4cc121150 con 0x7fa4d40d05c0 2024-09-10T10:02:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.948+0000 7fa4d9c00640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4b4067c20 0x7fa4b406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.948+0000 7fa4d9c00640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4b4067c20 0x7fa4b406a0e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009eb0 tx=0x7fa4c8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:06.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:06.952+0000 7fa4d1600640 1 -- 172.21.15.110:0/2368017571 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa4cc0165e0 con 0x7fa4d40d05c0 2024-09-10T10:02:07.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.140+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4d4071d80 con 0x7fa4d40d05c0 2024-09-10T10:02:07.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.140+0000 7fa4d1600640 1 -- 172.21.15.110:0/2368017571 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5237 (secure 0 0 0) 0x7fa4cc0f12f0 con 0x7fa4d40d05c0 2024-09-10T10:02:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:07.146 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":45,"btime":"2024-09-10T10:02:00:413261+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:07.147 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 45 2024-09-10T10:02:07.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4b4067c20 msgr2=0x7fa4b406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4b4067c20 0x7fa4b406a0e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009eb0 tx=0x7fa4c8009190 comp rx=0 tx=0).stop 2024-09-10T10:02:07.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40d05c0 msgr2=0x7fa4d4157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40d05c0 0x7fa4d4157550 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc01b070 tx=0x7fa4cc019760 comp rx=0 tx=0).stop 2024-09-10T10:02:07.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 shutdown_connections 2024-09-10T10:02:07.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fa4b4067c20 0x7fa4b406a0e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4d40d05c0 0x7fa4d4157550 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4d40cf540 0x7fa4d4157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 --2- 172.21.15.110:0/2368017571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4d40b59c0 0x7fa4d415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 >> 172.21.15.110:0/2368017571 conn(0x7fa4d4062ce0 msgr2=0x7fa4d406de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 shutdown_connections 2024-09-10T10:02:07.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.144+0000 7fa4db000640 1 -- 172.21.15.110:0/2368017571 wait complete. 2024-09-10T10:02:07.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:01:56.770054+0000', 'modified': '2024-09-10T10:01:59.790677+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6033}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6033': {'gid': 6033, 'name': 'd', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.145:6835/1773965791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6833', 'nonce': 1773965791}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 1773965791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6033, 'qdb_cluster': [6033]} 2024-09-10T10:02:07.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:02:07.569 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T10:02:07.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4847c00640 1 Processor -- start 2024-09-10T10:02:07.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4847c00640 1 -- start start 2024-09-10T10:02:07.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4847c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48400726e0 0x7f48400cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4847c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f484006efa0 0x7f48400b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4847c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f484006f410 con 0x7f48400b5430 2024-09-10T10:02:07.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4847c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f48400cee50 con 0x7f48400726e0 2024-09-10T10:02:07.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4847c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f48400b5860 con 0x7f484006efa0 2024-09-10T10:02:07.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4845200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48400726e0 0x7f48400cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4845200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48400726e0 0x7f48400cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44176/0 (socket says 172.21.15.110:44176) 2024-09-10T10:02:07.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.704+0000 7f4845200640 1 -- 172.21.15.110:0/101022199 learned_addr learned my addr 172.21.15.110:0/101022199 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:07.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4845200640 1 -- 172.21.15.110:0/101022199 >> v2:172.21.15.145:3300/0 conn(0x7f484006efa0 msgr2=0x7f48400b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4845200640 1 --2- 172.21.15.110:0/101022199 >> v2:172.21.15.145:3300/0 conn(0x7f484006efa0 0x7f48400b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4845200640 1 -- 172.21.15.110:0/101022199 >> v1:172.21.15.145:6789/0 conn(0x7f48400b5430 legacy=0x7f48400b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4845200640 1 -- 172.21.15.110:0/101022199 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484015fda0 con 0x7f48400726e0 2024-09-10T10:02:07.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4845200640 1 --2- 172.21.15.110:0/101022199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400726e0 0x7f48400cf2e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f484006ebf0 tx=0x7f483001c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a81e18d5ff8614f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f483f400640 1 -- 172.21.15.110:0/101022199 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300b0070 con 0x7f48400726e0 2024-09-10T10:02:07.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f483f400640 1 -- 172.21.15.110:0/101022199 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48300ab600 con 0x7f48400726e0 2024-09-10T10:02:07.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f483f400640 1 -- 172.21.15.110:0/101022199 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300ab9d0 con 0x7f48400726e0 2024-09-10T10:02:07.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4847c00640 1 -- 172.21.15.110:0/101022199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400726e0 msgr2=0x7f48400cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4847c00640 1 --2- 172.21.15.110:0/101022199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400726e0 0x7f48400cf2e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f484006ebf0 tx=0x7f483001c2d0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4847c00640 1 -- 172.21.15.110:0/101022199 shutdown_connections 2024-09-10T10:02:07.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4847c00640 1 --2- 172.21.15.110:0/101022199 >> v2:172.21.15.145:3300/0 conn(0x7f484006efa0 0x7f48400b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4847c00640 1 --2- 172.21.15.110:0/101022199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400726e0 0x7f48400cf2e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4847c00640 1 -- 172.21.15.110:0/101022199 >> 172.21.15.110:0/101022199 conn(0x7f4840062d10 msgr2=0x7f484006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4847c00640 1 -- 172.21.15.110:0/101022199 shutdown_connections 2024-09-10T10:02:07.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.708+0000 7f4847c00640 1 -- 172.21.15.110:0/101022199 wait complete. 2024-09-10T10:02:07.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 Processor -- start 2024-09-10T10:02:07.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 -- start start 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484006efa0 0x7f4840142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400726e0 0x7f4840157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b5430 0x7f4840158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f48400b3ff0 con 0x7f48400b5430 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4840073270 con 0x7f484006efa0 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f48400b4170 con 0x7f48400726e0 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b5430 0x7f4840158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b5430 0x7f4840158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44184/0 (socket says 172.21.15.110:44184) 2024-09-10T10:02:07.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 -- 172.21.15.110:0/3009433289 learned_addr learned my addr 172.21.15.110:0/3009433289 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845200640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484006efa0 0x7f4840142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f483fe00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400726e0 0x7f4840157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 -- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400726e0 msgr2=0x7f4840157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400726e0 0x7f4840157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 -- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484006efa0 msgr2=0x7f4840142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484006efa0 0x7f4840142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 -- 172.21.15.110:0/3009433289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4840158d00 con 0x7f48400b5430 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f483fe00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400726e0 0x7f4840157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4845c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b5430 0x7f4840158160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f483800b990 tx=0x7f483800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f483cc00640 1 -- 172.21.15.110:0/3009433289 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838019070 con 0x7f48400b5430 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f483cc00640 1 -- 172.21.15.110:0/3009433289 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483800cba0 con 0x7f48400b5430 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 -- 172.21.15.110:0/3009433289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4840158740 con 0x7f48400b5430 2024-09-10T10:02:07.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f483cc00640 1 -- 172.21.15.110:0/3009433289 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838014440 con 0x7f48400b5430 2024-09-10T10:02:07.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.712+0000 7f4847c00640 1 -- 172.21.15.110:0/3009433289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48400b01e0 con 0x7f48400b5430 2024-09-10T10:02:07.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f483cc00640 1 -- 172.21.15.110:0/3009433289 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f483801c070 con 0x7f48400b5430 2024-09-10T10:02:07.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f483cc00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4820067c20 0x7f482006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f483cc00640 1 -- 172.21.15.110:0/3009433289 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f4838085f20 con 0x7f48400b5430 2024-09-10T10:02:07.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f4845200640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4820067c20 0x7f482006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f484b098000 1 Processor -- start 2024-09-10T10:02:07.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f484b098000 1 -- start start 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f4845200640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4820067c20 0x7f482006a0e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f48300ad000 tx=0x7f48300a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f484b098000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3699eb20 0x563e3699ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f484b098000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e36954d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f484b098000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e36955290 0x563e36955670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f484b098000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563e3683cb60 con 0x563e36955290 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f484b098000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563e367a3c70 con 0x563e3699eb20 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.716+0000 7f484b098000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x563e36749ed0 con 0x563e3699f440 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e36955290 0x563e36955670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e36955290 0x563e36955670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44196/0 (socket says 172.21.15.110:44196) 2024-09-10T10:02:07.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 -- 172.21.15.110:0/1410651638 learned_addr learned my addr 172.21.15.110:0/1410651638 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f483fe00640 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e36954d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845200640 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3699eb20 0x563e3699ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 -- 172.21.15.110:0/1410651638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 msgr2=0x563e36954d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e36954d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 -- 172.21.15.110:0/1410651638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3699eb20 msgr2=0x563e3699ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3699eb20 0x563e3699ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 -- 172.21.15.110:0/1410651638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3675be50 con 0x563e36955290 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e36955290 0x563e36955670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4838000f90 tx=0x7f4838056780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3663d628cf739df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4835600640 1 -- 172.21.15.110:0/1410651638 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48380525b0 con 0x563e36955290 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4835600640 1 -- 172.21.15.110:0/1410651638 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483805e690 con 0x563e36955290 2024-09-10T10:02:07.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4835600640 1 -- 172.21.15.110:0/1410651638 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838052750 con 0x563e36955290 2024-09-10T10:02:07.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- 172.21.15.110:0/1410651638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e36955290 msgr2=0x563e36955670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e36955290 0x563e36955670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4838000f90 tx=0x7f4838056780 comp rx=0 tx=0).stop 2024-09-10T10:02:07.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- 172.21.15.110:0/1410651638 shutdown_connections 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e36955290 0x563e36955670 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e36954d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 --2- 172.21.15.110:0/1410651638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3699eb20 0x563e3699ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- 172.21.15.110:0/1410651638 >> 172.21.15.110:0/1410651638 conn(0x563e3695b600 msgr2=0x563e36953630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- 172.21.15.110:0/1410651638 shutdown_connections 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- 172.21.15.110:0/1410651638 wait complete. 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- start start 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 Processor -- start 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e36959e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e3695fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e36960230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563e36805b00 con 0x563e3699eb20 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563e3677e360 con 0x563e3695a390 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x563e3677b2d0 con 0x563e3699f440 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f484b098000 10 client.0 _ll_register_callbacks cb 0x563e36940820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e36959e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e36959e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44204/0 (socket says 172.21.15.110:44204) 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845200640 1 -- 172.21.15.110:0/2462528352 learned_addr learned my addr 172.21.15.110:0/2462528352 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f483fe00640 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e3695fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.720+0000 7f4845c00640 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e36960230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f4845200640 1 -- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 msgr2=0x563e3695fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f4845200640 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e3695fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f4845200640 1 -- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 msgr2=0x563e36960230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f4845200640 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e36960230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f4845200640 1 -- 172.21.15.110:0/2462528352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3674a120 con 0x563e3699eb20 2024-09-10T10:02:07.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f4845c00640 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e36960230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f4845200640 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e36959e50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f483001c7b0 tx=0x7f483001ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9b9031ddc2ff2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 1 -- 172.21.15.110:0/2462528352 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300b0040 con 0x563e3699eb20 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 1 -- 172.21.15.110:0/2462528352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f483805e690 con 0x563e3699eb20 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 1 -- 172.21.15.110:0/2462528352 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f48300b77a0 con 0x563e3699eb20 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 1 -- 172.21.15.110:0/2462528352 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f48300cf050 con 0x563e3699eb20 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 10 client.6177 fetch_fsmap learned FSMap version 45 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 1 -- 172.21.15.110:0/2462528352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x563e36743450 con 0x563e3699eb20 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 1 -- 172.21.15.110:0/2462528352 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483805e690 con 0x563e3699eb20 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 1 -- 172.21.15.110:0/2462528352 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f48300d2020 con 0x563e3699eb20 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 10 client.6177 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 10 client.6177 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T10:02:07.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 20 client.6177 populate_metadata read hostname 'smithi110' 2024-09-10T10:02:07.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] conn(0x563e369ad280 0x563e369af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 4 client.6177 mds_command: new command op to 6033 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:07.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 1 -- 172.21.15.110:0/2462528352 --> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e3673dc00 con 0x563e369ad280 2024-09-10T10:02:07.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f483fe00640 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] conn(0x563e369ad280 0x563e369af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f483fe00640 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] conn(0x563e369ad280 0x563e369af670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999ad4d421acac08 server_cookie=e36fc912723810a1 in_seq=0 out_seq=0 2024-09-10T10:02:07.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 10 client.6177 ms_handle_connect on v2:172.21.15.145:6833/1773965791 2024-09-10T10:02:07.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 1 -- 172.21.15.110:0/2462528352 <== mds.0 v2:172.21.15.145:6833/1773965791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e3673dc00 con 0x563e369ad280 2024-09-10T10:02:07.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f482a000640 10 client.6177 handle_command_reply: tid=0 2024-09-10T10:02:07.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 1 client.6177 shutdown 2024-09-10T10:02:07.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 1 -- 172.21.15.110:0/2462528352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 msgr2=0x563e36959e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.724+0000 7f484b098000 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e36959e50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f483001c7b0 tx=0x7f483001ab10 comp rx=0 tx=0).stop 2024-09-10T10:02:07.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 1 -- 172.21.15.110:0/2462528352 shutdown_connections 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] conn(0x563e369ad280 0x563e369af670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e36960230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e3695fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f482a000640 0 client.6177 ms_handle_reset on v2:172.21.15.145:6833/1773965791 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 1 --2- 172.21.15.110:0/2462528352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e36959e50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 1 -- 172.21.15.110:0/2462528352 >> 172.21.15.110:0/2462528352 conn(0x563e36955290 msgr2=0x563e36953d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 1 -- 172.21.15.110:0/2462528352 shutdown_connections 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 1 -- 172.21.15.110:0/2462528352 wait complete. 2024-09-10T10:02:07.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.728+0000 7f484b098000 20 client.6177 trim_cache size 0 max 16384 2024-09-10T10:02:07.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f484b098000 1 Processor -- start 2024-09-10T10:02:07.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f484b098000 1 -- start start 2024-09-10T10:02:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f484b098000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e3699dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f484b098000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699eb20 0x563e3699e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f484b098000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699f440 0x563e369af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f484b098000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4840203050 con 0x563e3699f440 2024-09-10T10:02:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f484b098000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f48402023e0 con 0x563e3695a390 2024-09-10T10:02:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f484b098000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4840202a30 con 0x563e3699eb20 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699f440 0x563e369af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699f440 0x563e369af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44212/0 (socket says 172.21.15.110:44212) 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845c00640 1 -- 172.21.15.110:0/2826418239 learned_addr learned my addr 172.21.15.110:0/2826418239 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845200640 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e3699dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f483fe00640 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699eb20 0x563e3699e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845c00640 1 -- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699eb20 msgr2=0x563e3699e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845c00640 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699eb20 0x563e3699e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845c00640 1 -- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 msgr2=0x563e3699dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845c00640 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e3699dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845c00640 1 -- 172.21.15.110:0/2826418239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3699f270 con 0x563e3699f440 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f4845200640 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e3699dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.744+0000 7f483fe00640 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699eb20 0x563e3699e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f4845c00640 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699f440 0x563e369af670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f483805a4b0 tx=0x7f4838020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce145e65fe34df5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f4834c00640 1 -- 172.21.15.110:0/2826418239 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838054240 con 0x563e3699f440 2024-09-10T10:02:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f4834c00640 1 -- 172.21.15.110:0/2826418239 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48380525b0 con 0x563e3699f440 2024-09-10T10:02:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f4834c00640 1 -- 172.21.15.110:0/2826418239 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483805e8b0 con 0x563e3699f440 2024-09-10T10:02:07.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- 172.21.15.110:0/2826418239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699f440 msgr2=0x563e369af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699f440 0x563e369af670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f483805a4b0 tx=0x7f4838020000 comp rx=0 tx=0).stop 2024-09-10T10:02:07.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- 172.21.15.110:0/2826418239 shutdown_connections 2024-09-10T10:02:07.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699f440 0x563e369af670 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699eb20 0x563e3699e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 --2- 172.21.15.110:0/2826418239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e3699dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- 172.21.15.110:0/2826418239 >> 172.21.15.110:0/2826418239 conn(0x563e36955290 msgr2=0x563e36959650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- 172.21.15.110:0/2826418239 shutdown_connections 2024-09-10T10:02:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- 172.21.15.110:0/2826418239 wait complete. 2024-09-10T10:02:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- start start 2024-09-10T10:02:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 Processor -- start 2024-09-10T10:02:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e369b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e369cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e369cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563e367a2f80 con 0x563e3699eb20 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563e3675bca0 con 0x563e3695a390 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x563e367a82a0 con 0x563e3699f440 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 10 client.0 _ll_register_callbacks cb 0x7f48300ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e369cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e369cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44224/0 (socket says 172.21.15.110:44224) 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 -- 172.21.15.110:0/1773509861 learned_addr learned my addr 172.21.15.110:0/1773509861 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f4845200640 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e369b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f4845c00640 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e369cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 -- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 msgr2=0x563e369cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e369cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 -- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 msgr2=0x563e369b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e369b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 -- 172.21.15.110:0/1773509861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3672afd0 con 0x563e3699eb20 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f4845200640 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e369b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f483fe00640 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e369cd960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f482c00ca30 tx=0x7f482c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e8511894a962f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f482a000640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f482a000640 1 -- 172.21.15.110:0/1773509861 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c016940 con 0x563e3699eb20 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- 172.21.15.110:0/1773509861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48380525b0 con 0x563e3699eb20 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f482a000640 1 -- 172.21.15.110:0/1773509861 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f482c016ae0 con 0x563e3699eb20 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f482a000640 1 -- 172.21.15.110:0/1773509861 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f482c02a050 con 0x563e3699eb20 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 10 client.6183 fetch_fsmap learned FSMap version 45 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.748+0000 7f484b098000 1 -- 172.21.15.110:0/1773509861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x563e36826750 con 0x563e3699eb20 2024-09-10T10:02:07.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f482a000640 1 -- 172.21.15.110:0/1773509861 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48380525b0 con 0x563e3699eb20 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f482a000640 1 -- 172.21.15.110:0/1773509861 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f482c02d020 con 0x563e3699eb20 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 10 client.6183 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 10 client.6183 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 20 client.6183 populate_metadata read hostname 'smithi110' 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] conn(0x563e369ce3e0 0x563e36a11de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 4 client.6183 mds_command: new command op to 6033 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 -- 172.21.15.110:0/1773509861 --> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] -- command(tid 0: {"prefix": "status"}) -- 0x563e367b13a0 con 0x563e369ce3e0 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4845200640 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] conn(0x563e369ce3e0 0x563e36a11de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4845200640 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] conn(0x563e369ce3e0 0x563e36a11de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99cd5bf68ba9a08 server_cookie=a083f2aaeb54169f in_seq=0 out_seq=0 2024-09-10T10:02:07.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f482a000640 10 client.6183 ms_handle_connect on v2:172.21.15.145:6833/1773965791 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f482a000640 1 -- 172.21.15.110:0/1773509861 <== mds.0 v2:172.21.15.145:6833/1773965791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563e367b13a0 con 0x563e369ce3e0 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f482a000640 10 client.6183 handle_command_reply: tid=0 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 client.6183 shutdown 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 -- 172.21.15.110:0/1773509861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 msgr2=0x563e369cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e369cd960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f482c00ca30 tx=0x7f482c00ceb0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 -- 172.21.15.110:0/1773509861 shutdown_connections 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x563e3699f440 0x563e369cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563e3699eb20 0x563e369cd960 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:6833/1773965791,v1:172.21.15.145:6835/1773965791] conn(0x563e369ce3e0 0x563e36a11de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 --2- 172.21.15.110:0/1773509861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563e3695a390 0x563e369b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f482a000640 0 client.6183 ms_handle_reset on v2:172.21.15.145:6833/1773965791 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 -- 172.21.15.110:0/1773509861 >> 172.21.15.110:0/1773509861 conn(0x563e36955290 msgr2=0x563e3699cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 -- 172.21.15.110:0/1773509861 shutdown_connections 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 1 -- 172.21.15.110:0/1773509861 wait complete. 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f484b098000 20 client.6183 trim_cache size 0 max 16384 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 6033, 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:02:07.762 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:02:07.763 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:02:07.763 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 10.365542425999999, 2024-09-10T10:02:07.763 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 45, 2024-09-10T10:02:07.763 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 71, 2024-09-10T10:02:07.763 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 69, 2024-09-10T10:02:07.763 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 15.219828112 2024-09-10T10:02:07.763 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 -- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4820067c20 msgr2=0x7f482006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4820067c20 0x7f482006a0e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f48300ad000 tx=0x7f48300a6000 comp rx=0 tx=0).stop 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 -- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b5430 msgr2=0x7f4840158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b5430 0x7f4840158160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f483800b990 tx=0x7f483800be50 comp rx=0 tx=0).stop 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 -- 172.21.15.110:0/3009433289 shutdown_connections 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4820067c20 0x7f482006a0e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b5430 0x7f4840158160 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400726e0 0x7f4840157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 --2- 172.21.15.110:0/3009433289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f484006efa0 0x7f4840142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 -- 172.21.15.110:0/3009433289 >> 172.21.15.110:0/3009433289 conn(0x7f4840062d10 msgr2=0x7f484015eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 -- 172.21.15.110:0/3009433289 shutdown_connections 2024-09-10T10:02:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.752+0000 7f4847c00640 1 -- 172.21.15.110:0/3009433289 wait complete. 2024-09-10T10:02:07.774 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:02:07.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.908+0000 7f53d5c00640 1 Processor -- start 2024-09-10T10:02:07.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.908+0000 7f53d5c00640 1 -- start start 2024-09-10T10:02:07.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53d5c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f53d0155970 0x7f53d00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53d5c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f53d006f1e0 0x7f53d00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53d5c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d0155d50 con 0x7f53d015c560 2024-09-10T10:02:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53d5c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f53d00b4570 con 0x7f53d006f1e0 2024-09-10T10:02:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53d5c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f53d00b34d0 con 0x7f53d0155970 2024-09-10T10:02:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53cf400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f53d006f1e0 0x7f53d00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53ce000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f53d015c560 0x7f53d006ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36908/0 (socket says 172.21.15.110:36908) 2024-09-10T10:02:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53ce000640 1 -- 172.21.15.110:0/2303706175 learned_addr learned my addr 172.21.15.110:0/2303706175 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53cea00640 1 --2- 172.21.15.110:0/2303706175 >> v2:172.21.15.145:3300/0 conn(0x7f53d0155970 0x7f53d00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53cf400640 1 -- 172.21.15.110:0/2303706175 >> v2:172.21.15.145:3300/0 conn(0x7f53d0155970 msgr2=0x7f53d00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:07.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53cf400640 1 --2- 172.21.15.110:0/2303706175 >> v2:172.21.15.145:3300/0 conn(0x7f53d0155970 0x7f53d00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53cf400640 1 -- 172.21.15.110:0/2303706175 >> v1:172.21.15.110:6789/0 conn(0x7f53d015c560 legacy=0x7f53d006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53cf400640 1 -- 172.21.15.110:0/2303706175 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d00b37b0 con 0x7f53d006f1e0 2024-09-10T10:02:07.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.912+0000 7f53cd600640 1 -- 172.21.15.110:0/2303706175 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2003756269 0 0) 0x7f53d0155d50 con 0x7f53d015c560 2024-09-10T10:02:07.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53cf400640 1 --2- 172.21.15.110:0/2303706175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d006f1e0 0x7f53d00ce420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f53d006cc30 tx=0x7f53c801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ba864932a4df2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53cd600640 1 -- 172.21.15.110:0/2303706175 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c80b2070 con 0x7f53d006f1e0 2024-09-10T10:02:07.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53cd600640 1 -- 172.21.15.110:0/2303706175 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c80a59c0 con 0x7f53d006f1e0 2024-09-10T10:02:07.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53cd600640 1 -- 172.21.15.110:0/2303706175 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c80a5d90 con 0x7f53d006f1e0 2024-09-10T10:02:07.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- 172.21.15.110:0/2303706175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d006f1e0 msgr2=0x7f53d00ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 --2- 172.21.15.110:0/2303706175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d006f1e0 0x7f53d00ce420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f53d006cc30 tx=0x7f53c801c900 comp rx=0 tx=0).stop 2024-09-10T10:02:07.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- 172.21.15.110:0/2303706175 shutdown_connections 2024-09-10T10:02:07.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 --2- 172.21.15.110:0/2303706175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d006f1e0 0x7f53d00ce420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 --2- 172.21.15.110:0/2303706175 >> v2:172.21.15.145:3300/0 conn(0x7f53d0155970 0x7f53d00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- 172.21.15.110:0/2303706175 >> 172.21.15.110:0/2303706175 conn(0x7f53d006bc50 msgr2=0x7f53d006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- 172.21.15.110:0/2303706175 shutdown_connections 2024-09-10T10:02:07.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- 172.21.15.110:0/2303706175 wait complete. 2024-09-10T10:02:07.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 Processor -- start 2024-09-10T10:02:07.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- start start 2024-09-10T10:02:07.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53d006f1e0 0x7f53d0164b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d0155970 0x7f53d0144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53d015c560 0x7f53d0144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53d00cf170 con 0x7f53d0155970 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f53d00cce00 con 0x7f53d006f1e0 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53d5c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f53d00cf2f0 con 0x7f53d015c560 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53ce000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d0155970 0x7f53d0144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53ce000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d0155970 0x7f53d0144920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44248/0 (socket says 172.21.15.110:44248) 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.916+0000 7f53ce000640 1 -- 172.21.15.110:0/53876271 learned_addr learned my addr 172.21.15.110:0/53876271 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53cea00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53d006f1e0 0x7f53d0164b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53cf400640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53d015c560 0x7f53d0144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53ce000640 1 -- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53d015c560 msgr2=0x7f53d0144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53ce000640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53d015c560 0x7f53d0144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53ce000640 1 -- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53d006f1e0 msgr2=0x7f53d0164b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53ce000640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53d006f1e0 0x7f53d0164b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53ce000640 1 -- 172.21.15.110:0/53876271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d0142930 con 0x7f53d0155970 2024-09-10T10:02:07.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53cea00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53d006f1e0 0x7f53d0164b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:07.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53ce000640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d0155970 0x7f53d0144920 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c480 tx=0x7f53bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53c6a00640 1 -- 172.21.15.110:0/53876271 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc018070 con 0x7f53d0155970 2024-09-10T10:02:07.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53c6a00640 1 -- 172.21.15.110:0/53876271 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53bc007500 con 0x7f53d0155970 2024-09-10T10:02:07.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53c6a00640 1 -- 172.21.15.110:0/53876271 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc004a30 con 0x7f53d0155970 2024-09-10T10:02:07.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d01616b0 con 0x7f53d0155970 2024-09-10T10:02:07.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d0142eb0 con 0x7f53d0155970 2024-09-10T10:02:07.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d01425b0 con 0x7f53d0155970 2024-09-10T10:02:07.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.920+0000 7f53c6a00640 1 -- 172.21.15.110:0/53876271 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f53bc004bd0 con 0x7f53d0155970 2024-09-10T10:02:07.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.924+0000 7f53c6a00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53b0067c20 0x7f53b006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.924+0000 7f53c6a00640 1 -- 172.21.15.110:0/53876271 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f53bc0854b0 con 0x7f53d0155970 2024-09-10T10:02:07.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.924+0000 7f53cea00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53b0067c20 0x7f53b006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.924+0000 7f53cea00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53b0067c20 0x7f53b006a0e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53c0007a70 tx=0x7f53c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:07.928+0000 7f53c6a00640 1 -- 172.21.15.110:0/53876271 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f53d01425b0 con 0x7f53d0155970 2024-09-10T10:02:08.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f53d00ce420 con 0x7f53d0155970 2024-09-10T10:02:08.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53c6a00640 1 -- 172.21.15.110:0/53876271 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7f53bc0554c0 con 0x7f53d0155970 2024-09-10T10:02:08.127 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53b0067c20 msgr2=0x7f53b006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53b0067c20 0x7f53b006a0e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53c0007a70 tx=0x7f53c0008040 comp rx=0 tx=0).stop 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d0155970 msgr2=0x7f53d0144920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d0155970 0x7f53d0144920 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c480 tx=0x7f53bc00c940 comp rx=0 tx=0).stop 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 shutdown_connections 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f53b0067c20 0x7f53b006a0e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53d015c560 0x7f53d0144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53d0155970 0x7f53d0144920 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 --2- 172.21.15.110:0/53876271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53d006f1e0 0x7f53d0164b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 >> 172.21.15.110:0/53876271 conn(0x7f53d006bc50 msgr2=0x7f53d006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 shutdown_connections 2024-09-10T10:02:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08.124+0000 7f53d5c00640 1 -- 172.21.15.110:0/53876271 wait complete. 2024-09-10T10:02:08.148 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:02:08.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:02:08.148 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:02:08.148 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:02:08.148 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:08.149 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:02:08.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:02:08.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:02:08.149 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:02:08.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:08.149 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:02:08.158 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:02:08.158 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:02:08.159 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42246sec preferred_lft 42246sec 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:02:08.160 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:08.161 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:02:08.161 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:02:08.161 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:02:08.161 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:08.161 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:02:08.161 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:02:08.161 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:02:08.161 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:08.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:08.164 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:02:08.222 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:02:08.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:08.224 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:02:08.279 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:08.280 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:02:08.281 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:02:08.281 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:02:08.281 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:02:08.281 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:02:08.281 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:02:08.281 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:02:08.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:02:08.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:02:08.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:08.418 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:02:08.473 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:02:08.473 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:02:08.474 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:02:08.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:08.474 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:02:08.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:02:08.536 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:08.584 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:02:08.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:02:08.585 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:08.632 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:02:08.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:08.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:02:08.743 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:02:08.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:08.743 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:08.763 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:02:08.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:02:08.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:02:08.871 INFO:teuthology.orchestra.run.smithi110.stdout:client5465 2024-09-10T10:02:08.871 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:08.871 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:08.880 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:02:09.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6438200640 1 Processor -- start 2024-09-10T10:02:09.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6438200640 1 -- start start 2024-09-10T10:02:09.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6438200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6430155970 0x7f64300cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6438200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643006f1e0 0x7f64300ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6438200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6430155d50 con 0x7f643015c560 2024-09-10T10:02:09.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6438200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f64300b4570 con 0x7f6430155970 2024-09-10T10:02:09.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6438200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f64300b34d0 con 0x7f643006f1e0 2024-09-10T10:02:09.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6435800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6430155970 0x7f64300cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.056+0000 7f6436200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643006f1e0 0x7f64300ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6435800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6430155970 0x7f64300cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:42732/0 (socket says 172.21.15.110:42732) 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6436200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643006f1e0 0x7f64300ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:38376/0 (socket says 172.21.15.110:38376) 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6435800640 1 -- 172.21.15.110:0/1774802907 learned_addr learned my addr 172.21.15.110:0/1774802907 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6436200640 1 -- 172.21.15.110:0/1774802907 >> v2:172.21.15.145:3300/0 conn(0x7f6430155970 msgr2=0x7f64300cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6436200640 1 --2- 172.21.15.110:0/1774802907 >> v2:172.21.15.145:3300/0 conn(0x7f6430155970 0x7f64300cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6436200640 1 -- 172.21.15.110:0/1774802907 >> v1:172.21.15.145:6789/0 conn(0x7f643015c560 legacy=0x7f643006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6436200640 1 -- 172.21.15.110:0/1774802907 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64300b37b0 con 0x7f643006f1e0 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f642fe00640 1 -- 172.21.15.110:0/1774802907 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 630543467 0 0) 0x7f6430155d50 con 0x7f643015c560 2024-09-10T10:02:09.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6435800640 1 --2- 172.21.15.110:0/1774802907 >> v2:172.21.15.145:3300/0 conn(0x7f6430155970 0x7f64300cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:09.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f6436200640 1 --2- 172.21.15.110:0/1774802907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643006f1e0 0x7f64300ce420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6428009d90 tx=0x7f642801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf055b2c7536f17e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f642fe00640 1 -- 172.21.15.110:0/1774802907 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64280b2070 con 0x7f643006f1e0 2024-09-10T10:02:09.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.060+0000 7f642fe00640 1 -- 172.21.15.110:0/1774802907 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64280a5a00 con 0x7f643006f1e0 2024-09-10T10:02:09.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f642fe00640 1 -- 172.21.15.110:0/1774802907 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64280a5dd0 con 0x7f643006f1e0 2024-09-10T10:02:09.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- 172.21.15.110:0/1774802907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643006f1e0 msgr2=0x7f64300ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 --2- 172.21.15.110:0/1774802907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643006f1e0 0x7f64300ce420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6428009d90 tx=0x7f642801c940 comp rx=0 tx=0).stop 2024-09-10T10:02:09.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- 172.21.15.110:0/1774802907 shutdown_connections 2024-09-10T10:02:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 --2- 172.21.15.110:0/1774802907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643006f1e0 0x7f64300ce420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 --2- 172.21.15.110:0/1774802907 >> v2:172.21.15.145:3300/0 conn(0x7f6430155970 0x7f64300cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- 172.21.15.110:0/1774802907 >> 172.21.15.110:0/1774802907 conn(0x7f643006bc50 msgr2=0x7f643006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- 172.21.15.110:0/1774802907 shutdown_connections 2024-09-10T10:02:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- 172.21.15.110:0/1774802907 wait complete. 2024-09-10T10:02:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 Processor -- start 2024-09-10T10:02:09.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- start start 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f643006f1e0 0x7f6430151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6430155970 0x7f643014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643015c560 0x7f643015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f64300cf170 con 0x7f643006f1e0 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f64300cce00 con 0x7f6430155970 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6438200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f64300cf2f0 con 0x7f643015c560 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6435800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f643006f1e0 0x7f6430151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6435800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f643006f1e0 0x7f6430151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55636/0 (socket says 172.21.15.110:55636) 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6435800640 1 -- 172.21.15.110:0/187216211 learned_addr learned my addr 172.21.15.110:0/187216211 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.064+0000 7f6434e00640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6430155970 0x7f643014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6436200640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643015c560 0x7f643015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6435800640 1 -- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643015c560 msgr2=0x7f643015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6435800640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643015c560 0x7f643015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6435800640 1 -- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6430155970 msgr2=0x7f643014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6435800640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6430155970 0x7f643014e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6435800640 1 -- 172.21.15.110:0/187216211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643014df50 con 0x7f643006f1e0 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6434e00640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6430155970 0x7f643014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6435800640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f643006f1e0 0x7f6430151d50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9d0 tx=0x7f642400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f642d600640 1 -- 172.21.15.110:0/187216211 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424017070 con 0x7f643006f1e0 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f642d600640 1 -- 172.21.15.110:0/187216211 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642400ca60 con 0x7f643006f1e0 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6430150ef0 con 0x7f643006f1e0 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f642d600640 1 -- 172.21.15.110:0/187216211 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642400ce10 con 0x7f643006f1e0 2024-09-10T10:02:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64301514b0 con 0x7f643006f1e0 2024-09-10T10:02:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64300ce420 con 0x7f643006f1e0 2024-09-10T10:02:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f642d600640 1 -- 172.21.15.110:0/187216211 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f642401a070 con 0x7f643006f1e0 2024-09-10T10:02:09.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f642d600640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6410067c20 0x7f641006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f642d600640 1 -- 172.21.15.110:0/187216211 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f6424086150 con 0x7f643006f1e0 2024-09-10T10:02:09.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.068+0000 7f6434e00640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6410067c20 0x7f641006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.072+0000 7f6434e00640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6410067c20 0x7f641006a0e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64300b1630 tx=0x7f641c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.076+0000 7f642d600640 1 -- 172.21.15.110:0/187216211 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6424056870 con 0x7f643006f1e0 2024-09-10T10:02:09.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f64300cf880 con 0x7f643006f1e0 2024-09-10T10:02:09.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f642d600640 1 -- 172.21.15.110:0/187216211 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7f6424056170 con 0x7f643006f1e0 2024-09-10T10:02:09.271 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6410067c20 msgr2=0x7f641006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6410067c20 0x7f641006a0e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64300b1630 tx=0x7f641c009040 comp rx=0 tx=0).stop 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f643006f1e0 msgr2=0x7f6430151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f643006f1e0 0x7f6430151d50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9d0 tx=0x7f642400ee90 comp rx=0 tx=0).stop 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 shutdown_connections 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f6410067c20 0x7f641006a0e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f643015c560 0x7f643015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6430155970 0x7f643014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 --2- 172.21.15.110:0/187216211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f643006f1e0 0x7f6430151d50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 >> 172.21.15.110:0/187216211 conn(0x7f643006bc50 msgr2=0x7f6430071580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:09.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 shutdown_connections 2024-09-10T10:02:09.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:09.268+0000 7f6438200640 1 -- 172.21.15.110:0/187216211 wait complete. 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:02:09.292 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:02:09.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:09.293 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:02:09.303 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:02:09.303 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:02:09.303 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:02:09.303 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:09.303 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:02:09.303 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:09.303 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:09.303 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42242sec preferred_lft 42242sec 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:02:09.304 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:02:09.305 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:09.306 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:02:09.306 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:02:09.306 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:02:09.306 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:09.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:09.307 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:02:09.374 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:02:09.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:09.377 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:02:09.431 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:02:09.431 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:09.432 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:02:09.433 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:02:09.433 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:02:09.433 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:02:09.433 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:02:09.433 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:02:09.433 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:02:09.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:02:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:02:09.570 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:02:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:02:09.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:09.577 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:02:09.587 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:02:09.588 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:02:09.588 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:02:09.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:09.588 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:02:09.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:02:09.654 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:09.706 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:02:09.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:02:09.706 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:09.762 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:02:09.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:09.762 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:09.851 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:09.851 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:02:09.865 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:09.865 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:02:09.865 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:02:09.865 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:02:09.865 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:02:09.869 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:02:09.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:09.869 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:09.888 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:02:09.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:02:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:02:09.984 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:02:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:02:09.990 INFO:teuthology.orchestra.run.smithi145.stdout:client5905 2024-09-10T10:02:09.991 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:09.991 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:10.047 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:02:10.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.192+0000 7f3b8be00640 1 Processor -- start 2024-09-10T10:02:10.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.192+0000 7f3b8be00640 1 -- start start 2024-09-10T10:02:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b8be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3b840cfe30 0x7f3b84071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b8be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3b840cd510 0x7f3b840cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b8be00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b8406d2d0 con 0x7f3b8406e090 2024-09-10T10:02:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b8be00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3b8415d9a0 con 0x7f3b840cfe30 2024-09-10T10:02:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b8be00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3b8406d5b0 con 0x7f3b840cd510 2024-09-10T10:02:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b89400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3b840cfe30 0x7f3b84071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b89400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3b840cfe30 0x7f3b84071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55648/0 (socket says 172.21.15.110:55648) 2024-09-10T10:02:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b89400640 1 -- 172.21.15.110:0/1089044697 learned_addr learned my addr 172.21.15.110:0/1089044697 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b88a00640 1 --2- 172.21.15.110:0/1089044697 >> v2:172.21.15.145:3300/0 conn(0x7f3b840cd510 0x7f3b840cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b89400640 1 -- 172.21.15.110:0/1089044697 >> v2:172.21.15.145:3300/0 conn(0x7f3b840cd510 msgr2=0x7f3b840cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b89400640 1 --2- 172.21.15.110:0/1089044697 >> v2:172.21.15.145:3300/0 conn(0x7f3b840cd510 0x7f3b840cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b89400640 1 -- 172.21.15.110:0/1089044697 >> v1:172.21.15.145:6789/0 conn(0x7f3b8406e090 legacy=0x7f3b8406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b89400640 1 -- 172.21.15.110:0/1089044697 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8406d890 con 0x7f3b840cfe30 2024-09-10T10:02:10.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.196+0000 7f3b89400640 1 --2- 172.21.15.110:0/1089044697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84071a40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3b840ce0f0 tx=0x7f3b7801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01006c6781f0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b83e00640 1 -- 172.21.15.110:0/1089044697 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780b1070 con 0x7f3b840cfe30 2024-09-10T10:02:10.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b83e00640 1 -- 172.21.15.110:0/1089044697 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b780ac600 con 0x7f3b840cfe30 2024-09-10T10:02:10.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b83e00640 1 -- 172.21.15.110:0/1089044697 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780ac9d0 con 0x7f3b840cfe30 2024-09-10T10:02:10.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- 172.21.15.110:0/1089044697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 msgr2=0x7f3b84071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 --2- 172.21.15.110:0/1089044697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84071a40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3b840ce0f0 tx=0x7f3b7801c490 comp rx=0 tx=0).stop 2024-09-10T10:02:10.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- 172.21.15.110:0/1089044697 shutdown_connections 2024-09-10T10:02:10.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 --2- 172.21.15.110:0/1089044697 >> v2:172.21.15.145:3300/0 conn(0x7f3b840cd510 0x7f3b840cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 --2- 172.21.15.110:0/1089044697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84071a40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- 172.21.15.110:0/1089044697 >> 172.21.15.110:0/1089044697 conn(0x7f3b8406bc50 msgr2=0x7f3b8406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:10.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- 172.21.15.110:0/1089044697 shutdown_connections 2024-09-10T10:02:10.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- 172.21.15.110:0/1089044697 wait complete. 2024-09-10T10:02:10.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 Processor -- start 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- start start 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b8406e090 0x7f3b841572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b840cd510 0x7f3b84152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b84072190 con 0x7f3b840cfe30 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8415d6c0 con 0x7f3b840cd510 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b8be00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3b84169180 con 0x7f3b8406e090 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b88a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b840cd510 0x7f3b84152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b89e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55654/0 (socket says 172.21.15.110:55654) 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b89e00640 1 -- 172.21.15.110:0/775304820 learned_addr learned my addr 172.21.15.110:0/775304820 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b89400640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b8406e090 0x7f3b841572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b88a00640 1 -- 172.21.15.110:0/775304820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b8406e090 msgr2=0x7f3b841572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b88a00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b8406e090 0x7f3b841572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b88a00640 1 -- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 msgr2=0x7f3b84152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b88a00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b88a00640 1 -- 172.21.15.110:0/775304820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b84157d40 con 0x7f3b840cd510 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.200+0000 7f3b89e00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b88a00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b840cd510 0x7f3b84152610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000e870 tx=0x7f3b7000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b81600640 1 -- 172.21.15.110:0/775304820 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70017070 con 0x7f3b840cd510 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b81600640 1 -- 172.21.15.110:0/775304820 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7000ee60 con 0x7f3b840cd510 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b84158020 con 0x7f3b840cd510 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b81600640 1 -- 172.21.15.110:0/775304820 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7000cb60 con 0x7f3b840cd510 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b84150b50 con 0x7f3b840cd510 2024-09-10T10:02:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b841584d0 con 0x7f3b840cd510 2024-09-10T10:02:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b81600640 1 -- 172.21.15.110:0/775304820 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f3b7001a070 con 0x7f3b840cd510 2024-09-10T10:02:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b81600640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3b64067c20 0x7f3b6406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b81600640 1 -- 172.21.15.110:0/775304820 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f3b700863a0 con 0x7f3b840cd510 2024-09-10T10:02:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.204+0000 7f3b89400640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3b64067c20 0x7f3b6406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.208+0000 7f3b89400640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3b64067c20 0x7f3b6406a0e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3b780ae000 tx=0x7f3b780a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.208+0000 7f3b81600640 1 -- 172.21.15.110:0/775304820 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3b700563e0 con 0x7f3b840cd510 2024-09-10T10:02:10.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.400+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f3b840b25b0 con 0x7f3b840cd510 2024-09-10T10:02:10.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b81600640 1 -- 172.21.15.110:0/775304820 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v46) ==== 136+0+0 (secure 0 0 0) 0x7f3b7005a6e0 con 0x7f3b840cd510 2024-09-10T10:02:10.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3b64067c20 msgr2=0x7f3b6406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3b64067c20 0x7f3b6406a0e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3b780ae000 tx=0x7f3b780a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:10.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b840cd510 msgr2=0x7f3b84152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b840cd510 0x7f3b84152610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000e870 tx=0x7f3b7000ed30 comp rx=0 tx=0).stop 2024-09-10T10:02:10.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 shutdown_connections 2024-09-10T10:02:10.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f3b64067c20 0x7f3b6406a0e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b840cfe30 0x7f3b84152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b840cd510 0x7f3b84152610 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 --2- 172.21.15.110:0/775304820 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b8406e090 0x7f3b841572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.792+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 >> 172.21.15.110:0/775304820 conn(0x7f3b8406bc50 msgr2=0x7f3b8415ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:10.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.796+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 shutdown_connections 2024-09-10T10:02:10.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.796+0000 7f3b8be00640 1 -- 172.21.15.110:0/775304820 wait complete. 2024-09-10T10:02:10.819 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:02:10.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.952+0000 7efeafa00640 1 Processor -- start 2024-09-10T10:02:10.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.952+0000 7efeafa00640 1 -- start start 2024-09-10T10:02:10.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.952+0000 7efeafa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea80cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeafa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efea80cff70 0x7efea80cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeafa00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efea815d4d0 con 0x7efea80cfb90 2024-09-10T10:02:10.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeafa00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7efea806d130 con 0x7efea80cff70 2024-09-10T10:02:10.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeafa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efea815d7b0 con 0x7efea80b3f40 2024-09-10T10:02:10.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efead000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7efea80cfb90 0x7efea815ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45034/0 (socket says 172.21.15.110:45034) 2024-09-10T10:02:10.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efead000640 1 -- 172.21.15.110:0/85621193 learned_addr learned my addr 172.21.15.110:0/85621193 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:10.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeada00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efea80cff70 0x7efea80cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efea7e00640 1 --2- 172.21.15.110:0/85621193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea80cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeada00640 1 -- 172.21.15.110:0/85621193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 msgr2=0x7efea80cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:10.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeada00640 1 --2- 172.21.15.110:0/85621193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea80cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeada00640 1 -- 172.21.15.110:0/85621193 >> v1:172.21.15.110:6789/0 conn(0x7efea80cfb90 legacy=0x7efea815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeada00640 1 -- 172.21.15.110:0/85621193 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea806d380 con 0x7efea80cff70 2024-09-10T10:02:10.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efea7400640 1 -- 172.21.15.110:0/85621193 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405918048 0 0) 0x7efea815d4d0 con 0x7efea80cfb90 2024-09-10T10:02:10.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efeada00640 1 --2- 172.21.15.110:0/85621193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea80cdae0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efea0009d90 tx=0x7efea001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f442ca34c8de9ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efea7400640 1 -- 172.21.15.110:0/85621193 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00b2070 con 0x7efea80cff70 2024-09-10T10:02:10.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efea7400640 1 -- 172.21.15.110:0/85621193 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea00a57a0 con 0x7efea80cff70 2024-09-10T10:02:10.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.956+0000 7efea7400640 1 -- 172.21.15.110:0/85621193 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00a5b70 con 0x7efea80cff70 2024-09-10T10:02:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- 172.21.15.110:0/85621193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 msgr2=0x7efea80cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 --2- 172.21.15.110:0/85621193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea80cdae0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efea0009d90 tx=0x7efea001c8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- 172.21.15.110:0/85621193 shutdown_connections 2024-09-10T10:02:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 --2- 172.21.15.110:0/85621193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea80cdae0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 --2- 172.21.15.110:0/85621193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea80cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- 172.21.15.110:0/85621193 >> 172.21.15.110:0/85621193 conn(0x7efea8062d10 msgr2=0x7efea806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- 172.21.15.110:0/85621193 shutdown_connections 2024-09-10T10:02:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- 172.21.15.110:0/85621193 wait complete. 2024-09-10T10:02:10.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 Processor -- start 2024-09-10T10:02:10.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- start start 2024-09-10T10:02:10.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea8159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea80cfb90 0x7efea8152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efea806e720 con 0x7efea80cff70 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efea806e5a0 con 0x7efea80cfb90 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeafa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efea806e8a0 con 0x7efea80b3f40 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeada00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeada00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea8156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55672/0 (socket says 172.21.15.110:55672) 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeada00640 1 -- 172.21.15.110:0/3317891471 learned_addr learned my addr 172.21.15.110:0/3317891471 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efea7e00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea80cfb90 0x7efea8152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efead000640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea8159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeada00640 1 -- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 msgr2=0x7efea8159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeada00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea8159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeada00640 1 -- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea80cfb90 msgr2=0x7efea8152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeada00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea80cfb90 0x7efea8152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efeada00640 1 -- 172.21.15.110:0/3317891471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea815b1b0 con 0x7efea80cff70 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efea7e00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea80cfb90 0x7efea8152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.960+0000 7efead000640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea8159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efeada00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea8156290 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7efea00b0040 tx=0x7efea0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efea4c00640 1 -- 172.21.15.110:0/3317891471 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00b2050 con 0x7efea80cff70 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efea4c00640 1 -- 172.21.15.110:0/3317891471 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea00b3040 con 0x7efea80cff70 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efea4c00640 1 -- 172.21.15.110:0/3317891471 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00ad650 con 0x7efea80cff70 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea815b490 con 0x7efea80cff70 2024-09-10T10:02:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea815ba50 con 0x7efea80cff70 2024-09-10T10:02:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea815ca10 con 0x7efea80cff70 2024-09-10T10:02:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efea4c00640 1 -- 172.21.15.110:0/3317891471 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7efea00c0020 con 0x7efea80cff70 2024-09-10T10:02:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efea4c00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe88067c20 0x7efe8806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efea4c00640 1 -- 172.21.15.110:0/3317891471 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7efea0120aa0 con 0x7efea80cff70 2024-09-10T10:02:10.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.964+0000 7efead000640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe88067c20 0x7efe8806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.968+0000 7efead000640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe88067c20 0x7efe8806a0e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe98007fd0 tx=0x7efe98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:10.968+0000 7efea4c00640 1 -- 172.21.15.110:0/3317891471 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efea0016560 con 0x7efea80cff70 2024-09-10T10:02:11.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.148+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7efea80b3a50 con 0x7efe88067c20 2024-09-10T10:02:11.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.148+0000 7efea4c00640 1 -- 172.21.15.110:0/3317891471 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7efea80b3a50 con 0x7efe88067c20 2024-09-10T10:02:11.152 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:02:11.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:11.152 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:02:11.152 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:11.152 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:02:11.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe88067c20 msgr2=0x7efe8806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe88067c20 0x7efe8806a0e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe98007fd0 tx=0x7efe98009040 comp rx=0 tx=0).stop 2024-09-10T10:02:11.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 msgr2=0x7efea8156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea8156290 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7efea00b0040 tx=0x7efea0019680 comp rx=0 tx=0).stop 2024-09-10T10:02:11.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 shutdown_connections 2024-09-10T10:02:11.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7efe88067c20 0x7efe8806a0e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea80cff70 0x7efea8156290 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea80cfb90 0x7efea8152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 --2- 172.21.15.110:0/3317891471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea80b3f40 0x7efea8159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 >> 172.21.15.110:0/3317891471 conn(0x7efea8062d10 msgr2=0x7efea80cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:11.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 shutdown_connections 2024-09-10T10:02:11.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.152+0000 7efeafa00640 1 -- 172.21.15.110:0/3317891471 wait complete. 2024-09-10T10:02:11.174 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:02:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.308+0000 7f90bbe00640 1 Processor -- start 2024-09-10T10:02:11.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.308+0000 7f90bbe00640 1 -- start start 2024-09-10T10:02:11.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90bbe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f90b415b330 0x7f90b406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b40ce4d0 0x7f90b406e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90bbe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f90b40b1c60 0x7f90b406e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90bbe00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f90b40b5570 con 0x7f90b40b1c60 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90bbe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f90b40b4960 con 0x7f90b415b330 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90bbe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f90b40b4c40 con 0x7f90b40ce4d0 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f90b40b1c60 0x7f90b406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f90b40b1c60 0x7f90b406e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55680/0 (socket says 172.21.15.110:55680) 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9e00640 1 -- 172.21.15.110:0/1505655051 learned_addr learned my addr 172.21.15.110:0/1505655051 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9400640 1 --2- 172.21.15.110:0/1505655051 >> v2:172.21.15.145:3300/0 conn(0x7f90b415b330 0x7f90b406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b8a00640 1 --2- 172.21.15.110:0/1505655051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b40ce4d0 0x7f90b406e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9e00640 1 -- 172.21.15.110:0/1505655051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b40ce4d0 msgr2=0x7f90b406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:11.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9e00640 1 --2- 172.21.15.110:0/1505655051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b40ce4d0 0x7f90b406e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9e00640 1 -- 172.21.15.110:0/1505655051 >> v2:172.21.15.145:3300/0 conn(0x7f90b415b330 msgr2=0x7f90b406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:11.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9e00640 1 --2- 172.21.15.110:0/1505655051 >> v2:172.21.15.145:3300/0 conn(0x7f90b415b330 0x7f90b406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9e00640 1 -- 172.21.15.110:0/1505655051 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b40b4f20 con 0x7f90b40b1c60 2024-09-10T10:02:11.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.312+0000 7f90b9400640 1 --2- 172.21.15.110:0/1505655051 >> v2:172.21.15.145:3300/0 conn(0x7f90b415b330 0x7f90b406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:11.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90b9e00640 1 --2- 172.21.15.110:0/1505655051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40b1c60 0x7f90b406e680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f90b406eca0 tx=0x7f90ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5d466101fa5bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:11.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90b3e00640 1 -- 172.21.15.110:0/1505655051 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0ab5f0 con 0x7f90b40b1c60 2024-09-10T10:02:11.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90b3e00640 1 -- 172.21.15.110:0/1505655051 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac0b0070 con 0x7f90b40b1c60 2024-09-10T10:02:11.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90b3e00640 1 -- 172.21.15.110:0/1505655051 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0ab9b0 con 0x7f90b40b1c60 2024-09-10T10:02:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 -- 172.21.15.110:0/1505655051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40b1c60 msgr2=0x7f90b406e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 --2- 172.21.15.110:0/1505655051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40b1c60 0x7f90b406e680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f90b406eca0 tx=0x7f90ac01c520 comp rx=0 tx=0).stop 2024-09-10T10:02:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 -- 172.21.15.110:0/1505655051 shutdown_connections 2024-09-10T10:02:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 --2- 172.21.15.110:0/1505655051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40b1c60 0x7f90b406e680 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 --2- 172.21.15.110:0/1505655051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b40ce4d0 0x7f90b406e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 --2- 172.21.15.110:0/1505655051 >> v2:172.21.15.145:3300/0 conn(0x7f90b415b330 0x7f90b406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 -- 172.21.15.110:0/1505655051 >> 172.21.15.110:0/1505655051 conn(0x7f90b4062e40 msgr2=0x7f90b406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:11.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 -- 172.21.15.110:0/1505655051 shutdown_connections 2024-09-10T10:02:11.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 -- 172.21.15.110:0/1505655051 wait complete. 2024-09-10T10:02:11.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 Processor -- start 2024-09-10T10:02:11.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.316+0000 7f90bbe00640 1 -- start start 2024-09-10T10:02:11.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90b40b1c60 0x7f90b4158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b415b330 0x7f90b4150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b415da40 0x7f90b40722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90b415fae0 con 0x7f90b415da40 2024-09-10T10:02:11.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f90b415f960 con 0x7f90b40b1c60 2024-09-10T10:02:11.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f90b415fc60 con 0x7f90b415b330 2024-09-10T10:02:11.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b415da40 0x7f90b40722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b415da40 0x7f90b40722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55692/0 (socket says 172.21.15.110:55692) 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 -- 172.21.15.110:0/3417407459 learned_addr learned my addr 172.21.15.110:0/3417407459 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b8a00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b415b330 0x7f90b4150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9400640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90b40b1c60 0x7f90b4158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 -- 172.21.15.110:0/3417407459 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b415b330 msgr2=0x7f90b4150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b415b330 0x7f90b4150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 -- 172.21.15.110:0/3417407459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90b40b1c60 msgr2=0x7f90b4158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90b40b1c60 0x7f90b4158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 -- 172.21.15.110:0/3417407459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b415c840 con 0x7f90b415da40 2024-09-10T10:02:11.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b9e00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b415da40 0x7f90b40722e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0ad040 tx=0x7f90ac01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:11.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b1600640 1 -- 172.21.15.110:0/3417407459 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0be3f0 con 0x7f90b415da40 2024-09-10T10:02:11.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b1600640 1 -- 172.21.15.110:0/3417407459 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac0be590 con 0x7f90b415da40 2024-09-10T10:02:11.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b1600640 1 -- 172.21.15.110:0/3417407459 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0be940 con 0x7f90b415da40 2024-09-10T10:02:11.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b415cf60 con 0x7f90b415da40 2024-09-10T10:02:11.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b415c4a0 con 0x7f90b415da40 2024-09-10T10:02:11.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b406c6a0 con 0x7f90b415da40 2024-09-10T10:02:11.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.320+0000 7f90b1600640 1 -- 172.21.15.110:0/3417407459 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f90ac0b43f0 con 0x7f90b415da40 2024-09-10T10:02:11.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.324+0000 7f90b1600640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9094067c20 0x7f909406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.324+0000 7f90b1600640 1 -- 172.21.15.110:0/3417407459 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f90ac1203e0 con 0x7f90b415da40 2024-09-10T10:02:11.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.324+0000 7f90b9400640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9094067c20 0x7f909406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.324+0000 7f90b9400640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9094067c20 0x7f909406a0e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f90b4072ed0 tx=0x7f90a8009870 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:11.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.328+0000 7f90b1600640 1 -- 172.21.15.110:0/3417407459 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f90ac016110 con 0x7f90b415da40 2024-09-10T10:02:11.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.488+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f90b41540e0 con 0x7f90b415da40 2024-09-10T10:02:11.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.488+0000 7f90b1600640 1 -- 172.21.15.110:0/3417407459 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f90ac0f0500 con 0x7f90b415da40 2024-09-10T10:02:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9094067c20 msgr2=0x7f909406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9094067c20 0x7f909406a0e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f90b4072ed0 tx=0x7f90a8009870 comp rx=0 tx=0).stop 2024-09-10T10:02:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b415da40 msgr2=0x7f90b40722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b415da40 0x7f90b40722e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0ad040 tx=0x7f90ac01ad60 comp rx=0 tx=0).stop 2024-09-10T10:02:11.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 shutdown_connections 2024-09-10T10:02:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f9094067c20 0x7f909406a0e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b415da40 0x7f90b40722e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90b415b330 0x7f90b4150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 --2- 172.21.15.110:0/3417407459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90b40b1c60 0x7f90b4158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 >> 172.21.15.110:0/3417407459 conn(0x7f90b4062e40 msgr2=0x7f90b41548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 shutdown_connections 2024-09-10T10:02:11.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.492+0000 7f90bbe00640 1 -- 172.21.15.110:0/3417407459 wait complete. 2024-09-10T10:02:11.513 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:11.513 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:11.522 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:11.522 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:11.577 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:02:11.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8043e00640 1 Processor -- start 2024-09-10T10:02:11.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8043e00640 1 -- start start 2024-09-10T10:02:11.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8043e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f803c0cf2e0 0x7f803c06efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8043e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c15fda0 con 0x7f803c0726e0 2024-09-10T10:02:11.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8043e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c0ce520 con 0x7f803c0b5600 2024-09-10T10:02:11.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8043e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f803c0cee50 con 0x7f803c0cf2e0 2024-09-10T10:02:11.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f803c0726e0 0x7f803c0b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45036/0 (socket says 172.21.15.110:45036) 2024-09-10T10:02:11.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041400640 1 -- 172.21.15.110:0/1950593246 learned_addr learned my addr 172.21.15.110:0/1950593246 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:11.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531939296 0 0) 0x7f803c15fda0 con 0x7f803c0726e0 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8024003530 con 0x7f803c0726e0 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041e00640 1 --2- 172.21.15.110:0/1950593246 >> v2:172.21.15.145:3300/0 conn(0x7f803c0cf2e0 0x7f803c06efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3112742317 0 0) 0x7f8024003530 con 0x7f803c0726e0 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 >> v2:172.21.15.145:3300/0 conn(0x7f803c0cf2e0 msgr2=0x7f803c06efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 --2- 172.21.15.110:0/1950593246 >> v2:172.21.15.145:3300/0 conn(0x7f803c0cf2e0 0x7f803c06efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 >> v1:172.21.15.145:6789/0 conn(0x7f803c0b5600 legacy=0x7f803c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c0ce800 con 0x7f803c0726e0 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f8030001680 con 0x7f803c0726e0 2024-09-10T10:02:11.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041e00640 1 --2- 172.21.15.110:0/1950593246 >> v2:172.21.15.145:3300/0 conn(0x7f803c0cf2e0 0x7f803c06efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 >> v1:172.21.15.110:6789/0 conn(0x7f803c0726e0 legacy=0x7f803c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240039c0 0x7f8024003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8024004810 0x7f8024004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80240051d0 0x7f802400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0cee50 con 0x7f80240051d0 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f802400c120 con 0x7f80240039c0 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f802400c320 con 0x7f8024004810 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041e00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80240051d0 0x7f802400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041400640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240039c0 0x7f8024003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8040a00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8024004810 0x7f8024004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041e00640 1 -- 172.21.15.110:0/1950593246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8024004810 msgr2=0x7f8024004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:11.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041e00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8024004810 0x7f8024004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041e00640 1 -- 172.21.15.110:0/1950593246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240039c0 msgr2=0x7f8024003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:11.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041e00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240039c0 0x7f8024003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.756+0000 7f8041e00640 1 -- 172.21.15.110:0/1950593246 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8024003530 con 0x7f80240051d0 2024-09-10T10:02:11.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8041e00640 1 -- 172.21.15.110:0/1950593246 reap_dead start 2024-09-10T10:02:11.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8041e00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80240051d0 0x7f802400b9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f80340073e0 tx=0x7f803401cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c122e83130eb87c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:11.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034009c80 con 0x7f80240051d0 2024-09-10T10:02:11.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80340a5e40 con 0x7f80240051d0 2024-09-10T10:02:11.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f803be00640 1 -- 172.21.15.110:0/1950593246 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80340ad660 con 0x7f80240051d0 2024-09-10T10:02:11.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8043e00640 1 -- 172.21.15.110:0/1950593246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80240051d0 msgr2=0x7f802400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8043e00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80240051d0 0x7f802400b9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f80340073e0 tx=0x7f803401cf40 comp rx=0 tx=0).stop 2024-09-10T10:02:11.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8043e00640 1 -- 172.21.15.110:0/1950593246 shutdown_connections 2024-09-10T10:02:11.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8043e00640 1 --2- 172.21.15.110:0/1950593246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80240051d0 0x7f802400b9d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8043e00640 1 -- 172.21.15.110:0/1950593246 >> 172.21.15.110:0/1950593246 conn(0x7f803c062d10 msgr2=0x7f803c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:11.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8043e00640 1 -- 172.21.15.110:0/1950593246 shutdown_connections 2024-09-10T10:02:11.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.760+0000 7f8043e00640 1 -- 172.21.15.110:0/1950593246 wait complete. 2024-09-10T10:02:11.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 Processor -- start 2024-09-10T10:02:11.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 -- start start 2024-09-10T10:02:11.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240051d0 0x7f803c154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f803c14e840 0x7f803c14e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f803c14fe10 0x7f803c1502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f803c073270 con 0x7f803c14fe10 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0cf750 con 0x7f80240051d0 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f803c072ac0 con 0x7f803c14e840 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f803c14fe10 0x7f803c1502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f803c14fe10 0x7f803c1502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55714/0 (socket says 172.21.15.110:55714) 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 -- 172.21.15.110:0/1297241334 learned_addr learned my addr 172.21.15.110:0/1297241334 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041400640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240051d0 0x7f803c154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8040a00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f803c14e840 0x7f803c14e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 -- 172.21.15.110:0/1297241334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f803c14e840 msgr2=0x7f803c14e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f803c14e840 0x7f803c14e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 -- 172.21.15.110:0/1297241334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240051d0 msgr2=0x7f803c154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240051d0 0x7f803c154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 -- 172.21.15.110:0/1297241334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c14ed70 con 0x7f803c14fe10 2024-09-10T10:02:11.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8041e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f803c14fe10 0x7f803c1502c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f803401cd70 tx=0x7f8034019ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:11.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8039600640 1 -- 172.21.15.110:0/1297241334 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034009c80 con 0x7f803c14fe10 2024-09-10T10:02:11.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8039600640 1 -- 172.21.15.110:0/1297241334 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80340ac5f0 con 0x7f803c14fe10 2024-09-10T10:02:11.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c14f050 con 0x7f803c14fe10 2024-09-10T10:02:11.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8039600640 1 -- 172.21.15.110:0/1297241334 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80340ac9a0 con 0x7f803c14fe10 2024-09-10T10:02:11.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.764+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c14f610 con 0x7f803c14fe10 2024-09-10T10:02:11.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.768+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f803c15f950 con 0x7f803c14fe10 2024-09-10T10:02:11.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.768+0000 7f8039600640 1 -- 172.21.15.110:0/1297241334 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f80340bf060 con 0x7f803c14fe10 2024-09-10T10:02:11.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.768+0000 7f8039600640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f801c067c50 0x7f801c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:11.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.768+0000 7f8039600640 1 -- 172.21.15.110:0/1297241334 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f8034120720 con 0x7f803c14fe10 2024-09-10T10:02:11.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.768+0000 7f8041400640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f801c067c50 0x7f801c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:11.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.768+0000 7f8041400640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f801c067c50 0x7f801c06a110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8030001060 tx=0x7f8030007900 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:11.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.772+0000 7f8039600640 1 -- 172.21.15.110:0/1297241334 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f80340b3070 con 0x7f803c14fe10 2024-09-10T10:02:11.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.944+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f803c06e740 con 0x7f803c14fe10 2024-09-10T10:02:11.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.944+0000 7f8039600640 1 -- 172.21.15.110:0/1297241334 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f8034016c50 con 0x7f803c14fe10 2024-09-10T10:02:11.948 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:02:11.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f801c067c50 msgr2=0x7f801c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f801c067c50 0x7f801c06a110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8030001060 tx=0x7f8030007900 comp rx=0 tx=0).stop 2024-09-10T10:02:11.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f803c14fe10 msgr2=0x7f803c1502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:11.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f803c14fe10 0x7f803c1502c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f803401cd70 tx=0x7f8034019ce0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 shutdown_connections 2024-09-10T10:02:11.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f801c067c50 0x7f801c06a110 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f803c14fe10 0x7f803c1502c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f803c14e840 0x7f803c14e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 --2- 172.21.15.110:0/1297241334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80240051d0 0x7f803c154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:11.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 >> 172.21.15.110:0/1297241334 conn(0x7f803c062d10 msgr2=0x7f803c0b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:11.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 shutdown_connections 2024-09-10T10:02:11.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:11.948+0000 7f8043e00640 1 -- 172.21.15.110:0/1297241334 wait complete. 2024-09-10T10:02:11.970 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:02:11.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:11.970 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:11.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:02:11.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:11.996 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:02:12.052 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:12.052 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:12.062 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:12.062 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:12.121 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:02:12.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.256+0000 7fd9b8200640 1 Processor -- start 2024-09-10T10:02:12.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.256+0000 7fd9b8200640 1 -- start start 2024-09-10T10:02:12.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.256+0000 7fd9b8200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd9b006beb0 0x7fd9b015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.256+0000 7fd9b8200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd9b015a2c0 0x7fd9b00ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.256+0000 7fd9b8200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b0072550 con 0x7fd9b00d10c0 2024-09-10T10:02:12.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.256+0000 7fd9b8200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b006e310 con 0x7fd9b006beb0 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.256+0000 7fd9b8200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b0072830 con 0x7fd9b015a2c0 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.256+0000 7fd9b5800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd9b006beb0 0x7fd9b015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9b5800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd9b006beb0 0x7fd9b015d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55730/0 (socket says 172.21.15.110:55730) 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9b5800640 1 -- 172.21.15.110:0/674058982 learned_addr learned my addr 172.21.15.110:0/674058982 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9b6200640 1 --2- 172.21.15.110:0/674058982 >> v2:172.21.15.145:3300/0 conn(0x7fd9b015a2c0 0x7fd9b00ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9b5800640 1 -- 172.21.15.110:0/674058982 >> v2:172.21.15.145:3300/0 conn(0x7fd9b015a2c0 msgr2=0x7fd9b00ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9b5800640 1 --2- 172.21.15.110:0/674058982 >> v2:172.21.15.145:3300/0 conn(0x7fd9b015a2c0 0x7fd9b00ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9b5800640 1 -- 172.21.15.110:0/674058982 >> v1:172.21.15.145:6789/0 conn(0x7fd9b00d10c0 legacy=0x7fd9b0159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:12.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9b5800640 1 -- 172.21.15.110:0/674058982 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b0072b10 con 0x7fd9b006beb0 2024-09-10T10:02:12.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9b5800640 1 --2- 172.21.15.110:0/674058982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b006beb0 0x7fd9b015d550 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00d0540 tx=0x7fd9a401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f49daaa9640813e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:12.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9afe00640 1 -- 172.21.15.110:0/674058982 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40a66a0 con 0x7fd9b006beb0 2024-09-10T10:02:12.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9afe00640 1 -- 172.21.15.110:0/674058982 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a40a6840 con 0x7fd9b006beb0 2024-09-10T10:02:12.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.260+0000 7fd9afe00640 1 -- 172.21.15.110:0/674058982 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40a6c10 con 0x7fd9b006beb0 2024-09-10T10:02:12.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- 172.21.15.110:0/674058982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b006beb0 msgr2=0x7fd9b015d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 --2- 172.21.15.110:0/674058982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b006beb0 0x7fd9b015d550 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00d0540 tx=0x7fd9a401ca80 comp rx=0 tx=0).stop 2024-09-10T10:02:12.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- 172.21.15.110:0/674058982 shutdown_connections 2024-09-10T10:02:12.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 --2- 172.21.15.110:0/674058982 >> v2:172.21.15.145:3300/0 conn(0x7fd9b015a2c0 0x7fd9b00ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 --2- 172.21.15.110:0/674058982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b006beb0 0x7fd9b015d550 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- 172.21.15.110:0/674058982 >> 172.21.15.110:0/674058982 conn(0x7fd9b006cd60 msgr2=0x7fd9b006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:12.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- 172.21.15.110:0/674058982 shutdown_connections 2024-09-10T10:02:12.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- 172.21.15.110:0/674058982 wait complete. 2024-09-10T10:02:12.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 Processor -- start 2024-09-10T10:02:12.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- start start 2024-09-10T10:02:12.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b006beb0 0x7fd9b014d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b00d10c0 0x7fd9b0155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd9b015a2c0 0x7fd9b0153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b00cf550 con 0x7fd9b00d10c0 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b00721e0 con 0x7fd9b006beb0 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b8200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b00ce6e0 con 0x7fd9b015a2c0 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b00d10c0 0x7fd9b0155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b00d10c0 0x7fd9b0155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55744/0 (socket says 172.21.15.110:55744) 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 -- 172.21.15.110:0/3209499133 learned_addr learned my addr 172.21.15.110:0/3209499133 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b5800640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b006beb0 0x7fd9b014d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b6200640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd9b015a2c0 0x7fd9b0153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 -- 172.21.15.110:0/3209499133 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd9b015a2c0 msgr2=0x7fd9b0153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd9b015a2c0 0x7fd9b0153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 -- 172.21.15.110:0/3209499133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b006beb0 msgr2=0x7fd9b014d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b006beb0 0x7fd9b014d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 -- 172.21.15.110:0/3209499133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b015fb80 con 0x7fd9b00d10c0 2024-09-10T10:02:12.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.264+0000 7fd9b4e00640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b00d10c0 0x7fd9b0155770 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00c430 tx=0x7fd99c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:12.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9ad600640 1 -- 172.21.15.110:0/3209499133 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0052d0 con 0x7fd9b00d10c0 2024-09-10T10:02:12.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9ad600640 1 -- 172.21.15.110:0/3209499133 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c005470 con 0x7fd9b00d10c0 2024-09-10T10:02:12.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b015e9f0 con 0x7fd9b00d10c0 2024-09-10T10:02:12.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9ad600640 1 -- 172.21.15.110:0/3209499133 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c007260 con 0x7fd9b00d10c0 2024-09-10T10:02:12.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b015f290 con 0x7fd9b00d10c0 2024-09-10T10:02:12.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b015ebc0 con 0x7fd9b00d10c0 2024-09-10T10:02:12.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9ad600640 1 -- 172.21.15.110:0/3209499133 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fd99c007400 con 0x7fd9b00d10c0 2024-09-10T10:02:12.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9ad600640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd990067c20 0x7fd99006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9ad600640 1 -- 172.21.15.110:0/3209499133 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7fd99c085600 con 0x7fd9b00d10c0 2024-09-10T10:02:12.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.268+0000 7fd9b5800640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd990067c20 0x7fd99006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.272+0000 7fd9b5800640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd990067c20 0x7fd99006a0e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40af000 tx=0x7fd9a40a8070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:12.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.272+0000 7fd9ad600640 1 -- 172.21.15.110:0/3209499133 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd99c055680 con 0x7fd9b00d10c0 2024-09-10T10:02:12.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.444+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd9b015d550 con 0x7fd9b00d10c0 2024-09-10T10:02:12.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9ad600640 1 -- 172.21.15.110:0/3209499133 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7fd99c059980 con 0x7fd9b00d10c0 2024-09-10T10:02:12.449 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:02:12.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd990067c20 msgr2=0x7fd99006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd990067c20 0x7fd99006a0e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40af000 tx=0x7fd9a40a8070 comp rx=0 tx=0).stop 2024-09-10T10:02:12.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b00d10c0 msgr2=0x7fd9b0155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b00d10c0 0x7fd9b0155770 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00c430 tx=0x7fd99c00c830 comp rx=0 tx=0).stop 2024-09-10T10:02:12.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 shutdown_connections 2024-09-10T10:02:12.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fd990067c20 0x7fd99006a0e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd9b015a2c0 0x7fd9b0153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b00d10c0 0x7fd9b0155770 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 --2- 172.21.15.110:0/3209499133 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9b006beb0 0x7fd9b014d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 >> 172.21.15.110:0/3209499133 conn(0x7fd9b006cd60 msgr2=0x7fd9b006d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:12.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 shutdown_connections 2024-09-10T10:02:12.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.448+0000 7fd9b8200640 1 -- 172.21.15.110:0/3209499133 wait complete. 2024-09-10T10:02:12.469 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:02:12.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:12.470 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:12.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:02:12.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:12.499 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:02:12.551 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99de800640 1 Processor -- start 2024-09-10T10:02:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99de800640 1 -- start start 2024-09-10T10:02:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99de800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f99d815c560 0x7f99d80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99de800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f99d80d13f0 0x7f99d8072a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99de800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99d80722d0 con 0x7f99d8072fd0 2024-09-10T10:02:12.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99de800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f99d815c940 con 0x7f99d80d13f0 2024-09-10T10:02:12.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99de800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f99d80d1bf0 con 0x7f99d815c560 2024-09-10T10:02:12.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99d6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f99d80d13f0 0x7f99d8072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99d7e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f99d8072fd0 0x7f99d806bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45042/0 (socket says 172.21.15.110:45042) 2024-09-10T10:02:12.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99d7e00640 1 -- 172.21.15.110:0/227711387 learned_addr learned my addr 172.21.15.110:0/227711387 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:12.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99d7400640 1 --2- 172.21.15.110:0/227711387 >> v2:172.21.15.145:3300/0 conn(0x7f99d815c560 0x7f99d80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99d6a00640 1 -- 172.21.15.110:0/227711387 >> v2:172.21.15.145:3300/0 conn(0x7f99d815c560 msgr2=0x7f99d80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:12.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99d6a00640 1 --2- 172.21.15.110:0/227711387 >> v2:172.21.15.145:3300/0 conn(0x7f99d815c560 0x7f99d80cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99d6a00640 1 -- 172.21.15.110:0/227711387 >> v1:172.21.15.110:6789/0 conn(0x7f99d8072fd0 legacy=0x7f99d806bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.684+0000 7f99d6a00640 1 -- 172.21.15.110:0/227711387 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d80d1ed0 con 0x7f99d80d13f0 2024-09-10T10:02:12.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99d6a00640 1 --2- 172.21.15.110:0/227711387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8072a90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f99d8073770 tx=0x7f99c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2a3e968c40d000 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:12.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99d6000640 1 -- 172.21.15.110:0/227711387 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c40b0070 con 0x7f99d80d13f0 2024-09-10T10:02:12.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99d6000640 1 -- 172.21.15.110:0/227711387 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c40ab600 con 0x7f99d80d13f0 2024-09-10T10:02:12.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99d6000640 1 -- 172.21.15.110:0/227711387 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c40ab9d0 con 0x7f99d80d13f0 2024-09-10T10:02:12.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99de800640 1 -- 172.21.15.110:0/227711387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 msgr2=0x7f99d8072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99de800640 1 --2- 172.21.15.110:0/227711387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8072a90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f99d8073770 tx=0x7f99c401c480 comp rx=0 tx=0).stop 2024-09-10T10:02:12.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99de800640 1 -- 172.21.15.110:0/227711387 shutdown_connections 2024-09-10T10:02:12.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99de800640 1 --2- 172.21.15.110:0/227711387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8072a90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99de800640 1 --2- 172.21.15.110:0/227711387 >> v2:172.21.15.145:3300/0 conn(0x7f99d815c560 0x7f99d80cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99de800640 1 -- 172.21.15.110:0/227711387 >> 172.21.15.110:0/227711387 conn(0x7f99d806cf30 msgr2=0x7f99d806d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:12.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99de800640 1 -- 172.21.15.110:0/227711387 shutdown_connections 2024-09-10T10:02:12.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.688+0000 7f99de800640 1 -- 172.21.15.110:0/227711387 wait complete. 2024-09-10T10:02:12.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 Processor -- start 2024-09-10T10:02:12.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 -- start start 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d8072fd0 0x7f99d8155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f99d815c560 0x7f99d806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99d806f8f0 con 0x7f99d80d13f0 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99d806f770 con 0x7f99d8072fd0 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f99d806c360 con 0x7f99d815c560 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55760/0 (socket says 172.21.15.110:55760) 2024-09-10T10:02:12.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 -- 172.21.15.110:0/2498782532 learned_addr learned my addr 172.21.15.110:0/2498782532 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d7400640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d8072fd0 0x7f99d8155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d7e00640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f99d815c560 0x7f99d806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 -- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f99d815c560 msgr2=0x7f99d806fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f99d815c560 0x7f99d806fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 -- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d8072fd0 msgr2=0x7f99d8155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d8072fd0 0x7f99d8155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 -- 172.21.15.110:0/2498782532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d8157ec0 con 0x7f99d80d13f0 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d7e00640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f99d815c560 0x7f99d806fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99d6a00640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8153a90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f99c40ae040 tx=0x7f99c401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99cf400640 1 -- 172.21.15.110:0/2498782532 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c40b0050 con 0x7f99d80d13f0 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99cf400640 1 -- 172.21.15.110:0/2498782532 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c40bc070 con 0x7f99d80d13f0 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99cf400640 1 -- 172.21.15.110:0/2498782532 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c40b5850 con 0x7f99d80d13f0 2024-09-10T10:02:12.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d8165920 con 0x7f99d80d13f0 2024-09-10T10:02:12.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.692+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d8165ee0 con 0x7f99d80d13f0 2024-09-10T10:02:12.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.696+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d8165740 con 0x7f99d80d13f0 2024-09-10T10:02:12.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.696+0000 7f99cf400640 1 -- 172.21.15.110:0/2498782532 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f99c40b59f0 con 0x7f99d80d13f0 2024-09-10T10:02:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.696+0000 7f99cf400640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f99b8067c50 0x7f99b806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.696+0000 7f99cf400640 1 -- 172.21.15.110:0/2498782532 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f99c4120ac0 con 0x7f99d80d13f0 2024-09-10T10:02:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.696+0000 7f99d7400640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f99b8067c50 0x7f99b806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.696+0000 7f99d7400640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f99b8067c50 0x7f99b806a110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f99d8070de0 tx=0x7f99c8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:12.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.700+0000 7f99cf400640 1 -- 172.21.15.110:0/2498782532 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f99d8165740 con 0x7f99d80d13f0 2024-09-10T10:02:12.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.892+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99d8072a90 con 0x7f99d80d13f0 2024-09-10T10:02:12.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99cf400640 1 -- 172.21.15.110:0/2498782532 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5237 (secure 0 0 0) 0x7f99c4016070 con 0x7f99d80d13f0 2024-09-10T10:02:12.897 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:12.898 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":46,"btime":"2024-09-10T10:02:10:790959+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:12.898 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 46 2024-09-10T10:02:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f99b8067c50 msgr2=0x7f99b806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f99b8067c50 0x7f99b806a110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f99d8070de0 tx=0x7f99c8009190 comp rx=0 tx=0).stop 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 msgr2=0x7f99d8153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8153a90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f99c40ae040 tx=0x7f99c401acc0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 shutdown_connections 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f99b8067c50 0x7f99b806a110 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f99d815c560 0x7f99d806fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99d80d13f0 0x7f99d8153a90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 --2- 172.21.15.110:0/2498782532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d8072fd0 0x7f99d8155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 >> 172.21.15.110:0/2498782532 conn(0x7f99d806cf30 msgr2=0x7f99d80b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 shutdown_connections 2024-09-10T10:02:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:12.896+0000 7f99de800640 1 -- 172.21.15.110:0/2498782532 wait complete. 2024-09-10T10:02:12.918 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T10:02:13.085 INFO:teuthology.orchestra.run.smithi145.stdout:[] 2024-09-10T10:02:13.085 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-10T10:02:13.086 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol20320 2024-09-10T10:02:13.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.220+0000 7fbc1e800640 1 Processor -- start 2024-09-10T10:02:13.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.220+0000 7fbc1e800640 1 -- start start 2024-09-10T10:02:13.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc1e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fbc180b2bb0 0x7fbc180cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc1e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc180cfbc0 0x7fbc180b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc1e800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc180b37c0 con 0x7fbc18145f30 2024-09-10T10:02:13.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc1e800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbc180b3aa0 con 0x7fbc180cfbc0 2024-09-10T10:02:13.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc1e800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fbc180b3d80 con 0x7fbc180b2bb0 2024-09-10T10:02:13.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc17e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc180cfbc0 0x7fbc180b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc17e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc180cfbc0 0x7fbc180b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55766/0 (socket says 172.21.15.110:55766) 2024-09-10T10:02:13.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc17e00640 1 -- 172.21.15.110:0/2364690077 learned_addr learned my addr 172.21.15.110:0/2364690077 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:13.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc16a00640 1 --2- 172.21.15.110:0/2364690077 >> v2:172.21.15.145:3300/0 conn(0x7fbc180b2bb0 0x7fbc180cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc17e00640 1 -- 172.21.15.110:0/2364690077 >> v2:172.21.15.145:3300/0 conn(0x7fbc180b2bb0 msgr2=0x7fbc180cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:13.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc17e00640 1 --2- 172.21.15.110:0/2364690077 >> v2:172.21.15.145:3300/0 conn(0x7fbc180b2bb0 0x7fbc180cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc17e00640 1 -- 172.21.15.110:0/2364690077 >> v1:172.21.15.145:6789/0 conn(0x7fbc18145f30 legacy=0x7fbc180b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.224+0000 7fbc17e00640 1 -- 172.21.15.110:0/2364690077 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc18146310 con 0x7fbc180cfbc0 2024-09-10T10:02:13.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc17e00640 1 --2- 172.21.15.110:0/2364690077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc180cfbc0 0x7fbc180b4580 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbc1806f4f0 tx=0x7fbc1001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47242fd21519a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc16000640 1 -- 172.21.15.110:0/2364690077 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a56c0 con 0x7fbc180cfbc0 2024-09-10T10:02:13.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc16000640 1 -- 172.21.15.110:0/2364690077 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc100a5860 con 0x7fbc180cfbc0 2024-09-10T10:02:13.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc16000640 1 -- 172.21.15.110:0/2364690077 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a5c30 con 0x7fbc180cfbc0 2024-09-10T10:02:13.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 -- 172.21.15.110:0/2364690077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc180cfbc0 msgr2=0x7fbc180b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 --2- 172.21.15.110:0/2364690077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc180cfbc0 0x7fbc180b4580 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbc1806f4f0 tx=0x7fbc1001c9f0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 -- 172.21.15.110:0/2364690077 shutdown_connections 2024-09-10T10:02:13.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 --2- 172.21.15.110:0/2364690077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc180cfbc0 0x7fbc180b4580 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 --2- 172.21.15.110:0/2364690077 >> v2:172.21.15.145:3300/0 conn(0x7fbc180b2bb0 0x7fbc180cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 -- 172.21.15.110:0/2364690077 >> 172.21.15.110:0/2364690077 conn(0x7fbc18062e40 msgr2=0x7fbc1806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:13.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 -- 172.21.15.110:0/2364690077 shutdown_connections 2024-09-10T10:02:13.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 -- 172.21.15.110:0/2364690077 wait complete. 2024-09-10T10:02:13.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 Processor -- start 2024-09-10T10:02:13.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 -- start start 2024-09-10T10:02:13.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.228+0000 7fbc1e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc180b2bb0 0x7fbc18144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc1e800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc180cfbc0 0x7fbc180aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc1e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18145f30 0x7fbc1813e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc1e800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180b5660 con 0x7fbc18145f30 2024-09-10T10:02:13.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc1e800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180b54e0 con 0x7fbc180b2bb0 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc1e800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbc180b57e0 con 0x7fbc180cfbc0 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18145f30 0x7fbc1813e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18145f30 0x7fbc1813e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55774/0 (socket says 172.21.15.110:55774) 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 -- 172.21.15.110:0/501965026 learned_addr learned my addr 172.21.15.110:0/501965026 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17400640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc180b2bb0 0x7fbc18144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc16a00640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc180cfbc0 0x7fbc180aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 -- 172.21.15.110:0/501965026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc180cfbc0 msgr2=0x7fbc180aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc180cfbc0 0x7fbc180aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 -- 172.21.15.110:0/501965026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc180b2bb0 msgr2=0x7fbc18144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc180b2bb0 0x7fbc18144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 -- 172.21.15.110:0/501965026 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc181437d0 con 0x7fbc18145f30 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc17e00640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18145f30 0x7fbc1813e150 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbc100af040 tx=0x7fbc10019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc0f400640 1 -- 172.21.15.110:0/501965026 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a56c0 con 0x7fbc18145f30 2024-09-10T10:02:13.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc0f400640 1 -- 172.21.15.110:0/501965026 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc100be070 con 0x7fbc18145f30 2024-09-10T10:02:13.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc18143a70 con 0x7fbc18145f30 2024-09-10T10:02:13.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc0f400640 1 -- 172.21.15.110:0/501965026 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100b8ac0 con 0x7fbc18145f30 2024-09-10T10:02:13.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc18144030 con 0x7fbc18145f30 2024-09-10T10:02:13.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc180cf680 con 0x7fbc18145f30 2024-09-10T10:02:13.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.232+0000 7fbc0f400640 1 -- 172.21.15.110:0/501965026 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fbc100b8c60 con 0x7fbc18145f30 2024-09-10T10:02:13.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.236+0000 7fbc0f400640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbbf8067c20 0x7fbbf806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.236+0000 7fbc0f400640 1 -- 172.21.15.110:0/501965026 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7fbc101212b0 con 0x7fbc18145f30 2024-09-10T10:02:13.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.236+0000 7fbc17400640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbbf8067c20 0x7fbbf806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.236+0000 7fbc17400640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbbf8067c20 0x7fbbf806a0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc08001770 tx=0x7fbc08009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.240+0000 7fbc0f400640 1 -- 172.21.15.110:0/501965026 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbc100165e0 con 0x7fbc18145f30 2024-09-10T10:02:13.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.424+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 --> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol20320", "target": ["mon-mgr", ""]}) -- 0x7fbc18144270 con 0x7fbbf8067c20 2024-09-10T10:02:13.434 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:13.432+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:13.435 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:13.432+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:13.435 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:13.432+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:13.435 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:13.432+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:13.435 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:13.432+0000 7f30db000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:13.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.460+0000 7fbc0f400640 1 -- 172.21.15.110:0/501965026 <== mgr.4108 v2:172.21.15.110:6832/952833175 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc18144270 con 0x7fbbf8067c20 2024-09-10T10:02:13.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbbf8067c20 msgr2=0x7fbbf806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbbf8067c20 0x7fbbf806a0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc08001770 tx=0x7fbc08009040 comp rx=0 tx=0).stop 2024-09-10T10:02:13.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18145f30 msgr2=0x7fbc1813e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18145f30 0x7fbc1813e150 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbc100af040 tx=0x7fbc10019790 comp rx=0 tx=0).stop 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 shutdown_connections 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7fbbf8067c20 0x7fbbf806a0e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18145f30 0x7fbc1813e150 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc180cfbc0 0x7fbc180aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 --2- 172.21.15.110:0/501965026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc180b2bb0 0x7fbc18144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 >> 172.21.15.110:0/501965026 conn(0x7fbc18062e40 msgr2=0x7fbc18071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 shutdown_connections 2024-09-10T10:02:13.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.464+0000 7fbc1e800640 1 -- 172.21.15.110:0/501965026 wait complete. 2024-09-10T10:02:13.485 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:13.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9011400640 1 Processor -- start 2024-09-10T10:02:13.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9011400640 1 -- start start 2024-09-10T10:02:13.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9011400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f900c06e890 0x7f900c06ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9011400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 0x7f900c072e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9011400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900c06eec0 con 0x7f900c15e700 2024-09-10T10:02:13.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9011400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f900c15eae0 con 0x7f900c06e890 2024-09-10T10:02:13.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9011400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f900c0af980 con 0x7f900c0729b0 2024-09-10T10:02:13.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9009e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f900c06e890 0x7f900c06ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9009e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f900c06e890 0x7f900c06ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55804/0 (socket says 172.21.15.110:55804) 2024-09-10T10:02:13.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9009e00640 1 -- 172.21.15.110:0/1630790028 learned_addr learned my addr 172.21.15.110:0/1630790028 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:13.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f900b200640 1 --2- 172.21.15.110:0/1630790028 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 0x7f900c072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9009e00640 1 -- 172.21.15.110:0/1630790028 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 msgr2=0x7f900c072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:13.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9009e00640 1 --2- 172.21.15.110:0/1630790028 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 0x7f900c072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9009e00640 1 -- 172.21.15.110:0/1630790028 >> v1:172.21.15.145:6789/0 conn(0x7f900c15e700 legacy=0x7f900c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.620+0000 7f9009e00640 1 -- 172.21.15.110:0/1630790028 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c06f1a0 con 0x7f900c06e890 2024-09-10T10:02:13.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9009e00640 1 --2- 172.21.15.110:0/1630790028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c06e890 0x7f900c06ff60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009870 tx=0x7f8ff801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adeef1098c78d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9009400640 1 -- 172.21.15.110:0/1630790028 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff80b2070 con 0x7f900c06e890 2024-09-10T10:02:13.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9009400640 1 -- 172.21.15.110:0/1630790028 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff80a9070 con 0x7f900c06e890 2024-09-10T10:02:13.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9009400640 1 -- 172.21.15.110:0/1630790028 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff80ad880 con 0x7f900c06e890 2024-09-10T10:02:13.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 -- 172.21.15.110:0/1630790028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c06e890 msgr2=0x7f900c06ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 --2- 172.21.15.110:0/1630790028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c06e890 0x7f900c06ff60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009870 tx=0x7f8ff801c560 comp rx=0 tx=0).stop 2024-09-10T10:02:13.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 -- 172.21.15.110:0/1630790028 shutdown_connections 2024-09-10T10:02:13.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 --2- 172.21.15.110:0/1630790028 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 0x7f900c072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 --2- 172.21.15.110:0/1630790028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c06e890 0x7f900c06ff60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 -- 172.21.15.110:0/1630790028 >> 172.21.15.110:0/1630790028 conn(0x7f900c06cb90 msgr2=0x7f900c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:13.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 -- 172.21.15.110:0/1630790028 shutdown_connections 2024-09-10T10:02:13.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 -- 172.21.15.110:0/1630790028 wait complete. 2024-09-10T10:02:13.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 Processor -- start 2024-09-10T10:02:13.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.624+0000 7f9011400640 1 -- start start 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c06e890 0x7f900c14f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 0x7f900c0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c15e700 0x7f900c1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f900c0b47e0 con 0x7f900c15e700 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f900c0b4660 con 0x7f900c06e890 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f900c0b4960 con 0x7f900c0729b0 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c15e700 0x7f900c1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c15e700 0x7f900c1542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55816/0 (socket says 172.21.15.110:55816) 2024-09-10T10:02:13.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9009e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 0x7f900c0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 -- 172.21.15.110:0/2207992475 learned_addr learned my addr 172.21.15.110:0/2207992475 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:13.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900a800640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c06e890 0x7f900c14f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 -- 172.21.15.110:0/2207992475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 msgr2=0x7f900c0b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 0x7f900c0b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 -- 172.21.15.110:0/2207992475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c06e890 msgr2=0x7f900c14f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c06e890 0x7f900c14f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 -- 172.21.15.110:0/2207992475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c0b1ce0 con 0x7f900c15e700 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f900b200640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c15e700 0x7f900c1542d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f900400efc0 tx=0x7f900400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9002a00640 1 -- 172.21.15.110:0/2207992475 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004018020 con 0x7f900c15e700 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9002a00640 1 -- 172.21.15.110:0/2207992475 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9004014480 con 0x7f900c15e700 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9002a00640 1 -- 172.21.15.110:0/2207992475 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004014870 con 0x7f900c15e700 2024-09-10T10:02:13.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c0b0830 con 0x7f900c15e700 2024-09-10T10:02:13.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c0b0df0 con 0x7f900c15e700 2024-09-10T10:02:13.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.628+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c072e00 con 0x7f900c15e700 2024-09-10T10:02:13.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.632+0000 7f9002a00640 1 -- 172.21.15.110:0/2207992475 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f900401b020 con 0x7f900c15e700 2024-09-10T10:02:13.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.632+0000 7f9002a00640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8fec067c20 0x7f8fec06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.632+0000 7f9002a00640 1 -- 172.21.15.110:0/2207992475 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f9004086ce0 con 0x7f900c15e700 2024-09-10T10:02:13.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.632+0000 7f900a800640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8fec067c20 0x7f8fec06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.632+0000 7f900a800640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8fec067c20 0x7f8fec06a0e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc007fd0 tx=0x7f8ffc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.636+0000 7f9002a00640 1 -- 172.21.15.110:0/2207992475 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9004022070 con 0x7f900c15e700 2024-09-10T10:02:13.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f900c0afb00 con 0x7f900c15e700 2024-09-10T10:02:13.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9002a00640 1 -- 172.21.15.110:0/2207992475 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5237 (secure 0 0 0) 0x7f9004010040 con 0x7f900c15e700 2024-09-10T10:02:13.830 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:13.830 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":46,"btime":"2024-09-10T10:02:10:790959+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:13.830 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 46 2024-09-10T10:02:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8fec067c20 msgr2=0x7f8fec06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8fec067c20 0x7f8fec06a0e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc007fd0 tx=0x7f8ffc009040 comp rx=0 tx=0).stop 2024-09-10T10:02:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c15e700 msgr2=0x7f900c1542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c15e700 0x7f900c1542d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f900400efc0 tx=0x7f900400c490 comp rx=0 tx=0).stop 2024-09-10T10:02:13.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 shutdown_connections 2024-09-10T10:02:13.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f8fec067c20 0x7f8fec06a0e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f900c15e700 0x7f900c1542d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f900c0729b0 0x7f900c0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 --2- 172.21.15.110:0/2207992475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f900c06e890 0x7f900c14f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 >> 172.21.15.110:0/2207992475 conn(0x7f900c06cb90 msgr2=0x7f900c15d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:13.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 shutdown_connections 2024-09-10T10:02:13.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:13.828+0000 7f9011400640 1 -- 172.21.15.110:0/2207992475 wait complete. 2024-09-10T10:02:13.850 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T10:02:14.010 INFO:teuthology.orchestra.run.smithi145.stdout:[{"id":6249,"entity":{"name":{"type":"client","num":6249},"addr":{"type":"any","addr":"172.21.15.110:0","nonce":698629357}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.55599225299999999,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.97237503709021,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6249 172.21.15.110:0/698629357","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"y","hostname":"smithi110","pid":"28893","root":"/"}}] 2024-09-10T10:02:14.011 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi110", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28893", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.110:0", "nonce": 698629357, "type": "any" }, "name": { "num": 6249, "type": "client" } }, "id": 6249, "inst": "client.6249 172.21.15.110:0/698629357", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.97237503709021 }, "state": "open", "uptime": 0.555992253 } ] 2024-09-10T10:02:14.011 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T10:02:14.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f485d400640 1 Processor -- start 2024-09-10T10:02:14.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f485d400640 1 -- start start 2024-09-10T10:02:14.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f485d400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f48581693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f485d400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48581482e0 con 0x7f48580aef50 2024-09-10T10:02:14.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f485d400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f485815ee20 con 0x7f4858147f00 2024-09-10T10:02:14.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f485d400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f48580b34d0 con 0x7f485806edf0 2024-09-10T10:02:14.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f4855e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f48580aef50 0x7f485806e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45046/0 (socket says 172.21.15.110:45046) 2024-09-10T10:02:14.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f4855e00640 1 -- 172.21.15.110:0/1586786221 learned_addr learned my addr 172.21.15.110:0/1586786221 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:14.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f4857200640 1 --2- 172.21.15.110:0/1586786221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f48581693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f4855400640 1 -- 172.21.15.110:0/1586786221 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637442929 0 0) 0x7f48581482e0 con 0x7f48580aef50 2024-09-10T10:02:14.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.144+0000 7f4855400640 1 -- 172.21.15.110:0/1586786221 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f483c003530 con 0x7f48580aef50 2024-09-10T10:02:14.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4855400640 1 -- 172.21.15.110:0/1586786221 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2066831423 0 0) 0x7f485815ee20 con 0x7f4858147f00 2024-09-10T10:02:14.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4855400640 1 -- 172.21.15.110:0/1586786221 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48581482e0 con 0x7f4858147f00 2024-09-10T10:02:14.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4855400640 1 -- 172.21.15.110:0/1586786221 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1178001131 0 0) 0x7f483c003530 con 0x7f48580aef50 2024-09-10T10:02:14.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4857200640 1 -- 172.21.15.110:0/1586786221 >> v1:172.21.15.145:6789/0 conn(0x7f4858147f00 legacy=0x7f48580ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4857200640 1 -- 172.21.15.110:0/1586786221 >> v1:172.21.15.110:6789/0 conn(0x7f48580aef50 legacy=0x7f485806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4857200640 1 -- 172.21.15.110:0/1586786221 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485815f100 con 0x7f485806edf0 2024-09-10T10:02:14.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4857200640 1 --2- 172.21.15.110:0/1586786221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f48581693f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f48500097f0 tx=0x7f485001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f96266ef718b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:14.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4855400640 1 -- 172.21.15.110:0/1586786221 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500b0070 con 0x7f485806edf0 2024-09-10T10:02:14.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4855400640 1 -- 172.21.15.110:0/1586786221 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48500ab400 con 0x7f485806edf0 2024-09-10T10:02:14.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.148+0000 7f4855400640 1 -- 172.21.15.110:0/1586786221 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844001680 con 0x7f485806edf0 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- 172.21.15.110:0/1586786221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 msgr2=0x7f48581693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 --2- 172.21.15.110:0/1586786221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f48581693f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f48500097f0 tx=0x7f485001c400 comp rx=0 tx=0).stop 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- 172.21.15.110:0/1586786221 shutdown_connections 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 --2- 172.21.15.110:0/1586786221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f48581693f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- 172.21.15.110:0/1586786221 >> 172.21.15.110:0/1586786221 conn(0x7f485806bc50 msgr2=0x7f485806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- 172.21.15.110:0/1586786221 shutdown_connections 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- 172.21.15.110:0/1586786221 wait complete. 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 Processor -- start 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- start start 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f4858161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48580aef50 0x7f485813f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4858147f00 0x7f485813d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f485816c6e0 con 0x7f4858147f00 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f485816c560 con 0x7f48580aef50 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f485d400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f485816c860 con 0x7f485806edf0 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4858147f00 0x7f485813d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4858147f00 0x7f485813d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55826/0 (socket says 172.21.15.110:55826) 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 -- 172.21.15.110:0/4216361069 learned_addr learned my addr 172.21.15.110:0/4216361069 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4855e00640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48580aef50 0x7f485813f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 -- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 msgr2=0x7f4858161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4856800640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f4858161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f4858161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 -- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48580aef50 msgr2=0x7f485813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48580aef50 0x7f485813f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 -- 172.21.15.110:0/4216361069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485813faa0 con 0x7f4858147f00 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4856800640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f4858161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4855e00640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48580aef50 0x7f485813f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.152+0000 7f4857200640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4858147f00 0x7f485813d970 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f48500ae040 tx=0x7f48500191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f484ea00640 1 -- 172.21.15.110:0/4216361069 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500b0040 con 0x7f4858147f00 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f484ea00640 1 -- 172.21.15.110:0/4216361069 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48500abcb0 con 0x7f4858147f00 2024-09-10T10:02:14.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f484ea00640 1 -- 172.21.15.110:0/4216361069 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850006e20 con 0x7f4858147f00 2024-09-10T10:02:14.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4858141140 con 0x7f4858147f00 2024-09-10T10:02:14.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4858141700 con 0x7f4858147f00 2024-09-10T10:02:14.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f485815ee20 con 0x7f4858147f00 2024-09-10T10:02:14.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f484ea00640 1 -- 172.21.15.110:0/4216361069 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f48500bc020 con 0x7f4858147f00 2024-09-10T10:02:14.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f484ea00640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4838067c20 0x7f483806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f484ea00640 1 -- 172.21.15.110:0/4216361069 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f485011f870 con 0x7f4858147f00 2024-09-10T10:02:14.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f4856800640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4838067c20 0x7f483806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.156+0000 7f4856800640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4838067c20 0x7f483806a0e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4848001570 tx=0x7f4848009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:14.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.160+0000 7f484ea00640 1 -- 172.21.15.110:0/4216361069 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4850015ff0 con 0x7f4858147f00 2024-09-10T10:02:14.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.356+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f485815e4d0 con 0x7f4858147f00 2024-09-10T10:02:14.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.364+0000 7f484ea00640 1 -- 172.21.15.110:0/4216361069 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) ==== 81+0+453585 (secure 0 0 0) 0x7f48500bb2c0 con 0x7f4858147f00 2024-09-10T10:02:14.371 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:14.371 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:02:14.371 INFO:teuthology.orchestra.run.smithi110.stdout: "epoch": 11, 2024-09-10T10:02:14.371 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.371 INFO:teuthology.orchestra.run.smithi110.stdout: "active_gid": 4108, 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "active_name": "y", 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "active_addrs": { 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:6832", 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 952833175 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v1", 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:6833", 2024-09-10T10:02:14.372 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 952833175 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "active_addr": "172.21.15.110:6833/952833175", 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "active_change": "2024-09-10T09:58:03.341235+0000", 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "available": true, 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "standbys": [ 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "gid": 4100, 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "x", 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:02:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: "available_modules": [ 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "alerts", 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.374 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.375 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.377 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.378 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.379 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_destination": { 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_destination", 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_from_name": { 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_from_name", 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.381 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "Ceph", 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email From: name", 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_host": { 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_host", 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.382 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP server", 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_password": { 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_password", 2024-09-10T10:02:14.383 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Password to authenticate with", 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.384 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_port": { 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_port", 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "465", 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP port", 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_sender": { 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_sender", 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_ssl": { 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_ssl", 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:02:14.387 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_user": { 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_user", 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.388 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "User to authenticate as", 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "balancer", 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.389 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "active": { 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "active", 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_time": { 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_time", 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0000", 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_weekday": { 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_weekday", 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:02:14.392 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_max_iterations": { 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "25", 2024-09-10T10:02:14.393 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "250", 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_metrics": { 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_metrics", 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.394 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_step": { 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_step", 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.395 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.5", 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0.001", 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "0.999", 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "end_time": { 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_time", 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2359", 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.397 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "end_weekday": { 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_weekday", 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-09-10T10:02:14.398 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.399 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.400 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.401 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.402 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.403 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.404 INFO:teuthology.orchestra.run.smithi110.stdout: "min_score": { 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_score", 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.405 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "upmap", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "crush-compat", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "none", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "read", 2024-09-10T10:02:14.406 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap", 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap-read" 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Balancer mode", 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_ids": { 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_ids", 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.407 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.408 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_deviation": { 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_deviation", 2024-09-10T10:02:14.409 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_optimizations": { 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:02:14.410 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm", 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_down_multiplier": { 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_down_multiplier", 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-09-10T10:02:14.412 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_refresh_rate": { 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_refresh_rate", 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "20", 2024-09-10T10:02:14.413 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_starting_port": { 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_starting_port", 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4721", 2024-09-10T10:02:14.414 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_ptrace": { 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_ptrace", 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.415 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_interval": { 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_interval", 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.416 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:02:14.417 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.7", 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm_log_destination": { 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm_log_destination", 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "file", 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "file,syslog", 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: "syslog" 2024-09-10T10:02:14.419 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "cgroups_split": { 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cgroups_split", 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.420 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "config_checks_enabled": { 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_checks_enabled", 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.421 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "config_dashboard": { 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_dashboard", 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.422 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_alertmanager": { 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_alertmanager", 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.423 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_base": { 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_base", 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_elasticsearch": { 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:02:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "elasticsearch container image", 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_grafana": { 2024-09-10T10:02:14.426 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_grafana", 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.427 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_haproxy": { 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_haproxy", 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "HAproxy container image", 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.428 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_agent": { 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.429 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_collector": { 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:02:14.430 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_query": { 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.431 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger query container image", 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_keepalived": { 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_keepalived", 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:02:14.432 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Keepalived container image", 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_loki": { 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_loki", 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.433 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Loki container image", 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_node_exporter": { 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_node_exporter", 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.434 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_nvmeof": { 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_nvmeof", 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.435 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Nvme-of container image", 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_prometheus": { 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_prometheus", 2024-09-10T10:02:14.436 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_promtail": { 2024-09-10T10:02:14.437 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_promtail", 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Promtail container image", 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.438 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_samba": { 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_samba", 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_snmp_gateway": { 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.440 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: "container_init": { 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_init", 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.441 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "daemon_cache_timeout": { 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:02:14.442 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "900", 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.444 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "default_registry": { 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_registry", 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io", 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.445 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "device_cache_timeout": { 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_cache_timeout", 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1800", 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.446 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: "device_enhanced_scan": { 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_enhanced_scan", 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.447 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "facts_cache_timeout": { 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "facts_cache_timeout", 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.449 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.450 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.450 INFO:teuthology.orchestra.run.smithi110.stdout: "grafana_dashboards_path": { 2024-09-10T10:02:14.450 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:02:14.450 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.450 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.450 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.450 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "host_check_interval": { 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "host_check_interval", 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.451 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "hw_monitoring": { 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hw_monitoring", 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.452 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "inventory_list_all": { 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "inventory_list_all", 2024-09-10T10:02:14.453 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.454 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.455 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "log_refresh_metadata": { 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_refresh_metadata", 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.456 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.457 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.458 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.459 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.460 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.461 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.462 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "*", 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "max_count_per_host": { 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_count_per_host", 2024-09-10T10:02:14.463 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "max_osd_draining_count": { 2024-09-10T10:02:14.464 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_osd_draining_count", 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.465 INFO:teuthology.orchestra.run.smithi110.stdout: "migration_current": { 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "migration_current", 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "internal - do not modify", 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "root", 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm-package", 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "root" 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.467 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "oob_default_addr": { 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "oob_default_addr", 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "169.254.1.1", 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.468 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "prometheus_alerts_path": { 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.469 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_insecure": { 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_insecure", 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.470 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_password": { 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_password", 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.471 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_url": { 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_url", 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.472 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_username": { 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_username", 2024-09-10T10:02:14.473 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "secure_monitoring_stack": { 2024-09-10T10:02:14.474 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "service_discovery_port": { 2024-09-10T10:02:14.475 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "service_discovery_port", 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8765", 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "ssh_config_file": { 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssh_config_file", 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "use_agent": { 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_agent", 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:02:14.478 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "use_repo_digest": { 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_repo_digest", 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.479 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_failed_host_check": { 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.480 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_daemons": { 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.481 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_hosts": { 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.482 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crash", 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.483 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.484 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.485 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.486 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.487 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.488 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "retain_interval": { 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retain_interval", 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.489 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "31536000", 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_recent_interval": { 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_recent_interval", 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.490 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1209600", 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.491 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dashboard", 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.492 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.493 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.494 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.495 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.496 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.497 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.498 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.499 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:02:14.500 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:02:14.501 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.502 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.504 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.505 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.506 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.507 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.508 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_URL": { 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.509 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.510 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.511 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.512 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.513 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.514 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:02:14.515 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.517 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.518 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.519 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.520 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.521 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:02:14.523 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:02:14.524 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:02:14.525 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.527 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.528 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8", 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.529 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "45", 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.530 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.531 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.532 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.533 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.534 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.535 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:02:14.536 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:02:14.537 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "cross_origin_url": { 2024-09-10T10:02:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cross_origin_url", 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.539 INFO:teuthology.orchestra.run.smithi110.stdout: "crt_file": { 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crt_file", 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "debug": { 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "debug", 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:02:14.541 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "jwt_token_ttl": { 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "jwt_token_ttl", 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "28800", 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.542 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.543 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.544 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.545 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.546 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.547 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.548 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "motd": { 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "motd", 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "The message of the day", 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8080", 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.553 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.554 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl_server_port": { 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl_server_port", 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8443", 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.555 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "redirect", 2024-09-10T10:02:14.556 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect" 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.557 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.558 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "url_prefix": { 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url_prefix", 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.559 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "devicehealth", 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_monitoring": { 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_monitoring", 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.561 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "monitor device health metrics", 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.562 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.564 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.565 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.566 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.567 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "mark_out_threshold": { 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mark_out_threshold", 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.568 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2419200", 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_name": { 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_name", 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.569 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "device_health_metrics", 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "retention_period": { 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retention_period", 2024-09-10T10:02:14.570 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15552000", 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_frequency": { 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_frequency", 2024-09-10T10:02:14.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: "self_heal": { 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "self_heal", 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_threshold": { 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_threshold", 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "7257600", 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.575 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "diskprediction_local", 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.576 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.578 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.579 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.580 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.581 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "predict_interval": { 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predict_interval", 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-09-10T10:02:14.582 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "predictor_model": { 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predictor_model", 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.583 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "prophetstor", 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "influx", 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": false, 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "batch_size": { 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "batch_size", 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5000", 2024-09-10T10:02:14.586 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "database": { 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "database", 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.587 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "ceph", 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "hostname": { 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hostname", 2024-09-10T10:02:14.588 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:14.589 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "30", 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "5", 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.590 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.591 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.592 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.593 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.594 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.595 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "password": { 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "password", 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.596 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "port": { 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "port", 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8086", 2024-09-10T10:02:14.597 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server port", 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.598 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "false", 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "threads": { 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threads", 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "32", 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "username": { 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "username", 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.600 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "verify_ssl": { 2024-09-10T10:02:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "verify_ssl", 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "true", 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "insights", 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.603 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.605 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.606 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.607 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "iostat", 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.609 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.610 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.611 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.612 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.613 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.614 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "localpool", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "failure_domain": { 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "failure_domain", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "host", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:02:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.616 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.617 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.618 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.619 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.620 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "min_size": { 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_size", 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.621 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "num_rep": { 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "num_rep", 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.622 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3", 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_num": { 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_num", 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.623 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "128", 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "prefix": { 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prefix", 2024-09-10T10:02:14.624 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.625 INFO:teuthology.orchestra.run.smithi110.stdout: "subtree": { 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "subtree", 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "rack", 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:02:14.626 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirroring", 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.627 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.628 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.629 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.630 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "nfs", 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.633 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.634 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.635 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.636 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.637 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.638 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "fail_fs": { 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "fail_fs", 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.639 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.640 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.641 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.642 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.643 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.644 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.645 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator": { 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm", 2024-09-10T10:02:14.646 INFO:teuthology.orchestra.run.smithi110.stdout: "rook", 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "test_orchestrator" 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Orchestrator backend", 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_perf_query", 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_support", 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.653 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.654 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.655 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.656 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.657 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.658 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_autoscaler", 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.661 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.662 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.664 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.665 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "threshold": { 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threshold", 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1.0", 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "scaling threshold", 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:02:14.666 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "progress", 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_pg_recovery_event": { 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.667 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.668 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.669 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.670 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.671 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.672 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.673 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.674 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "max_completed_events": { 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_completed_events", 2024-09-10T10:02:14.675 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "50", 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.676 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:02:14.677 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus", 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: "cache": { 2024-09-10T10:02:14.678 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cache", 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "exclude_perf_counters": { 2024-09-10T10:02:14.679 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "exclude_perf_counters", 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.681 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.682 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.683 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.683 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.683 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.683 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.683 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.683 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.683 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.683 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.684 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.685 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.686 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools": { 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools", 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.688 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "300", 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_interval": { 2024-09-10T10:02:14.689 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_interval", 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15.0", 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-09-10T10:02:14.690 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.691 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "9283", 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.692 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "stale_cache_strategy": { 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stale_cache_strategy", 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "log", 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.693 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "default", 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "default", 2024-09-10T10:02:14.694 INFO:teuthology.orchestra.run.smithi110.stdout: "error" 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-09-10T10:02:14.695 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_support", 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.696 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.697 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.698 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.699 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.700 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.701 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_snap_create": { 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:02:14.702 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.704 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "trash_purge_schedule": { 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "trash_purge_schedule", 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.705 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "restful", 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.706 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_auth": { 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_auth", 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.707 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.708 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.708 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-09-10T10:02:14.708 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-09-10T10:02:14.708 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.708 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.708 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.709 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.709 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.709 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.709 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.712 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.712 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.712 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.712 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.712 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.712 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.713 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.714 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.715 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.716 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.718 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.720 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "selftest", 2024-09-10T10:02:14.721 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.722 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.724 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.725 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "roption1": { 2024-09-10T10:02:14.727 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption1", 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.728 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "roption2": { 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption2", 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "xyz", 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.729 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption1": { 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption1", 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.730 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption2": { 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption2", 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.731 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption3": { 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption3", 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.732 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption4": { 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption4", 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.733 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption5": { 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption5", 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.734 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption6": { 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption6", 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.735 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption7": { 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption7", 2024-09-10T10:02:14.736 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "42", 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.737 INFO:teuthology.orchestra.run.smithi110.stdout: "testkey": { 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testkey", 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.738 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "testlkey": { 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testlkey", 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.739 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: "testnewline": { 2024-09-10T10:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testnewline", 2024-09-10T10:02:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.741 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snap_schedule", 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_m_granularity": { 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_m_granularity", 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.742 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "dump_on_update": { 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dump_on_update", 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.744 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.745 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.746 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.747 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.748 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.749 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.750 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stats", 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.751 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.752 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.753 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.756 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "status", 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.757 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.758 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.759 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.760 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.761 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.762 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telegraf", 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.763 INFO:teuthology.orchestra.run.smithi110.stdout: "address": { 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "address", 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.764 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15", 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.765 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.766 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.767 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.770 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telemetry", 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.771 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_basic": { 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_basic", 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.772 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_crash": { 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_crash", 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.773 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_device": { 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_device", 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_ident": { 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_ident", 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:02:14.775 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_perf": { 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_perf", 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "contact": { 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "contact", 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "description": { 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "description", 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.778 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "device_url": { 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_url", 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.779 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.780 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "24", 2024-09-10T10:02:14.781 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "8", 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "last_opt_revision": { 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "last_opt_revision", 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1", 2024-09-10T10:02:14.782 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard": { 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard", 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.783 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard_description": { 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard_description", 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.784 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.785 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.786 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.787 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.788 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.789 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.790 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "organization": { 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "organization", 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.791 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "proxy": { 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "proxy", 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.792 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "url": { 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url", 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:02:14.793 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "test_orchestrator", 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.794 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.795 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.796 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.797 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.798 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.799 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.800 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.801 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.803 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.804 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.805 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.806 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_clones": { 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_clones", 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4", 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "periodic_async_work": { 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "periodic_async_work", 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Periodically check for async work", 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.808 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_delay": { 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.809 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.810 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix", 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "discovery_interval": { 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "discovery_interval", 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.811 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "100", 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "identifier": { 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "identifier", 2024-09-10T10:02:14.812 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:14.813 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.814 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.815 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.816 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.817 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.818 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.819 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_host": { 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_host", 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.820 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_port": { 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_port", 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10051", 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.821 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_sender": { 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_sender", 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: "modules": [ 2024-09-10T10:02:14.823 INFO:teuthology.orchestra.run.smithi110.stdout: "iostat", 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "nfs", 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "restful" 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "available_modules": [ 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "alerts", 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:14.824 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.825 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.826 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.828 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_destination": { 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_destination", 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.831 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_from_name": { 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_from_name", 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "Ceph", 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email From: name", 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_host": { 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_host", 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP server", 2024-09-10T10:02:14.833 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_password": { 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_password", 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Password to authenticate with", 2024-09-10T10:02:14.834 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_port": { 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_port", 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "465", 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.835 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP port", 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_sender": { 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_sender", 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.836 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_ssl": { 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_ssl", 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.837 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_user": { 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_user", 2024-09-10T10:02:14.838 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "User to authenticate as", 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "balancer", 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "active": { 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "active", 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.840 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_time": { 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_time", 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0000", 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.841 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_weekday": { 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_weekday", 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.842 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_max_iterations": { 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "25", 2024-09-10T10:02:14.843 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "250", 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_metrics": { 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_metrics", 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_step": { 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_step", 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.845 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.5", 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0.001", 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "0.999", 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "end_time": { 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_time", 2024-09-10T10:02:14.846 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2359", 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "end_weekday": { 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_weekday", 2024-09-10T10:02:14.847 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.848 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.849 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.850 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.851 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.853 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "min_score": { 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_score", 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.854 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.855 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "upmap", 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "crush-compat", 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "none", 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "read", 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap", 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap-read" 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.856 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Balancer mode", 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_ids": { 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_ids", 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.857 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.858 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_deviation": { 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_deviation", 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-09-10T10:02:14.859 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_optimizations": { 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:02:14.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm", 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_down_multiplier": { 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_down_multiplier", 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.862 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_refresh_rate": { 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_refresh_rate", 2024-09-10T10:02:14.863 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "20", 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_starting_port": { 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_starting_port", 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4721", 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.865 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_ptrace": { 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_ptrace", 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.866 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_interval": { 2024-09-10T10:02:14.867 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_interval", 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.868 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.7", 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:02:14.869 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm_log_destination": { 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm_log_destination", 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "file", 2024-09-10T10:02:14.870 INFO:teuthology.orchestra.run.smithi110.stdout: "file,syslog", 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "syslog" 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "cgroups_split": { 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cgroups_split", 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.871 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "config_checks_enabled": { 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_checks_enabled", 2024-09-10T10:02:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "config_dashboard": { 2024-09-10T10:02:14.873 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_dashboard", 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_alertmanager": { 2024-09-10T10:02:14.874 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_alertmanager", 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.875 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_base": { 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_base", 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.876 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_elasticsearch": { 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "elasticsearch container image", 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.877 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_grafana": { 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_grafana", 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_haproxy": { 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_haproxy", 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "HAproxy container image", 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.879 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_agent": { 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.880 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_collector": { 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:02:14.881 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_query": { 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger query container image", 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_keepalived": { 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_keepalived", 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.883 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Keepalived container image", 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_loki": { 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_loki", 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.884 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Loki container image", 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_node_exporter": { 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_node_exporter", 2024-09-10T10:02:14.885 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.886 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_nvmeof": { 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_nvmeof", 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Nvme-of container image", 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_prometheus": { 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_prometheus", 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:02:14.888 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_promtail": { 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_promtail", 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.889 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Promtail container image", 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.890 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_samba": { 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_samba", 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.891 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_snmp_gateway": { 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.892 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "container_init": { 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_init", 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.893 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "daemon_cache_timeout": { 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:02:14.894 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "900", 2024-09-10T10:02:14.896 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "default_registry": { 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_registry", 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.897 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io", 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "device_cache_timeout": { 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_cache_timeout", 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.898 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1800", 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "device_enhanced_scan": { 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_enhanced_scan", 2024-09-10T10:02:14.899 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "facts_cache_timeout": { 2024-09-10T10:02:14.900 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "facts_cache_timeout", 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.901 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "grafana_dashboards_path": { 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.902 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "host_check_interval": { 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "host_check_interval", 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:02:14.903 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "hw_monitoring": { 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hw_monitoring", 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.904 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "inventory_list_all": { 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "inventory_list_all", 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.905 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.906 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "log_refresh_metadata": { 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_refresh_metadata", 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.908 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.909 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.910 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.911 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.913 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:02:14.914 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:02:14.914 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.916 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.916 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.916 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "*", 2024-09-10T10:02:14.916 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.916 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.916 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.916 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "max_count_per_host": { 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_count_per_host", 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.917 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: "max_osd_draining_count": { 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_osd_draining_count", 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.918 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "migration_current": { 2024-09-10T10:02:14.919 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "migration_current", 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "internal - do not modify", 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.920 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "root", 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.921 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm-package", 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "root" 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "oob_default_addr": { 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "oob_default_addr", 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.922 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "169.254.1.1", 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "prometheus_alerts_path": { 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.923 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_insecure": { 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_insecure", 2024-09-10T10:02:14.924 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_password": { 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_password", 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_url": { 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_url", 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.927 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.928 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_username": { 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_username", 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.929 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: "secure_monitoring_stack": { 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.930 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "service_discovery_port": { 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "service_discovery_port", 2024-09-10T10:02:14.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8765", 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "ssh_config_file": { 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssh_config_file", 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "use_agent": { 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_agent", 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "use_repo_digest": { 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_repo_digest", 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.935 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_failed_host_check": { 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.936 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_daemons": { 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.937 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_hosts": { 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.938 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crash", 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.940 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:14.941 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.944 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "retain_interval": { 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retain_interval", 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.945 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "31536000", 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_recent_interval": { 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_recent_interval", 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:14.946 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1209600", 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dashboard", 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.948 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.949 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.950 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.951 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.952 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.953 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.954 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.955 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:02:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:02:14.957 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:02:14.958 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:02:14.959 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:02:14.960 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.962 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.964 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_URL": { 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.965 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-09-10T10:02:14.966 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.967 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.968 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.969 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.971 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.972 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.973 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.975 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.976 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:02:14.977 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:02:14.978 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.979 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8", 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "45", 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.987 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.989 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:14.990 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:14.991 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.992 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "cross_origin_url": { 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cross_origin_url", 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.993 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "crt_file": { 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crt_file", 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.994 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "debug": { 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "debug", 2024-09-10T10:02:14.995 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: "jwt_token_ttl": { 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "jwt_token_ttl", 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "28800", 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.002 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.003 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "motd": { 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "motd", 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.004 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "The message of the day", 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.006 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8080", 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl_server_port": { 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl_server_port", 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8443", 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.010 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "redirect", 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect" 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.011 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.012 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "url_prefix": { 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url_prefix", 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.013 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "devicehealth", 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_monitoring": { 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_monitoring", 2024-09-10T10:02:15.014 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "monitor device health metrics", 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.015 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.017 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.018 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.019 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.020 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "mark_out_threshold": { 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mark_out_threshold", 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2419200", 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.021 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_name": { 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_name", 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "device_health_metrics", 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "retention_period": { 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retention_period", 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15552000", 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_frequency": { 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_frequency", 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.024 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "self_heal": { 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "self_heal", 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.025 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:15.026 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_threshold": { 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_threshold", 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "7257600", 2024-09-10T10:02:15.027 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "diskprediction_local", 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.028 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.029 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.032 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.033 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "predict_interval": { 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predict_interval", 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.034 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "predictor_model": { 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predictor_model", 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "prophetstor", 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.035 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-09-10T10:02:15.036 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "influx", 2024-09-10T10:02:15.037 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": false, 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "batch_size": { 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "batch_size", 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5000", 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:02:15.038 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "database": { 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "database", 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "ceph", 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.039 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "hostname": { 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hostname", 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.040 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "30", 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "5", 2024-09-10T10:02:15.041 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.042 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.043 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.044 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.045 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.046 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "password": { 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "password", 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.048 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "port": { 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "port", 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8086", 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server port", 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.049 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "false", 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:02:15.050 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "threads": { 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threads", 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "32", 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:02:15.051 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "username": { 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "username", 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.052 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "verify_ssl": { 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "verify_ssl", 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "true", 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.053 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "insights", 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.054 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.055 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.056 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.057 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.058 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.059 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "iostat", 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.060 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.061 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.062 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.063 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.064 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.065 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "localpool", 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "failure_domain": { 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "failure_domain", 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.066 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "host", 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.067 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.068 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.069 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.070 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.071 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "min_size": { 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_size", 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.073 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "num_rep": { 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "num_rep", 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3", 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.074 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_num": { 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_num", 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "128", 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.075 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "prefix": { 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prefix", 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:02:15.076 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "subtree": { 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "subtree", 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "rack", 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.077 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirroring", 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.078 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.079 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.080 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.081 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.082 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.083 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "nfs", 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.084 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.085 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.086 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.087 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.088 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "fail_fs": { 2024-09-10T10:02:15.090 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "fail_fs", 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.091 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.092 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.093 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.094 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.096 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator": { 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.097 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm", 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "rook", 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "test_orchestrator" 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Orchestrator backend", 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.098 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_perf_query", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.099 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.100 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.101 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.102 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.103 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_support", 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.104 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.105 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.107 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.108 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.109 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.110 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_autoscaler", 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.111 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.112 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.113 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.114 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.115 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.116 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "threshold": { 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threshold", 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1.0", 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.117 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "scaling threshold", 2024-09-10T10:02:15.120 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:02:15.120 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.120 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "progress", 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_pg_recovery_event": { 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.121 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.122 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.123 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.124 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.125 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.126 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.127 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.128 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "max_completed_events": { 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_completed_events", 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "50", 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.129 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.130 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus", 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.131 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "cache": { 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cache", 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "exclude_perf_counters": { 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "exclude_perf_counters", 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:02:15.133 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.134 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.135 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.136 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.137 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.138 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools": { 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools", 2024-09-10T10:02:15.139 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "300", 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.141 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_interval": { 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_interval", 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15.0", 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.143 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "9283", 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.144 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "stale_cache_strategy": { 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stale_cache_strategy", 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "log", 2024-09-10T10:02:15.145 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.146 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "default", 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "default", 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "error" 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-09-10T10:02:15.147 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_support", 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.149 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.150 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.151 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.152 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.153 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_snap_create": { 2024-09-10T10:02:15.154 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "trash_purge_schedule": { 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "trash_purge_schedule", 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.157 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "restful", 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.158 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_auth": { 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_auth", 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.159 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.160 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.161 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.162 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.163 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.164 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.165 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.166 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.167 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.168 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "selftest", 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.169 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.170 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.171 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.174 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "roption1": { 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption1", 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.175 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "roption2": { 2024-09-10T10:02:15.176 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption2", 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "xyz", 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.177 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption1": { 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption1", 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.178 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption2": { 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption2", 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.179 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption3": { 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption3", 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.180 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption4": { 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption4", 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.181 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption5": { 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption5", 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.182 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption6": { 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption6", 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.183 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption7": { 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption7", 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-09-10T10:02:15.184 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "42", 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "testkey": { 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testkey", 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.185 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "testlkey": { 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testlkey", 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.186 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "testnewline": { 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testnewline", 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.187 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snap_schedule", 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_m_granularity": { 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_m_granularity", 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.189 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "dump_on_update": { 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dump_on_update", 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.190 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.191 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.193 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.194 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.195 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.196 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stats", 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.197 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.198 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.199 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.200 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.201 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.202 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "status", 2024-09-10T10:02:15.203 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.204 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.205 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.206 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.207 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telegraf", 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "address": { 2024-09-10T10:02:15.209 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "address", 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.210 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15", 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.212 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.213 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.214 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.215 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.216 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telemetry", 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_basic": { 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_basic", 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.218 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_crash": { 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_crash", 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:02:15.219 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_device": { 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_device", 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.220 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_ident": { 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_ident", 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.221 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_perf": { 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_perf", 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.222 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "contact": { 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "contact", 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.223 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "description": { 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "description", 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "device_url": { 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_url", 2024-09-10T10:02:15.225 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-09-10T10:02:15.226 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.227 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "24", 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "8", 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.228 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "last_opt_revision": { 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "last_opt_revision", 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1", 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.229 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard": { 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard", 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.230 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard_description": { 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard_description", 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.231 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.232 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.233 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.234 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.235 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.237 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "organization": { 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "organization", 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.238 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "proxy": { 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "proxy", 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.239 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "url": { 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url", 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.240 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "test_orchestrator", 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.241 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.242 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.243 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.244 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.245 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.246 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.247 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.248 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.249 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.250 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.251 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.252 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_clones": { 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_clones", 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.253 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4", 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "periodic_async_work": { 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "periodic_async_work", 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.254 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Periodically check for async work", 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_delay": { 2024-09-10T10:02:15.255 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.256 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.257 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix", 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: "discovery_interval": { 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "discovery_interval", 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-09-10T10:02:15.258 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "100", 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "identifier": { 2024-09-10T10:02:15.259 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "identifier", 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.260 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_host": { 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_host", 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.268 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_port": { 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_port", 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.269 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10051", 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_sender": { 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_sender", 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "services": {}, 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "always_on_modules": { 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "octopus": [ 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-09-10T10:02:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "pacific": [ 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-09-10T10:02:15.273 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "quincy": [ 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-09-10T10:02:15.274 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "reef": [ 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "squid": [ 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-09-10T10:02:15.276 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: "last_failure_osd_epoch": 0, 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: "active_clients": [ 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "devicehealth", 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 2970528962 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.277 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "libcephsqlite", 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 3823528576 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_support", 2024-09-10T10:02:15.278 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 3209652493 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.279 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 3716204142 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 3695412383 2024-09-10T10:02:15.280 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 69305518 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.281 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 4153756567 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.282 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 543422288 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 1317276712 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.283 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 4258153839 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:15.284 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:15.285 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:02:15.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4838067c20 msgr2=0x7f483806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4838067c20 0x7f483806a0e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4848001570 tx=0x7f4848009040 comp rx=0 tx=0).stop 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4858147f00 msgr2=0x7f485813d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4858147f00 0x7f485813d970 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f48500ae040 tx=0x7f48500191a0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 shutdown_connections 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f4838067c20 0x7f483806a0e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4858147f00 0x7f485813d970 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48580aef50 0x7f485813f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 --2- 172.21.15.110:0/4216361069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806edf0 0x7f4858161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 >> 172.21.15.110:0/4216361069 conn(0x7f485806bc50 msgr2=0x7f485806d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 shutdown_connections 2024-09-10T10:02:15.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:14.368+0000 7f485d400640 1 -- 172.21.15.110:0/4216361069 wait complete. 2024-09-10T10:02:15.292 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail y 2024-09-10T10:02:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.436+0000 7f34e1200640 1 Processor -- start 2024-09-10T10:02:15.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.436+0000 7f34e1200640 1 -- start start 2024-09-10T10:02:15.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34e1200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc15aab0 0x7f34dc072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34e1200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f34dc0b4170 0x7f34dc0b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34e1200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f34dc0731c0 0x7f34dc0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34e1200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f34dc15ae90 con 0x7f34dc0731c0 2024-09-10T10:02:15.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34e1200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f34dc15b170 con 0x7f34dc0b4170 2024-09-10T10:02:15.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34e1200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc15b450 con 0x7f34dc15aab0 2024-09-10T10:02:15.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34da800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f34dc0731c0 0x7f34dc0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34da800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f34dc0731c0 0x7f34dc0b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55834/0 (socket says 172.21.15.110:55834) 2024-09-10T10:02:15.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34da800640 1 -- 172.21.15.110:0/1086977312 learned_addr learned my addr 172.21.15.110:0/1086977312 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:15.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34d9e00640 1 --2- 172.21.15.110:0/1086977312 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc15aab0 0x7f34dc072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34d9400640 1 --2- 172.21.15.110:0/1086977312 >> v2:172.21.15.145:3300/0 conn(0x7f34dc0b4170 0x7f34dc0b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34da800640 1 -- 172.21.15.110:0/1086977312 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc15aab0 msgr2=0x7f34dc072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:15.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34da800640 1 --2- 172.21.15.110:0/1086977312 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc15aab0 0x7f34dc072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34da800640 1 -- 172.21.15.110:0/1086977312 >> v2:172.21.15.145:3300/0 conn(0x7f34dc0b4170 msgr2=0x7f34dc0b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:15.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34da800640 1 --2- 172.21.15.110:0/1086977312 >> v2:172.21.15.145:3300/0 conn(0x7f34dc0b4170 0x7f34dc0b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.440+0000 7f34da800640 1 -- 172.21.15.110:0/1086977312 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc15b730 con 0x7f34dc0731c0 2024-09-10T10:02:15.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34da800640 1 --2- 172.21.15.110:0/1086977312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0731c0 0x7f34dc0b4b10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0afc80 tx=0x7f34d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abed1143825f458e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34d8a00640 1 -- 172.21.15.110:0/1086977312 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40ab5f0 con 0x7f34dc0731c0 2024-09-10T10:02:15.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34d8a00640 1 -- 172.21.15.110:0/1086977312 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d40b0070 con 0x7f34dc0731c0 2024-09-10T10:02:15.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34d8a00640 1 -- 172.21.15.110:0/1086977312 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40ab9b0 con 0x7f34dc0731c0 2024-09-10T10:02:15.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- 172.21.15.110:0/1086977312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0731c0 msgr2=0x7f34dc0b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 --2- 172.21.15.110:0/1086977312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0731c0 0x7f34dc0b4b10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0afc80 tx=0x7f34d401c520 comp rx=0 tx=0).stop 2024-09-10T10:02:15.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- 172.21.15.110:0/1086977312 shutdown_connections 2024-09-10T10:02:15.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 --2- 172.21.15.110:0/1086977312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0731c0 0x7f34dc0b4b10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 --2- 172.21.15.110:0/1086977312 >> v2:172.21.15.145:3300/0 conn(0x7f34dc0b4170 0x7f34dc0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 --2- 172.21.15.110:0/1086977312 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc15aab0 0x7f34dc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- 172.21.15.110:0/1086977312 >> 172.21.15.110:0/1086977312 conn(0x7f34dc062d10 msgr2=0x7f34dc06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- 172.21.15.110:0/1086977312 shutdown_connections 2024-09-10T10:02:15.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- 172.21.15.110:0/1086977312 wait complete. 2024-09-10T10:02:15.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 Processor -- start 2024-09-10T10:02:15.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- start start 2024-09-10T10:02:15.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc0731c0 0x7f34dc0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0b4170 0x7f34dc152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34dc15aab0 0x7f34dc0cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc06dcd0 con 0x7f34dc0b4170 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc06db50 con 0x7f34dc15aab0 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.444+0000 7f34e1200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc06de50 con 0x7f34dc0731c0 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0b4170 0x7f34dc152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0b4170 0x7f34dc152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55836/0 (socket says 172.21.15.110:55836) 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 -- 172.21.15.110:0/3834622382 learned_addr learned my addr 172.21.15.110:0/3834622382 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9e00640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc0731c0 0x7f34dc0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34da800640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34dc15aab0 0x7f34dc0cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 -- 172.21.15.110:0/3834622382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc0731c0 msgr2=0x7f34dc0b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc0731c0 0x7f34dc0b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 -- 172.21.15.110:0/3834622382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34dc15aab0 msgr2=0x7f34dc0cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34dc15aab0 0x7f34dc0cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 -- 172.21.15.110:0/3834622382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc0b1370 con 0x7f34dc0b4170 2024-09-10T10:02:15.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d9400640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0b4170 0x7f34dc152fb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f34c400e850 tx=0x7f34c400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d2000640 1 -- 172.21.15.110:0/3834622382 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4017070 con 0x7f34dc0b4170 2024-09-10T10:02:15.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d2000640 1 -- 172.21.15.110:0/3834622382 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c400c940 con 0x7f34dc0b4170 2024-09-10T10:02:15.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d2000640 1 -- 172.21.15.110:0/3834622382 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c400ccf0 con 0x7f34dc0b4170 2024-09-10T10:02:15.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc0c7100 con 0x7f34dc0b4170 2024-09-10T10:02:15.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc0c76c0 con 0x7f34dc0b4170 2024-09-10T10:02:15.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34dc072c80 con 0x7f34dc0b4170 2024-09-10T10:02:15.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.448+0000 7f34d2000640 1 -- 172.21.15.110:0/3834622382 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f34c401a070 con 0x7f34dc0b4170 2024-09-10T10:02:15.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.452+0000 7f34d2000640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34b8067c20 0x7f34b806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.452+0000 7f34d2000640 1 -- 172.21.15.110:0/3834622382 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5796+0+0 (secure 0 0 0) 0x7f34c4086170 con 0x7f34dc0b4170 2024-09-10T10:02:15.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.452+0000 7f34d9e00640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34b8067c20 0x7f34b806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.452+0000 7f34d9e00640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34b8067c20 0x7f34b806a0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f34cc011340 tx=0x7f34cc00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.456+0000 7f34d2000640 1 -- 172.21.15.110:0/3834622382 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f34c40561b0 con 0x7f34dc0b4170 2024-09-10T10:02:15.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.652+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr fail", "who": "y"} v 0) -- 0x7f34dc06f260 con 0x7f34dc0b4170 2024-09-10T10:02:15.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.656+0000 7f34d2000640 1 -- 172.21.15.110:0/3834622382 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "y"}]=0 v12) ==== 68+0+0 (secure 0 0 0) 0x7f34c405a4b0 con 0x7f34dc0b4170 2024-09-10T10:02:15.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.656+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34b8067c20 msgr2=0x7f34b806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.656+0000 7f34e1200640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34b8067c20 0x7f34b806a0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f34cc011340 tx=0x7f34cc00b040 comp rx=0 tx=0).stop 2024-09-10T10:02:15.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.656+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0b4170 msgr2=0x7f34dc152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.656+0000 7f34e1200640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0b4170 0x7f34dc152fb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f34c400e850 tx=0x7f34c400ed10 comp rx=0 tx=0).stop 2024-09-10T10:02:15.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.660+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 shutdown_connections 2024-09-10T10:02:15.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.660+0000 7f34e1200640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:6832/952833175,v1:172.21.15.110:6833/952833175] conn(0x7f34b8067c20 0x7f34b806a0e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.660+0000 7f34e1200640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f34dc15aab0 0x7f34dc0cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.660+0000 7f34e1200640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34dc0b4170 0x7f34dc152fb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.660+0000 7f34e1200640 1 --2- 172.21.15.110:0/3834622382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f34dc0731c0 0x7f34dc0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.660+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 >> 172.21.15.110:0/3834622382 conn(0x7f34dc062d10 msgr2=0x7f34dc155e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.660+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 shutdown_connections 2024-09-10T10:02:15.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.660+0000 7f34e1200640 1 -- 172.21.15.110:0/3834622382 wait complete. 2024-09-10T10:02:15.708 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:15.781 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.781 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.781 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.781 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.781 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.782 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.782 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.782 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.782 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.782 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:02:15.776+0000 7f95e3400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:02:15.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.856+0000 7feef3200640 1 Processor -- start 2024-09-10T10:02:15.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.856+0000 7feef3200640 1 -- start start 2024-09-10T10:02:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef3200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feeec145b00 0x7feeec0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef3200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec0b5690 0x7feeec0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef3200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7feeec15c940 0x7feeec15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef3200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7feeec0b6010 con 0x7feeec145b00 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef3200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7feeec071db0 con 0x7feeec15c940 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef3200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feeec0ce5f0 con 0x7feeec0b5690 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feeec145b00 0x7feeec0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feeec145b00 0x7feeec0cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55848/0 (socket says 172.21.15.110:55848) 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1e00640 1 -- 172.21.15.110:0/1032427990 learned_addr learned my addr 172.21.15.110:0/1032427990 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1400640 1 --2- 172.21.15.110:0/1032427990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec0b5690 0x7feeec0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef2800640 1 --2- 172.21.15.110:0/1032427990 >> v2:172.21.15.145:3300/0 conn(0x7feeec15c940 0x7feeec15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1e00640 1 -- 172.21.15.110:0/1032427990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec0b5690 msgr2=0x7feeec0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:15.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1e00640 1 --2- 172.21.15.110:0/1032427990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec0b5690 0x7feeec0b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1e00640 1 -- 172.21.15.110:0/1032427990 >> v2:172.21.15.145:3300/0 conn(0x7feeec15c940 msgr2=0x7feeec15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:15.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1e00640 1 --2- 172.21.15.110:0/1032427990 >> v2:172.21.15.145:3300/0 conn(0x7feeec15c940 0x7feeec15cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.860+0000 7feef1e00640 1 -- 172.21.15.110:0/1032427990 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec0ce8d0 con 0x7feeec145b00 2024-09-10T10:02:15.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef1e00640 1 --2- 172.21.15.110:0/1032427990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec145b00 0x7feeec0cf0d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7feeec0b63b0 tx=0x7feee001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6a1691cc384d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef0a00640 1 -- 172.21.15.110:0/1032427990 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00b0070 con 0x7feeec145b00 2024-09-10T10:02:15.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef0a00640 1 -- 172.21.15.110:0/1032427990 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee00ab600 con 0x7feeec145b00 2024-09-10T10:02:15.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef0a00640 1 -- 172.21.15.110:0/1032427990 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00ab9d0 con 0x7feeec145b00 2024-09-10T10:02:15.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 -- 172.21.15.110:0/1032427990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec145b00 msgr2=0x7feeec0cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 --2- 172.21.15.110:0/1032427990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec145b00 0x7feeec0cf0d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7feeec0b63b0 tx=0x7feee001c520 comp rx=0 tx=0).stop 2024-09-10T10:02:15.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 -- 172.21.15.110:0/1032427990 shutdown_connections 2024-09-10T10:02:15.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 --2- 172.21.15.110:0/1032427990 >> v2:172.21.15.145:3300/0 conn(0x7feeec15c940 0x7feeec15cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 --2- 172.21.15.110:0/1032427990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec0b5690 0x7feeec0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 --2- 172.21.15.110:0/1032427990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec145b00 0x7feeec0cf0d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 -- 172.21.15.110:0/1032427990 >> 172.21.15.110:0/1032427990 conn(0x7feeec06d0f0 msgr2=0x7feeec06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 -- 172.21.15.110:0/1032427990 shutdown_connections 2024-09-10T10:02:15.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 -- 172.21.15.110:0/1032427990 wait complete. 2024-09-10T10:02:15.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 Processor -- start 2024-09-10T10:02:15.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.864+0000 7feef3200640 1 -- start start 2024-09-10T10:02:15.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec0b5690 0x7feeec143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec145b00 0x7feeec15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feeec15c940 0x7feeec0aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feeec0cf790 con 0x7feeec0b5690 2024-09-10T10:02:15.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feeec0cf610 con 0x7feeec15c940 2024-09-10T10:02:15.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feeec071850 con 0x7feeec145b00 2024-09-10T10:02:15.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec0b5690 0x7feeec143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec0b5690 0x7feeec143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55860/0 (socket says 172.21.15.110:55860) 2024-09-10T10:02:15.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 -- 172.21.15.110:0/3207242382 learned_addr learned my addr 172.21.15.110:0/3207242382 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:15.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef2800640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feeec15c940 0x7feeec0aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1400640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec145b00 0x7feeec15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 -- 172.21.15.110:0/3207242382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec145b00 msgr2=0x7feeec15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec145b00 0x7feeec15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 -- 172.21.15.110:0/3207242382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feeec15c940 msgr2=0x7feeec0aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feeec15c940 0x7feeec0aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 -- 172.21.15.110:0/3207242382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec144460 con 0x7feeec0b5690 2024-09-10T10:02:15.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef1e00640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec0b5690 0x7feeec143c10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7feee00ae040 tx=0x7feee001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feee6000640 1 -- 172.21.15.110:0/3207242382 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00ba020 con 0x7feeec0b5690 2024-09-10T10:02:15.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feee6000640 1 -- 172.21.15.110:0/3207242382 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee00b6420 con 0x7feeec0b5690 2024-09-10T10:02:15.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feee6000640 1 -- 172.21.15.110:0/3207242382 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00b0040 con 0x7feeec0b5690 2024-09-10T10:02:15.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec144740 con 0x7feeec0b5690 2024-09-10T10:02:15.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec144d00 con 0x7feeec0b5690 2024-09-10T10:02:15.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feee6000640 1 -- 172.21.15.110:0/3207242382 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 12) ==== 43764+0+0 (secure 0 0 0) 0x7feee00bd020 con 0x7feeec0b5690 2024-09-10T10:02:15.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeec0cf0d0 con 0x7feeec0b5690 2024-09-10T10:02:15.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.868+0000 7feee6000640 1 -- 172.21.15.110:0/3207242382 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6355+0+0 (secure 0 0 0) 0x7feee00e2ec0 con 0x7feeec0b5690 2024-09-10T10:02:15.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:15.876+0000 7feee6000640 1 -- 172.21.15.110:0/3207242382 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feee00c4030 con 0x7feeec0b5690 2024-09-10T10:02:15.882 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:15.880+0000 7f37896c1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:02:16.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.076+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feeec06d7b0 con 0x7feeec0b5690 2024-09-10T10:02:16.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.076+0000 7feee6000640 1 -- 172.21.15.110:0/3207242382 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5237 (secure 0 0 0) 0x7feee0016110 con 0x7feeec0b5690 2024-09-10T10:02:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:16.081 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":46,"btime":"2024-09-10T10:02:10:790959+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:16.081 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 46 2024-09-10T10:02:16.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec0b5690 msgr2=0x7feeec143c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec0b5690 0x7feeec143c10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7feee00ae040 tx=0x7feee001ad60 comp rx=0 tx=0).stop 2024-09-10T10:02:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 shutdown_connections 2024-09-10T10:02:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feeec15c940 0x7feeec0aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feeec145b00 0x7feeec15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 --2- 172.21.15.110:0/3207242382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec0b5690 0x7feeec143c10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 >> 172.21.15.110:0/3207242382 conn(0x7feeec06d0f0 msgr2=0x7feeec0af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 shutdown_connections 2024-09-10T10:02:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.080+0000 7feef3200640 1 -- 172.21.15.110:0/3207242382 wait complete. 2024-09-10T10:02:16.103 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T10:02:16.262 INFO:teuthology.orchestra.run.smithi145.stdout:[{"id":5537,"entity":{"name":{"type":"client","num":5537},"addr":{"type":"any","addr":"172.21.15.145:0","nonce":3582326868}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.46399377600000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978755063165552,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5537 172.21.15.145:0/3582326868","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"x","hostname":"smithi145","pid":"28864","root":"/"}},{"id":6300,"entity":{"name":{"type":"client","num":6300},"addr":{"type":"any","addr":"172.21.15.145:0","nonce":3201797384}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.46399377600000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978755063165552,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6300 172.21.15.145:0/3201797384","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1cf819b7212a6048e92fc78c4eff963344f6d56b","ceph_version":"ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)","entity_id":"x","hostname":"smithi145","pid":"28864","root":"/"}}] 2024-09-10T10:02:16.262 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi145", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28864", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.145:0", "nonce": 3582326868, "type": "any" }, "name": { "num": 5537, "type": "client" } }, "id": 5537, "inst": "client.5537 172.21.15.145:0/3582326868", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978755063165552 }, "state": "open", "uptime": 0.463993776 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi145", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28864", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.145:0", "nonce": 3201797384, "type": "any" }, "name": { "num": 6300, "type": "client" } }, "id": 6300, "inst": "client.6300 172.21.15.145:0/3201797384", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978755063165552 }, "state": "open", "uptime": 0.463993776 } ] 2024-09-10T10:02:16.262 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:02:16.263 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:02:16.263 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:02:16.312 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:16.308+0000 7f37896c1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:02:16.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:02:16.425 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:02:16.425 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:02:16.425 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:16.425 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:16.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:02:16.434 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:02:16.435 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:16.435 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:16.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:02:16.444 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:02:16.444 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:16.444 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:16.462 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:16.460+0000 7f37896c1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:02:16.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:02:16.470 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:02:16.470 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:16.470 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:16.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:02:16.494 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:16.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf0d000640 1 Processor -- start 2024-09-10T10:02:16.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf0d000640 1 -- start start 2024-09-10T10:02:16.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf0d000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcf0806e7a0 0x7fcf080cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf0d000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf08072150 con 0x7fcf080cfe30 2024-09-10T10:02:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf0d000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0806d2d0 con 0x7fcf080cd510 2024-09-10T10:02:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf0d000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0815d9a0 con 0x7fcf0806e7a0 2024-09-10T10:02:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf05e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcf080cfe30 0x7fcf08071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45072/0 (socket says 172.21.15.110:45072) 2024-09-10T10:02:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf05e00640 1 -- 172.21.15.110:0/3917068304 learned_addr learned my addr 172.21.15.110:0/3917068304 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf06800640 1 --2- 172.21.15.110:0/3917068304 >> v2:172.21.15.110:3300/0 conn(0x7fcf0806e7a0 0x7fcf080cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf05400640 1 -- 172.21.15.110:0/3917068304 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 303709901 0 0) 0x7fcf08072150 con 0x7fcf080cfe30 2024-09-10T10:02:16.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf05400640 1 -- 172.21.15.110:0/3917068304 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceec003530 con 0x7fcf080cfe30 2024-09-10T10:02:16.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf06800640 1 -- 172.21.15.110:0/3917068304 >> v1:172.21.15.145:6789/0 conn(0x7fcf080cd510 legacy=0x7fcf0806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf06800640 1 -- 172.21.15.110:0/3917068304 >> v1:172.21.15.110:6789/0 conn(0x7fcf080cfe30 legacy=0x7fcf08071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.648+0000 7fcf06800640 1 -- 172.21.15.110:0/3917068304 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0806d5b0 con 0x7fcf0806e7a0 2024-09-10T10:02:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf06800640 1 --2- 172.21.15.110:0/3917068304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf0806e7a0 0x7fcf080cd9a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009d90 tx=0x7fcf0001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a240801fe83b30b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:16.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf05400640 1 -- 172.21.15.110:0/3917068304 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000b2070 con 0x7fcf0806e7a0 2024-09-10T10:02:16.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf05400640 1 -- 172.21.15.110:0/3917068304 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf000a58b0 con 0x7fcf0806e7a0 2024-09-10T10:02:16.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf05400640 1 -- 172.21.15.110:0/3917068304 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000a5c60 con 0x7fcf0806e7a0 2024-09-10T10:02:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf0d000640 1 -- 172.21.15.110:0/3917068304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf0806e7a0 msgr2=0x7fcf080cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf0d000640 1 --2- 172.21.15.110:0/3917068304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf0806e7a0 0x7fcf080cd9a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009d90 tx=0x7fcf0001c8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf0d000640 1 -- 172.21.15.110:0/3917068304 shutdown_connections 2024-09-10T10:02:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf0d000640 1 --2- 172.21.15.110:0/3917068304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf0806e7a0 0x7fcf080cd9a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf0d000640 1 -- 172.21.15.110:0/3917068304 >> 172.21.15.110:0/3917068304 conn(0x7fcf0806bc50 msgr2=0x7fcf0806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:16.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf0d000640 1 -- 172.21.15.110:0/3917068304 shutdown_connections 2024-09-10T10:02:16.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.652+0000 7fcf0d000640 1 -- 172.21.15.110:0/3917068304 wait complete. 2024-09-10T10:02:16.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 Processor -- start 2024-09-10T10:02:16.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 -- start start 2024-09-10T10:02:16.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf0806e7a0 0x7fcf08149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf080cd510 0x7fcf08157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf080cfe30 0x7fcf08155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0816a190 con 0x7fcf080cd510 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0815d700 con 0x7fcf0806e7a0 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0816a310 con 0x7fcf080cfe30 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf080cd510 0x7fcf08157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf080cd510 0x7fcf08157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55888/0 (socket says 172.21.15.110:55888) 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 -- 172.21.15.110:0/3467402488 learned_addr learned my addr 172.21.15.110:0/3467402488 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf05e00640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf0806e7a0 0x7fcf08149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf06800640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf080cfe30 0x7fcf08155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 -- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf080cfe30 msgr2=0x7fcf08155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf080cfe30 0x7fcf08155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 -- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf0806e7a0 msgr2=0x7fcf08149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf0806e7a0 0x7fcf08149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 -- 172.21.15.110:0/3467402488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0814f500 con 0x7fcf080cd510 2024-09-10T10:02:16.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fceffe00640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf080cd510 0x7fcf08157050 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c370 tx=0x7fcef000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:16.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcefe000640 1 -- 172.21.15.110:0/3467402488 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00052d0 con 0x7fcf080cd510 2024-09-10T10:02:16.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcefe000640 1 -- 172.21.15.110:0/3467402488 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef0005470 con 0x7fcf080cd510 2024-09-10T10:02:16.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcefe000640 1 -- 172.21.15.110:0/3467402488 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0007260 con 0x7fcf080cd510 2024-09-10T10:02:16.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf080b0410 con 0x7fcf080cd510 2024-09-10T10:02:16.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.656+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0814f120 con 0x7fcf080cd510 2024-09-10T10:02:16.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.660+0000 7fcefe000640 1 -- 172.21.15.110:0/3467402488 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 12) ==== 43764+0+0 (secure 0 0 0) 0x7fcef0007400 con 0x7fcf080cd510 2024-09-10T10:02:16.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.660+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf080b0200 con 0x7fcf080cd510 2024-09-10T10:02:16.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.660+0000 7fcefe000640 1 -- 172.21.15.110:0/3467402488 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6355+0+0 (secure 0 0 0) 0x7fcef00482d0 con 0x7fcf080cd510 2024-09-10T10:02:16.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.660+0000 7fcefe000640 1 -- 172.21.15.110:0/3467402488 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mgrmap(e 13) ==== 43891+0+0 (secure 0 0 0) 0x7fcef0003600 con 0x7fcf080cd510 2024-09-10T10:02:16.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.660+0000 7fcefe000640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcee403b6e0 0x7fcee403dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.664+0000 7fcf05e00640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcee403b6e0 0x7fcee403dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.668+0000 7fcf05e00640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcee403b6e0 0x7fcee403dad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcef80011f0 tx=0x7fcef8007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:16.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.668+0000 7fcefe000640 1 -- 172.21.15.110:0/3467402488 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcef001f170 con 0x7fcf080cd510 2024-09-10T10:02:16.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.864+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf0806e090 con 0x7fcf080cd510 2024-09-10T10:02:16.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.864+0000 7fcefe000640 1 -- 172.21.15.110:0/3467402488 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5237 (secure 0 0 0) 0x7fcef004ddb0 con 0x7fcf080cd510 2024-09-10T10:02:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:16.867 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":46,"btime":"2024-09-10T10:02:10:790959+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:16.867 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 46 2024-09-10T10:02:16.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.864+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcee403b6e0 msgr2=0x7fcee403dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.864+0000 7fcf0d000640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcee403b6e0 0x7fcee403dad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcef80011f0 tx=0x7fcef8007d40 comp rx=0 tx=0).stop 2024-09-10T10:02:16.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.864+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf080cd510 msgr2=0x7fcf08157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.864+0000 7fcf0d000640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf080cd510 0x7fcf08157050 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c370 tx=0x7fcef000c830 comp rx=0 tx=0).stop 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.868+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 shutdown_connections 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.868+0000 7fcf0d000640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcee403b6e0 0x7fcee403dad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.868+0000 7fcf0d000640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf080cfe30 0x7fcf08155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.868+0000 7fcf0d000640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf080cd510 0x7fcf08157050 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.868+0000 7fcf0d000640 1 --2- 172.21.15.110:0/3467402488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf0806e7a0 0x7fcf08149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.868+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 >> 172.21.15.110:0/3467402488 conn(0x7fcf0806bc50 msgr2=0x7fcf08071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.868+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 shutdown_connections 2024-09-10T10:02:16.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:16.868+0000 7fcf0d000640 1 -- 172.21.15.110:0/3467402488 wait complete. 2024-09-10T10:02:16.890 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:16.953 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:16.952+0000 7f37896c1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:02:17.013 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.012+0000 7f37896c1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:02:17.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18bce00640 1 Processor -- start 2024-09-10T10:02:17.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18bce00640 1 -- start start 2024-09-10T10:02:17.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18bce00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18bce00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f18b806f010 0x7f18b80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18bce00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b806f430 con 0x7f18b8155970 2024-09-10T10:02:17.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18bce00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f18b80b4570 con 0x7f18b806f010 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18bce00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f18b8155d50 con 0x7f18b815c560 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b5400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b6800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f18b806f010 0x7f18b80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b5400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:38574/0 (socket says 172.21.15.110:38574) 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b5400640 1 -- 172.21.15.110:0/2962093823 learned_addr learned my addr 172.21.15.110:0/2962093823 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b5e00640 1 --1- 172.21.15.110:0/2962093823 >> v1:172.21.15.145:6789/0 conn(0x7f18b8155970 0x7f18b80cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:36034/0 (socket says 172.21.15.110:36034) 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b5400640 1 -- 172.21.15.110:0/2962093823 >> v2:172.21.15.145:3300/0 conn(0x7f18b806f010 msgr2=0x7f18b80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b5400640 1 --2- 172.21.15.110:0/2962093823 >> v2:172.21.15.145:3300/0 conn(0x7f18b806f010 0x7f18b80ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b5400640 1 -- 172.21.15.110:0/2962093823 >> v1:172.21.15.145:6789/0 conn(0x7f18b8155970 legacy=0x7f18b80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b5400640 1 -- 172.21.15.110:0/2962093823 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b80b34d0 con 0x7f18b815c560 2024-09-10T10:02:17.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.028+0000 7f18b4a00640 1 -- 172.21.15.110:0/2962093823 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001746532 0 0) 0x7f18b806f430 con 0x7f18b8155970 2024-09-10T10:02:17.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18b5400640 1 --2- 172.21.15.110:0/2962093823 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b806ead0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009870 tx=0x7f18a401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a69e0d930aa560f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18b4a00640 1 -- 172.21.15.110:0/2962093823 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40aa600 con 0x7f18b815c560 2024-09-10T10:02:17.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18b4a00640 1 -- 172.21.15.110:0/2962093823 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a40af070 con 0x7f18b815c560 2024-09-10T10:02:17.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18b4a00640 1 -- 172.21.15.110:0/2962093823 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40aa9c0 con 0x7f18b815c560 2024-09-10T10:02:17.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18bce00640 1 -- 172.21.15.110:0/2962093823 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 msgr2=0x7f18b806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18bce00640 1 --2- 172.21.15.110:0/2962093823 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b806ead0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009870 tx=0x7f18a401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18bce00640 1 -- 172.21.15.110:0/2962093823 shutdown_connections 2024-09-10T10:02:17.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18bce00640 1 --2- 172.21.15.110:0/2962093823 >> v2:172.21.15.145:3300/0 conn(0x7f18b806f010 0x7f18b80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18bce00640 1 --2- 172.21.15.110:0/2962093823 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b806ead0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18bce00640 1 -- 172.21.15.110:0/2962093823 >> 172.21.15.110:0/2962093823 conn(0x7f18b806bc50 msgr2=0x7f18b806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:17.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18bce00640 1 -- 172.21.15.110:0/2962093823 shutdown_connections 2024-09-10T10:02:17.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.032+0000 7f18bce00640 1 -- 172.21.15.110:0/2962093823 wait complete. 2024-09-10T10:02:17.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 Processor -- start 2024-09-10T10:02:17.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 -- start start 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18b806f010 0x7f18b8152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18b8155970 0x7f18b814af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b8150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f18b80cf170 con 0x7f18b806f010 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18b80b4050 con 0x7f18b8155970 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f18b80cf2f0 con 0x7f18b815c560 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18b806f010 0x7f18b8152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18b806f010 0x7f18b8152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55904/0 (socket says 172.21.15.110:55904) 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 -- 172.21.15.110:0/355951635 learned_addr learned my addr 172.21.15.110:0/355951635 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5400640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18b8155970 0x7f18b814af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b6800640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b8150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 -- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 msgr2=0x7f18b8150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b8150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 -- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18b8155970 msgr2=0x7f18b814af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18b8155970 0x7f18b814af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 -- 172.21.15.110:0/355951635 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b8167240 con 0x7f18b806f010 2024-09-10T10:02:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b6800640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b8150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18b5e00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18b806f010 0x7f18b8152730 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c4d0 tx=0x7f18a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18ae000640 1 -- 172.21.15.110:0/355951635 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8018070 con 0x7f18b806f010 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18ae000640 1 -- 172.21.15.110:0/355951635 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a80074e0 con 0x7f18b806f010 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18ae000640 1 -- 172.21.15.110:0/355951635 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004a30 con 0x7f18b806f010 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b8167520 con 0x7f18b806f010 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b8167ae0 con 0x7f18b806f010 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18ae000640 1 -- 172.21.15.110:0/355951635 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 43891+0+0 (secure 0 0 0) 0x7f18a8004bd0 con 0x7f18b806f010 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.036+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b80ce420 con 0x7f18b806f010 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.040+0000 7f18ae000640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1898035d10 0x7f18980381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.040+0000 7f18ae000640 1 -- 172.21.15.110:0/355951635 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6355+0+0 (secure 0 0 0) 0x7f18a80482b0 con 0x7f18b806f010 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.040+0000 7f18b5400640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1898035d10 0x7f18980381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.040+0000 7f18b5400640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1898035d10 0x7f18980381d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f18a40ae000 tx=0x7f18a40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.044+0000 7f18ae000640 1 -- 172.21.15.110:0/355951635 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f18a80104c0 con 0x7f18b806f010 2024-09-10T10:02:17.073 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.072+0000 7f37896c1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:02:17.160 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.156+0000 7f37896c1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:02:17.217 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.212+0000 7f37896c1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:02:17.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18b814bc20 con 0x7f18b806f010 2024-09-10T10:02:17.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18ae000640 1 -- 172.21.15.110:0/355951635 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5237 (secure 0 0 0) 0x7f18a801bcf0 con 0x7f18b806f010 2024-09-10T10:02:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:17.239 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":46,"btime":"2024-09-10T10:02:10:790959+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:17.239 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 46 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1898035d10 msgr2=0x7f18980381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1898035d10 0x7f18980381d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f18a40ae000 tx=0x7f18a40a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18b806f010 msgr2=0x7f18b8152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18b806f010 0x7f18b8152730 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c4d0 tx=0x7f18a800c990 comp rx=0 tx=0).stop 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 shutdown_connections 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1898035d10 0x7f18980381d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f18b815c560 0x7f18b8150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18b8155970 0x7f18b814af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 --2- 172.21.15.110:0/355951635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18b806f010 0x7f18b8152730 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 >> 172.21.15.110:0/355951635 conn(0x7f18b806bc50 msgr2=0x7f18b806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:17.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 shutdown_connections 2024-09-10T10:02:17.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.236+0000 7f18bce00640 1 -- 172.21.15.110:0/355951635 wait complete. 2024-09-10T10:02:17.260 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:17.305 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.304+0000 7f37896c1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:02:17.368 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.364+0000 7f37896c1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:02:17.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16ff200640 1 Processor -- start 2024-09-10T10:02:17.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16ff200640 1 -- start start 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16ff200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16f8148120 0x7f16f806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16ff200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80ae840 0x7f16f80aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16ff200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f80702b0 con 0x7f16f80b0890 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16ff200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f16f806eb00 con 0x7f16f8148120 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16ff200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f16f806ede0 con 0x7f16f80ae840 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16f7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16f8148120 0x7f16f806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16f7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16f8148120 0x7f16f806e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55916/0 (socket says 172.21.15.110:55916) 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16f7e00640 1 -- 172.21.15.110:0/3926590259 learned_addr learned my addr 172.21.15.110:0/3926590259 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:17.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16fd200640 1 --2- 172.21.15.110:0/3926590259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80ae840 0x7f16f80aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16f7e00640 1 -- 172.21.15.110:0/3926590259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80ae840 msgr2=0x7f16f80aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:17.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16f7e00640 1 --2- 172.21.15.110:0/3926590259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80ae840 0x7f16f80aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16f7e00640 1 -- 172.21.15.110:0/3926590259 >> v1:172.21.15.145:6789/0 conn(0x7f16f80b0890 legacy=0x7f16f806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16f7e00640 1 -- 172.21.15.110:0/3926590259 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f806f0c0 con 0x7f16f8148120 2024-09-10T10:02:17.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.396+0000 7f16f7400640 1 -- 172.21.15.110:0/3926590259 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2784015486 0 0) 0x7f16f80702b0 con 0x7f16f80b0890 2024-09-10T10:02:17.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16f7e00640 1 --2- 172.21.15.110:0/3926590259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f806e1b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009870 tx=0x7f16e801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f05863e8b619b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16f7400640 1 -- 172.21.15.110:0/3926590259 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e80b2070 con 0x7f16f8148120 2024-09-10T10:02:17.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16f7400640 1 -- 172.21.15.110:0/3926590259 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e80a9070 con 0x7f16f8148120 2024-09-10T10:02:17.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16f7400640 1 -- 172.21.15.110:0/3926590259 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e80ad880 con 0x7f16f8148120 2024-09-10T10:02:17.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16ff200640 1 -- 172.21.15.110:0/3926590259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 msgr2=0x7f16f806e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16ff200640 1 --2- 172.21.15.110:0/3926590259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f806e1b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009870 tx=0x7f16e801c560 comp rx=0 tx=0).stop 2024-09-10T10:02:17.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16ff200640 1 -- 172.21.15.110:0/3926590259 shutdown_connections 2024-09-10T10:02:17.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16ff200640 1 --2- 172.21.15.110:0/3926590259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80ae840 0x7f16f80aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16ff200640 1 --2- 172.21.15.110:0/3926590259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f806e1b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16ff200640 1 -- 172.21.15.110:0/3926590259 >> 172.21.15.110:0/3926590259 conn(0x7f16f806cf60 msgr2=0x7f16f806d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:17.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16ff200640 1 -- 172.21.15.110:0/3926590259 shutdown_connections 2024-09-10T10:02:17.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.400+0000 7f16ff200640 1 -- 172.21.15.110:0/3926590259 wait complete. 2024-09-10T10:02:17.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 Processor -- start 2024-09-10T10:02:17.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 -- start start 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16f80ae840 0x7f16f813e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80b0890 0x7f16f815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f8143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16f816b550 con 0x7f16f8148120 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f16f816b3d0 con 0x7f16f80ae840 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f16f816b6d0 con 0x7f16f80b0890 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f8143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f8143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55924/0 (socket says 172.21.15.110:55924) 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 -- 172.21.15.110:0/528156511 learned_addr learned my addr 172.21.15.110:0/528156511 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16f7e00640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16f80ae840 0x7f16f813e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16efe00640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80b0890 0x7f16f815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 -- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80b0890 msgr2=0x7f16f815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80b0890 0x7f16f815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 -- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16f80ae840 msgr2=0x7f16f813e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16f80ae840 0x7f16f813e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 -- 172.21.15.110:0/528156511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f8146900 con 0x7f16f8148120 2024-09-10T10:02:17.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16efe00640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80b0890 0x7f16f815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:17.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16fd200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f8143650 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f16f000efc0 tx=0x7f16f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16f4c00640 1 -- 172.21.15.110:0/528156511 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0018020 con 0x7f16f8148120 2024-09-10T10:02:17.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16f4c00640 1 -- 172.21.15.110:0/528156511 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f0014480 con 0x7f16f8148120 2024-09-10T10:02:17.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f813e840 con 0x7f16f8148120 2024-09-10T10:02:17.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16f4c00640 1 -- 172.21.15.110:0/528156511 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0014870 con 0x7f16f8148120 2024-09-10T10:02:17.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.404+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f8146ea0 con 0x7f16f8148120 2024-09-10T10:02:17.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.408+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f80aecd0 con 0x7f16f8148120 2024-09-10T10:02:17.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.408+0000 7f16f4c00640 1 -- 172.21.15.110:0/528156511 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 43891+0+0 (secure 0 0 0) 0x7f16f001b020 con 0x7f16f8148120 2024-09-10T10:02:17.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.408+0000 7f16f4c00640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f16d4035d40 0x7f16d4038200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.408+0000 7f16f4c00640 1 -- 172.21.15.110:0/528156511 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6355+0+0 (secure 0 0 0) 0x7f16f0049930 con 0x7f16f8148120 2024-09-10T10:02:17.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.408+0000 7f16f7e00640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f16d4035d40 0x7f16d4038200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.408+0000 7f16f7e00640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f16d4035d40 0x7f16d4038200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f16e80af000 tx=0x7f16e80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.412+0000 7f16f4c00640 1 -- 172.21.15.110:0/528156511 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f16f0022070 con 0x7f16f8148120 2024-09-10T10:02:17.526 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.524+0000 7f37896c1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:02:17.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.572+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16f8141900 con 0x7f16f8148120 2024-09-10T10:02:17.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16f4c00640 1 -- 172.21.15.110:0/528156511 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+17728 (secure 0 0 0) 0x7f16f0010040 con 0x7f16f8148120 2024-09-10T10:02:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:17.579 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":72,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:15.654389+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:54.865555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:55.765126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2d","mappings":[{"from":6,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/698629357":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/69305518":"2024-09-11T10:02:15.654344+0000","172.21.15.110:6833/952833175":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/4153756567":"2024-09-11T10:02:15.654344+0000","172.21.15.110:6832/952833175":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3823528576":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3695412383":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3716204142":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/2970528962":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/543422288":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/4258153839":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/1317276712":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3209652493":"2024-09-11T10:02:15.654344+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f16d4035d40 msgr2=0x7f16d4038200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f16d4035d40 0x7f16d4038200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f16e80af000 tx=0x7f16e80a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 msgr2=0x7f16f8143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f8143650 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f16f000efc0 tx=0x7f16f000c490 comp rx=0 tx=0).stop 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 shutdown_connections 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f16d4035d40 0x7f16d4038200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16f8148120 0x7f16f8143650 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16f80b0890 0x7f16f815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 --2- 172.21.15.110:0/528156511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16f80ae840 0x7f16f813e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 >> 172.21.15.110:0/528156511 conn(0x7f16f806cf60 msgr2=0x7f16f80b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:17.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 shutdown_connections 2024-09-10T10:02:17.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.576+0000 7f16ff200640 1 -- 172.21.15.110:0/528156511 wait complete. 2024-09-10T10:02:17.586 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.584+0000 7f37896c1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:02:17.601 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:02:17.601 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:17.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.812+0000 7f3ef2400640 1 Processor -- start 2024-09-10T10:02:17.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.812+0000 7f3ef2400640 1 -- start start 2024-09-10T10:02:17.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3ef2400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3eec06eb30 0x7f3eec0d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3ef2400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eec0706a0 con 0x7f3eec163000 2024-09-10T10:02:17.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3ef2400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eec0d1370 con 0x7f3eec073290 2024-09-10T10:02:17.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3ef2400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3eec15ee90 con 0x7f3eec06eb30 2024-09-10T10:02:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3ee3e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3eec163000 0x7f3eec071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:45082/0 (socket says 172.21.15.110:45082) 2024-09-10T10:02:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3ee3e00640 1 -- 172.21.15.110:0/2060965171 learned_addr learned my addr 172.21.15.110:0/2060965171 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3eeb400640 1 --2- 172.21.15.110:0/2060965171 >> v2:172.21.15.145:3300/0 conn(0x7f3eec06eb30 0x7f3eec0d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3eeaa00640 1 -- 172.21.15.110:0/2060965171 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218442148 0 0) 0x7f3eec0706a0 con 0x7f3eec163000 2024-09-10T10:02:17.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3eeaa00640 1 -- 172.21.15.110:0/2060965171 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ed0003530 con 0x7f3eec163000 2024-09-10T10:02:17.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3eeaa00640 1 -- 172.21.15.110:0/2060965171 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1919859509 0 0) 0x7f3ed0003530 con 0x7f3eec163000 2024-09-10T10:02:17.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3eeb400640 1 -- 172.21.15.110:0/2060965171 >> v1:172.21.15.145:6789/0 conn(0x7f3eec073290 legacy=0x7f3eec06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3eeb400640 1 -- 172.21.15.110:0/2060965171 >> v1:172.21.15.110:6789/0 conn(0x7f3eec163000 legacy=0x7f3eec071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.816+0000 7f3eeb400640 1 -- 172.21.15.110:0/2060965171 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec0d1650 con 0x7f3eec06eb30 2024-09-10T10:02:17.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3eeb400640 1 --2- 172.21.15.110:0/2060965171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec06eb30 0x7f3eec0d0cb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0097c0 tx=0x7f3edc01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ebf8e43893349d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3eeaa00640 1 -- 172.21.15.110:0/2060965171 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0b0070 con 0x7f3eec06eb30 2024-09-10T10:02:17.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3eeaa00640 1 -- 172.21.15.110:0/2060965171 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3edc0ab400 con 0x7f3eec06eb30 2024-09-10T10:02:17.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3eeaa00640 1 -- 172.21.15.110:0/2060965171 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4001680 con 0x7f3eec06eb30 2024-09-10T10:02:17.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 -- 172.21.15.110:0/2060965171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec06eb30 msgr2=0x7f3eec0d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 --2- 172.21.15.110:0/2060965171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec06eb30 0x7f3eec0d0cb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0097c0 tx=0x7f3edc01c330 comp rx=0 tx=0).stop 2024-09-10T10:02:17.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 -- 172.21.15.110:0/2060965171 shutdown_connections 2024-09-10T10:02:17.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 --2- 172.21.15.110:0/2060965171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec06eb30 0x7f3eec0d0cb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 -- 172.21.15.110:0/2060965171 >> 172.21.15.110:0/2060965171 conn(0x7f3eec0d25c0 msgr2=0x7f3eec0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:17.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 -- 172.21.15.110:0/2060965171 shutdown_connections 2024-09-10T10:02:17.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 -- 172.21.15.110:0/2060965171 wait complete. 2024-09-10T10:02:17.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 Processor -- start 2024-09-10T10:02:17.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.820+0000 7f3ef2400640 1 -- start start 2024-09-10T10:02:17.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3eec06eb30 0x7f3eec0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eec073290 0x7f3eec155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec163000 0x7f3eec160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec073ab0 con 0x7f3eec073290 2024-09-10T10:02:17.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec06e070 con 0x7f3eec163000 2024-09-10T10:02:17.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3eec156200 con 0x7f3eec06eb30 2024-09-10T10:02:17.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3eeb400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3eec06eb30 0x7f3eec0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eec073290 0x7f3eec155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3eeb400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3eec06eb30 0x7f3eec0cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:38598/0 (socket says 172.21.15.110:38598) 2024-09-10T10:02:17.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3eeb400640 1 -- 172.21.15.110:0/3815991460 learned_addr learned my addr 172.21.15.110:0/3815991460 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:17.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3eebe00640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec163000 0x7f3eec160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3e00640 1 -- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3eec06eb30 msgr2=0x7f3eec0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3e00640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3eec06eb30 0x7f3eec0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3e00640 1 -- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec163000 msgr2=0x7f3eec160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3e00640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec163000 0x7f3eec160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3e00640 1 -- 172.21.15.110:0/3815991460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec161900 con 0x7f3eec073290 2024-09-10T10:02:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3e00640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eec073290 0x7f3eec155430 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c4a0 tx=0x7f3ed400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3400640 1 -- 172.21.15.110:0/3815991460 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4017070 con 0x7f3eec073290 2024-09-10T10:02:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3400640 1 -- 172.21.15.110:0/3815991460 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed4005cf0 con 0x7f3eec073290 2024-09-10T10:02:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3400640 1 -- 172.21.15.110:0/3815991460 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed40072b0 con 0x7f3eec073290 2024-09-10T10:02:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec161be0 con 0x7f3eec073290 2024-09-10T10:02:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec0cbe50 con 0x7f3eec073290 2024-09-10T10:02:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec0d0cb0 con 0x7f3eec073290 2024-09-10T10:02:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.824+0000 7f3ee3400640 1 -- 172.21.15.110:0/3815991460 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 43891+0+0 (secure 0 0 0) 0x7f3ed401a070 con 0x7f3eec073290 2024-09-10T10:02:17.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.828+0000 7f3ee3400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ec8035d10 0x7f3ec80381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:17.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.828+0000 7f3ee3400640 1 -- 172.21.15.110:0/3815991460 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6355+0+0 (secure 0 0 0) 0x7f3ed4048280 con 0x7f3eec073290 2024-09-10T10:02:17.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.828+0000 7f3eeb400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ec8035d10 0x7f3ec80381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:17.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.828+0000 7f3eeb400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ec8035d10 0x7f3ec80381d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3edc019f60 tx=0x7f3edc0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:17.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.828+0000 7f3ee3400640 1 -- 172.21.15.110:0/3815991460 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7f3ed401b8d0 con 0x7f3eec073290 2024-09-10T10:02:17.832 INFO:tasks.ceph.mgr.y.smithi110.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T10:02:17.832 INFO:tasks.ceph.mgr.y.smithi110.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T10:02:17.832 INFO:tasks.ceph.mgr.y.smithi110.stderr: from numpy import show_config as show_numpy_config 2024-09-10T10:02:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:17.836+0000 7f3ee3400640 1 -- 172.21.15.110:0/3815991460 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3ed4016e90 con 0x7f3eec073290 2024-09-10T10:02:17.842 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.840+0000 7f37896c1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:02:17.994 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:17.992+0000 7f37896c1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:02:18.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.028+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3eec071930 con 0x7f3eec073290 2024-09-10T10:02:18.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.028+0000 7f3ee3400640 1 -- 172.21.15.110:0/3815991460 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5237 (secure 0 0 0) 0x7f3ed4022490 con 0x7f3eec073290 2024-09-10T10:02:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:18.032 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":46,"btime":"2024-09-10T10:02:10:790959+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:18.032 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 46 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ec8035d10 msgr2=0x7f3ec80381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ec8035d10 0x7f3ec80381d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3edc019f60 tx=0x7f3edc0a6000 comp rx=0 tx=0).stop 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eec073290 msgr2=0x7f3eec155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eec073290 0x7f3eec155430 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c4a0 tx=0x7f3ed400c960 comp rx=0 tx=0).stop 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 shutdown_connections 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ec8035d10 0x7f3ec80381d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eec163000 0x7f3eec160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eec073290 0x7f3eec155430 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 --2- 172.21.15.110:0/3815991460 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3eec06eb30 0x7f3eec0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 >> 172.21.15.110:0/3815991460 conn(0x7f3eec0d25c0 msgr2=0x7f3eec0d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:18.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 shutdown_connections 2024-09-10T10:02:18.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.032+0000 7f3ef2400640 1 -- 172.21.15.110:0/3815991460 wait complete. 2024-09-10T10:02:18.054 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:18.123 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:18.120+0000 7f37896c1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:02:18.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.188+0000 7f5420800640 1 Processor -- start 2024-09-10T10:02:18.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.188+0000 7f5420800640 1 -- start start 2024-09-10T10:02:18.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f5420800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 0x7f54180d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f5420800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f54180b4a10 0x7f54180d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f5420800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f541815ce90 con 0x7f54180d0340 2024-09-10T10:02:18.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f5420800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f541815fef0 con 0x7f54180b4a10 2024-09-10T10:02:18.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f5420800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f541815d170 con 0x7f54180b1bc0 2024-09-10T10:02:18.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f541e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f54180b4a10 0x7f54180d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f541e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f54180b4a10 0x7f54180d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55946/0 (socket says 172.21.15.110:55946) 2024-09-10T10:02:18.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f541e800640 1 -- 172.21.15.110:0/3308348973 learned_addr learned my addr 172.21.15.110:0/3308348973 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:18.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f541de00640 1 --2- 172.21.15.110:0/3308348973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 0x7f54180d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f541e800640 1 -- 172.21.15.110:0/3308348973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 msgr2=0x7f54180d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:18.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f541e800640 1 --2- 172.21.15.110:0/3308348973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 0x7f54180d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f541e800640 1 -- 172.21.15.110:0/3308348973 >> v1:172.21.15.145:6789/0 conn(0x7f54180d0340 legacy=0x7f54180b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.192+0000 7f541e800640 1 -- 172.21.15.110:0/3308348973 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541815d450 con 0x7f54180b4a10 2024-09-10T10:02:18.198 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f37896c1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:02:18.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f541e800640 1 --2- 172.21.15.110:0/3308348973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180b4a10 0x7f54180d0760 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5414009e50 tx=0x7f541401d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f0d1940adbf6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f541ca00640 1 -- 172.21.15.110:0/3308348973 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140a5850 con 0x7f54180b4a10 2024-09-10T10:02:18.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f541ca00640 1 -- 172.21.15.110:0/3308348973 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54140a59f0 con 0x7f54180b4a10 2024-09-10T10:02:18.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f541ca00640 1 -- 172.21.15.110:0/3308348973 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140a5dc0 con 0x7f54180b4a10 2024-09-10T10:02:18.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f5420800640 1 -- 172.21.15.110:0/3308348973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180b4a10 msgr2=0x7f54180d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f5420800640 1 --2- 172.21.15.110:0/3308348973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180b4a10 0x7f54180d0760 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5414009e50 tx=0x7f541401d3d0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f5420800640 1 -- 172.21.15.110:0/3308348973 shutdown_connections 2024-09-10T10:02:18.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f5420800640 1 --2- 172.21.15.110:0/3308348973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180b4a10 0x7f54180d0760 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f5420800640 1 --2- 172.21.15.110:0/3308348973 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 0x7f54180d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f5420800640 1 -- 172.21.15.110:0/3308348973 >> 172.21.15.110:0/3308348973 conn(0x7f541806f420 msgr2=0x7f541806f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:18.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f5420800640 1 -- 172.21.15.110:0/3308348973 shutdown_connections 2024-09-10T10:02:18.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.196+0000 7f5420800640 1 -- 172.21.15.110:0/3308348973 wait complete. 2024-09-10T10:02:18.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 Processor -- start 2024-09-10T10:02:18.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 -- start start 2024-09-10T10:02:18.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 0x7f541814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54180b4a10 0x7f54181527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180d0340 0x7f541814be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5418160d10 con 0x7f54180d0340 2024-09-10T10:02:18.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5418160b90 con 0x7f54180b4a10 2024-09-10T10:02:18.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5418152d30 con 0x7f54180b1bc0 2024-09-10T10:02:18.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180d0340 0x7f541814be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 0x7f541814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180d0340 0x7f541814be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55948/0 (socket says 172.21.15.110:55948) 2024-09-10T10:02:18.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 -- 172.21.15.110:0/2980782526 learned_addr learned my addr 172.21.15.110:0/2980782526 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:18.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541d400640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54180b4a10 0x7f54181527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 -- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 msgr2=0x7f541814ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 0x7f541814ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 -- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54180b4a10 msgr2=0x7f54181527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54180b4a10 0x7f54181527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 -- 172.21.15.110:0/2980782526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541814f3f0 con 0x7f54180d0340 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f541e800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180d0340 0x7f541814be80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f54140af040 tx=0x7f5414019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5412000640 1 -- 172.21.15.110:0/2980782526 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140bc030 con 0x7f54180d0340 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5412000640 1 -- 172.21.15.110:0/2980782526 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5414009250 con 0x7f54180d0340 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5418156fc0 con 0x7f54180d0340 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5412000640 1 -- 172.21.15.110:0/2980782526 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140b88a0 con 0x7f54180d0340 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.200+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5418157580 con 0x7f54180d0340 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.204+0000 7f5412000640 1 -- 172.21.15.110:0/2980782526 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7f54140bf070 con 0x7f54180d0340 2024-09-10T10:02:18.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.204+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54180d1db0 con 0x7f54180d0340 2024-09-10T10:02:18.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.204+0000 7f5412000640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53fc035f50 0x7f53fc038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.204+0000 7f5412000640 1 -- 172.21.15.110:0/2980782526 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6355+0+0 (secure 0 0 0) 0x7f54140e3f30 con 0x7f54180d0340 2024-09-10T10:02:18.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.204+0000 7f541de00640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53fc035f50 0x7f53fc038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.204+0000 7f541de00640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53fc035f50 0x7f53fc038410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f540c007a40 tx=0x7f540c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.208+0000 7f5412000640 1 -- 172.21.15.110:0/2980782526 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f54140c7020 con 0x7f54180d0340 2024-09-10T10:02:18.351 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:18.348+0000 7f37896c1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:02:18.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5418069690 con 0x7f54180d0340 2024-09-10T10:02:18.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5412000640 1 -- 172.21.15.110:0/2980782526 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5237 (secure 0 0 0) 0x7f5414016620 con 0x7f54180d0340 2024-09-10T10:02:18.403 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:18.403 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":46,"btime":"2024-09-10T10:02:10:790959+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:01:56.770054+0000","modified":"2024-09-10T10:01:59.790677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6033},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6033":{"gid":6033,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1773965791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1773965791},{"type":"v1","addr":"172.21.15.145:6835","nonce":1773965791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6033,"qdb_cluster":[6033]},"id":5}]} 2024-09-10T10:02:18.403 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 46 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53fc035f50 msgr2=0x7f53fc038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53fc035f50 0x7f53fc038410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f540c007a40 tx=0x7f540c008040 comp rx=0 tx=0).stop 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180d0340 msgr2=0x7f541814be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180d0340 0x7f541814be80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f54140af040 tx=0x7f5414019710 comp rx=0 tx=0).stop 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 shutdown_connections 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53fc035f50 0x7f53fc038410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54180d0340 0x7f541814be80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54180b4a10 0x7f54181527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 --2- 172.21.15.110:0/2980782526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f54180b1bc0 0x7f541814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 >> 172.21.15.110:0/2980782526 conn(0x7f541806f420 msgr2=0x7f54180d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:18.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 shutdown_connections 2024-09-10T10:02:18.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.400+0000 7f5420800640 1 -- 172.21.15.110:0/2980782526 wait complete. 2024-09-10T10:02:18.425 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:18.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1056a00640 1 Processor -- start 2024-09-10T10:02:18.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1056a00640 1 -- start start 2024-09-10T10:02:18.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1056a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1050161f90 0x7f10500cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1056a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10500739a0 0x7f105006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1056a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10500706a0 con 0x7f1050162680 2024-09-10T10:02:18.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1056a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1050163060 con 0x7f1050161f90 2024-09-10T10:02:18.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1056a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1050163340 con 0x7f10500739a0 2024-09-10T10:02:18.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f104fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1050161f90 0x7f10500cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1047e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1050162680 0x7f1050073290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39954/0 (socket says 172.21.15.110:39954) 2024-09-10T10:02:18.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1047e00640 1 -- 172.21.15.110:0/370191526 learned_addr learned my addr 172.21.15.110:0/370191526 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:18.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f1054a00640 1 --2- 172.21.15.110:0/370191526 >> v2:172.21.15.145:3300/0 conn(0x7f10500739a0 0x7f105006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.560+0000 7f104fe00640 1 -- 172.21.15.110:0/370191526 >> v2:172.21.15.145:3300/0 conn(0x7f10500739a0 msgr2=0x7f105006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:18.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f104fe00640 1 --2- 172.21.15.110:0/370191526 >> v2:172.21.15.145:3300/0 conn(0x7f10500739a0 0x7f105006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f104fe00640 1 -- 172.21.15.110:0/370191526 >> v1:172.21.15.110:6789/0 conn(0x7f1050162680 legacy=0x7f1050073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f104fe00640 1 -- 172.21.15.110:0/370191526 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10500cf130 con 0x7f1050161f90 2024-09-10T10:02:18.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f104f400640 1 -- 172.21.15.110:0/370191526 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 992178051 0 0) 0x7f10500706a0 con 0x7f1050162680 2024-09-10T10:02:18.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f104fe00640 1 --2- 172.21.15.110:0/370191526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f10500cf930 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f105006ccf0 tx=0x7f104001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4aeb900033036b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f104f400640 1 -- 172.21.15.110:0/370191526 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400b0070 con 0x7f1050161f90 2024-09-10T10:02:18.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f104f400640 1 -- 172.21.15.110:0/370191526 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10400ab600 con 0x7f1050161f90 2024-09-10T10:02:18.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f104f400640 1 -- 172.21.15.110:0/370191526 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400ab9d0 con 0x7f1050161f90 2024-09-10T10:02:18.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f1056a00640 1 -- 172.21.15.110:0/370191526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 msgr2=0x7f10500cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f1056a00640 1 --2- 172.21.15.110:0/370191526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f10500cf930 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f105006ccf0 tx=0x7f104001c480 comp rx=0 tx=0).stop 2024-09-10T10:02:18.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f1056a00640 1 -- 172.21.15.110:0/370191526 shutdown_connections 2024-09-10T10:02:18.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f1056a00640 1 --2- 172.21.15.110:0/370191526 >> v2:172.21.15.145:3300/0 conn(0x7f10500739a0 0x7f105006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f1056a00640 1 --2- 172.21.15.110:0/370191526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f10500cf930 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f1056a00640 1 -- 172.21.15.110:0/370191526 >> 172.21.15.110:0/370191526 conn(0x7f10500d25c0 msgr2=0x7f10500d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:18.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f1056a00640 1 -- 172.21.15.110:0/370191526 shutdown_connections 2024-09-10T10:02:18.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.564+0000 7f1056a00640 1 -- 172.21.15.110:0/370191526 wait complete. 2024-09-10T10:02:18.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 Processor -- start 2024-09-10T10:02:18.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 -- start start 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10500739a0 0x7f105014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f1050161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050162680 0x7f105015a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10500cfe70 con 0x7f1050161f90 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10500741b0 con 0x7f1050162680 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f10500cfff0 con 0x7f10500739a0 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f1050161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f1050161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45472/0 (socket says 172.21.15.110:45472) 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 -- 172.21.15.110:0/1500289005 learned_addr learned my addr 172.21.15.110:0/1500289005 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1054a00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050162680 0x7f105015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f104fe00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10500739a0 0x7f105014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 -- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10500739a0 msgr2=0x7f105014e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10500739a0 0x7f105014e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 -- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050162680 msgr2=0x7f105015a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050162680 0x7f105015a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 -- 172.21.15.110:0/1500289005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10500cda70 con 0x7f1050161f90 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f104fe00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10500739a0 0x7f105014e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1054a00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050162680 0x7f105015a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1047e00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f1050161410 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f103800c480 tx=0x7f103800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f104cc00640 1 -- 172.21.15.110:0/1500289005 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038018070 con 0x7f1050161f90 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f104cc00640 1 -- 172.21.15.110:0/1500289005 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1038007500 con 0x7f1050161f90 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f104cc00640 1 -- 172.21.15.110:0/1500289005 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038004a30 con 0x7f1050161f90 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.568+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10500cdd30 con 0x7f1050161f90 2024-09-10T10:02:18.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.572+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10500ce2f0 con 0x7f1050161f90 2024-09-10T10:02:18.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.572+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105006d530 con 0x7f1050161f90 2024-09-10T10:02:18.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.572+0000 7f104cc00640 1 -- 172.21.15.110:0/1500289005 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7f1038004bd0 con 0x7f1050161f90 2024-09-10T10:02:18.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.572+0000 7f104cc00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f102c035f80 0x7f102c038440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.572+0000 7f104cc00640 1 -- 172.21.15.110:0/1500289005 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6355+0+0 (secure 0 0 0) 0x7f10380485f0 con 0x7f1050161f90 2024-09-10T10:02:18.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.572+0000 7f104fe00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f102c035f80 0x7f102c038440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.572+0000 7f104fe00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f102c035f80 0x7f102c038440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f10400ad000 tx=0x7f10400a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.580+0000 7f104cc00640 1 -- 172.21.15.110:0/1500289005 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1038011360 con 0x7f1050161f90 2024-09-10T10:02:18.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.740+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10500d0a10 con 0x7f1050161f90 2024-09-10T10:02:18.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f104cc00640 1 -- 172.21.15.110:0/1500289005 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+17728 (secure 0 0 0) 0x7f1038010d50 con 0x7f1050161f90 2024-09-10T10:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":72,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:15.654389+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:54.865555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:55.765126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2d","mappings":[{"from":6,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/698629357":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/69305518":"2024-09-11T10:02:15.654344+0000","172.21.15.110:6833/952833175":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/4153756567":"2024-09-11T10:02:15.654344+0000","172.21.15.110:6832/952833175":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3823528576":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3695412383":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3716204142":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/2970528962":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/543422288":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/4258153839":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/1317276712":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3209652493":"2024-09-11T10:02:15.654344+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f102c035f80 msgr2=0x7f102c038440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f102c035f80 0x7f102c038440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f10400ad000 tx=0x7f10400a6000 comp rx=0 tx=0).stop 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 msgr2=0x7f1050161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f1050161410 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f103800c480 tx=0x7f103800c940 comp rx=0 tx=0).stop 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 shutdown_connections 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f102c035f80 0x7f102c038440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050162680 0x7f105015a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1050161f90 0x7f1050161410 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 --2- 172.21.15.110:0/1500289005 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10500739a0 0x7f105014e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 >> 172.21.15.110:0/1500289005 conn(0x7f10500d25c0 msgr2=0x7f10500b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:18.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 shutdown_connections 2024-09-10T10:02:18.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.744+0000 7f1056a00640 1 -- 172.21.15.110:0/1500289005 wait complete. 2024-09-10T10:02:18.768 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:02:18.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb16000640 1 Processor -- start 2024-09-10T10:02:18.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb16000640 1 -- start start 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb16000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdb1006c360 0x7fdb100cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb16000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 0x7fdb10162840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb16000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb101632f0 con 0x7fdb100d0b90 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb16000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdb10162d80 con 0x7fdb1006c360 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb16000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdb10163010 con 0x7fdb10162390 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb0f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fdb100d0b90 0x7fdb1006bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39958/0 (socket says 172.21.15.110:39958) 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb0f400640 1 -- 172.21.15.110:0/2584819693 learned_addr learned my addr 172.21.15.110:0/2584819693 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb0fe00640 1 --2- 172.21.15.110:0/2584819693 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 0x7fdb10162840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb07e00640 1 --2- 172.21.15.110:0/2584819693 >> v2:172.21.15.145:3300/0 conn(0x7fdb1006c360 0x7fdb100cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb0ea00640 1 -- 172.21.15.110:0/2584819693 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165500974 0 0) 0x7fdb101632f0 con 0x7fdb100d0b90 2024-09-10T10:02:18.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb0ea00640 1 -- 172.21.15.110:0/2584819693 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaf4003530 con 0x7fdb100d0b90 2024-09-10T10:02:18.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb07e00640 1 -- 172.21.15.110:0/2584819693 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 msgr2=0x7fdb10162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:18.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb0ea00640 1 -- 172.21.15.110:0/2584819693 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3866926980 0 0) 0x7fdaf4003530 con 0x7fdb100d0b90 2024-09-10T10:02:18.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb07e00640 1 --2- 172.21.15.110:0/2584819693 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 0x7fdb10162840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb07e00640 1 -- 172.21.15.110:0/2584819693 >> v1:172.21.15.110:6789/0 conn(0x7fdb100d0b90 legacy=0x7fdb1006bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.904+0000 7fdb07e00640 1 -- 172.21.15.110:0/2584819693 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb100cdf30 con 0x7fdb1006c360 2024-09-10T10:02:18.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb07e00640 1 --2- 172.21.15.110:0/2584819693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 0x7fdb100cea10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097c0 tx=0x7fdaf801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cab37a66ef87772f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb0ea00640 1 -- 172.21.15.110:0/2584819693 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80b0070 con 0x7fdb1006c360 2024-09-10T10:02:18.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb0ea00640 1 -- 172.21.15.110:0/2584819693 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf80ab400 con 0x7fdb1006c360 2024-09-10T10:02:18.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb0ea00640 1 -- 172.21.15.110:0/2584819693 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00001680 con 0x7fdb1006c360 2024-09-10T10:02:18.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb16000640 1 -- 172.21.15.110:0/2584819693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 msgr2=0x7fdb100cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb16000640 1 --2- 172.21.15.110:0/2584819693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 0x7fdb100cea10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097c0 tx=0x7fdaf801c390 comp rx=0 tx=0).stop 2024-09-10T10:02:18.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb16000640 1 -- 172.21.15.110:0/2584819693 shutdown_connections 2024-09-10T10:02:18.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb16000640 1 --2- 172.21.15.110:0/2584819693 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 0x7fdb10162840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb16000640 1 --2- 172.21.15.110:0/2584819693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 0x7fdb100cea10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb16000640 1 -- 172.21.15.110:0/2584819693 >> 172.21.15.110:0/2584819693 conn(0x7fdb100b3100 msgr2=0x7fdb100b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:18.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb16000640 1 -- 172.21.15.110:0/2584819693 shutdown_connections 2024-09-10T10:02:18.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.908+0000 7fdb16000640 1 -- 172.21.15.110:0/2584819693 wait complete. 2024-09-10T10:02:18.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 Processor -- start 2024-09-10T10:02:18.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 -- start start 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 0x7fdb10141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb100d0b90 0x7fdb100af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 0x7fdb100b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10072d80 con 0x7fdb100d0b90 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10072c00 con 0x7fdb1006c360 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdb100cdaa0 con 0x7fdb10162390 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb100d0b90 0x7fdb100af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb100d0b90 0x7fdb100af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45474/0 (socket says 172.21.15.110:45474) 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 -- 172.21.15.110:0/1457101813 learned_addr learned my addr 172.21.15.110:0/1457101813 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb0fe00640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 0x7fdb100b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb0f400640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 0x7fdb10141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 -- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 msgr2=0x7fdb100b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 0x7fdb100b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 -- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 msgr2=0x7fdb10141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 0x7fdb10141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 -- 172.21.15.110:0/1457101813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb101408e0 con 0x7fdb100d0b90 2024-09-10T10:02:18.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb0f400640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 0x7fdb10141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:18.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07e00640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb100d0b90 0x7fdb100af0e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80ae040 tx=0x7fdaf801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07400640 1 -- 172.21.15.110:0/1457101813 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80b0050 con 0x7fdb100d0b90 2024-09-10T10:02:18.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07400640 1 -- 172.21.15.110:0/1457101813 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf80bc070 con 0x7fdb100d0b90 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb10140bc0 con 0x7fdb100d0b90 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb07400640 1 -- 172.21.15.110:0/1457101813 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80b78b0 con 0x7fdb100d0b90 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.912+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb100ae8e0 con 0x7fdb100d0b90 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.916+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb100cea10 con 0x7fdb100d0b90 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.916+0000 7fdb07400640 1 -- 172.21.15.110:0/1457101813 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7fdaf80b7a50 con 0x7fdb100d0b90 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.916+0000 7fdb07400640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdaec035f50 0x7fdaec038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.916+0000 7fdb07400640 1 -- 172.21.15.110:0/1457101813 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6355+0+0 (secure 0 0 0) 0x7fdaf80e3ac0 con 0x7fdb100d0b90 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.916+0000 7fdb0f400640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdaec035f50 0x7fdaec038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.916+0000 7fdb0f400640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdaec035f50 0x7fdaec038410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb00001ec0 tx=0x7fdb00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.920 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:18.916+0000 7f37896c1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:02:18.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:18.920+0000 7fdb07400640 1 -- 172.21.15.110:0/1457101813 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdaf8015fb0 con 0x7fdb100d0b90 2024-09-10T10:02:19.002 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:02:19.005 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:02:19.007 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:19.004+0000 7f37896c1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:02:19.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.112+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdb1015e6e0 con 0x7fdb100d0b90 2024-09-10T10:02:19.115 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:02:19.112+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:02:19.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb07400640 1 -- 172.21.15.110:0/1457101813 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7fdaf80b7d40 con 0x7fdb100d0b90 2024-09-10T10:02:19.117 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:02:19.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdaec035f50 msgr2=0x7fdaec038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdaec035f50 0x7fdaec038410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb00001ec0 tx=0x7fdb00009040 comp rx=0 tx=0).stop 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb100d0b90 msgr2=0x7fdb100af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb100d0b90 0x7fdb100af0e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80ae040 tx=0x7fdaf801abd0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 shutdown_connections 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdaec035f50 0x7fdaec038410 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb10162390 0x7fdb100b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb100d0b90 0x7fdb100af0e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 --2- 172.21.15.110:0/1457101813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb1006c360 0x7fdb10141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 >> 172.21.15.110:0/1457101813 conn(0x7fdb100b3100 msgr2=0x7fdb100cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 shutdown_connections 2024-09-10T10:02:19.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.116+0000 7fdb16000640 1 -- 172.21.15.110:0/1457101813 wait complete. 2024-09-10T10:02:19.134 INFO:tasks.ceph.mds.d.smithi145.stderr: -7601> 2024-09-10T10:01:59.372+0000 7fa59a600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:02:19.139 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:02:19.175 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:19.172+0000 7f37896c1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:02:19.186 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:02:19.261 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:19.256+0000 7f37896c1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:02:19.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9cad600640 1 Processor -- start 2024-09-10T10:02:19.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9cad600640 1 -- start start 2024-09-10T10:02:19.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9cad600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9ca8147f80 0x7f9ca806e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9cad600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b4af0 0x7f9ca80b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9cad600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca80b4f90 con 0x7f9ca80b0bf0 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9cad600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca80b5270 con 0x7f9ca8147f80 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9cad600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca80b5550 con 0x7f9ca80b4af0 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9ca6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9ca8147f80 0x7f9ca806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9ca6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9ca8147f80 0x7f9ca806e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45484/0 (socket says 172.21.15.110:45484) 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9ca6a00640 1 -- 172.21.15.110:0/3467393917 learned_addr learned my addr 172.21.15.110:0/3467393917 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9ca6000640 1 --2- 172.21.15.110:0/3467393917 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b4af0 0x7f9ca80b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9ca6a00640 1 -- 172.21.15.110:0/3467393917 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b4af0 msgr2=0x7f9ca80b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9ca6a00640 1 --2- 172.21.15.110:0/3467393917 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b4af0 0x7f9ca80b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9ca6a00640 1 -- 172.21.15.110:0/3467393917 >> v1:172.21.15.110:6789/0 conn(0x7f9ca80b0bf0 legacy=0x7f9ca80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.312+0000 7f9ca6a00640 1 -- 172.21.15.110:0/3467393917 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca80b5830 con 0x7f9ca8147f80 2024-09-10T10:02:19.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9ca6a00640 1 --2- 172.21.15.110:0/3467393917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca806e190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009870 tx=0x7f9c9801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805fa1348b24c5e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9ca5600640 1 -- 172.21.15.110:0/3467393917 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980a9610 con 0x7f9ca8147f80 2024-09-10T10:02:19.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9ca5600640 1 -- 172.21.15.110:0/3467393917 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c980a97b0 con 0x7f9ca8147f80 2024-09-10T10:02:19.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9ca5600640 1 -- 172.21.15.110:0/3467393917 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980a9b80 con 0x7f9ca8147f80 2024-09-10T10:02:19.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9cad600640 1 -- 172.21.15.110:0/3467393917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 msgr2=0x7f9ca806e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9cad600640 1 --2- 172.21.15.110:0/3467393917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca806e190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009870 tx=0x7f9c9801c530 comp rx=0 tx=0).stop 2024-09-10T10:02:19.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9cad600640 1 -- 172.21.15.110:0/3467393917 shutdown_connections 2024-09-10T10:02:19.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9cad600640 1 --2- 172.21.15.110:0/3467393917 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b4af0 0x7f9ca80b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9cad600640 1 --2- 172.21.15.110:0/3467393917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca806e190 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9cad600640 1 -- 172.21.15.110:0/3467393917 >> 172.21.15.110:0/3467393917 conn(0x7f9ca806d160 msgr2=0x7f9ca806d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9cad600640 1 -- 172.21.15.110:0/3467393917 shutdown_connections 2024-09-10T10:02:19.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.316+0000 7f9cad600640 1 -- 172.21.15.110:0/3467393917 wait complete. 2024-09-10T10:02:19.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 Processor -- start 2024-09-10T10:02:19.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 -- start start 2024-09-10T10:02:19.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b0bf0 0x7f9ca81638b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca80b4af0 0x7f9ca81468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca8146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80b6840 con 0x7f9ca8147f80 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca806e8e0 con 0x7f9ca80b4af0 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca80b69c0 con 0x7f9ca80b0bf0 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca8146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca8146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45488/0 (socket says 172.21.15.110:45488) 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 -- 172.21.15.110:0/1213472859 learned_addr learned my addr 172.21.15.110:0/1213472859 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca6a00640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b0bf0 0x7f9ca81638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca6000640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca80b4af0 0x7f9ca81468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 -- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b0bf0 msgr2=0x7f9ca81638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b0bf0 0x7f9ca81638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 -- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca80b4af0 msgr2=0x7f9ca81468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca80b4af0 0x7f9ca81468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 -- 172.21.15.110:0/1213472859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca8140520 con 0x7f9ca8147f80 2024-09-10T10:02:19.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca6a00640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b0bf0 0x7f9ca81638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:19.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9ca7400640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca8146df0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000c770 tx=0x7f9ca000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9c9ea00640 1 -- 172.21.15.110:0/1213472859 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0003b80 con 0x7f9ca8147f80 2024-09-10T10:02:19.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9c9ea00640 1 -- 172.21.15.110:0/1213472859 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca0003d20 con 0x7f9ca8147f80 2024-09-10T10:02:19.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9c9ea00640 1 -- 172.21.15.110:0/1213472859 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000aa90 con 0x7f9ca8147f80 2024-09-10T10:02:19.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca8140800 con 0x7f9ca8147f80 2024-09-10T10:02:19.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca8140dc0 con 0x7f9ca8147f80 2024-09-10T10:02:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.320+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca80af070 con 0x7f9ca8147f80 2024-09-10T10:02:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.324+0000 7f9c9ea00640 1 -- 172.21.15.110:0/1213472859 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7f9ca000ac30 con 0x7f9ca8147f80 2024-09-10T10:02:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.324+0000 7f9c9ea00640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9c88035f50 0x7f9c88038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.324+0000 7f9c9ea00640 1 -- 172.21.15.110:0/1213472859 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6441+0+0 (secure 0 0 0) 0x7f9ca0049250 con 0x7f9ca8147f80 2024-09-10T10:02:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.324+0000 7f9ca6a00640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9c88035f50 0x7f9c88038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.324+0000 7f9ca6a00640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9c88035f50 0x7f9c88038410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9c980ae000 tx=0x7f9c98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.332+0000 7f9c9ea00640 1 -- 172.21.15.110:0/1213472859 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9ca0012420 con 0x7f9ca8147f80 2024-09-10T10:02:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:19.524+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9ca80aec30 con 0x7f9ca8147f80 2024-09-10T10:02:19.594 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:19.592+0000 7f37896c1180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:02:19.656 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-10T10:02:19.652+0000 7f37896c1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:02:20.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.120+0000 7f9c9ea00640 1 -- 172.21.15.110:0/1213472859 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) ==== 104+0+0 (secure 0 0 0) 0x7f9ca0005090 con 0x7f9ca8147f80 2024-09-10T10:02:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9c88035f50 msgr2=0x7f9c88038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9c88035f50 0x7f9c88038410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9c980ae000 tx=0x7f9c98009040 comp rx=0 tx=0).stop 2024-09-10T10:02:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 msgr2=0x7f9ca8146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca8146df0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000c770 tx=0x7f9ca000cc30 comp rx=0 tx=0).stop 2024-09-10T10:02:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 shutdown_connections 2024-09-10T10:02:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9c88035f50 0x7f9c88038410 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ca8147f80 0x7f9ca8146df0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ca80b4af0 0x7f9ca81468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 --2- 172.21.15.110:0/1213472859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9ca80b0bf0 0x7f9ca81638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 >> 172.21.15.110:0/1213472859 conn(0x7f9ca806d160 msgr2=0x7f9ca80d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:20.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 shutdown_connections 2024-09-10T10:02:20.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.124+0000 7f9cad600640 1 -- 172.21.15.110:0/1213472859 wait complete. 2024-09-10T10:02:20.149 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:02:20.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.284+0000 7f13f4a00640 1 Processor -- start 2024-09-10T10:02:20.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.284+0000 7f13f4a00640 1 -- start start 2024-09-10T10:02:20.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13f4a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f13f00cfe30 0x7f13f0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13f4a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f13f00cd510 0x7f13f00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13f4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 0x7f13f006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13f4a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f13f015d9a0 con 0x7f13f00cd510 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13f4a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f13f006ea60 con 0x7f13f00cfe30 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13f4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f13f006d2d0 con 0x7f13f006e090 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ed400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f13f00cd510 0x7f13f00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ed400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f13f00cd510 0x7f13f00cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45500/0 (socket says 172.21.15.110:45500) 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ed400640 1 -- 172.21.15.110:0/81329570 learned_addr learned my addr 172.21.15.110:0/81329570 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ede00640 1 --2- 172.21.15.110:0/81329570 >> v2:172.21.15.145:3300/0 conn(0x7f13f00cfe30 0x7f13f0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ee800640 1 --2- 172.21.15.110:0/81329570 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 0x7f13f006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ed400640 1 -- 172.21.15.110:0/81329570 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 msgr2=0x7f13f006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:20.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ed400640 1 --2- 172.21.15.110:0/81329570 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 0x7f13f006e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ed400640 1 -- 172.21.15.110:0/81329570 >> v2:172.21.15.145:3300/0 conn(0x7f13f00cfe30 msgr2=0x7f13f0071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:20.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ed400640 1 --2- 172.21.15.110:0/81329570 >> v2:172.21.15.145:3300/0 conn(0x7f13f00cfe30 0x7f13f0071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.288+0000 7f13ed400640 1 -- 172.21.15.110:0/81329570 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f006d5b0 con 0x7f13f00cd510 2024-09-10T10:02:20.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13ed400640 1 --2- 172.21.15.110:0/81329570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cd510 0x7f13f00cd9a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f13f00ce120 tx=0x7f13dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4022447a132df98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:20.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13eca00640 1 -- 172.21.15.110:0/81329570 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0b2070 con 0x7f13f00cd510 2024-09-10T10:02:20.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13eca00640 1 -- 172.21.15.110:0/81329570 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13dc0a9070 con 0x7f13f00cd510 2024-09-10T10:02:20.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13eca00640 1 -- 172.21.15.110:0/81329570 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0ad880 con 0x7f13f00cd510 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 -- 172.21.15.110:0/81329570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cd510 msgr2=0x7f13f00cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 --2- 172.21.15.110:0/81329570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cd510 0x7f13f00cd9a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f13f00ce120 tx=0x7f13dc01c490 comp rx=0 tx=0).stop 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 -- 172.21.15.110:0/81329570 shutdown_connections 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 --2- 172.21.15.110:0/81329570 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 0x7f13f006e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 --2- 172.21.15.110:0/81329570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cd510 0x7f13f00cd9a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 --2- 172.21.15.110:0/81329570 >> v2:172.21.15.145:3300/0 conn(0x7f13f00cfe30 0x7f13f0071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 -- 172.21.15.110:0/81329570 >> 172.21.15.110:0/81329570 conn(0x7f13f006bc50 msgr2=0x7f13f006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 -- 172.21.15.110:0/81329570 shutdown_connections 2024-09-10T10:02:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 -- 172.21.15.110:0/81329570 wait complete. 2024-09-10T10:02:20.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 Processor -- start 2024-09-10T10:02:20.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.292+0000 7f13f4a00640 1 -- start start 2024-09-10T10:02:20.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 0x7f13f0146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13f00cd510 0x7f13f01664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cfe30 0x7f13f00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0072190 con 0x7f13f00cfe30 2024-09-10T10:02:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f13f015d680 con 0x7f13f00cd510 2024-09-10T10:02:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f13f0160010 con 0x7f13f006e090 2024-09-10T10:02:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cfe30 0x7f13f00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cfe30 0x7f13f00ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45506/0 (socket says 172.21.15.110:45506) 2024-09-10T10:02:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 -- 172.21.15.110:0/784602483 learned_addr learned my addr 172.21.15.110:0/784602483 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ed400640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13f00cd510 0x7f13f01664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ede00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 0x7f13f0146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 -- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 msgr2=0x7f13f0146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 0x7f13f0146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 -- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13f00cd510 msgr2=0x7f13f01664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13f00cd510 0x7f13f01664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 -- 172.21.15.110:0/784602483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f01f7b40 con 0x7f13f00cfe30 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13ee800640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cfe30 0x7f13f00ae840 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f13e800eed0 tx=0x7f13e800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13e6000640 1 -- 172.21.15.110:0/784602483 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8018020 con 0x7f13f00cfe30 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13e6000640 1 -- 172.21.15.110:0/784602483 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e8014400 con 0x7f13f00cfe30 2024-09-10T10:02:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13e6000640 1 -- 172.21.15.110:0/784602483 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80147f0 con 0x7f13f00cfe30 2024-09-10T10:02:20.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f01f7e20 con 0x7f13f00cfe30 2024-09-10T10:02:20.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f01f83e0 con 0x7f13f00cfe30 2024-09-10T10:02:20.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f015b570 con 0x7f13f00cfe30 2024-09-10T10:02:20.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.296+0000 7f13e6000640 1 -- 172.21.15.110:0/784602483 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f13e801b020 con 0x7f13f00cfe30 2024-09-10T10:02:20.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.300+0000 7f13e6000640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f13d00679e0 0x7f13d0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.300+0000 7f13e6000640 1 -- 172.21.15.110:0/784602483 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6441+0+0 (secure 0 0 0) 0x7f13e8086760 con 0x7f13f00cfe30 2024-09-10T10:02:20.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.300+0000 7f13ede00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f13d00679e0 0x7f13d0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.300+0000 7f13ede00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f13d00679e0 0x7f13d0069ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009ac0 tx=0x7f13e0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:20.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.304+0000 7f13e6000640 1 -- 172.21.15.110:0/784602483 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f13e8022070 con 0x7f13f00cfe30 2024-09-10T10:02:20.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:20.464+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f13f015b890 con 0x7f13f00cfe30 2024-09-10T10:02:21.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.124+0000 7f13e6000640 1 -- 172.21.15.110:0/784602483 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) ==== 181+0+0 (secure 0 0 0) 0x7f13e8056750 con 0x7f13f00cfe30 2024-09-10T10:02:21.127 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:02:21.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f13d00679e0 msgr2=0x7f13d0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f13d00679e0 0x7f13d0069ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009ac0 tx=0x7f13e0009190 comp rx=0 tx=0).stop 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cfe30 msgr2=0x7f13f00ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cfe30 0x7f13f00ae840 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f13e800eed0 tx=0x7f13e800b480 comp rx=0 tx=0).stop 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 shutdown_connections 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f13d00679e0 0x7f13d0069ea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13f00cfe30 0x7f13f00ae840 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13f00cd510 0x7f13f01664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 --2- 172.21.15.110:0/784602483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13f006e090 0x7f13f0146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 >> 172.21.15.110:0/784602483 conn(0x7f13f006bc50 msgr2=0x7f13f00ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 shutdown_connections 2024-09-10T10:02:21.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.128+0000 7f13f4a00640 1 -- 172.21.15.110:0/784602483 wait complete. 2024-09-10T10:02:21.153 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:02:21.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff137400640 1 Processor -- start 2024-09-10T10:02:21.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff137400640 1 -- start start 2024-09-10T10:02:21.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff137400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff13006f1e0 0x7ff130166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff137400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff130155d50 con 0x7ff13015c560 2024-09-10T10:02:21.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff137400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1300b34d0 con 0x7ff130155970 2024-09-10T10:02:21.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff137400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff1300b4570 con 0x7ff13006f1e0 2024-09-10T10:02:21.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff135400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff13006f1e0 0x7ff130166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff135400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff13006f1e0 0x7ff130166020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45518/0 (socket says 172.21.15.110:45518) 2024-09-10T10:02:21.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff135400640 1 -- 172.21.15.110:0/395276792 learned_addr learned my addr 172.21.15.110:0/395276792 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:21.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff135400640 1 -- 172.21.15.110:0/395276792 >> v1:172.21.15.145:6789/0 conn(0x7ff130155970 legacy=0x7ff1300cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff135400640 1 -- 172.21.15.110:0/395276792 >> v1:172.21.15.110:6789/0 conn(0x7ff13015c560 legacy=0x7ff13006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.296+0000 7ff135400640 1 -- 172.21.15.110:0/395276792 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300b37b0 con 0x7ff13006f1e0 2024-09-10T10:02:21.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff135400640 1 --2- 172.21.15.110:0/395276792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff13006f1e0 0x7ff130166020 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff128009e50 tx=0x7ff12801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffcf418b007bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff12f400640 1 -- 172.21.15.110:0/395276792 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1280a5880 con 0x7ff13006f1e0 2024-09-10T10:02:21.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff12f400640 1 -- 172.21.15.110:0/395276792 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1280a5a20 con 0x7ff13006f1e0 2024-09-10T10:02:21.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff12f400640 1 -- 172.21.15.110:0/395276792 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1280a5df0 con 0x7ff13006f1e0 2024-09-10T10:02:21.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff137400640 1 -- 172.21.15.110:0/395276792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff13006f1e0 msgr2=0x7ff130166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff137400640 1 --2- 172.21.15.110:0/395276792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff13006f1e0 0x7ff130166020 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff128009e50 tx=0x7ff12801c9c0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff137400640 1 -- 172.21.15.110:0/395276792 shutdown_connections 2024-09-10T10:02:21.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff137400640 1 --2- 172.21.15.110:0/395276792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff13006f1e0 0x7ff130166020 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff137400640 1 -- 172.21.15.110:0/395276792 >> 172.21.15.110:0/395276792 conn(0x7ff13006bc50 msgr2=0x7ff13006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff137400640 1 -- 172.21.15.110:0/395276792 shutdown_connections 2024-09-10T10:02:21.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.300+0000 7ff137400640 1 -- 172.21.15.110:0/395276792 wait complete. 2024-09-10T10:02:21.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 Processor -- start 2024-09-10T10:02:21.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 -- start start 2024-09-10T10:02:21.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff13006f1e0 0x7ff130148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff130155970 0x7ff130149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff13015c560 0x7ff13014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff130168f00 con 0x7ff130155970 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff130168d80 con 0x7ff13015c560 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff130169080 con 0x7ff13006f1e0 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff130155970 0x7ff130149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff130155970 0x7ff130149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45524/0 (socket says 172.21.15.110:45524) 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 -- 172.21.15.110:0/2189947587 learned_addr learned my addr 172.21.15.110:0/2189947587 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff134a00640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff13006f1e0 0x7ff130148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff135400640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff13015c560 0x7ff13014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 -- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff13006f1e0 msgr2=0x7ff130148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff13006f1e0 0x7ff130148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 -- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff13015c560 msgr2=0x7ff13014b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff13015c560 0x7ff13014b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 -- 172.21.15.110:0/2189947587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13015e010 con 0x7ff130155970 2024-09-10T10:02:21.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12fe00640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff130155970 0x7ff130149870 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c420 tx=0x7ff11c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12cc00640 1 -- 172.21.15.110:0/2189947587 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0052d0 con 0x7ff130155970 2024-09-10T10:02:21.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12cc00640 1 -- 172.21.15.110:0/2189947587 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c005470 con 0x7ff130155970 2024-09-10T10:02:21.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff12cc00640 1 -- 172.21.15.110:0/2189947587 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c007260 con 0x7ff130155970 2024-09-10T10:02:21.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13015e2f0 con 0x7ff130155970 2024-09-10T10:02:21.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.304+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13015e8b0 con 0x7ff130155970 2024-09-10T10:02:21.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.308+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1300cc8c0 con 0x7ff130155970 2024-09-10T10:02:21.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.308+0000 7ff12cc00640 1 -- 172.21.15.110:0/2189947587 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7ff11c007400 con 0x7ff130155970 2024-09-10T10:02:21.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.308+0000 7ff12cc00640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff1100679e0 0x7ff110069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.308+0000 7ff12cc00640 1 -- 172.21.15.110:0/2189947587 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6003+0+0 (secure 0 0 0) 0x7ff11c085510 con 0x7ff130155970 2024-09-10T10:02:21.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.308+0000 7ff134a00640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff1100679e0 0x7ff110069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.308+0000 7ff134a00640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff1100679e0 0x7ff110069ea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff12000b810 tx=0x7ff120008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.312+0000 7ff12cc00640 1 -- 172.21.15.110:0/2189947587 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff11c055d70 con 0x7ff130155970 2024-09-10T10:02:21.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:21.476+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff1300b1340 con 0x7ff130155970 2024-09-10T10:02:22.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.128+0000 7ff12cc00640 1 -- 172.21.15.110:0/2189947587 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) ==== 169+0+0 (secure 0 0 0) 0x7ff11c055670 con 0x7ff130155970 2024-09-10T10:02:22.133 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:02:22.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff1100679e0 msgr2=0x7ff110069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff1100679e0 0x7ff110069ea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff12000b810 tx=0x7ff120008040 comp rx=0 tx=0).stop 2024-09-10T10:02:22.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff130155970 msgr2=0x7ff130149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff130155970 0x7ff130149870 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c420 tx=0x7ff11c00c8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 shutdown_connections 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff1100679e0 0x7ff110069ea0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff13015c560 0x7ff13014b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff130155970 0x7ff130149870 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 --2- 172.21.15.110:0/2189947587 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff13006f1e0 0x7ff130148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 >> 172.21.15.110:0/2189947587 conn(0x7ff13006bc50 msgr2=0x7ff1300b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 shutdown_connections 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.132+0000 7ff137400640 1 -- 172.21.15.110:0/2189947587 wait complete. 2024-09-10T10:02:22.160 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:02:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47f6600640 1 Processor -- start 2024-09-10T10:02:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47f6600640 1 -- start start 2024-09-10T10:02:22.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47f6600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f47f00cf0f0 0x7f47f00cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47f6600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f47f00722b0 0x7f47f0072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47f6600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f00cd800 con 0x7f47f0155850 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47f6600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f47f00ce410 con 0x7f47f00cf0f0 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47f6600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f47f00cfa90 con 0x7f47f00722b0 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47efe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f47f00cf0f0 0x7f47f00cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47efe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f47f00cf0f0 0x7f47f00cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45528/0 (socket says 172.21.15.110:45528) 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47efe00640 1 -- 172.21.15.110:0/631636403 learned_addr learned my addr 172.21.15.110:0/631636403 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47f5c00640 1 --2- 172.21.15.110:0/631636403 >> v2:172.21.15.145:3300/0 conn(0x7f47f00722b0 0x7f47f0072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47efe00640 1 -- 172.21.15.110:0/631636403 >> v2:172.21.15.145:3300/0 conn(0x7f47f00722b0 msgr2=0x7f47f0072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47efe00640 1 --2- 172.21.15.110:0/631636403 >> v2:172.21.15.145:3300/0 conn(0x7f47f00722b0 0x7f47f0072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47efe00640 1 -- 172.21.15.110:0/631636403 >> v1:172.21.15.145:6789/0 conn(0x7f47f0155850 legacy=0x7f47f00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.296+0000 7f47efe00640 1 -- 172.21.15.110:0/631636403 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f00cdae0 con 0x7f47f00cf0f0 2024-09-10T10:02:22.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47efe00640 1 --2- 172.21.15.110:0/631636403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00cf0f0 0x7f47f00cf550 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f47f00ce890 tx=0x7f47dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e65944e845cfc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47ef400640 1 -- 172.21.15.110:0/631636403 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0ab610 con 0x7f47f00cf0f0 2024-09-10T10:02:22.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47ef400640 1 -- 172.21.15.110:0/631636403 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc0b0070 con 0x7f47f00cf0f0 2024-09-10T10:02:22.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47ef400640 1 -- 172.21.15.110:0/631636403 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0ab9d0 con 0x7f47f00cf0f0 2024-09-10T10:02:22.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47f6600640 1 -- 172.21.15.110:0/631636403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00cf0f0 msgr2=0x7f47f00cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47f6600640 1 --2- 172.21.15.110:0/631636403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00cf0f0 0x7f47f00cf550 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f47f00ce890 tx=0x7f47dc01c520 comp rx=0 tx=0).stop 2024-09-10T10:02:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47f6600640 1 -- 172.21.15.110:0/631636403 shutdown_connections 2024-09-10T10:02:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47f6600640 1 --2- 172.21.15.110:0/631636403 >> v2:172.21.15.145:3300/0 conn(0x7f47f00722b0 0x7f47f0072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47f6600640 1 --2- 172.21.15.110:0/631636403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00cf0f0 0x7f47f00cf550 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47f6600640 1 -- 172.21.15.110:0/631636403 >> 172.21.15.110:0/631636403 conn(0x7f47f006d710 msgr2=0x7f47f00b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47f6600640 1 -- 172.21.15.110:0/631636403 shutdown_connections 2024-09-10T10:02:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.300+0000 7f47f6600640 1 -- 172.21.15.110:0/631636403 wait complete. 2024-09-10T10:02:22.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 Processor -- start 2024-09-10T10:02:22.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 -- start start 2024-09-10T10:02:22.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00722b0 0x7f47f0154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47f00cf0f0 0x7f47f0158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47f0155850 0x7f47f015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f47f00b62b0 con 0x7f47f00722b0 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f47f00b6130 con 0x7f47f00cf0f0 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f47f0165e70 con 0x7f47f0155850 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00722b0 0x7f47f0154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00722b0 0x7f47f0154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45536/0 (socket says 172.21.15.110:45536) 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 -- 172.21.15.110:0/4243551530 learned_addr learned my addr 172.21.15.110:0/4243551530 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47efe00640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47f00cf0f0 0x7f47f0158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5c00640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47f0155850 0x7f47f015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 -- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47f0155850 msgr2=0x7f47f015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47f0155850 0x7f47f015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 -- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47f00cf0f0 msgr2=0x7f47f0158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47f00cf0f0 0x7f47f0158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 -- 172.21.15.110:0/4243551530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f0155030 con 0x7f47f00722b0 2024-09-10T10:02:22.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f5200640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00722b0 0x7f47f0154af0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f47e000c3c0 tx=0x7f47e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47ecc00640 1 -- 172.21.15.110:0/4243551530 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0017070 con 0x7f47f00722b0 2024-09-10T10:02:22.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47ecc00640 1 -- 172.21.15.110:0/4243551530 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e00052d0 con 0x7f47f00722b0 2024-09-10T10:02:22.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47ecc00640 1 -- 172.21.15.110:0/4243551530 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0003840 con 0x7f47f00722b0 2024-09-10T10:02:22.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f0156b30 con 0x7f47f00722b0 2024-09-10T10:02:22.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01570f0 con 0x7f47f00722b0 2024-09-10T10:02:22.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.304+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f015e4f0 con 0x7f47f00722b0 2024-09-10T10:02:22.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.308+0000 7f47ecc00640 1 -- 172.21.15.110:0/4243551530 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f47e001a070 con 0x7f47f00722b0 2024-09-10T10:02:22.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.308+0000 7f47ecc00640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47d00679e0 0x7f47d0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.308+0000 7f47ecc00640 1 -- 172.21.15.110:0/4243551530 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f47e0085010 con 0x7f47f00722b0 2024-09-10T10:02:22.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.308+0000 7f47efe00640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47d00679e0 0x7f47d0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.308+0000 7f47efe00640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47d00679e0 0x7f47d0069ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0af000 tx=0x7f47dc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.312+0000 7f47ecc00640 1 -- 172.21.15.110:0/4243551530 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f47f0155030 con 0x7f47f00722b0 2024-09-10T10:02:22.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.472+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f47f00cd090 con 0x7f47f00722b0 2024-09-10T10:02:22.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47ecc00640 1 -- 172.21.15.110:0/4243551530 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f47e0005480 con 0x7f47f00722b0 2024-09-10T10:02:22.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47d00679e0 msgr2=0x7f47d0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47d00679e0 0x7f47d0069ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0af000 tx=0x7f47dc0a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00722b0 msgr2=0x7f47f0154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00722b0 0x7f47f0154af0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f47e000c3c0 tx=0x7f47e000c880 comp rx=0 tx=0).stop 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 shutdown_connections 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47d00679e0 0x7f47d0069ea0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47f0155850 0x7f47f015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47f00cf0f0 0x7f47f0158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 --2- 172.21.15.110:0/4243551530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47f00722b0 0x7f47f0154af0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 >> 172.21.15.110:0/4243551530 conn(0x7f47f006d710 msgr2=0x7f47f015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 shutdown_connections 2024-09-10T10:02:22.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.476+0000 7f47f6600640 1 -- 172.21.15.110:0/4243551530 wait complete. 2024-09-10T10:02:22.498 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-10T10:02:22.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bd1600640 1 Processor -- start 2024-09-10T10:02:22.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bd1600640 1 -- start start 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bd1600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4bcc15c960 0x7f4bcc0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bd1600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4bcc1603c0 0x7f4bcc0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bd1600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bcc0706a0 con 0x7f4bcc0d04c0 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bd1600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4bcc15cd40 con 0x7f4bcc1603c0 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bd1600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4bcc0b6370 con 0x7f4bcc15c960 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bca000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4bcc1603c0 0x7f4bcc0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bca000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4bcc1603c0 0x7f4bcc0d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45538/0 (socket says 172.21.15.110:45538) 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bca000640 1 -- 172.21.15.110:0/4074081467 learned_addr learned my addr 172.21.15.110:0/4074081467 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bca000640 1 -- 172.21.15.110:0/4074081467 >> v2:172.21.15.145:3300/0 conn(0x7f4bcc15c960 msgr2=0x7f4bcc0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bca000640 1 --2- 172.21.15.110:0/4074081467 >> v2:172.21.15.145:3300/0 conn(0x7f4bcc15c960 0x7f4bcc0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bca000640 1 -- 172.21.15.110:0/4074081467 >> v1:172.21.15.145:6789/0 conn(0x7f4bcc0d04c0 legacy=0x7f4bcc070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.632+0000 7f4bca000640 1 -- 172.21.15.110:0/4074081467 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc070a80 con 0x7f4bcc1603c0 2024-09-10T10:02:22.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bca000640 1 --2- 172.21.15.110:0/4074081467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc0d0cd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0b64f0 tx=0x7f4bb801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5d84b1abf13c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bc9600640 1 -- 172.21.15.110:0/4074081467 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80b0070 con 0x7f4bcc1603c0 2024-09-10T10:02:22.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bc9600640 1 -- 172.21.15.110:0/4074081467 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb80ab600 con 0x7f4bcc1603c0 2024-09-10T10:02:22.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bc9600640 1 -- 172.21.15.110:0/4074081467 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80ab9d0 con 0x7f4bcc1603c0 2024-09-10T10:02:22.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 -- 172.21.15.110:0/4074081467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 msgr2=0x7f4bcc0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 --2- 172.21.15.110:0/4074081467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc0d0cd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0b64f0 tx=0x7f4bb801c390 comp rx=0 tx=0).stop 2024-09-10T10:02:22.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 -- 172.21.15.110:0/4074081467 shutdown_connections 2024-09-10T10:02:22.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 --2- 172.21.15.110:0/4074081467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc0d0cd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 --2- 172.21.15.110:0/4074081467 >> v2:172.21.15.145:3300/0 conn(0x7f4bcc15c960 0x7f4bcc0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 -- 172.21.15.110:0/4074081467 >> 172.21.15.110:0/4074081467 conn(0x7f4bcc0d25c0 msgr2=0x7f4bcc0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 -- 172.21.15.110:0/4074081467 shutdown_connections 2024-09-10T10:02:22.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 -- 172.21.15.110:0/4074081467 wait complete. 2024-09-10T10:02:22.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 Processor -- start 2024-09-10T10:02:22.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.636+0000 7f4bd1600640 1 -- start start 2024-09-10T10:02:22.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bcc0d04c0 0x7f4bcc153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bcc15c960 0x7f4bcc15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc0d3150 con 0x7f4bcc1603c0 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1611b0 con 0x7f4bcc0d04c0 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4bcc0d32d0 con 0x7f4bcc15c960 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcaa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bcc0d04c0 0x7f4bcc153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc143260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45540/0 (socket says 172.21.15.110:45540) 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 -- 172.21.15.110:0/2645939031 learned_addr learned my addr 172.21.15.110:0/2645939031 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:22.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bca000640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bcc15c960 0x7f4bcc15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 -- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bcc15c960 msgr2=0x7f4bcc15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bcc15c960 0x7f4bcc15d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 -- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bcc0d04c0 msgr2=0x7f4bcc153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bcc0d04c0 0x7f4bcc153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 -- 172.21.15.110:0/2645939031 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc15dad0 con 0x7f4bcc1603c0 2024-09-10T10:02:22.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bcb400640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc143260 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400c7b0 tx=0x7f4bc400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bc2a00640 1 -- 172.21.15.110:0/2645939031 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4017070 con 0x7f4bcc1603c0 2024-09-10T10:02:22.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bc2a00640 1 -- 172.21.15.110:0/2645939031 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4003db0 con 0x7f4bcc1603c0 2024-09-10T10:02:22.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bc2a00640 1 -- 172.21.15.110:0/2645939031 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400ce60 con 0x7f4bcc1603c0 2024-09-10T10:02:22.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc154700 con 0x7f4bcc1603c0 2024-09-10T10:02:22.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc156ea0 con 0x7f4bcc1603c0 2024-09-10T10:02:22.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bcc071a80 con 0x7f4bcc1603c0 2024-09-10T10:02:22.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.640+0000 7f4bc2a00640 1 -- 172.21.15.110:0/2645939031 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f4bc401a070 con 0x7f4bcc1603c0 2024-09-10T10:02:22.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.644+0000 7f4bc2a00640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4bac0679e0 0x7f4bac069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.644+0000 7f4bc2a00640 1 -- 172.21.15.110:0/2645939031 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f4bc4085220 con 0x7f4bcc1603c0 2024-09-10T10:02:22.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.644+0000 7f4bcaa00640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4bac0679e0 0x7f4bac069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.644+0000 7f4bcaa00640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4bac0679e0 0x7f4bac069ea0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009990 tx=0x7f4bbc009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.648+0000 7f4bc2a00640 1 -- 172.21.15.110:0/2645939031 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4bc4055760 con 0x7f4bcc1603c0 2024-09-10T10:02:22.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:22.856+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f4bcc0d34d0 con 0x7f4bcc1603c0 2024-09-10T10:02:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bc2a00640 1 -- 172.21.15.110:0/2645939031 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v221) ==== 131+0+0 (secure 0 0 0) 0x7f4bc4059a60 con 0x7f4bcc1603c0 2024-09-10T10:02:23.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4bac0679e0 msgr2=0x7f4bac069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4bac0679e0 0x7f4bac069ea0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009990 tx=0x7f4bbc009110 comp rx=0 tx=0).stop 2024-09-10T10:02:23.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 msgr2=0x7f4bcc143260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc143260 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400c7b0 tx=0x7f4bc400cc70 comp rx=0 tx=0).stop 2024-09-10T10:02:23.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 shutdown_connections 2024-09-10T10:02:23.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4bac0679e0 0x7f4bac069ea0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bcc1603c0 0x7f4bcc143260 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4bcc15c960 0x7f4bcc15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 --2- 172.21.15.110:0/2645939031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bcc0d04c0 0x7f4bcc153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 >> 172.21.15.110:0/2645939031 conn(0x7f4bcc0d25c0 msgr2=0x7f4bcc165da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:23.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 shutdown_connections 2024-09-10T10:02:23.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.132+0000 7f4bd1600640 1 -- 172.21.15.110:0/2645939031 wait complete. 2024-09-10T10:02:23.155 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-10T10:02:23.155 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-10T10:02:23.156 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:02:23.156 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:23.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc79a00640 1 Processor -- start 2024-09-10T10:02:23.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc79a00640 1 -- start start 2024-09-10T10:02:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc79a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc74155970 0x7fcc740cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc79a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc7415c560 0x7fcc7406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc79a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7406f430 con 0x7fcc7406f010 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc79a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcc740b4570 con 0x7fcc74155970 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc79a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc74155d50 con 0x7fcc7415c560 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc73400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcc7406f010 0x7fcc7415eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39986/0 (socket says 172.21.15.110:39986) 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc73400640 1 -- 172.21.15.110:0/1156792642 learned_addr learned my addr 172.21.15.110:0/1156792642 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc72a00640 1 --2- 172.21.15.110:0/1156792642 >> v2:172.21.15.145:3300/0 conn(0x7fcc74155970 0x7fcc740cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc72000640 1 --2- 172.21.15.110:0/1156792642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc7415c560 0x7fcc7406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc71600640 1 -- 172.21.15.110:0/1156792642 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1030940788 0 0) 0x7fcc7406f430 con 0x7fcc7406f010 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc71600640 1 -- 172.21.15.110:0/1156792642 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc58003530 con 0x7fcc7406f010 2024-09-10T10:02:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc72a00640 1 -- 172.21.15.110:0/1156792642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc7415c560 msgr2=0x7fcc7406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc72a00640 1 --2- 172.21.15.110:0/1156792642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc7415c560 0x7fcc7406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc72a00640 1 -- 172.21.15.110:0/1156792642 >> v1:172.21.15.110:6789/0 conn(0x7fcc7406f010 legacy=0x7fcc7415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.288+0000 7fcc72a00640 1 -- 172.21.15.110:0/1156792642 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc740b34d0 con 0x7fcc74155970 2024-09-10T10:02:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc72a00640 1 --2- 172.21.15.110:0/1156792642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc74155970 0x7fcc740cc8c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400bfa0 tx=0x7fcc6401eab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7d01b9b20626b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc71600640 1 -- 172.21.15.110:0/1156792642 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400b2d0 con 0x7fcc74155970 2024-09-10T10:02:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc71600640 1 -- 172.21.15.110:0/1156792642 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc640a7980 con 0x7fcc74155970 2024-09-10T10:02:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc71600640 1 -- 172.21.15.110:0/1156792642 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc640a7d50 con 0x7fcc74155970 2024-09-10T10:02:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc79a00640 1 -- 172.21.15.110:0/1156792642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc74155970 msgr2=0x7fcc740cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc79a00640 1 --2- 172.21.15.110:0/1156792642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc74155970 0x7fcc740cc8c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400bfa0 tx=0x7fcc6401eab0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc79a00640 1 -- 172.21.15.110:0/1156792642 shutdown_connections 2024-09-10T10:02:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc79a00640 1 --2- 172.21.15.110:0/1156792642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc7415c560 0x7fcc7406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc79a00640 1 --2- 172.21.15.110:0/1156792642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc74155970 0x7fcc740cc8c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc79a00640 1 -- 172.21.15.110:0/1156792642 >> 172.21.15.110:0/1156792642 conn(0x7fcc7406bc50 msgr2=0x7fcc7406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc79a00640 1 -- 172.21.15.110:0/1156792642 shutdown_connections 2024-09-10T10:02:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.292+0000 7fcc79a00640 1 -- 172.21.15.110:0/1156792642 wait complete. 2024-09-10T10:02:23.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 Processor -- start 2024-09-10T10:02:23.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 -- start start 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc7406f010 0x7fcc74143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc74155970 0x7fcc741fac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc7415c560 0x7fcc741fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740cce00 con 0x7fcc7415c560 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740b4050 con 0x7fcc7406f010 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7406d9c0 con 0x7fcc74155970 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc7415c560 0x7fcc741fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc7415c560 0x7fcc741fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45556/0 (socket says 172.21.15.110:45556) 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 -- 172.21.15.110:0/891636295 learned_addr learned my addr 172.21.15.110:0/891636295 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 -- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc74155970 msgr2=0x7fcc741fac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc74155970 0x7fcc741fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 -- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc7406f010 msgr2=0x7fcc74143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc72a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc7406f010 0x7fcc74143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc7406f010 0x7fcc74143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 -- 172.21.15.110:0/891636295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc74144270 con 0x7fcc7415c560 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc72a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc7406f010 0x7fcc74143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc73400640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc7415c560 0x7fcc741fb190 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c450 tx=0x7fcc6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc6aa00640 1 -- 172.21.15.110:0/891636295 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c017070 con 0x7fcc7415c560 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc6aa00640 1 -- 172.21.15.110:0/891636295 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c005cf0 con 0x7fcc7415c560 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc741fb7c0 con 0x7fcc7415c560 2024-09-10T10:02:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc6aa00640 1 -- 172.21.15.110:0/891636295 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0072b0 con 0x7fcc7415c560 2024-09-10T10:02:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.296+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc740718a0 con 0x7fcc7415c560 2024-09-10T10:02:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.300+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc740cc8c0 con 0x7fcc7415c560 2024-09-10T10:02:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.300+0000 7fcc6aa00640 1 -- 172.21.15.110:0/891636295 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fcc6c01a070 con 0x7fcc7415c560 2024-09-10T10:02:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.300+0000 7fcc6aa00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc540679e0 0x7fcc54069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.300+0000 7fcc6aa00640 1 -- 172.21.15.110:0/891636295 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7fcc6c084fe0 con 0x7fcc7415c560 2024-09-10T10:02:23.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.300+0000 7fcc72a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc540679e0 0x7fcc54069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.300+0000 7fcc72a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc540679e0 0x7fcc54069ea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc640b0000 tx=0x7fcc640a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.304+0000 7fcc6aa00640 1 -- 172.21.15.110:0/891636295 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcc6c055b10 con 0x7fcc7415c560 2024-09-10T10:02:23.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.476+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc7406edf0 con 0x7fcc7415c560 2024-09-10T10:02:23.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.476+0000 7fcc6aa00640 1 -- 172.21.15.110:0/891636295 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+13890 (secure 0 0 0) 0x7fcc6c055410 con 0x7fcc7415c560 2024-09-10T10:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":75,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:22.127810+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:6835/1773965791":"2024-09-11T10:02:19.115583+0000","172.21.15.145:6833/1773965791":"2024-09-11T10:02:19.115583+0000","172.21.15.110:0/1317276712":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/2970528962":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3716204142":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3695412383":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3823528576":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/3209652493":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/4153756567":"2024-09-11T10:02:15.654344+0000","172.21.15.110:6833/952833175":"2024-09-11T10:02:15.654344+0000","172.21.15.110:6832/952833175":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/69305518":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/4258153839":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/543422288":"2024-09-11T10:02:15.654344+0000","172.21.15.110:0/698629357":"2024-09-11T10:02:15.654344+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:23.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc540679e0 msgr2=0x7fcc54069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc540679e0 0x7fcc54069ea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc640b0000 tx=0x7fcc640a9070 comp rx=0 tx=0).stop 2024-09-10T10:02:23.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc7415c560 msgr2=0x7fcc741fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc7415c560 0x7fcc741fb190 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c450 tx=0x7fcc6c00c910 comp rx=0 tx=0).stop 2024-09-10T10:02:23.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 shutdown_connections 2024-09-10T10:02:23.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc540679e0 0x7fcc54069ea0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc7415c560 0x7fcc741fb190 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc74155970 0x7fcc741fac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 --2- 172.21.15.110:0/891636295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc7406f010 0x7fcc74143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 >> 172.21.15.110:0/891636295 conn(0x7fcc7406bc50 msgr2=0x7fcc7406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:23.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 shutdown_connections 2024-09-10T10:02:23.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.480+0000 7fcc79a00640 1 -- 172.21.15.110:0/891636295 wait complete. 2024-09-10T10:02:23.500 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:02:23.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:02:23.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69cbc00640 1 Processor -- start 2024-09-10T10:02:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69cbc00640 1 -- start start 2024-09-10T10:02:23.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69cbc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f69c40ce4d0 0x7f69c406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69cbc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c40b4960 con 0x7f69c415b330 2024-09-10T10:02:23.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69cbc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c40b4c40 con 0x7f69c406e850 2024-09-10T10:02:23.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69cbc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f69c40b5570 con 0x7f69c40ce4d0 2024-09-10T10:02:23.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69c9200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f69c415b330 0x7f69c406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39998/0 (socket says 172.21.15.110:39998) 2024-09-10T10:02:23.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69c9200640 1 -- 172.21.15.110:0/4168893655 learned_addr learned my addr 172.21.15.110:0/4168893655 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:23.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69c3e00640 1 --2- 172.21.15.110:0/4168893655 >> v2:172.21.15.110:3300/0 conn(0x7f69c40ce4d0 0x7f69c406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69c3e00640 1 -- 172.21.15.110:0/4168893655 >> v1:172.21.15.145:6789/0 conn(0x7f69c406e850 legacy=0x7f69c40b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69c3e00640 1 -- 172.21.15.110:0/4168893655 >> v1:172.21.15.110:6789/0 conn(0x7f69c415b330 legacy=0x7f69c406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.636+0000 7f69c3e00640 1 -- 172.21.15.110:0/4168893655 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c40b4f20 con 0x7f69c40ce4d0 2024-09-10T10:02:23.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69c3e00640 1 --2- 172.21.15.110:0/4168893655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c40ce4d0 0x7f69c406c6a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f69b00097c0 tx=0x7f69b001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85d50e28b601346 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69c3400640 1 -- 172.21.15.110:0/4168893655 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b00b1070 con 0x7f69c40ce4d0 2024-09-10T10:02:23.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69c3400640 1 -- 172.21.15.110:0/4168893655 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b00ac410 con 0x7f69c40ce4d0 2024-09-10T10:02:23.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69c3400640 1 -- 172.21.15.110:0/4168893655 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b00ac7e0 con 0x7f69c40ce4d0 2024-09-10T10:02:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69cbc00640 1 -- 172.21.15.110:0/4168893655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c40ce4d0 msgr2=0x7f69c406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69cbc00640 1 --2- 172.21.15.110:0/4168893655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c40ce4d0 0x7f69c406c6a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f69b00097c0 tx=0x7f69b001c4a0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69cbc00640 1 -- 172.21.15.110:0/4168893655 shutdown_connections 2024-09-10T10:02:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69cbc00640 1 --2- 172.21.15.110:0/4168893655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c40ce4d0 0x7f69c406c6a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69cbc00640 1 -- 172.21.15.110:0/4168893655 >> 172.21.15.110:0/4168893655 conn(0x7f69c4062e40 msgr2=0x7f69c406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69cbc00640 1 -- 172.21.15.110:0/4168893655 shutdown_connections 2024-09-10T10:02:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.640+0000 7f69cbc00640 1 -- 172.21.15.110:0/4168893655 wait complete. 2024-09-10T10:02:23.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 Processor -- start 2024-09-10T10:02:23.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 -- start start 2024-09-10T10:02:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69c406e850 0x7f69c414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c415b330 0x7f69c40ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69c4153370 0x7f69c40aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4169450 con 0x7f69c415b330 2024-09-10T10:02:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41692d0 con 0x7f69c406e850 2024-09-10T10:02:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f69c41695d0 con 0x7f69c4153370 2024-09-10T10:02:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c415b330 0x7f69c40ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c415b330 0x7f69c40ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45576/0 (socket says 172.21.15.110:45576) 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 -- 172.21.15.110:0/2825746670 learned_addr learned my addr 172.21.15.110:0/2825746670 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c9200640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69c406e850 0x7f69c414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c9c00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69c4153370 0x7f69c40aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 -- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69c4153370 msgr2=0x7f69c40aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69c4153370 0x7f69c40aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 -- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69c406e850 msgr2=0x7f69c414fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69c406e850 0x7f69c414fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 -- 172.21.15.110:0/2825746670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c41561c0 con 0x7f69c415b330 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c9200640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69c406e850 0x7f69c414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c9c00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69c4153370 0x7f69c40aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:23.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c3e00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c415b330 0x7f69c40ae840 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f69b00190b0 tx=0x7f69b0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c0c00640 1 -- 172.21.15.110:0/2825746670 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b00b1040 con 0x7f69c415b330 2024-09-10T10:02:23.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c0c00640 1 -- 172.21.15.110:0/2825746670 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b00bb070 con 0x7f69c415b330 2024-09-10T10:02:23.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c41564a0 con 0x7f69c415b330 2024-09-10T10:02:23.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69c0c00640 1 -- 172.21.15.110:0/2825746670 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b00b5630 con 0x7f69c415b330 2024-09-10T10:02:23.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.644+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c415c6b0 con 0x7f69c415b330 2024-09-10T10:02:23.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.648+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c406c6a0 con 0x7f69c415b330 2024-09-10T10:02:23.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.648+0000 7f69c0c00640 1 -- 172.21.15.110:0/2825746670 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f69b00b57d0 con 0x7f69c415b330 2024-09-10T10:02:23.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.648+0000 7f69c0c00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69a40679e0 0x7f69a4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.648+0000 7f69c0c00640 1 -- 172.21.15.110:0/2825746670 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f69b011faa0 con 0x7f69c415b330 2024-09-10T10:02:23.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.648+0000 7f69c9200640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69a40679e0 0x7f69a4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.648+0000 7f69c9200640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69a40679e0 0x7f69a4069ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69b4005c10 tx=0x7f69b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.652+0000 7f69c0c00640 1 -- 172.21.15.110:0/2825746670 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f69b0016090 con 0x7f69c415b330 2024-09-10T10:02:23.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69c415e4e0 con 0x7f69c415b330 2024-09-10T10:02:23.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69c0c00640 1 -- 172.21.15.110:0/2825746670 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f69b00bd310 con 0x7f69c415b330 2024-09-10T10:02:23.814 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:02:23.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69a40679e0 msgr2=0x7f69a4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69a40679e0 0x7f69a4069ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69b4005c10 tx=0x7f69b4009040 comp rx=0 tx=0).stop 2024-09-10T10:02:23.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c415b330 msgr2=0x7f69c40ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c415b330 0x7f69c40ae840 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f69b00190b0 tx=0x7f69b0019240 comp rx=0 tx=0).stop 2024-09-10T10:02:23.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 shutdown_connections 2024-09-10T10:02:23.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69a40679e0 0x7f69a4069ea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69c4153370 0x7f69c40aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c415b330 0x7f69c40ae840 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 --2- 172.21.15.110:0/2825746670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69c406e850 0x7f69c414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 >> 172.21.15.110:0/2825746670 conn(0x7f69c4062e40 msgr2=0x7f69c40cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:23.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 shutdown_connections 2024-09-10T10:02:23.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.812+0000 7f69cbc00640 1 -- 172.21.15.110:0/2825746670 wait complete. 2024-09-10T10:02:23.835 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-10T10:02:23.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.968+0000 7fcc8f400640 1 Processor -- start 2024-09-10T10:02:23.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.968+0000 7fcc8f400640 1 -- start start 2024-09-10T10:02:23.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.968+0000 7fcc8f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 0x7fcc8815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.968+0000 7fcc8f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc881629c0 0x7fcc8806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.968+0000 7fcc8f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc88162de0 0x7fcc8806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.968+0000 7fcc8f400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcc881419d0 con 0x7fcc88162de0 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.968+0000 7fcc8f400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcc880b3b10 con 0x7fcc881629c0 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.968+0000 7fcc8f400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc880ae840 con 0x7fcc881625e0 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc88162de0 0x7fcc8806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcc88162de0 0x7fcc8806c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45580/0 (socket says 172.21.15.110:45580) 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 -- 172.21.15.110:0/1550117070 learned_addr learned my addr 172.21.15.110:0/1550117070 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8ca00640 1 --2- 172.21.15.110:0/1550117070 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 0x7fcc8815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc87e00640 1 --2- 172.21.15.110:0/1550117070 >> v2:172.21.15.145:3300/0 conn(0x7fcc881629c0 0x7fcc8806bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 -- 172.21.15.110:0/1550117070 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 msgr2=0x7fcc8815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 --2- 172.21.15.110:0/1550117070 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 0x7fcc8815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 -- 172.21.15.110:0/1550117070 >> v2:172.21.15.145:3300/0 conn(0x7fcc881629c0 msgr2=0x7fcc8806bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:23.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 --2- 172.21.15.110:0/1550117070 >> v2:172.21.15.145:3300/0 conn(0x7fcc881629c0 0x7fcc8806bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 -- 172.21.15.110:0/1550117070 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8815f0e0 con 0x7fcc88162de0 2024-09-10T10:02:23.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc8d400640 1 --2- 172.21.15.110:0/1550117070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc88162de0 0x7fcc8806c190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcc880b34e0 tx=0x7fcc8001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cd7e99032f361d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc87400640 1 -- 172.21.15.110:0/1550117070 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc800a5840 con 0x7fcc88162de0 2024-09-10T10:02:23.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc87400640 1 -- 172.21.15.110:0/1550117070 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc800a59e0 con 0x7fcc88162de0 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.972+0000 7fcc87400640 1 -- 172.21.15.110:0/1550117070 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc800a5db0 con 0x7fcc88162de0 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- 172.21.15.110:0/1550117070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc88162de0 msgr2=0x7fcc8806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 --2- 172.21.15.110:0/1550117070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc88162de0 0x7fcc8806c190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcc880b34e0 tx=0x7fcc8001c9c0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- 172.21.15.110:0/1550117070 shutdown_connections 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 --2- 172.21.15.110:0/1550117070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc88162de0 0x7fcc8806c190 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 --2- 172.21.15.110:0/1550117070 >> v2:172.21.15.145:3300/0 conn(0x7fcc881629c0 0x7fcc8806bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 --2- 172.21.15.110:0/1550117070 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 0x7fcc8815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- 172.21.15.110:0/1550117070 >> 172.21.15.110:0/1550117070 conn(0x7fcc880b24e0 msgr2=0x7fcc880b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- 172.21.15.110:0/1550117070 shutdown_connections 2024-09-10T10:02:23.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- 172.21.15.110:0/1550117070 wait complete. 2024-09-10T10:02:23.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 Processor -- start 2024-09-10T10:02:23.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- start start 2024-09-10T10:02:23.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 0x7fcc880cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc881629c0 0x7fcc8813f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc88162de0 0x7fcc880725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88144670 con 0x7fcc881629c0 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881444f0 con 0x7fcc88162de0 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8f400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcc881447f0 con 0x7fcc881625e0 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc881629c0 0x7fcc8813f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc881629c0 0x7fcc8813f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45590/0 (socket says 172.21.15.110:45590) 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 -- 172.21.15.110:0/3829588362 learned_addr learned my addr 172.21.15.110:0/3829588362 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8ca00640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 0x7fcc880cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8d400640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc88162de0 0x7fcc880725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 -- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 msgr2=0x7fcc880cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 0x7fcc880cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 -- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc88162de0 msgr2=0x7fcc880725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc88162de0 0x7fcc880725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 -- 172.21.15.110:0/3829588362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc880cea00 con 0x7fcc881629c0 2024-09-10T10:02:23.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc8d400640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc88162de0 0x7fcc880725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:23.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.976+0000 7fcc87e00640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc881629c0 0x7fcc8813f050 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400d800 tx=0x7fcc7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc84c00640 1 -- 172.21.15.110:0/3829588362 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74004020 con 0x7fcc881629c0 2024-09-10T10:02:23.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc84c00640 1 -- 172.21.15.110:0/3829588362 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc740041c0 con 0x7fcc881629c0 2024-09-10T10:02:23.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc84c00640 1 -- 172.21.15.110:0/3829588362 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740043b0 con 0x7fcc881629c0 2024-09-10T10:02:23.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc880cece0 con 0x7fcc881629c0 2024-09-10T10:02:23.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc88140440 con 0x7fcc881629c0 2024-09-10T10:02:23.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8815eb00 con 0x7fcc881629c0 2024-09-10T10:02:23.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc84c00640 1 -- 172.21.15.110:0/3829588362 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fcc7401b020 con 0x7fcc881629c0 2024-09-10T10:02:23.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc84c00640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc680679e0 0x7fcc68069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:23.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc84c00640 1 -- 172.21.15.110:0/3829588362 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7fcc74085ae0 con 0x7fcc881629c0 2024-09-10T10:02:23.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.980+0000 7fcc8ca00640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc680679e0 0x7fcc68069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:23.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.984+0000 7fcc8ca00640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc680679e0 0x7fcc68069ea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc78006aa0 tx=0x7fcc78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:23.984+0000 7fcc84c00640 1 -- 172.21.15.110:0/3829588362 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcc74056550 con 0x7fcc881629c0 2024-09-10T10:02:24.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:24.200+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7fcc880d0580 con 0x7fcc881629c0 2024-09-10T10:02:25.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.140+0000 7fcc84c00640 1 -- 172.21.15.110:0/3829588362 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v223) ==== 149+0+0 (secure 0 0 0) 0x7fcc74055e50 con 0x7fcc881629c0 2024-09-10T10:02:25.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc680679e0 msgr2=0x7fcc68069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc680679e0 0x7fcc68069ea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc78006aa0 tx=0x7fcc78008040 comp rx=0 tx=0).stop 2024-09-10T10:02:25.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc881629c0 msgr2=0x7fcc8813f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc881629c0 0x7fcc8813f050 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400d800 tx=0x7fcc7400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 shutdown_connections 2024-09-10T10:02:25.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcc680679e0 0x7fcc68069ea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc88162de0 0x7fcc880725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc881629c0 0x7fcc8813f050 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 --2- 172.21.15.110:0/3829588362 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcc881625e0 0x7fcc880cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 >> 172.21.15.110:0/3829588362 conn(0x7fcc880b24e0 msgr2=0x7fcc880ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 shutdown_connections 2024-09-10T10:02:25.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.144+0000 7fcc8f400640 1 -- 172.21.15.110:0/3829588362 wait complete. 2024-09-10T10:02:25.171 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:02:25.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa783c00640 1 Processor -- start 2024-09-10T10:02:25.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa783c00640 1 -- start start 2024-09-10T10:02:25.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa783c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c15b330 0x7fa77c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa783c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa77c06e850 0x7fa77c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa783c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77c0b4960 con 0x7fa77c0ce4d0 2024-09-10T10:02:25.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa783c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa77c0b5570 con 0x7fa77c06e850 2024-09-10T10:02:25.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa783c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c0b4c40 con 0x7fa77c15b330 2024-09-10T10:02:25.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa77be00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa77c0ce4d0 0x7fa77c06e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40012/0 (socket says 172.21.15.110:40012) 2024-09-10T10:02:25.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa77be00640 1 -- 172.21.15.110:0/2014806158 learned_addr learned my addr 172.21.15.110:0/2014806158 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:25.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa781c00640 1 --2- 172.21.15.110:0/2014806158 >> v2:172.21.15.110:3300/0 conn(0x7fa77c06e850 0x7fa77c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa781200640 1 --2- 172.21.15.110:0/2014806158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c15b330 0x7fa77c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa781c00640 1 -- 172.21.15.110:0/2014806158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c15b330 msgr2=0x7fa77c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa781c00640 1 --2- 172.21.15.110:0/2014806158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c15b330 0x7fa77c06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa781c00640 1 -- 172.21.15.110:0/2014806158 >> v1:172.21.15.110:6789/0 conn(0x7fa77c0ce4d0 legacy=0x7fa77c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.312+0000 7fa781c00640 1 -- 172.21.15.110:0/2014806158 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c0b4f20 con 0x7fa77c06e850 2024-09-10T10:02:25.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa781c00640 1 --2- 172.21.15.110:0/2014806158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c06e850 0x7fa77c0b0860 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa774009d90 tx=0x7fa77401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c3b8813e00b195 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa77b400640 1 -- 172.21.15.110:0/2014806158 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740b2070 con 0x7fa77c06e850 2024-09-10T10:02:25.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa77b400640 1 -- 172.21.15.110:0/2014806158 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7740a5940 con 0x7fa77c06e850 2024-09-10T10:02:25.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa77b400640 1 -- 172.21.15.110:0/2014806158 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740a5d10 con 0x7fa77c06e850 2024-09-10T10:02:25.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 -- 172.21.15.110:0/2014806158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c06e850 msgr2=0x7fa77c0b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 --2- 172.21.15.110:0/2014806158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c06e850 0x7fa77c0b0860 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa774009d90 tx=0x7fa77401c970 comp rx=0 tx=0).stop 2024-09-10T10:02:25.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 -- 172.21.15.110:0/2014806158 shutdown_connections 2024-09-10T10:02:25.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 --2- 172.21.15.110:0/2014806158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c06e850 0x7fa77c0b0860 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 --2- 172.21.15.110:0/2014806158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c15b330 0x7fa77c06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 -- 172.21.15.110:0/2014806158 >> 172.21.15.110:0/2014806158 conn(0x7fa77c062e40 msgr2=0x7fa77c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 -- 172.21.15.110:0/2014806158 shutdown_connections 2024-09-10T10:02:25.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 -- 172.21.15.110:0/2014806158 wait complete. 2024-09-10T10:02:25.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 Processor -- start 2024-09-10T10:02:25.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.316+0000 7fa783c00640 1 -- start start 2024-09-10T10:02:25.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c06e850 0x7fa77c13cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa77c15b330 0x7fa77c0ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c0aa430 0x7fa77c0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c1680b0 con 0x7fa77c0aa430 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c0b17c0 con 0x7fa77c15b330 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c168230 con 0x7fa77c06e850 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c0aa430 0x7fa77c0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c06e850 0x7fa77c13cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c0aa430 0x7fa77c0aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45622/0 (socket says 172.21.15.110:45622) 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 -- 172.21.15.110:0/3550117020 learned_addr learned my addr 172.21.15.110:0/3550117020 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa77be00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa77c15b330 0x7fa77c0ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 -- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c06e850 msgr2=0x7fa77c13cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c06e850 0x7fa77c13cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 -- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa77c15b330 msgr2=0x7fa77c0ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa77c15b330 0x7fa77c0ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 -- 172.21.15.110:0/3550117020 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c0aa8c0 con 0x7fa77c0aa430 2024-09-10T10:02:25.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa781c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c0aa430 0x7fa77c0aea30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b0040 tx=0x7fa774019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa778c00640 1 -- 172.21.15.110:0/3550117020 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740b2040 con 0x7fa77c0aa430 2024-09-10T10:02:25.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa778c00640 1 -- 172.21.15.110:0/3550117020 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7740bf070 con 0x7fa77c0aa430 2024-09-10T10:02:25.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa778c00640 1 -- 172.21.15.110:0/3550117020 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740b88f0 con 0x7fa77c0aa430 2024-09-10T10:02:25.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c0aaba0 con 0x7fa77c0aa430 2024-09-10T10:02:25.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c0ab160 con 0x7fa77c0aa430 2024-09-10T10:02:25.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.320+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77c06c6a0 con 0x7fa77c0aa430 2024-09-10T10:02:25.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.324+0000 7fa778c00640 1 -- 172.21.15.110:0/3550117020 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fa7740b8a90 con 0x7fa77c0aa430 2024-09-10T10:02:25.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.324+0000 7fa778c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa75c0679e0 0x7fa75c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.324+0000 7fa778c00640 1 -- 172.21.15.110:0/3550117020 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7fa774120da0 con 0x7fa77c0aa430 2024-09-10T10:02:25.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.324+0000 7fa781200640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa75c0679e0 0x7fa75c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.324+0000 7fa781200640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa75c0679e0 0x7fa75c069ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009ac0 tx=0x7fa76c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.328+0000 7fa778c00640 1 -- 172.21.15.110:0/3550117020 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa774016560 con 0x7fa77c0aa430 2024-09-10T10:02:25.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.488+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa77c06e3f0 con 0x7fa77c0aa430 2024-09-10T10:02:25.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa778c00640 1 -- 172.21.15.110:0/3550117020 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fa7740f1370 con 0x7fa77c0aa430 2024-09-10T10:02:25.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa75c0679e0 msgr2=0x7fa75c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa75c0679e0 0x7fa75c069ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009ac0 tx=0x7fa76c009190 comp rx=0 tx=0).stop 2024-09-10T10:02:25.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c0aa430 msgr2=0x7fa77c0aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c0aa430 0x7fa77c0aea30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b0040 tx=0x7fa774019710 comp rx=0 tx=0).stop 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 shutdown_connections 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa75c0679e0 0x7fa75c069ea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77c0aa430 0x7fa77c0aea30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa77c15b330 0x7fa77c0ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 --2- 172.21.15.110:0/3550117020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa77c06e850 0x7fa77c13cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 >> 172.21.15.110:0/3550117020 conn(0x7fa77c062e40 msgr2=0x7fa77c154230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 shutdown_connections 2024-09-10T10:02:25.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.492+0000 7fa783c00640 1 -- 172.21.15.110:0/3550117020 wait complete. 2024-09-10T10:02:25.516 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:02:25.516 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:02:25.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:02:25.682 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:02:25.682 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:25.682 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:25.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:02:25.691 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:25.691 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:25.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:02:25.700 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:25.700 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:25.744 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:02:25.744 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:25.744 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:25.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:02:25.754 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:02:25.754 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:25.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.916+0000 7f8404a00640 1 Processor -- start 2024-09-10T10:02:25.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.916+0000 7f8404a00640 1 -- start start 2024-09-10T10:02:25.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.916+0000 7f8404a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8400155970 0x7f84000cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.916+0000 7f8404a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 0x7f84000ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.916+0000 7f8404a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8400155d50 con 0x7f840015c560 2024-09-10T10:02:25.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.916+0000 7f8404a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f84000b4570 con 0x7f8400155970 2024-09-10T10:02:25.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.916+0000 7f8404a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f84000b34d0 con 0x7f840006f1e0 2024-09-10T10:02:25.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fd400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f840015c560 0x7f840006ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40020/0 (socket says 172.21.15.110:40020) 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fd400640 1 -- 172.21.15.110:0/670086766 learned_addr learned my addr 172.21.15.110:0/670086766 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fe800640 1 --2- 172.21.15.110:0/670086766 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 0x7f84000ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fde00640 1 --2- 172.21.15.110:0/670086766 >> v2:172.21.15.145:3300/0 conn(0x7f8400155970 0x7f84000cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fca00640 1 -- 172.21.15.110:0/670086766 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709621881 0 0) 0x7f8400155d50 con 0x7f840015c560 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fca00640 1 -- 172.21.15.110:0/670086766 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83e4003530 con 0x7f840015c560 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fde00640 1 -- 172.21.15.110:0/670086766 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 msgr2=0x7f84000ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fde00640 1 --2- 172.21.15.110:0/670086766 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 0x7f84000ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fde00640 1 -- 172.21.15.110:0/670086766 >> v1:172.21.15.110:6789/0 conn(0x7f840015c560 legacy=0x7f840006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fde00640 1 -- 172.21.15.110:0/670086766 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84000b37b0 con 0x7f8400155970 2024-09-10T10:02:25.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.920+0000 7f83fde00640 1 --2- 172.21.15.110:0/670086766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8400155970 0x7f84000cc8c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f83f00097c0 tx=0x7f83f001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c79e4ed7b6410ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f83fca00640 1 -- 172.21.15.110:0/670086766 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f00a7070 con 0x7f8400155970 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f83fca00640 1 -- 172.21.15.110:0/670086766 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f00b0070 con 0x7f8400155970 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f83fca00640 1 -- 172.21.15.110:0/670086766 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f00ab890 con 0x7f8400155970 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- 172.21.15.110:0/670086766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8400155970 msgr2=0x7f84000cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 --2- 172.21.15.110:0/670086766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8400155970 0x7f84000cc8c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f83f00097c0 tx=0x7f83f001c470 comp rx=0 tx=0).stop 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- 172.21.15.110:0/670086766 shutdown_connections 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 --2- 172.21.15.110:0/670086766 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 0x7f84000ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 --2- 172.21.15.110:0/670086766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8400155970 0x7f84000cc8c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- 172.21.15.110:0/670086766 >> 172.21.15.110:0/670086766 conn(0x7f840006bc50 msgr2=0x7f840006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- 172.21.15.110:0/670086766 shutdown_connections 2024-09-10T10:02:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- 172.21.15.110:0/670086766 wait complete. 2024-09-10T10:02:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 Processor -- start 2024-09-10T10:02:25.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- start start 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 0x7f84001531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8400155970 0x7f8400153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f840015c560 0x7f840015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f84000cf170 con 0x7f8400155970 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84000cce00 con 0x7f840015c560 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f8404a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f84000cf2f0 con 0x7f840006f1e0 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f83fd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8400155970 0x7f8400153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f83fd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8400155970 0x7f8400153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45636/0 (socket says 172.21.15.110:45636) 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.924+0000 7f83fd400640 1 -- 172.21.15.110:0/2552125208 learned_addr learned my addr 172.21.15.110:0/2552125208 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:25.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83fe800640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f840015c560 0x7f840015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83fde00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 0x7f84001531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83fd400640 1 -- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 msgr2=0x7f84001531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83fd400640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 0x7f84001531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83fd400640 1 -- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f840015c560 msgr2=0x7f840015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83fd400640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f840015c560 0x7f840015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83fd400640 1 -- 172.21.15.110:0/2552125208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840015d8c0 con 0x7f8400155970 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83fd400640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8400155970 0x7f8400153bc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f83e80010f0 tx=0x7f83e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83f6000640 1 -- 172.21.15.110:0/2552125208 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8017070 con 0x7f8400155970 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83f6000640 1 -- 172.21.15.110:0/2552125208 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e8005d10 con 0x7f8400155970 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83f6000640 1 -- 172.21.15.110:0/2552125208 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8007310 con 0x7f8400155970 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840015bb70 con 0x7f8400155970 2024-09-10T10:02:25.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8400153870 con 0x7f8400155970 2024-09-10T10:02:25.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84000ce420 con 0x7f8400155970 2024-09-10T10:02:25.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83f6000640 1 -- 172.21.15.110:0/2552125208 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f83e801a070 con 0x7f8400155970 2024-09-10T10:02:25.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83f6000640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f83dc0679e0 0x7f83dc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.928+0000 7f83f6000640 1 -- 172.21.15.110:0/2552125208 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f83e8084ff0 con 0x7f8400155970 2024-09-10T10:02:25.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.932+0000 7f83fde00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f83dc0679e0 0x7f83dc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.932+0000 7f83fde00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f83dc0679e0 0x7f83dc069ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f83f00af000 tx=0x7f83f00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:25.932+0000 7f83f6000640 1 -- 172.21.15.110:0/2552125208 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f83e8055510 con 0x7f8400155970 2024-09-10T10:02:26.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.124+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f840015dd70 con 0x7f8400155970 2024-09-10T10:02:26.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.124+0000 7f83f6000640 1 -- 172.21.15.110:0/2552125208 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+3923 (secure 0 0 0) 0x7f83e8059810 con 0x7f8400155970 2024-09-10T10:02:26.128 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:26.128 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":48,"btime":"2024-09-10T10:02:20:118973+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6836/4040594595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":4040594595},{"type":"v1","addr":"172.21.15.110:6836","nonce":4040594595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1116913580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1116913580},{"type":"v1","addr":"172.21.15.145:6834","nonce":1116913580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2862560887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6835","nonce":2862560887},{"type":"v1","addr":"172.21.15.110:6837","nonce":2862560887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6076,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2408699090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":2408699090},{"type":"v1","addr":"172.21.15.145:6835","nonce":2408699090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[]} 2024-09-10T10:02:26.128 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 48 2024-09-10T10:02:26.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f83dc0679e0 msgr2=0x7f83dc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f83dc0679e0 0x7f83dc069ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f83f00af000 tx=0x7f83f00a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:26.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8400155970 msgr2=0x7f8400153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8400155970 0x7f8400153bc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f83e80010f0 tx=0x7f83e800c890 comp rx=0 tx=0).stop 2024-09-10T10:02:26.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 shutdown_connections 2024-09-10T10:02:26.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f83dc0679e0 0x7f83dc069ea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f840015c560 0x7f840015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8400155970 0x7f8400153bc0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 --2- 172.21.15.110:0/2552125208 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f840006f1e0 0x7f84001531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 >> 172.21.15.110:0/2552125208 conn(0x7f840006bc50 msgr2=0x7f840006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:26.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 shutdown_connections 2024-09-10T10:02:26.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.128+0000 7f8404a00640 1 -- 172.21.15.110:0/2552125208 wait complete. 2024-09-10T10:02:26.150 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:02:26.150 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:02:26.150 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:02:26.150 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:02:26.150 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:02:26.151 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:02:26.151 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:02:26.151 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:02:26.151 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:02:26.151 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:02:26.151 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:02:26.151 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:02:26.152 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:02:26.152 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:02:26.152 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:02:26.152 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:02:26.162 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:02:26.163 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:02:26.164 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:02:26.164 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:02:26.165 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:02:26.166 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:02:26.167 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:02:26.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:02:26.207 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:02:26.207 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:26.209 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:02:26.209 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:26.212 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:02:26.212 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:26.214 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:02:26.215 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:26.215 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:02:26.293 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:02:26.294 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:02:26.303 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:02:26.303 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:02:26.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.356+0000 7f8515000640 1 Processor -- start 2024-09-10T10:02:26.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.356+0000 7f8515000640 1 -- start start 2024-09-10T10:02:26.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f8515000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f85100cf280 0x7f851006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:26.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f8515000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f85100d0700 0x7f851015e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:26.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f8515000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f851006e4b0 con 0x7f85100cf660 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f8515000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f85100707b0 con 0x7f85100cf280 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f8515000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f85100d0b70 con 0x7f85100d0700 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f85100cf280 0x7f851006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f85100cf280 0x7f851006e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45658/0 (socket says 172.21.15.110:45658) 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f85100d0700 0x7f851015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850de00640 1 -- 172.21.15.110:0/231927594 learned_addr learned my addr 172.21.15.110:0/231927594 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850de00640 1 -- 172.21.15.110:0/231927594 >> v2:172.21.15.145:3300/0 conn(0x7f85100d0700 msgr2=0x7f851015e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850de00640 1 --2- 172.21.15.110:0/231927594 >> v2:172.21.15.145:3300/0 conn(0x7f85100d0700 0x7f851015e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850de00640 1 -- 172.21.15.110:0/231927594 >> v1:172.21.15.145:6789/0 conn(0x7f85100cf660 legacy=0x7f851006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850d400640 1 -- 172.21.15.110:0/231927594 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042385333 0 0) 0x7f851006e4b0 con 0x7f85100cf660 2024-09-10T10:02:26.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850de00640 1 -- 172.21.15.110:0/231927594 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851006def0 con 0x7f85100cf280 2024-09-10T10:02:26.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.360+0000 7f850de00640 1 --2- 172.21.15.110:0/231927594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf280 0x7f851006e9b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f85100cd800 tx=0x7f850001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9a370262bbe7eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:26.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f850d400640 1 -- 172.21.15.110:0/231927594 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85000aa650 con 0x7f85100cf280 2024-09-10T10:02:26.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f850d400640 1 -- 172.21.15.110:0/231927594 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85000af070 con 0x7f85100cf280 2024-09-10T10:02:26.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f850d400640 1 -- 172.21.15.110:0/231927594 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85000aaa10 con 0x7f85100cf280 2024-09-10T10:02:26.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- 172.21.15.110:0/231927594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf280 msgr2=0x7f851006e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 --2- 172.21.15.110:0/231927594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf280 0x7f851006e9b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f85100cd800 tx=0x7f850001c4f0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- 172.21.15.110:0/231927594 shutdown_connections 2024-09-10T10:02:26.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 --2- 172.21.15.110:0/231927594 >> v2:172.21.15.145:3300/0 conn(0x7f85100d0700 0x7f851015e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 --2- 172.21.15.110:0/231927594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf280 0x7f851006e9b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- 172.21.15.110:0/231927594 >> 172.21.15.110:0/231927594 conn(0x7f851009b9d0 msgr2=0x7f851009bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:26.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- 172.21.15.110:0/231927594 shutdown_connections 2024-09-10T10:02:26.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- 172.21.15.110:0/231927594 wait complete. 2024-09-10T10:02:26.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 Processor -- start 2024-09-10T10:02:26.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- start start 2024-09-10T10:02:26.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85100cf280 0x7f85100b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:26.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf660 0x7f8510159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:26.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85100d0700 0x7f85100b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:26.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85100ce7c0 con 0x7f85100cf660 2024-09-10T10:02:26.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85100ce640 con 0x7f85100d0700 2024-09-10T10:02:26.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8515000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f85100ce940 con 0x7f85100cf280 2024-09-10T10:02:26.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8507e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf660 0x7f8510159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:26.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8507e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf660 0x7f8510159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45668/0 (socket says 172.21.15.110:45668) 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8507e00640 1 -- 172.21.15.110:0/3366008114 learned_addr learned my addr 172.21.15.110:0/3366008114 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f850de00640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85100cf280 0x7f85100b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f850e800640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85100d0700 0x7f85100b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.364+0000 7f8507e00640 1 -- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85100cf280 msgr2=0x7f85100b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8507e00640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85100cf280 0x7f85100b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8507e00640 1 -- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85100d0700 msgr2=0x7f85100b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8507e00640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85100d0700 0x7f85100b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8507e00640 1 -- 172.21.15.110:0/3366008114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8510159670 con 0x7f85100cf660 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f850de00640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85100cf280 0x7f85100b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:26.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8507e00640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf660 0x7f8510159130 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c480 tx=0x7f84f800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:26.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8506000640 1 -- 172.21.15.110:0/3366008114 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8005480 con 0x7f85100cf660 2024-09-10T10:02:26.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8506000640 1 -- 172.21.15.110:0/3366008114 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8003600 con 0x7f85100cf660 2024-09-10T10:02:26.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8506000640 1 -- 172.21.15.110:0/3366008114 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80039f0 con 0x7f85100cf660 2024-09-10T10:02:26.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85100b8650 con 0x7f85100cf660 2024-09-10T10:02:26.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85100b5d50 con 0x7f85100cf660 2024-09-10T10:02:26.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851006e9b0 con 0x7f85100cf660 2024-09-10T10:02:26.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8506000640 1 -- 172.21.15.110:0/3366008114 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f84f8003b90 con 0x7f85100cf660 2024-09-10T10:02:26.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8506000640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f84ec0679e0 0x7f84ec069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:26.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f8506000640 1 -- 172.21.15.110:0/3366008114 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f84f80859a0 con 0x7f85100cf660 2024-09-10T10:02:26.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.368+0000 7f850de00640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f84ec0679e0 0x7f84ec069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:26.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.372+0000 7f850de00640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f84ec0679e0 0x7f84ec069ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85000ae000 tx=0x7f85000a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:26.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.372+0000 7f8506000640 1 -- 172.21.15.110:0/3366008114 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f84f8056530 con 0x7f85100cf660 2024-09-10T10:02:26.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:26.544+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f85100d0fd0 con 0x7f85100cf660 2024-09-10T10:02:27.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.504+0000 7f8506000640 1 -- 172.21.15.110:0/3366008114 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) ==== 97+0+0 (secure 0 0 0) 0x7f84f8055e30 con 0x7f85100cf660 2024-09-10T10:02:27.508 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:02:27.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f84ec0679e0 msgr2=0x7f84ec069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f84ec0679e0 0x7f84ec069ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85000ae000 tx=0x7f85000a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:27.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf660 msgr2=0x7f8510159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf660 0x7f8510159130 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c480 tx=0x7f84f800c940 comp rx=0 tx=0).stop 2024-09-10T10:02:27.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 shutdown_connections 2024-09-10T10:02:27.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f84ec0679e0 0x7f84ec069ea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85100d0700 0x7f85100b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85100cf660 0x7f8510159130 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 --2- 172.21.15.110:0/3366008114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85100cf280 0x7f85100b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 >> 172.21.15.110:0/3366008114 conn(0x7f851009b9d0 msgr2=0x7f851006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 shutdown_connections 2024-09-10T10:02:27.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.508+0000 7f8515000640 1 -- 172.21.15.110:0/3366008114 wait complete. 2024-09-10T10:02:27.534 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:02:27.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23e00640 1 Processor -- start 2024-09-10T10:02:27.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23e00640 1 -- start start 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fde240b1c60 0x7fde24142320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fde240731e0 0x7fde24073650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde2415c790 con 0x7fde2415fd80 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fde24073b90 con 0x7fde240b1c60 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fde240b2040 con 0x7fde240731e0 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde22a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fde240b1c60 0x7fde24142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde22000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fde2415fd80 0x7fde2413f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40044/0 (socket says 172.21.15.110:40044) 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde22000640 1 -- 172.21.15.110:0/3557809991 learned_addr learned my addr 172.21.15.110:0/3557809991 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:27.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23400640 1 --2- 172.21.15.110:0/3557809991 >> v2:172.21.15.145:3300/0 conn(0x7fde240731e0 0x7fde24073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde22a00640 1 -- 172.21.15.110:0/3557809991 >> v2:172.21.15.145:3300/0 conn(0x7fde240731e0 msgr2=0x7fde24073650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde22a00640 1 --2- 172.21.15.110:0/3557809991 >> v2:172.21.15.145:3300/0 conn(0x7fde240731e0 0x7fde24073650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde22a00640 1 -- 172.21.15.110:0/3557809991 >> v1:172.21.15.110:6789/0 conn(0x7fde2415fd80 legacy=0x7fde2413f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde22a00640 1 -- 172.21.15.110:0/3557809991 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde24142860 con 0x7fde240b1c60 2024-09-10T10:02:27.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde21600640 1 -- 172.21.15.110:0/3557809991 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3696585755 0 0) 0x7fde10001120 con 0x7fde2415fd80 2024-09-10T10:02:27.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.672+0000 7fde23400640 1 --2- 172.21.15.110:0/3557809991 >> v2:172.21.15.145:3300/0 conn(0x7fde240731e0 0x7fde24073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:27.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde22a00640 1 --2- 172.21.15.110:0/3557809991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde24142320 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fde24160e60 tx=0x7fde1401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc43b3eeb0a87074 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde21600640 1 -- 172.21.15.110:0/3557809991 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140b0070 con 0x7fde240b1c60 2024-09-10T10:02:27.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde21600640 1 -- 172.21.15.110:0/3557809991 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde140ab600 con 0x7fde240b1c60 2024-09-10T10:02:27.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde21600640 1 -- 172.21.15.110:0/3557809991 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140ab9d0 con 0x7fde240b1c60 2024-09-10T10:02:27.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 -- 172.21.15.110:0/3557809991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 msgr2=0x7fde24142320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 --2- 172.21.15.110:0/3557809991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde24142320 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fde24160e60 tx=0x7fde1401c480 comp rx=0 tx=0).stop 2024-09-10T10:02:27.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 -- 172.21.15.110:0/3557809991 shutdown_connections 2024-09-10T10:02:27.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 --2- 172.21.15.110:0/3557809991 >> v2:172.21.15.145:3300/0 conn(0x7fde240731e0 0x7fde24073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 --2- 172.21.15.110:0/3557809991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde24142320 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 -- 172.21.15.110:0/3557809991 >> 172.21.15.110:0/3557809991 conn(0x7fde240b42d0 msgr2=0x7fde240b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 -- 172.21.15.110:0/3557809991 shutdown_connections 2024-09-10T10:02:27.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 -- 172.21.15.110:0/3557809991 wait complete. 2024-09-10T10:02:27.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 Processor -- start 2024-09-10T10:02:27.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.676+0000 7fde23e00640 1 -- start start 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde240731e0 0x7fde24140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde240b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde2415fd80 0x7fde24137d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde2415d460 con 0x7fde240b1c60 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde24160b50 con 0x7fde240731e0 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fde2415d5e0 con 0x7fde2415fd80 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde240b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde240b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45686/0 (socket says 172.21.15.110:45686) 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 -- 172.21.15.110:0/3201896837 learned_addr learned my addr 172.21.15.110:0/3201896837 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22a00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde240731e0 0x7fde24140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23400640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde2415fd80 0x7fde24137d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 -- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde2415fd80 msgr2=0x7fde24137d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde2415fd80 0x7fde24137d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 -- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde240731e0 msgr2=0x7fde24140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde240731e0 0x7fde24140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 -- 172.21.15.110:0/3201896837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2406e110 con 0x7fde240b1c60 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22a00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde240731e0 0x7fde24140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde22000640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde240b6fa0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c7b0 tx=0x7fde1000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde1aa00640 1 -- 172.21.15.110:0/3201896837 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10018070 con 0x7fde240b1c60 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde1aa00640 1 -- 172.21.15.110:0/3201896837 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde100072f0 con 0x7fde240b1c60 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2406e3f0 con 0x7fde240b1c60 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde1aa00640 1 -- 172.21.15.110:0/3201896837 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde100047f0 con 0x7fde240b1c60 2024-09-10T10:02:27.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2406e9b0 con 0x7fde240b1c60 2024-09-10T10:02:27.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde24073650 con 0x7fde240b1c60 2024-09-10T10:02:27.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.680+0000 7fde1aa00640 1 -- 172.21.15.110:0/3201896837 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fde1001b070 con 0x7fde240b1c60 2024-09-10T10:02:27.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.684+0000 7fde1aa00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fde040679e0 0x7fde04069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.684+0000 7fde1aa00640 1 -- 172.21.15.110:0/3201896837 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7fde10084fb0 con 0x7fde240b1c60 2024-09-10T10:02:27.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.684+0000 7fde22a00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fde040679e0 0x7fde04069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.684+0000 7fde22a00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fde040679e0 0x7fde04069ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fde140ad000 tx=0x7fde140a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.688+0000 7fde1aa00640 1 -- 172.21.15.110:0/3201896837 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fde2415c790 con 0x7fde240b1c60 2024-09-10T10:02:27.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.896+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fde2415e750 con 0x7fde240b1c60 2024-09-10T10:02:27.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.896+0000 7fde1aa00640 1 -- 172.21.15.110:0/3201896837 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fde10055c90 con 0x7fde240b1c60 2024-09-10T10:02:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:27.900 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:02:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:02:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:02:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fde040679e0 msgr2=0x7fde04069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fde040679e0 0x7fde04069ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fde140ad000 tx=0x7fde140a6000 comp rx=0 tx=0).stop 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 msgr2=0x7fde240b6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde240b6fa0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c7b0 tx=0x7fde1000cc70 comp rx=0 tx=0).stop 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 shutdown_connections 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fde040679e0 0x7fde04069ea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde2415fd80 0x7fde24137d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde240b1c60 0x7fde240b6fa0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 --2- 172.21.15.110:0/3201896837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde240731e0 0x7fde24140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 >> 172.21.15.110:0/3201896837 conn(0x7fde240b42d0 msgr2=0x7fde240b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 shutdown_connections 2024-09-10T10:02:27.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:27.900+0000 7fde23e00640 1 -- 172.21.15.110:0/3201896837 wait complete. 2024-09-10T10:02:27.922 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:02:28.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff3e00640 1 Processor -- start 2024-09-10T10:02:28.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff3e00640 1 -- start start 2024-09-10T10:02:28.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff40ae840 0x7fdff406e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff3e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdff41621e0 0x7fdff4162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff3e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff415ec50 con 0x7fdff40aec20 2024-09-10T10:02:28.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff3e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdff415eef0 con 0x7fdff41621e0 2024-09-10T10:02:28.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff3e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdff415f1d0 con 0x7fdff40ae840 2024-09-10T10:02:28.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff2000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fdff40aec20 0x7fdff40b08a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:40048/0 (socket says 172.21.15.110:40048) 2024-09-10T10:02:28.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff2000640 1 -- 172.21.15.110:0/3077544815 learned_addr learned my addr 172.21.15.110:0/3077544815 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:28.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff2a00640 1 --2- 172.21.15.110:0/3077544815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff40ae840 0x7fdff406e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff3400640 1 --2- 172.21.15.110:0/3077544815 >> v2:172.21.15.145:3300/0 conn(0x7fdff41621e0 0x7fdff4162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff1600640 1 -- 172.21.15.110:0/3077544815 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785832763 0 0) 0x7fdff415ec50 con 0x7fdff40aec20 2024-09-10T10:02:28.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff1600640 1 -- 172.21.15.110:0/3077544815 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfd8003530 con 0x7fdff40aec20 2024-09-10T10:02:28.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff2a00640 1 -- 172.21.15.110:0/3077544815 >> v2:172.21.15.145:3300/0 conn(0x7fdff41621e0 msgr2=0x7fdff4162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:28.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff2a00640 1 --2- 172.21.15.110:0/3077544815 >> v2:172.21.15.145:3300/0 conn(0x7fdff41621e0 0x7fdff4162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff2a00640 1 -- 172.21.15.110:0/3077544815 >> v1:172.21.15.110:6789/0 conn(0x7fdff40aec20 legacy=0x7fdff40b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.056+0000 7fdff2a00640 1 -- 172.21.15.110:0/3077544815 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff415f4b0 con 0x7fdff40ae840 2024-09-10T10:02:28.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff2a00640 1 --2- 172.21.15.110:0/3077544815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff40ae840 0x7fdff406e140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009d90 tx=0x7fdfe401d3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9d3f1112ba26f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:28.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff1600640 1 -- 172.21.15.110:0/3077544815 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40b2070 con 0x7fdff40ae840 2024-09-10T10:02:28.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff1600640 1 -- 172.21.15.110:0/3077544815 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe40a5940 con 0x7fdff40ae840 2024-09-10T10:02:28.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff1600640 1 -- 172.21.15.110:0/3077544815 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40a5d10 con 0x7fdff40ae840 2024-09-10T10:02:28.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff3e00640 1 -- 172.21.15.110:0/3077544815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff40ae840 msgr2=0x7fdff406e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff3e00640 1 --2- 172.21.15.110:0/3077544815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff40ae840 0x7fdff406e140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009d90 tx=0x7fdfe401d3d0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff3e00640 1 -- 172.21.15.110:0/3077544815 shutdown_connections 2024-09-10T10:02:28.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff3e00640 1 --2- 172.21.15.110:0/3077544815 >> v2:172.21.15.145:3300/0 conn(0x7fdff41621e0 0x7fdff4162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff3e00640 1 --2- 172.21.15.110:0/3077544815 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff40ae840 0x7fdff406e140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff3e00640 1 -- 172.21.15.110:0/3077544815 >> 172.21.15.110:0/3077544815 conn(0x7fdff406cd60 msgr2=0x7fdff406d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:28.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff3e00640 1 -- 172.21.15.110:0/3077544815 shutdown_connections 2024-09-10T10:02:28.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.060+0000 7fdff3e00640 1 -- 172.21.15.110:0/3077544815 wait complete. 2024-09-10T10:02:28.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 Processor -- start 2024-09-10T10:02:28.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 -- start start 2024-09-10T10:02:28.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdff40ae840 0x7fdff40b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdff40aec20 0x7fdff40ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff41621e0 0x7fdff4161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40af020 con 0x7fdff40aec20 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40b0fb0 con 0x7fdff40ae840 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdff40af1a0 con 0x7fdff41621e0 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdff40aec20 0x7fdff40ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdff40aec20 0x7fdff40ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45688/0 (socket says 172.21.15.110:45688) 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 -- 172.21.15.110:0/2145094499 learned_addr learned my addr 172.21.15.110:0/2145094499 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2a00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdff40ae840 0x7fdff40b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3400640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff41621e0 0x7fdff4161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 -- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff41621e0 msgr2=0x7fdff4161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff41621e0 0x7fdff4161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 -- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdff40ae840 msgr2=0x7fdff40b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdff40ae840 0x7fdff40b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 -- 172.21.15.110:0/2145094499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff40cff30 con 0x7fdff40aec20 2024-09-10T10:02:28.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff2000640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdff40aec20 0x7fdff40ce270 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c450 tx=0x7fdfdc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:28.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdfeaa00640 1 -- 172.21.15.110:0/2145094499 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc017070 con 0x7fdff40aec20 2024-09-10T10:02:28.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdfeaa00640 1 -- 172.21.15.110:0/2145094499 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfdc005d10 con 0x7fdff40aec20 2024-09-10T10:02:28.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdfeaa00640 1 -- 172.21.15.110:0/2145094499 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0072b0 con 0x7fdff40aec20 2024-09-10T10:02:28.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff40b6e20 con 0x7fdff40aec20 2024-09-10T10:02:28.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff40cda70 con 0x7fdff40aec20 2024-09-10T10:02:28.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.064+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff406e140 con 0x7fdff40aec20 2024-09-10T10:02:28.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.068+0000 7fdfeaa00640 1 -- 172.21.15.110:0/2145094499 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fdfdc01a070 con 0x7fdff40aec20 2024-09-10T10:02:28.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.068+0000 7fdfeaa00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfd00679e0 0x7fdfd0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.068+0000 7fdfeaa00640 1 -- 172.21.15.110:0/2145094499 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7fdfdc084dd0 con 0x7fdff40aec20 2024-09-10T10:02:28.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.068+0000 7fdff2a00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfd00679e0 0x7fdfd0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.068+0000 7fdff2a00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfd00679e0 0x7fdfd0069ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40af000 tx=0x7fdfe4004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:28.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.072+0000 7fdfeaa00640 1 -- 172.21.15.110:0/2145094499 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdfdc089070 con 0x7fdff40aec20 2024-09-10T10:02:28.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.232+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdff40b4020 con 0x7fdff40aec20 2024-09-10T10:02:28.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.232+0000 7fdfeaa00640 1 -- 172.21.15.110:0/2145094499 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) ==== 84+0+65 (secure 0 0 0) 0x7fdfdc0555b0 con 0x7fdff40aec20 2024-09-10T10:02:28.236 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:28.236 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:02:28.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:28.236 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:02:28.236 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:02:28.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:28.237 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:02:28.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfd00679e0 msgr2=0x7fdfd0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfd00679e0 0x7fdfd0069ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40af000 tx=0x7fdfe4004ac0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdff40aec20 msgr2=0x7fdff40ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdff40aec20 0x7fdff40ce270 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c450 tx=0x7fdfdc00c910 comp rx=0 tx=0).stop 2024-09-10T10:02:28.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 shutdown_connections 2024-09-10T10:02:28.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfd00679e0 0x7fdfd0069ea0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdff41621e0 0x7fdff4161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdff40aec20 0x7fdff40ce270 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 --2- 172.21.15.110:0/2145094499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdff40ae840 0x7fdff40b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 >> 172.21.15.110:0/2145094499 conn(0x7fdff406cd60 msgr2=0x7fdff40b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:28.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 shutdown_connections 2024-09-10T10:02:28.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.236+0000 7fdff3e00640 1 -- 172.21.15.110:0/2145094499 wait complete. 2024-09-10T10:02:28.258 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:02:28.258 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:02:28.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f2651600640 1 Processor -- start 2024-09-10T10:02:28.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f2651600640 1 -- start start 2024-09-10T10:02:28.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f2651600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c155970 0x7f264c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f2651600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f264c15c560 0x7f264c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f2651600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f264c06f010 0x7f264c15eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f2651600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f264c0b4570 con 0x7f264c06f010 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f2651600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f264c06f450 con 0x7f264c15c560 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f2651600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f264c155d50 con 0x7f264c155970 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f264c06f010 0x7f264c15eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f264c06f010 0x7f264c15eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57080/0 (socket says 172.21.15.110:57080) 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264b400640 1 -- 172.21.15.110:0/2758089655 learned_addr learned my addr 172.21.15.110:0/2758089655 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264aa00640 1 --2- 172.21.15.110:0/2758089655 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c155970 0x7f264c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264a000640 1 --2- 172.21.15.110:0/2758089655 >> v2:172.21.15.145:3300/0 conn(0x7f264c15c560 0x7f264c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264b400640 1 -- 172.21.15.110:0/2758089655 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c155970 msgr2=0x7f264c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264b400640 1 --2- 172.21.15.110:0/2758089655 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c155970 0x7f264c0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264b400640 1 -- 172.21.15.110:0/2758089655 >> v2:172.21.15.145:3300/0 conn(0x7f264c15c560 msgr2=0x7f264c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:28.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264b400640 1 --2- 172.21.15.110:0/2758089655 >> v2:172.21.15.145:3300/0 conn(0x7f264c15c560 0x7f264c06ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.396+0000 7f264b400640 1 -- 172.21.15.110:0/2758089655 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0b34d0 con 0x7f264c06f010 2024-09-10T10:02:28.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f264b400640 1 --2- 172.21.15.110:0/2758089655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c15eea0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f264c06ca90 tx=0x7f264401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae817bac33b983e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:28.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2649600640 1 -- 172.21.15.110:0/2758089655 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440a6500 con 0x7f264c06f010 2024-09-10T10:02:28.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2649600640 1 -- 172.21.15.110:0/2758089655 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26440a66a0 con 0x7f264c06f010 2024-09-10T10:02:28.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2649600640 1 -- 172.21.15.110:0/2758089655 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440a6a70 con 0x7f264c06f010 2024-09-10T10:02:28.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 -- 172.21.15.110:0/2758089655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 msgr2=0x7f264c15eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 --2- 172.21.15.110:0/2758089655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c15eea0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f264c06ca90 tx=0x7f264401c670 comp rx=0 tx=0).stop 2024-09-10T10:02:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 -- 172.21.15.110:0/2758089655 shutdown_connections 2024-09-10T10:02:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 --2- 172.21.15.110:0/2758089655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c15eea0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 --2- 172.21.15.110:0/2758089655 >> v2:172.21.15.145:3300/0 conn(0x7f264c15c560 0x7f264c06ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 --2- 172.21.15.110:0/2758089655 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c155970 0x7f264c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 -- 172.21.15.110:0/2758089655 >> 172.21.15.110:0/2758089655 conn(0x7f264c06bc50 msgr2=0x7f264c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 -- 172.21.15.110:0/2758089655 shutdown_connections 2024-09-10T10:02:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.400+0000 7f2651600640 1 -- 172.21.15.110:0/2758089655 wait complete. 2024-09-10T10:02:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 Processor -- start 2024-09-10T10:02:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 -- start start 2024-09-10T10:02:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c155970 0x7f264c147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c15c560 0x7f264c1651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0cce00 con 0x7f264c06f010 2024-09-10T10:02:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0b4050 con 0x7f264c155970 2024-09-10T10:02:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0ce420 con 0x7f264c15c560 2024-09-10T10:02:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57084/0 (socket says 172.21.15.110:57084) 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 -- 172.21.15.110:0/2356658211 learned_addr learned my addr 172.21.15.110:0/2356658211 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264a000640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c155970 0x7f264c147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264b400640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c15c560 0x7f264c1651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 -- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c15c560 msgr2=0x7f264c1651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c15c560 0x7f264c1651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 -- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c155970 msgr2=0x7f264c147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c155970 0x7f264c147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 -- 172.21.15.110:0/2356658211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0b0ae0 con 0x7f264c06f010 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f264aa00640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c154a40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d800 tx=0x7f263c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2642a00640 1 -- 172.21.15.110:0/2356658211 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c018020 con 0x7f264c06f010 2024-09-10T10:02:28.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2642a00640 1 -- 172.21.15.110:0/2356658211 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c0096c0 con 0x7f264c06f010 2024-09-10T10:02:28.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2642a00640 1 -- 172.21.15.110:0/2356658211 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c009ab0 con 0x7f264c06f010 2024-09-10T10:02:28.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c156400 con 0x7f264c06f010 2024-09-10T10:02:28.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c1569c0 con 0x7f264c06f010 2024-09-10T10:02:28.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.404+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c0cc8c0 con 0x7f264c06f010 2024-09-10T10:02:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.408+0000 7f2642a00640 1 -- 172.21.15.110:0/2356658211 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f263c01b020 con 0x7f264c06f010 2024-09-10T10:02:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.408+0000 7f2642a00640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f262c067a10 0x7f262c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.408+0000 7f2642a00640 1 -- 172.21.15.110:0/2356658211 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7f263c085f60 con 0x7f264c06f010 2024-09-10T10:02:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.408+0000 7f264a000640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f262c067a10 0x7f262c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.408+0000 7f264a000640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f262c067a10 0x7f262c069ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f264c15f3b0 tx=0x7f2638008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:28.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.412+0000 7f2642a00640 1 -- 172.21.15.110:0/2356658211 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f263c022020 con 0x7f264c06f010 2024-09-10T10:02:28.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:28.572+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f264c06dc80 con 0x7f264c06f010 2024-09-10T10:02:29.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.512+0000 7f2642a00640 1 -- 172.21.15.110:0/2356658211 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) ==== 138+0+0 (secure 0 0 0) 0x7f263c0565f0 con 0x7f264c06f010 2024-09-10T10:02:29.515 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:02:29.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f262c067a10 msgr2=0x7f262c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f262c067a10 0x7f262c069ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f264c15f3b0 tx=0x7f2638008040 comp rx=0 tx=0).stop 2024-09-10T10:02:29.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 msgr2=0x7f264c154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c154a40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d800 tx=0x7f263c00dcc0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 shutdown_connections 2024-09-10T10:02:29.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f262c067a10 0x7f262c069ed0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f264c15c560 0x7f264c1651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f264c155970 0x7f264c147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 --2- 172.21.15.110:0/2356658211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c06f010 0x7f264c154a40 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 >> 172.21.15.110:0/2356658211 conn(0x7f264c06bc50 msgr2=0x7f264c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 shutdown_connections 2024-09-10T10:02:29.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.516+0000 7f2651600640 1 -- 172.21.15.110:0/2356658211 wait complete. 2024-09-10T10:02:29.547 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:02:29.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.688+0000 7fd751600640 1 Processor -- start 2024-09-10T10:02:29.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.688+0000 7fd751600640 1 -- start start 2024-09-10T10:02:29.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd751600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd74c06f1e0 0x7fd74c166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd751600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd74c155d50 con 0x7fd74c15c560 2024-09-10T10:02:29.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd751600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd74c0b34d0 con 0x7fd74c155970 2024-09-10T10:02:29.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd751600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd74c0b4570 con 0x7fd74c06f1e0 2024-09-10T10:02:29.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74a000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fd74c15c560 0x7fd74c06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52380/0 (socket says 172.21.15.110:52380) 2024-09-10T10:02:29.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74a000640 1 -- 172.21.15.110:0/4031251489 learned_addr learned my addr 172.21.15.110:0/4031251489 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:29.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74b400640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd74c06f1e0 0x7fd74c166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1993319312 0 0) 0x7fd74c155d50 con 0x7fd74c15c560 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd730003530 con 0x7fd74c15c560 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797914058 0 0) 0x7fd74c0b34d0 con 0x7fd74c155970 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd74c155d50 con 0x7fd74c155970 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1773966063 0 0) 0x7fd730003530 con 0x7fd74c15c560 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd74c06f1e0 msgr2=0x7fd74c166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd74c06f1e0 0x7fd74c166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 >> v1:172.21.15.145:6789/0 conn(0x7fd74c155970 legacy=0x7fd74c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c0b37b0 con 0x7fd74c15c560 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fd73c0016a0 con 0x7fd74c15c560 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 >> v1:172.21.15.110:6789/0 conn(0x7fd74c15c560 legacy=0x7fd74c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd730003b80 0x7fd730003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd730004810 0x7fd730004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd74c06f1e0 0x7fd7300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd73000ba80 con 0x7fd730003b80 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd73000bc00 con 0x7fd730004810 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd73000be00 con 0x7fd74c06f1e0 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74aa00640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd730003b80 0x7fd730003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74a000640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd730004810 0x7fd730004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74b400640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd74c06f1e0 0x7fd7300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74aa00640 1 -- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd74c06f1e0 msgr2=0x7fd7300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74aa00640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd74c06f1e0 0x7fd7300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74aa00640 1 -- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd730004810 msgr2=0x7fd730004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74aa00640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd730004810 0x7fd730004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.692+0000 7fd74aa00640 1 -- 172.21.15.110:0/4031251489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd730003530 con 0x7fd730003b80 2024-09-10T10:02:29.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd74aa00640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd730003b80 0x7fd730003f80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd734007fd0 tx=0x7fd73401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54ba5317d69d225 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340b0070 con 0x7fd730003b80 2024-09-10T10:02:29.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7340050d0 con 0x7fd730003b80 2024-09-10T10:02:29.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd749600640 1 -- 172.21.15.110:0/4031251489 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734001520 con 0x7fd730003b80 2024-09-10T10:02:29.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd751600640 1 -- 172.21.15.110:0/4031251489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd730003b80 msgr2=0x7fd730003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd751600640 1 --2- 172.21.15.110:0/4031251489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd730003b80 0x7fd730003f80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd734007fd0 tx=0x7fd73401c510 comp rx=0 tx=0).stop 2024-09-10T10:02:29.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd74b400640 1 -- 172.21.15.110:0/4031251489 reap_dead start 2024-09-10T10:02:29.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd751600640 1 -- 172.21.15.110:0/4031251489 shutdown_connections 2024-09-10T10:02:29.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd751600640 1 -- 172.21.15.110:0/4031251489 >> 172.21.15.110:0/4031251489 conn(0x7fd74c06bc50 msgr2=0x7fd74c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd751600640 1 -- 172.21.15.110:0/4031251489 shutdown_connections 2024-09-10T10:02:29.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.696+0000 7fd751600640 1 -- 172.21.15.110:0/4031251489 wait complete. 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 Processor -- start 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 -- start start 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd730003b80 0x7fd74c13db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd74c151940 0x7fd74c0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd74c0aed80 0x7fd74c13f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c1669b0 con 0x7fd74c151940 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c166830 con 0x7fd74c0aed80 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd74c166b30 con 0x7fd730003b80 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd74c151940 0x7fd74c0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd74c151940 0x7fd74c0ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57098/0 (socket says 172.21.15.110:57098) 2024-09-10T10:02:29.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 -- 172.21.15.110:0/4092706958 learned_addr learned my addr 172.21.15.110:0/4092706958 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74aa00640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd730003b80 0x7fd74c13db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74b400640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd74c0aed80 0x7fd74c13f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 -- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd730003b80 msgr2=0x7fd74c13db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd730003b80 0x7fd74c13db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 -- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd74c0aed80 msgr2=0x7fd74c13f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd74c0aed80 0x7fd74c13f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 -- 172.21.15.110:0/4092706958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c140450 con 0x7fd74c151940 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd74a000640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd74c151940 0x7fd74c0ae840 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0072f0 tx=0x7fd73c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd742a00640 1 -- 172.21.15.110:0/4092706958 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c01a070 con 0x7fd74c151940 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd742a00640 1 -- 172.21.15.110:0/4092706958 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c00f9c0 con 0x7fd74c151940 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd742a00640 1 -- 172.21.15.110:0/4092706958 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00fd90 con 0x7fd74c151940 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74c140730 con 0x7fd74c151940 2024-09-10T10:02:29.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74c153180 con 0x7fd74c151940 2024-09-10T10:02:29.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.700+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74c0cc8c0 con 0x7fd74c151940 2024-09-10T10:02:29.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.704+0000 7fd742a00640 1 -- 172.21.15.110:0/4092706958 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fd73c01d360 con 0x7fd74c151940 2024-09-10T10:02:29.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.704+0000 7fd742a00640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd7280679e0 0x7fd728069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.704+0000 7fd742a00640 1 -- 172.21.15.110:0/4092706958 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5227+0+0 (secure 0 0 0) 0x7fd73c085910 con 0x7fd74c151940 2024-09-10T10:02:29.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.704+0000 7fd74aa00640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd7280679e0 0x7fd728069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.704+0000 7fd74aa00640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd7280679e0 0x7fd728069ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd734007fd0 tx=0x7fd7340a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.708+0000 7fd742a00640 1 -- 172.21.15.110:0/4092706958 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd73c055ec0 con 0x7fd74c151940 2024-09-10T10:02:29.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:29.864+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd74c06c470 con 0x7fd74c151940 2024-09-10T10:02:30.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd742a00640 1 -- 172.21.15.110:0/4092706958 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) ==== 170+0+0 (secure 0 0 0) 0x7fd73c05a1c0 con 0x7fd74c151940 2024-09-10T10:02:30.522 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:02:30.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd7280679e0 msgr2=0x7fd728069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd7280679e0 0x7fd728069ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd734007fd0 tx=0x7fd7340a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:30.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd74c151940 msgr2=0x7fd74c0ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd74c151940 0x7fd74c0ae840 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0072f0 tx=0x7fd73c0077b0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 shutdown_connections 2024-09-10T10:02:30.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd7280679e0 0x7fd728069ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd74c0aed80 0x7fd74c13f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd74c151940 0x7fd74c0ae840 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 --2- 172.21.15.110:0/4092706958 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd730003b80 0x7fd74c13db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 >> 172.21.15.110:0/4092706958 conn(0x7fd74c06bc50 msgr2=0x7fd74c0b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:30.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.520+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 shutdown_connections 2024-09-10T10:02:30.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.524+0000 7fd751600640 1 -- 172.21.15.110:0/4092706958 wait complete. 2024-09-10T10:02:30.559 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:02:30.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd331e00640 1 Processor -- start 2024-09-10T10:02:30.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd331e00640 1 -- start start 2024-09-10T10:02:30.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd331e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd32c06e850 0x7fd32c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd331e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd32c0b4960 con 0x7fd32c0ce4d0 2024-09-10T10:02:30.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd331e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd32c0b4c40 con 0x7fd32c15b330 2024-09-10T10:02:30.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd331e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd32c0b5570 con 0x7fd32c06e850 2024-09-10T10:02:30.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32a000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fd32c0ce4d0 0x7fd32c0b0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52396/0 (socket says 172.21.15.110:52396) 2024-09-10T10:02:30.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32a000640 1 -- 172.21.15.110:0/467096788 learned_addr learned my addr 172.21.15.110:0/467096788 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32b400640 1 --2- 172.21.15.110:0/467096788 >> v2:172.21.15.145:3300/0 conn(0x7fd32c06e850 0x7fd32c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847500751 0 0) 0x7fd32c0b4960 con 0x7fd32c0ce4d0 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd310003530 con 0x7fd32c0ce4d0 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2019291681 0 0) 0x7fd32c0b4c40 con 0x7fd32c15b330 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd32c0b4960 con 0x7fd32c15b330 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1068786347 0 0) 0x7fd310003530 con 0x7fd32c0ce4d0 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 >> v2:172.21.15.145:3300/0 conn(0x7fd32c06e850 msgr2=0x7fd32c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 --2- 172.21.15.110:0/467096788 >> v2:172.21.15.145:3300/0 conn(0x7fd32c06e850 0x7fd32c06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 >> v1:172.21.15.145:6789/0 conn(0x7fd32c15b330 legacy=0x7fd32c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c0b4f20 con 0x7fd32c0ce4d0 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fd318001680 con 0x7fd32c0ce4d0 2024-09-10T10:02:30.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 >> v1:172.21.15.110:6789/0 conn(0x7fd32c0ce4d0 legacy=0x7fd32c0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd310003b50 0x7fd310003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd310004810 0x7fd310004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd31000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c0b5570 con 0x7fd3100051d0 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd31000e260 con 0x7fd310003b50 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd31000e460 con 0x7fd310004810 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32b400640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd31000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32aa00640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd310003b50 0x7fd310003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32a000640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd310004810 0x7fd310004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32b400640 1 -- 172.21.15.110:0/467096788 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd310004810 msgr2=0x7fd310004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32b400640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd310004810 0x7fd310004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32b400640 1 -- 172.21.15.110:0/467096788 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd310003b50 msgr2=0x7fd310003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:30.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32b400640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd310003b50 0x7fd310003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.700+0000 7fd32b400640 1 -- 172.21.15.110:0/467096788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd310003530 con 0x7fd3100051d0 2024-09-10T10:02:30.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd32b400640 1 -- 172.21.15.110:0/467096788 reap_dead start 2024-09-10T10:02:30.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd32b400640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd31000b9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0b0670 tx=0x7fd32401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff499db87700d8d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:30.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3240b2070 con 0x7fd3100051d0 2024-09-10T10:02:30.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3240a5c70 con 0x7fd3100051d0 2024-09-10T10:02:30.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd329600640 1 -- 172.21.15.110:0/467096788 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324009c10 con 0x7fd3100051d0 2024-09-10T10:02:30.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd331e00640 1 -- 172.21.15.110:0/467096788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 msgr2=0x7fd31000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd331e00640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd31000b9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0b0670 tx=0x7fd32401cc60 comp rx=0 tx=0).stop 2024-09-10T10:02:30.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd331e00640 1 -- 172.21.15.110:0/467096788 shutdown_connections 2024-09-10T10:02:30.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd331e00640 1 --2- 172.21.15.110:0/467096788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd31000b9d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd331e00640 1 -- 172.21.15.110:0/467096788 >> 172.21.15.110:0/467096788 conn(0x7fd32c062e40 msgr2=0x7fd32c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:30.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd331e00640 1 -- 172.21.15.110:0/467096788 shutdown_connections 2024-09-10T10:02:30.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.704+0000 7fd331e00640 1 -- 172.21.15.110:0/467096788 wait complete. 2024-09-10T10:02:30.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 Processor -- start 2024-09-10T10:02:30.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 -- start start 2024-09-10T10:02:30.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd32c0c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd32c15c2a0 0x7fd32c14ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd32c0c5ba0 0x7fd32c0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c0b1c60 con 0x7fd3100051d0 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c15b710 con 0x7fd32c15c2a0 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c0b1de0 con 0x7fd32c0c5ba0 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd32c0c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd32c0c6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57122/0 (socket says 172.21.15.110:57122) 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 -- 172.21.15.110:0/986496441 learned_addr learned my addr 172.21.15.110:0/986496441 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 -- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd32c0c5ba0 msgr2=0x7fd32c0c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd32c0c5ba0 0x7fd32c0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 -- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd32c15c2a0 msgr2=0x7fd32c14ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd32c15c2a0 0x7fd32c14ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 -- 172.21.15.110:0/986496441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c15ca60 con 0x7fd3100051d0 2024-09-10T10:02:30.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd32aa00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd32c0c6740 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00f7a0 tx=0x7fd31c00fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:30.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd323400640 1 -- 172.21.15.110:0/986496441 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c01a070 con 0x7fd3100051d0 2024-09-10T10:02:30.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd323400640 1 -- 172.21.15.110:0/986496441 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c005110 con 0x7fd3100051d0 2024-09-10T10:02:30.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c15cd40 con 0x7fd3100051d0 2024-09-10T10:02:30.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd323400640 1 -- 172.21.15.110:0/986496441 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0067e0 con 0x7fd3100051d0 2024-09-10T10:02:30.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.708+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c15e310 con 0x7fd3100051d0 2024-09-10T10:02:30.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.712+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c15e510 con 0x7fd3100051d0 2024-09-10T10:02:30.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.712+0000 7fd323400640 1 -- 172.21.15.110:0/986496441 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fd31c01d070 con 0x7fd3100051d0 2024-09-10T10:02:30.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.712+0000 7fd323400640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd30c067a10 0x7fd30c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.712+0000 7fd323400640 1 -- 172.21.15.110:0/986496441 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5609+0+0 (secure 0 0 0) 0x7fd31c0858e0 con 0x7fd3100051d0 2024-09-10T10:02:30.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.712+0000 7fd32a000640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd30c067a10 0x7fd30c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.712+0000 7fd32a000640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd30c067a10 0x7fd30c069ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd318004880 tx=0x7fd3180047f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:30.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.716+0000 7fd323400640 1 -- 172.21.15.110:0/986496441 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd32c15e510 con 0x7fd3100051d0 2024-09-10T10:02:30.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:30.908+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd32c15e100 con 0x7fd3100051d0 2024-09-10T10:02:31.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.520+0000 7fd323400640 1 -- 172.21.15.110:0/986496441 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v50) ==== 174+0+0 (secure 0 0 0) 0x7fd31c056350 con 0x7fd3100051d0 2024-09-10T10:02:31.525 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 12 and data pool 13 2024-09-10T10:02:31.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.524+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd30c067a10 msgr2=0x7fd30c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.524+0000 7fd331e00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd30c067a10 0x7fd30c069ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd318004880 tx=0x7fd3180047f0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.524+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 msgr2=0x7fd32c0c6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.524+0000 7fd331e00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd32c0c6740 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00f7a0 tx=0x7fd31c00fba0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.528+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 shutdown_connections 2024-09-10T10:02:31.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.528+0000 7fd331e00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd30c067a10 0x7fd30c069ed0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.528+0000 7fd331e00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd32c0c5ba0 0x7fd32c0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.528+0000 7fd331e00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd32c15c2a0 0x7fd32c14ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.528+0000 7fd331e00640 1 --2- 172.21.15.110:0/986496441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3100051d0 0x7fd32c0c6740 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.528+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 >> 172.21.15.110:0/986496441 conn(0x7fd32c062e40 msgr2=0x7fd32c06d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:31.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.528+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 shutdown_connections 2024-09-10T10:02:31.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.528+0000 7fd331e00640 1 -- 172.21.15.110:0/986496441 wait complete. 2024-09-10T10:02:31.581 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:31.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10ad200640 1 Processor -- start 2024-09-10T10:02:31.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10ad200640 1 -- start start 2024-09-10T10:02:31.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10ad200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cfe30 0x7f10a8071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10ad200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10a80cd510 0x7f10a80cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10ad200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10a806d2d0 con 0x7f10a806e090 2024-09-10T10:02:31.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10ad200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f10a80cdeb0 con 0x7f10a80cd510 2024-09-10T10:02:31.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10ad200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f10a815d9a0 con 0x7f10a80cfe30 2024-09-10T10:02:31.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10a5e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cfe30 0x7f10a8071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10a5400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10a80cd510 0x7f10a80cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10a6800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f10a806e090 0x7f10a806e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:49544/0 (socket says 172.21.15.110:49544) 2024-09-10T10:02:31.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.716+0000 7f10a6800640 1 -- 172.21.15.110:0/1997227990 learned_addr learned my addr 172.21.15.110:0/1997227990 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:31.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.720+0000 7f10a5e00640 1 -- 172.21.15.110:0/1997227990 >> v2:172.21.15.145:3300/0 conn(0x7f10a80cd510 msgr2=0x7f10a80cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:31.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.720+0000 7f10a5e00640 1 --2- 172.21.15.110:0/1997227990 >> v2:172.21.15.145:3300/0 conn(0x7f10a80cd510 0x7f10a80cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.720+0000 7f10a5e00640 1 -- 172.21.15.110:0/1997227990 >> v1:172.21.15.145:6789/0 conn(0x7f10a806e090 legacy=0x7f10a806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.720+0000 7f10a5e00640 1 -- 172.21.15.110:0/1997227990 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a806d5b0 con 0x7f10a80cfe30 2024-09-10T10:02:31.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.720+0000 7f10a4a00640 1 -- 172.21.15.110:0/1997227990 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890217899 0 0) 0x7f10a806d2d0 con 0x7f10a806e090 2024-09-10T10:02:31.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.720+0000 7f10a5e00640 1 --2- 172.21.15.110:0/1997227990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cfe30 0x7f10a8071a40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1098009d90 tx=0x7f109801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6182b1241264aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:31.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.720+0000 7f10a4a00640 1 -- 172.21.15.110:0/1997227990 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10980b2070 con 0x7f10a80cfe30 2024-09-10T10:02:31.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10a4a00640 1 -- 172.21.15.110:0/1997227990 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10980a58b0 con 0x7f10a80cfe30 2024-09-10T10:02:31.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10a4a00640 1 -- 172.21.15.110:0/1997227990 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10980a5c80 con 0x7f10a80cfe30 2024-09-10T10:02:31.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- 172.21.15.110:0/1997227990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cfe30 msgr2=0x7f10a8071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 --2- 172.21.15.110:0/1997227990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cfe30 0x7f10a8071a40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1098009d90 tx=0x7f109801c8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- 172.21.15.110:0/1997227990 shutdown_connections 2024-09-10T10:02:31.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 --2- 172.21.15.110:0/1997227990 >> v2:172.21.15.145:3300/0 conn(0x7f10a80cd510 0x7f10a80cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 --2- 172.21.15.110:0/1997227990 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cfe30 0x7f10a8071a40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- 172.21.15.110:0/1997227990 >> 172.21.15.110:0/1997227990 conn(0x7f10a806bc50 msgr2=0x7f10a806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:31.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- 172.21.15.110:0/1997227990 shutdown_connections 2024-09-10T10:02:31.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- 172.21.15.110:0/1997227990 wait complete. 2024-09-10T10:02:31.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 Processor -- start 2024-09-10T10:02:31.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- start start 2024-09-10T10:02:31.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10a806e090 0x7f10a81f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cd510 0x7f10a80b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10a80cfe30 0x7f10a8152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8072190 con 0x7f10a80cfe30 2024-09-10T10:02:31.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10a815d640 con 0x7f10a806e090 2024-09-10T10:02:31.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10ad200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f10a8169180 con 0x7f10a80cd510 2024-09-10T10:02:31.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10a5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10a806e090 0x7f10a81f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10a6800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10a80cfe30 0x7f10a8152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10a5400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cd510 0x7f10a80b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10a5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10a806e090 0x7f10a81f6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48148/0 (socket says 172.21.15.110:48148) 2024-09-10T10:02:31.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.724+0000 7f10a5e00640 1 -- 172.21.15.110:0/1222412247 learned_addr learned my addr 172.21.15.110:0/1222412247 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:31.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10a5400640 1 -- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10a806e090 msgr2=0x7f10a81f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10a5400640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10a806e090 0x7f10a81f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10a5400640 1 -- 172.21.15.110:0/1222412247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10a80cfe30 msgr2=0x7f10a8152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10a5400640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10a80cfe30 0x7f10a8152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10a5400640 1 -- 172.21.15.110:0/1222412247 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a80b4950 con 0x7f10a80cd510 2024-09-10T10:02:31.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10a5400640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cd510 0x7f10a80b4070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f109400e840 tx=0x7f109400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:31.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f109e000640 1 -- 172.21.15.110:0/1222412247 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094017070 con 0x7f10a80cd510 2024-09-10T10:02:31.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f109e000640 1 -- 172.21.15.110:0/1222412247 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109400c730 con 0x7f10a80cd510 2024-09-10T10:02:31.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f109e000640 1 -- 172.21.15.110:0/1222412247 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109400cb00 con 0x7f10a80cd510 2024-09-10T10:02:31.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a80b4c30 con 0x7f10a80cd510 2024-09-10T10:02:31.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a80b51f0 con 0x7f10a80cd510 2024-09-10T10:02:31.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a81f6a60 con 0x7f10a80cd510 2024-09-10T10:02:31.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.728+0000 7f109e000640 1 -- 172.21.15.110:0/1222412247 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f109401a070 con 0x7f10a80cd510 2024-09-10T10:02:31.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.732+0000 7f109e000640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1088067a10 0x7f1088069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.732+0000 7f109e000640 1 -- 172.21.15.110:0/1222412247 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f10940861b0 con 0x7f10a80cd510 2024-09-10T10:02:31.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.732+0000 7f10a5e00640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1088067a10 0x7f1088069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.732+0000 7f10a5e00640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1088067a10 0x7f1088069ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1098000f90 tx=0x7f10980a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:31.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.736+0000 7f109e000640 1 -- 172.21.15.110:0/1222412247 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1094056700 con 0x7f10a80cd510 2024-09-10T10:02:31.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.892+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10a80b5560 con 0x7f10a80cd510 2024-09-10T10:02:31.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.892+0000 7f109e000640 1 -- 172.21.15.110:0/1222412247 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16769 (secure 0 0 0) 0x7f109405aa00 con 0x7f10a80cd510 2024-09-10T10:02:31.896 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:31.897 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":83,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:30.912131+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:02:28.619234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T10:02:29.912872+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1088067a10 msgr2=0x7f1088069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1088067a10 0x7f1088069ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1098000f90 tx=0x7f10980a7070 comp rx=0 tx=0).stop 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cd510 msgr2=0x7f10a80b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cd510 0x7f10a80b4070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f109400e840 tx=0x7f109400ed00 comp rx=0 tx=0).stop 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 shutdown_connections 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1088067a10 0x7f1088069ed0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10a80cfe30 0x7f10a8152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f10a80cd510 0x7f10a80b4070 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 --2- 172.21.15.110:0/1222412247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10a806e090 0x7f10a81f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 >> 172.21.15.110:0/1222412247 conn(0x7f10a806bc50 msgr2=0x7f10a806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 shutdown_connections 2024-09-10T10:02:31.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:31.896+0000 7f10ad200640 1 -- 172.21.15.110:0/1222412247 wait complete. 2024-09-10T10:02:31.919 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:32.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f687ba00640 1 Processor -- start 2024-09-10T10:02:32.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f687ba00640 1 -- start start 2024-09-10T10:02:32.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f687ba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f6874163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f687ba00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6874073290 0x7f6874073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f687ba00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68740706a0 con 0x7f687406eb30 2024-09-10T10:02:32.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f687ba00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f687415ee90 con 0x7f6874073290 2024-09-10T10:02:32.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f687ba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f68740710d0 con 0x7f68740d0cb0 2024-09-10T10:02:32.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6879a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6874073290 0x7f6874073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6873e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f6874163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6879000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f687406eb30 0x7f6874071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:49556/0 (socket says 172.21.15.110:49556) 2024-09-10T10:02:32.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6879000640 1 -- 172.21.15.110:0/156158361 learned_addr learned my addr 172.21.15.110:0/156158361 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:32.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6873e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f6874163000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:48122/0 (socket says 172.21.15.110:48122) 2024-09-10T10:02:32.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6879a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6874073290 0x7f6874073740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48154/0 (socket says 172.21.15.110:48154) 2024-09-10T10:02:32.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6873e00640 1 -- 172.21.15.110:0/156158361 >> v2:172.21.15.145:3300/0 conn(0x7f6874073290 msgr2=0x7f6874073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:32.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6873e00640 1 --2- 172.21.15.110:0/156158361 >> v2:172.21.15.145:3300/0 conn(0x7f6874073290 0x7f6874073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6873e00640 1 -- 172.21.15.110:0/156158361 >> v1:172.21.15.145:6789/0 conn(0x7f687406eb30 legacy=0x7f6874071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.052+0000 7f6873e00640 1 -- 172.21.15.110:0/156158361 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68740d10f0 con 0x7f68740d0cb0 2024-09-10T10:02:32.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f6873e00640 1 --2- 172.21.15.110:0/156158361 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f6874163000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68600097c0 tx=0x7f686001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624f369c73d94a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:32.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f6873400640 1 -- 172.21.15.110:0/156158361 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600b0070 con 0x7f68740d0cb0 2024-09-10T10:02:32.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f6873400640 1 -- 172.21.15.110:0/156158361 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68600ab600 con 0x7f68740d0cb0 2024-09-10T10:02:32.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f6873400640 1 -- 172.21.15.110:0/156158361 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600ab9d0 con 0x7f68740d0cb0 2024-09-10T10:02:32.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f687ba00640 1 -- 172.21.15.110:0/156158361 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 msgr2=0x7f6874163000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f687ba00640 1 --2- 172.21.15.110:0/156158361 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f6874163000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68600097c0 tx=0x7f686001c410 comp rx=0 tx=0).stop 2024-09-10T10:02:32.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f687ba00640 1 -- 172.21.15.110:0/156158361 shutdown_connections 2024-09-10T10:02:32.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f687ba00640 1 --2- 172.21.15.110:0/156158361 >> v2:172.21.15.145:3300/0 conn(0x7f6874073290 0x7f6874073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f687ba00640 1 --2- 172.21.15.110:0/156158361 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f6874163000 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f687ba00640 1 -- 172.21.15.110:0/156158361 >> 172.21.15.110:0/156158361 conn(0x7f68740d25c0 msgr2=0x7f68740d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:32.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f687ba00640 1 -- 172.21.15.110:0/156158361 shutdown_connections 2024-09-10T10:02:32.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.056+0000 7f687ba00640 1 -- 172.21.15.110:0/156158361 wait complete. 2024-09-10T10:02:32.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 Processor -- start 2024-09-10T10:02:32.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 -- start start 2024-09-10T10:02:32.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f687406eb30 0x7f68740ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6874073290 0x7f68740ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f68740cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6874070d60 con 0x7f687406eb30 2024-09-10T10:02:32.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6874071c70 con 0x7f6874073290 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f687406dd60 con 0x7f68740d0cb0 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f687406eb30 0x7f68740ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f687406eb30 0x7f68740ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57140/0 (socket says 172.21.15.110:57140) 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 -- 172.21.15.110:0/211266874 learned_addr learned my addr 172.21.15.110:0/211266874 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6873e00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6874073290 0x7f68740ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879a00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f68740cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 -- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 msgr2=0x7f68740cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f68740cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 -- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6874073290 msgr2=0x7f68740ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6874073290 0x7f68740ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 -- 172.21.15.110:0/211266874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6874161670 con 0x7f687406eb30 2024-09-10T10:02:32.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6879000640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f687406eb30 0x7f68740ce990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f686400cbd0 tx=0x7f686400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:32.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6870c00640 1 -- 172.21.15.110:0/211266874 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864017070 con 0x7f687406eb30 2024-09-10T10:02:32.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6870c00640 1 -- 172.21.15.110:0/211266874 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686400ad40 con 0x7f687406eb30 2024-09-10T10:02:32.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f6870c00640 1 -- 172.21.15.110:0/211266874 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864012600 con 0x7f687406eb30 2024-09-10T10:02:32.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6874161950 con 0x7f687406eb30 2024-09-10T10:02:32.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.060+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6874161320 con 0x7f687406eb30 2024-09-10T10:02:32.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.064+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6874073740 con 0x7f687406eb30 2024-09-10T10:02:32.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.064+0000 7f6870c00640 1 -- 172.21.15.110:0/211266874 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f686401a070 con 0x7f687406eb30 2024-09-10T10:02:32.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.064+0000 7f6870c00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68540679e0 0x7f6854069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.064+0000 7f6870c00640 1 -- 172.21.15.110:0/211266874 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f6864085d70 con 0x7f687406eb30 2024-09-10T10:02:32.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.064+0000 7f6873e00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68540679e0 0x7f6854069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.064+0000 7f6873e00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68540679e0 0x7f6854069ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f68600ad000 tx=0x7f68600a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:32.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.068+0000 7f6870c00640 1 -- 172.21.15.110:0/211266874 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6864056140 con 0x7f687406eb30 2024-09-10T10:02:32.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.228+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6874155890 con 0x7f687406eb30 2024-09-10T10:02:32.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.228+0000 7f6870c00640 1 -- 172.21.15.110:0/211266874 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16769 (secure 0 0 0) 0x7f686405a440 con 0x7f687406eb30 2024-09-10T10:02:32.231 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:32.231 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":83,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:30.912131+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:02:28.619234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T10:02:29.912872+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:32.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68540679e0 msgr2=0x7f6854069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68540679e0 0x7f6854069ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f68600ad000 tx=0x7f68600a6000 comp rx=0 tx=0).stop 2024-09-10T10:02:32.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f687406eb30 msgr2=0x7f68740ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f687406eb30 0x7f68740ce990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f686400cbd0 tx=0x7f686400a430 comp rx=0 tx=0).stop 2024-09-10T10:02:32.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 shutdown_connections 2024-09-10T10:02:32.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68540679e0 0x7f6854069ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68740d0cb0 0x7f68740cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6874073290 0x7f68740ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 --2- 172.21.15.110:0/211266874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f687406eb30 0x7f68740ce990 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 >> 172.21.15.110:0/211266874 conn(0x7f68740d25c0 msgr2=0x7f6874074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:32.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 shutdown_connections 2024-09-10T10:02:32.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.232+0000 7f687ba00640 1 -- 172.21.15.110:0/211266874 wait complete. 2024-09-10T10:02:32.252 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:02:32.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc953600640 1 Processor -- start 2024-09-10T10:02:32.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc953600640 1 -- start start 2024-09-10T10:02:32.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc953600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc94c06f100 0x7fc94c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc953600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94c072690 con 0x7fc94c06ec70 2024-09-10T10:02:32.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc953600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94c072970 con 0x7fc94c15f010 2024-09-10T10:02:32.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc953600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc94c072c50 con 0x7fc94c06f100 2024-09-10T10:02:32.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc94be00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc94c06ec70 0x7fc94c0b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52404/0 (socket says 172.21.15.110:52404) 2024-09-10T10:02:32.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc94be00640 1 -- 172.21.15.110:0/1987192140 learned_addr learned my addr 172.21.15.110:0/1987192140 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:32.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.388+0000 7fc951600640 1 --2- 172.21.15.110:0/1987192140 >> v2:172.21.15.110:3300/0 conn(0x7fc94c06f100 0x7fc94c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc951600640 1 -- 172.21.15.110:0/1987192140 >> v1:172.21.15.145:6789/0 conn(0x7fc94c15f010 legacy=0x7fc94c15f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc951600640 1 -- 172.21.15.110:0/1987192140 >> v1:172.21.15.110:6789/0 conn(0x7fc94c06ec70 legacy=0x7fc94c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc951600640 1 -- 172.21.15.110:0/1987192140 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c072f30 con 0x7fc94c06f100 2024-09-10T10:02:32.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc94b400640 1 -- 172.21.15.110:0/1987192140 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3108776949 0 0) 0x7fc94c072690 con 0x7fc94c06ec70 2024-09-10T10:02:32.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc951600640 1 --2- 172.21.15.110:0/1987192140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06f100 0x7fc94c0b5390 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc944009d90 tx=0x7fc94401d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e7b1e97107a5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:32.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc94b400640 1 -- 172.21.15.110:0/1987192140 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440b2070 con 0x7fc94c06f100 2024-09-10T10:02:32.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc94b400640 1 -- 172.21.15.110:0/1987192140 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9440a5990 con 0x7fc94c06f100 2024-09-10T10:02:32.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc94b400640 1 -- 172.21.15.110:0/1987192140 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440a5d60 con 0x7fc94c06f100 2024-09-10T10:02:32.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc953600640 1 -- 172.21.15.110:0/1987192140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06f100 msgr2=0x7fc94c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc953600640 1 --2- 172.21.15.110:0/1987192140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06f100 0x7fc94c0b5390 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc944009d90 tx=0x7fc94401d310 comp rx=0 tx=0).stop 2024-09-10T10:02:32.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc953600640 1 -- 172.21.15.110:0/1987192140 shutdown_connections 2024-09-10T10:02:32.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc953600640 1 --2- 172.21.15.110:0/1987192140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06f100 0x7fc94c0b5390 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.392+0000 7fc953600640 1 -- 172.21.15.110:0/1987192140 >> 172.21.15.110:0/1987192140 conn(0x7fc94c06ca70 msgr2=0x7fc94c06ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:32.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 -- 172.21.15.110:0/1987192140 shutdown_connections 2024-09-10T10:02:32.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 -- 172.21.15.110:0/1987192140 wait complete. 2024-09-10T10:02:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 Processor -- start 2024-09-10T10:02:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 -- start start 2024-09-10T10:02:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06ec70 0x7fc94c1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc94c06f100 0x7fc94c147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc94c15f010 0x7fc94c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c062d10 con 0x7fc94c06ec70 2024-09-10T10:02:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c062b90 con 0x7fc94c06f100 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc953600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc94c0722d0 con 0x7fc94c15f010 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06ec70 0x7fc94c1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06ec70 0x7fc94c1492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57156/0 (socket says 172.21.15.110:57156) 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 -- 172.21.15.110:0/4168956426 learned_addr learned my addr 172.21.15.110:0/4168956426 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc951600640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc94c15f010 0x7fc94c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc94be00640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc94c06f100 0x7fc94c147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 -- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc94c15f010 msgr2=0x7fc94c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc94c15f010 0x7fc94c0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 -- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc94c06f100 msgr2=0x7fc94c147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc94c06f100 0x7fc94c147490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 -- 172.21.15.110:0/4168956426 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c149800 con 0x7fc94c06ec70 2024-09-10T10:02:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc950c00640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06ec70 0x7fc94c1492c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c530 tx=0x7fc94000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:32.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc948c00640 1 -- 172.21.15.110:0/4168956426 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940016020 con 0x7fc94c06ec70 2024-09-10T10:02:32.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc948c00640 1 -- 172.21.15.110:0/4168956426 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc940005220 con 0x7fc94c06ec70 2024-09-10T10:02:32.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.396+0000 7fc948c00640 1 -- 172.21.15.110:0/4168956426 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940007030 con 0x7fc94c06ec70 2024-09-10T10:02:32.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.400+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c14f7c0 con 0x7fc94c06ec70 2024-09-10T10:02:32.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.400+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c14fd80 con 0x7fc94c06ec70 2024-09-10T10:02:32.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.400+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94c0627b0 con 0x7fc94c06ec70 2024-09-10T10:02:32.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.400+0000 7fc948c00640 1 -- 172.21.15.110:0/4168956426 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fc9400071d0 con 0x7fc94c06ec70 2024-09-10T10:02:32.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.400+0000 7fc948c00640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc92c0679e0 0x7fc92c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.400+0000 7fc948c00640 1 -- 172.21.15.110:0/4168956426 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fc940085650 con 0x7fc94c06ec70 2024-09-10T10:02:32.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.400+0000 7fc94be00640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc92c0679e0 0x7fc92c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.400+0000 7fc94be00640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc92c0679e0 0x7fc92c069ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc94c14cc70 tx=0x7fc938007240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:32.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.404+0000 7fc948c00640 1 -- 172.21.15.110:0/4168956426 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc940020020 con 0x7fc94c06ec70 2024-09-10T10:02:32.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:32.600+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc94c15ca90 con 0x7fc94c06ec70 2024-09-10T10:02:33.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc948c00640 1 -- 172.21.15.110:0/4168956426 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) ==== 118+0+0 (secure 0 0 0) 0x7fc94c149800 con 0x7fc94c06ec70 2024-09-10T10:02:33.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc92c0679e0 msgr2=0x7fc92c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc92c0679e0 0x7fc92c069ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc94c14cc70 tx=0x7fc938007240 comp rx=0 tx=0).stop 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06ec70 msgr2=0x7fc94c1492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06ec70 0x7fc94c1492c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c530 tx=0x7fc94000c9f0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 shutdown_connections 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc92c0679e0 0x7fc92c069ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc94c15f010 0x7fc94c0cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc94c06f100 0x7fc94c147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 --2- 172.21.15.110:0/4168956426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c06ec70 0x7fc94c1492c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 >> 172.21.15.110:0/4168956426 conn(0x7fc94c06ca70 msgr2=0x7fc94c0b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 shutdown_connections 2024-09-10T10:02:33.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.548+0000 7fc953600640 1 -- 172.21.15.110:0/4168956426 wait complete. 2024-09-10T10:02:33.573 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:02:33.574 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:02:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.708+0000 7f924aa00640 1 Processor -- start 2024-09-10T10:02:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.708+0000 7f924aa00640 1 -- start start 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f924aa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9244145f30 0x7f92440b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f924aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f92440b29e0 0x7f92440cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f924aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f92440cfbc0 0x7f9244160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f924aa00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f92440b2e50 con 0x7f92440b29e0 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f924aa00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f92440b37c0 con 0x7f9244145f30 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f924aa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f92440b3aa0 con 0x7f92440cfbc0 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f92440b29e0 0x7f92440cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f92440b29e0 0x7f92440cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57166/0 (socket says 172.21.15.110:57166) 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243400640 1 -- 172.21.15.110:0/2637644685 learned_addr learned my addr 172.21.15.110:0/2637644685 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9248a00640 1 --2- 172.21.15.110:0/2637644685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f92440cfbc0 0x7f9244160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243e00640 1 --2- 172.21.15.110:0/2637644685 >> v2:172.21.15.145:3300/0 conn(0x7f9244145f30 0x7f92440b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243400640 1 -- 172.21.15.110:0/2637644685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f92440cfbc0 msgr2=0x7f9244160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243400640 1 --2- 172.21.15.110:0/2637644685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f92440cfbc0 0x7f9244160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243400640 1 -- 172.21.15.110:0/2637644685 >> v2:172.21.15.145:3300/0 conn(0x7f9244145f30 msgr2=0x7f92440b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243400640 1 --2- 172.21.15.110:0/2637644685 >> v2:172.21.15.145:3300/0 conn(0x7f9244145f30 0x7f92440b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243400640 1 -- 172.21.15.110:0/2637644685 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92440b3d80 con 0x7f92440b29e0 2024-09-10T10:02:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9248a00640 1 --2- 172.21.15.110:0/2637644685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f92440cfbc0 0x7f9244160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.712+0000 7f9243e00640 1 --2- 172.21.15.110:0/2637644685 >> v2:172.21.15.145:3300/0 conn(0x7f9244145f30 0x7f92440b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f9243400640 1 --2- 172.21.15.110:0/2637644685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92440cf680 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f92440b3000 tx=0x7f923001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f5112a98a11073 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f9242a00640 1 -- 172.21.15.110:0/2637644685 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92300b0070 con 0x7f92440b29e0 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f9242a00640 1 -- 172.21.15.110:0/2637644685 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92300ab600 con 0x7f92440b29e0 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f9242a00640 1 -- 172.21.15.110:0/2637644685 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92300ab9d0 con 0x7f92440b29e0 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 -- 172.21.15.110:0/2637644685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 msgr2=0x7f92440cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 --2- 172.21.15.110:0/2637644685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92440cf680 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f92440b3000 tx=0x7f923001c260 comp rx=0 tx=0).stop 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 -- 172.21.15.110:0/2637644685 shutdown_connections 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 --2- 172.21.15.110:0/2637644685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f92440cfbc0 0x7f9244160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 --2- 172.21.15.110:0/2637644685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92440cf680 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 --2- 172.21.15.110:0/2637644685 >> v2:172.21.15.145:3300/0 conn(0x7f9244145f30 0x7f92440b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 -- 172.21.15.110:0/2637644685 >> 172.21.15.110:0/2637644685 conn(0x7f9244062e40 msgr2=0x7f924406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 -- 172.21.15.110:0/2637644685 shutdown_connections 2024-09-10T10:02:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 -- 172.21.15.110:0/2637644685 wait complete. 2024-09-10T10:02:33.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 Processor -- start 2024-09-10T10:02:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.716+0000 7f924aa00640 1 -- start start 2024-09-10T10:02:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92441f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92440cfbc0 0x7f924415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9244145f30 0x7f9244140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92440b48f0 con 0x7f92440b29e0 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f92440b4770 con 0x7f92440cfbc0 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f92440b4a70 con 0x7f9244145f30 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92441f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92441f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57182/0 (socket says 172.21.15.110:57182) 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 -- 172.21.15.110:0/3529604077 learned_addr learned my addr 172.21.15.110:0/3529604077 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9248a00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9244145f30 0x7f9244140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 -- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9244145f30 msgr2=0x7f9244140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243400640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92440cfbc0 0x7f924415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9244145f30 0x7f9244140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 -- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92440cfbc0 msgr2=0x7f924415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92440cfbc0 0x7f924415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 -- 172.21.15.110:0/3529604077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92441f7d20 con 0x7f92440b29e0 2024-09-10T10:02:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9248a00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9244145f30 0x7f9244140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243400640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92440cfbc0 0x7f924415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f9243e00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92441f87c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f923400d800 tx=0x7f923400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f923be00640 1 -- 172.21.15.110:0/3529604077 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234018020 con 0x7f92440b29e0 2024-09-10T10:02:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f923be00640 1 -- 172.21.15.110:0/3529604077 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92340096c0 con 0x7f92440b29e0 2024-09-10T10:02:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924415f3c0 con 0x7f92440b29e0 2024-09-10T10:02:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f923be00640 1 -- 172.21.15.110:0/3529604077 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234009ab0 con 0x7f92440b29e0 2024-09-10T10:02:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924415f6a0 con 0x7f92440b29e0 2024-09-10T10:02:33.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.720+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9244160030 con 0x7f92440b29e0 2024-09-10T10:02:33.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.724+0000 7f923be00640 1 -- 172.21.15.110:0/3529604077 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f923401b020 con 0x7f92440b29e0 2024-09-10T10:02:33.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.724+0000 7f923be00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9224067a10 0x7f9224069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.724+0000 7f923be00640 1 -- 172.21.15.110:0/3529604077 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f9234086370 con 0x7f92440b29e0 2024-09-10T10:02:33.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.724+0000 7f9243400640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9224067a10 0x7f9224069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.724+0000 7f9243400640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9224067a10 0x7f9224069ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92300ad000 tx=0x7f9230008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.728+0000 7f923be00640 1 -- 172.21.15.110:0/3529604077 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9234022020 con 0x7f92440b29e0 2024-09-10T10:02:33.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:33.916+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f92441f83e0 con 0x7f92440b29e0 2024-09-10T10:02:34.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.552+0000 7f923be00640 1 -- 172.21.15.110:0/3529604077 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f9234056d00 con 0x7f92440b29e0 2024-09-10T10:02:34.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9224067a10 msgr2=0x7f9224069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9224067a10 0x7f9224069ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92300ad000 tx=0x7f9230008040 comp rx=0 tx=0).stop 2024-09-10T10:02:34.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 msgr2=0x7f92441f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92441f87c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f923400d800 tx=0x7f923400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 shutdown_connections 2024-09-10T10:02:34.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9224067a10 0x7f9224069ed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9244145f30 0x7f9244140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92440cfbc0 0x7f924415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 --2- 172.21.15.110:0/3529604077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92440b29e0 0x7f92441f87c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 >> 172.21.15.110:0/3529604077 conn(0x7f9244062e40 msgr2=0x7f924415c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 shutdown_connections 2024-09-10T10:02:34.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.556+0000 7f924aa00640 1 -- 172.21.15.110:0/3529604077 wait complete. 2024-09-10T10:02:34.580 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:02:34.580 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:34.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c8a00640 1 Processor -- start 2024-09-10T10:02:34.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c8a00640 1 -- start start 2024-09-10T10:02:34.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c8a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 0x7f53c40d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c8a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f53c4163000 0x7f53c4071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c8a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f53c4073290 0x7f53c4073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c8a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f53c415ee90 con 0x7f53c4163000 2024-09-10T10:02:34.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c8a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f53c40710d0 con 0x7f53c4073290 2024-09-10T10:02:34.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c8a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f53c40d1370 con 0x7f53c406eb30 2024-09-10T10:02:34.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f53c4163000 0x7f53c4071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f53c4163000 0x7f53c4071560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57184/0 (socket says 172.21.15.110:57184) 2024-09-10T10:02:34.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c1400640 1 -- 172.21.15.110:0/1203962288 learned_addr learned my addr 172.21.15.110:0/1203962288 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:34.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c1400640 1 -- 172.21.15.110:0/1203962288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 msgr2=0x7f53c40d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c1400640 1 --2- 172.21.15.110:0/1203962288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 0x7f53c40d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c1400640 1 -- 172.21.15.110:0/1203962288 >> v2:172.21.15.145:3300/0 conn(0x7f53c4073290 msgr2=0x7f53c4073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c2800640 1 --2- 172.21.15.110:0/1203962288 >> v2:172.21.15.145:3300/0 conn(0x7f53c4073290 0x7f53c4073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c1400640 1 --2- 172.21.15.110:0/1203962288 >> v2:172.21.15.145:3300/0 conn(0x7f53c4073290 0x7f53c4073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c1400640 1 -- 172.21.15.110:0/1203962288 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c40706a0 con 0x7f53c4163000 2024-09-10T10:02:34.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.716+0000 7f53c2800640 1 --2- 172.21.15.110:0/1203962288 >> v2:172.21.15.145:3300/0 conn(0x7f53c4073290 0x7f53c4073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:34.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c1400640 1 --2- 172.21.15.110:0/1203962288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4163000 0x7f53c4071560 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f53c4063650 tx=0x7f53b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b01d692df5d5ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c0a00640 1 -- 172.21.15.110:0/1203962288 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00b0070 con 0x7f53c4163000 2024-09-10T10:02:34.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c0a00640 1 -- 172.21.15.110:0/1203962288 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b00ab600 con 0x7f53c4163000 2024-09-10T10:02:34.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c0a00640 1 -- 172.21.15.110:0/1203962288 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00ab9d0 con 0x7f53c4163000 2024-09-10T10:02:34.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 -- 172.21.15.110:0/1203962288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4163000 msgr2=0x7f53c4071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 --2- 172.21.15.110:0/1203962288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4163000 0x7f53c4071560 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f53c4063650 tx=0x7f53b001c490 comp rx=0 tx=0).stop 2024-09-10T10:02:34.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 -- 172.21.15.110:0/1203962288 shutdown_connections 2024-09-10T10:02:34.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 --2- 172.21.15.110:0/1203962288 >> v2:172.21.15.145:3300/0 conn(0x7f53c4073290 0x7f53c4073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 --2- 172.21.15.110:0/1203962288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4163000 0x7f53c4071560 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 --2- 172.21.15.110:0/1203962288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 0x7f53c40d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 -- 172.21.15.110:0/1203962288 >> 172.21.15.110:0/1203962288 conn(0x7f53c40d25c0 msgr2=0x7f53c40d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 -- 172.21.15.110:0/1203962288 shutdown_connections 2024-09-10T10:02:34.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.720+0000 7f53c8a00640 1 -- 172.21.15.110:0/1203962288 wait complete. 2024-09-10T10:02:34.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 Processor -- start 2024-09-10T10:02:34.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 -- start start 2024-09-10T10:02:34.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 0x7f53c4152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4073290 0x7f53c4159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53c4163000 0x7f53c4156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4071aa0 con 0x7f53c4073290 2024-09-10T10:02:34.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4163440 con 0x7f53c4163000 2024-09-10T10:02:34.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f53c4071c20 con 0x7f53c406eb30 2024-09-10T10:02:34.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4073290 0x7f53c4159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4073290 0x7f53c4159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57200/0 (socket says 172.21.15.110:57200) 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 -- 172.21.15.110:0/575726236 learned_addr learned my addr 172.21.15.110:0/575726236 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1e00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 0x7f53c4152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c2800640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53c4163000 0x7f53c4156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 -- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 msgr2=0x7f53c4152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 0x7f53c4152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 -- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53c4163000 msgr2=0x7f53c4156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53c4163000 0x7f53c4156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 -- 172.21.15.110:0/575726236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c4155e00 con 0x7f53c4073290 2024-09-10T10:02:34.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c1400640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4073290 0x7f53c4159610 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f53b001c970 tx=0x7f53b0019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53ba000640 1 -- 172.21.15.110:0/575726236 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00bb070 con 0x7f53c4073290 2024-09-10T10:02:34.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53ba000640 1 -- 172.21.15.110:0/575726236 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b00b6640 con 0x7f53c4073290 2024-09-10T10:02:34.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53ba000640 1 -- 172.21.15.110:0/575726236 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00b0040 con 0x7f53c4073290 2024-09-10T10:02:34.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c41560a0 con 0x7f53c4073290 2024-09-10T10:02:34.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c4156660 con 0x7f53c4073290 2024-09-10T10:02:34.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.724+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c406dd60 con 0x7f53c4073290 2024-09-10T10:02:34.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.728+0000 7f53ba000640 1 -- 172.21.15.110:0/575726236 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f53b00ab4c0 con 0x7f53c4073290 2024-09-10T10:02:34.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.728+0000 7f53ba000640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53a40679e0 0x7f53a4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.728+0000 7f53ba000640 1 -- 172.21.15.110:0/575726236 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f53b011f7d0 con 0x7f53c4073290 2024-09-10T10:02:34.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.728+0000 7f53c1e00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53a40679e0 0x7f53a4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.728+0000 7f53c1e00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53a40679e0 0x7f53a4069ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53b400a400 tx=0x7f53b400d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.732+0000 7f53ba000640 1 -- 172.21.15.110:0/575726236 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f53b0016110 con 0x7f53c4073290 2024-09-10T10:02:34.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53c40d2d50 con 0x7f53c4073290 2024-09-10T10:02:34.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53ba000640 1 -- 172.21.15.110:0/575726236 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5237 (secure 0 0 0) 0x7f53b00efb10 con 0x7f53c4073290 2024-09-10T10:02:34.923 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:34.923 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":54,"btime":"2024-09-10T10:02:34:550856+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1164773100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1164773100},{"type":"v1","addr":"172.21.15.145:6834","nonce":1164773100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1195578364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1195578364},{"type":"v1","addr":"172.21.15.145:6835","nonce":1195578364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6447,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3463064995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3463064995},{"type":"v1","addr":"172.21.15.110:6834","nonce":3463064995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:02:30.912482+0000","modified":"2024-09-10T10:02:33.921168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1947643553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1947643553},{"type":"v1","addr":"172.21.15.110:6835","nonce":1947643553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":6}]} 2024-09-10T10:02:34.924 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 54 2024-09-10T10:02:34.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53a40679e0 msgr2=0x7f53a4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53a40679e0 0x7f53a4069ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53b400a400 tx=0x7f53b400d040 comp rx=0 tx=0).stop 2024-09-10T10:02:34.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4073290 msgr2=0x7f53c4159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4073290 0x7f53c4159610 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f53b001c970 tx=0x7f53b0019230 comp rx=0 tx=0).stop 2024-09-10T10:02:34.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 shutdown_connections 2024-09-10T10:02:34.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53a40679e0 0x7f53a4069ea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53c4163000 0x7f53c4156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53c4073290 0x7f53c4159610 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 --2- 172.21.15.110:0/575726236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53c406eb30 0x7f53c4152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.920+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 >> 172.21.15.110:0/575726236 conn(0x7f53c40d25c0 msgr2=0x7f53c40ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.924+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 shutdown_connections 2024-09-10T10:02:34.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:34.924+0000 7f53c8a00640 1 -- 172.21.15.110:0/575726236 wait complete. 2024-09-10T10:02:34.944 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:35.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.076+0000 7f78efe00640 1 Processor -- start 2024-09-10T10:02:35.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.076+0000 7f78efe00640 1 -- start start 2024-09-10T10:02:35.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.076+0000 7f78efe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f78f0155680 0x7f78f00ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.076+0000 7f78efe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f00cd260 0x7f78f0072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.076+0000 7f78efe00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f00cd7e0 con 0x7f78f00734b0 2024-09-10T10:02:35.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.076+0000 7f78efe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f78f00cdac0 con 0x7f78f0155680 2024-09-10T10:02:35.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.076+0000 7f78efe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f78f0072770 con 0x7f78f00cd260 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78eea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f78f0155680 0x7f78f00ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78ee000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f00cd260 0x7f78f0072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78eea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f78f0155680 0x7f78f00ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48202/0 (socket says 172.21.15.110:48202) 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78eea00640 1 -- 172.21.15.110:0/3889198380 learned_addr learned my addr 172.21.15.110:0/3889198380 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78ef400640 1 --1- 172.21.15.110:0/3889198380 >> v1:172.21.15.145:6789/0 conn(0x7f78f00734b0 0x7f78f00cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:49578/0 (socket says 172.21.15.110:49578) 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78eea00640 1 -- 172.21.15.110:0/3889198380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f00cd260 msgr2=0x7f78f0072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78eea00640 1 --2- 172.21.15.110:0/3889198380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f00cd260 0x7f78f0072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78eea00640 1 -- 172.21.15.110:0/3889198380 >> v1:172.21.15.145:6789/0 conn(0x7f78f00734b0 legacy=0x7f78f00cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78eea00640 1 -- 172.21.15.110:0/3889198380 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f0072a50 con 0x7f78f0155680 2024-09-10T10:02:35.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.080+0000 7f78ee000640 1 --2- 172.21.15.110:0/3889198380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f00cd260 0x7f78f0072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:35.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78eea00640 1 --2- 172.21.15.110:0/3889198380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f0155680 0x7f78f00ccd20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009e50 tx=0x7f78e001ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97f4b1d123bc0279 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78ed600640 1 -- 172.21.15.110:0/3889198380 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e00a7b30 con 0x7f78f0155680 2024-09-10T10:02:35.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78ed600640 1 -- 172.21.15.110:0/3889198380 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e00a7cd0 con 0x7f78f0155680 2024-09-10T10:02:35.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78ed600640 1 -- 172.21.15.110:0/3889198380 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e00af610 con 0x7f78f0155680 2024-09-10T10:02:35.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- 172.21.15.110:0/3889198380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f0155680 msgr2=0x7f78f00ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 --2- 172.21.15.110:0/3889198380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f0155680 0x7f78f00ccd20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009e50 tx=0x7f78e001ec40 comp rx=0 tx=0).stop 2024-09-10T10:02:35.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- 172.21.15.110:0/3889198380 shutdown_connections 2024-09-10T10:02:35.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 --2- 172.21.15.110:0/3889198380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f00cd260 0x7f78f0072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 --2- 172.21.15.110:0/3889198380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f0155680 0x7f78f00ccd20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- 172.21.15.110:0/3889198380 >> 172.21.15.110:0/3889198380 conn(0x7f78f006d0f0 msgr2=0x7f78f006d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- 172.21.15.110:0/3889198380 shutdown_connections 2024-09-10T10:02:35.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- 172.21.15.110:0/3889198380 wait complete. 2024-09-10T10:02:35.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 Processor -- start 2024-09-10T10:02:35.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- start start 2024-09-10T10:02:35.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00734b0 0x7f78f00b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f00cd260 0x7f78f0157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f0155680 0x7f78f0158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78f015d0e0 con 0x7f78f00734b0 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78f015cf60 con 0x7f78f00cd260 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78efe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f78f00cf9f0 con 0x7f78f0155680 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78eea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00734b0 0x7f78f00b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78eea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00734b0 0x7f78f00b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57210/0 (socket says 172.21.15.110:57210) 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.084+0000 7f78eea00640 1 -- 172.21.15.110:0/1002164775 learned_addr learned my addr 172.21.15.110:0/1002164775 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78ef400640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f0155680 0x7f78f0158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78ee000640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f00cd260 0x7f78f0157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78eea00640 1 -- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f0155680 msgr2=0x7f78f0158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78eea00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f0155680 0x7f78f0158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78eea00640 1 -- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f00cd260 msgr2=0x7f78f0157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78eea00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f00cd260 0x7f78f0157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78eea00640 1 -- 172.21.15.110:0/1002164775 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f0164f60 con 0x7f78f00734b0 2024-09-10T10:02:35.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78eea00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00734b0 0x7f78f00b55d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f78e001c870 tx=0x7f78e001b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78e7400640 1 -- 172.21.15.110:0/1002164775 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e00a7b30 con 0x7f78f00734b0 2024-09-10T10:02:35.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78e7400640 1 -- 172.21.15.110:0/1002164775 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e0007c70 con 0x7f78f00734b0 2024-09-10T10:02:35.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78e7400640 1 -- 172.21.15.110:0/1002164775 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0009510 con 0x7f78f00734b0 2024-09-10T10:02:35.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f0165240 con 0x7f78f00734b0 2024-09-10T10:02:35.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f0165800 con 0x7f78f00734b0 2024-09-10T10:02:35.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f0072f70 con 0x7f78f00734b0 2024-09-10T10:02:35.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78e7400640 1 -- 172.21.15.110:0/1002164775 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f78e00c0060 con 0x7f78f00734b0 2024-09-10T10:02:35.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78e7400640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78d00679e0 0x7f78d0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.088+0000 7f78e7400640 1 -- 172.21.15.110:0/1002164775 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f78e0120070 con 0x7f78f00734b0 2024-09-10T10:02:35.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.092+0000 7f78ee000640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78d00679e0 0x7f78d0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.092+0000 7f78ee000640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78d00679e0 0x7f78d0069ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78f014a800 tx=0x7f78dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.092+0000 7f78e7400640 1 -- 172.21.15.110:0/1002164775 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f78e0018830 con 0x7f78f00734b0 2024-09-10T10:02:35.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.252+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78f015b540 con 0x7f78f00734b0 2024-09-10T10:02:35.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.252+0000 7f78e7400640 1 -- 172.21.15.110:0/1002164775 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16756 (secure 0 0 0) 0x7f78e00f0b50 con 0x7f78f00734b0 2024-09-10T10:02:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:35.256 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":84,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:32.535945+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:02:28.619234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T10:02:29.912872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:35.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78d00679e0 msgr2=0x7f78d0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78d00679e0 0x7f78d0069ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78f014a800 tx=0x7f78dc008040 comp rx=0 tx=0).stop 2024-09-10T10:02:35.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00734b0 msgr2=0x7f78f00b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00734b0 0x7f78f00b55d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f78e001c870 tx=0x7f78e001b9e0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 shutdown_connections 2024-09-10T10:02:35.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78d00679e0 0x7f78d0069ea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78f0155680 0x7f78f0158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78f00cd260 0x7f78f0157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 --2- 172.21.15.110:0/1002164775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00734b0 0x7f78f00b55d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 >> 172.21.15.110:0/1002164775 conn(0x7f78f006d0f0 msgr2=0x7f78f0073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 shutdown_connections 2024-09-10T10:02:35.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.256+0000 7f78efe00640 1 -- 172.21.15.110:0/1002164775 wait complete. 2024-09-10T10:02:35.278 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:02:35.278 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:02:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f100ca00640 1 Processor -- start 2024-09-10T10:02:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f100ca00640 1 -- start start 2024-09-10T10:02:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f100ca00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f100806ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f100ca00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10080729b0 0x7f1008072e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f100ca00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f100806eec0 con 0x7f100815e700 2024-09-10T10:02:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f100ca00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f100815eae0 con 0x7f10080729b0 2024-09-10T10:02:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f100ca00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1008073360 con 0x7f100806e890 2024-09-10T10:02:35.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1006800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10080729b0 0x7f1008072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1006800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10080729b0 0x7f1008072e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:48220/0 (socket says 172.21.15.110:48220) 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1005400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f100806ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1006800640 1 -- 172.21.15.110:0/2413628348 learned_addr learned my addr 172.21.15.110:0/2413628348 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1005e00640 1 --1- 172.21.15.110:0/2413628348 >> v1:172.21.15.145:6789/0 conn(0x7f100815e700 0x7f10080af270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:49582/0 (socket says 172.21.15.110:49582) 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1005400640 1 -- 172.21.15.110:0/2413628348 >> v2:172.21.15.145:3300/0 conn(0x7f10080729b0 msgr2=0x7f1008072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1005400640 1 --2- 172.21.15.110:0/2413628348 >> v2:172.21.15.145:3300/0 conn(0x7f10080729b0 0x7f1008072e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1005400640 1 -- 172.21.15.110:0/2413628348 >> v1:172.21.15.145:6789/0 conn(0x7f100815e700 legacy=0x7f10080af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1005400640 1 -- 172.21.15.110:0/2413628348 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100806f1a0 con 0x7f100806e890 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.412+0000 7f1006800640 1 --2- 172.21.15.110:0/2413628348 >> v2:172.21.15.145:3300/0 conn(0x7f10080729b0 0x7f1008072e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:35.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f1005400640 1 --2- 172.21.15.110:0/2413628348 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f100806ff60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40097c0 tx=0x7f0ff401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d418fa121732e940 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f1004a00640 1 -- 172.21.15.110:0/2413628348 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40b2070 con 0x7f100806e890 2024-09-10T10:02:35.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f1004a00640 1 -- 172.21.15.110:0/2413628348 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff40ad640 con 0x7f100806e890 2024-09-10T10:02:35.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f1004a00640 1 -- 172.21.15.110:0/2413628348 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40ada10 con 0x7f100806e890 2024-09-10T10:02:35.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f100ca00640 1 -- 172.21.15.110:0/2413628348 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 msgr2=0x7f100806ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f100ca00640 1 --2- 172.21.15.110:0/2413628348 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f100806ff60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40097c0 tx=0x7f0ff401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f100ca00640 1 -- 172.21.15.110:0/2413628348 shutdown_connections 2024-09-10T10:02:35.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f100ca00640 1 --2- 172.21.15.110:0/2413628348 >> v2:172.21.15.145:3300/0 conn(0x7f10080729b0 0x7f1008072e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f100ca00640 1 --2- 172.21.15.110:0/2413628348 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f100806ff60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f100ca00640 1 -- 172.21.15.110:0/2413628348 >> 172.21.15.110:0/2413628348 conn(0x7f100806cb90 msgr2=0x7f100806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f100ca00640 1 -- 172.21.15.110:0/2413628348 shutdown_connections 2024-09-10T10:02:35.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.416+0000 7f100ca00640 1 -- 172.21.15.110:0/2413628348 wait complete. 2024-09-10T10:02:35.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 Processor -- start 2024-09-10T10:02:35.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 -- start start 2024-09-10T10:02:35.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f10080cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10080729b0 0x7f1008151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100815e700 0x7f1008152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10080b4660 con 0x7f100815e700 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10080af980 con 0x7f10080729b0 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f10080b47e0 con 0x7f100806e890 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100815e700 0x7f1008152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100815e700 0x7f1008152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57222/0 (socket says 172.21.15.110:57222) 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 -- 172.21.15.110:0/34303593 learned_addr learned my addr 172.21.15.110:0/34303593 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1005e00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f10080cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1005400640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10080729b0 0x7f1008151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 -- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 msgr2=0x7f10080cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f10080cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 -- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10080729b0 msgr2=0x7f1008151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10080729b0 0x7f1008151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 -- 172.21.15.110:0/34303593 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1008152ab0 con 0x7f100815e700 2024-09-10T10:02:35.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f1006800640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100815e700 0x7f1008152340 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f100000efc0 tx=0x7f100000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f0ffe000640 1 -- 172.21.15.110:0/34303593 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100000cbe0 con 0x7f100815e700 2024-09-10T10:02:35.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f0ffe000640 1 -- 172.21.15.110:0/34303593 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100000cd80 con 0x7f100815e700 2024-09-10T10:02:35.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1008152d90 con 0x7f100815e700 2024-09-10T10:02:35.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f0ffe000640 1 -- 172.21.15.110:0/34303593 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000003bc0 con 0x7f100815e700 2024-09-10T10:02:35.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.420+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10080cf600 con 0x7f100815e700 2024-09-10T10:02:35.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.424+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10080cf800 con 0x7f100815e700 2024-09-10T10:02:35.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.424+0000 7f0ffe000640 1 -- 172.21.15.110:0/34303593 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f100001b030 con 0x7f100815e700 2024-09-10T10:02:35.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.424+0000 7f0ffe000640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0fe80679e0 0x7f0fe8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.424+0000 7f0ffe000640 1 -- 172.21.15.110:0/34303593 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f1000086370 con 0x7f100815e700 2024-09-10T10:02:35.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.424+0000 7f1005e00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0fe80679e0 0x7f0fe8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.424+0000 7f1005e00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0fe80679e0 0x7f0fe8069ea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8000f70 tx=0x7f0ff8007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.428+0000 7f0ffe000640 1 -- 172.21.15.110:0/34303593 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f10080cf800 con 0x7f100815e700 2024-09-10T10:02:35.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.584+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10080d02e0 con 0x7f0fe80679e0 2024-09-10T10:02:35.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.592+0000 7f0ffe000640 1 -- 172.21.15.110:0/34303593 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252002 (secure 0 0 0) 0x7f10080d02e0 con 0x7f0fe80679e0 2024-09-10T10:02:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:35.599 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0fe80679e0 msgr2=0x7f0fe8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0fe80679e0 0x7f0fe8069ea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8000f70 tx=0x7f0ff8007cd0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100815e700 msgr2=0x7f1008152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100815e700 0x7f1008152340 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f100000efc0 tx=0x7f100000c490 comp rx=0 tx=0).stop 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 shutdown_connections 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0fe80679e0 0x7f0fe8069ea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100815e700 0x7f1008152340 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10080729b0 0x7f1008151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 --2- 172.21.15.110:0/34303593 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f100806e890 0x7f10080cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 >> 172.21.15.110:0/34303593 conn(0x7f100806cb90 msgr2=0x7f10080af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 shutdown_connections 2024-09-10T10:02:35.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.596+0000 7f100ca00640 1 -- 172.21.15.110:0/34303593 wait complete. 2024-09-10T10:02:35.618 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-10T10:02:33.693687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258500,"kb_used_data":8984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716220,"statfs":{"total":773094113280,"available":772829409280,"internally_reserved":0,"allocated":9199616,"data_stored":4588565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":255315500},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.173141"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.542215+0000","last_change":"2024-09-10T10:02:31.537543+0000","last_active":"2024-09-10T10:02:31.542215+0000","last_peered":"2024-09-10T10:02:31.542215+0000","last_clean":"2024-09-10T10:02:31.542215+0000","last_became_active":"2024-09-10T10:02:31.536424+0000","last_became_peered":"2024-09-10T10:02:31.536424+0000","last_unstale":"2024-09-10T10:02:31.542215+0000","last_undegraded":"2024-09-10T10:02:31.542215+0000","last_fullsized":"2024-09-10T10:02:31.542215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:23.746871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567932+0000","last_change":"2024-09-10T10:02:31.562606+0000","last_active":"2024-09-10T10:02:31.567932+0000","last_peered":"2024-09-10T10:02:31.567932+0000","last_clean":"2024-09-10T10:02:31.567932+0000","last_became_active":"2024-09-10T10:02:31.562331+0000","last_became_peered":"2024-09-10T10:02:31.562331+0000","last_unstale":"2024-09-10T10:02:31.567932+0000","last_undegraded":"2024-09-10T10:02:31.567932+0000","last_fullsized":"2024-09-10T10:02:31.567932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:03.821483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.553251+0000","last_change":"2024-09-10T10:02:31.548916+0000","last_active":"2024-09-10T10:02:31.553251+0000","last_peered":"2024-09-10T10:02:31.553251+0000","last_clean":"2024-09-10T10:02:31.553251+0000","last_became_active":"2024-09-10T10:02:31.548649+0000","last_became_peered":"2024-09-10T10:02:31.548649+0000","last_unstale":"2024-09-10T10:02:31.553251+0000","last_undegraded":"2024-09-10T10:02:31.553251+0000","last_fullsized":"2024-09-10T10:02:31.553251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:24.701614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552383+0000","last_change":"2024-09-10T10:02:31.546170+0000","last_active":"2024-09-10T10:02:31.552383+0000","last_peered":"2024-09-10T10:02:31.552383+0000","last_clean":"2024-09-10T10:02:31.552383+0000","last_became_active":"2024-09-10T10:02:31.545846+0000","last_became_peered":"2024-09-10T10:02:31.545846+0000","last_unstale":"2024-09-10T10:02:31.552383+0000","last_undegraded":"2024-09-10T10:02:31.552383+0000","last_fullsized":"2024-09-10T10:02:31.552383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:42.109552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552170+0000","last_change":"2024-09-10T10:02:31.548469+0000","last_active":"2024-09-10T10:02:31.552170+0000","last_peered":"2024-09-10T10:02:31.552170+0000","last_clean":"2024-09-10T10:02:31.552170+0000","last_became_active":"2024-09-10T10:02:31.548117+0000","last_became_peered":"2024-09-10T10:02:31.548117+0000","last_unstale":"2024-09-10T10:02:31.552170+0000","last_undegraded":"2024-09-10T10:02:31.552170+0000","last_fullsized":"2024-09-10T10:02:31.552170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:26.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556822+0000","last_change":"2024-09-10T10:02:31.549960+0000","last_active":"2024-09-10T10:02:31.556822+0000","last_peered":"2024-09-10T10:02:31.556822+0000","last_clean":"2024-09-10T10:02:31.556822+0000","last_became_active":"2024-09-10T10:02:31.549428+0000","last_became_peered":"2024-09-10T10:02:31.549428+0000","last_unstale":"2024-09-10T10:02:31.556822+0000","last_undegraded":"2024-09-10T10:02:31.556822+0000","last_fullsized":"2024-09-10T10:02:31.556822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:03.612516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556958+0000","last_change":"2024-09-10T10:02:31.548141+0000","last_active":"2024-09-10T10:02:31.556958+0000","last_peered":"2024-09-10T10:02:31.556958+0000","last_clean":"2024-09-10T10:02:31.556958+0000","last_became_active":"2024-09-10T10:02:31.547882+0000","last_became_peered":"2024-09-10T10:02:31.547882+0000","last_unstale":"2024-09-10T10:02:31.556958+0000","last_undegraded":"2024-09-10T10:02:31.556958+0000","last_fullsized":"2024-09-10T10:02:31.556958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:26.578162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.564513+0000","last_change":"2024-09-10T10:02:31.563038+0000","last_active":"2024-09-10T10:02:31.564513+0000","last_peered":"2024-09-10T10:02:31.564513+0000","last_clean":"2024-09-10T10:02:31.564513+0000","last_became_active":"2024-09-10T10:02:31.562751+0000","last_became_peered":"2024-09-10T10:02:31.562751+0000","last_unstale":"2024-09-10T10:02:31.564513+0000","last_undegraded":"2024-09-10T10:02:31.564513+0000","last_fullsized":"2024-09-10T10:02:31.564513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:23.870830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.562039+0000","last_change":"2024-09-10T10:02:31.559777+0000","last_active":"2024-09-10T10:02:31.562039+0000","last_peered":"2024-09-10T10:02:31.562039+0000","last_clean":"2024-09-10T10:02:31.562039+0000","last_became_active":"2024-09-10T10:02:31.559427+0000","last_became_peered":"2024-09-10T10:02:31.559427+0000","last_unstale":"2024-09-10T10:02:31.562039+0000","last_undegraded":"2024-09-10T10:02:31.562039+0000","last_fullsized":"2024-09-10T10:02:31.562039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:42.752143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.555358+0000","last_change":"2024-09-10T10:02:31.548708+0000","last_active":"2024-09-10T10:02:31.555358+0000","last_peered":"2024-09-10T10:02:31.555358+0000","last_clean":"2024-09-10T10:02:31.555358+0000","last_became_active":"2024-09-10T10:02:31.548429+0000","last_became_peered":"2024-09-10T10:02:31.548429+0000","last_unstale":"2024-09-10T10:02:31.555358+0000","last_undegraded":"2024-09-10T10:02:31.555358+0000","last_fullsized":"2024-09-10T10:02:31.555358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.144518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565730+0000","last_change":"2024-09-10T10:02:31.557439+0000","last_active":"2024-09-10T10:02:31.565730+0000","last_peered":"2024-09-10T10:02:31.565730+0000","last_clean":"2024-09-10T10:02:31.565730+0000","last_became_active":"2024-09-10T10:02:31.557198+0000","last_became_peered":"2024-09-10T10:02:31.557198+0000","last_unstale":"2024-09-10T10:02:31.565730+0000","last_undegraded":"2024-09-10T10:02:31.565730+0000","last_fullsized":"2024-09-10T10:02:31.565730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:50.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.549301+0000","last_change":"2024-09-10T10:02:31.543045+0000","last_active":"2024-09-10T10:02:31.549301+0000","last_peered":"2024-09-10T10:02:31.549301+0000","last_clean":"2024-09-10T10:02:31.549301+0000","last_became_active":"2024-09-10T10:02:31.542754+0000","last_became_peered":"2024-09-10T10:02:31.542754+0000","last_unstale":"2024-09-10T10:02:31.549301+0000","last_undegraded":"2024-09-10T10:02:31.549301+0000","last_fullsized":"2024-09-10T10:02:31.549301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:19.380565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556317+0000","last_change":"2024-09-10T10:02:31.554536+0000","last_active":"2024-09-10T10:02:31.556317+0000","last_peered":"2024-09-10T10:02:31.556317+0000","last_clean":"2024-09-10T10:02:31.556317+0000","last_became_active":"2024-09-10T10:02:31.554214+0000","last_became_peered":"2024-09-10T10:02:31.554214+0000","last_unstale":"2024-09-10T10:02:31.556317+0000","last_undegraded":"2024-09-10T10:02:31.556317+0000","last_fullsized":"2024-09-10T10:02:31.556317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:57.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.561133+0000","last_change":"2024-09-10T10:02:31.552948+0000","last_active":"2024-09-10T10:02:31.561133+0000","last_peered":"2024-09-10T10:02:31.561133+0000","last_clean":"2024-09-10T10:02:31.561133+0000","last_became_active":"2024-09-10T10:02:31.552690+0000","last_became_peered":"2024-09-10T10:02:31.552690+0000","last_unstale":"2024-09-10T10:02:31.561133+0000","last_undegraded":"2024-09-10T10:02:31.561133+0000","last_fullsized":"2024-09-10T10:02:31.561133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:25.567590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.555074+0000","last_change":"2024-09-10T10:02:31.547994+0000","last_active":"2024-09-10T10:02:31.555074+0000","last_peered":"2024-09-10T10:02:31.555074+0000","last_clean":"2024-09-10T10:02:31.555074+0000","last_became_active":"2024-09-10T10:02:31.547498+0000","last_became_peered":"2024-09-10T10:02:31.547498+0000","last_unstale":"2024-09-10T10:02:31.555074+0000","last_undegraded":"2024-09-10T10:02:31.555074+0000","last_fullsized":"2024-09-10T10:02:31.555074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:58.286607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.547356+0000","last_change":"2024-09-10T10:02:31.542408+0000","last_active":"2024-09-10T10:02:31.547356+0000","last_peered":"2024-09-10T10:02:31.547356+0000","last_clean":"2024-09-10T10:02:31.547356+0000","last_became_active":"2024-09-10T10:02:31.542105+0000","last_became_peered":"2024-09-10T10:02:31.542105+0000","last_unstale":"2024-09-10T10:02:31.547356+0000","last_undegraded":"2024-09-10T10:02:31.547356+0000","last_fullsized":"2024-09-10T10:02:31.547356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:42.973169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.551672+0000","last_change":"2024-09-10T10:02:31.548729+0000","last_active":"2024-09-10T10:02:31.551672+0000","last_peered":"2024-09-10T10:02:31.551672+0000","last_clean":"2024-09-10T10:02:31.551672+0000","last_became_active":"2024-09-10T10:02:31.548153+0000","last_became_peered":"2024-09-10T10:02:31.548153+0000","last_unstale":"2024-09-10T10:02:31.551672+0000","last_undegraded":"2024-09-10T10:02:31.551672+0000","last_fullsized":"2024-09-10T10:02:31.551672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:03.612516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552093+0000","last_change":"2024-09-10T10:02:31.545286+0000","last_active":"2024-09-10T10:02:31.552093+0000","last_peered":"2024-09-10T10:02:31.552093+0000","last_clean":"2024-09-10T10:02:31.552093+0000","last_became_active":"2024-09-10T10:02:31.544177+0000","last_became_peered":"2024-09-10T10:02:31.544177+0000","last_unstale":"2024-09-10T10:02:31.552093+0000","last_undegraded":"2024-09-10T10:02:31.552093+0000","last_fullsized":"2024-09-10T10:02:31.552093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:14.997921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.550344+0000","last_change":"2024-09-10T10:02:31.542334+0000","last_active":"2024-09-10T10:02:31.550344+0000","last_peered":"2024-09-10T10:02:31.550344+0000","last_clean":"2024-09-10T10:02:31.550344+0000","last_became_active":"2024-09-10T10:02:31.542062+0000","last_became_peered":"2024-09-10T10:02:31.542062+0000","last_unstale":"2024-09-10T10:02:31.550344+0000","last_undegraded":"2024-09-10T10:02:31.550344+0000","last_fullsized":"2024-09-10T10:02:31.550344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:10.136920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.566777+0000","last_change":"2024-09-10T10:02:31.552513+0000","last_active":"2024-09-10T10:02:31.566777+0000","last_peered":"2024-09-10T10:02:31.566777+0000","last_clean":"2024-09-10T10:02:31.566777+0000","last_became_active":"2024-09-10T10:02:31.552185+0000","last_became_peered":"2024-09-10T10:02:31.552185+0000","last_unstale":"2024-09-10T10:02:31.566777+0000","last_undegraded":"2024-09-10T10:02:31.566777+0000","last_fullsized":"2024-09-10T10:02:31.566777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:04.798551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.564893+0000","last_change":"2024-09-10T10:02:31.553625+0000","last_active":"2024-09-10T10:02:31.564893+0000","last_peered":"2024-09-10T10:02:31.564893+0000","last_clean":"2024-09-10T10:02:31.564893+0000","last_became_active":"2024-09-10T10:02:31.553327+0000","last_became_peered":"2024-09-10T10:02:31.553327+0000","last_unstale":"2024-09-10T10:02:31.564893+0000","last_undegraded":"2024-09-10T10:02:31.564893+0000","last_fullsized":"2024-09-10T10:02:31.564893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.159121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567850+0000","last_change":"2024-09-10T10:02:31.562283+0000","last_active":"2024-09-10T10:02:31.567850+0000","last_peered":"2024-09-10T10:02:31.567850+0000","last_clean":"2024-09-10T10:02:31.567850+0000","last_became_active":"2024-09-10T10:02:31.562020+0000","last_became_peered":"2024-09-10T10:02:31.562020+0000","last_unstale":"2024-09-10T10:02:31.567850+0000","last_undegraded":"2024-09-10T10:02:31.567850+0000","last_fullsized":"2024-09-10T10:02:31.567850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:27.739299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552006+0000","last_change":"2024-09-10T10:02:31.548505+0000","last_active":"2024-09-10T10:02:31.552006+0000","last_peered":"2024-09-10T10:02:31.552006+0000","last_clean":"2024-09-10T10:02:31.552006+0000","last_became_active":"2024-09-10T10:02:31.547909+0000","last_became_peered":"2024-09-10T10:02:31.547909+0000","last_unstale":"2024-09-10T10:02:31.552006+0000","last_undegraded":"2024-09-10T10:02:31.552006+0000","last_fullsized":"2024-09-10T10:02:31.552006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:01.152189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565670+0000","last_change":"2024-09-10T10:02:31.555494+0000","last_active":"2024-09-10T10:02:31.565670+0000","last_peered":"2024-09-10T10:02:31.565670+0000","last_clean":"2024-09-10T10:02:31.565670+0000","last_became_active":"2024-09-10T10:02:31.555240+0000","last_became_peered":"2024-09-10T10:02:31.555240+0000","last_unstale":"2024-09-10T10:02:31.565670+0000","last_undegraded":"2024-09-10T10:02:31.565670+0000","last_fullsized":"2024-09-10T10:02:31.565670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:13.169506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.569937+0000","last_change":"2024-09-10T10:02:31.566896+0000","last_active":"2024-09-10T10:02:31.569937+0000","last_peered":"2024-09-10T10:02:31.569937+0000","last_clean":"2024-09-10T10:02:31.569937+0000","last_became_active":"2024-09-10T10:02:31.564293+0000","last_became_peered":"2024-09-10T10:02:31.564293+0000","last_unstale":"2024-09-10T10:02:31.569937+0000","last_undegraded":"2024-09-10T10:02:31.569937+0000","last_fullsized":"2024-09-10T10:02:31.569937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:15.343407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552648+0000","last_change":"2024-09-10T10:02:31.547509+0000","last_active":"2024-09-10T10:02:31.552648+0000","last_peered":"2024-09-10T10:02:31.552648+0000","last_clean":"2024-09-10T10:02:31.552648+0000","last_became_active":"2024-09-10T10:02:31.545217+0000","last_became_peered":"2024-09-10T10:02:31.545217+0000","last_unstale":"2024-09-10T10:02:31.552648+0000","last_undegraded":"2024-09-10T10:02:31.552648+0000","last_fullsized":"2024-09-10T10:02:31.552648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:12.367366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.550807+0000","last_change":"2024-09-10T10:02:31.543777+0000","last_active":"2024-09-10T10:02:31.550807+0000","last_peered":"2024-09-10T10:02:31.550807+0000","last_clean":"2024-09-10T10:02:31.550807+0000","last_became_active":"2024-09-10T10:02:31.543531+0000","last_became_peered":"2024-09-10T10:02:31.543531+0000","last_unstale":"2024-09-10T10:02:31.550807+0000","last_undegraded":"2024-09-10T10:02:31.550807+0000","last_fullsized":"2024-09-10T10:02:31.550807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:21.481426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.542178+0000","last_change":"2024-09-10T10:02:31.537110+0000","last_active":"2024-09-10T10:02:31.542178+0000","last_peered":"2024-09-10T10:02:31.542178+0000","last_clean":"2024-09-10T10:02:31.542178+0000","last_became_active":"2024-09-10T10:02:31.536793+0000","last_became_peered":"2024-09-10T10:02:31.536793+0000","last_unstale":"2024-09-10T10:02:31.542178+0000","last_undegraded":"2024-09-10T10:02:31.542178+0000","last_fullsized":"2024-09-10T10:02:31.542178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:03.612516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567800+0000","last_change":"2024-09-10T10:02:31.559308+0000","last_active":"2024-09-10T10:02:31.567800+0000","last_peered":"2024-09-10T10:02:31.567800+0000","last_clean":"2024-09-10T10:02:31.567800+0000","last_became_active":"2024-09-10T10:02:31.559032+0000","last_became_peered":"2024-09-10T10:02:31.559032+0000","last_unstale":"2024-09-10T10:02:31.567800+0000","last_undegraded":"2024-09-10T10:02:31.567800+0000","last_fullsized":"2024-09-10T10:02:31.567800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:58.042203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560629+0000","last_change":"2024-09-10T10:02:31.554139+0000","last_active":"2024-09-10T10:02:31.560629+0000","last_peered":"2024-09-10T10:02:31.560629+0000","last_clean":"2024-09-10T10:02:31.560629+0000","last_became_active":"2024-09-10T10:02:31.553623+0000","last_became_peered":"2024-09-10T10:02:31.553623+0000","last_unstale":"2024-09-10T10:02:31.560629+0000","last_undegraded":"2024-09-10T10:02:31.560629+0000","last_fullsized":"2024-09-10T10:02:31.560629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:35.006717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.541471+0000","last_change":"2024-09-10T10:02:31.538144+0000","last_active":"2024-09-10T10:02:31.541471+0000","last_peered":"2024-09-10T10:02:31.541471+0000","last_clean":"2024-09-10T10:02:31.541471+0000","last_became_active":"2024-09-10T10:02:31.537870+0000","last_became_peered":"2024-09-10T10:02:31.537870+0000","last_unstale":"2024-09-10T10:02:31.541471+0000","last_undegraded":"2024-09-10T10:02:31.541471+0000","last_fullsized":"2024-09-10T10:02:31.541471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:19.696540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565350+0000","last_change":"2024-09-10T10:02:31.564283+0000","last_active":"2024-09-10T10:02:31.565350+0000","last_peered":"2024-09-10T10:02:31.565350+0000","last_clean":"2024-09-10T10:02:31.565350+0000","last_became_active":"2024-09-10T10:02:31.564023+0000","last_became_peered":"2024-09-10T10:02:31.564023+0000","last_unstale":"2024-09-10T10:02:31.565350+0000","last_undegraded":"2024-09-10T10:02:31.565350+0000","last_fullsized":"2024-09-10T10:02:31.565350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:12.367366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.547778+0000","last_change":"2024-09-10T10:02:31.540412+0000","last_active":"2024-09-10T10:02:31.547778+0000","last_peered":"2024-09-10T10:02:31.547778+0000","last_clean":"2024-09-10T10:02:31.547778+0000","last_became_active":"2024-09-10T10:02:31.540015+0000","last_became_peered":"2024-09-10T10:02:31.540015+0000","last_unstale":"2024-09-10T10:02:31.547778+0000","last_undegraded":"2024-09-10T10:02:31.547778+0000","last_fullsized":"2024-09-10T10:02:31.547778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:37.019878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568613+0000","last_change":"2024-09-10T10:02:30.546900+0000","last_active":"2024-09-10T10:02:31.568613+0000","last_peered":"2024-09-10T10:02:31.568613+0000","last_clean":"2024-09-10T10:02:31.568613+0000","last_became_active":"2024-09-10T10:02:30.546124+0000","last_became_peered":"2024-09-10T10:02:30.546124+0000","last_unstale":"2024-09-10T10:02:31.568613+0000","last_undegraded":"2024-09-10T10:02:31.568613+0000","last_fullsized":"2024-09-10T10:02:31.568613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:20.477932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.553056+0000","last_change":"2024-09-10T10:02:31.547790+0000","last_active":"2024-09-10T10:02:31.553056+0000","last_peered":"2024-09-10T10:02:31.553056+0000","last_clean":"2024-09-10T10:02:31.553056+0000","last_became_active":"2024-09-10T10:02:31.545182+0000","last_became_peered":"2024-09-10T10:02:31.545182+0000","last_unstale":"2024-09-10T10:02:31.553056+0000","last_undegraded":"2024-09-10T10:02:31.553056+0000","last_fullsized":"2024-09-10T10:02:31.553056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:23.870830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.563722+0000","last_change":"2024-09-10T10:02:30.538747+0000","last_active":"2024-09-10T10:02:31.563722+0000","last_peered":"2024-09-10T10:02:31.563722+0000","last_clean":"2024-09-10T10:02:31.563722+0000","last_became_active":"2024-09-10T10:02:30.538469+0000","last_became_peered":"2024-09-10T10:02:30.538469+0000","last_unstale":"2024-09-10T10:02:31.563722+0000","last_undegraded":"2024-09-10T10:02:31.563722+0000","last_fullsized":"2024-09-10T10:02:31.563722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:27.005499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.564981+0000","last_change":"2024-09-10T10:02:31.563536+0000","last_active":"2024-09-10T10:02:31.564981+0000","last_peered":"2024-09-10T10:02:31.564981+0000","last_clean":"2024-09-10T10:02:31.564981+0000","last_became_active":"2024-09-10T10:02:31.562898+0000","last_became_peered":"2024-09-10T10:02:31.562898+0000","last_unstale":"2024-09-10T10:02:31.564981+0000","last_undegraded":"2024-09-10T10:02:31.564981+0000","last_fullsized":"2024-09-10T10:02:31.564981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:16.389617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568622+0000","last_change":"2024-09-10T10:02:30.543972+0000","last_active":"2024-09-10T10:02:31.568622+0000","last_peered":"2024-09-10T10:02:31.568622+0000","last_clean":"2024-09-10T10:02:31.568622+0000","last_became_active":"2024-09-10T10:02:30.543640+0000","last_became_peered":"2024-09-10T10:02:30.543640+0000","last_unstale":"2024-09-10T10:02:31.568622+0000","last_undegraded":"2024-09-10T10:02:31.568622+0000","last_fullsized":"2024-09-10T10:02:31.568622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:56.190827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.559222+0000","last_change":"2024-09-10T10:02:31.553943+0000","last_active":"2024-09-10T10:02:31.559222+0000","last_peered":"2024-09-10T10:02:31.559222+0000","last_clean":"2024-09-10T10:02:31.559222+0000","last_became_active":"2024-09-10T10:02:31.553672+0000","last_became_peered":"2024-09-10T10:02:31.553672+0000","last_unstale":"2024-09-10T10:02:31.559222+0000","last_undegraded":"2024-09-10T10:02:31.559222+0000","last_fullsized":"2024-09-10T10:02:31.559222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:12.482386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568810+0000","last_change":"2024-09-10T10:02:30.544565+0000","last_active":"2024-09-10T10:02:31.568810+0000","last_peered":"2024-09-10T10:02:31.568810+0000","last_clean":"2024-09-10T10:02:31.568810+0000","last_became_active":"2024-09-10T10:02:30.544309+0000","last_became_peered":"2024-09-10T10:02:30.544309+0000","last_unstale":"2024-09-10T10:02:31.568810+0000","last_undegraded":"2024-09-10T10:02:31.568810+0000","last_fullsized":"2024-09-10T10:02:31.568810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:11.363871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560650+0000","last_change":"2024-09-10T10:02:31.554291+0000","last_active":"2024-09-10T10:02:31.560650+0000","last_peered":"2024-09-10T10:02:31.560650+0000","last_clean":"2024-09-10T10:02:31.560650+0000","last_became_active":"2024-09-10T10:02:31.553965+0000","last_became_peered":"2024-09-10T10:02:31.553965+0000","last_unstale":"2024-09-10T10:02:31.560650+0000","last_undegraded":"2024-09-10T10:02:31.560650+0000","last_fullsized":"2024-09-10T10:02:31.560650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:48.916497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565924+0000","last_change":"2024-09-10T10:02:30.540475+0000","last_active":"2024-09-10T10:02:31.565924+0000","last_peered":"2024-09-10T10:02:31.565924+0000","last_clean":"2024-09-10T10:02:31.565924+0000","last_became_active":"2024-09-10T10:02:30.538091+0000","last_became_peered":"2024-09-10T10:02:30.538091+0000","last_unstale":"2024-09-10T10:02:31.565924+0000","last_undegraded":"2024-09-10T10:02:31.565924+0000","last_fullsized":"2024-09-10T10:02:31.565924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:45.234728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552193+0000","last_change":"2024-09-10T10:02:31.548761+0000","last_active":"2024-09-10T10:02:31.552193+0000","last_peered":"2024-09-10T10:02:31.552193+0000","last_clean":"2024-09-10T10:02:31.552193+0000","last_became_active":"2024-09-10T10:02:31.548463+0000","last_became_peered":"2024-09-10T10:02:31.548463+0000","last_unstale":"2024-09-10T10:02:31.552193+0000","last_undegraded":"2024-09-10T10:02:31.552193+0000","last_fullsized":"2024-09-10T10:02:31.552193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:23.746871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560393+0000","last_change":"2024-09-10T10:02:30.542060+0000","last_active":"2024-09-10T10:02:31.560393+0000","last_peered":"2024-09-10T10:02:31.560393+0000","last_clean":"2024-09-10T10:02:31.560393+0000","last_became_active":"2024-09-10T10:02:30.541590+0000","last_became_peered":"2024-09-10T10:02:30.541590+0000","last_unstale":"2024-09-10T10:02:31.560393+0000","last_undegraded":"2024-09-10T10:02:31.560393+0000","last_fullsized":"2024-09-10T10:02:31.560393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:29.871870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565560+0000","last_change":"2024-09-10T10:02:31.563067+0000","last_active":"2024-09-10T10:02:31.565560+0000","last_peered":"2024-09-10T10:02:31.565560+0000","last_clean":"2024-09-10T10:02:31.565560+0000","last_became_active":"2024-09-10T10:02:31.562681+0000","last_became_peered":"2024-09-10T10:02:31.562681+0000","last_unstale":"2024-09-10T10:02:31.565560+0000","last_undegraded":"2024-09-10T10:02:31.565560+0000","last_fullsized":"2024-09-10T10:02:31.565560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:03.853165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.528182+0000","last_change":"2024-09-10T10:02:30.546298+0000","last_active":"2024-09-10T10:02:31.528182+0000","last_peered":"2024-09-10T10:02:31.528182+0000","last_clean":"2024-09-10T10:02:31.528182+0000","last_became_active":"2024-09-10T10:02:30.544005+0000","last_became_peered":"2024-09-10T10:02:30.544005+0000","last_unstale":"2024-09-10T10:02:31.528182+0000","last_undegraded":"2024-09-10T10:02:31.528182+0000","last_fullsized":"2024-09-10T10:02:31.528182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:50.393087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.554400+0000","last_change":"2024-09-10T10:02:31.547762+0000","last_active":"2024-09-10T10:02:31.554400+0000","last_peered":"2024-09-10T10:02:31.554400+0000","last_clean":"2024-09-10T10:02:31.554400+0000","last_became_active":"2024-09-10T10:02:31.547152+0000","last_became_peered":"2024-09-10T10:02:31.547152+0000","last_unstale":"2024-09-10T10:02:31.554400+0000","last_undegraded":"2024-09-10T10:02:31.554400+0000","last_fullsized":"2024-09-10T10:02:31.554400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:42.338238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.539412+0000","last_change":"2024-09-10T10:02:30.538882+0000","last_active":"2024-09-10T10:02:31.539412+0000","last_peered":"2024-09-10T10:02:31.539412+0000","last_clean":"2024-09-10T10:02:31.539412+0000","last_became_active":"2024-09-10T10:02:30.537793+0000","last_became_peered":"2024-09-10T10:02:30.537793+0000","last_unstale":"2024-09-10T10:02:31.539412+0000","last_undegraded":"2024-09-10T10:02:31.539412+0000","last_fullsized":"2024-09-10T10:02:31.539412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:11.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.559071+0000","last_change":"2024-09-10T10:02:31.552229+0000","last_active":"2024-09-10T10:02:31.559071+0000","last_peered":"2024-09-10T10:02:31.559071+0000","last_clean":"2024-09-10T10:02:31.559071+0000","last_became_active":"2024-09-10T10:02:31.551983+0000","last_became_peered":"2024-09-10T10:02:31.551983+0000","last_unstale":"2024-09-10T10:02:31.559071+0000","last_undegraded":"2024-09-10T10:02:31.559071+0000","last_fullsized":"2024-09-10T10:02:31.559071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:49.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534966+0000","last_change":"2024-09-10T10:02:30.546608+0000","last_active":"2024-09-10T10:02:31.534966+0000","last_peered":"2024-09-10T10:02:31.534966+0000","last_clean":"2024-09-10T10:02:31.534966+0000","last_became_active":"2024-09-10T10:02:30.545793+0000","last_became_peered":"2024-09-10T10:02:30.545793+0000","last_unstale":"2024-09-10T10:02:31.534966+0000","last_undegraded":"2024-09-10T10:02:31.534966+0000","last_fullsized":"2024-09-10T10:02:31.534966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:04.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.557730+0000","last_change":"2024-09-10T10:02:31.555156+0000","last_active":"2024-09-10T10:02:31.557730+0000","last_peered":"2024-09-10T10:02:31.557730+0000","last_clean":"2024-09-10T10:02:31.557730+0000","last_became_active":"2024-09-10T10:02:31.554830+0000","last_became_peered":"2024-09-10T10:02:31.554830+0000","last_unstale":"2024-09-10T10:02:31.557730+0000","last_undegraded":"2024-09-10T10:02:31.557730+0000","last_fullsized":"2024-09-10T10:02:31.557730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:19.380565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534582+0000","last_change":"2024-09-10T10:02:30.544730+0000","last_active":"2024-09-10T10:02:31.534582+0000","last_peered":"2024-09-10T10:02:31.534582+0000","last_clean":"2024-09-10T10:02:31.534582+0000","last_became_active":"2024-09-10T10:02:30.543586+0000","last_became_peered":"2024-09-10T10:02:30.543586+0000","last_unstale":"2024-09-10T10:02:31.534582+0000","last_undegraded":"2024-09-10T10:02:31.534582+0000","last_fullsized":"2024-09-10T10:02:31.534582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:41.969675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.550403+0000","last_change":"2024-09-10T10:02:31.544907+0000","last_active":"2024-09-10T10:02:31.550403+0000","last_peered":"2024-09-10T10:02:31.550403+0000","last_clean":"2024-09-10T10:02:31.550403+0000","last_became_active":"2024-09-10T10:02:31.543522+0000","last_became_peered":"2024-09-10T10:02:31.543522+0000","last_unstale":"2024-09-10T10:02:31.550403+0000","last_undegraded":"2024-09-10T10:02:31.550403+0000","last_fullsized":"2024-09-10T10:02:31.550403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:35.525434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568916+0000","last_change":"2024-09-10T10:02:30.545027+0000","last_active":"2024-09-10T10:02:31.568916+0000","last_peered":"2024-09-10T10:02:31.568916+0000","last_clean":"2024-09-10T10:02:31.568916+0000","last_became_active":"2024-09-10T10:02:30.544757+0000","last_became_peered":"2024-09-10T10:02:30.544757+0000","last_unstale":"2024-09-10T10:02:31.568916+0000","last_undegraded":"2024-09-10T10:02:31.568916+0000","last_fullsized":"2024-09-10T10:02:31.568916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:45.234728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.555452+0000","last_change":"2024-09-10T10:02:31.551445+0000","last_active":"2024-09-10T10:02:31.555452+0000","last_peered":"2024-09-10T10:02:31.555452+0000","last_clean":"2024-09-10T10:02:31.555452+0000","last_became_active":"2024-09-10T10:02:31.551027+0000","last_became_peered":"2024-09-10T10:02:31.551027+0000","last_unstale":"2024-09-10T10:02:31.555452+0000","last_undegraded":"2024-09-10T10:02:31.555452+0000","last_fullsized":"2024-09-10T10:02:31.555452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:42.752143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.570178+0000","last_change":"2024-09-10T10:02:30.548612+0000","last_active":"2024-09-10T10:02:31.570178+0000","last_peered":"2024-09-10T10:02:31.570178+0000","last_clean":"2024-09-10T10:02:31.570178+0000","last_became_active":"2024-09-10T10:02:30.548322+0000","last_became_peered":"2024-09-10T10:02:30.548322+0000","last_unstale":"2024-09-10T10:02:31.570178+0000","last_undegraded":"2024-09-10T10:02:31.570178+0000","last_fullsized":"2024-09-10T10:02:31.570178+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:20.477932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.541792+0000","last_change":"2024-09-10T10:02:31.536609+0000","last_active":"2024-09-10T10:02:31.541792+0000","last_peered":"2024-09-10T10:02:31.541792+0000","last_clean":"2024-09-10T10:02:31.541792+0000","last_became_active":"2024-09-10T10:02:31.536334+0000","last_became_peered":"2024-09-10T10:02:31.536334+0000","last_unstale":"2024-09-10T10:02:31.541792+0000","last_undegraded":"2024-09-10T10:02:31.541792+0000","last_fullsized":"2024-09-10T10:02:31.541792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:24.316984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567592+0000","last_change":"2024-09-10T10:02:30.540161+0000","last_active":"2024-09-10T10:02:31.567592+0000","last_peered":"2024-09-10T10:02:31.567592+0000","last_clean":"2024-09-10T10:02:31.567592+0000","last_became_active":"2024-09-10T10:02:30.539757+0000","last_became_peered":"2024-09-10T10:02:30.539757+0000","last_unstale":"2024-09-10T10:02:31.567592+0000","last_undegraded":"2024-09-10T10:02:31.567592+0000","last_fullsized":"2024-09-10T10:02:31.567592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:48.037362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.561170+0000","last_change":"2024-09-10T10:02:31.560329+0000","last_active":"2024-09-10T10:02:31.561170+0000","last_peered":"2024-09-10T10:02:31.561170+0000","last_clean":"2024-09-10T10:02:31.561170+0000","last_became_active":"2024-09-10T10:02:31.560059+0000","last_became_peered":"2024-09-10T10:02:31.560059+0000","last_unstale":"2024-09-10T10:02:31.561170+0000","last_undegraded":"2024-09-10T10:02:31.561170+0000","last_fullsized":"2024-09-10T10:02:31.561170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:35.525434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.533852+0000","last_change":"2024-09-10T10:02:30.544437+0000","last_active":"2024-09-10T10:02:31.533852+0000","last_peered":"2024-09-10T10:02:31.533852+0000","last_clean":"2024-09-10T10:02:31.533852+0000","last_became_active":"2024-09-10T10:02:30.543094+0000","last_became_peered":"2024-09-10T10:02:30.543094+0000","last_unstale":"2024-09-10T10:02:31.533852+0000","last_undegraded":"2024-09-10T10:02:31.533852+0000","last_fullsized":"2024-09-10T10:02:31.533852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:27.052155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.557977+0000","last_change":"2024-09-10T10:02:31.549747+0000","last_active":"2024-09-10T10:02:31.557977+0000","last_peered":"2024-09-10T10:02:31.557977+0000","last_clean":"2024-09-10T10:02:31.557977+0000","last_became_active":"2024-09-10T10:02:31.549471+0000","last_became_peered":"2024-09-10T10:02:31.549471+0000","last_unstale":"2024-09-10T10:02:31.557977+0000","last_undegraded":"2024-09-10T10:02:31.557977+0000","last_fullsized":"2024-09-10T10:02:31.557977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:01.886252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.540543+0000","last_change":"2024-09-10T10:02:30.548951+0000","last_active":"2024-09-10T10:02:31.540543+0000","last_peered":"2024-09-10T10:02:31.540543+0000","last_clean":"2024-09-10T10:02:31.540543+0000","last_became_active":"2024-09-10T10:02:30.548655+0000","last_became_peered":"2024-09-10T10:02:30.548655+0000","last_unstale":"2024-09-10T10:02:31.540543+0000","last_undegraded":"2024-09-10T10:02:31.540543+0000","last_fullsized":"2024-09-10T10:02:31.540543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:04.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567736+0000","last_change":"2024-09-10T10:02:31.561014+0000","last_active":"2024-09-10T10:02:31.567736+0000","last_peered":"2024-09-10T10:02:31.567736+0000","last_clean":"2024-09-10T10:02:31.567736+0000","last_became_active":"2024-09-10T10:02:31.560756+0000","last_became_peered":"2024-09-10T10:02:31.560756+0000","last_unstale":"2024-09-10T10:02:31.567736+0000","last_undegraded":"2024-09-10T10:02:31.567736+0000","last_fullsized":"2024-09-10T10:02:31.567736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:45.753900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567056+0000","last_change":"2024-09-10T10:02:30.549278+0000","last_active":"2024-09-10T10:02:31.567056+0000","last_peered":"2024-09-10T10:02:31.567056+0000","last_clean":"2024-09-10T10:02:31.567056+0000","last_became_active":"2024-09-10T10:02:30.548996+0000","last_became_peered":"2024-09-10T10:02:30.548996+0000","last_unstale":"2024-09-10T10:02:31.567056+0000","last_undegraded":"2024-09-10T10:02:31.567056+0000","last_fullsized":"2024-09-10T10:02:31.567056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.563160+0000","last_change":"2024-09-10T10:02:31.558223+0000","last_active":"2024-09-10T10:02:31.563160+0000","last_peered":"2024-09-10T10:02:31.563160+0000","last_clean":"2024-09-10T10:02:31.563160+0000","last_became_active":"2024-09-10T10:02:31.557905+0000","last_became_peered":"2024-09-10T10:02:31.557905+0000","last_unstale":"2024-09-10T10:02:31.563160+0000","last_undegraded":"2024-09-10T10:02:31.563160+0000","last_fullsized":"2024-09-10T10:02:31.563160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:26.502768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560090+0000","last_change":"2024-09-10T10:02:30.546372+0000","last_active":"2024-09-10T10:02:31.560090+0000","last_peered":"2024-09-10T10:02:31.560090+0000","last_clean":"2024-09-10T10:02:31.560090+0000","last_became_active":"2024-09-10T10:02:30.545491+0000","last_became_peered":"2024-09-10T10:02:30.545491+0000","last_unstale":"2024-09-10T10:02:31.560090+0000","last_undegraded":"2024-09-10T10:02:31.560090+0000","last_fullsized":"2024-09-10T10:02:31.560090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:25.529504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556759+0000","last_change":"2024-09-10T10:02:31.553625+0000","last_active":"2024-09-10T10:02:31.556759+0000","last_peered":"2024-09-10T10:02:31.556759+0000","last_clean":"2024-09-10T10:02:31.556759+0000","last_became_active":"2024-09-10T10:02:31.553288+0000","last_became_peered":"2024-09-10T10:02:31.553288+0000","last_unstale":"2024-09-10T10:02:31.556759+0000","last_undegraded":"2024-09-10T10:02:31.556759+0000","last_fullsized":"2024-09-10T10:02:31.556759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:51.396582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565999+0000","last_change":"2024-09-10T10:02:30.543434+0000","last_active":"2024-09-10T10:02:31.565999+0000","last_peered":"2024-09-10T10:02:31.565999+0000","last_clean":"2024-09-10T10:02:31.565999+0000","last_became_active":"2024-09-10T10:02:30.542135+0000","last_became_peered":"2024-09-10T10:02:30.542135+0000","last_unstale":"2024-09-10T10:02:31.565999+0000","last_undegraded":"2024-09-10T10:02:31.565999+0000","last_fullsized":"2024-09-10T10:02:31.565999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:11.478892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.555475+0000","last_change":"2024-09-10T10:02:31.552270+0000","last_active":"2024-09-10T10:02:31.555475+0000","last_peered":"2024-09-10T10:02:31.555475+0000","last_clean":"2024-09-10T10:02:31.555475+0000","last_became_active":"2024-09-10T10:02:31.551944+0000","last_became_peered":"2024-09-10T10:02:31.551944+0000","last_unstale":"2024-09-10T10:02:31.555475+0000","last_undegraded":"2024-09-10T10:02:31.555475+0000","last_fullsized":"2024-09-10T10:02:31.555475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:55.116672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.528810+0000","last_change":"2024-09-10T10:02:30.544170+0000","last_active":"2024-09-10T10:02:31.528810+0000","last_peered":"2024-09-10T10:02:31.528810+0000","last_clean":"2024-09-10T10:02:31.528810+0000","last_became_active":"2024-09-10T10:02:30.543876+0000","last_became_peered":"2024-09-10T10:02:30.543876+0000","last_unstale":"2024-09-10T10:02:31.528810+0000","last_undegraded":"2024-09-10T10:02:31.528810+0000","last_fullsized":"2024-09-10T10:02:31.528810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:23.698119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565897+0000","last_change":"2024-09-10T10:02:31.556818+0000","last_active":"2024-09-10T10:02:31.565897+0000","last_peered":"2024-09-10T10:02:31.565897+0000","last_clean":"2024-09-10T10:02:31.565897+0000","last_became_active":"2024-09-10T10:02:31.556553+0000","last_became_peered":"2024-09-10T10:02:31.556553+0000","last_unstale":"2024-09-10T10:02:31.565897+0000","last_undegraded":"2024-09-10T10:02:31.565897+0000","last_fullsized":"2024-09-10T10:02:31.565897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:39.978614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.569094+0000","last_change":"2024-09-10T10:02:30.544084+0000","last_active":"2024-09-10T10:02:31.569094+0000","last_peered":"2024-09-10T10:02:31.569094+0000","last_clean":"2024-09-10T10:02:31.569094+0000","last_became_active":"2024-09-10T10:02:30.543804+0000","last_became_peered":"2024-09-10T10:02:30.543804+0000","last_unstale":"2024-09-10T10:02:31.569094+0000","last_undegraded":"2024-09-10T10:02:31.569094+0000","last_fullsized":"2024-09-10T10:02:31.569094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:21.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556942+0000","last_change":"2024-09-10T10:02:31.550986+0000","last_active":"2024-09-10T10:02:31.556942+0000","last_peered":"2024-09-10T10:02:31.556942+0000","last_clean":"2024-09-10T10:02:31.556942+0000","last_became_active":"2024-09-10T10:02:31.550664+0000","last_became_peered":"2024-09-10T10:02:31.550664+0000","last_unstale":"2024-09-10T10:02:31.556942+0000","last_undegraded":"2024-09-10T10:02:31.556942+0000","last_fullsized":"2024-09-10T10:02:31.556942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:23.746871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.542864+0000","last_change":"2024-09-10T10:02:30.540953+0000","last_active":"2024-09-10T10:02:31.542864+0000","last_peered":"2024-09-10T10:02:31.542864+0000","last_clean":"2024-09-10T10:02:31.542864+0000","last_became_active":"2024-09-10T10:02:30.540677+0000","last_became_peered":"2024-09-10T10:02:30.540677+0000","last_unstale":"2024-09-10T10:02:31.542864+0000","last_undegraded":"2024-09-10T10:02:31.542864+0000","last_fullsized":"2024-09-10T10:02:31.542864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:40:31.074701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.545209+0000","last_change":"2024-09-10T10:02:31.542445+0000","last_active":"2024-09-10T10:02:31.545209+0000","last_peered":"2024-09-10T10:02:31.545209+0000","last_clean":"2024-09-10T10:02:31.545209+0000","last_became_active":"2024-09-10T10:02:31.542182+0000","last_became_peered":"2024-09-10T10:02:31.542182+0000","last_unstale":"2024-09-10T10:02:31.545209+0000","last_undegraded":"2024-09-10T10:02:31.545209+0000","last_fullsized":"2024-09-10T10:02:31.545209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:56.826789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.541703+0000","last_change":"2024-09-10T10:02:30.545974+0000","last_active":"2024-09-10T10:02:31.541703+0000","last_peered":"2024-09-10T10:02:31.541703+0000","last_clean":"2024-09-10T10:02:31.541703+0000","last_became_active":"2024-09-10T10:02:30.545694+0000","last_became_peered":"2024-09-10T10:02:30.545694+0000","last_unstale":"2024-09-10T10:02:31.541703+0000","last_undegraded":"2024-09-10T10:02:31.541703+0000","last_fullsized":"2024-09-10T10:02:31.541703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:17.117085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.564032+0000","last_change":"2024-09-10T10:02:31.560142+0000","last_active":"2024-09-10T10:02:31.564032+0000","last_peered":"2024-09-10T10:02:31.564032+0000","last_clean":"2024-09-10T10:02:31.564032+0000","last_became_active":"2024-09-10T10:02:31.559822+0000","last_became_peered":"2024-09-10T10:02:31.559822+0000","last_unstale":"2024-09-10T10:02:31.564032+0000","last_undegraded":"2024-09-10T10:02:31.564032+0000","last_fullsized":"2024-09-10T10:02:31.564032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:55.116672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534091+0000","last_change":"2024-09-10T10:02:30.541474+0000","last_active":"2024-09-10T10:02:31.534091+0000","last_peered":"2024-09-10T10:02:31.534091+0000","last_clean":"2024-09-10T10:02:31.534091+0000","last_became_active":"2024-09-10T10:02:30.540189+0000","last_became_peered":"2024-09-10T10:02:30.540189+0000","last_unstale":"2024-09-10T10:02:31.534091+0000","last_undegraded":"2024-09-10T10:02:31.534091+0000","last_fullsized":"2024-09-10T10:02:31.534091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:30.469708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.551722+0000","last_change":"2024-09-10T10:02:31.544741+0000","last_active":"2024-09-10T10:02:31.551722+0000","last_peered":"2024-09-10T10:02:31.551722+0000","last_clean":"2024-09-10T10:02:31.551722+0000","last_became_active":"2024-09-10T10:02:31.544502+0000","last_became_peered":"2024-09-10T10:02:31.544502+0000","last_unstale":"2024-09-10T10:02:31.551722+0000","last_undegraded":"2024-09-10T10:02:31.551722+0000","last_fullsized":"2024-09-10T10:02:31.551722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:05.222466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567419+0000","last_change":"2024-09-10T10:02:30.548421+0000","last_active":"2024-09-10T10:02:31.567419+0000","last_peered":"2024-09-10T10:02:31.567419+0000","last_clean":"2024-09-10T10:02:31.567419+0000","last_became_active":"2024-09-10T10:02:30.548072+0000","last_became_peered":"2024-09-10T10:02:30.548072+0000","last_unstale":"2024-09-10T10:02:31.567419+0000","last_undegraded":"2024-09-10T10:02:31.567419+0000","last_fullsized":"2024-09-10T10:02:31.567419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:08.652564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560745+0000","last_change":"2024-09-10T10:02:31.557943+0000","last_active":"2024-09-10T10:02:31.560745+0000","last_peered":"2024-09-10T10:02:31.560745+0000","last_clean":"2024-09-10T10:02:31.560745+0000","last_became_active":"2024-09-10T10:02:31.557626+0000","last_became_peered":"2024-09-10T10:02:31.557626+0000","last_unstale":"2024-09-10T10:02:31.560745+0000","last_undegraded":"2024-09-10T10:02:31.560745+0000","last_fullsized":"2024-09-10T10:02:31.560745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:10.136920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.544719+0000","last_change":"2024-09-10T10:02:30.542999+0000","last_active":"2024-09-10T10:02:31.544719+0000","last_peered":"2024-09-10T10:02:31.544719+0000","last_clean":"2024-09-10T10:02:31.544719+0000","last_became_active":"2024-09-10T10:02:30.542683+0000","last_became_peered":"2024-09-10T10:02:30.542683+0000","last_unstale":"2024-09-10T10:02:31.544719+0000","last_undegraded":"2024-09-10T10:02:31.544719+0000","last_fullsized":"2024-09-10T10:02:31.544719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:22.360524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.550839+0000","last_change":"2024-09-10T10:02:31.546740+0000","last_active":"2024-09-10T10:02:31.550839+0000","last_peered":"2024-09-10T10:02:31.550839+0000","last_clean":"2024-09-10T10:02:31.550839+0000","last_became_active":"2024-09-10T10:02:31.543386+0000","last_became_peered":"2024-09-10T10:02:31.543386+0000","last_unstale":"2024-09-10T10:02:31.550839+0000","last_undegraded":"2024-09-10T10:02:31.550839+0000","last_fullsized":"2024-09-10T10:02:31.550839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:55.116672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534398+0000","last_change":"2024-09-10T10:02:30.544353+0000","last_active":"2024-09-10T10:02:31.534398+0000","last_peered":"2024-09-10T10:02:31.534398+0000","last_clean":"2024-09-10T10:02:31.534398+0000","last_became_active":"2024-09-10T10:02:30.544032+0000","last_became_peered":"2024-09-10T10:02:30.544032+0000","last_unstale":"2024-09-10T10:02:31.534398+0000","last_undegraded":"2024-09-10T10:02:31.534398+0000","last_fullsized":"2024-09-10T10:02:31.534398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:18.693045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552946+0000","last_change":"2024-09-10T10:02:31.551236+0000","last_active":"2024-09-10T10:02:31.552946+0000","last_peered":"2024-09-10T10:02:31.552946+0000","last_clean":"2024-09-10T10:02:31.552946+0000","last_became_active":"2024-09-10T10:02:31.550614+0000","last_became_peered":"2024-09-10T10:02:31.550614+0000","last_unstale":"2024-09-10T10:02:31.552946+0000","last_undegraded":"2024-09-10T10:02:31.552946+0000","last_fullsized":"2024-09-10T10:02:31.552946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:19.380565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534562+0000","last_change":"2024-09-10T10:02:30.544644+0000","last_active":"2024-09-10T10:02:31.534562+0000","last_peered":"2024-09-10T10:02:31.534562+0000","last_clean":"2024-09-10T10:02:31.534562+0000","last_became_active":"2024-09-10T10:02:30.544332+0000","last_became_peered":"2024-09-10T10:02:30.544332+0000","last_unstale":"2024-09-10T10:02:31.534562+0000","last_undegraded":"2024-09-10T10:02:31.534562+0000","last_fullsized":"2024-09-10T10:02:31.534562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:36.016383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556116+0000","last_change":"2024-09-10T10:02:31.549701+0000","last_active":"2024-09-10T10:02:31.556116+0000","last_peered":"2024-09-10T10:02:31.556116+0000","last_clean":"2024-09-10T10:02:31.556116+0000","last_became_active":"2024-09-10T10:02:31.548944+0000","last_became_peered":"2024-09-10T10:02:31.548944+0000","last_unstale":"2024-09-10T10:02:31.556116+0000","last_undegraded":"2024-09-10T10:02:31.556116+0000","last_fullsized":"2024-09-10T10:02:31.556116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:26.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.528559+0000","last_change":"2024-09-10T10:02:30.549283+0000","last_active":"2024-09-10T10:02:31.528559+0000","last_peered":"2024-09-10T10:02:31.528559+0000","last_clean":"2024-09-10T10:02:31.528559+0000","last_became_active":"2024-09-10T10:02:30.549016+0000","last_became_peered":"2024-09-10T10:02:30.549016+0000","last_unstale":"2024-09-10T10:02:31.528559+0000","last_undegraded":"2024-09-10T10:02:31.528559+0000","last_fullsized":"2024-09-10T10:02:31.528559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:15.386122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.551302+0000","last_change":"2024-09-10T10:02:31.542526+0000","last_active":"2024-09-10T10:02:31.551302+0000","last_peered":"2024-09-10T10:02:31.551302+0000","last_clean":"2024-09-10T10:02:31.551302+0000","last_became_active":"2024-09-10T10:02:31.538982+0000","last_became_peered":"2024-09-10T10:02:31.538982+0000","last_unstale":"2024-09-10T10:02:31.551302+0000","last_undegraded":"2024-09-10T10:02:31.551302+0000","last_fullsized":"2024-09-10T10:02:31.551302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:26.502768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.533709+0000","last_change":"2024-09-10T10:02:30.537103+0000","last_active":"2024-09-10T10:02:31.533709+0000","last_peered":"2024-09-10T10:02:31.533709+0000","last_clean":"2024-09-10T10:02:31.533709+0000","last_became_active":"2024-09-10T10:02:30.536832+0000","last_became_peered":"2024-09-10T10:02:30.536832+0000","last_unstale":"2024-09-10T10:02:31.533709+0000","last_undegraded":"2024-09-10T10:02:31.533709+0000","last_fullsized":"2024-09-10T10:02:31.533709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:15.756536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.561035+0000","last_change":"2024-09-10T10:02:31.557294+0000","last_active":"2024-09-10T10:02:31.561035+0000","last_peered":"2024-09-10T10:02:31.561035+0000","last_clean":"2024-09-10T10:02:31.561035+0000","last_became_active":"2024-09-10T10:02:31.556994+0000","last_became_peered":"2024-09-10T10:02:31.556994+0000","last_unstale":"2024-09-10T10:02:31.561035+0000","last_undegraded":"2024-09-10T10:02:31.561035+0000","last_fullsized":"2024-09-10T10:02:31.561035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:24.316984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.533972+0000","last_change":"2024-09-10T10:02:30.536797+0000","last_active":"2024-09-10T10:02:31.533972+0000","last_peered":"2024-09-10T10:02:31.533972+0000","last_clean":"2024-09-10T10:02:31.533972+0000","last_became_active":"2024-09-10T10:02:30.536520+0000","last_became_peered":"2024-09-10T10:02:30.536520+0000","last_unstale":"2024-09-10T10:02:31.533972+0000","last_undegraded":"2024-09-10T10:02:31.533972+0000","last_fullsized":"2024-09-10T10:02:31.533972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:27.005499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.551962+0000","last_change":"2024-09-10T10:02:31.544360+0000","last_active":"2024-09-10T10:02:31.551962+0000","last_peered":"2024-09-10T10:02:31.551962+0000","last_clean":"2024-09-10T10:02:31.551962+0000","last_became_active":"2024-09-10T10:02:31.543605+0000","last_became_peered":"2024-09-10T10:02:31.543605+0000","last_unstale":"2024-09-10T10:02:31.551962+0000","last_undegraded":"2024-09-10T10:02:31.551962+0000","last_fullsized":"2024-09-10T10:02:31.551962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:42.752143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.532592+0000","last_change":"2024-09-10T10:02:30.547126+0000","last_active":"2024-09-10T10:02:31.532592+0000","last_peered":"2024-09-10T10:02:31.532592+0000","last_clean":"2024-09-10T10:02:31.532592+0000","last_became_active":"2024-09-10T10:02:30.543724+0000","last_became_peered":"2024-09-10T10:02:30.543724+0000","last_unstale":"2024-09-10T10:02:31.532592+0000","last_undegraded":"2024-09-10T10:02:31.532592+0000","last_fullsized":"2024-09-10T10:02:31.532592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:35.772565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568051+0000","last_change":"2024-09-10T10:02:31.563246+0000","last_active":"2024-09-10T10:02:31.568051+0000","last_peered":"2024-09-10T10:02:31.568051+0000","last_clean":"2024-09-10T10:02:31.568051+0000","last_became_active":"2024-09-10T10:02:31.562834+0000","last_became_peered":"2024-09-10T10:02:31.562834+0000","last_unstale":"2024-09-10T10:02:31.568051+0000","last_undegraded":"2024-09-10T10:02:31.568051+0000","last_fullsized":"2024-09-10T10:02:31.568051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:26.609221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568386+0000","last_change":"2024-09-10T10:02:30.549482+0000","last_active":"2024-09-10T10:02:31.568386+0000","last_peered":"2024-09-10T10:02:31.568386+0000","last_clean":"2024-09-10T10:02:31.568386+0000","last_became_active":"2024-09-10T10:02:30.549213+0000","last_became_peered":"2024-09-10T10:02:30.549213+0000","last_unstale":"2024-09-10T10:02:31.568386+0000","last_undegraded":"2024-09-10T10:02:31.568386+0000","last_fullsized":"2024-09-10T10:02:31.568386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:15.386122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.539427+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:02:31.539427+0000","last_peered":"2024-09-10T10:02:31.539427+0000","last_clean":"2024-09-10T10:02:31.539427+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:02:31.539427+0000","last_undegraded":"2024-09-10T10:02:31.539427+0000","last_fullsized":"2024-09-10T10:02:31.539427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1609728,"data_stored":1022336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542200,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1593344,"data_stored":1015491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:35.622 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:02:35.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.796+0000 7fcdf4600640 1 Processor -- start 2024-09-10T10:02:35.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.796+0000 7fcdf4600640 1 -- start start 2024-09-10T10:02:35.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf4600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcdec06bc50 0x7fcdec06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf4600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdec06c710 con 0x7fcdec0ae840 2024-09-10T10:02:35.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf4600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdec0b4680 con 0x7fcdec15e690 2024-09-10T10:02:35.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf4600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcdec0b4960 con 0x7fcdec06bc50 2024-09-10T10:02:35.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf2600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcdec0ae840 0x7fcdec0b5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52410/0 (socket says 172.21.15.110:52410) 2024-09-10T10:02:35.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf2600640 1 -- 172.21.15.110:0/1189008177 learned_addr learned my addr 172.21.15.110:0/1189008177 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1200640 1 --2- 172.21.15.110:0/1189008177 >> v2:172.21.15.145:3300/0 conn(0x7fcdec06bc50 0x7fcdec06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933380349 0 0) 0x7fcdec06c710 con 0x7fcdec0ae840 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdd4003530 con 0x7fcdec0ae840 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2815390514 0 0) 0x7fcdec0b4680 con 0x7fcdec15e690 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdec06c710 con 0x7fcdec15e690 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2000931664 0 0) 0x7fcdd4003530 con 0x7fcdec0ae840 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 >> v2:172.21.15.145:3300/0 conn(0x7fcdec06bc50 msgr2=0x7fcdec06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 --2- 172.21.15.110:0/1189008177 >> v2:172.21.15.145:3300/0 conn(0x7fcdec06bc50 0x7fcdec06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 >> v1:172.21.15.145:6789/0 conn(0x7fcdec15e690 legacy=0x7fcdec06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec0b4c40 con 0x7fcdec0ae840 2024-09-10T10:02:35.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fcde4001680 con 0x7fcdec0ae840 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 >> v1:172.21.15.110:6789/0 conn(0x7fcdec0ae840 legacy=0x7fcdec0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdd4003b50 0x7fcdd4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd4004810 0x7fcdd4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd40051d0 0x7fcdd400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec0b4960 con 0x7fcdd4003b50 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd400c050 con 0x7fcdd4004810 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd400c250 con 0x7fcdd40051d0 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1c00640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdd4003b50 0x7fcdd4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1200640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd4004810 0x7fcdd4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf2600640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd40051d0 0x7fcdd400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1c00640 1 -- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd40051d0 msgr2=0x7fcdd400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1c00640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd40051d0 0x7fcdd400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1c00640 1 -- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd4004810 msgr2=0x7fcdd4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:35.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1c00640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd4004810 0x7fcdd4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1c00640 1 -- 172.21.15.110:0/1189008177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd4003530 con 0x7fcdd4003b50 2024-09-10T10:02:35.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf1200640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd4004810 0x7fcdd4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:35.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf2600640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd40051d0 0x7fcdd400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:35.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.800+0000 7fcdf2600640 1 -- 172.21.15.110:0/1189008177 reap_dead start 2024-09-10T10:02:35.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf1c00640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdd4003b50 0x7fcdd4003f50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001fd0 tx=0x7fcde001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfeb45fe1f1ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0001c20 con 0x7fcdd4003b50 2024-09-10T10:02:35.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde0003ba0 con 0x7fcdd4003b50 2024-09-10T10:02:35.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdebe00640 1 -- 172.21.15.110:0/1189008177 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00050d0 con 0x7fcdd4003b50 2024-09-10T10:02:35.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 -- 172.21.15.110:0/1189008177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdd4003b50 msgr2=0x7fcdd4003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdd4003b50 0x7fcdd4003f50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001fd0 tx=0x7fcde001c8b0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 -- 172.21.15.110:0/1189008177 shutdown_connections 2024-09-10T10:02:35.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 --2- 172.21.15.110:0/1189008177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdd4003b50 0x7fcdd4003f50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 -- 172.21.15.110:0/1189008177 >> 172.21.15.110:0/1189008177 conn(0x7fcdec0d1530 msgr2=0x7fcdec0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 -- 172.21.15.110:0/1189008177 shutdown_connections 2024-09-10T10:02:35.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 -- 172.21.15.110:0/1189008177 wait complete. 2024-09-10T10:02:35.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 Processor -- start 2024-09-10T10:02:35.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.804+0000 7fcdf4600640 1 -- start start 2024-09-10T10:02:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd4003b50 0x7fcdec15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdec1fb3d0 0x7fcdec160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec1ec280 0x7fcdec1fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06e7d0 con 0x7fcdec1ec280 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06e650 con 0x7fcdec1fb3d0 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec15ea70 con 0x7fcdd4003b50 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec1ec280 0x7fcdec1fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec1ec280 0x7fcdec1fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57230/0 (socket says 172.21.15.110:57230) 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 -- 172.21.15.110:0/2551215487 learned_addr learned my addr 172.21.15.110:0/2551215487 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf1200640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdec1fb3d0 0x7fcdec160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf1c00640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd4003b50 0x7fcdec15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 -- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd4003b50 msgr2=0x7fcdec15ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd4003b50 0x7fcdec15ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 -- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdec1fb3d0 msgr2=0x7fcdec160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdec1fb3d0 0x7fcdec160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 -- 172.21.15.110:0/2551215487 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec161c60 con 0x7fcdec1ec280 2024-09-10T10:02:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf2600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec1ec280 0x7fcdec1fb920 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcde4000fc0 tx=0x7fcde400d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcde9600640 1 -- 172.21.15.110:0/2551215487 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4007440 con 0x7fcdec1ec280 2024-09-10T10:02:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcde9600640 1 -- 172.21.15.110:0/2551215487 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde40075e0 con 0x7fcdec1ec280 2024-09-10T10:02:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec1613c0 con 0x7fcdec1ec280 2024-09-10T10:02:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcde9600640 1 -- 172.21.15.110:0/2551215487 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4004a10 con 0x7fcdec1ec280 2024-09-10T10:02:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec161980 con 0x7fcdec1ec280 2024-09-10T10:02:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.808+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec0b5c50 con 0x7fcdec1ec280 2024-09-10T10:02:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.812+0000 7fcde9600640 1 -- 172.21.15.110:0/2551215487 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fcde4004bb0 con 0x7fcdec1ec280 2024-09-10T10:02:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.812+0000 7fcde9600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcdcc0679e0 0x7fcdcc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.812+0000 7fcde9600640 1 -- 172.21.15.110:0/2551215487 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fcde4085760 con 0x7fcdec1ec280 2024-09-10T10:02:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.812+0000 7fcdf1c00640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcdcc0679e0 0x7fcdcc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.812+0000 7fcdf1c00640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcdcc0679e0 0x7fcdcc069ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcde00ac000 tx=0x7fcde00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.816+0000 7fcde9600640 1 -- 172.21.15.110:0/2551215487 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcde401a060 con 0x7fcdec1ec280 2024-09-10T10:02:35.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.968+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdec0b68c0 con 0x7fcdcc0679e0 2024-09-10T10:02:35.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.976+0000 7fcde9600640 1 -- 172.21.15.110:0/2551215487 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252002 (secure 0 0 0) 0x7fcdec0b68c0 con 0x7fcdcc0679e0 2024-09-10T10:02:35.983 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:35.984 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:02:35.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcdcc0679e0 msgr2=0x7fcdcc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcdcc0679e0 0x7fcdcc069ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcde00ac000 tx=0x7fcde00a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec1ec280 msgr2=0x7fcdec1fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec1ec280 0x7fcdec1fb920 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcde4000fc0 tx=0x7fcde400d8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 shutdown_connections 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcdcc0679e0 0x7fcdcc069ea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcdec1ec280 0x7fcdec1fb920 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdec1fb3d0 0x7fcdec160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 --2- 172.21.15.110:0/2551215487 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcdd4003b50 0x7fcdec15ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 >> 172.21.15.110:0/2551215487 conn(0x7fcdec0d1530 msgr2=0x7fcdec15bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 shutdown_connections 2024-09-10T10:02:35.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:35.980+0000 7fcdf4600640 1 -- 172.21.15.110:0/2551215487 wait complete. 2024-09-10T10:02:36.003 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T10:02:35.694183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258500,"kb_used_data":8984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716220,"statfs":{"total":773094113280,"available":772829409280,"internally_reserved":0,"allocated":9199616,"data_stored":4588565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":255315500},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001618"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.542215+0000","last_change":"2024-09-10T10:02:31.537543+0000","last_active":"2024-09-10T10:02:31.542215+0000","last_peered":"2024-09-10T10:02:31.542215+0000","last_clean":"2024-09-10T10:02:31.542215+0000","last_became_active":"2024-09-10T10:02:31.536424+0000","last_became_peered":"2024-09-10T10:02:31.536424+0000","last_unstale":"2024-09-10T10:02:31.542215+0000","last_undegraded":"2024-09-10T10:02:31.542215+0000","last_fullsized":"2024-09-10T10:02:31.542215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:23.746871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567932+0000","last_change":"2024-09-10T10:02:31.562606+0000","last_active":"2024-09-10T10:02:31.567932+0000","last_peered":"2024-09-10T10:02:31.567932+0000","last_clean":"2024-09-10T10:02:31.567932+0000","last_became_active":"2024-09-10T10:02:31.562331+0000","last_became_peered":"2024-09-10T10:02:31.562331+0000","last_unstale":"2024-09-10T10:02:31.567932+0000","last_undegraded":"2024-09-10T10:02:31.567932+0000","last_fullsized":"2024-09-10T10:02:31.567932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:03.821483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.553251+0000","last_change":"2024-09-10T10:02:31.548916+0000","last_active":"2024-09-10T10:02:31.553251+0000","last_peered":"2024-09-10T10:02:31.553251+0000","last_clean":"2024-09-10T10:02:31.553251+0000","last_became_active":"2024-09-10T10:02:31.548649+0000","last_became_peered":"2024-09-10T10:02:31.548649+0000","last_unstale":"2024-09-10T10:02:31.553251+0000","last_undegraded":"2024-09-10T10:02:31.553251+0000","last_fullsized":"2024-09-10T10:02:31.553251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:24.701614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552383+0000","last_change":"2024-09-10T10:02:31.546170+0000","last_active":"2024-09-10T10:02:31.552383+0000","last_peered":"2024-09-10T10:02:31.552383+0000","last_clean":"2024-09-10T10:02:31.552383+0000","last_became_active":"2024-09-10T10:02:31.545846+0000","last_became_peered":"2024-09-10T10:02:31.545846+0000","last_unstale":"2024-09-10T10:02:31.552383+0000","last_undegraded":"2024-09-10T10:02:31.552383+0000","last_fullsized":"2024-09-10T10:02:31.552383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:42.109552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552170+0000","last_change":"2024-09-10T10:02:31.548469+0000","last_active":"2024-09-10T10:02:31.552170+0000","last_peered":"2024-09-10T10:02:31.552170+0000","last_clean":"2024-09-10T10:02:31.552170+0000","last_became_active":"2024-09-10T10:02:31.548117+0000","last_became_peered":"2024-09-10T10:02:31.548117+0000","last_unstale":"2024-09-10T10:02:31.552170+0000","last_undegraded":"2024-09-10T10:02:31.552170+0000","last_fullsized":"2024-09-10T10:02:31.552170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:26.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556822+0000","last_change":"2024-09-10T10:02:31.549960+0000","last_active":"2024-09-10T10:02:31.556822+0000","last_peered":"2024-09-10T10:02:31.556822+0000","last_clean":"2024-09-10T10:02:31.556822+0000","last_became_active":"2024-09-10T10:02:31.549428+0000","last_became_peered":"2024-09-10T10:02:31.549428+0000","last_unstale":"2024-09-10T10:02:31.556822+0000","last_undegraded":"2024-09-10T10:02:31.556822+0000","last_fullsized":"2024-09-10T10:02:31.556822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:03.612516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556958+0000","last_change":"2024-09-10T10:02:31.548141+0000","last_active":"2024-09-10T10:02:31.556958+0000","last_peered":"2024-09-10T10:02:31.556958+0000","last_clean":"2024-09-10T10:02:31.556958+0000","last_became_active":"2024-09-10T10:02:31.547882+0000","last_became_peered":"2024-09-10T10:02:31.547882+0000","last_unstale":"2024-09-10T10:02:31.556958+0000","last_undegraded":"2024-09-10T10:02:31.556958+0000","last_fullsized":"2024-09-10T10:02:31.556958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:26.578162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.564513+0000","last_change":"2024-09-10T10:02:31.563038+0000","last_active":"2024-09-10T10:02:31.564513+0000","last_peered":"2024-09-10T10:02:31.564513+0000","last_clean":"2024-09-10T10:02:31.564513+0000","last_became_active":"2024-09-10T10:02:31.562751+0000","last_became_peered":"2024-09-10T10:02:31.562751+0000","last_unstale":"2024-09-10T10:02:31.564513+0000","last_undegraded":"2024-09-10T10:02:31.564513+0000","last_fullsized":"2024-09-10T10:02:31.564513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:23.870830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.562039+0000","last_change":"2024-09-10T10:02:31.559777+0000","last_active":"2024-09-10T10:02:31.562039+0000","last_peered":"2024-09-10T10:02:31.562039+0000","last_clean":"2024-09-10T10:02:31.562039+0000","last_became_active":"2024-09-10T10:02:31.559427+0000","last_became_peered":"2024-09-10T10:02:31.559427+0000","last_unstale":"2024-09-10T10:02:31.562039+0000","last_undegraded":"2024-09-10T10:02:31.562039+0000","last_fullsized":"2024-09-10T10:02:31.562039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:42.752143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.555358+0000","last_change":"2024-09-10T10:02:31.548708+0000","last_active":"2024-09-10T10:02:31.555358+0000","last_peered":"2024-09-10T10:02:31.555358+0000","last_clean":"2024-09-10T10:02:31.555358+0000","last_became_active":"2024-09-10T10:02:31.548429+0000","last_became_peered":"2024-09-10T10:02:31.548429+0000","last_unstale":"2024-09-10T10:02:31.555358+0000","last_undegraded":"2024-09-10T10:02:31.555358+0000","last_fullsized":"2024-09-10T10:02:31.555358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:15.144518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565730+0000","last_change":"2024-09-10T10:02:31.557439+0000","last_active":"2024-09-10T10:02:31.565730+0000","last_peered":"2024-09-10T10:02:31.565730+0000","last_clean":"2024-09-10T10:02:31.565730+0000","last_became_active":"2024-09-10T10:02:31.557198+0000","last_became_peered":"2024-09-10T10:02:31.557198+0000","last_unstale":"2024-09-10T10:02:31.565730+0000","last_undegraded":"2024-09-10T10:02:31.565730+0000","last_fullsized":"2024-09-10T10:02:31.565730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:50.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.549301+0000","last_change":"2024-09-10T10:02:31.543045+0000","last_active":"2024-09-10T10:02:31.549301+0000","last_peered":"2024-09-10T10:02:31.549301+0000","last_clean":"2024-09-10T10:02:31.549301+0000","last_became_active":"2024-09-10T10:02:31.542754+0000","last_became_peered":"2024-09-10T10:02:31.542754+0000","last_unstale":"2024-09-10T10:02:31.549301+0000","last_undegraded":"2024-09-10T10:02:31.549301+0000","last_fullsized":"2024-09-10T10:02:31.549301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:19.380565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556317+0000","last_change":"2024-09-10T10:02:31.554536+0000","last_active":"2024-09-10T10:02:31.556317+0000","last_peered":"2024-09-10T10:02:31.556317+0000","last_clean":"2024-09-10T10:02:31.556317+0000","last_became_active":"2024-09-10T10:02:31.554214+0000","last_became_peered":"2024-09-10T10:02:31.554214+0000","last_unstale":"2024-09-10T10:02:31.556317+0000","last_undegraded":"2024-09-10T10:02:31.556317+0000","last_fullsized":"2024-09-10T10:02:31.556317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:57.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.561133+0000","last_change":"2024-09-10T10:02:31.552948+0000","last_active":"2024-09-10T10:02:31.561133+0000","last_peered":"2024-09-10T10:02:31.561133+0000","last_clean":"2024-09-10T10:02:31.561133+0000","last_became_active":"2024-09-10T10:02:31.552690+0000","last_became_peered":"2024-09-10T10:02:31.552690+0000","last_unstale":"2024-09-10T10:02:31.561133+0000","last_undegraded":"2024-09-10T10:02:31.561133+0000","last_fullsized":"2024-09-10T10:02:31.561133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:25.567590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.555074+0000","last_change":"2024-09-10T10:02:31.547994+0000","last_active":"2024-09-10T10:02:31.555074+0000","last_peered":"2024-09-10T10:02:31.555074+0000","last_clean":"2024-09-10T10:02:31.555074+0000","last_became_active":"2024-09-10T10:02:31.547498+0000","last_became_peered":"2024-09-10T10:02:31.547498+0000","last_unstale":"2024-09-10T10:02:31.555074+0000","last_undegraded":"2024-09-10T10:02:31.555074+0000","last_fullsized":"2024-09-10T10:02:31.555074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:58.286607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.547356+0000","last_change":"2024-09-10T10:02:31.542408+0000","last_active":"2024-09-10T10:02:31.547356+0000","last_peered":"2024-09-10T10:02:31.547356+0000","last_clean":"2024-09-10T10:02:31.547356+0000","last_became_active":"2024-09-10T10:02:31.542105+0000","last_became_peered":"2024-09-10T10:02:31.542105+0000","last_unstale":"2024-09-10T10:02:31.547356+0000","last_undegraded":"2024-09-10T10:02:31.547356+0000","last_fullsized":"2024-09-10T10:02:31.547356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:42.973169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.551672+0000","last_change":"2024-09-10T10:02:31.548729+0000","last_active":"2024-09-10T10:02:31.551672+0000","last_peered":"2024-09-10T10:02:31.551672+0000","last_clean":"2024-09-10T10:02:31.551672+0000","last_became_active":"2024-09-10T10:02:31.548153+0000","last_became_peered":"2024-09-10T10:02:31.548153+0000","last_unstale":"2024-09-10T10:02:31.551672+0000","last_undegraded":"2024-09-10T10:02:31.551672+0000","last_fullsized":"2024-09-10T10:02:31.551672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:03.612516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552093+0000","last_change":"2024-09-10T10:02:31.545286+0000","last_active":"2024-09-10T10:02:31.552093+0000","last_peered":"2024-09-10T10:02:31.552093+0000","last_clean":"2024-09-10T10:02:31.552093+0000","last_became_active":"2024-09-10T10:02:31.544177+0000","last_became_peered":"2024-09-10T10:02:31.544177+0000","last_unstale":"2024-09-10T10:02:31.552093+0000","last_undegraded":"2024-09-10T10:02:31.552093+0000","last_fullsized":"2024-09-10T10:02:31.552093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:14.997921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.550344+0000","last_change":"2024-09-10T10:02:31.542334+0000","last_active":"2024-09-10T10:02:31.550344+0000","last_peered":"2024-09-10T10:02:31.550344+0000","last_clean":"2024-09-10T10:02:31.550344+0000","last_became_active":"2024-09-10T10:02:31.542062+0000","last_became_peered":"2024-09-10T10:02:31.542062+0000","last_unstale":"2024-09-10T10:02:31.550344+0000","last_undegraded":"2024-09-10T10:02:31.550344+0000","last_fullsized":"2024-09-10T10:02:31.550344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:10.136920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.566777+0000","last_change":"2024-09-10T10:02:31.552513+0000","last_active":"2024-09-10T10:02:31.566777+0000","last_peered":"2024-09-10T10:02:31.566777+0000","last_clean":"2024-09-10T10:02:31.566777+0000","last_became_active":"2024-09-10T10:02:31.552185+0000","last_became_peered":"2024-09-10T10:02:31.552185+0000","last_unstale":"2024-09-10T10:02:31.566777+0000","last_undegraded":"2024-09-10T10:02:31.566777+0000","last_fullsized":"2024-09-10T10:02:31.566777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:04.798551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.564893+0000","last_change":"2024-09-10T10:02:31.553625+0000","last_active":"2024-09-10T10:02:31.564893+0000","last_peered":"2024-09-10T10:02:31.564893+0000","last_clean":"2024-09-10T10:02:31.564893+0000","last_became_active":"2024-09-10T10:02:31.553327+0000","last_became_peered":"2024-09-10T10:02:31.553327+0000","last_unstale":"2024-09-10T10:02:31.564893+0000","last_undegraded":"2024-09-10T10:02:31.564893+0000","last_fullsized":"2024-09-10T10:02:31.564893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.159121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567850+0000","last_change":"2024-09-10T10:02:31.562283+0000","last_active":"2024-09-10T10:02:31.567850+0000","last_peered":"2024-09-10T10:02:31.567850+0000","last_clean":"2024-09-10T10:02:31.567850+0000","last_became_active":"2024-09-10T10:02:31.562020+0000","last_became_peered":"2024-09-10T10:02:31.562020+0000","last_unstale":"2024-09-10T10:02:31.567850+0000","last_undegraded":"2024-09-10T10:02:31.567850+0000","last_fullsized":"2024-09-10T10:02:31.567850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:27.739299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552006+0000","last_change":"2024-09-10T10:02:31.548505+0000","last_active":"2024-09-10T10:02:31.552006+0000","last_peered":"2024-09-10T10:02:31.552006+0000","last_clean":"2024-09-10T10:02:31.552006+0000","last_became_active":"2024-09-10T10:02:31.547909+0000","last_became_peered":"2024-09-10T10:02:31.547909+0000","last_unstale":"2024-09-10T10:02:31.552006+0000","last_undegraded":"2024-09-10T10:02:31.552006+0000","last_fullsized":"2024-09-10T10:02:31.552006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:01.152189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565670+0000","last_change":"2024-09-10T10:02:31.555494+0000","last_active":"2024-09-10T10:02:31.565670+0000","last_peered":"2024-09-10T10:02:31.565670+0000","last_clean":"2024-09-10T10:02:31.565670+0000","last_became_active":"2024-09-10T10:02:31.555240+0000","last_became_peered":"2024-09-10T10:02:31.555240+0000","last_unstale":"2024-09-10T10:02:31.565670+0000","last_undegraded":"2024-09-10T10:02:31.565670+0000","last_fullsized":"2024-09-10T10:02:31.565670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:13.169506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.569937+0000","last_change":"2024-09-10T10:02:31.566896+0000","last_active":"2024-09-10T10:02:31.569937+0000","last_peered":"2024-09-10T10:02:31.569937+0000","last_clean":"2024-09-10T10:02:31.569937+0000","last_became_active":"2024-09-10T10:02:31.564293+0000","last_became_peered":"2024-09-10T10:02:31.564293+0000","last_unstale":"2024-09-10T10:02:31.569937+0000","last_undegraded":"2024-09-10T10:02:31.569937+0000","last_fullsized":"2024-09-10T10:02:31.569937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:15.343407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552648+0000","last_change":"2024-09-10T10:02:31.547509+0000","last_active":"2024-09-10T10:02:31.552648+0000","last_peered":"2024-09-10T10:02:31.552648+0000","last_clean":"2024-09-10T10:02:31.552648+0000","last_became_active":"2024-09-10T10:02:31.545217+0000","last_became_peered":"2024-09-10T10:02:31.545217+0000","last_unstale":"2024-09-10T10:02:31.552648+0000","last_undegraded":"2024-09-10T10:02:31.552648+0000","last_fullsized":"2024-09-10T10:02:31.552648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:12.367366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.550807+0000","last_change":"2024-09-10T10:02:31.543777+0000","last_active":"2024-09-10T10:02:31.550807+0000","last_peered":"2024-09-10T10:02:31.550807+0000","last_clean":"2024-09-10T10:02:31.550807+0000","last_became_active":"2024-09-10T10:02:31.543531+0000","last_became_peered":"2024-09-10T10:02:31.543531+0000","last_unstale":"2024-09-10T10:02:31.550807+0000","last_undegraded":"2024-09-10T10:02:31.550807+0000","last_fullsized":"2024-09-10T10:02:31.550807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:21.481426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.542178+0000","last_change":"2024-09-10T10:02:31.537110+0000","last_active":"2024-09-10T10:02:31.542178+0000","last_peered":"2024-09-10T10:02:31.542178+0000","last_clean":"2024-09-10T10:02:31.542178+0000","last_became_active":"2024-09-10T10:02:31.536793+0000","last_became_peered":"2024-09-10T10:02:31.536793+0000","last_unstale":"2024-09-10T10:02:31.542178+0000","last_undegraded":"2024-09-10T10:02:31.542178+0000","last_fullsized":"2024-09-10T10:02:31.542178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:03.612516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567800+0000","last_change":"2024-09-10T10:02:31.559308+0000","last_active":"2024-09-10T10:02:31.567800+0000","last_peered":"2024-09-10T10:02:31.567800+0000","last_clean":"2024-09-10T10:02:31.567800+0000","last_became_active":"2024-09-10T10:02:31.559032+0000","last_became_peered":"2024-09-10T10:02:31.559032+0000","last_unstale":"2024-09-10T10:02:31.567800+0000","last_undegraded":"2024-09-10T10:02:31.567800+0000","last_fullsized":"2024-09-10T10:02:31.567800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:58.042203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560629+0000","last_change":"2024-09-10T10:02:31.554139+0000","last_active":"2024-09-10T10:02:31.560629+0000","last_peered":"2024-09-10T10:02:31.560629+0000","last_clean":"2024-09-10T10:02:31.560629+0000","last_became_active":"2024-09-10T10:02:31.553623+0000","last_became_peered":"2024-09-10T10:02:31.553623+0000","last_unstale":"2024-09-10T10:02:31.560629+0000","last_undegraded":"2024-09-10T10:02:31.560629+0000","last_fullsized":"2024-09-10T10:02:31.560629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:35.006717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.541471+0000","last_change":"2024-09-10T10:02:31.538144+0000","last_active":"2024-09-10T10:02:31.541471+0000","last_peered":"2024-09-10T10:02:31.541471+0000","last_clean":"2024-09-10T10:02:31.541471+0000","last_became_active":"2024-09-10T10:02:31.537870+0000","last_became_peered":"2024-09-10T10:02:31.537870+0000","last_unstale":"2024-09-10T10:02:31.541471+0000","last_undegraded":"2024-09-10T10:02:31.541471+0000","last_fullsized":"2024-09-10T10:02:31.541471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:19.696540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565350+0000","last_change":"2024-09-10T10:02:31.564283+0000","last_active":"2024-09-10T10:02:31.565350+0000","last_peered":"2024-09-10T10:02:31.565350+0000","last_clean":"2024-09-10T10:02:31.565350+0000","last_became_active":"2024-09-10T10:02:31.564023+0000","last_became_peered":"2024-09-10T10:02:31.564023+0000","last_unstale":"2024-09-10T10:02:31.565350+0000","last_undegraded":"2024-09-10T10:02:31.565350+0000","last_fullsized":"2024-09-10T10:02:31.565350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:12.367366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.547778+0000","last_change":"2024-09-10T10:02:31.540412+0000","last_active":"2024-09-10T10:02:31.547778+0000","last_peered":"2024-09-10T10:02:31.547778+0000","last_clean":"2024-09-10T10:02:31.547778+0000","last_became_active":"2024-09-10T10:02:31.540015+0000","last_became_peered":"2024-09-10T10:02:31.540015+0000","last_unstale":"2024-09-10T10:02:31.547778+0000","last_undegraded":"2024-09-10T10:02:31.547778+0000","last_fullsized":"2024-09-10T10:02:31.547778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:37.019878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568613+0000","last_change":"2024-09-10T10:02:30.546900+0000","last_active":"2024-09-10T10:02:31.568613+0000","last_peered":"2024-09-10T10:02:31.568613+0000","last_clean":"2024-09-10T10:02:31.568613+0000","last_became_active":"2024-09-10T10:02:30.546124+0000","last_became_peered":"2024-09-10T10:02:30.546124+0000","last_unstale":"2024-09-10T10:02:31.568613+0000","last_undegraded":"2024-09-10T10:02:31.568613+0000","last_fullsized":"2024-09-10T10:02:31.568613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:20.477932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.553056+0000","last_change":"2024-09-10T10:02:31.547790+0000","last_active":"2024-09-10T10:02:31.553056+0000","last_peered":"2024-09-10T10:02:31.553056+0000","last_clean":"2024-09-10T10:02:31.553056+0000","last_became_active":"2024-09-10T10:02:31.545182+0000","last_became_peered":"2024-09-10T10:02:31.545182+0000","last_unstale":"2024-09-10T10:02:31.553056+0000","last_undegraded":"2024-09-10T10:02:31.553056+0000","last_fullsized":"2024-09-10T10:02:31.553056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:23.870830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.563722+0000","last_change":"2024-09-10T10:02:30.538747+0000","last_active":"2024-09-10T10:02:31.563722+0000","last_peered":"2024-09-10T10:02:31.563722+0000","last_clean":"2024-09-10T10:02:31.563722+0000","last_became_active":"2024-09-10T10:02:30.538469+0000","last_became_peered":"2024-09-10T10:02:30.538469+0000","last_unstale":"2024-09-10T10:02:31.563722+0000","last_undegraded":"2024-09-10T10:02:31.563722+0000","last_fullsized":"2024-09-10T10:02:31.563722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:27.005499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.564981+0000","last_change":"2024-09-10T10:02:31.563536+0000","last_active":"2024-09-10T10:02:31.564981+0000","last_peered":"2024-09-10T10:02:31.564981+0000","last_clean":"2024-09-10T10:02:31.564981+0000","last_became_active":"2024-09-10T10:02:31.562898+0000","last_became_peered":"2024-09-10T10:02:31.562898+0000","last_unstale":"2024-09-10T10:02:31.564981+0000","last_undegraded":"2024-09-10T10:02:31.564981+0000","last_fullsized":"2024-09-10T10:02:31.564981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:16.389617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568622+0000","last_change":"2024-09-10T10:02:30.543972+0000","last_active":"2024-09-10T10:02:31.568622+0000","last_peered":"2024-09-10T10:02:31.568622+0000","last_clean":"2024-09-10T10:02:31.568622+0000","last_became_active":"2024-09-10T10:02:30.543640+0000","last_became_peered":"2024-09-10T10:02:30.543640+0000","last_unstale":"2024-09-10T10:02:31.568622+0000","last_undegraded":"2024-09-10T10:02:31.568622+0000","last_fullsized":"2024-09-10T10:02:31.568622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:56.190827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.559222+0000","last_change":"2024-09-10T10:02:31.553943+0000","last_active":"2024-09-10T10:02:31.559222+0000","last_peered":"2024-09-10T10:02:31.559222+0000","last_clean":"2024-09-10T10:02:31.559222+0000","last_became_active":"2024-09-10T10:02:31.553672+0000","last_became_peered":"2024-09-10T10:02:31.553672+0000","last_unstale":"2024-09-10T10:02:31.559222+0000","last_undegraded":"2024-09-10T10:02:31.559222+0000","last_fullsized":"2024-09-10T10:02:31.559222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:12.482386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568810+0000","last_change":"2024-09-10T10:02:30.544565+0000","last_active":"2024-09-10T10:02:31.568810+0000","last_peered":"2024-09-10T10:02:31.568810+0000","last_clean":"2024-09-10T10:02:31.568810+0000","last_became_active":"2024-09-10T10:02:30.544309+0000","last_became_peered":"2024-09-10T10:02:30.544309+0000","last_unstale":"2024-09-10T10:02:31.568810+0000","last_undegraded":"2024-09-10T10:02:31.568810+0000","last_fullsized":"2024-09-10T10:02:31.568810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:11.363871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560650+0000","last_change":"2024-09-10T10:02:31.554291+0000","last_active":"2024-09-10T10:02:31.560650+0000","last_peered":"2024-09-10T10:02:31.560650+0000","last_clean":"2024-09-10T10:02:31.560650+0000","last_became_active":"2024-09-10T10:02:31.553965+0000","last_became_peered":"2024-09-10T10:02:31.553965+0000","last_unstale":"2024-09-10T10:02:31.560650+0000","last_undegraded":"2024-09-10T10:02:31.560650+0000","last_fullsized":"2024-09-10T10:02:31.560650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:48.916497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565924+0000","last_change":"2024-09-10T10:02:30.540475+0000","last_active":"2024-09-10T10:02:31.565924+0000","last_peered":"2024-09-10T10:02:31.565924+0000","last_clean":"2024-09-10T10:02:31.565924+0000","last_became_active":"2024-09-10T10:02:30.538091+0000","last_became_peered":"2024-09-10T10:02:30.538091+0000","last_unstale":"2024-09-10T10:02:31.565924+0000","last_undegraded":"2024-09-10T10:02:31.565924+0000","last_fullsized":"2024-09-10T10:02:31.565924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:45.234728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552193+0000","last_change":"2024-09-10T10:02:31.548761+0000","last_active":"2024-09-10T10:02:31.552193+0000","last_peered":"2024-09-10T10:02:31.552193+0000","last_clean":"2024-09-10T10:02:31.552193+0000","last_became_active":"2024-09-10T10:02:31.548463+0000","last_became_peered":"2024-09-10T10:02:31.548463+0000","last_unstale":"2024-09-10T10:02:31.552193+0000","last_undegraded":"2024-09-10T10:02:31.552193+0000","last_fullsized":"2024-09-10T10:02:31.552193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:23.746871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560393+0000","last_change":"2024-09-10T10:02:30.542060+0000","last_active":"2024-09-10T10:02:31.560393+0000","last_peered":"2024-09-10T10:02:31.560393+0000","last_clean":"2024-09-10T10:02:31.560393+0000","last_became_active":"2024-09-10T10:02:30.541590+0000","last_became_peered":"2024-09-10T10:02:30.541590+0000","last_unstale":"2024-09-10T10:02:31.560393+0000","last_undegraded":"2024-09-10T10:02:31.560393+0000","last_fullsized":"2024-09-10T10:02:31.560393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:29.871870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565560+0000","last_change":"2024-09-10T10:02:31.563067+0000","last_active":"2024-09-10T10:02:31.565560+0000","last_peered":"2024-09-10T10:02:31.565560+0000","last_clean":"2024-09-10T10:02:31.565560+0000","last_became_active":"2024-09-10T10:02:31.562681+0000","last_became_peered":"2024-09-10T10:02:31.562681+0000","last_unstale":"2024-09-10T10:02:31.565560+0000","last_undegraded":"2024-09-10T10:02:31.565560+0000","last_fullsized":"2024-09-10T10:02:31.565560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:03.853165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.528182+0000","last_change":"2024-09-10T10:02:30.546298+0000","last_active":"2024-09-10T10:02:31.528182+0000","last_peered":"2024-09-10T10:02:31.528182+0000","last_clean":"2024-09-10T10:02:31.528182+0000","last_became_active":"2024-09-10T10:02:30.544005+0000","last_became_peered":"2024-09-10T10:02:30.544005+0000","last_unstale":"2024-09-10T10:02:31.528182+0000","last_undegraded":"2024-09-10T10:02:31.528182+0000","last_fullsized":"2024-09-10T10:02:31.528182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:50.393087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.554400+0000","last_change":"2024-09-10T10:02:31.547762+0000","last_active":"2024-09-10T10:02:31.554400+0000","last_peered":"2024-09-10T10:02:31.554400+0000","last_clean":"2024-09-10T10:02:31.554400+0000","last_became_active":"2024-09-10T10:02:31.547152+0000","last_became_peered":"2024-09-10T10:02:31.547152+0000","last_unstale":"2024-09-10T10:02:31.554400+0000","last_undegraded":"2024-09-10T10:02:31.554400+0000","last_fullsized":"2024-09-10T10:02:31.554400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:42.338238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.539412+0000","last_change":"2024-09-10T10:02:30.538882+0000","last_active":"2024-09-10T10:02:31.539412+0000","last_peered":"2024-09-10T10:02:31.539412+0000","last_clean":"2024-09-10T10:02:31.539412+0000","last_became_active":"2024-09-10T10:02:30.537793+0000","last_became_peered":"2024-09-10T10:02:30.537793+0000","last_unstale":"2024-09-10T10:02:31.539412+0000","last_undegraded":"2024-09-10T10:02:31.539412+0000","last_fullsized":"2024-09-10T10:02:31.539412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:11.644067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.559071+0000","last_change":"2024-09-10T10:02:31.552229+0000","last_active":"2024-09-10T10:02:31.559071+0000","last_peered":"2024-09-10T10:02:31.559071+0000","last_clean":"2024-09-10T10:02:31.559071+0000","last_became_active":"2024-09-10T10:02:31.551983+0000","last_became_peered":"2024-09-10T10:02:31.551983+0000","last_unstale":"2024-09-10T10:02:31.559071+0000","last_undegraded":"2024-09-10T10:02:31.559071+0000","last_fullsized":"2024-09-10T10:02:31.559071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:49.802847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534966+0000","last_change":"2024-09-10T10:02:30.546608+0000","last_active":"2024-09-10T10:02:31.534966+0000","last_peered":"2024-09-10T10:02:31.534966+0000","last_clean":"2024-09-10T10:02:31.534966+0000","last_became_active":"2024-09-10T10:02:30.545793+0000","last_became_peered":"2024-09-10T10:02:30.545793+0000","last_unstale":"2024-09-10T10:02:31.534966+0000","last_undegraded":"2024-09-10T10:02:31.534966+0000","last_fullsized":"2024-09-10T10:02:31.534966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:04.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.557730+0000","last_change":"2024-09-10T10:02:31.555156+0000","last_active":"2024-09-10T10:02:31.557730+0000","last_peered":"2024-09-10T10:02:31.557730+0000","last_clean":"2024-09-10T10:02:31.557730+0000","last_became_active":"2024-09-10T10:02:31.554830+0000","last_became_peered":"2024-09-10T10:02:31.554830+0000","last_unstale":"2024-09-10T10:02:31.557730+0000","last_undegraded":"2024-09-10T10:02:31.557730+0000","last_fullsized":"2024-09-10T10:02:31.557730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:19.380565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534582+0000","last_change":"2024-09-10T10:02:30.544730+0000","last_active":"2024-09-10T10:02:31.534582+0000","last_peered":"2024-09-10T10:02:31.534582+0000","last_clean":"2024-09-10T10:02:31.534582+0000","last_became_active":"2024-09-10T10:02:30.543586+0000","last_became_peered":"2024-09-10T10:02:30.543586+0000","last_unstale":"2024-09-10T10:02:31.534582+0000","last_undegraded":"2024-09-10T10:02:31.534582+0000","last_fullsized":"2024-09-10T10:02:31.534582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:41.969675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.550403+0000","last_change":"2024-09-10T10:02:31.544907+0000","last_active":"2024-09-10T10:02:31.550403+0000","last_peered":"2024-09-10T10:02:31.550403+0000","last_clean":"2024-09-10T10:02:31.550403+0000","last_became_active":"2024-09-10T10:02:31.543522+0000","last_became_peered":"2024-09-10T10:02:31.543522+0000","last_unstale":"2024-09-10T10:02:31.550403+0000","last_undegraded":"2024-09-10T10:02:31.550403+0000","last_fullsized":"2024-09-10T10:02:31.550403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:35.525434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568916+0000","last_change":"2024-09-10T10:02:30.545027+0000","last_active":"2024-09-10T10:02:31.568916+0000","last_peered":"2024-09-10T10:02:31.568916+0000","last_clean":"2024-09-10T10:02:31.568916+0000","last_became_active":"2024-09-10T10:02:30.544757+0000","last_became_peered":"2024-09-10T10:02:30.544757+0000","last_unstale":"2024-09-10T10:02:31.568916+0000","last_undegraded":"2024-09-10T10:02:31.568916+0000","last_fullsized":"2024-09-10T10:02:31.568916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:45.234728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.555452+0000","last_change":"2024-09-10T10:02:31.551445+0000","last_active":"2024-09-10T10:02:31.555452+0000","last_peered":"2024-09-10T10:02:31.555452+0000","last_clean":"2024-09-10T10:02:31.555452+0000","last_became_active":"2024-09-10T10:02:31.551027+0000","last_became_peered":"2024-09-10T10:02:31.551027+0000","last_unstale":"2024-09-10T10:02:31.555452+0000","last_undegraded":"2024-09-10T10:02:31.555452+0000","last_fullsized":"2024-09-10T10:02:31.555452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:42.752143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.570178+0000","last_change":"2024-09-10T10:02:30.548612+0000","last_active":"2024-09-10T10:02:31.570178+0000","last_peered":"2024-09-10T10:02:31.570178+0000","last_clean":"2024-09-10T10:02:31.570178+0000","last_became_active":"2024-09-10T10:02:30.548322+0000","last_became_peered":"2024-09-10T10:02:30.548322+0000","last_unstale":"2024-09-10T10:02:31.570178+0000","last_undegraded":"2024-09-10T10:02:31.570178+0000","last_fullsized":"2024-09-10T10:02:31.570178+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:20.477932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.541792+0000","last_change":"2024-09-10T10:02:31.536609+0000","last_active":"2024-09-10T10:02:31.541792+0000","last_peered":"2024-09-10T10:02:31.541792+0000","last_clean":"2024-09-10T10:02:31.541792+0000","last_became_active":"2024-09-10T10:02:31.536334+0000","last_became_peered":"2024-09-10T10:02:31.536334+0000","last_unstale":"2024-09-10T10:02:31.541792+0000","last_undegraded":"2024-09-10T10:02:31.541792+0000","last_fullsized":"2024-09-10T10:02:31.541792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:24.316984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567592+0000","last_change":"2024-09-10T10:02:30.540161+0000","last_active":"2024-09-10T10:02:31.567592+0000","last_peered":"2024-09-10T10:02:31.567592+0000","last_clean":"2024-09-10T10:02:31.567592+0000","last_became_active":"2024-09-10T10:02:30.539757+0000","last_became_peered":"2024-09-10T10:02:30.539757+0000","last_unstale":"2024-09-10T10:02:31.567592+0000","last_undegraded":"2024-09-10T10:02:31.567592+0000","last_fullsized":"2024-09-10T10:02:31.567592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:48.037362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.561170+0000","last_change":"2024-09-10T10:02:31.560329+0000","last_active":"2024-09-10T10:02:31.561170+0000","last_peered":"2024-09-10T10:02:31.561170+0000","last_clean":"2024-09-10T10:02:31.561170+0000","last_became_active":"2024-09-10T10:02:31.560059+0000","last_became_peered":"2024-09-10T10:02:31.560059+0000","last_unstale":"2024-09-10T10:02:31.561170+0000","last_undegraded":"2024-09-10T10:02:31.561170+0000","last_fullsized":"2024-09-10T10:02:31.561170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:34:35.525434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.533852+0000","last_change":"2024-09-10T10:02:30.544437+0000","last_active":"2024-09-10T10:02:31.533852+0000","last_peered":"2024-09-10T10:02:31.533852+0000","last_clean":"2024-09-10T10:02:31.533852+0000","last_became_active":"2024-09-10T10:02:30.543094+0000","last_became_peered":"2024-09-10T10:02:30.543094+0000","last_unstale":"2024-09-10T10:02:31.533852+0000","last_undegraded":"2024-09-10T10:02:31.533852+0000","last_fullsized":"2024-09-10T10:02:31.533852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:27.052155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.557977+0000","last_change":"2024-09-10T10:02:31.549747+0000","last_active":"2024-09-10T10:02:31.557977+0000","last_peered":"2024-09-10T10:02:31.557977+0000","last_clean":"2024-09-10T10:02:31.557977+0000","last_became_active":"2024-09-10T10:02:31.549471+0000","last_became_peered":"2024-09-10T10:02:31.549471+0000","last_unstale":"2024-09-10T10:02:31.557977+0000","last_undegraded":"2024-09-10T10:02:31.557977+0000","last_fullsized":"2024-09-10T10:02:31.557977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:01.886252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.540543+0000","last_change":"2024-09-10T10:02:30.548951+0000","last_active":"2024-09-10T10:02:31.540543+0000","last_peered":"2024-09-10T10:02:31.540543+0000","last_clean":"2024-09-10T10:02:31.540543+0000","last_became_active":"2024-09-10T10:02:30.548655+0000","last_became_peered":"2024-09-10T10:02:30.548655+0000","last_unstale":"2024-09-10T10:02:31.540543+0000","last_undegraded":"2024-09-10T10:02:31.540543+0000","last_fullsized":"2024-09-10T10:02:31.540543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:04.218971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567736+0000","last_change":"2024-09-10T10:02:31.561014+0000","last_active":"2024-09-10T10:02:31.567736+0000","last_peered":"2024-09-10T10:02:31.567736+0000","last_clean":"2024-09-10T10:02:31.567736+0000","last_became_active":"2024-09-10T10:02:31.560756+0000","last_became_peered":"2024-09-10T10:02:31.560756+0000","last_unstale":"2024-09-10T10:02:31.567736+0000","last_undegraded":"2024-09-10T10:02:31.567736+0000","last_fullsized":"2024-09-10T10:02:31.567736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:45.753900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567056+0000","last_change":"2024-09-10T10:02:30.549278+0000","last_active":"2024-09-10T10:02:31.567056+0000","last_peered":"2024-09-10T10:02:31.567056+0000","last_clean":"2024-09-10T10:02:31.567056+0000","last_became_active":"2024-09-10T10:02:30.548996+0000","last_became_peered":"2024-09-10T10:02:30.548996+0000","last_unstale":"2024-09-10T10:02:31.567056+0000","last_undegraded":"2024-09-10T10:02:31.567056+0000","last_fullsized":"2024-09-10T10:02:31.567056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:14.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.563160+0000","last_change":"2024-09-10T10:02:31.558223+0000","last_active":"2024-09-10T10:02:31.563160+0000","last_peered":"2024-09-10T10:02:31.563160+0000","last_clean":"2024-09-10T10:02:31.563160+0000","last_became_active":"2024-09-10T10:02:31.557905+0000","last_became_peered":"2024-09-10T10:02:31.557905+0000","last_unstale":"2024-09-10T10:02:31.563160+0000","last_undegraded":"2024-09-10T10:02:31.563160+0000","last_fullsized":"2024-09-10T10:02:31.563160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:26.502768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560090+0000","last_change":"2024-09-10T10:02:30.546372+0000","last_active":"2024-09-10T10:02:31.560090+0000","last_peered":"2024-09-10T10:02:31.560090+0000","last_clean":"2024-09-10T10:02:31.560090+0000","last_became_active":"2024-09-10T10:02:30.545491+0000","last_became_peered":"2024-09-10T10:02:30.545491+0000","last_unstale":"2024-09-10T10:02:31.560090+0000","last_undegraded":"2024-09-10T10:02:31.560090+0000","last_fullsized":"2024-09-10T10:02:31.560090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:25.529504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556759+0000","last_change":"2024-09-10T10:02:31.553625+0000","last_active":"2024-09-10T10:02:31.556759+0000","last_peered":"2024-09-10T10:02:31.556759+0000","last_clean":"2024-09-10T10:02:31.556759+0000","last_became_active":"2024-09-10T10:02:31.553288+0000","last_became_peered":"2024-09-10T10:02:31.553288+0000","last_unstale":"2024-09-10T10:02:31.556759+0000","last_undegraded":"2024-09-10T10:02:31.556759+0000","last_fullsized":"2024-09-10T10:02:31.556759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:51.396582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565999+0000","last_change":"2024-09-10T10:02:30.543434+0000","last_active":"2024-09-10T10:02:31.565999+0000","last_peered":"2024-09-10T10:02:31.565999+0000","last_clean":"2024-09-10T10:02:31.565999+0000","last_became_active":"2024-09-10T10:02:30.542135+0000","last_became_peered":"2024-09-10T10:02:30.542135+0000","last_unstale":"2024-09-10T10:02:31.565999+0000","last_undegraded":"2024-09-10T10:02:31.565999+0000","last_fullsized":"2024-09-10T10:02:31.565999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:11.478892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.555475+0000","last_change":"2024-09-10T10:02:31.552270+0000","last_active":"2024-09-10T10:02:31.555475+0000","last_peered":"2024-09-10T10:02:31.555475+0000","last_clean":"2024-09-10T10:02:31.555475+0000","last_became_active":"2024-09-10T10:02:31.551944+0000","last_became_peered":"2024-09-10T10:02:31.551944+0000","last_unstale":"2024-09-10T10:02:31.555475+0000","last_undegraded":"2024-09-10T10:02:31.555475+0000","last_fullsized":"2024-09-10T10:02:31.555475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:55.116672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.528810+0000","last_change":"2024-09-10T10:02:30.544170+0000","last_active":"2024-09-10T10:02:31.528810+0000","last_peered":"2024-09-10T10:02:31.528810+0000","last_clean":"2024-09-10T10:02:31.528810+0000","last_became_active":"2024-09-10T10:02:30.543876+0000","last_became_peered":"2024-09-10T10:02:30.543876+0000","last_unstale":"2024-09-10T10:02:31.528810+0000","last_undegraded":"2024-09-10T10:02:31.528810+0000","last_fullsized":"2024-09-10T10:02:31.528810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:23.698119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.565897+0000","last_change":"2024-09-10T10:02:31.556818+0000","last_active":"2024-09-10T10:02:31.565897+0000","last_peered":"2024-09-10T10:02:31.565897+0000","last_clean":"2024-09-10T10:02:31.565897+0000","last_became_active":"2024-09-10T10:02:31.556553+0000","last_became_peered":"2024-09-10T10:02:31.556553+0000","last_unstale":"2024-09-10T10:02:31.565897+0000","last_undegraded":"2024-09-10T10:02:31.565897+0000","last_fullsized":"2024-09-10T10:02:31.565897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:39.978614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.569094+0000","last_change":"2024-09-10T10:02:30.544084+0000","last_active":"2024-09-10T10:02:31.569094+0000","last_peered":"2024-09-10T10:02:31.569094+0000","last_clean":"2024-09-10T10:02:31.569094+0000","last_became_active":"2024-09-10T10:02:30.543804+0000","last_became_peered":"2024-09-10T10:02:30.543804+0000","last_unstale":"2024-09-10T10:02:31.569094+0000","last_undegraded":"2024-09-10T10:02:31.569094+0000","last_fullsized":"2024-09-10T10:02:31.569094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:21.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556942+0000","last_change":"2024-09-10T10:02:31.550986+0000","last_active":"2024-09-10T10:02:31.556942+0000","last_peered":"2024-09-10T10:02:31.556942+0000","last_clean":"2024-09-10T10:02:31.556942+0000","last_became_active":"2024-09-10T10:02:31.550664+0000","last_became_peered":"2024-09-10T10:02:31.550664+0000","last_unstale":"2024-09-10T10:02:31.556942+0000","last_undegraded":"2024-09-10T10:02:31.556942+0000","last_fullsized":"2024-09-10T10:02:31.556942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:23.746871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.542864+0000","last_change":"2024-09-10T10:02:30.540953+0000","last_active":"2024-09-10T10:02:31.542864+0000","last_peered":"2024-09-10T10:02:31.542864+0000","last_clean":"2024-09-10T10:02:31.542864+0000","last_became_active":"2024-09-10T10:02:30.540677+0000","last_became_peered":"2024-09-10T10:02:30.540677+0000","last_unstale":"2024-09-10T10:02:31.542864+0000","last_undegraded":"2024-09-10T10:02:31.542864+0000","last_fullsized":"2024-09-10T10:02:31.542864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:40:31.074701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.545209+0000","last_change":"2024-09-10T10:02:31.542445+0000","last_active":"2024-09-10T10:02:31.545209+0000","last_peered":"2024-09-10T10:02:31.545209+0000","last_clean":"2024-09-10T10:02:31.545209+0000","last_became_active":"2024-09-10T10:02:31.542182+0000","last_became_peered":"2024-09-10T10:02:31.542182+0000","last_unstale":"2024-09-10T10:02:31.545209+0000","last_undegraded":"2024-09-10T10:02:31.545209+0000","last_fullsized":"2024-09-10T10:02:31.545209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:56.826789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.541703+0000","last_change":"2024-09-10T10:02:30.545974+0000","last_active":"2024-09-10T10:02:31.541703+0000","last_peered":"2024-09-10T10:02:31.541703+0000","last_clean":"2024-09-10T10:02:31.541703+0000","last_became_active":"2024-09-10T10:02:30.545694+0000","last_became_peered":"2024-09-10T10:02:30.545694+0000","last_unstale":"2024-09-10T10:02:31.541703+0000","last_undegraded":"2024-09-10T10:02:31.541703+0000","last_fullsized":"2024-09-10T10:02:31.541703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:17.117085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.564032+0000","last_change":"2024-09-10T10:02:31.560142+0000","last_active":"2024-09-10T10:02:31.564032+0000","last_peered":"2024-09-10T10:02:31.564032+0000","last_clean":"2024-09-10T10:02:31.564032+0000","last_became_active":"2024-09-10T10:02:31.559822+0000","last_became_peered":"2024-09-10T10:02:31.559822+0000","last_unstale":"2024-09-10T10:02:31.564032+0000","last_undegraded":"2024-09-10T10:02:31.564032+0000","last_fullsized":"2024-09-10T10:02:31.564032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:55.116672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534091+0000","last_change":"2024-09-10T10:02:30.541474+0000","last_active":"2024-09-10T10:02:31.534091+0000","last_peered":"2024-09-10T10:02:31.534091+0000","last_clean":"2024-09-10T10:02:31.534091+0000","last_became_active":"2024-09-10T10:02:30.540189+0000","last_became_peered":"2024-09-10T10:02:30.540189+0000","last_unstale":"2024-09-10T10:02:31.534091+0000","last_undegraded":"2024-09-10T10:02:31.534091+0000","last_fullsized":"2024-09-10T10:02:31.534091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:30.469708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.551722+0000","last_change":"2024-09-10T10:02:31.544741+0000","last_active":"2024-09-10T10:02:31.551722+0000","last_peered":"2024-09-10T10:02:31.551722+0000","last_clean":"2024-09-10T10:02:31.551722+0000","last_became_active":"2024-09-10T10:02:31.544502+0000","last_became_peered":"2024-09-10T10:02:31.544502+0000","last_unstale":"2024-09-10T10:02:31.551722+0000","last_undegraded":"2024-09-10T10:02:31.551722+0000","last_fullsized":"2024-09-10T10:02:31.551722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:05.222466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.567419+0000","last_change":"2024-09-10T10:02:30.548421+0000","last_active":"2024-09-10T10:02:31.567419+0000","last_peered":"2024-09-10T10:02:31.567419+0000","last_clean":"2024-09-10T10:02:31.567419+0000","last_became_active":"2024-09-10T10:02:30.548072+0000","last_became_peered":"2024-09-10T10:02:30.548072+0000","last_unstale":"2024-09-10T10:02:31.567419+0000","last_undegraded":"2024-09-10T10:02:31.567419+0000","last_fullsized":"2024-09-10T10:02:31.567419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:08.652564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.560745+0000","last_change":"2024-09-10T10:02:31.557943+0000","last_active":"2024-09-10T10:02:31.560745+0000","last_peered":"2024-09-10T10:02:31.560745+0000","last_clean":"2024-09-10T10:02:31.560745+0000","last_became_active":"2024-09-10T10:02:31.557626+0000","last_became_peered":"2024-09-10T10:02:31.557626+0000","last_unstale":"2024-09-10T10:02:31.560745+0000","last_undegraded":"2024-09-10T10:02:31.560745+0000","last_fullsized":"2024-09-10T10:02:31.560745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:10.136920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.544719+0000","last_change":"2024-09-10T10:02:30.542999+0000","last_active":"2024-09-10T10:02:31.544719+0000","last_peered":"2024-09-10T10:02:31.544719+0000","last_clean":"2024-09-10T10:02:31.544719+0000","last_became_active":"2024-09-10T10:02:30.542683+0000","last_became_peered":"2024-09-10T10:02:30.542683+0000","last_unstale":"2024-09-10T10:02:31.544719+0000","last_undegraded":"2024-09-10T10:02:31.544719+0000","last_fullsized":"2024-09-10T10:02:31.544719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:22.360524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.550839+0000","last_change":"2024-09-10T10:02:31.546740+0000","last_active":"2024-09-10T10:02:31.550839+0000","last_peered":"2024-09-10T10:02:31.550839+0000","last_clean":"2024-09-10T10:02:31.550839+0000","last_became_active":"2024-09-10T10:02:31.543386+0000","last_became_peered":"2024-09-10T10:02:31.543386+0000","last_unstale":"2024-09-10T10:02:31.550839+0000","last_undegraded":"2024-09-10T10:02:31.550839+0000","last_fullsized":"2024-09-10T10:02:31.550839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:55.116672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534398+0000","last_change":"2024-09-10T10:02:30.544353+0000","last_active":"2024-09-10T10:02:31.534398+0000","last_peered":"2024-09-10T10:02:31.534398+0000","last_clean":"2024-09-10T10:02:31.534398+0000","last_became_active":"2024-09-10T10:02:30.544032+0000","last_became_peered":"2024-09-10T10:02:30.544032+0000","last_unstale":"2024-09-10T10:02:31.534398+0000","last_undegraded":"2024-09-10T10:02:31.534398+0000","last_fullsized":"2024-09-10T10:02:31.534398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:18.693045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.552946+0000","last_change":"2024-09-10T10:02:31.551236+0000","last_active":"2024-09-10T10:02:31.552946+0000","last_peered":"2024-09-10T10:02:31.552946+0000","last_clean":"2024-09-10T10:02:31.552946+0000","last_became_active":"2024-09-10T10:02:31.550614+0000","last_became_peered":"2024-09-10T10:02:31.550614+0000","last_unstale":"2024-09-10T10:02:31.552946+0000","last_undegraded":"2024-09-10T10:02:31.552946+0000","last_fullsized":"2024-09-10T10:02:31.552946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:19.380565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.534562+0000","last_change":"2024-09-10T10:02:30.544644+0000","last_active":"2024-09-10T10:02:31.534562+0000","last_peered":"2024-09-10T10:02:31.534562+0000","last_clean":"2024-09-10T10:02:31.534562+0000","last_became_active":"2024-09-10T10:02:30.544332+0000","last_became_peered":"2024-09-10T10:02:30.544332+0000","last_unstale":"2024-09-10T10:02:31.534562+0000","last_undegraded":"2024-09-10T10:02:31.534562+0000","last_fullsized":"2024-09-10T10:02:31.534562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:36.016383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.556116+0000","last_change":"2024-09-10T10:02:31.549701+0000","last_active":"2024-09-10T10:02:31.556116+0000","last_peered":"2024-09-10T10:02:31.556116+0000","last_clean":"2024-09-10T10:02:31.556116+0000","last_became_active":"2024-09-10T10:02:31.548944+0000","last_became_peered":"2024-09-10T10:02:31.548944+0000","last_unstale":"2024-09-10T10:02:31.556116+0000","last_undegraded":"2024-09-10T10:02:31.556116+0000","last_fullsized":"2024-09-10T10:02:31.556116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:26.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.528559+0000","last_change":"2024-09-10T10:02:30.549283+0000","last_active":"2024-09-10T10:02:31.528559+0000","last_peered":"2024-09-10T10:02:31.528559+0000","last_clean":"2024-09-10T10:02:31.528559+0000","last_became_active":"2024-09-10T10:02:30.549016+0000","last_became_peered":"2024-09-10T10:02:30.549016+0000","last_unstale":"2024-09-10T10:02:31.528559+0000","last_undegraded":"2024-09-10T10:02:31.528559+0000","last_fullsized":"2024-09-10T10:02:31.528559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:15.386122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.551302+0000","last_change":"2024-09-10T10:02:31.542526+0000","last_active":"2024-09-10T10:02:31.551302+0000","last_peered":"2024-09-10T10:02:31.551302+0000","last_clean":"2024-09-10T10:02:31.551302+0000","last_became_active":"2024-09-10T10:02:31.538982+0000","last_became_peered":"2024-09-10T10:02:31.538982+0000","last_unstale":"2024-09-10T10:02:31.551302+0000","last_undegraded":"2024-09-10T10:02:31.551302+0000","last_fullsized":"2024-09-10T10:02:31.551302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:26.502768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.533709+0000","last_change":"2024-09-10T10:02:30.537103+0000","last_active":"2024-09-10T10:02:31.533709+0000","last_peered":"2024-09-10T10:02:31.533709+0000","last_clean":"2024-09-10T10:02:31.533709+0000","last_became_active":"2024-09-10T10:02:30.536832+0000","last_became_peered":"2024-09-10T10:02:30.536832+0000","last_unstale":"2024-09-10T10:02:31.533709+0000","last_undegraded":"2024-09-10T10:02:31.533709+0000","last_fullsized":"2024-09-10T10:02:31.533709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:15.756536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.561035+0000","last_change":"2024-09-10T10:02:31.557294+0000","last_active":"2024-09-10T10:02:31.561035+0000","last_peered":"2024-09-10T10:02:31.561035+0000","last_clean":"2024-09-10T10:02:31.561035+0000","last_became_active":"2024-09-10T10:02:31.556994+0000","last_became_peered":"2024-09-10T10:02:31.556994+0000","last_unstale":"2024-09-10T10:02:31.561035+0000","last_undegraded":"2024-09-10T10:02:31.561035+0000","last_fullsized":"2024-09-10T10:02:31.561035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:24.316984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.533972+0000","last_change":"2024-09-10T10:02:30.536797+0000","last_active":"2024-09-10T10:02:31.533972+0000","last_peered":"2024-09-10T10:02:31.533972+0000","last_clean":"2024-09-10T10:02:31.533972+0000","last_became_active":"2024-09-10T10:02:30.536520+0000","last_became_peered":"2024-09-10T10:02:30.536520+0000","last_unstale":"2024-09-10T10:02:31.533972+0000","last_undegraded":"2024-09-10T10:02:31.533972+0000","last_fullsized":"2024-09-10T10:02:31.533972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:27.005499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.551962+0000","last_change":"2024-09-10T10:02:31.544360+0000","last_active":"2024-09-10T10:02:31.551962+0000","last_peered":"2024-09-10T10:02:31.551962+0000","last_clean":"2024-09-10T10:02:31.551962+0000","last_became_active":"2024-09-10T10:02:31.543605+0000","last_became_peered":"2024-09-10T10:02:31.543605+0000","last_unstale":"2024-09-10T10:02:31.551962+0000","last_undegraded":"2024-09-10T10:02:31.551962+0000","last_fullsized":"2024-09-10T10:02:31.551962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:42.752143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.532592+0000","last_change":"2024-09-10T10:02:30.547126+0000","last_active":"2024-09-10T10:02:31.532592+0000","last_peered":"2024-09-10T10:02:31.532592+0000","last_clean":"2024-09-10T10:02:31.532592+0000","last_became_active":"2024-09-10T10:02:30.543724+0000","last_became_peered":"2024-09-10T10:02:30.543724+0000","last_unstale":"2024-09-10T10:02:31.532592+0000","last_undegraded":"2024-09-10T10:02:31.532592+0000","last_fullsized":"2024-09-10T10:02:31.532592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:35.772565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568051+0000","last_change":"2024-09-10T10:02:31.563246+0000","last_active":"2024-09-10T10:02:31.568051+0000","last_peered":"2024-09-10T10:02:31.568051+0000","last_clean":"2024-09-10T10:02:31.568051+0000","last_became_active":"2024-09-10T10:02:31.562834+0000","last_became_peered":"2024-09-10T10:02:31.562834+0000","last_unstale":"2024-09-10T10:02:31.568051+0000","last_undegraded":"2024-09-10T10:02:31.568051+0000","last_fullsized":"2024-09-10T10:02:31.568051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:30.516451+0000","last_clean_scrub_stamp":"2024-09-10T10:02:30.516451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:26.609221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.568386+0000","last_change":"2024-09-10T10:02:30.549482+0000","last_active":"2024-09-10T10:02:31.568386+0000","last_peered":"2024-09-10T10:02:31.568386+0000","last_clean":"2024-09-10T10:02:31.568386+0000","last_became_active":"2024-09-10T10:02:30.549213+0000","last_became_peered":"2024-09-10T10:02:30.549213+0000","last_unstale":"2024-09-10T10:02:31.568386+0000","last_undegraded":"2024-09-10T10:02:31.568386+0000","last_fullsized":"2024-09-10T10:02:31.568386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:02:29.512957+0000","last_clean_scrub_stamp":"2024-09-10T10:02:29.512957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:15.386122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.539427+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:02:31.539427+0000","last_peered":"2024-09-10T10:02:31.539427+0000","last_clean":"2024-09-10T10:02:31.539427+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:02:31.539427+0000","last_undegraded":"2024-09-10T10:02:31.539427+0000","last_fullsized":"2024-09-10T10:02:31.539427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1609728,"data_stored":1022336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542200,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":425123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1593344,"data_stored":1015491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:36.007 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:02:36.007 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:02:36.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f6472a00640 1 Processor -- start 2024-09-10T10:02:36.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f6472a00640 1 -- start start 2024-09-10T10:02:36.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f6472a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f646c15e690 0x7f646c06e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f6472a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f646c06bc50 0x7f646c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f6472a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646c06c710 con 0x7f646c0ae840 2024-09-10T10:02:36.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f6472a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f646c0b4680 con 0x7f646c15e690 2024-09-10T10:02:36.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f6472a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f646c0b4960 con 0x7f646c06bc50 2024-09-10T10:02:36.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f646be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f646c15e690 0x7f646c06e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f646be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f646c15e690 0x7f646c06e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57246/0 (socket says 172.21.15.110:57246) 2024-09-10T10:02:36.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f646be00640 1 -- 172.21.15.110:0/1441010453 learned_addr learned my addr 172.21.15.110:0/1441010453 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:36.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f646b400640 1 --2- 172.21.15.110:0/1441010453 >> v2:172.21.15.145:3300/0 conn(0x7f646c06bc50 0x7f646c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f646be00640 1 -- 172.21.15.110:0/1441010453 >> v2:172.21.15.145:3300/0 conn(0x7f646c06bc50 msgr2=0x7f646c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f646be00640 1 --2- 172.21.15.110:0/1441010453 >> v2:172.21.15.145:3300/0 conn(0x7f646c06bc50 0x7f646c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f646be00640 1 -- 172.21.15.110:0/1441010453 >> v1:172.21.15.145:6789/0 conn(0x7f646c0ae840 legacy=0x7f646c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.188+0000 7f646be00640 1 -- 172.21.15.110:0/1441010453 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c0b4c40 con 0x7f646c15e690 2024-09-10T10:02:36.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f646be00640 1 --2- 172.21.15.110:0/1441010453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c06e240 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f646c069820 tx=0x7f645c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c3d74d0dca51d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f646aa00640 1 -- 172.21.15.110:0/1441010453 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c0a5840 con 0x7f646c15e690 2024-09-10T10:02:36.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f646aa00640 1 -- 172.21.15.110:0/1441010453 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645c0a59e0 con 0x7f646c15e690 2024-09-10T10:02:36.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f646aa00640 1 -- 172.21.15.110:0/1441010453 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c0a5db0 con 0x7f646c15e690 2024-09-10T10:02:36.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f6472a00640 1 -- 172.21.15.110:0/1441010453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 msgr2=0x7f646c06e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f6472a00640 1 --2- 172.21.15.110:0/1441010453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c06e240 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f646c069820 tx=0x7f645c01c980 comp rx=0 tx=0).stop 2024-09-10T10:02:36.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f6472a00640 1 -- 172.21.15.110:0/1441010453 shutdown_connections 2024-09-10T10:02:36.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f6472a00640 1 --2- 172.21.15.110:0/1441010453 >> v2:172.21.15.145:3300/0 conn(0x7f646c06bc50 0x7f646c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f6472a00640 1 --2- 172.21.15.110:0/1441010453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c06e240 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f6472a00640 1 -- 172.21.15.110:0/1441010453 >> 172.21.15.110:0/1441010453 conn(0x7f646c0d1530 msgr2=0x7f646c0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f6472a00640 1 -- 172.21.15.110:0/1441010453 shutdown_connections 2024-09-10T10:02:36.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.192+0000 7f6472a00640 1 -- 172.21.15.110:0/1441010453 wait complete. 2024-09-10T10:02:36.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 Processor -- start 2024-09-10T10:02:36.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 -- start start 2024-09-10T10:02:36.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f646c06bc50 0x7f646c15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f646c0ae840 0x7f646c0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f646c0af0a0 con 0x7f646c15e690 2024-09-10T10:02:36.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f646c0b0aa0 con 0x7f646c06bc50 2024-09-10T10:02:36.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f646c15ea70 con 0x7f646c0ae840 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c14ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57250/0 (socket says 172.21.15.110:57250) 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 -- 172.21.15.110:0/3863744305 learned_addr learned my addr 172.21.15.110:0/3863744305 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f646b400640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f646c0ae840 0x7f646c0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f646be00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f646c06bc50 0x7f646c15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 -- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f646c0ae840 msgr2=0x7f646c0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f646c0ae840 0x7f646c0b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 -- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f646c06bc50 msgr2=0x7f646c15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f646c06bc50 0x7f646c15ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 -- 172.21.15.110:0/3863744305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c15a0c0 con 0x7f646c15e690 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f646be00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f646c06bc50 0x7f646c15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:36.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6470a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c14ff60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f646400c3c0 tx=0x7f646400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6463e00640 1 -- 172.21.15.110:0/3863744305 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464011790 con 0x7f646c15e690 2024-09-10T10:02:36.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6463e00640 1 -- 172.21.15.110:0/3863744305 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6464017070 con 0x7f646c15e690 2024-09-10T10:02:36.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646c15a3a0 con 0x7f646c15e690 2024-09-10T10:02:36.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6463e00640 1 -- 172.21.15.110:0/3863744305 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464011b70 con 0x7f646c15e690 2024-09-10T10:02:36.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646c15a960 con 0x7f646c15e690 2024-09-10T10:02:36.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.196+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646c06c090 con 0x7f646c15e690 2024-09-10T10:02:36.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.200+0000 7f6463e00640 1 -- 172.21.15.110:0/3863744305 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f6464011d10 con 0x7f646c15e690 2024-09-10T10:02:36.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.200+0000 7f6463e00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f644c067a10 0x7f644c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.200+0000 7f6463e00640 1 -- 172.21.15.110:0/3863744305 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f6464085b40 con 0x7f646c15e690 2024-09-10T10:02:36.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.200+0000 7f646be00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f644c067a10 0x7f644c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.200+0000 7f646be00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f644c067a10 0x7f644c069ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f645c0ae000 tx=0x7f645c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.204+0000 7f6463e00640 1 -- 172.21.15.110:0/3863744305 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6464056560 con 0x7f646c15e690 2024-09-10T10:02:36.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.416+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f646c15db80 con 0x7f646c15e690 2024-09-10T10:02:36.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6463e00640 1 -- 172.21.15.110:0/3863744305 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f6464055e60 con 0x7f646c15e690 2024-09-10T10:02:36.422 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:02:36.422 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:02:36.422 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:02:36.422 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:02:36.423 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:02:36.423 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:02:36.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f644c067a10 msgr2=0x7f644c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f644c067a10 0x7f644c069ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f645c0ae000 tx=0x7f645c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 msgr2=0x7f646c14ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c14ff60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f646400c3c0 tx=0x7f646400c880 comp rx=0 tx=0).stop 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 shutdown_connections 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f644c067a10 0x7f644c069ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646c15e690 0x7f646c14ff60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f646c0ae840 0x7f646c0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 --2- 172.21.15.110:0/3863744305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f646c06bc50 0x7f646c15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 >> 172.21.15.110:0/3863744305 conn(0x7f646c0d1530 msgr2=0x7f646c158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 shutdown_connections 2024-09-10T10:02:36.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.420+0000 7f6472a00640 1 -- 172.21.15.110:0/3863744305 wait complete. 2024-09-10T10:02:36.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:36.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33b1e00640 1 Processor -- start 2024-09-10T10:02:36.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33b1e00640 1 -- start start 2024-09-10T10:02:36.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0739a0 0x7f33ac163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33b1e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33ac0706a0 con 0x7f33ac06eb30 2024-09-10T10:02:36.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33b1e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33ac0d10f0 con 0x7f33ac0d0cb0 2024-09-10T10:02:36.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33b1e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac15ee90 con 0x7f33ac0739a0 2024-09-10T10:02:36.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33aaa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f33ac06eb30 0x7f33ac071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52420/0 (socket says 172.21.15.110:52420) 2024-09-10T10:02:36.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33aaa00640 1 -- 172.21.15.110:0/3798948128 learned_addr learned my addr 172.21.15.110:0/3798948128 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:36.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33ab400640 1 --2- 172.21.15.110:0/3798948128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0739a0 0x7f33ac163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33a9600640 1 -- 172.21.15.110:0/3798948128 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185190729 0 0) 0x7f33ac0706a0 con 0x7f33ac06eb30 2024-09-10T10:02:36.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33a9600640 1 -- 172.21.15.110:0/3798948128 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3390003530 con 0x7f33ac06eb30 2024-09-10T10:02:36.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33a9600640 1 -- 172.21.15.110:0/3798948128 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2881162954 0 0) 0x7f3398001120 con 0x7f33ac0d0cb0 2024-09-10T10:02:36.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33a9600640 1 -- 172.21.15.110:0/3798948128 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33ac0706a0 con 0x7f33ac0d0cb0 2024-09-10T10:02:36.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33ab400640 1 -- 172.21.15.110:0/3798948128 >> v1:172.21.15.145:6789/0 conn(0x7f33ac0d0cb0 legacy=0x7f33ac073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33ab400640 1 -- 172.21.15.110:0/3798948128 >> v1:172.21.15.110:6789/0 conn(0x7f33ac06eb30 legacy=0x7f33ac071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.580+0000 7f33ab400640 1 -- 172.21.15.110:0/3798948128 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac0d1370 con 0x7f33ac0739a0 2024-09-10T10:02:36.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33ab400640 1 --2- 172.21.15.110:0/3798948128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0739a0 0x7f33ac163000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f33a40097f0 tx=0x7f33a401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b35e921f3e22764 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33a9600640 1 -- 172.21.15.110:0/3798948128 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a40b0070 con 0x7f33ac0739a0 2024-09-10T10:02:36.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33a9600640 1 -- 172.21.15.110:0/3798948128 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a40ab600 con 0x7f33ac0739a0 2024-09-10T10:02:36.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33a9600640 1 -- 172.21.15.110:0/3798948128 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a40ab9d0 con 0x7f33ac0739a0 2024-09-10T10:02:36.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33b1e00640 1 -- 172.21.15.110:0/3798948128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0739a0 msgr2=0x7f33ac163000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33b1e00640 1 --2- 172.21.15.110:0/3798948128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0739a0 0x7f33ac163000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f33a40097f0 tx=0x7f33a401c450 comp rx=0 tx=0).stop 2024-09-10T10:02:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33b1e00640 1 -- 172.21.15.110:0/3798948128 shutdown_connections 2024-09-10T10:02:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33b1e00640 1 --2- 172.21.15.110:0/3798948128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0739a0 0x7f33ac163000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33b1e00640 1 -- 172.21.15.110:0/3798948128 >> 172.21.15.110:0/3798948128 conn(0x7f33ac0d25c0 msgr2=0x7f33ac0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33b1e00640 1 -- 172.21.15.110:0/3798948128 shutdown_connections 2024-09-10T10:02:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.584+0000 7f33b1e00640 1 -- 172.21.15.110:0/3798948128 wait complete. 2024-09-10T10:02:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 Processor -- start 2024-09-10T10:02:36.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 -- start start 2024-09-10T10:02:36.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac06eb30 0x7f33ac153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33ac0739a0 0x7f33ac152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0d0cb0 0x7f33ac0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac06dd60 con 0x7f33ac06eb30 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac071c70 con 0x7f33ac0739a0 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac06dee0 con 0x7f33ac0d0cb0 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac06eb30 0x7f33ac153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac06eb30 0x7f33ac153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57262/0 (socket says 172.21.15.110:57262) 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 -- 172.21.15.110:0/461313315 learned_addr learned my addr 172.21.15.110:0/461313315 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33ab400640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0d0cb0 0x7f33ac0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aa000640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33ac0739a0 0x7f33ac152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 -- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0d0cb0 msgr2=0x7f33ac0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0d0cb0 0x7f33ac0cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 -- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33ac0739a0 msgr2=0x7f33ac152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33ac0739a0 0x7f33ac152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 -- 172.21.15.110:0/461313315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac156470 con 0x7f33ac06eb30 2024-09-10T10:02:36.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33aaa00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac06eb30 0x7f33ac153340 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f339c00cb20 tx=0x7f339c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33a3400640 1 -- 172.21.15.110:0/461313315 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c017070 con 0x7f33ac06eb30 2024-09-10T10:02:36.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33a3400640 1 -- 172.21.15.110:0/461313315 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c00aca0 con 0x7f33ac06eb30 2024-09-10T10:02:36.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33a3400640 1 -- 172.21.15.110:0/461313315 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c012600 con 0x7f33ac06eb30 2024-09-10T10:02:36.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac156750 con 0x7f33ac06eb30 2024-09-10T10:02:36.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.588+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac156d10 con 0x7f33ac06eb30 2024-09-10T10:02:36.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.592+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ac163000 con 0x7f33ac06eb30 2024-09-10T10:02:36.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.592+0000 7f33a3400640 1 -- 172.21.15.110:0/461313315 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f339c01a070 con 0x7f33ac06eb30 2024-09-10T10:02:36.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.592+0000 7f33a3400640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f338c0679e0 0x7f338c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.592+0000 7f33a3400640 1 -- 172.21.15.110:0/461313315 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f339c085b30 con 0x7f33ac06eb30 2024-09-10T10:02:36.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.592+0000 7f33aa000640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f338c0679e0 0x7f338c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.592+0000 7f33aa000640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f338c0679e0 0x7f338c069ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33ac1fac70 tx=0x7f3398009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.596+0000 7f33a3400640 1 -- 172.21.15.110:0/461313315 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f339c055e70 con 0x7f33ac06eb30 2024-09-10T10:02:36.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.784+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33ac16a4d0 con 0x7f33ac06eb30 2024-09-10T10:02:36.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33a3400640 1 -- 172.21.15.110:0/461313315 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5237 (secure 0 0 0) 0x7f339c05a170 con 0x7f33ac06eb30 2024-09-10T10:02:36.789 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:36.789 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":54,"btime":"2024-09-10T10:02:34:550856+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1164773100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1164773100},{"type":"v1","addr":"172.21.15.145:6834","nonce":1164773100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1195578364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1195578364},{"type":"v1","addr":"172.21.15.145:6835","nonce":1195578364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6447,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3463064995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3463064995},{"type":"v1","addr":"172.21.15.110:6834","nonce":3463064995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:02:30.912482+0000","modified":"2024-09-10T10:02:33.921168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1947643553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1947643553},{"type":"v1","addr":"172.21.15.110:6835","nonce":1947643553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":6}]} 2024-09-10T10:02:36.789 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 54 2024-09-10T10:02:36.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f338c0679e0 msgr2=0x7f338c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f338c0679e0 0x7f338c069ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33ac1fac70 tx=0x7f3398009040 comp rx=0 tx=0).stop 2024-09-10T10:02:36.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac06eb30 msgr2=0x7f33ac153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac06eb30 0x7f33ac153340 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f339c00cb20 tx=0x7f339c00a060 comp rx=0 tx=0).stop 2024-09-10T10:02:36.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 shutdown_connections 2024-09-10T10:02:36.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f338c0679e0 0x7f338c069ea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33ac0d0cb0 0x7f33ac0cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33ac0739a0 0x7f33ac152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 --2- 172.21.15.110:0/461313315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33ac06eb30 0x7f33ac153340 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 >> 172.21.15.110:0/461313315 conn(0x7f33ac0d25c0 msgr2=0x7f33ac0b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 shutdown_connections 2024-09-10T10:02:36.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:36.788+0000 7f33b1e00640 1 -- 172.21.15.110:0/461313315 wait complete. 2024-09-10T10:02:37.157 INFO:tasks.ceph.mds.a.smithi110.stderr:2024-09-10T10:02:37.156+0000 7f5674e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:02:37.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:02:30.912482+0000', 'modified': '2024-09-10T10:02:33.921168+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6453}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6453': {'gid': 6453, 'name': 'a', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6835/1947643553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6833', 'nonce': 1947643553}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 1947643553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6453, 'qdb_cluster': [6453]} 2024-09-10T10:02:37.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:02:37.214 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:02:37.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9fa0000640 1 Processor -- start 2024-09-10T10:02:37.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9fa0000640 1 -- start start 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9fa0000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9f9815e5d0 0x7f9f980ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9fa0000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f980ceb40 0x7f9f9806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9fa0000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f98162330 con 0x7f9f9806c190 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9fa0000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9f98162610 con 0x7f9f9815e5d0 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9fa0000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9f981628f0 con 0x7f9f980ceb40 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9d600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9f9815e5d0 0x7f9f980ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9e000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9f9806c190 0x7f9f980d13c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52426/0 (socket says 172.21.15.110:52426) 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9e000640 1 -- 172.21.15.110:0/1736675463 learned_addr learned my addr 172.21.15.110:0/1736675463 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9d600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9f9815e5d0 0x7f9f980ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57276/0 (socket says 172.21.15.110:57276) 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/1736675463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f980ceb40 0x7f9f9806bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9d600640 1 -- 172.21.15.110:0/1736675463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f980ceb40 msgr2=0x7f9f9806bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9d600640 1 --2- 172.21.15.110:0/1736675463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f980ceb40 0x7f9f9806bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9d600640 1 -- 172.21.15.110:0/1736675463 >> v1:172.21.15.110:6789/0 conn(0x7f9f9806c190 legacy=0x7f9f980d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9d600640 1 -- 172.21.15.110:0/1736675463 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f98162bd0 con 0x7f9f9815e5d0 2024-09-10T10:02:37.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.352+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/1736675463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f980ceb40 0x7f9f9806bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:37.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9f9d600640 1 --2- 172.21.15.110:0/1736675463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f980ce070 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009d90 tx=0x7f9f8c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e95a0aea8a6dea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9f97e00640 1 -- 172.21.15.110:0/1736675463 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0b2070 con 0x7f9f9815e5d0 2024-09-10T10:02:37.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9f97e00640 1 -- 172.21.15.110:0/1736675463 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0a57a0 con 0x7f9f9815e5d0 2024-09-10T10:02:37.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9f97e00640 1 -- 172.21.15.110:0/1736675463 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0a5b70 con 0x7f9f9815e5d0 2024-09-10T10:02:37.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9fa0000640 1 -- 172.21.15.110:0/1736675463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 msgr2=0x7f9f980ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9fa0000640 1 --2- 172.21.15.110:0/1736675463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f980ce070 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009d90 tx=0x7f9f8c01c8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9fa0000640 1 -- 172.21.15.110:0/1736675463 shutdown_connections 2024-09-10T10:02:37.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9fa0000640 1 --2- 172.21.15.110:0/1736675463 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f980ceb40 0x7f9f9806bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9fa0000640 1 --2- 172.21.15.110:0/1736675463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f980ce070 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9fa0000640 1 -- 172.21.15.110:0/1736675463 >> 172.21.15.110:0/1736675463 conn(0x7f9f9806cd60 msgr2=0x7f9f9806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9fa0000640 1 -- 172.21.15.110:0/1736675463 shutdown_connections 2024-09-10T10:02:37.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.356+0000 7f9fa0000640 1 -- 172.21.15.110:0/1736675463 wait complete. 2024-09-10T10:02:37.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 Processor -- start 2024-09-10T10:02:37.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 -- start start 2024-09-10T10:02:37.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9806c190 0x7f9f980b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f980ceb40 0x7f9f9815fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f98160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9816b7b0 con 0x7f9f9815e5d0 2024-09-10T10:02:37.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9816b630 con 0x7f9f980ceb40 2024-09-10T10:02:37.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9816b930 con 0x7f9f9806c190 2024-09-10T10:02:37.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f98160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9806c190 0x7f9f980b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f98160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57282/0 (socket says 172.21.15.110:57282) 2024-09-10T10:02:37.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 -- 172.21.15.110:0/407697622 learned_addr learned my addr 172.21.15.110:0/407697622 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:37.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f980ceb40 0x7f9f9815fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 -- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9806c190 msgr2=0x7f9f980b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9806c190 0x7f9f980b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 -- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f980ceb40 msgr2=0x7f9f9815fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f980ceb40 0x7f9f9815fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 -- 172.21.15.110:0/407697622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f981fbad0 con 0x7f9f9815e5d0 2024-09-10T10:02:37.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f9e000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f98160150 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c3c0 tx=0x7f9f9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f95600640 1 -- 172.21.15.110:0/407697622 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90017070 con 0x7f9f9815e5d0 2024-09-10T10:02:37.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f95600640 1 -- 172.21.15.110:0/407697622 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f90005cf0 con 0x7f9f9815e5d0 2024-09-10T10:02:37.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9f95600640 1 -- 172.21.15.110:0/407697622 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900072f0 con 0x7f9f9815e5d0 2024-09-10T10:02:37.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 -- 172.21.15.110:0/407697622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f980b1b80 con 0x7f9f9815e5d0 2024-09-10T10:02:37.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.360+0000 7f9fa0000640 1 -- 172.21.15.110:0/407697622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f980b2140 con 0x7f9f9815e5d0 2024-09-10T10:02:37.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.364+0000 7f9f95600640 1 -- 172.21.15.110:0/407697622 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f9f9001a070 con 0x7f9f9815e5d0 2024-09-10T10:02:37.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.364+0000 7f9f95600640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9f780679e0 0x7f9f78069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.364+0000 7f9f95600640 1 -- 172.21.15.110:0/407697622 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f9f90084ff0 con 0x7f9f9815e5d0 2024-09-10T10:02:37.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.364+0000 7f9f9d600640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9f780679e0 0x7f9f78069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.364+0000 7f9fa34e6000 1 Processor -- start 2024-09-10T10:02:37.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.364+0000 7f9fa34e6000 1 -- start start 2024-09-10T10:02:37.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.364+0000 7f9f9d600640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9f780679e0 0x7f9f78069ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0af000 tx=0x7f9f8c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d24f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 0x5588b0cdad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0cdb290 0x5588b0cdb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5588b0bc2b60 con 0x5588b0d24b20 2024-09-10T10:02:37.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5588b0b29c70 con 0x5588b0cdb290 2024-09-10T10:02:37.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5588b0acfed0 con 0x5588b0d25440 2024-09-10T10:02:37.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d24f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d24f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57294/0 (socket says 172.21.15.110:57294) 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 -- 172.21.15.110:0/3169726259 learned_addr learned my addr 172.21.15.110:0/3169726259 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9e000640 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0cdb290 0x5588b0cdb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 0x5588b0cdad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 -- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 msgr2=0x5588b0cdad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 0x5588b0cdad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 -- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0cdb290 msgr2=0x5588b0cdb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0cdb290 0x5588b0cdb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 -- 172.21.15.110:0/3169726259 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588b0ae1e50 con 0x5588b0d24b20 2024-09-10T10:02:37.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9e000640 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0cdb290 0x5588b0cdb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:37.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f9d600640 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d24f00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009d90 tx=0x7f9f8c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0eeadd727fef12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f8a000640 1 -- 172.21.15.110:0/3169726259 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0cb070 con 0x5588b0d24b20 2024-09-10T10:02:37.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f8a000640 1 -- 172.21.15.110:0/3169726259 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0b9720 con 0x5588b0d24b20 2024-09-10T10:02:37.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9f8a000640 1 -- 172.21.15.110:0/3169726259 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0b2040 con 0x5588b0d24b20 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- 172.21.15.110:0/3169726259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 msgr2=0x5588b0d24f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d24f00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009d90 tx=0x7f9f8c004ac0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- 172.21.15.110:0/3169726259 shutdown_connections 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0cdb290 0x5588b0cdb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 0x5588b0cdad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/3169726259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d24f00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- 172.21.15.110:0/3169726259 >> 172.21.15.110:0/3169726259 conn(0x5588b0ce1600 msgr2=0x5588b0cd9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- 172.21.15.110:0/3169726259 shutdown_connections 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- 172.21.15.110:0/3169726259 wait complete. 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 -- start start 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.368+0000 7f9fa34e6000 1 Processor -- start 2024-09-10T10:02:37.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 0x5588b0cdfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0ce5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0ce6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5588b0b8bb00 con 0x5588b0d25440 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5588b0b04360 con 0x5588b0ce0330 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5588b0b012d0 con 0x5588b0d24b20 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 10 client.0 _ll_register_callbacks cb 0x5588b0cc6820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0ce5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0ce5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57306/0 (socket says 172.21.15.110:57306) 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 -- 172.21.15.110:0/2358156278 learned_addr learned my addr 172.21.15.110:0/2358156278 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9d600640 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 0x5588b0cdfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9e000640 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0ce6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 -- 172.21.15.110:0/2358156278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 msgr2=0x5588b0cdfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 0x5588b0cdfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 -- 172.21.15.110:0/2358156278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 msgr2=0x5588b0ce6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0ce6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 -- 172.21.15.110:0/2358156278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588b0ad0120 con 0x5588b0d25440 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0ce5cf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9f84003d60 tx=0x7f9f8400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f78b8079f47240 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2358156278 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84019070 con 0x5588b0d25440 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2358156278 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f9f84015660 con 0x5588b0d25440 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2358156278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f8c0b9720 con 0x5588b0d25440 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2358156278 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9f84024050 con 0x5588b0d25440 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 10 client.6567 fetch_fsmap learned FSMap version 54 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2358156278 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5588b0ac9450 con 0x5588b0d25440 2024-09-10T10:02:37.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2358156278 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0b9720 con 0x5588b0d25440 2024-09-10T10:02:37.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2358156278 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f9f84027020 con 0x5588b0d25440 2024-09-10T10:02:37.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 10 client.6567 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T10:02:37.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 10 client.6567 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:02:37.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 20 client.6567 populate_metadata read hostname 'smithi110' 2024-09-10T10:02:37.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] conn(0x5588b0d33280 0x5588b0d35670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 4 client.6567 mds_command: new command op to 6453 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:37.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2358156278 --> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588b0ac3c00 con 0x5588b0d33280 2024-09-10T10:02:37.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9d600640 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] conn(0x5588b0d33280 0x5588b0d35670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f9d600640 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] conn(0x5588b0d33280 0x5588b0d35670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05bcb51f3978463 server_cookie=515c57d6c5a5c334 in_seq=0 out_seq=0 2024-09-10T10:02:37.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 10 client.6567 ms_handle_connect on v2:172.21.15.110:6833/1947643553 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2358156278 <== mds.0 v2:172.21.15.110:6833/1947643553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588b0ac3c00 con 0x5588b0d33280 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 10 client.6567 handle_command_reply: tid=0 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 client.6567 shutdown 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2358156278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 msgr2=0x5588b0ce5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0ce5cf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9f84003d60 tx=0x7f9f8400b8d0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2358156278 shutdown_connections 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] conn(0x5588b0d33280 0x5588b0d35670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0ce6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0ce5cf0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2358156278 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 0x5588b0cdfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9f7ea00640 0 client.6567 ms_handle_reset on v2:172.21.15.110:6833/1947643553 2024-09-10T10:02:37.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2358156278 >> 172.21.15.110:0/2358156278 conn(0x5588b0cdb290 msgr2=0x5588b0cd9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.372+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2358156278 shutdown_connections 2024-09-10T10:02:37.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.376+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2358156278 wait complete. 2024-09-10T10:02:37.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.376+0000 7f9fa34e6000 20 client.6567 trim_cache size 0 max 16384 2024-09-10T10:02:37.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9fa34e6000 1 Processor -- start 2024-09-10T10:02:37.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9fa34e6000 1 -- start start 2024-09-10T10:02:37.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0d23c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 0x5588b0d241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0d35670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98202f60 con 0x5588b0d25440 2024-09-10T10:02:37.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9f982022c0 con 0x5588b0ce0330 2024-09-10T10:02:37.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9f98202910 con 0x5588b0d24b20 2024-09-10T10:02:37.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0d35670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0d35670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57314/0 (socket says 172.21.15.110:57314) 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9e000640 1 -- 172.21.15.110:0/2008311874 learned_addr learned my addr 172.21.15.110:0/2008311874 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9d600640 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0d23c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 0x5588b0d241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9e000640 1 -- 172.21.15.110:0/2008311874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 msgr2=0x5588b0d241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9e000640 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 0x5588b0d241d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9e000640 1 -- 172.21.15.110:0/2008311874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 msgr2=0x5588b0d23c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9e000640 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0d23c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.392+0000 7f9f9e000640 1 -- 172.21.15.110:0/2008311874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588b0ab0fd0 con 0x5588b0d25440 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9e000640 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0d35670 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9f9005da60 tx=0x7f9f90055520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd4a7dfa5609977 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f89600640 1 -- 172.21.15.110:0/2008311874 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90081070 con 0x5588b0d25440 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f89600640 1 -- 172.21.15.110:0/2008311874 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9005df90 con 0x5588b0d25440 2024-09-10T10:02:37.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f89600640 1 -- 172.21.15.110:0/2008311874 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90083510 con 0x5588b0d25440 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2008311874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 msgr2=0x5588b0d35670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0d35670 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9f9005da60 tx=0x7f9f90055520 comp rx=0 tx=0).stop 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2008311874 shutdown_connections 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d25440 0x5588b0d35670 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d24b20 0x5588b0d241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2008311874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0d23c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2008311874 >> 172.21.15.110:0/2008311874 conn(0x5588b0cdb290 msgr2=0x5588b0cdf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2008311874 shutdown_connections 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2008311874 wait complete. 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- start start 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 Processor -- start 2024-09-10T10:02:37.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0d38500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 0x5588b0d53ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5588b0b28f80 con 0x5588b0d24b20 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5588b0ae1ca0 con 0x5588b0ce0330 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5588b0b2e2a0 con 0x5588b0d25440 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9fa34e6000 10 client.0 _ll_register_callbacks cb 0x7f9f84004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d53960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d53960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57326/0 (socket says 172.21.15.110:57326) 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 -- 172.21.15.110:0/2861830608 learned_addr learned my addr 172.21.15.110:0/2861830608 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9d600640 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0d38500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9e000640 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 0x5588b0d53ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 -- 172.21.15.110:0/2861830608 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 msgr2=0x5588b0d53ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 0x5588b0d53ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 -- 172.21.15.110:0/2861830608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 msgr2=0x5588b0d38500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0d38500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 -- 172.21.15.110:0/2861830608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588b0bac750 con 0x5588b0d24b20 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f9cc00640 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d53960 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9f84003d60 tx=0x7f9f840046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f5c1f6f94bd1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.396+0000 7f9f7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2861830608 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8402c070 con 0x5588b0d24b20 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2861830608 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f9f8401e440 con 0x5588b0d24b20 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2861830608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9005df90 con 0x5588b0d24b20 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2861830608 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9f84019070 con 0x5588b0d24b20 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 10 client.6573 fetch_fsmap learned FSMap version 54 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2861830608 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5588b0ab7a30 con 0x5588b0d24b20 2024-09-10T10:02:37.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2861830608 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9005df90 con 0x5588b0d24b20 2024-09-10T10:02:37.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2861830608 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f9f84027040 con 0x5588b0d24b20 2024-09-10T10:02:37.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 10 client.6573 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T10:02:37.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 10 client.6573 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:02:37.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 20 client.6573 populate_metadata read hostname 'smithi110' 2024-09-10T10:02:37.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] conn(0x5588b0d543e0 0x5588b0d97de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 4 client.6573 mds_command: new command op to 6453 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:02:37.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2861830608 --> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] -- command(tid 0: {"prefix": "status"}) -- 0x5588b0b373a0 con 0x5588b0d543e0 2024-09-10T10:02:37.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f9d600640 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] conn(0x5588b0d543e0 0x5588b0d97de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 10 client.6573 ms_handle_connect on v2:172.21.15.110:6833/1947643553 2024-09-10T10:02:37.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f9d600640 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] conn(0x5588b0d543e0 0x5588b0d97de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b9867795703705b server_cookie=6be544b4826ff098 in_seq=0 out_seq=0 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 1 -- 172.21.15.110:0/2861830608 <== mds.0 v2:172.21.15.110:6833/1947643553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5588b0b373a0 con 0x5588b0d543e0 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 10 client.6573 handle_command_reply: tid=0 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 client.6573 shutdown 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2861830608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 msgr2=0x5588b0d53960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d53960 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9f84003d60 tx=0x7f9f840046d0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2861830608 shutdown_connections 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.110:6833/1947643553,v1:172.21.15.110:6835/1947643553] conn(0x5588b0d543e0 0x5588b0d97de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5588b0d25440 0x5588b0d53ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588b0d24b20 0x5588b0d53960 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 --2- 172.21.15.110:0/2861830608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588b0ce0330 0x5588b0d38500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9f7ea00640 0 client.6573 ms_handle_reset on v2:172.21.15.110:6833/1947643553 2024-09-10T10:02:37.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2861830608 >> 172.21.15.110:0/2861830608 conn(0x5588b0cdb290 msgr2=0x5588b0d22c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2861830608 shutdown_connections 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 1 -- 172.21.15.110:0/2861830608 wait complete. 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa34e6000 20 client.6573 trim_cache size 0 max 16384 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 6453, 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:02:37.408 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:02:37.409 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:02:37.409 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 5.8772073220000003, 2024-09-10T10:02:37.409 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 54, 2024-09-10T10:02:37.409 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 84, 2024-09-10T10:02:37.409 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 84, 2024-09-10T10:02:37.409 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 11.105826465 2024-09-10T10:02:37.409 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 -- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9f780679e0 msgr2=0x7f9f78069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9f780679e0 0x7f9f78069ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0af000 tx=0x7f9f8c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 -- 172.21.15.110:0/407697622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 msgr2=0x7f9f98160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f98160150 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c3c0 tx=0x7f9f9000c880 comp rx=0 tx=0).stop 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 -- 172.21.15.110:0/407697622 shutdown_connections 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9f780679e0 0x7f9f78069ea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f9815e5d0 0x7f9f98160150 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f980ceb40 0x7f9f9815fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 --2- 172.21.15.110:0/407697622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9f9806c190 0x7f9f980b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 -- 172.21.15.110:0/407697622 >> 172.21.15.110:0/407697622 conn(0x7f9f9806cd60 msgr2=0x7f9f980aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 -- 172.21.15.110:0/407697622 shutdown_connections 2024-09-10T10:02:37.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.400+0000 7f9fa0000640 1 -- 172.21.15.110:0/407697622 wait complete. 2024-09-10T10:02:37.422 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:02:37.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc67ce00640 1 Processor -- start 2024-09-10T10:02:37.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc67ce00640 1 -- start start 2024-09-10T10:02:37.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc67ce00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc67806f1e0 0x7fc678166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc67ce00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc678155d50 con 0x7fc678155970 2024-09-10T10:02:37.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc67ce00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6780b34d0 con 0x7fc67815c560 2024-09-10T10:02:37.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc67ce00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc6780b4570 con 0x7fc67806f1e0 2024-09-10T10:02:37.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc675e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc678155970 0x7fc6780cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52432/0 (socket says 172.21.15.110:52432) 2024-09-10T10:02:37.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc675e00640 1 -- 172.21.15.110:0/1770937700 learned_addr learned my addr 172.21.15.110:0/1770937700 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:37.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc676800640 1 --2- 172.21.15.110:0/1770937700 >> v2:172.21.15.145:3300/0 conn(0x7fc67806f1e0 0x7fc678166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc674a00640 1 -- 172.21.15.110:0/1770937700 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990280613 0 0) 0x7fc678155d50 con 0x7fc678155970 2024-09-10T10:02:37.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc674a00640 1 -- 172.21.15.110:0/1770937700 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc65c003530 con 0x7fc678155970 2024-09-10T10:02:37.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc676800640 1 -- 172.21.15.110:0/1770937700 >> v1:172.21.15.145:6789/0 conn(0x7fc67815c560 legacy=0x7fc67806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc676800640 1 -- 172.21.15.110:0/1770937700 >> v1:172.21.15.110:6789/0 conn(0x7fc678155970 legacy=0x7fc6780cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc674a00640 1 -- 172.21.15.110:0/1770937700 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1502137051 0 0) 0x7fc65c003530 con 0x7fc678155970 2024-09-10T10:02:37.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.556+0000 7fc676800640 1 -- 172.21.15.110:0/1770937700 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6780b37b0 con 0x7fc67806f1e0 2024-09-10T10:02:37.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.560+0000 7fc676800640 1 --2- 172.21.15.110:0/1770937700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc67806f1e0 0x7fc678166020 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc6700097c0 tx=0x7fc67001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb8206259cc879b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.560+0000 7fc674a00640 1 -- 172.21.15.110:0/1770937700 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6700b1070 con 0x7fc67806f1e0 2024-09-10T10:02:37.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.560+0000 7fc674a00640 1 -- 172.21.15.110:0/1770937700 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6700ac440 con 0x7fc67806f1e0 2024-09-10T10:02:37.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.560+0000 7fc674a00640 1 -- 172.21.15.110:0/1770937700 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668001a00 con 0x7fc67806f1e0 2024-09-10T10:02:37.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.560+0000 7fc67ce00640 1 -- 172.21.15.110:0/1770937700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc67806f1e0 msgr2=0x7fc678166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.560+0000 7fc67ce00640 1 --2- 172.21.15.110:0/1770937700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc67806f1e0 0x7fc678166020 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc6700097c0 tx=0x7fc67001c390 comp rx=0 tx=0).stop 2024-09-10T10:02:37.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 -- 172.21.15.110:0/1770937700 shutdown_connections 2024-09-10T10:02:37.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 --2- 172.21.15.110:0/1770937700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc67806f1e0 0x7fc678166020 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 -- 172.21.15.110:0/1770937700 >> 172.21.15.110:0/1770937700 conn(0x7fc67806bc50 msgr2=0x7fc67806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 -- 172.21.15.110:0/1770937700 shutdown_connections 2024-09-10T10:02:37.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 -- 172.21.15.110:0/1770937700 wait complete. 2024-09-10T10:02:37.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 Processor -- start 2024-09-10T10:02:37.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 -- start start 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67806f1e0 0x7fc67815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc678155970 0x7fc678157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc67815c560 0x7fc67806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc678168f00 con 0x7fc67806f1e0 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc678168d80 con 0x7fc678155970 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc67ce00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc678169080 con 0x7fc67815c560 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc675e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67806f1e0 0x7fc67815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc675e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67806f1e0 0x7fc67815ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57332/0 (socket says 172.21.15.110:57332) 2024-09-10T10:02:37.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc675e00640 1 -- 172.21.15.110:0/228908533 learned_addr learned my addr 172.21.15.110:0/228908533 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc675400640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc678155970 0x7fc678157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.564+0000 7fc676800640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc67815c560 0x7fc67806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc675e00640 1 -- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc67815c560 msgr2=0x7fc67806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc675e00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc67815c560 0x7fc67806e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc675e00640 1 -- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc678155970 msgr2=0x7fc678157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc675e00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc678155970 0x7fc678157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc675e00640 1 -- 172.21.15.110:0/228908533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc678153650 con 0x7fc67806f1e0 2024-09-10T10:02:37.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc675e00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67806f1e0 0x7fc67815ef60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc668001fe0 tx=0x7fc66800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc66e000640 1 -- 172.21.15.110:0/228908533 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668012400 con 0x7fc67806f1e0 2024-09-10T10:02:37.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc66e000640 1 -- 172.21.15.110:0/228908533 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6680125a0 con 0x7fc67806f1e0 2024-09-10T10:02:37.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc678153930 con 0x7fc67806f1e0 2024-09-10T10:02:37.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc66e000640 1 -- 172.21.15.110:0/228908533 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668012970 con 0x7fc67806f1e0 2024-09-10T10:02:37.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc678153ef0 con 0x7fc67806f1e0 2024-09-10T10:02:37.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6780cc8c0 con 0x7fc67806f1e0 2024-09-10T10:02:37.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc66e000640 1 -- 172.21.15.110:0/228908533 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fc668012b10 con 0x7fc67806f1e0 2024-09-10T10:02:37.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc66e000640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc658067a10 0x7fc658069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc66e000640 1 -- 172.21.15.110:0/228908533 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fc6680865f0 con 0x7fc67806f1e0 2024-09-10T10:02:37.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.568+0000 7fc675400640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc658067a10 0x7fc658069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.572+0000 7fc675400640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc658067a10 0x7fc658069ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc67815f4a0 tx=0x7fc664009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.572+0000 7fc66e000640 1 -- 172.21.15.110:0/228908533 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc6680569c0 con 0x7fc67806f1e0 2024-09-10T10:02:37.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.764+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc678150470 con 0x7fc67806f1e0 2024-09-10T10:02:37.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc66e000640 1 -- 172.21.15.110:0/228908533 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7fc66805acc0 con 0x7fc67806f1e0 2024-09-10T10:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:02:37.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc658067a10 msgr2=0x7fc658069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc658067a10 0x7fc658069ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc67815f4a0 tx=0x7fc664009040 comp rx=0 tx=0).stop 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67806f1e0 msgr2=0x7fc67815ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67806f1e0 0x7fc67815ef60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc668001fe0 tx=0x7fc66800a430 comp rx=0 tx=0).stop 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 shutdown_connections 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc658067a10 0x7fc658069ed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc67815c560 0x7fc67806e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc678155970 0x7fc678157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 --2- 172.21.15.110:0/228908533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67806f1e0 0x7fc67815ef60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 >> 172.21.15.110:0/228908533 conn(0x7fc67806bc50 msgr2=0x7fc6781515a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 shutdown_connections 2024-09-10T10:02:37.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:37.768+0000 7fc67ce00640 1 -- 172.21.15.110:0/228908533 wait complete. 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:02:37.790 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:02:37.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:37.791 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42217sec preferred_lft 42217sec 2024-09-10T10:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:37.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:37.803 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:02:37.858 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:02:37.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:37.860 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:02:37.920 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:02:37.920 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:02:37.920 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:02:37.920 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:02:37.921 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:02:37.921 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:02:37.921 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:02:37.921 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:37.921 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:37.921 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:37.921 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:02:37.921 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:02:37.921 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:02:37.921 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:02:37.921 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:02:37.922 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:02:37.922 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:02:38.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:02:38.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:02:38.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:38.053 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:02:38.061 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:02:38.062 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:02:38.062 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:02:38.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:38.063 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:02:38.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:02:38.124 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:38.172 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:02:38.173 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:02:38.173 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:38.224 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:02:38.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:38.224 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:38.312 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:38.312 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:02:38.325 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:38.325 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:02:38.326 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:02:38.326 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:02:38.326 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:02:38.329 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:02:38.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:38.330 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:38.392 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:02:38.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:02:38.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:02:38.496 INFO:teuthology.orchestra.run.smithi110.stdout:client5741 2024-09-10T10:02:38.496 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:38.497 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:38.505 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:02:38.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.684+0000 7fe814a00640 1 Processor -- start 2024-09-10T10:02:38.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.684+0000 7fe814a00640 1 -- start start 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe814a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe81015cb50 0x7fe810072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe814a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe8100728a0 0x7fe81015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe814a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe81015d470 0x7fe81006ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe814a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe810072ce0 con 0x7fe81015cb50 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe814a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe81006eab0 con 0x7fe8100728a0 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe814a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe810071710 con 0x7fe81015d470 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe80e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe81015cb50 0x7fe810072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe80e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe81015cb50 0x7fe810072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45796/0 (socket says 172.21.15.110:45796) 2024-09-10T10:02:38.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe80e800640 1 -- 172.21.15.110:0/3860707981 learned_addr learned my addr 172.21.15.110:0/3860707981 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:38.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe80e800640 1 -- 172.21.15.110:0/3860707981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe81015d470 msgr2=0x7fe81006ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:38.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe80e800640 1 --2- 172.21.15.110:0/3860707981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe81015d470 0x7fe81006ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe80e800640 1 -- 172.21.15.110:0/3860707981 >> v2:172.21.15.145:3300/0 conn(0x7fe8100728a0 msgr2=0x7fe81015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:38.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe80e800640 1 --2- 172.21.15.110:0/3860707981 >> v2:172.21.15.145:3300/0 conn(0x7fe8100728a0 0x7fe81015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.688+0000 7fe80e800640 1 -- 172.21.15.110:0/3860707981 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81006e180 con 0x7fe81015cb50 2024-09-10T10:02:38.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe80e800640 1 --2- 172.21.15.110:0/3860707981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe810072360 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe8100d0600 tx=0x7fe80001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331a01731442fbf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:38.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe80d400640 1 -- 172.21.15.110:0/3860707981 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8000a5840 con 0x7fe81015cb50 2024-09-10T10:02:38.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe80d400640 1 -- 172.21.15.110:0/3860707981 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8000a59e0 con 0x7fe81015cb50 2024-09-10T10:02:38.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe80d400640 1 -- 172.21.15.110:0/3860707981 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8000a5db0 con 0x7fe81015cb50 2024-09-10T10:02:38.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 -- 172.21.15.110:0/3860707981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 msgr2=0x7fe810072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:38.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 --2- 172.21.15.110:0/3860707981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe810072360 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe8100d0600 tx=0x7fe80001c9c0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 -- 172.21.15.110:0/3860707981 shutdown_connections 2024-09-10T10:02:38.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 --2- 172.21.15.110:0/3860707981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe81015d470 0x7fe81006ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 --2- 172.21.15.110:0/3860707981 >> v2:172.21.15.145:3300/0 conn(0x7fe8100728a0 0x7fe81015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 --2- 172.21.15.110:0/3860707981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe810072360 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 -- 172.21.15.110:0/3860707981 >> 172.21.15.110:0/3860707981 conn(0x7fe8100b23e0 msgr2=0x7fe8100b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:38.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 -- 172.21.15.110:0/3860707981 shutdown_connections 2024-09-10T10:02:38.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 -- 172.21.15.110:0/3860707981 wait complete. 2024-09-10T10:02:38.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 Processor -- start 2024-09-10T10:02:38.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.692+0000 7fe814a00640 1 -- start start 2024-09-10T10:02:38.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe8100728a0 0x7fe8100cff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:38.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe81014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:38.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe81015d470 0x7fe810150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:38.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8100ce530 con 0x7fe81015cb50 2024-09-10T10:02:38.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe810072f00 con 0x7fe81015d470 2024-09-10T10:02:38.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe8100ce6b0 con 0x7fe8100728a0 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe81014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80e800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe8100728a0 0x7fe8100cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe81014fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45808/0 (socket says 172.21.15.110:45808) 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 -- 172.21.15.110:0/3425674431 learned_addr learned my addr 172.21.15.110:0/3425674431 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80f200640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe81015d470 0x7fe810150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 -- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe8100728a0 msgr2=0x7fe8100cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe8100728a0 0x7fe8100cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 -- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe81015d470 msgr2=0x7fe810150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe81015d470 0x7fe810150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 -- 172.21.15.110:0/3425674431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8100d04a0 con 0x7fe81015cb50 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe80de00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe81014fff0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00b440 tx=0x7fe7fc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe806a00640 1 -- 172.21.15.110:0/3425674431 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc015020 con 0x7fe81015cb50 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe806a00640 1 -- 172.21.15.110:0/3425674431 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7fc0118b0 con 0x7fe81015cb50 2024-09-10T10:02:38.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81014d120 con 0x7fe81015cb50 2024-09-10T10:02:38.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe806a00640 1 -- 172.21.15.110:0/3425674431 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc011ca0 con 0x7fe81015cb50 2024-09-10T10:02:38.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81014d6e0 con 0x7fe81015cb50 2024-09-10T10:02:38.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe810071ed0 con 0x7fe81015cb50 2024-09-10T10:02:38.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.696+0000 7fe806a00640 1 -- 172.21.15.110:0/3425674431 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fe7fc018020 con 0x7fe81015cb50 2024-09-10T10:02:38.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.700+0000 7fe806a00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7f00679e0 0x7fe7f0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:38.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.700+0000 7fe806a00640 1 -- 172.21.15.110:0/3425674431 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fe7fc085d90 con 0x7fe81015cb50 2024-09-10T10:02:38.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.700+0000 7fe80e800640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7f00679e0 0x7fe7f0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:38.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.700+0000 7fe80e800640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7f00679e0 0x7fe7f0069ea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe8000ae000 tx=0x7fe8000a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:38.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.704+0000 7fe806a00640 1 -- 172.21.15.110:0/3425674431 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe7fc01f020 con 0x7fe81015cb50 2024-09-10T10:02:38.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.892+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe81015c7b0 con 0x7fe81015cb50 2024-09-10T10:02:38.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.892+0000 7fe806a00640 1 -- 172.21.15.110:0/3425674431 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7fe7fc00d040 con 0x7fe81015cb50 2024-09-10T10:02:38.897 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7f00679e0 msgr2=0x7fe7f0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7f00679e0 0x7fe7f0069ea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe8000ae000 tx=0x7fe8000a7070 comp rx=0 tx=0).stop 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 msgr2=0x7fe81014fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe81014fff0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00b440 tx=0x7fe7fc00b900 comp rx=0 tx=0).stop 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 shutdown_connections 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7f00679e0 0x7fe7f0069ea0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe81015d470 0x7fe810150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe81015cb50 0x7fe81014fff0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 --2- 172.21.15.110:0/3425674431 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe8100728a0 0x7fe8100cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:38.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 >> 172.21.15.110:0/3425674431 conn(0x7fe8100b23e0 msgr2=0x7fe81015b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:38.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 shutdown_connections 2024-09-10T10:02:38.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:38.896+0000 7fe814a00640 1 -- 172.21.15.110:0/3425674431 wait complete. 2024-09-10T10:02:38.917 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:02:38.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:02:38.917 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:02:38.917 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:02:38.917 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:38.917 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:02:38.917 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:02:38.918 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:02:38.918 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:02:38.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:38.918 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:02:38.928 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:02:38.928 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:02:38.928 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:02:38.928 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:38.928 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:02:38.928 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:38.928 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42212sec preferred_lft 42212sec 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:02:38.929 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:38.930 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:02:38.931 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:02:38.931 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:02:38.931 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:02:38.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:38.932 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:02:38.994 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:02:38.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:38.996 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:39.055 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:39.056 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:02:39.056 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:02:39.056 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:02:39.056 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:02:39.056 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:02:39.056 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:02:39.056 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:02:39.056 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:02:39.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:02:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:02:39.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:02:39.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:39.205 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:02:39.214 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:02:39.214 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:02:39.214 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:02:39.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:39.215 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:02:39.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:02:39.278 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:39.330 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:02:39.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:02:39.331 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:39.385 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:02:39.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:39.386 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:39.474 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:39.474 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:02:39.488 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:02:39.488 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:02:39.488 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:02:39.488 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:02:39.488 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:02:39.492 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:02:39.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:39.493 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:39.512 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:02:39.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:02:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:02:39.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:02:39.615 INFO:teuthology.orchestra.run.smithi145.stdout:client6280 2024-09-10T10:02:39.616 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:39.616 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:39.624 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:02:39.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e83e00640 1 Processor -- start 2024-09-10T10:02:39.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e83e00640 1 -- start start 2024-09-10T10:02:39.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e83e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8e8415e830 0x7f8e840ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e83e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e840aed80 0x7f8e840b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e83e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e840af160 con 0x7f8e8415e450 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e83e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8e84073290 con 0x7f8e8415e830 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e83e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8406e5e0 con 0x7f8e840aed80 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e82000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8e8415e830 0x7f8e840ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e82a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f8e8415e450 0x7f8e84072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37506/0 (socket says 172.21.15.110:37506) 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e82a00640 1 -- 172.21.15.110:0/336125873 learned_addr learned my addr 172.21.15.110:0/336125873 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e83400640 1 --2- 172.21.15.110:0/336125873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e840aed80 0x7f8e840b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e82000640 1 -- 172.21.15.110:0/336125873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e840aed80 msgr2=0x7f8e840b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e82000640 1 --2- 172.21.15.110:0/336125873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e840aed80 0x7f8e840b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e81600640 1 -- 172.21.15.110:0/336125873 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3607703404 0 0) 0x7f8e840af160 con 0x7f8e8415e450 2024-09-10T10:02:39.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e82000640 1 -- 172.21.15.110:0/336125873 >> v1:172.21.15.110:6789/0 conn(0x7f8e8415e450 legacy=0x7f8e84072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.764+0000 7f8e82000640 1 -- 172.21.15.110:0/336125873 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8406e8c0 con 0x7f8e8415e830 2024-09-10T10:02:39.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e82000640 1 --2- 172.21.15.110:0/336125873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8415e830 0x7f8e840ae840 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8e700097c0 tx=0x7f8e7001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc99918119dd292d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:39.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e81600640 1 -- 172.21.15.110:0/336125873 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e700a7070 con 0x7f8e8415e830 2024-09-10T10:02:39.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e81600640 1 -- 172.21.15.110:0/336125873 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e700b0070 con 0x7f8e8415e830 2024-09-10T10:02:39.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e81600640 1 -- 172.21.15.110:0/336125873 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e700ab890 con 0x7f8e8415e830 2024-09-10T10:02:39.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e83e00640 1 -- 172.21.15.110:0/336125873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8415e830 msgr2=0x7f8e840ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e83e00640 1 --2- 172.21.15.110:0/336125873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8415e830 0x7f8e840ae840 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8e700097c0 tx=0x7f8e7001c470 comp rx=0 tx=0).stop 2024-09-10T10:02:39.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e83e00640 1 -- 172.21.15.110:0/336125873 shutdown_connections 2024-09-10T10:02:39.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e83e00640 1 --2- 172.21.15.110:0/336125873 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e840aed80 0x7f8e840b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e83e00640 1 --2- 172.21.15.110:0/336125873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e8415e830 0x7f8e840ae840 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e83e00640 1 -- 172.21.15.110:0/336125873 >> 172.21.15.110:0/336125873 conn(0x7f8e8406cd60 msgr2=0x7f8e8406d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:39.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e83e00640 1 -- 172.21.15.110:0/336125873 shutdown_connections 2024-09-10T10:02:39.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.768+0000 7f8e83e00640 1 -- 172.21.15.110:0/336125873 wait complete. 2024-09-10T10:02:39.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 Processor -- start 2024-09-10T10:02:39.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 -- start start 2024-09-10T10:02:39.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e840aed80 0x7f8e8414a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e8415e450 0x7f8e8415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e8415e830 0x7f8e8415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8406e3c0 con 0x7f8e840aed80 2024-09-10T10:02:39.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8406e240 con 0x7f8e8415e450 2024-09-10T10:02:39.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8e840b52a0 con 0x7f8e8415e830 2024-09-10T10:02:39.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e840aed80 0x7f8e8414a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e840aed80 0x7f8e8414a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45824/0 (socket says 172.21.15.110:45824) 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 -- 172.21.15.110:0/2265683685 learned_addr learned my addr 172.21.15.110:0/2265683685 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 -- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e8415e830 msgr2=0x7f8e8415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e8415e830 0x7f8e8415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 -- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e8415e450 msgr2=0x7f8e8415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e8415e450 0x7f8e8415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 -- 172.21.15.110:0/2265683685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e84158f80 con 0x7f8e840aed80 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e82a00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e840aed80 0x7f8e8414a380 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8e74001570 tx=0x7f8e7400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e7aa00640 1 -- 172.21.15.110:0/2265683685 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74018070 con 0x7f8e840aed80 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e7aa00640 1 -- 172.21.15.110:0/2265683685 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7400a5e0 con 0x7f8e840aed80 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e84159260 con 0x7f8e840aed80 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e7aa00640 1 -- 172.21.15.110:0/2265683685 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400a9d0 con 0x7f8e840aed80 2024-09-10T10:02:39.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.772+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8415ab10 con 0x7f8e840aed80 2024-09-10T10:02:39.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.776+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8415a620 con 0x7f8e840aed80 2024-09-10T10:02:39.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.776+0000 7f8e7aa00640 1 -- 172.21.15.110:0/2265683685 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f8e7401b070 con 0x7f8e840aed80 2024-09-10T10:02:39.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.776+0000 7f8e7aa00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8e640679e0 0x7f8e64069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.776+0000 7f8e7aa00640 1 -- 172.21.15.110:0/2265683685 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f8e740854d0 con 0x7f8e840aed80 2024-09-10T10:02:39.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.776+0000 7f8e82000640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8e640679e0 0x7f8e64069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:39.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.776+0000 7f8e82000640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8e640679e0 0x7f8e64069ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e700af000 tx=0x7f8e70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:39.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.780+0000 7f8e7aa00640 1 -- 172.21.15.110:0/2265683685 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8e74055f00 con 0x7f8e840aed80 2024-09-10T10:02:39.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:39.972+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f8e840b5700 con 0x7f8e840aed80 2024-09-10T10:02:40.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e7aa00640 1 -- 172.21.15.110:0/2265683685 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v55) ==== 136+0+0 (secure 0 0 0) 0x7f8e74083e20 con 0x7f8e840aed80 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8e640679e0 msgr2=0x7f8e64069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8e640679e0 0x7f8e64069ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e700af000 tx=0x7f8e70008040 comp rx=0 tx=0).stop 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e840aed80 msgr2=0x7f8e8414a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e840aed80 0x7f8e8414a380 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8e74001570 tx=0x7f8e7400cd20 comp rx=0 tx=0).stop 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 shutdown_connections 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8e640679e0 0x7f8e64069ea0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8e8415e830 0x7f8e8415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e8415e450 0x7f8e8415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 --2- 172.21.15.110:0/2265683685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e840aed80 0x7f8e8414a380 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 >> 172.21.15.110:0/2265683685 conn(0x7f8e8406cd60 msgr2=0x7f8e8414c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 shutdown_connections 2024-09-10T10:02:40.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.580+0000 7f8e83e00640 1 -- 172.21.15.110:0/2265683685 wait complete. 2024-09-10T10:02:40.605 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:02:40.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.740+0000 7f4067000640 1 Processor -- start 2024-09-10T10:02:40.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.740+0000 7f4067000640 1 -- start start 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.740+0000 7f4067000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f40600cf270 0x7f40600b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.740+0000 7f4067000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f406015b4a0 0x7f40600d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.740+0000 7f4067000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4060073200 con 0x7f40600cf650 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.740+0000 7f4067000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f40600734e0 con 0x7f40600cf270 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.740+0000 7f4067000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f40600cff40 con 0x7f406015b4a0 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.740+0000 7f4065c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f40600cf270 0x7f40600b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4066600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f40600cf650 0x7f4060071fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37516/0 (socket says 172.21.15.110:37516) 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4066600640 1 -- 172.21.15.110:0/4061563470 learned_addr learned my addr 172.21.15.110:0/4061563470 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4065200640 1 --2- 172.21.15.110:0/4061563470 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f406015b4a0 0x7f40600d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4065c00640 1 -- 172.21.15.110:0/4061563470 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f406015b4a0 msgr2=0x7f40600d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4065c00640 1 --2- 172.21.15.110:0/4061563470 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f406015b4a0 0x7f40600d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4065c00640 1 -- 172.21.15.110:0/4061563470 >> v1:172.21.15.110:6789/0 conn(0x7f40600cf650 legacy=0x7f4060071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4065c00640 1 -- 172.21.15.110:0/4061563470 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40600d0220 con 0x7f40600cf270 2024-09-10T10:02:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4065200640 1 --2- 172.21.15.110:0/4061563470 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f406015b4a0 0x7f40600d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f4065c00640 1 --2- 172.21.15.110:0/4061563470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf270 0x7f40600b0af0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f40540097c0 tx=0x7f405401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba4d829f844aa35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f405fe00640 1 -- 172.21.15.110:0/4061563470 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540b0070 con 0x7f40600cf270 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f405fe00640 1 -- 172.21.15.110:0/4061563470 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40540ab600 con 0x7f40600cf270 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.744+0000 7f405fe00640 1 -- 172.21.15.110:0/4061563470 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540ab9d0 con 0x7f40600cf270 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- 172.21.15.110:0/4061563470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf270 msgr2=0x7f40600b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 --2- 172.21.15.110:0/4061563470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf270 0x7f40600b0af0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f40540097c0 tx=0x7f405401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- 172.21.15.110:0/4061563470 shutdown_connections 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 --2- 172.21.15.110:0/4061563470 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f406015b4a0 0x7f40600d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 --2- 172.21.15.110:0/4061563470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf270 0x7f40600b0af0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- 172.21.15.110:0/4061563470 >> 172.21.15.110:0/4061563470 conn(0x7f406006cd60 msgr2=0x7f406006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- 172.21.15.110:0/4061563470 shutdown_connections 2024-09-10T10:02:40.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- 172.21.15.110:0/4061563470 wait complete. 2024-09-10T10:02:40.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 Processor -- start 2024-09-10T10:02:40.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- start start 2024-09-10T10:02:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40600cf270 0x7f406013b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf650 0x7f40600aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40600aa970 0x7f4060150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4060168170 con 0x7f40600cf650 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4060167ff0 con 0x7f40600aa970 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4067000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f40601682f0 con 0x7f40600cf270 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf650 0x7f40600aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf650 0x7f40600aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45834/0 (socket says 172.21.15.110:45834) 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 -- 172.21.15.110:0/591077735 learned_addr learned my addr 172.21.15.110:0/591077735 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065c00640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40600cf270 0x7f406013b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 -- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40600cf270 msgr2=0x7f406013b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40600cf270 0x7f406013b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 -- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40600aa970 msgr2=0x7f4060150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4066600640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40600aa970 0x7f4060150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40600aa970 0x7f4060150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 -- 172.21.15.110:0/591077735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406013be50 con 0x7f40600cf650 2024-09-10T10:02:40.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065c00640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40600cf270 0x7f406013b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:40.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4066600640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40600aa970 0x7f4060150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:40.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.748+0000 7f4065200640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf650 0x7f40600aa430 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f404c00e840 tx=0x7f404c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f405d600640 1 -- 172.21.15.110:0/591077735 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c017070 con 0x7f40600cf650 2024-09-10T10:02:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f405d600640 1 -- 172.21.15.110:0/591077735 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c00c8d0 con 0x7f40600cf650 2024-09-10T10:02:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f405d600640 1 -- 172.21.15.110:0/591077735 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c00f040 con 0x7f40600cf650 2024-09-10T10:02:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4060151ba0 con 0x7f40600cf650 2024-09-10T10:02:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406013c2a0 con 0x7f40600cf650 2024-09-10T10:02:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40600aad80 con 0x7f40600cf650 2024-09-10T10:02:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f405d600640 1 -- 172.21.15.110:0/591077735 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f404c01a070 con 0x7f40600cf650 2024-09-10T10:02:40.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f405d600640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f40400679e0 0x7f4040069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f405d600640 1 -- 172.21.15.110:0/591077735 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f404c085bb0 con 0x7f40600cf650 2024-09-10T10:02:40.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.752+0000 7f4065c00640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f40400679e0 0x7f4040069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.756+0000 7f4065c00640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f40400679e0 0x7f4040069ea0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40540ad000 tx=0x7f4054008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.756+0000 7f405d600640 1 -- 172.21.15.110:0/591077735 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f40600aad80 con 0x7f40600cf650 2024-09-10T10:02:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.936+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f40600b0af0 con 0x7f40400679e0 2024-09-10T10:02:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.936+0000 7f405d600640 1 -- 172.21.15.110:0/591077735 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f40600b0af0 con 0x7f40400679e0 2024-09-10T10:02:40.939 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:02:40.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:02:40.940 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:02:40.940 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:02:40.940 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:02:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f40400679e0 msgr2=0x7f4040069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f40400679e0 0x7f4040069ea0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40540ad000 tx=0x7f4054008040 comp rx=0 tx=0).stop 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf650 msgr2=0x7f40600aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf650 0x7f40600aa430 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f404c00e840 tx=0x7f404c00ed00 comp rx=0 tx=0).stop 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 shutdown_connections 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f40400679e0 0x7f4040069ea0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40600aa970 0x7f4060150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40600cf650 0x7f40600aa430 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 --2- 172.21.15.110:0/591077735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40600cf270 0x7f406013b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 >> 172.21.15.110:0/591077735 conn(0x7f406006cd60 msgr2=0x7f40600b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 shutdown_connections 2024-09-10T10:02:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:40.940+0000 7f4067000640 1 -- 172.21.15.110:0/591077735 wait complete. 2024-09-10T10:02:40.960 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:02:41.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54c600640 1 Processor -- start 2024-09-10T10:02:41.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54c600640 1 -- start start 2024-09-10T10:02:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54c600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd54406e850 0x7fd54406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54c600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5440b4960 con 0x7fd54415b330 2024-09-10T10:02:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54c600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5440b4c40 con 0x7fd5440ce4d0 2024-09-10T10:02:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54c600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd5440b5570 con 0x7fd54406e850 2024-09-10T10:02:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54a600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd54406e850 0x7fd54406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd549c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fd54415b330 0x7fd54406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37518/0 (socket says 172.21.15.110:37518) 2024-09-10T10:02:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd549c00640 1 -- 172.21.15.110:0/3342225558 learned_addr learned my addr 172.21.15.110:0/3342225558 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:41.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54a600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd54406e850 0x7fd54406c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45840/0 (socket says 172.21.15.110:45840) 2024-09-10T10:02:41.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54a600640 1 -- 172.21.15.110:0/3342225558 >> v1:172.21.15.145:6789/0 conn(0x7fd5440ce4d0 legacy=0x7fd5440b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54a600640 1 -- 172.21.15.110:0/3342225558 >> v1:172.21.15.110:6789/0 conn(0x7fd54415b330 legacy=0x7fd54406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd54a600640 1 -- 172.21.15.110:0/3342225558 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5440b4f20 con 0x7fd54406e850 2024-09-10T10:02:41.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.096+0000 7fd543e00640 1 -- 172.21.15.110:0/3342225558 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 323577430 0 0) 0x7fd5440b4960 con 0x7fd54415b330 2024-09-10T10:02:41.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd54a600640 1 --2- 172.21.15.110:0/3342225558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54406e850 0x7fd54406c6a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00bfa0 tx=0x7fd53c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c6dcb13a6b5838 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd543e00640 1 -- 172.21.15.110:0/3342225558 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00b2d0 con 0x7fd54406e850 2024-09-10T10:02:41.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd543e00640 1 -- 172.21.15.110:0/3342225558 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53c0a7950 con 0x7fd54406e850 2024-09-10T10:02:41.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd543e00640 1 -- 172.21.15.110:0/3342225558 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0a7d20 con 0x7fd54406e850 2024-09-10T10:02:41.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd54c600640 1 -- 172.21.15.110:0/3342225558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54406e850 msgr2=0x7fd54406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd54c600640 1 --2- 172.21.15.110:0/3342225558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54406e850 0x7fd54406c6a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00bfa0 tx=0x7fd53c01eb70 comp rx=0 tx=0).stop 2024-09-10T10:02:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd54c600640 1 -- 172.21.15.110:0/3342225558 shutdown_connections 2024-09-10T10:02:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd54c600640 1 --2- 172.21.15.110:0/3342225558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54406e850 0x7fd54406c6a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd54c600640 1 -- 172.21.15.110:0/3342225558 >> 172.21.15.110:0/3342225558 conn(0x7fd544062e40 msgr2=0x7fd54406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd54c600640 1 -- 172.21.15.110:0/3342225558 shutdown_connections 2024-09-10T10:02:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.100+0000 7fd54c600640 1 -- 172.21.15.110:0/3342225558 wait complete. 2024-09-10T10:02:41.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 Processor -- start 2024-09-10T10:02:41.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 -- start start 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd54406e850 0x7fd544142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd54415b330 0x7fd54415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54415d080 0x7fd5440ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd5440b3b00 con 0x7fd54415d080 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd5440b3980 con 0x7fd54406e850 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd5440b3c80 con 0x7fd54415b330 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54415d080 0x7fd5440ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54415d080 0x7fd5440ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45848/0 (socket says 172.21.15.110:45848) 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 -- 172.21.15.110:0/2720829325 learned_addr learned my addr 172.21.15.110:0/2720829325 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd549c00640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd54406e850 0x7fd544142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd549200640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd54415b330 0x7fd54415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 -- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd54415b330 msgr2=0x7fd54415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd54415b330 0x7fd54415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 -- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd54406e850 msgr2=0x7fd544142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd54406e850 0x7fd544142d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 -- 172.21.15.110:0/2720829325 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54406e240 con 0x7fd54415d080 2024-09-10T10:02:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54a600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54415d080 0x7fd5440ae840 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0b1040 tx=0x7fd53c01b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd541600640 1 -- 172.21.15.110:0/2720829325 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0bd020 con 0x7fd54415d080 2024-09-10T10:02:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd541600640 1 -- 172.21.15.110:0/2720829325 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53c009950 con 0x7fd54415d080 2024-09-10T10:02:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd544153cf0 con 0x7fd54415d080 2024-09-10T10:02:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd541600640 1 -- 172.21.15.110:0/2720829325 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c009d40 con 0x7fd54415d080 2024-09-10T10:02:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.104+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd544143860 con 0x7fd54415d080 2024-09-10T10:02:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.108+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54406c6a0 con 0x7fd54415d080 2024-09-10T10:02:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.108+0000 7fd541600640 1 -- 172.21.15.110:0/2720829325 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fd53c0c0070 con 0x7fd54415d080 2024-09-10T10:02:41.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.108+0000 7fd541600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd5240679e0 0x7fd524069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.108+0000 7fd541600640 1 -- 172.21.15.110:0/2720829325 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fd53c1202e0 con 0x7fd54415d080 2024-09-10T10:02:41.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.108+0000 7fd549c00640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd5240679e0 0x7fd524069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.108+0000 7fd549c00640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd5240679e0 0x7fd524069ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd538000f70 tx=0x7fd538007240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.112+0000 7fd541600640 1 -- 172.21.15.110:0/2720829325 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd53c018760 con 0x7fd54415d080 2024-09-10T10:02:41.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.268+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd54406bf00 con 0x7fd54415d080 2024-09-10T10:02:41.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.268+0000 7fd541600640 1 -- 172.21.15.110:0/2720829325 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fd53c0f06b0 con 0x7fd54415d080 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd5240679e0 msgr2=0x7fd524069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd5240679e0 0x7fd524069ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd538000f70 tx=0x7fd538007240 comp rx=0 tx=0).stop 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54415d080 msgr2=0x7fd5440ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54415d080 0x7fd5440ae840 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0b1040 tx=0x7fd53c01b910 comp rx=0 tx=0).stop 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 shutdown_connections 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd5240679e0 0x7fd524069ea0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54415d080 0x7fd5440ae840 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd54415b330 0x7fd54415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 --2- 172.21.15.110:0/2720829325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd54406e850 0x7fd544142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 >> 172.21.15.110:0/2720829325 conn(0x7fd544062e40 msgr2=0x7fd544071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:41.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 shutdown_connections 2024-09-10T10:02:41.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.272+0000 7fd54c600640 1 -- 172.21.15.110:0/2720829325 wait complete. 2024-09-10T10:02:41.294 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-10T10:02:41.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.428+0000 7f05b7a00640 1 Processor -- start 2024-09-10T10:02:41.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.428+0000 7f05b7a00640 1 -- start start 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.428+0000 7f05b7a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f05b015b330 0x7f05b006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.428+0000 7f05b7a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b00ce4d0 0x7f05b006e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.428+0000 7f05b7a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b00b1870 con 0x7f05b00b1c60 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.428+0000 7f05b7a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f05b00b5570 con 0x7f05b015b330 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.428+0000 7f05b7a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f05b00b4960 con 0x7f05b00ce4d0 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05b5a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f05b00b1c60 0x7f05b015ef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37532/0 (socket says 172.21.15.110:37532) 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05b5a00640 1 -- 172.21.15.110:0/1693000905 learned_addr learned my addr 172.21.15.110:0/1693000905 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05afe00640 1 --2- 172.21.15.110:0/1693000905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b00ce4d0 0x7f05b006e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05b5000640 1 --2- 172.21.15.110:0/1693000905 >> v2:172.21.15.145:3300/0 conn(0x7f05b015b330 0x7f05b006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05af400640 1 -- 172.21.15.110:0/1693000905 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3216444924 0 0) 0x7f05b00b1870 con 0x7f05b00b1c60 2024-09-10T10:02:41.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05af400640 1 -- 172.21.15.110:0/1693000905 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0594003530 con 0x7f05b00b1c60 2024-09-10T10:02:41.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05afe00640 1 -- 172.21.15.110:0/1693000905 >> v2:172.21.15.145:3300/0 conn(0x7f05b015b330 msgr2=0x7f05b006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:41.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05afe00640 1 --2- 172.21.15.110:0/1693000905 >> v2:172.21.15.145:3300/0 conn(0x7f05b015b330 0x7f05b006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05afe00640 1 -- 172.21.15.110:0/1693000905 >> v1:172.21.15.110:6789/0 conn(0x7f05b00b1c60 legacy=0x7f05b015ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05af400640 1 -- 172.21.15.110:0/1693000905 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3393991643 0 0) 0x7f0594003530 con 0x7f05b00b1c60 2024-09-10T10:02:41.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.432+0000 7f05afe00640 1 -- 172.21.15.110:0/1693000905 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b00b4c40 con 0x7f05b00ce4d0 2024-09-10T10:02:41.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05afe00640 1 --2- 172.21.15.110:0/1693000905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b00ce4d0 0x7f05b006e140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f059c0097c0 tx=0x7f059c01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676ae58a985debdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05af400640 1 -- 172.21.15.110:0/1693000905 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0b0070 con 0x7f05b00ce4d0 2024-09-10T10:02:41.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05af400640 1 -- 172.21.15.110:0/1693000905 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059c0ab400 con 0x7f05b00ce4d0 2024-09-10T10:02:41.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05af400640 1 -- 172.21.15.110:0/1693000905 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8001680 con 0x7f05b00ce4d0 2024-09-10T10:02:41.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- 172.21.15.110:0/1693000905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b00ce4d0 msgr2=0x7f05b006e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 --2- 172.21.15.110:0/1693000905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b00ce4d0 0x7f05b006e140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f059c0097c0 tx=0x7f059c01c3c0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- 172.21.15.110:0/1693000905 shutdown_connections 2024-09-10T10:02:41.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 --2- 172.21.15.110:0/1693000905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b00ce4d0 0x7f05b006e140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 --2- 172.21.15.110:0/1693000905 >> v2:172.21.15.145:3300/0 conn(0x7f05b015b330 0x7f05b006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- 172.21.15.110:0/1693000905 >> 172.21.15.110:0/1693000905 conn(0x7f05b0062e40 msgr2=0x7f05b006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:41.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- 172.21.15.110:0/1693000905 shutdown_connections 2024-09-10T10:02:41.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- 172.21.15.110:0/1693000905 wait complete. 2024-09-10T10:02:41.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 Processor -- start 2024-09-10T10:02:41.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- start start 2024-09-10T10:02:41.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05b00b1c60 0x7f05b0153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b015b330 0x7f05b0160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b0153880 0x7f05b01557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05b00b3830 con 0x7f05b015b330 2024-09-10T10:02:41.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05b00b36b0 con 0x7f05b00b1c60 2024-09-10T10:02:41.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.436+0000 7f05b7a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f05b00b39b0 con 0x7f05b0153880 2024-09-10T10:02:41.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b015b330 0x7f05b0160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b015b330 0x7f05b0160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45860/0 (socket says 172.21.15.110:45860) 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 -- 172.21.15.110:0/20982181 learned_addr learned my addr 172.21.15.110:0/20982181 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 -- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b0153880 msgr2=0x7f05b01557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05b5000640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05b00b1c60 0x7f05b0153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05b5a00640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b0153880 0x7f05b01557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b0153880 0x7f05b01557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 -- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05b00b1c60 msgr2=0x7f05b0153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05b00b1c60 0x7f05b0153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 -- 172.21.15.110:0/20982181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b0154710 con 0x7f05b015b330 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05b5000640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05b00b1c60 0x7f05b0153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05b5a00640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05b0153880 0x7f05b01557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05afe00640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b015b330 0x7f05b0160070 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f059c0ae040 tx=0x7f059c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05acc00640 1 -- 172.21.15.110:0/20982181 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0b0050 con 0x7f05b015b330 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05acc00640 1 -- 172.21.15.110:0/20982181 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059c0bc070 con 0x7f05b015b330 2024-09-10T10:02:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05acc00640 1 -- 172.21.15.110:0/20982181 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0b78b0 con 0x7f05b015b330 2024-09-10T10:02:41.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05b7a00640 1 -- 172.21.15.110:0/20982181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b01549f0 con 0x7f05b015b330 2024-09-10T10:02:41.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05b7a00640 1 -- 172.21.15.110:0/20982181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b0154fb0 con 0x7f05b015b330 2024-09-10T10:02:41.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05b7a00640 1 -- 172.21.15.110:0/20982181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b006e140 con 0x7f05b015b330 2024-09-10T10:02:41.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.440+0000 7f05acc00640 1 -- 172.21.15.110:0/20982181 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f059c0b7a50 con 0x7f05b015b330 2024-09-10T10:02:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.444+0000 7f05acc00640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0590067a10 0x7f0590069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.444+0000 7f05acc00640 1 -- 172.21.15.110:0/20982181 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f059c1207b0 con 0x7f05b015b330 2024-09-10T10:02:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.444+0000 7f05b5000640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0590067a10 0x7f0590069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.444+0000 7f05b5000640 1 --2- 172.21.15.110:0/20982181 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0590067a10 0x7f0590069ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f05a0007440 tx=0x7f05a000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.448+0000 7f05acc00640 1 -- 172.21.15.110:0/20982181 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f059c015fb0 con 0x7f05b015b330 2024-09-10T10:02:41.629 INFO:teuthology.orchestra.run.smithi110.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-10T10:02:41.629 INFO:teuthology.orchestra.run.smithi110.stderr:fs volume create [] : Create a CephFS volume 2024-09-10T10:02:41.629 INFO:teuthology.orchestra.run.smithi110.stderr:Error EINVAL: invalid command 2024-09-10T10:02:41.637 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T10:02:41.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-10T10:02:41.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bfc800640 1 Processor -- start 2024-09-10T10:02:41.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bfc800640 1 -- start start 2024-09-10T10:02:41.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bfc800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1bf40ce4d0 0x7f1bf406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bfc800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bf40b4960 con 0x7f1bf415b330 2024-09-10T10:02:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bfc800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bf40b4c40 con 0x7f1bf406e850 2024-09-10T10:02:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bfc800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1bf40b5570 con 0x7f1bf40ce4d0 2024-09-10T10:02:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bf9e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1bf415b330 0x7f1bf406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37546/0 (socket says 172.21.15.110:37546) 2024-09-10T10:02:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bf9e00640 1 -- 172.21.15.110:0/3101457626 learned_addr learned my addr 172.21.15.110:0/3101457626 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.776+0000 7f1bf9400640 1 --2- 172.21.15.110:0/3101457626 >> v2:172.21.15.110:3300/0 conn(0x7f1bf40ce4d0 0x7f1bf406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.780+0000 7f1bf9400640 1 -- 172.21.15.110:0/3101457626 >> v1:172.21.15.145:6789/0 conn(0x7f1bf406e850 legacy=0x7f1bf40b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.780+0000 7f1bf9400640 1 -- 172.21.15.110:0/3101457626 >> v1:172.21.15.110:6789/0 conn(0x7f1bf415b330 legacy=0x7f1bf406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.780+0000 7f1bf9400640 1 -- 172.21.15.110:0/3101457626 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf40b4f20 con 0x7f1bf40ce4d0 2024-09-10T10:02:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.780+0000 7f1bf9400640 1 --2- 172.21.15.110:0/3101457626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf40ce4d0 0x7f1bf406c6a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1be40097c0 tx=0x7f1be401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984228f25e165552 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.780+0000 7f1bf8a00640 1 -- 172.21.15.110:0/3101457626 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be40b1070 con 0x7f1bf40ce4d0 2024-09-10T10:02:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.780+0000 7f1bf8a00640 1 -- 172.21.15.110:0/3101457626 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1be40ac410 con 0x7f1bf40ce4d0 2024-09-10T10:02:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.780+0000 7f1bf8a00640 1 -- 172.21.15.110:0/3101457626 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be40ac7e0 con 0x7f1bf40ce4d0 2024-09-10T10:02:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- 172.21.15.110:0/3101457626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf40ce4d0 msgr2=0x7f1bf406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 --2- 172.21.15.110:0/3101457626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf40ce4d0 0x7f1bf406c6a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1be40097c0 tx=0x7f1be401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- 172.21.15.110:0/3101457626 shutdown_connections 2024-09-10T10:02:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 --2- 172.21.15.110:0/3101457626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf40ce4d0 0x7f1bf406c6a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- 172.21.15.110:0/3101457626 >> 172.21.15.110:0/3101457626 conn(0x7f1bf4062e40 msgr2=0x7f1bf406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- 172.21.15.110:0/3101457626 shutdown_connections 2024-09-10T10:02:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- 172.21.15.110:0/3101457626 wait complete. 2024-09-10T10:02:41.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 Processor -- start 2024-09-10T10:02:41.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- start start 2024-09-10T10:02:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1bf406e850 0x7f1bf4161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf415b330 0x7f1bf415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf415bf00 0x7f1bf4157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4169450 con 0x7f1bf415b330 2024-09-10T10:02:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41692d0 con 0x7f1bf415bf00 2024-09-10T10:02:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfc800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf41695d0 con 0x7f1bf406e850 2024-09-10T10:02:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf415b330 0x7f1bf415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf415b330 0x7f1bf415b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45880/0 (socket says 172.21.15.110:45880) 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 -- 172.21.15.110:0/2554428272 learned_addr learned my addr 172.21.15.110:0/2554428272 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfa800640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf415bf00 0x7f1bf4157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9e00640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1bf406e850 0x7f1bf4161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 -- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1bf406e850 msgr2=0x7f1bf4161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1bf406e850 0x7f1bf4161c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 -- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf415bf00 msgr2=0x7f1bf4157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf415bf00 0x7f1bf4157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 -- 172.21.15.110:0/2554428272 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf415d6d0 con 0x7f1bf415b330 2024-09-10T10:02:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bfa800640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf415bf00 0x7f1bf4157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.784+0000 7f1bf9400640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf415b330 0x7f1bf415b9c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1be40190b0 tx=0x7f1be4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bee000640 1 -- 172.21.15.110:0/2554428272 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be40b9020 con 0x7f1bf415b330 2024-09-10T10:02:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bee000640 1 -- 172.21.15.110:0/2554428272 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1be40ace40 con 0x7f1bf415b330 2024-09-10T10:02:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bee000640 1 -- 172.21.15.110:0/2554428272 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be40b1040 con 0x7f1bf415b330 2024-09-10T10:02:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf415c4d0 con 0x7f1bf415b330 2024-09-10T10:02:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf4163c10 con 0x7f1bf415b330 2024-09-10T10:02:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf406c6a0 con 0x7f1bf415b330 2024-09-10T10:02:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bee000640 1 -- 172.21.15.110:0/2554428272 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f1be40bc020 con 0x7f1bf415b330 2024-09-10T10:02:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bee000640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1bd80679e0 0x7f1bd8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bee000640 1 -- 172.21.15.110:0/2554428272 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f1be411fa40 con 0x7f1bf415b330 2024-09-10T10:02:41.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bf9e00640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1bd80679e0 0x7f1bd8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:41.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.788+0000 7f1bf9e00640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1bd80679e0 0x7f1bd8069ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005890 tx=0x7f1be8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:41.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.792+0000 7f1bee000640 1 -- 172.21.15.110:0/2554428272 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1be40c3030 con 0x7f1bf415b330 2024-09-10T10:02:41.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:41.972+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7f1bf406bdf0 con 0x7f1bd80679e0 2024-09-10T10:02:44.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.620+0000 7f1bee000640 1 -- 172.21.15.110:0/2554428272 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7f1bf406bdf0 con 0x7f1bd80679e0 2024-09-10T10:02:44.626 INFO:teuthology.orchestra.run.smithi110.stderr:Volume created successfully (no MDS daemons created) 2024-09-10T10:02:44.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1bd80679e0 msgr2=0x7f1bd8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:44.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1bd80679e0 0x7f1bd8069ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005890 tx=0x7f1be8009040 comp rx=0 tx=0).stop 2024-09-10T10:02:44.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf415b330 msgr2=0x7f1bf415b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:44.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf415b330 0x7f1bf415b9c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1be40190b0 tx=0x7f1be4019240 comp rx=0 tx=0).stop 2024-09-10T10:02:44.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 shutdown_connections 2024-09-10T10:02:44.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1bd80679e0 0x7f1bd8069ea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf415bf00 0x7f1bf4157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1bf415b330 0x7f1bf415b9c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 --2- 172.21.15.110:0/2554428272 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1bf406e850 0x7f1bf4161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 >> 172.21.15.110:0/2554428272 conn(0x7f1bf4062e40 msgr2=0x7f1bf40ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:44.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 shutdown_connections 2024-09-10T10:02:44.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.628+0000 7f1bfc800640 1 -- 172.21.15.110:0/2554428272 wait complete. 2024-09-10T10:02:44.660 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-10T10:02:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e4b800640 1 Processor -- start 2024-09-10T10:02:44.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e4b800640 1 -- start start 2024-09-10T10:02:44.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e4b800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e4415f870 0x7f9e440b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:44.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e4b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 0x7f9e4406e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:44.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e4b800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e440b1e90 con 0x7f9e440cfce0 2024-09-10T10:02:44.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e4b800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9e4406ead0 con 0x7f9e4415f870 2024-09-10T10:02:44.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e4b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e440d00c0 con 0x7f9e4406e140 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e43e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e4415f870 0x7f9e440b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e48e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f9e440cfce0 0x7f9e440ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:58300/0 (socket says 172.21.15.110:58300) 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e48e00640 1 -- 172.21.15.110:0/2109909260 learned_addr learned my addr 172.21.15.110:0/2109909260 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e43e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e4415f870 0x7f9e440b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:55132/0 (socket says 172.21.15.110:55132) 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e49800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 0x7f9e4406e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e43e00640 1 -- 172.21.15.110:0/2109909260 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 msgr2=0x7f9e4406e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e43e00640 1 --2- 172.21.15.110:0/2109909260 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 0x7f9e4406e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e43e00640 1 -- 172.21.15.110:0/2109909260 >> v1:172.21.15.145:6789/0 conn(0x7f9e440cfce0 legacy=0x7f9e440ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:44.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.800+0000 7f9e43e00640 1 -- 172.21.15.110:0/2109909260 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e440b43d0 con 0x7f9e4415f870 2024-09-10T10:02:44.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e43e00640 1 --2- 172.21.15.110:0/2109909260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e4415f870 0x7f9e440b3b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e300097c0 tx=0x7f9e3001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a23713ea931f74de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e43400640 1 -- 172.21.15.110:0/2109909260 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300b0070 con 0x7f9e4415f870 2024-09-10T10:02:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e43400640 1 -- 172.21.15.110:0/2109909260 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e300ab600 con 0x7f9e4415f870 2024-09-10T10:02:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e43400640 1 -- 172.21.15.110:0/2109909260 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300ab9d0 con 0x7f9e4415f870 2024-09-10T10:02:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e4b800640 1 -- 172.21.15.110:0/2109909260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e4415f870 msgr2=0x7f9e440b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:44.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e4b800640 1 --2- 172.21.15.110:0/2109909260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e4415f870 0x7f9e440b3b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e300097c0 tx=0x7f9e3001c4a0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e4b800640 1 -- 172.21.15.110:0/2109909260 shutdown_connections 2024-09-10T10:02:44.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e4b800640 1 --2- 172.21.15.110:0/2109909260 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 0x7f9e4406e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e4b800640 1 --2- 172.21.15.110:0/2109909260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e4415f870 0x7f9e440b3b80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e4b800640 1 -- 172.21.15.110:0/2109909260 >> 172.21.15.110:0/2109909260 conn(0x7f9e44062e40 msgr2=0x7f9e4406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:44.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e4b800640 1 -- 172.21.15.110:0/2109909260 shutdown_connections 2024-09-10T10:02:44.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.804+0000 7f9e4b800640 1 -- 172.21.15.110:0/2109909260 wait complete. 2024-09-10T10:02:44.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e4b800640 1 Processor -- start 2024-09-10T10:02:44.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e4b800640 1 -- start start 2024-09-10T10:02:44.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e4b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 0x7f9e44158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:44.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e4b800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e440cfce0 0x7f9e4415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:44.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e4b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e4415f870 0x7f9e44150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e4b800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e440b2fc0 con 0x7f9e440cfce0 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e4b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e440b2e40 con 0x7f9e4415f870 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e4b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e440b3140 con 0x7f9e4406e140 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e440cfce0 0x7f9e4415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e440cfce0 0x7f9e4415b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45882/0 (socket says 172.21.15.110:45882) 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 -- 172.21.15.110:0/1234845621 learned_addr learned my addr 172.21.15.110:0/1234845621 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e48e00640 1 --2- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 0x7f9e44158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e49800640 1 --2- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e4415f870 0x7f9e44150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 -- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 msgr2=0x7f9e44158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 --2- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e4406e140 0x7f9e44158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 -- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e4415f870 msgr2=0x7f9e44150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:44.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 --2- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e4415f870 0x7f9e44150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 -- 172.21.15.110:0/1234845621 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4415f490 con 0x7f9e440cfce0 2024-09-10T10:02:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.808+0000 7f9e43e00640 1 --2- 172.21.15.110:0/1234845621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e440cfce0 0x7f9e4415b490 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9e3001a0d0 tx=0x7f9e30019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e40c00640 1 -- 172.21.15.110:0/1234845621 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300b0040 con 0x7f9e440cfce0 2024-09-10T10:02:44.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e40c00640 1 -- 172.21.15.110:0/1234845621 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e300ba070 con 0x7f9e440cfce0 2024-09-10T10:02:44.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e4b800640 1 -- 172.21.15.110:0/1234845621 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4415a9d0 con 0x7f9e440cfce0 2024-09-10T10:02:44.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e40c00640 1 -- 172.21.15.110:0/1234845621 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300b4630 con 0x7f9e440cfce0 2024-09-10T10:02:44.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e4b800640 1 -- 172.21.15.110:0/1234845621 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e44151610 con 0x7f9e440cfce0 2024-09-10T10:02:44.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e4b800640 1 -- 172.21.15.110:0/1234845621 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e440b3960 con 0x7f9e440cfce0 2024-09-10T10:02:44.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e40c00640 1 -- 172.21.15.110:0/1234845621 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f9e300b47d0 con 0x7f9e440cfce0 2024-09-10T10:02:44.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e40c00640 1 --2- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e240679e0 0x7f9e24069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:44.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e40c00640 1 -- 172.21.15.110:0/1234845621 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6543+0+0 (secure 0 0 0) 0x7f9e301208c0 con 0x7f9e440cfce0 2024-09-10T10:02:44.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.812+0000 7f9e48e00640 1 --2- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e240679e0 0x7f9e24069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:44.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.816+0000 7f9e48e00640 1 --2- 172.21.15.110:0/1234845621 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e240679e0 0x7f9e24069ea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e34000f70 tx=0x7f9e34007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:44.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:44.816+0000 7f9e40c00640 1 -- 172.21.15.110:0/1234845621 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9e30016090 con 0x7f9e440cfce0 2024-09-10T10:02:44.998 INFO:teuthology.orchestra.run.smithi110.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-10T10:02:44.998 INFO:teuthology.orchestra.run.smithi110.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-10T10:02:44.999 INFO:teuthology.orchestra.run.smithi110.stderr:Error EINVAL: invalid command 2024-09-10T10:02:45.007 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T10:02:45.007 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-10T10:02:45.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0554000640 1 Processor -- start 2024-09-10T10:02:45.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0554000640 1 -- start start 2024-09-10T10:02:45.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0554000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f054c0ce4d0 0x7f054c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0554000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f054c0b4960 con 0x7f054c15b330 2024-09-10T10:02:45.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0554000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f054c0b4c40 con 0x7f054c06e850 2024-09-10T10:02:45.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0554000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f054c0b5570 con 0x7f054c0ce4d0 2024-09-10T10:02:45.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0551600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f054c15b330 0x7f054c06e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37562/0 (socket says 172.21.15.110:37562) 2024-09-10T10:02:45.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0551600640 1 -- 172.21.15.110:0/3412432227 learned_addr learned my addr 172.21.15.110:0/3412432227 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:45.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0550c00640 1 --2- 172.21.15.110:0/3412432227 >> v2:172.21.15.145:3300/0 conn(0x7f054c0ce4d0 0x7f054c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705278444 0 0) 0x7f054c0b4960 con 0x7f054c15b330 2024-09-10T10:02:45.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0534003530 con 0x7f054c15b330 2024-09-10T10:02:45.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3243744460 0 0) 0x7f0534003530 con 0x7f054c15b330 2024-09-10T10:02:45.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 >> v2:172.21.15.145:3300/0 conn(0x7f054c0ce4d0 msgr2=0x7f054c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 --2- 172.21.15.110:0/3412432227 >> v2:172.21.15.145:3300/0 conn(0x7f054c0ce4d0 0x7f054c06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 >> v1:172.21.15.145:6789/0 conn(0x7f054c06e850 legacy=0x7f054c0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c0b4f20 con 0x7f054c15b330 2024-09-10T10:02:45.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f0540001680 con 0x7f054c15b330 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 >> v1:172.21.15.110:6789/0 conn(0x7f054c15b330 legacy=0x7f054c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05340039c0 0x7f0534003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0534004810 0x7f0534004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f053400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f054c0b5570 con 0x7f05340051d0 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f053400c070 con 0x7f0534004810 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f053400c270 con 0x7f05340039c0 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0552000640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f053400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0550c00640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0534004810 0x7f0534004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0551600640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05340039c0 0x7f0534003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0552000640 1 -- 172.21.15.110:0/3412432227 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05340039c0 msgr2=0x7f0534003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0552000640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05340039c0 0x7f0534003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0552000640 1 -- 172.21.15.110:0/3412432227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0534004810 msgr2=0x7f0534004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0552000640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0534004810 0x7f0534004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.144+0000 7f0552000640 1 -- 172.21.15.110:0/3412432227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0534003530 con 0x7f05340051d0 2024-09-10T10:02:45.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0552000640 1 -- 172.21.15.110:0/3412432227 reap_dead start 2024-09-10T10:02:45.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0552000640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f053400b9d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f054c0b0670 tx=0x7f054401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a7fe8107cd5017 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544001a90 con 0x7f05340051d0 2024-09-10T10:02:45.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0544003b80 con 0x7f05340051d0 2024-09-10T10:02:45.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f054be00640 1 -- 172.21.15.110:0/3412432227 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544005020 con 0x7f05340051d0 2024-09-10T10:02:45.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0554000640 1 -- 172.21.15.110:0/3412432227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 msgr2=0x7f053400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0554000640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f053400b9d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f054c0b0670 tx=0x7f054401c720 comp rx=0 tx=0).stop 2024-09-10T10:02:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0554000640 1 -- 172.21.15.110:0/3412432227 shutdown_connections 2024-09-10T10:02:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0554000640 1 --2- 172.21.15.110:0/3412432227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f053400b9d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0554000640 1 -- 172.21.15.110:0/3412432227 >> 172.21.15.110:0/3412432227 conn(0x7f054c062e40 msgr2=0x7f054c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0554000640 1 -- 172.21.15.110:0/3412432227 shutdown_connections 2024-09-10T10:02:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.148+0000 7f0554000640 1 -- 172.21.15.110:0/3412432227 wait complete. 2024-09-10T10:02:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 Processor -- start 2024-09-10T10:02:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 -- start start 2024-09-10T10:02:45.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f054c1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f054c14df90 0x7f054c14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f054c14eb90 0x7f054c0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f054c0b1c60 con 0x7f05340051d0 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f054c15b710 con 0x7f054c14df90 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f054c0b1de0 con 0x7f054c14eb90 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f054c1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f054c1580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45892/0 (socket says 172.21.15.110:45892) 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 -- 172.21.15.110:0/2756909225 learned_addr learned my addr 172.21.15.110:0/2756909225 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0552000640 1 --2- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f054c14eb90 0x7f054c0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0550c00640 1 --2- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f054c14df90 0x7f054c14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 -- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f054c14eb90 msgr2=0x7f054c0b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 --2- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f054c14eb90 0x7f054c0b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 -- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f054c14df90 msgr2=0x7f054c14e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 --2- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f054c14df90 0x7f054c14e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 -- 172.21.15.110:0/2756909225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c157680 con 0x7f05340051d0 2024-09-10T10:02:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0551600640 1 --2- 172.21.15.110:0/2756909225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05340051d0 0x7f054c1580f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0540007060 tx=0x7f0540007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0549600640 1 -- 172.21.15.110:0/2756909225 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054001a070 con 0x7f05340051d0 2024-09-10T10:02:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0549600640 1 -- 172.21.15.110:0/2756909225 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054000f9c0 con 0x7f05340051d0 2024-09-10T10:02:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 -- 172.21.15.110:0/2756909225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c14e410 con 0x7f05340051d0 2024-09-10T10:02:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0549600640 1 -- 172.21.15.110:0/2756909225 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054000fd90 con 0x7f05340051d0 2024-09-10T10:02:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.152+0000 7f0554000640 1 -- 172.21.15.110:0/2756909225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c15c640 con 0x7f05340051d0 2024-09-10T10:02:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.156+0000 7f0554000640 1 -- 172.21.15.110:0/2756909225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c06c6a0 con 0x7f05340051d0 2024-09-10T10:02:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.156+0000 7f0549600640 1 -- 172.21.15.110:0/2756909225 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f054001d070 con 0x7f05340051d0 2024-09-10T10:02:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.156+0000 7f0549600640 1 --2- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f052c067a10 0x7f052c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.156+0000 7f0549600640 1 -- 172.21.15.110:0/2756909225 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6543+0+0 (secure 0 0 0) 0x7f05400859d0 con 0x7f05340051d0 2024-09-10T10:02:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.156+0000 7f0550c00640 1 --2- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f052c067a10 0x7f052c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.156+0000 7f0550c00640 1 --2- 172.21.15.110:0/2756909225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f052c067a10 0x7f052c069ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f054c0cd060 tx=0x7f0538008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.160+0000 7f0549600640 1 -- 172.21.15.110:0/2756909225 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f05400560b0 con 0x7f05340051d0 2024-09-10T10:02:45.349 INFO:teuthology.orchestra.run.smithi110.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-10T10:02:45.349 INFO:teuthology.orchestra.run.smithi110.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-10T10:02:45.349 INFO:teuthology.orchestra.run.smithi110.stderr:Error EINVAL: invalid command 2024-09-10T10:02:45.356 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T10:02:45.356 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-10T10:02:45.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.492+0000 7faf0da00640 1 Processor -- start 2024-09-10T10:02:45.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.492+0000 7faf0da00640 1 -- start start 2024-09-10T10:02:45.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf0da00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 0x7faf080b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf0da00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7faf080cfd90 0x7faf080b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf0da00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf080b37c0 con 0x7faf080b29e0 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf0da00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7faf080b2e20 con 0x7faf080cfd90 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf0da00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7faf080b3aa0 con 0x7faf08145f30 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf07400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7faf080cfd90 0x7faf080b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf06000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7faf080b29e0 0x7faf080cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37570/0 (socket says 172.21.15.110:37570) 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf06000640 1 -- 172.21.15.110:0/3170636360 learned_addr learned my addr 172.21.15.110:0/3170636360 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf06a00640 1 --2- 172.21.15.110:0/3170636360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 0x7faf080b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf07400640 1 -- 172.21.15.110:0/3170636360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 msgr2=0x7faf080b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:45.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf07400640 1 --2- 172.21.15.110:0/3170636360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 0x7faf080b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf07400640 1 -- 172.21.15.110:0/3170636360 >> v1:172.21.15.110:6789/0 conn(0x7faf080b29e0 legacy=0x7faf080cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf07400640 1 -- 172.21.15.110:0/3170636360 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf080b3d80 con 0x7faf080cfd90 2024-09-10T10:02:45.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.496+0000 7faf05600640 1 -- 172.21.15.110:0/3170636360 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3859831329 0 0) 0x7faf080b37c0 con 0x7faf080b29e0 2024-09-10T10:02:45.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf07400640 1 --2- 172.21.15.110:0/3170636360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf080b4580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7faf00009d90 tx=0x7faf0001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c53cc1711a96401 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf05600640 1 -- 172.21.15.110:0/3170636360 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf000b2070 con 0x7faf080cfd90 2024-09-10T10:02:45.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf05600640 1 -- 172.21.15.110:0/3170636360 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf000a5830 con 0x7faf080cfd90 2024-09-10T10:02:45.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf05600640 1 -- 172.21.15.110:0/3170636360 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf000a5c00 con 0x7faf080cfd90 2024-09-10T10:02:45.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- 172.21.15.110:0/3170636360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 msgr2=0x7faf080b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 --2- 172.21.15.110:0/3170636360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf080b4580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7faf00009d90 tx=0x7faf0001c970 comp rx=0 tx=0).stop 2024-09-10T10:02:45.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- 172.21.15.110:0/3170636360 shutdown_connections 2024-09-10T10:02:45.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 --2- 172.21.15.110:0/3170636360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf080b4580 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 --2- 172.21.15.110:0/3170636360 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 0x7faf080b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- 172.21.15.110:0/3170636360 >> 172.21.15.110:0/3170636360 conn(0x7faf08062e40 msgr2=0x7faf0806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- 172.21.15.110:0/3170636360 shutdown_connections 2024-09-10T10:02:45.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- 172.21.15.110:0/3170636360 wait complete. 2024-09-10T10:02:45.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 Processor -- start 2024-09-10T10:02:45.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- start start 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf080b29e0 0x7faf0815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf081435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 0x7faf08143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf080b56a0 con 0x7faf080cfd90 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf080b5520 con 0x7faf080b29e0 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf0da00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7faf080b5820 con 0x7faf08145f30 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf06000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf081435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf06000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf081435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45908/0 (socket says 172.21.15.110:45908) 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.500+0000 7faf06000640 1 -- 172.21.15.110:0/1345195310 learned_addr learned my addr 172.21.15.110:0/1345195310 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf06a00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf080b29e0 0x7faf0815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf07400640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 0x7faf08143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf06000640 1 -- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 msgr2=0x7faf08143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf06000640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 0x7faf08143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf06000640 1 -- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf080b29e0 msgr2=0x7faf0815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf06000640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf080b29e0 0x7faf0815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf06000640 1 -- 172.21.15.110:0/1345195310 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf081410b0 con 0x7faf080cfd90 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf06000640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf081435d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7faef400c4b0 tx=0x7faef400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faefea00640 1 -- 172.21.15.110:0/1345195310 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4018070 con 0x7faf080cfd90 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faefea00640 1 -- 172.21.15.110:0/1345195310 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef4007500 con 0x7faf080cfd90 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf08141390 con 0x7faf080cfd90 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faefea00640 1 -- 172.21.15.110:0/1345195310 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4004a30 con 0x7faf080cfd90 2024-09-10T10:02:45.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0815c080 con 0x7faf080cfd90 2024-09-10T10:02:45.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf080b4580 con 0x7faf080cfd90 2024-09-10T10:02:45.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faefea00640 1 -- 172.21.15.110:0/1345195310 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7faef401b070 con 0x7faf080cfd90 2024-09-10T10:02:45.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faefea00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faee8067a10 0x7faee8069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.504+0000 7faefea00640 1 -- 172.21.15.110:0/1345195310 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6543+0+0 (secure 0 0 0) 0x7faef40853c0 con 0x7faf080cfd90 2024-09-10T10:02:45.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.508+0000 7faf06a00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faee8067a10 0x7faee8069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.508+0000 7faf06a00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faee8067a10 0x7faee8069ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf081f9e10 tx=0x7faef8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.508+0000 7faefea00640 1 -- 172.21.15.110:0/1345195310 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7faef40553c0 con 0x7faf080cfd90 2024-09-10T10:02:45.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:45.692+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7faf08140e60 con 0x7faee8067a10 2024-09-10T10:02:45.702 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:02:45.700+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:02:45.747 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:02:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.720+0000 7faefea00640 1 -- 172.21.15.110:0/1345195310 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7faf08140e60 con 0x7faee8067a10 2024-09-10T10:02:48.723 INFO:teuthology.orchestra.run.smithi110.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-10T10:02:48.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faee8067a10 msgr2=0x7faee8069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faee8067a10 0x7faee8069ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf081f9e10 tx=0x7faef8009190 comp rx=0 tx=0).stop 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 msgr2=0x7faf081435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf081435d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7faef400c4b0 tx=0x7faef400c970 comp rx=0 tx=0).stop 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 shutdown_connections 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faee8067a10 0x7faee8069ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf08145f30 0x7faf08143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf080cfd90 0x7faf081435d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 --2- 172.21.15.110:0/1345195310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf080b29e0 0x7faf0815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 >> 172.21.15.110:0/1345195310 conn(0x7faf08062e40 msgr2=0x7faf080ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 shutdown_connections 2024-09-10T10:02:48.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:48.724+0000 7faf0da00640 1 -- 172.21.15.110:0/1345195310 wait complete. 2024-09-10T10:02:48.751 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:02:48.751 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:02:48.859 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:02:48.949 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:02:48.950 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:02:48.950 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:48.950 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:48.959 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:48.960 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:49.009 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:02:49.087 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:02:49.103 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:02:49.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.191+0000 7f7022600640 1 Processor -- start 2024-09-10T10:02:49.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.191+0000 7f7022600640 1 -- start start 2024-09-10T10:02:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.191+0000 7f7022600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f701c0726e0 0x7f701c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f7022600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c06efa0 0x7f701c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f7022600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f701c06f3e0 con 0x7f701c0b5430 2024-09-10T10:02:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f7022600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f701c0b5880 con 0x7f701c0726e0 2024-09-10T10:02:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f7022600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f701c0cee50 con 0x7f701c06efa0 2024-09-10T10:02:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f701be00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f701c0b5430 0x7f701c0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:35556/0 (socket says 172.21.15.110:35556) 2024-09-10T10:02:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f701be00640 1 -- 172.21.15.110:0/4076336699 learned_addr learned my addr 172.21.15.110:0/4076336699 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f701b400640 1 --2- 172.21.15.110:0/4076336699 >> v2:172.21.15.145:3300/0 conn(0x7f701c0726e0 0x7f701c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f701aa00640 1 --2- 172.21.15.110:0/4076336699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c06efa0 0x7f701c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f701b400640 1 -- 172.21.15.110:0/4076336699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c06efa0 msgr2=0x7f701c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f701b400640 1 --2- 172.21.15.110:0/4076336699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c06efa0 0x7f701c0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f701b400640 1 -- 172.21.15.110:0/4076336699 >> v1:172.21.15.145:6789/0 conn(0x7f701c0b5430 legacy=0x7f701c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.195+0000 7f701b400640 1 -- 172.21.15.110:0/4076336699 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c15fda0 con 0x7f701c0726e0 2024-09-10T10:02:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f701b400640 1 --2- 172.21.15.110:0/4076336699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c0726e0 0x7f701c0cf2e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f700c009d90 tx=0x7f700c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f34242bf3cda5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:49.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f701a000640 1 -- 172.21.15.110:0/4076336699 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c0b2070 con 0x7f701c0726e0 2024-09-10T10:02:49.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f701a000640 1 -- 172.21.15.110:0/4076336699 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c0a5a00 con 0x7f701c0726e0 2024-09-10T10:02:49.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f701a000640 1 -- 172.21.15.110:0/4076336699 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c0a5dd0 con 0x7f701c0726e0 2024-09-10T10:02:49.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 -- 172.21.15.110:0/4076336699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c0726e0 msgr2=0x7f701c0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 --2- 172.21.15.110:0/4076336699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c0726e0 0x7f701c0cf2e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f700c009d90 tx=0x7f700c01c940 comp rx=0 tx=0).stop 2024-09-10T10:02:49.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 -- 172.21.15.110:0/4076336699 shutdown_connections 2024-09-10T10:02:49.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 --2- 172.21.15.110:0/4076336699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c06efa0 0x7f701c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 --2- 172.21.15.110:0/4076336699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c0726e0 0x7f701c0cf2e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 -- 172.21.15.110:0/4076336699 >> 172.21.15.110:0/4076336699 conn(0x7f701c062d10 msgr2=0x7f701c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:49.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 -- 172.21.15.110:0/4076336699 shutdown_connections 2024-09-10T10:02:49.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 -- 172.21.15.110:0/4076336699 wait complete. 2024-09-10T10:02:49.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 Processor -- start 2024-09-10T10:02:49.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.199+0000 7f7022600640 1 -- start start 2024-09-10T10:02:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c06efa0 0x7f701c150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c0726e0 0x7f701c0ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f701c0b5430 0x7f701c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f701c0b3ff0 con 0x7f701c0b5430 2024-09-10T10:02:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f701c073270 con 0x7f701c06efa0 2024-09-10T10:02:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f701c0b4170 con 0x7f701c0726e0 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f701c0b5430 0x7f701c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f701c0b5430 0x7f701c155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34014/0 (socket says 172.21.15.110:34014) 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 -- 172.21.15.110:0/3342595275 learned_addr learned my addr 172.21.15.110:0/3342595275 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701b400640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c06efa0 0x7f701c150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701aa00640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c0726e0 0x7f701c0ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 -- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c0726e0 msgr2=0x7f701c0ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c0726e0 0x7f701c0ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 -- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c06efa0 msgr2=0x7f701c150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c06efa0 0x7f701c150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 -- 172.21.15.110:0/3342595275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c1553e0 con 0x7f701c0b5430 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f701be00640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f701c0b5430 0x7f701c155810 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f701400c400 tx=0x7f701400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7013400640 1 -- 172.21.15.110:0/3342595275 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014017070 con 0x7f701c0b5430 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7013400640 1 -- 172.21.15.110:0/3342595275 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7014005cf0 con 0x7f701c0b5430 2024-09-10T10:02:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7013400640 1 -- 172.21.15.110:0/3342595275 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140072d0 con 0x7f701c0b5430 2024-09-10T10:02:49.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c0ab420 con 0x7f701c0b5430 2024-09-10T10:02:49.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c154620 con 0x7f701c0b5430 2024-09-10T10:02:49.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c072ac0 con 0x7f701c0b5430 2024-09-10T10:02:49.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.203+0000 7f7013400640 1 -- 172.21.15.110:0/3342595275 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f701401a070 con 0x7f701c0b5430 2024-09-10T10:02:49.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.207+0000 7f7013400640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ffc0679e0 0x7f6ffc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.207+0000 7f7013400640 1 -- 172.21.15.110:0/3342595275 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f7014085140 con 0x7f701c0b5430 2024-09-10T10:02:49.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.207+0000 7f701b400640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ffc0679e0 0x7f6ffc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.207+0000 7f701b400640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ffc0679e0 0x7f6ffc069ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f700c0af000 tx=0x7f700c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:49.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.211+0000 7f7013400640 1 -- 172.21.15.110:0/3342595275 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f70140554a0 con 0x7f701c0b5430 2024-09-10T10:02:49.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.387+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f701c0b5090 con 0x7f701c0b5430 2024-09-10T10:02:49.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.387+0000 7f7013400640 1 -- 172.21.15.110:0/3342595275 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+124 (secure 0 0 0) 0x7f70140597a0 con 0x7f701c0b5430 2024-09-10T10:02:49.393 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6834/3463064995 2024-09-11T10:02:45.702580+0000 2024-09-10T10:02:49.393 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6832/3463064995 2024-09-11T10:02:45.702580+0000 2024-09-10T10:02:49.393 INFO:teuthology.orchestra.run.smithi110.stderr:listed 2 entries 2024-09-10T10:02:49.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ffc0679e0 msgr2=0x7f6ffc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ffc0679e0 0x7f6ffc069ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f700c0af000 tx=0x7f700c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:02:49.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f701c0b5430 msgr2=0x7f701c155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f701c0b5430 0x7f701c155810 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f701400c400 tx=0x7f701400c8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 shutdown_connections 2024-09-10T10:02:49.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ffc0679e0 0x7f6ffc069ea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f701c0b5430 0x7f701c155810 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f701c0726e0 0x7f701c0ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 --2- 172.21.15.110:0/3342595275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f701c06efa0 0x7f701c150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 >> 172.21.15.110:0/3342595275 conn(0x7f701c062d10 msgr2=0x7f701c15eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:49.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 shutdown_connections 2024-09-10T10:02:49.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.391+0000 7f7022600640 1 -- 172.21.15.110:0/3342595275 wait complete. 2024-09-10T10:02:49.414 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:02:49.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:49.414 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:02:49.439 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:02:49.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:49.440 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:02:49.489 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:02:49.489 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:49.489 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:49.498 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:49.499 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:49.555 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:02:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.687+0000 7f7f1d000640 1 Processor -- start 2024-09-10T10:02:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.687+0000 7f7f1d000640 1 -- start start 2024-09-10T10:02:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.687+0000 7f7f1d000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7f180726e0 0x7f7f180cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.687+0000 7f7f1d000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f1806efa0 0x7f7f180b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.687+0000 7f7f1d000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f1806f3e0 con 0x7f7f180b5430 2024-09-10T10:02:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.687+0000 7f7f1d000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f7f180b5880 con 0x7f7f180726e0 2024-09-10T10:02:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.687+0000 7f7f1d000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7f180cee50 con 0x7f7f1806efa0 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.687+0000 7f7f15e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7f180726e0 0x7f7f180cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7f180726e0 0x7f7f180cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34024/0 (socket says 172.21.15.110:34024) 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15e00640 1 -- 172.21.15.110:0/531776144 learned_addr learned my addr 172.21.15.110:0/531776144 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15400640 1 --2- 172.21.15.110:0/531776144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f1806efa0 0x7f7f180b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15e00640 1 -- 172.21.15.110:0/531776144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f1806efa0 msgr2=0x7f7f180b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15e00640 1 --2- 172.21.15.110:0/531776144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f1806efa0 0x7f7f180b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15e00640 1 -- 172.21.15.110:0/531776144 >> v1:172.21.15.145:6789/0 conn(0x7f7f180b5430 legacy=0x7f7f180b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15e00640 1 -- 172.21.15.110:0/531776144 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1815fda0 con 0x7f7f180726e0 2024-09-10T10:02:49.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15400640 1 --2- 172.21.15.110:0/531776144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f1806efa0 0x7f7f180b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:49.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f15e00640 1 --2- 172.21.15.110:0/531776144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f180726e0 0x7f7f180cf2e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009e50 tx=0x7f7f0801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bef218751851c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:49.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f14a00640 1 -- 172.21.15.110:0/531776144 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080a58d0 con 0x7f7f180726e0 2024-09-10T10:02:49.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f14a00640 1 -- 172.21.15.110:0/531776144 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f080a5a70 con 0x7f7f180726e0 2024-09-10T10:02:49.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f14a00640 1 -- 172.21.15.110:0/531776144 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080a5e40 con 0x7f7f180726e0 2024-09-10T10:02:49.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f1d000640 1 -- 172.21.15.110:0/531776144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f180726e0 msgr2=0x7f7f180cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.691+0000 7f7f1d000640 1 --2- 172.21.15.110:0/531776144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f180726e0 0x7f7f180cf2e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009e50 tx=0x7f7f0801ca00 comp rx=0 tx=0).stop 2024-09-10T10:02:49.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- 172.21.15.110:0/531776144 shutdown_connections 2024-09-10T10:02:49.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 --2- 172.21.15.110:0/531776144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f1806efa0 0x7f7f180b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 --2- 172.21.15.110:0/531776144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f180726e0 0x7f7f180cf2e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- 172.21.15.110:0/531776144 >> 172.21.15.110:0/531776144 conn(0x7f7f18062d10 msgr2=0x7f7f1806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:49.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- 172.21.15.110:0/531776144 shutdown_connections 2024-09-10T10:02:49.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- 172.21.15.110:0/531776144 wait complete. 2024-09-10T10:02:49.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 Processor -- start 2024-09-10T10:02:49.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- start start 2024-09-10T10:02:49.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f1806efa0 0x7f7f1814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f180726e0 0x7f7f18155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f180b5430 0x7f7f18156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7f180b3ff0 con 0x7f7f1806efa0 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18073270 con 0x7f7f180b5430 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7f180b4170 con 0x7f7f180726e0 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f1806efa0 0x7f7f1814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f1806efa0 0x7f7f1814ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34034/0 (socket says 172.21.15.110:34034) 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 -- 172.21.15.110:0/1765885439 learned_addr learned my addr 172.21.15.110:0/1765885439 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15400640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f180726e0 0x7f7f18155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f16800640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f180b5430 0x7f7f18156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 -- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f180726e0 msgr2=0x7f7f18155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f180726e0 0x7f7f18155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 -- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f180b5430 msgr2=0x7f7f18156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f180b5430 0x7f7f18156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 -- 172.21.15.110:0/1765885439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f18155f70 con 0x7f7f1806efa0 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f16800640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f180b5430 0x7f7f18156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f15e00640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f1806efa0 0x7f7f1814ff60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7f0801a630 tx=0x7f7f080197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f0e000640 1 -- 172.21.15.110:0/1765885439 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080a58d0 con 0x7f7f1806efa0 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f0e000640 1 -- 172.21.15.110:0/1765885439 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f080090d0 con 0x7f7f1806efa0 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f18156210 con 0x7f7f1806efa0 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.695+0000 7f7f0e000640 1 -- 172.21.15.110:0/1765885439 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08009510 con 0x7f7f1806efa0 2024-09-10T10:02:49.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.699+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f181567d0 con 0x7f7f1806efa0 2024-09-10T10:02:49.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.699+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f18072ac0 con 0x7f7f1806efa0 2024-09-10T10:02:49.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.699+0000 7f7f0e000640 1 -- 172.21.15.110:0/1765885439 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f7f080c0020 con 0x7f7f1806efa0 2024-09-10T10:02:49.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.699+0000 7f7f0e000640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7ef8067a10 0x7f7ef8069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:49.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.699+0000 7f7f0e000640 1 -- 172.21.15.110:0/1765885439 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f7f08120ca0 con 0x7f7f1806efa0 2024-09-10T10:02:49.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.699+0000 7f7f15400640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7ef8067a10 0x7f7ef8069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:49.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.703+0000 7f7f15400640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7ef8067a10 0x7f7ef8069ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f1815b810 tx=0x7f7f04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:49.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.703+0000 7f7f0e000640 1 -- 172.21.15.110:0/1765885439 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7f080c7060 con 0x7f7f1806efa0 2024-09-10T10:02:49.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7f1806e500 con 0x7f7f1806efa0 2024-09-10T10:02:49.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f0e000640 1 -- 172.21.15.110:0/1765885439 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+124 (secure 0 0 0) 0x7f7f08016620 con 0x7f7f1806efa0 2024-09-10T10:02:49.883 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6834/3463064995 2024-09-11T10:02:45.702580+0000 2024-09-10T10:02:49.883 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6832/3463064995 2024-09-11T10:02:45.702580+0000 2024-09-10T10:02:49.883 INFO:teuthology.orchestra.run.smithi110.stderr:listed 2 entries 2024-09-10T10:02:49.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7ef8067a10 msgr2=0x7f7ef8069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7ef8067a10 0x7f7ef8069ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f1815b810 tx=0x7f7f04008040 comp rx=0 tx=0).stop 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f1806efa0 msgr2=0x7f7f1814ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f1806efa0 0x7f7f1814ff60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7f0801a630 tx=0x7f7f080197a0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 shutdown_connections 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7ef8067a10 0x7f7ef8069ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f180b5430 0x7f7f18156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f180726e0 0x7f7f18155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 --2- 172.21.15.110:0/1765885439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f1806efa0 0x7f7f1814ff60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 >> 172.21.15.110:0/1765885439 conn(0x7f7f18062d10 msgr2=0x7f7f180b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 shutdown_connections 2024-09-10T10:02:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:49.879+0000 7f7f1d000640 1 -- 172.21.15.110:0/1765885439 wait complete. 2024-09-10T10:02:49.904 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:02:49.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:49.904 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:02:49.932 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:02:49.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:02:49.932 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:02:49.988 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:02:49.988 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:49.988 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:49.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:02:49.997 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:02:49.997 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:49.997 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:50.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:02:50.043 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feaeb800640 1 Processor -- start 2024-09-10T10:02:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feaeb800640 1 -- start start 2024-09-10T10:02:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feaeb800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7feae415c560 0x7feae406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feaeb800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feae406f010 0x7feae40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feaeb800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae406f430 con 0x7feae4155970 2024-09-10T10:02:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feaeb800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7feae40b4570 con 0x7feae406f010 2024-09-10T10:02:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feaeb800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7feae4155d50 con 0x7feae415c560 2024-09-10T10:02:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feae8e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7feae4155970 0x7feae40cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49740/0 (socket says 172.21.15.110:49740) 2024-09-10T10:02:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feae8e00640 1 -- 172.21.15.110:0/1583626710 learned_addr learned my addr 172.21.15.110:0/1583626710 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feae9800640 1 --2- 172.21.15.110:0/1583626710 >> v2:172.21.15.110:3300/0 conn(0x7feae406f010 0x7feae40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.175+0000 7feae3e00640 1 --2- 172.21.15.110:0/1583626710 >> v2:172.21.15.145:3300/0 conn(0x7feae415c560 0x7feae406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae9800640 1 -- 172.21.15.110:0/1583626710 >> v2:172.21.15.145:3300/0 conn(0x7feae415c560 msgr2=0x7feae406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae9800640 1 --2- 172.21.15.110:0/1583626710 >> v2:172.21.15.145:3300/0 conn(0x7feae415c560 0x7feae406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae9800640 1 -- 172.21.15.110:0/1583626710 >> v1:172.21.15.110:6789/0 conn(0x7feae4155970 legacy=0x7feae40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae9800640 1 -- 172.21.15.110:0/1583626710 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae40b34d0 con 0x7feae406f010 2024-09-10T10:02:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae3400640 1 -- 172.21.15.110:0/1583626710 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577003615 0 0) 0x7feae406f430 con 0x7feae4155970 2024-09-10T10:02:50.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae9800640 1 --2- 172.21.15.110:0/1583626710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae40ce420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feae406d930 tx=0x7feadc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38534a8ee73b7769 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae3400640 1 -- 172.21.15.110:0/1583626710 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc0ab650 con 0x7feae406f010 2024-09-10T10:02:50.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae3400640 1 -- 172.21.15.110:0/1583626710 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc0b0070 con 0x7feae406f010 2024-09-10T10:02:50.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feae3400640 1 -- 172.21.15.110:0/1583626710 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc0aba10 con 0x7feae406f010 2024-09-10T10:02:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feaeb800640 1 -- 172.21.15.110:0/1583626710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 msgr2=0x7feae40ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.179+0000 7feaeb800640 1 --2- 172.21.15.110:0/1583626710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae40ce420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feae406d930 tx=0x7feadc01c450 comp rx=0 tx=0).stop 2024-09-10T10:02:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 -- 172.21.15.110:0/1583626710 shutdown_connections 2024-09-10T10:02:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 --2- 172.21.15.110:0/1583626710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae40ce420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 --2- 172.21.15.110:0/1583626710 >> v2:172.21.15.145:3300/0 conn(0x7feae415c560 0x7feae406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 -- 172.21.15.110:0/1583626710 >> 172.21.15.110:0/1583626710 conn(0x7feae406bc50 msgr2=0x7feae406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 -- 172.21.15.110:0/1583626710 shutdown_connections 2024-09-10T10:02:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 -- 172.21.15.110:0/1583626710 wait complete. 2024-09-10T10:02:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 Processor -- start 2024-09-10T10:02:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 -- start start 2024-09-10T10:02:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae41452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feae4155970 0x7feae4072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feae415c560 0x7feae4145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feae40cf170 con 0x7feae406f010 2024-09-10T10:02:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feae40b4050 con 0x7feae4155970 2024-09-10T10:02:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feaeb800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feae40cf2f0 con 0x7feae415c560 2024-09-10T10:02:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae41452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae41452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34060/0 (socket says 172.21.15.110:34060) 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 -- 172.21.15.110:0/4107231096 learned_addr learned my addr 172.21.15.110:0/4107231096 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae3e00640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feae4155970 0x7feae4072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae9800640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feae415c560 0x7feae4145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 -- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feae415c560 msgr2=0x7feae4145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feae415c560 0x7feae4145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 -- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feae4155970 msgr2=0x7feae4072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feae4155970 0x7feae4072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 -- 172.21.15.110:0/4107231096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae41457e0 con 0x7feae406f010 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae9800640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feae415c560 0x7feae4145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae8e00640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae41452a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fead400c890 tx=0x7fead400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae0c00640 1 -- 172.21.15.110:0/4107231096 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4018070 con 0x7feae406f010 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae0c00640 1 -- 172.21.15.110:0/4107231096 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead400a640 con 0x7feae406f010 2024-09-10T10:02:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.183+0000 7feae0c00640 1 -- 172.21.15.110:0/4107231096 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead400aa30 con 0x7feae406f010 2024-09-10T10:02:50.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.187+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae4142ba0 con 0x7feae406f010 2024-09-10T10:02:50.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.187+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae4143160 con 0x7feae406f010 2024-09-10T10:02:50.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.187+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae40cf650 con 0x7feae406f010 2024-09-10T10:02:50.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.187+0000 7feae0c00640 1 -- 172.21.15.110:0/4107231096 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fead400abd0 con 0x7feae406f010 2024-09-10T10:02:50.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.187+0000 7feae0c00640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feac40679e0 0x7feac4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.187+0000 7feae0c00640 1 -- 172.21.15.110:0/4107231096 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fead40854b0 con 0x7feae406f010 2024-09-10T10:02:50.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.187+0000 7feae3e00640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feac40679e0 0x7feac4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.191+0000 7feae3e00640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feac40679e0 0x7feac4069ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feae40730e0 tx=0x7fead0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.191+0000 7feae0c00640 1 -- 172.21.15.110:0/4107231096 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feae41457e0 con 0x7feae406f010 2024-09-10T10:02:50.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.379+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feae406eb00 con 0x7feae406f010 2024-09-10T10:02:50.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.379+0000 7feae0c00640 1 -- 172.21.15.110:0/4107231096 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5237 (secure 0 0 0) 0x7fead4055790 con 0x7feae406f010 2024-09-10T10:02:50.385 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:50.385 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":60,"btime":"2024-09-10T10:02:46:704608+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1164773100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1164773100},{"type":"v1","addr":"172.21.15.145:6834","nonce":1164773100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1195578364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1195578364},{"type":"v1","addr":"172.21.15.145:6835","nonce":1195578364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6651,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3822482715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3822482715},{"type":"v1","addr":"172.21.15.110:6834","nonce":3822482715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:02:30.912482+0000","modified":"2024-09-10T10:02:33.921168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1947643553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1947643553},{"type":"v1","addr":"172.21.15.110:6835","nonce":1947643553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":6}]} 2024-09-10T10:02:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 60 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feac40679e0 msgr2=0x7feac4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feac40679e0 0x7feac4069ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feae40730e0 tx=0x7fead0009190 comp rx=0 tx=0).stop 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 msgr2=0x7feae41452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae41452a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fead400c890 tx=0x7fead400cd50 comp rx=0 tx=0).stop 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 shutdown_connections 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feac40679e0 0x7feac4069ea0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feae415c560 0x7feae4145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feae4155970 0x7feae4072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 --2- 172.21.15.110:0/4107231096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feae406f010 0x7feae41452a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 >> 172.21.15.110:0/4107231096 conn(0x7feae406bc50 msgr2=0x7feae40cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 shutdown_connections 2024-09-10T10:02:50.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.383+0000 7feaeb800640 1 -- 172.21.15.110:0/4107231096 wait complete. 2024-09-10T10:02:50.405 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:50.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.535+0000 7fa542c00640 1 Processor -- start 2024-09-10T10:02:50.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.535+0000 7fa542c00640 1 -- start start 2024-09-10T10:02:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa542c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa53c15c330 0x7fa53c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa542c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa53c15c710 0x7fa53c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa542c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c0cff20 0x7fa53c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa542c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa53c0703d0 con 0x7fa53c15c710 2024-09-10T10:02:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa542c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa53c0ceab0 con 0x7fa53c15c330 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa542c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa53c0b0510 con 0x7fa53c0cff20 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa53c15c710 0x7fa53c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa53c15c710 0x7fa53c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34066/0 (socket says 172.21.15.110:34066) 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53b400640 1 -- 172.21.15.110:0/2512800199 learned_addr learned my addr 172.21.15.110:0/2512800199 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa540c00640 1 --2- 172.21.15.110:0/2512800199 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c0cff20 0x7fa53c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53be00640 1 --2- 172.21.15.110:0/2512800199 >> v2:172.21.15.145:3300/0 conn(0x7fa53c15c330 0x7fa53c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53b400640 1 -- 172.21.15.110:0/2512800199 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c0cff20 msgr2=0x7fa53c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53b400640 1 --2- 172.21.15.110:0/2512800199 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c0cff20 0x7fa53c0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53b400640 1 -- 172.21.15.110:0/2512800199 >> v2:172.21.15.145:3300/0 conn(0x7fa53c15c330 msgr2=0x7fa53c06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:50.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53b400640 1 --2- 172.21.15.110:0/2512800199 >> v2:172.21.15.145:3300/0 conn(0x7fa53c15c330 0x7fa53c06fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.539+0000 7fa53b400640 1 -- 172.21.15.110:0/2512800199 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c15d930 con 0x7fa53c15c710 2024-09-10T10:02:50.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa53b400640 1 --2- 172.21.15.110:0/2512800199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c710 0x7fa53c06dc60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0744a0 tx=0x7fa52801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9317d3df3650ff88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa53aa00640 1 -- 172.21.15.110:0/2512800199 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280aa610 con 0x7fa53c15c710 2024-09-10T10:02:50.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa53aa00640 1 -- 172.21.15.110:0/2512800199 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5280af070 con 0x7fa53c15c710 2024-09-10T10:02:50.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa53aa00640 1 -- 172.21.15.110:0/2512800199 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280aa9d0 con 0x7fa53c15c710 2024-09-10T10:02:50.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 -- 172.21.15.110:0/2512800199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c710 msgr2=0x7fa53c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 --2- 172.21.15.110:0/2512800199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c710 0x7fa53c06dc60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0744a0 tx=0x7fa52801c4f0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 -- 172.21.15.110:0/2512800199 shutdown_connections 2024-09-10T10:02:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 --2- 172.21.15.110:0/2512800199 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c0cff20 0x7fa53c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 --2- 172.21.15.110:0/2512800199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c710 0x7fa53c06dc60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 --2- 172.21.15.110:0/2512800199 >> v2:172.21.15.145:3300/0 conn(0x7fa53c15c330 0x7fa53c06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 -- 172.21.15.110:0/2512800199 >> 172.21.15.110:0/2512800199 conn(0x7fa53c06cb90 msgr2=0x7fa53c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 -- 172.21.15.110:0/2512800199 shutdown_connections 2024-09-10T10:02:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 -- 172.21.15.110:0/2512800199 wait complete. 2024-09-10T10:02:50.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 Processor -- start 2024-09-10T10:02:50.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.543+0000 7fa542c00640 1 -- start start 2024-09-10T10:02:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa53c0cff20 0x7fa53c14b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c330 0x7fa53c14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c15c710 0x7fa53c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c0d0890 con 0x7fa53c15c330 2024-09-10T10:02:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c06e1a0 con 0x7fa53c0cff20 2024-09-10T10:02:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa53c15d500 con 0x7fa53c15c710 2024-09-10T10:02:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c330 0x7fa53c14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c330 0x7fa53c14fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34072/0 (socket says 172.21.15.110:34072) 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 -- 172.21.15.110:0/3624740560 learned_addr learned my addr 172.21.15.110:0/3624740560 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53be00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa53c0cff20 0x7fa53c14b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa540c00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c15c710 0x7fa53c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 -- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c15c710 msgr2=0x7fa53c14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c15c710 0x7fa53c14ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 -- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa53c0cff20 msgr2=0x7fa53c14b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa53c0cff20 0x7fa53c14b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 -- 172.21.15.110:0/3624740560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c156a60 con 0x7fa53c15c330 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa53b400640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c330 0x7fa53c14fa90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa52801aba0 tx=0x7fa52801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa533e00640 1 -- 172.21.15.110:0/3624740560 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280bd5f0 con 0x7fa53c15c330 2024-09-10T10:02:50.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa533e00640 1 -- 172.21.15.110:0/3624740560 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5280bd790 con 0x7fa53c15c330 2024-09-10T10:02:50.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa533e00640 1 -- 172.21.15.110:0/3624740560 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280bdb80 con 0x7fa53c15c330 2024-09-10T10:02:50.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c150ea0 con 0x7fa53c15c330 2024-09-10T10:02:50.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c1508e0 con 0x7fa53c15c330 2024-09-10T10:02:50.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53c1f6e00 con 0x7fa53c15c330 2024-09-10T10:02:50.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa533e00640 1 -- 172.21.15.110:0/3624740560 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fa5280b33f0 con 0x7fa53c15c330 2024-09-10T10:02:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.547+0000 7fa533e00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa51c0679e0 0x7fa51c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.551+0000 7fa533e00640 1 -- 172.21.15.110:0/3624740560 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fa5281207a0 con 0x7fa53c15c330 2024-09-10T10:02:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.551+0000 7fa53be00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa51c0679e0 0x7fa51c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.551+0000 7fa53be00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa51c0679e0 0x7fa51c069ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0074a0 tx=0x7fa52c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.551+0000 7fa533e00640 1 -- 172.21.15.110:0/3624740560 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa5280a7070 con 0x7fa53c15c330 2024-09-10T10:02:50.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa53c156c30 con 0x7fa53c15c330 2024-09-10T10:02:50.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa533e00640 1 -- 172.21.15.110:0/3624740560 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5237 (secure 0 0 0) 0x7fa528016110 con 0x7fa53c15c330 2024-09-10T10:02:50.746 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:50.746 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":60,"btime":"2024-09-10T10:02:46:704608+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1164773100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1164773100},{"type":"v1","addr":"172.21.15.145:6834","nonce":1164773100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1195578364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1195578364},{"type":"v1","addr":"172.21.15.145:6835","nonce":1195578364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6651,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3822482715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3822482715},{"type":"v1","addr":"172.21.15.110:6834","nonce":3822482715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:02:30.912482+0000","modified":"2024-09-10T10:02:33.921168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1947643553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1947643553},{"type":"v1","addr":"172.21.15.110:6835","nonce":1947643553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":6}]} 2024-09-10T10:02:50.746 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 60 2024-09-10T10:02:50.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa51c0679e0 msgr2=0x7fa51c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa51c0679e0 0x7fa51c069ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0074a0 tx=0x7fa52c00b040 comp rx=0 tx=0).stop 2024-09-10T10:02:50.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c330 msgr2=0x7fa53c14fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c330 0x7fa53c14fa90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa52801aba0 tx=0x7fa52801ad30 comp rx=0 tx=0).stop 2024-09-10T10:02:50.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 shutdown_connections 2024-09-10T10:02:50.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa51c0679e0 0x7fa51c069ea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa53c15c710 0x7fa53c14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa53c15c330 0x7fa53c14fa90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 --2- 172.21.15.110:0/3624740560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa53c0cff20 0x7fa53c14b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 >> 172.21.15.110:0/3624740560 conn(0x7fa53c06cb90 msgr2=0x7fa53c06cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 shutdown_connections 2024-09-10T10:02:50.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.743+0000 7fa542c00640 1 -- 172.21.15.110:0/3624740560 wait complete. 2024-09-10T10:02:50.767 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:50.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5aaa00640 1 Processor -- start 2024-09-10T10:02:50.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5aaa00640 1 -- start start 2024-09-10T10:02:50.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5aaa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5aaa00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a406f430 con 0x7fe5a406f010 2024-09-10T10:02:50.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5aaa00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a4155d50 con 0x7fe5a4155970 2024-09-10T10:02:50.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5aaa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a40b4570 con 0x7fe5a415c560 2024-09-10T10:02:50.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5a8a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe5a406f010 0x7fe5a40ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49756/0 (socket says 172.21.15.110:49756) 2024-09-10T10:02:50.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5a8a00640 1 -- 172.21.15.110:0/334652877 learned_addr learned my addr 172.21.15.110:0/334652877 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:50.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe59b400640 1 --2- 172.21.15.110:0/334652877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5a3400640 1 -- 172.21.15.110:0/334652877 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994351313 0 0) 0x7fe5a406f430 con 0x7fe5a406f010 2024-09-10T10:02:50.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5a3400640 1 -- 172.21.15.110:0/334652877 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe588003530 con 0x7fe5a406f010 2024-09-10T10:02:50.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5a3400640 1 -- 172.21.15.110:0/334652877 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2117823770 0 0) 0x7fe5a4155d50 con 0x7fe5a4155970 2024-09-10T10:02:50.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe5a3400640 1 -- 172.21.15.110:0/334652877 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5a406f430 con 0x7fe5a4155970 2024-09-10T10:02:50.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe59b400640 1 -- 172.21.15.110:0/334652877 >> v1:172.21.15.145:6789/0 conn(0x7fe5a4155970 legacy=0x7fe5a40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe59b400640 1 -- 172.21.15.110:0/334652877 >> v1:172.21.15.110:6789/0 conn(0x7fe5a406f010 legacy=0x7fe5a40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.899+0000 7fe59b400640 1 -- 172.21.15.110:0/334652877 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a40b34d0 con 0x7fe5a415c560 2024-09-10T10:02:50.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe59b400640 1 --2- 172.21.15.110:0/334652877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a406ead0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0097f0 tx=0x7fe58c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab68de0535ef7975 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5a3400640 1 -- 172.21.15.110:0/334652877 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0ab620 con 0x7fe5a415c560 2024-09-10T10:02:50.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5a3400640 1 -- 172.21.15.110:0/334652877 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58c0b0070 con 0x7fe5a415c560 2024-09-10T10:02:50.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5a3400640 1 -- 172.21.15.110:0/334652877 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0ab9e0 con 0x7fe5a415c560 2024-09-10T10:02:50.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5aaa00640 1 -- 172.21.15.110:0/334652877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 msgr2=0x7fe5a406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5aaa00640 1 --2- 172.21.15.110:0/334652877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a406ead0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0097f0 tx=0x7fe58c01c420 comp rx=0 tx=0).stop 2024-09-10T10:02:50.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5aaa00640 1 -- 172.21.15.110:0/334652877 shutdown_connections 2024-09-10T10:02:50.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5aaa00640 1 --2- 172.21.15.110:0/334652877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a406ead0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5aaa00640 1 -- 172.21.15.110:0/334652877 >> 172.21.15.110:0/334652877 conn(0x7fe5a406bc50 msgr2=0x7fe5a406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5aaa00640 1 -- 172.21.15.110:0/334652877 shutdown_connections 2024-09-10T10:02:50.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.903+0000 7fe5aaa00640 1 -- 172.21.15.110:0/334652877 wait complete. 2024-09-10T10:02:50.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 Processor -- start 2024-09-10T10:02:50.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 -- start start 2024-09-10T10:02:50.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a406f010 0x7fe5a41583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5a4155970 0x7fe5a4156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a4146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40cf340 con 0x7fe5a406f010 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40b4050 con 0x7fe5a4155970 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a40cf4c0 con 0x7fe5a415c560 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a406f010 0x7fe5a41583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a406f010 0x7fe5a41583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34088/0 (socket says 172.21.15.110:34088) 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 -- 172.21.15.110:0/319184641 learned_addr learned my addr 172.21.15.110:0/319184641 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe59b400640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5a4155970 0x7fe5a4156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a8a00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a4146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 -- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 msgr2=0x7fe5a4146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a4146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 -- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5a4155970 msgr2=0x7fe5a4156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5a4155970 0x7fe5a4156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 -- 172.21.15.110:0/319184641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a415f6f0 con 0x7fe5a406f010 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a3e00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a406f010 0x7fe5a41583c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe59400c490 tx=0x7fe59400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a0c00640 1 -- 172.21.15.110:0/319184641 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940052d0 con 0x7fe5a406f010 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a0c00640 1 -- 172.21.15.110:0/319184641 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe594005470 con 0x7fe5a406f010 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a41560a0 con 0x7fe5a406f010 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5a0c00640 1 -- 172.21.15.110:0/319184641 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594012870 con 0x7fe5a406f010 2024-09-10T10:02:50.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a4156660 con 0x7fe5a406f010 2024-09-10T10:02:50.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.907+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a406ead0 con 0x7fe5a406f010 2024-09-10T10:02:50.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.911+0000 7fe5a0c00640 1 -- 172.21.15.110:0/319184641 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fe594012a10 con 0x7fe5a406f010 2024-09-10T10:02:50.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.911+0000 7fe5a0c00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5800679e0 0x7fe580069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.911+0000 7fe5a0c00640 1 -- 172.21.15.110:0/319184641 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fe594085a70 con 0x7fe5a406f010 2024-09-10T10:02:50.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.911+0000 7fe59b400640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5800679e0 0x7fe580069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.911+0000 7fe59b400640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5800679e0 0x7fe580069ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0af000 tx=0x7fe58c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:50.915+0000 7fe5a0c00640 1 -- 172.21.15.110:0/319184641 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe594055d60 con 0x7fe5a406f010 2024-09-10T10:02:51.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.071+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5a40b0570 con 0x7fe5a406f010 2024-09-10T10:02:51.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.071+0000 7fe5a0c00640 1 -- 172.21.15.110:0/319184641 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16887 (secure 0 0 0) 0x7fe59405a060 con 0x7fe5a406f010 2024-09-10T10:02:51.077 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:51.077 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":91,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:48.709541+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:02:28.619234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T10:02:29.912872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6834/3463064995":"2024-09-11T10:02:45.702580+0000","172.21.15.110:6832/3463064995":"2024-09-11T10:02:45.702580+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:51.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5800679e0 msgr2=0x7fe580069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5800679e0 0x7fe580069ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0af000 tx=0x7fe58c0a7000 comp rx=0 tx=0).stop 2024-09-10T10:02:51.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a406f010 msgr2=0x7fe5a41583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a406f010 0x7fe5a41583c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe59400c490 tx=0x7fe59400c950 comp rx=0 tx=0).stop 2024-09-10T10:02:51.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 shutdown_connections 2024-09-10T10:02:51.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5800679e0 0x7fe580069ea0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5a415c560 0x7fe5a4146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5a4155970 0x7fe5a4156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 --2- 172.21.15.110:0/319184641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a406f010 0x7fe5a41583c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 >> 172.21.15.110:0/319184641 conn(0x7fe5a406bc50 msgr2=0x7fe5a40ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:51.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 shutdown_connections 2024-09-10T10:02:51.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.075+0000 7fe5aaa00640 1 -- 172.21.15.110:0/319184641 wait complete. 2024-09-10T10:02:51.098 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:02:51.098 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:51.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd9e000640 1 Processor -- start 2024-09-10T10:02:51.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd9e000640 1 -- start start 2024-09-10T10:02:51.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd9e000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efd98144640 0x7efd980719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd9e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd980b5380 0x7efd98072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd9e000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd980722c0 con 0x7efd98073240 2024-09-10T10:02:51.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd9e000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7efd980725a0 con 0x7efd98144640 2024-09-10T10:02:51.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd9e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efd98072880 con 0x7efd980b5380 2024-09-10T10:02:51.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd97400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efd98144640 0x7efd980719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd97400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efd98144640 0x7efd980719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34102/0 (socket says 172.21.15.110:34102) 2024-09-10T10:02:51.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd97400640 1 -- 172.21.15.110:0/3028797051 learned_addr learned my addr 172.21.15.110:0/3028797051 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:51.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd96a00640 1 --2- 172.21.15.110:0/3028797051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd980b5380 0x7efd98072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd97400640 1 -- 172.21.15.110:0/3028797051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd980b5380 msgr2=0x7efd98072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:51.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd97400640 1 --2- 172.21.15.110:0/3028797051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd980b5380 0x7efd98072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd97400640 1 -- 172.21.15.110:0/3028797051 >> v1:172.21.15.145:6789/0 conn(0x7efd98073240 legacy=0x7efd980b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:51.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.231+0000 7efd97400640 1 -- 172.21.15.110:0/3028797051 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd980ce600 con 0x7efd98144640 2024-09-10T10:02:51.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd97400640 1 --2- 172.21.15.110:0/3028797051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98144640 0x7efd980719c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7efd88006fd0 tx=0x7efd8801e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a275541700923c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd96000640 1 -- 172.21.15.110:0/3028797051 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880ad070 con 0x7efd98144640 2024-09-10T10:02:51.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd96000640 1 -- 172.21.15.110:0/3028797051 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd88004230 con 0x7efd98144640 2024-09-10T10:02:51.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd96000640 1 -- 172.21.15.110:0/3028797051 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88003620 con 0x7efd98144640 2024-09-10T10:02:51.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 -- 172.21.15.110:0/3028797051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98144640 msgr2=0x7efd980719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 --2- 172.21.15.110:0/3028797051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98144640 0x7efd980719c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7efd88006fd0 tx=0x7efd8801e9b0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 -- 172.21.15.110:0/3028797051 shutdown_connections 2024-09-10T10:02:51.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 --2- 172.21.15.110:0/3028797051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd980b5380 0x7efd98072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 --2- 172.21.15.110:0/3028797051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98144640 0x7efd980719c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 -- 172.21.15.110:0/3028797051 >> 172.21.15.110:0/3028797051 conn(0x7efd9806cb90 msgr2=0x7efd9806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:51.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 -- 172.21.15.110:0/3028797051 shutdown_connections 2024-09-10T10:02:51.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 -- 172.21.15.110:0/3028797051 wait complete. 2024-09-10T10:02:51.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 Processor -- start 2024-09-10T10:02:51.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.235+0000 7efd9e000640 1 -- start start 2024-09-10T10:02:51.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98073240 0x7efd98158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd980b5380 0x7efd98142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd98144640 0x7efd9813c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd980b4880 con 0x7efd98073240 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd980b4700 con 0x7efd980b5380 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efd980b4a00 con 0x7efd98144640 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98073240 0x7efd98158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98073240 0x7efd98158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34108/0 (socket says 172.21.15.110:34108) 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 -- 172.21.15.110:0/1158307399 learned_addr learned my addr 172.21.15.110:0/1158307399 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97e00640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd98144640 0x7efd9813c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd96a00640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd980b5380 0x7efd98142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 -- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd98144640 msgr2=0x7efd9813c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd98144640 0x7efd9813c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 -- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd980b5380 msgr2=0x7efd98142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd980b5380 0x7efd98142d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 -- 172.21.15.110:0/1158307399 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9815e750 con 0x7efd98073240 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97e00640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd98144640 0x7efd9813c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd97400640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98073240 0x7efd98158c60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7efd88000c10 tx=0x7efd8801b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd8f400640 1 -- 172.21.15.110:0/1158307399 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880c1070 con 0x7efd98073240 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd8f400640 1 -- 172.21.15.110:0/1158307399 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd880a75c0 con 0x7efd98073240 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd8f400640 1 -- 172.21.15.110:0/1158307399 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880ad040 con 0x7efd98073240 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9815ea30 con 0x7efd98073240 2024-09-10T10:02:51.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9815eff0 con 0x7efd98073240 2024-09-10T10:02:51.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd98072d00 con 0x7efd98073240 2024-09-10T10:02:51.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.239+0000 7efd8f400640 1 -- 172.21.15.110:0/1158307399 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7efd880c4070 con 0x7efd98073240 2024-09-10T10:02:51.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.243+0000 7efd8f400640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efd780679e0 0x7efd78069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.243+0000 7efd8f400640 1 -- 172.21.15.110:0/1158307399 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7efd881221c0 con 0x7efd98073240 2024-09-10T10:02:51.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.243+0000 7efd96a00640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efd780679e0 0x7efd78069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.243+0000 7efd96a00640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efd780679e0 0x7efd78069ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd98071ed0 tx=0x7efd84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.247+0000 7efd8f400640 1 -- 172.21.15.110:0/1158307399 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efd880185a0 con 0x7efd98073240 2024-09-10T10:02:51.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd980d0820 con 0x7efd98073240 2024-09-10T10:02:51.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd8f400640 1 -- 172.21.15.110:0/1158307399 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5237 (secure 0 0 0) 0x7efd880f2af0 con 0x7efd98073240 2024-09-10T10:02:51.438 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:51.438 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":60,"btime":"2024-09-10T10:02:46:704608+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1164773100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1164773100},{"type":"v1","addr":"172.21.15.145:6834","nonce":1164773100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1195578364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1195578364},{"type":"v1","addr":"172.21.15.145:6835","nonce":1195578364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6651,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3822482715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3822482715},{"type":"v1","addr":"172.21.15.110:6834","nonce":3822482715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:02:30.912482+0000","modified":"2024-09-10T10:02:33.921168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1947643553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1947643553},{"type":"v1","addr":"172.21.15.110:6835","nonce":1947643553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":6}]} 2024-09-10T10:02:51.439 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 60 2024-09-10T10:02:51.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efd780679e0 msgr2=0x7efd78069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efd780679e0 0x7efd78069ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd98071ed0 tx=0x7efd84008040 comp rx=0 tx=0).stop 2024-09-10T10:02:51.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98073240 msgr2=0x7efd98158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98073240 0x7efd98158c60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7efd88000c10 tx=0x7efd8801b750 comp rx=0 tx=0).stop 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 shutdown_connections 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efd780679e0 0x7efd78069ea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efd98144640 0x7efd9813c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd980b5380 0x7efd98142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 --2- 172.21.15.110:0/1158307399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98073240 0x7efd98158c60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 >> 172.21.15.110:0/1158307399 conn(0x7efd9806cb90 msgr2=0x7efd9815cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 shutdown_connections 2024-09-10T10:02:51.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.435+0000 7efd9e000640 1 -- 172.21.15.110:0/1158307399 wait complete. 2024-09-10T10:02:51.459 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:51.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b5bc00640 1 Processor -- start 2024-09-10T10:02:51.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b5bc00640 1 -- start start 2024-09-10T10:02:51.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b5bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 0x7f8b54071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b5bc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8b540cd510 0x7f8b540cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b5bc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b5406d2d0 con 0x7f8b5406e090 2024-09-10T10:02:51.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b5bc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8b540cdeb0 con 0x7f8b540cd510 2024-09-10T10:02:51.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b5bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5415d9a0 con 0x7f8b540cfe30 2024-09-10T10:02:51.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b53e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8b540cd510 0x7f8b540cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b53e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8b540cd510 0x7f8b540cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34122/0 (socket says 172.21.15.110:34122) 2024-09-10T10:02:51.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b53e00640 1 -- 172.21.15.110:0/3416527972 learned_addr learned my addr 172.21.15.110:0/3416527972 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:51.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.591+0000 7f8b59200640 1 --2- 172.21.15.110:0/3416527972 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 0x7f8b54071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b53e00640 1 -- 172.21.15.110:0/3416527972 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 msgr2=0x7f8b54071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:51.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b53e00640 1 --2- 172.21.15.110:0/3416527972 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 0x7f8b54071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b53e00640 1 -- 172.21.15.110:0/3416527972 >> v1:172.21.15.145:6789/0 conn(0x7f8b5406e090 legacy=0x7f8b5406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:51.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b53e00640 1 -- 172.21.15.110:0/3416527972 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5406d5b0 con 0x7f8b540cd510 2024-09-10T10:02:51.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b53e00640 1 --2- 172.21.15.110:0/3416527972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b540cd970 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009870 tx=0x7f8b4001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b51d1f6c828e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b53400640 1 -- 172.21.15.110:0/3416527972 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b1070 con 0x7f8b540cd510 2024-09-10T10:02:51.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b53400640 1 -- 172.21.15.110:0/3416527972 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b400a8050 con 0x7f8b540cd510 2024-09-10T10:02:51.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b53400640 1 -- 172.21.15.110:0/3416527972 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b400ac860 con 0x7f8b540cd510 2024-09-10T10:02:51.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b5bc00640 1 -- 172.21.15.110:0/3416527972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 msgr2=0x7f8b540cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.595+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/3416527972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b540cd970 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009870 tx=0x7f8b4001c4d0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 -- 172.21.15.110:0/3416527972 shutdown_connections 2024-09-10T10:02:51.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/3416527972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b540cd970 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/3416527972 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 0x7f8b54071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 -- 172.21.15.110:0/3416527972 >> 172.21.15.110:0/3416527972 conn(0x7f8b5406bc50 msgr2=0x7f8b5406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:51.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 -- 172.21.15.110:0/3416527972 shutdown_connections 2024-09-10T10:02:51.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 -- 172.21.15.110:0/3416527972 wait complete. 2024-09-10T10:02:51.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 Processor -- start 2024-09-10T10:02:51.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 -- start start 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b5406e090 0x7f8b541562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b54157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 0x7f8b54157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54072190 con 0x7f8b540cd510 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5415d640 con 0x7f8b5406e090 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b5bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8b54169180 con 0x7f8b540cfe30 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b54157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b54157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34136/0 (socket says 172.21.15.110:34136) 2024-09-10T10:02:51.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b59200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b5406e090 0x7f8b541562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 -- 172.21.15.110:0/1727838492 learned_addr learned my addr 172.21.15.110:0/1727838492 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b59c00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 0x7f8b54157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 -- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 msgr2=0x7f8b54157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 0x7f8b54157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 -- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b5406e090 msgr2=0x7f8b541562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b5406e090 0x7f8b541562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 -- 172.21.15.110:0/1727838492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b54153b00 con 0x7f8b540cd510 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.599+0000 7f8b53e00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b54157150 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8b4001ab80 tx=0x7f8b4001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b50c00640 1 -- 172.21.15.110:0/1727838492 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b400bc070 con 0x7f8b540cd510 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b50c00640 1 -- 172.21.15.110:0/1727838492 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b400b7630 con 0x7f8b540cd510 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b50c00640 1 -- 172.21.15.110:0/1727838492 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b1040 con 0x7f8b540cd510 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b54153de0 con 0x7f8b540cd510 2024-09-10T10:02:51.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b54154820 con 0x7f8b540cd510 2024-09-10T10:02:51.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b540cd970 con 0x7f8b540cd510 2024-09-10T10:02:51.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b50c00640 1 -- 172.21.15.110:0/1727838492 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f8b400ab3e0 con 0x7f8b540cd510 2024-09-10T10:02:51.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b50c00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8b340679e0 0x7f8b34069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b50c00640 1 -- 172.21.15.110:0/1727838492 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f8b4011f960 con 0x7f8b540cd510 2024-09-10T10:02:51.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.603+0000 7f8b59200640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8b340679e0 0x7f8b34069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.607+0000 7f8b59200640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8b340679e0 0x7f8b34069ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b54156d70 tx=0x7f8b44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.607+0000 7f8b50c00640 1 -- 172.21.15.110:0/1727838492 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8b40016150 con 0x7f8b540cd510 2024-09-10T10:02:51.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.803+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b54154a20 con 0x7f8b540cd510 2024-09-10T10:02:51.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.803+0000 7f8b50c00640 1 -- 172.21.15.110:0/1727838492 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5237 (secure 0 0 0) 0x7f8b4011f020 con 0x7f8b540cd510 2024-09-10T10:02:51.808 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:51.808 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":60,"btime":"2024-09-10T10:02:46:704608+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1164773100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1164773100},{"type":"v1","addr":"172.21.15.145:6834","nonce":1164773100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1195578364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1195578364},{"type":"v1","addr":"172.21.15.145:6835","nonce":1195578364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6651,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3822482715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3822482715},{"type":"v1","addr":"172.21.15.110:6834","nonce":3822482715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:02:30.912482+0000","modified":"2024-09-10T10:02:33.921168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6453},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6453":{"gid":6453,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1947643553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1947643553},{"type":"v1","addr":"172.21.15.110:6835","nonce":1947643553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6453,"qdb_cluster":[6453]},"id":6}]} 2024-09-10T10:02:51.808 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 60 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8b340679e0 msgr2=0x7f8b34069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8b340679e0 0x7f8b34069ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b54156d70 tx=0x7f8b44008040 comp rx=0 tx=0).stop 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 msgr2=0x7f8b54157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b54157150 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8b4001ab80 tx=0x7f8b4001ad10 comp rx=0 tx=0).stop 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 shutdown_connections 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8b340679e0 0x7f8b34069ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8b540cfe30 0x7f8b54157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b540cd510 0x7f8b54157150 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 --2- 172.21.15.110:0/1727838492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8b5406e090 0x7f8b541562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 >> 172.21.15.110:0/1727838492 conn(0x7f8b5406bc50 msgr2=0x7f8b5406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 shutdown_connections 2024-09-10T10:02:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.807+0000 7f8b5bc00640 1 -- 172.21.15.110:0/1727838492 wait complete. 2024-09-10T10:02:51.829 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:51.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.959+0000 7fcf24800640 1 Processor -- start 2024-09-10T10:02:51.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.959+0000 7fcf24800640 1 -- start start 2024-09-10T10:02:51.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf24800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 0x7fcf1c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf24800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcf1c06f010 0x7fcf1c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf24800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1c06f430 con 0x7fcf1c155970 2024-09-10T10:02:51.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf24800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcf1c0b4570 con 0x7fcf1c06f010 2024-09-10T10:02:51.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf24800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c155d50 con 0x7fcf1c15c560 2024-09-10T10:02:51.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf22800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcf1c06f010 0x7fcf1c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf21400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 0x7fcf1c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf22800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcf1c06f010 0x7fcf1c0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:45260/0 (socket says 172.21.15.110:45260) 2024-09-10T10:02:51.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf22800640 1 -- 172.21.15.110:0/1349528910 learned_addr learned my addr 172.21.15.110:0/1349528910 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:51.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf21e00640 1 --1- 172.21.15.110:0/1349528910 >> v1:172.21.15.145:6789/0 conn(0x7fcf1c155970 0x7fcf1c0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:35618/0 (socket says 172.21.15.110:35618) 2024-09-10T10:02:51.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf22800640 1 -- 172.21.15.110:0/1349528910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 msgr2=0x7fcf1c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:51.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf22800640 1 --2- 172.21.15.110:0/1349528910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 0x7fcf1c06ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf22800640 1 -- 172.21.15.110:0/1349528910 >> v1:172.21.15.145:6789/0 conn(0x7fcf1c155970 legacy=0x7fcf1c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.963+0000 7fcf22800640 1 -- 172.21.15.110:0/1349528910 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c0b34d0 con 0x7fcf1c06f010 2024-09-10T10:02:51.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf22800640 1 --2- 172.21.15.110:0/1349528910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c06f010 0x7fcf1c0ce420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009e50 tx=0x7fcf1801eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed172a3f6356042 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf20a00640 1 -- 172.21.15.110:0/1349528910 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf180a7a50 con 0x7fcf1c06f010 2024-09-10T10:02:51.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf20a00640 1 -- 172.21.15.110:0/1349528910 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf180a7bf0 con 0x7fcf1c06f010 2024-09-10T10:02:51.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf20a00640 1 -- 172.21.15.110:0/1349528910 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf180af3f0 con 0x7fcf1c06f010 2024-09-10T10:02:51.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 -- 172.21.15.110:0/1349528910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c06f010 msgr2=0x7fcf1c0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 --2- 172.21.15.110:0/1349528910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c06f010 0x7fcf1c0ce420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009e50 tx=0x7fcf1801eb80 comp rx=0 tx=0).stop 2024-09-10T10:02:51.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 -- 172.21.15.110:0/1349528910 shutdown_connections 2024-09-10T10:02:51.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 --2- 172.21.15.110:0/1349528910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c06f010 0x7fcf1c0ce420 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 --2- 172.21.15.110:0/1349528910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 0x7fcf1c06ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 -- 172.21.15.110:0/1349528910 >> 172.21.15.110:0/1349528910 conn(0x7fcf1c06bc50 msgr2=0x7fcf1c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:51.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 -- 172.21.15.110:0/1349528910 shutdown_connections 2024-09-10T10:02:51.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 -- 172.21.15.110:0/1349528910 wait complete. 2024-09-10T10:02:51.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 Processor -- start 2024-09-10T10:02:51.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.967+0000 7fcf24800640 1 -- start start 2024-09-10T10:02:51.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf1c06f010 0x7fcf1c14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c155970 0x7fcf1c156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 0x7fcf1c1643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0cf170 con 0x7fcf1c06f010 2024-09-10T10:02:51.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0b4050 con 0x7fcf1c155970 2024-09-10T10:02:51.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c0cf2f0 con 0x7fcf1c15c560 2024-09-10T10:02:51.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf1c06f010 0x7fcf1c14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf1c06f010 0x7fcf1c14ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34152/0 (socket says 172.21.15.110:34152) 2024-09-10T10:02:51.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 -- 172.21.15.110:0/2783157658 learned_addr learned my addr 172.21.15.110:0/2783157658 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:51.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf22800640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 0x7fcf1c1643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21400640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c155970 0x7fcf1c156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 -- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 msgr2=0x7fcf1c1643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 0x7fcf1c1643e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 -- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c155970 msgr2=0x7fcf1c156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c155970 0x7fcf1c156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 -- 172.21.15.110:0/2783157658 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c156000 con 0x7fcf1c06f010 2024-09-10T10:02:51.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21400640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c155970 0x7fcf1c156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:51.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf21e00640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf1c06f010 0x7fcf1c14ded0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c420 tx=0x7fcf1000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf16000640 1 -- 172.21.15.110:0/2783157658 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf100074e0 con 0x7fcf1c06f010 2024-09-10T10:02:51.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf16000640 1 -- 172.21.15.110:0/2783157658 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf10007680 con 0x7fcf1c06f010 2024-09-10T10:02:51.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf16000640 1 -- 172.21.15.110:0/2783157658 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10004a30 con 0x7fcf1c06f010 2024-09-10T10:02:51.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c1562e0 con 0x7fcf1c06f010 2024-09-10T10:02:51.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c14e410 con 0x7fcf1c06f010 2024-09-10T10:02:51.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c0ce420 con 0x7fcf1c06f010 2024-09-10T10:02:51.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.971+0000 7fcf16000640 1 -- 172.21.15.110:0/2783157658 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fcf1001a020 con 0x7fcf1c06f010 2024-09-10T10:02:51.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.975+0000 7fcf16000640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf00067a10 0x7fcf00069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.975+0000 7fcf16000640 1 -- 172.21.15.110:0/2783157658 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fcf100855d0 con 0x7fcf1c06f010 2024-09-10T10:02:51.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.975+0000 7fcf21400640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf00067a10 0x7fcf00069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.975+0000 7fcf21400640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf00067a10 0x7fcf00069ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c15ff00 tx=0x7fcf0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:51.979+0000 7fcf16000640 1 -- 172.21.15.110:0/2783157658 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcf10055f90 con 0x7fcf1c06f010 2024-09-10T10:02:52.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.139+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf1c075560 con 0x7fcf1c06f010 2024-09-10T10:02:52.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf16000640 1 -- 172.21.15.110:0/2783157658 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16887 (secure 0 0 0) 0x7fcf10083e20 con 0x7fcf1c06f010 2024-09-10T10:02:52.145 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:52.146 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":91,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:48.709541+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:02:28.619234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T10:02:29.912872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6834/3463064995":"2024-09-11T10:02:45.702580+0000","172.21.15.110:6832/3463064995":"2024-09-11T10:02:45.702580+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:52.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf00067a10 msgr2=0x7fcf00069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf00067a10 0x7fcf00069ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c15ff00 tx=0x7fcf0c008040 comp rx=0 tx=0).stop 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf1c06f010 msgr2=0x7fcf1c14ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf1c06f010 0x7fcf1c14ded0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c420 tx=0x7fcf1000c8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 shutdown_connections 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf00067a10 0x7fcf00069ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf1c15c560 0x7fcf1c1643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf1c155970 0x7fcf1c156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 --2- 172.21.15.110:0/2783157658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf1c06f010 0x7fcf1c14ded0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 >> 172.21.15.110:0/2783157658 conn(0x7fcf1c06bc50 msgr2=0x7fcf1c0cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 shutdown_connections 2024-09-10T10:02:52.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.143+0000 7fcf24800640 1 -- 172.21.15.110:0/2783157658 wait complete. 2024-09-10T10:02:52.167 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:02:52.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.303+0000 7fa0ef200640 1 Processor -- start 2024-09-10T10:02:52.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.303+0000 7fa0ef200640 1 -- start start 2024-09-10T10:02:52.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.303+0000 7fa0ef200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa0e80cfb90 0x7fa0e815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.303+0000 7fa0ef200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa0e80b3f40 0x7fa0e80cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.303+0000 7fa0ef200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cff70 0x7fa0e80cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.303+0000 7fa0ef200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e806d130 con 0x7fa0e80cfb90 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.303+0000 7fa0ef200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e815d4d0 con 0x7fa0e80b3f40 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.303+0000 7fa0ef200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e815d7b0 con 0x7fa0e80cff70 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa0e80cfb90 0x7fa0e815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa0e80cfb90 0x7fa0e815ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34162/0 (socket says 172.21.15.110:34162) 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 -- 172.21.15.110:0/729137301 learned_addr learned my addr 172.21.15.110:0/729137301 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0ed200640 1 --2- 172.21.15.110:0/729137301 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cff70 0x7fa0e80cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7400640 1 --2- 172.21.15.110:0/729137301 >> v2:172.21.15.145:3300/0 conn(0x7fa0e80b3f40 0x7fa0e80cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 -- 172.21.15.110:0/729137301 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cff70 msgr2=0x7fa0e80cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 --2- 172.21.15.110:0/729137301 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cff70 0x7fa0e80cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 -- 172.21.15.110:0/729137301 >> v2:172.21.15.145:3300/0 conn(0x7fa0e80b3f40 msgr2=0x7fa0e80cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 --2- 172.21.15.110:0/729137301 >> v2:172.21.15.145:3300/0 conn(0x7fa0e80b3f40 0x7fa0e80cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 -- 172.21.15.110:0/729137301 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e806d380 con 0x7fa0e80cfb90 2024-09-10T10:02:52.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e7e00640 1 --2- 172.21.15.110:0/729137301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80cfb90 0x7fa0e815ca10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80cf000 tx=0x7fa0d801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985d92fd6d469d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:52.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e6a00640 1 -- 172.21.15.110:0/729137301 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80a55e0 con 0x7fa0e80cfb90 2024-09-10T10:02:52.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e6a00640 1 -- 172.21.15.110:0/729137301 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d80a5780 con 0x7fa0e80cfb90 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.307+0000 7fa0e6a00640 1 -- 172.21.15.110:0/729137301 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80a5b50 con 0x7fa0e80cfb90 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- 172.21.15.110:0/729137301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80cfb90 msgr2=0x7fa0e815ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 --2- 172.21.15.110:0/729137301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80cfb90 0x7fa0e815ca10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80cf000 tx=0x7fa0d801c960 comp rx=0 tx=0).stop 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- 172.21.15.110:0/729137301 shutdown_connections 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 --2- 172.21.15.110:0/729137301 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cff70 0x7fa0e80cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 --2- 172.21.15.110:0/729137301 >> v2:172.21.15.145:3300/0 conn(0x7fa0e80b3f40 0x7fa0e80cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 --2- 172.21.15.110:0/729137301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80cfb90 0x7fa0e815ca10 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- 172.21.15.110:0/729137301 >> 172.21.15.110:0/729137301 conn(0x7fa0e8062d10 msgr2=0x7fa0e806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- 172.21.15.110:0/729137301 shutdown_connections 2024-09-10T10:02:52.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- 172.21.15.110:0/729137301 wait complete. 2024-09-10T10:02:52.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 Processor -- start 2024-09-10T10:02:52.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- start start 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80b3f40 0x7fa0e8140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cfb90 0x7fa0e815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0e80cff70 0x7fa0e8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e8160ba0 con 0x7fa0e80b3f40 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e8160a20 con 0x7fa0e80cff70 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ef200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e8160d20 con 0x7fa0e80cfb90 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80b3f40 0x7fa0e8140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80b3f40 0x7fa0e8140d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34164/0 (socket says 172.21.15.110:34164) 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7e00640 1 -- 172.21.15.110:0/957936671 learned_addr learned my addr 172.21.15.110:0/957936671 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7400640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cfb90 0x7fa0e815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0ed200640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0e80cff70 0x7fa0e8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7e00640 1 -- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cfb90 msgr2=0x7fa0e815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7e00640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cfb90 0x7fa0e815f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7e00640 1 -- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0e80cff70 msgr2=0x7fa0e8159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7e00640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0e80cff70 0x7fa0e8159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.311+0000 7fa0e7e00640 1 -- 172.21.15.110:0/957936671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e815f600 con 0x7fa0e80b3f40 2024-09-10T10:02:52.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0e7e00640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80b3f40 0x7fa0e8140d70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa0d801d200 tx=0x7fa0d8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:52.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0dfe00640 1 -- 172.21.15.110:0/957936671 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80bc020 con 0x7fa0e80b3f40 2024-09-10T10:02:52.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0dfe00640 1 -- 172.21.15.110:0/957936671 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d80b88a0 con 0x7fa0e80b3f40 2024-09-10T10:02:52.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e8071620 con 0x7fa0e80b3f40 2024-09-10T10:02:52.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0dfe00640 1 -- 172.21.15.110:0/957936671 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8009510 con 0x7fa0e80b3f40 2024-09-10T10:02:52.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e8071be0 con 0x7fa0e80b3f40 2024-09-10T10:02:52.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e815fb70 con 0x7fa0e80b3f40 2024-09-10T10:02:52.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0dfe00640 1 -- 172.21.15.110:0/957936671 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fa0d80bf020 con 0x7fa0e80b3f40 2024-09-10T10:02:52.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.315+0000 7fa0dfe00640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa0c80679e0 0x7fa0c8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.319+0000 7fa0dfe00640 1 -- 172.21.15.110:0/957936671 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fa0d8120f70 con 0x7fa0e80b3f40 2024-09-10T10:02:52.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.319+0000 7fa0e7400640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa0c80679e0 0x7fa0c8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.319+0000 7fa0e7400640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa0c80679e0 0x7fa0c8069ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8166210 tx=0x7fa0d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:52.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.323+0000 7fa0dfe00640 1 -- 172.21.15.110:0/957936671 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa0d80c6030 con 0x7fa0e80b3f40 2024-09-10T10:02:52.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.511+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa0e815f890 con 0x7fa0e80b3f40 2024-09-10T10:02:52.516 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:02:52.511+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:02:52.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.515+0000 7fa0dfe00640 1 -- 172.21.15.110:0/957936671 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) ==== 161+0+0 (secure 0 0 0) 0x7fa0d80165e0 con 0x7fa0e80b3f40 2024-09-10T10:02:52.518 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:02:52.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.515+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa0c80679e0 msgr2=0x7fa0c8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa0c80679e0 0x7fa0c8069ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8166210 tx=0x7fa0d4008040 comp rx=0 tx=0).stop 2024-09-10T10:02:52.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80b3f40 msgr2=0x7fa0e8140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80b3f40 0x7fa0e8140d70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa0d801d200 tx=0x7fa0d8019700 comp rx=0 tx=0).stop 2024-09-10T10:02:52.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 shutdown_connections 2024-09-10T10:02:52.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa0c80679e0 0x7fa0c8069ea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0e80cff70 0x7fa0e8159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0e80cfb90 0x7fa0e815f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 --2- 172.21.15.110:0/957936671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0e80b3f40 0x7fa0e8140d70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 >> 172.21.15.110:0/957936671 conn(0x7fa0e8062d10 msgr2=0x7fa0e80af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:52.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 shutdown_connections 2024-09-10T10:02:52.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.519+0000 7fa0ef200640 1 -- 172.21.15.110:0/957936671 wait complete. 2024-09-10T10:02:52.527 INFO:tasks.ceph.mds.a.smithi110.stderr: -2504> 2024-09-10T10:02:37.156+0000 7f5674e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:02:52.542 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:02:52.573 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:02:52.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f04a5600640 1 Processor -- start 2024-09-10T10:02:52.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f04a5600640 1 -- start start 2024-09-10T10:02:52.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f04a5600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04a015b330 0x7f04a006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f04a5600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a00b4960 con 0x7f04a00ce4d0 2024-09-10T10:02:52.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f04a5600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a00b4c40 con 0x7f04a006e850 2024-09-10T10:02:52.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f04a5600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f04a00b5570 con 0x7f04a015b330 2024-09-10T10:02:52.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f049ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04a015b330 0x7f04a006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f049ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04a015b330 0x7f04a006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34188/0 (socket says 172.21.15.110:34188) 2024-09-10T10:02:52.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f049ea00640 1 -- 172.21.15.110:0/1114875748 learned_addr learned my addr 172.21.15.110:0/1114875748 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:52.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f049e000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f04a00ce4d0 0x7f04a006e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49762/0 (socket says 172.21.15.110:49762) 2024-09-10T10:02:52.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f049ea00640 1 -- 172.21.15.110:0/1114875748 >> v1:172.21.15.145:6789/0 conn(0x7f04a006e850 legacy=0x7f04a00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:52.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f049ea00640 1 -- 172.21.15.110:0/1114875748 >> v1:172.21.15.110:6789/0 conn(0x7f04a00ce4d0 legacy=0x7f04a006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f049ea00640 1 -- 172.21.15.110:0/1114875748 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a00b4f20 con 0x7f04a015b330 2024-09-10T10:02:52.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.679+0000 7f049d600640 1 -- 172.21.15.110:0/1114875748 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 915668745 0 0) 0x7f04a00b4960 con 0x7f04a00ce4d0 2024-09-10T10:02:52.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f049ea00640 1 --2- 172.21.15.110:0/1114875748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a006c6a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0490009e50 tx=0x7f049001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fae29b447e3179 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:52.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f049d600640 1 -- 172.21.15.110:0/1114875748 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900a57d0 con 0x7f04a015b330 2024-09-10T10:02:52.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f049d600640 1 -- 172.21.15.110:0/1114875748 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04900a5970 con 0x7f04a015b330 2024-09-10T10:02:52.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f049d600640 1 -- 172.21.15.110:0/1114875748 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900a5d40 con 0x7f04a015b330 2024-09-10T10:02:52.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f04a5600640 1 -- 172.21.15.110:0/1114875748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 msgr2=0x7f04a006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f04a5600640 1 --2- 172.21.15.110:0/1114875748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a006c6a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0490009e50 tx=0x7f049001c9f0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f04a5600640 1 -- 172.21.15.110:0/1114875748 shutdown_connections 2024-09-10T10:02:52.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f04a5600640 1 --2- 172.21.15.110:0/1114875748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a006c6a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f04a5600640 1 -- 172.21.15.110:0/1114875748 >> 172.21.15.110:0/1114875748 conn(0x7f04a0062e40 msgr2=0x7f04a006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:52.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f04a5600640 1 -- 172.21.15.110:0/1114875748 shutdown_connections 2024-09-10T10:02:52.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.683+0000 7f04a5600640 1 -- 172.21.15.110:0/1114875748 wait complete. 2024-09-10T10:02:52.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 Processor -- start 2024-09-10T10:02:52.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 -- start start 2024-09-10T10:02:52.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04a006e850 0x7f04a0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a01524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04a0152a20 0x7f04a01544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04a0169450 con 0x7f04a015b330 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04a01692d0 con 0x7f04a0152a20 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04a01695d0 con 0x7f04a006e850 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a01524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a01524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34192/0 (socket says 172.21.15.110:34192) 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 -- 172.21.15.110:0/41582480 learned_addr learned my addr 172.21.15.110:0/41582480 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 -- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04a006e850 msgr2=0x7f04a0150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04a006e850 0x7f04a0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 -- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04a0152a20 msgr2=0x7f04a01544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:52.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049f400640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04a0152a20 0x7f04a01544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04a0152a20 0x7f04a01544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:52.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 -- 172.21.15.110:0/41582480 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a0152f60 con 0x7f04a015b330 2024-09-10T10:02:52.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049f400640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04a0152a20 0x7f04a01544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:52.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f049e000640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a01524e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f048c0011f0 tx=0x7f048c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:52.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f0496a00640 1 -- 172.21.15.110:0/41582480 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c018070 con 0x7f04a015b330 2024-09-10T10:02:52.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f0496a00640 1 -- 172.21.15.110:0/41582480 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0074e0 con 0x7f04a015b330 2024-09-10T10:02:52.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a0153730 con 0x7f04a015b330 2024-09-10T10:02:52.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f0496a00640 1 -- 172.21.15.110:0/41582480 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c004a10 con 0x7f04a015b330 2024-09-10T10:02:52.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.687+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a0153cf0 con 0x7f04a015b330 2024-09-10T10:02:52.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.691+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a006c6a0 con 0x7f04a015b330 2024-09-10T10:02:52.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.691+0000 7f0496a00640 1 -- 172.21.15.110:0/41582480 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f048c01b070 con 0x7f04a015b330 2024-09-10T10:02:52.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.691+0000 7f0496a00640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04800679e0 0x7f0480069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:52.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.691+0000 7f0496a00640 1 -- 172.21.15.110:0/41582480 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f048c0852b0 con 0x7f04a015b330 2024-09-10T10:02:52.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.691+0000 7f049ea00640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04800679e0 0x7f0480069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:52.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.691+0000 7f049ea00640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04800679e0 0x7f0480069ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f04900ae000 tx=0x7f0490004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:52.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.695+0000 7f0496a00640 1 -- 172.21.15.110:0/41582480 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f048c055b90 con 0x7f04a015b330 2024-09-10T10:02:52.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:52.887+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f04a00d00f0 con 0x7f04a015b330 2024-09-10T10:02:53.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.515+0000 7f0496a00640 1 -- 172.21.15.110:0/41582480 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) ==== 104+0+0 (secure 0 0 0) 0x7f048c055490 con 0x7f04a015b330 2024-09-10T10:02:53.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04800679e0 msgr2=0x7f0480069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04800679e0 0x7f0480069ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f04900ae000 tx=0x7f0490004ac0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 msgr2=0x7f04a01524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a01524e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f048c0011f0 tx=0x7f048c00c9b0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 shutdown_connections 2024-09-10T10:02:53.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04800679e0 0x7f0480069ea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04a0152a20 0x7f04a01544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04a015b330 0x7f04a01524e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 --2- 172.21.15.110:0/41582480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04a006e850 0x7f04a0150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 >> 172.21.15.110:0/41582480 conn(0x7f04a0062e40 msgr2=0x7f04a006df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:53.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 shutdown_connections 2024-09-10T10:02:53.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.519+0000 7f04a5600640 1 -- 172.21.15.110:0/41582480 wait complete. 2024-09-10T10:02:53.548 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:02:53.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.731+0000 7f9e15800640 1 Processor -- start 2024-09-10T10:02:53.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.731+0000 7f9e15800640 1 -- start start 2024-09-10T10:02:53.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e15800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e100b32b0 0x7f9e100b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e15800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e10155b60 con 0x7f9e1015b8b0 2024-09-10T10:02:53.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e15800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e100b3bd0 con 0x7f9e1006e140 2024-09-10T10:02:53.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e15800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9e1015c600 con 0x7f9e100b32b0 2024-09-10T10:02:53.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e0fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9e100b32b0 0x7f9e100b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e0f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9e1015b8b0 0x7f9e1015bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49776/0 (socket says 172.21.15.110:49776) 2024-09-10T10:02:53.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e0f400640 1 -- 172.21.15.110:0/1279031412 learned_addr learned my addr 172.21.15.110:0/1279031412 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:53.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e0fe00640 1 -- 172.21.15.110:0/1279031412 >> v1:172.21.15.145:6789/0 conn(0x7f9e1006e140 legacy=0x7f9e1006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e0fe00640 1 -- 172.21.15.110:0/1279031412 >> v1:172.21.15.110:6789/0 conn(0x7f9e1015b8b0 legacy=0x7f9e1015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e0fe00640 1 -- 172.21.15.110:0/1279031412 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e100b4660 con 0x7f9e100b32b0 2024-09-10T10:02:53.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.735+0000 7f9e0ea00640 1 -- 172.21.15.110:0/1279031412 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962235441 0 0) 0x7f9dfc001120 con 0x7f9e1015b8b0 2024-09-10T10:02:53.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e0fe00640 1 --2- 172.21.15.110:0/1279031412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e100b32b0 0x7f9e100b3690 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097c0 tx=0x7f9e0001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a077278fdbb28fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:53.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e0ea00640 1 -- 172.21.15.110:0/1279031412 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000b1070 con 0x7f9e100b32b0 2024-09-10T10:02:53.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e0ea00640 1 -- 172.21.15.110:0/1279031412 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e000ac410 con 0x7f9e100b32b0 2024-09-10T10:02:53.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e0ea00640 1 -- 172.21.15.110:0/1279031412 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000ac7e0 con 0x7f9e100b32b0 2024-09-10T10:02:53.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 -- 172.21.15.110:0/1279031412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e100b32b0 msgr2=0x7f9e100b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 --2- 172.21.15.110:0/1279031412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e100b32b0 0x7f9e100b3690 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9e000097c0 tx=0x7f9e0001c4a0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 -- 172.21.15.110:0/1279031412 shutdown_connections 2024-09-10T10:02:53.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 --2- 172.21.15.110:0/1279031412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e100b32b0 0x7f9e100b3690 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 -- 172.21.15.110:0/1279031412 >> 172.21.15.110:0/1279031412 conn(0x7f9e1006cd60 msgr2=0x7f9e1006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:53.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 -- 172.21.15.110:0/1279031412 shutdown_connections 2024-09-10T10:02:53.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 -- 172.21.15.110:0/1279031412 wait complete. 2024-09-10T10:02:53.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 Processor -- start 2024-09-10T10:02:53.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.739+0000 7f9e15800640 1 -- start start 2024-09-10T10:02:53.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1006e140 0x7f9e1014d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e100b32b0 0x7f9e1014e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1015b8b0 0x7f9e1014ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e100b5420 con 0x7f9e1006e140 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e100b4f00 con 0x7f9e1015b8b0 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e100b55a0 con 0x7f9e100b32b0 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1006e140 0x7f9e1014d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1006e140 0x7f9e1014d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34200/0 (socket says 172.21.15.110:34200) 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 -- 172.21.15.110:0/4069507983 learned_addr learned my addr 172.21.15.110:0/4069507983 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0f400640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e100b32b0 0x7f9e1014e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e14e00640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1015b8b0 0x7f9e1014ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 -- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e100b32b0 msgr2=0x7f9e1014e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e100b32b0 0x7f9e1014e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 -- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1015b8b0 msgr2=0x7f9e1014ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1015b8b0 0x7f9e1014ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 -- 172.21.15.110:0/4069507983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e100b2530 con 0x7f9e1006e140 2024-09-10T10:02:53.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e14e00640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1015b8b0 0x7f9e1014ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:53.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e0fe00640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1006e140 0x7f9e1014d170 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9e000af040 tx=0x7f9e0001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:53.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e07e00640 1 -- 172.21.15.110:0/4069507983 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000b1050 con 0x7f9e1006e140 2024-09-10T10:02:53.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e07e00640 1 -- 172.21.15.110:0/4069507983 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e000b2040 con 0x7f9e1006e140 2024-09-10T10:02:53.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e100b2810 con 0x7f9e1006e140 2024-09-10T10:02:53.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e07e00640 1 -- 172.21.15.110:0/4069507983 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00004240 con 0x7f9e1006e140 2024-09-10T10:02:53.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e101516c0 con 0x7f9e1006e140 2024-09-10T10:02:53.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e100b62e0 con 0x7f9e1006e140 2024-09-10T10:02:53.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.743+0000 7f9e07e00640 1 -- 172.21.15.110:0/4069507983 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f9e000bf060 con 0x7f9e1006e140 2024-09-10T10:02:53.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.747+0000 7f9e07e00640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9df00679e0 0x7f9df0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.747+0000 7f9e07e00640 1 -- 172.21.15.110:0/4069507983 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f9e0011f9b0 con 0x7f9e1006e140 2024-09-10T10:02:53.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.747+0000 7f9e0f400640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9df00679e0 0x7f9df0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.747+0000 7f9e0f400640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9df00679e0 0x7f9df0069ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e100733f0 tx=0x7f9dfc007d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:53.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.751+0000 7f9e07e00640 1 -- 172.21.15.110:0/4069507983 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9e00016090 con 0x7f9e1006e140 2024-09-10T10:02:53.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:53.907+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9e10151ae0 con 0x7f9e1006e140 2024-09-10T10:02:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.731+0000 7f9e07e00640 1 -- 172.21.15.110:0/4069507983 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f9e0011f020 con 0x7f9e1006e140 2024-09-10T10:02:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:02:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9df00679e0 msgr2=0x7f9df0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9df00679e0 0x7f9df0069ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e100733f0 tx=0x7f9dfc007d10 comp rx=0 tx=0).stop 2024-09-10T10:02:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1006e140 msgr2=0x7f9e1014d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1006e140 0x7f9e1014d170 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9e000af040 tx=0x7f9e0001ace0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 shutdown_connections 2024-09-10T10:02:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9df00679e0 0x7f9df0069ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1015b8b0 0x7f9e1014ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e100b32b0 0x7f9e1014e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 --2- 172.21.15.110:0/4069507983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1006e140 0x7f9e1014d170 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 >> 172.21.15.110:0/4069507983 conn(0x7f9e1006cd60 msgr2=0x7f9e1006e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 shutdown_connections 2024-09-10T10:02:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.735+0000 7f9e15800640 1 -- 172.21.15.110:0/4069507983 wait complete. 2024-09-10T10:02:54.763 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:02:54.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b3000640 1 Processor -- start 2024-09-10T10:02:54.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b3000640 1 -- start start 2024-09-10T10:02:54.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b3000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cfe30 0x7fe9ac071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b3000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe9ac06e7a0 0x7fe9ac0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b3000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ac06d2d0 con 0x7fe9ac0cd510 2024-09-10T10:02:54.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b3000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ac0cdeb0 con 0x7fe9ac06e7a0 2024-09-10T10:02:54.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b3000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac15d9a0 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe9ac06e7a0 0x7fe9ac0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe9ac06e7a0 0x7fe9ac0cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34210/0 (socket says 172.21.15.110:34210) 2024-09-10T10:02:54.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b1000640 1 -- 172.21.15.110:0/3290491039 learned_addr learned my addr 172.21.15.110:0/3290491039 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:54.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9abe00640 1 --2- 172.21.15.110:0/3290491039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cfe30 0x7fe9ac071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b1000640 1 -- 172.21.15.110:0/3290491039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cfe30 msgr2=0x7fe9ac071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:54.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b1000640 1 --2- 172.21.15.110:0/3290491039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cfe30 0x7fe9ac071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b1000640 1 -- 172.21.15.110:0/3290491039 >> v1:172.21.15.145:6789/0 conn(0x7fe9ac0cd510 legacy=0x7fe9ac06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9b1000640 1 -- 172.21.15.110:0/3290491039 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac06d5b0 con 0x7fe9ac06e7a0 2024-09-10T10:02:54.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.903+0000 7fe9abe00640 1 --2- 172.21.15.110:0/3290491039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cfe30 0x7fe9ac071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:54.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b1000640 1 --2- 172.21.15.110:0/3290491039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac06e7a0 0x7fe9ac0cd970 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009e50 tx=0x7fe9a401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f507d21654c80b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9aaa00640 1 -- 172.21.15.110:0/3290491039 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40a5770 con 0x7fe9ac06e7a0 2024-09-10T10:02:54.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9aaa00640 1 -- 172.21.15.110:0/3290491039 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a40a5910 con 0x7fe9ac06e7a0 2024-09-10T10:02:54.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9aaa00640 1 -- 172.21.15.110:0/3290491039 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40a5ce0 con 0x7fe9ac06e7a0 2024-09-10T10:02:54.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b3000640 1 -- 172.21.15.110:0/3290491039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac06e7a0 msgr2=0x7fe9ac0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b3000640 1 --2- 172.21.15.110:0/3290491039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac06e7a0 0x7fe9ac0cd970 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009e50 tx=0x7fe9a401c9a0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b3000640 1 -- 172.21.15.110:0/3290491039 shutdown_connections 2024-09-10T10:02:54.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b3000640 1 --2- 172.21.15.110:0/3290491039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac06e7a0 0x7fe9ac0cd970 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b3000640 1 --2- 172.21.15.110:0/3290491039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cfe30 0x7fe9ac071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b3000640 1 -- 172.21.15.110:0/3290491039 >> 172.21.15.110:0/3290491039 conn(0x7fe9ac06bc50 msgr2=0x7fe9ac06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b3000640 1 -- 172.21.15.110:0/3290491039 shutdown_connections 2024-09-10T10:02:54.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.907+0000 7fe9b3000640 1 -- 172.21.15.110:0/3290491039 wait complete. 2024-09-10T10:02:54.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 Processor -- start 2024-09-10T10:02:54.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 -- start start 2024-09-10T10:02:54.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac06e7a0 0x7fe9ac150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cd510 0x7fe9ac14f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac0cfe30 0x7fe9ac154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac072190 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac15d640 con 0x7fe9ac06e7a0 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac169180 con 0x7fe9ac0cd510 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac0cfe30 0x7fe9ac154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac0cfe30 0x7fe9ac154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34220/0 (socket says 172.21.15.110:34220) 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 -- 172.21.15.110:0/889546546 learned_addr learned my addr 172.21.15.110:0/889546546 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9abe00640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac06e7a0 0x7fe9ac150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9ab400640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cd510 0x7fe9ac14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 -- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cd510 msgr2=0x7fe9ac14f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cd510 0x7fe9ac14f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 -- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac06e7a0 msgr2=0x7fe9ac150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac06e7a0 0x7fe9ac150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 -- 172.21.15.110:0/889546546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac14f970 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b1000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac0cfe30 0x7fe9ac154b50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe9a401a5d0 tx=0x7fe9a4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9a3e00640 1 -- 172.21.15.110:0/889546546 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40bc020 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9a3e00640 1 -- 172.21.15.110:0/889546546 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a40b8670 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9a3e00640 1 -- 172.21.15.110:0/889546546 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40b8a40 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac14fed0 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac150490 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.911+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac0cd970 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.915+0000 7fe9a3e00640 1 -- 172.21.15.110:0/889546546 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fe9a40bf020 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.915+0000 7fe9a3e00640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe98c067a10 0x7fe98c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.915+0000 7fe9a3e00640 1 -- 172.21.15.110:0/889546546 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5443+0+0 (secure 0 0 0) 0x7fe9a4120b70 con 0x7fe9ac0cfe30 2024-09-10T10:02:54.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.915+0000 7fe9abe00640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe98c067a10 0x7fe98c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.915+0000 7fe9abe00640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe98c067a10 0x7fe98c069ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0b1490 tx=0x7fe99c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:54.919+0000 7fe9a3e00640 1 -- 172.21.15.110:0/889546546 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe9a40c6030 con 0x7fe9ac0cfe30 2024-09-10T10:02:55.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.075+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe9ac071190 con 0x7fe9ac0cfe30 2024-09-10T10:02:55.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.735+0000 7fe9a3e00640 1 -- 172.21.15.110:0/889546546 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7fe9a4016620 con 0x7fe9ac0cfe30 2024-09-10T10:02:55.741 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:02:55.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe98c067a10 msgr2=0x7fe98c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:55.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe98c067a10 0x7fe98c069ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0b1490 tx=0x7fe99c008040 comp rx=0 tx=0).stop 2024-09-10T10:02:55.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac0cfe30 msgr2=0x7fe9ac154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:55.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac0cfe30 0x7fe9ac154b50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe9a401a5d0 tx=0x7fe9a4019740 comp rx=0 tx=0).stop 2024-09-10T10:02:55.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 shutdown_connections 2024-09-10T10:02:55.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe98c067a10 0x7fe98c069ed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:55.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9ac0cfe30 0x7fe9ac154b50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:55.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe9ac0cd510 0x7fe9ac14f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:55.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 --2- 172.21.15.110:0/889546546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac06e7a0 0x7fe9ac150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:55.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 >> 172.21.15.110:0/889546546 conn(0x7fe9ac06bc50 msgr2=0x7fe9ac15a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:55.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 shutdown_connections 2024-09-10T10:02:55.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.739+0000 7fe9b3000640 1 -- 172.21.15.110:0/889546546 wait complete. 2024-09-10T10:02:55.765 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:02:55.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0cae000640 1 Processor -- start 2024-09-10T10:02:55.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0cae000640 1 -- start start 2024-09-10T10:02:55.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0cae000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0ca80d20b0 0x7f0ca806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:55.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0cae000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca8071470 con 0x7f0ca815e940 2024-09-10T10:02:55.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0cae000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca8071710 con 0x7f0ca80b71c0 2024-09-10T10:02:55.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0cae000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca806cd60 con 0x7f0ca80d20b0 2024-09-10T10:02:55.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0ca7400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0ca815e940 0x7f0ca80b6ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49784/0 (socket says 172.21.15.110:49784) 2024-09-10T10:02:55.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0ca6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0ca80d20b0 0x7f0ca806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:55.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.899+0000 7f0ca7400640 1 -- 172.21.15.110:0/3980887722 learned_addr learned my addr 172.21.15.110:0/3980887722 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:55.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.903+0000 7f0ca6a00640 1 -- 172.21.15.110:0/3980887722 >> v1:172.21.15.145:6789/0 conn(0x7f0ca80b71c0 legacy=0x7f0ca806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:55.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.903+0000 7f0ca6a00640 1 -- 172.21.15.110:0/3980887722 >> v1:172.21.15.110:6789/0 conn(0x7f0ca815e940 legacy=0x7f0ca80b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:55.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.903+0000 7f0ca6a00640 1 -- 172.21.15.110:0/3980887722 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80719f0 con 0x7f0ca80d20b0 2024-09-10T10:02:55.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.903+0000 7f0ca6000640 1 -- 172.21.15.110:0/3980887722 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342648797 0 0) 0x7f0ca8071470 con 0x7f0ca815e940 2024-09-10T10:02:55.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.903+0000 7f0ca6a00640 1 --2- 172.21.15.110:0/3980887722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80d20b0 0x7f0ca806d7b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8071640 tx=0x7f0ca806ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd66ebe16a776327 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:55.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.903+0000 7f0ca6000640 1 -- 172.21.15.110:0/3980887722 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940ab5f0 con 0x7f0ca80d20b0 2024-09-10T10:02:55.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.903+0000 7f0ca6000640 1 -- 172.21.15.110:0/3980887722 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c940b0070 con 0x7f0ca80d20b0 2024-09-10T10:02:55.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.903+0000 7f0ca6000640 1 -- 172.21.15.110:0/3980887722 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940ab9b0 con 0x7f0ca80d20b0 2024-09-10T10:02:55.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- 172.21.15.110:0/3980887722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80d20b0 msgr2=0x7f0ca806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:55.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 --2- 172.21.15.110:0/3980887722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80d20b0 0x7f0ca806d7b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8071640 tx=0x7f0ca806ec10 comp rx=0 tx=0).stop 2024-09-10T10:02:55.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- 172.21.15.110:0/3980887722 shutdown_connections 2024-09-10T10:02:55.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 --2- 172.21.15.110:0/3980887722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80d20b0 0x7f0ca806d7b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:55.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- 172.21.15.110:0/3980887722 >> 172.21.15.110:0/3980887722 conn(0x7f0ca80d25a0 msgr2=0x7f0ca80d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:55.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- 172.21.15.110:0/3980887722 shutdown_connections 2024-09-10T10:02:55.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- 172.21.15.110:0/3980887722 wait complete. 2024-09-10T10:02:55.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 Processor -- start 2024-09-10T10:02:55.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- start start 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ca80b71c0 0x7f0ca8148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ca80d20b0 0x7f0ca81514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca815e940 0x7f0ca814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca816c6d0 con 0x7f0ca815e940 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca816c550 con 0x7f0ca80d20b0 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0cae000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca816c850 con 0x7f0ca80b71c0 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca815e940 0x7f0ca814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca815e940 0x7f0ca814f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34244/0 (socket says 172.21.15.110:34244) 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 -- 172.21.15.110:0/3777894652 learned_addr learned my addr 172.21.15.110:0/3777894652 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:55.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca6a00640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ca80d20b0 0x7f0ca81514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7400640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ca80b71c0 0x7f0ca8148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 -- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ca80b71c0 msgr2=0x7f0ca8148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ca80b71c0 0x7f0ca8148100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 -- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ca80d20b0 msgr2=0x7f0ca81514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ca80d20b0 0x7f0ca81514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 -- 172.21.15.110:0/3777894652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca8150410 con 0x7f0ca815e940 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0ca7e00640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca815e940 0x7f0ca814f770 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000c410 tx=0x7f0ca000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.907+0000 7f0c9f400640 1 -- 172.21.15.110:0/3777894652 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0017070 con 0x7f0ca815e940 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0c9f400640 1 -- 172.21.15.110:0/3777894652 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca0003600 con 0x7f0ca815e940 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0c9f400640 1 -- 172.21.15.110:0/3777894652 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0003a30 con 0x7f0ca815e940 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca81506f0 con 0x7f0ca815e940 2024-09-10T10:02:55.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca8150cb0 con 0x7f0ca815e940 2024-09-10T10:02:55.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca80b6ab0 con 0x7f0ca815e940 2024-09-10T10:02:55.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0c9f400640 1 -- 172.21.15.110:0/3777894652 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f0ca0003bd0 con 0x7f0ca815e940 2024-09-10T10:02:55.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0c9f400640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c88067a10 0x7f0c88069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:55.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0c9f400640 1 -- 172.21.15.110:0/3777894652 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f0ca0084c30 con 0x7f0ca815e940 2024-09-10T10:02:55.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0ca7400640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c88067a10 0x7f0c88069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:55.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.911+0000 7f0ca7400640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c88067a10 0x7f0c88069ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c980012f0 tx=0x7f0c98001690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:55.915+0000 7f0c9f400640 1 -- 172.21.15.110:0/3777894652 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0ca0055340 con 0x7f0ca815e940 2024-09-10T10:02:56.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.075+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0ca8073aa0 con 0x7f0ca815e940 2024-09-10T10:02:56.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0c9f400640 1 -- 172.21.15.110:0/3777894652 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f0ca0059640 con 0x7f0ca815e940 2024-09-10T10:02:56.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c88067a10 msgr2=0x7f0c88069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c88067a10 0x7f0c88069ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c980012f0 tx=0x7f0c98001690 comp rx=0 tx=0).stop 2024-09-10T10:02:56.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca815e940 msgr2=0x7f0ca814f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca815e940 0x7f0ca814f770 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000c410 tx=0x7f0ca000c8d0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 shutdown_connections 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c88067a10 0x7f0c88069ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca815e940 0x7f0ca814f770 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ca80d20b0 0x7f0ca81514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 --2- 172.21.15.110:0/3777894652 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ca80b71c0 0x7f0ca8148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 >> 172.21.15.110:0/3777894652 conn(0x7f0ca80d25a0 msgr2=0x7f0ca8070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 shutdown_connections 2024-09-10T10:02:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.079+0000 7f0cae000640 1 -- 172.21.15.110:0/3777894652 wait complete. 2024-09-10T10:02:56.103 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-10T10:02:56.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfe1400640 1 Processor -- start 2024-09-10T10:02:56.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfe1400640 1 -- start start 2024-09-10T10:02:56.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfe1400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfe1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcfdc06f010 0x7fcfdc0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfe1400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfdc06f430 con 0x7fcfdc155970 2024-09-10T10:02:56.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfe1400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcfdc0b4570 con 0x7fcfdc06f010 2024-09-10T10:02:56.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfe1400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc155d50 con 0x7fcfdc15c560 2024-09-10T10:02:56.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfdb200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcfdc06f010 0x7fcfdc0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfdb200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcfdc06f010 0x7fcfdc0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34248/0 (socket says 172.21.15.110:34248) 2024-09-10T10:02:56.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfdb200640 1 -- 172.21.15.110:0/521545238 learned_addr learned my addr 172.21.15.110:0/521545238 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:56.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfd9e00640 1 --2- 172.21.15.110:0/521545238 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfdb200640 1 -- 172.21.15.110:0/521545238 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 msgr2=0x7fcfdc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfdb200640 1 --2- 172.21.15.110:0/521545238 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc06ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfdb200640 1 -- 172.21.15.110:0/521545238 >> v1:172.21.15.145:6789/0 conn(0x7fcfdc155970 legacy=0x7fcfdc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfdb200640 1 -- 172.21.15.110:0/521545238 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc0b34d0 con 0x7fcfdc06f010 2024-09-10T10:02:56.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.235+0000 7fcfd9e00640 1 --2- 172.21.15.110:0/521545238 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc06ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:56.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfdb200640 1 --2- 172.21.15.110:0/521545238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0ce420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009e50 tx=0x7fcfd401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead958711b1926b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfd9400640 1 -- 172.21.15.110:0/521545238 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40a57b0 con 0x7fcfdc06f010 2024-09-10T10:02:56.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfd9400640 1 -- 172.21.15.110:0/521545238 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd40a5950 con 0x7fcfdc06f010 2024-09-10T10:02:56.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfd9400640 1 -- 172.21.15.110:0/521545238 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40a5d20 con 0x7fcfdc06f010 2024-09-10T10:02:56.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfe1400640 1 -- 172.21.15.110:0/521545238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 msgr2=0x7fcfdc0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfe1400640 1 --2- 172.21.15.110:0/521545238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0ce420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009e50 tx=0x7fcfd401c930 comp rx=0 tx=0).stop 2024-09-10T10:02:56.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfe1400640 1 -- 172.21.15.110:0/521545238 shutdown_connections 2024-09-10T10:02:56.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfe1400640 1 --2- 172.21.15.110:0/521545238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0ce420 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfe1400640 1 --2- 172.21.15.110:0/521545238 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc06ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfe1400640 1 -- 172.21.15.110:0/521545238 >> 172.21.15.110:0/521545238 conn(0x7fcfdc06bc50 msgr2=0x7fcfdc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfe1400640 1 -- 172.21.15.110:0/521545238 shutdown_connections 2024-09-10T10:02:56.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.239+0000 7fcfe1400640 1 -- 172.21.15.110:0/521545238 wait complete. 2024-09-10T10:02:56.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 Processor -- start 2024-09-10T10:02:56.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 -- start start 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc155970 0x7fcfdc158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc0cf170 con 0x7fcfdc06f010 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc0b4050 con 0x7fcfdc155970 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc0cf2f0 con 0x7fcfdc15c560 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34264/0 (socket says 172.21.15.110:34264) 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 -- 172.21.15.110:0/659406173 learned_addr learned my addr 172.21.15.110:0/659406173 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfdb200640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfd9e00640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc155970 0x7fcfdc158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 -- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 msgr2=0x7fcfdc152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 -- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc155970 msgr2=0x7fcfdc158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc155970 0x7fcfdc158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 -- 172.21.15.110:0/659406173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc1533b0 con 0x7fcfdc06f010 2024-09-10T10:02:56.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfd9e00640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc155970 0x7fcfdc158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:56.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfda800640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0aec10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00c450 tx=0x7fcfcc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfd2a00640 1 -- 172.21.15.110:0/659406173 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc011790 con 0x7fcfdc06f010 2024-09-10T10:02:56.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfd2a00640 1 -- 172.21.15.110:0/659406173 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfcc017070 con 0x7fcfdc06f010 2024-09-10T10:02:56.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc157b80 con 0x7fcfdc06f010 2024-09-10T10:02:56.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfd2a00640 1 -- 172.21.15.110:0/659406173 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc011b90 con 0x7fcfdc06f010 2024-09-10T10:02:56.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.243+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc158140 con 0x7fcfdc06f010 2024-09-10T10:02:56.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.247+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc0ce420 con 0x7fcfdc06f010 2024-09-10T10:02:56.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.247+0000 7fcfd2a00640 1 -- 172.21.15.110:0/659406173 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fcfcc011d30 con 0x7fcfdc06f010 2024-09-10T10:02:56.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.247+0000 7fcfd2a00640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfbc0679e0 0x7fcfbc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.247+0000 7fcfd2a00640 1 -- 172.21.15.110:0/659406173 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fcfcc0858a0 con 0x7fcfdc06f010 2024-09-10T10:02:56.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.247+0000 7fcfd9e00640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfbc0679e0 0x7fcfbc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.247+0000 7fcfd9e00640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfbc0679e0 0x7fcfbc069ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0af3d0 tx=0x7fcfc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.251+0000 7fcfd2a00640 1 -- 172.21.15.110:0/659406173 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcfcc056560 con 0x7fcfdc06f010 2024-09-10T10:02:56.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.463+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7fcfdc06ddb0 con 0x7fcfdc06f010 2024-09-10T10:02:56.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.739+0000 7fcfd2a00640 1 -- 172.21.15.110:0/659406173 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v254) ==== 146+0+0 (secure 0 0 0) 0x7fcfcc055e60 con 0x7fcfdc06f010 2024-09-10T10:02:56.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfbc0679e0 msgr2=0x7fcfbc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfbc0679e0 0x7fcfbc069ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0af3d0 tx=0x7fcfc8008040 comp rx=0 tx=0).stop 2024-09-10T10:02:56.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 msgr2=0x7fcfdc0aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0aec10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00c450 tx=0x7fcfcc00c910 comp rx=0 tx=0).stop 2024-09-10T10:02:56.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 shutdown_connections 2024-09-10T10:02:56.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfbc0679e0 0x7fcfbc069ea0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfdc15c560 0x7fcfdc152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc155970 0x7fcfdc158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 --2- 172.21.15.110:0/659406173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfdc06f010 0x7fcfdc0aec10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 >> 172.21.15.110:0/659406173 conn(0x7fcfdc06bc50 msgr2=0x7fcfdc06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 shutdown_connections 2024-09-10T10:02:56.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.743+0000 7fcfe1400640 1 -- 172.21.15.110:0/659406173 wait complete. 2024-09-10T10:02:56.764 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-10T10:02:56.765 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-10T10:02:56.765 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:02:56.765 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:02:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.895+0000 7fee9b800640 1 Processor -- start 2024-09-10T10:02:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.895+0000 7fee9b800640 1 -- start start 2024-09-10T10:02:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee9b800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee94072840 0x7fee94140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee9b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee94140a60 0x7fee9415b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee9b800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee940ccf40 con 0x7fee940ccb60 2024-09-10T10:02:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee9b800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fee941412a0 con 0x7fee94072840 2024-09-10T10:02:56.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee9b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fee9415e760 con 0x7fee94140a60 2024-09-10T10:02:56.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee93e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee94072840 0x7fee94140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee93e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee94072840 0x7fee94140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34272/0 (socket says 172.21.15.110:34272) 2024-09-10T10:02:56.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee93e00640 1 -- 172.21.15.110:0/2061594410 learned_addr learned my addr 172.21.15.110:0/2061594410 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:56.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee93e00640 1 -- 172.21.15.110:0/2061594410 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee94140a60 msgr2=0x7fee9415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee93e00640 1 --2- 172.21.15.110:0/2061594410 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee94140a60 0x7fee9415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee93e00640 1 -- 172.21.15.110:0/2061594410 >> v1:172.21.15.145:6789/0 conn(0x7fee940ccb60 legacy=0x7fee94072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:56.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.899+0000 7fee93e00640 1 -- 172.21.15.110:0/2061594410 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9415ea40 con 0x7fee94072840 2024-09-10T10:02:56.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee93e00640 1 --2- 172.21.15.110:0/2061594410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee94140520 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fee8000aa10 tx=0x7fee8001d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9185d3458bf47ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee93400640 1 -- 172.21.15.110:0/2061594410 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee800b3070 con 0x7fee94072840 2024-09-10T10:02:56.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee93400640 1 -- 172.21.15.110:0/2061594410 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee800a6930 con 0x7fee94072840 2024-09-10T10:02:56.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee93400640 1 -- 172.21.15.110:0/2061594410 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8000b040 con 0x7fee94072840 2024-09-10T10:02:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 -- 172.21.15.110:0/2061594410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 msgr2=0x7fee94140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 --2- 172.21.15.110:0/2061594410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee94140520 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fee8000aa10 tx=0x7fee8001d8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 -- 172.21.15.110:0/2061594410 shutdown_connections 2024-09-10T10:02:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 --2- 172.21.15.110:0/2061594410 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee94140a60 0x7fee9415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 --2- 172.21.15.110:0/2061594410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee94140520 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 -- 172.21.15.110:0/2061594410 >> 172.21.15.110:0/2061594410 conn(0x7fee9406bc50 msgr2=0x7fee9406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 -- 172.21.15.110:0/2061594410 shutdown_connections 2024-09-10T10:02:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 -- 172.21.15.110:0/2061594410 wait complete. 2024-09-10T10:02:56.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 Processor -- start 2024-09-10T10:02:56.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.903+0000 7fee9b800640 1 -- start start 2024-09-10T10:02:56.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee941529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee940ccb60 0x7fee94154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee94140a60 0x7fee9413f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee94141030 con 0x7fee94072840 2024-09-10T10:02:56.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fee94140eb0 con 0x7fee94140a60 2024-09-10T10:02:56.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fee9415b660 con 0x7fee940ccb60 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee941529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee941529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34286/0 (socket says 172.21.15.110:34286) 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 -- 172.21.15.110:0/2344774889 learned_addr learned my addr 172.21.15.110:0/2344774889 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee93e00640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee940ccb60 0x7fee94154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee99800640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee94140a60 0x7fee9413f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 -- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee940ccb60 msgr2=0x7fee94154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee940ccb60 0x7fee94154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 -- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee94140a60 msgr2=0x7fee9413f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee94140a60 0x7fee9413f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 -- 172.21.15.110:0/2344774889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee94155350 con 0x7fee94072840 2024-09-10T10:02:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee98e00640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee941529c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fee8400c7a0 tx=0x7fee8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee90c00640 1 -- 172.21.15.110:0/2344774889 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84017070 con 0x7fee94072840 2024-09-10T10:02:56.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee90c00640 1 -- 172.21.15.110:0/2344774889 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee84003db0 con 0x7fee94072840 2024-09-10T10:02:56.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee90c00640 1 -- 172.21.15.110:0/2344774889 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840052b0 con 0x7fee94072840 2024-09-10T10:02:56.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee94153d70 con 0x7fee94072840 2024-09-10T10:02:56.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee94154330 con 0x7fee94072840 2024-09-10T10:02:56.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee94140520 con 0x7fee94072840 2024-09-10T10:02:56.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.907+0000 7fee90c00640 1 -- 172.21.15.110:0/2344774889 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fee84005450 con 0x7fee94072840 2024-09-10T10:02:56.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.911+0000 7fee90c00640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee74067a10 0x7fee74069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.911+0000 7fee90c00640 1 -- 172.21.15.110:0/2344774889 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fee84085120 con 0x7fee94072840 2024-09-10T10:02:56.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.911+0000 7fee93e00640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee74067a10 0x7fee74069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.911+0000 7fee93e00640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee74067a10 0x7fee74069ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee800b0000 tx=0x7fee80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:56.915+0000 7fee90c00640 1 -- 172.21.15.110:0/2344774889 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fee84055700 con 0x7fee94072840 2024-09-10T10:02:57.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.071+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee940722d0 con 0x7fee94072840 2024-09-10T10:02:57.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee90c00640 1 -- 172.21.15.110:0/2344774889 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+13203 (secure 0 0 0) 0x7fee84059a00 con 0x7fee94072840 2024-09-10T10:02:57.078 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:57.078 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":94,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:02:55.737120+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6835/1947643553":"2024-09-11T10:02:52.516941+0000","172.21.15.110:6833/1947643553":"2024-09-11T10:02:52.516941+0000","172.21.15.110:6834/3463064995":"2024-09-11T10:02:45.702580+0000","172.21.15.110:6832/3463064995":"2024-09-11T10:02:45.702580+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee74067a10 msgr2=0x7fee74069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee74067a10 0x7fee74069ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee800b0000 tx=0x7fee80008040 comp rx=0 tx=0).stop 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 msgr2=0x7fee941529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee941529c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fee8400c7a0 tx=0x7fee8400cc60 comp rx=0 tx=0).stop 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 shutdown_connections 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee74067a10 0x7fee74069ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee94140a60 0x7fee9413f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee940ccb60 0x7fee94154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 --2- 172.21.15.110:0/2344774889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94072840 0x7fee941529c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 >> 172.21.15.110:0/2344774889 conn(0x7fee9406bc50 msgr2=0x7fee94074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:57.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 shutdown_connections 2024-09-10T10:02:57.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.075+0000 7fee9b800640 1 -- 172.21.15.110:0/2344774889 wait complete. 2024-09-10T10:02:57.099 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:02:57.099 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:02:57.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.239+0000 7f733bc00640 1 Processor -- start 2024-09-10T10:02:57.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.239+0000 7f733bc00640 1 -- start start 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.239+0000 7f733bc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7334145f30 0x7f73340b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.239+0000 7f733bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73340cfd90 0x7f73340b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.239+0000 7f733bc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73340b37c0 con 0x7f73340b29e0 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.239+0000 7f733bc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f73340b2e50 con 0x7f7334145f30 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.239+0000 7f733bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73340b3aa0 con 0x7f73340cfd90 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7334145f30 0x7f73340b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7334145f30 0x7f73340b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34298/0 (socket says 172.21.15.110:34298) 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339200640 1 -- 172.21.15.110:0/2545451585 learned_addr learned my addr 172.21.15.110:0/2545451585 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:57.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339c00640 1 --2- 172.21.15.110:0/2545451585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73340cfd90 0x7f73340b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339200640 1 -- 172.21.15.110:0/2545451585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73340cfd90 msgr2=0x7f73340b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:57.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339200640 1 --2- 172.21.15.110:0/2545451585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73340cfd90 0x7f73340b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339200640 1 -- 172.21.15.110:0/2545451585 >> v1:172.21.15.145:6789/0 conn(0x7f73340b29e0 legacy=0x7f73340cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339200640 1 -- 172.21.15.110:0/2545451585 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73340b3d80 con 0x7f7334145f30 2024-09-10T10:02:57.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7339200640 1 --2- 172.21.15.110:0/2545451585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7334145f30 0x7f73340b24a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7324009e50 tx=0x7f732401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c612dcffe88e3daa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:57.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7333400640 1 -- 172.21.15.110:0/2545451585 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73240a5700 con 0x7f7334145f30 2024-09-10T10:02:57.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7333400640 1 -- 172.21.15.110:0/2545451585 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73240a58a0 con 0x7f7334145f30 2024-09-10T10:02:57.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.243+0000 7f7333400640 1 -- 172.21.15.110:0/2545451585 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73240a5c70 con 0x7f7334145f30 2024-09-10T10:02:57.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- 172.21.15.110:0/2545451585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7334145f30 msgr2=0x7f73340b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 --2- 172.21.15.110:0/2545451585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7334145f30 0x7f73340b24a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7324009e50 tx=0x7f732401ca30 comp rx=0 tx=0).stop 2024-09-10T10:02:57.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- 172.21.15.110:0/2545451585 shutdown_connections 2024-09-10T10:02:57.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 --2- 172.21.15.110:0/2545451585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f73340cfd90 0x7f73340b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 --2- 172.21.15.110:0/2545451585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7334145f30 0x7f73340b24a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- 172.21.15.110:0/2545451585 >> 172.21.15.110:0/2545451585 conn(0x7f7334062e40 msgr2=0x7f733406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:57.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- 172.21.15.110:0/2545451585 shutdown_connections 2024-09-10T10:02:57.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- 172.21.15.110:0/2545451585 wait complete. 2024-09-10T10:02:57.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 Processor -- start 2024-09-10T10:02:57.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- start start 2024-09-10T10:02:57.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73340b29e0 0x7f7334142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73340cfd90 0x7f73341447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7334145f30 0x7f733413d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73340b5660 con 0x7f73340cfd90 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f73340b54e0 con 0x7f73340b29e0 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f733bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f73340b57e0 con 0x7f7334145f30 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73340cfd90 0x7f73341447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73340cfd90 0x7f73341447b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34314/0 (socket says 172.21.15.110:34314) 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 -- 172.21.15.110:0/3134745846 learned_addr learned my addr 172.21.15.110:0/3134745846 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7339200640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73340b29e0 0x7f7334142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7339c00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7334145f30 0x7f733413d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 -- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7334145f30 msgr2=0x7f733413d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7334145f30 0x7f733413d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 -- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73340b29e0 msgr2=0x7f7334142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73340b29e0 0x7f7334142000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 -- 172.21.15.110:0/3134745846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7334142990 con 0x7f73340cfd90 2024-09-10T10:02:57.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.247+0000 7f7333e00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73340cfd90 0x7f73341447b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f732000c390 tx=0x7f732000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:57.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f7330c00640 1 -- 172.21.15.110:0/3134745846 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320017070 con 0x7f73340cfd90 2024-09-10T10:02:57.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f7330c00640 1 -- 172.21.15.110:0/3134745846 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73200052f0 con 0x7f73340cfd90 2024-09-10T10:02:57.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7334142c30 con 0x7f73340cfd90 2024-09-10T10:02:57.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f7330c00640 1 -- 172.21.15.110:0/3134745846 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320003840 con 0x7f73340cfd90 2024-09-10T10:02:57.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7334142f10 con 0x7f73340cfd90 2024-09-10T10:02:57.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73340b4560 con 0x7f73340cfd90 2024-09-10T10:02:57.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f7330c00640 1 -- 172.21.15.110:0/3134745846 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f732001a070 con 0x7f73340cfd90 2024-09-10T10:02:57.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f7330c00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f73140679e0 0x7f7314069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f7330c00640 1 -- 172.21.15.110:0/3134745846 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f7320084e10 con 0x7f73340cfd90 2024-09-10T10:02:57.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.251+0000 7f7339200640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f73140679e0 0x7f7314069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.255+0000 7f7339200640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f73140679e0 0x7f7314069ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73240ae000 tx=0x7f73240a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:57.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.255+0000 7f7330c00640 1 -- 172.21.15.110:0/3134745846 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7320088030 con 0x7f73340cfd90 2024-09-10T10:02:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7334071ad0 con 0x7f73340cfd90 2024-09-10T10:02:57.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f7330c00640 1 -- 172.21.15.110:0/3134745846 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f7320055480 con 0x7f73340cfd90 2024-09-10T10:02:57.418 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:02:57.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f73140679e0 msgr2=0x7f7314069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f73140679e0 0x7f7314069ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73240ae000 tx=0x7f73240a7070 comp rx=0 tx=0).stop 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73340cfd90 msgr2=0x7f73341447b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73340cfd90 0x7f73341447b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f732000c390 tx=0x7f732000c850 comp rx=0 tx=0).stop 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 shutdown_connections 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f73140679e0 0x7f7314069ea0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7334145f30 0x7f733413d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73340cfd90 0x7f73341447b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 --2- 172.21.15.110:0/3134745846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73340b29e0 0x7f7334142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 >> 172.21.15.110:0/3134745846 conn(0x7f7334062e40 msgr2=0x7f73340727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 shutdown_connections 2024-09-10T10:02:57.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.415+0000 7f733bc00640 1 -- 172.21.15.110:0/3134745846 wait complete. 2024-09-10T10:02:57.438 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-10T10:02:57.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e6a00640 1 Processor -- start 2024-09-10T10:02:57.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e6a00640 1 -- start start 2024-09-10T10:02:57.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e6a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa7e015c960 0x7fa7e00b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa7e01603c0 0x7fa7e00d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e6a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e00706a0 con 0x7fa7e00d04c0 2024-09-10T10:02:57.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e6a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e015cd40 con 0x7fa7e01603c0 2024-09-10T10:02:57.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e6a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e00b6370 con 0x7fa7e015c960 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7df400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa7e01603c0 0x7fa7e00d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e4a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa7e00d04c0 0x7fa7e0070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:49800/0 (socket says 172.21.15.110:49800) 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7e4a00640 1 -- 172.21.15.110:0/1653817193 learned_addr learned my addr 172.21.15.110:0/1653817193 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7dfe00640 1 --2- 172.21.15.110:0/1653817193 >> v2:172.21.15.145:3300/0 conn(0x7fa7e015c960 0x7fa7e00b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7df400640 1 -- 172.21.15.110:0/1653817193 >> v2:172.21.15.145:3300/0 conn(0x7fa7e015c960 msgr2=0x7fa7e00b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7df400640 1 --2- 172.21.15.110:0/1653817193 >> v2:172.21.15.145:3300/0 conn(0x7fa7e015c960 0x7fa7e00b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7df400640 1 -- 172.21.15.110:0/1653817193 >> v1:172.21.15.110:6789/0 conn(0x7fa7e00d04c0 legacy=0x7fa7e0070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7dea00640 1 -- 172.21.15.110:0/1653817193 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939605639 0 0) 0x7fa7e00706a0 con 0x7fa7e00d04c0 2024-09-10T10:02:57.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.571+0000 7fa7df400640 1 -- 172.21.15.110:0/1653817193 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e0070a80 con 0x7fa7e01603c0 2024-09-10T10:02:57.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7df400640 1 --2- 172.21.15.110:0/1653817193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e01603c0 0x7fa7e00d0cd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00b64f0 tx=0x7fa7cc01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af38ca5eb5ff748 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7dea00640 1 -- 172.21.15.110:0/1653817193 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0b0070 con 0x7fa7e01603c0 2024-09-10T10:02:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7dea00640 1 -- 172.21.15.110:0/1653817193 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7cc0ab600 con 0x7fa7e01603c0 2024-09-10T10:02:57.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7dea00640 1 -- 172.21.15.110:0/1653817193 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0ab9d0 con 0x7fa7e01603c0 2024-09-10T10:02:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7e6a00640 1 -- 172.21.15.110:0/1653817193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e01603c0 msgr2=0x7fa7e00d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/1653817193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e01603c0 0x7fa7e00d0cd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00b64f0 tx=0x7fa7cc01c310 comp rx=0 tx=0).stop 2024-09-10T10:02:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7e6a00640 1 -- 172.21.15.110:0/1653817193 shutdown_connections 2024-09-10T10:02:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/1653817193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e01603c0 0x7fa7e00d0cd0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/1653817193 >> v2:172.21.15.145:3300/0 conn(0x7fa7e015c960 0x7fa7e00b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7e6a00640 1 -- 172.21.15.110:0/1653817193 >> 172.21.15.110:0/1653817193 conn(0x7fa7e00d25c0 msgr2=0x7fa7e00d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7e6a00640 1 -- 172.21.15.110:0/1653817193 shutdown_connections 2024-09-10T10:02:57.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.575+0000 7fa7e6a00640 1 -- 172.21.15.110:0/1653817193 wait complete. 2024-09-10T10:02:57.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 Processor -- start 2024-09-10T10:02:57.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 -- start start 2024-09-10T10:02:57.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e00d04c0 0x7fa7e015f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7e015c960 0x7fa7e015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e01603c0 0x7fa7e015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e00d3150 con 0x7fa7e00d04c0 2024-09-10T10:02:57.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e01611b0 con 0x7fa7e01603c0 2024-09-10T10:02:57.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e00d32d0 con 0x7fa7e015c960 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e00d04c0 0x7fa7e015f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e00d04c0 0x7fa7e015f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34334/0 (socket says 172.21.15.110:34334) 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 -- 172.21.15.110:0/408932086 learned_addr learned my addr 172.21.15.110:0/408932086 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7df400640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7e015c960 0x7fa7e015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e4a00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e01603c0 0x7fa7e015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 -- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7e015c960 msgr2=0x7fa7e015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7e015c960 0x7fa7e015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 -- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e01603c0 msgr2=0x7fa7e015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e01603c0 0x7fa7e015d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 -- 172.21.15.110:0/408932086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e015e550 con 0x7fa7e00d04c0 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7dfe00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e00d04c0 0x7fa7e015f5f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000e930 tx=0x7fa7d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:57.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7d7e00640 1 -- 172.21.15.110:0/408932086 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0017070 con 0x7fa7e00d04c0 2024-09-10T10:02:57.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7d7e00640 1 -- 172.21.15.110:0/408932086 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d000ca00 con 0x7fa7e00d04c0 2024-09-10T10:02:57.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7d7e00640 1 -- 172.21.15.110:0/408932086 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000cdb0 con 0x7fa7e00d04c0 2024-09-10T10:02:57.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e015e830 con 0x7fa7e00d04c0 2024-09-10T10:02:57.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e015edf0 con 0x7fa7e00d04c0 2024-09-10T10:02:57.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.579+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e0071a80 con 0x7fa7e00d04c0 2024-09-10T10:02:57.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.583+0000 7fa7d7e00640 1 -- 172.21.15.110:0/408932086 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fa7d001a070 con 0x7fa7e00d04c0 2024-09-10T10:02:57.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.583+0000 7fa7d7e00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa7c0067a10 0x7fa7c0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.583+0000 7fa7d7e00640 1 -- 172.21.15.110:0/408932086 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fa7d0085f80 con 0x7fa7e00d04c0 2024-09-10T10:02:57.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.583+0000 7fa7df400640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa7c0067a10 0x7fa7c0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.583+0000 7fa7df400640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa7c0067a10 0x7fa7c0069ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0ad000 tx=0x7fa7cc0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:57.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.587+0000 7fa7d7e00640 1 -- 172.21.15.110:0/408932086 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa7d0056d00 con 0x7fa7e00d04c0 2024-09-10T10:02:57.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:57.799+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fa7e00d34d0 con 0x7fa7e00d04c0 2024-09-10T10:02:58.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7d7e00640 1 -- 172.21.15.110:0/408932086 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v256) ==== 152+0+0 (secure 0 0 0) 0x7fa7d0056600 con 0x7fa7e00d04c0 2024-09-10T10:02:58.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa7c0067a10 msgr2=0x7fa7c0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:58.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa7c0067a10 0x7fa7c0069ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0ad000 tx=0x7fa7cc0a6000 comp rx=0 tx=0).stop 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e00d04c0 msgr2=0x7fa7e015f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e00d04c0 0x7fa7e015f5f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000e930 tx=0x7fa7d000edf0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 shutdown_connections 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa7c0067a10 0x7fa7c0069ed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7e01603c0 0x7fa7e015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7e015c960 0x7fa7e015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 --2- 172.21.15.110:0/408932086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7e00d04c0 0x7fa7e015f5f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 >> 172.21.15.110:0/408932086 conn(0x7fa7e00d25c0 msgr2=0x7fa7e0073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 shutdown_connections 2024-09-10T10:02:58.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.747+0000 7fa7e6a00640 1 -- 172.21.15.110:0/408932086 wait complete. 2024-09-10T10:02:58.776 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:02:58.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.915+0000 7ff589400640 1 Processor -- start 2024-09-10T10:02:58.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.915+0000 7ff589400640 1 -- start start 2024-09-10T10:02:58.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.915+0000 7ff589400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff584145f30 0x7ff5840b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:58.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.915+0000 7ff589400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5840b29e0 0x7ff5840cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:58.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.915+0000 7ff589400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff5840cfbc0 0x7ff584160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:58.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff589400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff5840b2e20 con 0x7ff584145f30 2024-09-10T10:02:58.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff589400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff5840b37c0 con 0x7ff5840cfbc0 2024-09-10T10:02:58.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff589400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5840b3aa0 con 0x7ff5840b29e0 2024-09-10T10:02:58.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff584145f30 0x7ff5840b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:58.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff584145f30 0x7ff5840b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47800/0 (socket says 172.21.15.110:47800) 2024-09-10T10:02:58.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 -- 172.21.15.110:0/65974217 learned_addr learned my addr 172.21.15.110:0/65974217 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:58.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff579e00640 1 --2- 172.21.15.110:0/65974217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5840b29e0 0x7ff5840cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:58.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 -- 172.21.15.110:0/65974217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5840b29e0 msgr2=0x7ff5840cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff583200640 1 --2- 172.21.15.110:0/65974217 >> v2:172.21.15.145:3300/0 conn(0x7ff5840cfbc0 0x7ff584160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 --2- 172.21.15.110:0/65974217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5840b29e0 0x7ff5840cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 -- 172.21.15.110:0/65974217 >> v2:172.21.15.145:3300/0 conn(0x7ff5840cfbc0 msgr2=0x7ff584160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 --2- 172.21.15.110:0/65974217 >> v2:172.21.15.145:3300/0 conn(0x7ff5840cfbc0 0x7ff584160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 -- 172.21.15.110:0/65974217 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5840b3d80 con 0x7ff584145f30 2024-09-10T10:02:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff579e00640 1 --2- 172.21.15.110:0/65974217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5840b29e0 0x7ff5840cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff583200640 1 --2- 172.21.15.110:0/65974217 >> v2:172.21.15.145:3300/0 conn(0x7ff5840cfbc0 0x7ff584160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:58.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff582800640 1 --2- 172.21.15.110:0/65974217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff584145f30 0x7ff5840b24a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff5841610d0 tx=0x7ff57401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea476cca457bcf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:58.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff581e00640 1 -- 172.21.15.110:0/65974217 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5740b0070 con 0x7ff584145f30 2024-09-10T10:02:58.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff581e00640 1 -- 172.21.15.110:0/65974217 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5740ab600 con 0x7ff584145f30 2024-09-10T10:02:58.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.919+0000 7ff581e00640 1 -- 172.21.15.110:0/65974217 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5740ab9d0 con 0x7ff584145f30 2024-09-10T10:02:58.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- 172.21.15.110:0/65974217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff584145f30 msgr2=0x7ff5840b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 --2- 172.21.15.110:0/65974217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff584145f30 0x7ff5840b24a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff5841610d0 tx=0x7ff57401c260 comp rx=0 tx=0).stop 2024-09-10T10:02:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- 172.21.15.110:0/65974217 shutdown_connections 2024-09-10T10:02:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 --2- 172.21.15.110:0/65974217 >> v2:172.21.15.145:3300/0 conn(0x7ff5840cfbc0 0x7ff584160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 --2- 172.21.15.110:0/65974217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5840b29e0 0x7ff5840cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 --2- 172.21.15.110:0/65974217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff584145f30 0x7ff5840b24a0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- 172.21.15.110:0/65974217 >> 172.21.15.110:0/65974217 conn(0x7ff584062e40 msgr2=0x7ff58406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- 172.21.15.110:0/65974217 shutdown_connections 2024-09-10T10:02:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- 172.21.15.110:0/65974217 wait complete. 2024-09-10T10:02:58.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 Processor -- start 2024-09-10T10:02:58.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- start start 2024-09-10T10:02:58.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5840b29e0 0x7ff5840b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:58.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5840cfbc0 0x7ff58415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff584145f30 0x7ff58413cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5840b4910 con 0x7ff5840b29e0 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5840b4790 con 0x7ff5840cfbc0 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff589400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5840b4a90 con 0x7ff584145f30 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff582800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5840b29e0 0x7ff5840b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff582800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5840b29e0 0x7ff5840b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47810/0 (socket says 172.21.15.110:47810) 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff582800640 1 -- 172.21.15.110:0/3586256576 learned_addr learned my addr 172.21.15.110:0/3586256576 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff579e00640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5840cfbc0 0x7ff58415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff583200640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff584145f30 0x7ff58413cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff582800640 1 -- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff584145f30 msgr2=0x7ff58413cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff582800640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff584145f30 0x7ff58413cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff582800640 1 -- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5840cfbc0 msgr2=0x7ff58415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:58.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff582800640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5840cfbc0 0x7ff58415c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.923+0000 7ff582800640 1 -- 172.21.15.110:0/3586256576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5840b1e20 con 0x7ff5840b29e0 2024-09-10T10:02:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff582800640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5840b29e0 0x7ff5840b1800 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff5740ae040 tx=0x7ff574019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff57b400640 1 -- 172.21.15.110:0/3586256576 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5740b0040 con 0x7ff5840b29e0 2024-09-10T10:02:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff57b400640 1 -- 172.21.15.110:0/3586256576 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5740b9070 con 0x7ff5840b29e0 2024-09-10T10:02:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5840b2100 con 0x7ff5840b29e0 2024-09-10T10:02:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff57b400640 1 -- 172.21.15.110:0/3586256576 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5740b48a0 con 0x7ff5840b29e0 2024-09-10T10:02:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff584143d90 con 0x7ff5840b29e0 2024-09-10T10:02:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff584160030 con 0x7ff5840b29e0 2024-09-10T10:02:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff57b400640 1 -- 172.21.15.110:0/3586256576 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7ff5740ab4c0 con 0x7ff5840b29e0 2024-09-10T10:02:58.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff57b400640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5600679e0 0x7ff560069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:58.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff57b400640 1 -- 172.21.15.110:0/3586256576 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7ff57411faa0 con 0x7ff5840b29e0 2024-09-10T10:02:58.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.927+0000 7ff579e00640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5600679e0 0x7ff560069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:58.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.931+0000 7ff579e00640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5600679e0 0x7ff560069ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff584144ce0 tx=0x7ff56c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:58.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:58.931+0000 7ff57b400640 1 -- 172.21.15.110:0/3586256576 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff574016110 con 0x7ff5840b29e0 2024-09-10T10:02:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.091+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff58413f9c0 con 0x7ff5840b29e0 2024-09-10T10:02:59.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff57b400640 1 -- 172.21.15.110:0/3586256576 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7ff5740f0650 con 0x7ff5840b29e0 2024-09-10T10:02:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5600679e0 msgr2=0x7ff560069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5600679e0 0x7ff560069ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff584144ce0 tx=0x7ff56c008040 comp rx=0 tx=0).stop 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5840b29e0 msgr2=0x7ff5840b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5840b29e0 0x7ff5840b1800 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff5740ae040 tx=0x7ff574019000 comp rx=0 tx=0).stop 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 shutdown_connections 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5600679e0 0x7ff560069ea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff584145f30 0x7ff58413cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5840cfbc0 0x7ff58415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 --2- 172.21.15.110:0/3586256576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5840b29e0 0x7ff5840b1800 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 >> 172.21.15.110:0/3586256576 conn(0x7ff584062e40 msgr2=0x7ff5840728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 shutdown_connections 2024-09-10T10:02:59.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.095+0000 7ff589400640 1 -- 172.21.15.110:0/3586256576 wait complete. 2024-09-10T10:02:59.120 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:02:59.120 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:02:59.203 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:02:59.301 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:02:59.301 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:59.301 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:59.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:02:59.310 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:59.310 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:59.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:02:59.319 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:02:59.319 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:59.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:02:59.360 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:02:59.360 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:02:59.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:02:59.371 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:02:59.371 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:02:59.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.531+0000 7ff26b000640 1 Processor -- start 2024-09-10T10:02:59.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.531+0000 7ff26b000640 1 -- start start 2024-09-10T10:02:59.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff26b000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff2640cfe30 0x7ff264071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff26b000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff26406e7a0 0x7ff2640cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff26b000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26406d2d0 con 0x7ff2640cd510 2024-09-10T10:02:59.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff26b000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff26415d9a0 con 0x7ff26406e7a0 2024-09-10T10:02:59.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff26b000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff26406d5b0 con 0x7ff2640cfe30 2024-09-10T10:02:59.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff269000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff26406e7a0 0x7ff2640cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff269000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff26406e7a0 0x7ff2640cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47818/0 (socket says 172.21.15.110:47818) 2024-09-10T10:02:59.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff269000640 1 -- 172.21.15.110:0/532505488 learned_addr learned my addr 172.21.15.110:0/532505488 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:59.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff263e00640 1 --2- 172.21.15.110:0/532505488 >> v2:172.21.15.145:3300/0 conn(0x7ff2640cfe30 0x7ff264071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff269000640 1 -- 172.21.15.110:0/532505488 >> v2:172.21.15.145:3300/0 conn(0x7ff2640cfe30 msgr2=0x7ff264071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:59.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff269000640 1 --2- 172.21.15.110:0/532505488 >> v2:172.21.15.145:3300/0 conn(0x7ff2640cfe30 0x7ff264071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff269000640 1 -- 172.21.15.110:0/532505488 >> v1:172.21.15.145:6789/0 conn(0x7ff2640cd510 legacy=0x7ff26406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:59.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff269000640 1 -- 172.21.15.110:0/532505488 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26406d890 con 0x7ff26406e7a0 2024-09-10T10:02:59.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.535+0000 7ff263e00640 1 --2- 172.21.15.110:0/532505488 >> v2:172.21.15.145:3300/0 conn(0x7ff2640cfe30 0x7ff264071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:59.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff269000640 1 --2- 172.21.15.110:0/532505488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff2640cd9a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff2640ce120 tx=0x7ff25c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403db2a3a4409641 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:59.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff263400640 1 -- 172.21.15.110:0/532505488 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0a56f0 con 0x7ff26406e7a0 2024-09-10T10:02:59.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff263400640 1 -- 172.21.15.110:0/532505488 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c0a5890 con 0x7ff26406e7a0 2024-09-10T10:02:59.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff263400640 1 -- 172.21.15.110:0/532505488 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0a5c60 con 0x7ff26406e7a0 2024-09-10T10:02:59.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- 172.21.15.110:0/532505488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 msgr2=0x7ff2640cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 --2- 172.21.15.110:0/532505488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff2640cd9a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff2640ce120 tx=0x7ff25c01c920 comp rx=0 tx=0).stop 2024-09-10T10:02:59.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- 172.21.15.110:0/532505488 shutdown_connections 2024-09-10T10:02:59.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 --2- 172.21.15.110:0/532505488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff2640cd9a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 --2- 172.21.15.110:0/532505488 >> v2:172.21.15.145:3300/0 conn(0x7ff2640cfe30 0x7ff264071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- 172.21.15.110:0/532505488 >> 172.21.15.110:0/532505488 conn(0x7ff26406bc50 msgr2=0x7ff26406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:59.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- 172.21.15.110:0/532505488 shutdown_connections 2024-09-10T10:02:59.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- 172.21.15.110:0/532505488 wait complete. 2024-09-10T10:02:59.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 Processor -- start 2024-09-10T10:02:59.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- start start 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff26414e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2640cd510 0x7ff26414c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2640cfe30 0x7ff264148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff264072190 con 0x7ff26406e7a0 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff26415d6c0 con 0x7ff2640cd510 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.539+0000 7ff26b000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff264169180 con 0x7ff2640cfe30 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff26414e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff26414e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47832/0 (socket says 172.21.15.110:47832) 2024-09-10T10:02:59.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 -- 172.21.15.110:0/2309099148 learned_addr learned my addr 172.21.15.110:0/2309099148 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff25be00640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2640cd510 0x7ff26414c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff269000640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2640cfe30 0x7ff264148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 -- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2640cfe30 msgr2=0x7ff264148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2640cfe30 0x7ff264148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 -- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2640cd510 msgr2=0x7ff26414c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2640cd510 0x7ff26414c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 -- 172.21.15.110:0/2309099148 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff264146300 con 0x7ff26406e7a0 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff269000640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2640cfe30 0x7ff264148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff263e00640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff26414e3f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c7f0 tx=0x7ff25400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff260c00640 1 -- 172.21.15.110:0/2309099148 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254015070 con 0x7ff26406e7a0 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff260c00640 1 -- 172.21.15.110:0/2309099148 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff254007820 con 0x7ff26406e7a0 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26414b540 con 0x7ff26406e7a0 2024-09-10T10:02:59.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff260c00640 1 -- 172.21.15.110:0/2309099148 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254007c10 con 0x7ff26406e7a0 2024-09-10T10:02:59.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26414b820 con 0x7ff26406e7a0 2024-09-10T10:02:59.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26406c370 con 0x7ff26406e7a0 2024-09-10T10:02:59.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff260c00640 1 -- 172.21.15.110:0/2309099148 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7ff254018070 con 0x7ff26406e7a0 2024-09-10T10:02:59.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.543+0000 7ff260c00640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2400679e0 0x7ff240069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.547+0000 7ff260c00640 1 -- 172.21.15.110:0/2309099148 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7ff254085db0 con 0x7ff26406e7a0 2024-09-10T10:02:59.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.547+0000 7ff25be00640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2400679e0 0x7ff240069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.547+0000 7ff25be00640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2400679e0 0x7ff240069ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff26414aa30 tx=0x7ff24c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:59.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.547+0000 7ff260c00640 1 -- 172.21.15.110:0/2309099148 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff25408a070 con 0x7ff26406e7a0 2024-09-10T10:02:59.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff26414e990 con 0x7ff26406e7a0 2024-09-10T10:02:59.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff260c00640 1 -- 172.21.15.110:0/2309099148 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+3923 (secure 0 0 0) 0x7ff254056960 con 0x7ff26406e7a0 2024-09-10T10:02:59.743 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:02:59.743 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":62,"btime":"2024-09-10T10:02:53:519311+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1164773100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1164773100},{"type":"v1","addr":"172.21.15.145:6834","nonce":1164773100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1195578364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1195578364},{"type":"v1","addr":"172.21.15.145:6835","nonce":1195578364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6651,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3822482715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3822482715},{"type":"v1","addr":"172.21.15.110:6834","nonce":3822482715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6708,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2516018317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2516018317},{"type":"v1","addr":"172.21.15.110:6835","nonce":2516018317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[]} 2024-09-10T10:02:59.743 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 62 2024-09-10T10:02:59.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2400679e0 msgr2=0x7ff240069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2400679e0 0x7ff240069ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff26414aa30 tx=0x7ff24c008040 comp rx=0 tx=0).stop 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 msgr2=0x7ff26414e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff26414e3f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c7f0 tx=0x7ff25400ccb0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 shutdown_connections 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2400679e0 0x7ff240069ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2640cfe30 0x7ff264148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff2640cd510 0x7ff26414c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 --2- 172.21.15.110:0/2309099148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff26406e7a0 0x7ff26414e3f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 >> 172.21.15.110:0/2309099148 conn(0x7ff26406bc50 msgr2=0x7ff26415a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 shutdown_connections 2024-09-10T10:02:59.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.739+0000 7ff26b000640 1 -- 172.21.15.110:0/2309099148 wait complete. 2024-09-10T10:02:59.765 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:02:59.765 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:02:59.765 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:02:59.765 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:02:59.765 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:02:59.765 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:02:59.766 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:02:59.766 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:02:59.766 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:02:59.766 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:02:59.766 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:02:59.766 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:02:59.766 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:02:59.767 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:02:59.767 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:02:59.767 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:02:59.776 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:02:59.777 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:02:59.778 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:02:59.778 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:02:59.781 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:02:59.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:02:59.782 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:02:59.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:02:59.825 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:02:59.825 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:59.826 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:02:59.827 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:59.827 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:02:59.828 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:59.828 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:02:59.829 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:59.829 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:02:59.914 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:02:59.915 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:02:59.917 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:02:59.917 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:02:59.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6373400640 1 Processor -- start 2024-09-10T10:02:59.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6373400640 1 -- start start 2024-09-10T10:02:59.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6373400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f636c0726e0 0x7f636c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6373400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c06efa0 0x7f636c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6373400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f636c0b5430 0x7f636c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6373400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f636c0b5880 con 0x7f636c0b5430 2024-09-10T10:02:59.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6373400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f636c0cee50 con 0x7f636c0726e0 2024-09-10T10:02:59.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6373400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f636c06f3e0 con 0x7f636c06efa0 2024-09-10T10:02:59.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6371400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f636c0b5430 0x7f636c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6371400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f636c0b5430 0x7f636c0b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47892/0 (socket says 172.21.15.110:47892) 2024-09-10T10:02:59.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.967+0000 7f6371400640 1 -- 172.21.15.110:0/3619467177 learned_addr learned my addr 172.21.15.110:0/3619467177 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:59.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f636be00640 1 --2- 172.21.15.110:0/3619467177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c06efa0 0x7f636c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6370a00640 1 --2- 172.21.15.110:0/3619467177 >> v2:172.21.15.145:3300/0 conn(0x7f636c0726e0 0x7f636c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6371400640 1 -- 172.21.15.110:0/3619467177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c06efa0 msgr2=0x7f636c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:59.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6371400640 1 --2- 172.21.15.110:0/3619467177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c06efa0 0x7f636c0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6371400640 1 -- 172.21.15.110:0/3619467177 >> v2:172.21.15.145:3300/0 conn(0x7f636c0726e0 msgr2=0x7f636c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:59.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6371400640 1 --2- 172.21.15.110:0/3619467177 >> v2:172.21.15.145:3300/0 conn(0x7f636c0726e0 0x7f636c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6371400640 1 -- 172.21.15.110:0/3619467177 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c15fda0 con 0x7f636c0b5430 2024-09-10T10:02:59.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6371400640 1 --2- 172.21.15.110:0/3619467177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c0b5430 0x7f636c0b34d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f636c06c8f0 tx=0x7f636401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675f0bfa049c168b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:59.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f636b400640 1 -- 172.21.15.110:0/3619467177 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640aa610 con 0x7f636c0b5430 2024-09-10T10:02:59.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f636b400640 1 -- 172.21.15.110:0/3619467177 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63640af070 con 0x7f636c0b5430 2024-09-10T10:02:59.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f636b400640 1 -- 172.21.15.110:0/3619467177 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640aa9d0 con 0x7f636c0b5430 2024-09-10T10:02:59.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6373400640 1 -- 172.21.15.110:0/3619467177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c0b5430 msgr2=0x7f636c0b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.971+0000 7f6373400640 1 --2- 172.21.15.110:0/3619467177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c0b5430 0x7f636c0b34d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f636c06c8f0 tx=0x7f636401c4f0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 -- 172.21.15.110:0/3619467177 shutdown_connections 2024-09-10T10:02:59.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 --2- 172.21.15.110:0/3619467177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c0b5430 0x7f636c0b34d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 --2- 172.21.15.110:0/3619467177 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c06efa0 0x7f636c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 --2- 172.21.15.110:0/3619467177 >> v2:172.21.15.145:3300/0 conn(0x7f636c0726e0 0x7f636c0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 -- 172.21.15.110:0/3619467177 >> 172.21.15.110:0/3619467177 conn(0x7f636c062d10 msgr2=0x7f636c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:59.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 -- 172.21.15.110:0/3619467177 shutdown_connections 2024-09-10T10:02:59.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 -- 172.21.15.110:0/3619467177 wait complete. 2024-09-10T10:02:59.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 Processor -- start 2024-09-10T10:02:59.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 -- start start 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c06efa0 0x7f636c147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636c0726e0 0x7f636c157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c0b5430 0x7f636c158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f636c072ac0 con 0x7f636c06efa0 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f636c073270 con 0x7f636c0726e0 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6373400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f636c072c40 con 0x7f636c0b5430 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c06efa0 0x7f636c147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c06efa0 0x7f636c147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47894/0 (socket says 172.21.15.110:47894) 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 -- 172.21.15.110:0/2543944344 learned_addr learned my addr 172.21.15.110:0/2543944344 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:02:59.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6371400640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c0b5430 0x7f636c158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f636be00640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636c0726e0 0x7f636c157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 -- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c0b5430 msgr2=0x7f636c158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c0b5430 0x7f636c158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 -- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636c0726e0 msgr2=0x7f636c157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636c0726e0 0x7f636c157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 -- 172.21.15.110:0/2543944344 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c148290 con 0x7f636c06efa0 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f636be00640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636c0726e0 0x7f636c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.975+0000 7f6370a00640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c06efa0 0x7f636c147d50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f635c007ef0 tx=0x7f635c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6368c00640 1 -- 172.21.15.110:0/2543944344 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c014620 con 0x7f636c06efa0 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6368c00640 1 -- 172.21.15.110:0/2543944344 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635c019070 con 0x7f636c06efa0 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6368c00640 1 -- 172.21.15.110:0/2543944344 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c014a00 con 0x7f636c06efa0 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c0b0ec0 con 0x7f636c06efa0 2024-09-10T10:02:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c0b1480 con 0x7f636c06efa0 2024-09-10T10:02:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c15f950 con 0x7f636c06efa0 2024-09-10T10:02:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6368c00640 1 -- 172.21.15.110:0/2543944344 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f635c014ba0 con 0x7f636c06efa0 2024-09-10T10:02:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6368c00640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f634c0679e0 0x7f634c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f6368c00640 1 -- 172.21.15.110:0/2543944344 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f635c086570 con 0x7f636c06efa0 2024-09-10T10:02:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f636be00640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f634c0679e0 0x7f634c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.979+0000 7f636be00640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f634c0679e0 0x7f634c069ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f636c159a90 tx=0x7f6358008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:59.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:02:59.983+0000 7f6368c00640 1 -- 172.21.15.110:0/2543944344 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f635c0571a0 con 0x7f636c06efa0 2024-09-10T10:03:00.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:00.159+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f636c1597e0 con 0x7f636c06efa0 2024-09-10T10:03:01.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.111+0000 7f6368c00640 1 -- 172.21.15.110:0/2543944344 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7f635c056aa0 con 0x7f636c06efa0 2024-09-10T10:03:01.116 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:03:01.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f634c0679e0 msgr2=0x7f634c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f634c0679e0 0x7f634c069ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f636c159a90 tx=0x7f6358008040 comp rx=0 tx=0).stop 2024-09-10T10:03:01.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c06efa0 msgr2=0x7f636c147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c06efa0 0x7f636c147d50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f635c007ef0 tx=0x7f635c00e4b0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 shutdown_connections 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f634c0679e0 0x7f634c069ea0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f636c0b5430 0x7f636c158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f636c0726e0 0x7f636c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 --2- 172.21.15.110:0/2543944344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f636c06efa0 0x7f636c147d50 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 >> 172.21.15.110:0/2543944344 conn(0x7f636c062d10 msgr2=0x7f636c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 shutdown_connections 2024-09-10T10:03:01.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.115+0000 7f6373400640 1 -- 172.21.15.110:0/2543944344 wait complete. 2024-09-10T10:03:01.139 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:03:01.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc39c600640 1 Processor -- start 2024-09-10T10:03:01.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc39c600640 1 -- start start 2024-09-10T10:03:01.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc39c600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc3940ccb60 0x7fc394072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc39c600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3940ccf40 con 0x7fc394072670 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc39c600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3941412a0 con 0x7fc394140c30 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc39c600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc394141080 con 0x7fc3940ccb60 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc399c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc3940ccb60 0x7fc394072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc399c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc3940ccb60 0x7fc394072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47902/0 (socket says 172.21.15.110:47902) 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc399c00640 1 -- 172.21.15.110:0/895640582 learned_addr learned my addr 172.21.15.110:0/895640582 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc399c00640 1 -- 172.21.15.110:0/895640582 >> v1:172.21.15.145:6789/0 conn(0x7fc394140c30 legacy=0x7fc39415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.275+0000 7fc399c00640 1 -- 172.21.15.110:0/895640582 >> v1:172.21.15.110:6789/0 conn(0x7fc394072670 legacy=0x7fc394140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc399c00640 1 -- 172.21.15.110:0/895640582 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39415e760 con 0x7fc3940ccb60 2024-09-10T10:03:01.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc399c00640 1 --2- 172.21.15.110:0/895640582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3940ccb60 0x7fc394072130 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc388009e50 tx=0x7fc38801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7f0fb40bc654c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:01.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc393e00640 1 -- 172.21.15.110:0/895640582 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880a57a0 con 0x7fc3940ccb60 2024-09-10T10:03:01.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc393e00640 1 -- 172.21.15.110:0/895640582 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3880a5940 con 0x7fc3940ccb60 2024-09-10T10:03:01.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc393e00640 1 -- 172.21.15.110:0/895640582 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880a5d10 con 0x7fc3940ccb60 2024-09-10T10:03:01.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc39c600640 1 -- 172.21.15.110:0/895640582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3940ccb60 msgr2=0x7fc394072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc39c600640 1 --2- 172.21.15.110:0/895640582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3940ccb60 0x7fc394072130 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc388009e50 tx=0x7fc38801c9f0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc39c600640 1 -- 172.21.15.110:0/895640582 shutdown_connections 2024-09-10T10:03:01.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc39c600640 1 --2- 172.21.15.110:0/895640582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3940ccb60 0x7fc394072130 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc39c600640 1 -- 172.21.15.110:0/895640582 >> 172.21.15.110:0/895640582 conn(0x7fc39406bc50 msgr2=0x7fc39406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:01.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc39c600640 1 -- 172.21.15.110:0/895640582 shutdown_connections 2024-09-10T10:03:01.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.279+0000 7fc39c600640 1 -- 172.21.15.110:0/895640582 wait complete. 2024-09-10T10:03:01.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 Processor -- start 2024-09-10T10:03:01.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 -- start start 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc394072670 0x7fc39413fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3940ccb60 0x7fc39415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc394140c30 0x7fc394158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc39415f520 con 0x7fc394140c30 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc39415b830 con 0x7fc394072670 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc39415cb30 con 0x7fc3940ccb60 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc394140c30 0x7fc394158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc394140c30 0x7fc394158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47916/0 (socket says 172.21.15.110:47916) 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc399c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc394072670 0x7fc39413fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 -- 172.21.15.110:0/427300384 learned_addr learned my addr 172.21.15.110:0/427300384 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:01.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc399200640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3940ccb60 0x7fc39415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 -- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3940ccb60 msgr2=0x7fc39415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3940ccb60 0x7fc39415c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 -- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc394072670 msgr2=0x7fc39413fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc394072670 0x7fc39413fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 -- 172.21.15.110:0/427300384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39415ce10 con 0x7fc394140c30 2024-09-10T10:03:01.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc399200640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3940ccb60 0x7fc39415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:01.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39a600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc394140c30 0x7fc394158120 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00b410 tx=0x7fc38c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:01.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc391600640 1 -- 172.21.15.110:0/427300384 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c015020 con 0x7fc394140c30 2024-09-10T10:03:01.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc391600640 1 -- 172.21.15.110:0/427300384 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38c011830 con 0x7fc394140c30 2024-09-10T10:03:01.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc391600640 1 -- 172.21.15.110:0/427300384 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c011c20 con 0x7fc394140c30 2024-09-10T10:03:01.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39415d0f0 con 0x7fc394140c30 2024-09-10T10:03:01.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.283+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39415d6b0 con 0x7fc394140c30 2024-09-10T10:03:01.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.287+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc394072130 con 0x7fc394140c30 2024-09-10T10:03:01.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.287+0000 7fc391600640 1 -- 172.21.15.110:0/427300384 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fc38c018020 con 0x7fc394140c30 2024-09-10T10:03:01.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.287+0000 7fc391600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc374067a10 0x7fc374069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.287+0000 7fc391600640 1 -- 172.21.15.110:0/427300384 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7fc38c0858f0 con 0x7fc394140c30 2024-09-10T10:03:01.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.287+0000 7fc399c00640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc374067a10 0x7fc374069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.287+0000 7fc399c00640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc374067a10 0x7fc374069ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ae000 tx=0x7fc3880a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:01.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.291+0000 7fc391600640 1 -- 172.21.15.110:0/427300384 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc38c01f020 con 0x7fc394140c30 2024-09-10T10:03:01.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.503+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc394075450 con 0x7fc394140c30 2024-09-10T10:03:01.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.503+0000 7fc391600640 1 -- 172.21.15.110:0/427300384 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fc38c056660 con 0x7fc394140c30 2024-09-10T10:03:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:01.507 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:03:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:03:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:03:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:03:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:03:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:03:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:03:01.521 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:01.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc374067a10 msgr2=0x7fc374069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc374067a10 0x7fc374069ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ae000 tx=0x7fc3880a7070 comp rx=0 tx=0).stop 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc394140c30 msgr2=0x7fc394158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc394140c30 0x7fc394158120 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00b410 tx=0x7fc38c00b8d0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 shutdown_connections 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc374067a10 0x7fc374069ed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc394140c30 0x7fc394158120 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3940ccb60 0x7fc39415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 --2- 172.21.15.110:0/427300384 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc394072670 0x7fc39413fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 >> 172.21.15.110:0/427300384 conn(0x7fc39406bc50 msgr2=0x7fc3940b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 shutdown_connections 2024-09-10T10:03:01.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.507+0000 7fc39c600640 1 -- 172.21.15.110:0/427300384 wait complete. 2024-09-10T10:03:01.528 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:03:01.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.659+0000 7fbdda600640 1 Processor -- start 2024-09-10T10:03:01.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.659+0000 7fbdda600640 1 -- start start 2024-09-10T10:03:01.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.659+0000 7fbdda600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.659+0000 7fbdda600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd40703a0 con 0x7fbdd406ff60 2024-09-10T10:03:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdda600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd406eec0 con 0x7fbdd40729b0 2024-09-10T10:03:01.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdda600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd415eae0 con 0x7fbdd415e700 2024-09-10T10:03:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd2a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fbdd406ff60 0x7fbdd40af270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52814/0 (socket says 172.21.15.110:52814) 2024-09-10T10:03:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd2a00640 1 -- 172.21.15.110:0/64024879 learned_addr learned my addr 172.21.15.110:0/64024879 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd3400640 1 --2- 172.21.15.110:0/64024879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd2000640 1 -- 172.21.15.110:0/64024879 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2815431859 0 0) 0x7fbdc0001120 con 0x7fbdd406ff60 2024-09-10T10:03:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd2000640 1 -- 172.21.15.110:0/64024879 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdb8003530 con 0x7fbdd406ff60 2024-09-10T10:03:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd3400640 1 -- 172.21.15.110:0/64024879 >> v1:172.21.15.145:6789/0 conn(0x7fbdd40729b0 legacy=0x7fbdd40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd3400640 1 -- 172.21.15.110:0/64024879 >> v1:172.21.15.110:6789/0 conn(0x7fbdd406ff60 legacy=0x7fbdd40af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd3400640 1 -- 172.21.15.110:0/64024879 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd406f1a0 con 0x7fbdd415e700 2024-09-10T10:03:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.663+0000 7fbdd3400640 1 --2- 172.21.15.110:0/64024879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd406e890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009dc0 tx=0x7fbdc401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=117f5ac93a48c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdd2000640 1 -- 172.21.15.110:0/64024879 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40b2070 con 0x7fbdd415e700 2024-09-10T10:03:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdd2000640 1 -- 172.21.15.110:0/64024879 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc40a5a20 con 0x7fbdd415e700 2024-09-10T10:03:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdd2000640 1 -- 172.21.15.110:0/64024879 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40a5dd0 con 0x7fbdd415e700 2024-09-10T10:03:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- 172.21.15.110:0/64024879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 msgr2=0x7fbdd406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 --2- 172.21.15.110:0/64024879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd406e890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009dc0 tx=0x7fbdc401c960 comp rx=0 tx=0).stop 2024-09-10T10:03:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- 172.21.15.110:0/64024879 shutdown_connections 2024-09-10T10:03:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 --2- 172.21.15.110:0/64024879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd406e890 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- 172.21.15.110:0/64024879 >> 172.21.15.110:0/64024879 conn(0x7fbdd406cb90 msgr2=0x7fbdd406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- 172.21.15.110:0/64024879 shutdown_connections 2024-09-10T10:03:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- 172.21.15.110:0/64024879 wait complete. 2024-09-10T10:03:01.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 Processor -- start 2024-09-10T10:03:01.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- start start 2024-09-10T10:03:01.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbdd406ff60 0x7fbdd415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd40729b0 0x7fbdd415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd40731d0 con 0x7fbdd40729b0 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd40af980 con 0x7fbdd406ff60 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdda600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd415d580 con 0x7fbdd415e700 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdd2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd40729b0 0x7fbdd415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdd2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd40729b0 0x7fbdd415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47918/0 (socket says 172.21.15.110:47918) 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdd2a00640 1 -- 172.21.15.110:0/3274579965 learned_addr learned my addr 172.21.15.110:0/3274579965 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.667+0000 7fbdd3400640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbdd406ff60 0x7fbdd415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdd3e00640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdd2a00640 1 -- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 msgr2=0x7fbdd414efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdd2a00640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd414efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdd2a00640 1 -- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbdd406ff60 msgr2=0x7fbdd415d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdd2a00640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbdd406ff60 0x7fbdd415d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdd2a00640 1 -- 172.21.15.110:0/3274579965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd415df20 con 0x7fbdd40729b0 2024-09-10T10:03:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdd2a00640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd40729b0 0x7fbdd415d040 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c730 tx=0x7fbdc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdcb400640 1 -- 172.21.15.110:0/3274579965 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0017070 con 0x7fbdd40729b0 2024-09-10T10:03:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdcb400640 1 -- 172.21.15.110:0/3274579965 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc0005ac0 con 0x7fbdd40729b0 2024-09-10T10:03:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdcb400640 1 -- 172.21.15.110:0/3274579965 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0007070 con 0x7fbdd40729b0 2024-09-10T10:03:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd415c040 con 0x7fbdd40729b0 2024-09-10T10:03:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd415c600 con 0x7fbdd40729b0 2024-09-10T10:03:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd40b4a00 con 0x7fbdd40729b0 2024-09-10T10:03:01.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdcb400640 1 -- 172.21.15.110:0/3274579965 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fbdc001a070 con 0x7fbdd40729b0 2024-09-10T10:03:01.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdcb400640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbdb40679e0 0x7fbdb4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdcb400640 1 -- 172.21.15.110:0/3274579965 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7fbdc0084f10 con 0x7fbdd40729b0 2024-09-10T10:03:01.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.671+0000 7fbdd3400640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbdb40679e0 0x7fbdb4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.675+0000 7fbdd3400640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbdb40679e0 0x7fbdb4069ea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40af000 tx=0x7fbdc40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:01.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.675+0000 7fbdcb400640 1 -- 172.21.15.110:0/3274579965 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbdd40703a0 con 0x7fbdd40729b0 2024-09-10T10:03:01.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.835+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbdd40afb00 con 0x7fbdd40729b0 2024-09-10T10:03:01.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.835+0000 7fbdcb400640 1 -- 172.21.15.110:0/3274579965 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) ==== 84+0+65 (secure 0 0 0) 0x7fbdc0055c80 con 0x7fbdd40729b0 2024-09-10T10:03:01.841 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:01.841 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:03:01.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:01.841 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:03:01.841 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:03:01.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:01.841 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:03:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbdb40679e0 msgr2=0x7fbdb4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbdb40679e0 0x7fbdb4069ea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40af000 tx=0x7fbdc40a7070 comp rx=0 tx=0).stop 2024-09-10T10:03:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd40729b0 msgr2=0x7fbdd415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd40729b0 0x7fbdd415d040 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c730 tx=0x7fbdc000cbf0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 shutdown_connections 2024-09-10T10:03:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbdb40679e0 0x7fbdb4069ea0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbdd415e700 0x7fbdd414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdd40729b0 0x7fbdd415d040 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 --2- 172.21.15.110:0/3274579965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbdd406ff60 0x7fbdd415d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 >> 172.21.15.110:0/3274579965 conn(0x7fbdd406cb90 msgr2=0x7fbdd40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:01.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 shutdown_connections 2024-09-10T10:03:01.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.839+0000 7fbdda600640 1 -- 172.21.15.110:0/3274579965 wait complete. 2024-09-10T10:03:01.861 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:03:01.861 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:03:02.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.995+0000 7f4ebf400640 1 Processor -- start 2024-09-10T10:03:02.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.995+0000 7f4ebf400640 1 -- start start 2024-09-10T10:03:02.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebf400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4eb8155970 0x7f4eb80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebf400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb8155d50 con 0x7f4eb806f1e0 2024-09-10T10:03:02.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebf400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb80b34d0 con 0x7f4eb815c560 2024-09-10T10:03:02.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebf400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb80b4570 con 0x7f4eb8155970 2024-09-10T10:03:02.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebca00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4eb8155970 0x7f4eb80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebd400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f4eb806f1e0 0x7f4eb80ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52828/0 (socket says 172.21.15.110:52828) 2024-09-10T10:03:02.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebd400640 1 -- 172.21.15.110:0/3007273916 learned_addr learned my addr 172.21.15.110:0/3007273916 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:02.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebca00640 1 -- 172.21.15.110:0/3007273916 >> v1:172.21.15.145:6789/0 conn(0x7f4eb815c560 legacy=0x7f4eb806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:02.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebca00640 1 -- 172.21.15.110:0/3007273916 >> v1:172.21.15.110:6789/0 conn(0x7f4eb806f1e0 legacy=0x7f4eb80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:02.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4ebca00640 1 -- 172.21.15.110:0/3007273916 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb80b37b0 con 0x7f4eb8155970 2024-09-10T10:03:02.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:01.999+0000 7f4eb7400640 1 -- 172.21.15.110:0/3007273916 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1964761626 0 0) 0x7f4eb8155d50 con 0x7f4eb806f1e0 2024-09-10T10:03:02.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebca00640 1 --2- 172.21.15.110:0/3007273916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb80cc8c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80097c0 tx=0x7f4ea801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2938db7982bf4b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:02.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4eb7400640 1 -- 172.21.15.110:0/3007273916 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80a7070 con 0x7f4eb8155970 2024-09-10T10:03:02.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4eb7400640 1 -- 172.21.15.110:0/3007273916 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea80b0070 con 0x7f4eb8155970 2024-09-10T10:03:02.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4eb7400640 1 -- 172.21.15.110:0/3007273916 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80ab890 con 0x7f4eb8155970 2024-09-10T10:03:02.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- 172.21.15.110:0/3007273916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 msgr2=0x7f4eb80cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:02.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 --2- 172.21.15.110:0/3007273916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb80cc8c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80097c0 tx=0x7f4ea801c470 comp rx=0 tx=0).stop 2024-09-10T10:03:02.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- 172.21.15.110:0/3007273916 shutdown_connections 2024-09-10T10:03:02.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 --2- 172.21.15.110:0/3007273916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb80cc8c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:02.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- 172.21.15.110:0/3007273916 >> 172.21.15.110:0/3007273916 conn(0x7f4eb806bc50 msgr2=0x7f4eb806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:02.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- 172.21.15.110:0/3007273916 shutdown_connections 2024-09-10T10:03:02.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- 172.21.15.110:0/3007273916 wait complete. 2024-09-10T10:03:02.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 Processor -- start 2024-09-10T10:03:02.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- start start 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4eb806f1e0 0x7f4eb814b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb81500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eb815c560 0x7f4eb80b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80cf340 con 0x7f4eb8155970 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80cce00 con 0x7f4eb815c560 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4ebf400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb80cf4c0 con 0x7f4eb806f1e0 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb81500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb81500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47938/0 (socket says 172.21.15.110:47938) 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.003+0000 7f4eb7e00640 1 -- 172.21.15.110:0/3521068256 learned_addr learned my addr 172.21.15.110:0/3521068256 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4ebca00640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4eb806f1e0 0x7f4eb814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4ebd400640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eb815c560 0x7f4eb80b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb7e00640 1 -- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4eb806f1e0 msgr2=0x7f4eb814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb7e00640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4eb806f1e0 0x7f4eb814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb7e00640 1 -- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eb815c560 msgr2=0x7f4eb80b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb7e00640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eb815c560 0x7f4eb80b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb7e00640 1 -- 172.21.15.110:0/3521068256 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb814bba0 con 0x7f4eb8155970 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb7e00640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb81500f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c3c0 tx=0x7f4ea400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb4c00640 1 -- 172.21.15.110:0/3521068256 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4017070 con 0x7f4eb8155970 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb4c00640 1 -- 172.21.15.110:0/3521068256 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea40052d0 con 0x7f4eb8155970 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb814be80 con 0x7f4eb8155970 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb4c00640 1 -- 172.21.15.110:0/3521068256 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4003840 con 0x7f4eb8155970 2024-09-10T10:03:02.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb814c440 con 0x7f4eb8155970 2024-09-10T10:03:02.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb80cc8c0 con 0x7f4eb8155970 2024-09-10T10:03:02.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb4c00640 1 -- 172.21.15.110:0/3521068256 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f4ea401a070 con 0x7f4eb8155970 2024-09-10T10:03:02.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb4c00640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4e980679e0 0x7f4e98069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4eb4c00640 1 -- 172.21.15.110:0/3521068256 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f4ea4084e70 con 0x7f4eb8155970 2024-09-10T10:03:02.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.007+0000 7f4ebca00640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4e980679e0 0x7f4e98069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.011+0000 7f4ebca00640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4e980679e0 0x7f4e98069ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80af000 tx=0x7f4ea80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:02.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.011+0000 7f4eb4c00640 1 -- 172.21.15.110:0/3521068256 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4ea4005480 con 0x7f4eb8155970 2024-09-10T10:03:02.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:02.171+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4eb806c500 con 0x7f4eb8155970 2024-09-10T10:03:03.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.115+0000 7f4eb4c00640 1 -- 172.21.15.110:0/3521068256 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) ==== 138+0+0 (secure 0 0 0) 0x7f4ea4055b50 con 0x7f4eb8155970 2024-09-10T10:03:03.120 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:03:03.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4e980679e0 msgr2=0x7f4e98069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:03.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4e980679e0 0x7f4e98069ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80af000 tx=0x7f4ea80a7000 comp rx=0 tx=0).stop 2024-09-10T10:03:03.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 msgr2=0x7f4eb81500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:03.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb81500f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c3c0 tx=0x7f4ea400c880 comp rx=0 tx=0).stop 2024-09-10T10:03:03.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 shutdown_connections 2024-09-10T10:03:03.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4e980679e0 0x7f4e98069ea0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eb815c560 0x7f4eb80b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4eb8155970 0x7f4eb81500f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 --2- 172.21.15.110:0/3521068256 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4eb806f1e0 0x7f4eb814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 >> 172.21.15.110:0/3521068256 conn(0x7f4eb806bc50 msgr2=0x7f4eb8074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:03.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 shutdown_connections 2024-09-10T10:03:03.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.119+0000 7f4ebf400640 1 -- 172.21.15.110:0/3521068256 wait complete. 2024-09-10T10:03:03.152 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:03:03.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5f2000640 1 Processor -- start 2024-09-10T10:03:03.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5f2000640 1 -- start start 2024-09-10T10:03:03.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5f2000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff5ec0cfe30 0x7ff5ec071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:03.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5f2000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec0cd510 0x7ff5ec0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:03.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5f2000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ec06d2d0 con 0x7ff5ec06e090 2024-09-10T10:03:03.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5f2000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff5ec0cdeb0 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5f2000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec15d9a0 con 0x7ff5ec0cd510 2024-09-10T10:03:03.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5eaa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec0cd510 0x7ff5ec0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:03.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5eb400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff5ec0cfe30 0x7ff5ec071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:03.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5ebe00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ff5ec06e090 0x7ff5ec06e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:59014/0 (socket says 172.21.15.110:59014) 2024-09-10T10:03:03.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.295+0000 7ff5ebe00640 1 -- 172.21.15.110:0/1241189380 learned_addr learned my addr 172.21.15.110:0/1241189380 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:03.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5eaa00640 1 -- 172.21.15.110:0/1241189380 >> v2:172.21.15.145:3300/0 conn(0x7ff5ec0cfe30 msgr2=0x7ff5ec071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:03.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5eaa00640 1 --2- 172.21.15.110:0/1241189380 >> v2:172.21.15.145:3300/0 conn(0x7ff5ec0cfe30 0x7ff5ec071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5ea000640 1 -- 172.21.15.110:0/1241189380 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 610926234 0 0) 0x7ff5ec06d2d0 con 0x7ff5ec06e090 2024-09-10T10:03:03.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5eaa00640 1 -- 172.21.15.110:0/1241189380 >> v1:172.21.15.145:6789/0 conn(0x7ff5ec06e090 legacy=0x7ff5ec06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:03.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5eaa00640 1 -- 172.21.15.110:0/1241189380 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec06d5b0 con 0x7ff5ec0cd510 2024-09-10T10:03:03.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5eaa00640 1 --2- 172.21.15.110:0/1241189380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec0cd510 0x7ff5ec0cd970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80097c0 tx=0x7ff5d801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5fd855e96bee03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:03.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5ea000640 1 -- 172.21.15.110:0/1241189380 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80b0070 con 0x7ff5ec0cd510 2024-09-10T10:03:03.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5ea000640 1 -- 172.21.15.110:0/1241189380 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d80ab600 con 0x7ff5ec0cd510 2024-09-10T10:03:03.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.299+0000 7ff5ea000640 1 -- 172.21.15.110:0/1241189380 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80ab9d0 con 0x7ff5ec0cd510 2024-09-10T10:03:03.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- 172.21.15.110:0/1241189380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec0cd510 msgr2=0x7ff5ec0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:03.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1241189380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec0cd510 0x7ff5ec0cd970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80097c0 tx=0x7ff5d801c410 comp rx=0 tx=0).stop 2024-09-10T10:03:03.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- 172.21.15.110:0/1241189380 shutdown_connections 2024-09-10T10:03:03.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1241189380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec0cd510 0x7ff5ec0cd970 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1241189380 >> v2:172.21.15.145:3300/0 conn(0x7ff5ec0cfe30 0x7ff5ec071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- 172.21.15.110:0/1241189380 >> 172.21.15.110:0/1241189380 conn(0x7ff5ec06bc50 msgr2=0x7ff5ec06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:03.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- 172.21.15.110:0/1241189380 shutdown_connections 2024-09-10T10:03:03.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- 172.21.15.110:0/1241189380 wait complete. 2024-09-10T10:03:03.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 Processor -- start 2024-09-10T10:03:03.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- start start 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec06e090 0x7ff5ec158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5ec0cd510 0x7ff5ec1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5ec0cfe30 0x7ff5ec156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec072190 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec15d640 con 0x7ff5ec0cd510 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5f2000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec169180 con 0x7ff5ec06e090 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5ec0cfe30 0x7ff5ec156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5ec0cfe30 0x7ff5ec156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47952/0 (socket says 172.21.15.110:47952) 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 -- 172.21.15.110:0/1292628225 learned_addr learned my addr 172.21.15.110:0/1292628225 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:03.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5eb400640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec06e090 0x7ff5ec158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:03.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5eaa00640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5ec0cd510 0x7ff5ec1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 -- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec06e090 msgr2=0x7ff5ec158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec06e090 0x7ff5ec158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 -- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5ec0cd510 msgr2=0x7ff5ec1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5ec0cd510 0x7ff5ec1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 -- 172.21.15.110:0/1292628225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec1559f0 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.303+0000 7ff5ebe00640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5ec0cfe30 0x7ff5ec156190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c410 tx=0x7ff5e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.307+0000 7ff5e3400640 1 -- 172.21.15.110:0/1292628225 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40074e0 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.307+0000 7ff5e3400640 1 -- 172.21.15.110:0/1292628225 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e40047f0 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.307+0000 7ff5e3400640 1 -- 172.21.15.110:0/1292628225 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4004be0 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.307+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec158320 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.307+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec155630 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.307+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec0cd970 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.307+0000 7ff5e3400640 1 -- 172.21.15.110:0/1292628225 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7ff5e4011070 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.311+0000 7ff5e3400640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5cc0679e0 0x7ff5cc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.311+0000 7ff5e3400640 1 -- 172.21.15.110:0/1292628225 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5227+0+0 (secure 0 0 0) 0x7ff5e4085d60 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.311+0000 7ff5eb400640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5cc0679e0 0x7ff5cc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:03.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.311+0000 7ff5eb400640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5cc0679e0 0x7ff5cc069ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec167040 tx=0x7ff5dc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.315+0000 7ff5e3400640 1 -- 172.21.15.110:0/1292628225 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff5e4056280 con 0x7ff5ec0cfe30 2024-09-10T10:03:03.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:03.471+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff5ec075560 con 0x7ff5ec0cfe30 2024-09-10T10:03:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.123+0000 7ff5e3400640 1 -- 172.21.15.110:0/1292628225 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) ==== 170+0+0 (secure 0 0 0) 0x7ff5e405a580 con 0x7ff5ec0cfe30 2024-09-10T10:03:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:03:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5cc0679e0 msgr2=0x7ff5cc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5cc0679e0 0x7ff5cc069ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec167040 tx=0x7ff5dc00b040 comp rx=0 tx=0).stop 2024-09-10T10:03:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5ec0cfe30 msgr2=0x7ff5ec156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5ec0cfe30 0x7ff5ec156190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c410 tx=0x7ff5e400c8d0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 shutdown_connections 2024-09-10T10:03:04.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5cc0679e0 0x7ff5cc069ea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5ec0cfe30 0x7ff5ec156190 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5ec0cd510 0x7ff5ec1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 --2- 172.21.15.110:0/1292628225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5ec06e090 0x7ff5ec158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 >> 172.21.15.110:0/1292628225 conn(0x7ff5ec06bc50 msgr2=0x7ff5ec06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:04.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 shutdown_connections 2024-09-10T10:03:04.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.127+0000 7ff5f2000640 1 -- 172.21.15.110:0/1292628225 wait complete. 2024-09-10T10:03:04.168 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:03:04.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.303+0000 7f48f9600640 1 Processor -- start 2024-09-10T10:03:04.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.303+0000 7f48f9600640 1 -- start start 2024-09-10T10:03:04.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.303+0000 7f48f9600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f48f415b330 0x7f48f406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.303+0000 7f48f9600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f40b4960 con 0x7f48f40ce4d0 2024-09-10T10:03:04.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f9600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f40b4c40 con 0x7f48f406e850 2024-09-10T10:03:04.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f9600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f48f40b5570 con 0x7f48f415b330 2024-09-10T10:03:04.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f48f40ce4d0 0x7f48f406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52836/0 (socket says 172.21.15.110:52836) 2024-09-10T10:03:04.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2000640 1 -- 172.21.15.110:0/2283289530 learned_addr learned my addr 172.21.15.110:0/2283289530 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2a00640 1 --2- 172.21.15.110:0/2283289530 >> v2:172.21.15.145:3300/0 conn(0x7f48f415b330 0x7f48f406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617031797 0 0) 0x7f48f40b4960 con 0x7f48f40ce4d0 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48d8003530 con 0x7f48f40ce4d0 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 557142342 0 0) 0x7f48f40b4c40 con 0x7f48f406e850 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48f40b4960 con 0x7f48f406e850 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2443256991 0 0) 0x7f48d8003530 con 0x7f48f40ce4d0 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 >> v2:172.21.15.145:3300/0 conn(0x7f48f415b330 msgr2=0x7f48f406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 --2- 172.21.15.110:0/2283289530 >> v2:172.21.15.145:3300/0 conn(0x7f48f415b330 0x7f48f406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 >> v1:172.21.15.145:6789/0 conn(0x7f48f406e850 legacy=0x7f48f40b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f40b4f20 con 0x7f48f40ce4d0 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f48e0001680 con 0x7f48f40ce4d0 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2a00640 1 --2- 172.21.15.110:0/2283289530 >> v2:172.21.15.145:3300/0 conn(0x7f48f415b330 0x7f48f406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 >> v1:172.21.15.110:6789/0 conn(0x7f48f40ce4d0 legacy=0x7f48f406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48d8003b50 0x7f48d8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48d8004810 0x7f48d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d80051d0 0x7f48d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f48f40b5570 con 0x7f48d8004810 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f48d800c050 con 0x7f48d80051d0 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f48d800c250 con 0x7f48d8003b50 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2000640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48d8004810 0x7f48d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2a00640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48d8003b50 0x7f48d8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f3400640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d80051d0 0x7f48d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2000640 1 -- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48d8003b50 msgr2=0x7f48d8003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2000640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48d8003b50 0x7f48d8003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2000640 1 -- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d80051d0 msgr2=0x7f48d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2000640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d80051d0 0x7f48d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2000640 1 -- 172.21.15.110:0/2283289530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d8003530 con 0x7f48d8004810 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f2a00640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48d8003b50 0x7f48d8003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f3400640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d80051d0 0x7f48d800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:04.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.307+0000 7f48f3400640 1 -- 172.21.15.110:0/2283289530 reap_dead start 2024-09-10T10:03:04.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f2000640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48d8004810 0x7f48d8004c90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f48e0007cb0 tx=0x7f48e001ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cb5f46148ab337 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0007cf0 con 0x7f48d8004810 2024-09-10T10:03:04.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e00ab020 con 0x7f48d8004810 2024-09-10T10:03:04.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f1600640 1 -- 172.21.15.110:0/2283289530 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e00a7d30 con 0x7f48d8004810 2024-09-10T10:03:04.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 -- 172.21.15.110:0/2283289530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48d8004810 msgr2=0x7f48d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48d8004810 0x7f48d8004c90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f48e0007cb0 tx=0x7f48e001ee10 comp rx=0 tx=0).stop 2024-09-10T10:03:04.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 -- 172.21.15.110:0/2283289530 shutdown_connections 2024-09-10T10:03:04.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 --2- 172.21.15.110:0/2283289530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48d8004810 0x7f48d8004c90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 -- 172.21.15.110:0/2283289530 >> 172.21.15.110:0/2283289530 conn(0x7f48f4062e40 msgr2=0x7f48f406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:04.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 -- 172.21.15.110:0/2283289530 shutdown_connections 2024-09-10T10:03:04.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 -- 172.21.15.110:0/2283289530 wait complete. 2024-09-10T10:03:04.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 Processor -- start 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.311+0000 7f48f9600640 1 -- start start 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d8004810 0x7f48f4144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48f415eea0 0x7f48f415c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48f415cd80 0x7f48f40715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f48f40b1c60 con 0x7f48f415cd80 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f48f415b710 con 0x7f48d8004810 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f48f40b1de0 con 0x7f48f415eea0 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48f415cd80 0x7f48f40715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48f415cd80 0x7f48f40715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47970/0 (socket says 172.21.15.110:47970) 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 -- 172.21.15.110:0/1007138357 learned_addr learned my addr 172.21.15.110:0/1007138357 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f2000640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48f415eea0 0x7f48f415c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f2a00640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d8004810 0x7f48f4144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 -- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48f415eea0 msgr2=0x7f48f415c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48f415eea0 0x7f48f415c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 -- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d8004810 msgr2=0x7f48f4144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d8004810 0x7f48f4144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 -- 172.21.15.110:0/1007138357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f4154fc0 con 0x7f48f415cd80 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f2a00640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d8004810 0x7f48f4144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f3400640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48f415cd80 0x7f48f40715a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f48ec007af0 tx=0x7f48ec00f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48eaa00640 1 -- 172.21.15.110:0/1007138357 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec010040 con 0x7f48f415cd80 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48eaa00640 1 -- 172.21.15.110:0/1007138357 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48ec01a070 con 0x7f48f415cd80 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48eaa00640 1 -- 172.21.15.110:0/1007138357 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec015410 con 0x7f48f415cd80 2024-09-10T10:03:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f415f420 con 0x7f48f415cd80 2024-09-10T10:03:04.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f415f9e0 con 0x7f48f415cd80 2024-09-10T10:03:04.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f4155190 con 0x7f48f415cd80 2024-09-10T10:03:04.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.315+0000 7f48eaa00640 1 -- 172.21.15.110:0/1007138357 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f48ec0155b0 con 0x7f48f415cd80 2024-09-10T10:03:04.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.319+0000 7f48eaa00640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48d40679e0 0x7f48d4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.319+0000 7f48eaa00640 1 -- 172.21.15.110:0/1007138357 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5609+0+0 (secure 0 0 0) 0x7f48ec089280 con 0x7f48f415cd80 2024-09-10T10:03:04.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.319+0000 7f48f2a00640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48d40679e0 0x7f48d4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.319+0000 7f48f2a00640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48d40679e0 0x7f48d4069ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009fd0 tx=0x7f48e400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.323+0000 7f48eaa00640 1 -- 172.21.15.110:0/1007138357 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f48ec059d00 con 0x7f48f415cd80 2024-09-10T10:03:04.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:04.511+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f48f406c6a0 con 0x7f48f415cd80 2024-09-10T10:03:05.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.127+0000 7f48eaa00640 1 -- 172.21.15.110:0/1007138357 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v64) ==== 174+0+0 (secure 0 0 0) 0x7f48ec059600 con 0x7f48f415cd80 2024-09-10T10:03:05.130 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 16 and data pool 17 2024-09-10T10:03:05.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.127+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48d40679e0 msgr2=0x7f48d4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.127+0000 7f48f9600640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48d40679e0 0x7f48d4069ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009fd0 tx=0x7f48e400b040 comp rx=0 tx=0).stop 2024-09-10T10:03:05.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.127+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48f415cd80 msgr2=0x7f48f40715a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.127+0000 7f48f9600640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48f415cd80 0x7f48f40715a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f48ec007af0 tx=0x7f48ec00f540 comp rx=0 tx=0).stop 2024-09-10T10:03:05.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.131+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 shutdown_connections 2024-09-10T10:03:05.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.131+0000 7f48f9600640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48d40679e0 0x7f48d4069ea0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.131+0000 7f48f9600640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48f415cd80 0x7f48f40715a0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.131+0000 7f48f9600640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48f415eea0 0x7f48f415c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.131+0000 7f48f9600640 1 --2- 172.21.15.110:0/1007138357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48d8004810 0x7f48f4144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.131+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 >> 172.21.15.110:0/1007138357 conn(0x7f48f4062e40 msgr2=0x7f48f406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:05.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.131+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 shutdown_connections 2024-09-10T10:03:05.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.131+0000 7f48f9600640 1 -- 172.21.15.110:0/1007138357 wait complete. 2024-09-10T10:03:05.183 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:03:05.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f775e200640 1 Processor -- start 2024-09-10T10:03:05.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f775e200640 1 -- start start 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f775e200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f77580b5690 0x7f77580b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f775e200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77580cf7e0 con 0x7f7758145b00 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f775e200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7758071db0 con 0x7f775815c940 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f775e200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f77580b6010 con 0x7f77580b5690 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f7757e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f77580b5690 0x7f77580b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f7757e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f77580b5690 0x7f77580b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47974/0 (socket says 172.21.15.110:47974) 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f7757e00640 1 -- 172.21.15.110:0/62809988 learned_addr learned my addr 172.21.15.110:0/62809988 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.315+0000 7f775ce00640 1 --1- 172.21.15.110:0/62809988 >> v1:172.21.15.110:6789/0 conn(0x7f7758145b00 0x7f77580cf0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52850/0 (socket says 172.21.15.110:52850) 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f7757e00640 1 -- 172.21.15.110:0/62809988 >> v1:172.21.15.145:6789/0 conn(0x7f775815c940 legacy=0x7f775815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f7757e00640 1 -- 172.21.15.110:0/62809988 >> v1:172.21.15.110:6789/0 conn(0x7f7758145b00 legacy=0x7f77580cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f7757e00640 1 -- 172.21.15.110:0/62809988 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77580ce5f0 con 0x7f77580b5690 2024-09-10T10:03:05.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f7757e00640 1 --2- 172.21.15.110:0/62809988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f77580b5ad0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f77580b6190 tx=0x7f774401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b48df7b6da6bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f7757400640 1 -- 172.21.15.110:0/62809988 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77440b0070 con 0x7f77580b5690 2024-09-10T10:03:05.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f7757400640 1 -- 172.21.15.110:0/62809988 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77440ab600 con 0x7f77580b5690 2024-09-10T10:03:05.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f7757400640 1 -- 172.21.15.110:0/62809988 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77440ab9d0 con 0x7f77580b5690 2024-09-10T10:03:05.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f775e200640 1 -- 172.21.15.110:0/62809988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 msgr2=0x7f77580b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f775e200640 1 --2- 172.21.15.110:0/62809988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f77580b5ad0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f77580b6190 tx=0x7f774401c540 comp rx=0 tx=0).stop 2024-09-10T10:03:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f775e200640 1 -- 172.21.15.110:0/62809988 shutdown_connections 2024-09-10T10:03:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f775e200640 1 --2- 172.21.15.110:0/62809988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f77580b5ad0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f775e200640 1 -- 172.21.15.110:0/62809988 >> 172.21.15.110:0/62809988 conn(0x7f775806d0f0 msgr2=0x7f775806d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.319+0000 7f775e200640 1 -- 172.21.15.110:0/62809988 shutdown_connections 2024-09-10T10:03:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 -- 172.21.15.110:0/62809988 wait complete. 2024-09-10T10:03:05.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 Processor -- start 2024-09-10T10:03:05.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 -- start start 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f775813eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7758145b00 0x7f7758159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f775815c940 0x7f775813dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77580719d0 con 0x7f77580b5690 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7758071850 con 0x7f7758145b00 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7758071b50 con 0x7f775815c940 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f775813eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f775813eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47978/0 (socket says 172.21.15.110:47978) 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 -- 172.21.15.110:0/3144347390 learned_addr learned my addr 172.21.15.110:0/3144347390 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:05.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f7757e00640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7758145b00 0x7f7758159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775d800640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f775815c940 0x7f775813dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 -- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f775815c940 msgr2=0x7f775813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f775815c940 0x7f775813dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 -- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7758145b00 msgr2=0x7f7758159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7758145b00 0x7f7758159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 -- 172.21.15.110:0/3144347390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775813f030 con 0x7f77580b5690 2024-09-10T10:03:05.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775ce00640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f775813eaf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f774800c4c0 tx=0x7f774800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f7754c00640 1 -- 172.21.15.110:0/3144347390 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748018070 con 0x7f77580b5690 2024-09-10T10:03:05.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f7754c00640 1 -- 172.21.15.110:0/3144347390 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7748007480 con 0x7f77580b5690 2024-09-10T10:03:05.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775815a380 con 0x7f77580b5690 2024-09-10T10:03:05.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f7754c00640 1 -- 172.21.15.110:0/3144347390 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480047f0 con 0x7f77580b5690 2024-09-10T10:03:05.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.323+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77580ae4b0 con 0x7f77580b5690 2024-09-10T10:03:05.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.327+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77580b5ad0 con 0x7f77580b5690 2024-09-10T10:03:05.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.327+0000 7f7754c00640 1 -- 172.21.15.110:0/3144347390 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f774801b070 con 0x7f77580b5690 2024-09-10T10:03:05.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.327+0000 7f7754c00640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f77380679e0 0x7f7738069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.327+0000 7f7754c00640 1 -- 172.21.15.110:0/3144347390 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f7748084fe0 con 0x7f77580b5690 2024-09-10T10:03:05.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.327+0000 7f7757e00640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f77380679e0 0x7f7738069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.331+0000 7f7757e00640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f77380679e0 0x7f7738069ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77580b6640 tx=0x7f7744008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.331+0000 7f7754c00640 1 -- 172.21.15.110:0/3144347390 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f775813f030 con 0x7f77580b5690 2024-09-10T10:03:05.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77580aa540 con 0x7f77580b5690 2024-09-10T10:03:05.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f7754c00640 1 -- 172.21.15.110:0/3144347390 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16736 (secure 0 0 0) 0x7f7748055360 con 0x7f77580b5690 2024-09-10T10:03:05.494 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:05.495 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":102,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:04.515844+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:02.221093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:03.519991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:05.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f77380679e0 msgr2=0x7f7738069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f77380679e0 0x7f7738069ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77580b6640 tx=0x7f7744008040 comp rx=0 tx=0).stop 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 msgr2=0x7f775813eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f775813eaf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f774800c4c0 tx=0x7f774800c980 comp rx=0 tx=0).stop 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 shutdown_connections 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f77380679e0 0x7f7738069ea0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f775815c940 0x7f775813dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7758145b00 0x7f7758159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 --2- 172.21.15.110:0/3144347390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77580b5690 0x7f775813eaf0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 >> 172.21.15.110:0/3144347390 conn(0x7f775806d0f0 msgr2=0x7f775813f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 shutdown_connections 2024-09-10T10:03:05.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.491+0000 7f775e200640 1 -- 172.21.15.110:0/3144347390 wait complete. 2024-09-10T10:03:05.516 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:03:05.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4870a00640 1 Processor -- start 2024-09-10T10:03:05.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4870a00640 1 -- start start 2024-09-10T10:03:05.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4870a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f486c147f00 0x7f486c0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4870a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c0aed80 0x7f486c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4870a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c0af1a0 con 0x7f486c06ec20 2024-09-10T10:03:05.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4870a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f486c0b34d0 con 0x7f486c147f00 2024-09-10T10:03:05.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4870a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f486c1482e0 con 0x7f486c0aed80 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f486a800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f486c06ec20 0x7f486c06d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52858/0 (socket says 172.21.15.110:52858) 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f486a800640 1 -- 172.21.15.110:0/3513582761 learned_addr learned my addr 172.21.15.110:0/3513582761 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4869400640 1 --2- 172.21.15.110:0/3513582761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c0aed80 0x7f486c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4869e00640 1 --2- 172.21.15.110:0/3513582761 >> v2:172.21.15.145:3300/0 conn(0x7f486c147f00 0x7f486c0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4868a00640 1 -- 172.21.15.110:0/3513582761 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1640154241 0 0) 0x7f486c0af1a0 con 0x7f486c06ec20 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.647+0000 7f4868a00640 1 -- 172.21.15.110:0/3513582761 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4850003530 con 0x7f486c06ec20 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4869400640 1 -- 172.21.15.110:0/3513582761 >> v2:172.21.15.145:3300/0 conn(0x7f486c147f00 msgr2=0x7f486c0ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4869400640 1 --2- 172.21.15.110:0/3513582761 >> v2:172.21.15.145:3300/0 conn(0x7f486c147f00 0x7f486c0ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4869400640 1 -- 172.21.15.110:0/3513582761 >> v1:172.21.15.110:6789/0 conn(0x7f486c06ec20 legacy=0x7f486c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4869400640 1 -- 172.21.15.110:0/3513582761 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c15ee20 con 0x7f486c0aed80 2024-09-10T10:03:05.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4869400640 1 --2- 172.21.15.110:0/3513582761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c0aed80 0x7f486c06e6e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f48580097c0 tx=0x7f485801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f216aee816131185 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4868a00640 1 -- 172.21.15.110:0/3513582761 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48580b0070 con 0x7f486c0aed80 2024-09-10T10:03:05.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4868a00640 1 -- 172.21.15.110:0/3513582761 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48580ab600 con 0x7f486c0aed80 2024-09-10T10:03:05.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4868a00640 1 -- 172.21.15.110:0/3513582761 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48580ab9b0 con 0x7f486c0aed80 2024-09-10T10:03:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4870a00640 1 -- 172.21.15.110:0/3513582761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c0aed80 msgr2=0x7f486c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4870a00640 1 --2- 172.21.15.110:0/3513582761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c0aed80 0x7f486c06e6e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f48580097c0 tx=0x7f485801c3c0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4870a00640 1 -- 172.21.15.110:0/3513582761 shutdown_connections 2024-09-10T10:03:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4870a00640 1 --2- 172.21.15.110:0/3513582761 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c0aed80 0x7f486c06e6e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4870a00640 1 --2- 172.21.15.110:0/3513582761 >> v2:172.21.15.145:3300/0 conn(0x7f486c147f00 0x7f486c0ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.651+0000 7f4870a00640 1 -- 172.21.15.110:0/3513582761 >> 172.21.15.110:0/3513582761 conn(0x7f486c06bc50 msgr2=0x7f486c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 -- 172.21.15.110:0/3513582761 shutdown_connections 2024-09-10T10:03:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 -- 172.21.15.110:0/3513582761 wait complete. 2024-09-10T10:03:05.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 Processor -- start 2024-09-10T10:03:05.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 -- start start 2024-09-10T10:03:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c06ec20 0x7f486c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f486c0aed80 0x7f486c143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f486c147f00 0x7f486c146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f486c073320 con 0x7f486c0aed80 2024-09-10T10:03:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06f090 con 0x7f486c147f00 2024-09-10T10:03:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f486c0734a0 con 0x7f486c06ec20 2024-09-10T10:03:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f486c0aed80 0x7f486c143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f486c0aed80 0x7f486c143dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47980/0 (socket says 172.21.15.110:47980) 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 -- 172.21.15.110:0/2350709201 learned_addr learned my addr 172.21.15.110:0/2350709201 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869e00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c06ec20 0x7f486c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f486a800640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f486c147f00 0x7f486c146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 -- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c06ec20 msgr2=0x7f486c15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c06ec20 0x7f486c15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 -- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f486c147f00 msgr2=0x7f486c146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f486c147f00 0x7f486c146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 -- 172.21.15.110:0/2350709201 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c13b6c0 con 0x7f486c0aed80 2024-09-10T10:03:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4869400640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f486c0aed80 0x7f486c143dd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4858000c10 tx=0x7f4858019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4862000640 1 -- 172.21.15.110:0/2350709201 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48580ba070 con 0x7f486c0aed80 2024-09-10T10:03:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4862000640 1 -- 172.21.15.110:0/2350709201 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48580ab9b0 con 0x7f486c0aed80 2024-09-10T10:03:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4862000640 1 -- 172.21.15.110:0/2350709201 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48580b0040 con 0x7f486c0aed80 2024-09-10T10:03:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.655+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c13b9a0 con 0x7f486c0aed80 2024-09-10T10:03:05.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.659+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c13cd10 con 0x7f486c0aed80 2024-09-10T10:03:05.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.659+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c13b440 con 0x7f486c0aed80 2024-09-10T10:03:05.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.659+0000 7f4862000640 1 -- 172.21.15.110:0/2350709201 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f48580ab4c0 con 0x7f486c0aed80 2024-09-10T10:03:05.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.659+0000 7f4862000640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f484c0679e0 0x7f484c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.659+0000 7f4862000640 1 -- 172.21.15.110:0/2350709201 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f485811eff0 con 0x7f486c0aed80 2024-09-10T10:03:05.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.659+0000 7f4869e00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f484c0679e0 0x7f484c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.663+0000 7f4869e00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f484c0679e0 0x7f484c069ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f485c009b60 tx=0x7f485c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.663+0000 7f4862000640 1 -- 172.21.15.110:0/2350709201 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f486c13b440 con 0x7f486c0aed80 2024-09-10T10:03:05.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.823+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f486c06e6e0 con 0x7f486c0aed80 2024-09-10T10:03:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.823+0000 7f4862000640 1 -- 172.21.15.110:0/2350709201 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16736 (secure 0 0 0) 0x7f4858015fb0 con 0x7f486c0aed80 2024-09-10T10:03:05.828 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:05.828 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":102,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:04.515844+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:02.221093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:03.519991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f484c0679e0 msgr2=0x7f484c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f484c0679e0 0x7f484c069ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f485c009b60 tx=0x7f485c009190 comp rx=0 tx=0).stop 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f486c0aed80 msgr2=0x7f486c143dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f486c0aed80 0x7f486c143dd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4858000c10 tx=0x7f4858019160 comp rx=0 tx=0).stop 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 shutdown_connections 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f484c0679e0 0x7f484c069ea0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f486c147f00 0x7f486c146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f486c0aed80 0x7f486c143dd0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 --2- 172.21.15.110:0/2350709201 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f486c06ec20 0x7f486c15c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 >> 172.21.15.110:0/2350709201 conn(0x7f486c06bc50 msgr2=0x7f486c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:05.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 shutdown_connections 2024-09-10T10:03:05.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.827+0000 7f4870a00640 1 -- 172.21.15.110:0/2350709201 wait complete. 2024-09-10T10:03:05.850 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:03:05.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.983+0000 7fe167800640 1 Processor -- start 2024-09-10T10:03:05.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.983+0000 7fe167800640 1 -- start start 2024-09-10T10:03:05.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe167800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe160145f30 0x7fe1600b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe167800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe1600cfd90 0x7fe1600b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe167800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1600b37c0 con 0x7fe1600b29e0 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe167800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe1600b2e50 con 0x7fe160145f30 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe167800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe1600d0170 con 0x7fe1600cfd90 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe164e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe160145f30 0x7fe1600b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe164e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe160145f30 0x7fe1600b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47996/0 (socket says 172.21.15.110:47996) 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe164e00640 1 -- 172.21.15.110:0/4174490763 learned_addr learned my addr 172.21.15.110:0/4174490763 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe165800640 1 --2- 172.21.15.110:0/4174490763 >> v2:172.21.15.145:3300/0 conn(0x7fe1600cfd90 0x7fe1600b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe164e00640 1 -- 172.21.15.110:0/4174490763 >> v2:172.21.15.145:3300/0 conn(0x7fe1600cfd90 msgr2=0x7fe1600b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe164e00640 1 --2- 172.21.15.110:0/4174490763 >> v2:172.21.15.145:3300/0 conn(0x7fe1600cfd90 0x7fe1600b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe164e00640 1 -- 172.21.15.110:0/4174490763 >> v1:172.21.15.145:6789/0 conn(0x7fe1600b29e0 legacy=0x7fe1600cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:05.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe164e00640 1 -- 172.21.15.110:0/4174490763 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1600b3aa0 con 0x7fe160145f30 2024-09-10T10:03:05.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.987+0000 7fe164e00640 1 --2- 172.21.15.110:0/4174490763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe160145f30 0x7fe1600b24a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe1600b3000 tx=0x7fe14801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45914d2176b3711 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe15f400640 1 -- 172.21.15.110:0/4174490763 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b0070 con 0x7fe160145f30 2024-09-10T10:03:05.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe15f400640 1 -- 172.21.15.110:0/4174490763 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1480ab600 con 0x7fe160145f30 2024-09-10T10:03:05.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe15f400640 1 -- 172.21.15.110:0/4174490763 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1480ab9d0 con 0x7fe160145f30 2024-09-10T10:03:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- 172.21.15.110:0/4174490763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe160145f30 msgr2=0x7fe1600b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 --2- 172.21.15.110:0/4174490763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe160145f30 0x7fe1600b24a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe1600b3000 tx=0x7fe14801c4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- 172.21.15.110:0/4174490763 shutdown_connections 2024-09-10T10:03:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 --2- 172.21.15.110:0/4174490763 >> v2:172.21.15.145:3300/0 conn(0x7fe1600cfd90 0x7fe1600b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 --2- 172.21.15.110:0/4174490763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe160145f30 0x7fe1600b24a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- 172.21.15.110:0/4174490763 >> 172.21.15.110:0/4174490763 conn(0x7fe160062e40 msgr2=0x7fe16006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- 172.21.15.110:0/4174490763 shutdown_connections 2024-09-10T10:03:05.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- 172.21.15.110:0/4174490763 wait complete. 2024-09-10T10:03:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 Processor -- start 2024-09-10T10:03:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- start start 2024-09-10T10:03:05.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1600b29e0 0x7fe16013e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1600cfd90 0x7fe160137470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe160145f30 0x7fe1601f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1600b5660 con 0x7fe1600cfd90 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1600b54e0 con 0x7fe1600b29e0 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe167800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe1600b57e0 con 0x7fe160145f30 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe15fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1600cfd90 0x7fe160137470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe15fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1600cfd90 0x7fe160137470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48004/0 (socket says 172.21.15.110:48004) 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe15fe00640 1 -- 172.21.15.110:0/1953224921 learned_addr learned my addr 172.21.15.110:0/1953224921 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.991+0000 7fe164e00640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1600b29e0 0x7fe16013e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe165800640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe160145f30 0x7fe1601f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15fe00640 1 -- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe160145f30 msgr2=0x7fe1601f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15fe00640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe160145f30 0x7fe1601f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15fe00640 1 -- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1600b29e0 msgr2=0x7fe16013e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15fe00640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1600b29e0 0x7fe16013e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15fe00640 1 -- 172.21.15.110:0/1953224921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16013ee40 con 0x7fe1600cfd90 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe165800640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe160145f30 0x7fe1601f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15fe00640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1600cfd90 0x7fe160137470 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b410 tx=0x7fe15000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15cc00640 1 -- 172.21.15.110:0/1953224921 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150015030 con 0x7fe1600cfd90 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15cc00640 1 -- 172.21.15.110:0/1953224921 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe150004020 con 0x7fe1600cfd90 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1601f8190 con 0x7fe1600cfd90 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15cc00640 1 -- 172.21.15.110:0/1953224921 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150004410 con 0x7fe1600cfd90 2024-09-10T10:03:05.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16013e3e0 con 0x7fe1600cfd90 2024-09-10T10:03:06.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16013e5e0 con 0x7fe1600cfd90 2024-09-10T10:03:06.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15cc00640 1 -- 172.21.15.110:0/1953224921 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fe150018070 con 0x7fe1600cfd90 2024-09-10T10:03:06.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15cc00640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe13c0679e0 0x7fe13c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:06.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.995+0000 7fe15cc00640 1 -- 172.21.15.110:0/1953224921 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fe1500858c0 con 0x7fe1600cfd90 2024-09-10T10:03:06.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.999+0000 7fe164e00640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe13c0679e0 0x7fe13c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:06.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.999+0000 7fe164e00640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe13c0679e0 0x7fe13c069ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe1480ad000 tx=0x7fe148008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:06.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:05.999+0000 7fe15cc00640 1 -- 172.21.15.110:0/1953224921 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe16013e5e0 con 0x7fe1600cfd90 2024-09-10T10:03:06.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:06.191+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe1600b4560 con 0x7fe1600cfd90 2024-09-10T10:03:07.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.155+0000 7fe15cc00640 1 -- 172.21.15.110:0/1953224921 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) ==== 118+0+0 (secure 0 0 0) 0x7fe1601f8190 con 0x7fe1600cfd90 2024-09-10T10:03:07.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe13c0679e0 msgr2=0x7fe13c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe13c0679e0 0x7fe13c069ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe1480ad000 tx=0x7fe148008040 comp rx=0 tx=0).stop 2024-09-10T10:03:07.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1600cfd90 msgr2=0x7fe160137470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1600cfd90 0x7fe160137470 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b410 tx=0x7fe15000b8d0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 shutdown_connections 2024-09-10T10:03:07.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe13c0679e0 0x7fe13c069ea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe160145f30 0x7fe1601f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1600cfd90 0x7fe160137470 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 --2- 172.21.15.110:0/1953224921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1600b29e0 0x7fe16013e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 >> 172.21.15.110:0/1953224921 conn(0x7fe160062e40 msgr2=0x7fe1600723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:07.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 shutdown_connections 2024-09-10T10:03:07.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.159+0000 7fe167800640 1 -- 172.21.15.110:0/1953224921 wait complete. 2024-09-10T10:03:07.186 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:03:07.187 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:03:07.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.323+0000 7f05ffc00640 1 Processor -- start 2024-09-10T10:03:07.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05ffc00640 1 -- start start 2024-09-10T10:03:07.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05ffc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80b2bb0 0x7f05f80cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05ffc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f05f80cfbc0 0x7f05f80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05ffc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05f80b37c0 con 0x7f05f8145f30 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05ffc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f05f80b3aa0 con 0x7f05f80cfbc0 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05ffc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f05f80b3d80 con 0x7f05f80b2bb0 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05fdc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f05f80cfbc0 0x7f05f80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05fdc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f05f80cfbc0 0x7f05f80b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48010/0 (socket says 172.21.15.110:48010) 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05fdc00640 1 -- 172.21.15.110:0/770441862 learned_addr learned my addr 172.21.15.110:0/770441862 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05f7e00640 1 --2- 172.21.15.110:0/770441862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80b2bb0 0x7f05f80cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05fdc00640 1 -- 172.21.15.110:0/770441862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80b2bb0 msgr2=0x7f05f80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05fdc00640 1 --2- 172.21.15.110:0/770441862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80b2bb0 0x7f05f80cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05fdc00640 1 -- 172.21.15.110:0/770441862 >> v1:172.21.15.145:6789/0 conn(0x7f05f8145f30 legacy=0x7f05f80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:07.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.327+0000 7f05fdc00640 1 -- 172.21.15.110:0/770441862 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f8146310 con 0x7f05f80cfbc0 2024-09-10T10:03:07.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05fdc00640 1 --2- 172.21.15.110:0/770441862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80cfbc0 0x7f05f80b4580 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009870 tx=0x7f05f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee14ff0990b8b1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05f7400640 1 -- 172.21.15.110:0/770441862 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f00b0070 con 0x7f05f80cfbc0 2024-09-10T10:03:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05f7400640 1 -- 172.21.15.110:0/770441862 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f00ab600 con 0x7f05f80cfbc0 2024-09-10T10:03:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05f7400640 1 -- 172.21.15.110:0/770441862 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f00ab9d0 con 0x7f05f80cfbc0 2024-09-10T10:03:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 -- 172.21.15.110:0/770441862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80cfbc0 msgr2=0x7f05f80b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 --2- 172.21.15.110:0/770441862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80cfbc0 0x7f05f80b4580 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009870 tx=0x7f05f001c520 comp rx=0 tx=0).stop 2024-09-10T10:03:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 -- 172.21.15.110:0/770441862 shutdown_connections 2024-09-10T10:03:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 --2- 172.21.15.110:0/770441862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80cfbc0 0x7f05f80b4580 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 --2- 172.21.15.110:0/770441862 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80b2bb0 0x7f05f80cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 -- 172.21.15.110:0/770441862 >> 172.21.15.110:0/770441862 conn(0x7f05f8062e40 msgr2=0x7f05f806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 -- 172.21.15.110:0/770441862 shutdown_connections 2024-09-10T10:03:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 -- 172.21.15.110:0/770441862 wait complete. 2024-09-10T10:03:07.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 Processor -- start 2024-09-10T10:03:07.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.331+0000 7f05ffc00640 1 -- start start 2024-09-10T10:03:07.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80b2bb0 0x7f05f81411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80cfbc0 0x7f05f815b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05f8145f30 0x7f05f80ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05f80b56a0 con 0x7f05f80b2bb0 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05f80b5520 con 0x7f05f8145f30 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f05f80b5820 con 0x7f05f80cfbc0 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80b2bb0 0x7f05f81411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80b2bb0 0x7f05f81411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48024/0 (socket says 172.21.15.110:48024) 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 -- 172.21.15.110:0/1449673989 learned_addr learned my addr 172.21.15.110:0/1449673989 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05f7e00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80cfbc0 0x7f05f815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fdc00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05f8145f30 0x7f05f80ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 -- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80cfbc0 msgr2=0x7f05f815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80cfbc0 0x7f05f815b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 -- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05f8145f30 msgr2=0x7f05f80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05f8145f30 0x7f05f80ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 -- 172.21.15.110:0/1449673989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f813d420 con 0x7f05f80b2bb0 2024-09-10T10:03:07.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05f7e00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80cfbc0 0x7f05f815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:07.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fdc00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05f8145f30 0x7f05f80ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:07.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05fd200640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80b2bb0 0x7f05f81411e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f05e800ba50 tx=0x7f05e800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:07.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05f4c00640 1 -- 172.21.15.110:0/1449673989 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8019070 con 0x7f05f80b2bb0 2024-09-10T10:03:07.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05f4c00640 1 -- 172.21.15.110:0/1449673989 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e800cc20 con 0x7f05f80b2bb0 2024-09-10T10:03:07.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f813d100 con 0x7f05f80b2bb0 2024-09-10T10:03:07.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05f4c00640 1 -- 172.21.15.110:0/1449673989 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8014440 con 0x7f05f80b2bb0 2024-09-10T10:03:07.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f813da50 con 0x7f05f80b2bb0 2024-09-10T10:03:07.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f80cf680 con 0x7f05f80b2bb0 2024-09-10T10:03:07.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.335+0000 7f05f4c00640 1 -- 172.21.15.110:0/1449673989 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f05e801c070 con 0x7f05f80b2bb0 2024-09-10T10:03:07.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.339+0000 7f05f4c00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05d80679e0 0x7f05d8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.339+0000 7f05f4c00640 1 -- 172.21.15.110:0/1449673989 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f05e8085a40 con 0x7f05f80b2bb0 2024-09-10T10:03:07.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.339+0000 7f05f7e00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05d80679e0 0x7f05d8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.339+0000 7f05f7e00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05d80679e0 0x7f05d8069ea0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05f80b2470 tx=0x7f05e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:07.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.343+0000 7f05f4c00640 1 -- 172.21.15.110:0/1449673989 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f05e8055e10 con 0x7f05f80b2bb0 2024-09-10T10:03:07.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:07.531+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f05f8071c10 con 0x7f05f80b2bb0 2024-09-10T10:03:08.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.163+0000 7f05f4c00640 1 -- 172.21.15.110:0/1449673989 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) ==== 122+0+0 (secure 0 0 0) 0x7f05e805a110 con 0x7f05f80b2bb0 2024-09-10T10:03:08.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.163+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05d80679e0 msgr2=0x7f05d8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05d80679e0 0x7f05d8069ea0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05f80b2470 tx=0x7f05e4008040 comp rx=0 tx=0).stop 2024-09-10T10:03:08.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80b2bb0 msgr2=0x7f05f81411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80b2bb0 0x7f05f81411e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f05e800ba50 tx=0x7f05e800bf10 comp rx=0 tx=0).stop 2024-09-10T10:03:08.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 shutdown_connections 2024-09-10T10:03:08.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05d80679e0 0x7f05d8069ea0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05f8145f30 0x7f05f80ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f05f80cfbc0 0x7f05f815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 --2- 172.21.15.110:0/1449673989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05f80b2bb0 0x7f05f81411e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 >> 172.21.15.110:0/1449673989 conn(0x7f05f8062e40 msgr2=0x7f05f8071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 shutdown_connections 2024-09-10T10:03:08.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.167+0000 7f05ffc00640 1 -- 172.21.15.110:0/1449673989 wait complete. 2024-09-10T10:03:08.192 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:03:08.192 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:03:08.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.327+0000 7ff97e800640 1 Processor -- start 2024-09-10T10:03:08.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.327+0000 7ff97e800640 1 -- start start 2024-09-10T10:03:08.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff9780b6710 0x7ff97806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97e800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97806d060 con 0x7ff9780cf1b0 2024-09-10T10:03:08.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97e800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97806d340 con 0x7ff97815e620 2024-09-10T10:03:08.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97e800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff9780d1e90 con 0x7ff9780b6710 2024-09-10T10:03:08.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff9780b6710 0x7ff97806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff9780b6710 0x7ff97806e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54652/0 (socket says 172.21.15.110:54652) 2024-09-10T10:03:08.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97de00640 1 -- 172.21.15.110:0/814580685 learned_addr learned my addr 172.21.15.110:0/814580685 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:08.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97de00640 1 -- 172.21.15.110:0/814580685 >> v1:172.21.15.145:6789/0 conn(0x7ff97815e620 legacy=0x7ff9780b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97de00640 1 -- 172.21.15.110:0/814580685 >> v1:172.21.15.110:6789/0 conn(0x7ff9780cf1b0 legacy=0x7ff9780b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.331+0000 7ff97de00640 1 -- 172.21.15.110:0/814580685 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97806d620 con 0x7ff9780b6710 2024-09-10T10:03:08.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97de00640 1 --2- 172.21.15.110:0/814580685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9780b6710 0x7ff97806e100 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff970009e50 tx=0x7ff97001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca0ad29bf126169 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff977e00640 1 -- 172.21.15.110:0/814580685 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700a5620 con 0x7ff9780b6710 2024-09-10T10:03:08.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff977e00640 1 -- 172.21.15.110:0/814580685 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9700a57c0 con 0x7ff9780b6710 2024-09-10T10:03:08.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff977e00640 1 -- 172.21.15.110:0/814580685 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700a5b90 con 0x7ff9780b6710 2024-09-10T10:03:08.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- 172.21.15.110:0/814580685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9780b6710 msgr2=0x7ff97806e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 --2- 172.21.15.110:0/814580685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9780b6710 0x7ff97806e100 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff970009e50 tx=0x7ff97001c960 comp rx=0 tx=0).stop 2024-09-10T10:03:08.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- 172.21.15.110:0/814580685 shutdown_connections 2024-09-10T10:03:08.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 --2- 172.21.15.110:0/814580685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9780b6710 0x7ff97806e100 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- 172.21.15.110:0/814580685 >> 172.21.15.110:0/814580685 conn(0x7ff97806f440 msgr2=0x7ff97806f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- 172.21.15.110:0/814580685 shutdown_connections 2024-09-10T10:03:08.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- 172.21.15.110:0/814580685 wait complete. 2024-09-10T10:03:08.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 Processor -- start 2024-09-10T10:03:08.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- start start 2024-09-10T10:03:08.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9780b6710 0x7ff9781535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9780cf1b0 0x7ff978154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff97815e620 0x7ff9780709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9780cf5c0 con 0x7ff97815e620 2024-09-10T10:03:08.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff97806ce10 con 0x7ff9780b6710 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97e800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff97816c300 con 0x7ff9780cf1b0 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.335+0000 7ff97de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff97815e620 0x7ff9780709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff97815e620 0x7ff9780709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54660/0 (socket says 172.21.15.110:54660) 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97de00640 1 -- 172.21.15.110:0/258571389 learned_addr learned my addr 172.21.15.110:0/258571389 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97d400640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9780b6710 0x7ff9781535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97ca00640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9780cf1b0 0x7ff978154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97de00640 1 -- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9780cf1b0 msgr2=0x7ff978154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97de00640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9780cf1b0 0x7ff978154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97de00640 1 -- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9780b6710 msgr2=0x7ff9781535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97de00640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9780b6710 0x7ff9781535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97de00640 1 -- 172.21.15.110:0/258571389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff978155460 con 0x7ff97815e620 2024-09-10T10:03:08.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97de00640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff97815e620 0x7ff9780709c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff9700af040 tx=0x7ff970019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff975600640 1 -- 172.21.15.110:0/258571389 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700bc020 con 0x7ff97815e620 2024-09-10T10:03:08.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff975600640 1 -- 172.21.15.110:0/258571389 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9700b8670 con 0x7ff97815e620 2024-09-10T10:03:08.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff975600640 1 -- 172.21.15.110:0/258571389 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700b8a60 con 0x7ff97815e620 2024-09-10T10:03:08.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff978155720 con 0x7ff97815e620 2024-09-10T10:03:08.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97814caa0 con 0x7ff97815e620 2024-09-10T10:03:08.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97806e100 con 0x7ff97815e620 2024-09-10T10:03:08.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff975600640 1 -- 172.21.15.110:0/258571389 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7ff9700bf020 con 0x7ff97815e620 2024-09-10T10:03:08.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff975600640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff9580679e0 0x7ff958069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.339+0000 7ff975600640 1 -- 172.21.15.110:0/258571389 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff970120e10 con 0x7ff97815e620 2024-09-10T10:03:08.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.343+0000 7ff97d400640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff9580679e0 0x7ff958069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.343+0000 7ff97d400640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff9580679e0 0x7ff958069ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff978071430 tx=0x7ff96c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.347+0000 7ff975600640 1 -- 172.21.15.110:0/258571389 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff9700c6030 con 0x7ff97815e620 2024-09-10T10:03:08.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.535+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff97814edc0 con 0x7ff97815e620 2024-09-10T10:03:08.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.535+0000 7ff975600640 1 -- 172.21.15.110:0/258571389 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5228 (secure 0 0 0) 0x7ff9700165e0 con 0x7ff97815e620 2024-09-10T10:03:08.540 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:08.540 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":68,"btime":"2024-09-10T10:03:08:162006+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/59288243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":59288243},{"type":"v1","addr":"172.21.15.145:6834","nonce":59288243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2534764627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2534764627},{"type":"v1","addr":"172.21.15.145:6835","nonce":2534764627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6798,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3565143399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":3565143399},{"type":"v1","addr":"172.21.15.110:6834","nonce":3565143399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:04.516174+0000","modified":"2024-09-10T10:03:07.538032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/100964295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":100964295},{"type":"v1","addr":"172.21.15.110:6835","nonce":100964295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":8}]} 2024-09-10T10:03:08.541 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 68 2024-09-10T10:03:08.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff9580679e0 msgr2=0x7ff958069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff9580679e0 0x7ff958069ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff978071430 tx=0x7ff96c008040 comp rx=0 tx=0).stop 2024-09-10T10:03:08.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff97815e620 msgr2=0x7ff9780709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff97815e620 0x7ff9780709c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff9700af040 tx=0x7ff970019700 comp rx=0 tx=0).stop 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 shutdown_connections 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff9580679e0 0x7ff958069ea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff97815e620 0x7ff9780709c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9780cf1b0 0x7ff978154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 --2- 172.21.15.110:0/258571389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9780b6710 0x7ff9781535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 >> 172.21.15.110:0/258571389 conn(0x7ff97806f440 msgr2=0x7ff9780b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 shutdown_connections 2024-09-10T10:03:08.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.539+0000 7ff97e800640 1 -- 172.21.15.110:0/258571389 wait complete. 2024-09-10T10:03:08.562 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:03:08.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58dbe00640 1 Processor -- start 2024-09-10T10:03:08.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58dbe00640 1 -- start start 2024-09-10T10:03:08.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58dbe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f58d415d4b0 0x7f58d40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58dbe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f58d40d1660 0x7f58d40d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58dbe00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d40d1fb0 con 0x7f58d415c3d0 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58dbe00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f58d415d920 con 0x7f58d40d1660 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58dbe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f58d415c7b0 con 0x7f58d415d4b0 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f58d40d1660 0x7f58d40d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f58d40d1660 0x7f58d40d1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54674/0 (socket says 172.21.15.110:54674) 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d9e00640 1 -- 172.21.15.110:0/1350698232 learned_addr learned my addr 172.21.15.110:0/1350698232 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d9e00640 1 -- 172.21.15.110:0/1350698232 >> v2:172.21.15.145:3300/0 conn(0x7f58d415d4b0 msgr2=0x7f58d40d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d8a00640 1 --2- 172.21.15.110:0/1350698232 >> v2:172.21.15.145:3300/0 conn(0x7f58d415d4b0 0x7f58d40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d9e00640 1 --2- 172.21.15.110:0/1350698232 >> v2:172.21.15.145:3300/0 conn(0x7f58d415d4b0 0x7f58d40d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d9e00640 1 -- 172.21.15.110:0/1350698232 >> v1:172.21.15.145:6789/0 conn(0x7f58d415c3d0 legacy=0x7f58d40cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d9e00640 1 -- 172.21.15.110:0/1350698232 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d40cfb30 con 0x7f58d40d1660 2024-09-10T10:03:08.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.695+0000 7f58d8a00640 1 --2- 172.21.15.110:0/1350698232 >> v2:172.21.15.145:3300/0 conn(0x7f58d415d4b0 0x7f58d40d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:08.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58d9e00640 1 --2- 172.21.15.110:0/1350698232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d40d1660 0x7f58d40d1a70 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f58d40b0590 tx=0x7f58cc01c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4964b1643de2c4c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58d3e00640 1 -- 172.21.15.110:0/1350698232 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0a5660 con 0x7f58d40d1660 2024-09-10T10:03:08.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58d3e00640 1 -- 172.21.15.110:0/1350698232 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc0a5800 con 0x7f58d40d1660 2024-09-10T10:03:08.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58d3e00640 1 -- 172.21.15.110:0/1350698232 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0a5bd0 con 0x7f58d40d1660 2024-09-10T10:03:08.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58dbe00640 1 -- 172.21.15.110:0/1350698232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d40d1660 msgr2=0x7f58d40d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58dbe00640 1 --2- 172.21.15.110:0/1350698232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d40d1660 0x7f58d40d1a70 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f58d40b0590 tx=0x7f58cc01c7a0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58dbe00640 1 -- 172.21.15.110:0/1350698232 shutdown_connections 2024-09-10T10:03:08.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58dbe00640 1 --2- 172.21.15.110:0/1350698232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d40d1660 0x7f58d40d1a70 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58dbe00640 1 --2- 172.21.15.110:0/1350698232 >> v2:172.21.15.145:3300/0 conn(0x7f58d415d4b0 0x7f58d40d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58dbe00640 1 -- 172.21.15.110:0/1350698232 >> 172.21.15.110:0/1350698232 conn(0x7f58d406cf60 msgr2=0x7f58d406d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58dbe00640 1 -- 172.21.15.110:0/1350698232 shutdown_connections 2024-09-10T10:03:08.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.699+0000 7f58dbe00640 1 -- 172.21.15.110:0/1350698232 wait complete. 2024-09-10T10:03:08.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 Processor -- start 2024-09-10T10:03:08.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 -- start start 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58d40d1660 0x7f58d414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d415c3d0 0x7f58d40afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58d415d4b0 0x7f58d41569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58d415e300 con 0x7f58d415c3d0 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40d06b0 con 0x7f58d40d1660 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f58d415e480 con 0x7f58d415d4b0 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d415c3d0 0x7f58d40afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d415c3d0 0x7f58d40afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54686/0 (socket says 172.21.15.110:54686) 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 -- 172.21.15.110:0/574260963 learned_addr learned my addr 172.21.15.110:0/574260963 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d9400640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58d40d1660 0x7f58d414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d9e00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58d415d4b0 0x7f58d41569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 -- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58d415d4b0 msgr2=0x7f58d41569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58d415d4b0 0x7f58d41569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 -- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58d40d1660 msgr2=0x7f58d414ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58d40d1660 0x7f58d414ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 -- 172.21.15.110:0/574260963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d414c500 con 0x7f58d415c3d0 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d8a00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d415c3d0 0x7f58d40afca0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f58c000c7f0 tx=0x7f58c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d1600640 1 -- 172.21.15.110:0/574260963 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0015070 con 0x7f58d415c3d0 2024-09-10T10:03:08.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d1600640 1 -- 172.21.15.110:0/574260963 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c0007820 con 0x7f58d415c3d0 2024-09-10T10:03:08.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58d1600640 1 -- 172.21.15.110:0/574260963 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0007c10 con 0x7f58d415c3d0 2024-09-10T10:03:08.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d40aec00 con 0x7f58d415c3d0 2024-09-10T10:03:08.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d40af1c0 con 0x7f58d415c3d0 2024-09-10T10:03:08.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.703+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d40d1a70 con 0x7f58d415c3d0 2024-09-10T10:03:08.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.707+0000 7f58d1600640 1 -- 172.21.15.110:0/574260963 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f58c0018070 con 0x7f58d415c3d0 2024-09-10T10:03:08.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.707+0000 7f58d1600640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f58b40679e0 0x7f58b4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.707+0000 7f58d1600640 1 -- 172.21.15.110:0/574260963 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f58c0085ff0 con 0x7f58d415c3d0 2024-09-10T10:03:08.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.707+0000 7f58d9400640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f58b40679e0 0x7f58b4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.707+0000 7f58d9400640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f58b40679e0 0x7f58b4069ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58d4157540 tx=0x7f58c8007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.711+0000 7f58d1600640 1 -- 172.21.15.110:0/574260963 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f58c0056280 con 0x7f58d415c3d0 2024-09-10T10:03:08.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.867+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58d4070c00 con 0x7f58d415c3d0 2024-09-10T10:03:08.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58d1600640 1 -- 172.21.15.110:0/574260963 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16723 (secure 0 0 0) 0x7f58c005a580 con 0x7f58d415c3d0 2024-09-10T10:03:08.874 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:08.874 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":103,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:06.145077+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:02.221093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:03.519991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:08.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f58b40679e0 msgr2=0x7f58b4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f58b40679e0 0x7f58b4069ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58d4157540 tx=0x7f58c8007e40 comp rx=0 tx=0).stop 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d415c3d0 msgr2=0x7f58d40afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d415c3d0 0x7f58d40afca0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f58c000c7f0 tx=0x7f58c000ccb0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 shutdown_connections 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f58b40679e0 0x7f58b4069ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f58d415d4b0 0x7f58d41569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58d415c3d0 0x7f58d40afca0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 --2- 172.21.15.110:0/574260963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58d40d1660 0x7f58d414ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 >> 172.21.15.110:0/574260963 conn(0x7f58d406cf60 msgr2=0x7f58d40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 shutdown_connections 2024-09-10T10:03:08.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:08.871+0000 7f58dbe00640 1 -- 172.21.15.110:0/574260963 wait complete. 2024-09-10T10:03:08.895 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:03:08.896 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:03:09.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.027+0000 7fb5e6000640 1 Processor -- start 2024-09-10T10:03:09.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.027+0000 7fb5e6000640 1 -- start start 2024-09-10T10:03:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5e6000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb5e006e7a0 0x7fb5e00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5e6000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e0072150 con 0x7fb5e00cfe30 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5e6000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e006d2d0 con 0x7fb5e00cd510 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5e6000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e015d9a0 con 0x7fb5e006e7a0 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5df400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb5e00cfe30 0x7fb5e0071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52038/0 (socket says 172.21.15.110:52038) 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5df400640 1 -- 172.21.15.110:0/3819358259 learned_addr learned my addr 172.21.15.110:0/3819358259 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5dfe00640 1 --2- 172.21.15.110:0/3819358259 >> v2:172.21.15.145:3300/0 conn(0x7fb5e006e7a0 0x7fb5e00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5de000640 1 -- 172.21.15.110:0/3819358259 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3482782229 0 0) 0x7fb5e0072150 con 0x7fb5e00cfe30 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5de000640 1 -- 172.21.15.110:0/3819358259 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5c4003530 con 0x7fb5e00cfe30 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5dfe00640 1 -- 172.21.15.110:0/3819358259 >> v1:172.21.15.145:6789/0 conn(0x7fb5e00cd510 legacy=0x7fb5e006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5de000640 1 -- 172.21.15.110:0/3819358259 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2069934348 0 0) 0x7fb5c4003530 con 0x7fb5e00cfe30 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5dfe00640 1 -- 172.21.15.110:0/3819358259 >> v1:172.21.15.110:6789/0 conn(0x7fb5e00cfe30 legacy=0x7fb5e0071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.031+0000 7fb5dfe00640 1 -- 172.21.15.110:0/3819358259 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e006d5b0 con 0x7fb5e006e7a0 2024-09-10T10:03:09.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5dfe00640 1 --2- 172.21.15.110:0/3819358259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e006e7a0 0x7fb5e00cd9a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80097c0 tx=0x7fb5d801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54ff6f14c7af38db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5de000640 1 -- 172.21.15.110:0/3819358259 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80b0070 con 0x7fb5e006e7a0 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5de000640 1 -- 172.21.15.110:0/3819358259 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d80a4e50 con 0x7fb5e006e7a0 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5de000640 1 -- 172.21.15.110:0/3819358259 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0001a00 con 0x7fb5e006e7a0 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 -- 172.21.15.110:0/3819358259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e006e7a0 msgr2=0x7fb5e00cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 --2- 172.21.15.110:0/3819358259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e006e7a0 0x7fb5e00cd9a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80097c0 tx=0x7fb5d801c330 comp rx=0 tx=0).stop 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 -- 172.21.15.110:0/3819358259 shutdown_connections 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 --2- 172.21.15.110:0/3819358259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e006e7a0 0x7fb5e00cd9a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 -- 172.21.15.110:0/3819358259 >> 172.21.15.110:0/3819358259 conn(0x7fb5e006bc50 msgr2=0x7fb5e006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 -- 172.21.15.110:0/3819358259 shutdown_connections 2024-09-10T10:03:09.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 -- 172.21.15.110:0/3819358259 wait complete. 2024-09-10T10:03:09.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 Processor -- start 2024-09-10T10:03:09.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.035+0000 7fb5e6000640 1 -- start start 2024-09-10T10:03:09.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5e006e7a0 0x7fb5e01fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e00cd510 0x7fb5e01face0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e00cfe30 0x7fb5e014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e016a190 con 0x7fb5e00cfe30 2024-09-10T10:03:09.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e015d700 con 0x7fb5e00cd510 2024-09-10T10:03:09.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e016a310 con 0x7fb5e006e7a0 2024-09-10T10:03:09.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e00cfe30 0x7fb5e014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e00cfe30 0x7fb5e014c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54688/0 (socket says 172.21.15.110:54688) 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 -- 172.21.15.110:0/1675619249 learned_addr learned my addr 172.21.15.110:0/1675619249 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dea00640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e00cd510 0x7fb5e01face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5df400640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5e006e7a0 0x7fb5e01fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 -- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5e006e7a0 msgr2=0x7fb5e01fa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5e006e7a0 0x7fb5e01fa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 -- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e00cd510 msgr2=0x7fb5e01face0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e00cd510 0x7fb5e01face0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 -- 172.21.15.110:0/1675619249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e014a860 con 0x7fb5e00cfe30 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5dfe00640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e00cfe30 0x7fb5e014c1e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80ae040 tx=0x7fb5d80190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5d7400640 1 -- 172.21.15.110:0/1675619249 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80b9020 con 0x7fb5e00cfe30 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5d7400640 1 -- 172.21.15.110:0/1675619249 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d8003a30 con 0x7fb5e00cfe30 2024-09-10T10:03:09.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5d7400640 1 -- 172.21.15.110:0/1675619249 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80b0040 con 0x7fb5e00cfe30 2024-09-10T10:03:09.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e0151750 con 0x7fb5e00cfe30 2024-09-10T10:03:09.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e01fb220 con 0x7fb5e00cfe30 2024-09-10T10:03:09.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e006e090 con 0x7fb5e00cfe30 2024-09-10T10:03:09.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.039+0000 7fb5d7400640 1 -- 172.21.15.110:0/1675619249 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fb5d80bc020 con 0x7fb5e00cfe30 2024-09-10T10:03:09.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.043+0000 7fb5d7400640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb5c00679e0 0x7fb5c0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.043+0000 7fb5d7400640 1 -- 172.21.15.110:0/1675619249 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fb5d811fd90 con 0x7fb5e00cfe30 2024-09-10T10:03:09.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.043+0000 7fb5df400640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb5c00679e0 0x7fb5c0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.043+0000 7fb5df400640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb5c00679e0 0x7fb5c0069ea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0005c10 tx=0x7fb5d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.047+0000 7fb5d7400640 1 -- 172.21.15.110:0/1675619249 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb5d80c3030 con 0x7fb5e00cfe30 2024-09-10T10:03:09.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.199+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5e0151520 con 0x7fb5c00679e0 2024-09-10T10:03:09.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.207+0000 7fb5d7400640 1 -- 172.21.15.110:0/1675619249 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252453 (secure 0 0 0) 0x7fb5e0151520 con 0x7fb5c00679e0 2024-09-10T10:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:09.216 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:03:09.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb5c00679e0 msgr2=0x7fb5c0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb5c00679e0 0x7fb5c0069ea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0005c10 tx=0x7fb5d0009040 comp rx=0 tx=0).stop 2024-09-10T10:03:09.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e00cfe30 msgr2=0x7fb5e014c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e00cfe30 0x7fb5e014c1e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80ae040 tx=0x7fb5d80190d0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 shutdown_connections 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb5c00679e0 0x7fb5c0069ea0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5e00cfe30 0x7fb5e014c1e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5e00cd510 0x7fb5e01face0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 --2- 172.21.15.110:0/1675619249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5e006e7a0 0x7fb5e01fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 >> 172.21.15.110:0/1675619249 conn(0x7fb5e006bc50 msgr2=0x7fb5e006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 shutdown_connections 2024-09-10T10:03:09.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.211+0000 7fb5e6000640 1 -- 172.21.15.110:0/1675619249 wait complete. 2024-09-10T10:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-10T10:03:07.701889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264800,"kb_used_data":11104,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709920,"statfs":{"total":773094113280,"available":772822958080,"internally_reserved":0,"allocated":11370496,"data_stored":5598452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":259575340},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.580114"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157015+0000","last_change":"2024-09-10T10:03:05.153139+0000","last_active":"2024-09-10T10:03:06.157015+0000","last_peered":"2024-09-10T10:03:06.157015+0000","last_clean":"2024-09-10T10:03:06.157015+0000","last_became_active":"2024-09-10T10:03:05.152819+0000","last_became_peered":"2024-09-10T10:03:05.152819+0000","last_unstale":"2024-09-10T10:03:06.157015+0000","last_undegraded":"2024-09-10T10:03:06.157015+0000","last_fullsized":"2024-09-10T10:03:06.157015+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:03.568760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158495+0000","last_change":"2024-09-10T10:03:05.153077+0000","last_active":"2024-09-10T10:03:06.158495+0000","last_peered":"2024-09-10T10:03:06.158495+0000","last_clean":"2024-09-10T10:03:06.158495+0000","last_became_active":"2024-09-10T10:03:05.152814+0000","last_became_peered":"2024-09-10T10:03:05.152814+0000","last_unstale":"2024-09-10T10:03:06.158495+0000","last_undegraded":"2024-09-10T10:03:06.158495+0000","last_fullsized":"2024-09-10T10:03:06.158495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:55.040526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.160932+0000","last_change":"2024-09-10T10:03:05.153608+0000","last_active":"2024-09-10T10:03:05.160932+0000","last_peered":"2024-09-10T10:03:05.160932+0000","last_clean":"2024-09-10T10:03:05.160932+0000","last_became_active":"2024-09-10T10:03:05.152587+0000","last_became_peered":"2024-09-10T10:03:05.152587+0000","last_unstale":"2024-09-10T10:03:05.160932+0000","last_undegraded":"2024-09-10T10:03:05.160932+0000","last_fullsized":"2024-09-10T10:03:05.160932+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:59.828450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157261+0000","last_change":"2024-09-10T10:03:05.164865+0000","last_active":"2024-09-10T10:03:06.157261+0000","last_peered":"2024-09-10T10:03:06.157261+0000","last_clean":"2024-09-10T10:03:06.157261+0000","last_became_active":"2024-09-10T10:03:05.164299+0000","last_became_peered":"2024-09-10T10:03:05.164299+0000","last_unstale":"2024-09-10T10:03:06.157261+0000","last_undegraded":"2024-09-10T10:03:06.157261+0000","last_fullsized":"2024-09-10T10:03:06.157261+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:25.203158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.168132+0000","last_change":"2024-09-10T10:03:05.161405+0000","last_active":"2024-09-10T10:03:05.168132+0000","last_peered":"2024-09-10T10:03:05.168132+0000","last_clean":"2024-09-10T10:03:05.168132+0000","last_became_active":"2024-09-10T10:03:05.158623+0000","last_became_peered":"2024-09-10T10:03:05.158623+0000","last_unstale":"2024-09-10T10:03:05.168132+0000","last_undegraded":"2024-09-10T10:03:05.168132+0000","last_fullsized":"2024-09-10T10:03:05.168132+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:37.615059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.157725+0000","last_change":"2024-09-10T10:03:05.155356+0000","last_active":"2024-09-10T10:03:05.157725+0000","last_peered":"2024-09-10T10:03:05.157725+0000","last_clean":"2024-09-10T10:03:05.157725+0000","last_became_active":"2024-09-10T10:03:05.155084+0000","last_became_peered":"2024-09-10T10:03:05.155084+0000","last_unstale":"2024-09-10T10:03:05.157725+0000","last_undegraded":"2024-09-10T10:03:05.157725+0000","last_fullsized":"2024-09-10T10:03:05.157725+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:05.551493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.158838+0000","last_change":"2024-09-10T10:03:05.152778+0000","last_active":"2024-09-10T10:03:05.158838+0000","last_peered":"2024-09-10T10:03:05.158838+0000","last_clean":"2024-09-10T10:03:05.158838+0000","last_became_active":"2024-09-10T10:03:05.151979+0000","last_became_peered":"2024-09-10T10:03:05.151979+0000","last_unstale":"2024-09-10T10:03:05.158838+0000","last_undegraded":"2024-09-10T10:03:05.158838+0000","last_fullsized":"2024-09-10T10:03:05.158838+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:39.875964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.161038+0000","last_change":"2024-09-10T10:03:05.154560+0000","last_active":"2024-09-10T10:03:05.161038+0000","last_peered":"2024-09-10T10:03:05.161038+0000","last_clean":"2024-09-10T10:03:05.161038+0000","last_became_active":"2024-09-10T10:03:05.154200+0000","last_became_peered":"2024-09-10T10:03:05.154200+0000","last_unstale":"2024-09-10T10:03:05.161038+0000","last_undegraded":"2024-09-10T10:03:05.161038+0000","last_fullsized":"2024-09-10T10:03:05.161038+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:01.258634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157156+0000","last_change":"2024-09-10T10:03:05.170093+0000","last_active":"2024-09-10T10:03:06.157156+0000","last_peered":"2024-09-10T10:03:06.157156+0000","last_clean":"2024-09-10T10:03:06.157156+0000","last_became_active":"2024-09-10T10:03:05.169820+0000","last_became_peered":"2024-09-10T10:03:05.169820+0000","last_unstale":"2024-09-10T10:03:06.157156+0000","last_undegraded":"2024-09-10T10:03:06.157156+0000","last_fullsized":"2024-09-10T10:03:06.157156+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:12:38.718259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157684+0000","last_change":"2024-09-10T10:03:05.171648+0000","last_active":"2024-09-10T10:03:06.157684+0000","last_peered":"2024-09-10T10:03:06.157684+0000","last_clean":"2024-09-10T10:03:06.157684+0000","last_became_active":"2024-09-10T10:03:05.168315+0000","last_became_peered":"2024-09-10T10:03:05.168315+0000","last_unstale":"2024-09-10T10:03:06.157684+0000","last_undegraded":"2024-09-10T10:03:06.157684+0000","last_fullsized":"2024-09-10T10:03:06.157684+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:01.892673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158776+0000","last_change":"2024-09-10T10:03:05.156448+0000","last_active":"2024-09-10T10:03:06.158776+0000","last_peered":"2024-09-10T10:03:06.158776+0000","last_clean":"2024-09-10T10:03:06.158776+0000","last_became_active":"2024-09-10T10:03:05.156135+0000","last_became_peered":"2024-09-10T10:03:05.156135+0000","last_unstale":"2024-09-10T10:03:06.158776+0000","last_undegraded":"2024-09-10T10:03:06.158776+0000","last_fullsized":"2024-09-10T10:03:06.158776+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:23.566054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157282+0000","last_change":"2024-09-10T10:03:05.170099+0000","last_active":"2024-09-10T10:03:06.157282+0000","last_peered":"2024-09-10T10:03:06.157282+0000","last_clean":"2024-09-10T10:03:06.157282+0000","last_became_active":"2024-09-10T10:03:05.169819+0000","last_became_peered":"2024-09-10T10:03:05.169819+0000","last_unstale":"2024-09-10T10:03:06.157282+0000","last_undegraded":"2024-09-10T10:03:06.157282+0000","last_fullsized":"2024-09-10T10:03:06.157282+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:16.369813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.161110+0000","last_change":"2024-09-10T10:03:05.155436+0000","last_active":"2024-09-10T10:03:05.161110+0000","last_peered":"2024-09-10T10:03:05.161110+0000","last_clean":"2024-09-10T10:03:05.161110+0000","last_became_active":"2024-09-10T10:03:05.155126+0000","last_became_peered":"2024-09-10T10:03:05.155126+0000","last_unstale":"2024-09-10T10:03:05.161110+0000","last_undegraded":"2024-09-10T10:03:05.161110+0000","last_fullsized":"2024-09-10T10:03:05.161110+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:23.566054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.158349+0000","last_change":"2024-09-10T10:03:05.151278+0000","last_active":"2024-09-10T10:03:05.158349+0000","last_peered":"2024-09-10T10:03:05.158349+0000","last_clean":"2024-09-10T10:03:05.158349+0000","last_became_active":"2024-09-10T10:03:05.150010+0000","last_became_peered":"2024-09-10T10:03:05.150010+0000","last_unstale":"2024-09-10T10:03:05.158349+0000","last_undegraded":"2024-09-10T10:03:05.158349+0000","last_fullsized":"2024-09-10T10:03:05.158349+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:05.657996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.156080+0000","last_change":"2024-09-10T10:03:05.152255+0000","last_active":"2024-09-10T10:03:05.156080+0000","last_peered":"2024-09-10T10:03:05.156080+0000","last_clean":"2024-09-10T10:03:05.156080+0000","last_became_active":"2024-09-10T10:03:05.151987+0000","last_became_peered":"2024-09-10T10:03:05.151987+0000","last_unstale":"2024-09-10T10:03:05.156080+0000","last_undegraded":"2024-09-10T10:03:05.156080+0000","last_fullsized":"2024-09-10T10:03:05.156080+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:43.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156540+0000","last_change":"2024-09-10T10:03:05.172860+0000","last_active":"2024-09-10T10:03:06.156540+0000","last_peered":"2024-09-10T10:03:06.156540+0000","last_clean":"2024-09-10T10:03:06.156540+0000","last_became_active":"2024-09-10T10:03:05.172220+0000","last_became_peered":"2024-09-10T10:03:05.172220+0000","last_unstale":"2024-09-10T10:03:06.156540+0000","last_undegraded":"2024-09-10T10:03:06.156540+0000","last_fullsized":"2024-09-10T10:03:06.156540+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:53.040198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158940+0000","last_change":"2024-09-10T10:03:05.156392+0000","last_active":"2024-09-10T10:03:06.158940+0000","last_peered":"2024-09-10T10:03:06.158940+0000","last_clean":"2024-09-10T10:03:06.158940+0000","last_became_active":"2024-09-10T10:03:05.156118+0000","last_became_peered":"2024-09-10T10:03:05.156118+0000","last_unstale":"2024-09-10T10:03:06.158940+0000","last_undegraded":"2024-09-10T10:03:06.158940+0000","last_fullsized":"2024-09-10T10:03:06.158940+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:16.016706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157923+0000","last_change":"2024-09-10T10:03:05.160880+0000","last_active":"2024-09-10T10:03:06.157923+0000","last_peered":"2024-09-10T10:03:06.157923+0000","last_clean":"2024-09-10T10:03:06.157923+0000","last_became_active":"2024-09-10T10:03:05.160458+0000","last_became_peered":"2024-09-10T10:03:05.160458+0000","last_unstale":"2024-09-10T10:03:06.157923+0000","last_undegraded":"2024-09-10T10:03:06.157923+0000","last_fullsized":"2024-09-10T10:03:06.157923+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:16.016706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.160954+0000","last_change":"2024-09-10T10:03:05.153863+0000","last_active":"2024-09-10T10:03:05.160954+0000","last_peered":"2024-09-10T10:03:05.160954+0000","last_clean":"2024-09-10T10:03:05.160954+0000","last_became_active":"2024-09-10T10:03:05.153001+0000","last_became_peered":"2024-09-10T10:03:05.153001+0000","last_unstale":"2024-09-10T10:03:05.160954+0000","last_undegraded":"2024-09-10T10:03:05.160954+0000","last_fullsized":"2024-09-10T10:03:05.160954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:55.040526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158753+0000","last_change":"2024-09-10T10:03:05.155437+0000","last_active":"2024-09-10T10:03:06.158753+0000","last_peered":"2024-09-10T10:03:06.158753+0000","last_clean":"2024-09-10T10:03:06.158753+0000","last_became_active":"2024-09-10T10:03:05.155145+0000","last_became_peered":"2024-09-10T10:03:05.155145+0000","last_unstale":"2024-09-10T10:03:06.158753+0000","last_undegraded":"2024-09-10T10:03:06.158753+0000","last_fullsized":"2024-09-10T10:03:06.158753+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:01.258634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.159478+0000","last_change":"2024-09-10T10:03:05.157161+0000","last_active":"2024-09-10T10:03:06.159478+0000","last_peered":"2024-09-10T10:03:06.159478+0000","last_clean":"2024-09-10T10:03:06.159478+0000","last_became_active":"2024-09-10T10:03:05.156832+0000","last_became_peered":"2024-09-10T10:03:05.156832+0000","last_unstale":"2024-09-10T10:03:06.159478+0000","last_undegraded":"2024-09-10T10:03:06.159478+0000","last_fullsized":"2024-09-10T10:03:06.159478+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:45.445940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634324+0000","last_change":"2024-09-10T10:03:05.145571+0000","last_active":"2024-09-10T10:03:06.634324+0000","last_peered":"2024-09-10T10:03:06.634324+0000","last_clean":"2024-09-10T10:03:06.634324+0000","last_became_active":"2024-09-10T10:03:05.145283+0000","last_became_peered":"2024-09-10T10:03:05.145283+0000","last_unstale":"2024-09-10T10:03:06.634324+0000","last_undegraded":"2024-09-10T10:03:06.634324+0000","last_fullsized":"2024-09-10T10:03:06.634324+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:15.640226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158043+0000","last_change":"2024-09-10T10:03:05.158224+0000","last_active":"2024-09-10T10:03:06.158043+0000","last_peered":"2024-09-10T10:03:06.158043+0000","last_clean":"2024-09-10T10:03:06.158043+0000","last_became_active":"2024-09-10T10:03:05.157697+0000","last_became_peered":"2024-09-10T10:03:05.157697+0000","last_unstale":"2024-09-10T10:03:06.158043+0000","last_undegraded":"2024-09-10T10:03:06.158043+0000","last_fullsized":"2024-09-10T10:03:06.158043+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:51.628666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634090+0000","last_change":"2024-09-10T10:03:05.150113+0000","last_active":"2024-09-10T10:03:06.634090+0000","last_peered":"2024-09-10T10:03:06.634090+0000","last_clean":"2024-09-10T10:03:06.634090+0000","last_became_active":"2024-09-10T10:03:05.149844+0000","last_became_peered":"2024-09-10T10:03:05.149844+0000","last_unstale":"2024-09-10T10:03:06.634090+0000","last_undegraded":"2024-09-10T10:03:06.634090+0000","last_fullsized":"2024-09-10T10:03:06.634090+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:49.366205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.159027+0000","last_change":"2024-09-10T10:03:05.161669+0000","last_active":"2024-09-10T10:03:06.159027+0000","last_peered":"2024-09-10T10:03:06.159027+0000","last_clean":"2024-09-10T10:03:06.159027+0000","last_became_active":"2024-09-10T10:03:05.161324+0000","last_became_peered":"2024-09-10T10:03:05.161324+0000","last_unstale":"2024-09-10T10:03:06.159027+0000","last_undegraded":"2024-09-10T10:03:06.159027+0000","last_fullsized":"2024-09-10T10:03:06.159027+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:27.289059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157825+0000","last_change":"2024-09-10T10:03:05.172663+0000","last_active":"2024-09-10T10:03:06.157825+0000","last_peered":"2024-09-10T10:03:06.157825+0000","last_clean":"2024-09-10T10:03:06.157825+0000","last_became_active":"2024-09-10T10:03:05.172417+0000","last_became_peered":"2024-09-10T10:03:05.172417+0000","last_unstale":"2024-09-10T10:03:06.157825+0000","last_undegraded":"2024-09-10T10:03:06.157825+0000","last_fullsized":"2024-09-10T10:03:06.157825+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:11.523586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.170947+0000","last_change":"2024-09-10T10:03:05.167884+0000","last_active":"2024-09-10T10:03:05.170947+0000","last_peered":"2024-09-10T10:03:05.170947+0000","last_clean":"2024-09-10T10:03:05.170947+0000","last_became_active":"2024-09-10T10:03:05.166536+0000","last_became_peered":"2024-09-10T10:03:05.166536+0000","last_unstale":"2024-09-10T10:03:05.170947+0000","last_undegraded":"2024-09-10T10:03:05.170947+0000","last_fullsized":"2024-09-10T10:03:05.170947+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:30.151714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634244+0000","last_change":"2024-09-10T10:03:05.168406+0000","last_active":"2024-09-10T10:03:06.634244+0000","last_peered":"2024-09-10T10:03:06.634244+0000","last_clean":"2024-09-10T10:03:06.634244+0000","last_became_active":"2024-09-10T10:03:05.168051+0000","last_became_peered":"2024-09-10T10:03:05.168051+0000","last_unstale":"2024-09-10T10:03:06.634244+0000","last_undegraded":"2024-09-10T10:03:06.634244+0000","last_fullsized":"2024-09-10T10:03:06.634244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:12.424333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158133+0000","last_change":"2024-09-10T10:03:05.157033+0000","last_active":"2024-09-10T10:03:06.158133+0000","last_peered":"2024-09-10T10:03:06.158133+0000","last_clean":"2024-09-10T10:03:06.158133+0000","last_became_active":"2024-09-10T10:03:05.156466+0000","last_became_peered":"2024-09-10T10:03:05.156466+0000","last_unstale":"2024-09-10T10:03:06.158133+0000","last_undegraded":"2024-09-10T10:03:06.158133+0000","last_fullsized":"2024-09-10T10:03:06.158133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:15.379112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.167829+0000","last_change":"2024-09-10T10:03:05.162355+0000","last_active":"2024-09-10T10:03:05.167829+0000","last_peered":"2024-09-10T10:03:05.167829+0000","last_clean":"2024-09-10T10:03:05.167829+0000","last_became_active":"2024-09-10T10:03:05.162081+0000","last_became_peered":"2024-09-10T10:03:05.162081+0000","last_unstale":"2024-09-10T10:03:05.167829+0000","last_undegraded":"2024-09-10T10:03:05.167829+0000","last_fullsized":"2024-09-10T10:03:05.167829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:51.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157021+0000","last_change":"2024-09-10T10:03:05.174547+0000","last_active":"2024-09-10T10:03:06.157021+0000","last_peered":"2024-09-10T10:03:06.157021+0000","last_clean":"2024-09-10T10:03:06.157021+0000","last_became_active":"2024-09-10T10:03:05.174092+0000","last_became_peered":"2024-09-10T10:03:05.174092+0000","last_unstale":"2024-09-10T10:03:06.157021+0000","last_undegraded":"2024-09-10T10:03:06.157021+0000","last_fullsized":"2024-09-10T10:03:06.157021+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:50.034187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.157727+0000","last_change":"2024-09-10T10:03:05.156000+0000","last_active":"2024-09-10T10:03:05.157727+0000","last_peered":"2024-09-10T10:03:05.157727+0000","last_clean":"2024-09-10T10:03:05.157727+0000","last_became_active":"2024-09-10T10:03:05.155710+0000","last_became_peered":"2024-09-10T10:03:05.155710+0000","last_unstale":"2024-09-10T10:03:05.157727+0000","last_undegraded":"2024-09-10T10:03:05.157727+0000","last_fullsized":"2024-09-10T10:03:05.157727+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:32.646817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.167957+0000","last_change":"2024-09-10T10:03:05.157669+0000","last_active":"2024-09-10T10:03:05.167957+0000","last_peered":"2024-09-10T10:03:05.167957+0000","last_clean":"2024-09-10T10:03:05.167957+0000","last_became_active":"2024-09-10T10:03:05.157222+0000","last_became_peered":"2024-09-10T10:03:05.157222+0000","last_unstale":"2024-09-10T10:03:05.167957+0000","last_undegraded":"2024-09-10T10:03:05.167957+0000","last_fullsized":"2024-09-10T10:03:05.167957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:15.418491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.168342+0000","last_change":"2024-09-10T10:03:04.149739+0000","last_active":"2024-09-10T10:03:05.168342+0000","last_peered":"2024-09-10T10:03:05.168342+0000","last_clean":"2024-09-10T10:03:05.168342+0000","last_became_active":"2024-09-10T10:03:04.149240+0000","last_became_peered":"2024-09-10T10:03:04.149240+0000","last_unstale":"2024-09-10T10:03:05.168342+0000","last_undegraded":"2024-09-10T10:03:05.168342+0000","last_fullsized":"2024-09-10T10:03:05.168342+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:12.890195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.159130+0000","last_change":"2024-09-10T10:03:05.156642+0000","last_active":"2024-09-10T10:03:06.159130+0000","last_peered":"2024-09-10T10:03:06.159130+0000","last_clean":"2024-09-10T10:03:06.159130+0000","last_became_active":"2024-09-10T10:03:05.155830+0000","last_became_peered":"2024-09-10T10:03:05.155830+0000","last_unstale":"2024-09-10T10:03:06.159130+0000","last_undegraded":"2024-09-10T10:03:06.159130+0000","last_fullsized":"2024-09-10T10:03:06.159130+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:57.949604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.163115+0000","last_change":"2024-09-10T10:03:04.151632+0000","last_active":"2024-09-10T10:03:05.163115+0000","last_peered":"2024-09-10T10:03:05.163115+0000","last_clean":"2024-09-10T10:03:05.163115+0000","last_became_active":"2024-09-10T10:03:04.151278+0000","last_became_peered":"2024-09-10T10:03:04.151278+0000","last_unstale":"2024-09-10T10:03:05.163115+0000","last_undegraded":"2024-09-10T10:03:05.163115+0000","last_fullsized":"2024-09-10T10:03:05.163115+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:30.731113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.172361+0000","last_change":"2024-09-10T10:03:05.169522+0000","last_active":"2024-09-10T10:03:05.172361+0000","last_peered":"2024-09-10T10:03:05.172361+0000","last_clean":"2024-09-10T10:03:05.172361+0000","last_became_active":"2024-09-10T10:03:05.169100+0000","last_became_peered":"2024-09-10T10:03:05.169100+0000","last_unstale":"2024-09-10T10:03:05.172361+0000","last_undegraded":"2024-09-10T10:03:05.172361+0000","last_fullsized":"2024-09-10T10:03:05.172361+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:26.001196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.173210+0000","last_change":"2024-09-10T10:03:04.153849+0000","last_active":"2024-09-10T10:03:05.173210+0000","last_peered":"2024-09-10T10:03:05.173210+0000","last_clean":"2024-09-10T10:03:05.173210+0000","last_became_active":"2024-09-10T10:03:04.153483+0000","last_became_peered":"2024-09-10T10:03:04.153483+0000","last_unstale":"2024-09-10T10:03:05.173210+0000","last_undegraded":"2024-09-10T10:03:05.173210+0000","last_fullsized":"2024-09-10T10:03:05.173210+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:14.386747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.160972+0000","last_change":"2024-09-10T10:03:05.154091+0000","last_active":"2024-09-10T10:03:05.160972+0000","last_peered":"2024-09-10T10:03:05.160972+0000","last_clean":"2024-09-10T10:03:05.160972+0000","last_became_active":"2024-09-10T10:03:05.153367+0000","last_became_peered":"2024-09-10T10:03:05.153367+0000","last_unstale":"2024-09-10T10:03:05.160972+0000","last_undegraded":"2024-09-10T10:03:05.160972+0000","last_fullsized":"2024-09-10T10:03:05.160972+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.719505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.172984+0000","last_change":"2024-09-10T10:03:04.155558+0000","last_active":"2024-09-10T10:03:05.172984+0000","last_peered":"2024-09-10T10:03:05.172984+0000","last_clean":"2024-09-10T10:03:05.172984+0000","last_became_active":"2024-09-10T10:03:04.155306+0000","last_became_peered":"2024-09-10T10:03:04.155306+0000","last_unstale":"2024-09-10T10:03:05.172984+0000","last_undegraded":"2024-09-10T10:03:05.172984+0000","last_fullsized":"2024-09-10T10:03:05.172984+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:36.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156730+0000","last_change":"2024-09-10T10:03:05.173577+0000","last_active":"2024-09-10T10:03:06.156730+0000","last_peered":"2024-09-10T10:03:06.156730+0000","last_clean":"2024-09-10T10:03:06.156730+0000","last_became_active":"2024-09-10T10:03:05.170423+0000","last_became_peered":"2024-09-10T10:03:05.170423+0000","last_unstale":"2024-09-10T10:03:06.156730+0000","last_undegraded":"2024-09-10T10:03:06.156730+0000","last_fullsized":"2024-09-10T10:03:06.156730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:16.207967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.170331+0000","last_change":"2024-09-10T10:03:04.156743+0000","last_active":"2024-09-10T10:03:05.170331+0000","last_peered":"2024-09-10T10:03:05.170331+0000","last_clean":"2024-09-10T10:03:05.170331+0000","last_became_active":"2024-09-10T10:03:04.156471+0000","last_became_peered":"2024-09-10T10:03:04.156471+0000","last_unstale":"2024-09-10T10:03:05.170331+0000","last_undegraded":"2024-09-10T10:03:05.170331+0000","last_fullsized":"2024-09-10T10:03:05.170331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:40.063337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158299+0000","last_change":"2024-09-10T10:03:05.153846+0000","last_active":"2024-09-10T10:03:06.158299+0000","last_peered":"2024-09-10T10:03:06.158299+0000","last_clean":"2024-09-10T10:03:06.158299+0000","last_became_active":"2024-09-10T10:03:05.153586+0000","last_became_peered":"2024-09-10T10:03:05.153586+0000","last_unstale":"2024-09-10T10:03:06.158299+0000","last_undegraded":"2024-09-10T10:03:06.158299+0000","last_fullsized":"2024-09-10T10:03:06.158299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:05.657996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634354+0000","last_change":"2024-09-10T10:03:04.149728+0000","last_active":"2024-09-10T10:03:06.634354+0000","last_peered":"2024-09-10T10:03:06.634354+0000","last_clean":"2024-09-10T10:03:06.634354+0000","last_became_active":"2024-09-10T10:03:04.149405+0000","last_became_peered":"2024-09-10T10:03:04.149405+0000","last_unstale":"2024-09-10T10:03:06.634354+0000","last_undegraded":"2024-09-10T10:03:06.634354+0000","last_fullsized":"2024-09-10T10:03:06.634354+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:25.651067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156546+0000","last_change":"2024-09-10T10:03:05.154294+0000","last_active":"2024-09-10T10:03:06.156546+0000","last_peered":"2024-09-10T10:03:06.156546+0000","last_clean":"2024-09-10T10:03:06.156546+0000","last_became_active":"2024-09-10T10:03:05.154003+0000","last_became_peered":"2024-09-10T10:03:05.154003+0000","last_unstale":"2024-09-10T10:03:06.156546+0000","last_undegraded":"2024-09-10T10:03:06.156546+0000","last_fullsized":"2024-09-10T10:03:06.156546+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:33.308961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158155+0000","last_change":"2024-09-10T10:03:04.149416+0000","last_active":"2024-09-10T10:03:06.158155+0000","last_peered":"2024-09-10T10:03:06.158155+0000","last_clean":"2024-09-10T10:03:06.158155+0000","last_became_active":"2024-09-10T10:03:04.149110+0000","last_became_peered":"2024-09-10T10:03:04.149110+0000","last_unstale":"2024-09-10T10:03:06.158155+0000","last_undegraded":"2024-09-10T10:03:06.158155+0000","last_fullsized":"2024-09-10T10:03:06.158155+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:33.863849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.158860+0000","last_change":"2024-09-10T10:03:05.153003+0000","last_active":"2024-09-10T10:03:05.158860+0000","last_peered":"2024-09-10T10:03:05.158860+0000","last_clean":"2024-09-10T10:03:05.158860+0000","last_became_active":"2024-09-10T10:03:05.152509+0000","last_became_peered":"2024-09-10T10:03:05.152509+0000","last_unstale":"2024-09-10T10:03:05.158860+0000","last_undegraded":"2024-09-10T10:03:05.158860+0000","last_fullsized":"2024-09-10T10:03:05.158860+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:26.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156831+0000","last_change":"2024-09-10T10:03:04.158095+0000","last_active":"2024-09-10T10:03:06.156831+0000","last_peered":"2024-09-10T10:03:06.156831+0000","last_clean":"2024-09-10T10:03:06.156831+0000","last_became_active":"2024-09-10T10:03:04.157806+0000","last_became_peered":"2024-09-10T10:03:04.157806+0000","last_unstale":"2024-09-10T10:03:06.156831+0000","last_undegraded":"2024-09-10T10:03:06.156831+0000","last_fullsized":"2024-09-10T10:03:06.156831+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:20:48.641017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.164819+0000","last_change":"2024-09-10T10:03:05.156751+0000","last_active":"2024-09-10T10:03:05.164819+0000","last_peered":"2024-09-10T10:03:05.164819+0000","last_clean":"2024-09-10T10:03:05.164819+0000","last_became_active":"2024-09-10T10:03:05.156488+0000","last_became_peered":"2024-09-10T10:03:05.156488+0000","last_unstale":"2024-09-10T10:03:05.164819+0000","last_undegraded":"2024-09-10T10:03:05.164819+0000","last_fullsized":"2024-09-10T10:03:05.164819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:18.871682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.138753+0000","last_change":"2024-09-10T10:03:04.153843+0000","last_active":"2024-09-10T10:03:05.138753+0000","last_peered":"2024-09-10T10:03:05.138753+0000","last_clean":"2024-09-10T10:03:05.138753+0000","last_became_active":"2024-09-10T10:03:04.153565+0000","last_became_peered":"2024-09-10T10:03:04.153565+0000","last_unstale":"2024-09-10T10:03:05.138753+0000","last_undegraded":"2024-09-10T10:03:05.138753+0000","last_fullsized":"2024-09-10T10:03:05.138753+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:36.813763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.159240+0000","last_change":"2024-09-10T10:03:05.156861+0000","last_active":"2024-09-10T10:03:06.159240+0000","last_peered":"2024-09-10T10:03:06.159240+0000","last_clean":"2024-09-10T10:03:06.159240+0000","last_became_active":"2024-09-10T10:03:05.156359+0000","last_became_peered":"2024-09-10T10:03:05.156359+0000","last_unstale":"2024-09-10T10:03:06.159240+0000","last_undegraded":"2024-09-10T10:03:06.159240+0000","last_fullsized":"2024-09-10T10:03:06.159240+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:37.947530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634226+0000","last_change":"2024-09-10T10:03:04.148096+0000","last_active":"2024-09-10T10:03:06.634226+0000","last_peered":"2024-09-10T10:03:06.634226+0000","last_clean":"2024-09-10T10:03:06.634226+0000","last_became_active":"2024-09-10T10:03:04.146168+0000","last_became_peered":"2024-09-10T10:03:04.146168+0000","last_unstale":"2024-09-10T10:03:06.634226+0000","last_undegraded":"2024-09-10T10:03:06.634226+0000","last_fullsized":"2024-09-10T10:03:06.634226+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:04.150107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634390+0000","last_change":"2024-09-10T10:03:05.159737+0000","last_active":"2024-09-10T10:03:06.634390+0000","last_peered":"2024-09-10T10:03:06.634390+0000","last_clean":"2024-09-10T10:03:06.634390+0000","last_became_active":"2024-09-10T10:03:05.155486+0000","last_became_peered":"2024-09-10T10:03:05.155486+0000","last_unstale":"2024-09-10T10:03:06.634390+0000","last_undegraded":"2024-09-10T10:03:06.634390+0000","last_fullsized":"2024-09-10T10:03:06.634390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:26.391372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156821+0000","last_change":"2024-09-10T10:03:04.153779+0000","last_active":"2024-09-10T10:03:06.156821+0000","last_peered":"2024-09-10T10:03:06.156821+0000","last_clean":"2024-09-10T10:03:06.156821+0000","last_became_active":"2024-09-10T10:03:04.153470+0000","last_became_peered":"2024-09-10T10:03:04.153470+0000","last_unstale":"2024-09-10T10:03:06.156821+0000","last_undegraded":"2024-09-10T10:03:06.156821+0000","last_fullsized":"2024-09-10T10:03:06.156821+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:04.147964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.156167+0000","last_change":"2024-09-10T10:03:05.146195+0000","last_active":"2024-09-10T10:03:05.156167+0000","last_peered":"2024-09-10T10:03:05.156167+0000","last_clean":"2024-09-10T10:03:05.156167+0000","last_became_active":"2024-09-10T10:03:05.145891+0000","last_became_peered":"2024-09-10T10:03:05.145891+0000","last_unstale":"2024-09-10T10:03:05.156167+0000","last_undegraded":"2024-09-10T10:03:05.156167+0000","last_fullsized":"2024-09-10T10:03:05.156167+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:19.294841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157390+0000","last_change":"2024-09-10T10:03:04.152717+0000","last_active":"2024-09-10T10:03:06.157390+0000","last_peered":"2024-09-10T10:03:06.157390+0000","last_clean":"2024-09-10T10:03:06.157390+0000","last_became_active":"2024-09-10T10:03:04.152450+0000","last_became_peered":"2024-09-10T10:03:04.152450+0000","last_unstale":"2024-09-10T10:03:06.157390+0000","last_undegraded":"2024-09-10T10:03:06.157390+0000","last_fullsized":"2024-09-10T10:03:06.157390+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:10.648351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634258+0000","last_change":"2024-09-10T10:03:05.155170+0000","last_active":"2024-09-10T10:03:06.634258+0000","last_peered":"2024-09-10T10:03:06.634258+0000","last_clean":"2024-09-10T10:03:06.634258+0000","last_became_active":"2024-09-10T10:03:05.154230+0000","last_became_peered":"2024-09-10T10:03:05.154230+0000","last_unstale":"2024-09-10T10:03:06.634258+0000","last_undegraded":"2024-09-10T10:03:06.634258+0000","last_fullsized":"2024-09-10T10:03:06.634258+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:36.451626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.172804+0000","last_change":"2024-09-10T10:03:04.152645+0000","last_active":"2024-09-10T10:03:05.172804+0000","last_peered":"2024-09-10T10:03:05.172804+0000","last_clean":"2024-09-10T10:03:05.172804+0000","last_became_active":"2024-09-10T10:03:04.152390+0000","last_became_peered":"2024-09-10T10:03:04.152390+0000","last_unstale":"2024-09-10T10:03:05.172804+0000","last_undegraded":"2024-09-10T10:03:05.172804+0000","last_fullsized":"2024-09-10T10:03:05.172804+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:32:29.083600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634237+0000","last_change":"2024-09-10T10:03:05.153282+0000","last_active":"2024-09-10T10:03:06.634237+0000","last_peered":"2024-09-10T10:03:06.634237+0000","last_clean":"2024-09-10T10:03:06.634237+0000","last_became_active":"2024-09-10T10:03:05.150536+0000","last_became_peered":"2024-09-10T10:03:05.150536+0000","last_unstale":"2024-09-10T10:03:06.634237+0000","last_undegraded":"2024-09-10T10:03:06.634237+0000","last_fullsized":"2024-09-10T10:03:06.634237+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:53.249360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158154+0000","last_change":"2024-09-10T10:03:04.146776+0000","last_active":"2024-09-10T10:03:06.158154+0000","last_peered":"2024-09-10T10:03:06.158154+0000","last_clean":"2024-09-10T10:03:06.158154+0000","last_became_active":"2024-09-10T10:03:04.146500+0000","last_became_peered":"2024-09-10T10:03:04.146500+0000","last_unstale":"2024-09-10T10:03:06.158154+0000","last_undegraded":"2024-09-10T10:03:06.158154+0000","last_fullsized":"2024-09-10T10:03:06.158154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:38.689419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.633938+0000","last_change":"2024-09-10T10:03:05.153510+0000","last_active":"2024-09-10T10:03:06.633938+0000","last_peered":"2024-09-10T10:03:06.633938+0000","last_clean":"2024-09-10T10:03:06.633938+0000","last_became_active":"2024-09-10T10:03:05.153225+0000","last_became_peered":"2024-09-10T10:03:05.153225+0000","last_unstale":"2024-09-10T10:03:06.633938+0000","last_undegraded":"2024-09-10T10:03:06.633938+0000","last_fullsized":"2024-09-10T10:03:06.633938+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:03.412549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.177246+0000","last_change":"2024-09-10T10:03:04.148457+0000","last_active":"2024-09-10T10:03:05.177246+0000","last_peered":"2024-09-10T10:03:05.177246+0000","last_clean":"2024-09-10T10:03:05.177246+0000","last_became_active":"2024-09-10T10:03:04.148162+0000","last_became_peered":"2024-09-10T10:03:04.148162+0000","last_unstale":"2024-09-10T10:03:05.177246+0000","last_undegraded":"2024-09-10T10:03:05.177246+0000","last_fullsized":"2024-09-10T10:03:05.177246+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:30.108499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.159801+0000","last_change":"2024-09-10T10:03:05.158316+0000","last_active":"2024-09-10T10:03:05.159801+0000","last_peered":"2024-09-10T10:03:05.159801+0000","last_clean":"2024-09-10T10:03:05.159801+0000","last_became_active":"2024-09-10T10:03:05.158035+0000","last_became_peered":"2024-09-10T10:03:05.158035+0000","last_unstale":"2024-09-10T10:03:05.159801+0000","last_undegraded":"2024-09-10T10:03:05.159801+0000","last_fullsized":"2024-09-10T10:03:05.159801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:39.121779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157389+0000","last_change":"2024-09-10T10:03:04.155803+0000","last_active":"2024-09-10T10:03:06.157389+0000","last_peered":"2024-09-10T10:03:06.157389+0000","last_clean":"2024-09-10T10:03:06.157389+0000","last_became_active":"2024-09-10T10:03:04.152633+0000","last_became_peered":"2024-09-10T10:03:04.152633+0000","last_unstale":"2024-09-10T10:03:06.157389+0000","last_undegraded":"2024-09-10T10:03:06.157389+0000","last_fullsized":"2024-09-10T10:03:06.157389+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:03.814589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.157772+0000","last_change":"2024-09-10T10:03:05.154740+0000","last_active":"2024-09-10T10:03:05.157772+0000","last_peered":"2024-09-10T10:03:05.157772+0000","last_clean":"2024-09-10T10:03:05.157772+0000","last_became_active":"2024-09-10T10:03:05.154480+0000","last_became_peered":"2024-09-10T10:03:05.154480+0000","last_unstale":"2024-09-10T10:03:05.157772+0000","last_undegraded":"2024-09-10T10:03:05.157772+0000","last_fullsized":"2024-09-10T10:03:05.157772+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:03.182798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.169442+0000","last_change":"2024-09-10T10:03:04.139308+0000","last_active":"2024-09-10T10:03:05.169442+0000","last_peered":"2024-09-10T10:03:05.169442+0000","last_clean":"2024-09-10T10:03:05.169442+0000","last_became_active":"2024-09-10T10:03:04.139035+0000","last_became_peered":"2024-09-10T10:03:04.139035+0000","last_unstale":"2024-09-10T10:03:05.169442+0000","last_undegraded":"2024-09-10T10:03:05.169442+0000","last_fullsized":"2024-09-10T10:03:05.169442+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:34.306326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.175449+0000","last_change":"2024-09-10T10:03:05.174553+0000","last_active":"2024-09-10T10:03:05.175449+0000","last_peered":"2024-09-10T10:03:05.175449+0000","last_clean":"2024-09-10T10:03:05.175449+0000","last_became_active":"2024-09-10T10:03:05.174244+0000","last_became_peered":"2024-09-10T10:03:05.174244+0000","last_unstale":"2024-09-10T10:03:05.175449+0000","last_undegraded":"2024-09-10T10:03:05.175449+0000","last_fullsized":"2024-09-10T10:03:05.175449+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:27.289059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158238+0000","last_change":"2024-09-10T10:03:04.147450+0000","last_active":"2024-09-10T10:03:06.158238+0000","last_peered":"2024-09-10T10:03:06.158238+0000","last_clean":"2024-09-10T10:03:06.158238+0000","last_became_active":"2024-09-10T10:03:04.147067+0000","last_became_peered":"2024-09-10T10:03:04.147067+0000","last_unstale":"2024-09-10T10:03:06.158238+0000","last_undegraded":"2024-09-10T10:03:06.158238+0000","last_fullsized":"2024-09-10T10:03:06.158238+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:50.625533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.150555+0000","last_change":"2024-09-10T10:03:05.145982+0000","last_active":"2024-09-10T10:03:05.150555+0000","last_peered":"2024-09-10T10:03:05.150555+0000","last_clean":"2024-09-10T10:03:05.150555+0000","last_became_active":"2024-09-10T10:03:05.145484+0000","last_became_peered":"2024-09-10T10:03:05.145484+0000","last_unstale":"2024-09-10T10:03:05.150555+0000","last_undegraded":"2024-09-10T10:03:05.150555+0000","last_fullsized":"2024-09-10T10:03:05.150555+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:39.692552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.140605+0000","last_change":"2024-09-10T10:03:04.153017+0000","last_active":"2024-09-10T10:03:05.140605+0000","last_peered":"2024-09-10T10:03:05.140605+0000","last_clean":"2024-09-10T10:03:05.140605+0000","last_became_active":"2024-09-10T10:03:04.152668+0000","last_became_peered":"2024-09-10T10:03:04.152668+0000","last_unstale":"2024-09-10T10:03:05.140605+0000","last_undegraded":"2024-09-10T10:03:05.140605+0000","last_fullsized":"2024-09-10T10:03:05.140605+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:52.510664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.161462+0000","last_change":"2024-09-10T10:03:05.155639+0000","last_active":"2024-09-10T10:03:05.161462+0000","last_peered":"2024-09-10T10:03:05.161462+0000","last_clean":"2024-09-10T10:03:05.161462+0000","last_became_active":"2024-09-10T10:03:05.155372+0000","last_became_peered":"2024-09-10T10:03:05.155372+0000","last_unstale":"2024-09-10T10:03:05.161462+0000","last_undegraded":"2024-09-10T10:03:05.161462+0000","last_fullsized":"2024-09-10T10:03:05.161462+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:16.016706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.172785+0000","last_change":"2024-09-10T10:03:04.151877+0000","last_active":"2024-09-10T10:03:05.172785+0000","last_peered":"2024-09-10T10:03:05.172785+0000","last_clean":"2024-09-10T10:03:05.172785+0000","last_became_active":"2024-09-10T10:03:04.151612+0000","last_became_peered":"2024-09-10T10:03:04.151612+0000","last_unstale":"2024-09-10T10:03:05.172785+0000","last_undegraded":"2024-09-10T10:03:05.172785+0000","last_fullsized":"2024-09-10T10:03:05.172785+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:11.317940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156980+0000","last_change":"2024-09-10T10:03:05.172567+0000","last_active":"2024-09-10T10:03:06.156980+0000","last_peered":"2024-09-10T10:03:06.156980+0000","last_clean":"2024-09-10T10:03:06.156980+0000","last_became_active":"2024-09-10T10:03:05.172178+0000","last_became_peered":"2024-09-10T10:03:05.172178+0000","last_unstale":"2024-09-10T10:03:06.156980+0000","last_undegraded":"2024-09-10T10:03:06.156980+0000","last_fullsized":"2024-09-10T10:03:06.156980+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:14.633611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.138665+0000","last_change":"2024-09-10T10:03:04.157375+0000","last_active":"2024-09-10T10:03:05.138665+0000","last_peered":"2024-09-10T10:03:05.138665+0000","last_clean":"2024-09-10T10:03:05.138665+0000","last_became_active":"2024-09-10T10:03:04.157141+0000","last_became_peered":"2024-09-10T10:03:04.157141+0000","last_unstale":"2024-09-10T10:03:05.138665+0000","last_undegraded":"2024-09-10T10:03:05.138665+0000","last_fullsized":"2024-09-10T10:03:05.138665+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:02.694978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.154453+0000","last_change":"2024-09-10T10:03:05.148704+0000","last_active":"2024-09-10T10:03:05.154453+0000","last_peered":"2024-09-10T10:03:05.154453+0000","last_clean":"2024-09-10T10:03:05.154453+0000","last_became_active":"2024-09-10T10:03:05.148399+0000","last_became_peered":"2024-09-10T10:03:05.148399+0000","last_unstale":"2024-09-10T10:03:05.154453+0000","last_undegraded":"2024-09-10T10:03:05.154453+0000","last_fullsized":"2024-09-10T10:03:05.154453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:15.379112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.141863+0000","last_change":"2024-09-10T10:03:04.154634+0000","last_active":"2024-09-10T10:03:05.141863+0000","last_peered":"2024-09-10T10:03:05.141863+0000","last_clean":"2024-09-10T10:03:05.141863+0000","last_became_active":"2024-09-10T10:03:04.154121+0000","last_became_peered":"2024-09-10T10:03:04.154121+0000","last_unstale":"2024-09-10T10:03:05.141863+0000","last_undegraded":"2024-09-10T10:03:05.141863+0000","last_fullsized":"2024-09-10T10:03:05.141863+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:46.763023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634070+0000","last_change":"2024-09-10T10:03:05.166035+0000","last_active":"2024-09-10T10:03:06.634070+0000","last_peered":"2024-09-10T10:03:06.634070+0000","last_clean":"2024-09-10T10:03:06.634070+0000","last_became_active":"2024-09-10T10:03:05.165711+0000","last_became_peered":"2024-09-10T10:03:05.165711+0000","last_unstale":"2024-09-10T10:03:06.634070+0000","last_undegraded":"2024-09-10T10:03:06.634070+0000","last_fullsized":"2024-09-10T10:03:06.634070+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:24.751202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156811+0000","last_change":"2024-09-10T10:03:04.145276+0000","last_active":"2024-09-10T10:03:06.156811+0000","last_peered":"2024-09-10T10:03:06.156811+0000","last_clean":"2024-09-10T10:03:06.156811+0000","last_became_active":"2024-09-10T10:03:04.144537+0000","last_became_peered":"2024-09-10T10:03:04.144537+0000","last_unstale":"2024-09-10T10:03:06.156811+0000","last_undegraded":"2024-09-10T10:03:06.156811+0000","last_fullsized":"2024-09-10T10:03:06.156811+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:34.893656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158687+0000","last_change":"2024-09-10T10:03:05.153683+0000","last_active":"2024-09-10T10:03:06.158687+0000","last_peered":"2024-09-10T10:03:06.158687+0000","last_clean":"2024-09-10T10:03:06.158687+0000","last_became_active":"2024-09-10T10:03:05.153411+0000","last_became_peered":"2024-09-10T10:03:05.153411+0000","last_unstale":"2024-09-10T10:03:06.158687+0000","last_undegraded":"2024-09-10T10:03:06.158687+0000","last_fullsized":"2024-09-10T10:03:06.158687+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:45.445940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634206+0000","last_change":"2024-09-10T10:03:04.150472+0000","last_active":"2024-09-10T10:03:06.634206+0000","last_peered":"2024-09-10T10:03:06.634206+0000","last_clean":"2024-09-10T10:03:06.634206+0000","last_became_active":"2024-09-10T10:03:04.150224+0000","last_became_peered":"2024-09-10T10:03:04.150224+0000","last_unstale":"2024-09-10T10:03:06.634206+0000","last_undegraded":"2024-09-10T10:03:06.634206+0000","last_fullsized":"2024-09-10T10:03:06.634206+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:51.039468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157298+0000","last_change":"2024-09-10T10:03:05.162999+0000","last_active":"2024-09-10T10:03:06.157298+0000","last_peered":"2024-09-10T10:03:06.157298+0000","last_clean":"2024-09-10T10:03:06.157298+0000","last_became_active":"2024-09-10T10:03:05.162769+0000","last_became_peered":"2024-09-10T10:03:05.162769+0000","last_unstale":"2024-09-10T10:03:06.157298+0000","last_undegraded":"2024-09-10T10:03:06.157298+0000","last_fullsized":"2024-09-10T10:03:06.157298+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:39.748066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158678+0000","last_change":"2024-09-10T10:03:04.149970+0000","last_active":"2024-09-10T10:03:06.158678+0000","last_peered":"2024-09-10T10:03:06.158678+0000","last_clean":"2024-09-10T10:03:06.158678+0000","last_became_active":"2024-09-10T10:03:04.149656+0000","last_became_peered":"2024-09-10T10:03:04.149656+0000","last_unstale":"2024-09-10T10:03:06.158678+0000","last_undegraded":"2024-09-10T10:03:06.158678+0000","last_fullsized":"2024-09-10T10:03:06.158678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:14.375979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157406+0000","last_change":"2024-09-10T10:03:05.154296+0000","last_active":"2024-09-10T10:03:06.157406+0000","last_peered":"2024-09-10T10:03:06.157406+0000","last_clean":"2024-09-10T10:03:06.157406+0000","last_became_active":"2024-09-10T10:03:05.154015+0000","last_became_peered":"2024-09-10T10:03:05.154015+0000","last_unstale":"2024-09-10T10:03:06.157406+0000","last_undegraded":"2024-09-10T10:03:06.157406+0000","last_fullsized":"2024-09-10T10:03:06.157406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:56.609614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156912+0000","last_change":"2024-09-10T10:03:04.149363+0000","last_active":"2024-09-10T10:03:06.156912+0000","last_peered":"2024-09-10T10:03:06.156912+0000","last_clean":"2024-09-10T10:03:06.156912+0000","last_became_active":"2024-09-10T10:03:04.149107+0000","last_became_peered":"2024-09-10T10:03:04.149107+0000","last_unstale":"2024-09-10T10:03:06.156912+0000","last_undegraded":"2024-09-10T10:03:06.156912+0000","last_fullsized":"2024-09-10T10:03:06.156912+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:11:34.524613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158119+0000","last_change":"2024-09-10T10:03:05.152802+0000","last_active":"2024-09-10T10:03:06.158119+0000","last_peered":"2024-09-10T10:03:06.158119+0000","last_clean":"2024-09-10T10:03:06.158119+0000","last_became_active":"2024-09-10T10:03:05.152497+0000","last_became_peered":"2024-09-10T10:03:05.152497+0000","last_unstale":"2024-09-10T10:03:06.158119+0000","last_undegraded":"2024-09-10T10:03:06.158119+0000","last_fullsized":"2024-09-10T10:03:06.158119+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:27.289059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634345+0000","last_change":"2024-09-10T10:03:04.158156+0000","last_active":"2024-09-10T10:03:06.634345+0000","last_peered":"2024-09-10T10:03:06.634345+0000","last_clean":"2024-09-10T10:03:06.634345+0000","last_became_active":"2024-09-10T10:03:04.157858+0000","last_became_peered":"2024-09-10T10:03:04.157858+0000","last_unstale":"2024-09-10T10:03:06.634345+0000","last_undegraded":"2024-09-10T10:03:06.634345+0000","last_fullsized":"2024-09-10T10:03:06.634345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:39.617729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158685+0000","last_change":"2024-09-10T10:03:05.151283+0000","last_active":"2024-09-10T10:03:06.158685+0000","last_peered":"2024-09-10T10:03:06.158685+0000","last_clean":"2024-09-10T10:03:06.158685+0000","last_became_active":"2024-09-10T10:03:05.150980+0000","last_became_peered":"2024-09-10T10:03:05.150980+0000","last_unstale":"2024-09-10T10:03:06.158685+0000","last_undegraded":"2024-09-10T10:03:06.158685+0000","last_fullsized":"2024-09-10T10:03:06.158685+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.719505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.140411+0000","last_change":"2024-09-10T10:03:04.152342+0000","last_active":"2024-09-10T10:03:05.140411+0000","last_peered":"2024-09-10T10:03:05.140411+0000","last_clean":"2024-09-10T10:03:05.140411+0000","last_became_active":"2024-09-10T10:03:04.152008+0000","last_became_peered":"2024-09-10T10:03:04.152008+0000","last_unstale":"2024-09-10T10:03:05.140411+0000","last_undegraded":"2024-09-10T10:03:05.140411+0000","last_fullsized":"2024-09-10T10:03:05.140411+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:22.305412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158482+0000","last_change":"2024-09-10T10:03:05.162965+0000","last_active":"2024-09-10T10:03:06.158482+0000","last_peered":"2024-09-10T10:03:06.158482+0000","last_clean":"2024-09-10T10:03:06.158482+0000","last_became_active":"2024-09-10T10:03:05.162546+0000","last_became_peered":"2024-09-10T10:03:05.162546+0000","last_unstale":"2024-09-10T10:03:06.158482+0000","last_undegraded":"2024-09-10T10:03:06.158482+0000","last_fullsized":"2024-09-10T10:03:06.158482+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:59.828450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158815+0000","last_change":"2024-09-10T10:03:04.152147+0000","last_active":"2024-09-10T10:03:06.158815+0000","last_peered":"2024-09-10T10:03:06.158815+0000","last_clean":"2024-09-10T10:03:06.158815+0000","last_became_active":"2024-09-10T10:03:04.151776+0000","last_became_peered":"2024-09-10T10:03:04.151776+0000","last_unstale":"2024-09-10T10:03:06.158815+0000","last_undegraded":"2024-09-10T10:03:06.158815+0000","last_fullsized":"2024-09-10T10:03:06.158815+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:22.305412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.154432+0000","last_change":"2024-09-10T10:03:05.148105+0000","last_active":"2024-09-10T10:03:05.154432+0000","last_peered":"2024-09-10T10:03:05.154432+0000","last_clean":"2024-09-10T10:03:05.154432+0000","last_became_active":"2024-09-10T10:03:05.147839+0000","last_became_peered":"2024-09-10T10:03:05.147839+0000","last_unstale":"2024-09-10T10:03:05.154432+0000","last_undegraded":"2024-09-10T10:03:05.154432+0000","last_fullsized":"2024-09-10T10:03:05.154432+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:51.628666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.138848+0000","last_change":"2024-09-10T10:03:04.150860+0000","last_active":"2024-09-10T10:03:05.138848+0000","last_peered":"2024-09-10T10:03:05.138848+0000","last_clean":"2024-09-10T10:03:05.138848+0000","last_became_active":"2024-09-10T10:03:04.150581+0000","last_became_peered":"2024-09-10T10:03:04.150581+0000","last_unstale":"2024-09-10T10:03:05.138848+0000","last_undegraded":"2024-09-10T10:03:05.138848+0000","last_fullsized":"2024-09-10T10:03:05.138848+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:48.938577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156715+0000","last_change":"2024-09-10T10:03:05.157317+0000","last_active":"2024-09-10T10:03:06.156715+0000","last_peered":"2024-09-10T10:03:06.156715+0000","last_clean":"2024-09-10T10:03:06.156715+0000","last_became_active":"2024-09-10T10:03:05.156977+0000","last_became_peered":"2024-09-10T10:03:05.156977+0000","last_unstale":"2024-09-10T10:03:06.156715+0000","last_undegraded":"2024-09-10T10:03:06.156715+0000","last_fullsized":"2024-09-10T10:03:06.156715+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:09.740426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156895+0000","last_change":"2024-09-10T10:03:04.146461+0000","last_active":"2024-09-10T10:03:06.156895+0000","last_peered":"2024-09-10T10:03:06.156895+0000","last_clean":"2024-09-10T10:03:06.156895+0000","last_became_active":"2024-09-10T10:03:04.146196+0000","last_became_peered":"2024-09-10T10:03:04.146196+0000","last_unstale":"2024-09-10T10:03:06.156895+0000","last_undegraded":"2024-09-10T10:03:06.156895+0000","last_fullsized":"2024-09-10T10:03:06.156895+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:23.917121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634004+0000","last_change":"2024-09-10T10:03:05.147095+0000","last_active":"2024-09-10T10:03:06.634004+0000","last_peered":"2024-09-10T10:03:06.634004+0000","last_clean":"2024-09-10T10:03:06.634004+0000","last_became_active":"2024-09-10T10:03:05.146811+0000","last_became_peered":"2024-09-10T10:03:05.146811+0000","last_unstale":"2024-09-10T10:03:06.634004+0000","last_undegraded":"2024-09-10T10:03:06.634004+0000","last_fullsized":"2024-09-10T10:03:06.634004+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:17.098721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.140487+0000","last_change":"2024-09-10T10:03:04.151026+0000","last_active":"2024-09-10T10:03:05.140487+0000","last_peered":"2024-09-10T10:03:05.140487+0000","last_clean":"2024-09-10T10:03:05.140487+0000","last_became_active":"2024-09-10T10:03:04.150687+0000","last_became_peered":"2024-09-10T10:03:04.150687+0000","last_unstale":"2024-09-10T10:03:05.140487+0000","last_undegraded":"2024-09-10T10:03:05.140487+0000","last_fullsized":"2024-09-10T10:03:05.140487+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:33.863849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":354,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.139306+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:03:05.139306+0000","last_peered":"2024-09-10T10:03:05.139306+0000","last_clean":"2024-09-10T10:03:05.139306+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:03:05.139306+0000","last_undegraded":"2024-09-10T10:03:05.139306+0000","last_fullsized":"2024-09-10T10:03:05.139306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1863680,"data_stored":1141021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1265664,"data_stored":550107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1269760,"data_stored":550641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1273856,"data_stored":556337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542207,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1277952,"data_stored":551719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1261568,"data_stored":550037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":550161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":1148429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:03:09.243 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:03:09.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.411+0000 7fcf96200640 1 Processor -- start 2024-09-10T10:03:09.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.411+0000 7fcf96200640 1 -- start start 2024-09-10T10:03:09.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.411+0000 7fcf96200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf900cfe30 0x7fcf90071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf96200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf9006d2d0 con 0x7fcf9006e7a0 2024-09-10T10:03:09.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf96200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf9006d5b0 con 0x7fcf900cd510 2024-09-10T10:03:09.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf96200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9015d9a0 con 0x7fcf900cfe30 2024-09-10T10:03:09.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8fe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcf9006e7a0 0x7fcf9015cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52052/0 (socket says 172.21.15.110:52052) 2024-09-10T10:03:09.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8fe00640 1 -- 172.21.15.110:0/2644902059 learned_addr learned my addr 172.21.15.110:0/2644902059 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8f400640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf900cfe30 0x7fcf90071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2496068020 0 0) 0x7fcf9006d2d0 con 0x7fcf9006e7a0 2024-09-10T10:03:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf74003530 con 0x7fcf9006e7a0 2024-09-10T10:03:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2482429402 0 0) 0x7fcf74003530 con 0x7fcf9006e7a0 2024-09-10T10:03:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf900cfe30 msgr2=0x7fcf90071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:09.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf900cfe30 0x7fcf90071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 >> v1:172.21.15.145:6789/0 conn(0x7fcf900cd510 legacy=0x7fcf9006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9006d890 con 0x7fcf9006e7a0 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fcf88001a00 con 0x7fcf9006e7a0 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 >> v1:172.21.15.110:6789/0 conn(0x7fcf9006e7a0 legacy=0x7fcf9015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf740039f0 0x7fcf74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf900cfe30 0x7fcf74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf74004d50 0x7fcf740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9015d9a0 con 0x7fcf740039f0 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7400ba80 con 0x7fcf74004d50 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7400bc80 con 0x7fcf900cfe30 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8f400640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf740039f0 0x7fcf74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf87e00640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf900cfe30 0x7fcf74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8fe00640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf74004d50 0x7fcf740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8f400640 1 -- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf900cfe30 msgr2=0x7fcf74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8f400640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf900cfe30 0x7fcf74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8f400640 1 -- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf74004d50 msgr2=0x7fcf740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8f400640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf74004d50 0x7fcf740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8f400640 1 -- 172.21.15.110:0/2644902059 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf74003530 con 0x7fcf740039f0 2024-09-10T10:03:09.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.415+0000 7fcf8fe00640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf74004d50 0x7fcf740051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf8f400640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf740039f0 0x7fcf74003dd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009ae0 tx=0x7fcf8001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1612fcfe1a5b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf800b2070 con 0x7fcf740039f0 2024-09-10T10:03:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf800a5550 con 0x7fcf740039f0 2024-09-10T10:03:09.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf8ea00640 1 -- 172.21.15.110:0/2644902059 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf800096b0 con 0x7fcf740039f0 2024-09-10T10:03:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf96200640 1 -- 172.21.15.110:0/2644902059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf740039f0 msgr2=0x7fcf74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf96200640 1 --2- 172.21.15.110:0/2644902059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf740039f0 0x7fcf74003dd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009ae0 tx=0x7fcf8001ca70 comp rx=0 tx=0).stop 2024-09-10T10:03:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf8fe00640 1 -- 172.21.15.110:0/2644902059 reap_dead start 2024-09-10T10:03:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf96200640 1 -- 172.21.15.110:0/2644902059 shutdown_connections 2024-09-10T10:03:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf96200640 1 -- 172.21.15.110:0/2644902059 >> 172.21.15.110:0/2644902059 conn(0x7fcf9006bc50 msgr2=0x7fcf9006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf96200640 1 -- 172.21.15.110:0/2644902059 shutdown_connections 2024-09-10T10:03:09.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf96200640 1 -- 172.21.15.110:0/2644902059 wait complete. 2024-09-10T10:03:09.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf96200640 1 Processor -- start 2024-09-10T10:03:09.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.419+0000 7fcf96200640 1 -- start start 2024-09-10T10:03:09.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf740039f0 0x7fcf90159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf90151770 0x7fcf90158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf90158560 0x7fcf90152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90168510 con 0x7fcf90151770 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90168390 con 0x7fcf740039f0 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf90168690 con 0x7fcf90158560 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf90151770 0x7fcf90158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf90151770 0x7fcf90158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54700/0 (socket says 172.21.15.110:54700) 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 -- 172.21.15.110:0/2434502850 learned_addr learned my addr 172.21.15.110:0/2434502850 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf8fe00640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf90158560 0x7fcf90152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf8f400640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf740039f0 0x7fcf90159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 -- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf90158560 msgr2=0x7fcf90152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf90158560 0x7fcf90152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 -- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf740039f0 msgr2=0x7fcf90159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf740039f0 0x7fcf90159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 -- 172.21.15.110:0/2434502850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90158a70 con 0x7fcf90151770 2024-09-10T10:03:09.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87e00640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf90151770 0x7fcf90158020 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800f770 tx=0x7fcf7800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87400640 1 -- 172.21.15.110:0/2434502850 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800fe80 con 0x7fcf90151770 2024-09-10T10:03:09.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87400640 1 -- 172.21.15.110:0/2434502850 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78018070 con 0x7fcf90151770 2024-09-10T10:03:09.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf90158d50 con 0x7fcf90151770 2024-09-10T10:03:09.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87400640 1 -- 172.21.15.110:0/2434502850 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800d6d0 con 0x7fcf90151770 2024-09-10T10:03:09.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf90159310 con 0x7fcf90151770 2024-09-10T10:03:09.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9015cc20 con 0x7fcf90151770 2024-09-10T10:03:09.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.423+0000 7fcf87400640 1 -- 172.21.15.110:0/2434502850 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fcf7800d870 con 0x7fcf90151770 2024-09-10T10:03:09.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.427+0000 7fcf87400640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf6c067a10 0x7fcf6c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.427+0000 7fcf87400640 1 -- 172.21.15.110:0/2434502850 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fcf78085d00 con 0x7fcf90151770 2024-09-10T10:03:09.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.427+0000 7fcf8f400640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf6c067a10 0x7fcf6c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.427+0000 7fcf8f400640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf6c067a10 0x7fcf6c069ed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf800af000 tx=0x7fcf800046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.431+0000 7fcf87400640 1 -- 172.21.15.110:0/2434502850 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcf78056690 con 0x7fcf90151770 2024-09-10T10:03:09.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.583+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf9015a050 con 0x7fcf6c067a10 2024-09-10T10:03:09.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.591+0000 7fcf87400640 1 -- 172.21.15.110:0/2434502850 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252453 (secure 0 0 0) 0x7fcf800096b0 con 0x7fcf6c067a10 2024-09-10T10:03:09.596 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:09.598 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf6c067a10 msgr2=0x7fcf6c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf6c067a10 0x7fcf6c069ed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf800af000 tx=0x7fcf800046d0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf90151770 msgr2=0x7fcf90158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf90151770 0x7fcf90158020 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800f770 tx=0x7fcf7800fc30 comp rx=0 tx=0).stop 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 shutdown_connections 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf6c067a10 0x7fcf6c069ed0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf90158560 0x7fcf90152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf90151770 0x7fcf90158020 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 --2- 172.21.15.110:0/2434502850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf740039f0 0x7fcf90159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 >> 172.21.15.110:0/2434502850 conn(0x7fcf9006bc50 msgr2=0x7fcf90167a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:09.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 shutdown_connections 2024-09-10T10:03:09.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.595+0000 7fcf96200640 1 -- 172.21.15.110:0/2434502850 wait complete. 2024-09-10T10:03:09.617 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-10T10:03:07.701889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264800,"kb_used_data":11104,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709920,"statfs":{"total":773094113280,"available":772822958080,"internally_reserved":0,"allocated":11370496,"data_stored":5598452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":259575340},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.580114"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157015+0000","last_change":"2024-09-10T10:03:05.153139+0000","last_active":"2024-09-10T10:03:06.157015+0000","last_peered":"2024-09-10T10:03:06.157015+0000","last_clean":"2024-09-10T10:03:06.157015+0000","last_became_active":"2024-09-10T10:03:05.152819+0000","last_became_peered":"2024-09-10T10:03:05.152819+0000","last_unstale":"2024-09-10T10:03:06.157015+0000","last_undegraded":"2024-09-10T10:03:06.157015+0000","last_fullsized":"2024-09-10T10:03:06.157015+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:03.568760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158495+0000","last_change":"2024-09-10T10:03:05.153077+0000","last_active":"2024-09-10T10:03:06.158495+0000","last_peered":"2024-09-10T10:03:06.158495+0000","last_clean":"2024-09-10T10:03:06.158495+0000","last_became_active":"2024-09-10T10:03:05.152814+0000","last_became_peered":"2024-09-10T10:03:05.152814+0000","last_unstale":"2024-09-10T10:03:06.158495+0000","last_undegraded":"2024-09-10T10:03:06.158495+0000","last_fullsized":"2024-09-10T10:03:06.158495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:55.040526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.160932+0000","last_change":"2024-09-10T10:03:05.153608+0000","last_active":"2024-09-10T10:03:05.160932+0000","last_peered":"2024-09-10T10:03:05.160932+0000","last_clean":"2024-09-10T10:03:05.160932+0000","last_became_active":"2024-09-10T10:03:05.152587+0000","last_became_peered":"2024-09-10T10:03:05.152587+0000","last_unstale":"2024-09-10T10:03:05.160932+0000","last_undegraded":"2024-09-10T10:03:05.160932+0000","last_fullsized":"2024-09-10T10:03:05.160932+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:59.828450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157261+0000","last_change":"2024-09-10T10:03:05.164865+0000","last_active":"2024-09-10T10:03:06.157261+0000","last_peered":"2024-09-10T10:03:06.157261+0000","last_clean":"2024-09-10T10:03:06.157261+0000","last_became_active":"2024-09-10T10:03:05.164299+0000","last_became_peered":"2024-09-10T10:03:05.164299+0000","last_unstale":"2024-09-10T10:03:06.157261+0000","last_undegraded":"2024-09-10T10:03:06.157261+0000","last_fullsized":"2024-09-10T10:03:06.157261+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:25.203158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.168132+0000","last_change":"2024-09-10T10:03:05.161405+0000","last_active":"2024-09-10T10:03:05.168132+0000","last_peered":"2024-09-10T10:03:05.168132+0000","last_clean":"2024-09-10T10:03:05.168132+0000","last_became_active":"2024-09-10T10:03:05.158623+0000","last_became_peered":"2024-09-10T10:03:05.158623+0000","last_unstale":"2024-09-10T10:03:05.168132+0000","last_undegraded":"2024-09-10T10:03:05.168132+0000","last_fullsized":"2024-09-10T10:03:05.168132+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:37.615059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.157725+0000","last_change":"2024-09-10T10:03:05.155356+0000","last_active":"2024-09-10T10:03:05.157725+0000","last_peered":"2024-09-10T10:03:05.157725+0000","last_clean":"2024-09-10T10:03:05.157725+0000","last_became_active":"2024-09-10T10:03:05.155084+0000","last_became_peered":"2024-09-10T10:03:05.155084+0000","last_unstale":"2024-09-10T10:03:05.157725+0000","last_undegraded":"2024-09-10T10:03:05.157725+0000","last_fullsized":"2024-09-10T10:03:05.157725+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:05.551493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.158838+0000","last_change":"2024-09-10T10:03:05.152778+0000","last_active":"2024-09-10T10:03:05.158838+0000","last_peered":"2024-09-10T10:03:05.158838+0000","last_clean":"2024-09-10T10:03:05.158838+0000","last_became_active":"2024-09-10T10:03:05.151979+0000","last_became_peered":"2024-09-10T10:03:05.151979+0000","last_unstale":"2024-09-10T10:03:05.158838+0000","last_undegraded":"2024-09-10T10:03:05.158838+0000","last_fullsized":"2024-09-10T10:03:05.158838+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:39.875964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.161038+0000","last_change":"2024-09-10T10:03:05.154560+0000","last_active":"2024-09-10T10:03:05.161038+0000","last_peered":"2024-09-10T10:03:05.161038+0000","last_clean":"2024-09-10T10:03:05.161038+0000","last_became_active":"2024-09-10T10:03:05.154200+0000","last_became_peered":"2024-09-10T10:03:05.154200+0000","last_unstale":"2024-09-10T10:03:05.161038+0000","last_undegraded":"2024-09-10T10:03:05.161038+0000","last_fullsized":"2024-09-10T10:03:05.161038+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:01.258634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157156+0000","last_change":"2024-09-10T10:03:05.170093+0000","last_active":"2024-09-10T10:03:06.157156+0000","last_peered":"2024-09-10T10:03:06.157156+0000","last_clean":"2024-09-10T10:03:06.157156+0000","last_became_active":"2024-09-10T10:03:05.169820+0000","last_became_peered":"2024-09-10T10:03:05.169820+0000","last_unstale":"2024-09-10T10:03:06.157156+0000","last_undegraded":"2024-09-10T10:03:06.157156+0000","last_fullsized":"2024-09-10T10:03:06.157156+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:12:38.718259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157684+0000","last_change":"2024-09-10T10:03:05.171648+0000","last_active":"2024-09-10T10:03:06.157684+0000","last_peered":"2024-09-10T10:03:06.157684+0000","last_clean":"2024-09-10T10:03:06.157684+0000","last_became_active":"2024-09-10T10:03:05.168315+0000","last_became_peered":"2024-09-10T10:03:05.168315+0000","last_unstale":"2024-09-10T10:03:06.157684+0000","last_undegraded":"2024-09-10T10:03:06.157684+0000","last_fullsized":"2024-09-10T10:03:06.157684+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:01.892673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158776+0000","last_change":"2024-09-10T10:03:05.156448+0000","last_active":"2024-09-10T10:03:06.158776+0000","last_peered":"2024-09-10T10:03:06.158776+0000","last_clean":"2024-09-10T10:03:06.158776+0000","last_became_active":"2024-09-10T10:03:05.156135+0000","last_became_peered":"2024-09-10T10:03:05.156135+0000","last_unstale":"2024-09-10T10:03:06.158776+0000","last_undegraded":"2024-09-10T10:03:06.158776+0000","last_fullsized":"2024-09-10T10:03:06.158776+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:23.566054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157282+0000","last_change":"2024-09-10T10:03:05.170099+0000","last_active":"2024-09-10T10:03:06.157282+0000","last_peered":"2024-09-10T10:03:06.157282+0000","last_clean":"2024-09-10T10:03:06.157282+0000","last_became_active":"2024-09-10T10:03:05.169819+0000","last_became_peered":"2024-09-10T10:03:05.169819+0000","last_unstale":"2024-09-10T10:03:06.157282+0000","last_undegraded":"2024-09-10T10:03:06.157282+0000","last_fullsized":"2024-09-10T10:03:06.157282+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:16.369813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.161110+0000","last_change":"2024-09-10T10:03:05.155436+0000","last_active":"2024-09-10T10:03:05.161110+0000","last_peered":"2024-09-10T10:03:05.161110+0000","last_clean":"2024-09-10T10:03:05.161110+0000","last_became_active":"2024-09-10T10:03:05.155126+0000","last_became_peered":"2024-09-10T10:03:05.155126+0000","last_unstale":"2024-09-10T10:03:05.161110+0000","last_undegraded":"2024-09-10T10:03:05.161110+0000","last_fullsized":"2024-09-10T10:03:05.161110+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:23.566054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.158349+0000","last_change":"2024-09-10T10:03:05.151278+0000","last_active":"2024-09-10T10:03:05.158349+0000","last_peered":"2024-09-10T10:03:05.158349+0000","last_clean":"2024-09-10T10:03:05.158349+0000","last_became_active":"2024-09-10T10:03:05.150010+0000","last_became_peered":"2024-09-10T10:03:05.150010+0000","last_unstale":"2024-09-10T10:03:05.158349+0000","last_undegraded":"2024-09-10T10:03:05.158349+0000","last_fullsized":"2024-09-10T10:03:05.158349+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:05.657996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.156080+0000","last_change":"2024-09-10T10:03:05.152255+0000","last_active":"2024-09-10T10:03:05.156080+0000","last_peered":"2024-09-10T10:03:05.156080+0000","last_clean":"2024-09-10T10:03:05.156080+0000","last_became_active":"2024-09-10T10:03:05.151987+0000","last_became_peered":"2024-09-10T10:03:05.151987+0000","last_unstale":"2024-09-10T10:03:05.156080+0000","last_undegraded":"2024-09-10T10:03:05.156080+0000","last_fullsized":"2024-09-10T10:03:05.156080+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:43.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156540+0000","last_change":"2024-09-10T10:03:05.172860+0000","last_active":"2024-09-10T10:03:06.156540+0000","last_peered":"2024-09-10T10:03:06.156540+0000","last_clean":"2024-09-10T10:03:06.156540+0000","last_became_active":"2024-09-10T10:03:05.172220+0000","last_became_peered":"2024-09-10T10:03:05.172220+0000","last_unstale":"2024-09-10T10:03:06.156540+0000","last_undegraded":"2024-09-10T10:03:06.156540+0000","last_fullsized":"2024-09-10T10:03:06.156540+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:53.040198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158940+0000","last_change":"2024-09-10T10:03:05.156392+0000","last_active":"2024-09-10T10:03:06.158940+0000","last_peered":"2024-09-10T10:03:06.158940+0000","last_clean":"2024-09-10T10:03:06.158940+0000","last_became_active":"2024-09-10T10:03:05.156118+0000","last_became_peered":"2024-09-10T10:03:05.156118+0000","last_unstale":"2024-09-10T10:03:06.158940+0000","last_undegraded":"2024-09-10T10:03:06.158940+0000","last_fullsized":"2024-09-10T10:03:06.158940+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:16.016706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157923+0000","last_change":"2024-09-10T10:03:05.160880+0000","last_active":"2024-09-10T10:03:06.157923+0000","last_peered":"2024-09-10T10:03:06.157923+0000","last_clean":"2024-09-10T10:03:06.157923+0000","last_became_active":"2024-09-10T10:03:05.160458+0000","last_became_peered":"2024-09-10T10:03:05.160458+0000","last_unstale":"2024-09-10T10:03:06.157923+0000","last_undegraded":"2024-09-10T10:03:06.157923+0000","last_fullsized":"2024-09-10T10:03:06.157923+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:16.016706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.160954+0000","last_change":"2024-09-10T10:03:05.153863+0000","last_active":"2024-09-10T10:03:05.160954+0000","last_peered":"2024-09-10T10:03:05.160954+0000","last_clean":"2024-09-10T10:03:05.160954+0000","last_became_active":"2024-09-10T10:03:05.153001+0000","last_became_peered":"2024-09-10T10:03:05.153001+0000","last_unstale":"2024-09-10T10:03:05.160954+0000","last_undegraded":"2024-09-10T10:03:05.160954+0000","last_fullsized":"2024-09-10T10:03:05.160954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:55.040526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158753+0000","last_change":"2024-09-10T10:03:05.155437+0000","last_active":"2024-09-10T10:03:06.158753+0000","last_peered":"2024-09-10T10:03:06.158753+0000","last_clean":"2024-09-10T10:03:06.158753+0000","last_became_active":"2024-09-10T10:03:05.155145+0000","last_became_peered":"2024-09-10T10:03:05.155145+0000","last_unstale":"2024-09-10T10:03:06.158753+0000","last_undegraded":"2024-09-10T10:03:06.158753+0000","last_fullsized":"2024-09-10T10:03:06.158753+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:01.258634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.159478+0000","last_change":"2024-09-10T10:03:05.157161+0000","last_active":"2024-09-10T10:03:06.159478+0000","last_peered":"2024-09-10T10:03:06.159478+0000","last_clean":"2024-09-10T10:03:06.159478+0000","last_became_active":"2024-09-10T10:03:05.156832+0000","last_became_peered":"2024-09-10T10:03:05.156832+0000","last_unstale":"2024-09-10T10:03:06.159478+0000","last_undegraded":"2024-09-10T10:03:06.159478+0000","last_fullsized":"2024-09-10T10:03:06.159478+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:45.445940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634324+0000","last_change":"2024-09-10T10:03:05.145571+0000","last_active":"2024-09-10T10:03:06.634324+0000","last_peered":"2024-09-10T10:03:06.634324+0000","last_clean":"2024-09-10T10:03:06.634324+0000","last_became_active":"2024-09-10T10:03:05.145283+0000","last_became_peered":"2024-09-10T10:03:05.145283+0000","last_unstale":"2024-09-10T10:03:06.634324+0000","last_undegraded":"2024-09-10T10:03:06.634324+0000","last_fullsized":"2024-09-10T10:03:06.634324+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:15.640226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158043+0000","last_change":"2024-09-10T10:03:05.158224+0000","last_active":"2024-09-10T10:03:06.158043+0000","last_peered":"2024-09-10T10:03:06.158043+0000","last_clean":"2024-09-10T10:03:06.158043+0000","last_became_active":"2024-09-10T10:03:05.157697+0000","last_became_peered":"2024-09-10T10:03:05.157697+0000","last_unstale":"2024-09-10T10:03:06.158043+0000","last_undegraded":"2024-09-10T10:03:06.158043+0000","last_fullsized":"2024-09-10T10:03:06.158043+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:51.628666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634090+0000","last_change":"2024-09-10T10:03:05.150113+0000","last_active":"2024-09-10T10:03:06.634090+0000","last_peered":"2024-09-10T10:03:06.634090+0000","last_clean":"2024-09-10T10:03:06.634090+0000","last_became_active":"2024-09-10T10:03:05.149844+0000","last_became_peered":"2024-09-10T10:03:05.149844+0000","last_unstale":"2024-09-10T10:03:06.634090+0000","last_undegraded":"2024-09-10T10:03:06.634090+0000","last_fullsized":"2024-09-10T10:03:06.634090+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:49.366205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.159027+0000","last_change":"2024-09-10T10:03:05.161669+0000","last_active":"2024-09-10T10:03:06.159027+0000","last_peered":"2024-09-10T10:03:06.159027+0000","last_clean":"2024-09-10T10:03:06.159027+0000","last_became_active":"2024-09-10T10:03:05.161324+0000","last_became_peered":"2024-09-10T10:03:05.161324+0000","last_unstale":"2024-09-10T10:03:06.159027+0000","last_undegraded":"2024-09-10T10:03:06.159027+0000","last_fullsized":"2024-09-10T10:03:06.159027+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:27.289059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157825+0000","last_change":"2024-09-10T10:03:05.172663+0000","last_active":"2024-09-10T10:03:06.157825+0000","last_peered":"2024-09-10T10:03:06.157825+0000","last_clean":"2024-09-10T10:03:06.157825+0000","last_became_active":"2024-09-10T10:03:05.172417+0000","last_became_peered":"2024-09-10T10:03:05.172417+0000","last_unstale":"2024-09-10T10:03:06.157825+0000","last_undegraded":"2024-09-10T10:03:06.157825+0000","last_fullsized":"2024-09-10T10:03:06.157825+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:11.523586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.170947+0000","last_change":"2024-09-10T10:03:05.167884+0000","last_active":"2024-09-10T10:03:05.170947+0000","last_peered":"2024-09-10T10:03:05.170947+0000","last_clean":"2024-09-10T10:03:05.170947+0000","last_became_active":"2024-09-10T10:03:05.166536+0000","last_became_peered":"2024-09-10T10:03:05.166536+0000","last_unstale":"2024-09-10T10:03:05.170947+0000","last_undegraded":"2024-09-10T10:03:05.170947+0000","last_fullsized":"2024-09-10T10:03:05.170947+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:30.151714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634244+0000","last_change":"2024-09-10T10:03:05.168406+0000","last_active":"2024-09-10T10:03:06.634244+0000","last_peered":"2024-09-10T10:03:06.634244+0000","last_clean":"2024-09-10T10:03:06.634244+0000","last_became_active":"2024-09-10T10:03:05.168051+0000","last_became_peered":"2024-09-10T10:03:05.168051+0000","last_unstale":"2024-09-10T10:03:06.634244+0000","last_undegraded":"2024-09-10T10:03:06.634244+0000","last_fullsized":"2024-09-10T10:03:06.634244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:12.424333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158133+0000","last_change":"2024-09-10T10:03:05.157033+0000","last_active":"2024-09-10T10:03:06.158133+0000","last_peered":"2024-09-10T10:03:06.158133+0000","last_clean":"2024-09-10T10:03:06.158133+0000","last_became_active":"2024-09-10T10:03:05.156466+0000","last_became_peered":"2024-09-10T10:03:05.156466+0000","last_unstale":"2024-09-10T10:03:06.158133+0000","last_undegraded":"2024-09-10T10:03:06.158133+0000","last_fullsized":"2024-09-10T10:03:06.158133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:15.379112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.167829+0000","last_change":"2024-09-10T10:03:05.162355+0000","last_active":"2024-09-10T10:03:05.167829+0000","last_peered":"2024-09-10T10:03:05.167829+0000","last_clean":"2024-09-10T10:03:05.167829+0000","last_became_active":"2024-09-10T10:03:05.162081+0000","last_became_peered":"2024-09-10T10:03:05.162081+0000","last_unstale":"2024-09-10T10:03:05.167829+0000","last_undegraded":"2024-09-10T10:03:05.167829+0000","last_fullsized":"2024-09-10T10:03:05.167829+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:51.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157021+0000","last_change":"2024-09-10T10:03:05.174547+0000","last_active":"2024-09-10T10:03:06.157021+0000","last_peered":"2024-09-10T10:03:06.157021+0000","last_clean":"2024-09-10T10:03:06.157021+0000","last_became_active":"2024-09-10T10:03:05.174092+0000","last_became_peered":"2024-09-10T10:03:05.174092+0000","last_unstale":"2024-09-10T10:03:06.157021+0000","last_undegraded":"2024-09-10T10:03:06.157021+0000","last_fullsized":"2024-09-10T10:03:06.157021+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:50.034187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.157727+0000","last_change":"2024-09-10T10:03:05.156000+0000","last_active":"2024-09-10T10:03:05.157727+0000","last_peered":"2024-09-10T10:03:05.157727+0000","last_clean":"2024-09-10T10:03:05.157727+0000","last_became_active":"2024-09-10T10:03:05.155710+0000","last_became_peered":"2024-09-10T10:03:05.155710+0000","last_unstale":"2024-09-10T10:03:05.157727+0000","last_undegraded":"2024-09-10T10:03:05.157727+0000","last_fullsized":"2024-09-10T10:03:05.157727+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:32.646817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.167957+0000","last_change":"2024-09-10T10:03:05.157669+0000","last_active":"2024-09-10T10:03:05.167957+0000","last_peered":"2024-09-10T10:03:05.167957+0000","last_clean":"2024-09-10T10:03:05.167957+0000","last_became_active":"2024-09-10T10:03:05.157222+0000","last_became_peered":"2024-09-10T10:03:05.157222+0000","last_unstale":"2024-09-10T10:03:05.167957+0000","last_undegraded":"2024-09-10T10:03:05.167957+0000","last_fullsized":"2024-09-10T10:03:05.167957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:15.418491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.168342+0000","last_change":"2024-09-10T10:03:04.149739+0000","last_active":"2024-09-10T10:03:05.168342+0000","last_peered":"2024-09-10T10:03:05.168342+0000","last_clean":"2024-09-10T10:03:05.168342+0000","last_became_active":"2024-09-10T10:03:04.149240+0000","last_became_peered":"2024-09-10T10:03:04.149240+0000","last_unstale":"2024-09-10T10:03:05.168342+0000","last_undegraded":"2024-09-10T10:03:05.168342+0000","last_fullsized":"2024-09-10T10:03:05.168342+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:12.890195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.159130+0000","last_change":"2024-09-10T10:03:05.156642+0000","last_active":"2024-09-10T10:03:06.159130+0000","last_peered":"2024-09-10T10:03:06.159130+0000","last_clean":"2024-09-10T10:03:06.159130+0000","last_became_active":"2024-09-10T10:03:05.155830+0000","last_became_peered":"2024-09-10T10:03:05.155830+0000","last_unstale":"2024-09-10T10:03:06.159130+0000","last_undegraded":"2024-09-10T10:03:06.159130+0000","last_fullsized":"2024-09-10T10:03:06.159130+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:57.949604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.163115+0000","last_change":"2024-09-10T10:03:04.151632+0000","last_active":"2024-09-10T10:03:05.163115+0000","last_peered":"2024-09-10T10:03:05.163115+0000","last_clean":"2024-09-10T10:03:05.163115+0000","last_became_active":"2024-09-10T10:03:04.151278+0000","last_became_peered":"2024-09-10T10:03:04.151278+0000","last_unstale":"2024-09-10T10:03:05.163115+0000","last_undegraded":"2024-09-10T10:03:05.163115+0000","last_fullsized":"2024-09-10T10:03:05.163115+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:30.731113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.172361+0000","last_change":"2024-09-10T10:03:05.169522+0000","last_active":"2024-09-10T10:03:05.172361+0000","last_peered":"2024-09-10T10:03:05.172361+0000","last_clean":"2024-09-10T10:03:05.172361+0000","last_became_active":"2024-09-10T10:03:05.169100+0000","last_became_peered":"2024-09-10T10:03:05.169100+0000","last_unstale":"2024-09-10T10:03:05.172361+0000","last_undegraded":"2024-09-10T10:03:05.172361+0000","last_fullsized":"2024-09-10T10:03:05.172361+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:26.001196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.173210+0000","last_change":"2024-09-10T10:03:04.153849+0000","last_active":"2024-09-10T10:03:05.173210+0000","last_peered":"2024-09-10T10:03:05.173210+0000","last_clean":"2024-09-10T10:03:05.173210+0000","last_became_active":"2024-09-10T10:03:04.153483+0000","last_became_peered":"2024-09-10T10:03:04.153483+0000","last_unstale":"2024-09-10T10:03:05.173210+0000","last_undegraded":"2024-09-10T10:03:05.173210+0000","last_fullsized":"2024-09-10T10:03:05.173210+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:14.386747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.160972+0000","last_change":"2024-09-10T10:03:05.154091+0000","last_active":"2024-09-10T10:03:05.160972+0000","last_peered":"2024-09-10T10:03:05.160972+0000","last_clean":"2024-09-10T10:03:05.160972+0000","last_became_active":"2024-09-10T10:03:05.153367+0000","last_became_peered":"2024-09-10T10:03:05.153367+0000","last_unstale":"2024-09-10T10:03:05.160972+0000","last_undegraded":"2024-09-10T10:03:05.160972+0000","last_fullsized":"2024-09-10T10:03:05.160972+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.719505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.172984+0000","last_change":"2024-09-10T10:03:04.155558+0000","last_active":"2024-09-10T10:03:05.172984+0000","last_peered":"2024-09-10T10:03:05.172984+0000","last_clean":"2024-09-10T10:03:05.172984+0000","last_became_active":"2024-09-10T10:03:04.155306+0000","last_became_peered":"2024-09-10T10:03:04.155306+0000","last_unstale":"2024-09-10T10:03:05.172984+0000","last_undegraded":"2024-09-10T10:03:05.172984+0000","last_fullsized":"2024-09-10T10:03:05.172984+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:36.454870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156730+0000","last_change":"2024-09-10T10:03:05.173577+0000","last_active":"2024-09-10T10:03:06.156730+0000","last_peered":"2024-09-10T10:03:06.156730+0000","last_clean":"2024-09-10T10:03:06.156730+0000","last_became_active":"2024-09-10T10:03:05.170423+0000","last_became_peered":"2024-09-10T10:03:05.170423+0000","last_unstale":"2024-09-10T10:03:06.156730+0000","last_undegraded":"2024-09-10T10:03:06.156730+0000","last_fullsized":"2024-09-10T10:03:06.156730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:16.207967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.170331+0000","last_change":"2024-09-10T10:03:04.156743+0000","last_active":"2024-09-10T10:03:05.170331+0000","last_peered":"2024-09-10T10:03:05.170331+0000","last_clean":"2024-09-10T10:03:05.170331+0000","last_became_active":"2024-09-10T10:03:04.156471+0000","last_became_peered":"2024-09-10T10:03:04.156471+0000","last_unstale":"2024-09-10T10:03:05.170331+0000","last_undegraded":"2024-09-10T10:03:05.170331+0000","last_fullsized":"2024-09-10T10:03:05.170331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:40.063337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158299+0000","last_change":"2024-09-10T10:03:05.153846+0000","last_active":"2024-09-10T10:03:06.158299+0000","last_peered":"2024-09-10T10:03:06.158299+0000","last_clean":"2024-09-10T10:03:06.158299+0000","last_became_active":"2024-09-10T10:03:05.153586+0000","last_became_peered":"2024-09-10T10:03:05.153586+0000","last_unstale":"2024-09-10T10:03:06.158299+0000","last_undegraded":"2024-09-10T10:03:06.158299+0000","last_fullsized":"2024-09-10T10:03:06.158299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:05.657996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634354+0000","last_change":"2024-09-10T10:03:04.149728+0000","last_active":"2024-09-10T10:03:06.634354+0000","last_peered":"2024-09-10T10:03:06.634354+0000","last_clean":"2024-09-10T10:03:06.634354+0000","last_became_active":"2024-09-10T10:03:04.149405+0000","last_became_peered":"2024-09-10T10:03:04.149405+0000","last_unstale":"2024-09-10T10:03:06.634354+0000","last_undegraded":"2024-09-10T10:03:06.634354+0000","last_fullsized":"2024-09-10T10:03:06.634354+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:25.651067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156546+0000","last_change":"2024-09-10T10:03:05.154294+0000","last_active":"2024-09-10T10:03:06.156546+0000","last_peered":"2024-09-10T10:03:06.156546+0000","last_clean":"2024-09-10T10:03:06.156546+0000","last_became_active":"2024-09-10T10:03:05.154003+0000","last_became_peered":"2024-09-10T10:03:05.154003+0000","last_unstale":"2024-09-10T10:03:06.156546+0000","last_undegraded":"2024-09-10T10:03:06.156546+0000","last_fullsized":"2024-09-10T10:03:06.156546+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:33.308961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158155+0000","last_change":"2024-09-10T10:03:04.149416+0000","last_active":"2024-09-10T10:03:06.158155+0000","last_peered":"2024-09-10T10:03:06.158155+0000","last_clean":"2024-09-10T10:03:06.158155+0000","last_became_active":"2024-09-10T10:03:04.149110+0000","last_became_peered":"2024-09-10T10:03:04.149110+0000","last_unstale":"2024-09-10T10:03:06.158155+0000","last_undegraded":"2024-09-10T10:03:06.158155+0000","last_fullsized":"2024-09-10T10:03:06.158155+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:33.863849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.158860+0000","last_change":"2024-09-10T10:03:05.153003+0000","last_active":"2024-09-10T10:03:05.158860+0000","last_peered":"2024-09-10T10:03:05.158860+0000","last_clean":"2024-09-10T10:03:05.158860+0000","last_became_active":"2024-09-10T10:03:05.152509+0000","last_became_peered":"2024-09-10T10:03:05.152509+0000","last_unstale":"2024-09-10T10:03:05.158860+0000","last_undegraded":"2024-09-10T10:03:05.158860+0000","last_fullsized":"2024-09-10T10:03:05.158860+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:26.850490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156831+0000","last_change":"2024-09-10T10:03:04.158095+0000","last_active":"2024-09-10T10:03:06.156831+0000","last_peered":"2024-09-10T10:03:06.156831+0000","last_clean":"2024-09-10T10:03:06.156831+0000","last_became_active":"2024-09-10T10:03:04.157806+0000","last_became_peered":"2024-09-10T10:03:04.157806+0000","last_unstale":"2024-09-10T10:03:06.156831+0000","last_undegraded":"2024-09-10T10:03:06.156831+0000","last_fullsized":"2024-09-10T10:03:06.156831+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:20:48.641017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.164819+0000","last_change":"2024-09-10T10:03:05.156751+0000","last_active":"2024-09-10T10:03:05.164819+0000","last_peered":"2024-09-10T10:03:05.164819+0000","last_clean":"2024-09-10T10:03:05.164819+0000","last_became_active":"2024-09-10T10:03:05.156488+0000","last_became_peered":"2024-09-10T10:03:05.156488+0000","last_unstale":"2024-09-10T10:03:05.164819+0000","last_undegraded":"2024-09-10T10:03:05.164819+0000","last_fullsized":"2024-09-10T10:03:05.164819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:18.871682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.138753+0000","last_change":"2024-09-10T10:03:04.153843+0000","last_active":"2024-09-10T10:03:05.138753+0000","last_peered":"2024-09-10T10:03:05.138753+0000","last_clean":"2024-09-10T10:03:05.138753+0000","last_became_active":"2024-09-10T10:03:04.153565+0000","last_became_peered":"2024-09-10T10:03:04.153565+0000","last_unstale":"2024-09-10T10:03:05.138753+0000","last_undegraded":"2024-09-10T10:03:05.138753+0000","last_fullsized":"2024-09-10T10:03:05.138753+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:36.813763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.159240+0000","last_change":"2024-09-10T10:03:05.156861+0000","last_active":"2024-09-10T10:03:06.159240+0000","last_peered":"2024-09-10T10:03:06.159240+0000","last_clean":"2024-09-10T10:03:06.159240+0000","last_became_active":"2024-09-10T10:03:05.156359+0000","last_became_peered":"2024-09-10T10:03:05.156359+0000","last_unstale":"2024-09-10T10:03:06.159240+0000","last_undegraded":"2024-09-10T10:03:06.159240+0000","last_fullsized":"2024-09-10T10:03:06.159240+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:37.947530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634226+0000","last_change":"2024-09-10T10:03:04.148096+0000","last_active":"2024-09-10T10:03:06.634226+0000","last_peered":"2024-09-10T10:03:06.634226+0000","last_clean":"2024-09-10T10:03:06.634226+0000","last_became_active":"2024-09-10T10:03:04.146168+0000","last_became_peered":"2024-09-10T10:03:04.146168+0000","last_unstale":"2024-09-10T10:03:06.634226+0000","last_undegraded":"2024-09-10T10:03:06.634226+0000","last_fullsized":"2024-09-10T10:03:06.634226+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:04.150107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634390+0000","last_change":"2024-09-10T10:03:05.159737+0000","last_active":"2024-09-10T10:03:06.634390+0000","last_peered":"2024-09-10T10:03:06.634390+0000","last_clean":"2024-09-10T10:03:06.634390+0000","last_became_active":"2024-09-10T10:03:05.155486+0000","last_became_peered":"2024-09-10T10:03:05.155486+0000","last_unstale":"2024-09-10T10:03:06.634390+0000","last_undegraded":"2024-09-10T10:03:06.634390+0000","last_fullsized":"2024-09-10T10:03:06.634390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:26.391372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156821+0000","last_change":"2024-09-10T10:03:04.153779+0000","last_active":"2024-09-10T10:03:06.156821+0000","last_peered":"2024-09-10T10:03:06.156821+0000","last_clean":"2024-09-10T10:03:06.156821+0000","last_became_active":"2024-09-10T10:03:04.153470+0000","last_became_peered":"2024-09-10T10:03:04.153470+0000","last_unstale":"2024-09-10T10:03:06.156821+0000","last_undegraded":"2024-09-10T10:03:06.156821+0000","last_fullsized":"2024-09-10T10:03:06.156821+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:04.147964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.156167+0000","last_change":"2024-09-10T10:03:05.146195+0000","last_active":"2024-09-10T10:03:05.156167+0000","last_peered":"2024-09-10T10:03:05.156167+0000","last_clean":"2024-09-10T10:03:05.156167+0000","last_became_active":"2024-09-10T10:03:05.145891+0000","last_became_peered":"2024-09-10T10:03:05.145891+0000","last_unstale":"2024-09-10T10:03:05.156167+0000","last_undegraded":"2024-09-10T10:03:05.156167+0000","last_fullsized":"2024-09-10T10:03:05.156167+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:19.294841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157390+0000","last_change":"2024-09-10T10:03:04.152717+0000","last_active":"2024-09-10T10:03:06.157390+0000","last_peered":"2024-09-10T10:03:06.157390+0000","last_clean":"2024-09-10T10:03:06.157390+0000","last_became_active":"2024-09-10T10:03:04.152450+0000","last_became_peered":"2024-09-10T10:03:04.152450+0000","last_unstale":"2024-09-10T10:03:06.157390+0000","last_undegraded":"2024-09-10T10:03:06.157390+0000","last_fullsized":"2024-09-10T10:03:06.157390+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:10.648351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634258+0000","last_change":"2024-09-10T10:03:05.155170+0000","last_active":"2024-09-10T10:03:06.634258+0000","last_peered":"2024-09-10T10:03:06.634258+0000","last_clean":"2024-09-10T10:03:06.634258+0000","last_became_active":"2024-09-10T10:03:05.154230+0000","last_became_peered":"2024-09-10T10:03:05.154230+0000","last_unstale":"2024-09-10T10:03:06.634258+0000","last_undegraded":"2024-09-10T10:03:06.634258+0000","last_fullsized":"2024-09-10T10:03:06.634258+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:36.451626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.172804+0000","last_change":"2024-09-10T10:03:04.152645+0000","last_active":"2024-09-10T10:03:05.172804+0000","last_peered":"2024-09-10T10:03:05.172804+0000","last_clean":"2024-09-10T10:03:05.172804+0000","last_became_active":"2024-09-10T10:03:04.152390+0000","last_became_peered":"2024-09-10T10:03:04.152390+0000","last_unstale":"2024-09-10T10:03:05.172804+0000","last_undegraded":"2024-09-10T10:03:05.172804+0000","last_fullsized":"2024-09-10T10:03:05.172804+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:32:29.083600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634237+0000","last_change":"2024-09-10T10:03:05.153282+0000","last_active":"2024-09-10T10:03:06.634237+0000","last_peered":"2024-09-10T10:03:06.634237+0000","last_clean":"2024-09-10T10:03:06.634237+0000","last_became_active":"2024-09-10T10:03:05.150536+0000","last_became_peered":"2024-09-10T10:03:05.150536+0000","last_unstale":"2024-09-10T10:03:06.634237+0000","last_undegraded":"2024-09-10T10:03:06.634237+0000","last_fullsized":"2024-09-10T10:03:06.634237+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:53.249360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158154+0000","last_change":"2024-09-10T10:03:04.146776+0000","last_active":"2024-09-10T10:03:06.158154+0000","last_peered":"2024-09-10T10:03:06.158154+0000","last_clean":"2024-09-10T10:03:06.158154+0000","last_became_active":"2024-09-10T10:03:04.146500+0000","last_became_peered":"2024-09-10T10:03:04.146500+0000","last_unstale":"2024-09-10T10:03:06.158154+0000","last_undegraded":"2024-09-10T10:03:06.158154+0000","last_fullsized":"2024-09-10T10:03:06.158154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:38.689419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.633938+0000","last_change":"2024-09-10T10:03:05.153510+0000","last_active":"2024-09-10T10:03:06.633938+0000","last_peered":"2024-09-10T10:03:06.633938+0000","last_clean":"2024-09-10T10:03:06.633938+0000","last_became_active":"2024-09-10T10:03:05.153225+0000","last_became_peered":"2024-09-10T10:03:05.153225+0000","last_unstale":"2024-09-10T10:03:06.633938+0000","last_undegraded":"2024-09-10T10:03:06.633938+0000","last_fullsized":"2024-09-10T10:03:06.633938+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:03.412549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.177246+0000","last_change":"2024-09-10T10:03:04.148457+0000","last_active":"2024-09-10T10:03:05.177246+0000","last_peered":"2024-09-10T10:03:05.177246+0000","last_clean":"2024-09-10T10:03:05.177246+0000","last_became_active":"2024-09-10T10:03:04.148162+0000","last_became_peered":"2024-09-10T10:03:04.148162+0000","last_unstale":"2024-09-10T10:03:05.177246+0000","last_undegraded":"2024-09-10T10:03:05.177246+0000","last_fullsized":"2024-09-10T10:03:05.177246+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:30.108499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.159801+0000","last_change":"2024-09-10T10:03:05.158316+0000","last_active":"2024-09-10T10:03:05.159801+0000","last_peered":"2024-09-10T10:03:05.159801+0000","last_clean":"2024-09-10T10:03:05.159801+0000","last_became_active":"2024-09-10T10:03:05.158035+0000","last_became_peered":"2024-09-10T10:03:05.158035+0000","last_unstale":"2024-09-10T10:03:05.159801+0000","last_undegraded":"2024-09-10T10:03:05.159801+0000","last_fullsized":"2024-09-10T10:03:05.159801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:39.121779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157389+0000","last_change":"2024-09-10T10:03:04.155803+0000","last_active":"2024-09-10T10:03:06.157389+0000","last_peered":"2024-09-10T10:03:06.157389+0000","last_clean":"2024-09-10T10:03:06.157389+0000","last_became_active":"2024-09-10T10:03:04.152633+0000","last_became_peered":"2024-09-10T10:03:04.152633+0000","last_unstale":"2024-09-10T10:03:06.157389+0000","last_undegraded":"2024-09-10T10:03:06.157389+0000","last_fullsized":"2024-09-10T10:03:06.157389+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:03.814589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.157772+0000","last_change":"2024-09-10T10:03:05.154740+0000","last_active":"2024-09-10T10:03:05.157772+0000","last_peered":"2024-09-10T10:03:05.157772+0000","last_clean":"2024-09-10T10:03:05.157772+0000","last_became_active":"2024-09-10T10:03:05.154480+0000","last_became_peered":"2024-09-10T10:03:05.154480+0000","last_unstale":"2024-09-10T10:03:05.157772+0000","last_undegraded":"2024-09-10T10:03:05.157772+0000","last_fullsized":"2024-09-10T10:03:05.157772+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:03.182798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.169442+0000","last_change":"2024-09-10T10:03:04.139308+0000","last_active":"2024-09-10T10:03:05.169442+0000","last_peered":"2024-09-10T10:03:05.169442+0000","last_clean":"2024-09-10T10:03:05.169442+0000","last_became_active":"2024-09-10T10:03:04.139035+0000","last_became_peered":"2024-09-10T10:03:04.139035+0000","last_unstale":"2024-09-10T10:03:05.169442+0000","last_undegraded":"2024-09-10T10:03:05.169442+0000","last_fullsized":"2024-09-10T10:03:05.169442+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:34.306326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.175449+0000","last_change":"2024-09-10T10:03:05.174553+0000","last_active":"2024-09-10T10:03:05.175449+0000","last_peered":"2024-09-10T10:03:05.175449+0000","last_clean":"2024-09-10T10:03:05.175449+0000","last_became_active":"2024-09-10T10:03:05.174244+0000","last_became_peered":"2024-09-10T10:03:05.174244+0000","last_unstale":"2024-09-10T10:03:05.175449+0000","last_undegraded":"2024-09-10T10:03:05.175449+0000","last_fullsized":"2024-09-10T10:03:05.175449+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:27.289059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158238+0000","last_change":"2024-09-10T10:03:04.147450+0000","last_active":"2024-09-10T10:03:06.158238+0000","last_peered":"2024-09-10T10:03:06.158238+0000","last_clean":"2024-09-10T10:03:06.158238+0000","last_became_active":"2024-09-10T10:03:04.147067+0000","last_became_peered":"2024-09-10T10:03:04.147067+0000","last_unstale":"2024-09-10T10:03:06.158238+0000","last_undegraded":"2024-09-10T10:03:06.158238+0000","last_fullsized":"2024-09-10T10:03:06.158238+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:50.625533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.150555+0000","last_change":"2024-09-10T10:03:05.145982+0000","last_active":"2024-09-10T10:03:05.150555+0000","last_peered":"2024-09-10T10:03:05.150555+0000","last_clean":"2024-09-10T10:03:05.150555+0000","last_became_active":"2024-09-10T10:03:05.145484+0000","last_became_peered":"2024-09-10T10:03:05.145484+0000","last_unstale":"2024-09-10T10:03:05.150555+0000","last_undegraded":"2024-09-10T10:03:05.150555+0000","last_fullsized":"2024-09-10T10:03:05.150555+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:39.692552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.140605+0000","last_change":"2024-09-10T10:03:04.153017+0000","last_active":"2024-09-10T10:03:05.140605+0000","last_peered":"2024-09-10T10:03:05.140605+0000","last_clean":"2024-09-10T10:03:05.140605+0000","last_became_active":"2024-09-10T10:03:04.152668+0000","last_became_peered":"2024-09-10T10:03:04.152668+0000","last_unstale":"2024-09-10T10:03:05.140605+0000","last_undegraded":"2024-09-10T10:03:05.140605+0000","last_fullsized":"2024-09-10T10:03:05.140605+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:52.510664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.161462+0000","last_change":"2024-09-10T10:03:05.155639+0000","last_active":"2024-09-10T10:03:05.161462+0000","last_peered":"2024-09-10T10:03:05.161462+0000","last_clean":"2024-09-10T10:03:05.161462+0000","last_became_active":"2024-09-10T10:03:05.155372+0000","last_became_peered":"2024-09-10T10:03:05.155372+0000","last_unstale":"2024-09-10T10:03:05.161462+0000","last_undegraded":"2024-09-10T10:03:05.161462+0000","last_fullsized":"2024-09-10T10:03:05.161462+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:16.016706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.172785+0000","last_change":"2024-09-10T10:03:04.151877+0000","last_active":"2024-09-10T10:03:05.172785+0000","last_peered":"2024-09-10T10:03:05.172785+0000","last_clean":"2024-09-10T10:03:05.172785+0000","last_became_active":"2024-09-10T10:03:04.151612+0000","last_became_peered":"2024-09-10T10:03:04.151612+0000","last_unstale":"2024-09-10T10:03:05.172785+0000","last_undegraded":"2024-09-10T10:03:05.172785+0000","last_fullsized":"2024-09-10T10:03:05.172785+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:11.317940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156980+0000","last_change":"2024-09-10T10:03:05.172567+0000","last_active":"2024-09-10T10:03:06.156980+0000","last_peered":"2024-09-10T10:03:06.156980+0000","last_clean":"2024-09-10T10:03:06.156980+0000","last_became_active":"2024-09-10T10:03:05.172178+0000","last_became_peered":"2024-09-10T10:03:05.172178+0000","last_unstale":"2024-09-10T10:03:06.156980+0000","last_undegraded":"2024-09-10T10:03:06.156980+0000","last_fullsized":"2024-09-10T10:03:06.156980+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:14.633611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.138665+0000","last_change":"2024-09-10T10:03:04.157375+0000","last_active":"2024-09-10T10:03:05.138665+0000","last_peered":"2024-09-10T10:03:05.138665+0000","last_clean":"2024-09-10T10:03:05.138665+0000","last_became_active":"2024-09-10T10:03:04.157141+0000","last_became_peered":"2024-09-10T10:03:04.157141+0000","last_unstale":"2024-09-10T10:03:05.138665+0000","last_undegraded":"2024-09-10T10:03:05.138665+0000","last_fullsized":"2024-09-10T10:03:05.138665+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:02.694978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.154453+0000","last_change":"2024-09-10T10:03:05.148704+0000","last_active":"2024-09-10T10:03:05.154453+0000","last_peered":"2024-09-10T10:03:05.154453+0000","last_clean":"2024-09-10T10:03:05.154453+0000","last_became_active":"2024-09-10T10:03:05.148399+0000","last_became_peered":"2024-09-10T10:03:05.148399+0000","last_unstale":"2024-09-10T10:03:05.154453+0000","last_undegraded":"2024-09-10T10:03:05.154453+0000","last_fullsized":"2024-09-10T10:03:05.154453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:15.379112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.141863+0000","last_change":"2024-09-10T10:03:04.154634+0000","last_active":"2024-09-10T10:03:05.141863+0000","last_peered":"2024-09-10T10:03:05.141863+0000","last_clean":"2024-09-10T10:03:05.141863+0000","last_became_active":"2024-09-10T10:03:04.154121+0000","last_became_peered":"2024-09-10T10:03:04.154121+0000","last_unstale":"2024-09-10T10:03:05.141863+0000","last_undegraded":"2024-09-10T10:03:05.141863+0000","last_fullsized":"2024-09-10T10:03:05.141863+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:46.763023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634070+0000","last_change":"2024-09-10T10:03:05.166035+0000","last_active":"2024-09-10T10:03:06.634070+0000","last_peered":"2024-09-10T10:03:06.634070+0000","last_clean":"2024-09-10T10:03:06.634070+0000","last_became_active":"2024-09-10T10:03:05.165711+0000","last_became_peered":"2024-09-10T10:03:05.165711+0000","last_unstale":"2024-09-10T10:03:06.634070+0000","last_undegraded":"2024-09-10T10:03:06.634070+0000","last_fullsized":"2024-09-10T10:03:06.634070+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:24.751202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156811+0000","last_change":"2024-09-10T10:03:04.145276+0000","last_active":"2024-09-10T10:03:06.156811+0000","last_peered":"2024-09-10T10:03:06.156811+0000","last_clean":"2024-09-10T10:03:06.156811+0000","last_became_active":"2024-09-10T10:03:04.144537+0000","last_became_peered":"2024-09-10T10:03:04.144537+0000","last_unstale":"2024-09-10T10:03:06.156811+0000","last_undegraded":"2024-09-10T10:03:06.156811+0000","last_fullsized":"2024-09-10T10:03:06.156811+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:34.893656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158687+0000","last_change":"2024-09-10T10:03:05.153683+0000","last_active":"2024-09-10T10:03:06.158687+0000","last_peered":"2024-09-10T10:03:06.158687+0000","last_clean":"2024-09-10T10:03:06.158687+0000","last_became_active":"2024-09-10T10:03:05.153411+0000","last_became_peered":"2024-09-10T10:03:05.153411+0000","last_unstale":"2024-09-10T10:03:06.158687+0000","last_undegraded":"2024-09-10T10:03:06.158687+0000","last_fullsized":"2024-09-10T10:03:06.158687+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:45.445940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634206+0000","last_change":"2024-09-10T10:03:04.150472+0000","last_active":"2024-09-10T10:03:06.634206+0000","last_peered":"2024-09-10T10:03:06.634206+0000","last_clean":"2024-09-10T10:03:06.634206+0000","last_became_active":"2024-09-10T10:03:04.150224+0000","last_became_peered":"2024-09-10T10:03:04.150224+0000","last_unstale":"2024-09-10T10:03:06.634206+0000","last_undegraded":"2024-09-10T10:03:06.634206+0000","last_fullsized":"2024-09-10T10:03:06.634206+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:51.039468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157298+0000","last_change":"2024-09-10T10:03:05.162999+0000","last_active":"2024-09-10T10:03:06.157298+0000","last_peered":"2024-09-10T10:03:06.157298+0000","last_clean":"2024-09-10T10:03:06.157298+0000","last_became_active":"2024-09-10T10:03:05.162769+0000","last_became_peered":"2024-09-10T10:03:05.162769+0000","last_unstale":"2024-09-10T10:03:06.157298+0000","last_undegraded":"2024-09-10T10:03:06.157298+0000","last_fullsized":"2024-09-10T10:03:06.157298+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:39.748066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158678+0000","last_change":"2024-09-10T10:03:04.149970+0000","last_active":"2024-09-10T10:03:06.158678+0000","last_peered":"2024-09-10T10:03:06.158678+0000","last_clean":"2024-09-10T10:03:06.158678+0000","last_became_active":"2024-09-10T10:03:04.149656+0000","last_became_peered":"2024-09-10T10:03:04.149656+0000","last_unstale":"2024-09-10T10:03:06.158678+0000","last_undegraded":"2024-09-10T10:03:06.158678+0000","last_fullsized":"2024-09-10T10:03:06.158678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:14.375979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.157406+0000","last_change":"2024-09-10T10:03:05.154296+0000","last_active":"2024-09-10T10:03:06.157406+0000","last_peered":"2024-09-10T10:03:06.157406+0000","last_clean":"2024-09-10T10:03:06.157406+0000","last_became_active":"2024-09-10T10:03:05.154015+0000","last_became_peered":"2024-09-10T10:03:05.154015+0000","last_unstale":"2024-09-10T10:03:06.157406+0000","last_undegraded":"2024-09-10T10:03:06.157406+0000","last_fullsized":"2024-09-10T10:03:06.157406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:56.609614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156912+0000","last_change":"2024-09-10T10:03:04.149363+0000","last_active":"2024-09-10T10:03:06.156912+0000","last_peered":"2024-09-10T10:03:06.156912+0000","last_clean":"2024-09-10T10:03:06.156912+0000","last_became_active":"2024-09-10T10:03:04.149107+0000","last_became_peered":"2024-09-10T10:03:04.149107+0000","last_unstale":"2024-09-10T10:03:06.156912+0000","last_undegraded":"2024-09-10T10:03:06.156912+0000","last_fullsized":"2024-09-10T10:03:06.156912+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:11:34.524613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158119+0000","last_change":"2024-09-10T10:03:05.152802+0000","last_active":"2024-09-10T10:03:06.158119+0000","last_peered":"2024-09-10T10:03:06.158119+0000","last_clean":"2024-09-10T10:03:06.158119+0000","last_became_active":"2024-09-10T10:03:05.152497+0000","last_became_peered":"2024-09-10T10:03:05.152497+0000","last_unstale":"2024-09-10T10:03:06.158119+0000","last_undegraded":"2024-09-10T10:03:06.158119+0000","last_fullsized":"2024-09-10T10:03:06.158119+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:27.289059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634345+0000","last_change":"2024-09-10T10:03:04.158156+0000","last_active":"2024-09-10T10:03:06.634345+0000","last_peered":"2024-09-10T10:03:06.634345+0000","last_clean":"2024-09-10T10:03:06.634345+0000","last_became_active":"2024-09-10T10:03:04.157858+0000","last_became_peered":"2024-09-10T10:03:04.157858+0000","last_unstale":"2024-09-10T10:03:06.634345+0000","last_undegraded":"2024-09-10T10:03:06.634345+0000","last_fullsized":"2024-09-10T10:03:06.634345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:39.617729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158685+0000","last_change":"2024-09-10T10:03:05.151283+0000","last_active":"2024-09-10T10:03:06.158685+0000","last_peered":"2024-09-10T10:03:06.158685+0000","last_clean":"2024-09-10T10:03:06.158685+0000","last_became_active":"2024-09-10T10:03:05.150980+0000","last_became_peered":"2024-09-10T10:03:05.150980+0000","last_unstale":"2024-09-10T10:03:06.158685+0000","last_undegraded":"2024-09-10T10:03:06.158685+0000","last_fullsized":"2024-09-10T10:03:06.158685+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:04.719505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.140411+0000","last_change":"2024-09-10T10:03:04.152342+0000","last_active":"2024-09-10T10:03:05.140411+0000","last_peered":"2024-09-10T10:03:05.140411+0000","last_clean":"2024-09-10T10:03:05.140411+0000","last_became_active":"2024-09-10T10:03:04.152008+0000","last_became_peered":"2024-09-10T10:03:04.152008+0000","last_unstale":"2024-09-10T10:03:05.140411+0000","last_undegraded":"2024-09-10T10:03:05.140411+0000","last_fullsized":"2024-09-10T10:03:05.140411+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:22.305412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158482+0000","last_change":"2024-09-10T10:03:05.162965+0000","last_active":"2024-09-10T10:03:06.158482+0000","last_peered":"2024-09-10T10:03:06.158482+0000","last_clean":"2024-09-10T10:03:06.158482+0000","last_became_active":"2024-09-10T10:03:05.162546+0000","last_became_peered":"2024-09-10T10:03:05.162546+0000","last_unstale":"2024-09-10T10:03:06.158482+0000","last_undegraded":"2024-09-10T10:03:06.158482+0000","last_fullsized":"2024-09-10T10:03:06.158482+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:59.828450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.158815+0000","last_change":"2024-09-10T10:03:04.152147+0000","last_active":"2024-09-10T10:03:06.158815+0000","last_peered":"2024-09-10T10:03:06.158815+0000","last_clean":"2024-09-10T10:03:06.158815+0000","last_became_active":"2024-09-10T10:03:04.151776+0000","last_became_peered":"2024-09-10T10:03:04.151776+0000","last_unstale":"2024-09-10T10:03:06.158815+0000","last_undegraded":"2024-09-10T10:03:06.158815+0000","last_fullsized":"2024-09-10T10:03:06.158815+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:22.305412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.154432+0000","last_change":"2024-09-10T10:03:05.148105+0000","last_active":"2024-09-10T10:03:05.154432+0000","last_peered":"2024-09-10T10:03:05.154432+0000","last_clean":"2024-09-10T10:03:05.154432+0000","last_became_active":"2024-09-10T10:03:05.147839+0000","last_became_peered":"2024-09-10T10:03:05.147839+0000","last_unstale":"2024-09-10T10:03:05.154432+0000","last_undegraded":"2024-09-10T10:03:05.154432+0000","last_fullsized":"2024-09-10T10:03:05.154432+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:51.628666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.138848+0000","last_change":"2024-09-10T10:03:04.150860+0000","last_active":"2024-09-10T10:03:05.138848+0000","last_peered":"2024-09-10T10:03:05.138848+0000","last_clean":"2024-09-10T10:03:05.138848+0000","last_became_active":"2024-09-10T10:03:04.150581+0000","last_became_peered":"2024-09-10T10:03:04.150581+0000","last_unstale":"2024-09-10T10:03:05.138848+0000","last_undegraded":"2024-09-10T10:03:05.138848+0000","last_fullsized":"2024-09-10T10:03:05.138848+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:48.938577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156715+0000","last_change":"2024-09-10T10:03:05.157317+0000","last_active":"2024-09-10T10:03:06.156715+0000","last_peered":"2024-09-10T10:03:06.156715+0000","last_clean":"2024-09-10T10:03:06.156715+0000","last_became_active":"2024-09-10T10:03:05.156977+0000","last_became_peered":"2024-09-10T10:03:05.156977+0000","last_unstale":"2024-09-10T10:03:06.156715+0000","last_undegraded":"2024-09-10T10:03:06.156715+0000","last_fullsized":"2024-09-10T10:03:06.156715+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:09.740426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.156895+0000","last_change":"2024-09-10T10:03:04.146461+0000","last_active":"2024-09-10T10:03:06.156895+0000","last_peered":"2024-09-10T10:03:06.156895+0000","last_clean":"2024-09-10T10:03:06.156895+0000","last_became_active":"2024-09-10T10:03:04.146196+0000","last_became_peered":"2024-09-10T10:03:04.146196+0000","last_unstale":"2024-09-10T10:03:06.156895+0000","last_undegraded":"2024-09-10T10:03:06.156895+0000","last_fullsized":"2024-09-10T10:03:06.156895+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:23.917121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T10:03:06.634004+0000","last_change":"2024-09-10T10:03:05.147095+0000","last_active":"2024-09-10T10:03:06.634004+0000","last_peered":"2024-09-10T10:03:06.634004+0000","last_clean":"2024-09-10T10:03:06.634004+0000","last_became_active":"2024-09-10T10:03:05.146811+0000","last_became_peered":"2024-09-10T10:03:05.146811+0000","last_unstale":"2024-09-10T10:03:06.634004+0000","last_undegraded":"2024-09-10T10:03:06.634004+0000","last_fullsized":"2024-09-10T10:03:06.634004+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:04.121290+0000","last_clean_scrub_stamp":"2024-09-10T10:03:04.121290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:17.098721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.140487+0000","last_change":"2024-09-10T10:03:04.151026+0000","last_active":"2024-09-10T10:03:05.140487+0000","last_peered":"2024-09-10T10:03:05.140487+0000","last_clean":"2024-09-10T10:03:05.140487+0000","last_became_active":"2024-09-10T10:03:04.150687+0000","last_became_peered":"2024-09-10T10:03:04.150687+0000","last_unstale":"2024-09-10T10:03:05.140487+0000","last_undegraded":"2024-09-10T10:03:05.140487+0000","last_fullsized":"2024-09-10T10:03:05.140487+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:03.118157+0000","last_clean_scrub_stamp":"2024-09-10T10:03:03.118157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:33.863849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":354,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T10:03:05.139306+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:03:05.139306+0000","last_peered":"2024-09-10T10:03:05.139306+0000","last_clean":"2024-09-10T10:03:05.139306+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:03:05.139306+0000","last_undegraded":"2024-09-10T10:03:05.139306+0000","last_fullsized":"2024-09-10T10:03:05.139306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1863680,"data_stored":1141021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1265664,"data_stored":550107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1269760,"data_stored":550641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1273856,"data_stored":556337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542207,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1277952,"data_stored":551719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1261568,"data_stored":550037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":550161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":1148429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:03:09.621 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:03:09.621 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:03:09.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.799+0000 7fa514200640 1 Processor -- start 2024-09-10T10:03:09.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.799+0000 7fa514200640 1 -- start start 2024-09-10T10:03:09.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.799+0000 7fa514200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c06f1e0 0x7fa50c166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.799+0000 7fa514200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa50c155d50 con 0x7fa50c155970 2024-09-10T10:03:09.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.799+0000 7fa514200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa50c0b34d0 con 0x7fa50c15c560 2024-09-10T10:03:09.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.799+0000 7fa514200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c0b4570 con 0x7fa50c06f1e0 2024-09-10T10:03:09.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.799+0000 7fa511800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa50c155970 0x7fa50c0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52056/0 (socket says 172.21.15.110:52056) 2024-09-10T10:03:09.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.799+0000 7fa511800640 1 -- 172.21.15.110:0/1700236193 learned_addr learned my addr 172.21.15.110:0/1700236193 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:09.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa512200640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c06f1e0 0x7fa50c166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 431055844 0 0) 0x7fa50c155d50 con 0x7fa50c155970 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4f4003530 con 0x7fa50c155970 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3574553255 0 0) 0x7fa4f4003530 con 0x7fa50c155970 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c06f1e0 msgr2=0x7fa50c166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c06f1e0 0x7fa50c166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 >> v1:172.21.15.145:6789/0 conn(0x7fa50c15c560 legacy=0x7fa50c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c0b37b0 con 0x7fa50c155970 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fa500001a20 con 0x7fa50c155970 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 >> v1:172.21.15.110:6789/0 conn(0x7fa50c155970 legacy=0x7fa50c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f40039f0 0x7fa4f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c06f1e0 0x7fa4f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4f4004d50 0x7fa4f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f400b9d0 con 0x7fa4f40039f0 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f400bb80 con 0x7fa4f4004d50 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f400bd80 con 0x7fa50c06f1e0 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa511800640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f40039f0 0x7fa4f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa510e00640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c06f1e0 0x7fa4f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa512200640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4f4004d50 0x7fa4f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa511800640 1 -- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c06f1e0 msgr2=0x7fa4f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa511800640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c06f1e0 0x7fa4f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa511800640 1 -- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4f4004d50 msgr2=0x7fa4f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa511800640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4f4004d50 0x7fa4f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.803+0000 7fa511800640 1 -- 172.21.15.110:0/1700236193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4003530 con 0x7fa4f40039f0 2024-09-10T10:03:09.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa511800640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f40039f0 0x7fa4f4003dd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa50000a880 tx=0x7fa50001d290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b49ad3ef6f87dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50000a4d0 con 0x7fa4f40039f0 2024-09-10T10:03:09.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5000072d0 con 0x7fa4f40039f0 2024-09-10T10:03:09.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa50be00640 1 -- 172.21.15.110:0/1700236193 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5000076c0 con 0x7fa4f40039f0 2024-09-10T10:03:09.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- 172.21.15.110:0/1700236193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f40039f0 msgr2=0x7fa4f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 --2- 172.21.15.110:0/1700236193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f40039f0 0x7fa4f4003dd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa50000a880 tx=0x7fa50001d290 comp rx=0 tx=0).stop 2024-09-10T10:03:09.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa512200640 1 -- 172.21.15.110:0/1700236193 reap_dead start 2024-09-10T10:03:09.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- 172.21.15.110:0/1700236193 shutdown_connections 2024-09-10T10:03:09.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- 172.21.15.110:0/1700236193 >> 172.21.15.110:0/1700236193 conn(0x7fa50c06bc50 msgr2=0x7fa50c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:09.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- 172.21.15.110:0/1700236193 shutdown_connections 2024-09-10T10:03:09.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- 172.21.15.110:0/1700236193 wait complete. 2024-09-10T10:03:09.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 Processor -- start 2024-09-10T10:03:09.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- start start 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4f40039f0 0x7fa50c072f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c14d1d0 0x7fa50c13bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c1fb660 0x7fa50c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1669b0 con 0x7fa50c14d1d0 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c166830 con 0x7fa50c1fb660 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa514200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c166b30 con 0x7fa4f40039f0 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa510e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c14d1d0 0x7fa50c13bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa510e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c14d1d0 0x7fa50c13bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54704/0 (socket says 172.21.15.110:54704) 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa510e00640 1 -- 172.21.15.110:0/354595505 learned_addr learned my addr 172.21.15.110:0/354595505 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:09.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.807+0000 7fa511800640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4f40039f0 0x7fa50c072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa512200640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c1fb660 0x7fa50c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa510e00640 1 -- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4f40039f0 msgr2=0x7fa50c072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa510e00640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4f40039f0 0x7fa50c072f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa510e00640 1 -- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c1fb660 msgr2=0x7fa50c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:09.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa510e00640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c1fb660 0x7fa50c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:09.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa510e00640 1 -- 172.21.15.110:0/354595505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c14d650 con 0x7fa50c14d1d0 2024-09-10T10:03:09.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa510e00640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c14d1d0 0x7fa50c13bd20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800e860 tx=0x7fa4f800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa509600640 1 -- 172.21.15.110:0/354595505 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8001d30 con 0x7fa50c14d1d0 2024-09-10T10:03:09.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa509600640 1 -- 172.21.15.110:0/354595505 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f8018070 con 0x7fa50c14d1d0 2024-09-10T10:03:09.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c15d460 con 0x7fa50c14d1d0 2024-09-10T10:03:09.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa509600640 1 -- 172.21.15.110:0/354595505 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800c6c0 con 0x7fa50c14d1d0 2024-09-10T10:03:09.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c1fae60 con 0x7fa50c14d1d0 2024-09-10T10:03:09.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50c0b34d0 con 0x7fa50c14d1d0 2024-09-10T10:03:09.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa509600640 1 -- 172.21.15.110:0/354595505 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fa4f800c860 con 0x7fa50c14d1d0 2024-09-10T10:03:09.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa509600640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec0679e0 0x7fa4ec069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:09.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.811+0000 7fa509600640 1 -- 172.21.15.110:0/354595505 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fa4f8085ae0 con 0x7fa50c14d1d0 2024-09-10T10:03:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.815+0000 7fa511800640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec0679e0 0x7fa4ec069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:09.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.815+0000 7fa511800640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec0679e0 0x7fa4ec069ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa5000ad000 tx=0x7fa500009050 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:09.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:09.815+0000 7fa509600640 1 -- 172.21.15.110:0/354595505 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa4f8055eb0 con 0x7fa50c14d1d0 2024-09-10T10:03:10.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.031+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa50c15d980 con 0x7fa50c14d1d0 2024-09-10T10:03:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa509600640 1 -- 172.21.15.110:0/354595505 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7fa4f805a1b0 con 0x7fa50c14d1d0 2024-09-10T10:03:10.037 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:03:10.037 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:03:10.037 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:03:10.037 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:03:10.037 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:03:10.038 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:03:10.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec0679e0 msgr2=0x7fa4ec069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec0679e0 0x7fa4ec069ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa5000ad000 tx=0x7fa500009050 comp rx=0 tx=0).stop 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c14d1d0 msgr2=0x7fa50c13bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c14d1d0 0x7fa50c13bd20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800e860 tx=0x7fa4f800ed20 comp rx=0 tx=0).stop 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 shutdown_connections 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec0679e0 0x7fa4ec069ea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c1fb660 0x7fa50c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c14d1d0 0x7fa50c13bd20 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 --2- 172.21.15.110:0/354595505 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4f40039f0 0x7fa50c072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 >> 172.21.15.110:0/354595505 conn(0x7fa50c06bc50 msgr2=0x7fa50c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 shutdown_connections 2024-09-10T10:03:10.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.035+0000 7fa514200640 1 -- 172.21.15.110:0/354595505 wait complete. 2024-09-10T10:03:10.058 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:03:10.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f6290000640 1 Processor -- start 2024-09-10T10:03:10.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f6290000640 1 -- start start 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f6290000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6288155970 0x7f62880cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f6290000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f628815c560 0x7f628806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f6290000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628806f010 0x7f628815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f6290000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f62880b4570 con 0x7f628815c560 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f6290000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f628806f430 con 0x7f6288155970 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f6290000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6288155d50 con 0x7f628806f010 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f628cc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f628815c560 0x7f628806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f628cc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f628815c560 0x7f628806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54716/0 (socket says 172.21.15.110:54716) 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f628cc00640 1 -- 172.21.15.110:0/135476762 learned_addr learned my addr 172.21.15.110:0/135476762 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f628d600640 1 --2- 172.21.15.110:0/135476762 >> v2:172.21.15.145:3300/0 conn(0x7f6288155970 0x7f62880cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.191+0000 7f628e000640 1 --2- 172.21.15.110:0/135476762 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628806f010 0x7f628815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f628cc00640 1 -- 172.21.15.110:0/135476762 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628806f010 msgr2=0x7f628815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f628cc00640 1 --2- 172.21.15.110:0/135476762 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628806f010 0x7f628815eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f628cc00640 1 -- 172.21.15.110:0/135476762 >> v2:172.21.15.145:3300/0 conn(0x7f6288155970 msgr2=0x7f62880cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f628cc00640 1 --2- 172.21.15.110:0/135476762 >> v2:172.21.15.145:3300/0 conn(0x7f6288155970 0x7f62880cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f628cc00640 1 -- 172.21.15.110:0/135476762 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62880b34d0 con 0x7f628815c560 2024-09-10T10:03:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f628e000640 1 --2- 172.21.15.110:0/135476762 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628806f010 0x7f628815eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:10.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f628d600640 1 --2- 172.21.15.110:0/135476762 >> v2:172.21.15.145:3300/0 conn(0x7f6288155970 0x7f62880cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:10.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f628cc00640 1 --2- 172.21.15.110:0/135476762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628815c560 0x7f628806ead0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f628806cc30 tx=0x7f627401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa044dc1590b3a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6287e00640 1 -- 172.21.15.110:0/135476762 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740b0070 con 0x7f628815c560 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6287e00640 1 -- 172.21.15.110:0/135476762 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62740ab600 con 0x7f628815c560 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6287e00640 1 -- 172.21.15.110:0/135476762 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740ab9d0 con 0x7f628815c560 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6290000640 1 -- 172.21.15.110:0/135476762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628815c560 msgr2=0x7f628806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6290000640 1 --2- 172.21.15.110:0/135476762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628815c560 0x7f628806ead0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f628806cc30 tx=0x7f627401c230 comp rx=0 tx=0).stop 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6290000640 1 -- 172.21.15.110:0/135476762 shutdown_connections 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6290000640 1 --2- 172.21.15.110:0/135476762 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628806f010 0x7f628815eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6290000640 1 --2- 172.21.15.110:0/135476762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628815c560 0x7f628806ead0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6290000640 1 --2- 172.21.15.110:0/135476762 >> v2:172.21.15.145:3300/0 conn(0x7f6288155970 0x7f62880cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6290000640 1 -- 172.21.15.110:0/135476762 >> 172.21.15.110:0/135476762 conn(0x7f628806bc50 msgr2=0x7f628806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.195+0000 7f6290000640 1 -- 172.21.15.110:0/135476762 shutdown_connections 2024-09-10T10:03:10.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 -- 172.21.15.110:0/135476762 wait complete. 2024-09-10T10:03:10.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 Processor -- start 2024-09-10T10:03:10.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 -- start start 2024-09-10T10:03:10.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628806f010 0x7f6288156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6288155970 0x7f62880ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628815c560 0x7f62880ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62880cce00 con 0x7f628806f010 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62880b4050 con 0x7f6288155970 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f62880ce420 con 0x7f628815c560 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628806f010 0x7f6288156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628806f010 0x7f6288156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54724/0 (socket says 172.21.15.110:54724) 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 -- 172.21.15.110:0/2339235239 learned_addr learned my addr 172.21.15.110:0/2339235239 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628cc00640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6288155970 0x7f62880ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628e000640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628815c560 0x7f62880ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 -- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628815c560 msgr2=0x7f62880ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628815c560 0x7f62880ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 -- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6288155970 msgr2=0x7f62880ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6288155970 0x7f62880ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 -- 172.21.15.110:0/2339235239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62881587a0 con 0x7f628806f010 2024-09-10T10:03:10.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f628d600640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628806f010 0x7f6288156e30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f627c00e840 tx=0x7f627c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:10.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6285600640 1 -- 172.21.15.110:0/2339235239 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c017070 con 0x7f628806f010 2024-09-10T10:03:10.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6285600640 1 -- 172.21.15.110:0/2339235239 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627c00c8d0 con 0x7f628806f010 2024-09-10T10:03:10.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6288156070 con 0x7f628806f010 2024-09-10T10:03:10.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.199+0000 7f6285600640 1 -- 172.21.15.110:0/2339235239 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c00f040 con 0x7f628806f010 2024-09-10T10:03:10.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.203+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6288156630 con 0x7f628806f010 2024-09-10T10:03:10.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.203+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62880cc8c0 con 0x7f628806f010 2024-09-10T10:03:10.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.203+0000 7f6285600640 1 -- 172.21.15.110:0/2339235239 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f627c01a070 con 0x7f628806f010 2024-09-10T10:03:10.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.203+0000 7f6285600640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62680679e0 0x7f6268069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.203+0000 7f6285600640 1 -- 172.21.15.110:0/2339235239 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f627c085b10 con 0x7f628806f010 2024-09-10T10:03:10.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.203+0000 7f628cc00640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62680679e0 0x7f6268069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.203+0000 7f628cc00640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62680679e0 0x7f6268069ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62740ad000 tx=0x7f6274008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.207+0000 7f6285600640 1 -- 172.21.15.110:0/2339235239 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f627c055d40 con 0x7f628806f010 2024-09-10T10:03:10.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.395+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62880af360 con 0x7f628806f010 2024-09-10T10:03:10.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6285600640 1 -- 172.21.15.110:0/2339235239 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5228 (secure 0 0 0) 0x7f627c05a040 con 0x7f628806f010 2024-09-10T10:03:10.401 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:10.401 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":68,"btime":"2024-09-10T10:03:08:162006+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/59288243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":59288243},{"type":"v1","addr":"172.21.15.145:6834","nonce":59288243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2534764627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2534764627},{"type":"v1","addr":"172.21.15.145:6835","nonce":2534764627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6798,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3565143399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":3565143399},{"type":"v1","addr":"172.21.15.110:6834","nonce":3565143399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:04.516174+0000","modified":"2024-09-10T10:03:07.538032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/100964295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":100964295},{"type":"v1","addr":"172.21.15.110:6835","nonce":100964295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":8}]} 2024-09-10T10:03:10.402 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 68 2024-09-10T10:03:10.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62680679e0 msgr2=0x7f6268069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62680679e0 0x7f6268069ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62740ad000 tx=0x7f6274008040 comp rx=0 tx=0).stop 2024-09-10T10:03:10.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628806f010 msgr2=0x7f6288156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628806f010 0x7f6288156e30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f627c00e840 tx=0x7f627c00ed00 comp rx=0 tx=0).stop 2024-09-10T10:03:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 shutdown_connections 2024-09-10T10:03:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62680679e0 0x7f6268069ea0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f628815c560 0x7f62880ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6288155970 0x7f62880ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 --2- 172.21.15.110:0/2339235239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f628806f010 0x7f6288156e30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 >> 172.21.15.110:0/2339235239 conn(0x7f628806bc50 msgr2=0x7f62880b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 shutdown_connections 2024-09-10T10:03:10.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.399+0000 7f6290000640 1 -- 172.21.15.110:0/2339235239 wait complete. 2024-09-10T10:03:10.769 INFO:tasks.ceph.mds.c.smithi110.stderr:2024-09-10T10:03:10.767+0000 7f7725200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:03:10.824 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:03:04.516174+0000', 'modified': '2024-09-10T10:03:07.538032+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6801}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6801': {'gid': 6801, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6835/100964295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6832', 'nonce': 100964295}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 100964295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6801, 'qdb_cluster': [6801]} 2024-09-10T10:03:10.825 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:03:10.825 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:03:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f3991400640 1 Processor -- start 2024-09-10T10:03:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f3991400640 1 -- start start 2024-09-10T10:03:10.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f3991400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c15c520 0x7f398c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f3991400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c0b5aa0 con 0x7f398c070540 2024-09-10T10:03:10.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f3991400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c06f780 con 0x7f398c0b64c0 2024-09-10T10:03:10.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f3991400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f398c15c900 con 0x7f398c15c520 2024-09-10T10:03:10.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f3983400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f398c070540 0x7f398c0b5db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52066/0 (socket says 172.21.15.110:52066) 2024-09-10T10:03:10.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f3983400640 1 -- 172.21.15.110:0/1365289183 learned_addr learned my addr 172.21.15.110:0/1365289183 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:10.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f398be00640 1 --2- 172.21.15.110:0/1365289183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c15c520 0x7f398c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f398b400640 1 -- 172.21.15.110:0/1365289183 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222078981 0 0) 0x7f398c0b5aa0 con 0x7f398c070540 2024-09-10T10:03:10.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f398b400640 1 -- 172.21.15.110:0/1365289183 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3970003530 con 0x7f398c070540 2024-09-10T10:03:10.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f398b400640 1 -- 172.21.15.110:0/1365289183 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126611874 0 0) 0x7f398c06f780 con 0x7f398c0b64c0 2024-09-10T10:03:10.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f398b400640 1 -- 172.21.15.110:0/1365289183 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f398c0b5aa0 con 0x7f398c0b64c0 2024-09-10T10:03:10.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f398be00640 1 -- 172.21.15.110:0/1365289183 >> v1:172.21.15.145:6789/0 conn(0x7f398c0b64c0 legacy=0x7f398c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f398be00640 1 -- 172.21.15.110:0/1365289183 >> v1:172.21.15.110:6789/0 conn(0x7f398c070540 legacy=0x7f398c0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.959+0000 7f398be00640 1 -- 172.21.15.110:0/1365289183 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c06fa60 con 0x7f398c15c520 2024-09-10T10:03:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f398be00640 1 --2- 172.21.15.110:0/1365289183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c15c520 0x7f398c0b54c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f397c009dc0 tx=0x7f397c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1732a71a2026f4f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f398b400640 1 -- 172.21.15.110:0/1365289183 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0b2070 con 0x7f398c15c520 2024-09-10T10:03:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f398b400640 1 -- 172.21.15.110:0/1365289183 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c0a57c0 con 0x7f398c15c520 2024-09-10T10:03:10.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f398b400640 1 -- 172.21.15.110:0/1365289183 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0a5b70 con 0x7f398c15c520 2024-09-10T10:03:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f3991400640 1 -- 172.21.15.110:0/1365289183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c15c520 msgr2=0x7f398c0b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f3991400640 1 --2- 172.21.15.110:0/1365289183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c15c520 0x7f398c0b54c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f397c009dc0 tx=0x7f397c01c900 comp rx=0 tx=0).stop 2024-09-10T10:03:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f3991400640 1 -- 172.21.15.110:0/1365289183 shutdown_connections 2024-09-10T10:03:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f3991400640 1 --2- 172.21.15.110:0/1365289183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c15c520 0x7f398c0b54c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f3991400640 1 -- 172.21.15.110:0/1365289183 >> 172.21.15.110:0/1365289183 conn(0x7f398c06cd60 msgr2=0x7f398c06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:10.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f3991400640 1 -- 172.21.15.110:0/1365289183 shutdown_connections 2024-09-10T10:03:10.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.963+0000 7f3991400640 1 -- 172.21.15.110:0/1365289183 wait complete. 2024-09-10T10:03:10.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 Processor -- start 2024-09-10T10:03:10.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 -- start start 2024-09-10T10:03:10.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f398c070540 0x7f398c146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c0b64c0 0x7f398c152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f398c15c520 0x7f398c152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f398c16a390 con 0x7f398c070540 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f398c16a210 con 0x7f398c15c520 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f398c16a510 con 0x7f398c0b64c0 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f398c070540 0x7f398c146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f398c070540 0x7f398c146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54732/0 (socket says 172.21.15.110:54732) 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 -- 172.21.15.110:0/1071375385 learned_addr learned my addr 172.21.15.110:0/1071375385 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3983400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c0b64c0 0x7f398c152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3990a00640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f398c15c520 0x7f398c152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 -- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c0b64c0 msgr2=0x7f398c152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c0b64c0 0x7f398c152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 -- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f398c15c520 msgr2=0x7f398c152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f398c15c520 0x7f398c152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 -- 172.21.15.110:0/1071375385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c14e9f0 con 0x7f398c070540 2024-09-10T10:03:10.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f398be00640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f398c070540 0x7f398c146d10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f397c000f90 tx=0x7f397c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:10.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3988c00640 1 -- 172.21.15.110:0/1071375385 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0bd030 con 0x7f398c070540 2024-09-10T10:03:10.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3988c00640 1 -- 172.21.15.110:0/1071375385 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c0a55c0 con 0x7f398c070540 2024-09-10T10:03:10.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 -- 172.21.15.110:0/1071375385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398c14ecd0 con 0x7f398c070540 2024-09-10T10:03:10.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3988c00640 1 -- 172.21.15.110:0/1071375385 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0b2040 con 0x7f398c070540 2024-09-10T10:03:10.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.967+0000 7f3991400640 1 -- 172.21.15.110:0/1071375385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398c14ad30 con 0x7f398c070540 2024-09-10T10:03:10.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.971+0000 7f3988c00640 1 -- 172.21.15.110:0/1071375385 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f397c0c0070 con 0x7f398c070540 2024-09-10T10:03:10.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.971+0000 7f3988c00640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3968067a10 0x7f3968069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.971+0000 7f3988c00640 1 -- 172.21.15.110:0/1071375385 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f397c1206f0 con 0x7f398c070540 2024-09-10T10:03:10.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.971+0000 7f3983400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3968067a10 0x7f3968069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.971+0000 7f3995e35000 1 Processor -- start 2024-09-10T10:03:10.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.971+0000 7f3995e35000 1 -- start start 2024-09-10T10:03:10.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3983400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3968067a10 0x7f3968069ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3974007fd0 tx=0x7f3974007d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:10.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 0x55fb1f97ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f935d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f936290 0x55fb1f936670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fb1f81db60 con 0x55fb1f936290 2024-09-10T10:03:10.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fb1f784c70 con 0x55fb1f980440 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55fb1f72aed0 con 0x55fb1f97fb20 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f936290 0x55fb1f936670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f936290 0x55fb1f936670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54744/0 (socket says 172.21.15.110:54744) 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 -- 172.21.15.110:0/2537667477 learned_addr learned my addr 172.21.15.110:0/2537667477 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3983400640 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f935d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f398be00640 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 0x55fb1f97ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 -- 172.21.15.110:0/2537667477 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 msgr2=0x55fb1f97ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 0x55fb1f97ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 -- 172.21.15.110:0/2537667477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 msgr2=0x55fb1f935d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f935d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 -- 172.21.15.110:0/2537667477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb1f73ce50 con 0x55fb1f936290 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3990a00640 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f936290 0x55fb1f936670 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3984001870 tx=0x7f3984007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2599dede9bc3b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:10.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3980c00640 1 -- 172.21.15.110:0/2537667477 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39840052e0 con 0x55fb1f936290 2024-09-10T10:03:10.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3980c00640 1 -- 172.21.15.110:0/2537667477 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3984018070 con 0x55fb1f936290 2024-09-10T10:03:10.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3980c00640 1 -- 172.21.15.110:0/2537667477 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398400fab0 con 0x55fb1f936290 2024-09-10T10:03:10.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- 172.21.15.110:0/2537667477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f936290 msgr2=0x55fb1f936670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f936290 0x55fb1f936670 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3984001870 tx=0x7f3984007cb0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- 172.21.15.110:0/2537667477 shutdown_connections 2024-09-10T10:03:10.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f936290 0x55fb1f936670 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f935d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 --2- 172.21.15.110:0/2537667477 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 0x55fb1f97ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- 172.21.15.110:0/2537667477 >> 172.21.15.110:0/2537667477 conn(0x55fb1f93c600 msgr2=0x55fb1f934630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:10.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- 172.21.15.110:0/2537667477 shutdown_connections 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- 172.21.15.110:0/2537667477 wait complete. 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 -- start start 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.975+0000 7f3995e35000 1 Processor -- start 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f97fb20 0x55fb1f93ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f980440 0x55fb1f940cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f93b390 0x55fb1f941230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fb1f7e6b00 con 0x55fb1f980440 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fb1f75f360 con 0x55fb1f97fb20 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55fb1f75c2d0 con 0x55fb1f93b390 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 10 client.0 _ll_register_callbacks cb 0x55fb1f921820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f980440 0x55fb1f940cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f980440 0x55fb1f940cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54758/0 (socket says 172.21.15.110:54758) 2024-09-10T10:03:10.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 -- 172.21.15.110:0/2137596756 learned_addr learned my addr 172.21.15.110:0/2137596756 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f398be00640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f97fb20 0x55fb1f93ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3990a00640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f93b390 0x55fb1f941230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 -- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f93b390 msgr2=0x55fb1f941230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f93b390 0x55fb1f941230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 -- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f97fb20 msgr2=0x55fb1f93ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f97fb20 0x55fb1f93ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 -- 172.21.15.110:0/2137596756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1f72b120 con 0x55fb1f980440 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f398be00640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f97fb20 0x55fb1f93ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3990a00640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f93b390 0x55fb1f941230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3983400640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f980440 0x55fb1f940cf0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f398c0d12e0 tx=0x7f3974001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319328aca25c05d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 1 -- 172.21.15.110:0/2137596756 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974005a70 con 0x55fb1f980440 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 1 -- 172.21.15.110:0/2137596756 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f3974005c10 con 0x55fb1f980440 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- 172.21.15.110:0/2137596756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3984018070 con 0x55fb1f980440 2024-09-10T10:03:10.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 1 -- 172.21.15.110:0/2137596756 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3974005e10 con 0x55fb1f980440 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 10 client.6924 fetch_fsmap learned FSMap version 68 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- 172.21.15.110:0/2137596756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55fb1f724450 con 0x55fb1f980440 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 1 -- 172.21.15.110:0/2137596756 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3984018070 con 0x55fb1f980440 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 1 -- 172.21.15.110:0/2137596756 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7f3974020040 con 0x55fb1f980440 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 10 client.6924 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 10 client.6924 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 20 client.6924 populate_metadata read hostname 'smithi110' 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] conn(0x55fb1f985f50 0x55fb1f986330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 4 client.6924 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- 172.21.15.110:0/2137596756 --> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb1f71ec00 con 0x55fb1f985f50 2024-09-10T10:03:10.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f398be00640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] conn(0x55fb1f985f50 0x55fb1f986330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:10.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f398be00640 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] conn(0x55fb1f985f50 0x55fb1f986330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805a9005f1e2c14a server_cookie=32632c8c50777d19 in_seq=0 out_seq=0 2024-09-10T10:03:10.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 10 client.6924 ms_handle_connect on v2:172.21.15.110:6832/100964295 2024-09-10T10:03:10.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 1 -- 172.21.15.110:0/2137596756 <== mds.0 v2:172.21.15.110:6832/100964295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb1f71ec00 con 0x55fb1f985f50 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 10 client.6924 handle_command_reply: tid=0 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 client.6924 shutdown 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- 172.21.15.110:0/2137596756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f980440 msgr2=0x55fb1f940cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f980440 0x55fb1f940cf0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f398c0d12e0 tx=0x7f3974001160 comp rx=0 tx=0).stop 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- 172.21.15.110:0/2137596756 shutdown_connections 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f93b390 0x55fb1f941230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] conn(0x55fb1f985f50 0x55fb1f986330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f980440 0x55fb1f940cf0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 --2- 172.21.15.110:0/2137596756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f97fb20 0x55fb1f93ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3979600640 0 client.6924 ms_handle_reset on v2:172.21.15.110:6832/100964295 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.979+0000 7f3995e35000 1 -- 172.21.15.110:0/2137596756 >> 172.21.15.110:0/2137596756 conn(0x55fb1f936290 msgr2=0x55fb1f934d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.983+0000 7f3995e35000 1 -- 172.21.15.110:0/2137596756 shutdown_connections 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.983+0000 7f3995e35000 1 -- 172.21.15.110:0/2137596756 wait complete. 2024-09-10T10:03:10.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.983+0000 7f3995e35000 20 client.6924 trim_cache size 0 max 16384 2024-09-10T10:03:11.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3995e35000 1 Processor -- start 2024-09-10T10:03:11.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3995e35000 1 -- start start 2024-09-10T10:03:11.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3995e35000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f93b390 0x55fb1f996850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3995e35000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 0x55fb1f97ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3995e35000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f97f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3995e35000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0b2870 con 0x55fb1f93b390 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3995e35000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f398c150370 con 0x55fb1f980440 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3995e35000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f398c0b2220 con 0x55fb1f97fb20 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f93b390 0x55fb1f996850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f93b390 0x55fb1f996850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54764/0 (socket says 172.21.15.110:54764) 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 -- 172.21.15.110:0/443150421 learned_addr learned my addr 172.21.15.110:0/443150421 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3990a00640 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f97f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3983400640 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 0x55fb1f97ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 -- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 msgr2=0x55fb1f97ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 0x55fb1f97ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 -- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 msgr2=0x55fb1f97f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f97f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 -- 172.21.15.110:0/443150421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb1f70bfd0 con 0x55fb1f93b390 2024-09-10T10:03:11.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f3990a00640 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f97f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:11.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:10.999+0000 7f398be00640 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f93b390 0x55fb1f996850 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f397c01a530 tx=0x7f397c0ed050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8e026b2272d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f397be00640 1 -- 172.21.15.110:0/443150421 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0f97b0 con 0x55fb1f93b390 2024-09-10T10:03:11.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f397be00640 1 -- 172.21.15.110:0/443150421 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c0eed20 con 0x55fb1f93b390 2024-09-10T10:03:11.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f397be00640 1 -- 172.21.15.110:0/443150421 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0ed2f0 con 0x55fb1f93b390 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- 172.21.15.110:0/443150421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f93b390 msgr2=0x55fb1f996850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f93b390 0x55fb1f996850 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f397c01a530 tx=0x7f397c0ed050 comp rx=0 tx=0).stop 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- 172.21.15.110:0/443150421 shutdown_connections 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f980440 0x55fb1f97f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f97fb20 0x55fb1f97ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 --2- 172.21.15.110:0/443150421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f93b390 0x55fb1f996850 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- 172.21.15.110:0/443150421 >> 172.21.15.110:0/443150421 conn(0x55fb1f936290 msgr2=0x55fb1f93a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- 172.21.15.110:0/443150421 shutdown_connections 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- 172.21.15.110:0/443150421 wait complete. 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- start start 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 Processor -- start 2024-09-10T10:03:11.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f93b390 0x55fb1f9f3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f97fb20 0x55fb1f9f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f980440 0x55fb1f9897f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fb1f783f80 con 0x55fb1f97fb20 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fb1f73cca0 con 0x55fb1f93b390 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55fb1f7892a0 con 0x55fb1f980440 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 10 client.0 _ll_register_callbacks cb 0x7f3974006c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f97fb20 0x55fb1f9f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f97fb20 0x55fb1f9f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54780/0 (socket says 172.21.15.110:54780) 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 -- 172.21.15.110:0/229646933 learned_addr learned my addr 172.21.15.110:0/229646933 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3990a00640 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f980440 0x55fb1f9897f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f398be00640 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f93b390 0x55fb1f9f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 -- 172.21.15.110:0/229646933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f980440 msgr2=0x55fb1f9897f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f980440 0x55fb1f9897f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 -- 172.21.15.110:0/229646933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f93b390 msgr2=0x55fb1f9f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f93b390 0x55fb1f9f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 -- 172.21.15.110:0/229646933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1f807750 con 0x55fb1f97fb20 2024-09-10T10:03:11.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3983400640 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f97fb20 0x55fb1f9f35a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f39740067b0 tx=0x7f3974015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a9881a19a9ffbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3979600640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:03:11.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3979600640 1 -- 172.21.15.110:0/229646933 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740062e0 con 0x55fb1f97fb20 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3979600640 1 -- 172.21.15.110:0/229646933 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f39740236e0 con 0x55fb1f97fb20 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 1 -- 172.21.15.110:0/229646933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397c0eed20 con 0x55fb1f97fb20 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3979600640 1 -- 172.21.15.110:0/229646933 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f39740238e0 con 0x55fb1f97fb20 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.003+0000 7f3995e35000 10 client.6930 fetch_fsmap learned FSMap version 68 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 -- 172.21.15.110:0/229646933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55fb1f712a30 con 0x55fb1f97fb20 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3979600640 1 -- 172.21.15.110:0/229646933 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c0eed20 con 0x55fb1f97fb20 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3979600640 1 -- 172.21.15.110:0/229646933 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7f3974020040 con 0x55fb1f97fb20 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 10 client.6930 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 10 client.6930 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 20 client.6930 populate_metadata read hostname 'smithi110' 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] conn(0x55fb1f989d30 0x55fb1f98a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 4 client.6930 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 -- 172.21.15.110:0/229646933 --> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] -- command(tid 0: {"prefix": "status"}) -- 0x55fb1f7923a0 con 0x55fb1f989d30 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f398be00640 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] conn(0x55fb1f989d30 0x55fb1f98a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3979600640 10 client.6930 ms_handle_connect on v2:172.21.15.110:6832/100964295 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f398be00640 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] conn(0x55fb1f989d30 0x55fb1f98a110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3566d85810704b83 server_cookie=2739052f0423381e in_seq=0 out_seq=0 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3979600640 1 -- 172.21.15.110:0/229646933 <== mds.0 v2:172.21.15.110:6832/100964295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55fb1f7923a0 con 0x55fb1f989d30 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3979600640 10 client.6930 handle_command_reply: tid=0 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 client.6930 shutdown 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 -- 172.21.15.110:0/229646933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f97fb20 msgr2=0x55fb1f9f35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f97fb20 0x55fb1f9f35a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f39740067b0 tx=0x7f3974015000 comp rx=0 tx=0).stop 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 -- 172.21.15.110:0/229646933 shutdown_connections 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55fb1f980440 0x55fb1f9897f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.110:6832/100964295,v1:172.21.15.110:6835/100964295] conn(0x55fb1f989d30 0x55fb1f98a110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb1f97fb20 0x55fb1f9f35a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 --2- 172.21.15.110:0/229646933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fb1f93b390 0x55fb1f9f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3979600640 0 client.6930 ms_handle_reset on v2:172.21.15.110:6832/100964295 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 -- 172.21.15.110:0/229646933 >> 172.21.15.110:0/229646933 conn(0x55fb1f936290 msgr2=0x55fb1f97dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 -- 172.21.15.110:0/229646933 shutdown_connections 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 1 -- 172.21.15.110:0/229646933 wait complete. 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3995e35000 20 client.6930 trim_cache size 0 max 16384 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 6801, 2024-09-10T10:03:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:03:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:03:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:03:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 5.8795361660000003, 2024-09-10T10:03:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 68, 2024-09-10T10:03:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 103, 2024-09-10T10:03:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 103, 2024-09-10T10:03:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 11.092679394999999 2024-09-10T10:03:11.016 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 -- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3968067a10 msgr2=0x7f3968069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3968067a10 0x7f3968069ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3974007fd0 tx=0x7f3974007d80 comp rx=0 tx=0).stop 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 -- 172.21.15.110:0/1071375385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f398c070540 msgr2=0x7f398c146d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f398c070540 0x7f398c146d10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f397c000f90 tx=0x7f397c0196a0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 -- 172.21.15.110:0/1071375385 shutdown_connections 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3968067a10 0x7f3968069ed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f398c15c520 0x7f398c152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f398c0b64c0 0x7f398c152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 --2- 172.21.15.110:0/1071375385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f398c070540 0x7f398c146d10 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 -- 172.21.15.110:0/1071375385 >> 172.21.15.110:0/1071375385 conn(0x7f398c06cd60 msgr2=0x7f398c157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 -- 172.21.15.110:0/1071375385 shutdown_connections 2024-09-10T10:03:11.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.007+0000 7f3991400640 1 -- 172.21.15.110:0/1071375385 wait complete. 2024-09-10T10:03:11.029 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:03:11.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a4c600640 1 Processor -- start 2024-09-10T10:03:11.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a4c600640 1 -- start start 2024-09-10T10:03:11.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a4c600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a44155970 0x7f1a440cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a4c600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1a4415c560 0x7f1a4406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a4c600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a44155d50 con 0x7f1a4406f010 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a4c600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1a440b4570 con 0x7f1a4415c560 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a4c600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4406f450 con 0x7f1a44155970 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a49c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a44155970 0x7f1a440cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a49200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1a4415c560 0x7f1a4406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a49c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a44155970 0x7f1a440cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:46472/0 (socket says 172.21.15.110:46472) 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a49c00640 1 -- 172.21.15.110:0/2328182661 learned_addr learned my addr 172.21.15.110:0/2328182661 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a49c00640 1 -- 172.21.15.110:0/2328182661 >> v2:172.21.15.145:3300/0 conn(0x7f1a4415c560 msgr2=0x7f1a4406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a49c00640 1 --2- 172.21.15.110:0/2328182661 >> v2:172.21.15.145:3300/0 conn(0x7f1a4415c560 0x7f1a4406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a49c00640 1 -- 172.21.15.110:0/2328182661 >> v1:172.21.15.145:6789/0 conn(0x7f1a4406f010 legacy=0x7f1a4415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.163+0000 7f1a49c00640 1 -- 172.21.15.110:0/2328182661 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a440b34d0 con 0x7f1a44155970 2024-09-10T10:03:11.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a49c00640 1 --2- 172.21.15.110:0/2328182661 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a44155970 0x7f1a440cc8c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a4406f5d0 tx=0x7f1a3801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685ec78eab88e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a43e00640 1 -- 172.21.15.110:0/2328182661 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a380a5810 con 0x7f1a44155970 2024-09-10T10:03:11.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a43e00640 1 -- 172.21.15.110:0/2328182661 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a380a59b0 con 0x7f1a44155970 2024-09-10T10:03:11.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a43e00640 1 -- 172.21.15.110:0/2328182661 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a380a5d80 con 0x7f1a44155970 2024-09-10T10:03:11.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a4c600640 1 -- 172.21.15.110:0/2328182661 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a44155970 msgr2=0x7f1a440cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a4c600640 1 --2- 172.21.15.110:0/2328182661 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a44155970 0x7f1a440cc8c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a4406f5d0 tx=0x7f1a3801c950 comp rx=0 tx=0).stop 2024-09-10T10:03:11.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a4c600640 1 -- 172.21.15.110:0/2328182661 shutdown_connections 2024-09-10T10:03:11.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a4c600640 1 --2- 172.21.15.110:0/2328182661 >> v2:172.21.15.145:3300/0 conn(0x7f1a4415c560 0x7f1a4406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a4c600640 1 --2- 172.21.15.110:0/2328182661 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a44155970 0x7f1a440cc8c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a4c600640 1 -- 172.21.15.110:0/2328182661 >> 172.21.15.110:0/2328182661 conn(0x7f1a4406bc50 msgr2=0x7f1a4406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a4c600640 1 -- 172.21.15.110:0/2328182661 shutdown_connections 2024-09-10T10:03:11.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.167+0000 7f1a4c600640 1 -- 172.21.15.110:0/2328182661 wait complete. 2024-09-10T10:03:11.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4c600640 1 Processor -- start 2024-09-10T10:03:11.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4c600640 1 -- start start 2024-09-10T10:03:11.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4c600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4406f010 0x7f1a4415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4c600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a44155970 0x7f1a44156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4c600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4415c560 0x7f1a440b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4c600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1a440cce00 con 0x7f1a44155970 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4c600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1a440b4050 con 0x7f1a4406f010 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4c600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1a44166020 con 0x7f1a4415c560 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a44155970 0x7f1a44156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a44155970 0x7f1a44156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54796/0 (socket says 172.21.15.110:54796) 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 -- 172.21.15.110:0/392096546 learned_addr learned my addr 172.21.15.110:0/392096546 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49c00640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4406f010 0x7f1a4415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a4a600640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4415c560 0x7f1a440b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 -- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4415c560 msgr2=0x7f1a440b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4415c560 0x7f1a440b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 -- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4406f010 msgr2=0x7f1a4415eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4406f010 0x7f1a4415eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 -- 172.21.15.110:0/392096546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a441573e0 con 0x7f1a44155970 2024-09-10T10:03:11.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.171+0000 7f1a49200640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a44155970 0x7f1a44156b70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e840 tx=0x7f1a3000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.175+0000 7f1a41600640 1 -- 172.21.15.110:0/392096546 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30017070 con 0x7f1a44155970 2024-09-10T10:03:11.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.175+0000 7f1a41600640 1 -- 172.21.15.110:0/392096546 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3000ee90 con 0x7f1a44155970 2024-09-10T10:03:11.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.175+0000 7f1a41600640 1 -- 172.21.15.110:0/392096546 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000cb60 con 0x7f1a44155970 2024-09-10T10:03:11.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.175+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a441576c0 con 0x7f1a44155970 2024-09-10T10:03:11.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.175+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a44157c80 con 0x7f1a44155970 2024-09-10T10:03:11.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.175+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a440cc8c0 con 0x7f1a44155970 2024-09-10T10:03:11.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.175+0000 7f1a41600640 1 -- 172.21.15.110:0/392096546 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f1a3001a070 con 0x7f1a44155970 2024-09-10T10:03:11.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.179+0000 7f1a41600640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1a240679e0 0x7f1a24069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.179+0000 7f1a41600640 1 -- 172.21.15.110:0/392096546 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f1a30085f10 con 0x7f1a44155970 2024-09-10T10:03:11.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.179+0000 7f1a49c00640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1a240679e0 0x7f1a24069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.179+0000 7f1a49c00640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1a240679e0 0x7f1a24069ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1a380ae000 tx=0x7f1a380a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.183+0000 7f1a41600640 1 -- 172.21.15.110:0/392096546 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1a30056360 con 0x7f1a44155970 2024-09-10T10:03:11.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1a4406df50 con 0x7f1a44155970 2024-09-10T10:03:11.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a41600640 1 -- 172.21.15.110:0/392096546 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) ==== 53+0+73 (secure 0 0 0) 0x7f1a3005a660 con 0x7f1a44155970 2024-09-10T10:03:11.378 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:03:11.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1a240679e0 msgr2=0x7f1a24069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1a240679e0 0x7f1a24069ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1a380ae000 tx=0x7f1a380a7070 comp rx=0 tx=0).stop 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a44155970 msgr2=0x7f1a44156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a44155970 0x7f1a44156b70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e840 tx=0x7f1a3000ec40 comp rx=0 tx=0).stop 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 shutdown_connections 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1a240679e0 0x7f1a24069ea0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a4415c560 0x7f1a440b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1a44155970 0x7f1a44156b70 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 --2- 172.21.15.110:0/392096546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a4406f010 0x7f1a4415eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 >> 172.21.15.110:0/392096546 conn(0x7f1a4406bc50 msgr2=0x7f1a4406e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 shutdown_connections 2024-09-10T10:03:11.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11.375+0000 7f1a4c600640 1 -- 172.21.15.110:0/392096546 wait complete. 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:03:11.400 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:03:11.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:11.401 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:03:11.410 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:03:11.410 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:03:11.410 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:03:11.410 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42183sec preferred_lft 42183sec 2024-09-10T10:03:11.411 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:03:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:03:11.413 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:03:11.413 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:11.413 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:03:11.413 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:03:11.413 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:03:11.413 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:11.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:11.413 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:03:11.474 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:03:11.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:11.476 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:03:11.527 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:03:11.527 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:03:11.527 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:03:11.527 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:03:11.527 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:03:11.527 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:03:11.528 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:03:11.528 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:03:11.528 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:03:11.528 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:03:11.528 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:03:11.528 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:03:11.528 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:03:11.528 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:03:11.528 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:03:11.528 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:03:11.528 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:03:11.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:03:11.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:11.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:11.661 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:03:11.670 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:03:11.670 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:03:11.671 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:03:11.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:11.671 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:03:11.732 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:03:11.732 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:11.780 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:03:11.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:03:11.781 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:11.835 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:03:11.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:11.835 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:03:11.928 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:03:11.928 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:03:11.942 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:03:11.942 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:03:11.942 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:03:11.942 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:03:11.942 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:03:11.946 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:03:11.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:11.946 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:11.966 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:03:12.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:03:12.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:12.075 INFO:teuthology.orchestra.run.smithi110.stdout:client5966 2024-09-10T10:03:12.075 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:12.075 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:12.129 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:03:12.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.303+0000 7f4d17e00640 1 Processor -- start 2024-09-10T10:03:12.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d17e00640 1 -- start start 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d17e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4d180ce730 0x7f4d180b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d17e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4d180b62b0 0x7f4d1806dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d17e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d180cebf0 con 0x7f4d180b5000 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d17e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4d180b6720 con 0x7f4d180ce730 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d17e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4d180ceed0 con 0x7f4d180b62b0 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d16000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4d180ce730 0x7f4d180b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d16000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4d180ce730 0x7f4d180b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54800/0 (socket says 172.21.15.110:54800) 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d16000640 1 -- 172.21.15.110:0/3559620552 learned_addr learned my addr 172.21.15.110:0/3559620552 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:12.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d17400640 1 --2- 172.21.15.110:0/3559620552 >> v2:172.21.15.145:3300/0 conn(0x7f4d180b62b0 0x7f4d1806dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d16000640 1 -- 172.21.15.110:0/3559620552 >> v2:172.21.15.145:3300/0 conn(0x7f4d180b62b0 msgr2=0x7f4d1806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:12.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d16000640 1 --2- 172.21.15.110:0/3559620552 >> v2:172.21.15.145:3300/0 conn(0x7f4d180b62b0 0x7f4d1806dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d16000640 1 -- 172.21.15.110:0/3559620552 >> v1:172.21.15.145:6789/0 conn(0x7f4d180b5000 legacy=0x7f4d1815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:12.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d16000640 1 -- 172.21.15.110:0/3559620552 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d180cf1b0 con 0x7f4d180ce730 2024-09-10T10:03:12.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.307+0000 7f4d17400640 1 --2- 172.21.15.110:0/3559620552 >> v2:172.21.15.145:3300/0 conn(0x7f4d180b62b0 0x7f4d1806dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:12.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d16000640 1 --2- 172.21.15.110:0/3559620552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180ce730 0x7f4d180b5d70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4d180ce270 tx=0x7f4d0401eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1b446907e5dea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d15600640 1 -- 172.21.15.110:0/3559620552 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d040b4070 con 0x7f4d180ce730 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d15600640 1 -- 172.21.15.110:0/3559620552 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d040a79f0 con 0x7f4d180ce730 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d15600640 1 -- 172.21.15.110:0/3559620552 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d040a7dc0 con 0x7f4d180ce730 2024-09-10T10:03:12.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 -- 172.21.15.110:0/3559620552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180ce730 msgr2=0x7f4d180b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 --2- 172.21.15.110:0/3559620552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180ce730 0x7f4d180b5d70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4d180ce270 tx=0x7f4d0401eb20 comp rx=0 tx=0).stop 2024-09-10T10:03:12.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 -- 172.21.15.110:0/3559620552 shutdown_connections 2024-09-10T10:03:12.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 --2- 172.21.15.110:0/3559620552 >> v2:172.21.15.145:3300/0 conn(0x7f4d180b62b0 0x7f4d1806dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 --2- 172.21.15.110:0/3559620552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180ce730 0x7f4d180b5d70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 -- 172.21.15.110:0/3559620552 >> 172.21.15.110:0/3559620552 conn(0x7f4d1806c590 msgr2=0x7f4d1806c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:12.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 -- 172.21.15.110:0/3559620552 shutdown_connections 2024-09-10T10:03:12.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 -- 172.21.15.110:0/3559620552 wait complete. 2024-09-10T10:03:12.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 Processor -- start 2024-09-10T10:03:12.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.311+0000 7f4d17e00640 1 -- start start 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180b5000 0x7f4d181489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d180b62b0 0x7f4d1814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d180ce730 0x7f4d1814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1806e9b0 con 0x7f4d180b5000 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4d180b55f0 con 0x7f4d180ce730 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1806eb30 con 0x7f4d180b62b0 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180b5000 0x7f4d181489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180b5000 0x7f4d181489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54816/0 (socket says 172.21.15.110:54816) 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 -- 172.21.15.110:0/136581561 learned_addr learned my addr 172.21.15.110:0/136581561 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17400640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d180ce730 0x7f4d1814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16000640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d180b62b0 0x7f4d1814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 -- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d180b62b0 msgr2=0x7f4d1814eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d180b62b0 0x7f4d1814eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 -- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d180ce730 msgr2=0x7f4d1814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d180ce730 0x7f4d1814f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 -- 172.21.15.110:0/136581561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1814dae0 con 0x7f4d180b5000 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17400640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d180ce730 0x7f4d1814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16000640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d180b62b0 0x7f4d1814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d16a00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180b5000 0x7f4d181489c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800c7f0 tx=0x7f4d0800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d0ea00640 1 -- 172.21.15.110:0/136581561 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08016020 con 0x7f4d180b5000 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d0ea00640 1 -- 172.21.15.110:0/136581561 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0800aa50 con 0x7f4d180b5000 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d0ea00640 1 -- 172.21.15.110:0/136581561 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800ae40 con 0x7f4d180b5000 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1814dd80 con 0x7f4d180b5000 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1814e340 con 0x7f4d180b5000 2024-09-10T10:03:12.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1806dc60 con 0x7f4d180b5000 2024-09-10T10:03:12.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.315+0000 7f4d0ea00640 1 -- 172.21.15.110:0/136581561 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f4d08019020 con 0x7f4d180b5000 2024-09-10T10:03:12.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.319+0000 7f4d0ea00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4cf80679e0 0x7f4cf8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.319+0000 7f4d0ea00640 1 -- 172.21.15.110:0/136581561 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f4d08085ea0 con 0x7f4d180b5000 2024-09-10T10:03:12.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.319+0000 7f4d16000640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4cf80679e0 0x7f4cf8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.319+0000 7f4d16000640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4cf80679e0 0x7f4cf8069ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4d040b1000 tx=0x7f4d040a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:12.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.323+0000 7f4d0ea00640 1 -- 172.21.15.110:0/136581561 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4d08020030 con 0x7f4d180b5000 2024-09-10T10:03:12.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4d18075760 con 0x7f4d180b5000 2024-09-10T10:03:12.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d0ea00640 1 -- 172.21.15.110:0/136581561 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) ==== 53+0+73 (secure 0 0 0) 0x7f4d080568c0 con 0x7f4d180b5000 2024-09-10T10:03:12.518 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:03:12.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4cf80679e0 msgr2=0x7f4cf8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4cf80679e0 0x7f4cf8069ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4d040b1000 tx=0x7f4d040a9070 comp rx=0 tx=0).stop 2024-09-10T10:03:12.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180b5000 msgr2=0x7f4d181489c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180b5000 0x7f4d181489c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800c7f0 tx=0x7f4d0800ccb0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 shutdown_connections 2024-09-10T10:03:12.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4cf80679e0 0x7f4cf8069ea0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d180ce730 0x7f4d1814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d180b62b0 0x7f4d1814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 --2- 172.21.15.110:0/136581561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d180b5000 0x7f4d181489c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 >> 172.21.15.110:0/136581561 conn(0x7f4d1806c590 msgr2=0x7f4d18159800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:12.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 shutdown_connections 2024-09-10T10:03:12.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:12.515+0000 7f4d17e00640 1 -- 172.21.15.110:0/136581561 wait complete. 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:03:12.541 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:03:12.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:12.542 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:03:12.552 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:03:12.552 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:03:12.552 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:03:12.552 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:12.552 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:03:12.552 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:12.552 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:03:12.552 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42179sec preferred_lft 42179sec 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:12.553 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:03:12.554 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:12.555 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:03:12.555 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:03:12.555 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:03:12.555 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:12.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:12.556 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:03:12.619 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:03:12.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:12.622 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:03:12.679 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:03:12.679 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:03:12.680 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:03:12.681 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:03:12.681 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:03:12.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:03:12.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:03:12.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:03:12.681 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:03:12.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:03:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:03:12.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:12.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:12.825 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:03:12.834 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:03:12.834 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:03:12.835 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:03:12.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:12.835 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:03:12.902 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:03:12.902 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:03:12.954 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:03:12.954 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:03:12.954 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:03:13.006 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:03:13.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:13.006 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:03:13.098 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:03:13.098 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:03:13.111 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:03:13.112 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:03:13.112 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:03:13.112 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:03:13.112 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:03:13.116 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:03:13.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:13.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:03:13.136 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:03:13.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:03:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:03:13.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:03:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:13.234 INFO:teuthology.orchestra.run.smithi145.stdout:client6942 2024-09-10T10:03:13.234 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:03:13.235 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:13.243 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:03:13.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab3a00640 1 Processor -- start 2024-09-10T10:03:13.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab3a00640 1 -- start start 2024-09-10T10:03:13.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab3a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9aac0cfe30 0x7f9aac071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab3a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 0x7f9aac0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab3a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9aac06e090 0x7f9aac06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab3a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9aac0cdeb0 con 0x7f9aac0cfe30 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab3a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9aac15d9a0 con 0x7f9aac06e090 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab3a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac06ea60 con 0x7f9aac0cd510 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9aac0cfe30 0x7f9aac071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9aac0cfe30 0x7f9aac071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54818/0 (socket says 172.21.15.110:54818) 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab1000640 1 -- 172.21.15.110:0/2152254135 learned_addr learned my addr 172.21.15.110:0/2152254135 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9aabe00640 1 --2- 172.21.15.110:0/2152254135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 0x7f9aac0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.379+0000 7f9ab1a00640 1 --2- 172.21.15.110:0/2152254135 >> v2:172.21.15.145:3300/0 conn(0x7f9aac06e090 0x7f9aac06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9ab1000640 1 -- 172.21.15.110:0/2152254135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 msgr2=0x7f9aac0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:13.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9ab1000640 1 --2- 172.21.15.110:0/2152254135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 0x7f9aac0cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9ab1000640 1 -- 172.21.15.110:0/2152254135 >> v2:172.21.15.145:3300/0 conn(0x7f9aac06e090 msgr2=0x7f9aac06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:13.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9ab1000640 1 --2- 172.21.15.110:0/2152254135 >> v2:172.21.15.145:3300/0 conn(0x7f9aac06e090 0x7f9aac06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9ab1000640 1 -- 172.21.15.110:0/2152254135 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac06d2d0 con 0x7f9aac0cfe30 2024-09-10T10:03:13.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9ab1000640 1 --2- 172.21.15.110:0/2152254135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac0cfe30 0x7f9aac071a40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9aac15a560 tx=0x7f9a9c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6293074ebb3ef58c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9aab400640 1 -- 172.21.15.110:0/2152254135 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c0b2070 con 0x7f9aac0cfe30 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9aab400640 1 -- 172.21.15.110:0/2152254135 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a9c0a9070 con 0x7f9aac0cfe30 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9aab400640 1 -- 172.21.15.110:0/2152254135 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c0ad880 con 0x7f9aac0cfe30 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9ab3a00640 1 -- 172.21.15.110:0/2152254135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac0cfe30 msgr2=0x7f9aac071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.383+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/2152254135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac0cfe30 0x7f9aac071a40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9aac15a560 tx=0x7f9a9c01c490 comp rx=0 tx=0).stop 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- 172.21.15.110:0/2152254135 shutdown_connections 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/2152254135 >> v2:172.21.15.145:3300/0 conn(0x7f9aac06e090 0x7f9aac06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/2152254135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 0x7f9aac0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/2152254135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac0cfe30 0x7f9aac071a40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- 172.21.15.110:0/2152254135 >> 172.21.15.110:0/2152254135 conn(0x7f9aac06bc50 msgr2=0x7f9aac06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:13.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- 172.21.15.110:0/2152254135 shutdown_connections 2024-09-10T10:03:13.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- 172.21.15.110:0/2152254135 wait complete. 2024-09-10T10:03:13.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 Processor -- start 2024-09-10T10:03:13.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- start start 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac06e090 0x7f9aac14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 0x7f9aac0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9aac0cfe30 0x7f9aac157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac15d430 con 0x7f9aac06e090 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac072190 con 0x7f9aac0cfe30 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac15d5b0 con 0x7f9aac0cd510 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac06e090 0x7f9aac14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac06e090 0x7f9aac14fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54826/0 (socket says 172.21.15.110:54826) 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 -- 172.21.15.110:0/3840103872 learned_addr learned my addr 172.21.15.110:0/3840103872 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:13.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9aabe00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 0x7f9aac0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1a00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9aac0cfe30 0x7f9aac157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 -- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 msgr2=0x7f9aac0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 0x7f9aac0b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 -- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9aac0cfe30 msgr2=0x7f9aac157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9aac0cfe30 0x7f9aac157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 -- 172.21.15.110:0/3840103872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac150100 con 0x7f9aac06e090 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab1000640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac06e090 0x7f9aac14fbc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c01ab40 tx=0x7f9a9c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9aa8c00640 1 -- 172.21.15.110:0/3840103872 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c0b2040 con 0x7f9aac06e090 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9aa8c00640 1 -- 172.21.15.110:0/3840103872 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a9c0bb070 con 0x7f9aac06e090 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9aa8c00640 1 -- 172.21.15.110:0/3840103872 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c0b6440 con 0x7f9aac06e090 2024-09-10T10:03:13.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.387+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac0b19f0 con 0x7f9aac06e090 2024-09-10T10:03:13.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.391+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac0b1fb0 con 0x7f9aac06e090 2024-09-10T10:03:13.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.391+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac06e520 con 0x7f9aac06e090 2024-09-10T10:03:13.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.391+0000 7f9aa8c00640 1 -- 172.21.15.110:0/3840103872 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f9a9c0ab070 con 0x7f9aac06e090 2024-09-10T10:03:13.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.391+0000 7f9aa8c00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a8c0679e0 0x7f9a8c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.391+0000 7f9aa8c00640 1 -- 172.21.15.110:0/3840103872 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f9a9c11f730 con 0x7f9aac06e090 2024-09-10T10:03:13.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.391+0000 7f9aabe00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a8c0679e0 0x7f9a8c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.391+0000 7f9aabe00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a8c0679e0 0x7f9a8c069ea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9aac1fa930 tx=0x7f9a98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.395+0000 7f9aa8c00640 1 -- 172.21.15.110:0/3840103872 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9a9c016110 con 0x7f9aac06e090 2024-09-10T10:03:13.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:13.587+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f9aac0710f0 con 0x7f9aac06e090 2024-09-10T10:03:14.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9aa8c00640 1 -- 172.21.15.110:0/3840103872 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v69) ==== 136+0+0 (secure 0 0 0) 0x7f9a9c0f0160 con 0x7f9aac06e090 2024-09-10T10:03:14.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a8c0679e0 msgr2=0x7f9a8c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a8c0679e0 0x7f9a8c069ea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9aac1fa930 tx=0x7f9a98008040 comp rx=0 tx=0).stop 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac06e090 msgr2=0x7f9aac14fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac06e090 0x7f9aac14fbc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c01ab40 tx=0x7f9a9c019230 comp rx=0 tx=0).stop 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 shutdown_connections 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a8c0679e0 0x7f9a8c069ea0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9aac0cfe30 0x7f9aac157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9aac0cd510 0x7f9aac0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 --2- 172.21.15.110:0/3840103872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9aac06e090 0x7f9aac14fbc0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 >> 172.21.15.110:0/3840103872 conn(0x7f9aac06bc50 msgr2=0x7f9aac152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 shutdown_connections 2024-09-10T10:03:14.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.187+0000 7f9ab3a00640 1 -- 172.21.15.110:0/3840103872 wait complete. 2024-09-10T10:03:14.213 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:03:14.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb4a400640 1 Processor -- start 2024-09-10T10:03:14.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb4a400640 1 -- start start 2024-09-10T10:03:14.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb4a400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb440b5600 0x7feb440cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb4a400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feb4406efa0 0x7feb440b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb4a400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4415fda0 con 0x7feb440726e0 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb4a400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7feb440cee50 con 0x7feb4406efa0 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb4a400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feb440ce520 con 0x7feb440b5600 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb43400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7feb440726e0 0x7feb440b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52068/0 (socket says 172.21.15.110:52068) 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb43400640 1 -- 172.21.15.110:0/685223916 learned_addr learned my addr 172.21.15.110:0/685223916 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb43e00640 1 --2- 172.21.15.110:0/685223916 >> v2:172.21.15.110:3300/0 conn(0x7feb4406efa0 0x7feb440b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb42a00640 1 --2- 172.21.15.110:0/685223916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb440b5600 0x7feb440cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb43e00640 1 -- 172.21.15.110:0/685223916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb440b5600 msgr2=0x7feb440cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb43e00640 1 --2- 172.21.15.110:0/685223916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb440b5600 0x7feb440cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb43e00640 1 -- 172.21.15.110:0/685223916 >> v1:172.21.15.110:6789/0 conn(0x7feb440726e0 legacy=0x7feb440b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb43e00640 1 -- 172.21.15.110:0/685223916 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb440ce800 con 0x7feb4406efa0 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb42000640 1 -- 172.21.15.110:0/685223916 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443592361 0 0) 0x7feb4415fda0 con 0x7feb440726e0 2024-09-10T10:03:14.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.347+0000 7feb42a00640 1 --2- 172.21.15.110:0/685223916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb440b5600 0x7feb440cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:14.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb43e00640 1 --2- 172.21.15.110:0/685223916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb4406efa0 0x7feb440b34d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0097c0 tx=0x7feb3c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bcf83ea3aa6874 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb42000640 1 -- 172.21.15.110:0/685223916 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0ab650 con 0x7feb4406efa0 2024-09-10T10:03:14.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb42000640 1 -- 172.21.15.110:0/685223916 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3c0b0070 con 0x7feb4406efa0 2024-09-10T10:03:14.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb42000640 1 -- 172.21.15.110:0/685223916 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0aba10 con 0x7feb4406efa0 2024-09-10T10:03:14.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb4a400640 1 -- 172.21.15.110:0/685223916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb4406efa0 msgr2=0x7feb440b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb4a400640 1 --2- 172.21.15.110:0/685223916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb4406efa0 0x7feb440b34d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0097c0 tx=0x7feb3c01c470 comp rx=0 tx=0).stop 2024-09-10T10:03:14.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb4a400640 1 -- 172.21.15.110:0/685223916 shutdown_connections 2024-09-10T10:03:14.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb4a400640 1 --2- 172.21.15.110:0/685223916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb4406efa0 0x7feb440b34d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb4a400640 1 --2- 172.21.15.110:0/685223916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb440b5600 0x7feb440cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb4a400640 1 -- 172.21.15.110:0/685223916 >> 172.21.15.110:0/685223916 conn(0x7feb44062d10 msgr2=0x7feb4406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb4a400640 1 -- 172.21.15.110:0/685223916 shutdown_connections 2024-09-10T10:03:14.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.351+0000 7feb4a400640 1 -- 172.21.15.110:0/685223916 wait complete. 2024-09-10T10:03:14.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 Processor -- start 2024-09-10T10:03:14.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 -- start start 2024-09-10T10:03:14.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb4406efa0 0x7feb440b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb440726e0 0x7feb4415fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb440b5600 0x7feb441602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb44072c40 con 0x7feb440b5600 2024-09-10T10:03:14.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb44072ac0 con 0x7feb440726e0 2024-09-10T10:03:14.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feb4415f950 con 0x7feb4406efa0 2024-09-10T10:03:14.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb440b5600 0x7feb441602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb440b5600 0x7feb441602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54838/0 (socket says 172.21.15.110:54838) 2024-09-10T10:03:14.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 -- 172.21.15.110:0/577392129 learned_addr learned my addr 172.21.15.110:0/577392129 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:14.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb4406efa0 0x7feb440b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb42a00640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb440726e0 0x7feb4415fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 -- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb4406efa0 msgr2=0x7feb440b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb4406efa0 0x7feb440b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 -- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb440726e0 msgr2=0x7feb4415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb440726e0 0x7feb4415fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 -- 172.21.15.110:0/577392129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb440b12e0 con 0x7feb440b5600 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb43e00640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb440b5600 0x7feb441602e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0097c0 tx=0x7feb3c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb3b400640 1 -- 172.21.15.110:0/577392129 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c006e20 con 0x7feb440b5600 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb3b400640 1 -- 172.21.15.110:0/577392129 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3c004230 con 0x7feb440b5600 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb3b400640 1 -- 172.21.15.110:0/577392129 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c003620 con 0x7feb440b5600 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb440b15c0 con 0x7feb440b5600 2024-09-10T10:03:14.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb440b1b80 con 0x7feb440b5600 2024-09-10T10:03:14.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.355+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb440b34d0 con 0x7feb440b5600 2024-09-10T10:03:14.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.359+0000 7feb3b400640 1 -- 172.21.15.110:0/577392129 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7feb3c0037c0 con 0x7feb440b5600 2024-09-10T10:03:14.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.359+0000 7feb3b400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb240679e0 0x7feb24069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.359+0000 7feb3b400640 1 -- 172.21.15.110:0/577392129 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7feb3c120910 con 0x7feb440b5600 2024-09-10T10:03:14.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.359+0000 7feb43400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb240679e0 0x7feb24069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.359+0000 7feb43400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb240679e0 0x7feb24069ea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feb340012f0 tx=0x7feb34001770 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.363+0000 7feb3b400640 1 -- 172.21.15.110:0/577392129 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feb3c016090 con 0x7feb440b5600 2024-09-10T10:03:14.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.539+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7feb4406bfd0 con 0x7feb240679e0 2024-09-10T10:03:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb3b400640 1 -- 172.21.15.110:0/577392129 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7feb4406bfd0 con 0x7feb240679e0 2024-09-10T10:03:14.549 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:03:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:03:14.549 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:14.549 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb240679e0 msgr2=0x7feb24069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb240679e0 0x7feb24069ea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feb340012f0 tx=0x7feb34001770 comp rx=0 tx=0).stop 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb440b5600 msgr2=0x7feb441602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb440b5600 0x7feb441602e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0097c0 tx=0x7feb3c01acb0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 shutdown_connections 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb240679e0 0x7feb24069ea0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb440b5600 0x7feb441602e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb440726e0 0x7feb4415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 --2- 172.21.15.110:0/577392129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb4406efa0 0x7feb440b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 >> 172.21.15.110:0/577392129 conn(0x7feb44062d10 msgr2=0x7feb4415e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 shutdown_connections 2024-09-10T10:03:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.543+0000 7feb4a400640 1 -- 172.21.15.110:0/577392129 wait complete. 2024-09-10T10:03:14.566 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:03:14.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee33a00640 1 Processor -- start 2024-09-10T10:03:14.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee33a00640 1 -- start start 2024-09-10T10:03:14.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee33a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee33a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee2c0b29e0 0x7fee2c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee33a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee2c0b37c0 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee33a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fee2c0b2e20 con 0x7fee2c0b29e0 2024-09-10T10:03:14.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee33a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c0b3aa0 con 0x7fee2c145f30 2024-09-10T10:03:14.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee2be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee2c0b29e0 0x7fee2c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee31a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fee2c0cfbc0 0x7fee2c160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52082/0 (socket says 172.21.15.110:52082) 2024-09-10T10:03:14.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.707+0000 7fee31a00640 1 -- 172.21.15.110:0/2735606172 learned_addr learned my addr 172.21.15.110:0/2735606172 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:14.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee31000640 1 --2- 172.21.15.110:0/2735606172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee2be00640 1 -- 172.21.15.110:0/2735606172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 msgr2=0x7fee2c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:14.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee2be00640 1 --2- 172.21.15.110:0/2735606172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee2be00640 1 -- 172.21.15.110:0/2735606172 >> v1:172.21.15.110:6789/0 conn(0x7fee2c0cfbc0 legacy=0x7fee2c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee2be00640 1 -- 172.21.15.110:0/2735606172 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c0b3d80 con 0x7fee2c0b29e0 2024-09-10T10:03:14.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee31000640 1 --2- 172.21.15.110:0/2735606172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:14.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee2be00640 1 --2- 172.21.15.110:0/2735606172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0b29e0 0x7fee2c0cf680 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fee180097c0 tx=0x7fee1801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d88a1f1f4935cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee2b400640 1 -- 172.21.15.110:0/2735606172 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180b0070 con 0x7fee2c0b29e0 2024-09-10T10:03:14.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee2b400640 1 -- 172.21.15.110:0/2735606172 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee180ab600 con 0x7fee2c0b29e0 2024-09-10T10:03:14.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee2b400640 1 -- 172.21.15.110:0/2735606172 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180ab9d0 con 0x7fee2c0b29e0 2024-09-10T10:03:14.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.711+0000 7fee33a00640 1 -- 172.21.15.110:0/2735606172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0b29e0 msgr2=0x7fee2c0cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 --2- 172.21.15.110:0/2735606172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0b29e0 0x7fee2c0cf680 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fee180097c0 tx=0x7fee1801c4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- 172.21.15.110:0/2735606172 shutdown_connections 2024-09-10T10:03:14.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 --2- 172.21.15.110:0/2735606172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0b29e0 0x7fee2c0cf680 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 --2- 172.21.15.110:0/2735606172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- 172.21.15.110:0/2735606172 >> 172.21.15.110:0/2735606172 conn(0x7fee2c062e40 msgr2=0x7fee2c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- 172.21.15.110:0/2735606172 shutdown_connections 2024-09-10T10:03:14.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- 172.21.15.110:0/2735606172 wait complete. 2024-09-10T10:03:14.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 Processor -- start 2024-09-10T10:03:14.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- start start 2024-09-10T10:03:14.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee2c0b29e0 0x7fee2c13dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0cfbc0 0x7fee2c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c169530 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c1693b0 con 0x7fee2c0b29e0 2024-09-10T10:03:14.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c1696b0 con 0x7fee2c145f30 2024-09-10T10:03:14.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0cfbc0 0x7fee2c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0cfbc0 0x7fee2c0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54850/0 (socket says 172.21.15.110:54850) 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 -- 172.21.15.110:0/1510849733 learned_addr learned my addr 172.21.15.110:0/1510849733 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee31000640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee2c0b29e0 0x7fee2c13dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee31a00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 -- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 msgr2=0x7fee2c15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c15bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 -- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee2c0b29e0 msgr2=0x7fee2c13dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee2c0b29e0 0x7fee2c13dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 -- 172.21.15.110:0/1510849733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c0b0470 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee2be00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0cfbc0 0x7fee2c0b1bc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fee180ae040 tx=0x7fee18019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee28c00640 1 -- 172.21.15.110:0/1510849733 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180b8030 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee28c00640 1 -- 172.21.15.110:0/1510849733 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee180b4410 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee28c00640 1 -- 172.21.15.110:0/1510849733 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180b0040 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.715+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c0b0750 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.719+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c15e900 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.719+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c0cf680 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.719+0000 7fee28c00640 1 -- 172.21.15.110:0/1510849733 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fee180bb070 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.719+0000 7fee28c00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee0c0679e0 0x7fee0c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.719+0000 7fee28c00640 1 -- 172.21.15.110:0/1510849733 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fee1811f800 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.719+0000 7fee31000640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee0c0679e0 0x7fee0c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.723+0000 7fee31000640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee0c0679e0 0x7fee0c069ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fee20006fd0 tx=0x7fee20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.723+0000 7fee28c00640 1 -- 172.21.15.110:0/1510849733 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fee180c2070 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fee2c143190 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee28c00640 1 -- 172.21.15.110:0/1510849733 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fee18016090 con 0x7fee2c0cfbc0 2024-09-10T10:03:14.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee0c0679e0 msgr2=0x7fee0c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee0c0679e0 0x7fee0c069ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fee20006fd0 tx=0x7fee20008040 comp rx=0 tx=0).stop 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0cfbc0 msgr2=0x7fee2c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0cfbc0 0x7fee2c0b1bc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fee180ae040 tx=0x7fee18019240 comp rx=0 tx=0).stop 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 shutdown_connections 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fee0c0679e0 0x7fee0c069ea0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fee2c145f30 0x7fee2c15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee2c0cfbc0 0x7fee2c0b1bc0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 --2- 172.21.15.110:0/1510849733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee2c0b29e0 0x7fee2c13dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 >> 172.21.15.110:0/1510849733 conn(0x7fee2c062e40 msgr2=0x7fee2c072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 shutdown_connections 2024-09-10T10:03:14.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:14.883+0000 7fee33a00640 1 -- 172.21.15.110:0/1510849733 wait complete. 2024-09-10T10:03:14.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:14.908 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322) 2024-09-10T10:03:14.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:14.935 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/fake) 2024-09-10T10:03:15.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:15.003 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T10:03:15.054 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:03:15.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:15.054 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322) 2024-09-10T10:03:15.120 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:15.120 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol58330/.meta 2024-09-10T10:03:15.184 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol58330 2024-09-10T10:03:15.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca9600640 1 Processor -- start 2024-09-10T10:03:15.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca9600640 1 -- start start 2024-09-10T10:03:15.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca9600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca40ce4d0 0x7f5ca406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca9600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca40b4960 con 0x7f5ca415b330 2024-09-10T10:03:15.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca9600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca40b4c40 con 0x7f5ca406e850 2024-09-10T10:03:15.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca9600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca40b5570 con 0x7f5ca40ce4d0 2024-09-10T10:03:15.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca2a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5ca415b330 0x7f5ca406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52088/0 (socket says 172.21.15.110:52088) 2024-09-10T10:03:15.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca2a00640 1 -- 172.21.15.110:0/3831398514 learned_addr learned my addr 172.21.15.110:0/3831398514 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:15.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2690823815 0 0) 0x7f5ca40b4960 con 0x7f5ca415b330 2024-09-10T10:03:15.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca2000640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca40ce4d0 0x7f5ca406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c88003530 con 0x7f5ca415b330 2024-09-10T10:03:15.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 796124637 0 0) 0x7f5c88003530 con 0x7f5ca415b330 2024-09-10T10:03:15.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca40ce4d0 msgr2=0x7f5ca406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:15.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca40ce4d0 0x7f5ca406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 >> v1:172.21.15.145:6789/0 conn(0x7f5ca406e850 legacy=0x7f5ca40b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca40b4f20 con 0x7f5ca415b330 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f5c94001680 con 0x7f5ca415b330 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 >> v1:172.21.15.110:6789/0 conn(0x7f5ca415b330 legacy=0x7f5ca406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880039f0 0x7f5c88003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c88004810 0x7f5c88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c88005190 0x7f5c8800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40b5570 con 0x7f5c88005190 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8800e120 con 0x7f5c880039f0 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8800e2d0 con 0x7f5c88004810 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca3400640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c88005190 0x7f5c8800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca2a00640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880039f0 0x7f5c88003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.359+0000 7f5ca2000640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c88004810 0x7f5c88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca3400640 1 -- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c88004810 msgr2=0x7f5c88004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca3400640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5c88004810 0x7f5c88004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca3400640 1 -- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880039f0 msgr2=0x7f5c88003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:15.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca3400640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c880039f0 0x7f5c88003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca3400640 1 -- 172.21.15.110:0/3831398514 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c88003530 con 0x7f5c88005190 2024-09-10T10:03:15.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca3400640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c88005190 0x7f5c8800dbe0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4070ed0 tx=0x7f5c9c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6da22105aa80a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:15.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c001a90 con 0x7f5c88005190 2024-09-10T10:03:15.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c003b80 con 0x7f5c88005190 2024-09-10T10:03:15.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca1600640 1 -- 172.21.15.110:0/3831398514 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c005080 con 0x7f5c88005190 2024-09-10T10:03:15.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca9600640 1 -- 172.21.15.110:0/3831398514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c88005190 msgr2=0x7f5c8800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca9600640 1 --2- 172.21.15.110:0/3831398514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c88005190 0x7f5c8800dbe0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4070ed0 tx=0x7f5c9c01c730 comp rx=0 tx=0).stop 2024-09-10T10:03:15.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca3400640 1 -- 172.21.15.110:0/3831398514 reap_dead start 2024-09-10T10:03:15.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca9600640 1 -- 172.21.15.110:0/3831398514 shutdown_connections 2024-09-10T10:03:15.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.363+0000 7f5ca9600640 1 -- 172.21.15.110:0/3831398514 >> 172.21.15.110:0/3831398514 conn(0x7f5ca4062e40 msgr2=0x7f5ca406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:15.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 -- 172.21.15.110:0/3831398514 shutdown_connections 2024-09-10T10:03:15.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 -- 172.21.15.110:0/3831398514 wait complete. 2024-09-10T10:03:15.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 Processor -- start 2024-09-10T10:03:15.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 -- start start 2024-09-10T10:03:15.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c88005190 0x7f5ca415b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca414c020 0x7f5ca415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ca415c3a0 0x7f5ca415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca406c6a0 con 0x7f5ca415c3a0 2024-09-10T10:03:15.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40b1780 con 0x7f5c88005190 2024-09-10T10:03:15.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca406c820 con 0x7f5ca414c020 2024-09-10T10:03:15.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ca415c3a0 0x7f5ca415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ca415c3a0 0x7f5ca415d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54872/0 (socket says 172.21.15.110:54872) 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 -- 172.21.15.110:0/728650379 learned_addr learned my addr 172.21.15.110:0/728650379 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca2a00640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c88005190 0x7f5ca415b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca2000640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca414c020 0x7f5ca415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 -- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca414c020 msgr2=0x7f5ca415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca414c020 0x7f5ca415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 -- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c88005190 msgr2=0x7f5ca415b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c88005190 0x7f5ca415b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 -- 172.21.15.110:0/728650379 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca414d3d0 con 0x7f5ca415c3a0 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca2a00640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c88005190 0x7f5ca415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca2000640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca414c020 0x7f5ca415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:15.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca3400640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ca415c3a0 0x7f5ca415d720 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c01cfd0 tx=0x7f5c88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:15.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5c9aa00640 1 -- 172.21.15.110:0/728650379 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c001a90 con 0x7f5ca415c3a0 2024-09-10T10:03:15.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5c9aa00640 1 -- 172.21.15.110:0/728650379 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c005610 con 0x7f5ca415c3a0 2024-09-10T10:03:15.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca414d6b0 con 0x7f5ca415c3a0 2024-09-10T10:03:15.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.367+0000 7f5c9aa00640 1 -- 172.21.15.110:0/728650379 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c007dc0 con 0x7f5ca415c3a0 2024-09-10T10:03:15.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.371+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca414dc70 con 0x7f5ca415c3a0 2024-09-10T10:03:15.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.371+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca40ce4d0 con 0x7f5ca415c3a0 2024-09-10T10:03:15.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.371+0000 7f5c9aa00640 1 -- 172.21.15.110:0/728650379 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f5c9c0a7070 con 0x7f5ca415c3a0 2024-09-10T10:03:15.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.371+0000 7f5c9aa00640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5c84067a10 0x7f5c84069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.371+0000 7f5c9aa00640 1 -- 172.21.15.110:0/728650379 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f5c9c1207e0 con 0x7f5ca415c3a0 2024-09-10T10:03:15.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.371+0000 7f5ca2a00640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5c84067a10 0x7f5c84069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.375+0000 7f5ca2a00640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5c84067a10 0x7f5c84069ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000c10 tx=0x7f5c94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:15.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.375+0000 7f5c9aa00640 1 -- 172.21.15.110:0/728650379 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5c9c0ad020 con 0x7f5ca415c3a0 2024-09-10T10:03:15.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.563+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol58330", "target": ["mon-mgr", ""]}) -- 0x7f5ca414d180 con 0x7f5c84067a10 2024-09-10T10:03:15.575 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:15.575+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:15.575 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:15.575+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:15.575 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:15.575+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:15.575 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:15.575+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:15.575 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:15.575+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:15.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5c9aa00640 1 -- 172.21.15.110:0/728650379 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f5ca414d180 con 0x7f5c84067a10 2024-09-10T10:03:15.726 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5c84067a10 msgr2=0x7f5c84069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5c84067a10 0x7f5c84069ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000c10 tx=0x7f5c94009040 comp rx=0 tx=0).stop 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ca415c3a0 msgr2=0x7f5ca415d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ca415c3a0 0x7f5ca415d720 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c01cfd0 tx=0x7f5c88005840 comp rx=0 tx=0).stop 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 shutdown_connections 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5c84067a10 0x7f5c84069ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ca415c3a0 0x7f5ca415d720 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5ca414c020 0x7f5ca415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 --2- 172.21.15.110:0/728650379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c88005190 0x7f5ca415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 >> 172.21.15.110:0/728650379 conn(0x7f5ca4062e40 msgr2=0x7f5ca40727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 shutdown_connections 2024-09-10T10:03:15.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.723+0000 7f5ca9600640 1 -- 172.21.15.110:0/728650379 wait complete. 2024-09-10T10:03:15.748 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-10T10:03:15.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e9c00640 1 Processor -- start 2024-09-10T10:03:15.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e9c00640 1 -- start start 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e9c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f04e40b4ea0 0x7f04e406e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4147be0 0x7f04e40ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e9c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04e4148130 0x7f04e40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e9c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f04e40b5280 con 0x7f04e4148130 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e9c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f04e40b6a70 con 0x7f04e40b4ea0 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e9c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04e40b6d50 con 0x7f04e4147be0 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e3400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04e4148130 0x7f04e40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e3400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04e4148130 0x7f04e40ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54876/0 (socket says 172.21.15.110:54876) 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e3400640 1 -- 172.21.15.110:0/2117802302 learned_addr learned my addr 172.21.15.110:0/2117802302 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e2000640 1 --2- 172.21.15.110:0/2117802302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4147be0 0x7f04e40ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e2a00640 1 --2- 172.21.15.110:0/2117802302 >> v2:172.21.15.145:3300/0 conn(0x7f04e40b4ea0 0x7f04e406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e3400640 1 -- 172.21.15.110:0/2117802302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4147be0 msgr2=0x7f04e40ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:15.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e3400640 1 --2- 172.21.15.110:0/2117802302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4147be0 0x7f04e40ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e3400640 1 -- 172.21.15.110:0/2117802302 >> v2:172.21.15.145:3300/0 conn(0x7f04e40b4ea0 msgr2=0x7f04e406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:15.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e3400640 1 --2- 172.21.15.110:0/2117802302 >> v2:172.21.15.145:3300/0 conn(0x7f04e40b4ea0 0x7f04e406e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e3400640 1 -- 172.21.15.110:0/2117802302 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e40b7030 con 0x7f04e4148130 2024-09-10T10:03:15.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.883+0000 7f04e2a00640 1 --2- 172.21.15.110:0/2117802302 >> v2:172.21.15.145:3300/0 conn(0x7f04e40b4ea0 0x7f04e406e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:15.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e3400640 1 --2- 172.21.15.110:0/2117802302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e4148130 0x7f04e40ce740 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f04e4072770 tx=0x7f04dc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6e362b6796581a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:15.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e1600640 1 -- 172.21.15.110:0/2117802302 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0ab640 con 0x7f04e4148130 2024-09-10T10:03:15.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e1600640 1 -- 172.21.15.110:0/2117802302 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc0b0070 con 0x7f04e4148130 2024-09-10T10:03:15.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e1600640 1 -- 172.21.15.110:0/2117802302 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0aba00 con 0x7f04e4148130 2024-09-10T10:03:15.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 -- 172.21.15.110:0/2117802302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e4148130 msgr2=0x7f04e40ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2117802302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e4148130 0x7f04e40ce740 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f04e4072770 tx=0x7f04dc01c460 comp rx=0 tx=0).stop 2024-09-10T10:03:15.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 -- 172.21.15.110:0/2117802302 shutdown_connections 2024-09-10T10:03:15.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2117802302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e4148130 0x7f04e40ce740 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2117802302 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4147be0 0x7f04e40ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2117802302 >> v2:172.21.15.145:3300/0 conn(0x7f04e40b4ea0 0x7f04e406e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 -- 172.21.15.110:0/2117802302 >> 172.21.15.110:0/2117802302 conn(0x7f04e406cf30 msgr2=0x7f04e406d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:15.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 -- 172.21.15.110:0/2117802302 shutdown_connections 2024-09-10T10:03:15.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.887+0000 7f04e9c00640 1 -- 172.21.15.110:0/2117802302 wait complete. 2024-09-10T10:03:15.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 Processor -- start 2024-09-10T10:03:15.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 -- start start 2024-09-10T10:03:15.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e40b4ea0 0x7f04e40cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4147be0 0x7f04e4143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4148130 0x7f04e4144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4163000 con 0x7f04e40b4ea0 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4162e80 con 0x7f04e4147be0 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04e4163180 con 0x7f04e4148130 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e40b4ea0 0x7f04e40cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e40b4ea0 0x7f04e40cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54884/0 (socket says 172.21.15.110:54884) 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 -- 172.21.15.110:0/2559886069 learned_addr learned my addr 172.21.15.110:0/2559886069 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2000640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4147be0 0x7f04e4143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e3400640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4148130 0x7f04e4144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 -- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4148130 msgr2=0x7f04e4144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4148130 0x7f04e4144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 -- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4147be0 msgr2=0x7f04e4143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4147be0 0x7f04e4143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 -- 172.21.15.110:0/2559886069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e40cadb0 con 0x7f04e40b4ea0 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e2a00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e40b4ea0 0x7f04e40cbe50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f04d4007e40 tx=0x7f04d400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04daa00640 1 -- 172.21.15.110:0/2559886069 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d401a070 con 0x7f04e40b4ea0 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04daa00640 1 -- 172.21.15.110:0/2559886069 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d4003ba0 con 0x7f04e40b4ea0 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e40cb090 con 0x7f04e40b4ea0 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04daa00640 1 -- 172.21.15.110:0/2559886069 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d400de80 con 0x7f04e40b4ea0 2024-09-10T10:03:15.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e40cb650 con 0x7f04e40b4ea0 2024-09-10T10:03:15.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.891+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e406e240 con 0x7f04e40b4ea0 2024-09-10T10:03:15.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.895+0000 7f04daa00640 1 -- 172.21.15.110:0/2559886069 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f04d401d070 con 0x7f04e40b4ea0 2024-09-10T10:03:15.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.895+0000 7f04daa00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04c40679e0 0x7f04c4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:15.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.895+0000 7f04daa00640 1 -- 172.21.15.110:0/2559886069 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f04d40860a0 con 0x7f04e40b4ea0 2024-09-10T10:03:15.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.895+0000 7f04e2000640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04c40679e0 0x7f04c4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:15.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.895+0000 7f04e2000640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04c40679e0 0x7f04c4069ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f04e4163530 tx=0x7f04d0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:15.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:15.899+0000 7f04daa00640 1 -- 172.21.15.110:0/2559886069 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f04d4010070 con 0x7f04e40b4ea0 2024-09-10T10:03:16.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.079+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f04e4162070 con 0x7f04c40679e0 2024-09-10T10:03:16.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.083+0000 7f04daa00640 1 -- 172.21.15.110:0/2559886069 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f04e4162070 con 0x7f04c40679e0 2024-09-10T10:03:16.087 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:03:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "subvol58330" 2024-09-10T10:03:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:16.087 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:03:16.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04c40679e0 msgr2=0x7f04c4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04c40679e0 0x7f04c4069ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f04e4163530 tx=0x7f04d0009190 comp rx=0 tx=0).stop 2024-09-10T10:03:16.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e40b4ea0 msgr2=0x7f04e40cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e40b4ea0 0x7f04e40cbe50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f04d4007e40 tx=0x7f04d400d4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 shutdown_connections 2024-09-10T10:03:16.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04c40679e0 0x7f04c4069ea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04e4148130 0x7f04e4144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04e4147be0 0x7f04e4143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 --2- 172.21.15.110:0/2559886069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04e40b4ea0 0x7f04e40cbe50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 >> 172.21.15.110:0/2559886069 conn(0x7f04e406cf30 msgr2=0x7f04e40ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:16.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 shutdown_connections 2024-09-10T10:03:16.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.087+0000 7f04e9c00640 1 -- 172.21.15.110:0/2559886069 wait complete. 2024-09-10T10:03:16.108 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol58330 2024-09-10T10:03:16.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f048e600640 1 Processor -- start 2024-09-10T10:03:16.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f048e600640 1 -- start start 2024-09-10T10:03:16.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f048e600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0488145f30 0x7f04880b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f048e600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04880b37c0 con 0x7f04880b29e0 2024-09-10T10:03:16.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f048e600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04880b3aa0 con 0x7f04880cfd90 2024-09-10T10:03:16.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f048e600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f04880b2e50 con 0x7f0488145f30 2024-09-10T10:03:16.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0486a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f04880b29e0 0x7f04880cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52098/0 (socket says 172.21.15.110:52098) 2024-09-10T10:03:16.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0486a00640 1 -- 172.21.15.110:0/902867873 learned_addr learned my addr 172.21.15.110:0/902867873 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:16.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0487400640 1 --2- 172.21.15.110:0/902867873 >> v2:172.21.15.145:3300/0 conn(0x7f0488145f30 0x7f04880b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0486000640 1 -- 172.21.15.110:0/902867873 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4179647323 0 0) 0x7f04880b37c0 con 0x7f04880b29e0 2024-09-10T10:03:16.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0486000640 1 -- 172.21.15.110:0/902867873 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f046c003530 con 0x7f04880b29e0 2024-09-10T10:03:16.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0486000640 1 -- 172.21.15.110:0/902867873 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2001846247 0 0) 0x7f046c003530 con 0x7f04880b29e0 2024-09-10T10:03:16.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0487400640 1 -- 172.21.15.110:0/902867873 >> v1:172.21.15.145:6789/0 conn(0x7f04880cfd90 legacy=0x7f04880b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0487400640 1 -- 172.21.15.110:0/902867873 >> v1:172.21.15.110:6789/0 conn(0x7f04880b29e0 legacy=0x7f04880cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.251+0000 7f0487400640 1 -- 172.21.15.110:0/902867873 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04880b3d80 con 0x7f0488145f30 2024-09-10T10:03:16.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f0487400640 1 --2- 172.21.15.110:0/902867873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 0x7f04880b24a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f04780097c0 tx=0x7f047801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6c48f968a4ee161 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:16.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f0486000640 1 -- 172.21.15.110:0/902867873 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04780b0070 con 0x7f0488145f30 2024-09-10T10:03:16.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f0486000640 1 -- 172.21.15.110:0/902867873 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04780ab400 con 0x7f0488145f30 2024-09-10T10:03:16.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f0486000640 1 -- 172.21.15.110:0/902867873 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04740016a0 con 0x7f0488145f30 2024-09-10T10:03:16.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f048e600640 1 -- 172.21.15.110:0/902867873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 msgr2=0x7f04880b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f048e600640 1 --2- 172.21.15.110:0/902867873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 0x7f04880b24a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f04780097c0 tx=0x7f047801c3c0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f048e600640 1 -- 172.21.15.110:0/902867873 shutdown_connections 2024-09-10T10:03:16.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f048e600640 1 --2- 172.21.15.110:0/902867873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 0x7f04880b24a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f048e600640 1 -- 172.21.15.110:0/902867873 >> 172.21.15.110:0/902867873 conn(0x7f0488062e40 msgr2=0x7f048806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:16.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f048e600640 1 -- 172.21.15.110:0/902867873 shutdown_connections 2024-09-10T10:03:16.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.255+0000 7f048e600640 1 -- 172.21.15.110:0/902867873 wait complete. 2024-09-10T10:03:16.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 Processor -- start 2024-09-10T10:03:16.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 -- start start 2024-09-10T10:03:16.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04880b29e0 0x7f0488142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04880cfd90 0x7f048815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 0x7f04881f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04880b56f0 con 0x7f04880cfd90 2024-09-10T10:03:16.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04880d0170 con 0x7f0488145f30 2024-09-10T10:03:16.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04880b5870 con 0x7f04880b29e0 2024-09-10T10:03:16.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04880cfd90 0x7f048815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04880cfd90 0x7f048815faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54886/0 (socket says 172.21.15.110:54886) 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 -- 172.21.15.110:0/3219582289 learned_addr learned my addr 172.21.15.110:0/3219582289 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0487400640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04880b29e0 0x7f0488142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0487e00640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 0x7f04881f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 -- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04880b29e0 msgr2=0x7f0488142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04880b29e0 0x7f0488142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 -- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 msgr2=0x7f04881f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 0x7f04881f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 -- 172.21.15.110:0/3219582289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488143760 con 0x7f04880cfd90 2024-09-10T10:03:16.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f0486a00640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04880cfd90 0x7f048815faf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f047400c4e0 tx=0x7f047400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:16.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f047f400640 1 -- 172.21.15.110:0/3219582289 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474018070 con 0x7f04880cfd90 2024-09-10T10:03:16.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f047f400640 1 -- 172.21.15.110:0/3219582289 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04740074e0 con 0x7f04880cfd90 2024-09-10T10:03:16.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f047f400640 1 -- 172.21.15.110:0/3219582289 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474004a30 con 0x7f04880cfd90 2024-09-10T10:03:16.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0488143ef0 con 0x7f04880cfd90 2024-09-10T10:03:16.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0488143450 con 0x7f04880cfd90 2024-09-10T10:03:16.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.259+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04880b24a0 con 0x7f04880cfd90 2024-09-10T10:03:16.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.263+0000 7f047f400640 1 -- 172.21.15.110:0/3219582289 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f0474004bd0 con 0x7f04880cfd90 2024-09-10T10:03:16.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.263+0000 7f047f400640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04680679e0 0x7f0468069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.263+0000 7f047f400640 1 -- 172.21.15.110:0/3219582289 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f04740852d0 con 0x7f04880cfd90 2024-09-10T10:03:16.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.263+0000 7f0487400640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04680679e0 0x7f0468069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.263+0000 7f0487400640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04680679e0 0x7f0468069ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f04780ad000 tx=0x7f0478008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:16.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.267+0000 7f047f400640 1 -- 172.21.15.110:0/3219582289 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f04740555d0 con 0x7f04880cfd90 2024-09-10T10:03:16.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.451+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol58330", "target": ["mon-mgr", ""]}) -- 0x7f0488071550 con 0x7f04680679e0 2024-09-10T10:03:16.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.479+0000 7f047f400640 1 -- 172.21.15.110:0/3219582289 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+687 (secure 0 0 0) 0x7f0488071550 con 0x7f04680679e0 2024-09-10T10:03:16.483 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "atime": "2024-09-10 10:03:14", 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_pcent": "undefined", 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_quota": "infinite", 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_used": 0, 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "created_at": "2024-09-10 10:03:14", 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "ctime": "2024-09-10 10:03:15", 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "data_pool": "cephfs_data", 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "features": [ 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot-clone", 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot-autoprotect" 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "flavor": 1, 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "gid": 0, 2024-09-10T10:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": 16877, 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "mon_addrs": [ 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.110:6789", 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.145:6789", 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.145:6790" 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "mtime": "2024-09-10 10:03:14", 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "path": "/volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322", 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_namespace": "", 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "complete", 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "subvolume", 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout: "uid": 0 2024-09-10T10:03:16.485 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:16.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04680679e0 msgr2=0x7f0468069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04680679e0 0x7f0468069ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f04780ad000 tx=0x7f0478008040 comp rx=0 tx=0).stop 2024-09-10T10:03:16.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04880cfd90 msgr2=0x7f048815faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04880cfd90 0x7f048815faf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f047400c4e0 tx=0x7f047400c9a0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 shutdown_connections 2024-09-10T10:03:16.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04680679e0 0x7f0468069ea0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0488145f30 0x7f04881f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04880cfd90 0x7f048815faf0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 --2- 172.21.15.110:0/3219582289 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04880b29e0 0x7f0488142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 >> 172.21.15.110:0/3219582289 conn(0x7f0488062e40 msgr2=0x7f04880cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:16.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 shutdown_connections 2024-09-10T10:03:16.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.483+0000 7f048e600640 1 -- 172.21.15.110:0/3219582289 wait complete. 2024-09-10T10:03:16.505 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol58330 10485760 2024-09-10T10:03:16.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.639+0000 7fc48b200640 1 Processor -- start 2024-09-10T10:03:16.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.639+0000 7fc48b200640 1 -- start start 2024-09-10T10:03:16.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc48b200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc484155970 0x7fc4840cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc48b200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc48415c560 0x7fc48406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc48b200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc48406f430 con 0x7fc48406f010 2024-09-10T10:03:16.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc48b200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc4840b4570 con 0x7fc484155970 2024-09-10T10:03:16.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc48b200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc484155d50 con 0x7fc48415c560 2024-09-10T10:03:16.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc483e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc484155970 0x7fc4840cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc489200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc48406f010 0x7fc48415eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52112/0 (socket says 172.21.15.110:52112) 2024-09-10T10:03:16.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc489200640 1 -- 172.21.15.110:0/2683416719 learned_addr learned my addr 172.21.15.110:0/2683416719 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:16.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc483400640 1 --2- 172.21.15.110:0/2683416719 >> v2:172.21.15.145:3300/0 conn(0x7fc48415c560 0x7fc48406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc483e00640 1 -- 172.21.15.110:0/2683416719 >> v2:172.21.15.145:3300/0 conn(0x7fc48415c560 msgr2=0x7fc48406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:16.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc483e00640 1 --2- 172.21.15.110:0/2683416719 >> v2:172.21.15.145:3300/0 conn(0x7fc48415c560 0x7fc48406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc483e00640 1 -- 172.21.15.110:0/2683416719 >> v1:172.21.15.110:6789/0 conn(0x7fc48406f010 legacy=0x7fc48415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.643+0000 7fc483e00640 1 -- 172.21.15.110:0/2683416719 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4840b34d0 con 0x7fc484155970 2024-09-10T10:03:16.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc483e00640 1 --2- 172.21.15.110:0/2683416719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc484155970 0x7fc4840cc8c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc48406d930 tx=0x7fc47401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1eca07724869ece server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:16.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc482a00640 1 -- 172.21.15.110:0/2683416719 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4740ab650 con 0x7fc484155970 2024-09-10T10:03:16.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc482a00640 1 -- 172.21.15.110:0/2683416719 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4740b0070 con 0x7fc484155970 2024-09-10T10:03:16.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc482a00640 1 -- 172.21.15.110:0/2683416719 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4740aba10 con 0x7fc484155970 2024-09-10T10:03:16.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 -- 172.21.15.110:0/2683416719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc484155970 msgr2=0x7fc4840cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 --2- 172.21.15.110:0/2683416719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc484155970 0x7fc4840cc8c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc48406d930 tx=0x7fc47401c450 comp rx=0 tx=0).stop 2024-09-10T10:03:16.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 -- 172.21.15.110:0/2683416719 shutdown_connections 2024-09-10T10:03:16.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 --2- 172.21.15.110:0/2683416719 >> v2:172.21.15.145:3300/0 conn(0x7fc48415c560 0x7fc48406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 --2- 172.21.15.110:0/2683416719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc484155970 0x7fc4840cc8c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 -- 172.21.15.110:0/2683416719 >> 172.21.15.110:0/2683416719 conn(0x7fc48406bc50 msgr2=0x7fc48406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 -- 172.21.15.110:0/2683416719 shutdown_connections 2024-09-10T10:03:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 -- 172.21.15.110:0/2683416719 wait complete. 2024-09-10T10:03:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 Processor -- start 2024-09-10T10:03:16.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.647+0000 7fc48b200640 1 -- start start 2024-09-10T10:03:16.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc48406f010 0x7fc484150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc484155970 0x7fc4840ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc48415c560 0x7fc4840ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4840cce00 con 0x7fc48415c560 2024-09-10T10:03:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4840b4050 con 0x7fc484155970 2024-09-10T10:03:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc484168030 con 0x7fc48406f010 2024-09-10T10:03:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc48415c560 0x7fc4840ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc48415c560 0x7fc4840ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54910/0 (socket says 172.21.15.110:54910) 2024-09-10T10:03:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 -- 172.21.15.110:0/1307721921 learned_addr learned my addr 172.21.15.110:0/1307721921 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc483400640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc484155970 0x7fc4840ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc483e00640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc48406f010 0x7fc484150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 -- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc48406f010 msgr2=0x7fc484150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc48406f010 0x7fc484150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 -- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc484155970 msgr2=0x7fc4840ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc484155970 0x7fc4840ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 -- 172.21.15.110:0/1307721921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4841512b0 con 0x7fc48415c560 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc489200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc48415c560 0x7fc4840ab9f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00ca80 tx=0x7fc47c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc47be00640 1 -- 172.21.15.110:0/1307721921 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c017070 con 0x7fc48415c560 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc47be00640 1 -- 172.21.15.110:0/1307721921 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47c00ac30 con 0x7fc48415c560 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc47be00640 1 -- 172.21.15.110:0/1307721921 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c012600 con 0x7fc48415c560 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48415de10 con 0x7fc48415c560 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48415e3d0 con 0x7fc48415c560 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4840ad2f0 con 0x7fc48415c560 2024-09-10T10:03:16.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.651+0000 7fc47be00640 1 -- 172.21.15.110:0/1307721921 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fc47c01a070 con 0x7fc48415c560 2024-09-10T10:03:16.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.655+0000 7fc47be00640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc4640679e0 0x7fc464069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:16.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.655+0000 7fc47be00640 1 -- 172.21.15.110:0/1307721921 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fc47c085ac0 con 0x7fc48415c560 2024-09-10T10:03:16.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.655+0000 7fc483e00640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc4640679e0 0x7fc464069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:16.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.655+0000 7fc483e00640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc4640679e0 0x7fc464069ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc4740af000 tx=0x7fc4740a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:16.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.659+0000 7fc47be00640 1 -- 172.21.15.110:0/1307721921 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc4840ad2f0 con 0x7fc48415c560 2024-09-10T10:03:16.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.843+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol58330", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7fc4840cc8c0 con 0x7fc4640679e0 2024-09-10T10:03:16.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc47be00640 1 -- 172.21.15.110:0/1307721921 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7fc4840cc8c0 con 0x7fc4640679e0 2024-09-10T10:03:16.877 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_used": 0 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_quota": 10485760 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_pcent": "0.00" 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:16.878 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:03:16.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc4640679e0 msgr2=0x7fc464069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc4640679e0 0x7fc464069ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc4740af000 tx=0x7fc4740a7000 comp rx=0 tx=0).stop 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc48415c560 msgr2=0x7fc4840ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc48415c560 0x7fc4840ab9f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00ca80 tx=0x7fc47c00cf40 comp rx=0 tx=0).stop 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 shutdown_connections 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc4640679e0 0x7fc464069ea0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc48415c560 0x7fc4840ab9f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc484155970 0x7fc4840ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 --2- 172.21.15.110:0/1307721921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc48406f010 0x7fc484150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 >> 172.21.15.110:0/1307721921 conn(0x7fc48406bc50 msgr2=0x7fc484154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 shutdown_connections 2024-09-10T10:03:16.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:16.875+0000 7fc48b200640 1 -- 172.21.15.110:0/1307721921 wait complete. 2024-09-10T10:03:16.898 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol58330 2024-09-10T10:03:17.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc8600640 1 Processor -- start 2024-09-10T10:03:17.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc8600640 1 -- start start 2024-09-10T10:03:17.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc8600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2bc00ce4d0 0x7f2bc006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc8600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc00b4960 con 0x7f2bc006e850 2024-09-10T10:03:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc8600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc00b4c40 con 0x7f2bc015b330 2024-09-10T10:03:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc8600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2bc00b5570 con 0x7f2bc00ce4d0 2024-09-10T10:03:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc5200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2bc00ce4d0 0x7f2bc006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc6600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2bc006e850 0x7f2bc00b0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52116/0 (socket says 172.21.15.110:52116) 2024-09-10T10:03:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc6600640 1 -- 172.21.15.110:0/859292174 learned_addr learned my addr 172.21.15.110:0/859292174 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc5200640 1 -- 172.21.15.110:0/859292174 >> v1:172.21.15.145:6789/0 conn(0x7f2bc015b330 legacy=0x7f2bc006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc5200640 1 -- 172.21.15.110:0/859292174 >> v1:172.21.15.110:6789/0 conn(0x7f2bc006e850 legacy=0x7f2bc00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bc5200640 1 -- 172.21.15.110:0/859292174 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc00b4f20 con 0x7f2bc00ce4d0 2024-09-10T10:03:17.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.035+0000 7f2bbfe00640 1 -- 172.21.15.110:0/859292174 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72178597 0 0) 0x7f2bc00b4960 con 0x7f2bc006e850 2024-09-10T10:03:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc5200640 1 --2- 172.21.15.110:0/859292174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc00ce4d0 0x7f2bc006c6a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0097c0 tx=0x7f2bac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d463e1d401722da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bbfe00640 1 -- 172.21.15.110:0/859292174 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0b1070 con 0x7f2bc00ce4d0 2024-09-10T10:03:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bbfe00640 1 -- 172.21.15.110:0/859292174 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bac0ac410 con 0x7f2bc00ce4d0 2024-09-10T10:03:17.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bbfe00640 1 -- 172.21.15.110:0/859292174 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0ac7e0 con 0x7f2bc00ce4d0 2024-09-10T10:03:17.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 -- 172.21.15.110:0/859292174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc00ce4d0 msgr2=0x7f2bc006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 --2- 172.21.15.110:0/859292174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc00ce4d0 0x7f2bc006c6a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0097c0 tx=0x7f2bac01c4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 -- 172.21.15.110:0/859292174 shutdown_connections 2024-09-10T10:03:17.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 --2- 172.21.15.110:0/859292174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc00ce4d0 0x7f2bc006c6a0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 -- 172.21.15.110:0/859292174 >> 172.21.15.110:0/859292174 conn(0x7f2bc0062e40 msgr2=0x7f2bc006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:17.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 -- 172.21.15.110:0/859292174 shutdown_connections 2024-09-10T10:03:17.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 -- 172.21.15.110:0/859292174 wait complete. 2024-09-10T10:03:17.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 Processor -- start 2024-09-10T10:03:17.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.039+0000 7f2bc8600640 1 -- start start 2024-09-10T10:03:17.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bc006e850 0x7f2bc014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bc015b330 0x7f2bc0072770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc014f2a0 0x7f2bc00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0169450 con 0x7f2bc014f2a0 2024-09-10T10:03:17.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01692d0 con 0x7f2bc015b330 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc01695d0 con 0x7f2bc006e850 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc014f2a0 0x7f2bc00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc014f2a0 0x7f2bc00ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54932/0 (socket says 172.21.15.110:54932) 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 -- 172.21.15.110:0/2613302794 learned_addr learned my addr 172.21.15.110:0/2613302794 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc5200640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bc015b330 0x7f2bc0072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc5c00640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bc006e850 0x7f2bc014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 -- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bc006e850 msgr2=0x7f2bc014e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bc006e850 0x7f2bc014e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 -- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bc015b330 msgr2=0x7f2bc0072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bc015b330 0x7f2bc0072770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 -- 172.21.15.110:0/2613302794 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc0154a10 con 0x7f2bc014f2a0 2024-09-10T10:03:17.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc6600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc014f2a0 0x7f2bc00ae840 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c750 tx=0x7f2bb800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bbd600640 1 -- 172.21.15.110:0/2613302794 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8018070 con 0x7f2bc014f2a0 2024-09-10T10:03:17.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bbd600640 1 -- 172.21.15.110:0/2613302794 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb800a5e0 con 0x7f2bc014f2a0 2024-09-10T10:03:17.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc0154cf0 con 0x7f2bc014f2a0 2024-09-10T10:03:17.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bbd600640 1 -- 172.21.15.110:0/2613302794 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800ce60 con 0x7f2bc014f2a0 2024-09-10T10:03:17.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc014ea40 con 0x7f2bc014f2a0 2024-09-10T10:03:17.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.043+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc006c6a0 con 0x7f2bc014f2a0 2024-09-10T10:03:17.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.047+0000 7f2bbd600640 1 -- 172.21.15.110:0/2613302794 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7f2bb801b070 con 0x7f2bc014f2a0 2024-09-10T10:03:17.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.047+0000 7f2bbd600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ba0067a10 0x7f2ba0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.047+0000 7f2bbd600640 1 -- 172.21.15.110:0/2613302794 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f2bb80853b0 con 0x7f2bc014f2a0 2024-09-10T10:03:17.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.047+0000 7f2bc5c00640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ba0067a10 0x7f2ba0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.047+0000 7f2bc5c00640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ba0067a10 0x7f2ba0069ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007fd0 tx=0x7f2bb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.051+0000 7f2bbd600640 1 -- 172.21.15.110:0/2613302794 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2bb80556b0 con 0x7f2bc014f2a0 2024-09-10T10:03:17.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.239+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol58330", "target": ["mon-mgr", ""]}) -- 0x7f2bc006e1a0 con 0x7f2ba0067a10 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bbd600640 1 -- 172.21.15.110:0/2613302794 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+680 (secure 0 0 0) 0x7f2bc006e1a0 con 0x7f2ba0067a10 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "atime": "2024-09-10 10:03:14", 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_pcent": "0.00", 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_quota": 10485760, 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_used": 0, 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "created_at": "2024-09-10 10:03:14", 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "ctime": "2024-09-10 10:03:16", 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "data_pool": "cephfs_data", 2024-09-10T10:03:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "features": [ 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot-clone", 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot-autoprotect" 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "flavor": 1, 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "gid": 0, 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": 16877, 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "mon_addrs": [ 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.110:6789", 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.145:6789", 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.145:6790" 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "mtime": "2024-09-10 10:03:14", 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "path": "/volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322", 2024-09-10T10:03:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_namespace": "", 2024-09-10T10:03:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "complete", 2024-09-10T10:03:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "subvolume", 2024-09-10T10:03:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "uid": 0 2024-09-10T10:03:17.272 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ba0067a10 msgr2=0x7f2ba0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ba0067a10 0x7f2ba0069ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007fd0 tx=0x7f2bb4009040 comp rx=0 tx=0).stop 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc014f2a0 msgr2=0x7f2bc00ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc014f2a0 0x7f2bc00ae840 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c750 tx=0x7f2bb800cc10 comp rx=0 tx=0).stop 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 shutdown_connections 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ba0067a10 0x7f2ba0069ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bc014f2a0 0x7f2bc00ae840 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bc015b330 0x7f2bc0072770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 --2- 172.21.15.110:0/2613302794 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bc006e850 0x7f2bc014e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 >> 172.21.15.110:0/2613302794 conn(0x7f2bc0062e40 msgr2=0x7f2bc00b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 shutdown_connections 2024-09-10T10:03:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.267+0000 7f2bc8600640 1 -- 172.21.15.110:0/2613302794 wait complete. 2024-09-10T10:03:17.291 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol58330 --mode 777 --uid 1000 --gid 1000 2024-09-10T10:03:17.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2e1c00640 1 Processor -- start 2024-09-10T10:03:17.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2e1c00640 1 -- start start 2024-09-10T10:03:17.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2e1c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe2dc151a80 0x7fe2dc06c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2e1c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2dc15b5a0 con 0x7fe2dc15afc0 2024-09-10T10:03:17.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2e1c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2dc15b880 con 0x7fe2dc0710b0 2024-09-10T10:03:17.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2e1c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe2dc15bb60 con 0x7fe2dc151a80 2024-09-10T10:03:17.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2daa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe2dc15afc0 0x7fe2dc072da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52124/0 (socket says 172.21.15.110:52124) 2024-09-10T10:03:17.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2daa00640 1 -- 172.21.15.110:0/3169131971 learned_addr learned my addr 172.21.15.110:0/3169131971 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:17.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2da000640 1 --2- 172.21.15.110:0/3169131971 >> v2:172.21.15.145:3300/0 conn(0x7fe2dc151a80 0x7fe2dc06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2421783854 0 0) 0x7fe2dc15b5a0 con 0x7fe2dc15afc0 2024-09-10T10:03:17.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2c0003530 con 0x7fe2dc15afc0 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.435+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1797918544 0 0) 0x7fe2dc15b880 con 0x7fe2dc0710b0 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2dc15b5a0 con 0x7fe2dc0710b0 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793778010 0 0) 0x7fe2c0003530 con 0x7fe2dc15afc0 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 >> v2:172.21.15.145:3300/0 conn(0x7fe2dc151a80 msgr2=0x7fe2dc06c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 --2- 172.21.15.110:0/3169131971 >> v2:172.21.15.145:3300/0 conn(0x7fe2dc151a80 0x7fe2dc06c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 >> v1:172.21.15.145:6789/0 conn(0x7fe2dc0710b0 legacy=0x7fe2dc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc0b3420 con 0x7fe2dc15afc0 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fe2cc001a00 con 0x7fe2dc15afc0 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 >> v1:172.21.15.110:6789/0 conn(0x7fe2dc15afc0 legacy=0x7fe2dc072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0003bd0 0x7fe2c0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2c0004810 0x7fe2c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2c0005220 0x7fe2c000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc15bb60 con 0x7fe2c0004810 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c000e380 con 0x7fe2c0003bd0 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c000e580 con 0x7fe2c0005220 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2da000640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2c0004810 0x7fe2c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2daa00640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0003bd0 0x7fe2c0003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2db400640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2c0005220 0x7fe2c000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2da000640 1 -- 172.21.15.110:0/3169131971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2c0005220 msgr2=0x7fe2c000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2da000640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2c0005220 0x7fe2c000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2da000640 1 -- 172.21.15.110:0/3169131971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0003bd0 msgr2=0x7fe2c0003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2da000640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0003bd0 0x7fe2c0003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2da000640 1 -- 172.21.15.110:0/3169131971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c0003530 con 0x7fe2c0004810 2024-09-10T10:03:17.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2db400640 1 -- 172.21.15.110:0/3169131971 reap_dead start 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2da000640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2c0004810 0x7fe2c0004c90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009ae0 tx=0x7fe2c801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a264e0ce903f5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80096b0 con 0x7fe2c0004810 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c80a64e0 con 0x7fe2c0004810 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.439+0000 7fe2d9600640 1 -- 172.21.15.110:0/3169131971 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80a68d0 con 0x7fe2c0004810 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- 172.21.15.110:0/3169131971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2c0004810 msgr2=0x7fe2c0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2c0004810 0x7fe2c0004c90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009ae0 tx=0x7fe2c801ca70 comp rx=0 tx=0).stop 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- 172.21.15.110:0/3169131971 shutdown_connections 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 --2- 172.21.15.110:0/3169131971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2c0004810 0x7fe2c0004c90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- 172.21.15.110:0/3169131971 >> 172.21.15.110:0/3169131971 conn(0x7fe2dc0b5ea0 msgr2=0x7fe2dc06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- 172.21.15.110:0/3169131971 shutdown_connections 2024-09-10T10:03:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- 172.21.15.110:0/3169131971 wait complete. 2024-09-10T10:03:17.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 Processor -- start 2024-09-10T10:03:17.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- start start 2024-09-10T10:03:17.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0004810 0x7fe2dc1f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2dc0b05d0 0x7fe2dc14b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2dc14b8c0 0x7fe2dc153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc0b3b50 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc0718a0 con 0x7fe2c0004810 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2e1c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe2dc0b3cd0 con 0x7fe2dc0b05d0 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2db400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2dc14b8c0 0x7fe2dc153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2db400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2dc14b8c0 0x7fe2dc153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54952/0 (socket says 172.21.15.110:54952) 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2db400640 1 -- 172.21.15.110:0/7761897 learned_addr learned my addr 172.21.15.110:0/7761897 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2da000640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2dc0b05d0 0x7fe2dc14b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.443+0000 7fe2daa00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0004810 0x7fe2dc1f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2db400640 1 -- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2dc0b05d0 msgr2=0x7fe2dc14b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2db400640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2dc0b05d0 0x7fe2dc14b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2db400640 1 -- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0004810 msgr2=0x7fe2dc1f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2db400640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0004810 0x7fe2dc1f77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2db400640 1 -- 172.21.15.110:0/7761897 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1f7d10 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2db400640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2dc14b8c0 0x7fe2dc153090 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0b49d0 tx=0x7fe2d400fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2d2a00640 1 -- 172.21.15.110:0/7761897 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4019070 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2d2a00640 1 -- 172.21.15.110:0/7761897 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d4005030 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc0b0a50 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2d2a00640 1 -- 172.21.15.110:0/7761897 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4005400 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc0ad930 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2dc0b4060 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2d2a00640 1 -- 172.21.15.110:0/7761897 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7fe2d401c070 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2d2a00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe2bc0679e0 0x7fe2bc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2d2a00640 1 -- 172.21.15.110:0/7761897 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fe2d4084d80 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.447+0000 7fe2daa00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe2bc0679e0 0x7fe2bc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.451+0000 7fe2daa00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe2bc0679e0 0x7fe2bc069ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc001590 tx=0x7fe2cc006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.451+0000 7fe2d2a00640 1 -- 172.21.15.110:0/7761897 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe2dc1f7d10 con 0x7fe2dc14b8c0 2024-09-10T10:03:17.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.635+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol58330", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7fe2dc0762a0 con 0x7fe2bc0679e0 2024-09-10T10:03:17.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2d2a00640 1 -- 172.21.15.110:0/7761897 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe2dc0762a0 con 0x7fe2bc0679e0 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe2bc0679e0 msgr2=0x7fe2bc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe2bc0679e0 0x7fe2bc069ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc001590 tx=0x7fe2cc006040 comp rx=0 tx=0).stop 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2dc14b8c0 msgr2=0x7fe2dc153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2dc14b8c0 0x7fe2dc153090 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0b49d0 tx=0x7fe2d400fb50 comp rx=0 tx=0).stop 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 shutdown_connections 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe2bc0679e0 0x7fe2bc069ea0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2dc14b8c0 0x7fe2dc153090 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe2dc0b05d0 0x7fe2dc14b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 --2- 172.21.15.110:0/7761897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c0004810 0x7fe2dc1f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 >> 172.21.15.110:0/7761897 conn(0x7fe2dc0b5ea0 msgr2=0x7fe2dc0aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:17.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 shutdown_connections 2024-09-10T10:03:17.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.663+0000 7fe2e1c00640 1 -- 172.21.15.110:0/7761897 wait complete. 2024-09-10T10:03:17.687 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol58330 2024-09-10T10:03:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff63e000640 1 Processor -- start 2024-09-10T10:03:17.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff63e000640 1 -- start start 2024-09-10T10:03:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff63e000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff6380ce4d0 0x7ff63806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff63e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63806e850 0x7ff6380b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff63e000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6380b4960 con 0x7ff63815b330 2024-09-10T10:03:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff63e000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff6380b5570 con 0x7ff6380ce4d0 2024-09-10T10:03:17.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff63e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff6380b4c40 con 0x7ff63806e850 2024-09-10T10:03:17.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff637400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff63815b330 0x7ff63806e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52132/0 (socket says 172.21.15.110:52132) 2024-09-10T10:03:17.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff637400640 1 -- 172.21.15.110:0/3415415574 learned_addr learned my addr 172.21.15.110:0/3415415574 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:17.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff637e00640 1 --2- 172.21.15.110:0/3415415574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63806e850 0x7ff6380b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.823+0000 7ff636a00640 1 --2- 172.21.15.110:0/3415415574 >> v2:172.21.15.145:3300/0 conn(0x7ff6380ce4d0 0x7ff63806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff636000640 1 -- 172.21.15.110:0/3415415574 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 845080054 0 0) 0x7ff6380b4960 con 0x7ff63815b330 2024-09-10T10:03:17.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff636000640 1 -- 172.21.15.110:0/3415415574 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff61c003530 con 0x7ff63815b330 2024-09-10T10:03:17.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff637e00640 1 -- 172.21.15.110:0/3415415574 >> v2:172.21.15.145:3300/0 conn(0x7ff6380ce4d0 msgr2=0x7ff63806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:17.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff637e00640 1 --2- 172.21.15.110:0/3415415574 >> v2:172.21.15.145:3300/0 conn(0x7ff6380ce4d0 0x7ff63806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff637e00640 1 -- 172.21.15.110:0/3415415574 >> v1:172.21.15.110:6789/0 conn(0x7ff63815b330 legacy=0x7ff63806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff637e00640 1 -- 172.21.15.110:0/3415415574 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6380b4f20 con 0x7ff63806e850 2024-09-10T10:03:17.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff636a00640 1 --2- 172.21.15.110:0/3415415574 >> v2:172.21.15.145:3300/0 conn(0x7ff6380ce4d0 0x7ff63806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:17.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff637e00640 1 --2- 172.21.15.110:0/3415415574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63806e850 0x7ff6380b0860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff63001eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ba076a0d91bca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff636000640 1 -- 172.21.15.110:0/3415415574 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300b4070 con 0x7ff63806e850 2024-09-10T10:03:17.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff636000640 1 -- 172.21.15.110:0/3415415574 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6300a7b60 con 0x7ff63806e850 2024-09-10T10:03:17.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.827+0000 7ff636000640 1 -- 172.21.15.110:0/3415415574 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300af440 con 0x7ff63806e850 2024-09-10T10:03:17.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- 172.21.15.110:0/3415415574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63806e850 msgr2=0x7ff6380b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 --2- 172.21.15.110:0/3415415574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63806e850 0x7ff6380b0860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff63001eb80 comp rx=0 tx=0).stop 2024-09-10T10:03:17.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- 172.21.15.110:0/3415415574 shutdown_connections 2024-09-10T10:03:17.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 --2- 172.21.15.110:0/3415415574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63806e850 0x7ff6380b0860 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 --2- 172.21.15.110:0/3415415574 >> v2:172.21.15.145:3300/0 conn(0x7ff6380ce4d0 0x7ff63806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- 172.21.15.110:0/3415415574 >> 172.21.15.110:0/3415415574 conn(0x7ff638062e40 msgr2=0x7ff63806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:17.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- 172.21.15.110:0/3415415574 shutdown_connections 2024-09-10T10:03:17.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- 172.21.15.110:0/3415415574 wait complete. 2024-09-10T10:03:17.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 Processor -- start 2024-09-10T10:03:17.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- start start 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63806e850 0x7ff638141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63815b330 0x7ff63815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6381528f0 0x7ff6381550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380b36b0 con 0x7ff6381528f0 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380b1800 con 0x7ff63806e850 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff63e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff6380b3830 con 0x7ff63815b330 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6381528f0 0x7ff6381550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6381528f0 0x7ff6381550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54962/0 (socket says 172.21.15.110:54962) 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 -- 172.21.15.110:0/1382399304 learned_addr learned my addr 172.21.15.110:0/1382399304 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:17.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637400640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63806e850 0x7ff638141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff636a00640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63815b330 0x7ff63815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 -- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63815b330 msgr2=0x7ff63815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63815b330 0x7ff63815c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 -- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63806e850 msgr2=0x7ff638141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63806e850 0x7ff638141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 -- 172.21.15.110:0/1382399304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63815c7d0 con 0x7ff6381528f0 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.831+0000 7ff637e00640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6381528f0 0x7ff6381550e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff6300b2040 tx=0x7ff63001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff62f400640 1 -- 172.21.15.110:0/1382399304 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300b4040 con 0x7ff6381528f0 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff62f400640 1 -- 172.21.15.110:0/1382399304 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6300bd070 con 0x7ff6381528f0 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff62f400640 1 -- 172.21.15.110:0/1382399304 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300afa60 con 0x7ff6381528f0 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff638151b30 con 0x7ff6381528f0 2024-09-10T10:03:17.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6381520f0 con 0x7ff6381528f0 2024-09-10T10:03:17.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63806c6a0 con 0x7ff6381528f0 2024-09-10T10:03:17.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff62f400640 1 -- 172.21.15.110:0/1382399304 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 87547+0+0 (secure 0 0 0) 0x7ff6300a7d10 con 0x7ff6381528f0 2024-09-10T10:03:17.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff62f400640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff6180679e0 0x7ff618069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:17.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff62f400640 1 -- 172.21.15.110:0/1382399304 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7ff630120480 con 0x7ff6381528f0 2024-09-10T10:03:17.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff637400640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff6180679e0 0x7ff618069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:17.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.835+0000 7ff637400640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff6180679e0 0x7ff618069ea0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff628007fd0 tx=0x7ff628009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:17.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:17.839+0000 7ff62f400640 1 -- 172.21.15.110:0/1382399304 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff63815c7d0 con 0x7ff6381528f0 2024-09-10T10:03:18.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.023+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol58330", "target": ["mon-mgr", ""]}) -- 0x7ff6380cd830 con 0x7ff6180679e0 2024-09-10T10:03:18.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff62f400640 1 -- 172.21.15.110:0/1382399304 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7ff6380cd830 con 0x7ff6180679e0 2024-09-10T10:03:18.054 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322 2024-09-10T10:03:18.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff6180679e0 msgr2=0x7ff618069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:18.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff6180679e0 0x7ff618069ea0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff628007fd0 tx=0x7ff628009040 comp rx=0 tx=0).stop 2024-09-10T10:03:18.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6381528f0 msgr2=0x7ff6381550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:18.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6381528f0 0x7ff6381550e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff6300b2040 tx=0x7ff63001b920 comp rx=0 tx=0).stop 2024-09-10T10:03:18.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 shutdown_connections 2024-09-10T10:03:18.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff6180679e0 0x7ff618069ea0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:18.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6381528f0 0x7ff6381550e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:18.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff63815b330 0x7ff63815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:18.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 --2- 172.21.15.110:0/1382399304 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63806e850 0x7ff638141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:18.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.051+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 >> 172.21.15.110:0/1382399304 conn(0x7ff638062e40 msgr2=0x7ff6380ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:18.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.055+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 shutdown_connections 2024-09-10T10:03:18.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:18.055+0000 7ff63e000640 1 -- 172.21.15.110:0/1382399304 wait complete. 2024-09-10T10:03:18.075 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol58330 with 8 files each 1MB size under directory volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322 2024-09-10T10:03:18.076 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:18.076 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:18.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:18.085 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T10:03:18.160 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T10:03:18.161 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T10:03:18.161 INFO:teuthology.orchestra.run.smithi110.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207157 s, 50.6 MB/s 2024-09-10T10:03:18.162 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:18.162 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:18.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:18.218 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T10:03:18.291 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T10:03:18.291 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T10:03:18.291 INFO:teuthology.orchestra.run.smithi110.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217859 s, 48.1 MB/s 2024-09-10T10:03:18.292 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:18.292 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:18.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:18.345 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T10:03:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T10:03:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T10:03:18.416 INFO:teuthology.orchestra.run.smithi110.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019113 s, 54.9 MB/s 2024-09-10T10:03:18.417 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:18.417 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:18.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:18.469 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T10:03:18.538 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T10:03:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T10:03:18.539 INFO:teuthology.orchestra.run.smithi110.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174849 s, 60.0 MB/s 2024-09-10T10:03:18.539 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:18.540 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:18.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:18.589 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T10:03:18.655 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T10:03:18.655 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T10:03:18.655 INFO:teuthology.orchestra.run.smithi110.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145048 s, 72.3 MB/s 2024-09-10T10:03:18.656 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:18.656 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:18.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:18.709 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T10:03:18.783 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T10:03:18.783 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T10:03:18.783 INFO:teuthology.orchestra.run.smithi110.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178735 s, 58.7 MB/s 2024-09-10T10:03:18.784 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:18.784 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:18.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:18.833 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T10:03:18.904 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T10:03:18.904 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T10:03:18.904 INFO:teuthology.orchestra.run.smithi110.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018808 s, 55.8 MB/s 2024-09-10T10:03:18.905 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:18.905 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:18.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:18.957 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T10:03:19.027 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-10T10:03:19.027 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-10T10:03:19.027 INFO:teuthology.orchestra.run.smithi110.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184764 s, 56.8 MB/s 2024-09-10T10:03:19.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol58330 subvol_snap58400 2024-09-10T10:03:19.154 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:03:19.157 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:03:19.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.207+0000 7f8d14800640 1 Processor -- start 2024-09-10T10:03:19.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.207+0000 7f8d14800640 1 -- start start 2024-09-10T10:03:19.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.207+0000 7f8d14800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8d0c06bc50 0x7f8d0c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.207+0000 7f8d14800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c0ae840 0x7f8d0c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.207+0000 7f8d14800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d0c06c710 con 0x7f8d0c15e690 2024-09-10T10:03:19.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.207+0000 7f8d14800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0c0b4680 con 0x7f8d0c06bc50 2024-09-10T10:03:19.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.207+0000 7f8d14800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c0b4960 con 0x7f8d0c0ae840 2024-09-10T10:03:19.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8d0c06bc50 0x7f8d0c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f8d0c15e690 0x7f8d0c06e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:43064/0 (socket says 172.21.15.110:43064) 2024-09-10T10:03:19.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11e00640 1 -- 172.21.15.110:0/4148327502 learned_addr learned my addr 172.21.15.110:0/4148327502 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:19.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8d0c06bc50 0x7f8d0c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37318/0 (socket says 172.21.15.110:37318) 2024-09-10T10:03:19.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d12800640 1 --2- 172.21.15.110:0/4148327502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c0ae840 0x7f8d0c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11400640 1 -- 172.21.15.110:0/4148327502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c0ae840 msgr2=0x7f8d0c0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:19.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11400640 1 --2- 172.21.15.110:0/4148327502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c0ae840 0x7f8d0c0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11400640 1 -- 172.21.15.110:0/4148327502 >> v1:172.21.15.145:6789/0 conn(0x7f8d0c15e690 legacy=0x7f8d0c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11400640 1 -- 172.21.15.110:0/4148327502 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c0b4c40 con 0x7f8d0c06bc50 2024-09-10T10:03:19.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d11400640 1 --2- 172.21.15.110:0/4148327502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c06bc50 0x7f8d0c06c090 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0097c0 tx=0x7f8cfc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382a48f3df8eaf71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:19.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d10a00640 1 -- 172.21.15.110:0/4148327502 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0b1070 con 0x7f8d0c06bc50 2024-09-10T10:03:19.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d10a00640 1 -- 172.21.15.110:0/4148327502 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc0ac410 con 0x7f8d0c06bc50 2024-09-10T10:03:19.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.211+0000 7f8d10a00640 1 -- 172.21.15.110:0/4148327502 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0ac7e0 con 0x7f8d0c06bc50 2024-09-10T10:03:19.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- 172.21.15.110:0/4148327502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c06bc50 msgr2=0x7f8d0c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 --2- 172.21.15.110:0/4148327502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c06bc50 0x7f8d0c06c090 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0097c0 tx=0x7f8cfc01c4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- 172.21.15.110:0/4148327502 shutdown_connections 2024-09-10T10:03:19.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 --2- 172.21.15.110:0/4148327502 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c0ae840 0x7f8d0c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 --2- 172.21.15.110:0/4148327502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c06bc50 0x7f8d0c06c090 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- 172.21.15.110:0/4148327502 >> 172.21.15.110:0/4148327502 conn(0x7f8d0c0d1530 msgr2=0x7f8d0c0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:19.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- 172.21.15.110:0/4148327502 shutdown_connections 2024-09-10T10:03:19.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- 172.21.15.110:0/4148327502 wait complete. 2024-09-10T10:03:19.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 Processor -- start 2024-09-10T10:03:19.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- start start 2024-09-10T10:03:19.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d0c06bc50 0x7f8d0c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c0ae840 0x7f8d0c158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c15e690 0x7f8d0c15af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c0b5dd0 con 0x7f8d0c0ae840 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c0b5c50 con 0x7f8d0c06bc50 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d14800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c0b5f50 con 0x7f8d0c15e690 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d11400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c0ae840 0x7f8d0c158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d11400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c0ae840 0x7f8d0c158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37330/0 (socket says 172.21.15.110:37330) 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d11400640 1 -- 172.21.15.110:0/2974581632 learned_addr learned my addr 172.21.15.110:0/2974581632 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d11e00640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d0c06bc50 0x7f8d0c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.215+0000 7f8d12800640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c15e690 0x7f8d0c15af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d11400640 1 -- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c15e690 msgr2=0x7f8d0c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d11400640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c15e690 0x7f8d0c15af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d11400640 1 -- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d0c06bc50 msgr2=0x7f8d0c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d11400640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d0c06bc50 0x7f8d0c15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d11400640 1 -- 172.21.15.110:0/2974581632 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c161220 con 0x7f8d0c0ae840 2024-09-10T10:03:19.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d11400640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c0ae840 0x7f8d0c158c80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc01a0d0 tx=0x7f8cfc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:19.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d06000640 1 -- 172.21.15.110:0/2974581632 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0b9020 con 0x7f8d0c0ae840 2024-09-10T10:03:19.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d06000640 1 -- 172.21.15.110:0/2974581632 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc0acd90 con 0x7f8d0c0ae840 2024-09-10T10:03:19.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d06000640 1 -- 172.21.15.110:0/2974581632 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0b1040 con 0x7f8d0c0ae840 2024-09-10T10:03:19.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c161500 con 0x7f8d0c0ae840 2024-09-10T10:03:19.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c15b470 con 0x7f8d0c0ae840 2024-09-10T10:03:19.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c06c090 con 0x7f8d0c0ae840 2024-09-10T10:03:19.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d06000640 1 -- 172.21.15.110:0/2974581632 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 87598+0+0 (secure 0 0 0) 0x7f8cfc0bc020 con 0x7f8d0c0ae840 2024-09-10T10:03:19.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.219+0000 7f8d06000640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8cf0067a70 0x7f8cf0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.223+0000 7f8d06000640 1 -- 172.21.15.110:0/2974581632 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f8cfc11f9a0 con 0x7f8d0c0ae840 2024-09-10T10:03:19.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.223+0000 7f8d11e00640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8cf0067a70 0x7f8cf0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.223+0000 7f8d11e00640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8cf0067a70 0x7f8cf0069f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c1692c0 tx=0x7f8d00007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:19.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.227+0000 7f8d06000640 1 -- 172.21.15.110:0/2974581632 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8cfc0c3030 con 0x7f8d0c0ae840 2024-09-10T10:03:19.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.411+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol58330", "snap_name": "subvol_snap58400", "target": ["mon-mgr", ""]}) -- 0x7f8d0c159420 con 0x7f8cf0067a70 2024-09-10T10:03:19.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.447+0000 7f8d06000640 1 -- 172.21.15.110:0/2974581632 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d0c159420 con 0x7f8cf0067a70 2024-09-10T10:03:19.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8cf0067a70 msgr2=0x7f8cf0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8cf0067a70 0x7f8cf0069f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c1692c0 tx=0x7f8d00007e90 comp rx=0 tx=0).stop 2024-09-10T10:03:19.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c0ae840 msgr2=0x7f8d0c158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c0ae840 0x7f8d0c158c80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc01a0d0 tx=0x7f8cfc019240 comp rx=0 tx=0).stop 2024-09-10T10:03:19.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 shutdown_connections 2024-09-10T10:03:19.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8cf0067a70 0x7f8cf0069f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d0c15e690 0x7f8d0c15af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d0c0ae840 0x7f8d0c158c80 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 --2- 172.21.15.110:0/2974581632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d0c06bc50 0x7f8d0c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 >> 172.21.15.110:0/2974581632 conn(0x7f8d0c0d1530 msgr2=0x7f8d0c06e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:19.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 shutdown_connections 2024-09-10T10:03:19.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.451+0000 7f8d14800640 1 -- 172.21.15.110:0/2974581632 wait complete. 2024-09-10T10:03:19.473 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol58330 subvol_snap58400 subvol_clone81830 2024-09-10T10:03:19.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc8ae00640 1 Processor -- start 2024-09-10T10:03:19.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc8ae00640 1 -- start start 2024-09-10T10:03:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc8ae00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efc840af710 0x7efc840b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc8ae00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efc84070470 0x7efc840b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc8ae00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8406ce00 con 0x7efc8406f980 2024-09-10T10:03:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc8ae00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7efc8406da10 con 0x7efc840af710 2024-09-10T10:03:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc8ae00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7efc8406d0e0 con 0x7efc84070470 2024-09-10T10:03:19.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc83400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efc840af710 0x7efc840b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc83400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efc840af710 0x7efc840b6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37340/0 (socket says 172.21.15.110:37340) 2024-09-10T10:03:19.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc83400640 1 -- 172.21.15.110:0/3314256076 learned_addr learned my addr 172.21.15.110:0/3314256076 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:19.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc88e00640 1 --2- 172.21.15.110:0/3314256076 >> v2:172.21.15.145:3300/0 conn(0x7efc84070470 0x7efc840b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc83400640 1 -- 172.21.15.110:0/3314256076 >> v2:172.21.15.145:3300/0 conn(0x7efc84070470 msgr2=0x7efc840b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:19.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc83400640 1 --2- 172.21.15.110:0/3314256076 >> v2:172.21.15.145:3300/0 conn(0x7efc84070470 0x7efc840b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc83400640 1 -- 172.21.15.110:0/3314256076 >> v1:172.21.15.145:6789/0 conn(0x7efc8406f980 legacy=0x7efc8406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.655+0000 7efc83400640 1 -- 172.21.15.110:0/3314256076 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8406d3c0 con 0x7efc840af710 2024-09-10T10:03:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc83400640 1 --2- 172.21.15.110:0/3314256076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc840af710 0x7efc840b6ae0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7efc840b2310 tx=0x7efc7001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3138015a62e26972 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc82a00640 1 -- 172.21.15.110:0/3314256076 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc700ab680 con 0x7efc840af710 2024-09-10T10:03:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc82a00640 1 -- 172.21.15.110:0/3314256076 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc700b0070 con 0x7efc840af710 2024-09-10T10:03:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc82a00640 1 -- 172.21.15.110:0/3314256076 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc700aba40 con 0x7efc840af710 2024-09-10T10:03:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc8ae00640 1 -- 172.21.15.110:0/3314256076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc840af710 msgr2=0x7efc840b6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3314256076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc840af710 0x7efc840b6ae0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7efc840b2310 tx=0x7efc7001c460 comp rx=0 tx=0).stop 2024-09-10T10:03:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc8ae00640 1 -- 172.21.15.110:0/3314256076 shutdown_connections 2024-09-10T10:03:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3314256076 >> v2:172.21.15.145:3300/0 conn(0x7efc84070470 0x7efc840b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3314256076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc840af710 0x7efc840b6ae0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc8ae00640 1 -- 172.21.15.110:0/3314256076 >> 172.21.15.110:0/3314256076 conn(0x7efc84063160 msgr2=0x7efc8409b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc8ae00640 1 -- 172.21.15.110:0/3314256076 shutdown_connections 2024-09-10T10:03:19.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.659+0000 7efc8ae00640 1 -- 172.21.15.110:0/3314256076 wait complete. 2024-09-10T10:03:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 Processor -- start 2024-09-10T10:03:19.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 -- start start 2024-09-10T10:03:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc8406f980 0x7efc8414caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc84070470 0x7efc84149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc840af710 0x7efc84151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc8416ae20 con 0x7efc8406f980 2024-09-10T10:03:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efc8416aca0 con 0x7efc840af710 2024-09-10T10:03:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efc8416afa0 con 0x7efc84070470 2024-09-10T10:03:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc8406f980 0x7efc8414caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc8406f980 0x7efc8414caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37346/0 (socket says 172.21.15.110:37346) 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 -- 172.21.15.110:0/3922543704 learned_addr learned my addr 172.21.15.110:0/3922543704 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83400640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc84070470 0x7efc84149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc88e00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc840af710 0x7efc84151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 -- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc84070470 msgr2=0x7efc84149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc84070470 0x7efc84149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 -- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc840af710 msgr2=0x7efc84151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc840af710 0x7efc84151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 -- 172.21.15.110:0/3922543704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc84152490 con 0x7efc8406f980 2024-09-10T10:03:19.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc88e00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc840af710 0x7efc84151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc83e00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc8406f980 0x7efc8414caa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c760 tx=0x7efc7400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc7be00640 1 -- 172.21.15.110:0/3922543704 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74016020 con 0x7efc8406f980 2024-09-10T10:03:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc7be00640 1 -- 172.21.15.110:0/3922543704 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc74006d60 con 0x7efc8406f980 2024-09-10T10:03:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8414d680 con 0x7efc8406f980 2024-09-10T10:03:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc7be00640 1 -- 172.21.15.110:0/3922543704 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74003bc0 con 0x7efc8406f980 2024-09-10T10:03:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8414dc40 con 0x7efc8406f980 2024-09-10T10:03:19.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.663+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8406fd60 con 0x7efc8406f980 2024-09-10T10:03:19.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.667+0000 7efc7be00640 1 -- 172.21.15.110:0/3922543704 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 87598+0+0 (secure 0 0 0) 0x7efc7400a850 con 0x7efc8406f980 2024-09-10T10:03:19.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.667+0000 7efc7be00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efc64067a70 0x7efc64069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:19.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.667+0000 7efc7be00640 1 -- 172.21.15.110:0/3922543704 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7efc74085c10 con 0x7efc8406f980 2024-09-10T10:03:19.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.667+0000 7efc83400640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efc64067a70 0x7efc64069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:19.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.667+0000 7efc83400640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efc64067a70 0x7efc64069f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc700af000 tx=0x7efc700a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:19.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.671+0000 7efc7be00640 1 -- 172.21.15.110:0/3922543704 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efc74020020 con 0x7efc8406f980 2024-09-10T10:03:19.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.859+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol58330", "snap_name": "subvol_snap58400", "target_sub_name": "subvol_clone81830", "target": ["mon-mgr", ""]}) -- 0x7efc840b3290 con 0x7efc64067a70 2024-09-10T10:03:19.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc7be00640 1 -- 172.21.15.110:0/3922543704 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc840b3290 con 0x7efc64067a70 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efc64067a70 msgr2=0x7efc64069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efc64067a70 0x7efc64069f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc700af000 tx=0x7efc700a7000 comp rx=0 tx=0).stop 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc8406f980 msgr2=0x7efc8414caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc8406f980 0x7efc8414caa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c760 tx=0x7efc7400cc20 comp rx=0 tx=0).stop 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 shutdown_connections 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efc64067a70 0x7efc64069f30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc840af710 0x7efc84151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efc84070470 0x7efc84149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 --2- 172.21.15.110:0/3922543704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc8406f980 0x7efc8414caa0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 >> 172.21.15.110:0/3922543704 conn(0x7efc84063160 msgr2=0x7efc840b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 shutdown_connections 2024-09-10T10:03:19.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:19.947+0000 7efc8ae00640 1 -- 172.21.15.110:0/3922543704 wait complete. 2024-09-10T10:03:19.955 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.955+0000 7f95e3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.955 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.955+0000 7f95e3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.955 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.955+0000 7f95e3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.955 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.955+0000 7f95e3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.955 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.955+0000 7f95e3e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.971 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone81830 2024-09-10T10:03:19.972 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.971+0000 7f95e4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.972 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.971+0000 7f95e4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.972 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.971+0000 7f95e4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.972 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.971+0000 7f95e4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:19.972 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:19.971+0000 7f95e4800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:20.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc3d800640 1 Processor -- start 2024-09-10T10:03:20.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc3d800640 1 -- start start 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc3d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc380ce4d0 0x7fdc3806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc3d800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdc3806e850 0x7fdc380b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc3d800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc380b4960 con 0x7fdc3815b330 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc3d800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdc380b5570 con 0x7fdc3806e850 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc3d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdc380b4c40 con 0x7fdc380ce4d0 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc36a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fdc3815b330 0x7fdc3806e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36452/0 (socket says 172.21.15.110:36452) 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc36a00640 1 -- 172.21.15.110:0/1344599881 learned_addr learned my addr 172.21.15.110:0/1344599881 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc37400640 1 --2- 172.21.15.110:0/1344599881 >> v2:172.21.15.110:3300/0 conn(0x7fdc3806e850 0x7fdc380b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc2fe00640 1 --2- 172.21.15.110:0/1344599881 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc380ce4d0 0x7fdc3806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:20.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc36000640 1 -- 172.21.15.110:0/1344599881 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795797884 0 0) 0x7fdc380b4960 con 0x7fdc3815b330 2024-09-10T10:03:20.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc36000640 1 -- 172.21.15.110:0/1344599881 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc1c003530 con 0x7fdc3815b330 2024-09-10T10:03:20.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc2fe00640 1 -- 172.21.15.110:0/1344599881 >> v2:172.21.15.110:3300/0 conn(0x7fdc3806e850 msgr2=0x7fdc380b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:20.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc2fe00640 1 --2- 172.21.15.110:0/1344599881 >> v2:172.21.15.110:3300/0 conn(0x7fdc3806e850 0x7fdc380b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:20.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc2fe00640 1 -- 172.21.15.110:0/1344599881 >> v1:172.21.15.110:6789/0 conn(0x7fdc3815b330 legacy=0x7fdc3806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:20.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc2fe00640 1 -- 172.21.15.110:0/1344599881 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc380b4f20 con 0x7fdc380ce4d0 2024-09-10T10:03:20.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.127+0000 7fdc37400640 1 --2- 172.21.15.110:0/1344599881 >> v2:172.21.15.110:3300/0 conn(0x7fdc3806e850 0x7fdc380b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:20.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.131+0000 7fdc2fe00640 1 --2- 172.21.15.110:0/1344599881 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc380ce4d0 0x7fdc3806c6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc200097c0 tx=0x7fdc2001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b25698c9debf10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:20.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc36000640 1 -- 172.21.15.110:0/1344599881 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200b1070 con 0x7fdc380ce4d0 2024-09-10T10:03:20.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc36000640 1 -- 172.21.15.110:0/1344599881 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc200ac410 con 0x7fdc380ce4d0 2024-09-10T10:03:20.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc36000640 1 -- 172.21.15.110:0/1344599881 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200ac7e0 con 0x7fdc380ce4d0 2024-09-10T10:03:20.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- 172.21.15.110:0/1344599881 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc380ce4d0 msgr2=0x7fdc3806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:20.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 --2- 172.21.15.110:0/1344599881 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc380ce4d0 0x7fdc3806c6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc200097c0 tx=0x7fdc2001c4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:20.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- 172.21.15.110:0/1344599881 shutdown_connections 2024-09-10T10:03:20.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 --2- 172.21.15.110:0/1344599881 >> v2:172.21.15.110:3300/0 conn(0x7fdc3806e850 0x7fdc380b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:20.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 --2- 172.21.15.110:0/1344599881 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc380ce4d0 0x7fdc3806c6a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:20.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- 172.21.15.110:0/1344599881 >> 172.21.15.110:0/1344599881 conn(0x7fdc38062e40 msgr2=0x7fdc3806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:20.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- 172.21.15.110:0/1344599881 shutdown_connections 2024-09-10T10:03:20.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- 172.21.15.110:0/1344599881 wait complete. 2024-09-10T10:03:20.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 Processor -- start 2024-09-10T10:03:20.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- start start 2024-09-10T10:03:20.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc3806e850 0x7fdc381565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:20.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc3815b330 0x7fdc381646e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:20.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc38154880 0x7fdc3814f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:20.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381680b0 con 0x7fdc3815b330 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdc380b17c0 con 0x7fdc38154880 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc3d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdc38168230 con 0x7fdc3806e850 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc2fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc3815b330 0x7fdc381646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc2fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc3815b330 0x7fdc381646e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37370/0 (socket says 172.21.15.110:37370) 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.135+0000 7fdc2fe00640 1 -- 172.21.15.110:0/4283692561 learned_addr learned my addr 172.21.15.110:0/4283692561 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc36a00640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc3806e850 0x7fdc381565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc37400640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc38154880 0x7fdc3814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2fe00640 1 -- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc3806e850 msgr2=0x7fdc381565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2fe00640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc3806e850 0x7fdc381565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2fe00640 1 -- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc38154880 msgr2=0x7fdc3814f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2fe00640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc38154880 0x7fdc3814f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2fe00640 1 -- 172.21.15.110:0/4283692561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc38154d90 con 0x7fdc3815b330 2024-09-10T10:03:20.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2fe00640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc3815b330 0x7fdc381646e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fdc200af040 tx=0x7fdc20019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:20.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2ea00640 1 -- 172.21.15.110:0/4283692561 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200b1040 con 0x7fdc3815b330 2024-09-10T10:03:20.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2ea00640 1 -- 172.21.15.110:0/4283692561 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc200bc070 con 0x7fdc3815b330 2024-09-10T10:03:20.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2ea00640 1 -- 172.21.15.110:0/4283692561 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200b5410 con 0x7fdc3815b330 2024-09-10T10:03:20.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3815b9c0 con 0x7fdc3815b330 2024-09-10T10:03:20.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc3815bf80 con 0x7fdc3815b330 2024-09-10T10:03:20.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc3806c6a0 con 0x7fdc3815b330 2024-09-10T10:03:20.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.139+0000 7fdc2ea00640 1 -- 172.21.15.110:0/4283692561 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 87598+0+0 (secure 0 0 0) 0x7fdc200b55b0 con 0x7fdc3815b330 2024-09-10T10:03:20.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.143+0000 7fdc2ea00640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc14067aa0 0x7fdc14069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:20.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.143+0000 7fdc2ea00640 1 -- 172.21.15.110:0/4283692561 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fdc2011faa0 con 0x7fdc3815b330 2024-09-10T10:03:20.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.143+0000 7fdc36a00640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc14067aa0 0x7fdc14069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:20.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.143+0000 7fdc36a00640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc14067aa0 0x7fdc14069f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdc28007fd0 tx=0x7fdc28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.147+0000 7fdc2ea00640 1 -- 172.21.15.110:0/4283692561 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdc20016090 con 0x7fdc3815b330 2024-09-10T10:03:20.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:20.355+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone81830", "target": ["mon-mgr", ""]}) -- 0x7fdc38150a40 con 0x7fdc14067aa0 2024-09-10T10:03:22.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:22.227+0000 7fdc2ea00640 1 -- 172.21.15.110:0/4283692561 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7fdc200ec1a0 con 0x7fdc3815b330 2024-09-10T10:03:24.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc2ea00640 1 -- 172.21.15.110:0/4283692561 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fdc28001480 con 0x7fdc14067aa0 2024-09-10T10:03:24.934 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "status": { 2024-09-10T10:03:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "complete" 2024-09-10T10:03:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:24.934 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc14067aa0 msgr2=0x7fdc14069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc14067aa0 0x7fdc14069f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdc28007fd0 tx=0x7fdc28009040 comp rx=0 tx=0).stop 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc3815b330 msgr2=0x7fdc381646e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc3815b330 0x7fdc381646e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fdc200af040 tx=0x7fdc20019240 comp rx=0 tx=0).stop 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 shutdown_connections 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc14067aa0 0x7fdc14069f60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc38154880 0x7fdc3814f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc3815b330 0x7fdc381646e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 --2- 172.21.15.110:0/4283692561 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc3806e850 0x7fdc381565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.931+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 >> 172.21.15.110:0/4283692561 conn(0x7fdc38062e40 msgr2=0x7fdc3806d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:24.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.935+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 shutdown_connections 2024-09-10T10:03:24.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:24.935+0000 7fdc3d800640 1 -- 172.21.15.110:0/4283692561 wait complete. 2024-09-10T10:03:24.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:24.960 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone81830/.meta) 2024-09-10T10:03:24.982 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:03:24.982 INFO:teuthology.orchestra.run.smithi110.stdout:version = 2 2024-09-10T10:03:24.983 INFO:teuthology.orchestra.run.smithi110.stdout:type = clone 2024-09-10T10:03:24.983 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669 2024-09-10T10:03:24.983 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:03:24.983 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:24.985 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol58330 2024-09-10T10:03:25.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f884d200640 1 Processor -- start 2024-09-10T10:03:25.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f884d200640 1 -- start start 2024-09-10T10:03:25.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f884d200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88480ce4d0 0x7f884806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f884d200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f884806e850 0x7f88480b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f884d200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88480b4960 con 0x7f884815b330 2024-09-10T10:03:25.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f884d200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f88480b5570 con 0x7f884806e850 2024-09-10T10:03:25.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f884d200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f88480b4c40 con 0x7f88480ce4d0 2024-09-10T10:03:25.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8845e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f884815b330 0x7f884806e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36454/0 (socket says 172.21.15.110:36454) 2024-09-10T10:03:25.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8845e00640 1 -- 172.21.15.110:0/3869522636 learned_addr learned my addr 172.21.15.110:0/3869522636 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:25.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8846800640 1 --2- 172.21.15.110:0/3869522636 >> v2:172.21.15.110:3300/0 conn(0x7f884806e850 0x7f88480b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8846800640 1 -- 172.21.15.110:0/3869522636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88480ce4d0 msgr2=0x7f884806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:25.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8845400640 1 --2- 172.21.15.110:0/3869522636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88480ce4d0 0x7f884806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8846800640 1 --2- 172.21.15.110:0/3869522636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88480ce4d0 0x7f884806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8844a00640 1 -- 172.21.15.110:0/3869522636 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570147748 0 0) 0x7f88480b4960 con 0x7f884815b330 2024-09-10T10:03:25.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8846800640 1 -- 172.21.15.110:0/3869522636 >> v1:172.21.15.110:6789/0 conn(0x7f884815b330 legacy=0x7f884806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8846800640 1 -- 172.21.15.110:0/3869522636 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88480b4f20 con 0x7f884806e850 2024-09-10T10:03:25.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.167+0000 7f8845400640 1 --2- 172.21.15.110:0/3869522636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88480ce4d0 0x7f884806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:25.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f8846800640 1 --2- 172.21.15.110:0/3869522636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884806e850 0x7f88480b0860 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8840009d90 tx=0x7f884001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ef6f9fab559819 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:25.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f8844a00640 1 -- 172.21.15.110:0/3869522636 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88400b2070 con 0x7f884806e850 2024-09-10T10:03:25.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f8844a00640 1 -- 172.21.15.110:0/3869522636 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88400a5940 con 0x7f884806e850 2024-09-10T10:03:25.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f8844a00640 1 -- 172.21.15.110:0/3869522636 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88400a5d10 con 0x7f884806e850 2024-09-10T10:03:25.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 -- 172.21.15.110:0/3869522636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884806e850 msgr2=0x7f88480b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 --2- 172.21.15.110:0/3869522636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884806e850 0x7f88480b0860 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8840009d90 tx=0x7f884001c970 comp rx=0 tx=0).stop 2024-09-10T10:03:25.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 -- 172.21.15.110:0/3869522636 shutdown_connections 2024-09-10T10:03:25.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 --2- 172.21.15.110:0/3869522636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884806e850 0x7f88480b0860 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 --2- 172.21.15.110:0/3869522636 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88480ce4d0 0x7f884806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 -- 172.21.15.110:0/3869522636 >> 172.21.15.110:0/3869522636 conn(0x7f8848062e40 msgr2=0x7f884806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:25.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 -- 172.21.15.110:0/3869522636 shutdown_connections 2024-09-10T10:03:25.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 -- 172.21.15.110:0/3869522636 wait complete. 2024-09-10T10:03:25.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.171+0000 7f884d200640 1 Processor -- start 2024-09-10T10:03:25.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 -- start start 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f884806e850 0x7f8848156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884815b330 0x7f88481f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88481f7120 0x7f884815d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88481680b0 con 0x7f884815b330 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f88480b17c0 con 0x7f884806e850 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8848168230 con 0x7f88481f7120 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884815b330 0x7f88481f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884815b330 0x7f88481f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37400/0 (socket says 172.21.15.110:37400) 2024-09-10T10:03:25.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 -- 172.21.15.110:0/152281730 learned_addr learned my addr 172.21.15.110:0/152281730 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845e00640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f884806e850 0x7f8848156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8846800640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88481f7120 0x7f884815d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 -- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88481f7120 msgr2=0x7f884815d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88481f7120 0x7f884815d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 -- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f884806e850 msgr2=0x7f8848156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f884806e850 0x7f8848156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 -- 172.21.15.110:0/152281730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88481574f0 con 0x7f884815b330 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f8845400640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884815b330 0x7f88481f6be0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f883400c7f0 tx=0x7f883400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:25.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f883e000640 1 -- 172.21.15.110:0/152281730 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834015070 con 0x7f884815b330 2024-09-10T10:03:25.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f883e000640 1 -- 172.21.15.110:0/152281730 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8834007a30 con 0x7f884815b330 2024-09-10T10:03:25.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88481f7700 con 0x7f884815b330 2024-09-10T10:03:25.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f883e000640 1 -- 172.21.15.110:0/152281730 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834007e20 con 0x7f884815b330 2024-09-10T10:03:25.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8848163900 con 0x7f884815b330 2024-09-10T10:03:25.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.175+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88480b0840 con 0x7f884815b330 2024-09-10T10:03:25.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.179+0000 7f883e000640 1 -- 172.21.15.110:0/152281730 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f8834018070 con 0x7f884815b330 2024-09-10T10:03:25.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.179+0000 7f883e000640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8828067bc0 0x7f882806a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.179+0000 7f883e000640 1 -- 172.21.15.110:0/152281730 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f8834085dd0 con 0x7f884815b330 2024-09-10T10:03:25.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.179+0000 7f8845e00640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8828067bc0 0x7f882806a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.179+0000 7f8845e00640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8828067bc0 0x7f882806a080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8838000f70 tx=0x7f8838007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:25.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.183+0000 7f883e000640 1 -- 172.21.15.110:0/152281730 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f88340564c0 con 0x7f884815b330 2024-09-10T10:03:25.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.363+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol58330", "target": ["mon-mgr", ""]}) -- 0x7f8848063600 con 0x7f8828067bc0 2024-09-10T10:03:25.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f883e000640 1 -- 172.21.15.110:0/152281730 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f8848063600 con 0x7f8828067bc0 2024-09-10T10:03:25.389 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322 2024-09-10T10:03:25.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8828067bc0 msgr2=0x7f882806a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8828067bc0 0x7f882806a080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8838000f70 tx=0x7f8838007c10 comp rx=0 tx=0).stop 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884815b330 msgr2=0x7f88481f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884815b330 0x7f88481f6be0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f883400c7f0 tx=0x7f883400ccb0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 shutdown_connections 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8828067bc0 0x7f882806a080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88481f7120 0x7f884815d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884815b330 0x7f88481f6be0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 --2- 172.21.15.110:0/152281730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f884806e850 0x7f8848156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 >> 172.21.15.110:0/152281730 conn(0x7f8848062e40 msgr2=0x7f88480ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 shutdown_connections 2024-09-10T10:03:25.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.387+0000 7f884d200640 1 -- 172.21.15.110:0/152281730 wait complete. 2024-09-10T10:03:25.410 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone81830 2024-09-10T10:03:25.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae32400640 1 Processor -- start 2024-09-10T10:03:25.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae32400640 1 -- start start 2024-09-10T10:03:25.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae32400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fae2c0b2bb0 0x7fae2c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae32400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c0cfbc0 0x7fae2c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae32400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae2c0b37c0 con 0x7fae2c145f30 2024-09-10T10:03:25.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae32400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fae2c0b3aa0 con 0x7fae2c0b2bb0 2024-09-10T10:03:25.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae32400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fae2c0b3d80 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae2aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fae2c0b2bb0 0x7fae2c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae2b400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fae2c145f30 0x7fae2c0b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36458/0 (socket says 172.21.15.110:36458) 2024-09-10T10:03:25.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae2b400640 1 -- 172.21.15.110:0/421584676 learned_addr learned my addr 172.21.15.110:0/421584676 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:25.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae2aa00640 1 -- 172.21.15.110:0/421584676 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c0cfbc0 msgr2=0x7fae2c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:25.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae2aa00640 1 --2- 172.21.15.110:0/421584676 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c0cfbc0 0x7fae2c0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae2aa00640 1 -- 172.21.15.110:0/421584676 >> v1:172.21.15.110:6789/0 conn(0x7fae2c145f30 legacy=0x7fae2c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae2aa00640 1 -- 172.21.15.110:0/421584676 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c146310 con 0x7fae2c0b2bb0 2024-09-10T10:03:25.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.547+0000 7fae2a000640 1 -- 172.21.15.110:0/421584676 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1453404937 0 0) 0x7fae2c0b37c0 con 0x7fae2c145f30 2024-09-10T10:03:25.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae2aa00640 1 --2- 172.21.15.110:0/421584676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0b2bb0 0x7fae2c0cf680 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fae180097c0 tx=0x7fae1801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5a42648e1fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:25.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae2a000640 1 -- 172.21.15.110:0/421584676 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180b0070 con 0x7fae2c0b2bb0 2024-09-10T10:03:25.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae2a000640 1 -- 172.21.15.110:0/421584676 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae180ab600 con 0x7fae2c0b2bb0 2024-09-10T10:03:25.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae2a000640 1 -- 172.21.15.110:0/421584676 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180ab9d0 con 0x7fae2c0b2bb0 2024-09-10T10:03:25.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae32400640 1 -- 172.21.15.110:0/421584676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0b2bb0 msgr2=0x7fae2c0cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae32400640 1 --2- 172.21.15.110:0/421584676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0b2bb0 0x7fae2c0cf680 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fae180097c0 tx=0x7fae1801c4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae32400640 1 -- 172.21.15.110:0/421584676 shutdown_connections 2024-09-10T10:03:25.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae32400640 1 --2- 172.21.15.110:0/421584676 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c0cfbc0 0x7fae2c0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae32400640 1 --2- 172.21.15.110:0/421584676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0b2bb0 0x7fae2c0cf680 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae32400640 1 -- 172.21.15.110:0/421584676 >> 172.21.15.110:0/421584676 conn(0x7fae2c062e40 msgr2=0x7fae2c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:25.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae32400640 1 -- 172.21.15.110:0/421584676 shutdown_connections 2024-09-10T10:03:25.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.551+0000 7fae32400640 1 -- 172.21.15.110:0/421584676 wait complete. 2024-09-10T10:03:25.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 Processor -- start 2024-09-10T10:03:25.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 -- start start 2024-09-10T10:03:25.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2c0b2bb0 0x7fae2c15bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0cfbc0 0x7fae2c0aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c145f30 0x7fae2c15e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c0b5660 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c0b54e0 con 0x7fae2c0b2bb0 2024-09-10T10:03:25.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fae2c0b57e0 con 0x7fae2c145f30 2024-09-10T10:03:25.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0cfbc0 0x7fae2c0aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0cfbc0 0x7fae2c0aac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37418/0 (socket says 172.21.15.110:37418) 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 -- 172.21.15.110:0/399138552 learned_addr learned my addr 172.21.15.110:0/399138552 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2b400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2c0b2bb0 0x7fae2c15bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2be00640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c145f30 0x7fae2c15e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 -- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c145f30 msgr2=0x7fae2c15e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c145f30 0x7fae2c15e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 -- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2c0b2bb0 msgr2=0x7fae2c15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2c0b2bb0 0x7fae2c15bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 -- 172.21.15.110:0/399138552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c0ab6e0 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2be00640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c145f30 0x7fae2c15e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae2aa00640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0cfbc0 0x7fae2c0aac20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fae1801a0d0 tx=0x7fae18019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae23400640 1 -- 172.21.15.110:0/399138552 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180b0040 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae23400640 1 -- 172.21.15.110:0/399138552 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae180ba070 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae23400640 1 -- 172.21.15.110:0/399138552 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180aa9f0 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2c13e290 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2c13e850 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.555+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae2c0cf680 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.559+0000 7fae23400640 1 -- 172.21.15.110:0/399138552 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7fae180aab90 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.559+0000 7fae23400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fae0c067b90 0x7fae0c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:25.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.559+0000 7fae23400640 1 -- 172.21.15.110:0/399138552 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fae18120230 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.559+0000 7fae2b400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fae0c067b90 0x7fae0c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:25.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.559+0000 7fae2b400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fae0c067b90 0x7fae0c06a050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0012f0 tx=0x7fae1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:25.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.563+0000 7fae23400640 1 -- 172.21.15.110:0/399138552 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fae18016090 con 0x7fae2c0cfbc0 2024-09-10T10:03:25.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.743+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone81830", "target": ["mon-mgr", ""]}) -- 0x7fae2c13ea20 con 0x7fae0c067b90 2024-09-10T10:03:25.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.751+0000 7fae23400640 1 -- 172.21.15.110:0/399138552 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7fae2c13ea20 con 0x7fae0c067b90 2024-09-10T10:03:25.755 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fae0c067b90 msgr2=0x7fae0c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fae0c067b90 0x7fae0c06a050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0012f0 tx=0x7fae1c009040 comp rx=0 tx=0).stop 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0cfbc0 msgr2=0x7fae2c0aac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0cfbc0 0x7fae2c0aac20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fae1801a0d0 tx=0x7fae18019240 comp rx=0 tx=0).stop 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 shutdown_connections 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fae0c067b90 0x7fae0c06a050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae2c145f30 0x7fae2c15e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae2c0cfbc0 0x7fae2c0aac20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 --2- 172.21.15.110:0/399138552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae2c0b2bb0 0x7fae2c15bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 >> 172.21.15.110:0/399138552 conn(0x7fae2c062e40 msgr2=0x7fae2c1607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:25.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 shutdown_connections 2024-09-10T10:03:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:25.755+0000 7fae32400640 1 -- 172.21.15.110:0/399138552 wait complete. 2024-09-10T10:03:25.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:25.776 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:25.787 INFO:teuthology.orchestra.run.smithi110.stdout:9 2024-09-10T10:03:25.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:25.788 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669) 2024-09-10T10:03:25.838 INFO:teuthology.orchestra.run.smithi110.stdout:9 2024-09-10T10:03:25.838 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone81830 2024-09-10T10:03:26.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.019+0000 7fe469c00640 1 Processor -- start 2024-09-10T10:03:26.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.019+0000 7fe469c00640 1 -- start start 2024-09-10T10:03:26.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe469c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:26.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe469c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4640b4960 con 0x7fe4640ce4d0 2024-09-10T10:03:26.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe469c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4640b4c40 con 0x7fe46415b330 2024-09-10T10:03:26.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe469c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe4640b5570 con 0x7fe46406e850 2024-09-10T10:03:26.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe462000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe4640ce4d0 0x7fe4640b0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:36466/0 (socket says 172.21.15.110:36466) 2024-09-10T10:03:26.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe462000640 1 -- 172.21.15.110:0/3084265781 learned_addr learned my addr 172.21.15.110:0/3084265781 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:26.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe463400640 1 --2- 172.21.15.110:0/3084265781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:26.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe461600640 1 -- 172.21.15.110:0/3084265781 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198560176 0 0) 0x7fe4640b4960 con 0x7fe4640ce4d0 2024-09-10T10:03:26.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe461600640 1 -- 172.21.15.110:0/3084265781 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe448003530 con 0x7fe4640ce4d0 2024-09-10T10:03:26.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe461600640 1 -- 172.21.15.110:0/3084265781 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2098499934 0 0) 0x7fe4640b4c40 con 0x7fe46415b330 2024-09-10T10:03:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe461600640 1 -- 172.21.15.110:0/3084265781 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4640b4960 con 0x7fe46415b330 2024-09-10T10:03:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe463400640 1 -- 172.21.15.110:0/3084265781 >> v1:172.21.15.145:6789/0 conn(0x7fe46415b330 legacy=0x7fe46406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe461600640 1 -- 172.21.15.110:0/3084265781 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3562818768 0 0) 0x7fe448003530 con 0x7fe4640ce4d0 2024-09-10T10:03:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe463400640 1 -- 172.21.15.110:0/3084265781 >> v1:172.21.15.110:6789/0 conn(0x7fe4640ce4d0 legacy=0x7fe4640b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.023+0000 7fe463400640 1 -- 172.21.15.110:0/3084265781 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4640b4f20 con 0x7fe46406e850 2024-09-10T10:03:26.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe463400640 1 --2- 172.21.15.110:0/3084265781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46406c6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0097f0 tx=0x7fe45c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2c81d5df50673d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:26.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe461600640 1 -- 172.21.15.110:0/3084265781 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c0b0070 con 0x7fe46406e850 2024-09-10T10:03:26.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe461600640 1 -- 172.21.15.110:0/3084265781 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c0ab400 con 0x7fe46406e850 2024-09-10T10:03:26.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe461600640 1 -- 172.21.15.110:0/3084265781 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450001680 con 0x7fe46406e850 2024-09-10T10:03:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- 172.21.15.110:0/3084265781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 msgr2=0x7fe46406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 --2- 172.21.15.110:0/3084265781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46406c6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0097f0 tx=0x7fe45c01c400 comp rx=0 tx=0).stop 2024-09-10T10:03:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- 172.21.15.110:0/3084265781 shutdown_connections 2024-09-10T10:03:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 --2- 172.21.15.110:0/3084265781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46406c6a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- 172.21.15.110:0/3084265781 >> 172.21.15.110:0/3084265781 conn(0x7fe464062e40 msgr2=0x7fe46406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- 172.21.15.110:0/3084265781 shutdown_connections 2024-09-10T10:03:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- 172.21.15.110:0/3084265781 wait complete. 2024-09-10T10:03:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 Processor -- start 2024-09-10T10:03:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- start start 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe46415b330 0x7fe4640aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe464156ab0 0x7fe464156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe464169450 con 0x7fe464156ab0 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe4641692d0 con 0x7fe46415b330 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe469c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe4641695d0 con 0x7fe46406e850 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe463400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe464156ab0 0x7fe464156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe463400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe464156ab0 0x7fe464156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37434/0 (socket says 172.21.15.110:37434) 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.027+0000 7fe463400640 1 -- 172.21.15.110:0/2357798825 learned_addr learned my addr 172.21.15.110:0/2357798825 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe463400640 1 -- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 msgr2=0x7fe46415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe462a00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe462000640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe46415b330 0x7fe4640aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe463400640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe463400640 1 -- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe46415b330 msgr2=0x7fe4640aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe463400640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe46415b330 0x7fe4640aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe463400640 1 -- 172.21.15.110:0/2357798825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe464157480 con 0x7fe464156ab0 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe462000640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe46415b330 0x7fe4640aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe462a00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe463400640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe464156ab0 0x7fe464156f40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0ae040 tx=0x7fe45c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe45aa00640 1 -- 172.21.15.110:0/2357798825 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c0b0040 con 0x7fe464156ab0 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe45aa00640 1 -- 172.21.15.110:0/2357798825 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c0abcb0 con 0x7fe464156ab0 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe45aa00640 1 -- 172.21.15.110:0/2357798825 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c006e20 con 0x7fe464156ab0 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe464159460 con 0x7fe464156ab0 2024-09-10T10:03:26.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe464159f40 con 0x7fe464156ab0 2024-09-10T10:03:26.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46406c6a0 con 0x7fe464156ab0 2024-09-10T10:03:26.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe45aa00640 1 -- 172.21.15.110:0/2357798825 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7fe45c0bc020 con 0x7fe464156ab0 2024-09-10T10:03:26.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe45aa00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe444067b90 0x7fe44406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:26.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe45aa00640 1 -- 172.21.15.110:0/2357798825 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fe45c11f760 con 0x7fe464156ab0 2024-09-10T10:03:26.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.031+0000 7fe462a00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe444067b90 0x7fe44406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:26.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.035+0000 7fe462a00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe444067b90 0x7fe44406a050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe454001570 tx=0x7fe454009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:26.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.039+0000 7fe45aa00640 1 -- 172.21.15.110:0/2357798825 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe45c015ff0 con 0x7fe464156ab0 2024-09-10T10:03:26.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.219+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone81830", "target": ["mon-mgr", ""]}) -- 0x7fe46415e790 con 0x7fe444067b90 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe45aa00640 1 -- 172.21.15.110:0/2357798825 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+725 (secure 0 0 0) 0x7fe46415e790 con 0x7fe444067b90 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "atime": "2024-09-10 10:03:14", 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_pcent": "80.00", 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_quota": 10485760, 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "bytes_used": 8388608, 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "created_at": "2024-09-10 10:03:19", 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "ctime": "2024-09-10 10:03:20", 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "data_pool": "cephfs_data", 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "features": [ 2024-09-10T10:03:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot-clone", 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot-autoprotect", 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot-retention" 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "flavor": 2, 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "gid": 1000, 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": 16895, 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "mon_addrs": [ 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.110:6789", 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.145:6789", 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "172.21.15.145:6790" 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-10T10:03:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "mtime": "2024-09-10 10:03:19", 2024-09-10T10:03:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "path": "/volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669", 2024-09-10T10:03:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_namespace": "", 2024-09-10T10:03:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "complete", 2024-09-10T10:03:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "clone", 2024-09-10T10:03:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "uid": 1000 2024-09-10T10:03:26.235 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:26.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe444067b90 msgr2=0x7fe44406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:26.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe444067b90 0x7fe44406a050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe454001570 tx=0x7fe454009040 comp rx=0 tx=0).stop 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe464156ab0 msgr2=0x7fe464156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe464156ab0 0x7fe464156f40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0ae040 tx=0x7fe45c0191a0 comp rx=0 tx=0).stop 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 shutdown_connections 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe444067b90 0x7fe44406a050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe464156ab0 0x7fe464156f40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe46415b330 0x7fe4640aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 --2- 172.21.15.110:0/2357798825 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe46406e850 0x7fe46415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 >> 172.21.15.110:0/2357798825 conn(0x7fe464062e40 msgr2=0x7fe4640cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 shutdown_connections 2024-09-10T10:03:26.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:26.231+0000 7fe469c00640 1 -- 172.21.15.110:0/2357798825 wait complete. 2024-09-10T10:03:26.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.255 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:26.263 INFO:teuthology.orchestra.run.smithi110.stderr:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400: ceph.quota.max_bytes: No such attribute 2024-09-10T10:03:26.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:03:26.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.264 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:26.313 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:03:26.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.313 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:26.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.369 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:26.422 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400 2024-09-10T10:03:26.422 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.2 2024-09-10T10:03:26.422 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.5 2024-09-10T10:03:26.422 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.4 2024-09-10T10:03:26.422 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.7 2024-09-10T10:03:26.423 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.6 2024-09-10T10:03:26.423 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.0 2024-09-10T10:03:26.423 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.3 2024-09-10T10:03:26.423 INFO:teuthology.orchestra.run.smithi110.stdout:volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.1 2024-09-10T10:03:26.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.424 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:26.472 INFO:teuthology.orchestra.run.smithi110.stdout:41ff 2024-09-10T10:03:26.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.473 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/) 2024-09-10T10:03:26.527 INFO:teuthology.orchestra.run.smithi110.stdout:41ff 2024-09-10T10:03:26.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.529 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:26.580 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:26.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.581 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/) 2024-09-10T10:03:26.632 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:26.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.633 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:26.684 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:26.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.685 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/) 2024-09-10T10:03:26.736 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:26.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.737 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400) 2024-09-10T10:03:26.788 INFO:teuthology.orchestra.run.smithi110.stdout:1725962599 2024-09-10T10:03:26.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.789 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/) 2024-09-10T10:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout:1725962599 2024-09-10T10:03:26.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.841 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.2) 2024-09-10T10:03:26.893 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:26.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.894 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.2) 2024-09-10T10:03:26.947 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:26.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:26.948 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.2) 2024-09-10T10:03:27.000 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:27.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.001 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.2) 2024-09-10T10:03:27.052 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:27.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.053 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.2) 2024-09-10T10:03:27.104 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:27.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.105 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.2) 2024-09-10T10:03:27.156 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:27.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.157 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.2) 2024-09-10T10:03:27.212 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:27.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.213 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.2) 2024-09-10T10:03:27.269 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:27.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.270 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.5) 2024-09-10T10:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:27.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.326 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.5) 2024-09-10T10:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:27.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.381 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.5) 2024-09-10T10:03:27.436 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:27.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.437 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.5) 2024-09-10T10:03:27.492 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:27.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.493 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.5) 2024-09-10T10:03:27.544 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:27.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.545 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.5) 2024-09-10T10:03:27.596 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:27.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.598 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.5) 2024-09-10T10:03:27.657 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:27.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.658 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.5) 2024-09-10T10:03:27.712 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:27.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.713 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.4) 2024-09-10T10:03:27.765 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:27.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.766 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.4) 2024-09-10T10:03:27.817 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:27.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.818 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.4) 2024-09-10T10:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:27.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.869 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.4) 2024-09-10T10:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:27.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.921 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.4) 2024-09-10T10:03:27.972 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:27.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:27.974 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.4) 2024-09-10T10:03:28.028 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:28.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.030 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.4) 2024-09-10T10:03:28.084 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:28.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.085 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.4) 2024-09-10T10:03:28.141 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:28.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.142 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.7) 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.195 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.7) 2024-09-10T10:03:28.249 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:28.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.250 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.7) 2024-09-10T10:03:28.300 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:28.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.302 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.7) 2024-09-10T10:03:28.357 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:28.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.358 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.7) 2024-09-10T10:03:28.408 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:28.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.409 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.7) 2024-09-10T10:03:28.460 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:28.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.461 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.7) 2024-09-10T10:03:28.514 INFO:teuthology.orchestra.run.smithi110.stdout:1725962599 2024-09-10T10:03:28.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.516 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.7) 2024-09-10T10:03:28.564 INFO:teuthology.orchestra.run.smithi110.stdout:1725962599 2024-09-10T10:03:28.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.565 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.6) 2024-09-10T10:03:28.617 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:28.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.618 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.6) 2024-09-10T10:03:28.669 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:28.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.670 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.6) 2024-09-10T10:03:28.720 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:28.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.721 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.6) 2024-09-10T10:03:28.772 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:28.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.773 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.6) 2024-09-10T10:03:28.824 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:28.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.825 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.6) 2024-09-10T10:03:28.877 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:28.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.878 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.6) 2024-09-10T10:03:28.932 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:28.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.933 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.6) 2024-09-10T10:03:28.989 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:28.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:28.990 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.0) 2024-09-10T10:03:29.045 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:29.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.047 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.0) 2024-09-10T10:03:29.097 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:29.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.099 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.0) 2024-09-10T10:03:29.149 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:29.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.150 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.0) 2024-09-10T10:03:29.201 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:29.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.202 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.0) 2024-09-10T10:03:29.253 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:29.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.254 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.0) 2024-09-10T10:03:29.305 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:29.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.306 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.0) 2024-09-10T10:03:29.359 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:29.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.360 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.0) 2024-09-10T10:03:29.412 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:29.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.413 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.3) 2024-09-10T10:03:29.466 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:29.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.467 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.3) 2024-09-10T10:03:29.517 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:29.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.518 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.3) 2024-09-10T10:03:29.568 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:29.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.570 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.3) 2024-09-10T10:03:29.621 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:29.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.622 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.3) 2024-09-10T10:03:29.677 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:29.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.678 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.3) 2024-09-10T10:03:29.733 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:29.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.734 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.3) 2024-09-10T10:03:29.793 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:29.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.794 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.3) 2024-09-10T10:03:29.845 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:29.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.846 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.1) 2024-09-10T10:03:29.898 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:29.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.899 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.1) 2024-09-10T10:03:29.950 INFO:teuthology.orchestra.run.smithi110.stdout:81b4 2024-09-10T10:03:29.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:29.951 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.1) 2024-09-10T10:03:30.005 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:30.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:30.006 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.1) 2024-09-10T10:03:30.057 INFO:teuthology.orchestra.run.smithi110.stdout:1000 2024-09-10T10:03:30.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:30.058 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.1) 2024-09-10T10:03:30.113 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:30.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:30.114 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.1) 2024-09-10T10:03:30.169 INFO:teuthology.orchestra.run.smithi110.stdout:1271 2024-09-10T10:03:30.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:30.170 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322/.snap/subvol_snap58400/subvolume_file.1) 2024-09-10T10:03:30.221 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:30.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:30.222 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone81830/dc4881bb-022d-4b83-b1bb-cda6a5fb5669/subvolume_file.1) 2024-09-10T10:03:30.273 INFO:teuthology.orchestra.run.smithi110.stdout:1725962598 2024-09-10T10:03:30.274 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol58330 fake subvol_clone83191 2024-09-10T10:03:30.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.463+0000 7f33a8c00640 1 Processor -- start 2024-09-10T10:03:30.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.463+0000 7f33a8c00640 1 -- start start 2024-09-10T10:03:30.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a8c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f33a415c560 0x7f33a406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a8c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a406f430 con 0x7f33a406f010 2024-09-10T10:03:30.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a8c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a4155d50 con 0x7f33a4155970 2024-09-10T10:03:30.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a8c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f33a40b4570 con 0x7f33a415c560 2024-09-10T10:03:30.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a2800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f33a406f010 0x7f33a40ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41654/0 (socket says 172.21.15.110:41654) 2024-09-10T10:03:30.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a2800640 1 -- 172.21.15.110:0/4283994179 learned_addr learned my addr 172.21.15.110:0/4283994179 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:30.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a1400640 1 --2- 172.21.15.110:0/4283994179 >> v2:172.21.15.145:3300/0 conn(0x7f33a415c560 0x7f33a406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a0a00640 1 -- 172.21.15.110:0/4283994179 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3723047211 0 0) 0x7f33a406f430 con 0x7f33a406f010 2024-09-10T10:03:30.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a0a00640 1 -- 172.21.15.110:0/4283994179 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3388003530 con 0x7f33a406f010 2024-09-10T10:03:30.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a1400640 1 -- 172.21.15.110:0/4283994179 >> v1:172.21.15.145:6789/0 conn(0x7f33a4155970 legacy=0x7f33a40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a1400640 1 -- 172.21.15.110:0/4283994179 >> v1:172.21.15.110:6789/0 conn(0x7f33a406f010 legacy=0x7f33a40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.467+0000 7f33a1400640 1 -- 172.21.15.110:0/4283994179 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a40b34d0 con 0x7f33a415c560 2024-09-10T10:03:30.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a1400640 1 --2- 172.21.15.110:0/4283994179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a415c560 0x7f33a406ead0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f33900097c0 tx=0x7f339001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5330a7ae71e01fdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:30.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a0a00640 1 -- 172.21.15.110:0/4283994179 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33900a7070 con 0x7f33a415c560 2024-09-10T10:03:30.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a0a00640 1 -- 172.21.15.110:0/4283994179 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33900b0070 con 0x7f33a415c560 2024-09-10T10:03:30.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a0a00640 1 -- 172.21.15.110:0/4283994179 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33900ab890 con 0x7f33a415c560 2024-09-10T10:03:30.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 -- 172.21.15.110:0/4283994179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a415c560 msgr2=0x7f33a406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 --2- 172.21.15.110:0/4283994179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a415c560 0x7f33a406ead0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f33900097c0 tx=0x7f339001c470 comp rx=0 tx=0).stop 2024-09-10T10:03:30.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 -- 172.21.15.110:0/4283994179 shutdown_connections 2024-09-10T10:03:30.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 --2- 172.21.15.110:0/4283994179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a415c560 0x7f33a406ead0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 -- 172.21.15.110:0/4283994179 >> 172.21.15.110:0/4283994179 conn(0x7f33a406bc50 msgr2=0x7f33a406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:30.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 -- 172.21.15.110:0/4283994179 shutdown_connections 2024-09-10T10:03:30.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 -- 172.21.15.110:0/4283994179 wait complete. 2024-09-10T10:03:30.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 Processor -- start 2024-09-10T10:03:30.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.471+0000 7f33a8c00640 1 -- start start 2024-09-10T10:03:30.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a406f010 0x7f33a413ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33a4155970 0x7f33a40aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33a415c560 0x7f33a40accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40cf340 con 0x7f33a415c560 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40b4050 con 0x7f33a406f010 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f33a40cf4c0 con 0x7f33a4155970 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33a415c560 0x7f33a40accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a406f010 0x7f33a413ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a1400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33a4155970 0x7f33a40aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33a415c560 0x7f33a40accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57340/0 (socket says 172.21.15.110:57340) 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 -- 172.21.15.110:0/1750750422 learned_addr learned my addr 172.21.15.110:0/1750750422 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 -- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33a4155970 msgr2=0x7f33a40aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33a4155970 0x7f33a40aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 -- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a406f010 msgr2=0x7f33a413ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a406f010 0x7f33a413ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 -- 172.21.15.110:0/1750750422 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a40af260 con 0x7f33a415c560 2024-09-10T10:03:30.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a2800640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33a415c560 0x7f33a40accd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f339c00cb70 tx=0x7f339c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:30.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f339a000640 1 -- 172.21.15.110:0/1750750422 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c017070 con 0x7f33a415c560 2024-09-10T10:03:30.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f339a000640 1 -- 172.21.15.110:0/1750750422 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c00ad40 con 0x7f33a415c560 2024-09-10T10:03:30.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f339a000640 1 -- 172.21.15.110:0/1750750422 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c012620 con 0x7f33a415c560 2024-09-10T10:03:30.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a414e960 con 0x7f33a415c560 2024-09-10T10:03:30.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a414ef20 con 0x7f33a415c560 2024-09-10T10:03:30.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a406ead0 con 0x7f33a415c560 2024-09-10T10:03:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.475+0000 7f339a000640 1 -- 172.21.15.110:0/1750750422 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f339c01a070 con 0x7f33a415c560 2024-09-10T10:03:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.479+0000 7f339a000640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3384067b90 0x7f338406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.479+0000 7f339a000640 1 -- 172.21.15.110:0/1750750422 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f339c085f10 con 0x7f33a415c560 2024-09-10T10:03:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.479+0000 7f33a1e00640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3384067b90 0x7f338406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.479+0000 7f33a1e00640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3384067b90 0x7f338406a050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3394007fd0 tx=0x7f3394009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:30.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.483+0000 7f339a000640 1 -- 172.21.15.110:0/1750750422 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f33a40af260 con 0x7f33a415c560 2024-09-10T10:03:30.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.667+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol58330", "snap_name": "fake", "target_sub_name": "subvol_clone83191", "target": ["mon-mgr", ""]}) -- 0x7f33a4075560 con 0x7f3384067b90 2024-09-10T10:03:30.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.743+0000 7f339a000640 1 -- 172.21.15.110:0/1750750422 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33a4075560 con 0x7f3384067b90 2024-09-10T10:03:30.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3384067b90 msgr2=0x7f338406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3384067b90 0x7f338406a050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3394007fd0 tx=0x7f3394009040 comp rx=0 tx=0).stop 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33a415c560 msgr2=0x7f33a40accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33a415c560 0x7f33a40accd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f339c00cb70 tx=0x7f339c00a430 comp rx=0 tx=0).stop 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 shutdown_connections 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3384067b90 0x7f338406a050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33a415c560 0x7f33a40accd0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f33a4155970 0x7f33a40aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 --2- 172.21.15.110:0/1750750422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33a406f010 0x7f33a413ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 >> 172.21.15.110:0/1750750422 conn(0x7f33a406bc50 msgr2=0x7f33a406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 shutdown_connections 2024-09-10T10:03:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.747+0000 7f33a8c00640 1 -- 172.21.15.110:0/1750750422 wait complete. 2024-09-10T10:03:30.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone83191 2024-09-10T10:03:30.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3e7400640 1 Processor -- start 2024-09-10T10:03:30.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3e7400640 1 -- start start 2024-09-10T10:03:30.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3e7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa3e00b2bb0 0x7fa3e00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3e7400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e00b37c0 con 0x7fa3e0145f30 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3e7400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e00b3aa0 con 0x7fa3e00cfbc0 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3e7400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e00b3d80 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3dfe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa3e00b2bb0 0x7fa3e00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3e4a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa3e0145f30 0x7fa3e00b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41670/0 (socket says 172.21.15.110:41670) 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3e4a00640 1 -- 172.21.15.110:0/3318806838 learned_addr learned my addr 172.21.15.110:0/3318806838 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3dfe00640 1 -- 172.21.15.110:0/3318806838 >> v1:172.21.15.145:6789/0 conn(0x7fa3e00cfbc0 legacy=0x7fa3e00b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3dfe00640 1 -- 172.21.15.110:0/3318806838 >> v1:172.21.15.110:6789/0 conn(0x7fa3e0145f30 legacy=0x7fa3e00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3dfe00640 1 -- 172.21.15.110:0/3318806838 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e0146310 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.907+0000 7fa3df400640 1 -- 172.21.15.110:0/3318806838 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724785276 0 0) 0x7fa3e00b37c0 con 0x7fa3e0145f30 2024-09-10T10:03:30.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3dfe00640 1 --2- 172.21.15.110:0/3318806838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e00cf680 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097c0 tx=0x7fa3cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232b1b0db94e61c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:30.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3df400640 1 -- 172.21.15.110:0/3318806838 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0b0070 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3df400640 1 -- 172.21.15.110:0/3318806838 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0ab600 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3df400640 1 -- 172.21.15.110:0/3318806838 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0ab9d0 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3e7400640 1 -- 172.21.15.110:0/3318806838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 msgr2=0x7fa3e00cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3e7400640 1 --2- 172.21.15.110:0/3318806838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e00cf680 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097c0 tx=0x7fa3cc01c4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3e7400640 1 -- 172.21.15.110:0/3318806838 shutdown_connections 2024-09-10T10:03:30.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3e7400640 1 --2- 172.21.15.110:0/3318806838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e00cf680 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3e7400640 1 -- 172.21.15.110:0/3318806838 >> 172.21.15.110:0/3318806838 conn(0x7fa3e0062e40 msgr2=0x7fa3e006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:30.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3e7400640 1 -- 172.21.15.110:0/3318806838 shutdown_connections 2024-09-10T10:03:30.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.911+0000 7fa3e7400640 1 -- 172.21.15.110:0/3318806838 wait complete. 2024-09-10T10:03:30.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 Processor -- start 2024-09-10T10:03:30.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 -- start start 2024-09-10T10:03:30.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3e00cfbc0 0x7fa3e015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3e0145f30 0x7fa3e0161b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e00b58b0 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e00b5730 con 0x7fa3e00cfbc0 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e016a1b0 con 0x7fa3e0145f30 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e015f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57350/0 (socket says 172.21.15.110:57350) 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 -- 172.21.15.110:0/976692079 learned_addr learned my addr 172.21.15.110:0/976692079 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3dfe00640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3e00cfbc0 0x7fa3e015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e5400640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3e0145f30 0x7fa3e0161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 -- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3e0145f30 msgr2=0x7fa3e0161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3e0145f30 0x7fa3e0161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 -- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3e00cfbc0 msgr2=0x7fa3e015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:30.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3e00cfbc0 0x7fa3e015e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:30.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 -- 172.21.15.110:0/976692079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e01670f0 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e4a00640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e015f0d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c4a0 tx=0x7fa3d000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:30.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3dcc00640 1 -- 172.21.15.110:0/976692079 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0018070 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3dcc00640 1 -- 172.21.15.110:0/976692079 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d00074e0 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01673d0 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3dcc00640 1 -- 172.21.15.110:0/976692079 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0004a30 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e0167990 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.915+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e00cf680 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.919+0000 7fa3dcc00640 1 -- 172.21.15.110:0/976692079 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7fa3d001b070 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.919+0000 7fa3dcc00640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa3c0067bc0 0x7fa3c006a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:30.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.919+0000 7fa3dcc00640 1 -- 172.21.15.110:0/976692079 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fa3d00853a0 con 0x7fa3e00b2bb0 2024-09-10T10:03:30.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.919+0000 7fa3dfe00640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa3c0067bc0 0x7fa3c006a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:30.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.919+0000 7fa3dfe00640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa3c0067bc0 0x7fa3c006a080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ad000 tx=0x7fa3cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:30.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:30.923+0000 7fa3dcc00640 1 -- 172.21.15.110:0/976692079 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa3d0055500 con 0x7fa3e00b2bb0 2024-09-10T10:03:31.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:31.111+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone83191", "target": ["mon-mgr", ""]}) -- 0x7fa3e0144c80 con 0x7fa3c0067bc0 2024-09-10T10:03:34.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3dcc00640 1 -- 172.21.15.110:0/976692079 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7fa3e0144c80 con 0x7fa3c0067bc0 2024-09-10T10:03:34.942 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "status": { 2024-09-10T10:03:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "pending", 2024-09-10T10:03:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "source": { 2024-09-10T10:03:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "volume": "cephfs", 2024-09-10T10:03:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "subvolume": "subvol58330", 2024-09-10T10:03:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot": "fake" 2024-09-10T10:03:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:34.943 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa3c0067bc0 msgr2=0x7fa3c006a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa3c0067bc0 0x7fa3c006a080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ad000 tx=0x7fa3cc008040 comp rx=0 tx=0).stop 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 msgr2=0x7fa3e015f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e015f0d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c4a0 tx=0x7fa3d000c960 comp rx=0 tx=0).stop 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 shutdown_connections 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa3c0067bc0 0x7fa3c006a080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa3e0145f30 0x7fa3e0161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3e00cfbc0 0x7fa3e015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 --2- 172.21.15.110:0/976692079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3e00b2bb0 0x7fa3e015f0d0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 >> 172.21.15.110:0/976692079 conn(0x7fa3e0062e40 msgr2=0x7fa3e0143760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:34.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 shutdown_connections 2024-09-10T10:03:34.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:34.939+0000 7fa3e7400640 1 -- 172.21.15.110:0/976692079 wait complete. 2024-09-10T10:03:35.974 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone83191 2024-09-10T10:03:36.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0443a00640 1 Processor -- start 2024-09-10T10:03:36.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0443a00640 1 -- start start 2024-09-10T10:03:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0443a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f043c0b5590 0x7f043c0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0443a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f043c0b37c0 con 0x7f043c145f30 2024-09-10T10:03:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0443a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f043c0b3aa0 con 0x7f043c0b2bb0 2024-09-10T10:03:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0443a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f043c0b3d80 con 0x7f043c0b5590 2024-09-10T10:03:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0441000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f043c145f30 0x7f043c0b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41676/0 (socket says 172.21.15.110:41676) 2024-09-10T10:03:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0441000640 1 -- 172.21.15.110:0/1476307249 learned_addr learned my addr 172.21.15.110:0/1476307249 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0441a00640 1 --2- 172.21.15.110:0/1476307249 >> v2:172.21.15.145:3300/0 conn(0x7f043c0b5590 0x7f043c0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3361504743 0 0) 0x7f043c0b37c0 con 0x7f043c145f30 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0420003530 con 0x7f043c145f30 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2513018312 0 0) 0x7f0420003530 con 0x7f043c145f30 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 >> v2:172.21.15.145:3300/0 conn(0x7f043c0b5590 msgr2=0x7f043c0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 --2- 172.21.15.110:0/1476307249 >> v2:172.21.15.145:3300/0 conn(0x7f043c0b5590 0x7f043c0cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 >> v1:172.21.15.145:6789/0 conn(0x7f043c0b2bb0 legacy=0x7f043c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c146310 con 0x7f043c145f30 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f042c001a20 con 0x7f043c145f30 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 >> v1:172.21.15.110:6789/0 conn(0x7f043c145f30 legacy=0x7f043c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200039c0 0x7f0420003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0420004810 0x7f0420004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04200051d0 0x7f042000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f043c0b3d80 con 0x7f04200051d0 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f042000c170 con 0x7f04200039c0 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f042000c370 con 0x7f0420004810 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0441a00640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04200051d0 0x7f042000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f043be00640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0420004810 0x7f0420004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.123+0000 7f0441000640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200039c0 0x7f0420003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0441a00640 1 -- 172.21.15.110:0/1476307249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0420004810 msgr2=0x7f0420004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0441a00640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0420004810 0x7f0420004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0441a00640 1 -- 172.21.15.110:0/1476307249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200039c0 msgr2=0x7f0420003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0441a00640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200039c0 0x7f0420003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0441a00640 1 -- 172.21.15.110:0/1476307249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0420003530 con 0x7f04200051d0 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0441a00640 1 -- 172.21.15.110:0/1476307249 reap_dead start 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0441a00640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04200051d0 0x7f042000b9d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0434009ae0 tx=0x7f043401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a694428004e5710f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04340096b0 con 0x7f04200051d0 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0434004020 con 0x7f04200051d0 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f043b400640 1 -- 172.21.15.110:0/1476307249 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434004410 con 0x7f04200051d0 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0443a00640 1 -- 172.21.15.110:0/1476307249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04200051d0 msgr2=0x7f042000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0443a00640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04200051d0 0x7f042000b9d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0434009ae0 tx=0x7f043401cab0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0443a00640 1 -- 172.21.15.110:0/1476307249 shutdown_connections 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0443a00640 1 --2- 172.21.15.110:0/1476307249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04200051d0 0x7f042000b9d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0443a00640 1 -- 172.21.15.110:0/1476307249 >> 172.21.15.110:0/1476307249 conn(0x7f043c062e40 msgr2=0x7f043c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:36.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0443a00640 1 -- 172.21.15.110:0/1476307249 shutdown_connections 2024-09-10T10:03:36.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.127+0000 7f0443a00640 1 -- 172.21.15.110:0/1476307249 wait complete. 2024-09-10T10:03:36.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 Processor -- start 2024-09-10T10:03:36.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 -- start start 2024-09-10T10:03:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200051d0 0x7f043c1f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c13ff90 0x7f043c1403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c15dc30 0x7f043c15e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f043c0b2830 con 0x7f043c13ff90 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f043c0b26b0 con 0x7f04200051d0 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f043c0b29b0 con 0x7f043c15dc30 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c13ff90 0x7f043c1403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c13ff90 0x7f043c1403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57368/0 (socket says 172.21.15.110:57368) 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 -- 172.21.15.110:0/2684936297 learned_addr learned my addr 172.21.15.110:0/2684936297 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0441000640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200051d0 0x7f043c1f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 -- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c15dc30 msgr2=0x7f043c15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0441a00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c15dc30 0x7f043c15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c15dc30 0x7f043c15e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 -- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200051d0 msgr2=0x7f043c1f14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200051d0 0x7f043c1f14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 -- 172.21.15.110:0/2684936297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c1417a0 con 0x7f043c13ff90 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0441000640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200051d0 0x7f043c1f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f043be00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c13ff90 0x7f043c1403f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f042800f510 tx=0x7f042800f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0438c00640 1 -- 172.21.15.110:0/2684936297 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04280052e0 con 0x7f043c13ff90 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0438c00640 1 -- 172.21.15.110:0/2684936297 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0428018070 con 0x7f043c13ff90 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0438c00640 1 -- 172.21.15.110:0/2684936297 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428005480 con 0x7f043c13ff90 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043c13e750 con 0x7f043c13ff90 2024-09-10T10:03:36.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.131+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f043c140f00 con 0x7f043c13ff90 2024-09-10T10:03:36.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.135+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043c0cf680 con 0x7f043c13ff90 2024-09-10T10:03:36.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.135+0000 7f0438c00640 1 -- 172.21.15.110:0/2684936297 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f0428003820 con 0x7f043c13ff90 2024-09-10T10:03:36.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.135+0000 7f0438c00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c067b90 0x7f041c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.135+0000 7f0438c00640 1 -- 172.21.15.110:0/2684936297 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f0428085e30 con 0x7f043c13ff90 2024-09-10T10:03:36.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.135+0000 7f0441000640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c067b90 0x7f041c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.135+0000 7f0441000640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c067b90 0x7f041c06a050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f042c0015b0 tx=0x7f042c001540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.139+0000 7f0438c00640 1 -- 172.21.15.110:0/2684936297 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f04280565a0 con 0x7f043c13ff90 2024-09-10T10:03:36.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:36.331+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone83191", "target": ["mon-mgr", ""]}) -- 0x7f043c13acb0 con 0x7f041c067b90 2024-09-10T10:03:39.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.927+0000 7f0438c00640 1 -- 172.21.15.110:0/2684936297 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f043c13acb0 con 0x7f041c067b90 2024-09-10T10:03:39.932 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:39.932 INFO:teuthology.orchestra.run.smithi110.stdout: "status": { 2024-09-10T10:03:39.932 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "complete" 2024-09-10T10:03:39.932 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:39.932 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c067b90 msgr2=0x7f041c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c067b90 0x7f041c06a050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f042c0015b0 tx=0x7f042c001540 comp rx=0 tx=0).stop 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c13ff90 msgr2=0x7f043c1403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c13ff90 0x7f043c1403f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f042800f510 tx=0x7f042800f9d0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 shutdown_connections 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c067b90 0x7f041c06a050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c15dc30 0x7f043c15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c13ff90 0x7f043c1403f0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 --2- 172.21.15.110:0/2684936297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04200051d0 0x7f043c1f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 >> 172.21.15.110:0/2684936297 conn(0x7f043c062e40 msgr2=0x7f043c0cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 shutdown_connections 2024-09-10T10:03:39.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:39.931+0000 7f0443a00640 1 -- 172.21.15.110:0/2684936297 wait complete. 2024-09-10T10:03:39.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:39.958 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone83191/.meta) 2024-09-10T10:03:39.980 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:03:39.980 INFO:teuthology.orchestra.run.smithi110.stdout:version = 2 2024-09-10T10:03:39.980 INFO:teuthology.orchestra.run.smithi110.stdout:type = clone 2024-09-10T10:03:39.980 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/_nogroup/subvol_clone83191/18c6c0a7-35a2-4e79-98f6-309b63b747e9 2024-09-10T10:03:39.980 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:03:39.980 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:39.982 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol58330 subvol_snap58400 2024-09-10T10:03:40.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.163+0000 7fddce000640 1 Processor -- start 2024-09-10T10:03:40.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.163+0000 7fddce000640 1 -- start start 2024-09-10T10:03:40.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddce000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fddc80aef50 0x7fddc806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddce000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc806ec20 0x7fddc8072310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddce000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc81482e0 con 0x7fddc8147f00 2024-09-10T10:03:40.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddce000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fddc80b34d0 con 0x7fddc80aef50 2024-09-10T10:03:40.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddce000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fddc815ee20 con 0x7fddc806ec20 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc7400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fddc8147f00 0x7fddc80ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60312/0 (socket says 172.21.15.110:60312) 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc7400640 1 -- 172.21.15.110:0/4238822763 learned_addr learned my addr 172.21.15.110:0/4238822763 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc7e00640 1 --2- 172.21.15.110:0/4238822763 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc806ec20 0x7fddc8072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc6a00640 1 --2- 172.21.15.110:0/4238822763 >> v2:172.21.15.145:3300/0 conn(0x7fddc80aef50 0x7fddc806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc6000640 1 -- 172.21.15.110:0/4238822763 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3120937730 0 0) 0x7fddc81482e0 con 0x7fddc8147f00 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc6000640 1 -- 172.21.15.110:0/4238822763 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddac003530 con 0x7fddc8147f00 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc7e00640 1 -- 172.21.15.110:0/4238822763 >> v2:172.21.15.145:3300/0 conn(0x7fddc80aef50 msgr2=0x7fddc806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc7e00640 1 --2- 172.21.15.110:0/4238822763 >> v2:172.21.15.145:3300/0 conn(0x7fddc80aef50 0x7fddc806e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc7e00640 1 -- 172.21.15.110:0/4238822763 >> v1:172.21.15.110:6789/0 conn(0x7fddc8147f00 legacy=0x7fddc80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc6000640 1 -- 172.21.15.110:0/4238822763 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688491210 0 0) 0x7fddac003530 con 0x7fddc8147f00 2024-09-10T10:03:40.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.167+0000 7fddc7e00640 1 -- 172.21.15.110:0/4238822763 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc815f100 con 0x7fddc806ec20 2024-09-10T10:03:40.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddc7e00640 1 --2- 172.21.15.110:0/4238822763 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc806ec20 0x7fddc8072310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009d90 tx=0x7fddc001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca08da60ee5222c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddc6000640 1 -- 172.21.15.110:0/4238822763 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00ac070 con 0x7fddc806ec20 2024-09-10T10:03:40.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddc6000640 1 -- 172.21.15.110:0/4238822763 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc00a64a0 con 0x7fddc806ec20 2024-09-10T10:03:40.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddc6000640 1 -- 172.21.15.110:0/4238822763 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8001680 con 0x7fddc806ec20 2024-09-10T10:03:40.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 -- 172.21.15.110:0/4238822763 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc806ec20 msgr2=0x7fddc8072310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 --2- 172.21.15.110:0/4238822763 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc806ec20 0x7fddc8072310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009d90 tx=0x7fddc001c890 comp rx=0 tx=0).stop 2024-09-10T10:03:40.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 -- 172.21.15.110:0/4238822763 shutdown_connections 2024-09-10T10:03:40.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 --2- 172.21.15.110:0/4238822763 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc806ec20 0x7fddc8072310 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 --2- 172.21.15.110:0/4238822763 >> v2:172.21.15.145:3300/0 conn(0x7fddc80aef50 0x7fddc806e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 -- 172.21.15.110:0/4238822763 >> 172.21.15.110:0/4238822763 conn(0x7fddc806bc50 msgr2=0x7fddc806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:40.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 -- 172.21.15.110:0/4238822763 shutdown_connections 2024-09-10T10:03:40.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 -- 172.21.15.110:0/4238822763 wait complete. 2024-09-10T10:03:40.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 Processor -- start 2024-09-10T10:03:40.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.171+0000 7fddce000640 1 -- start start 2024-09-10T10:03:40.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddc806ec20 0x7fddc8145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc80aef50 0x7fddc81378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddc8147f00 0x7fddc81433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fddc80731e0 con 0x7fddc806ec20 2024-09-10T10:03:40.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8073060 con 0x7fddc8147f00 2024-09-10T10:03:40.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fddc8073360 con 0x7fddc80aef50 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddc806ec20 0x7fddc8145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddc806ec20 0x7fddc8145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59654/0 (socket says 172.21.15.110:59654) 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 -- 172.21.15.110:0/542448846 learned_addr learned my addr 172.21.15.110:0/542448846 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc6a00640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc80aef50 0x7fddc81378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7e00640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddc8147f00 0x7fddc81433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 -- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc80aef50 msgr2=0x7fddc81378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc80aef50 0x7fddc81378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 -- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddc8147f00 msgr2=0x7fddc81433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddc8147f00 0x7fddc81433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 -- 172.21.15.110:0/542448846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc8144a40 con 0x7fddc806ec20 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc6a00640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc80aef50 0x7fddc81378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddc7400640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddc806ec20 0x7fddc8145f80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c6d0 tx=0x7fddb800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddbf400640 1 -- 172.21.15.110:0/542448846 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8017070 con 0x7fddc806ec20 2024-09-10T10:03:40.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddbf400640 1 -- 172.21.15.110:0/542448846 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb8005ae0 con 0x7fddc806ec20 2024-09-10T10:03:40.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc8144d20 con 0x7fddc806ec20 2024-09-10T10:03:40.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddbf400640 1 -- 172.21.15.110:0/542448846 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8007070 con 0x7fddc806ec20 2024-09-10T10:03:40.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc8145a80 con 0x7fddc806ec20 2024-09-10T10:03:40.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc80722f0 con 0x7fddc806ec20 2024-09-10T10:03:40.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.175+0000 7fddbf400640 1 -- 172.21.15.110:0/542448846 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7fddb801a070 con 0x7fddc806ec20 2024-09-10T10:03:40.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.179+0000 7fddbf400640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdda8067b90 0x7fdda806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.179+0000 7fddbf400640 1 -- 172.21.15.110:0/542448846 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fddb80854f0 con 0x7fddc806ec20 2024-09-10T10:03:40.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.179+0000 7fddc6a00640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdda8067b90 0x7fdda806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.179+0000 7fddc6a00640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdda8067b90 0x7fdda806a050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fddc81464c0 tx=0x7fddb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.183+0000 7fddbf400640 1 -- 172.21.15.110:0/542448846 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fddb80556e0 con 0x7fddc806ec20 2024-09-10T10:03:40.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.367+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol58330", "snap_name": "subvol_snap58400", "target": ["mon-mgr", ""]}) -- 0x7fddc80b3b90 con 0x7fdda8067b90 2024-09-10T10:03:40.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddbf400640 1 -- 172.21.15.110:0/542448846 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7fddc80b3b90 con 0x7fdda8067b90 2024-09-10T10:03:40.390 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "created_at": "2024-09-10 10:03:19.447855", 2024-09-10T10:03:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "data_pool": "cephfs_data", 2024-09-10T10:03:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "has_pending_clones": "no" 2024-09-10T10:03:40.390 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:40.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdda8067b90 msgr2=0x7fdda806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdda8067b90 0x7fdda806a050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fddc81464c0 tx=0x7fddb4008040 comp rx=0 tx=0).stop 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddc806ec20 msgr2=0x7fddc8145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddc806ec20 0x7fddc8145f80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c6d0 tx=0x7fddb800cb90 comp rx=0 tx=0).stop 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 shutdown_connections 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdda8067b90 0x7fdda806a050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fddc8147f00 0x7fddc81433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fddc80aef50 0x7fddc81378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 --2- 172.21.15.110:0/542448846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddc806ec20 0x7fddc8145f80 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 >> 172.21.15.110:0/542448846 conn(0x7fddc806bc50 msgr2=0x7fddc80b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 shutdown_connections 2024-09-10T10:03:40.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.387+0000 7fddce000640 1 -- 172.21.15.110:0/542448846 wait complete. 2024-09-10T10:03:40.411 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol58330 2024-09-10T10:03:40.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.547+0000 7f0013a00640 1 Processor -- start 2024-09-10T10:03:40.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.547+0000 7f0013a00640 1 -- start start 2024-09-10T10:03:40.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0013a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f000c06e7a0 0x7f000c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0013a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000c072150 con 0x7f000c0cfe30 2024-09-10T10:03:40.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0013a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000c06d2d0 con 0x7f000c0cd510 2024-09-10T10:03:40.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0013a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f000c15d9a0 con 0x7f000c06e7a0 2024-09-10T10:03:40.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0011000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f000c0cfe30 0x7f000c071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60320/0 (socket says 172.21.15.110:60320) 2024-09-10T10:03:40.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0011000640 1 -- 172.21.15.110:0/2900575929 learned_addr learned my addr 172.21.15.110:0/2900575929 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:40.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0011a00640 1 --2- 172.21.15.110:0/2900575929 >> v2:172.21.15.110:3300/0 conn(0x7f000c06e7a0 0x7f000c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f000b400640 1 -- 172.21.15.110:0/2900575929 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4284168554 0 0) 0x7f000c072150 con 0x7f000c0cfe30 2024-09-10T10:03:40.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f000b400640 1 -- 172.21.15.110:0/2900575929 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efff0003530 con 0x7f000c0cfe30 2024-09-10T10:03:40.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0011a00640 1 -- 172.21.15.110:0/2900575929 >> v1:172.21.15.145:6789/0 conn(0x7f000c0cd510 legacy=0x7f000c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0011a00640 1 -- 172.21.15.110:0/2900575929 >> v1:172.21.15.110:6789/0 conn(0x7f000c0cfe30 legacy=0x7f000c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.551+0000 7f0011a00640 1 -- 172.21.15.110:0/2900575929 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c06d5b0 con 0x7f000c06e7a0 2024-09-10T10:03:40.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0011a00640 1 --2- 172.21.15.110:0/2900575929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c06e7a0 0x7f000c0cd9a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0004009d90 tx=0x7f000401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df70beb14856227 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f000b400640 1 -- 172.21.15.110:0/2900575929 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040b2070 con 0x7f000c06e7a0 2024-09-10T10:03:40.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f000b400640 1 -- 172.21.15.110:0/2900575929 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00040a58b0 con 0x7f000c06e7a0 2024-09-10T10:03:40.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f000b400640 1 -- 172.21.15.110:0/2900575929 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040a5c80 con 0x7f000c06e7a0 2024-09-10T10:03:40.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- 172.21.15.110:0/2900575929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c06e7a0 msgr2=0x7f000c0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 --2- 172.21.15.110:0/2900575929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c06e7a0 0x7f000c0cd9a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0004009d90 tx=0x7f000401c8e0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- 172.21.15.110:0/2900575929 shutdown_connections 2024-09-10T10:03:40.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 --2- 172.21.15.110:0/2900575929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c06e7a0 0x7f000c0cd9a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- 172.21.15.110:0/2900575929 >> 172.21.15.110:0/2900575929 conn(0x7f000c06bc50 msgr2=0x7f000c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:40.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- 172.21.15.110:0/2900575929 shutdown_connections 2024-09-10T10:03:40.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- 172.21.15.110:0/2900575929 wait complete. 2024-09-10T10:03:40.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 Processor -- start 2024-09-10T10:03:40.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- start start 2024-09-10T10:03:40.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000c06e7a0 0x7f000c147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f000c0cd510 0x7f000c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c0cfe30 0x7f000c0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f000c16a190 con 0x7f000c0cfe30 2024-09-10T10:03:40.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f000c15d700 con 0x7f000c0cd510 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.555+0000 7f0013a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f000c16a310 con 0x7f000c06e7a0 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c0cfe30 0x7f000c0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c0cfe30 0x7f000c0b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59672/0 (socket says 172.21.15.110:59672) 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 -- 172.21.15.110:0/4001386489 learned_addr learned my addr 172.21.15.110:0/4001386489 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011000640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000c06e7a0 0x7f000c147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f000be00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f000c0cd510 0x7f000c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 -- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000c06e7a0 msgr2=0x7f000c147d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000c06e7a0 0x7f000c147d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 -- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f000c0cd510 msgr2=0x7f000c0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f000c0cd510 0x7f000c0b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 -- 172.21.15.110:0/4001386489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c148290 con 0x7f000c0cfe30 2024-09-10T10:03:40.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0011a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c0cfe30 0x7f000c0b24c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f00040b0040 tx=0x7f0004019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0008c00640 1 -- 172.21.15.110:0/4001386489 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040b2050 con 0x7f000c0cfe30 2024-09-10T10:03:40.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0008c00640 1 -- 172.21.15.110:0/4001386489 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00040be070 con 0x7f000c0cfe30 2024-09-10T10:03:40.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0008c00640 1 -- 172.21.15.110:0/4001386489 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004009450 con 0x7f000c0cfe30 2024-09-10T10:03:40.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c157610 con 0x7f000c0cfe30 2024-09-10T10:03:40.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c15a040 con 0x7f000c0cfe30 2024-09-10T10:03:40.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c06e090 con 0x7f000c0cfe30 2024-09-10T10:03:40.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0008c00640 1 -- 172.21.15.110:0/4001386489 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f00040adc00 con 0x7f000c0cfe30 2024-09-10T10:03:40.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0008c00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7effec067bc0 0x7effec06a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.559+0000 7f0008c00640 1 -- 172.21.15.110:0/4001386489 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f00041211c0 con 0x7f000c0cfe30 2024-09-10T10:03:40.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.563+0000 7f0011000640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7effec067bc0 0x7effec06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.563+0000 7f0011000640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7effec067bc0 0x7effec06a080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efffc0011f0 tx=0x7efffc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.567+0000 7f0008c00640 1 -- 172.21.15.110:0/4001386489 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0004016560 con 0x7f000c0cfe30 2024-09-10T10:03:40.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.747+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol58330", "target": ["mon-mgr", ""]}) -- 0x7f000c071bc0 con 0x7effec067bc0 2024-09-10T10:03:40.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0008c00640 1 -- 172.21.15.110:0/4001386489 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7f000c071bc0 con 0x7effec067bc0 2024-09-10T10:03:40.766 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:03:40.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:40.767 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "fake" 2024-09-10T10:03:40.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:40.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:40.767 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "subvol_snap58400" 2024-09-10T10:03:40.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:40.767 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:03:40.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7effec067bc0 msgr2=0x7effec06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7effec067bc0 0x7effec06a080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efffc0011f0 tx=0x7efffc009040 comp rx=0 tx=0).stop 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c0cfe30 msgr2=0x7f000c0b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c0cfe30 0x7f000c0b24c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f00040b0040 tx=0x7f0004019680 comp rx=0 tx=0).stop 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 shutdown_connections 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7effec067bc0 0x7effec06a080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f000c0cfe30 0x7f000c0b24c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f000c0cd510 0x7f000c0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 --2- 172.21.15.110:0/4001386489 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f000c06e7a0 0x7f000c147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 >> 172.21.15.110:0/4001386489 conn(0x7f000c06bc50 msgr2=0x7f000c15c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 shutdown_connections 2024-09-10T10:03:40.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.763+0000 7f0013a00640 1 -- 172.21.15.110:0/4001386489 wait complete. 2024-09-10T10:03:40.787 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol58330 subvol_snap58400 2024-09-10T10:03:40.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.923+0000 7f1149600640 1 Processor -- start 2024-09-10T10:03:40.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.923+0000 7f1149600640 1 -- start start 2024-09-10T10:03:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.923+0000 7f1149600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f11440cfe30 0x7f1144071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.923+0000 7f1149600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114406e7a0 0x7f11440cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.923+0000 7f1149600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114406d2d0 con 0x7f11440cd510 2024-09-10T10:03:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.923+0000 7f1149600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f114415d9a0 con 0x7f11440cfe30 2024-09-10T10:03:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.923+0000 7f1149600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f114406d5b0 con 0x7f114406e7a0 2024-09-10T10:03:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1142000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f11440cd510 0x7f114406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60332/0 (socket says 172.21.15.110:60332) 2024-09-10T10:03:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1142a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f11440cfe30 0x7f1144071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1142000640 1 -- 172.21.15.110:0/228914584 learned_addr learned my addr 172.21.15.110:0/228914584 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1143400640 1 --2- 172.21.15.110:0/228914584 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114406e7a0 0x7f11440cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1142a00640 1 -- 172.21.15.110:0/228914584 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114406e7a0 msgr2=0x7f11440cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1142a00640 1 --2- 172.21.15.110:0/228914584 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114406e7a0 0x7f11440cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1142a00640 1 -- 172.21.15.110:0/228914584 >> v1:172.21.15.110:6789/0 conn(0x7f11440cd510 legacy=0x7f114406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1142a00640 1 -- 172.21.15.110:0/228914584 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114406d890 con 0x7f11440cfe30 2024-09-10T10:03:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1141600640 1 -- 172.21.15.110:0/228914584 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602056959 0 0) 0x7f114406d2d0 con 0x7f11440cd510 2024-09-10T10:03:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1143400640 1 --2- 172.21.15.110:0/228914584 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114406e7a0 0x7f11440cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:40.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1142a00640 1 --2- 172.21.15.110:0/228914584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11440cfe30 0x7f1144071a40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f11340097c0 tx=0x7f113401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcf77db9451597f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1141600640 1 -- 172.21.15.110:0/228914584 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11340b0070 con 0x7f11440cfe30 2024-09-10T10:03:40.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1141600640 1 -- 172.21.15.110:0/228914584 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11340ab600 con 0x7f11440cfe30 2024-09-10T10:03:40.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.927+0000 7f1141600640 1 -- 172.21.15.110:0/228914584 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11340ab9d0 con 0x7f11440cfe30 2024-09-10T10:03:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- 172.21.15.110:0/228914584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11440cfe30 msgr2=0x7f1144071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 --2- 172.21.15.110:0/228914584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11440cfe30 0x7f1144071a40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f11340097c0 tx=0x7f113401c410 comp rx=0 tx=0).stop 2024-09-10T10:03:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- 172.21.15.110:0/228914584 shutdown_connections 2024-09-10T10:03:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 --2- 172.21.15.110:0/228914584 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114406e7a0 0x7f11440cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 --2- 172.21.15.110:0/228914584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11440cfe30 0x7f1144071a40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- 172.21.15.110:0/228914584 >> 172.21.15.110:0/228914584 conn(0x7f114406bc50 msgr2=0x7f114406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:40.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- 172.21.15.110:0/228914584 shutdown_connections 2024-09-10T10:03:40.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- 172.21.15.110:0/228914584 wait complete. 2024-09-10T10:03:40.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 Processor -- start 2024-09-10T10:03:40.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- start start 2024-09-10T10:03:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114406e7a0 0x7f11441526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11440cd510 0x7f11441fa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11440cfe30 0x7f11440b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1144072190 con 0x7f114406e7a0 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f114415d680 con 0x7f11440cd510 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1149600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1144169180 con 0x7f11440cfe30 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1142a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114406e7a0 0x7f11441526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1142a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114406e7a0 0x7f11441526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59692/0 (socket says 172.21.15.110:59692) 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.931+0000 7f1142a00640 1 -- 172.21.15.110:0/901916263 learned_addr learned my addr 172.21.15.110:0/901916263 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1142000640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11440cd510 0x7f11441fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1143400640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11440cfe30 0x7f11440b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1142a00640 1 -- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11440cfe30 msgr2=0x7f11440b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1142a00640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11440cfe30 0x7f11440b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1142a00640 1 -- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11440cd510 msgr2=0x7f11441fa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1142a00640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11440cd510 0x7f11441fa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1142a00640 1 -- 172.21.15.110:0/901916263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11441fb370 con 0x7f114406e7a0 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1142000640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11440cd510 0x7f11441fa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1142a00640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114406e7a0 0x7f11441526c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f11340ae040 tx=0x7f113401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f113aa00640 1 -- 172.21.15.110:0/901916263 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11340b0040 con 0x7f114406e7a0 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f113aa00640 1 -- 172.21.15.110:0/901916263 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11340abe10 con 0x7f114406e7a0 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f113aa00640 1 -- 172.21.15.110:0/901916263 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134004220 con 0x7f114406e7a0 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11441faad0 con 0x7f114406e7a0 2024-09-10T10:03:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11441fb090 con 0x7f114406e7a0 2024-09-10T10:03:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11440b3670 con 0x7f114406e7a0 2024-09-10T10:03:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.935+0000 7f113aa00640 1 -- 172.21.15.110:0/901916263 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f11340bf060 con 0x7f114406e7a0 2024-09-10T10:03:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.939+0000 7f113aa00640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1124067b90 0x7f112406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.939+0000 7f113aa00640 1 -- 172.21.15.110:0/901916263 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f113411fe50 con 0x7f114406e7a0 2024-09-10T10:03:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.939+0000 7f1142000640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1124067b90 0x7f112406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.939+0000 7f1142000640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1124067b90 0x7f112406a050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f11440cd0b0 tx=0x7f1130007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:40.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:40.943+0000 7f113aa00640 1 -- 172.21.15.110:0/901916263 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1134016090 con 0x7f114406e7a0 2024-09-10T10:03:41.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.131+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol58330", "snap_name": "subvol_snap58400", "target": ["mon-mgr", ""]}) -- 0x7f11440ce480 con 0x7f1124067b90 2024-09-10T10:03:41.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.159+0000 7f113aa00640 1 -- 172.21.15.110:0/901916263 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f11440ce480 con 0x7f1124067b90 2024-09-10T10:03:41.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1124067b90 msgr2=0x7f112406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1124067b90 0x7f112406a050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f11440cd0b0 tx=0x7f1130007cb0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114406e7a0 msgr2=0x7f11441526c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114406e7a0 0x7f11441526c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f11340ae040 tx=0x7f113401ac50 comp rx=0 tx=0).stop 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 shutdown_connections 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1124067b90 0x7f112406a050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f11440cfe30 0x7f11440b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11440cd510 0x7f11441fa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 --2- 172.21.15.110:0/901916263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114406e7a0 0x7f11441526c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 >> 172.21.15.110:0/901916263 conn(0x7f114406bc50 msgr2=0x7f11440b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 shutdown_connections 2024-09-10T10:03:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.163+0000 7f1149600640 1 -- 172.21.15.110:0/901916263 wait complete. 2024-09-10T10:03:41.186 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol58330 fake 2024-09-10T10:03:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8056a00640 1 Processor -- start 2024-09-10T10:03:41.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8056a00640 1 -- start start 2024-09-10T10:03:41.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8056a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f805015e970 0x7f805015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8056a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f805006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8056a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8050073750 con 0x7f8050072f00 2024-09-10T10:03:41.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8056a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f80500b4f10 con 0x7f805015e970 2024-09-10T10:03:41.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8056a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f80500b51f0 con 0x7f805006eb60 2024-09-10T10:03:41.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f804fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f805015e970 0x7f805015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8054a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f805006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8054a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f805006eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:37496/0 (socket says 172.21.15.110:37496) 2024-09-10T10:03:41.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8054a00640 1 -- 172.21.15.110:0/4213922399 learned_addr learned my addr 172.21.15.110:0/4213922399 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:41.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8054a00640 1 -- 172.21.15.110:0/4213922399 >> v2:172.21.15.145:3300/0 conn(0x7f805015e970 msgr2=0x7f805015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8054a00640 1 --2- 172.21.15.110:0/4213922399 >> v2:172.21.15.145:3300/0 conn(0x7f805015e970 0x7f805015ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8054a00640 1 -- 172.21.15.110:0/4213922399 >> v1:172.21.15.145:6789/0 conn(0x7f8050072f00 legacy=0x7f80500cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f8054a00640 1 -- 172.21.15.110:0/4213922399 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8050073a30 con 0x7f805006eb60 2024-09-10T10:03:41.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.327+0000 7f804fe00640 1 --2- 172.21.15.110:0/4213922399 >> v2:172.21.15.145:3300/0 conn(0x7f805015e970 0x7f805015ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:41.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.331+0000 7f8054a00640 1 --2- 172.21.15.110:0/4213922399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f805006eff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8048009e50 tx=0x7f804801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fbcac09589d4bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.331+0000 7f804f400640 1 -- 172.21.15.110:0/4213922399 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480a5810 con 0x7f805006eb60 2024-09-10T10:03:41.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.331+0000 7f804f400640 1 -- 172.21.15.110:0/4213922399 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80480a59b0 con 0x7f805006eb60 2024-09-10T10:03:41.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.331+0000 7f804f400640 1 -- 172.21.15.110:0/4213922399 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480a5d80 con 0x7f805006eb60 2024-09-10T10:03:41.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- 172.21.15.110:0/4213922399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 msgr2=0x7f805006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 --2- 172.21.15.110:0/4213922399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f805006eff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8048009e50 tx=0x7f804801ca30 comp rx=0 tx=0).stop 2024-09-10T10:03:41.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- 172.21.15.110:0/4213922399 shutdown_connections 2024-09-10T10:03:41.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 --2- 172.21.15.110:0/4213922399 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f805006eff0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 --2- 172.21.15.110:0/4213922399 >> v2:172.21.15.145:3300/0 conn(0x7f805015e970 0x7f805015ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- 172.21.15.110:0/4213922399 >> 172.21.15.110:0/4213922399 conn(0x7f805006cb50 msgr2=0x7f805006cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- 172.21.15.110:0/4213922399 shutdown_connections 2024-09-10T10:03:41.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- 172.21.15.110:0/4213922399 wait complete. 2024-09-10T10:03:41.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 Processor -- start 2024-09-10T10:03:41.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- start start 2024-09-10T10:03:41.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f8050146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8050072f00 0x7f80501f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f805015e970 0x7f8050164370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8050167fc0 con 0x7f8050072f00 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f80500b6100 con 0x7f805015e970 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8056a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8050168140 con 0x7f805006eb60 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8047e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8050072f00 0x7f80501f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8047e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8050072f00 0x7f80501f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59702/0 (socket says 172.21.15.110:59702) 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8047e00640 1 -- 172.21.15.110:0/2005746500 learned_addr learned my addr 172.21.15.110:0/2005746500 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f8054a00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f805015e970 0x7f8050164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.335+0000 7f804fe00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f8050146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8047e00640 1 -- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 msgr2=0x7f8050146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8047e00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f8050146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8047e00640 1 -- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f805015e970 msgr2=0x7f8050164370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8047e00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f805015e970 0x7f8050164370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8047e00640 1 -- 172.21.15.110:0/2005746500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8050154db0 con 0x7f8050072f00 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8054a00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f805015e970 0x7f8050164370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8047e00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8050072f00 0x7f80501f7540 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f803800c370 tx=0x7f803800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f804cc00640 1 -- 172.21.15.110:0/2005746500 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380052d0 con 0x7f8050072f00 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f804cc00640 1 -- 172.21.15.110:0/2005746500 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038005470 con 0x7f8050072f00 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f804cc00640 1 -- 172.21.15.110:0/2005746500 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038007260 con 0x7f8050072f00 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8050155090 con 0x7f8050072f00 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8050154a60 con 0x7f8050072f00 2024-09-10T10:03:41.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805006eff0 con 0x7f8050072f00 2024-09-10T10:03:41.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f804cc00640 1 -- 172.21.15.110:0/2005746500 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f8038007400 con 0x7f8050072f00 2024-09-10T10:03:41.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f804cc00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f802c067b90 0x7f802c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.339+0000 7f804cc00640 1 -- 172.21.15.110:0/2005746500 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f8038085530 con 0x7f8050072f00 2024-09-10T10:03:41.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.343+0000 7f804fe00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f802c067b90 0x7f802c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.343+0000 7f804fe00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f802c067b90 0x7f802c06a050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8040009580 tx=0x7f8040009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.347+0000 7f804cc00640 1 -- 172.21.15.110:0/2005746500 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8038055650 con 0x7f8050072f00 2024-09-10T10:03:41.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.535+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol58330", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f8050155260 con 0x7f802c067b90 2024-09-10T10:03:41.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f804cc00640 1 -- 172.21.15.110:0/2005746500 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8050155260 con 0x7f802c067b90 2024-09-10T10:03:41.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f802c067b90 msgr2=0x7f802c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f802c067b90 0x7f802c06a050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8040009580 tx=0x7f8040009190 comp rx=0 tx=0).stop 2024-09-10T10:03:41.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8050072f00 msgr2=0x7f80501f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8050072f00 0x7f80501f7540 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f803800c370 tx=0x7f803800c830 comp rx=0 tx=0).stop 2024-09-10T10:03:41.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 shutdown_connections 2024-09-10T10:03:41.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f802c067b90 0x7f802c06a050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f805015e970 0x7f8050164370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8050072f00 0x7f80501f7540 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 --2- 172.21.15.110:0/2005746500 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f805006eb60 0x7f8050146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 >> 172.21.15.110:0/2005746500 conn(0x7f805006cb50 msgr2=0x7f805015dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 shutdown_connections 2024-09-10T10:03:41.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.567+0000 7f8056a00640 1 -- 172.21.15.110:0/2005746500 wait complete. 2024-09-10T10:03:41.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:41.592 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol58330/.meta) 2024-09-10T10:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T10:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/_nogroup/subvol58330/b24bacef-40c9-47bf-a3e4-75800ffe0322 2024-09-10T10:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:41.615 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol58330 2024-09-10T10:03:41.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.791+0000 7ff7fa400640 1 Processor -- start 2024-09-10T10:03:41.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.791+0000 7ff7fa400640 1 -- start start 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.791+0000 7ff7fa400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40b5590 0x7ff7f40cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.791+0000 7ff7fa400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f40b37c0 con 0x7ff7f4145f30 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.791+0000 7ff7fa400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f40b3aa0 con 0x7ff7f40b2bb0 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.791+0000 7ff7fa400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f40b3d80 con 0x7ff7f40b5590 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.791+0000 7ff7f3400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff7f4145f30 0x7ff7f40b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60346/0 (socket says 172.21.15.110:60346) 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.791+0000 7ff7f3400640 1 -- 172.21.15.110:0/2698512526 learned_addr learned my addr 172.21.15.110:0/2698512526 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f3e00640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40b5590 0x7ff7f40cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 917688013 0 0) 0x7ff7f40b37c0 con 0x7ff7f4145f30 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d8003530 con 0x7ff7f4145f30 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3001050461 0 0) 0x7ff7f40b3aa0 con 0x7ff7f40b2bb0 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7f40b37c0 con 0x7ff7f40b2bb0 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2178348392 0 0) 0x7ff7d8003530 con 0x7ff7f4145f30 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40b5590 msgr2=0x7ff7f40cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40b5590 0x7ff7f40cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 >> v1:172.21.15.145:6789/0 conn(0x7ff7f40b2bb0 legacy=0x7ff7f40cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f4146310 con 0x7ff7f4145f30 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7ff7e4001a00 con 0x7ff7f4145f30 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 >> v1:172.21.15.110:6789/0 conn(0x7ff7f4145f30 legacy=0x7ff7f40b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7d8003b80 0x7ff7d8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40b5590 0x7ff7d8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40b3d80 con 0x7ff7d8004810 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d800bb60 con 0x7ff7d8003b80 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d800bd30 con 0x7ff7f40b5590 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2a00640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f3400640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7d8003b80 0x7ff7d8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f3e00640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40b5590 0x7ff7d8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2a00640 1 -- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40b5590 msgr2=0x7ff7d8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2a00640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40b5590 0x7ff7d8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2a00640 1 -- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7d8003b80 msgr2=0x7ff7d8003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2a00640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7d8003b80 0x7ff7d8003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.795+0000 7ff7f2a00640 1 -- 172.21.15.110:0/2698512526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d8003530 con 0x7ff7d8004810 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7f2a00640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7d8004c90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00018d0 tx=0x7ff7e001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a8fe203ae68f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0001520 con 0x7ff7d8004810 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e00052f0 con 0x7ff7d8004810 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7f2000640 1 -- 172.21.15.110:0/2698512526 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0003840 con 0x7ff7d8004810 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- 172.21.15.110:0/2698512526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 msgr2=0x7ff7d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 --2- 172.21.15.110:0/2698512526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7d8004c90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00018d0 tx=0x7ff7e001c540 comp rx=0 tx=0).stop 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7f3e00640 1 -- 172.21.15.110:0/2698512526 reap_dead start 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- 172.21.15.110:0/2698512526 shutdown_connections 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- 172.21.15.110:0/2698512526 >> 172.21.15.110:0/2698512526 conn(0x7ff7f4062e40 msgr2=0x7ff7f406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- 172.21.15.110:0/2698512526 shutdown_connections 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- 172.21.15.110:0/2698512526 wait complete. 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 Processor -- start 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- start start 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7f413b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f415bf00 0x7ff7f40ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40aedb0 0x7ff7f40b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4167d30 con 0x7ff7d8004810 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4167bb0 con 0x7ff7f415bf00 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7fa400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f4167eb0 con 0x7ff7f40aedb0 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.799+0000 7ff7f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7f413b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7f413b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59718/0 (socket says 172.21.15.110:59718) 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3400640 1 -- 172.21.15.110:0/309994282 learned_addr learned my addr 172.21.15.110:0/309994282 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f2a00640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f415bf00 0x7ff7f40ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3e00640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40aedb0 0x7ff7f40b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3400640 1 -- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40aedb0 msgr2=0x7ff7f40b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40aedb0 0x7ff7f40b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3400640 1 -- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f415bf00 msgr2=0x7ff7f40ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f415bf00 0x7ff7f40ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3400640 1 -- 172.21.15.110:0/309994282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f40b2290 con 0x7ff7d8004810 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3e00640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40aedb0 0x7ff7f40b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7f3400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7f413b880 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40079a0 tx=0x7ff7e400a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7eb400640 1 -- 172.21.15.110:0/309994282 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4017020 con 0x7ff7d8004810 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7eb400640 1 -- 172.21.15.110:0/309994282 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e4007a90 con 0x7ff7d8004810 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7eb400640 1 -- 172.21.15.110:0/309994282 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4013610 con 0x7ff7d8004810 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f415cf90 con 0x7ff7d8004810 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f415d550 con 0x7ff7d8004810 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f40cf680 con 0x7ff7d8004810 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7eb400640 1 -- 172.21.15.110:0/309994282 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7ff7e401a020 con 0x7ff7d8004810 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7eb400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff7d4067bc0 0x7ff7d406a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.803+0000 7ff7eb400640 1 -- 172.21.15.110:0/309994282 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7ff7e4086a90 con 0x7ff7d8004810 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.807+0000 7ff7f2a00640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff7d4067bc0 0x7ff7d406a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.807+0000 7ff7f2a00640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff7d4067bc0 0x7ff7d406a080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00ac000 tx=0x7ff7e00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.811+0000 7ff7eb400640 1 -- 172.21.15.110:0/309994282 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff7e4021020 con 0x7ff7d8004810 2024-09-10T10:03:41.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:41.991+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol58330", "target": ["mon-mgr", ""]}) -- 0x7ff7f40af190 con 0x7ff7d4067bc0 2024-09-10T10:03:42.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7eb400640 1 -- 172.21.15.110:0/309994282 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff7f40af190 con 0x7ff7d4067bc0 2024-09-10T10:03:42.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff7d4067bc0 msgr2=0x7ff7d406a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff7d4067bc0 0x7ff7d406a080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00ac000 tx=0x7ff7e00a6000 comp rx=0 tx=0).stop 2024-09-10T10:03:42.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 msgr2=0x7ff7f413b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7f413b880 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40079a0 tx=0x7ff7e400a600 comp rx=0 tx=0).stop 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 shutdown_connections 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff7d4067bc0 0x7ff7d406a080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff7f40aedb0 0x7ff7f40b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7f415bf00 0x7ff7f40ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 --2- 172.21.15.110:0/309994282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7d8004810 0x7ff7f413b880 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 >> 172.21.15.110:0/309994282 conn(0x7ff7f4062e40 msgr2=0x7ff7f40b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 shutdown_connections 2024-09-10T10:03:42.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.031+0000 7ff7fa400640 1 -- 172.21.15.110:0/309994282 wait complete. 2024-09-10T10:03:42.039 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.038+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.039 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.038+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.039 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.038+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.039 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.038+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.039 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.038+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.058 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.054+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.058 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.054+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.058 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.054+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.058 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.054+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.058 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:03:42.054+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:42.060 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone81830 2024-09-10T10:03:42.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.203+0000 7f1553200640 1 Processor -- start 2024-09-10T10:03:42.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.203+0000 7f1553200640 1 -- start start 2024-09-10T10:03:42.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1553200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f155415bc30 0x7f155415c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1553200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15540d0ee0 con 0x7f155406beb0 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1553200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155406f2d0 con 0x7f155406c290 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1553200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f155406eff0 con 0x7f155415bc30 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1551e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f155406beb0 0x7f155415d970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60350/0 (socket says 172.21.15.110:60350) 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1551e00640 1 -- 172.21.15.110:0/654534606 learned_addr learned my addr 172.21.15.110:0/654534606 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1552800640 1 --2- 172.21.15.110:0/654534606 >> v2:172.21.15.110:3300/0 conn(0x7f155415bc30 0x7f155415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1550a00640 1 -- 172.21.15.110:0/654534606 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999799108 0 0) 0x7f15540d0ee0 con 0x7f155406beb0 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1550a00640 1 -- 172.21.15.110:0/654534606 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1538003530 con 0x7f155406beb0 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1552800640 1 -- 172.21.15.110:0/654534606 >> v1:172.21.15.145:6789/0 conn(0x7f155406c290 legacy=0x7f155406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1552800640 1 -- 172.21.15.110:0/654534606 >> v1:172.21.15.110:6789/0 conn(0x7f155406beb0 legacy=0x7f155415d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1552800640 1 -- 172.21.15.110:0/654534606 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15540d1180 con 0x7f155415bc30 2024-09-10T10:03:42.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.207+0000 7f1552800640 1 --2- 172.21.15.110:0/654534606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155415bc30 0x7f155415c060 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f154c0097c0 tx=0x7f154c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c4b68a1c491367 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:42.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1550a00640 1 -- 172.21.15.110:0/654534606 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c0b1070 con 0x7f155415bc30 2024-09-10T10:03:42.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1550a00640 1 -- 172.21.15.110:0/654534606 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154c0ac410 con 0x7f155415bc30 2024-09-10T10:03:42.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1550a00640 1 -- 172.21.15.110:0/654534606 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c0ac7c0 con 0x7f155415bc30 2024-09-10T10:03:42.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- 172.21.15.110:0/654534606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155415bc30 msgr2=0x7f155415c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 --2- 172.21.15.110:0/654534606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155415bc30 0x7f155415c060 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f154c0097c0 tx=0x7f154c01c4a0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- 172.21.15.110:0/654534606 shutdown_connections 2024-09-10T10:03:42.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 --2- 172.21.15.110:0/654534606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155415bc30 0x7f155415c060 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- 172.21.15.110:0/654534606 >> 172.21.15.110:0/654534606 conn(0x7f155406cd60 msgr2=0x7f155406d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:42.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- 172.21.15.110:0/654534606 shutdown_connections 2024-09-10T10:03:42.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- 172.21.15.110:0/654534606 wait complete. 2024-09-10T10:03:42.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 Processor -- start 2024-09-10T10:03:42.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- start start 2024-09-10T10:03:42.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155406beb0 0x7f155414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f155406c290 0x7f15540cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f155415bc30 0x7f1554140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f155406ed90 con 0x7f155406beb0 2024-09-10T10:03:42.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f155415d690 con 0x7f155415bc30 2024-09-10T10:03:42.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1553200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f155416a170 con 0x7f155406c290 2024-09-10T10:03:42.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1551e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155406beb0 0x7f155414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1551e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155406beb0 0x7f155414e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59744/0 (socket says 172.21.15.110:59744) 2024-09-10T10:03:42.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.211+0000 7f1551e00640 1 -- 172.21.15.110:0/688906741 learned_addr learned my addr 172.21.15.110:0/688906741 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:42.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1552800640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f155415bc30 0x7f1554140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1551400640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f155406c290 0x7f15540cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1551e00640 1 -- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f155406c290 msgr2=0x7f15540cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1551e00640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f155406c290 0x7f15540cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1551e00640 1 -- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f155415bc30 msgr2=0x7f1554140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1551e00640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f155415bc30 0x7f1554140140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1551e00640 1 -- 172.21.15.110:0/688906741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155414f220 con 0x7f155406beb0 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1551400640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f155406c290 0x7f15540cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1551e00640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155406beb0 0x7f155414e5e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f154400cbe0 tx=0x7f154400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f154a000640 1 -- 172.21.15.110:0/688906741 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544017070 con 0x7f155406beb0 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f154a000640 1 -- 172.21.15.110:0/688906741 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154400ad40 con 0x7f155406beb0 2024-09-10T10:03:42.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15540d0110 con 0x7f155406beb0 2024-09-10T10:03:42.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f154a000640 1 -- 172.21.15.110:0/688906741 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544012600 con 0x7f155406beb0 2024-09-10T10:03:42.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15540ccd30 con 0x7f155406beb0 2024-09-10T10:03:42.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155415c060 con 0x7f155406beb0 2024-09-10T10:03:42.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f154a000640 1 -- 172.21.15.110:0/688906741 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f154401a070 con 0x7f155406beb0 2024-09-10T10:03:42.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.215+0000 7f154a000640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1534067b90 0x7f153406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.219+0000 7f154a000640 1 -- 172.21.15.110:0/688906741 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f1544085e80 con 0x7f155406beb0 2024-09-10T10:03:42.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.219+0000 7f1551400640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1534067b90 0x7f153406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.219+0000 7f1551400640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1534067b90 0x7f153406a050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f15540b3c10 tx=0x7f1540009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:42.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.223+0000 7f154a000640 1 -- 172.21.15.110:0/688906741 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1544056030 con 0x7f155406beb0 2024-09-10T10:03:42.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.403+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone81830", "target": ["mon-mgr", ""]}) -- 0x7f155414f420 con 0x7f1534067b90 2024-09-10T10:03:42.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.423+0000 7f154a000640 1 -- 172.21.15.110:0/688906741 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1540001d40 con 0x7f1534067b90 2024-09-10T10:03:42.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1534067b90 msgr2=0x7f153406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1534067b90 0x7f153406a050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f15540b3c10 tx=0x7f1540009040 comp rx=0 tx=0).stop 2024-09-10T10:03:42.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155406beb0 msgr2=0x7f155414e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155406beb0 0x7f155414e5e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f154400cbe0 tx=0x7f154400a430 comp rx=0 tx=0).stop 2024-09-10T10:03:42.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 shutdown_connections 2024-09-10T10:03:42.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1534067b90 0x7f153406a050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f155415bc30 0x7f1554140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f155406c290 0x7f15540cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 --2- 172.21.15.110:0/688906741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155406beb0 0x7f155414e5e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 >> 172.21.15.110:0/688906741 conn(0x7f155406cd60 msgr2=0x7f155415ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:42.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 shutdown_connections 2024-09-10T10:03:42.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.427+0000 7f1553200640 1 -- 172.21.15.110:0/688906741 wait complete. 2024-09-10T10:03:42.450 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone83191 2024-09-10T10:03:42.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb7a3600640 1 Processor -- start 2024-09-10T10:03:42.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb7a3600640 1 -- start start 2024-09-10T10:03:42.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb7a3600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb79c145f30 0x7fb79c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb7a3600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79c0b37c0 con 0x7fb79c0b29e0 2024-09-10T10:03:42.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb7a3600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79c0b3aa0 con 0x7fb79c0cfd90 2024-09-10T10:03:42.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb7a3600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb79c0b2e50 con 0x7fb79c145f30 2024-09-10T10:03:42.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb7a0c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb79c145f30 0x7fb79c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb79be00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb79c0b29e0 0x7fb79c0cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60358/0 (socket says 172.21.15.110:60358) 2024-09-10T10:03:42.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.591+0000 7fb79be00640 1 -- 172.21.15.110:0/2711382155 learned_addr learned my addr 172.21.15.110:0/2711382155 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:42.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a0c00640 1 -- 172.21.15.110:0/2711382155 >> v1:172.21.15.145:6789/0 conn(0x7fb79c0cfd90 legacy=0x7fb79c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a0c00640 1 -- 172.21.15.110:0/2711382155 >> v1:172.21.15.110:6789/0 conn(0x7fb79c0b29e0 legacy=0x7fb79c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a0c00640 1 -- 172.21.15.110:0/2711382155 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c0b3d80 con 0x7fb79c145f30 2024-09-10T10:03:42.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb79b400640 1 -- 172.21.15.110:0/2711382155 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3874814185 0 0) 0x7fb79c0b37c0 con 0x7fb79c0b29e0 2024-09-10T10:03:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a0c00640 1 --2- 172.21.15.110:0/2711382155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c0b24a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d90 tx=0x7fb78c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4602d51cafe07e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb79b400640 1 -- 172.21.15.110:0/2711382155 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0b2070 con 0x7fb79c145f30 2024-09-10T10:03:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb79b400640 1 -- 172.21.15.110:0/2711382155 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c0a5830 con 0x7fb79c145f30 2024-09-10T10:03:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb79b400640 1 -- 172.21.15.110:0/2711382155 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0a5c00 con 0x7fb79c145f30 2024-09-10T10:03:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a3600640 1 -- 172.21.15.110:0/2711382155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 msgr2=0x7fb79c0b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a3600640 1 --2- 172.21.15.110:0/2711382155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c0b24a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d90 tx=0x7fb78c01c970 comp rx=0 tx=0).stop 2024-09-10T10:03:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a3600640 1 -- 172.21.15.110:0/2711382155 shutdown_connections 2024-09-10T10:03:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a3600640 1 --2- 172.21.15.110:0/2711382155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c0b24a0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.595+0000 7fb7a3600640 1 -- 172.21.15.110:0/2711382155 >> 172.21.15.110:0/2711382155 conn(0x7fb79c062e40 msgr2=0x7fb79c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 -- 172.21.15.110:0/2711382155 shutdown_connections 2024-09-10T10:03:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 -- 172.21.15.110:0/2711382155 wait complete. 2024-09-10T10:03:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 Processor -- start 2024-09-10T10:03:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 -- start start 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79c0b29e0 0x7fb79c144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79c0cfd90 0x7fb79c13ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c0b56b0 con 0x7fb79c145f30 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c0d0170 con 0x7fb79c0cfd90 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a3600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb79c0b5830 con 0x7fb79c0b29e0 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c15fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59748/0 (socket says 172.21.15.110:59748) 2024-09-10T10:03:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 -- 172.21.15.110:0/2719971226 learned_addr learned my addr 172.21.15.110:0/2719971226 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a0c00640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79c0b29e0 0x7fb79c144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb79be00640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79c0cfd90 0x7fb79c13ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 -- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79c0b29e0 msgr2=0x7fb79c144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79c0b29e0 0x7fb79c144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 -- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79c0cfd90 msgr2=0x7fb79c13ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79c0cfd90 0x7fb79c13ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 -- 172.21.15.110:0/2719971226 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c0b0330 con 0x7fb79c145f30 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a0c00640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79c0b29e0 0x7fb79c144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb79be00640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79c0cfd90 0x7fb79c13ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb7a1600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c15fab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb79400c340 tx=0x7fb79400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.599+0000 7fb798c00640 1 -- 172.21.15.110:0/2719971226 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794005220 con 0x7fb79c145f30 2024-09-10T10:03:42.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb798c00640 1 -- 172.21.15.110:0/2719971226 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7940053c0 con 0x7fb79c145f30 2024-09-10T10:03:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb798c00640 1 -- 172.21.15.110:0/2719971226 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794007270 con 0x7fb79c145f30 2024-09-10T10:03:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c0b0610 con 0x7fb79c145f30 2024-09-10T10:03:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c144da0 con 0x7fb79c145f30 2024-09-10T10:03:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c072530 con 0x7fb79c145f30 2024-09-10T10:03:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb798c00640 1 -- 172.21.15.110:0/2719971226 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7fb794019020 con 0x7fb79c145f30 2024-09-10T10:03:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb798c00640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb77c067b90 0x7fb77c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb798c00640 1 -- 172.21.15.110:0/2719971226 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fb794085470 con 0x7fb79c145f30 2024-09-10T10:03:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb7a0c00640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb77c067b90 0x7fb77c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.603+0000 7fb7a0c00640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb77c067b90 0x7fb77c06a050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0af000 tx=0x7fb78c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:42.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.607+0000 7fb798c00640 1 -- 172.21.15.110:0/2719971226 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb794055590 con 0x7fb79c145f30 2024-09-10T10:03:42.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.787+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone83191", "target": ["mon-mgr", ""]}) -- 0x7fb79c0716c0 con 0x7fb77c067b90 2024-09-10T10:03:42.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb798c00640 1 -- 172.21.15.110:0/2719971226 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb79c0716c0 con 0x7fb77c067b90 2024-09-10T10:03:42.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb77c067b90 msgr2=0x7fb77c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb77c067b90 0x7fb77c06a050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0af000 tx=0x7fb78c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:03:42.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 msgr2=0x7fb79c15fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:42.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c15fab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb79400c340 tx=0x7fb79400c800 comp rx=0 tx=0).stop 2024-09-10T10:03:42.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 shutdown_connections 2024-09-10T10:03:42.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb77c067b90 0x7fb77c06a050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb79c145f30 0x7fb79c15fab0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb79c0cfd90 0x7fb79c13ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 --2- 172.21.15.110:0/2719971226 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb79c0b29e0 0x7fb79c144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:42.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 >> 172.21.15.110:0/2719971226 conn(0x7fb79c062e40 msgr2=0x7fb79c0b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:42.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 shutdown_connections 2024-09-10T10:03:42.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42.807+0000 7fb7a3600640 1 -- 172.21.15.110:0/2719971226 wait complete. 2024-09-10T10:03:42.834 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T10:03:42.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:03:42.895 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-10T10:03:42.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:42.902 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:03:42.902 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:03:42.983 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:03:43.073 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:03:43.073 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:03:43.073 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:43.073 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:43.082 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:43.082 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:43.136 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:03:43.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f0732c00640 1 Processor -- start 2024-09-10T10:03:43.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f0732c00640 1 -- start start 2024-09-10T10:03:43.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f0732c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f072c0b5600 0x7f072c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f0732c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f072c06efa0 0x7f072c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f0732c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f072c06f410 con 0x7f072c0726e0 2024-09-10T10:03:43.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f0732c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f072c0cee50 con 0x7f072c0b5600 2024-09-10T10:03:43.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f0732c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f072c15fda0 con 0x7f072c06efa0 2024-09-10T10:03:43.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f072b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f072c0b5600 0x7f072c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f072b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f072c0b5600 0x7f072c0cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59756/0 (socket says 172.21.15.110:59756) 2024-09-10T10:03:43.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f072b400640 1 -- 172.21.15.110:0/2495349852 learned_addr learned my addr 172.21.15.110:0/2495349852 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:43.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f0730c00640 1 --2- 172.21.15.110:0/2495349852 >> v2:172.21.15.145:3300/0 conn(0x7f072c06efa0 0x7f072c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f072b400640 1 -- 172.21.15.110:0/2495349852 >> v2:172.21.15.145:3300/0 conn(0x7f072c06efa0 msgr2=0x7f072c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f072b400640 1 --2- 172.21.15.110:0/2495349852 >> v2:172.21.15.145:3300/0 conn(0x7f072c06efa0 0x7f072c0b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f072b400640 1 -- 172.21.15.110:0/2495349852 >> v1:172.21.15.145:6789/0 conn(0x7f072c0726e0 legacy=0x7f072c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.315+0000 7f072b400640 1 -- 172.21.15.110:0/2495349852 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c0ce520 con 0x7f072c0b5600 2024-09-10T10:03:43.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f072b400640 1 --2- 172.21.15.110:0/2495349852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c0cf2e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f072c0d02e0 tx=0x7f071401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f72f5052a9e6f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f072aa00640 1 -- 172.21.15.110:0/2495349852 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07140aa650 con 0x7f072c0b5600 2024-09-10T10:03:43.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f072aa00640 1 -- 172.21.15.110:0/2495349852 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07140af070 con 0x7f072c0b5600 2024-09-10T10:03:43.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f072aa00640 1 -- 172.21.15.110:0/2495349852 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07140aaa10 con 0x7f072c0b5600 2024-09-10T10:03:43.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 -- 172.21.15.110:0/2495349852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 msgr2=0x7f072c0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 --2- 172.21.15.110:0/2495349852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c0cf2e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f072c0d02e0 tx=0x7f071401c4f0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 -- 172.21.15.110:0/2495349852 shutdown_connections 2024-09-10T10:03:43.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 --2- 172.21.15.110:0/2495349852 >> v2:172.21.15.145:3300/0 conn(0x7f072c06efa0 0x7f072c0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 --2- 172.21.15.110:0/2495349852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c0cf2e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 -- 172.21.15.110:0/2495349852 >> 172.21.15.110:0/2495349852 conn(0x7f072c062d10 msgr2=0x7f072c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 -- 172.21.15.110:0/2495349852 shutdown_connections 2024-09-10T10:03:43.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 -- 172.21.15.110:0/2495349852 wait complete. 2024-09-10T10:03:43.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 Processor -- start 2024-09-10T10:03:43.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.319+0000 7f0732c00640 1 -- start start 2024-09-10T10:03:43.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f072c06efa0 0x7f072c1f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f072c0726e0 0x7f072c152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f072c072c40 con 0x7f072c0b5600 2024-09-10T10:03:43.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f072c072ac0 con 0x7f072c06efa0 2024-09-10T10:03:43.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f072c15f950 con 0x7f072c0726e0 2024-09-10T10:03:43.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59764/0 (socket says 172.21.15.110:59764) 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 -- 172.21.15.110:0/1857457188 learned_addr learned my addr 172.21.15.110:0/1857457188 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f072b400640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f072c0726e0 0x7f072c152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f072be00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f072c06efa0 0x7f072c1f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 -- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f072c0726e0 msgr2=0x7f072c152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f072c0726e0 0x7f072c152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 -- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f072c06efa0 msgr2=0x7f072c1f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f072c06efa0 0x7f072c1f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 -- 172.21.15.110:0/1857457188 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c0b10a0 con 0x7f072c0b5600 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f072b400640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f072c0726e0 0x7f072c152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f072be00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f072c06efa0 0x7f072c1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:43.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0730c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c147830 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f072400d560 tx=0x7f072400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0723e00640 1 -- 172.21.15.110:0/1857457188 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07240125f0 con 0x7f072c0b5600 2024-09-10T10:03:43.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0723e00640 1 -- 172.21.15.110:0/1857457188 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0724019070 con 0x7f072c0b5600 2024-09-10T10:03:43.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072c0b1380 con 0x7f072c0b5600 2024-09-10T10:03:43.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0723e00640 1 -- 172.21.15.110:0/1857457188 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07240129f0 con 0x7f072c0b5600 2024-09-10T10:03:43.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072c0b1c20 con 0x7f072c0b5600 2024-09-10T10:03:43.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f072c0b34d0 con 0x7f072c0b5600 2024-09-10T10:03:43.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.323+0000 7f0723e00640 1 -- 172.21.15.110:0/1857457188 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f0724012b90 con 0x7f072c0b5600 2024-09-10T10:03:43.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.327+0000 7f0723e00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0708067b90 0x7f070806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.327+0000 7f0723e00640 1 -- 172.21.15.110:0/1857457188 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f0724086530 con 0x7f072c0b5600 2024-09-10T10:03:43.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.327+0000 7f072be00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0708067b90 0x7f070806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.327+0000 7f072be00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0708067b90 0x7f070806a050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f071c001e10 tx=0x7f071c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.331+0000 7f0723e00640 1 -- 172.21.15.110:0/1857457188 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0724056ca0 con 0x7f072c0b5600 2024-09-10T10:03:43.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.499+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f072c0b1550 con 0x7f072c0b5600 2024-09-10T10:03:43.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0723e00640 1 -- 172.21.15.110:0/1857457188 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f07240565a0 con 0x7f072c0b5600 2024-09-10T10:03:43.505 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:03:43.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0708067b90 msgr2=0x7f070806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0708067b90 0x7f070806a050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f071c001e10 tx=0x7f071c009040 comp rx=0 tx=0).stop 2024-09-10T10:03:43.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 msgr2=0x7f072c147830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c147830 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f072400d560 tx=0x7f072400da20 comp rx=0 tx=0).stop 2024-09-10T10:03:43.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 shutdown_connections 2024-09-10T10:03:43.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0708067b90 0x7f070806a050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c0b5600 0x7f072c147830 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f072c0726e0 0x7f072c152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 --2- 172.21.15.110:0/1857457188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f072c06efa0 0x7f072c1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 >> 172.21.15.110:0/1857457188 conn(0x7f072c062d10 msgr2=0x7f072c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 shutdown_connections 2024-09-10T10:03:43.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.503+0000 7f0732c00640 1 -- 172.21.15.110:0/1857457188 wait complete. 2024-09-10T10:03:43.526 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:03:43.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:43.526 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:43.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:03:43.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:43.576 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:03:43.591 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:03:43.592 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:03:43.592 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:43.604 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:03:43.604 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:43.654 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:03:43.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f5706800640 1 Processor -- start 2024-09-10T10:03:43.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f5706800640 1 -- start start 2024-09-10T10:03:43.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f5706800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f57000cfb90 0x7f570015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f5706800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f57000b3f40 0x7f57000cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f5706800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f570015d4d0 con 0x7f57000cff70 2024-09-10T10:03:43.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f5706800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f570006d130 con 0x7f57000cfb90 2024-09-10T10:03:43.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f5706800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f570015d7b0 con 0x7f57000b3f40 2024-09-10T10:03:43.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56ff400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f57000cfb90 0x7f570015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56ffe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f57000cff70 0x7f5700160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60366/0 (socket says 172.21.15.110:60366) 2024-09-10T10:03:43.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56ffe00640 1 -- 172.21.15.110:0/4080643852 learned_addr learned my addr 172.21.15.110:0/4080643852 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:43.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56fea00640 1 --2- 172.21.15.110:0/4080643852 >> v2:172.21.15.145:3300/0 conn(0x7f57000b3f40 0x7f57000cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56fe000640 1 -- 172.21.15.110:0/4080643852 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093786342 0 0) 0x7f570015d4d0 con 0x7f57000cff70 2024-09-10T10:03:43.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56fe000640 1 -- 172.21.15.110:0/4080643852 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56e4003530 con 0x7f57000cff70 2024-09-10T10:03:43.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56ff400640 1 -- 172.21.15.110:0/4080643852 >> v2:172.21.15.145:3300/0 conn(0x7f57000b3f40 msgr2=0x7f57000cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:43.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56ff400640 1 --2- 172.21.15.110:0/4080643852 >> v2:172.21.15.145:3300/0 conn(0x7f57000b3f40 0x7f57000cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56ff400640 1 -- 172.21.15.110:0/4080643852 >> v1:172.21.15.110:6789/0 conn(0x7f57000cff70 legacy=0x7f5700160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.787+0000 7f56ff400640 1 -- 172.21.15.110:0/4080643852 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570006d380 con 0x7f57000cfb90 2024-09-10T10:03:43.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f56ff400640 1 --2- 172.21.15.110:0/4080643852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f570015ca10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f57000cf000 tx=0x7f56f001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f64ba48fe728c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f56fe000640 1 -- 172.21.15.110:0/4080643852 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00b0070 con 0x7f57000cfb90 2024-09-10T10:03:43.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f56fe000640 1 -- 172.21.15.110:0/4080643852 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f00ab600 con 0x7f57000cfb90 2024-09-10T10:03:43.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f56fe000640 1 -- 172.21.15.110:0/4080643852 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00ab9b0 con 0x7f57000cfb90 2024-09-10T10:03:43.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f5706800640 1 -- 172.21.15.110:0/4080643852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 msgr2=0x7f570015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f5706800640 1 --2- 172.21.15.110:0/4080643852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f570015ca10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f57000cf000 tx=0x7f56f001c3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f5706800640 1 -- 172.21.15.110:0/4080643852 shutdown_connections 2024-09-10T10:03:43.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f5706800640 1 --2- 172.21.15.110:0/4080643852 >> v2:172.21.15.145:3300/0 conn(0x7f57000b3f40 0x7f57000cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f5706800640 1 --2- 172.21.15.110:0/4080643852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f570015ca10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f5706800640 1 -- 172.21.15.110:0/4080643852 >> 172.21.15.110:0/4080643852 conn(0x7f5700062d10 msgr2=0x7f570006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f5706800640 1 -- 172.21.15.110:0/4080643852 shutdown_connections 2024-09-10T10:03:43.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.791+0000 7f5706800640 1 -- 172.21.15.110:0/4080643852 wait complete. 2024-09-10T10:03:43.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 Processor -- start 2024-09-10T10:03:43.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 -- start start 2024-09-10T10:03:43.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57000b3f40 0x7f57001565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f5700143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57000cff70 0x7f5700165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5700169500 con 0x7f57000cfb90 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5700169380 con 0x7f57000cff70 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5700169680 con 0x7f57000b3f40 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f5700143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f5700143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59780/0 (socket says 172.21.15.110:59780) 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 -- 172.21.15.110:0/1140786683 learned_addr learned my addr 172.21.15.110:0/1140786683 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56ff400640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57000b3f40 0x7f57001565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56ffe00640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57000cff70 0x7f5700165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 -- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57000b3f40 msgr2=0x7f57001565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57000b3f40 0x7f57001565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 -- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57000cff70 msgr2=0x7f5700165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57000cff70 0x7f5700165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 -- 172.21.15.110:0/1140786683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5700071150 con 0x7f57000cfb90 2024-09-10T10:03:43.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56ffe00640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57000cff70 0x7f5700165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:43.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56fea00640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f5700143b90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00e900 tx=0x7f56ec00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56f7400640 1 -- 172.21.15.110:0/1140786683 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec017070 con 0x7f57000cfb90 2024-09-10T10:03:43.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56f7400640 1 -- 172.21.15.110:0/1140786683 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec00ca00 con 0x7f57000cfb90 2024-09-10T10:03:43.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f56f7400640 1 -- 172.21.15.110:0/1140786683 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00cdf0 con 0x7f57000cfb90 2024-09-10T10:03:43.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57000713f0 con 0x7f57000cfb90 2024-09-10T10:03:43.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.795+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57000719b0 con 0x7f57000cfb90 2024-09-10T10:03:43.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.799+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57000cd5a0 con 0x7f57000cfb90 2024-09-10T10:03:43.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.799+0000 7f56f7400640 1 -- 172.21.15.110:0/1140786683 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 87700+0+0 (secure 0 0 0) 0x7f56ec01a070 con 0x7f57000cfb90 2024-09-10T10:03:43.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.799+0000 7f56f7400640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56e0067b90 0x7f56e006a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:43.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.799+0000 7f56f7400640 1 -- 172.21.15.110:0/1140786683 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f56ec086050 con 0x7f57000cfb90 2024-09-10T10:03:43.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.799+0000 7f56ff400640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56e0067b90 0x7f56e006a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:43.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.799+0000 7f56ff400640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56e0067b90 0x7f56e006a050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56f00ad000 tx=0x7f56f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:43.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.803+0000 7f56f7400640 1 -- 172.21.15.110:0/1140786683 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f56ec056200 con 0x7f57000cfb90 2024-09-10T10:03:43.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5700166f40 con 0x7f57000cfb90 2024-09-10T10:03:43.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f56f7400640 1 -- 172.21.15.110:0/1140786683 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f56ec05a500 con 0x7f57000cfb90 2024-09-10T10:03:43.978 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:03:43.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56e0067b90 msgr2=0x7f56e006a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56e0067b90 0x7f56e006a050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56f00ad000 tx=0x7f56f0008040 comp rx=0 tx=0).stop 2024-09-10T10:03:43.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 msgr2=0x7f5700143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:43.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f5700143b90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00e900 tx=0x7f56ec00edc0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 shutdown_connections 2024-09-10T10:03:43.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56e0067b90 0x7f56e006a050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57000cff70 0x7f5700165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57000cfb90 0x7f5700143b90 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 --2- 172.21.15.110:0/1140786683 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57000b3f40 0x7f57001565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:43.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 >> 172.21.15.110:0/1140786683 conn(0x7f5700062d10 msgr2=0x7f570006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:43.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 shutdown_connections 2024-09-10T10:03:43.981 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:43.975+0000 7f5706800640 1 -- 172.21.15.110:0/1140786683 wait complete. 2024-09-10T10:03:43.999 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:03:43.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:43.999 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:03:44.027 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:03:44.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:44.027 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:03:44.078 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:03:44.079 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:44.079 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:44.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:03:44.087 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:03:44.088 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:03:44.088 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:44.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:03:44.130 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:03:44.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.267+0000 7ff61d800640 1 Processor -- start 2024-09-10T10:03:44.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.267+0000 7ff61d800640 1 -- start start 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff61d800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff6180ccb60 0x7ff618072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff61d800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff618072670 0x7ff618140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff61d800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6180ccf40 con 0x7ff618140a60 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff61d800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff618072ad0 con 0x7ff6180ccb60 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff61d800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff61815c130 con 0x7ff618072670 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff616a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff6180ccb60 0x7ff618072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff616a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff6180ccb60 0x7ff618072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59790/0 (socket says 172.21.15.110:59790) 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff616a00640 1 -- 172.21.15.110:0/780587340 learned_addr learned my addr 172.21.15.110:0/780587340 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:44.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff617400640 1 --1- 172.21.15.110:0/780587340 >> v1:172.21.15.110:6789/0 conn(0x7ff618140a60 0x7ff6180710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60374/0 (socket says 172.21.15.110:60374) 2024-09-10T10:03:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff616000640 1 --2- 172.21.15.110:0/780587340 >> v2:172.21.15.145:3300/0 conn(0x7ff618072670 0x7ff618140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff616a00640 1 -- 172.21.15.110:0/780587340 >> v2:172.21.15.145:3300/0 conn(0x7ff618072670 msgr2=0x7ff618140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff616a00640 1 --2- 172.21.15.110:0/780587340 >> v2:172.21.15.145:3300/0 conn(0x7ff618072670 0x7ff618140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff616a00640 1 -- 172.21.15.110:0/780587340 >> v1:172.21.15.110:6789/0 conn(0x7ff618140a60 legacy=0x7ff6180710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.271+0000 7ff616a00640 1 -- 172.21.15.110:0/780587340 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6181412a0 con 0x7ff6180ccb60 2024-09-10T10:03:44.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff616a00640 1 --2- 172.21.15.110:0/780587340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6180ccb60 0x7ff618072130 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff6180b59a0 tx=0x7ff60801dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b9565f40c7318f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:44.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff615600640 1 -- 172.21.15.110:0/780587340 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60800a2d0 con 0x7ff6180ccb60 2024-09-10T10:03:44.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff615600640 1 -- 172.21.15.110:0/780587340 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6080a68a0 con 0x7ff6180ccb60 2024-09-10T10:03:44.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff615600640 1 -- 172.21.15.110:0/780587340 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6080a6c70 con 0x7ff6180ccb60 2024-09-10T10:03:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 -- 172.21.15.110:0/780587340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6180ccb60 msgr2=0x7ff618072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 --2- 172.21.15.110:0/780587340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6180ccb60 0x7ff618072130 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff6180b59a0 tx=0x7ff60801dac0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 -- 172.21.15.110:0/780587340 shutdown_connections 2024-09-10T10:03:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 --2- 172.21.15.110:0/780587340 >> v2:172.21.15.145:3300/0 conn(0x7ff618072670 0x7ff618140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 --2- 172.21.15.110:0/780587340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6180ccb60 0x7ff618072130 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 -- 172.21.15.110:0/780587340 >> 172.21.15.110:0/780587340 conn(0x7ff61806bc50 msgr2=0x7ff61806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:44.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 -- 172.21.15.110:0/780587340 shutdown_connections 2024-09-10T10:03:44.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 -- 172.21.15.110:0/780587340 wait complete. 2024-09-10T10:03:44.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 Processor -- start 2024-09-10T10:03:44.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 -- start start 2024-09-10T10:03:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.275+0000 7ff61d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff618072670 0x7ff6180b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff61d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff6180ccb60 0x7ff618156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff61d800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff618140a60 0x7ff6180cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff61d800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff61815f520 con 0x7ff618140a60 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff61d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff61815f000 con 0x7ff618072670 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff61d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff618158190 con 0x7ff6180ccb60 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff617400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff618140a60 0x7ff6180cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff618072670 0x7ff6180b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff6180ccb60 0x7ff618156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff618072670 0x7ff6180b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:43802/0 (socket says 172.21.15.110:43802) 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 -- 172.21.15.110:0/2213740614 learned_addr learned my addr 172.21.15.110:0/2213740614 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 -- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff6180ccb60 msgr2=0x7ff618156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff6180ccb60 0x7ff618156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 -- 172.21.15.110:0/2213740614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff618140a60 msgr2=0x7ff6180cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff618140a60 0x7ff6180cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 -- 172.21.15.110:0/2213740614 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff618157580 con 0x7ff618072670 2024-09-10T10:03:44.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616000640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff6180ccb60 0x7ff618156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff616a00640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff618072670 0x7ff6180b1ae0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff60801a6d0 tx=0x7ff60801a860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff60ea00640 1 -- 172.21.15.110:0/2213740614 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608008700 con 0x7ff618072670 2024-09-10T10:03:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff618157860 con 0x7ff618072670 2024-09-10T10:03:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff618157e20 con 0x7ff618072670 2024-09-10T10:03:44.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.279+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618140e70 con 0x7ff618072670 2024-09-10T10:03:44.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.287+0000 7ff60ea00640 1 -- 172.21.15.110:0/2213740614 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6080bd070 con 0x7ff618072670 2024-09-10T10:03:44.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.287+0000 7ff60ea00640 1 -- 172.21.15.110:0/2213740614 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608008aa0 con 0x7ff618072670 2024-09-10T10:03:44.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.291+0000 7ff60ea00640 1 -- 172.21.15.110:0/2213740614 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7ff608008c40 con 0x7ff618072670 2024-09-10T10:03:44.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.291+0000 7ff60ea00640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5f8067cb0 0x7ff5f806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.291+0000 7ff60ea00640 1 -- 172.21.15.110:0/2213740614 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7ff608121f00 con 0x7ff618072670 2024-09-10T10:03:44.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.291+0000 7ff616000640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5f8067cb0 0x7ff5f806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.291+0000 7ff616000640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5f8067cb0 0x7ff5f806a170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff6180ce7a0 tx=0x7ff604008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:44.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.295+0000 7ff60ea00640 1 -- 172.21.15.110:0/2213740614 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff6080176e0 con 0x7ff618072670 2024-09-10T10:03:44.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.483+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff618156620 con 0x7ff618072670 2024-09-10T10:03:44.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.483+0000 7ff60ea00640 1 -- 172.21.15.110:0/2213740614 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5228 (secure 0 0 0) 0x7ff6080f1f60 con 0x7ff618072670 2024-09-10T10:03:44.488 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:44.488 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":69,"btime":"2024-09-10T10:03:14:185413+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/59288243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":59288243},{"type":"v1","addr":"172.21.15.145:6834","nonce":59288243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2534764627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2534764627},{"type":"v1","addr":"172.21.15.145:6835","nonce":2534764627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6798,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3565143399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":3565143399},{"type":"v1","addr":"172.21.15.110:6834","nonce":3565143399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:04.516174+0000","modified":"2024-09-10T10:03:07.538032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/100964295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":100964295},{"type":"v1","addr":"172.21.15.110:6835","nonce":100964295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":8}]} 2024-09-10T10:03:44.488 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 69 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5f8067cb0 msgr2=0x7ff5f806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5f8067cb0 0x7ff5f806a170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff6180ce7a0 tx=0x7ff604008040 comp rx=0 tx=0).stop 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff618072670 msgr2=0x7ff6180b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff618072670 0x7ff6180b1ae0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff60801a6d0 tx=0x7ff60801a860 comp rx=0 tx=0).stop 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 shutdown_connections 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5f8067cb0 0x7ff5f806a170 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff618140a60 0x7ff6180cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff6180ccb60 0x7ff618156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 --2- 172.21.15.110:0/2213740614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff618072670 0x7ff6180b1ae0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 >> 172.21.15.110:0/2213740614 conn(0x7ff61806bc50 msgr2=0x7ff6180710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:44.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 shutdown_connections 2024-09-10T10:03:44.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.487+0000 7ff61d800640 1 -- 172.21.15.110:0/2213740614 wait complete. 2024-09-10T10:03:44.509 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:03:44.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f3a00640 1 Processor -- start 2024-09-10T10:03:44.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f3a00640 1 -- start start 2024-09-10T10:03:44.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f3a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f75ec06e7a0 0x7f75ec0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f3a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75ec072150 con 0x7f75ec0cd510 2024-09-10T10:03:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f3a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75ec06d2d0 con 0x7f75ec0cfe30 2024-09-10T10:03:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f3a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f75ec15d9a0 con 0x7f75ec06e7a0 2024-09-10T10:03:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75ebe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f75ec0cd510 0x7f75ec06e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60386/0 (socket says 172.21.15.110:60386) 2024-09-10T10:03:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75ebe00640 1 -- 172.21.15.110:0/1481254197 learned_addr learned my addr 172.21.15.110:0/1481254197 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:44.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f1a00640 1 --2- 172.21.15.110:0/1481254197 >> v2:172.21.15.145:3300/0 conn(0x7f75ec06e7a0 0x7f75ec0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75eb400640 1 -- 172.21.15.110:0/1481254197 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3387248306 0 0) 0x7f75ec072150 con 0x7f75ec0cd510 2024-09-10T10:03:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75eb400640 1 -- 172.21.15.110:0/1481254197 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75d4003530 con 0x7f75ec0cd510 2024-09-10T10:03:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f1a00640 1 -- 172.21.15.110:0/1481254197 >> v1:172.21.15.145:6789/0 conn(0x7f75ec0cfe30 legacy=0x7f75ec071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f1a00640 1 -- 172.21.15.110:0/1481254197 >> v1:172.21.15.110:6789/0 conn(0x7f75ec0cd510 legacy=0x7f75ec06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.643+0000 7f75f1a00640 1 -- 172.21.15.110:0/1481254197 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec06d5b0 con 0x7f75ec06e7a0 2024-09-10T10:03:44.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75f1a00640 1 --2- 172.21.15.110:0/1481254197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec06e7a0 0x7f75ec0cd9a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009d90 tx=0x7f75e401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d03e382008bfb0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:44.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75eb400640 1 -- 172.21.15.110:0/1481254197 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e40b2070 con 0x7f75ec06e7a0 2024-09-10T10:03:44.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75eb400640 1 -- 172.21.15.110:0/1481254197 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e40a58b0 con 0x7f75ec06e7a0 2024-09-10T10:03:44.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75eb400640 1 -- 172.21.15.110:0/1481254197 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e40a5c80 con 0x7f75ec06e7a0 2024-09-10T10:03:44.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75f3a00640 1 -- 172.21.15.110:0/1481254197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec06e7a0 msgr2=0x7f75ec0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75f3a00640 1 --2- 172.21.15.110:0/1481254197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec06e7a0 0x7f75ec0cd9a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009d90 tx=0x7f75e401c8e0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75f3a00640 1 -- 172.21.15.110:0/1481254197 shutdown_connections 2024-09-10T10:03:44.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75f3a00640 1 --2- 172.21.15.110:0/1481254197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec06e7a0 0x7f75ec0cd9a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75f3a00640 1 -- 172.21.15.110:0/1481254197 >> 172.21.15.110:0/1481254197 conn(0x7f75ec06bc50 msgr2=0x7f75ec06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:44.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75f3a00640 1 -- 172.21.15.110:0/1481254197 shutdown_connections 2024-09-10T10:03:44.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.647+0000 7f75f3a00640 1 -- 172.21.15.110:0/1481254197 wait complete. 2024-09-10T10:03:44.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 Processor -- start 2024-09-10T10:03:44.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 -- start start 2024-09-10T10:03:44.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75ec06e7a0 0x7f75ec14eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75ec0cd510 0x7f75ec14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec0cfe30 0x7f75ec1482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec16a190 con 0x7f75ec06e7a0 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec15d700 con 0x7f75ec0cfe30 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec16a310 con 0x7f75ec0cd510 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75ec06e7a0 0x7f75ec14eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75ec06e7a0 0x7f75ec14eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59808/0 (socket says 172.21.15.110:59808) 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 -- 172.21.15.110:0/522479094 learned_addr learned my addr 172.21.15.110:0/522479094 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1a00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec0cfe30 0x7f75ec1482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75ebe00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75ec0cd510 0x7f75ec14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 -- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75ec0cd510 msgr2=0x7f75ec14f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75ec0cd510 0x7f75ec14f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 -- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec0cfe30 msgr2=0x7f75ec1482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec0cfe30 0x7f75ec1482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 -- 172.21.15.110:0/522479094 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec0b2fd0 con 0x7f75ec06e7a0 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1a00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec0cfe30 0x7f75ec1482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75ebe00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75ec0cd510 0x7f75ec14f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f1000640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75ec06e7a0 0x7f75ec14eeb0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c3c0 tx=0x7f75e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75e8c00640 1 -- 172.21.15.110:0/522479094 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0017070 con 0x7f75ec06e7a0 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75e8c00640 1 -- 172.21.15.110:0/522479094 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e00052d0 con 0x7f75ec06e7a0 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec156680 con 0x7f75ec06e7a0 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75e8c00640 1 -- 172.21.15.110:0/522479094 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0003840 con 0x7f75ec06e7a0 2024-09-10T10:03:44.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec154e60 con 0x7f75ec06e7a0 2024-09-10T10:03:44.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.651+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec06e090 con 0x7f75ec06e7a0 2024-09-10T10:03:44.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.655+0000 7f75e8c00640 1 -- 172.21.15.110:0/522479094 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f75e001a070 con 0x7f75ec06e7a0 2024-09-10T10:03:44.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.655+0000 7f75e8c00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f75cc067cb0 0x7f75cc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:44.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.655+0000 7f75e8c00640 1 -- 172.21.15.110:0/522479094 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f75e0085780 con 0x7f75ec06e7a0 2024-09-10T10:03:44.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.655+0000 7f75ebe00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f75cc067cb0 0x7f75cc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:44.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.655+0000 7f75ebe00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f75cc067cb0 0x7f75cc06a170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75ec071e30 tx=0x7f75d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:44.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.659+0000 7f75e8c00640 1 -- 172.21.15.110:0/522479094 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f75e0005480 con 0x7f75ec06e7a0 2024-09-10T10:03:44.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.847+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75ec0ccf20 con 0x7f75ec06e7a0 2024-09-10T10:03:44.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.847+0000 7f75e8c00640 1 -- 172.21.15.110:0/522479094 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5228 (secure 0 0 0) 0x7f75e0055cc0 con 0x7f75ec06e7a0 2024-09-10T10:03:44.851 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:44.851 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":69,"btime":"2024-09-10T10:03:14:185413+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/59288243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":59288243},{"type":"v1","addr":"172.21.15.145:6834","nonce":59288243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2534764627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2534764627},{"type":"v1","addr":"172.21.15.145:6835","nonce":2534764627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6798,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3565143399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":3565143399},{"type":"v1","addr":"172.21.15.110:6834","nonce":3565143399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:04.516174+0000","modified":"2024-09-10T10:03:07.538032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/100964295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":100964295},{"type":"v1","addr":"172.21.15.110:6835","nonce":100964295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":8}]} 2024-09-10T10:03:44.852 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 69 2024-09-10T10:03:44.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f75cc067cb0 msgr2=0x7f75cc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f75cc067cb0 0x7f75cc06a170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75ec071e30 tx=0x7f75d8009040 comp rx=0 tx=0).stop 2024-09-10T10:03:44.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75ec06e7a0 msgr2=0x7f75ec14eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75ec06e7a0 0x7f75ec14eeb0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c3c0 tx=0x7f75e000c880 comp rx=0 tx=0).stop 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 shutdown_connections 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f75cc067cb0 0x7f75cc06a170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75ec0cfe30 0x7f75ec1482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75ec0cd510 0x7f75ec14f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 --2- 172.21.15.110:0/522479094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75ec06e7a0 0x7f75ec14eeb0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 >> 172.21.15.110:0/522479094 conn(0x7f75ec06bc50 msgr2=0x7f75ec06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 shutdown_connections 2024-09-10T10:03:44.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:44.851+0000 7f75f3a00640 1 -- 172.21.15.110:0/522479094 wait complete. 2024-09-10T10:03:44.873 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:03:45.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.007+0000 7f45ed600640 1 Processor -- start 2024-09-10T10:03:45.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.007+0000 7f45ed600640 1 -- start start 2024-09-10T10:03:45.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45ed600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f45e8072840 0x7f45e8140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45ed600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f45e8140a60 0x7f45e815b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45ed600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45e80ccf40 con 0x7f45e80ccb60 2024-09-10T10:03:45.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45ed600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f45e81412a0 con 0x7f45e8072840 2024-09-10T10:03:45.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45ed600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f45e815e760 con 0x7f45e8140a60 2024-09-10T10:03:45.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45dfe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f45e8072840 0x7f45e8140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45dfe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f45e8072840 0x7f45e8140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59814/0 (socket says 172.21.15.110:59814) 2024-09-10T10:03:45.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45dfe00640 1 -- 172.21.15.110:0/1634755855 learned_addr learned my addr 172.21.15.110:0/1634755855 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:45.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45e7400640 1 --2- 172.21.15.110:0/1634755855 >> v2:172.21.15.145:3300/0 conn(0x7f45e8140a60 0x7f45e815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45dfe00640 1 -- 172.21.15.110:0/1634755855 >> v2:172.21.15.145:3300/0 conn(0x7f45e8140a60 msgr2=0x7f45e815b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:45.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45dfe00640 1 --2- 172.21.15.110:0/1634755855 >> v2:172.21.15.145:3300/0 conn(0x7f45e8140a60 0x7f45e815b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45dfe00640 1 -- 172.21.15.110:0/1634755855 >> v1:172.21.15.145:6789/0 conn(0x7f45e80ccb60 legacy=0x7f45e8072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.011+0000 7f45dfe00640 1 -- 172.21.15.110:0/1634755855 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e815ea40 con 0x7f45e8072840 2024-09-10T10:03:45.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45dfe00640 1 --2- 172.21.15.110:0/1634755855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8072840 0x7f45e8140520 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f45e806f4b0 tx=0x7f45d001d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aebe6a8bf39e797 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45e6000640 1 -- 172.21.15.110:0/1634755855 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d00b3070 con 0x7f45e8072840 2024-09-10T10:03:45.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45e6000640 1 -- 172.21.15.110:0/1634755855 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d00a6970 con 0x7f45e8072840 2024-09-10T10:03:45.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45e6000640 1 -- 172.21.15.110:0/1634755855 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d000b040 con 0x7f45e8072840 2024-09-10T10:03:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 -- 172.21.15.110:0/1634755855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8072840 msgr2=0x7f45e8140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 --2- 172.21.15.110:0/1634755855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8072840 0x7f45e8140520 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f45e806f4b0 tx=0x7f45d001d8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 -- 172.21.15.110:0/1634755855 shutdown_connections 2024-09-10T10:03:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 --2- 172.21.15.110:0/1634755855 >> v2:172.21.15.145:3300/0 conn(0x7f45e8140a60 0x7f45e815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 --2- 172.21.15.110:0/1634755855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e8072840 0x7f45e8140520 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 -- 172.21.15.110:0/1634755855 >> 172.21.15.110:0/1634755855 conn(0x7f45e806bc50 msgr2=0x7f45e806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 -- 172.21.15.110:0/1634755855 shutdown_connections 2024-09-10T10:03:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 -- 172.21.15.110:0/1634755855 wait complete. 2024-09-10T10:03:45.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 Processor -- start 2024-09-10T10:03:45.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.015+0000 7f45ed600640 1 -- start start 2024-09-10T10:03:45.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45e8072840 0x7f45e814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e80ccb60 0x7f45e81fabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45e8140a60 0x7f45e80ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8140ff0 con 0x7f45e80ccb60 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8140e70 con 0x7f45e8072840 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f45e815b660 con 0x7f45e8140a60 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e80ccb60 0x7f45e81fabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e80ccb60 0x7f45e81fabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59820/0 (socket says 172.21.15.110:59820) 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 -- 172.21.15.110:0/34963994 learned_addr learned my addr 172.21.15.110:0/34963994 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45e6a00640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45e8072840 0x7f45e814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45e7400640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45e8140a60 0x7f45e80ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 -- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45e8140a60 msgr2=0x7f45e80ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45e8140a60 0x7f45e80ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 -- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45e8072840 msgr2=0x7f45e814f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45e8072840 0x7f45e814f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 -- 172.21.15.110:0/34963994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e814f620 con 0x7f45e80ccb60 2024-09-10T10:03:45.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dfe00640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e80ccb60 0x7f45e81fabe0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f45d000a1c0 tx=0x7f45d001c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dea00640 1 -- 172.21.15.110:0/34963994 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d00b3070 con 0x7f45e80ccb60 2024-09-10T10:03:45.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dea00640 1 -- 172.21.15.110:0/34963994 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d00b5070 con 0x7f45e80ccb60 2024-09-10T10:03:45.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e81522d0 con 0x7f45e80ccb60 2024-09-10T10:03:45.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dea00640 1 -- 172.21.15.110:0/34963994 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d00aeb70 con 0x7f45e80ccb60 2024-09-10T10:03:45.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e81517f0 con 0x7f45e80ccb60 2024-09-10T10:03:45.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e8140520 con 0x7f45e80ccb60 2024-09-10T10:03:45.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.019+0000 7f45dea00640 1 -- 172.21.15.110:0/34963994 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f45d00aed10 con 0x7f45e80ccb60 2024-09-10T10:03:45.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.023+0000 7f45dea00640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f45c4067ce0 0x7f45c406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.023+0000 7f45dea00640 1 -- 172.21.15.110:0/34963994 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f45d0120520 con 0x7f45e80ccb60 2024-09-10T10:03:45.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.023+0000 7f45e6a00640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f45c4067ce0 0x7f45c406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.023+0000 7f45e6a00640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f45c4067ce0 0x7f45c406a1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000f70 tx=0x7f45d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.027+0000 7f45dea00640 1 -- 172.21.15.110:0/34963994 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f45d00174e0 con 0x7f45e80ccb60 2024-09-10T10:03:45.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.191+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45e8158970 con 0x7f45e80ccb60 2024-09-10T10:03:45.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.191+0000 7f45dea00640 1 -- 172.21.15.110:0/34963994 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16876 (secure 0 0 0) 0x7f45d00f0a10 con 0x7f45e80ccb60 2024-09-10T10:03:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:45.196 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":105,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:17.198463+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:02.221093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:03.519991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:45.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f45c4067ce0 msgr2=0x7f45c406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f45c4067ce0 0x7f45c406a1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000f70 tx=0x7f45d8009040 comp rx=0 tx=0).stop 2024-09-10T10:03:45.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e80ccb60 msgr2=0x7f45e81fabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e80ccb60 0x7f45e81fabe0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f45d000a1c0 tx=0x7f45d001c100 comp rx=0 tx=0).stop 2024-09-10T10:03:45.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 shutdown_connections 2024-09-10T10:03:45.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f45c4067ce0 0x7f45c406a1a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f45e8140a60 0x7f45e80ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e80ccb60 0x7f45e81fabe0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 --2- 172.21.15.110:0/34963994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45e8072840 0x7f45e814f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 >> 172.21.15.110:0/34963994 conn(0x7f45e806bc50 msgr2=0x7f45e814e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 shutdown_connections 2024-09-10T10:03:45.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.195+0000 7f45ed600640 1 -- 172.21.15.110:0/34963994 wait complete. 2024-09-10T10:03:45.218 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:03:45.218 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:03:45.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.355+0000 7f00e4c00640 1 Processor -- start 2024-09-10T10:03:45.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.355+0000 7f00e4c00640 1 -- start start 2024-09-10T10:03:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00e4c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f00e00cfe30 0x7f00e0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00e4c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e006d2d0 con 0x7f00e006e7a0 2024-09-10T10:03:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00e4c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e006d5b0 con 0x7f00e00cd510 2024-09-10T10:03:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00e4c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f00e015d9a0 con 0x7f00e00cfe30 2024-09-10T10:03:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00dde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f00e00cfe30 0x7f00e0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00dde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f00e00cfe30 0x7f00e0071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59830/0 (socket says 172.21.15.110:59830) 2024-09-10T10:03:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00dde00640 1 -- 172.21.15.110:0/4028282293 learned_addr learned my addr 172.21.15.110:0/4028282293 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:45.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00dde00640 1 -- 172.21.15.110:0/4028282293 >> v1:172.21.15.145:6789/0 conn(0x7f00e00cd510 legacy=0x7f00e006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00dde00640 1 -- 172.21.15.110:0/4028282293 >> v1:172.21.15.110:6789/0 conn(0x7f00e006e7a0 legacy=0x7f00e015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.359+0000 7f00dde00640 1 -- 172.21.15.110:0/4028282293 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e006d890 con 0x7f00e00cfe30 2024-09-10T10:03:45.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00dde00640 1 --2- 172.21.15.110:0/4028282293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0071a40 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009870 tx=0x7f00d001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0a4a363cf6cc87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00dd400640 1 -- 172.21.15.110:0/4028282293 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d00b1070 con 0x7f00e00cfe30 2024-09-10T10:03:45.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00dd400640 1 -- 172.21.15.110:0/4028282293 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d00a8050 con 0x7f00e00cfe30 2024-09-10T10:03:45.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00dd400640 1 -- 172.21.15.110:0/4028282293 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d00ac860 con 0x7f00e00cfe30 2024-09-10T10:03:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00e4c00640 1 -- 172.21.15.110:0/4028282293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 msgr2=0x7f00e0071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00e4c00640 1 --2- 172.21.15.110:0/4028282293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0071a40 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009870 tx=0x7f00d001c4d0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00e4c00640 1 -- 172.21.15.110:0/4028282293 shutdown_connections 2024-09-10T10:03:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00e4c00640 1 --2- 172.21.15.110:0/4028282293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0071a40 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00e4c00640 1 -- 172.21.15.110:0/4028282293 >> 172.21.15.110:0/4028282293 conn(0x7f00e006bc50 msgr2=0x7f00e006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00e4c00640 1 -- 172.21.15.110:0/4028282293 shutdown_connections 2024-09-10T10:03:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.363+0000 7f00e4c00640 1 -- 172.21.15.110:0/4028282293 wait complete. 2024-09-10T10:03:45.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 Processor -- start 2024-09-10T10:03:45.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 -- start start 2024-09-10T10:03:45.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00e006e7a0 0x7f00e0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00e00cd510 0x7f00e0159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00e015d740 con 0x7f00e00cfe30 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0072190 con 0x7f00e00cd510 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f00e016abb0 con 0x7f00e006e7a0 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59840/0 (socket says 172.21.15.110:59840) 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 -- 172.21.15.110:0/3447946039 learned_addr learned my addr 172.21.15.110:0/3447946039 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00dde00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00e006e7a0 0x7f00e0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00d7e00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00e00cd510 0x7f00e0159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 -- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00e006e7a0 msgr2=0x7f00e0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00e006e7a0 0x7f00e0158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 -- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00e00cd510 msgr2=0x7f00e0159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00e00cd510 0x7f00e0159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 -- 172.21.15.110:0/3447946039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e0154760 con 0x7f00e00cfe30 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00de800640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0157050 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f00d800c760 tx=0x7f00d800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00d6000640 1 -- 172.21.15.110:0/3447946039 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8003b80 con 0x7f00e00cfe30 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00d6000640 1 -- 172.21.15.110:0/3447946039 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d8003d20 con 0x7f00e00cfe30 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00d6000640 1 -- 172.21.15.110:0/3447946039 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d800aa30 con 0x7f00e00cfe30 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e0154a40 con 0x7f00e00cfe30 2024-09-10T10:03:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e0155000 con 0x7f00e00cfe30 2024-09-10T10:03:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.367+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e00afef0 con 0x7f00e00cfe30 2024-09-10T10:03:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.371+0000 7f00d6000640 1 -- 172.21.15.110:0/3447946039 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f00d800abd0 con 0x7f00e00cfe30 2024-09-10T10:03:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.371+0000 7f00d6000640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00bc067cb0 0x7f00bc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.371+0000 7f00d6000640 1 -- 172.21.15.110:0/3447946039 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5823+0+0 (secure 0 0 0) 0x7f00d80862d0 con 0x7f00e00cfe30 2024-09-10T10:03:45.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.371+0000 7f00dde00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00bc067cb0 0x7f00bc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.371+0000 7f00dde00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00bc067cb0 0x7f00bc06a170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f00d00ae000 tx=0x7f00d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.375+0000 7f00d6000640 1 -- 172.21.15.110:0/3447946039 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f00e00afef0 con 0x7f00e00cfe30 2024-09-10T10:03:45.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.563+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00e0156b80 con 0x7f00e00cfe30 2024-09-10T10:03:45.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.563+0000 7f00d6000640 1 -- 172.21.15.110:0/3447946039 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5228 (secure 0 0 0) 0x7f00d8056780 con 0x7f00e00cfe30 2024-09-10T10:03:45.568 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:45.568 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":69,"btime":"2024-09-10T10:03:14:185413+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/59288243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":59288243},{"type":"v1","addr":"172.21.15.145:6834","nonce":59288243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2534764627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2534764627},{"type":"v1","addr":"172.21.15.145:6835","nonce":2534764627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6798,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3565143399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":3565143399},{"type":"v1","addr":"172.21.15.110:6834","nonce":3565143399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:04.516174+0000","modified":"2024-09-10T10:03:07.538032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/100964295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":100964295},{"type":"v1","addr":"172.21.15.110:6835","nonce":100964295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":8}]} 2024-09-10T10:03:45.568 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 69 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00bc067cb0 msgr2=0x7f00bc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00bc067cb0 0x7f00bc06a170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f00d00ae000 tx=0x7f00d0008040 comp rx=0 tx=0).stop 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 msgr2=0x7f00e0157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0157050 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f00d800c760 tx=0x7f00d800cc20 comp rx=0 tx=0).stop 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 shutdown_connections 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00bc067cb0 0x7f00bc06a170 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00e00cfe30 0x7f00e0157050 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00e00cd510 0x7f00e0159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 --2- 172.21.15.110:0/3447946039 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00e006e7a0 0x7f00e0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 >> 172.21.15.110:0/3447946039 conn(0x7f00e006bc50 msgr2=0x7f00e015ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 shutdown_connections 2024-09-10T10:03:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.567+0000 7f00e4c00640 1 -- 172.21.15.110:0/3447946039 wait complete. 2024-09-10T10:03:45.588 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:03:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1151200640 1 Processor -- start 2024-09-10T10:03:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1151200640 1 -- start start 2024-09-10T10:03:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1151200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0b1bc0 0x7f114c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1151200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114c15ce90 con 0x7f114c0b4a10 2024-09-10T10:03:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1151200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114c15d170 con 0x7f114c0d0340 2024-09-10T10:03:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1151200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f114c15fef0 con 0x7f114c0b1bc0 2024-09-10T10:03:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f114a800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f114c0b4a10 0x7f114c160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60402/0 (socket says 172.21.15.110:60402) 2024-09-10T10:03:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f114a800640 1 -- 172.21.15.110:0/74219569 learned_addr learned my addr 172.21.15.110:0/74219569 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1149e00640 1 --2- 172.21.15.110:0/74219569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0b1bc0 0x7f114c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1148a00640 1 -- 172.21.15.110:0/74219569 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2720196464 0 0) 0x7f114c15ce90 con 0x7f114c0b4a10 2024-09-10T10:03:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1148a00640 1 -- 172.21.15.110:0/74219569 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1130003530 con 0x7f114c0b4a10 2024-09-10T10:03:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1148a00640 1 -- 172.21.15.110:0/74219569 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2391545092 0 0) 0x7f114c15d170 con 0x7f114c0d0340 2024-09-10T10:03:45.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1149e00640 1 -- 172.21.15.110:0/74219569 >> v1:172.21.15.145:6789/0 conn(0x7f114c0d0340 legacy=0x7f114c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1149e00640 1 -- 172.21.15.110:0/74219569 >> v1:172.21.15.110:6789/0 conn(0x7f114c0b4a10 legacy=0x7f114c160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.727+0000 7f1149e00640 1 -- 172.21.15.110:0/74219569 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c15d450 con 0x7f114c0b1bc0 2024-09-10T10:03:45.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1149e00640 1 --2- 172.21.15.110:0/74219569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0b1bc0 0x7f114c0d1db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f113c0097f0 tx=0x7f113c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6df014e7a2454f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1148a00640 1 -- 172.21.15.110:0/74219569 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0b0070 con 0x7f114c0b1bc0 2024-09-10T10:03:45.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1148a00640 1 -- 172.21.15.110:0/74219569 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c0ab600 con 0x7f114c0b1bc0 2024-09-10T10:03:45.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1148a00640 1 -- 172.21.15.110:0/74219569 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0ab9b0 con 0x7f114c0b1bc0 2024-09-10T10:03:45.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1151200640 1 -- 172.21.15.110:0/74219569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0b1bc0 msgr2=0x7f114c0d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1151200640 1 --2- 172.21.15.110:0/74219569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0b1bc0 0x7f114c0d1db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f113c0097f0 tx=0x7f113c01c420 comp rx=0 tx=0).stop 2024-09-10T10:03:45.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1151200640 1 -- 172.21.15.110:0/74219569 shutdown_connections 2024-09-10T10:03:45.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1151200640 1 --2- 172.21.15.110:0/74219569 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0b1bc0 0x7f114c0d1db0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1151200640 1 -- 172.21.15.110:0/74219569 >> 172.21.15.110:0/74219569 conn(0x7f114c06f420 msgr2=0x7f114c06f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1151200640 1 -- 172.21.15.110:0/74219569 shutdown_connections 2024-09-10T10:03:45.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.731+0000 7f1151200640 1 -- 172.21.15.110:0/74219569 wait complete. 2024-09-10T10:03:45.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 Processor -- start 2024-09-10T10:03:45.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 -- start start 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c0b1bc0 0x7f114c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114c0b4a10 0x7f114c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0d0340 0x7f114c166160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f114c15ca70 con 0x7f114c0b4a10 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f114c0d0b30 con 0x7f114c0b1bc0 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f114c15cbf0 con 0x7f114c0d0340 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114c0b4a10 0x7f114c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114c0b4a10 0x7f114c0b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59850/0 (socket says 172.21.15.110:59850) 2024-09-10T10:03:45.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 -- 172.21.15.110:0/1174616877 learned_addr learned my addr 172.21.15.110:0/1174616877 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f114a800640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0d0340 0x7f114c166160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149e00640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c0b1bc0 0x7f114c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 -- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0d0340 msgr2=0x7f114c166160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0d0340 0x7f114c166160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 -- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c0b1bc0 msgr2=0x7f114c0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c0b1bc0 0x7f114c0adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 -- 172.21.15.110:0/1174616877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c0b13d0 con 0x7f114c0b4a10 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1149400640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114c0b4a10 0x7f114c0b0430 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f113800c4d0 tx=0x7f113800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1142000640 1 -- 172.21.15.110:0/1174616877 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138017070 con 0x7f114c0b4a10 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1142000640 1 -- 172.21.15.110:0/1174616877 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1138005480 con 0x7f114c0b4a10 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1142000640 1 -- 172.21.15.110:0/1174616877 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138003840 con 0x7f114c0b4a10 2024-09-10T10:03:45.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c147ce0 con 0x7f114c0b4a10 2024-09-10T10:03:45.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.735+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c0b10f0 con 0x7f114c0b4a10 2024-09-10T10:03:45.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.739+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114c0d1db0 con 0x7f114c0b4a10 2024-09-10T10:03:45.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.739+0000 7f1142000640 1 -- 172.21.15.110:0/1174616877 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f113801a070 con 0x7f114c0b4a10 2024-09-10T10:03:45.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.739+0000 7f1142000640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f112c067ce0 0x7f112c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:45.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.739+0000 7f1142000640 1 -- 172.21.15.110:0/1174616877 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5823+0+0 (secure 0 0 0) 0x7f1138085870 con 0x7f114c0b4a10 2024-09-10T10:03:45.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.739+0000 7f1149e00640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f112c067ce0 0x7f112c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:45.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.739+0000 7f1149e00640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f112c067ce0 0x7f112c06a1a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f113c0ad000 tx=0x7f113c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:45.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.743+0000 7f1142000640 1 -- 172.21.15.110:0/1174616877 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1138055df0 con 0x7f114c0b4a10 2024-09-10T10:03:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.931+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f114c06ea50 con 0x7f114c0b4a10 2024-09-10T10:03:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1142000640 1 -- 172.21.15.110:0/1174616877 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5228 (secure 0 0 0) 0x7f11380556f0 con 0x7f114c0b4a10 2024-09-10T10:03:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:45.937 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":69,"btime":"2024-09-10T10:03:14:185413+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/59288243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":59288243},{"type":"v1","addr":"172.21.15.145:6834","nonce":59288243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2534764627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2534764627},{"type":"v1","addr":"172.21.15.145:6835","nonce":2534764627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6798,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3565143399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":3565143399},{"type":"v1","addr":"172.21.15.110:6834","nonce":3565143399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:04.516174+0000","modified":"2024-09-10T10:03:07.538032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/100964295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":100964295},{"type":"v1","addr":"172.21.15.110:6835","nonce":100964295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":8}]} 2024-09-10T10:03:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 69 2024-09-10T10:03:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f112c067ce0 msgr2=0x7f112c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f112c067ce0 0x7f112c06a1a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f113c0ad000 tx=0x7f113c008040 comp rx=0 tx=0).stop 2024-09-10T10:03:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114c0b4a10 msgr2=0x7f114c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114c0b4a10 0x7f114c0b0430 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f113800c4d0 tx=0x7f113800c990 comp rx=0 tx=0).stop 2024-09-10T10:03:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 shutdown_connections 2024-09-10T10:03:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f112c067ce0 0x7f112c06a1a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f114c0d0340 0x7f114c166160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f114c0b4a10 0x7f114c0b0430 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 --2- 172.21.15.110:0/1174616877 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c0b1bc0 0x7f114c0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:45.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 >> 172.21.15.110:0/1174616877 conn(0x7f114c06f420 msgr2=0x7f114c0d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:45.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 shutdown_connections 2024-09-10T10:03:45.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:45.935+0000 7f1151200640 1 -- 172.21.15.110:0/1174616877 wait complete. 2024-09-10T10:03:45.957 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:03:46.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a3c00640 1 Processor -- start 2024-09-10T10:03:46.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a3c00640 1 -- start start 2024-09-10T10:03:46.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a3c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa39c15c960 0x7fa39c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a3c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39c0706a0 con 0x7fa39c0d13e0 2024-09-10T10:03:46.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a3c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39c070a80 con 0x7fa39c1603c0 2024-09-10T10:03:46.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a3c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa39c15cd40 con 0x7fa39c15c960 2024-09-10T10:03:46.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a1200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa39c15c960 0x7fa39c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a1200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa39c15c960 0x7fa39c0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59866/0 (socket says 172.21.15.110:59866) 2024-09-10T10:03:46.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a1200640 1 -- 172.21.15.110:0/4161601586 learned_addr learned my addr 172.21.15.110:0/4161601586 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:46.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a1200640 1 -- 172.21.15.110:0/4161601586 >> v1:172.21.15.145:6789/0 conn(0x7fa39c1603c0 legacy=0x7fa39c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:46.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a1200640 1 -- 172.21.15.110:0/4161601586 >> v1:172.21.15.110:6789/0 conn(0x7fa39c0d13e0 legacy=0x7fa39c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.091+0000 7fa3a1200640 1 -- 172.21.15.110:0/4161601586 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c160910 con 0x7fa39c15c960 2024-09-10T10:03:46.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa3a1200640 1 --2- 172.21.15.110:0/4161601586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c0b5e30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009e50 tx=0x7fa38c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed1914745ef5c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa39b400640 1 -- 172.21.15.110:0/4161601586 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0a5680 con 0x7fa39c15c960 2024-09-10T10:03:46.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa39b400640 1 -- 172.21.15.110:0/4161601586 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38c0a5820 con 0x7fa39c15c960 2024-09-10T10:03:46.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa39b400640 1 -- 172.21.15.110:0/4161601586 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0a5bf0 con 0x7fa39c15c960 2024-09-10T10:03:46.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa3a3c00640 1 -- 172.21.15.110:0/4161601586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 msgr2=0x7fa39c0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa3a3c00640 1 --2- 172.21.15.110:0/4161601586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c0b5e30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009e50 tx=0x7fa38c01c8d0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa3a3c00640 1 -- 172.21.15.110:0/4161601586 shutdown_connections 2024-09-10T10:03:46.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa3a3c00640 1 --2- 172.21.15.110:0/4161601586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c0b5e30 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa3a3c00640 1 -- 172.21.15.110:0/4161601586 >> 172.21.15.110:0/4161601586 conn(0x7fa39c0d25c0 msgr2=0x7fa39c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:46.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa3a3c00640 1 -- 172.21.15.110:0/4161601586 shutdown_connections 2024-09-10T10:03:46.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.095+0000 7fa3a3c00640 1 -- 172.21.15.110:0/4161601586 wait complete. 2024-09-10T10:03:46.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 Processor -- start 2024-09-10T10:03:46.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 -- start start 2024-09-10T10:03:46.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa39c0d13e0 0x7fa39c1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa39c1603c0 0x7fa39c15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c0d32d0 con 0x7fa39c15c960 2024-09-10T10:03:46.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c0d3150 con 0x7fa39c1603c0 2024-09-10T10:03:46.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa39c071a80 con 0x7fa39c0d13e0 2024-09-10T10:03:46.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59872/0 (socket says 172.21.15.110:59872) 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 -- 172.21.15.110:0/1256321431 learned_addr learned my addr 172.21.15.110:0/1256321431 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a1200640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa39c0d13e0 0x7fa39c1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a1c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa39c1603c0 0x7fa39c15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 -- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa39c0d13e0 msgr2=0x7fa39c1542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa39c0d13e0 0x7fa39c1542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 -- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa39c1603c0 msgr2=0x7fa39c15b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa39c1603c0 0x7fa39c15b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 -- 172.21.15.110:0/1256321431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c153750 con 0x7fa39c15c960 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a1c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa39c1603c0 0x7fa39c15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa39be00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c153210 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa38800c410 tx=0x7fa38800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa398c00640 1 -- 172.21.15.110:0/1256321431 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388011790 con 0x7fa39c15c960 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa398c00640 1 -- 172.21.15.110:0/1256321431 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa388017070 con 0x7fa39c15c960 2024-09-10T10:03:46.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa398c00640 1 -- 172.21.15.110:0/1256321431 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388011bb0 con 0x7fa39c15c960 2024-09-10T10:03:46.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39c15cff0 con 0x7fa39c15c960 2024-09-10T10:03:46.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39c15ed40 con 0x7fa39c15c960 2024-09-10T10:03:46.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.099+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39c15ef40 con 0x7fa39c15c960 2024-09-10T10:03:46.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.103+0000 7fa398c00640 1 -- 172.21.15.110:0/1256321431 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7fa38801c460 con 0x7fa39c15c960 2024-09-10T10:03:46.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.103+0000 7fa398c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa37c067cb0 0x7fa37c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.103+0000 7fa398c00640 1 -- 172.21.15.110:0/1256321431 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5823+0+0 (secure 0 0 0) 0x7fa388085fa0 con 0x7fa39c15c960 2024-09-10T10:03:46.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.103+0000 7fa3a1200640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa37c067cb0 0x7fa37c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.103+0000 7fa3a1200640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa37c067cb0 0x7fa37c06a170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0ae000 tx=0x7fa38c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.107+0000 7fa398c00640 1 -- 172.21.15.110:0/1256321431 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa388021020 con 0x7fa39c15c960 2024-09-10T10:03:46.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.271+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa39c0b5e30 con 0x7fa39c15c960 2024-09-10T10:03:46.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.271+0000 7fa398c00640 1 -- 172.21.15.110:0/1256321431 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16987 (secure 0 0 0) 0x7fa388056520 con 0x7fa39c15c960 2024-09-10T10:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":106,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:45.283383+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:02.221093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:03.519991+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa37c067cb0 msgr2=0x7fa37c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa37c067cb0 0x7fa37c06a170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0ae000 tx=0x7fa38c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 msgr2=0x7fa39c153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c153210 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa38800c410 tx=0x7fa38800c8d0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 shutdown_connections 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa37c067cb0 0x7fa37c06a170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa39c1603c0 0x7fa39c15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa39c15c960 0x7fa39c153210 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 --2- 172.21.15.110:0/1256321431 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa39c0d13e0 0x7fa39c1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 >> 172.21.15.110:0/1256321431 conn(0x7fa39c0d25c0 msgr2=0x7fa39c0b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:46.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 shutdown_connections 2024-09-10T10:03:46.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.275+0000 7fa3a3c00640 1 -- 172.21.15.110:0/1256321431 wait complete. 2024-09-10T10:03:46.298 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:03:46.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.431+0000 7f6f93a00640 1 Processor -- start 2024-09-10T10:03:46.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.431+0000 7f6f93a00640 1 -- start start 2024-09-10T10:03:46.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f93a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c06bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f93a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6f8c0aecd0 0x7f6f8c06c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f93a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c0af220 con 0x7f6f8c1480c0 2024-09-10T10:03:46.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f93a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8c0af500 con 0x7f6f8c0aecd0 2024-09-10T10:03:46.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f93a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c162480 con 0x7f6f8c0ae840 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f91000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6f8c1480c0 0x7f6f8c162c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60412/0 (socket says 172.21.15.110:60412) 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f91000640 1 -- 172.21.15.110:0/713481413 learned_addr learned my addr 172.21.15.110:0/713481413 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f91a00640 1 --2- 172.21.15.110:0/713481413 >> v2:172.21.15.110:3300/0 conn(0x7f6f8c0aecd0 0x7f6f8c06c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f8be00640 1 --2- 172.21.15.110:0/713481413 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c06bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f91a00640 1 -- 172.21.15.110:0/713481413 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 msgr2=0x7f6f8c06bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f91a00640 1 --2- 172.21.15.110:0/713481413 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c06bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f91a00640 1 -- 172.21.15.110:0/713481413 >> v1:172.21.15.110:6789/0 conn(0x7f6f8c1480c0 legacy=0x7f6f8c162c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f91a00640 1 -- 172.21.15.110:0/713481413 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c162760 con 0x7f6f8c0aecd0 2024-09-10T10:03:46.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.435+0000 7f6f8b400640 1 -- 172.21.15.110:0/713481413 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3881200147 0 0) 0x7f6f8c0af220 con 0x7f6f8c1480c0 2024-09-10T10:03:46.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f91a00640 1 --2- 172.21.15.110:0/713481413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0aecd0 0x7f6f8c06c190 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009d90 tx=0x7f6f8401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bd9a3d8cbf6cad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f8b400640 1 -- 172.21.15.110:0/713481413 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840b2070 con 0x7f6f8c0aecd0 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f8b400640 1 -- 172.21.15.110:0/713481413 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f840a5940 con 0x7f6f8c0aecd0 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f8b400640 1 -- 172.21.15.110:0/713481413 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840a5d10 con 0x7f6f8c0aecd0 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- 172.21.15.110:0/713481413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0aecd0 msgr2=0x7f6f8c06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 --2- 172.21.15.110:0/713481413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0aecd0 0x7f6f8c06c190 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009d90 tx=0x7f6f8401c970 comp rx=0 tx=0).stop 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- 172.21.15.110:0/713481413 shutdown_connections 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 --2- 172.21.15.110:0/713481413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c0aecd0 0x7f6f8c06c190 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 --2- 172.21.15.110:0/713481413 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c06bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- 172.21.15.110:0/713481413 >> 172.21.15.110:0/713481413 conn(0x7f6f8c06cd60 msgr2=0x7f6f8c06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- 172.21.15.110:0/713481413 shutdown_connections 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- 172.21.15.110:0/713481413 wait complete. 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 Processor -- start 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- start start 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c0aecd0 0x7f6f8c13f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c1480c0 0x7f6f8c146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c15f910 con 0x7f6f8c1480c0 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c15f790 con 0x7f6f8c0aecd0 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.439+0000 7f6f93a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c16b680 con 0x7f6f8c0ae840 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c1480c0 0x7f6f8c146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c1480c0 0x7f6f8c146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59902/0 (socket says 172.21.15.110:59902) 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 -- 172.21.15.110:0/797914859 learned_addr learned my addr 172.21.15.110:0/797914859 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91000640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 -- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 msgr2=0x7f6f8c141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c141bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 -- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c0aecd0 msgr2=0x7f6f8c13f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f8be00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c0aecd0 0x7f6f8c13f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c0aecd0 0x7f6f8c13f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 -- 172.21.15.110:0/797914859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c0b5ef0 con 0x7f6f8c1480c0 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91000640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f8be00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c0aecd0 0x7f6f8c13f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f91a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c1480c0 0x7f6f8c146b70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009d90 tx=0x7f6f84019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f88c00640 1 -- 172.21.15.110:0/797914859 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840bd020 con 0x7f6f8c1480c0 2024-09-10T10:03:46.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f88c00640 1 -- 172.21.15.110:0/797914859 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f840b3040 con 0x7f6f8c1480c0 2024-09-10T10:03:46.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c1421d0 con 0x7f6f8c1480c0 2024-09-10T10:03:46.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f88c00640 1 -- 172.21.15.110:0/797914859 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f840b2050 con 0x7f6f8c1480c0 2024-09-10T10:03:46.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c142790 con 0x7f6f8c1480c0 2024-09-10T10:03:46.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c162c80 con 0x7f6f8c1480c0 2024-09-10T10:03:46.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f88c00640 1 -- 172.21.15.110:0/797914859 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f6f840adca0 con 0x7f6f8c1480c0 2024-09-10T10:03:46.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f88c00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f6c067cb0 0x7f6f6c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.443+0000 7f6f88c00640 1 -- 172.21.15.110:0/797914859 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5823+0+0 (secure 0 0 0) 0x7f6f84120ec0 con 0x7f6f8c1480c0 2024-09-10T10:03:46.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.447+0000 7f6f91000640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f6c067cb0 0x7f6f6c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.447+0000 7f6f91000640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f6c067cb0 0x7f6f6c06a170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007fd0 tx=0x7f6f7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.451+0000 7f6f88c00640 1 -- 172.21.15.110:0/797914859 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6f840c7060 con 0x7f6f8c1480c0 2024-09-10T10:03:46.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.651+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6f8c06e200 con 0x7f6f8c1480c0 2024-09-10T10:03:46.653 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:03:46.651+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:03:46.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.651+0000 7f6f88c00640 1 -- 172.21.15.110:0/797914859 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) ==== 161+0+0 (secure 0 0 0) 0x7f6f84016560 con 0x7f6f8c1480c0 2024-09-10T10:03:46.656 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:03:46.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f6c067cb0 msgr2=0x7f6f6c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f6c067cb0 0x7f6f6c06a170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007fd0 tx=0x7f6f7c009040 comp rx=0 tx=0).stop 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c1480c0 msgr2=0x7f6f8c146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c1480c0 0x7f6f8c146b70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009d90 tx=0x7f6f84019710 comp rx=0 tx=0).stop 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 shutdown_connections 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f6c067cb0 0x7f6f6c06a170 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f8c1480c0 0x7f6f8c146b70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f8c0aecd0 0x7f6f8c13f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 --2- 172.21.15.110:0/797914859 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f8c0ae840 0x7f6f8c141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 >> 172.21.15.110:0/797914859 conn(0x7f6f8c06cd60 msgr2=0x7f6f8c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 shutdown_connections 2024-09-10T10:03:46.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.655+0000 7f6f93a00640 1 -- 172.21.15.110:0/797914859 wait complete. 2024-09-10T10:03:46.681 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:03:46.728 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:03:46.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.823+0000 7f56e9600640 1 Processor -- start 2024-09-10T10:03:46.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.823+0000 7f56e9600640 1 -- start start 2024-09-10T10:03:46.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e9600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f56e40cfce0 0x7f56e415f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e9600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f56e40ae840 0x7f56e40b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e9600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56e40b1e90 con 0x7f56e406e140 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e9600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f56e40d00c0 con 0x7f56e40cfce0 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e9600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f56e40b21c0 con 0x7f56e40ae840 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f56e40cfce0 0x7f56e415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f56e40cfce0 0x7f56e415f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59918/0 (socket says 172.21.15.110:59918) 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2a00640 1 -- 172.21.15.110:0/630268194 learned_addr learned my addr 172.21.15.110:0/630268194 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e3400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f56e406e140 0x7f56e40b2650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60430/0 (socket says 172.21.15.110:60430) 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2000640 1 --2- 172.21.15.110:0/630268194 >> v2:172.21.15.145:3300/0 conn(0x7f56e40ae840 0x7f56e40b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2a00640 1 -- 172.21.15.110:0/630268194 >> v2:172.21.15.145:3300/0 conn(0x7f56e40ae840 msgr2=0x7f56e40b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:46.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2a00640 1 --2- 172.21.15.110:0/630268194 >> v2:172.21.15.145:3300/0 conn(0x7f56e40ae840 0x7f56e40b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2a00640 1 -- 172.21.15.110:0/630268194 >> v1:172.21.15.110:6789/0 conn(0x7f56e406e140 legacy=0x7f56e40b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2a00640 1 -- 172.21.15.110:0/630268194 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e40b43d0 con 0x7f56e40cfce0 2024-09-10T10:03:46.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2000640 1 --2- 172.21.15.110:0/630268194 >> v2:172.21.15.145:3300/0 conn(0x7f56e40ae840 0x7f56e40b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:46.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e1600640 1 -- 172.21.15.110:0/630268194 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667583425 0 0) 0x7f56e40b1e90 con 0x7f56e406e140 2024-09-10T10:03:46.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.827+0000 7f56e2a00640 1 --2- 172.21.15.110:0/630268194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40cfce0 0x7f56e415f870 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f56e40d0240 tx=0x7f56d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9d28636719fbd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e1600640 1 -- 172.21.15.110:0/630268194 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d40a57a0 con 0x7f56e40cfce0 2024-09-10T10:03:46.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e1600640 1 -- 172.21.15.110:0/630268194 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d40a5940 con 0x7f56e40cfce0 2024-09-10T10:03:46.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e1600640 1 -- 172.21.15.110:0/630268194 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d40a5d10 con 0x7f56e40cfce0 2024-09-10T10:03:46.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- 172.21.15.110:0/630268194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40cfce0 msgr2=0x7f56e415f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 --2- 172.21.15.110:0/630268194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40cfce0 0x7f56e415f870 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f56e40d0240 tx=0x7f56d401c9f0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- 172.21.15.110:0/630268194 shutdown_connections 2024-09-10T10:03:46.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 --2- 172.21.15.110:0/630268194 >> v2:172.21.15.145:3300/0 conn(0x7f56e40ae840 0x7f56e40b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 --2- 172.21.15.110:0/630268194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40cfce0 0x7f56e415f870 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- 172.21.15.110:0/630268194 >> 172.21.15.110:0/630268194 conn(0x7f56e4062e40 msgr2=0x7f56e406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:46.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- 172.21.15.110:0/630268194 shutdown_connections 2024-09-10T10:03:46.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- 172.21.15.110:0/630268194 wait complete. 2024-09-10T10:03:46.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 Processor -- start 2024-09-10T10:03:46.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- start start 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56e406e140 0x7f56e4134930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40ae840 0x7f56e4146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56e40cfce0 0x7f56e41592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56e406e960 con 0x7f56e40ae840 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f56e40b40c0 con 0x7f56e406e140 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e9600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f56e4159c50 con 0x7f56e40cfce0 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.831+0000 7f56e2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40ae840 0x7f56e4146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40ae840 0x7f56e4146be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59924/0 (socket says 172.21.15.110:59924) 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2000640 1 -- 172.21.15.110:0/3479273646 learned_addr learned my addr 172.21.15.110:0/3479273646 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e3400640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56e40cfce0 0x7f56e41592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2a00640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56e406e140 0x7f56e4134930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2000640 1 -- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56e40cfce0 msgr2=0x7f56e41592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2000640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56e40cfce0 0x7f56e41592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2000640 1 -- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56e406e140 msgr2=0x7f56e4134930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2000640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56e406e140 0x7f56e4134930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2000640 1 -- 172.21.15.110:0/3479273646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e4133e50 con 0x7f56e40ae840 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e2000640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40ae840 0x7f56e4146be0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e810 tx=0x7f56d000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56daa00640 1 -- 172.21.15.110:0/3479273646 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0017070 con 0x7f56e40ae840 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56daa00640 1 -- 172.21.15.110:0/3479273646 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d000c8d0 con 0x7f56e40ae840 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e4134130 con 0x7f56e40ae840 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56daa00640 1 -- 172.21.15.110:0/3479273646 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d000f040 con 0x7f56e40ae840 2024-09-10T10:03:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e4159f30 con 0x7f56e40ae840 2024-09-10T10:03:46.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e40aed90 con 0x7f56e40ae840 2024-09-10T10:03:46.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56daa00640 1 -- 172.21.15.110:0/3479273646 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f56d001a070 con 0x7f56e40ae840 2024-09-10T10:03:46.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56daa00640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56c4067ce0 0x7f56c406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.835+0000 7f56daa00640 1 -- 172.21.15.110:0/3479273646 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5881+0+0 (secure 0 0 0) 0x7f56d0086100 con 0x7f56e40ae840 2024-09-10T10:03:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.839+0000 7f56e2a00640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56c4067ce0 0x7f56c406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.839+0000 7f56e2a00640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56c4067ce0 0x7f56c406a1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f56d40ae000 tx=0x7f56d40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:46.839+0000 7f56daa00640 1 -- 172.21.15.110:0/3479273646 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f56d00564f0 con 0x7f56e40ae840 2024-09-10T10:03:47.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.031+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f56e415e9f0 con 0x7f56e40ae840 2024-09-10T10:03:47.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56daa00640 1 -- 172.21.15.110:0/3479273646 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) ==== 104+0+0 (secure 0 0 0) 0x7f56d0055df0 con 0x7f56e40ae840 2024-09-10T10:03:47.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56c4067ce0 msgr2=0x7f56c406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56c4067ce0 0x7f56c406a1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f56d40ae000 tx=0x7f56d40a7070 comp rx=0 tx=0).stop 2024-09-10T10:03:47.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40ae840 msgr2=0x7f56e4146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40ae840 0x7f56e4146be0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e810 tx=0x7f56d000ecd0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 shutdown_connections 2024-09-10T10:03:47.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f56c4067ce0 0x7f56c406a1a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f56e40cfce0 0x7f56e41592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56e40ae840 0x7f56e4146be0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 --2- 172.21.15.110:0/3479273646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f56e406e140 0x7f56e4134930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.655+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 >> 172.21.15.110:0/3479273646 conn(0x7f56e4062e40 msgr2=0x7f56e4072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:47.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.659+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 shutdown_connections 2024-09-10T10:03:47.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.659+0000 7f56e9600640 1 -- 172.21.15.110:0/3479273646 wait complete. 2024-09-10T10:03:47.682 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:03:47.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3edae00640 1 Processor -- start 2024-09-10T10:03:47.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3edae00640 1 -- start start 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3edae00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3ed415b170 0x7f3ed4073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3edae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 0x7f3ed40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3edae00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed40b3b40 con 0x7f3ed415ed40 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3edae00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed40b3910 con 0x7f3ed415b170 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3edae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed40cfc60 con 0x7f3ed40cf2e0 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3eda400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3ed415ed40 0x7f3ed415f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60436/0 (socket says 172.21.15.110:60436) 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3eda400640 1 -- 172.21.15.110:0/1004374244 learned_addr learned my addr 172.21.15.110:0/1004374244 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed9a00640 1 --2- 172.21.15.110:0/1004374244 >> v2:172.21.15.145:3300/0 conn(0x7f3ed415b170 0x7f3ed4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed9000640 1 --2- 172.21.15.110:0/1004374244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 0x7f3ed40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed3e00640 1 -- 172.21.15.110:0/1004374244 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627922813 0 0) 0x7f3ed40b3b40 con 0x7f3ed415ed40 2024-09-10T10:03:47.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed3e00640 1 -- 172.21.15.110:0/1004374244 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ebc003530 con 0x7f3ed415ed40 2024-09-10T10:03:47.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed9a00640 1 -- 172.21.15.110:0/1004374244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 msgr2=0x7f3ed40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:47.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed9a00640 1 --2- 172.21.15.110:0/1004374244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 0x7f3ed40cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed9a00640 1 -- 172.21.15.110:0/1004374244 >> v1:172.21.15.110:6789/0 conn(0x7f3ed415ed40 legacy=0x7f3ed415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed3e00640 1 -- 172.21.15.110:0/1004374244 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1277063974 0 0) 0x7f3ebc003530 con 0x7f3ed415ed40 2024-09-10T10:03:47.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.863+0000 7f3ed9a00640 1 -- 172.21.15.110:0/1004374244 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed40b3e20 con 0x7f3ed415b170 2024-09-10T10:03:47.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3ed9a00640 1 --2- 172.21.15.110:0/1004374244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed415b170 0x7f3ed4073390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009d90 tx=0x7f3ec801c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dca42de3cb0657 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:47.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3ed3e00640 1 -- 172.21.15.110:0/1004374244 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80b2070 con 0x7f3ed415b170 2024-09-10T10:03:47.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3ed3e00640 1 -- 172.21.15.110:0/1004374244 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec80a5660 con 0x7f3ed415b170 2024-09-10T10:03:47.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3ed3e00640 1 -- 172.21.15.110:0/1004374244 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc001680 con 0x7f3ed415b170 2024-09-10T10:03:47.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3edae00640 1 -- 172.21.15.110:0/1004374244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed415b170 msgr2=0x7f3ed4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3edae00640 1 --2- 172.21.15.110:0/1004374244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed415b170 0x7f3ed4073390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009d90 tx=0x7f3ec801c860 comp rx=0 tx=0).stop 2024-09-10T10:03:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3edae00640 1 -- 172.21.15.110:0/1004374244 shutdown_connections 2024-09-10T10:03:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3edae00640 1 --2- 172.21.15.110:0/1004374244 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 0x7f3ed40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3edae00640 1 --2- 172.21.15.110:0/1004374244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed415b170 0x7f3ed4073390 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3edae00640 1 -- 172.21.15.110:0/1004374244 >> 172.21.15.110:0/1004374244 conn(0x7f3ed406d0f0 msgr2=0x7f3ed406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3edae00640 1 -- 172.21.15.110:0/1004374244 shutdown_connections 2024-09-10T10:03:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.867+0000 7f3edae00640 1 -- 172.21.15.110:0/1004374244 wait complete. 2024-09-10T10:03:47.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 Processor -- start 2024-09-10T10:03:47.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 -- start start 2024-09-10T10:03:47.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 0x7f3ed4162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed415b170 0x7f3ed4160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed415ed40 0x7f3ed40ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4072680 con 0x7f3ed415b170 2024-09-10T10:03:47.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed415b550 con 0x7f3ed415ed40 2024-09-10T10:03:47.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed4072800 con 0x7f3ed40cf2e0 2024-09-10T10:03:47.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed415b170 0x7f3ed4160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed415b170 0x7f3ed4160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59940/0 (socket says 172.21.15.110:59940) 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 -- 172.21.15.110:0/803692510 learned_addr learned my addr 172.21.15.110:0/803692510 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 -- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 msgr2=0x7f3ed4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 0x7f3ed4162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 -- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed415ed40 msgr2=0x7f3ed40ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed415ed40 0x7f3ed40ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 -- 172.21.15.110:0/803692510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed4160710 con 0x7f3ed415b170 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed9000640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed415b170 0x7f3ed4160010 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000e960 tx=0x7f3ec000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed1600640 1 -- 172.21.15.110:0/803692510 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0017070 con 0x7f3ed415b170 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed1600640 1 -- 172.21.15.110:0/803692510 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec000ca00 con 0x7f3ed415b170 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3ed1600640 1 -- 172.21.15.110:0/803692510 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec000cdb0 con 0x7f3ed415b170 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed41609f0 con 0x7f3ed415b170 2024-09-10T10:03:47.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed40acfb0 con 0x7f3ed415b170 2024-09-10T10:03:47.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.871+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed40cf720 con 0x7f3ed415b170 2024-09-10T10:03:47.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.875+0000 7f3ed1600640 1 -- 172.21.15.110:0/803692510 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f3ec001a070 con 0x7f3ed415b170 2024-09-10T10:03:47.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.875+0000 7f3ed1600640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3eb4067cb0 0x7f3eb406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:47.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.875+0000 7f3ed1600640 1 -- 172.21.15.110:0/803692510 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5881+0+0 (secure 0 0 0) 0x7f3ec00867e0 con 0x7f3ed415b170 2024-09-10T10:03:47.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.875+0000 7f3ed9a00640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3eb4067cb0 0x7f3eb406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:47.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.875+0000 7f3ed9a00640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3eb4067cb0 0x7f3eb406a170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80af000 tx=0x7f3ec8004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:47.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:47.879+0000 7f3ed1600640 1 -- 172.21.15.110:0/803692510 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3ec0056d40 con 0x7f3ed415b170 2024-09-10T10:03:48.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.039+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ed4153200 con 0x7f3ed415b170 2024-09-10T10:03:48.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.659+0000 7f3ed1600640 1 -- 172.21.15.110:0/803692510 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f3ec0056640 con 0x7f3ed415b170 2024-09-10T10:03:48.665 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:03:48.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3eb4067cb0 msgr2=0x7f3eb406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:48.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3eb4067cb0 0x7f3eb406a170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80af000 tx=0x7f3ec8004ac0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed415b170 msgr2=0x7f3ed4160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:48.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed415b170 0x7f3ed4160010 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000e960 tx=0x7f3ec000ee20 comp rx=0 tx=0).stop 2024-09-10T10:03:48.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 shutdown_connections 2024-09-10T10:03:48.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3eb4067cb0 0x7f3eb406a170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ed415ed40 0x7f3ed40ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ed415b170 0x7f3ed4160010 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 --2- 172.21.15.110:0/803692510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ed40cf2e0 0x7f3ed4162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 >> 172.21.15.110:0/803692510 conn(0x7f3ed406d0f0 msgr2=0x7f3ed4153720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:48.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 shutdown_connections 2024-09-10T10:03:48.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.663+0000 7f3edae00640 1 -- 172.21.15.110:0/803692510 wait complete. 2024-09-10T10:03:48.690 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:03:48.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e86e00640 1 Processor -- start 2024-09-10T10:03:48.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e86e00640 1 -- start start 2024-09-10T10:03:48.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e86e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 0x7f5e800cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:48.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e86e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5e8006f1e0 0x7f5e800ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:48.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e86e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e80155d50 con 0x7f5e8015c560 2024-09-10T10:03:48.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e86e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5e800b4570 con 0x7f5e8006f1e0 2024-09-10T10:03:48.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e86e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5e800b34d0 con 0x7f5e80155970 2024-09-10T10:03:48.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e84e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5e8006f1e0 0x7f5e800ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:48.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e84e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5e8006f1e0 0x7f5e800ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34850/0 (socket says 172.21.15.110:34850) 2024-09-10T10:03:48.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e84e00640 1 -- 172.21.15.110:0/3235922782 learned_addr learned my addr 172.21.15.110:0/3235922782 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:48.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e7fe00640 1 --2- 172.21.15.110:0/3235922782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 0x7f5e800cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:48.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e84e00640 1 -- 172.21.15.110:0/3235922782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 msgr2=0x7f5e800cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:48.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e84e00640 1 --2- 172.21.15.110:0/3235922782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 0x7f5e800cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e84e00640 1 -- 172.21.15.110:0/3235922782 >> v1:172.21.15.145:6789/0 conn(0x7f5e8015c560 legacy=0x7f5e8006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:48.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.831+0000 7f5e84e00640 1 -- 172.21.15.110:0/3235922782 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e800b37b0 con 0x7f5e8006f1e0 2024-09-10T10:03:48.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e84e00640 1 --2- 172.21.15.110:0/3235922782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8006f1e0 0x7f5e800ce420 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009870 tx=0x7f5e7801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf20f4d71cf8068 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:48.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e7ea00640 1 -- 172.21.15.110:0/3235922782 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780ab620 con 0x7f5e8006f1e0 2024-09-10T10:03:48.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e7ea00640 1 -- 172.21.15.110:0/3235922782 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e780b0070 con 0x7f5e8006f1e0 2024-09-10T10:03:48.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e7ea00640 1 -- 172.21.15.110:0/3235922782 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780ab9e0 con 0x7f5e8006f1e0 2024-09-10T10:03:48.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e86e00640 1 -- 172.21.15.110:0/3235922782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8006f1e0 msgr2=0x7f5e800ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:48.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e86e00640 1 --2- 172.21.15.110:0/3235922782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8006f1e0 0x7f5e800ce420 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009870 tx=0x7f5e7801c460 comp rx=0 tx=0).stop 2024-09-10T10:03:48.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e86e00640 1 -- 172.21.15.110:0/3235922782 shutdown_connections 2024-09-10T10:03:48.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e86e00640 1 --2- 172.21.15.110:0/3235922782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8006f1e0 0x7f5e800ce420 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e86e00640 1 --2- 172.21.15.110:0/3235922782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 0x7f5e800cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e86e00640 1 -- 172.21.15.110:0/3235922782 >> 172.21.15.110:0/3235922782 conn(0x7f5e8006bc50 msgr2=0x7f5e8006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:48.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e86e00640 1 -- 172.21.15.110:0/3235922782 shutdown_connections 2024-09-10T10:03:48.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.835+0000 7f5e86e00640 1 -- 172.21.15.110:0/3235922782 wait complete. 2024-09-10T10:03:48.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 Processor -- start 2024-09-10T10:03:48.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 -- start start 2024-09-10T10:03:48.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e8006f1e0 0x7f5e80150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:48.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 0x7f5e8015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:48.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8015c560 0x7f5e80154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:48.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e800cf170 con 0x7f5e8015c560 2024-09-10T10:03:48.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5e800cce00 con 0x7f5e8006f1e0 2024-09-10T10:03:48.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5e800cf2f0 con 0x7f5e80155970 2024-09-10T10:03:48.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8015c560 0x7f5e80154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8015c560 0x7f5e80154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34864/0 (socket says 172.21.15.110:34864) 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 -- 172.21.15.110:0/2628722710 learned_addr learned my addr 172.21.15.110:0/2628722710 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e7fe00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e8006f1e0 0x7f5e80150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e7f400640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 0x7f5e8015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 -- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 msgr2=0x7f5e8015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 0x7f5e8015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 -- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e8006f1e0 msgr2=0x7f5e80150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e8006f1e0 0x7f5e80150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 -- 172.21.15.110:0/2628722710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8015d430 con 0x7f5e8015c560 2024-09-10T10:03:48.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e84e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8015c560 0x7f5e80154ab0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5e7801ab10 tx=0x7f5e7801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:48.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e77e00640 1 -- 172.21.15.110:0/2628722710 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780abd30 con 0x7f5e8015c560 2024-09-10T10:03:48.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e77e00640 1 -- 172.21.15.110:0/2628722710 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e780be3f0 con 0x7f5e8015c560 2024-09-10T10:03:48.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e77e00640 1 -- 172.21.15.110:0/2628722710 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780be7a0 con 0x7f5e8015c560 2024-09-10T10:03:48.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8015d710 con 0x7f5e8015c560 2024-09-10T10:03:48.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.839+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8015dcd0 con 0x7f5e8015c560 2024-09-10T10:03:48.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.843+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e80156e10 con 0x7f5e8015c560 2024-09-10T10:03:48.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.843+0000 7f5e77e00640 1 -- 172.21.15.110:0/2628722710 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f5e780b43f0 con 0x7f5e8015c560 2024-09-10T10:03:48.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.843+0000 7f5e77e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e60067cb0 0x7f5e6006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:48.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.843+0000 7f5e77e00640 1 -- 172.21.15.110:0/2628722710 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5443+0+0 (secure 0 0 0) 0x7f5e78120450 con 0x7f5e8015c560 2024-09-10T10:03:48.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.843+0000 7f5e7fe00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e60067cb0 0x7f5e6006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:48.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.843+0000 7f5e7fe00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e60067cb0 0x7f5e6006a170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5e70003b90 tx=0x7f5e700098d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:48.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:48.847+0000 7f5e77e00640 1 -- 172.21.15.110:0/2628722710 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5e780c4060 con 0x7f5e8015c560 2024-09-10T10:03:49.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.007+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5e800ce420 con 0x7f5e8015c560 2024-09-10T10:03:49.209 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:03:49.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:03:49.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.663+0000 7f5e77e00640 1 -- 172.21.15.110:0/2628722710 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f5e78016110 con 0x7f5e8015c560 2024-09-10T10:03:49.666 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:03:49.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e60067cb0 msgr2=0x7f5e6006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e60067cb0 0x7f5e6006a170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5e70003b90 tx=0x7f5e700098d0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8015c560 msgr2=0x7f5e80154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8015c560 0x7f5e80154ab0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5e7801ab10 tx=0x7f5e7801aca0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 shutdown_connections 2024-09-10T10:03:49.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e60067cb0 0x7f5e6006a170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e8015c560 0x7f5e80154ab0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e80155970 0x7f5e8015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 --2- 172.21.15.110:0/2628722710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e8006f1e0 0x7f5e80150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 >> 172.21.15.110:0/2628722710 conn(0x7f5e8006bc50 msgr2=0x7f5e800b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:49.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 shutdown_connections 2024-09-10T10:03:49.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.667+0000 7f5e86e00640 1 -- 172.21.15.110:0/2628722710 wait complete. 2024-09-10T10:03:49.691 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:03:49.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.831+0000 7f485d800640 1 Processor -- start 2024-09-10T10:03:49.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.831+0000 7f485d800640 1 -- start start 2024-09-10T10:03:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.831+0000 7f485d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485815b330 0x7f485806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f485d800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f485806e850 0x7f48580b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f485d800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48580b4960 con 0x7f48580ce4d0 2024-09-10T10:03:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f485d800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f48580b5570 con 0x7f485806e850 2024-09-10T10:03:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f485d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f48580b4c40 con 0x7f485815b330 2024-09-10T10:03:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4856a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485815b330 0x7f485806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4856a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485815b330 0x7f485806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:51660/0 (socket says 172.21.15.110:51660) 2024-09-10T10:03:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4856a00640 1 -- 172.21.15.110:0/3988354048 learned_addr learned my addr 172.21.15.110:0/3988354048 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4857400640 1 --2- 172.21.15.110:0/3988354048 >> v2:172.21.15.145:3300/0 conn(0x7f485806e850 0x7f48580b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4856a00640 1 -- 172.21.15.110:0/3988354048 >> v2:172.21.15.145:3300/0 conn(0x7f485806e850 msgr2=0x7f48580b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4856a00640 1 --2- 172.21.15.110:0/3988354048 >> v2:172.21.15.145:3300/0 conn(0x7f485806e850 0x7f48580b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4855600640 1 -- 172.21.15.110:0/3988354048 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847063079 0 0) 0x7f48580b4960 con 0x7f48580ce4d0 2024-09-10T10:03:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4856a00640 1 -- 172.21.15.110:0/3988354048 >> v1:172.21.15.145:6789/0 conn(0x7f48580ce4d0 legacy=0x7f485806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.835+0000 7f4856a00640 1 -- 172.21.15.110:0/3988354048 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48580b4f20 con 0x7f485815b330 2024-09-10T10:03:49.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f4856a00640 1 --2- 172.21.15.110:0/3988354048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485815b330 0x7f485806c6a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4848009870 tx=0x7f484801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b391649bce85ae7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f4855600640 1 -- 172.21.15.110:0/3988354048 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48480a7070 con 0x7f485815b330 2024-09-10T10:03:49.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f4855600640 1 -- 172.21.15.110:0/3988354048 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48480b0070 con 0x7f485815b330 2024-09-10T10:03:49.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f4855600640 1 -- 172.21.15.110:0/3988354048 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48480ab890 con 0x7f485815b330 2024-09-10T10:03:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- 172.21.15.110:0/3988354048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485815b330 msgr2=0x7f485806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 --2- 172.21.15.110:0/3988354048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485815b330 0x7f485806c6a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4848009870 tx=0x7f484801c560 comp rx=0 tx=0).stop 2024-09-10T10:03:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- 172.21.15.110:0/3988354048 shutdown_connections 2024-09-10T10:03:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 --2- 172.21.15.110:0/3988354048 >> v2:172.21.15.145:3300/0 conn(0x7f485806e850 0x7f48580b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 --2- 172.21.15.110:0/3988354048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485815b330 0x7f485806c6a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- 172.21.15.110:0/3988354048 >> 172.21.15.110:0/3988354048 conn(0x7f4858062e40 msgr2=0x7f485806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:49.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- 172.21.15.110:0/3988354048 shutdown_connections 2024-09-10T10:03:49.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- 172.21.15.110:0/3988354048 wait complete. 2024-09-10T10:03:49.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 Processor -- start 2024-09-10T10:03:49.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- start start 2024-09-10T10:03:49.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806e850 0x7f48580ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485815b330 0x7f48580aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4858160070 0x7f48581459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f48581680b0 con 0x7f485815b330 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f48580b17c0 con 0x7f4858160070 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f485d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4858168230 con 0x7f485806e850 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.839+0000 7f4856000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485815b330 0x7f48580aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485815b330 0x7f48580aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34880/0 (socket says 172.21.15.110:34880) 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856000640 1 -- 172.21.15.110:0/2529022262 learned_addr learned my addr 172.21.15.110:0/2529022262 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4857400640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4858160070 0x7f48581459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856a00640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806e850 0x7f48580ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856000640 1 -- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806e850 msgr2=0x7f48580ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856000640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806e850 0x7f48580ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856000640 1 -- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4858160070 msgr2=0x7f48581459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856000640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4858160070 0x7f48581459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856000640 1 -- 172.21.15.110:0/2529022262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4858160530 con 0x7f485815b330 2024-09-10T10:03:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f4856000640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485815b330 0x7f48580aed80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f484400c450 tx=0x7f484400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f484ea00640 1 -- 172.21.15.110:0/2529022262 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844005480 con 0x7f485815b330 2024-09-10T10:03:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f484ea00640 1 -- 172.21.15.110:0/2529022262 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4844003600 con 0x7f485815b330 2024-09-10T10:03:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485815ba60 con 0x7f485815b330 2024-09-10T10:03:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f484ea00640 1 -- 172.21.15.110:0/2529022262 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48440039b0 con 0x7f485815b330 2024-09-10T10:03:49.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4858160810 con 0x7f485815b330 2024-09-10T10:03:49.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48580b0840 con 0x7f485815b330 2024-09-10T10:03:49.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f484ea00640 1 -- 172.21.15.110:0/2529022262 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f4844003b50 con 0x7f485815b330 2024-09-10T10:03:49.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.843+0000 7f484ea00640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4838067cb0 0x7f483806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.847+0000 7f484ea00640 1 -- 172.21.15.110:0/2529022262 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f4844085b20 con 0x7f485815b330 2024-09-10T10:03:49.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.847+0000 7f4856a00640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4838067cb0 0x7f483806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.847+0000 7f4856a00640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4838067cb0 0x7f483806a170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48480af000 tx=0x7f48480a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:49.851+0000 7f484ea00640 1 -- 172.21.15.110:0/2529022262 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4844055da0 con 0x7f485815b330 2024-09-10T10:03:50.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.007+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f485815eaa0 con 0x7f485815b330 2024-09-10T10:03:50.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.011+0000 7f484ea00640 1 -- 172.21.15.110:0/2529022262 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f484405a0a0 con 0x7f485815b330 2024-09-10T10:03:50.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4838067cb0 msgr2=0x7f483806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4838067cb0 0x7f483806a170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48480af000 tx=0x7f48480a7000 comp rx=0 tx=0).stop 2024-09-10T10:03:50.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485815b330 msgr2=0x7f48580aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485815b330 0x7f48580aed80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f484400c450 tx=0x7f484400c910 comp rx=0 tx=0).stop 2024-09-10T10:03:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 shutdown_connections 2024-09-10T10:03:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4838067cb0 0x7f483806a170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4858160070 0x7f48581459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485815b330 0x7f48580aed80 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 --2- 172.21.15.110:0/2529022262 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f485806e850 0x7f48580ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.015+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 >> 172.21.15.110:0/2529022262 conn(0x7f4858062e40 msgr2=0x7f485815dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.019+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 shutdown_connections 2024-09-10T10:03:50.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.019+0000 7f485d800640 1 -- 172.21.15.110:0/2529022262 wait complete. 2024-09-10T10:03:50.044 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-10T10:03:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.179+0000 7f81e1600640 1 Processor -- start 2024-09-10T10:03:50.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.179+0000 7f81e1600640 1 -- start start 2024-09-10T10:03:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81e1600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81dc0cd510 0x7f81dc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81e1600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81dc072150 con 0x7f81dc06e090 2024-09-10T10:03:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81e1600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81dc06d2d0 con 0x7f81dc0cfe30 2024-09-10T10:03:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81e1600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f81dc15d9a0 con 0x7f81dc0cd510 2024-09-10T10:03:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81d3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81dc0cd510 0x7f81dc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81d3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81dc0cd510 0x7f81dc0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34896/0 (socket says 172.21.15.110:34896) 2024-09-10T10:03:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81d3e00640 1 -- 172.21.15.110:0/244793075 learned_addr learned my addr 172.21.15.110:0/244793075 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81d3e00640 1 -- 172.21.15.110:0/244793075 >> v1:172.21.15.145:6789/0 conn(0x7f81dc0cfe30 legacy=0x7f81dc071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81d3e00640 1 -- 172.21.15.110:0/244793075 >> v1:172.21.15.110:6789/0 conn(0x7f81dc06e090 legacy=0x7f81dc06e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81d3e00640 1 -- 172.21.15.110:0/244793075 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc06d5b0 con 0x7f81dc0cd510 2024-09-10T10:03:50.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81da000640 1 -- 172.21.15.110:0/244793075 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530319266 0 0) 0x7f81dc072150 con 0x7f81dc06e090 2024-09-10T10:03:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81d3e00640 1 --2- 172.21.15.110:0/244793075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cd510 0x7f81dc0cd9a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009870 tx=0x7f81c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec7bc991b417f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.183+0000 7f81da000640 1 -- 172.21.15.110:0/244793075 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40b1070 con 0x7f81dc0cd510 2024-09-10T10:03:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81da000640 1 -- 172.21.15.110:0/244793075 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c40a8050 con 0x7f81dc0cd510 2024-09-10T10:03:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81da000640 1 -- 172.21.15.110:0/244793075 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40ac860 con 0x7f81dc0cd510 2024-09-10T10:03:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- 172.21.15.110:0/244793075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cd510 msgr2=0x7f81dc0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 --2- 172.21.15.110:0/244793075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cd510 0x7f81dc0cd9a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009870 tx=0x7f81c401c4d0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- 172.21.15.110:0/244793075 shutdown_connections 2024-09-10T10:03:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 --2- 172.21.15.110:0/244793075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cd510 0x7f81dc0cd9a0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- 172.21.15.110:0/244793075 >> 172.21.15.110:0/244793075 conn(0x7f81dc06bc50 msgr2=0x7f81dc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- 172.21.15.110:0/244793075 shutdown_connections 2024-09-10T10:03:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- 172.21.15.110:0/244793075 wait complete. 2024-09-10T10:03:50.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 Processor -- start 2024-09-10T10:03:50.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- start start 2024-09-10T10:03:50.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc06e090 0x7f81dc147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81dc0cd510 0x7f81dc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cfe30 0x7f81dc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc16a190 con 0x7f81dc0cfe30 2024-09-10T10:03:50.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc15d700 con 0x7f81dc06e090 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81e1600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc16a310 con 0x7f81dc0cd510 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81db400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cfe30 0x7f81dc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81db400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cfe30 0x7f81dc0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34902/0 (socket says 172.21.15.110:34902) 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81db400640 1 -- 172.21.15.110:0/3807179281 learned_addr learned my addr 172.21.15.110:0/3807179281 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81daa00640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc06e090 0x7f81dc147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81db400640 1 -- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81dc0cd510 msgr2=0x7f81dc0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81d3e00640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81dc0cd510 0x7f81dc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81db400640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81dc0cd510 0x7f81dc0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81db400640 1 -- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc06e090 msgr2=0x7f81dc147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81db400640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc06e090 0x7f81dc147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81db400640 1 -- 172.21.15.110:0/3807179281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc155520 con 0x7f81dc0cfe30 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.187+0000 7f81daa00640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc06e090 0x7f81dc147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81db400640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cfe30 0x7f81dc0b1230 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c780 tx=0x7f81d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81d2a00640 1 -- 172.21.15.110:0/3807179281 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4003db0 con 0x7f81dc0cfe30 2024-09-10T10:03:50.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81d2a00640 1 -- 172.21.15.110:0/3807179281 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d400ce30 con 0x7f81dc0cfe30 2024-09-10T10:03:50.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81d2a00640 1 -- 172.21.15.110:0/3807179281 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4005290 con 0x7f81dc0cfe30 2024-09-10T10:03:50.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc154760 con 0x7f81dc0cfe30 2024-09-10T10:03:50.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc154d20 con 0x7f81dc0cfe30 2024-09-10T10:03:50.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc0cd9a0 con 0x7f81dc0cfe30 2024-09-10T10:03:50.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81d2a00640 1 -- 172.21.15.110:0/3807179281 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f81d4005430 con 0x7f81dc0cfe30 2024-09-10T10:03:50.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81d2a00640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81b8067ce0 0x7f81b806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81d2a00640 1 -- 172.21.15.110:0/3807179281 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f81d4085fc0 con 0x7f81dc0cfe30 2024-09-10T10:03:50.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.191+0000 7f81daa00640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81b8067ce0 0x7f81b806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.195+0000 7f81daa00640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81b8067ce0 0x7f81b806a1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f81cc000f70 tx=0x7f81cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.195+0000 7f81d2a00640 1 -- 172.21.15.110:0/3807179281 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f81d4019300 con 0x7f81dc0cfe30 2024-09-10T10:03:50.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.411+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f81dc15e8e0 con 0x7f81dc0cfe30 2024-09-10T10:03:50.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.667+0000 7f81d2a00640 1 -- 172.21.15.110:0/3807179281 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v298) ==== 149+0+0 (secure 0 0 0) 0x7f81d40567c0 con 0x7f81dc0cfe30 2024-09-10T10:03:50.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81b8067ce0 msgr2=0x7f81b806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81b8067ce0 0x7f81b806a1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f81cc000f70 tx=0x7f81cc009040 comp rx=0 tx=0).stop 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cfe30 msgr2=0x7f81dc0b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cfe30 0x7f81dc0b1230 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c780 tx=0x7f81d400cc40 comp rx=0 tx=0).stop 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 shutdown_connections 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81b8067ce0 0x7f81b806a1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81dc0cfe30 0x7f81dc0b1230 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81dc0cd510 0x7f81dc0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 --2- 172.21.15.110:0/3807179281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc06e090 0x7f81dc147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 >> 172.21.15.110:0/3807179281 conn(0x7f81dc06bc50 msgr2=0x7f81dc15ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 shutdown_connections 2024-09-10T10:03:50.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.671+0000 7f81e1600640 1 -- 172.21.15.110:0/3807179281 wait complete. 2024-09-10T10:03:50.693 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-10T10:03:50.694 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-10T10:03:50.694 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:03:50.694 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:03:50.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3db5e00640 1 Processor -- start 2024-09-10T10:03:50.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3db5e00640 1 -- start start 2024-09-10T10:03:50.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3db5e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3db00d0cb0 0x7f3db0163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3db5e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3db0073290 0x7f3db00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3db5e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3db00706a0 con 0x7f3db006eb30 2024-09-10T10:03:50.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3db5e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3db015ee90 con 0x7f3db00d0cb0 2024-09-10T10:03:50.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3db5e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3db0073c20 con 0x7f3db0073290 2024-09-10T10:03:50.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3dae000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3db00d0cb0 0x7f3db0163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3dae000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3db00d0cb0 0x7f3db0163000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34914/0 (socket says 172.21.15.110:34914) 2024-09-10T10:03:50.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3dae000640 1 -- 172.21.15.110:0/2538053359 learned_addr learned my addr 172.21.15.110:0/2538053359 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:50.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3daea00640 1 --1- 172.21.15.110:0/2538053359 >> v1:172.21.15.110:6789/0 conn(0x7f3db006eb30 0x7f3db0071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47820/0 (socket says 172.21.15.110:47820) 2024-09-10T10:03:50.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3daf400640 1 --2- 172.21.15.110:0/2538053359 >> v2:172.21.15.145:3300/0 conn(0x7f3db0073290 0x7f3db00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3dae000640 1 -- 172.21.15.110:0/2538053359 >> v2:172.21.15.145:3300/0 conn(0x7f3db0073290 msgr2=0x7f3db00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3dae000640 1 --2- 172.21.15.110:0/2538053359 >> v2:172.21.15.145:3300/0 conn(0x7f3db0073290 0x7f3db00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3dae000640 1 -- 172.21.15.110:0/2538053359 >> v1:172.21.15.110:6789/0 conn(0x7f3db006eb30 legacy=0x7f3db0071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3dae000640 1 -- 172.21.15.110:0/2538053359 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db00d10f0 con 0x7f3db00d0cb0 2024-09-10T10:03:50.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.827+0000 7f3dad600640 1 -- 172.21.15.110:0/2538053359 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612820547 0 0) 0x7f3db00706a0 con 0x7f3db006eb30 2024-09-10T10:03:50.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3dae000640 1 --2- 172.21.15.110:0/2538053359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db00d0cb0 0x7f3db0163000 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3db00b8010 tx=0x7f3d9c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4958891581bf344 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3dad600640 1 -- 172.21.15.110:0/2538053359 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0b0070 con 0x7f3db00d0cb0 2024-09-10T10:03:50.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3dad600640 1 -- 172.21.15.110:0/2538053359 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9c0ab600 con 0x7f3db00d0cb0 2024-09-10T10:03:50.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3dad600640 1 -- 172.21.15.110:0/2538053359 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0ab9d0 con 0x7f3db00d0cb0 2024-09-10T10:03:50.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3db5e00640 1 -- 172.21.15.110:0/2538053359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db00d0cb0 msgr2=0x7f3db0163000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3db5e00640 1 --2- 172.21.15.110:0/2538053359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db00d0cb0 0x7f3db0163000 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3db00b8010 tx=0x7f3d9c01c490 comp rx=0 tx=0).stop 2024-09-10T10:03:50.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3db5e00640 1 -- 172.21.15.110:0/2538053359 shutdown_connections 2024-09-10T10:03:50.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3db5e00640 1 --2- 172.21.15.110:0/2538053359 >> v2:172.21.15.145:3300/0 conn(0x7f3db0073290 0x7f3db00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3db5e00640 1 --2- 172.21.15.110:0/2538053359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db00d0cb0 0x7f3db0163000 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3db5e00640 1 -- 172.21.15.110:0/2538053359 >> 172.21.15.110:0/2538053359 conn(0x7f3db00d25c0 msgr2=0x7f3db00d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:50.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3db5e00640 1 -- 172.21.15.110:0/2538053359 shutdown_connections 2024-09-10T10:03:50.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.831+0000 7f3db5e00640 1 -- 172.21.15.110:0/2538053359 wait complete. 2024-09-10T10:03:50.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 Processor -- start 2024-09-10T10:03:50.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 -- start start 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db006eb30 0x7f3db00b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3db0073290 0x7f3db015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3db00d0cb0 0x7f3db015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0070d60 con 0x7f3db006eb30 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0071c70 con 0x7f3db0073290 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3db006dd60 con 0x7f3db00d0cb0 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db006eb30 0x7f3db00b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db006eb30 0x7f3db00b7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34924/0 (socket says 172.21.15.110:34924) 2024-09-10T10:03:50.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 -- 172.21.15.110:0/3310881639 learned_addr learned my addr 172.21.15.110:0/3310881639 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3dae000640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3db0073290 0x7f3db015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daf400640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3db00d0cb0 0x7f3db015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 -- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3db00d0cb0 msgr2=0x7f3db015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3db00d0cb0 0x7f3db015a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 -- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3db0073290 msgr2=0x7f3db015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3db0073290 0x7f3db015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 -- 172.21.15.110:0/3310881639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db015ac20 con 0x7f3db006eb30 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3daea00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db006eb30 0x7f3db00b7320 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3da000c520 tx=0x7f3da000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3da7400640 1 -- 172.21.15.110:0/3310881639 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0018070 con 0x7f3db006eb30 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3da7400640 1 -- 172.21.15.110:0/3310881639 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da00074e0 con 0x7f3db006eb30 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3da7400640 1 -- 172.21.15.110:0/3310881639 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da00049f0 con 0x7f3db006eb30 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db00cc480 con 0x7f3db006eb30 2024-09-10T10:03:50.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.835+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db00b6e00 con 0x7f3db006eb30 2024-09-10T10:03:50.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.839+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db01692b0 con 0x7f3db006eb30 2024-09-10T10:03:50.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.839+0000 7f3da7400640 1 -- 172.21.15.110:0/3310881639 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f3da0004b90 con 0x7f3db006eb30 2024-09-10T10:03:50.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.839+0000 7f3da7400640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d90067ce0 0x7f3d9006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.839+0000 7f3da7400640 1 -- 172.21.15.110:0/3310881639 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f3da00853e0 con 0x7f3db006eb30 2024-09-10T10:03:50.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.839+0000 7f3dae000640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d90067ce0 0x7f3d9006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.839+0000 7f3dae000640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d90067ce0 0x7f3d9006a1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0ad000 tx=0x7f3d9c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.843+0000 7f3da7400640 1 -- 172.21.15.110:0/3310881639 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3da0055cd0 con 0x7f3db006eb30 2024-09-10T10:03:51.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:50.999+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3db00736a0 con 0x7f3db006eb30 2024-09-10T10:03:51.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3da7400640 1 -- 172.21.15.110:0/3310881639 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13122 (secure 0 0 0) 0x7f3da00555d0 con 0x7f3db006eb30 2024-09-10T10:03:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:51.006 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":109,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:49.665208+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6835/100964295":"2024-09-11T10:03:46.654758+0000","172.21.15.110:6832/100964295":"2024-09-11T10:03:46.654758+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d90067ce0 msgr2=0x7f3d9006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d90067ce0 0x7f3d9006a1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0ad000 tx=0x7f3d9c0a6000 comp rx=0 tx=0).stop 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db006eb30 msgr2=0x7f3db00b7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db006eb30 0x7f3db00b7320 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3da000c520 tx=0x7f3da000c9e0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 shutdown_connections 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d90067ce0 0x7f3d9006a1a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3db00d0cb0 0x7f3db015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3db0073290 0x7f3db015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 --2- 172.21.15.110:0/3310881639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db006eb30 0x7f3db00b7320 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 >> 172.21.15.110:0/3310881639 conn(0x7f3db00d25c0 msgr2=0x7f3db00b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:51.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 shutdown_connections 2024-09-10T10:03:51.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.003+0000 7f3db5e00640 1 -- 172.21.15.110:0/3310881639 wait complete. 2024-09-10T10:03:51.026 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:03:51.027 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:03:51.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d73e00640 1 Processor -- start 2024-09-10T10:03:51.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d73e00640 1 -- start start 2024-09-10T10:03:51.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d73e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3d7415c2e0 0x7f3d740b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d73e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3d740b63d0 0x7f3d740b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d73e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d7415d370 con 0x7f3d740b4c00 2024-09-10T10:03:51.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d73e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3d740b5070 con 0x7f3d7415c2e0 2024-09-10T10:03:51.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d73e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3d7415d930 con 0x7f3d740b63d0 2024-09-10T10:03:51.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3d7415c2e0 0x7f3d740b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3d7415c2e0 0x7f3d740b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34930/0 (socket says 172.21.15.110:34930) 2024-09-10T10:03:51.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72a00640 1 -- 172.21.15.110:0/557593655 learned_addr learned my addr 172.21.15.110:0/557593655 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:51.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72a00640 1 -- 172.21.15.110:0/557593655 >> v2:172.21.15.145:3300/0 conn(0x7f3d740b63d0 msgr2=0x7f3d740b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:51.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72000640 1 --2- 172.21.15.110:0/557593655 >> v2:172.21.15.145:3300/0 conn(0x7f3d740b63d0 0x7f3d740b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72a00640 1 --2- 172.21.15.110:0/557593655 >> v2:172.21.15.145:3300/0 conn(0x7f3d740b63d0 0x7f3d740b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72a00640 1 -- 172.21.15.110:0/557593655 >> v1:172.21.15.145:6789/0 conn(0x7f3d740b4c00 legacy=0x7f3d7415c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:51.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72a00640 1 -- 172.21.15.110:0/557593655 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7415d650 con 0x7f3d7415c2e0 2024-09-10T10:03:51.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.163+0000 7f3d72000640 1 --2- 172.21.15.110:0/557593655 >> v2:172.21.15.145:3300/0 conn(0x7f3d740b63d0 0x7f3d740b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d72a00640 1 --2- 172.21.15.110:0/557593655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d7415c2e0 0x7f3d740b5e90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3d74070470 tx=0x7f3d6401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744f349826d0bb7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d71600640 1 -- 172.21.15.110:0/557593655 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640aa610 con 0x7f3d7415c2e0 2024-09-10T10:03:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d71600640 1 -- 172.21.15.110:0/557593655 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d640af070 con 0x7f3d7415c2e0 2024-09-10T10:03:51.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d71600640 1 -- 172.21.15.110:0/557593655 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640aa9d0 con 0x7f3d7415c2e0 2024-09-10T10:03:51.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 -- 172.21.15.110:0/557593655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d7415c2e0 msgr2=0x7f3d740b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 --2- 172.21.15.110:0/557593655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d7415c2e0 0x7f3d740b5e90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3d74070470 tx=0x7f3d6401c4b0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 -- 172.21.15.110:0/557593655 shutdown_connections 2024-09-10T10:03:51.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 --2- 172.21.15.110:0/557593655 >> v2:172.21.15.145:3300/0 conn(0x7f3d740b63d0 0x7f3d740b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 --2- 172.21.15.110:0/557593655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d7415c2e0 0x7f3d740b5e90 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 -- 172.21.15.110:0/557593655 >> 172.21.15.110:0/557593655 conn(0x7f3d740b7be0 msgr2=0x7f3d740b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 -- 172.21.15.110:0/557593655 shutdown_connections 2024-09-10T10:03:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 -- 172.21.15.110:0/557593655 wait complete. 2024-09-10T10:03:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 Processor -- start 2024-09-10T10:03:51.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.167+0000 7f3d73e00640 1 -- start start 2024-09-10T10:03:51.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d740b4c00 0x7f3d741583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d740b63d0 0x7f3d740cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d7415c2e0 0x7f3d74159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d740d0910 con 0x7f3d740b63d0 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d740d0790 con 0x7f3d7415c2e0 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3d740cd920 con 0x7f3d740b4c00 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d740b63d0 0x7f3d740cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d740b4c00 0x7f3d741583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d740b63d0 0x7f3d740cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34942/0 (socket says 172.21.15.110:34942) 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 -- 172.21.15.110:0/3831385672 learned_addr learned my addr 172.21.15.110:0/3831385672 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:51.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73400640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d7415c2e0 0x7f3d74159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 -- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d740b4c00 msgr2=0x7f3d741583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d740b4c00 0x7f3d741583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 -- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d7415c2e0 msgr2=0x7f3d74159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d7415c2e0 0x7f3d74159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 -- 172.21.15.110:0/3831385672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d740cbab0 con 0x7f3d740b63d0 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d72000640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d740b63d0 0x7f3d740cd3e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c7f0 tx=0x7f3d6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d6aa00640 1 -- 172.21.15.110:0/3831385672 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60015070 con 0x7f3d740b63d0 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d6aa00640 1 -- 172.21.15.110:0/3831385672 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d60007a30 con 0x7f3d740b63d0 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d740cbd90 con 0x7f3d740b63d0 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d6aa00640 1 -- 172.21.15.110:0/3831385672 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60007e00 con 0x7f3d740b63d0 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d740c8d20 con 0x7f3d740b63d0 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d740cbf60 con 0x7f3d740b63d0 2024-09-10T10:03:51.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.171+0000 7f3d6aa00640 1 -- 172.21.15.110:0/3831385672 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f3d60018070 con 0x7f3d740b63d0 2024-09-10T10:03:51.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.175+0000 7f3d6aa00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d54067cb0 0x7f3d5406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.175+0000 7f3d6aa00640 1 -- 172.21.15.110:0/3831385672 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f3d60086150 con 0x7f3d740b63d0 2024-09-10T10:03:51.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.175+0000 7f3d72a00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d54067cb0 0x7f3d5406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.175+0000 7f3d72a00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d54067cb0 0x7f3d5406a170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d640ae000 tx=0x7f3d640a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:51.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.179+0000 7f3d6aa00640 1 -- 172.21.15.110:0/3831385672 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3d600561f0 con 0x7f3d740b63d0 2024-09-10T10:03:51.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.335+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3d740b46c0 con 0x7f3d740b63d0 2024-09-10T10:03:51.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.335+0000 7f3d6aa00640 1 -- 172.21.15.110:0/3831385672 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f3d6005a4f0 con 0x7f3d740b63d0 2024-09-10T10:03:51.339 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d54067cb0 msgr2=0x7f3d5406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d54067cb0 0x7f3d5406a170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d640ae000 tx=0x7f3d640a7000 comp rx=0 tx=0).stop 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d740b63d0 msgr2=0x7f3d740cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d740b63d0 0x7f3d740cd3e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c7f0 tx=0x7f3d6000ccb0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 shutdown_connections 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d54067cb0 0x7f3d5406a170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d7415c2e0 0x7f3d74159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d740b63d0 0x7f3d740cd3e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 --2- 172.21.15.110:0/3831385672 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d740b4c00 0x7f3d741583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 >> 172.21.15.110:0/3831385672 conn(0x7f3d740b7be0 msgr2=0x7f3d7415b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 shutdown_connections 2024-09-10T10:03:51.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.339+0000 7f3d73e00640 1 -- 172.21.15.110:0/3831385672 wait complete. 2024-09-10T10:03:51.361 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-10T10:03:51.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb0c000640 1 Processor -- start 2024-09-10T10:03:51.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb0c000640 1 -- start start 2024-09-10T10:03:51.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb0c000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7feb0415c960 0x7feb040b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb0c000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb041603c0 0x7feb040d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb0c000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb040706a0 con 0x7feb040d04c0 2024-09-10T10:03:51.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb0c000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7feb0415cd40 con 0x7feb0415c960 2024-09-10T10:03:51.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb0c000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feb040b6370 con 0x7feb041603c0 2024-09-10T10:03:51.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb08c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb041603c0 0x7feb040d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb08c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb041603c0 0x7feb040d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:51696/0 (socket says 172.21.15.110:51696) 2024-09-10T10:03:51.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb08c00640 1 -- 172.21.15.110:0/815900259 learned_addr learned my addr 172.21.15.110:0/815900259 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:51.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb09600640 1 --2- 172.21.15.110:0/815900259 >> v2:172.21.15.145:3300/0 conn(0x7feb0415c960 0x7feb040b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb08c00640 1 -- 172.21.15.110:0/815900259 >> v2:172.21.15.145:3300/0 conn(0x7feb0415c960 msgr2=0x7feb040b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:51.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb08c00640 1 --2- 172.21.15.110:0/815900259 >> v2:172.21.15.145:3300/0 conn(0x7feb0415c960 0x7feb040b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb08c00640 1 -- 172.21.15.110:0/815900259 >> v1:172.21.15.145:6789/0 conn(0x7feb040d04c0 legacy=0x7feb04070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb08c00640 1 -- 172.21.15.110:0/815900259 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb04070a80 con 0x7feb041603c0 2024-09-10T10:03:51.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.495+0000 7feb03e00640 1 -- 172.21.15.110:0/815900259 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49074116 0 0) 0x7feb040706a0 con 0x7feb040d04c0 2024-09-10T10:03:51.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb08c00640 1 --2- 172.21.15.110:0/815900259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb041603c0 0x7feb040d0cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009870 tx=0x7feaf001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb971b0d0cc489c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb03e00640 1 -- 172.21.15.110:0/815900259 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00b0070 con 0x7feb041603c0 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb03e00640 1 -- 172.21.15.110:0/815900259 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf00ab600 con 0x7feb041603c0 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb03e00640 1 -- 172.21.15.110:0/815900259 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00ab9d0 con 0x7feb041603c0 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb0c000640 1 -- 172.21.15.110:0/815900259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb041603c0 msgr2=0x7feb040d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb0c000640 1 --2- 172.21.15.110:0/815900259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb041603c0 0x7feb040d0cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009870 tx=0x7feaf001c410 comp rx=0 tx=0).stop 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb0c000640 1 -- 172.21.15.110:0/815900259 shutdown_connections 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb0c000640 1 --2- 172.21.15.110:0/815900259 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb041603c0 0x7feb040d0cd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb0c000640 1 --2- 172.21.15.110:0/815900259 >> v2:172.21.15.145:3300/0 conn(0x7feb0415c960 0x7feb040b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb0c000640 1 -- 172.21.15.110:0/815900259 >> 172.21.15.110:0/815900259 conn(0x7feb040d25c0 msgr2=0x7feb040d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:51.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb0c000640 1 -- 172.21.15.110:0/815900259 shutdown_connections 2024-09-10T10:03:51.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.499+0000 7feb0c000640 1 -- 172.21.15.110:0/815900259 wait complete. 2024-09-10T10:03:51.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 Processor -- start 2024-09-10T10:03:51.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 -- start start 2024-09-10T10:03:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb040d04c0 0x7feb041558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb0415c960 0x7feb041536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb041603c0 0x7feb041569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb040d3150 con 0x7feb041603c0 2024-09-10T10:03:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb041611b0 con 0x7feb0415c960 2024-09-10T10:03:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feb040d32d0 con 0x7feb040d04c0 2024-09-10T10:03:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb041603c0 0x7feb041569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb041603c0 0x7feb041569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34944/0 (socket says 172.21.15.110:34944) 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 -- 172.21.15.110:0/1174832748 learned_addr learned my addr 172.21.15.110:0/1174832748 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb09600640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb040d04c0 0x7feb041558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb08c00640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb0415c960 0x7feb041536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 -- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb040d04c0 msgr2=0x7feb041558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb040d04c0 0x7feb041558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 -- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb0415c960 msgr2=0x7feb041536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb0415c960 0x7feb041536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 -- 172.21.15.110:0/1174832748 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0415bbb0 con 0x7feb041603c0 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0a000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb041603c0 0x7feb041569c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c780 tx=0x7feafc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb01600640 1 -- 172.21.15.110:0/1174832748 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc003db0 con 0x7feb041603c0 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb01600640 1 -- 172.21.15.110:0/1174832748 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feafc00ce30 con 0x7feb041603c0 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb01600640 1 -- 172.21.15.110:0/1174832748 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc005290 con 0x7feb041603c0 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0415be90 con 0x7feb041603c0 2024-09-10T10:03:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.503+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb041553c0 con 0x7feb041603c0 2024-09-10T10:03:51.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.507+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb04071a80 con 0x7feb041603c0 2024-09-10T10:03:51.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.507+0000 7feb01600640 1 -- 172.21.15.110:0/1174832748 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7feafc005430 con 0x7feb041603c0 2024-09-10T10:03:51.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.507+0000 7feb01600640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feae4067ce0 0x7feae406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:51.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.507+0000 7feb01600640 1 -- 172.21.15.110:0/1174832748 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7feafc085e40 con 0x7feb041603c0 2024-09-10T10:03:51.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.507+0000 7feb09600640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feae4067ce0 0x7feae406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:51.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.507+0000 7feb09600640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feae4067ce0 0x7feae406a1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009ac0 tx=0x7feaf8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:51.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.511+0000 7feb01600640 1 -- 172.21.15.110:0/1174832748 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feafc056080 con 0x7feb041603c0 2024-09-10T10:03:51.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:51.723+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7feb040d34d0 con 0x7feb041603c0 2024-09-10T10:03:52.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb01600640 1 -- 172.21.15.110:0/1174832748 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v300) ==== 151+0+0 (secure 0 0 0) 0x7feafc05a380 con 0x7feb041603c0 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feae4067ce0 msgr2=0x7feae406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feae4067ce0 0x7feae406a1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009ac0 tx=0x7feaf8009190 comp rx=0 tx=0).stop 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb041603c0 msgr2=0x7feb041569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb041603c0 0x7feb041569c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c780 tx=0x7feafc00cc40 comp rx=0 tx=0).stop 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 shutdown_connections 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feae4067ce0 0x7feae406a1a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb041603c0 0x7feb041569c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb0415c960 0x7feb041536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 --2- 172.21.15.110:0/1174832748 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feb040d04c0 0x7feb041558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.675+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 >> 172.21.15.110:0/1174832748 conn(0x7feb040d25c0 msgr2=0x7feb040b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.679+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 shutdown_connections 2024-09-10T10:03:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.679+0000 7feb0c000640 1 -- 172.21.15.110:0/1174832748 wait complete. 2024-09-10T10:03:52.701 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:03:52.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f888ee00640 1 Processor -- start 2024-09-10T10:03:52.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f888ee00640 1 -- start start 2024-09-10T10:03:52.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f888ee00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8888145f30 0x7f88880b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:52.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f888ee00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f88880b29e0 0x7f88880cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:52.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f888ee00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88880b37c0 con 0x7f88880cfbc0 2024-09-10T10:03:52.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f888ee00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f88880b2e20 con 0x7f88880b29e0 2024-09-10T10:03:52.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f888ee00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f88880b3aa0 con 0x7f8888145f30 2024-09-10T10:03:52.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f8887e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8888145f30 0x7f88880b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:52.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f8887e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8888145f30 0x7f88880b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:51716/0 (socket says 172.21.15.110:51716) 2024-09-10T10:03:52.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f8887e00640 1 -- 172.21.15.110:0/3248428065 learned_addr learned my addr 172.21.15.110:0/3248428065 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:52.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f888ce00640 1 --1- 172.21.15.110:0/3248428065 >> v1:172.21.15.145:6789/0 conn(0x7f88880cfbc0 0x7f8888160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:35378/0 (socket says 172.21.15.110:35378) 2024-09-10T10:03:52.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f8887400640 1 --2- 172.21.15.110:0/3248428065 >> v2:172.21.15.145:3300/0 conn(0x7f88880b29e0 0x7f88880cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:52.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f8887e00640 1 -- 172.21.15.110:0/3248428065 >> v2:172.21.15.145:3300/0 conn(0x7f88880b29e0 msgr2=0x7f88880cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:52.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f8887e00640 1 --2- 172.21.15.110:0/3248428065 >> v2:172.21.15.145:3300/0 conn(0x7f88880b29e0 0x7f88880cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f8887e00640 1 -- 172.21.15.110:0/3248428065 >> v1:172.21.15.145:6789/0 conn(0x7f88880cfbc0 legacy=0x7f8888160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:52.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.839+0000 7f8887e00640 1 -- 172.21.15.110:0/3248428065 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88880b3d80 con 0x7f8888145f30 2024-09-10T10:03:52.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f8887e00640 1 --2- 172.21.15.110:0/3248428065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8888145f30 0x7f88880b24a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8878009870 tx=0x7f887801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c430c7fbafcb1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:52.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f8886a00640 1 -- 172.21.15.110:0/3248428065 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88780b2070 con 0x7f8888145f30 2024-09-10T10:03:52.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f8886a00640 1 -- 172.21.15.110:0/3248428065 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88780a9070 con 0x7f8888145f30 2024-09-10T10:03:52.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f8886a00640 1 -- 172.21.15.110:0/3248428065 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88780ad880 con 0x7f8888145f30 2024-09-10T10:03:52.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f888ee00640 1 -- 172.21.15.110:0/3248428065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8888145f30 msgr2=0x7f88880b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:52.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f888ee00640 1 --2- 172.21.15.110:0/3248428065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8888145f30 0x7f88880b24a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8878009870 tx=0x7f887801c560 comp rx=0 tx=0).stop 2024-09-10T10:03:52.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f888ee00640 1 -- 172.21.15.110:0/3248428065 shutdown_connections 2024-09-10T10:03:52.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f888ee00640 1 --2- 172.21.15.110:0/3248428065 >> v2:172.21.15.145:3300/0 conn(0x7f88880b29e0 0x7f88880cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f888ee00640 1 --2- 172.21.15.110:0/3248428065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8888145f30 0x7f88880b24a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f888ee00640 1 -- 172.21.15.110:0/3248428065 >> 172.21.15.110:0/3248428065 conn(0x7f8888062e40 msgr2=0x7f888806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:52.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f888ee00640 1 -- 172.21.15.110:0/3248428065 shutdown_connections 2024-09-10T10:03:52.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.843+0000 7f888ee00640 1 -- 172.21.15.110:0/3248428065 wait complete. 2024-09-10T10:03:52.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 Processor -- start 2024-09-10T10:03:52.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 -- start start 2024-09-10T10:03:52.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88880b29e0 0x7f888815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:52.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88880cfbc0 0x7f888813df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:52.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888145f30 0x7f8888145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:52.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8888169530 con 0x7f8888145f30 2024-09-10T10:03:52.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f88881693b0 con 0x7f88880b29e0 2024-09-10T10:03:52.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f88881696b0 con 0x7f88880cfbc0 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888145f30 0x7f8888145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f8887400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88880cfbc0 0x7f888813df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f8887e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88880b29e0 0x7f888815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f8887400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88880cfbc0 0x7f888813df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:51732/0 (socket says 172.21.15.110:51732) 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f8887400640 1 -- 172.21.15.110:0/842835981 learned_addr learned my addr 172.21.15.110:0/842835981 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f8887e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88880b29e0 0x7f888815bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:60864/0 (socket says 172.21.15.110:60864) 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888145f30 0x7f8888145140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34954/0 (socket says 172.21.15.110:34954) 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ce00640 1 -- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88880cfbc0 msgr2=0x7f888813df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ce00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88880cfbc0 0x7f888813df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ce00640 1 -- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88880b29e0 msgr2=0x7f888815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ce00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88880b29e0 0x7f888815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ce00640 1 -- 172.21.15.110:0/842835981 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88880b0db0 con 0x7f8888145f30 2024-09-10T10:03:52.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f8887400640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88880cfbc0 0x7f888813df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:52.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f8887e00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88880b29e0 0x7f888815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:52.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ce00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888145f30 0x7f8888145140 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f888000c730 tx=0x7f888000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:52.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f887fe00640 1 -- 172.21.15.110:0/842835981 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880018070 con 0x7f8888145f30 2024-09-10T10:03:52.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f887fe00640 1 -- 172.21.15.110:0/842835981 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8880004d70 con 0x7f8888145f30 2024-09-10T10:03:52.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88880b1090 con 0x7f8888145f30 2024-09-10T10:03:52.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f887fe00640 1 -- 172.21.15.110:0/842835981 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888000a860 con 0x7f8888145f30 2024-09-10T10:03:52.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.847+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88880b22d0 con 0x7f8888145f30 2024-09-10T10:03:52.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.851+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88880cf680 con 0x7f8888145f30 2024-09-10T10:03:52.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.851+0000 7f887fe00640 1 -- 172.21.15.110:0/842835981 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f888001b070 con 0x7f8888145f30 2024-09-10T10:03:52.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.851+0000 7f887fe00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8868067cb0 0x7f886806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:52.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.851+0000 7f887fe00640 1 -- 172.21.15.110:0/842835981 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f88800856b0 con 0x7f8888145f30 2024-09-10T10:03:52.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.851+0000 7f8887e00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8868067cb0 0x7f886806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:52.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.851+0000 7f8887e00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8868067cb0 0x7f886806a170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f88780af000 tx=0x7f8878008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:52.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:52.855+0000 7f887fe00640 1 -- 172.21.15.110:0/842835981 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f88800558f0 con 0x7f8888145f30 2024-09-10T10:03:53.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.011+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f888813f9c0 con 0x7f8888145f30 2024-09-10T10:03:53.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.015+0000 7f887fe00640 1 -- 172.21.15.110:0/842835981 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f888005c110 con 0x7f8888145f30 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8868067cb0 msgr2=0x7f886806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8868067cb0 0x7f886806a170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f88780af000 tx=0x7f8878008040 comp rx=0 tx=0).stop 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888145f30 msgr2=0x7f8888145140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888145f30 0x7f8888145140 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f888000c730 tx=0x7f888000cbf0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 shutdown_connections 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8868067cb0 0x7f886806a170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8888145f30 0x7f8888145140 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88880cfbc0 0x7f888813df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 --2- 172.21.15.110:0/842835981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88880b29e0 0x7f888815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 >> 172.21.15.110:0/842835981 conn(0x7f8888062e40 msgr2=0x7f88880725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 shutdown_connections 2024-09-10T10:03:53.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.019+0000 7f888ee00640 1 -- 172.21.15.110:0/842835981 wait complete. 2024-09-10T10:03:53.041 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:03:53.041 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:03:53.135 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:03:53.220 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:03:53.221 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:53.221 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:53.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:03:53.229 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:03:53.230 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:53.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:03:53.239 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:03:53.239 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:53.280 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:03:53.280 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:03:53.280 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:53.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:03:53.290 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:03:53.290 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:03:53.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f0790200640 1 Processor -- start 2024-09-10T10:03:53.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f0790200640 1 -- start start 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f0790200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078806f1e0 0x7f0788166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f0790200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0788155d50 con 0x7f0788155970 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f0790200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07880b34d0 con 0x7f078815c560 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f0790200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f07880b4570 con 0x7f078806f1e0 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f078d800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0788155970 0x7f07880cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47824/0 (socket says 172.21.15.110:47824) 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f078d800640 1 -- 172.21.15.110:0/3213928736 learned_addr learned my addr 172.21.15.110:0/3213928736 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.455+0000 7f078e200640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078806f1e0 0x7f0788166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1084100678 0 0) 0x7f0788155d50 con 0x7f0788155970 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0770005740 con 0x7f0788155970 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387528691 0 0) 0x7f07880b34d0 con 0x7f078815c560 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0788155d50 con 0x7f078815c560 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 902175493 0 0) 0x7f0770005740 con 0x7f0788155970 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078806f1e0 msgr2=0x7f0788166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078806f1e0 0x7f0788166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 >> v1:172.21.15.145:6789/0 conn(0x7f078815c560 legacy=0x7f078806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07880b37b0 con 0x7f0788155970 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f077c001a20 con 0x7f0788155970 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 >> v1:172.21.15.110:6789/0 conn(0x7f0788155970 legacy=0x7f07880cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f0770006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0770006a20 0x7f0770006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078806f1e0 0x7f07700073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f077000dc90 con 0x7f0770005d90 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f077000de40 con 0x7f0770006a20 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f077000e040 con 0x7f078806f1e0 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f078d800640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f0770006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f078ce00640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0770006a20 0x7f0770006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f078e200640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078806f1e0 0x7f07700073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f078d800640 1 -- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078806f1e0 msgr2=0x7f07700073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f078d800640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078806f1e0 0x7f07700073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f078d800640 1 -- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0770006a20 msgr2=0x7f0770006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f078d800640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0770006a20 0x7f0770006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.459+0000 7f078d800640 1 -- 172.21.15.110:0/3213928736 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0770005740 con 0x7f0770005d90 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f078d800640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f0770006190 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f07880b4dd0 tx=0x7f077c01d210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296acf92211863fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c00a3b0 con 0x7f0770005d90 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c0ad630 con 0x7f0770005d90 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0787e00640 1 -- 172.21.15.110:0/3213928736 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c0ada20 con 0x7f0770005d90 2024-09-10T10:03:53.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- 172.21.15.110:0/3213928736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 msgr2=0x7f0770006190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 --2- 172.21.15.110:0/3213928736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f0770006190 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f07880b4dd0 tx=0x7f077c01d210 comp rx=0 tx=0).stop 2024-09-10T10:03:53.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f078e200640 1 -- 172.21.15.110:0/3213928736 reap_dead start 2024-09-10T10:03:53.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- 172.21.15.110:0/3213928736 shutdown_connections 2024-09-10T10:03:53.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- 172.21.15.110:0/3213928736 >> 172.21.15.110:0/3213928736 conn(0x7f078806bc50 msgr2=0x7f078806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- 172.21.15.110:0/3213928736 shutdown_connections 2024-09-10T10:03:53.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- 172.21.15.110:0/3213928736 wait complete. 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 Processor -- start 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- start start 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f07881480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07881471b0 0x7f0788146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078814c9f0 0x7f07880b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f078806e350 con 0x7f0770005d90 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f078806e1d0 con 0x7f07881471b0 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f0790200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f078806e4d0 con 0x7f078814c9f0 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f078d800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f07881480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f078d800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f07881480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34980/0 (socket says 172.21.15.110:34980) 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.463+0000 7f078d800640 1 -- 172.21.15.110:0/1692042380 learned_addr learned my addr 172.21.15.110:0/1692042380 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f078ce00640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07881471b0 0x7f0788146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f078e200640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078814c9f0 0x7f07880b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f078d800640 1 -- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078814c9f0 msgr2=0x7f07880b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f078d800640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078814c9f0 0x7f07880b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f078d800640 1 -- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07881471b0 msgr2=0x7f0788146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f078d800640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07881471b0 0x7f0788146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f078d800640 1 -- 172.21.15.110:0/1692042380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078814bc30 con 0x7f0770005d90 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f078d800640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f07881480a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f077c01c180 tx=0x7f0770007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0785600640 1 -- 172.21.15.110:0/1692042380 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c0bd070 con 0x7f0770005d90 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0785600640 1 -- 172.21.15.110:0/1692042380 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c001820 con 0x7f0770005d90 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0785600640 1 -- 172.21.15.110:0/1692042380 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c0b2040 con 0x7f0770005d90 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078814bf10 con 0x7f0770005d90 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f078814b920 con 0x7f0770005d90 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07880cc8c0 con 0x7f0770005d90 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0785600640 1 -- 172.21.15.110:0/1692042380 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f077c0c0070 con 0x7f0770005d90 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0785600640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0768067cb0 0x7f076806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.467+0000 7f0785600640 1 -- 172.21.15.110:0/1692042380 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f077c120710 con 0x7f0770005d90 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.471+0000 7f078ce00640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0768067cb0 0x7f076806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.471+0000 7f078ce00640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0768067cb0 0x7f076806a170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f07880b2eb0 tx=0x7f0774009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.475+0000 7f0785600640 1 -- 172.21.15.110:0/1692042380 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f077c016f50 con 0x7f0770005d90 2024-09-10T10:03:53.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.663+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f078806ec60 con 0x7f0770005d90 2024-09-10T10:03:53.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0785600640 1 -- 172.21.15.110:0/1692042380 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+3911 (secure 0 0 0) 0x7f077c0f0840 con 0x7f0770005d90 2024-09-10T10:03:53.669 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:53.669 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":71,"btime":"2024-09-10T10:03:47:657089+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/59288243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":59288243},{"type":"v1","addr":"172.21.15.145:6834","nonce":59288243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6427,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2534764627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2534764627},{"type":"v1","addr":"172.21.15.145:6835","nonce":2534764627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6798,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/3565143399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":3565143399},{"type":"v1","addr":"172.21.15.110:6834","nonce":3565143399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/69128020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":69128020},{"type":"v1","addr":"172.21.15.110:6835","nonce":69128020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[]} 2024-09-10T10:03:53.669 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 71 2024-09-10T10:03:53.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0768067cb0 msgr2=0x7f076806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0768067cb0 0x7f076806a170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f07880b2eb0 tx=0x7f0774009040 comp rx=0 tx=0).stop 2024-09-10T10:03:53.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 msgr2=0x7f07881480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f07881480a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f077c01c180 tx=0x7f0770007a50 comp rx=0 tx=0).stop 2024-09-10T10:03:53.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 shutdown_connections 2024-09-10T10:03:53.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0768067cb0 0x7f076806a170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f078814c9f0 0x7f07880b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07881471b0 0x7f0788146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 --2- 172.21.15.110:0/1692042380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0770005d90 0x7f07881480a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 >> 172.21.15.110:0/1692042380 conn(0x7f078806bc50 msgr2=0x7f07881547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 shutdown_connections 2024-09-10T10:03:53.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.667+0000 7f0790200640 1 -- 172.21.15.110:0/1692042380 wait complete. 2024-09-10T10:03:53.691 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:03:53.691 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:03:53.691 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:03:53.692 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:03:53.692 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:03:53.692 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:03:53.692 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:03:53.692 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:03:53.692 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:03:53.692 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:03:53.693 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:03:53.693 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:03:53.693 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:03:53.693 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:03:53.693 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:03:53.693 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:03:53.703 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:03:53.704 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:03:53.705 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:03:53.705 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:03:53.707 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:03:53.708 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:03:53.709 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:03:53.709 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:03:53.752 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:03:53.753 DEBUG:teuthology.parallel:result is None 2024-09-10T10:03:53.754 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:03:53.754 DEBUG:teuthology.parallel:result is None 2024-09-10T10:03:53.755 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:03:53.755 DEBUG:teuthology.parallel:result is None 2024-09-10T10:03:53.757 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:03:53.757 DEBUG:teuthology.parallel:result is None 2024-09-10T10:03:53.757 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:03:53.840 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:03:53.842 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:03:53.843 INFO:tasks.ceph.mds.a.smithi110.stderr:2024-09-10T10:03:53.839+0000 7f1cf9a00640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6833/0: (98) Address already in use 2024-09-10T10:03:53.844 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:03:53.845 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:03:53.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c03400640 1 Processor -- start 2024-09-10T10:03:53.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c03400640 1 -- start start 2024-09-10T10:03:53.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c03400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3bfc0b3f40 0x7f3bfc0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c03400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0cff70 0x7f3bfc0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c03400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bfc15d4d0 con 0x7f3bfc0cfb90 2024-09-10T10:03:53.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c03400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3bfc06d130 con 0x7f3bfc0b3f40 2024-09-10T10:03:53.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c03400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3bfc15d7b0 con 0x7f3bfc0cff70 2024-09-10T10:03:53.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c00a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3bfc0cfb90 0x7f3bfc15ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47836/0 (socket says 172.21.15.110:47836) 2024-09-10T10:03:53.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c00a00640 1 -- 172.21.15.110:0/3332385674 learned_addr learned my addr 172.21.15.110:0/3332385674 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:53.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3bfbe00640 1 --2- 172.21.15.110:0/3332385674 >> v2:172.21.15.110:3300/0 conn(0x7f3bfc0b3f40 0x7f3bfc0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c01400640 1 --2- 172.21.15.110:0/3332385674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0cff70 0x7f3bfc0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3bfbe00640 1 -- 172.21.15.110:0/3332385674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0cff70 msgr2=0x7f3bfc0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3bfbe00640 1 --2- 172.21.15.110:0/3332385674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0cff70 0x7f3bfc0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3bfbe00640 1 -- 172.21.15.110:0/3332385674 >> v1:172.21.15.110:6789/0 conn(0x7f3bfc0cfb90 legacy=0x7f3bfc15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3bfbe00640 1 -- 172.21.15.110:0/3332385674 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc06d380 con 0x7f3bfc0b3f40 2024-09-10T10:03:53.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.899+0000 7f3c01400640 1 --2- 172.21.15.110:0/3332385674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0cff70 0x7f3bfc0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:53.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3bfbe00640 1 --2- 172.21.15.110:0/3332385674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0b3f40 0x7f3bfc0cd5a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097c0 tx=0x7f3be801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9b79a90a351102 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3bfb400640 1 -- 172.21.15.110:0/3332385674 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be80b0070 con 0x7f3bfc0b3f40 2024-09-10T10:03:53.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3bfb400640 1 -- 172.21.15.110:0/3332385674 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be80ab600 con 0x7f3bfc0b3f40 2024-09-10T10:03:53.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3bfb400640 1 -- 172.21.15.110:0/3332385674 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be80ab9d0 con 0x7f3bfc0b3f40 2024-09-10T10:03:53.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3c03400640 1 -- 172.21.15.110:0/3332385674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0b3f40 msgr2=0x7f3bfc0cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3c03400640 1 --2- 172.21.15.110:0/3332385674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0b3f40 0x7f3bfc0cd5a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097c0 tx=0x7f3be801c410 comp rx=0 tx=0).stop 2024-09-10T10:03:53.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3c03400640 1 -- 172.21.15.110:0/3332385674 shutdown_connections 2024-09-10T10:03:53.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3c03400640 1 --2- 172.21.15.110:0/3332385674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0cff70 0x7f3bfc0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3c03400640 1 --2- 172.21.15.110:0/3332385674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0b3f40 0x7f3bfc0cd5a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3c03400640 1 -- 172.21.15.110:0/3332385674 >> 172.21.15.110:0/3332385674 conn(0x7f3bfc062d10 msgr2=0x7f3bfc06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3c03400640 1 -- 172.21.15.110:0/3332385674 shutdown_connections 2024-09-10T10:03:53.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.903+0000 7f3c03400640 1 -- 172.21.15.110:0/3332385674 wait complete. 2024-09-10T10:03:53.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 Processor -- start 2024-09-10T10:03:53.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 -- start start 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0b3f40 0x7f3bfc1f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bfc0cfb90 0x7f3bfc0aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0cff70 0x7f3bfc0aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc06e720 con 0x7f3bfc0cff70 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc06e5a0 con 0x7f3bfc0cfb90 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3bfc06e8a0 con 0x7f3bfc0b3f40 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0cff70 0x7f3bfc0aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0cff70 0x7f3bfc0aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35020/0 (socket says 172.21.15.110:35020) 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 -- 172.21.15.110:0/1716910778 learned_addr learned my addr 172.21.15.110:0/1716910778 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:53.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c00a00640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0b3f40 0x7f3bfc1f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3bfbe00640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bfc0cfb90 0x7f3bfc0aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 -- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0b3f40 msgr2=0x7f3bfc1f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0b3f40 0x7f3bfc1f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 -- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bfc0cfb90 msgr2=0x7f3bfc0aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bfc0cfb90 0x7f3bfc0aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 -- 172.21.15.110:0/1716910778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc0ab2b0 con 0x7f3bfc0cff70 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c01400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0cff70 0x7f3bfc0aa970 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400e950 tx=0x7f3bf400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3bf8c00640 1 -- 172.21.15.110:0/1716910778 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4017070 con 0x7f3bfc0cff70 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3bf8c00640 1 -- 172.21.15.110:0/1716910778 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bf400ca00 con 0x7f3bfc0cff70 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bfc1f6180 con 0x7f3bfc0cff70 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3bf8c00640 1 -- 172.21.15.110:0/1716910778 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf400cdd0 con 0x7f3bfc0cff70 2024-09-10T10:03:53.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.907+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bfc1f6740 con 0x7f3bfc0cff70 2024-09-10T10:03:53.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.911+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bfc15ca10 con 0x7f3bfc0cff70 2024-09-10T10:03:53.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.911+0000 7f3bf8c00640 1 -- 172.21.15.110:0/1716910778 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f3bf4013470 con 0x7f3bfc0cff70 2024-09-10T10:03:53.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.911+0000 7f3bf8c00640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bdc067cb0 0x7f3bdc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.911+0000 7f3bf8c00640 1 -- 172.21.15.110:0/1716910778 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f3bf4085bb0 con 0x7f3bfc0cff70 2024-09-10T10:03:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.911+0000 7f3c00a00640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bdc067cb0 0x7f3bdc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.911+0000 7f3c00a00640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bdc067cb0 0x7f3bdc06a170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0cc990 tx=0x7f3bec001a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:53.915+0000 7f3bf8c00640 1 -- 172.21.15.110:0/1716910778 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3bf4055df0 con 0x7f3bfc0cff70 2024-09-10T10:03:54.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.083+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3bfc0cdbb0 con 0x7f3bfc0cff70 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3bf8c00640 1 -- 172.21.15.110:0/1716910778 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) ==== 97+0+0 (secure 0 0 0) 0x7f3bf405a0f0 con 0x7f3bfc0cff70 2024-09-10T10:03:54.697 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bdc067cb0 msgr2=0x7f3bdc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bdc067cb0 0x7f3bdc06a170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0cc990 tx=0x7f3bec001a40 comp rx=0 tx=0).stop 2024-09-10T10:03:54.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0cff70 msgr2=0x7f3bfc0aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0cff70 0x7f3bfc0aa970 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400e950 tx=0x7f3bf400ee10 comp rx=0 tx=0).stop 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 shutdown_connections 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bdc067cb0 0x7f3bdc06a170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bfc0cff70 0x7f3bfc0aa970 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bfc0cfb90 0x7f3bfc0aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 --2- 172.21.15.110:0/1716910778 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3bfc0b3f40 0x7f3bfc1f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 >> 172.21.15.110:0/1716910778 conn(0x7f3bfc062d10 msgr2=0x7f3bfc0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 shutdown_connections 2024-09-10T10:03:54.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.695+0000 7f3c03400640 1 -- 172.21.15.110:0/1716910778 wait complete. 2024-09-10T10:03:54.726 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:03:54.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.859+0000 7f691f000640 1 Processor -- start 2024-09-10T10:03:54.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.859+0000 7f691f000640 1 -- start start 2024-09-10T10:03:54.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f691f000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6918155970 0x7f69180cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f691f000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f691806f1e0 0x7f69180ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f691f000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6918155d50 con 0x7f691815c560 2024-09-10T10:03:54.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f691f000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f69180b4570 con 0x7f691806f1e0 2024-09-10T10:03:54.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f691f000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f69180b34d0 con 0x7f6918155970 2024-09-10T10:03:54.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f6917e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6918155970 0x7f69180cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f691d000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f691806f1e0 0x7f69180ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f6917400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f691815c560 0x7f691806ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:35414/0 (socket says 172.21.15.110:35414) 2024-09-10T10:03:54.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f6917400640 1 -- 172.21.15.110:0/1965725552 learned_addr learned my addr 172.21.15.110:0/1965725552 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:54.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f6917e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6918155970 0x7f69180cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:51860/0 (socket says 172.21.15.110:51860) 2024-09-10T10:03:54.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f6917e00640 1 -- 172.21.15.110:0/1965725552 >> v2:172.21.15.145:3300/0 conn(0x7f691806f1e0 msgr2=0x7f69180ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f6917e00640 1 --2- 172.21.15.110:0/1965725552 >> v2:172.21.15.145:3300/0 conn(0x7f691806f1e0 0x7f69180ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f6917e00640 1 -- 172.21.15.110:0/1965725552 >> v1:172.21.15.145:6789/0 conn(0x7f691815c560 legacy=0x7f691806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.863+0000 7f6917e00640 1 -- 172.21.15.110:0/1965725552 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69180b37b0 con 0x7f6918155970 2024-09-10T10:03:54.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f6917e00640 1 --2- 172.21.15.110:0/1965725552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6918155970 0x7f69180cc8c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f69080097c0 tx=0x7f690801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc305a746456da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f6916a00640 1 -- 172.21.15.110:0/1965725552 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080b1070 con 0x7f6918155970 2024-09-10T10:03:54.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f6916a00640 1 -- 172.21.15.110:0/1965725552 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69080a8050 con 0x7f6918155970 2024-09-10T10:03:54.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f6916a00640 1 -- 172.21.15.110:0/1965725552 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080ac860 con 0x7f6918155970 2024-09-10T10:03:54.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 -- 172.21.15.110:0/1965725552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6918155970 msgr2=0x7f69180cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 --2- 172.21.15.110:0/1965725552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6918155970 0x7f69180cc8c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f69080097c0 tx=0x7f690801c3e0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 -- 172.21.15.110:0/1965725552 shutdown_connections 2024-09-10T10:03:54.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 --2- 172.21.15.110:0/1965725552 >> v2:172.21.15.145:3300/0 conn(0x7f691806f1e0 0x7f69180ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 --2- 172.21.15.110:0/1965725552 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6918155970 0x7f69180cc8c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 -- 172.21.15.110:0/1965725552 >> 172.21.15.110:0/1965725552 conn(0x7f691806bc50 msgr2=0x7f691806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 -- 172.21.15.110:0/1965725552 shutdown_connections 2024-09-10T10:03:54.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 -- 172.21.15.110:0/1965725552 wait complete. 2024-09-10T10:03:54.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 Processor -- start 2024-09-10T10:03:54.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.867+0000 7f691f000640 1 -- start start 2024-09-10T10:03:54.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f691806f1e0 0x7f691813a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6918155970 0x7f691815cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f691815c560 0x7f69180ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69180cf170 con 0x7f6918155970 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f69180cce00 con 0x7f691815c560 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f69180cf2f0 con 0x7f691806f1e0 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6918155970 0x7f691815cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6918155970 0x7f691815cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35024/0 (socket says 172.21.15.110:35024) 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 -- 172.21.15.110:0/3719204840 learned_addr learned my addr 172.21.15.110:0/3719204840 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691d000640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f691815c560 0x7f69180ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917e00640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f691806f1e0 0x7f691813a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 -- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f691806f1e0 msgr2=0x7f691813a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f691806f1e0 0x7f691813a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 -- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f691815c560 msgr2=0x7f69180ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f691815c560 0x7f69180ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 -- 172.21.15.110:0/3719204840 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691815e750 con 0x7f6918155970 2024-09-10T10:03:54.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f6917400640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6918155970 0x7f691815cbe0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f690400c370 tx=0x7f690400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f690fe00640 1 -- 172.21.15.110:0/3719204840 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904017070 con 0x7f6918155970 2024-09-10T10:03:54.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f690fe00640 1 -- 172.21.15.110:0/3719204840 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6904005cf0 con 0x7f6918155970 2024-09-10T10:03:54.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691815e9f0 con 0x7f6918155970 2024-09-10T10:03:54.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f690fe00640 1 -- 172.21.15.110:0/3719204840 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69040072f0 con 0x7f6918155970 2024-09-10T10:03:54.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691815efb0 con 0x7f6918155970 2024-09-10T10:03:54.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69180ce420 con 0x7f6918155970 2024-09-10T10:03:54.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.871+0000 7f690fe00640 1 -- 172.21.15.110:0/3719204840 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f690401a070 con 0x7f6918155970 2024-09-10T10:03:54.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.875+0000 7f690fe00640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68f8067cb0 0x7f68f806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.875+0000 7f690fe00640 1 -- 172.21.15.110:0/3719204840 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4885+0+0 (secure 0 0 0) 0x7f6904085390 con 0x7f6918155970 2024-09-10T10:03:54.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.875+0000 7f6917e00640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68f8067cb0 0x7f68f806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.875+0000 7f6917e00640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68f8067cb0 0x7f68f806a170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f69080ae000 tx=0x7f6908008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:54.879+0000 7f690fe00640 1 -- 172.21.15.110:0/3719204840 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6904055560 con 0x7f6918155970 2024-09-10T10:03:55.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f691806eb40 con 0x7f6918155970 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f690fe00640 1 -- 172.21.15.110:0/3719204840 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f6904059860 con 0x7f6918155970 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:55.094 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:55.095 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:55.096 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:03:55.098 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:55.099 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.100 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:55.101 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:03:55.103 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:03:55.104 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:03:55.105 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:03:55.108 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:03:55.109 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:03:55.109 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:03:55.109 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:03:55.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.109 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:03:55.109 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:03:55.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68f8067cb0 msgr2=0x7f68f806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68f8067cb0 0x7f68f806a170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f69080ae000 tx=0x7f6908008040 comp rx=0 tx=0).stop 2024-09-10T10:03:55.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6918155970 msgr2=0x7f691815cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6918155970 0x7f691815cbe0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f690400c370 tx=0x7f690400c830 comp rx=0 tx=0).stop 2024-09-10T10:03:55.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 shutdown_connections 2024-09-10T10:03:55.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68f8067cb0 0x7f68f806a170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f691815c560 0x7f69180ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6918155970 0x7f691815cbe0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 --2- 172.21.15.110:0/3719204840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f691806f1e0 0x7f691813a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 >> 172.21.15.110:0/3719204840 conn(0x7f691806bc50 msgr2=0x7f69180710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:55.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 shutdown_connections 2024-09-10T10:03:55.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.091+0000 7f691f000640 1 -- 172.21.15.110:0/3719204840 wait complete. 2024-09-10T10:03:55.117 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:03:55.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.247+0000 7f529d400640 1 Processor -- start 2024-09-10T10:03:55.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.247+0000 7f529d400640 1 -- start start 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f529d400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5298155970 0x7f52980cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f529d400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f529815c560 0x7f529806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f529d400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5298155d50 con 0x7f529806f010 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f529d400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f52980b4570 con 0x7f529815c560 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f529d400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f529806f450 con 0x7f5298155970 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5295e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f529815c560 0x7f529806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5295e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f529815c560 0x7f529806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35040/0 (socket says 172.21.15.110:35040) 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5295e00640 1 -- 172.21.15.110:0/617062315 learned_addr learned my addr 172.21.15.110:0/617062315 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:55.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5295e00640 1 -- 172.21.15.110:0/617062315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5298155970 msgr2=0x7f52980cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5296800640 1 --2- 172.21.15.110:0/617062315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5298155970 0x7f52980cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5295e00640 1 --2- 172.21.15.110:0/617062315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5298155970 0x7f52980cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5295e00640 1 -- 172.21.15.110:0/617062315 >> v1:172.21.15.145:6789/0 conn(0x7f529806f010 legacy=0x7f529815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5295e00640 1 -- 172.21.15.110:0/617062315 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52980b34d0 con 0x7f529815c560 2024-09-10T10:03:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.251+0000 7f5296800640 1 --2- 172.21.15.110:0/617062315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5298155970 0x7f52980cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:55.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5295e00640 1 --2- 172.21.15.110:0/617062315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529806ead0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f529806f5d0 tx=0x7f528401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a220e3c3b2119481 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:55.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5295400640 1 -- 172.21.15.110:0/617062315 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52840ab680 con 0x7f529815c560 2024-09-10T10:03:55.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5295400640 1 -- 172.21.15.110:0/617062315 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52840b0070 con 0x7f529815c560 2024-09-10T10:03:55.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5295400640 1 -- 172.21.15.110:0/617062315 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52840aba40 con 0x7f529815c560 2024-09-10T10:03:55.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- 172.21.15.110:0/617062315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 msgr2=0x7f529806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 --2- 172.21.15.110:0/617062315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529806ead0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f529806f5d0 tx=0x7f528401c480 comp rx=0 tx=0).stop 2024-09-10T10:03:55.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- 172.21.15.110:0/617062315 shutdown_connections 2024-09-10T10:03:55.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 --2- 172.21.15.110:0/617062315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529806ead0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 --2- 172.21.15.110:0/617062315 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5298155970 0x7f52980cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- 172.21.15.110:0/617062315 >> 172.21.15.110:0/617062315 conn(0x7f529806bc50 msgr2=0x7f529806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:55.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- 172.21.15.110:0/617062315 shutdown_connections 2024-09-10T10:03:55.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- 172.21.15.110:0/617062315 wait complete. 2024-09-10T10:03:55.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 Processor -- start 2024-09-10T10:03:55.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- start start 2024-09-10T10:03:55.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529806f010 0x7f529815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5298155970 0x7f5298168b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52980cce00 con 0x7f529815c560 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52980b4050 con 0x7f5298155970 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f529d400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5298166020 con 0x7f529806f010 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5297200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5296800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529806f010 0x7f529815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5297200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529815ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35048/0 (socket says 172.21.15.110:35048) 2024-09-10T10:03:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5297200640 1 -- 172.21.15.110:0/3879120307 learned_addr learned my addr 172.21.15.110:0/3879120307 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.255+0000 7f5295e00640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5298155970 0x7f5298168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f5297200640 1 -- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529806f010 msgr2=0x7f529815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f5297200640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529806f010 0x7f529815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f5297200640 1 -- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5298155970 msgr2=0x7f5298168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f5297200640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5298155970 0x7f5298168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f5297200640 1 -- 172.21.15.110:0/3879120307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5298167030 con 0x7f529815c560 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f5297200640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529815ca50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f529000c7a0 tx=0x7f529000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f528ea00640 1 -- 172.21.15.110:0/3879120307 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290003b80 con 0x7f529815c560 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f528ea00640 1 -- 172.21.15.110:0/3879120307 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5290003d20 con 0x7f529815c560 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f528ea00640 1 -- 172.21.15.110:0/3879120307 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529000aa90 con 0x7f529815c560 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529815d1a0 con 0x7f529815c560 2024-09-10T10:03:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5298153030 con 0x7f529815c560 2024-09-10T10:03:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52980cc8c0 con 0x7f529815c560 2024-09-10T10:03:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f528ea00640 1 -- 172.21.15.110:0/3879120307 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f529000ac30 con 0x7f529815c560 2024-09-10T10:03:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f528ea00640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5278067cb0 0x7f527806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f528ea00640 1 -- 172.21.15.110:0/3879120307 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4885+0+0 (secure 0 0 0) 0x7f5290085e90 con 0x7f529815c560 2024-09-10T10:03:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.259+0000 7f5296800640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5278067cb0 0x7f527806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.263+0000 7f5296800640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5278067cb0 0x7f527806a170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f528800ab60 tx=0x7f5288008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.263+0000 7f528ea00640 1 -- 172.21.15.110:0/3879120307 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5298167030 con 0x7f529815c560 2024-09-10T10:03:55.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.427+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f529806e5c0 con 0x7f529815c560 2024-09-10T10:03:55.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.427+0000 7f528ea00640 1 -- 172.21.15.110:0/3879120307 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) ==== 84+0+65 (secure 0 0 0) 0x7f5290056090 con 0x7f529815c560 2024-09-10T10:03:55.431 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:55.432 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:03:55.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:03:55.432 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:03:55.432 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:03:55.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:03:55.432 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:03:55.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5278067cb0 msgr2=0x7f527806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5278067cb0 0x7f527806a170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f528800ab60 tx=0x7f5288008040 comp rx=0 tx=0).stop 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 msgr2=0x7f529815ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529815ca50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f529000c7a0 tx=0x7f529000cc60 comp rx=0 tx=0).stop 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 shutdown_connections 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5278067cb0 0x7f527806a170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f529815c560 0x7f529815ca50 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5298155970 0x7f5298168b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 --2- 172.21.15.110:0/3879120307 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f529806f010 0x7f529815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 >> 172.21.15.110:0/3879120307 conn(0x7f529806bc50 msgr2=0x7f5298166450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 shutdown_connections 2024-09-10T10:03:55.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.431+0000 7f529d400640 1 -- 172.21.15.110:0/3879120307 wait complete. 2024-09-10T10:03:55.452 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:03:55.453 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:03:55.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3263a00640 1 Processor -- start 2024-09-10T10:03:55.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3263a00640 1 -- start start 2024-09-10T10:03:55.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3263a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f325c15e700 0x7f325c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3263a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f325c06ff60 0x7f325c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3263a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f325c0703a0 con 0x7f325c0729b0 2024-09-10T10:03:55.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3263a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f325c15eae0 con 0x7f325c15e700 2024-09-10T10:03:55.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3263a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f325c06eec0 con 0x7f325c06ff60 2024-09-10T10:03:55.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3261000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f325c15e700 0x7f325c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3261000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f325c15e700 0x7f325c06e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35052/0 (socket says 172.21.15.110:35052) 2024-09-10T10:03:55.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.587+0000 7f3261000640 1 -- 172.21.15.110:0/800912235 learned_addr learned my addr 172.21.15.110:0/800912235 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:55.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f325be00640 1 --2- 172.21.15.110:0/800912235 >> v2:172.21.15.145:3300/0 conn(0x7f325c06ff60 0x7f325c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f3261000640 1 -- 172.21.15.110:0/800912235 >> v2:172.21.15.145:3300/0 conn(0x7f325c06ff60 msgr2=0x7f325c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:55.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f3261000640 1 --2- 172.21.15.110:0/800912235 >> v2:172.21.15.145:3300/0 conn(0x7f325c06ff60 0x7f325c0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f3261000640 1 -- 172.21.15.110:0/800912235 >> v1:172.21.15.110:6789/0 conn(0x7f325c0729b0 legacy=0x7f325c072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f3261000640 1 -- 172.21.15.110:0/800912235 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325c06f1a0 con 0x7f325c15e700 2024-09-10T10:03:55.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f3261000640 1 --2- 172.21.15.110:0/800912235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c15e700 0x7f325c06e890 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f325c06e040 tx=0x7f324c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac457bb85a67eabb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:55.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f325b400640 1 -- 172.21.15.110:0/800912235 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0a5880 con 0x7f325c15e700 2024-09-10T10:03:55.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f325b400640 1 -- 172.21.15.110:0/800912235 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324c0a5a20 con 0x7f325c15e700 2024-09-10T10:03:55.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.591+0000 7f325b400640 1 -- 172.21.15.110:0/800912235 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0a5df0 con 0x7f325c15e700 2024-09-10T10:03:55.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- 172.21.15.110:0/800912235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c15e700 msgr2=0x7f325c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 --2- 172.21.15.110:0/800912235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c15e700 0x7f325c06e890 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f325c06e040 tx=0x7f324c01c9c0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- 172.21.15.110:0/800912235 shutdown_connections 2024-09-10T10:03:55.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 --2- 172.21.15.110:0/800912235 >> v2:172.21.15.145:3300/0 conn(0x7f325c06ff60 0x7f325c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 --2- 172.21.15.110:0/800912235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c15e700 0x7f325c06e890 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- 172.21.15.110:0/800912235 >> 172.21.15.110:0/800912235 conn(0x7f325c06cb90 msgr2=0x7f325c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:55.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- 172.21.15.110:0/800912235 shutdown_connections 2024-09-10T10:03:55.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- 172.21.15.110:0/800912235 wait complete. 2024-09-10T10:03:55.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 Processor -- start 2024-09-10T10:03:55.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- start start 2024-09-10T10:03:55.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f325c06ff60 0x7f325c14af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c0729b0 0x7f325c154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f325c15e700 0x7f325c154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f325c0af930 con 0x7f325c0729b0 2024-09-10T10:03:55.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f325c0af7b0 con 0x7f325c15e700 2024-09-10T10:03:55.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f3263a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f325c0b4660 con 0x7f325c06ff60 2024-09-10T10:03:55.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c0729b0 0x7f325c154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c0729b0 0x7f325c154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35066/0 (socket says 172.21.15.110:35066) 2024-09-10T10:03:55.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 -- 172.21.15.110:0/3958858705 learned_addr learned my addr 172.21.15.110:0/3958858705 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:55.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 -- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f325c06ff60 msgr2=0x7f325c14af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f325c06ff60 0x7f325c14af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 -- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f325c15e700 msgr2=0x7f325c154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:55.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f325c15e700 0x7f325c154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:55.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 -- 172.21.15.110:0/3958858705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325c14b730 con 0x7f325c0729b0 2024-09-10T10:03:55.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.595+0000 7f325be00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c0729b0 0x7f325c154430 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f324800e960 tx=0x7f324800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3258c00640 1 -- 172.21.15.110:0/3958858705 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248017070 con 0x7f325c0729b0 2024-09-10T10:03:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3258c00640 1 -- 172.21.15.110:0/3958858705 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324800ca00 con 0x7f325c0729b0 2024-09-10T10:03:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3258c00640 1 -- 172.21.15.110:0/3958858705 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324800cdb0 con 0x7f325c0729b0 2024-09-10T10:03:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325c0cf5a0 con 0x7f325c0729b0 2024-09-10T10:03:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325c14b450 con 0x7f325c0729b0 2024-09-10T10:03:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f325c0b4990 con 0x7f325c0729b0 2024-09-10T10:03:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3258c00640 1 -- 172.21.15.110:0/3958858705 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f324801a070 con 0x7f325c0729b0 2024-09-10T10:03:55.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3258c00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f323c067ce0 0x7f323c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:55.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3258c00640 1 -- 172.21.15.110:0/3958858705 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4885+0+0 (secure 0 0 0) 0x7f3248086440 con 0x7f325c0729b0 2024-09-10T10:03:55.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.599+0000 7f3261000640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f323c067ce0 0x7f323c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:55.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.603+0000 7f3261000640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f323c067ce0 0x7f323c06a1a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f324c0ae000 tx=0x7f324c004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:55.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.603+0000 7f3258c00640 1 -- 172.21.15.110:0/3958858705 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3248056cf0 con 0x7f325c0729b0 2024-09-10T10:03:55.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:55.763+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f325c0afb00 con 0x7f325c0729b0 2024-09-10T10:03:56.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3258c00640 1 -- 172.21.15.110:0/3958858705 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) ==== 138+0+0 (secure 0 0 0) 0x7f32480565f0 con 0x7f325c0729b0 2024-09-10T10:03:56.701 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:03:56.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f323c067ce0 msgr2=0x7f323c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:56.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f323c067ce0 0x7f323c06a1a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f324c0ae000 tx=0x7f324c004ac0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c0729b0 msgr2=0x7f325c154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:56.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c0729b0 0x7f325c154430 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f324800e960 tx=0x7f324800ee20 comp rx=0 tx=0).stop 2024-09-10T10:03:56.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 shutdown_connections 2024-09-10T10:03:56.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f323c067ce0 0x7f323c06a1a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f325c15e700 0x7f325c154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f325c0729b0 0x7f325c154430 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 --2- 172.21.15.110:0/3958858705 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f325c06ff60 0x7f325c14af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 >> 172.21.15.110:0/3958858705 conn(0x7f325c06cb90 msgr2=0x7f325c0b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:56.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 shutdown_connections 2024-09-10T10:03:56.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.699+0000 7f3263a00640 1 -- 172.21.15.110:0/3958858705 wait complete. 2024-09-10T10:03:56.730 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:03:56.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b54000640 1 Processor -- start 2024-09-10T10:03:56.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b54000640 1 -- start start 2024-09-10T10:03:56.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b54000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cfe30 0x7f6b4c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:56.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b54000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6b4c0cd510 0x7f6b4c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b54000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b4c06d2d0 con 0x7f6b4c06e090 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b54000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6b4c0cdeb0 con 0x7f6b4c0cd510 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b54000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4c15d9a0 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b50c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6b4c0cd510 0x7f6b4c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b50c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6b4c0cd510 0x7f6b4c0cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35076/0 (socket says 172.21.15.110:35076) 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b50c00640 1 -- 172.21.15.110:0/477169217 learned_addr learned my addr 172.21.15.110:0/477169217 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b51600640 1 --2- 172.21.15.110:0/477169217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cfe30 0x7f6b4c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b50c00640 1 -- 172.21.15.110:0/477169217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cfe30 msgr2=0x7f6b4c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b50c00640 1 --2- 172.21.15.110:0/477169217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cfe30 0x7f6b4c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b50c00640 1 -- 172.21.15.110:0/477169217 >> v1:172.21.15.110:6789/0 conn(0x7f6b4c06e090 legacy=0x7f6b4c06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:56.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.871+0000 7f6b50c00640 1 -- 172.21.15.110:0/477169217 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c06d5b0 con 0x7f6b4c0cd510 2024-09-10T10:03:56.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b50c00640 1 --2- 172.21.15.110:0/477169217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cd510 0x7f6b4c0cd970 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009870 tx=0x7f6b3801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2e4f134ddb9cbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:56.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b4be00640 1 -- 172.21.15.110:0/477169217 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380b1070 con 0x7f6b4c0cd510 2024-09-10T10:03:56.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b4be00640 1 -- 172.21.15.110:0/477169217 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b380a8050 con 0x7f6b4c0cd510 2024-09-10T10:03:56.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b4be00640 1 -- 172.21.15.110:0/477169217 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380ac860 con 0x7f6b4c0cd510 2024-09-10T10:03:56.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b54000640 1 -- 172.21.15.110:0/477169217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cd510 msgr2=0x7f6b4c0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:56.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b54000640 1 --2- 172.21.15.110:0/477169217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cd510 0x7f6b4c0cd970 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009870 tx=0x7f6b3801c4d0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b54000640 1 -- 172.21.15.110:0/477169217 shutdown_connections 2024-09-10T10:03:56.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b54000640 1 --2- 172.21.15.110:0/477169217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cd510 0x7f6b4c0cd970 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b54000640 1 --2- 172.21.15.110:0/477169217 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cfe30 0x7f6b4c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b54000640 1 -- 172.21.15.110:0/477169217 >> 172.21.15.110:0/477169217 conn(0x7f6b4c06bc50 msgr2=0x7f6b4c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:56.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b54000640 1 -- 172.21.15.110:0/477169217 shutdown_connections 2024-09-10T10:03:56.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.875+0000 7f6b54000640 1 -- 172.21.15.110:0/477169217 wait complete. 2024-09-10T10:03:56.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 Processor -- start 2024-09-10T10:03:56.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 -- start start 2024-09-10T10:03:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b4c06e090 0x7f6b4c14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cd510 0x7f6b4c159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cfe30 0x7f6b4c0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c072190 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c15d640 con 0x7f6b4c06e090 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4c169180 con 0x7f6b4c0cd510 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cfe30 0x7f6b4c0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cfe30 0x7f6b4c0ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35092/0 (socket says 172.21.15.110:35092) 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 -- 172.21.15.110:0/1376919670 learned_addr learned my addr 172.21.15.110:0/1376919670 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b50c00640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cd510 0x7f6b4c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b51600640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b4c06e090 0x7f6b4c14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 -- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cd510 msgr2=0x7f6b4c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:56.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cd510 0x7f6b4c159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 -- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b4c06e090 msgr2=0x7f6b4c14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b4c06e090 0x7f6b4c14faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 -- 172.21.15.110:0/1376919670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c0b25f0 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b51600640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b4c06e090 0x7f6b4c14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b52000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cfe30 0x7f6b4c0ae840 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400c340 tx=0x7f6b4400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b49600640 1 -- 172.21.15.110:0/1376919670 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44005220 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b49600640 1 -- 172.21.15.110:0/1376919670 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b440053c0 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b49600640 1 -- 172.21.15.110:0/1376919670 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44007270 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c0b28d0 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.879+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c0b2e90 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.883+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b4c0cd970 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.883+0000 7f6b49600640 1 -- 172.21.15.110:0/1376919670 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f6b44019020 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.883+0000 7f6b49600640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b2c067cb0 0x7f6b2c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:56.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.883+0000 7f6b49600640 1 -- 172.21.15.110:0/1376919670 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5255+0+0 (secure 0 0 0) 0x7f6b44085860 con 0x7f6b4c0cfe30 2024-09-10T10:03:56.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.883+0000 7f6b51600640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b2c067cb0 0x7f6b2c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:56.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.883+0000 7f6b51600640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b2c067cb0 0x7f6b2c06a170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6b40003fd0 tx=0x7f6b400039e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:56.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:56.887+0000 7f6b49600640 1 -- 172.21.15.110:0/1376919670 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6b44020020 con 0x7f6b4c0cfe30 2024-09-10T10:03:57.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.043+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6b4c166670 con 0x7f6b4c0cfe30 2024-09-10T10:03:57.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b49600640 1 -- 172.21.15.110:0/1376919670 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) ==== 170+0+0 (secure 0 0 0) 0x7f6b44056050 con 0x7f6b4c0cfe30 2024-09-10T10:03:57.705 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:03:57.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b2c067cb0 msgr2=0x7f6b2c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b2c067cb0 0x7f6b2c06a170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6b40003fd0 tx=0x7f6b400039e0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cfe30 msgr2=0x7f6b4c0ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cfe30 0x7f6b4c0ae840 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400c340 tx=0x7f6b4400c800 comp rx=0 tx=0).stop 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 shutdown_connections 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b2c067cb0 0x7f6b2c06a170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b4c0cfe30 0x7f6b4c0ae840 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6b4c0cd510 0x7f6b4c159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 --2- 172.21.15.110:0/1376919670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b4c06e090 0x7f6b4c14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 >> 172.21.15.110:0/1376919670 conn(0x7f6b4c06bc50 msgr2=0x7f6b4c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 shutdown_connections 2024-09-10T10:03:57.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.703+0000 7f6b54000640 1 -- 172.21.15.110:0/1376919670 wait complete. 2024-09-10T10:03:57.748 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:03:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78e2200640 1 Processor -- start 2024-09-10T10:03:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78e2200640 1 -- start start 2024-09-10T10:03:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78e2200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f78dc145f30 0x7f78dc0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78e2200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f78dc0cfd90 0x7f78dc0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78e2200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78dc0b37c0 con 0x7f78dc0b29e0 2024-09-10T10:03:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78e2200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f78dc0b2e50 con 0x7f78dc145f30 2024-09-10T10:03:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78e2200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f78dc0d0170 con 0x7f78dc0cfd90 2024-09-10T10:03:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78db400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f78dc145f30 0x7f78dc0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78db400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f78dc145f30 0x7f78dc0b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35094/0 (socket says 172.21.15.110:35094) 2024-09-10T10:03:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78db400640 1 -- 172.21.15.110:0/2944204536 learned_addr learned my addr 172.21.15.110:0/2944204536 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78dbe00640 1 --2- 172.21.15.110:0/2944204536 >> v2:172.21.15.145:3300/0 conn(0x7f78dc0cfd90 0x7f78dc0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78db400640 1 -- 172.21.15.110:0/2944204536 >> v2:172.21.15.145:3300/0 conn(0x7f78dc0cfd90 msgr2=0x7f78dc0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78db400640 1 --2- 172.21.15.110:0/2944204536 >> v2:172.21.15.145:3300/0 conn(0x7f78dc0cfd90 0x7f78dc0b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78db400640 1 -- 172.21.15.110:0/2944204536 >> v1:172.21.15.110:6789/0 conn(0x7f78dc0b29e0 legacy=0x7f78dc0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:57.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.887+0000 7f78db400640 1 -- 172.21.15.110:0/2944204536 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc0b3aa0 con 0x7f78dc145f30 2024-09-10T10:03:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78db400640 1 --2- 172.21.15.110:0/2944204536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc145f30 0x7f78dc0b24a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0b5c50 tx=0x7f78cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c84e1339da8cf11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78daa00640 1 -- 172.21.15.110:0/2944204536 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0b0070 con 0x7f78dc145f30 2024-09-10T10:03:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78daa00640 1 -- 172.21.15.110:0/2944204536 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78cc0ab600 con 0x7f78dc145f30 2024-09-10T10:03:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78daa00640 1 -- 172.21.15.110:0/2944204536 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0ab9d0 con 0x7f78dc145f30 2024-09-10T10:03:57.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 -- 172.21.15.110:0/2944204536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc145f30 msgr2=0x7f78dc0b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:57.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 --2- 172.21.15.110:0/2944204536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc145f30 0x7f78dc0b24a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0b5c50 tx=0x7f78cc01c520 comp rx=0 tx=0).stop 2024-09-10T10:03:57.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 -- 172.21.15.110:0/2944204536 shutdown_connections 2024-09-10T10:03:57.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 --2- 172.21.15.110:0/2944204536 >> v2:172.21.15.145:3300/0 conn(0x7f78dc0cfd90 0x7f78dc0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 --2- 172.21.15.110:0/2944204536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc145f30 0x7f78dc0b24a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 -- 172.21.15.110:0/2944204536 >> 172.21.15.110:0/2944204536 conn(0x7f78dc062e40 msgr2=0x7f78dc06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:57.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 -- 172.21.15.110:0/2944204536 shutdown_connections 2024-09-10T10:03:57.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 -- 172.21.15.110:0/2944204536 wait complete. 2024-09-10T10:03:57.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 Processor -- start 2024-09-10T10:03:57.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.891+0000 7f78e2200640 1 -- start start 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc0b29e0 0x7f78dc13b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78dc0cfd90 0x7f78dc13ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78dc145f30 0x7f78dc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc0b56a0 con 0x7f78dc0b29e0 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc0b5520 con 0x7f78dc0cfd90 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f78dc0b5820 con 0x7f78dc145f30 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc0b29e0 0x7f78dc13b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc0b29e0 0x7f78dc13b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35106/0 (socket says 172.21.15.110:35106) 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 -- 172.21.15.110:0/345010236 learned_addr learned my addr 172.21.15.110:0/345010236 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 -- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78dc145f30 msgr2=0x7f78dc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78dc145f30 0x7f78dc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 -- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78dc0cfd90 msgr2=0x7f78dc13ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78d3e00640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78dc0cfd90 0x7f78dc13ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78dc0cfd90 0x7f78dc13ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 -- 172.21.15.110:0/345010236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc071060 con 0x7f78dc0b29e0 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78d3e00640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78dc0cfd90 0x7f78dc13ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78db400640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc0b29e0 0x7f78dc13b1d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f78cc01abd0 tx=0x7f78cc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78d3400640 1 -- 172.21.15.110:0/345010236 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0b0040 con 0x7f78dc0b29e0 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78d3400640 1 -- 172.21.15.110:0/345010236 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78cc0abe20 con 0x7f78dc0b29e0 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc15bce0 con 0x7f78dc0b29e0 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78d3400640 1 -- 172.21.15.110:0/345010236 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0b4440 con 0x7f78dc0b29e0 2024-09-10T10:03:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc156f20 con 0x7f78dc0b29e0 2024-09-10T10:03:57.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.895+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78dc0b4580 con 0x7f78dc0b29e0 2024-09-10T10:03:57.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.899+0000 7f78d3400640 1 -- 172.21.15.110:0/345010236 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f78cc0bb020 con 0x7f78dc0b29e0 2024-09-10T10:03:57.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.899+0000 7f78d3400640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78b8067ce0 0x7f78b806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:57.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.899+0000 7f78d3400640 1 -- 172.21.15.110:0/345010236 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5637+0+0 (secure 0 0 0) 0x7f78cc11ff20 con 0x7f78dc0b29e0 2024-09-10T10:03:57.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.899+0000 7f78d3e00640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78b8067ce0 0x7f78b806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:57.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.899+0000 7f78d3e00640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78b8067ce0 0x7f78b806a1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0aefb0 tx=0x7f78c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:57.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:57.903+0000 7f78d3400640 1 -- 172.21.15.110:0/345010236 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f78cc0c2020 con 0x7f78dc0b29e0 2024-09-10T10:03:58.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.091+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f78dc0aed00 con 0x7f78dc0b29e0 2024-09-10T10:03:58.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.714+0000 7f78d3400640 1 -- 172.21.15.110:0/345010236 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v74) ==== 174+0+0 (secure 0 0 0) 0x7f78cc016110 con 0x7f78dc0b29e0 2024-09-10T10:03:58.721 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 18 and data pool 19 2024-09-10T10:03:58.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.718+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78b8067ce0 msgr2=0x7f78b806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:58.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.718+0000 7f78e2200640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78b8067ce0 0x7f78b806a1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0aefb0 tx=0x7f78c4009040 comp rx=0 tx=0).stop 2024-09-10T10:03:58.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.718+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc0b29e0 msgr2=0x7f78dc13b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:58.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.718+0000 7f78e2200640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc0b29e0 0x7f78dc13b1d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f78cc01abd0 tx=0x7f78cc0192c0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.722+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 shutdown_connections 2024-09-10T10:03:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.722+0000 7f78e2200640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f78b8067ce0 0x7f78b806a1a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.722+0000 7f78e2200640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f78dc145f30 0x7f78dc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.722+0000 7f78e2200640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78dc0cfd90 0x7f78dc13ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.722+0000 7f78e2200640 1 --2- 172.21.15.110:0/345010236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78dc0b29e0 0x7f78dc13b1d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.722+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 >> 172.21.15.110:0/345010236 conn(0x7f78dc062e40 msgr2=0x7f78dc0cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.726+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 shutdown_connections 2024-09-10T10:03:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.730+0000 7f78e2200640 1 -- 172.21.15.110:0/345010236 wait complete. 2024-09-10T10:03:58.783 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:03:58.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.914+0000 7fc0aae00640 1 Processor -- start 2024-09-10T10:03:58.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.914+0000 7fc0aae00640 1 -- start start 2024-09-10T10:03:58.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0aae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a40b1c60 0x7fc0a4142320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:58.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0aae00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc0a415fd80 0x7fc0a413f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:58.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0aae00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a415c790 con 0x7fc0a40731e0 2024-09-10T10:03:58.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0aae00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc0a40b2040 con 0x7fc0a415fd80 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0aae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc0a4142860 con 0x7fc0a40b1c60 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0aa400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc0a40731e0 0x7fc0a415cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43748/0 (socket says 172.21.15.110:43748) 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0aa400640 1 -- 172.21.15.110:0/3322898945 learned_addr learned my addr 172.21.15.110:0/3322898945 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a9000640 1 --2- 172.21.15.110:0/3322898945 >> v2:172.21.15.145:3300/0 conn(0x7fc0a415fd80 0x7fc0a413f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a9a00640 1 --2- 172.21.15.110:0/3322898945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a40b1c60 0x7fc0a4142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a3e00640 1 -- 172.21.15.110:0/3322898945 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1225869825 0 0) 0x7fc09c0014a0 con 0x7fc0a40731e0 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a3e00640 1 -- 172.21.15.110:0/3322898945 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc08c003530 con 0x7fc0a40731e0 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a3e00640 1 -- 172.21.15.110:0/3322898945 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1940662626 0 0) 0x7fc08c003530 con 0x7fc0a40731e0 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a9000640 1 -- 172.21.15.110:0/3322898945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a40b1c60 msgr2=0x7fc0a4142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a9000640 1 --2- 172.21.15.110:0/3322898945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a40b1c60 0x7fc0a4142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a9000640 1 -- 172.21.15.110:0/3322898945 >> v1:172.21.15.110:6789/0 conn(0x7fc0a40731e0 legacy=0x7fc0a415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.918+0000 7fc0a9000640 1 -- 172.21.15.110:0/3322898945 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a41602b0 con 0x7fc0a415fd80 2024-09-10T10:03:58.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0a9000640 1 --2- 172.21.15.110:0/3322898945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a415fd80 0x7fc0a413f8e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0900097c0 tx=0x7fc09001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ed786e2347d54a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:58.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0a3e00640 1 -- 172.21.15.110:0/3322898945 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900b0070 con 0x7fc0a415fd80 2024-09-10T10:03:58.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0a3e00640 1 -- 172.21.15.110:0/3322898945 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0900ab400 con 0x7fc0a415fd80 2024-09-10T10:03:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0a3e00640 1 -- 172.21.15.110:0/3322898945 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c001bd0 con 0x7fc0a415fd80 2024-09-10T10:03:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 -- 172.21.15.110:0/3322898945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a415fd80 msgr2=0x7fc0a413f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 --2- 172.21.15.110:0/3322898945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a415fd80 0x7fc0a413f8e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0900097c0 tx=0x7fc09001c3c0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 -- 172.21.15.110:0/3322898945 shutdown_connections 2024-09-10T10:03:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 --2- 172.21.15.110:0/3322898945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a415fd80 0x7fc0a413f8e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 --2- 172.21.15.110:0/3322898945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a40b1c60 0x7fc0a4142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 -- 172.21.15.110:0/3322898945 >> 172.21.15.110:0/3322898945 conn(0x7fc0a40b42d0 msgr2=0x7fc0a40b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:58.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 -- 172.21.15.110:0/3322898945 shutdown_connections 2024-09-10T10:03:58.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 -- 172.21.15.110:0/3322898945 wait complete. 2024-09-10T10:03:58.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 Processor -- start 2024-09-10T10:03:58.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.922+0000 7fc0aae00640 1 -- start start 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40731e0 0x7fc0a415f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a40b1c60 0x7fc0a413ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a415fd80 0x7fc0a41417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a413fe20 con 0x7fc0a40b1c60 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a415d770 con 0x7fc0a40731e0 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc0a4073a30 con 0x7fc0a415fd80 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a40b1c60 0x7fc0a413ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a40b1c60 0x7fc0a413ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50374/0 (socket says 172.21.15.110:50374) 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 -- 172.21.15.110:0/1865252427 learned_addr learned my addr 172.21.15.110:0/1865252427 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:58.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aa400640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a415fd80 0x7fc0a41417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9a00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40731e0 0x7fc0a415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 -- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a415fd80 msgr2=0x7fc0a41417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a415fd80 0x7fc0a41417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 -- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40731e0 msgr2=0x7fc0a415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40731e0 0x7fc0a415f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 -- 172.21.15.110:0/1865252427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a40b3fa0 con 0x7fc0a40b1c60 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9a00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40731e0 0x7fc0a415f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aa400640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a415fd80 0x7fc0a41417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a9000640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a40b1c60 0x7fc0a413ffa0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc09001cc60 tx=0x7fc090019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a1600640 1 -- 172.21.15.110:0/1865252427 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900b0040 con 0x7fc0a40b1c60 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a1600640 1 -- 172.21.15.110:0/1865252427 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0900b5460 con 0x7fc0a40b1c60 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a4141d20 con 0x7fc0a40b1c60 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a1600640 1 -- 172.21.15.110:0/1865252427 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900b5850 con 0x7fc0a40b1c60 2024-09-10T10:03:58.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a4140a20 con 0x7fc0a40b1c60 2024-09-10T10:03:58.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a4142320 con 0x7fc0a40b1c60 2024-09-10T10:03:58.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.926+0000 7fc0a1600640 1 -- 172.21.15.110:0/1865252427 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7fc0900be030 con 0x7fc0a40b1c60 2024-09-10T10:03:58.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.930+0000 7fc0a1600640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc084067ce0 0x7fc08406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:58.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.930+0000 7fc0a1600640 1 -- 172.21.15.110:0/1865252427 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5737+0+0 (secure 0 0 0) 0x7fc090120060 con 0x7fc0a40b1c60 2024-09-10T10:03:58.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.930+0000 7fc0a9a00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc084067ce0 0x7fc08406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:58.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.930+0000 7fc0a9a00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc084067ce0 0x7fc08406a1a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc098009bb0 tx=0x7fc098008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:58.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:58.934+0000 7fc0a1600640 1 -- 172.21.15.110:0/1865252427 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc090015fb0 con 0x7fc0a40b1c60 2024-09-10T10:03:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.090+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0a40711b0 con 0x7fc0a40b1c60 2024-09-10T10:03:59.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.090+0000 7fc0a1600640 1 -- 172.21.15.110:0/1865252427 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16832 (secure 0 0 0) 0x7fc0900f04e0 con 0x7fc0a40b1c60 2024-09-10T10:03:59.096 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:59.097 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":116,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:58.095735+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:55.813835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:57.093856+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:59.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc084067ce0 msgr2=0x7fc08406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc084067ce0 0x7fc08406a1a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc098009bb0 tx=0x7fc098008040 comp rx=0 tx=0).stop 2024-09-10T10:03:59.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a40b1c60 msgr2=0x7fc0a413ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a40b1c60 0x7fc0a413ffa0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc09001cc60 tx=0x7fc090019160 comp rx=0 tx=0).stop 2024-09-10T10:03:59.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 shutdown_connections 2024-09-10T10:03:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc084067ce0 0x7fc08406a1a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0a415fd80 0x7fc0a41417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a40b1c60 0x7fc0a413ffa0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 --2- 172.21.15.110:0/1865252427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40731e0 0x7fc0a415f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 >> 172.21.15.110:0/1865252427 conn(0x7fc0a40b42d0 msgr2=0x7fc0a40b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 shutdown_connections 2024-09-10T10:03:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.094+0000 7fc0aae00640 1 -- 172.21.15.110:0/1865252427 wait complete. 2024-09-10T10:03:59.118 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:03:59.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac6b400640 1 Processor -- start 2024-09-10T10:03:59.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac6b400640 1 -- start start 2024-09-10T10:03:59.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac6b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 0x7fac640ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac6b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fac6406edf0 0x7fac64072310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac6b400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac640af1a0 con 0x7fac640aed80 2024-09-10T10:03:59.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac6b400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fac640b34d0 con 0x7fac6406edf0 2024-09-10T10:03:59.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac6b400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fac641482e0 con 0x7fac64147f00 2024-09-10T10:03:59.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac69400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fac6406edf0 0x7fac64072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac69400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fac6406edf0 0x7fac64072310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50386/0 (socket says 172.21.15.110:50386) 2024-09-10T10:03:59.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac69400640 1 -- 172.21.15.110:0/2905609171 learned_addr learned my addr 172.21.15.110:0/2905609171 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:59.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac68a00640 1 --2- 172.21.15.110:0/2905609171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 0x7fac640ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac69400640 1 -- 172.21.15.110:0/2905609171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 msgr2=0x7fac640ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:59.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac69400640 1 --2- 172.21.15.110:0/2905609171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 0x7fac640ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac69400640 1 -- 172.21.15.110:0/2905609171 >> v1:172.21.15.110:6789/0 conn(0x7fac640aed80 legacy=0x7fac6406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.250+0000 7fac69400640 1 -- 172.21.15.110:0/2905609171 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6415ee20 con 0x7fac6406edf0 2024-09-10T10:03:59.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac69400640 1 --2- 172.21.15.110:0/2905609171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac64072310 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009870 tx=0x7fac5c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b904b83ce94710 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:59.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac63400640 1 -- 172.21.15.110:0/2905609171 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0b2070 con 0x7fac6406edf0 2024-09-10T10:03:59.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac63400640 1 -- 172.21.15.110:0/2905609171 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5c0a9070 con 0x7fac6406edf0 2024-09-10T10:03:59.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac63400640 1 -- 172.21.15.110:0/2905609171 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0ad880 con 0x7fac6406edf0 2024-09-10T10:03:59.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac6b400640 1 -- 172.21.15.110:0/2905609171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 msgr2=0x7fac64072310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac6b400640 1 --2- 172.21.15.110:0/2905609171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac64072310 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009870 tx=0x7fac5c01c560 comp rx=0 tx=0).stop 2024-09-10T10:03:59.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac6b400640 1 -- 172.21.15.110:0/2905609171 shutdown_connections 2024-09-10T10:03:59.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac6b400640 1 --2- 172.21.15.110:0/2905609171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac64072310 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac6b400640 1 --2- 172.21.15.110:0/2905609171 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 0x7fac640ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac6b400640 1 -- 172.21.15.110:0/2905609171 >> 172.21.15.110:0/2905609171 conn(0x7fac6406bc50 msgr2=0x7fac6406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:59.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac6b400640 1 -- 172.21.15.110:0/2905609171 shutdown_connections 2024-09-10T10:03:59.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.254+0000 7fac6b400640 1 -- 172.21.15.110:0/2905609171 wait complete. 2024-09-10T10:03:59.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 Processor -- start 2024-09-10T10:03:59.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 -- start start 2024-09-10T10:03:59.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac640b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac640aed80 0x7fac64140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 0x7fac641405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fac640731e0 con 0x7fac6406edf0 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fac64073060 con 0x7fac640aed80 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fac64073360 con 0x7fac64147f00 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac640b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac640b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50398/0 (socket says 172.21.15.110:50398) 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 -- 172.21.15.110:0/3300813674 learned_addr learned my addr 172.21.15.110:0/3300813674 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac69400640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 0x7fac641405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac63e00640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac640aed80 0x7fac64140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 -- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 msgr2=0x7fac641405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 0x7fac641405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 -- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac640aed80 msgr2=0x7fac64140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac640aed80 0x7fac64140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 -- 172.21.15.110:0/3300813674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac640b1e40 con 0x7fac6406edf0 2024-09-10T10:03:59.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac68a00640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac640b1900 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fac5400efc0 tx=0x7fac5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:59.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac60c00640 1 -- 172.21.15.110:0/3300813674 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54018020 con 0x7fac6406edf0 2024-09-10T10:03:59.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac60c00640 1 -- 172.21.15.110:0/3300813674 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac54014480 con 0x7fac6406edf0 2024-09-10T10:03:59.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac64160a10 con 0x7fac6406edf0 2024-09-10T10:03:59.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac60c00640 1 -- 172.21.15.110:0/3300813674 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54014870 con 0x7fac6406edf0 2024-09-10T10:03:59.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac64160fd0 con 0x7fac6406edf0 2024-09-10T10:03:59.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.258+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac64072310 con 0x7fac6406edf0 2024-09-10T10:03:59.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.262+0000 7fac60c00640 1 -- 172.21.15.110:0/3300813674 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7fac5401b020 con 0x7fac6406edf0 2024-09-10T10:03:59.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.262+0000 7fac60c00640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fac44067cb0 0x7fac4406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.262+0000 7fac60c00640 1 -- 172.21.15.110:0/3300813674 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5737+0+0 (secure 0 0 0) 0x7fac54086f70 con 0x7fac6406edf0 2024-09-10T10:03:59.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.262+0000 7fac63e00640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fac44067cb0 0x7fac4406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.262+0000 7fac63e00640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fac44067cb0 0x7fac4406a170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac640b39b0 tx=0x7fac50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:59.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.266+0000 7fac60c00640 1 -- 172.21.15.110:0/3300813674 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fac54022070 con 0x7fac6406edf0 2024-09-10T10:03:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.426+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac640b39e0 con 0x7fac6406edf0 2024-09-10T10:03:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.426+0000 7fac60c00640 1 -- 172.21.15.110:0/3300813674 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16832 (secure 0 0 0) 0x7fac54010040 con 0x7fac6406edf0 2024-09-10T10:03:59.432 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:03:59.432 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":116,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:58.095735+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:55.813835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:57.093856+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:59.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fac44067cb0 msgr2=0x7fac4406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fac44067cb0 0x7fac4406a170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac640b39b0 tx=0x7fac50009040 comp rx=0 tx=0).stop 2024-09-10T10:03:59.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 msgr2=0x7fac640b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac640b1900 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fac5400efc0 tx=0x7fac5400c490 comp rx=0 tx=0).stop 2024-09-10T10:03:59.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 shutdown_connections 2024-09-10T10:03:59.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fac44067cb0 0x7fac4406a170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fac64147f00 0x7fac641405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac640aed80 0x7fac64140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 --2- 172.21.15.110:0/3300813674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac6406edf0 0x7fac640b1900 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 >> 172.21.15.110:0/3300813674 conn(0x7fac6406bc50 msgr2=0x7fac6415d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:59.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 shutdown_connections 2024-09-10T10:03:59.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.430+0000 7fac6b400640 1 -- 172.21.15.110:0/3300813674 wait complete. 2024-09-10T10:03:59.454 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:03:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034d600640 1 Processor -- start 2024-09-10T10:03:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034d600640 1 -- start start 2024-09-10T10:03:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034d600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0348148fe0 0x7f0348072fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034d600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03480735e0 con 0x7f03480ce1c0 2024-09-10T10:03:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034d600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03480738c0 con 0x7f03480cdde0 2024-09-10T10:03:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034d600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0348073ba0 con 0x7f0348148fe0 2024-09-10T10:03:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034cc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0348148fe0 0x7f0348072fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034cc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0348148fe0 0x7f0348072fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50412/0 (socket says 172.21.15.110:50412) 2024-09-10T10:03:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034cc00640 1 -- 172.21.15.110:0/3310027431 learned_addr learned my addr 172.21.15.110:0/3310027431 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:59.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034cc00640 1 -- 172.21.15.110:0/3310027431 >> v1:172.21.15.145:6789/0 conn(0x7f03480cdde0 legacy=0x7f0348163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034cc00640 1 -- 172.21.15.110:0/3310027431 >> v1:172.21.15.110:6789/0 conn(0x7f03480ce1c0 legacy=0x7f03481488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.590+0000 7f034cc00640 1 -- 172.21.15.110:0/3310027431 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0348073e80 con 0x7f0348148fe0 2024-09-10T10:03:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f034cc00640 1 --2- 172.21.15.110:0/3310027431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0348148fe0 0x7f0348072fe0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0340009e50 tx=0x7f034001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6900fc33db58f556 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f0346a00640 1 -- 172.21.15.110:0/3310027431 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400a58f0 con 0x7f0348148fe0 2024-09-10T10:03:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f0346a00640 1 -- 172.21.15.110:0/3310027431 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03400a5a90 con 0x7f0348148fe0 2024-09-10T10:03:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f0346a00640 1 -- 172.21.15.110:0/3310027431 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400a5e60 con 0x7f0348148fe0 2024-09-10T10:03:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f034d600640 1 -- 172.21.15.110:0/3310027431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0348148fe0 msgr2=0x7f0348072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f034d600640 1 --2- 172.21.15.110:0/3310027431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0348148fe0 0x7f0348072fe0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0340009e50 tx=0x7f034001ca00 comp rx=0 tx=0).stop 2024-09-10T10:03:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f034d600640 1 -- 172.21.15.110:0/3310027431 shutdown_connections 2024-09-10T10:03:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f034d600640 1 --2- 172.21.15.110:0/3310027431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0348148fe0 0x7f0348072fe0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f034d600640 1 -- 172.21.15.110:0/3310027431 >> 172.21.15.110:0/3310027431 conn(0x7f034806d710 msgr2=0x7f03480b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f034d600640 1 -- 172.21.15.110:0/3310027431 shutdown_connections 2024-09-10T10:03:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.594+0000 7f034d600640 1 -- 172.21.15.110:0/3310027431 wait complete. 2024-09-10T10:03:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 Processor -- start 2024-09-10T10:03:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 -- start start 2024-09-10T10:03:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03480cdde0 0x7f0348140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03480ce1c0 0x7f0348142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348148fe0 0x7f034813b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03481701a0 con 0x7f03480cdde0 2024-09-10T10:03:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0348170020 con 0x7f0348148fe0 2024-09-10T10:03:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0348170320 con 0x7f03480ce1c0 2024-09-10T10:03:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03480cdde0 0x7f0348140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03480cdde0 0x7f0348140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50428/0 (socket says 172.21.15.110:50428) 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 -- 172.21.15.110:0/2450436218 learned_addr learned my addr 172.21.15.110:0/2450436218 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034cc00640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348148fe0 0x7f034813b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347400640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03480ce1c0 0x7f0348142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 -- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03480ce1c0 msgr2=0x7f0348142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03480ce1c0 0x7f0348142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 -- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348148fe0 msgr2=0x7f034813b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348148fe0 0x7f034813b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 -- 172.21.15.110:0/2450436218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034815b320 con 0x7f03480cdde0 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f0347e00640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03480cdde0 0x7f0348140950 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f033800c450 tx=0x7f033800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f033fe00640 1 -- 172.21.15.110:0/2450436218 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03380052d0 con 0x7f03480cdde0 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f033fe00640 1 -- 172.21.15.110:0/2450436218 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0338005470 con 0x7f03480cdde0 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034815c270 con 0x7f03480cdde0 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f033fe00640 1 -- 172.21.15.110:0/2450436218 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338007260 con 0x7f03480cdde0 2024-09-10T10:03:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034815b8a0 con 0x7f03480cdde0 2024-09-10T10:03:59.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.598+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0348072fe0 con 0x7f03480cdde0 2024-09-10T10:03:59.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.602+0000 7f033fe00640 1 -- 172.21.15.110:0/2450436218 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f0338007400 con 0x7f03480cdde0 2024-09-10T10:03:59.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.602+0000 7f033fe00640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0328067cb0 0x7f032806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:59.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.602+0000 7f033fe00640 1 -- 172.21.15.110:0/2450436218 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5737+0+0 (secure 0 0 0) 0x7f0338085880 con 0x7f03480cdde0 2024-09-10T10:03:59.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.602+0000 7f0347400640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0328067cb0 0x7f032806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:59.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.602+0000 7f0347400640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0328067cb0 0x7f032806a170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0334000f70 tx=0x7f0334009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:59.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.606+0000 7f033fe00640 1 -- 172.21.15.110:0/2450436218 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0338055e50 con 0x7f03480cdde0 2024-09-10T10:03:59.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:03:59.802+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f03481430e0 con 0x7f03480cdde0 2024-09-10T10:04:00.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.738+0000 7f033fe00640 1 -- 172.21.15.110:0/2450436218 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) ==== 118+0+0 (secure 0 0 0) 0x7f0338085070 con 0x7f03480cdde0 2024-09-10T10:04:00.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0328067cb0 msgr2=0x7f032806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:00.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0328067cb0 0x7f032806a170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0334000f70 tx=0x7f0334009040 comp rx=0 tx=0).stop 2024-09-10T10:04:00.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03480cdde0 msgr2=0x7f0348140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:00.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03480cdde0 0x7f0348140950 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f033800c450 tx=0x7f033800c910 comp rx=0 tx=0).stop 2024-09-10T10:04:00.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 shutdown_connections 2024-09-10T10:04:00.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0328067cb0 0x7f032806a170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0348148fe0 0x7f034813b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03480ce1c0 0x7f0348142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 --2- 172.21.15.110:0/2450436218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03480cdde0 0x7f0348140950 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 >> 172.21.15.110:0/2450436218 conn(0x7f034806d710 msgr2=0x7f0348144bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:00.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 shutdown_connections 2024-09-10T10:04:00.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.742+0000 7f034d600640 1 -- 172.21.15.110:0/2450436218 wait complete. 2024-09-10T10:04:00.765 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:04:00.765 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:04:00.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.898+0000 7f169e000640 1 Processor -- start 2024-09-10T10:04:00.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.898+0000 7f169e000640 1 -- start start 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f169e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1698155850 0x7f16980ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f169e000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16980cd2f0 0x7f16980cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f169e000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16980cd800 con 0x7f16980cf630 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f169e000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f16980ce410 con 0x7f16980cd2f0 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f169e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1698155c30 con 0x7f1698155850 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f1697e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16980cd2f0 0x7f16980cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f1697e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16980cd2f0 0x7f16980cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50432/0 (socket says 172.21.15.110:50432) 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f1697e00640 1 -- 172.21.15.110:0/2928912057 learned_addr learned my addr 172.21.15.110:0/2928912057 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:00.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f169cc00640 1 --2- 172.21.15.110:0/2928912057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1698155850 0x7f16980ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:00.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f1697e00640 1 -- 172.21.15.110:0/2928912057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1698155850 msgr2=0x7f16980ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:00.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f1697e00640 1 --2- 172.21.15.110:0/2928912057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1698155850 0x7f16980ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f1697e00640 1 -- 172.21.15.110:0/2928912057 >> v1:172.21.15.145:6789/0 conn(0x7f16980cf630 legacy=0x7f169815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:00.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.902+0000 7f1697e00640 1 -- 172.21.15.110:0/2928912057 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16980cdae0 con 0x7f16980cd2f0 2024-09-10T10:04:00.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f1697e00640 1 --2- 172.21.15.110:0/2928912057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16980cd2f0 0x7f16980cf0f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1684009870 tx=0x7f168401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62aab1b15e469d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:00.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f1697400640 1 -- 172.21.15.110:0/2928912057 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840ab650 con 0x7f16980cd2f0 2024-09-10T10:04:00.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f1697400640 1 -- 172.21.15.110:0/2928912057 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16840b0070 con 0x7f16980cd2f0 2024-09-10T10:04:00.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f1697400640 1 -- 172.21.15.110:0/2928912057 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840aba10 con 0x7f16980cd2f0 2024-09-10T10:04:00.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- 172.21.15.110:0/2928912057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16980cd2f0 msgr2=0x7f16980cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:00.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 --2- 172.21.15.110:0/2928912057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16980cd2f0 0x7f16980cf0f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1684009870 tx=0x7f168401c560 comp rx=0 tx=0).stop 2024-09-10T10:04:00.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- 172.21.15.110:0/2928912057 shutdown_connections 2024-09-10T10:04:00.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 --2- 172.21.15.110:0/2928912057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16980cd2f0 0x7f16980cf0f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 --2- 172.21.15.110:0/2928912057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1698155850 0x7f16980ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- 172.21.15.110:0/2928912057 >> 172.21.15.110:0/2928912057 conn(0x7f169806d710 msgr2=0x7f16980b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:00.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- 172.21.15.110:0/2928912057 shutdown_connections 2024-09-10T10:04:00.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- 172.21.15.110:0/2928912057 wait complete. 2024-09-10T10:04:00.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 Processor -- start 2024-09-10T10:04:00.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- start start 2024-09-10T10:04:00.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16980cd2f0 0x7f169813b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16980cf630 0x7f16981645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1698155850 0x7f1698164b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1698168000 con 0x7f1698155850 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1698167e80 con 0x7f16980cf630 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1698168180 con 0x7f16980cd2f0 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f169d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1698155850 0x7f1698164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.906+0000 7f1697e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16980cf630 0x7f16981645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1698155850 0x7f1698164b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50438/0 (socket says 172.21.15.110:50438) 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169d600640 1 -- 172.21.15.110:0/4196933184 learned_addr learned my addr 172.21.15.110:0/4196933184 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:00.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169cc00640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16980cd2f0 0x7f169813b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:00.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169d600640 1 -- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16980cd2f0 msgr2=0x7f169813b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169d600640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16980cd2f0 0x7f169813b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169d600640 1 -- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16980cf630 msgr2=0x7f16981645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169d600640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16980cf630 0x7f16981645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169d600640 1 -- 172.21.15.110:0/4196933184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1698159c90 con 0x7f1698155850 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169d600640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1698155850 0x7f1698164b10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f169000c770 tx=0x7f169000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f1694c00640 1 -- 172.21.15.110:0/4196933184 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690003b80 con 0x7f1698155850 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f1694c00640 1 -- 172.21.15.110:0/4196933184 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1690003d20 con 0x7f1698155850 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169813ea50 con 0x7f1698155850 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f1694c00640 1 -- 172.21.15.110:0/4196933184 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169000aa90 con 0x7f1698155850 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169813c110 con 0x7f1698155850 2024-09-10T10:04:00.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16980cf0f0 con 0x7f1698155850 2024-09-10T10:04:00.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f1694c00640 1 -- 172.21.15.110:0/4196933184 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f169000ac30 con 0x7f1698155850 2024-09-10T10:04:00.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f1694c00640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1678067cb0 0x7f167806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:00.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.910+0000 7f1694c00640 1 -- 172.21.15.110:0/4196933184 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f1690086240 con 0x7f1698155850 2024-09-10T10:04:00.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.914+0000 7f169cc00640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1678067cb0 0x7f167806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:00.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.914+0000 7f169cc00640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1678067cb0 0x7f167806a170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1688007a60 tx=0x7f1688008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:00.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:00.918+0000 7f1694c00640 1 -- 172.21.15.110:0/4196933184 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1698159c90 con 0x7f1698155850 2024-09-10T10:04:01.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.106+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f16980b2fc0 con 0x7f1698155850 2024-09-10T10:04:01.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.742+0000 7f1694c00640 1 -- 172.21.15.110:0/4196933184 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) ==== 122+0+0 (secure 0 0 0) 0x7f16900560a0 con 0x7f1698155850 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1678067cb0 msgr2=0x7f167806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1678067cb0 0x7f167806a170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1688007a60 tx=0x7f1688008040 comp rx=0 tx=0).stop 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1698155850 msgr2=0x7f1698164b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1698155850 0x7f1698164b10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f169000c770 tx=0x7f169000cc30 comp rx=0 tx=0).stop 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 shutdown_connections 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1678067cb0 0x7f167806a170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1698155850 0x7f1698164b10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16980cf630 0x7f16981645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 --2- 172.21.15.110:0/4196933184 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f16980cd2f0 0x7f169813b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 >> 172.21.15.110:0/4196933184 conn(0x7f169806d710 msgr2=0x7f1698073760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:01.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 shutdown_connections 2024-09-10T10:04:01.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.746+0000 7f169e000640 1 -- 172.21.15.110:0/4196933184 wait complete. 2024-09-10T10:04:01.770 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:04:01.770 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:01.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe282200640 1 Processor -- start 2024-09-10T10:04:01.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe282200640 1 -- start start 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe282200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe27c0cd510 0x7fe27c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe282200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c06e090 0x7fe27c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe282200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe27c072150 con 0x7fe27c0cfe30 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe282200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe27c15d9a0 con 0x7fe27c0cd510 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe282200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe27c06ea60 con 0x7fe27c06e090 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe27c0cd510 0x7fe27c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe27c0cd510 0x7fe27c0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50446/0 (socket says 172.21.15.110:50446) 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27aa00640 1 -- 172.21.15.110:0/1126504057 learned_addr learned my addr 172.21.15.110:0/1126504057 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:01.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27be00640 1 --2- 172.21.15.110:0/1126504057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c06e090 0x7fe27c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:01.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27aa00640 1 -- 172.21.15.110:0/1126504057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c06e090 msgr2=0x7fe27c06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:01.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27aa00640 1 --2- 172.21.15.110:0/1126504057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c06e090 0x7fe27c06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27aa00640 1 -- 172.21.15.110:0/1126504057 >> v1:172.21.15.145:6789/0 conn(0x7fe27c0cfe30 legacy=0x7fe27c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:01.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27aa00640 1 -- 172.21.15.110:0/1126504057 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c06d2d0 con 0x7fe27c0cd510 2024-09-10T10:04:01.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.902+0000 7fe27be00640 1 --2- 172.21.15.110:0/1126504057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c06e090 0x7fe27c06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:01.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe27aa00640 1 --2- 172.21.15.110:0/1126504057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cd510 0x7fe27c0cd9a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe268009870 tx=0x7fe26801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6aff8545af4d51a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:01.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe27a000640 1 -- 172.21.15.110:0/1126504057 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680b1070 con 0x7fe27c0cd510 2024-09-10T10:04:01.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe27a000640 1 -- 172.21.15.110:0/1126504057 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2680a8050 con 0x7fe27c0cd510 2024-09-10T10:04:01.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe27a000640 1 -- 172.21.15.110:0/1126504057 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680ac860 con 0x7fe27c0cd510 2024-09-10T10:04:01.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe282200640 1 -- 172.21.15.110:0/1126504057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cd510 msgr2=0x7fe27c0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:01.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe282200640 1 --2- 172.21.15.110:0/1126504057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cd510 0x7fe27c0cd9a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe268009870 tx=0x7fe26801c4d0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe282200640 1 -- 172.21.15.110:0/1126504057 shutdown_connections 2024-09-10T10:04:01.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe282200640 1 --2- 172.21.15.110:0/1126504057 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c06e090 0x7fe27c06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe282200640 1 --2- 172.21.15.110:0/1126504057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cd510 0x7fe27c0cd9a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe282200640 1 -- 172.21.15.110:0/1126504057 >> 172.21.15.110:0/1126504057 conn(0x7fe27c06bc50 msgr2=0x7fe27c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:01.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe282200640 1 -- 172.21.15.110:0/1126504057 shutdown_connections 2024-09-10T10:04:01.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.906+0000 7fe282200640 1 -- 172.21.15.110:0/1126504057 wait complete. 2024-09-10T10:04:01.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 Processor -- start 2024-09-10T10:04:01.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 -- start start 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe27c06e090 0x7fe27c141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c0cd510 0x7fe27c14c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cfe30 0x7fe27c14c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c169180 con 0x7fe27c0cfe30 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c15d6c0 con 0x7fe27c06e090 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe27c169300 con 0x7fe27c0cd510 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cfe30 0x7fe27c14c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cfe30 0x7fe27c14c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50454/0 (socket says 172.21.15.110:50454) 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 -- 172.21.15.110:0/259046421 learned_addr learned my addr 172.21.15.110:0/259046421 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27b400640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe27c06e090 0x7fe27c141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:01.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27aa00640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c0cd510 0x7fe27c14c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:01.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 -- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c0cd510 msgr2=0x7fe27c14c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:01.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c0cd510 0x7fe27c14c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 -- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe27c06e090 msgr2=0x7fe27c141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe27c06e090 0x7fe27c141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 -- 172.21.15.110:0/259046421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c146130 con 0x7fe27c0cfe30 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe27be00640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cfe30 0x7fe27c14c5d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe27400d800 tx=0x7fe27400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe273400640 1 -- 172.21.15.110:0/259046421 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2740096a0 con 0x7fe27c0cfe30 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe273400640 1 -- 172.21.15.110:0/259046421 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe274019070 con 0x7fe27c0cfe30 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c146410 con 0x7fe27c0cfe30 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe273400640 1 -- 172.21.15.110:0/259046421 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274009a80 con 0x7fe27c0cfe30 2024-09-10T10:04:01.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.910+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c1469d0 con 0x7fe27c0cfe30 2024-09-10T10:04:01.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.914+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe27c06e500 con 0x7fe27c0cfe30 2024-09-10T10:04:01.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.914+0000 7fe273400640 1 -- 172.21.15.110:0/259046421 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7fe274009c20 con 0x7fe27c0cfe30 2024-09-10T10:04:01.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.914+0000 7fe273400640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe25c067cb0 0x7fe25c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:01.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.914+0000 7fe273400640 1 -- 172.21.15.110:0/259046421 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7fe274086620 con 0x7fe27c0cfe30 2024-09-10T10:04:01.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.914+0000 7fe27b400640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe25c067cb0 0x7fe25c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:01.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.914+0000 7fe27b400640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe25c067cb0 0x7fe25c06a170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe26c001de0 tx=0x7fe26c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:01.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:01.918+0000 7fe273400640 1 -- 172.21.15.110:0/259046421 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe274056580 con 0x7fe27c0cfe30 2024-09-10T10:04:02.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.106+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe27c071c90 con 0x7fe27c0cfe30 2024-09-10T10:04:02.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.106+0000 7fe273400640 1 -- 172.21.15.110:0/259046421 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5237 (secure 0 0 0) 0x7fe27405a880 con 0x7fe27c0cfe30 2024-09-10T10:04:02.112 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:02.112 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":78,"btime":"2024-09-10T10:04:01:743659+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1172125083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1172125083},{"type":"v1","addr":"172.21.15.145:6834","nonce":1172125083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6194,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3529040969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3529040969},{"type":"v1","addr":"172.21.15.145:6835","nonce":3529040969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2459795528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2459795528},{"type":"v1","addr":"172.21.15.110:6835","nonce":2459795528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:58.096051+0000","modified":"2024-09-10T10:04:01.113268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6834/1757537079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1757537079},{"type":"v1","addr":"172.21.15.110:6834","nonce":1757537079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":9}]} 2024-09-10T10:04:02.113 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 78 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe25c067cb0 msgr2=0x7fe25c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe25c067cb0 0x7fe25c06a170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe26c001de0 tx=0x7fe26c009040 comp rx=0 tx=0).stop 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cfe30 msgr2=0x7fe27c14c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cfe30 0x7fe27c14c5d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe27400d800 tx=0x7fe27400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 shutdown_connections 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe25c067cb0 0x7fe25c06a170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27c0cfe30 0x7fe27c14c5d0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe27c0cd510 0x7fe27c14c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 --2- 172.21.15.110:0/259046421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe27c06e090 0x7fe27c141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 >> 172.21.15.110:0/259046421 conn(0x7fe27c06bc50 msgr2=0x7fe27c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:02.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 shutdown_connections 2024-09-10T10:04:02.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.110+0000 7fe282200640 1 -- 172.21.15.110:0/259046421 wait complete. 2024-09-10T10:04:02.134 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:02.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8659600640 1 Processor -- start 2024-09-10T10:04:02.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8659600640 1 -- start start 2024-09-10T10:04:02.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8659600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f86540b1c60 0x7f8654142320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8659600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865415c790 con 0x7f86540731e0 2024-09-10T10:04:02.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8659600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86540b2040 con 0x7f865415fd80 2024-09-10T10:04:02.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8659600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8654142860 con 0x7f86540b1c60 2024-09-10T10:04:02.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8653e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f86540b1c60 0x7f8654142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8653e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f86540b1c60 0x7f8654142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50466/0 (socket says 172.21.15.110:50466) 2024-09-10T10:04:02.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8653e00640 1 -- 172.21.15.110:0/937641862 learned_addr learned my addr 172.21.15.110:0/937641862 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:02.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8653e00640 1 -- 172.21.15.110:0/937641862 >> v1:172.21.15.145:6789/0 conn(0x7f865415fd80 legacy=0x7f865413f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8653e00640 1 -- 172.21.15.110:0/937641862 >> v1:172.21.15.110:6789/0 conn(0x7f86540731e0 legacy=0x7f865415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.266+0000 7f8653e00640 1 -- 172.21.15.110:0/937641862 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86541602b0 con 0x7f86540b1c60 2024-09-10T10:04:02.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8653e00640 1 --2- 172.21.15.110:0/937641862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654142320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8644009870 tx=0x7f864401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1807a554dc0849a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8652a00640 1 -- 172.21.15.110:0/937641862 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440b2070 con 0x7f86540b1c60 2024-09-10T10:04:02.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8652a00640 1 -- 172.21.15.110:0/937641862 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86440a9070 con 0x7f86540b1c60 2024-09-10T10:04:02.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8652a00640 1 -- 172.21.15.110:0/937641862 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440ad880 con 0x7f86540b1c60 2024-09-10T10:04:02.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 -- 172.21.15.110:0/937641862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 msgr2=0x7f8654142320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 --2- 172.21.15.110:0/937641862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654142320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8644009870 tx=0x7f864401c560 comp rx=0 tx=0).stop 2024-09-10T10:04:02.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 -- 172.21.15.110:0/937641862 shutdown_connections 2024-09-10T10:04:02.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 --2- 172.21.15.110:0/937641862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654142320 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 -- 172.21.15.110:0/937641862 >> 172.21.15.110:0/937641862 conn(0x7f86540b42d0 msgr2=0x7f86540b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:02.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 -- 172.21.15.110:0/937641862 shutdown_connections 2024-09-10T10:04:02.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 -- 172.21.15.110:0/937641862 wait complete. 2024-09-10T10:04:02.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 Processor -- start 2024-09-10T10:04:02.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.270+0000 7f8659600640 1 -- start start 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86540731e0 0x7f865415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f865415fd80 0x7f865415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8654073a00 con 0x7f86540b1c60 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f865415d770 con 0x7f86540731e0 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f865415e350 con 0x7f865415fd80 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50480/0 (socket says 172.21.15.110:50480) 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 -- 172.21.15.110:0/3568229568 learned_addr learned my addr 172.21.15.110:0/3568229568 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653e00640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86540731e0 0x7f865415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8658c00640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f865415fd80 0x7f865415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 -- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f865415fd80 msgr2=0x7f865415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f865415fd80 0x7f865415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 -- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86540731e0 msgr2=0x7f865415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86540731e0 0x7f865415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 -- 172.21.15.110:0/3568229568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865415da20 con 0x7f86540b1c60 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8658c00640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f865415fd80 0x7f865415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8653400640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654152b60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f864000c370 tx=0x7f864000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f864be00640 1 -- 172.21.15.110:0/3568229568 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400052d0 con 0x7f86540b1c60 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f864be00640 1 -- 172.21.15.110:0/3568229568 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8640005470 con 0x7f86540b1c60 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865415dd00 con 0x7f86540b1c60 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f864be00640 1 -- 172.21.15.110:0/3568229568 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640007260 con 0x7f86540b1c60 2024-09-10T10:04:02.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865415f780 con 0x7f86540b1c60 2024-09-10T10:04:02.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.274+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8654140a40 con 0x7f86540b1c60 2024-09-10T10:04:02.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.278+0000 7f864be00640 1 -- 172.21.15.110:0/3568229568 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f8640007400 con 0x7f86540b1c60 2024-09-10T10:04:02.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.278+0000 7f864be00640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8634067cb0 0x7f863406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.278+0000 7f864be00640 1 -- 172.21.15.110:0/3568229568 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f8640085630 con 0x7f86540b1c60 2024-09-10T10:04:02.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.278+0000 7f8653e00640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8634067cb0 0x7f863406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.278+0000 7f8653e00640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8634067cb0 0x7f863406a170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86440af000 tx=0x7f86440a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.282+0000 7f864be00640 1 -- 172.21.15.110:0/3568229568 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f86400556b0 con 0x7f86540b1c60 2024-09-10T10:04:02.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.438+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8654140760 con 0x7f86540b1c60 2024-09-10T10:04:02.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f864be00640 1 -- 172.21.15.110:0/3568229568 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16819 (secure 0 0 0) 0x7f86400599b0 con 0x7f86540b1c60 2024-09-10T10:04:02.446 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:02.446 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":117,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:03:59.729984+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:55.813835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:57.093856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:02.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8634067cb0 msgr2=0x7f863406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8634067cb0 0x7f863406a170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86440af000 tx=0x7f86440a7000 comp rx=0 tx=0).stop 2024-09-10T10:04:02.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 msgr2=0x7f8654152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654152b60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f864000c370 tx=0x7f864000c830 comp rx=0 tx=0).stop 2024-09-10T10:04:02.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 shutdown_connections 2024-09-10T10:04:02.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8634067cb0 0x7f863406a170 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f865415fd80 0x7f865415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86540b1c60 0x7f8654152b60 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 --2- 172.21.15.110:0/3568229568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86540731e0 0x7f865415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 >> 172.21.15.110:0/3568229568 conn(0x7f86540b42d0 msgr2=0x7f86540b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:02.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 shutdown_connections 2024-09-10T10:04:02.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.442+0000 7f8659600640 1 -- 172.21.15.110:0/3568229568 wait complete. 2024-09-10T10:04:02.467 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:04:02.468 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:04:02.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b0b400640 1 Processor -- start 2024-09-10T10:04:02.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b0b400640 1 -- start start 2024-09-10T10:04:02.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b0b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5b0415c560 0x7f5b0406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b0b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5b0406f010 0x7f5b040ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b0b400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0406f430 con 0x7f5b04155970 2024-09-10T10:04:02.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b0b400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5b040b4570 con 0x7f5b0415c560 2024-09-10T10:04:02.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b0b400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5b04155d50 con 0x7f5b0406f010 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b03e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5b0415c560 0x7f5b0406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b03e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5b0415c560 0x7f5b0406ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50492/0 (socket says 172.21.15.110:50492) 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b03e00640 1 -- 172.21.15.110:0/3442876769 learned_addr learned my addr 172.21.15.110:0/3442876769 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b09400640 1 --2- 172.21.15.110:0/3442876769 >> v2:172.21.15.145:3300/0 conn(0x7f5b0406f010 0x7f5b040ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b03e00640 1 -- 172.21.15.110:0/3442876769 >> v2:172.21.15.145:3300/0 conn(0x7f5b0406f010 msgr2=0x7f5b040ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b03e00640 1 --2- 172.21.15.110:0/3442876769 >> v2:172.21.15.145:3300/0 conn(0x7f5b0406f010 0x7f5b040ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b03e00640 1 -- 172.21.15.110:0/3442876769 >> v1:172.21.15.145:6789/0 conn(0x7f5b04155970 legacy=0x7f5b040cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b03e00640 1 -- 172.21.15.110:0/3442876769 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b040b34d0 con 0x7f5b0415c560 2024-09-10T10:04:02.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.598+0000 7f5b09400640 1 --2- 172.21.15.110:0/3442876769 >> v2:172.21.15.145:3300/0 conn(0x7f5b0406f010 0x7f5b040ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:02.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b03e00640 1 --2- 172.21.15.110:0/3442876769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b0406ead0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5b040735e0 tx=0x7f5af001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9693a32f15c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b03400640 1 -- 172.21.15.110:0/3442876769 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af00b4070 con 0x7f5b0415c560 2024-09-10T10:04:02.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b03400640 1 -- 172.21.15.110:0/3442876769 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af00a7c50 con 0x7f5b0415c560 2024-09-10T10:04:02.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b03400640 1 -- 172.21.15.110:0/3442876769 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af00af440 con 0x7f5b0415c560 2024-09-10T10:04:02.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b0b400640 1 -- 172.21.15.110:0/3442876769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 msgr2=0x7f5b0406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b0b400640 1 --2- 172.21.15.110:0/3442876769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b0406ead0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5b040735e0 tx=0x7f5af001eb80 comp rx=0 tx=0).stop 2024-09-10T10:04:02.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b0b400640 1 -- 172.21.15.110:0/3442876769 shutdown_connections 2024-09-10T10:04:02.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b0b400640 1 --2- 172.21.15.110:0/3442876769 >> v2:172.21.15.145:3300/0 conn(0x7f5b0406f010 0x7f5b040ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b0b400640 1 --2- 172.21.15.110:0/3442876769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b0406ead0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b0b400640 1 -- 172.21.15.110:0/3442876769 >> 172.21.15.110:0/3442876769 conn(0x7f5b0406bc50 msgr2=0x7f5b0406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:02.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b0b400640 1 -- 172.21.15.110:0/3442876769 shutdown_connections 2024-09-10T10:04:02.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.602+0000 7f5b0b400640 1 -- 172.21.15.110:0/3442876769 wait complete. 2024-09-10T10:04:02.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 Processor -- start 2024-09-10T10:04:02.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 -- start start 2024-09-10T10:04:02.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b0406f010 0x7f5b0414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5b04155970 0x7f5b0415f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b041569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b040cf170 con 0x7f5b0415c560 2024-09-10T10:04:02.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b040b4050 con 0x7f5b0406f010 2024-09-10T10:04:02.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5b040cf2f0 con 0x7f5b04155970 2024-09-10T10:04:02.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b041569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b041569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50504/0 (socket says 172.21.15.110:50504) 2024-09-10T10:04:02.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 -- 172.21.15.110:0/4260521999 learned_addr learned my addr 172.21.15.110:0/4260521999 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:02.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b08a00640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b0406f010 0x7f5b0414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b03e00640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5b04155970 0x7f5b0415f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 -- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5b04155970 msgr2=0x7f5b0415f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5b04155970 0x7f5b0415f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 -- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b0406f010 msgr2=0x7f5b0414f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b0406f010 0x7f5b0414f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 -- 172.21.15.110:0/4260521999 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0415f970 con 0x7f5b0415c560 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b09400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b041569b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c7f0 tx=0x7f5afc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b00c00640 1 -- 172.21.15.110:0/4260521999 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc015070 con 0x7f5b0415c560 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b00c00640 1 -- 172.21.15.110:0/4260521999 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc007860 con 0x7f5b0415c560 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b00c00640 1 -- 172.21.15.110:0/4260521999 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc007c50 con 0x7f5b0415c560 2024-09-10T10:04:02.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0415e980 con 0x7f5b0415c560 2024-09-10T10:04:02.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.606+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b04155ed0 con 0x7f5b0415c560 2024-09-10T10:04:02.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.610+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b04069bb0 con 0x7f5b0415c560 2024-09-10T10:04:02.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.610+0000 7f5b00c00640 1 -- 172.21.15.110:0/4260521999 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f5afc018070 con 0x7f5b0415c560 2024-09-10T10:04:02.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.610+0000 7f5b00c00640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5ae4067cb0 0x7f5ae406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.610+0000 7f5b00c00640 1 -- 172.21.15.110:0/4260521999 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f5afc086650 con 0x7f5b0415c560 2024-09-10T10:04:02.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.610+0000 7f5b08a00640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5ae4067cb0 0x7f5ae406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.610+0000 7f5b08a00640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5ae4067cb0 0x7f5ae406a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b04076370 tx=0x7f5af4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.614+0000 7f5b00c00640 1 -- 172.21.15.110:0/4260521999 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5afc056ad0 con 0x7f5b0415c560 2024-09-10T10:04:02.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.770+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b040cf680 con 0x7f5ae4067cb0 2024-09-10T10:04:02.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.774+0000 7f5b00c00640 1 -- 172.21.15.110:0/4260521999 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f5b040cf680 con 0x7f5ae4067cb0 2024-09-10T10:04:02.782 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:02.784 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:04:02.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5ae4067cb0 msgr2=0x7f5ae406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5ae4067cb0 0x7f5ae406a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b04076370 tx=0x7f5af4009040 comp rx=0 tx=0).stop 2024-09-10T10:04:02.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 msgr2=0x7f5b041569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b041569b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c7f0 tx=0x7f5afc00ccb0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 shutdown_connections 2024-09-10T10:04:02.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5ae4067cb0 0x7f5ae406a170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b0415c560 0x7f5b041569b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5b04155970 0x7f5b0415f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 --2- 172.21.15.110:0/4260521999 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b0406f010 0x7f5b0414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 >> 172.21.15.110:0/4260521999 conn(0x7f5b0406bc50 msgr2=0x7f5b0406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:02.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 shutdown_connections 2024-09-10T10:04:02.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.778+0000 7f5b0b400640 1 -- 172.21.15.110:0/4260521999 wait complete. 2024-09-10T10:04:02.802 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-10T10:04:01.716089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270428,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704292,"statfs":{"total":773094113280,"available":772817195008,"internally_reserved":0,"allocated":12886016,"data_stored":6293116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":263835180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.012428"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605273+0000","last_change":"2024-09-10T10:03:58.757686+0000","last_active":"2024-09-10T10:04:00.605273+0000","last_peered":"2024-09-10T10:04:00.605273+0000","last_clean":"2024-09-10T10:04:00.605273+0000","last_became_active":"2024-09-10T10:03:58.756250+0000","last_became_peered":"2024-09-10T10:03:58.756250+0000","last_unstale":"2024-09-10T10:04:00.605273+0000","last_undegraded":"2024-09-10T10:04:00.605273+0000","last_fullsized":"2024-09-10T10:04:00.605273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:19.336634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357047+0000","last_change":"2024-09-10T10:03:58.757540+0000","last_active":"2024-09-10T10:04:00.357047+0000","last_peered":"2024-09-10T10:04:00.357047+0000","last_clean":"2024-09-10T10:04:00.357047+0000","last_became_active":"2024-09-10T10:03:58.757247+0000","last_became_peered":"2024-09-10T10:03:58.757247+0000","last_unstale":"2024-09-10T10:04:00.357047+0000","last_undegraded":"2024-09-10T10:04:00.357047+0000","last_fullsized":"2024-09-10T10:04:00.357047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:03.321352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740838+0000","last_change":"2024-09-10T10:03:58.747149+0000","last_active":"2024-09-10T10:03:59.740838+0000","last_peered":"2024-09-10T10:03:59.740838+0000","last_clean":"2024-09-10T10:03:59.740838+0000","last_became_active":"2024-09-10T10:03:58.746862+0000","last_became_peered":"2024-09-10T10:03:58.746862+0000","last_unstale":"2024-09-10T10:03:59.740838+0000","last_undegraded":"2024-09-10T10:03:59.740838+0000","last_fullsized":"2024-09-10T10:03:59.740838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:15.755172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605044+0000","last_change":"2024-09-10T10:03:58.746228+0000","last_active":"2024-09-10T10:04:00.605044+0000","last_peered":"2024-09-10T10:04:00.605044+0000","last_clean":"2024-09-10T10:04:00.605044+0000","last_became_active":"2024-09-10T10:03:58.746027+0000","last_became_peered":"2024-09-10T10:03:58.746027+0000","last_unstale":"2024-09-10T10:04:00.605044+0000","last_undegraded":"2024-09-10T10:04:00.605044+0000","last_fullsized":"2024-09-10T10:04:00.605044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:20.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022413+0000","last_change":"2024-09-10T10:03:58.740901+0000","last_active":"2024-09-10T10:04:00.022413+0000","last_peered":"2024-09-10T10:04:00.022413+0000","last_clean":"2024-09-10T10:04:00.022413+0000","last_became_active":"2024-09-10T10:03:58.740388+0000","last_became_peered":"2024-09-10T10:03:58.740388+0000","last_unstale":"2024-09-10T10:04:00.022413+0000","last_undegraded":"2024-09-10T10:04:00.022413+0000","last_fullsized":"2024-09-10T10:04:00.022413+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:37.996548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739882+0000","last_change":"2024-09-10T10:03:58.743514+0000","last_active":"2024-09-10T10:03:59.739882+0000","last_peered":"2024-09-10T10:03:59.739882+0000","last_clean":"2024-09-10T10:03:59.739882+0000","last_became_active":"2024-09-10T10:03:58.743189+0000","last_became_peered":"2024-09-10T10:03:58.743189+0000","last_unstale":"2024-09-10T10:03:59.739882+0000","last_undegraded":"2024-09-10T10:03:59.739882+0000","last_fullsized":"2024-09-10T10:03:59.739882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:34.470602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739567+0000","last_change":"2024-09-10T10:03:58.744711+0000","last_active":"2024-09-10T10:03:59.739567+0000","last_peered":"2024-09-10T10:03:59.739567+0000","last_clean":"2024-09-10T10:03:59.739567+0000","last_became_active":"2024-09-10T10:03:58.744424+0000","last_became_peered":"2024-09-10T10:03:58.744424+0000","last_unstale":"2024-09-10T10:03:59.739567+0000","last_undegraded":"2024-09-10T10:03:59.739567+0000","last_fullsized":"2024-09-10T10:03:59.739567+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:07.188145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739862+0000","last_change":"2024-09-10T10:03:58.745002+0000","last_active":"2024-09-10T10:03:59.739862+0000","last_peered":"2024-09-10T10:03:59.739862+0000","last_clean":"2024-09-10T10:03:59.739862+0000","last_became_active":"2024-09-10T10:03:58.741497+0000","last_became_peered":"2024-09-10T10:03:58.741497+0000","last_unstale":"2024-09-10T10:03:59.739862+0000","last_undegraded":"2024-09-10T10:03:59.739862+0000","last_fullsized":"2024-09-10T10:03:59.739862+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:54.517670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741206+0000","last_change":"2024-09-10T10:03:58.741533+0000","last_active":"2024-09-10T10:03:59.741206+0000","last_peered":"2024-09-10T10:03:59.741206+0000","last_clean":"2024-09-10T10:03:59.741206+0000","last_became_active":"2024-09-10T10:03:58.741171+0000","last_became_peered":"2024-09-10T10:03:58.741171+0000","last_unstale":"2024-09-10T10:03:59.741206+0000","last_undegraded":"2024-09-10T10:03:59.741206+0000","last_fullsized":"2024-09-10T10:03:59.741206+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:42.036490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357097+0000","last_change":"2024-09-10T10:03:58.757376+0000","last_active":"2024-09-10T10:04:00.357097+0000","last_peered":"2024-09-10T10:04:00.357097+0000","last_clean":"2024-09-10T10:04:00.357097+0000","last_became_active":"2024-09-10T10:03:58.757110+0000","last_became_peered":"2024-09-10T10:03:58.757110+0000","last_unstale":"2024-09-10T10:04:00.357097+0000","last_undegraded":"2024-09-10T10:04:00.357097+0000","last_fullsized":"2024-09-10T10:04:00.357097+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:09.788894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989512+0000","last_change":"2024-09-10T10:03:58.763248+0000","last_active":"2024-09-10T10:04:00.989512+0000","last_peered":"2024-09-10T10:04:00.989512+0000","last_clean":"2024-09-10T10:04:00.989512+0000","last_became_active":"2024-09-10T10:03:58.762938+0000","last_became_peered":"2024-09-10T10:03:58.762938+0000","last_unstale":"2024-09-10T10:04:00.989512+0000","last_undegraded":"2024-09-10T10:04:00.989512+0000","last_fullsized":"2024-09-10T10:04:00.989512+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:54.062122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740568+0000","last_change":"2024-09-10T10:03:58.742063+0000","last_active":"2024-09-10T10:03:59.740568+0000","last_peered":"2024-09-10T10:03:59.740568+0000","last_clean":"2024-09-10T10:03:59.740568+0000","last_became_active":"2024-09-10T10:03:58.741760+0000","last_became_peered":"2024-09-10T10:03:58.741760+0000","last_unstale":"2024-09-10T10:03:59.740568+0000","last_undegraded":"2024-09-10T10:03:59.740568+0000","last_fullsized":"2024-09-10T10:03:59.740568+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:34.470602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605030+0000","last_change":"2024-09-10T10:03:58.752792+0000","last_active":"2024-09-10T10:04:00.605030+0000","last_peered":"2024-09-10T10:04:00.605030+0000","last_clean":"2024-09-10T10:04:00.605030+0000","last_became_active":"2024-09-10T10:03:58.752525+0000","last_became_peered":"2024-09-10T10:03:58.752525+0000","last_unstale":"2024-09-10T10:04:00.605030+0000","last_undegraded":"2024-09-10T10:04:00.605030+0000","last_fullsized":"2024-09-10T10:04:00.605030+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:09.597632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356720+0000","last_change":"2024-09-10T10:03:58.761971+0000","last_active":"2024-09-10T10:04:00.356720+0000","last_peered":"2024-09-10T10:04:00.356720+0000","last_clean":"2024-09-10T10:04:00.356720+0000","last_became_active":"2024-09-10T10:03:58.756915+0000","last_became_peered":"2024-09-10T10:03:58.756915+0000","last_unstale":"2024-09-10T10:04:00.356720+0000","last_undegraded":"2024-09-10T10:04:00.356720+0000","last_fullsized":"2024-09-10T10:04:00.356720+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:07.445143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.604872+0000","last_change":"2024-09-10T10:03:58.744977+0000","last_active":"2024-09-10T10:04:00.604872+0000","last_peered":"2024-09-10T10:04:00.604872+0000","last_clean":"2024-09-10T10:04:00.604872+0000","last_became_active":"2024-09-10T10:03:58.744681+0000","last_became_peered":"2024-09-10T10:03:58.744681+0000","last_unstale":"2024-09-10T10:04:00.604872+0000","last_undegraded":"2024-09-10T10:04:00.604872+0000","last_fullsized":"2024-09-10T10:04:00.604872+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:58.300431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737770+0000","last_change":"2024-09-10T10:03:58.753812+0000","last_active":"2024-09-10T10:03:59.737770+0000","last_peered":"2024-09-10T10:03:59.737770+0000","last_clean":"2024-09-10T10:03:59.737770+0000","last_became_active":"2024-09-10T10:03:58.753555+0000","last_became_peered":"2024-09-10T10:03:58.753555+0000","last_unstale":"2024-09-10T10:03:59.737770+0000","last_undegraded":"2024-09-10T10:03:59.737770+0000","last_fullsized":"2024-09-10T10:03:59.737770+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:20.244238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740498+0000","last_change":"2024-09-10T10:03:58.739905+0000","last_active":"2024-09-10T10:03:59.740498+0000","last_peered":"2024-09-10T10:03:59.740498+0000","last_clean":"2024-09-10T10:03:59.740498+0000","last_became_active":"2024-09-10T10:03:58.739640+0000","last_became_peered":"2024-09-10T10:03:58.739640+0000","last_unstale":"2024-09-10T10:03:59.740498+0000","last_undegraded":"2024-09-10T10:03:59.740498+0000","last_fullsized":"2024-09-10T10:03:59.740498+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:42.036490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738411+0000","last_change":"2024-09-10T10:03:58.753277+0000","last_active":"2024-09-10T10:03:59.738411+0000","last_peered":"2024-09-10T10:03:59.738411+0000","last_clean":"2024-09-10T10:03:59.738411+0000","last_became_active":"2024-09-10T10:03:58.753038+0000","last_became_peered":"2024-09-10T10:03:58.753038+0000","last_unstale":"2024-09-10T10:03:59.738411+0000","last_undegraded":"2024-09-10T10:03:59.738411+0000","last_fullsized":"2024-09-10T10:03:59.738411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:07.566000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989538+0000","last_change":"2024-09-10T10:03:58.758170+0000","last_active":"2024-09-10T10:04:00.989538+0000","last_peered":"2024-09-10T10:04:00.989538+0000","last_clean":"2024-09-10T10:04:00.989538+0000","last_became_active":"2024-09-10T10:03:58.757820+0000","last_became_peered":"2024-09-10T10:03:58.757820+0000","last_unstale":"2024-09-10T10:04:00.989538+0000","last_undegraded":"2024-09-10T10:04:00.989538+0000","last_fullsized":"2024-09-10T10:04:00.989538+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:41.391186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738050+0000","last_change":"2024-09-10T10:03:58.762564+0000","last_active":"2024-09-10T10:03:59.738050+0000","last_peered":"2024-09-10T10:03:59.738050+0000","last_clean":"2024-09-10T10:03:59.738050+0000","last_became_active":"2024-09-10T10:03:58.762289+0000","last_became_peered":"2024-09-10T10:03:58.762289+0000","last_unstale":"2024-09-10T10:03:59.738050+0000","last_undegraded":"2024-09-10T10:03:59.738050+0000","last_fullsized":"2024-09-10T10:03:59.738050+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:35.272830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022183+0000","last_change":"2024-09-10T10:03:58.740663+0000","last_active":"2024-09-10T10:04:00.022183+0000","last_peered":"2024-09-10T10:04:00.022183+0000","last_clean":"2024-09-10T10:04:00.022183+0000","last_became_active":"2024-09-10T10:03:58.739953+0000","last_became_peered":"2024-09-10T10:03:58.739953+0000","last_unstale":"2024-09-10T10:04:00.022183+0000","last_undegraded":"2024-09-10T10:04:00.022183+0000","last_fullsized":"2024-09-10T10:04:00.022183+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:00.549552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989295+0000","last_change":"2024-09-10T10:03:58.764137+0000","last_active":"2024-09-10T10:04:00.989295+0000","last_peered":"2024-09-10T10:04:00.989295+0000","last_clean":"2024-09-10T10:04:00.989295+0000","last_became_active":"2024-09-10T10:03:58.763782+0000","last_became_peered":"2024-09-10T10:03:58.763782+0000","last_unstale":"2024-09-10T10:04:00.989295+0000","last_undegraded":"2024-09-10T10:04:00.989295+0000","last_fullsized":"2024-09-10T10:04:00.989295+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:05.264657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740628+0000","last_change":"2024-09-10T10:03:58.743508+0000","last_active":"2024-09-10T10:03:59.740628+0000","last_peered":"2024-09-10T10:03:59.740628+0000","last_clean":"2024-09-10T10:03:59.740628+0000","last_became_active":"2024-09-10T10:03:58.743224+0000","last_became_peered":"2024-09-10T10:03:58.743224+0000","last_unstale":"2024-09-10T10:03:59.740628+0000","last_undegraded":"2024-09-10T10:03:59.740628+0000","last_fullsized":"2024-09-10T10:03:59.740628+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:27.634373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989276+0000","last_change":"2024-09-10T10:03:58.741356+0000","last_active":"2024-09-10T10:04:00.989276+0000","last_peered":"2024-09-10T10:04:00.989276+0000","last_clean":"2024-09-10T10:04:00.989276+0000","last_became_active":"2024-09-10T10:03:58.741087+0000","last_became_peered":"2024-09-10T10:03:58.741087+0000","last_unstale":"2024-09-10T10:04:00.989276+0000","last_undegraded":"2024-09-10T10:04:00.989276+0000","last_fullsized":"2024-09-10T10:04:00.989276+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:27.401807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357105+0000","last_change":"2024-09-10T10:03:58.755723+0000","last_active":"2024-09-10T10:04:00.357105+0000","last_peered":"2024-09-10T10:04:00.357105+0000","last_clean":"2024-09-10T10:04:00.357105+0000","last_became_active":"2024-09-10T10:03:58.755475+0000","last_became_peered":"2024-09-10T10:03:58.755475+0000","last_unstale":"2024-09-10T10:04:00.357105+0000","last_undegraded":"2024-09-10T10:04:00.357105+0000","last_fullsized":"2024-09-10T10:04:00.357105+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:57.149687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738416+0000","last_change":"2024-09-10T10:03:58.753138+0000","last_active":"2024-09-10T10:03:59.738416+0000","last_peered":"2024-09-10T10:03:59.738416+0000","last_clean":"2024-09-10T10:03:59.738416+0000","last_became_active":"2024-09-10T10:03:58.752664+0000","last_became_peered":"2024-09-10T10:03:58.752664+0000","last_unstale":"2024-09-10T10:03:59.738416+0000","last_undegraded":"2024-09-10T10:03:59.738416+0000","last_fullsized":"2024-09-10T10:03:59.738416+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:16.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605199+0000","last_change":"2024-09-10T10:03:58.751327+0000","last_active":"2024-09-10T10:04:00.605199+0000","last_peered":"2024-09-10T10:04:00.605199+0000","last_clean":"2024-09-10T10:04:00.605199+0000","last_became_active":"2024-09-10T10:03:58.750908+0000","last_became_peered":"2024-09-10T10:03:58.750908+0000","last_unstale":"2024-09-10T10:04:00.605199+0000","last_undegraded":"2024-09-10T10:04:00.605199+0000","last_fullsized":"2024-09-10T10:04:00.605199+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:51.530530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572284+0000","last_change":"2024-09-10T10:03:58.739405+0000","last_active":"2024-09-10T10:04:00.572284+0000","last_peered":"2024-09-10T10:04:00.572284+0000","last_clean":"2024-09-10T10:04:00.572284+0000","last_became_active":"2024-09-10T10:03:58.738947+0000","last_became_peered":"2024-09-10T10:03:58.738947+0000","last_unstale":"2024-09-10T10:04:00.572284+0000","last_undegraded":"2024-09-10T10:04:00.572284+0000","last_fullsized":"2024-09-10T10:04:00.572284+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:57.149687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605047+0000","last_change":"2024-09-10T10:03:58.746915+0000","last_active":"2024-09-10T10:04:00.605047+0000","last_peered":"2024-09-10T10:04:00.605047+0000","last_clean":"2024-09-10T10:04:00.605047+0000","last_became_active":"2024-09-10T10:03:58.746607+0000","last_became_peered":"2024-09-10T10:03:58.746607+0000","last_unstale":"2024-09-10T10:04:00.605047+0000","last_undegraded":"2024-09-10T10:04:00.605047+0000","last_fullsized":"2024-09-10T10:04:00.605047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:39.026866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.604951+0000","last_change":"2024-09-10T10:03:58.745328+0000","last_active":"2024-09-10T10:04:00.604951+0000","last_peered":"2024-09-10T10:04:00.604951+0000","last_clean":"2024-09-10T10:04:00.604951+0000","last_became_active":"2024-09-10T10:03:58.745050+0000","last_became_peered":"2024-09-10T10:03:58.745050+0000","last_unstale":"2024-09-10T10:04:00.604951+0000","last_undegraded":"2024-09-10T10:04:00.604951+0000","last_fullsized":"2024-09-10T10:04:00.604951+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:54.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022112+0000","last_change":"2024-09-10T10:03:58.747797+0000","last_active":"2024-09-10T10:04:00.022112+0000","last_peered":"2024-09-10T10:04:00.022112+0000","last_clean":"2024-09-10T10:04:00.022112+0000","last_became_active":"2024-09-10T10:03:58.747501+0000","last_became_peered":"2024-09-10T10:03:58.747501+0000","last_unstale":"2024-09-10T10:04:00.022112+0000","last_undegraded":"2024-09-10T10:04:00.022112+0000","last_fullsized":"2024-09-10T10:04:00.022112+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:00.526867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740124+0000","last_change":"2024-09-10T10:03:58.737821+0000","last_active":"2024-09-10T10:03:59.740124+0000","last_peered":"2024-09-10T10:03:59.740124+0000","last_clean":"2024-09-10T10:03:59.740124+0000","last_became_active":"2024-09-10T10:03:58.737471+0000","last_became_peered":"2024-09-10T10:03:58.737471+0000","last_unstale":"2024-09-10T10:03:59.740124+0000","last_undegraded":"2024-09-10T10:03:59.740124+0000","last_fullsized":"2024-09-10T10:03:59.740124+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:27.634373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738486+0000","last_change":"2024-09-10T10:03:58.741943+0000","last_active":"2024-09-10T10:03:59.738486+0000","last_peered":"2024-09-10T10:03:59.738486+0000","last_clean":"2024-09-10T10:03:59.738486+0000","last_became_active":"2024-09-10T10:03:58.741299+0000","last_became_peered":"2024-09-10T10:03:58.741299+0000","last_unstale":"2024-09-10T10:03:59.738486+0000","last_undegraded":"2024-09-10T10:03:59.738486+0000","last_fullsized":"2024-09-10T10:03:59.738486+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:07.887129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"116'4","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022488+0000","last_change":"2024-09-10T10:03:57.733868+0000","last_active":"2024-09-10T10:04:00.022488+0000","last_peered":"2024-09-10T10:04:00.022488+0000","last_clean":"2024-09-10T10:04:00.022488+0000","last_became_active":"2024-09-10T10:03:57.733606+0000","last_became_peered":"2024-09-10T10:03:57.733606+0000","last_unstale":"2024-09-10T10:04:00.022488+0000","last_undegraded":"2024-09-10T10:04:00.022488+0000","last_fullsized":"2024-09-10T10:04:00.022488+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:18.217435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741272+0000","last_change":"2024-09-10T10:03:58.749838+0000","last_active":"2024-09-10T10:03:59.741272+0000","last_peered":"2024-09-10T10:03:59.741272+0000","last_clean":"2024-09-10T10:03:59.741272+0000","last_became_active":"2024-09-10T10:03:58.746742+0000","last_became_peered":"2024-09-10T10:03:58.746742+0000","last_unstale":"2024-09-10T10:03:59.741272+0000","last_undegraded":"2024-09-10T10:03:59.741272+0000","last_fullsized":"2024-09-10T10:03:59.741272+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:11.829596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572404+0000","last_change":"2024-09-10T10:03:57.734427+0000","last_active":"2024-09-10T10:04:00.572404+0000","last_peered":"2024-09-10T10:04:00.572404+0000","last_clean":"2024-09-10T10:04:00.572404+0000","last_became_active":"2024-09-10T10:03:57.734144+0000","last_became_peered":"2024-09-10T10:03:57.734144+0000","last_unstale":"2024-09-10T10:04:00.572404+0000","last_undegraded":"2024-09-10T10:04:00.572404+0000","last_fullsized":"2024-09-10T10:04:00.572404+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:15.520169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022439+0000","last_change":"2024-09-10T10:03:58.758463+0000","last_active":"2024-09-10T10:04:00.022439+0000","last_peered":"2024-09-10T10:04:00.022439+0000","last_clean":"2024-09-10T10:04:00.022439+0000","last_became_active":"2024-09-10T10:03:58.755771+0000","last_became_peered":"2024-09-10T10:03:58.755771+0000","last_unstale":"2024-09-10T10:04:00.022439+0000","last_undegraded":"2024-09-10T10:04:00.022439+0000","last_fullsized":"2024-09-10T10:04:00.022439+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:22.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989568+0000","last_change":"2024-09-10T10:03:57.725070+0000","last_active":"2024-09-10T10:04:00.989568+0000","last_peered":"2024-09-10T10:04:00.989568+0000","last_clean":"2024-09-10T10:04:00.989568+0000","last_became_active":"2024-09-10T10:03:57.724795+0000","last_became_peered":"2024-09-10T10:03:57.724795+0000","last_unstale":"2024-09-10T10:04:00.989568+0000","last_undegraded":"2024-09-10T10:04:00.989568+0000","last_fullsized":"2024-09-10T10:04:00.989568+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:03.379284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022320+0000","last_change":"2024-09-10T10:03:58.754249+0000","last_active":"2024-09-10T10:04:00.022320+0000","last_peered":"2024-09-10T10:04:00.022320+0000","last_clean":"2024-09-10T10:04:00.022320+0000","last_became_active":"2024-09-10T10:03:58.753894+0000","last_became_peered":"2024-09-10T10:03:58.753894+0000","last_unstale":"2024-09-10T10:04:00.022320+0000","last_undegraded":"2024-09-10T10:04:00.022320+0000","last_fullsized":"2024-09-10T10:04:00.022320+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:07.929163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357053+0000","last_change":"2024-09-10T10:03:57.730941+0000","last_active":"2024-09-10T10:04:00.357053+0000","last_peered":"2024-09-10T10:04:00.357053+0000","last_clean":"2024-09-10T10:04:00.357053+0000","last_became_active":"2024-09-10T10:03:57.730647+0000","last_became_peered":"2024-09-10T10:03:57.730647+0000","last_unstale":"2024-09-10T10:04:00.357053+0000","last_undegraded":"2024-09-10T10:04:00.357053+0000","last_fullsized":"2024-09-10T10:04:00.357053+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:04.226652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737801+0000","last_change":"2024-09-10T10:03:58.759861+0000","last_active":"2024-09-10T10:03:59.737801+0000","last_peered":"2024-09-10T10:03:59.737801+0000","last_clean":"2024-09-10T10:03:59.737801+0000","last_became_active":"2024-09-10T10:03:58.759592+0000","last_became_peered":"2024-09-10T10:03:58.759592+0000","last_unstale":"2024-09-10T10:03:59.737801+0000","last_undegraded":"2024-09-10T10:03:59.737801+0000","last_fullsized":"2024-09-10T10:03:59.737801+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:20.615050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356697+0000","last_change":"2024-09-10T10:03:57.728434+0000","last_active":"2024-09-10T10:04:00.356697+0000","last_peered":"2024-09-10T10:04:00.356697+0000","last_clean":"2024-09-10T10:04:00.356697+0000","last_became_active":"2024-09-10T10:03:57.728160+0000","last_became_peered":"2024-09-10T10:03:57.728160+0000","last_unstale":"2024-09-10T10:04:00.356697+0000","last_undegraded":"2024-09-10T10:04:00.356697+0000","last_fullsized":"2024-09-10T10:04:00.356697+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:28.102915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740987+0000","last_change":"2024-09-10T10:03:58.740526+0000","last_active":"2024-09-10T10:03:59.740987+0000","last_peered":"2024-09-10T10:03:59.740987+0000","last_clean":"2024-09-10T10:03:59.740987+0000","last_became_active":"2024-09-10T10:03:58.740217+0000","last_became_peered":"2024-09-10T10:03:58.740217+0000","last_unstale":"2024-09-10T10:03:59.740987+0000","last_undegraded":"2024-09-10T10:03:59.740987+0000","last_fullsized":"2024-09-10T10:03:59.740987+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:11.829596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737904+0000","last_change":"2024-09-10T10:03:57.728618+0000","last_active":"2024-09-10T10:03:59.737904+0000","last_peered":"2024-09-10T10:03:59.737904+0000","last_clean":"2024-09-10T10:03:59.737904+0000","last_became_active":"2024-09-10T10:03:57.728326+0000","last_became_peered":"2024-09-10T10:03:57.728326+0000","last_unstale":"2024-09-10T10:03:59.737904+0000","last_undegraded":"2024-09-10T10:03:59.737904+0000","last_fullsized":"2024-09-10T10:03:59.737904+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:03.701145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739963+0000","last_change":"2024-09-10T10:03:58.748663+0000","last_active":"2024-09-10T10:03:59.739963+0000","last_peered":"2024-09-10T10:03:59.739963+0000","last_clean":"2024-09-10T10:03:59.739963+0000","last_became_active":"2024-09-10T10:03:58.741211+0000","last_became_peered":"2024-09-10T10:03:58.741211+0000","last_unstale":"2024-09-10T10:03:59.739963+0000","last_undegraded":"2024-09-10T10:03:59.739963+0000","last_fullsized":"2024-09-10T10:03:59.739963+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:11.616816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739633+0000","last_change":"2024-09-10T10:03:57.730168+0000","last_active":"2024-09-10T10:03:59.739633+0000","last_peered":"2024-09-10T10:03:59.739633+0000","last_clean":"2024-09-10T10:03:59.739633+0000","last_became_active":"2024-09-10T10:03:57.729878+0000","last_became_peered":"2024-09-10T10:03:57.729878+0000","last_unstale":"2024-09-10T10:03:59.739633+0000","last_undegraded":"2024-09-10T10:03:59.739633+0000","last_fullsized":"2024-09-10T10:03:59.739633+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:31.293427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356738+0000","last_change":"2024-09-10T10:03:58.761674+0000","last_active":"2024-09-10T10:04:00.356738+0000","last_peered":"2024-09-10T10:04:00.356738+0000","last_clean":"2024-09-10T10:04:00.356738+0000","last_became_active":"2024-09-10T10:03:58.756662+0000","last_became_peered":"2024-09-10T10:03:58.756662+0000","last_unstale":"2024-09-10T10:04:00.356738+0000","last_undegraded":"2024-09-10T10:04:00.356738+0000","last_fullsized":"2024-09-10T10:04:00.356738+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:51.425956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356814+0000","last_change":"2024-09-10T10:03:57.730627+0000","last_active":"2024-09-10T10:04:00.356814+0000","last_peered":"2024-09-10T10:04:00.356814+0000","last_clean":"2024-09-10T10:04:00.356814+0000","last_became_active":"2024-09-10T10:03:57.730334+0000","last_became_peered":"2024-09-10T10:03:57.730334+0000","last_unstale":"2024-09-10T10:04:00.356814+0000","last_undegraded":"2024-09-10T10:04:00.356814+0000","last_fullsized":"2024-09-10T10:04:00.356814+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:28.471958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605219+0000","last_change":"2024-09-10T10:03:58.734305+0000","last_active":"2024-09-10T10:04:00.605219+0000","last_peered":"2024-09-10T10:04:00.605219+0000","last_clean":"2024-09-10T10:04:00.605219+0000","last_became_active":"2024-09-10T10:03:58.733998+0000","last_became_peered":"2024-09-10T10:03:58.733998+0000","last_unstale":"2024-09-10T10:04:00.605219+0000","last_undegraded":"2024-09-10T10:04:00.605219+0000","last_fullsized":"2024-09-10T10:04:00.605219+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:31.528456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572497+0000","last_change":"2024-09-10T10:03:57.734752+0000","last_active":"2024-09-10T10:04:00.572497+0000","last_peered":"2024-09-10T10:04:00.572497+0000","last_clean":"2024-09-10T10:04:00.572497+0000","last_became_active":"2024-09-10T10:03:57.734368+0000","last_became_peered":"2024-09-10T10:03:57.734368+0000","last_unstale":"2024-09-10T10:04:00.572497+0000","last_undegraded":"2024-09-10T10:04:00.572497+0000","last_fullsized":"2024-09-10T10:04:00.572497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:48.285278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572377+0000","last_change":"2024-09-10T10:03:58.739626+0000","last_active":"2024-09-10T10:04:00.572377+0000","last_peered":"2024-09-10T10:04:00.572377+0000","last_clean":"2024-09-10T10:04:00.572377+0000","last_became_active":"2024-09-10T10:03:58.739176+0000","last_became_peered":"2024-09-10T10:03:58.739176+0000","last_unstale":"2024-09-10T10:04:00.572377+0000","last_undegraded":"2024-09-10T10:04:00.572377+0000","last_fullsized":"2024-09-10T10:04:00.572377+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:26:43.615114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738507+0000","last_change":"2024-09-10T10:03:57.720975+0000","last_active":"2024-09-10T10:03:59.738507+0000","last_peered":"2024-09-10T10:03:59.738507+0000","last_clean":"2024-09-10T10:03:59.738507+0000","last_became_active":"2024-09-10T10:03:57.720698+0000","last_became_peered":"2024-09-10T10:03:57.720698+0000","last_unstale":"2024-09-10T10:03:59.738507+0000","last_undegraded":"2024-09-10T10:03:59.738507+0000","last_fullsized":"2024-09-10T10:03:59.738507+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:39.207283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022591+0000","last_change":"2024-09-10T10:03:58.740073+0000","last_active":"2024-09-10T10:04:00.022591+0000","last_peered":"2024-09-10T10:04:00.022591+0000","last_clean":"2024-09-10T10:04:00.022591+0000","last_became_active":"2024-09-10T10:03:58.739782+0000","last_became_peered":"2024-09-10T10:03:58.739782+0000","last_unstale":"2024-09-10T10:04:00.022591+0000","last_undegraded":"2024-09-10T10:04:00.022591+0000","last_fullsized":"2024-09-10T10:04:00.022591+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.811982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"116'3","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572182+0000","last_change":"2024-09-10T10:03:57.731228+0000","last_active":"2024-09-10T10:04:00.572182+0000","last_peered":"2024-09-10T10:04:00.572182+0000","last_clean":"2024-09-10T10:04:00.572182+0000","last_became_active":"2024-09-10T10:03:57.730947+0000","last_became_peered":"2024-09-10T10:03:57.730947+0000","last_unstale":"2024-09-10T10:04:00.572182+0000","last_undegraded":"2024-09-10T10:04:00.572182+0000","last_fullsized":"2024-09-10T10:04:00.572182+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:48.478042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022422+0000","last_change":"2024-09-10T10:03:58.746359+0000","last_active":"2024-09-10T10:04:00.022422+0000","last_peered":"2024-09-10T10:04:00.022422+0000","last_clean":"2024-09-10T10:04:00.022422+0000","last_became_active":"2024-09-10T10:03:58.745992+0000","last_became_peered":"2024-09-10T10:03:58.745992+0000","last_unstale":"2024-09-10T10:04:00.022422+0000","last_undegraded":"2024-09-10T10:04:00.022422+0000","last_fullsized":"2024-09-10T10:04:00.022422+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:59.017348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"116'6","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740637+0000","last_change":"2024-09-10T10:03:57.730625+0000","last_active":"2024-09-10T10:03:59.740637+0000","last_peered":"2024-09-10T10:03:59.740637+0000","last_clean":"2024-09-10T10:03:59.740637+0000","last_became_active":"2024-09-10T10:03:57.730333+0000","last_became_peered":"2024-09-10T10:03:57.730333+0000","last_unstale":"2024-09-10T10:03:59.740637+0000","last_undegraded":"2024-09-10T10:03:59.740637+0000","last_fullsized":"2024-09-10T10:03:59.740637+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:26:42.609356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605272+0000","last_change":"2024-09-10T10:03:58.737717+0000","last_active":"2024-09-10T10:04:00.605272+0000","last_peered":"2024-09-10T10:04:00.605272+0000","last_clean":"2024-09-10T10:04:00.605272+0000","last_became_active":"2024-09-10T10:03:58.737382+0000","last_became_peered":"2024-09-10T10:03:58.737382+0000","last_unstale":"2024-09-10T10:04:00.605272+0000","last_undegraded":"2024-09-10T10:04:00.605272+0000","last_fullsized":"2024-09-10T10:04:00.605272+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:21.786863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605049+0000","last_change":"2024-09-10T10:03:57.718706+0000","last_active":"2024-09-10T10:04:00.605049+0000","last_peered":"2024-09-10T10:04:00.605049+0000","last_clean":"2024-09-10T10:04:00.605049+0000","last_became_active":"2024-09-10T10:03:57.718454+0000","last_became_peered":"2024-09-10T10:03:57.718454+0000","last_unstale":"2024-09-10T10:04:00.605049+0000","last_undegraded":"2024-09-10T10:04:00.605049+0000","last_fullsized":"2024-09-10T10:04:00.605049+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:32.451132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356700+0000","last_change":"2024-09-10T10:03:58.760468+0000","last_active":"2024-09-10T10:04:00.356700+0000","last_peered":"2024-09-10T10:04:00.356700+0000","last_clean":"2024-09-10T10:04:00.356700+0000","last_became_active":"2024-09-10T10:03:58.760155+0000","last_became_peered":"2024-09-10T10:03:58.760155+0000","last_unstale":"2024-09-10T10:04:00.356700+0000","last_undegraded":"2024-09-10T10:04:00.356700+0000","last_fullsized":"2024-09-10T10:04:00.356700+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:50.827741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356841+0000","last_change":"2024-09-10T10:03:57.733675+0000","last_active":"2024-09-10T10:04:00.356841+0000","last_peered":"2024-09-10T10:04:00.356841+0000","last_clean":"2024-09-10T10:04:00.356841+0000","last_became_active":"2024-09-10T10:03:57.733348+0000","last_became_peered":"2024-09-10T10:03:57.733348+0000","last_unstale":"2024-09-10T10:04:00.356841+0000","last_undegraded":"2024-09-10T10:04:00.356841+0000","last_fullsized":"2024-09-10T10:04:00.356841+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:21:42.219319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740446+0000","last_change":"2024-09-10T10:03:58.742303+0000","last_active":"2024-09-10T10:03:59.740446+0000","last_peered":"2024-09-10T10:03:59.740446+0000","last_clean":"2024-09-10T10:03:59.740446+0000","last_became_active":"2024-09-10T10:03:58.742017+0000","last_became_peered":"2024-09-10T10:03:58.742017+0000","last_unstale":"2024-09-10T10:03:59.740446+0000","last_undegraded":"2024-09-10T10:03:59.740446+0000","last_fullsized":"2024-09-10T10:03:59.740446+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:07.188145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"116'8","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739978+0000","last_change":"2024-09-10T10:03:57.718897+0000","last_active":"2024-09-10T10:03:59.739978+0000","last_peered":"2024-09-10T10:03:59.739978+0000","last_clean":"2024-09-10T10:03:59.739978+0000","last_became_active":"2024-09-10T10:03:57.718648+0000","last_became_peered":"2024-09-10T10:03:57.718648+0000","last_unstale":"2024-09-10T10:03:59.739978+0000","last_undegraded":"2024-09-10T10:03:59.739978+0000","last_fullsized":"2024-09-10T10:03:59.739978+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:31.293427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741260+0000","last_change":"2024-09-10T10:03:58.753799+0000","last_active":"2024-09-10T10:03:59.741260+0000","last_peered":"2024-09-10T10:03:59.741260+0000","last_clean":"2024-09-10T10:03:59.741260+0000","last_became_active":"2024-09-10T10:03:58.751826+0000","last_became_peered":"2024-09-10T10:03:58.751826+0000","last_unstale":"2024-09-10T10:03:59.741260+0000","last_undegraded":"2024-09-10T10:03:59.741260+0000","last_fullsized":"2024-09-10T10:03:59.741260+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:19.223193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572531+0000","last_change":"2024-09-10T10:03:57.733712+0000","last_active":"2024-09-10T10:04:00.572531+0000","last_peered":"2024-09-10T10:04:00.572531+0000","last_clean":"2024-09-10T10:04:00.572531+0000","last_became_active":"2024-09-10T10:03:57.733429+0000","last_became_peered":"2024-09-10T10:03:57.733429+0000","last_unstale":"2024-09-10T10:04:00.572531+0000","last_undegraded":"2024-09-10T10:04:00.572531+0000","last_fullsized":"2024-09-10T10:04:00.572531+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:52.770882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572464+0000","last_change":"2024-09-10T10:03:58.741757+0000","last_active":"2024-09-10T10:04:00.572464+0000","last_peered":"2024-09-10T10:04:00.572464+0000","last_clean":"2024-09-10T10:04:00.572464+0000","last_became_active":"2024-09-10T10:03:58.741484+0000","last_became_peered":"2024-09-10T10:03:58.741484+0000","last_unstale":"2024-09-10T10:04:00.572464+0000","last_undegraded":"2024-09-10T10:04:00.572464+0000","last_fullsized":"2024-09-10T10:04:00.572464+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:32.299185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989522+0000","last_change":"2024-09-10T10:03:57.732023+0000","last_active":"2024-09-10T10:04:00.989522+0000","last_peered":"2024-09-10T10:04:00.989522+0000","last_clean":"2024-09-10T10:04:00.989522+0000","last_became_active":"2024-09-10T10:03:57.730132+0000","last_became_peered":"2024-09-10T10:03:57.730132+0000","last_unstale":"2024-09-10T10:04:00.989522+0000","last_undegraded":"2024-09-10T10:04:00.989522+0000","last_fullsized":"2024-09-10T10:04:00.989522+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:03.522101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738118+0000","last_change":"2024-09-10T10:03:58.753910+0000","last_active":"2024-09-10T10:03:59.738118+0000","last_peered":"2024-09-10T10:03:59.738118+0000","last_clean":"2024-09-10T10:03:59.738118+0000","last_became_active":"2024-09-10T10:03:58.753631+0000","last_became_peered":"2024-09-10T10:03:58.753631+0000","last_unstale":"2024-09-10T10:03:59.738118+0000","last_undegraded":"2024-09-10T10:03:59.738118+0000","last_fullsized":"2024-09-10T10:03:59.738118+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:33.892796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741196+0000","last_change":"2024-09-10T10:03:57.721762+0000","last_active":"2024-09-10T10:03:59.741196+0000","last_peered":"2024-09-10T10:03:59.741196+0000","last_clean":"2024-09-10T10:03:59.741196+0000","last_became_active":"2024-09-10T10:03:57.721449+0000","last_became_peered":"2024-09-10T10:03:57.721449+0000","last_unstale":"2024-09-10T10:03:59.741196+0000","last_undegraded":"2024-09-10T10:03:59.741196+0000","last_fullsized":"2024-09-10T10:03:59.741196+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:17.778326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357106+0000","last_change":"2024-09-10T10:03:58.760795+0000","last_active":"2024-09-10T10:04:00.357106+0000","last_peered":"2024-09-10T10:04:00.357106+0000","last_clean":"2024-09-10T10:04:00.357106+0000","last_became_active":"2024-09-10T10:03:58.760472+0000","last_became_peered":"2024-09-10T10:03:58.760472+0000","last_unstale":"2024-09-10T10:04:00.357106+0000","last_undegraded":"2024-09-10T10:04:00.357106+0000","last_fullsized":"2024-09-10T10:04:00.357106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:08.214538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572311+0000","last_change":"2024-09-10T10:03:57.733091+0000","last_active":"2024-09-10T10:04:00.572311+0000","last_peered":"2024-09-10T10:04:00.572311+0000","last_clean":"2024-09-10T10:04:00.572311+0000","last_became_active":"2024-09-10T10:03:57.732834+0000","last_became_peered":"2024-09-10T10:03:57.732834+0000","last_unstale":"2024-09-10T10:04:00.572311+0000","last_undegraded":"2024-09-10T10:04:00.572311+0000","last_fullsized":"2024-09-10T10:04:00.572311+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:49.821984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.604962+0000","last_change":"2024-09-10T10:03:58.754290+0000","last_active":"2024-09-10T10:04:00.604962+0000","last_peered":"2024-09-10T10:04:00.604962+0000","last_clean":"2024-09-10T10:04:00.604962+0000","last_became_active":"2024-09-10T10:03:58.753982+0000","last_became_peered":"2024-09-10T10:03:58.753982+0000","last_unstale":"2024-09-10T10:04:00.604962+0000","last_undegraded":"2024-09-10T10:04:00.604962+0000","last_fullsized":"2024-09-10T10:04:00.604962+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:08:17.146980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572157+0000","last_change":"2024-09-10T10:03:57.735283+0000","last_active":"2024-09-10T10:04:00.572157+0000","last_peered":"2024-09-10T10:04:00.572157+0000","last_clean":"2024-09-10T10:04:00.572157+0000","last_became_active":"2024-09-10T10:03:57.734668+0000","last_became_peered":"2024-09-10T10:03:57.734668+0000","last_unstale":"2024-09-10T10:04:00.572157+0000","last_undegraded":"2024-09-10T10:04:00.572157+0000","last_fullsized":"2024-09-10T10:04:00.572157+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:04.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740374+0000","last_change":"2024-09-10T10:03:58.745639+0000","last_active":"2024-09-10T10:03:59.740374+0000","last_peered":"2024-09-10T10:03:59.740374+0000","last_clean":"2024-09-10T10:03:59.740374+0000","last_became_active":"2024-09-10T10:03:58.745380+0000","last_became_peered":"2024-09-10T10:03:58.745380+0000","last_unstale":"2024-09-10T10:03:59.740374+0000","last_undegraded":"2024-09-10T10:03:59.740374+0000","last_fullsized":"2024-09-10T10:03:59.740374+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:15.755172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737913+0000","last_change":"2024-09-10T10:03:57.718663+0000","last_active":"2024-09-10T10:03:59.737913+0000","last_peered":"2024-09-10T10:03:59.737913+0000","last_clean":"2024-09-10T10:03:59.737913+0000","last_became_active":"2024-09-10T10:03:57.718381+0000","last_became_peered":"2024-09-10T10:03:57.718381+0000","last_unstale":"2024-09-10T10:03:59.737913+0000","last_undegraded":"2024-09-10T10:03:59.737913+0000","last_fullsized":"2024-09-10T10:03:59.737913+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:21.131893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356706+0000","last_change":"2024-09-10T10:03:58.750354+0000","last_active":"2024-09-10T10:04:00.356706+0000","last_peered":"2024-09-10T10:04:00.356706+0000","last_clean":"2024-09-10T10:04:00.356706+0000","last_became_active":"2024-09-10T10:03:58.750010+0000","last_became_peered":"2024-09-10T10:03:58.750010+0000","last_unstale":"2024-09-10T10:04:00.356706+0000","last_undegraded":"2024-09-10T10:04:00.356706+0000","last_fullsized":"2024-09-10T10:04:00.356706+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:49.483800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737717+0000","last_change":"2024-09-10T10:03:57.732729+0000","last_active":"2024-09-10T10:03:59.737717+0000","last_peered":"2024-09-10T10:03:59.737717+0000","last_clean":"2024-09-10T10:03:59.737717+0000","last_became_active":"2024-09-10T10:03:57.732475+0000","last_became_peered":"2024-09-10T10:03:57.732475+0000","last_unstale":"2024-09-10T10:03:59.737717+0000","last_undegraded":"2024-09-10T10:03:59.737717+0000","last_fullsized":"2024-09-10T10:03:59.737717+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:49.760077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022234+0000","last_change":"2024-09-10T10:03:58.748742+0000","last_active":"2024-09-10T10:04:00.022234+0000","last_peered":"2024-09-10T10:04:00.022234+0000","last_clean":"2024-09-10T10:04:00.022234+0000","last_became_active":"2024-09-10T10:03:58.748041+0000","last_became_peered":"2024-09-10T10:03:58.748041+0000","last_unstale":"2024-09-10T10:04:00.022234+0000","last_undegraded":"2024-09-10T10:04:00.022234+0000","last_fullsized":"2024-09-10T10:04:00.022234+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:24.709279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989233+0000","last_change":"2024-09-10T10:03:57.726476+0000","last_active":"2024-09-10T10:04:00.989233+0000","last_peered":"2024-09-10T10:04:00.989233+0000","last_clean":"2024-09-10T10:04:00.989233+0000","last_became_active":"2024-09-10T10:03:57.726149+0000","last_became_peered":"2024-09-10T10:03:57.726149+0000","last_unstale":"2024-09-10T10:04:00.989233+0000","last_undegraded":"2024-09-10T10:04:00.989233+0000","last_fullsized":"2024-09-10T10:04:00.989233+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:41.858327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740956+0000","last_change":"2024-09-10T10:03:58.742589+0000","last_active":"2024-09-10T10:03:59.740956+0000","last_peered":"2024-09-10T10:03:59.740956+0000","last_clean":"2024-09-10T10:03:59.740956+0000","last_became_active":"2024-09-10T10:03:58.742319+0000","last_became_peered":"2024-09-10T10:03:58.742319+0000","last_unstale":"2024-09-10T10:03:59.740956+0000","last_undegraded":"2024-09-10T10:03:59.740956+0000","last_fullsized":"2024-09-10T10:03:59.740956+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:19.223193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605331+0000","last_change":"2024-09-10T10:03:57.734198+0000","last_active":"2024-09-10T10:04:00.605331+0000","last_peered":"2024-09-10T10:04:00.605331+0000","last_clean":"2024-09-10T10:04:00.605331+0000","last_became_active":"2024-09-10T10:03:57.733799+0000","last_became_peered":"2024-09-10T10:03:57.733799+0000","last_unstale":"2024-09-10T10:04:00.605331+0000","last_undegraded":"2024-09-10T10:04:00.605331+0000","last_fullsized":"2024-09-10T10:04:00.605331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:19.425659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605314+0000","last_change":"2024-09-10T10:03:58.744898+0000","last_active":"2024-09-10T10:04:00.605314+0000","last_peered":"2024-09-10T10:04:00.605314+0000","last_clean":"2024-09-10T10:04:00.605314+0000","last_became_active":"2024-09-10T10:03:58.744626+0000","last_became_peered":"2024-09-10T10:03:58.744626+0000","last_unstale":"2024-09-10T10:04:00.605314+0000","last_undegraded":"2024-09-10T10:04:00.605314+0000","last_fullsized":"2024-09-10T10:04:00.605314+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:24.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"117'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.732875+0000","last_change":"2024-09-10T10:03:57.733131+0000","last_active":"2024-09-10T10:04:00.732875+0000","last_peered":"2024-09-10T10:04:00.732875+0000","last_clean":"2024-09-10T10:04:00.732875+0000","last_became_active":"2024-09-10T10:03:57.732834+0000","last_became_peered":"2024-09-10T10:03:57.732834+0000","last_unstale":"2024-09-10T10:04:00.732875+0000","last_undegraded":"2024-09-10T10:04:00.732875+0000","last_fullsized":"2024-09-10T10:04:00.732875+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:41.030732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357008+0000","last_change":"2024-09-10T10:03:58.758443+0000","last_active":"2024-09-10T10:04:00.357008+0000","last_peered":"2024-09-10T10:04:00.357008+0000","last_clean":"2024-09-10T10:04:00.357008+0000","last_became_active":"2024-09-10T10:03:58.754351+0000","last_became_peered":"2024-09-10T10:03:58.754351+0000","last_unstale":"2024-09-10T10:04:00.357008+0000","last_undegraded":"2024-09-10T10:04:00.357008+0000","last_fullsized":"2024-09-10T10:04:00.357008+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:26.889887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357058+0000","last_change":"2024-09-10T10:03:57.733776+0000","last_active":"2024-09-10T10:04:00.357058+0000","last_peered":"2024-09-10T10:04:00.357058+0000","last_clean":"2024-09-10T10:04:00.357058+0000","last_became_active":"2024-09-10T10:03:57.730896+0000","last_became_peered":"2024-09-10T10:03:57.730896+0000","last_unstale":"2024-09-10T10:04:00.357058+0000","last_undegraded":"2024-09-10T10:04:00.357058+0000","last_fullsized":"2024-09-10T10:04:00.357058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:17.495422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741060+0000","last_change":"2024-09-10T10:03:58.732754+0000","last_active":"2024-09-10T10:03:59.741060+0000","last_peered":"2024-09-10T10:03:59.741060+0000","last_clean":"2024-09-10T10:03:59.741060+0000","last_became_active":"2024-09-10T10:03:58.732469+0000","last_became_peered":"2024-09-10T10:03:58.732469+0000","last_unstale":"2024-09-10T10:03:59.741060+0000","last_undegraded":"2024-09-10T10:03:59.741060+0000","last_fullsized":"2024-09-10T10:03:59.741060+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:00.860201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989331+0000","last_change":"2024-09-10T10:03:57.730736+0000","last_active":"2024-09-10T10:04:00.989331+0000","last_peered":"2024-09-10T10:04:00.989331+0000","last_clean":"2024-09-10T10:04:00.989331+0000","last_became_active":"2024-09-10T10:03:57.730437+0000","last_became_peered":"2024-09-10T10:03:57.730437+0000","last_unstale":"2024-09-10T10:04:00.989331+0000","last_undegraded":"2024-09-10T10:04:00.989331+0000","last_fullsized":"2024-09-10T10:04:00.989331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:54.908884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022359+0000","last_change":"2024-09-10T10:03:58.757454+0000","last_active":"2024-09-10T10:04:00.022359+0000","last_peered":"2024-09-10T10:04:00.022359+0000","last_clean":"2024-09-10T10:04:00.022359+0000","last_became_active":"2024-09-10T10:03:58.757154+0000","last_became_peered":"2024-09-10T10:03:58.757154+0000","last_unstale":"2024-09-10T10:04:00.022359+0000","last_undegraded":"2024-09-10T10:04:00.022359+0000","last_fullsized":"2024-09-10T10:04:00.022359+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:17:02.778168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356844+0000","last_change":"2024-09-10T10:03:57.733419+0000","last_active":"2024-09-10T10:04:00.356844+0000","last_peered":"2024-09-10T10:04:00.356844+0000","last_clean":"2024-09-10T10:04:00.356844+0000","last_became_active":"2024-09-10T10:03:57.733145+0000","last_became_peered":"2024-09-10T10:03:57.733145+0000","last_unstale":"2024-09-10T10:04:00.356844+0000","last_undegraded":"2024-09-10T10:04:00.356844+0000","last_fullsized":"2024-09-10T10:04:00.356844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:57.726265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572204+0000","last_change":"2024-09-10T10:03:58.749552+0000","last_active":"2024-09-10T10:04:00.572204+0000","last_peered":"2024-09-10T10:04:00.572204+0000","last_clean":"2024-09-10T10:04:00.572204+0000","last_became_active":"2024-09-10T10:03:58.749329+0000","last_became_peered":"2024-09-10T10:03:58.749329+0000","last_unstale":"2024-09-10T10:04:00.572204+0000","last_undegraded":"2024-09-10T10:04:00.572204+0000","last_fullsized":"2024-09-10T10:04:00.572204+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:08.214538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022141+0000","last_change":"2024-09-10T10:03:57.735891+0000","last_active":"2024-09-10T10:04:00.022141+0000","last_peered":"2024-09-10T10:04:00.022141+0000","last_clean":"2024-09-10T10:04:00.022141+0000","last_became_active":"2024-09-10T10:03:57.735484+0000","last_became_peered":"2024-09-10T10:03:57.735484+0000","last_unstale":"2024-09-10T10:04:00.022141+0000","last_undegraded":"2024-09-10T10:04:00.022141+0000","last_fullsized":"2024-09-10T10:04:00.022141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:59.854443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738301+0000","last_change":"2024-09-10T10:03:58.747711+0000","last_active":"2024-09-10T10:03:59.738301+0000","last_peered":"2024-09-10T10:03:59.738301+0000","last_clean":"2024-09-10T10:03:59.738301+0000","last_became_active":"2024-09-10T10:03:58.747389+0000","last_became_peered":"2024-09-10T10:03:58.747389+0000","last_unstale":"2024-09-10T10:03:59.738301+0000","last_undegraded":"2024-09-10T10:03:59.738301+0000","last_fullsized":"2024-09-10T10:03:59.738301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:29.183472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737819+0000","last_change":"2024-09-10T10:03:57.720632+0000","last_active":"2024-09-10T10:03:59.737819+0000","last_peered":"2024-09-10T10:03:59.737819+0000","last_clean":"2024-09-10T10:03:59.737819+0000","last_became_active":"2024-09-10T10:03:57.720348+0000","last_became_peered":"2024-09-10T10:03:57.720348+0000","last_unstale":"2024-09-10T10:03:59.737819+0000","last_undegraded":"2024-09-10T10:03:59.737819+0000","last_fullsized":"2024-09-10T10:03:59.737819+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:22.754697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356738+0000","last_change":"2024-09-10T10:03:58.755159+0000","last_active":"2024-09-10T10:04:00.356738+0000","last_peered":"2024-09-10T10:04:00.356738+0000","last_clean":"2024-09-10T10:04:00.356738+0000","last_became_active":"2024-09-10T10:03:58.754915+0000","last_became_peered":"2024-09-10T10:03:58.754915+0000","last_unstale":"2024-09-10T10:04:00.356738+0000","last_undegraded":"2024-09-10T10:04:00.356738+0000","last_fullsized":"2024-09-10T10:04:00.356738+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:46.621125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989332+0000","last_change":"2024-09-10T10:03:57.732231+0000","last_active":"2024-09-10T10:04:00.989332+0000","last_peered":"2024-09-10T10:04:00.989332+0000","last_clean":"2024-09-10T10:04:00.989332+0000","last_became_active":"2024-09-10T10:03:57.731815+0000","last_became_peered":"2024-09-10T10:03:57.731815+0000","last_unstale":"2024-09-10T10:04:00.989332+0000","last_undegraded":"2024-09-10T10:04:00.989332+0000","last_fullsized":"2024-09-10T10:04:00.989332+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:49.394741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022006+0000","last_change":"2024-09-10T10:03:58.742743+0000","last_active":"2024-09-10T10:04:00.022006+0000","last_peered":"2024-09-10T10:04:00.022006+0000","last_clean":"2024-09-10T10:04:00.022006+0000","last_became_active":"2024-09-10T10:03:58.741777+0000","last_became_peered":"2024-09-10T10:03:58.741777+0000","last_unstale":"2024-09-10T10:04:00.022006+0000","last_undegraded":"2024-09-10T10:04:00.022006+0000","last_fullsized":"2024-09-10T10:04:00.022006+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:32.220934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022318+0000","last_change":"2024-09-10T10:03:57.735434+0000","last_active":"2024-09-10T10:04:00.022318+0000","last_peered":"2024-09-10T10:04:00.022318+0000","last_clean":"2024-09-10T10:04:00.022318+0000","last_became_active":"2024-09-10T10:03:57.734831+0000","last_became_peered":"2024-09-10T10:03:57.734831+0000","last_unstale":"2024-09-10T10:04:00.022318+0000","last_undegraded":"2024-09-10T10:04:00.022318+0000","last_fullsized":"2024-09-10T10:04:00.022318+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:10.823838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":382,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.021797+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:04:00.021797+0000","last_peered":"2024-09-10T10:04:00.021797+0000","last_clean":"2024-09-10T10:04:00.021797+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:04:00.021797+0000","last_undegraded":"2024-09-10T10:04:00.021797+0000","last_fullsized":"2024-09-10T10:04:00.021797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":1231063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1466368,"data_stored":640057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":638557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1458176,"data_stored":638515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542218,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1462272,"data_stored":638605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":639061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":638445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2048000,"data_stored":1228813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:04:02.806 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:04:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29c6200640 1 Processor -- start 2024-09-10T10:04:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29c6200640 1 -- start start 2024-09-10T10:04:02.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29c6200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c00cfe30 0x7f29c0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29c6200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f29c006e7a0 0x7f29c00cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29c6200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29c006d2d0 con 0x7f29c00cd510 2024-09-10T10:04:02.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29c6200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f29c00cdeb0 con 0x7f29c006e7a0 2024-09-10T10:04:02.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29c6200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f29c015d9a0 con 0x7f29c00cfe30 2024-09-10T10:04:02.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29bea00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f29c00cd510 0x7f29c006e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43794/0 (socket says 172.21.15.110:43794) 2024-09-10T10:04:02.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29bea00640 1 -- 172.21.15.110:0/4171060664 learned_addr learned my addr 172.21.15.110:0/4171060664 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29bfe00640 1 --2- 172.21.15.110:0/4171060664 >> v2:172.21.15.145:3300/0 conn(0x7f29c006e7a0 0x7f29c00cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29bf400640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c00cfe30 0x7f29c0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146855843 0 0) 0x7f29c006d2d0 con 0x7f29c00cd510 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.974+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29a4003530 con 0x7f29c00cd510 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2598474944 0 0) 0x7f29a4003530 con 0x7f29c00cd510 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c00cfe30 msgr2=0x7f29c0071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c00cfe30 0x7f29c0071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 >> v2:172.21.15.145:3300/0 conn(0x7f29c006e7a0 msgr2=0x7f29c00cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 --2- 172.21.15.110:0/4171060664 >> v2:172.21.15.145:3300/0 conn(0x7f29c006e7a0 0x7f29c00cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c006d5b0 con 0x7f29c00cd510 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f29ac001680 con 0x7f29c00cd510 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 >> v1:172.21.15.110:6789/0 conn(0x7f29c00cd510 legacy=0x7f29c006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c00cfe30 0x7f29a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29a4004d50 0x7f29a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29c015d9a0 con 0x7f29a40039c0 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f29a400fde0 con 0x7f29a4004d50 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f29a400ff60 con 0x7f29c00cfe30 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bf400640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bea00640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c00cfe30 0x7f29a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bfe00640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29a4004d50 0x7f29a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bf400640 1 -- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c00cfe30 msgr2=0x7f29a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bf400640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c00cfe30 0x7f29a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bf400640 1 -- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29a4004d50 msgr2=0x7f29a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bf400640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29a4004d50 0x7f29a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bf400640 1 -- 172.21.15.110:0/4171060664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a4003530 con 0x7f29a40039c0 2024-09-10T10:04:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.978+0000 7f29bf400640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29a4003de0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009ae0 tx=0x7f29b001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dd00cd874f354f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00096b0 con 0x7f29a40039c0 2024-09-10T10:04:02.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00a6480 con 0x7f29a40039c0 2024-09-10T10:04:02.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29be000640 1 -- 172.21.15.110:0/4171060664 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00a6870 con 0x7f29a40039c0 2024-09-10T10:04:02.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- 172.21.15.110:0/4171060664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 msgr2=0x7f29a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 --2- 172.21.15.110:0/4171060664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29a4003de0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009ae0 tx=0x7f29b001ca70 comp rx=0 tx=0).stop 2024-09-10T10:04:02.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bfe00640 1 -- 172.21.15.110:0/4171060664 reap_dead start 2024-09-10T10:04:02.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- 172.21.15.110:0/4171060664 shutdown_connections 2024-09-10T10:04:02.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- 172.21.15.110:0/4171060664 >> 172.21.15.110:0/4171060664 conn(0x7f29c006bc50 msgr2=0x7f29c006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:02.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- 172.21.15.110:0/4171060664 shutdown_connections 2024-09-10T10:04:02.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- 172.21.15.110:0/4171060664 wait complete. 2024-09-10T10:04:02.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 Processor -- start 2024-09-10T10:04:02.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- start start 2024-09-10T10:04:02.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29c0153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c014b1f0 0x7f29c014aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c01562f0 0x7f29c01516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00b35d0 con 0x7f29a40039c0 2024-09-10T10:04:02.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f29c015d640 con 0x7f29c01562f0 2024-09-10T10:04:02.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29c6200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f29c00b3750 con 0x7f29c014b1f0 2024-09-10T10:04:02.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29c0153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bf400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29c0153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50520/0 (socket says 172.21.15.110:50520) 2024-09-10T10:04:02.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bf400640 1 -- 172.21.15.110:0/2809934100 learned_addr learned my addr 172.21.15.110:0/2809934100 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:02.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bfe00640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c01562f0 0x7f29c01516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bea00640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c014b1f0 0x7f29c014aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bf400640 1 -- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c014b1f0 msgr2=0x7f29c014aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bf400640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c014b1f0 0x7f29c014aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bf400640 1 -- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c01562f0 msgr2=0x7f29c01516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bf400640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c01562f0 0x7f29c01516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.982+0000 7f29bf400640 1 -- 172.21.15.110:0/2809934100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c0154cc0 con 0x7f29a40039c0 2024-09-10T10:04:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29bf400640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29c0153b00 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009ae0 tx=0x7f29a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29b7400640 1 -- 172.21.15.110:0/2809934100 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00a6e30 con 0x7f29a40039c0 2024-09-10T10:04:02.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29b7400640 1 -- 172.21.15.110:0/2809934100 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b0004020 con 0x7f29a40039c0 2024-09-10T10:04:02.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c0154f60 con 0x7f29a40039c0 2024-09-10T10:04:02.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29b7400640 1 -- 172.21.15.110:0/2809934100 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0004450 con 0x7f29a40039c0 2024-09-10T10:04:02.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c0155af0 con 0x7f29a40039c0 2024-09-10T10:04:02.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c00cd970 con 0x7f29a40039c0 2024-09-10T10:04:02.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29b7400640 1 -- 172.21.15.110:0/2809934100 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f29b0003620 con 0x7f29a40039c0 2024-09-10T10:04:02.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29b7400640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29a0067ce0 0x7f29a006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.986+0000 7f29b7400640 1 -- 172.21.15.110:0/2809934100 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f29b0120820 con 0x7f29a40039c0 2024-09-10T10:04:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.990+0000 7f29bea00640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29a0067ce0 0x7f29a006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.990+0000 7f29bea00640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29a0067ce0 0x7f29a006a1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29c015cbb0 tx=0x7f29ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:02.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:02.994+0000 7f29b7400640 1 -- 172.21.15.110:0/2809934100 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f29b00b6070 con 0x7f29a40039c0 2024-09-10T10:04:03.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.146+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29c014b020 con 0x7f29a0067ce0 2024-09-10T10:04:03.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.154+0000 7f29b7400640 1 -- 172.21.15.110:0/2809934100 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f29c014b020 con 0x7f29a0067ce0 2024-09-10T10:04:03.160 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:03.162 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29a0067ce0 msgr2=0x7f29a006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29a0067ce0 0x7f29a006a1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29c015cbb0 tx=0x7f29ac009040 comp rx=0 tx=0).stop 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 msgr2=0x7f29c0153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29c0153b00 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009ae0 tx=0x7f29a4005840 comp rx=0 tx=0).stop 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 shutdown_connections 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29a0067ce0 0x7f29a006a1a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c01562f0 0x7f29c01516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29c014b1f0 0x7f29c014aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 --2- 172.21.15.110:0/2809934100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29a40039c0 0x7f29c0153b00 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 >> 172.21.15.110:0/2809934100 conn(0x7f29c006bc50 msgr2=0x7f29c0153050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:03.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 shutdown_connections 2024-09-10T10:04:03.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.158+0000 7f29c6200640 1 -- 172.21.15.110:0/2809934100 wait complete. 2024-09-10T10:04:03.181 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-10T10:04:01.716089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270428,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704292,"statfs":{"total":773094113280,"available":772817195008,"internally_reserved":0,"allocated":12886016,"data_stored":6293116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":263835180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.012428"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605273+0000","last_change":"2024-09-10T10:03:58.757686+0000","last_active":"2024-09-10T10:04:00.605273+0000","last_peered":"2024-09-10T10:04:00.605273+0000","last_clean":"2024-09-10T10:04:00.605273+0000","last_became_active":"2024-09-10T10:03:58.756250+0000","last_became_peered":"2024-09-10T10:03:58.756250+0000","last_unstale":"2024-09-10T10:04:00.605273+0000","last_undegraded":"2024-09-10T10:04:00.605273+0000","last_fullsized":"2024-09-10T10:04:00.605273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:19.336634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357047+0000","last_change":"2024-09-10T10:03:58.757540+0000","last_active":"2024-09-10T10:04:00.357047+0000","last_peered":"2024-09-10T10:04:00.357047+0000","last_clean":"2024-09-10T10:04:00.357047+0000","last_became_active":"2024-09-10T10:03:58.757247+0000","last_became_peered":"2024-09-10T10:03:58.757247+0000","last_unstale":"2024-09-10T10:04:00.357047+0000","last_undegraded":"2024-09-10T10:04:00.357047+0000","last_fullsized":"2024-09-10T10:04:00.357047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:03.321352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740838+0000","last_change":"2024-09-10T10:03:58.747149+0000","last_active":"2024-09-10T10:03:59.740838+0000","last_peered":"2024-09-10T10:03:59.740838+0000","last_clean":"2024-09-10T10:03:59.740838+0000","last_became_active":"2024-09-10T10:03:58.746862+0000","last_became_peered":"2024-09-10T10:03:58.746862+0000","last_unstale":"2024-09-10T10:03:59.740838+0000","last_undegraded":"2024-09-10T10:03:59.740838+0000","last_fullsized":"2024-09-10T10:03:59.740838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:15.755172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605044+0000","last_change":"2024-09-10T10:03:58.746228+0000","last_active":"2024-09-10T10:04:00.605044+0000","last_peered":"2024-09-10T10:04:00.605044+0000","last_clean":"2024-09-10T10:04:00.605044+0000","last_became_active":"2024-09-10T10:03:58.746027+0000","last_became_peered":"2024-09-10T10:03:58.746027+0000","last_unstale":"2024-09-10T10:04:00.605044+0000","last_undegraded":"2024-09-10T10:04:00.605044+0000","last_fullsized":"2024-09-10T10:04:00.605044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:20.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022413+0000","last_change":"2024-09-10T10:03:58.740901+0000","last_active":"2024-09-10T10:04:00.022413+0000","last_peered":"2024-09-10T10:04:00.022413+0000","last_clean":"2024-09-10T10:04:00.022413+0000","last_became_active":"2024-09-10T10:03:58.740388+0000","last_became_peered":"2024-09-10T10:03:58.740388+0000","last_unstale":"2024-09-10T10:04:00.022413+0000","last_undegraded":"2024-09-10T10:04:00.022413+0000","last_fullsized":"2024-09-10T10:04:00.022413+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:37.996548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739882+0000","last_change":"2024-09-10T10:03:58.743514+0000","last_active":"2024-09-10T10:03:59.739882+0000","last_peered":"2024-09-10T10:03:59.739882+0000","last_clean":"2024-09-10T10:03:59.739882+0000","last_became_active":"2024-09-10T10:03:58.743189+0000","last_became_peered":"2024-09-10T10:03:58.743189+0000","last_unstale":"2024-09-10T10:03:59.739882+0000","last_undegraded":"2024-09-10T10:03:59.739882+0000","last_fullsized":"2024-09-10T10:03:59.739882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:34.470602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739567+0000","last_change":"2024-09-10T10:03:58.744711+0000","last_active":"2024-09-10T10:03:59.739567+0000","last_peered":"2024-09-10T10:03:59.739567+0000","last_clean":"2024-09-10T10:03:59.739567+0000","last_became_active":"2024-09-10T10:03:58.744424+0000","last_became_peered":"2024-09-10T10:03:58.744424+0000","last_unstale":"2024-09-10T10:03:59.739567+0000","last_undegraded":"2024-09-10T10:03:59.739567+0000","last_fullsized":"2024-09-10T10:03:59.739567+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:07.188145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739862+0000","last_change":"2024-09-10T10:03:58.745002+0000","last_active":"2024-09-10T10:03:59.739862+0000","last_peered":"2024-09-10T10:03:59.739862+0000","last_clean":"2024-09-10T10:03:59.739862+0000","last_became_active":"2024-09-10T10:03:58.741497+0000","last_became_peered":"2024-09-10T10:03:58.741497+0000","last_unstale":"2024-09-10T10:03:59.739862+0000","last_undegraded":"2024-09-10T10:03:59.739862+0000","last_fullsized":"2024-09-10T10:03:59.739862+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:54.517670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741206+0000","last_change":"2024-09-10T10:03:58.741533+0000","last_active":"2024-09-10T10:03:59.741206+0000","last_peered":"2024-09-10T10:03:59.741206+0000","last_clean":"2024-09-10T10:03:59.741206+0000","last_became_active":"2024-09-10T10:03:58.741171+0000","last_became_peered":"2024-09-10T10:03:58.741171+0000","last_unstale":"2024-09-10T10:03:59.741206+0000","last_undegraded":"2024-09-10T10:03:59.741206+0000","last_fullsized":"2024-09-10T10:03:59.741206+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:42.036490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357097+0000","last_change":"2024-09-10T10:03:58.757376+0000","last_active":"2024-09-10T10:04:00.357097+0000","last_peered":"2024-09-10T10:04:00.357097+0000","last_clean":"2024-09-10T10:04:00.357097+0000","last_became_active":"2024-09-10T10:03:58.757110+0000","last_became_peered":"2024-09-10T10:03:58.757110+0000","last_unstale":"2024-09-10T10:04:00.357097+0000","last_undegraded":"2024-09-10T10:04:00.357097+0000","last_fullsized":"2024-09-10T10:04:00.357097+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:09.788894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989512+0000","last_change":"2024-09-10T10:03:58.763248+0000","last_active":"2024-09-10T10:04:00.989512+0000","last_peered":"2024-09-10T10:04:00.989512+0000","last_clean":"2024-09-10T10:04:00.989512+0000","last_became_active":"2024-09-10T10:03:58.762938+0000","last_became_peered":"2024-09-10T10:03:58.762938+0000","last_unstale":"2024-09-10T10:04:00.989512+0000","last_undegraded":"2024-09-10T10:04:00.989512+0000","last_fullsized":"2024-09-10T10:04:00.989512+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:54.062122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740568+0000","last_change":"2024-09-10T10:03:58.742063+0000","last_active":"2024-09-10T10:03:59.740568+0000","last_peered":"2024-09-10T10:03:59.740568+0000","last_clean":"2024-09-10T10:03:59.740568+0000","last_became_active":"2024-09-10T10:03:58.741760+0000","last_became_peered":"2024-09-10T10:03:58.741760+0000","last_unstale":"2024-09-10T10:03:59.740568+0000","last_undegraded":"2024-09-10T10:03:59.740568+0000","last_fullsized":"2024-09-10T10:03:59.740568+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:34.470602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605030+0000","last_change":"2024-09-10T10:03:58.752792+0000","last_active":"2024-09-10T10:04:00.605030+0000","last_peered":"2024-09-10T10:04:00.605030+0000","last_clean":"2024-09-10T10:04:00.605030+0000","last_became_active":"2024-09-10T10:03:58.752525+0000","last_became_peered":"2024-09-10T10:03:58.752525+0000","last_unstale":"2024-09-10T10:04:00.605030+0000","last_undegraded":"2024-09-10T10:04:00.605030+0000","last_fullsized":"2024-09-10T10:04:00.605030+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:09.597632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356720+0000","last_change":"2024-09-10T10:03:58.761971+0000","last_active":"2024-09-10T10:04:00.356720+0000","last_peered":"2024-09-10T10:04:00.356720+0000","last_clean":"2024-09-10T10:04:00.356720+0000","last_became_active":"2024-09-10T10:03:58.756915+0000","last_became_peered":"2024-09-10T10:03:58.756915+0000","last_unstale":"2024-09-10T10:04:00.356720+0000","last_undegraded":"2024-09-10T10:04:00.356720+0000","last_fullsized":"2024-09-10T10:04:00.356720+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:07.445143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.604872+0000","last_change":"2024-09-10T10:03:58.744977+0000","last_active":"2024-09-10T10:04:00.604872+0000","last_peered":"2024-09-10T10:04:00.604872+0000","last_clean":"2024-09-10T10:04:00.604872+0000","last_became_active":"2024-09-10T10:03:58.744681+0000","last_became_peered":"2024-09-10T10:03:58.744681+0000","last_unstale":"2024-09-10T10:04:00.604872+0000","last_undegraded":"2024-09-10T10:04:00.604872+0000","last_fullsized":"2024-09-10T10:04:00.604872+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:58.300431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737770+0000","last_change":"2024-09-10T10:03:58.753812+0000","last_active":"2024-09-10T10:03:59.737770+0000","last_peered":"2024-09-10T10:03:59.737770+0000","last_clean":"2024-09-10T10:03:59.737770+0000","last_became_active":"2024-09-10T10:03:58.753555+0000","last_became_peered":"2024-09-10T10:03:58.753555+0000","last_unstale":"2024-09-10T10:03:59.737770+0000","last_undegraded":"2024-09-10T10:03:59.737770+0000","last_fullsized":"2024-09-10T10:03:59.737770+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:20.244238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740498+0000","last_change":"2024-09-10T10:03:58.739905+0000","last_active":"2024-09-10T10:03:59.740498+0000","last_peered":"2024-09-10T10:03:59.740498+0000","last_clean":"2024-09-10T10:03:59.740498+0000","last_became_active":"2024-09-10T10:03:58.739640+0000","last_became_peered":"2024-09-10T10:03:58.739640+0000","last_unstale":"2024-09-10T10:03:59.740498+0000","last_undegraded":"2024-09-10T10:03:59.740498+0000","last_fullsized":"2024-09-10T10:03:59.740498+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:42.036490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738411+0000","last_change":"2024-09-10T10:03:58.753277+0000","last_active":"2024-09-10T10:03:59.738411+0000","last_peered":"2024-09-10T10:03:59.738411+0000","last_clean":"2024-09-10T10:03:59.738411+0000","last_became_active":"2024-09-10T10:03:58.753038+0000","last_became_peered":"2024-09-10T10:03:58.753038+0000","last_unstale":"2024-09-10T10:03:59.738411+0000","last_undegraded":"2024-09-10T10:03:59.738411+0000","last_fullsized":"2024-09-10T10:03:59.738411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:07.566000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989538+0000","last_change":"2024-09-10T10:03:58.758170+0000","last_active":"2024-09-10T10:04:00.989538+0000","last_peered":"2024-09-10T10:04:00.989538+0000","last_clean":"2024-09-10T10:04:00.989538+0000","last_became_active":"2024-09-10T10:03:58.757820+0000","last_became_peered":"2024-09-10T10:03:58.757820+0000","last_unstale":"2024-09-10T10:04:00.989538+0000","last_undegraded":"2024-09-10T10:04:00.989538+0000","last_fullsized":"2024-09-10T10:04:00.989538+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:41.391186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738050+0000","last_change":"2024-09-10T10:03:58.762564+0000","last_active":"2024-09-10T10:03:59.738050+0000","last_peered":"2024-09-10T10:03:59.738050+0000","last_clean":"2024-09-10T10:03:59.738050+0000","last_became_active":"2024-09-10T10:03:58.762289+0000","last_became_peered":"2024-09-10T10:03:58.762289+0000","last_unstale":"2024-09-10T10:03:59.738050+0000","last_undegraded":"2024-09-10T10:03:59.738050+0000","last_fullsized":"2024-09-10T10:03:59.738050+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:35.272830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022183+0000","last_change":"2024-09-10T10:03:58.740663+0000","last_active":"2024-09-10T10:04:00.022183+0000","last_peered":"2024-09-10T10:04:00.022183+0000","last_clean":"2024-09-10T10:04:00.022183+0000","last_became_active":"2024-09-10T10:03:58.739953+0000","last_became_peered":"2024-09-10T10:03:58.739953+0000","last_unstale":"2024-09-10T10:04:00.022183+0000","last_undegraded":"2024-09-10T10:04:00.022183+0000","last_fullsized":"2024-09-10T10:04:00.022183+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:00.549552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989295+0000","last_change":"2024-09-10T10:03:58.764137+0000","last_active":"2024-09-10T10:04:00.989295+0000","last_peered":"2024-09-10T10:04:00.989295+0000","last_clean":"2024-09-10T10:04:00.989295+0000","last_became_active":"2024-09-10T10:03:58.763782+0000","last_became_peered":"2024-09-10T10:03:58.763782+0000","last_unstale":"2024-09-10T10:04:00.989295+0000","last_undegraded":"2024-09-10T10:04:00.989295+0000","last_fullsized":"2024-09-10T10:04:00.989295+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:05.264657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740628+0000","last_change":"2024-09-10T10:03:58.743508+0000","last_active":"2024-09-10T10:03:59.740628+0000","last_peered":"2024-09-10T10:03:59.740628+0000","last_clean":"2024-09-10T10:03:59.740628+0000","last_became_active":"2024-09-10T10:03:58.743224+0000","last_became_peered":"2024-09-10T10:03:58.743224+0000","last_unstale":"2024-09-10T10:03:59.740628+0000","last_undegraded":"2024-09-10T10:03:59.740628+0000","last_fullsized":"2024-09-10T10:03:59.740628+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:27.634373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989276+0000","last_change":"2024-09-10T10:03:58.741356+0000","last_active":"2024-09-10T10:04:00.989276+0000","last_peered":"2024-09-10T10:04:00.989276+0000","last_clean":"2024-09-10T10:04:00.989276+0000","last_became_active":"2024-09-10T10:03:58.741087+0000","last_became_peered":"2024-09-10T10:03:58.741087+0000","last_unstale":"2024-09-10T10:04:00.989276+0000","last_undegraded":"2024-09-10T10:04:00.989276+0000","last_fullsized":"2024-09-10T10:04:00.989276+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:27.401807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357105+0000","last_change":"2024-09-10T10:03:58.755723+0000","last_active":"2024-09-10T10:04:00.357105+0000","last_peered":"2024-09-10T10:04:00.357105+0000","last_clean":"2024-09-10T10:04:00.357105+0000","last_became_active":"2024-09-10T10:03:58.755475+0000","last_became_peered":"2024-09-10T10:03:58.755475+0000","last_unstale":"2024-09-10T10:04:00.357105+0000","last_undegraded":"2024-09-10T10:04:00.357105+0000","last_fullsized":"2024-09-10T10:04:00.357105+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:57.149687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738416+0000","last_change":"2024-09-10T10:03:58.753138+0000","last_active":"2024-09-10T10:03:59.738416+0000","last_peered":"2024-09-10T10:03:59.738416+0000","last_clean":"2024-09-10T10:03:59.738416+0000","last_became_active":"2024-09-10T10:03:58.752664+0000","last_became_peered":"2024-09-10T10:03:58.752664+0000","last_unstale":"2024-09-10T10:03:59.738416+0000","last_undegraded":"2024-09-10T10:03:59.738416+0000","last_fullsized":"2024-09-10T10:03:59.738416+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:16.941985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605199+0000","last_change":"2024-09-10T10:03:58.751327+0000","last_active":"2024-09-10T10:04:00.605199+0000","last_peered":"2024-09-10T10:04:00.605199+0000","last_clean":"2024-09-10T10:04:00.605199+0000","last_became_active":"2024-09-10T10:03:58.750908+0000","last_became_peered":"2024-09-10T10:03:58.750908+0000","last_unstale":"2024-09-10T10:04:00.605199+0000","last_undegraded":"2024-09-10T10:04:00.605199+0000","last_fullsized":"2024-09-10T10:04:00.605199+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:51.530530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572284+0000","last_change":"2024-09-10T10:03:58.739405+0000","last_active":"2024-09-10T10:04:00.572284+0000","last_peered":"2024-09-10T10:04:00.572284+0000","last_clean":"2024-09-10T10:04:00.572284+0000","last_became_active":"2024-09-10T10:03:58.738947+0000","last_became_peered":"2024-09-10T10:03:58.738947+0000","last_unstale":"2024-09-10T10:04:00.572284+0000","last_undegraded":"2024-09-10T10:04:00.572284+0000","last_fullsized":"2024-09-10T10:04:00.572284+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:57.149687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605047+0000","last_change":"2024-09-10T10:03:58.746915+0000","last_active":"2024-09-10T10:04:00.605047+0000","last_peered":"2024-09-10T10:04:00.605047+0000","last_clean":"2024-09-10T10:04:00.605047+0000","last_became_active":"2024-09-10T10:03:58.746607+0000","last_became_peered":"2024-09-10T10:03:58.746607+0000","last_unstale":"2024-09-10T10:04:00.605047+0000","last_undegraded":"2024-09-10T10:04:00.605047+0000","last_fullsized":"2024-09-10T10:04:00.605047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:39.026866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.604951+0000","last_change":"2024-09-10T10:03:58.745328+0000","last_active":"2024-09-10T10:04:00.604951+0000","last_peered":"2024-09-10T10:04:00.604951+0000","last_clean":"2024-09-10T10:04:00.604951+0000","last_became_active":"2024-09-10T10:03:58.745050+0000","last_became_peered":"2024-09-10T10:03:58.745050+0000","last_unstale":"2024-09-10T10:04:00.604951+0000","last_undegraded":"2024-09-10T10:04:00.604951+0000","last_fullsized":"2024-09-10T10:04:00.604951+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:54.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022112+0000","last_change":"2024-09-10T10:03:58.747797+0000","last_active":"2024-09-10T10:04:00.022112+0000","last_peered":"2024-09-10T10:04:00.022112+0000","last_clean":"2024-09-10T10:04:00.022112+0000","last_became_active":"2024-09-10T10:03:58.747501+0000","last_became_peered":"2024-09-10T10:03:58.747501+0000","last_unstale":"2024-09-10T10:04:00.022112+0000","last_undegraded":"2024-09-10T10:04:00.022112+0000","last_fullsized":"2024-09-10T10:04:00.022112+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:00.526867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740124+0000","last_change":"2024-09-10T10:03:58.737821+0000","last_active":"2024-09-10T10:03:59.740124+0000","last_peered":"2024-09-10T10:03:59.740124+0000","last_clean":"2024-09-10T10:03:59.740124+0000","last_became_active":"2024-09-10T10:03:58.737471+0000","last_became_peered":"2024-09-10T10:03:58.737471+0000","last_unstale":"2024-09-10T10:03:59.740124+0000","last_undegraded":"2024-09-10T10:03:59.740124+0000","last_fullsized":"2024-09-10T10:03:59.740124+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:27.634373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738486+0000","last_change":"2024-09-10T10:03:58.741943+0000","last_active":"2024-09-10T10:03:59.738486+0000","last_peered":"2024-09-10T10:03:59.738486+0000","last_clean":"2024-09-10T10:03:59.738486+0000","last_became_active":"2024-09-10T10:03:58.741299+0000","last_became_peered":"2024-09-10T10:03:58.741299+0000","last_unstale":"2024-09-10T10:03:59.738486+0000","last_undegraded":"2024-09-10T10:03:59.738486+0000","last_fullsized":"2024-09-10T10:03:59.738486+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:07.887129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"116'4","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022488+0000","last_change":"2024-09-10T10:03:57.733868+0000","last_active":"2024-09-10T10:04:00.022488+0000","last_peered":"2024-09-10T10:04:00.022488+0000","last_clean":"2024-09-10T10:04:00.022488+0000","last_became_active":"2024-09-10T10:03:57.733606+0000","last_became_peered":"2024-09-10T10:03:57.733606+0000","last_unstale":"2024-09-10T10:04:00.022488+0000","last_undegraded":"2024-09-10T10:04:00.022488+0000","last_fullsized":"2024-09-10T10:04:00.022488+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:18.217435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741272+0000","last_change":"2024-09-10T10:03:58.749838+0000","last_active":"2024-09-10T10:03:59.741272+0000","last_peered":"2024-09-10T10:03:59.741272+0000","last_clean":"2024-09-10T10:03:59.741272+0000","last_became_active":"2024-09-10T10:03:58.746742+0000","last_became_peered":"2024-09-10T10:03:58.746742+0000","last_unstale":"2024-09-10T10:03:59.741272+0000","last_undegraded":"2024-09-10T10:03:59.741272+0000","last_fullsized":"2024-09-10T10:03:59.741272+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:11.829596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572404+0000","last_change":"2024-09-10T10:03:57.734427+0000","last_active":"2024-09-10T10:04:00.572404+0000","last_peered":"2024-09-10T10:04:00.572404+0000","last_clean":"2024-09-10T10:04:00.572404+0000","last_became_active":"2024-09-10T10:03:57.734144+0000","last_became_peered":"2024-09-10T10:03:57.734144+0000","last_unstale":"2024-09-10T10:04:00.572404+0000","last_undegraded":"2024-09-10T10:04:00.572404+0000","last_fullsized":"2024-09-10T10:04:00.572404+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:15.520169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022439+0000","last_change":"2024-09-10T10:03:58.758463+0000","last_active":"2024-09-10T10:04:00.022439+0000","last_peered":"2024-09-10T10:04:00.022439+0000","last_clean":"2024-09-10T10:04:00.022439+0000","last_became_active":"2024-09-10T10:03:58.755771+0000","last_became_peered":"2024-09-10T10:03:58.755771+0000","last_unstale":"2024-09-10T10:04:00.022439+0000","last_undegraded":"2024-09-10T10:04:00.022439+0000","last_fullsized":"2024-09-10T10:04:00.022439+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:22.861633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989568+0000","last_change":"2024-09-10T10:03:57.725070+0000","last_active":"2024-09-10T10:04:00.989568+0000","last_peered":"2024-09-10T10:04:00.989568+0000","last_clean":"2024-09-10T10:04:00.989568+0000","last_became_active":"2024-09-10T10:03:57.724795+0000","last_became_peered":"2024-09-10T10:03:57.724795+0000","last_unstale":"2024-09-10T10:04:00.989568+0000","last_undegraded":"2024-09-10T10:04:00.989568+0000","last_fullsized":"2024-09-10T10:04:00.989568+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:03.379284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022320+0000","last_change":"2024-09-10T10:03:58.754249+0000","last_active":"2024-09-10T10:04:00.022320+0000","last_peered":"2024-09-10T10:04:00.022320+0000","last_clean":"2024-09-10T10:04:00.022320+0000","last_became_active":"2024-09-10T10:03:58.753894+0000","last_became_peered":"2024-09-10T10:03:58.753894+0000","last_unstale":"2024-09-10T10:04:00.022320+0000","last_undegraded":"2024-09-10T10:04:00.022320+0000","last_fullsized":"2024-09-10T10:04:00.022320+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:07.929163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357053+0000","last_change":"2024-09-10T10:03:57.730941+0000","last_active":"2024-09-10T10:04:00.357053+0000","last_peered":"2024-09-10T10:04:00.357053+0000","last_clean":"2024-09-10T10:04:00.357053+0000","last_became_active":"2024-09-10T10:03:57.730647+0000","last_became_peered":"2024-09-10T10:03:57.730647+0000","last_unstale":"2024-09-10T10:04:00.357053+0000","last_undegraded":"2024-09-10T10:04:00.357053+0000","last_fullsized":"2024-09-10T10:04:00.357053+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:04.226652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737801+0000","last_change":"2024-09-10T10:03:58.759861+0000","last_active":"2024-09-10T10:03:59.737801+0000","last_peered":"2024-09-10T10:03:59.737801+0000","last_clean":"2024-09-10T10:03:59.737801+0000","last_became_active":"2024-09-10T10:03:58.759592+0000","last_became_peered":"2024-09-10T10:03:58.759592+0000","last_unstale":"2024-09-10T10:03:59.737801+0000","last_undegraded":"2024-09-10T10:03:59.737801+0000","last_fullsized":"2024-09-10T10:03:59.737801+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:20.615050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356697+0000","last_change":"2024-09-10T10:03:57.728434+0000","last_active":"2024-09-10T10:04:00.356697+0000","last_peered":"2024-09-10T10:04:00.356697+0000","last_clean":"2024-09-10T10:04:00.356697+0000","last_became_active":"2024-09-10T10:03:57.728160+0000","last_became_peered":"2024-09-10T10:03:57.728160+0000","last_unstale":"2024-09-10T10:04:00.356697+0000","last_undegraded":"2024-09-10T10:04:00.356697+0000","last_fullsized":"2024-09-10T10:04:00.356697+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:28.102915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740987+0000","last_change":"2024-09-10T10:03:58.740526+0000","last_active":"2024-09-10T10:03:59.740987+0000","last_peered":"2024-09-10T10:03:59.740987+0000","last_clean":"2024-09-10T10:03:59.740987+0000","last_became_active":"2024-09-10T10:03:58.740217+0000","last_became_peered":"2024-09-10T10:03:58.740217+0000","last_unstale":"2024-09-10T10:03:59.740987+0000","last_undegraded":"2024-09-10T10:03:59.740987+0000","last_fullsized":"2024-09-10T10:03:59.740987+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:11.829596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737904+0000","last_change":"2024-09-10T10:03:57.728618+0000","last_active":"2024-09-10T10:03:59.737904+0000","last_peered":"2024-09-10T10:03:59.737904+0000","last_clean":"2024-09-10T10:03:59.737904+0000","last_became_active":"2024-09-10T10:03:57.728326+0000","last_became_peered":"2024-09-10T10:03:57.728326+0000","last_unstale":"2024-09-10T10:03:59.737904+0000","last_undegraded":"2024-09-10T10:03:59.737904+0000","last_fullsized":"2024-09-10T10:03:59.737904+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:03.701145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739963+0000","last_change":"2024-09-10T10:03:58.748663+0000","last_active":"2024-09-10T10:03:59.739963+0000","last_peered":"2024-09-10T10:03:59.739963+0000","last_clean":"2024-09-10T10:03:59.739963+0000","last_became_active":"2024-09-10T10:03:58.741211+0000","last_became_peered":"2024-09-10T10:03:58.741211+0000","last_unstale":"2024-09-10T10:03:59.739963+0000","last_undegraded":"2024-09-10T10:03:59.739963+0000","last_fullsized":"2024-09-10T10:03:59.739963+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:11.616816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739633+0000","last_change":"2024-09-10T10:03:57.730168+0000","last_active":"2024-09-10T10:03:59.739633+0000","last_peered":"2024-09-10T10:03:59.739633+0000","last_clean":"2024-09-10T10:03:59.739633+0000","last_became_active":"2024-09-10T10:03:57.729878+0000","last_became_peered":"2024-09-10T10:03:57.729878+0000","last_unstale":"2024-09-10T10:03:59.739633+0000","last_undegraded":"2024-09-10T10:03:59.739633+0000","last_fullsized":"2024-09-10T10:03:59.739633+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:31.293427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356738+0000","last_change":"2024-09-10T10:03:58.761674+0000","last_active":"2024-09-10T10:04:00.356738+0000","last_peered":"2024-09-10T10:04:00.356738+0000","last_clean":"2024-09-10T10:04:00.356738+0000","last_became_active":"2024-09-10T10:03:58.756662+0000","last_became_peered":"2024-09-10T10:03:58.756662+0000","last_unstale":"2024-09-10T10:04:00.356738+0000","last_undegraded":"2024-09-10T10:04:00.356738+0000","last_fullsized":"2024-09-10T10:04:00.356738+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:51.425956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356814+0000","last_change":"2024-09-10T10:03:57.730627+0000","last_active":"2024-09-10T10:04:00.356814+0000","last_peered":"2024-09-10T10:04:00.356814+0000","last_clean":"2024-09-10T10:04:00.356814+0000","last_became_active":"2024-09-10T10:03:57.730334+0000","last_became_peered":"2024-09-10T10:03:57.730334+0000","last_unstale":"2024-09-10T10:04:00.356814+0000","last_undegraded":"2024-09-10T10:04:00.356814+0000","last_fullsized":"2024-09-10T10:04:00.356814+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:28.471958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605219+0000","last_change":"2024-09-10T10:03:58.734305+0000","last_active":"2024-09-10T10:04:00.605219+0000","last_peered":"2024-09-10T10:04:00.605219+0000","last_clean":"2024-09-10T10:04:00.605219+0000","last_became_active":"2024-09-10T10:03:58.733998+0000","last_became_peered":"2024-09-10T10:03:58.733998+0000","last_unstale":"2024-09-10T10:04:00.605219+0000","last_undegraded":"2024-09-10T10:04:00.605219+0000","last_fullsized":"2024-09-10T10:04:00.605219+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:31.528456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572497+0000","last_change":"2024-09-10T10:03:57.734752+0000","last_active":"2024-09-10T10:04:00.572497+0000","last_peered":"2024-09-10T10:04:00.572497+0000","last_clean":"2024-09-10T10:04:00.572497+0000","last_became_active":"2024-09-10T10:03:57.734368+0000","last_became_peered":"2024-09-10T10:03:57.734368+0000","last_unstale":"2024-09-10T10:04:00.572497+0000","last_undegraded":"2024-09-10T10:04:00.572497+0000","last_fullsized":"2024-09-10T10:04:00.572497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:48.285278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572377+0000","last_change":"2024-09-10T10:03:58.739626+0000","last_active":"2024-09-10T10:04:00.572377+0000","last_peered":"2024-09-10T10:04:00.572377+0000","last_clean":"2024-09-10T10:04:00.572377+0000","last_became_active":"2024-09-10T10:03:58.739176+0000","last_became_peered":"2024-09-10T10:03:58.739176+0000","last_unstale":"2024-09-10T10:04:00.572377+0000","last_undegraded":"2024-09-10T10:04:00.572377+0000","last_fullsized":"2024-09-10T10:04:00.572377+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:26:43.615114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738507+0000","last_change":"2024-09-10T10:03:57.720975+0000","last_active":"2024-09-10T10:03:59.738507+0000","last_peered":"2024-09-10T10:03:59.738507+0000","last_clean":"2024-09-10T10:03:59.738507+0000","last_became_active":"2024-09-10T10:03:57.720698+0000","last_became_peered":"2024-09-10T10:03:57.720698+0000","last_unstale":"2024-09-10T10:03:59.738507+0000","last_undegraded":"2024-09-10T10:03:59.738507+0000","last_fullsized":"2024-09-10T10:03:59.738507+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:39.207283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022591+0000","last_change":"2024-09-10T10:03:58.740073+0000","last_active":"2024-09-10T10:04:00.022591+0000","last_peered":"2024-09-10T10:04:00.022591+0000","last_clean":"2024-09-10T10:04:00.022591+0000","last_became_active":"2024-09-10T10:03:58.739782+0000","last_became_peered":"2024-09-10T10:03:58.739782+0000","last_unstale":"2024-09-10T10:04:00.022591+0000","last_undegraded":"2024-09-10T10:04:00.022591+0000","last_fullsized":"2024-09-10T10:04:00.022591+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:34.811982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"116'3","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572182+0000","last_change":"2024-09-10T10:03:57.731228+0000","last_active":"2024-09-10T10:04:00.572182+0000","last_peered":"2024-09-10T10:04:00.572182+0000","last_clean":"2024-09-10T10:04:00.572182+0000","last_became_active":"2024-09-10T10:03:57.730947+0000","last_became_peered":"2024-09-10T10:03:57.730947+0000","last_unstale":"2024-09-10T10:04:00.572182+0000","last_undegraded":"2024-09-10T10:04:00.572182+0000","last_fullsized":"2024-09-10T10:04:00.572182+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:48.478042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022422+0000","last_change":"2024-09-10T10:03:58.746359+0000","last_active":"2024-09-10T10:04:00.022422+0000","last_peered":"2024-09-10T10:04:00.022422+0000","last_clean":"2024-09-10T10:04:00.022422+0000","last_became_active":"2024-09-10T10:03:58.745992+0000","last_became_peered":"2024-09-10T10:03:58.745992+0000","last_unstale":"2024-09-10T10:04:00.022422+0000","last_undegraded":"2024-09-10T10:04:00.022422+0000","last_fullsized":"2024-09-10T10:04:00.022422+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:59.017348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"116'6","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740637+0000","last_change":"2024-09-10T10:03:57.730625+0000","last_active":"2024-09-10T10:03:59.740637+0000","last_peered":"2024-09-10T10:03:59.740637+0000","last_clean":"2024-09-10T10:03:59.740637+0000","last_became_active":"2024-09-10T10:03:57.730333+0000","last_became_peered":"2024-09-10T10:03:57.730333+0000","last_unstale":"2024-09-10T10:03:59.740637+0000","last_undegraded":"2024-09-10T10:03:59.740637+0000","last_fullsized":"2024-09-10T10:03:59.740637+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:26:42.609356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605272+0000","last_change":"2024-09-10T10:03:58.737717+0000","last_active":"2024-09-10T10:04:00.605272+0000","last_peered":"2024-09-10T10:04:00.605272+0000","last_clean":"2024-09-10T10:04:00.605272+0000","last_became_active":"2024-09-10T10:03:58.737382+0000","last_became_peered":"2024-09-10T10:03:58.737382+0000","last_unstale":"2024-09-10T10:04:00.605272+0000","last_undegraded":"2024-09-10T10:04:00.605272+0000","last_fullsized":"2024-09-10T10:04:00.605272+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:21.786863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605049+0000","last_change":"2024-09-10T10:03:57.718706+0000","last_active":"2024-09-10T10:04:00.605049+0000","last_peered":"2024-09-10T10:04:00.605049+0000","last_clean":"2024-09-10T10:04:00.605049+0000","last_became_active":"2024-09-10T10:03:57.718454+0000","last_became_peered":"2024-09-10T10:03:57.718454+0000","last_unstale":"2024-09-10T10:04:00.605049+0000","last_undegraded":"2024-09-10T10:04:00.605049+0000","last_fullsized":"2024-09-10T10:04:00.605049+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:32.451132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356700+0000","last_change":"2024-09-10T10:03:58.760468+0000","last_active":"2024-09-10T10:04:00.356700+0000","last_peered":"2024-09-10T10:04:00.356700+0000","last_clean":"2024-09-10T10:04:00.356700+0000","last_became_active":"2024-09-10T10:03:58.760155+0000","last_became_peered":"2024-09-10T10:03:58.760155+0000","last_unstale":"2024-09-10T10:04:00.356700+0000","last_undegraded":"2024-09-10T10:04:00.356700+0000","last_fullsized":"2024-09-10T10:04:00.356700+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:50.827741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356841+0000","last_change":"2024-09-10T10:03:57.733675+0000","last_active":"2024-09-10T10:04:00.356841+0000","last_peered":"2024-09-10T10:04:00.356841+0000","last_clean":"2024-09-10T10:04:00.356841+0000","last_became_active":"2024-09-10T10:03:57.733348+0000","last_became_peered":"2024-09-10T10:03:57.733348+0000","last_unstale":"2024-09-10T10:04:00.356841+0000","last_undegraded":"2024-09-10T10:04:00.356841+0000","last_fullsized":"2024-09-10T10:04:00.356841+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:21:42.219319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740446+0000","last_change":"2024-09-10T10:03:58.742303+0000","last_active":"2024-09-10T10:03:59.740446+0000","last_peered":"2024-09-10T10:03:59.740446+0000","last_clean":"2024-09-10T10:03:59.740446+0000","last_became_active":"2024-09-10T10:03:58.742017+0000","last_became_peered":"2024-09-10T10:03:58.742017+0000","last_unstale":"2024-09-10T10:03:59.740446+0000","last_undegraded":"2024-09-10T10:03:59.740446+0000","last_fullsized":"2024-09-10T10:03:59.740446+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:07.188145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"116'8","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.739978+0000","last_change":"2024-09-10T10:03:57.718897+0000","last_active":"2024-09-10T10:03:59.739978+0000","last_peered":"2024-09-10T10:03:59.739978+0000","last_clean":"2024-09-10T10:03:59.739978+0000","last_became_active":"2024-09-10T10:03:57.718648+0000","last_became_peered":"2024-09-10T10:03:57.718648+0000","last_unstale":"2024-09-10T10:03:59.739978+0000","last_undegraded":"2024-09-10T10:03:59.739978+0000","last_fullsized":"2024-09-10T10:03:59.739978+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:31.293427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741260+0000","last_change":"2024-09-10T10:03:58.753799+0000","last_active":"2024-09-10T10:03:59.741260+0000","last_peered":"2024-09-10T10:03:59.741260+0000","last_clean":"2024-09-10T10:03:59.741260+0000","last_became_active":"2024-09-10T10:03:58.751826+0000","last_became_peered":"2024-09-10T10:03:58.751826+0000","last_unstale":"2024-09-10T10:03:59.741260+0000","last_undegraded":"2024-09-10T10:03:59.741260+0000","last_fullsized":"2024-09-10T10:03:59.741260+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:19.223193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572531+0000","last_change":"2024-09-10T10:03:57.733712+0000","last_active":"2024-09-10T10:04:00.572531+0000","last_peered":"2024-09-10T10:04:00.572531+0000","last_clean":"2024-09-10T10:04:00.572531+0000","last_became_active":"2024-09-10T10:03:57.733429+0000","last_became_peered":"2024-09-10T10:03:57.733429+0000","last_unstale":"2024-09-10T10:04:00.572531+0000","last_undegraded":"2024-09-10T10:04:00.572531+0000","last_fullsized":"2024-09-10T10:04:00.572531+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:52.770882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572464+0000","last_change":"2024-09-10T10:03:58.741757+0000","last_active":"2024-09-10T10:04:00.572464+0000","last_peered":"2024-09-10T10:04:00.572464+0000","last_clean":"2024-09-10T10:04:00.572464+0000","last_became_active":"2024-09-10T10:03:58.741484+0000","last_became_peered":"2024-09-10T10:03:58.741484+0000","last_unstale":"2024-09-10T10:04:00.572464+0000","last_undegraded":"2024-09-10T10:04:00.572464+0000","last_fullsized":"2024-09-10T10:04:00.572464+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:32.299185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989522+0000","last_change":"2024-09-10T10:03:57.732023+0000","last_active":"2024-09-10T10:04:00.989522+0000","last_peered":"2024-09-10T10:04:00.989522+0000","last_clean":"2024-09-10T10:04:00.989522+0000","last_became_active":"2024-09-10T10:03:57.730132+0000","last_became_peered":"2024-09-10T10:03:57.730132+0000","last_unstale":"2024-09-10T10:04:00.989522+0000","last_undegraded":"2024-09-10T10:04:00.989522+0000","last_fullsized":"2024-09-10T10:04:00.989522+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:03.522101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738118+0000","last_change":"2024-09-10T10:03:58.753910+0000","last_active":"2024-09-10T10:03:59.738118+0000","last_peered":"2024-09-10T10:03:59.738118+0000","last_clean":"2024-09-10T10:03:59.738118+0000","last_became_active":"2024-09-10T10:03:58.753631+0000","last_became_peered":"2024-09-10T10:03:58.753631+0000","last_unstale":"2024-09-10T10:03:59.738118+0000","last_undegraded":"2024-09-10T10:03:59.738118+0000","last_fullsized":"2024-09-10T10:03:59.738118+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:33.892796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741196+0000","last_change":"2024-09-10T10:03:57.721762+0000","last_active":"2024-09-10T10:03:59.741196+0000","last_peered":"2024-09-10T10:03:59.741196+0000","last_clean":"2024-09-10T10:03:59.741196+0000","last_became_active":"2024-09-10T10:03:57.721449+0000","last_became_peered":"2024-09-10T10:03:57.721449+0000","last_unstale":"2024-09-10T10:03:59.741196+0000","last_undegraded":"2024-09-10T10:03:59.741196+0000","last_fullsized":"2024-09-10T10:03:59.741196+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:17.778326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357106+0000","last_change":"2024-09-10T10:03:58.760795+0000","last_active":"2024-09-10T10:04:00.357106+0000","last_peered":"2024-09-10T10:04:00.357106+0000","last_clean":"2024-09-10T10:04:00.357106+0000","last_became_active":"2024-09-10T10:03:58.760472+0000","last_became_peered":"2024-09-10T10:03:58.760472+0000","last_unstale":"2024-09-10T10:04:00.357106+0000","last_undegraded":"2024-09-10T10:04:00.357106+0000","last_fullsized":"2024-09-10T10:04:00.357106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:08.214538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572311+0000","last_change":"2024-09-10T10:03:57.733091+0000","last_active":"2024-09-10T10:04:00.572311+0000","last_peered":"2024-09-10T10:04:00.572311+0000","last_clean":"2024-09-10T10:04:00.572311+0000","last_became_active":"2024-09-10T10:03:57.732834+0000","last_became_peered":"2024-09-10T10:03:57.732834+0000","last_unstale":"2024-09-10T10:04:00.572311+0000","last_undegraded":"2024-09-10T10:04:00.572311+0000","last_fullsized":"2024-09-10T10:04:00.572311+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:49.821984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.604962+0000","last_change":"2024-09-10T10:03:58.754290+0000","last_active":"2024-09-10T10:04:00.604962+0000","last_peered":"2024-09-10T10:04:00.604962+0000","last_clean":"2024-09-10T10:04:00.604962+0000","last_became_active":"2024-09-10T10:03:58.753982+0000","last_became_peered":"2024-09-10T10:03:58.753982+0000","last_unstale":"2024-09-10T10:04:00.604962+0000","last_undegraded":"2024-09-10T10:04:00.604962+0000","last_fullsized":"2024-09-10T10:04:00.604962+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:08:17.146980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572157+0000","last_change":"2024-09-10T10:03:57.735283+0000","last_active":"2024-09-10T10:04:00.572157+0000","last_peered":"2024-09-10T10:04:00.572157+0000","last_clean":"2024-09-10T10:04:00.572157+0000","last_became_active":"2024-09-10T10:03:57.734668+0000","last_became_peered":"2024-09-10T10:03:57.734668+0000","last_unstale":"2024-09-10T10:04:00.572157+0000","last_undegraded":"2024-09-10T10:04:00.572157+0000","last_fullsized":"2024-09-10T10:04:00.572157+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:04.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740374+0000","last_change":"2024-09-10T10:03:58.745639+0000","last_active":"2024-09-10T10:03:59.740374+0000","last_peered":"2024-09-10T10:03:59.740374+0000","last_clean":"2024-09-10T10:03:59.740374+0000","last_became_active":"2024-09-10T10:03:58.745380+0000","last_became_peered":"2024-09-10T10:03:58.745380+0000","last_unstale":"2024-09-10T10:03:59.740374+0000","last_undegraded":"2024-09-10T10:03:59.740374+0000","last_fullsized":"2024-09-10T10:03:59.740374+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:15.755172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737913+0000","last_change":"2024-09-10T10:03:57.718663+0000","last_active":"2024-09-10T10:03:59.737913+0000","last_peered":"2024-09-10T10:03:59.737913+0000","last_clean":"2024-09-10T10:03:59.737913+0000","last_became_active":"2024-09-10T10:03:57.718381+0000","last_became_peered":"2024-09-10T10:03:57.718381+0000","last_unstale":"2024-09-10T10:03:59.737913+0000","last_undegraded":"2024-09-10T10:03:59.737913+0000","last_fullsized":"2024-09-10T10:03:59.737913+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:21.131893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356706+0000","last_change":"2024-09-10T10:03:58.750354+0000","last_active":"2024-09-10T10:04:00.356706+0000","last_peered":"2024-09-10T10:04:00.356706+0000","last_clean":"2024-09-10T10:04:00.356706+0000","last_became_active":"2024-09-10T10:03:58.750010+0000","last_became_peered":"2024-09-10T10:03:58.750010+0000","last_unstale":"2024-09-10T10:04:00.356706+0000","last_undegraded":"2024-09-10T10:04:00.356706+0000","last_fullsized":"2024-09-10T10:04:00.356706+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:49.483800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737717+0000","last_change":"2024-09-10T10:03:57.732729+0000","last_active":"2024-09-10T10:03:59.737717+0000","last_peered":"2024-09-10T10:03:59.737717+0000","last_clean":"2024-09-10T10:03:59.737717+0000","last_became_active":"2024-09-10T10:03:57.732475+0000","last_became_peered":"2024-09-10T10:03:57.732475+0000","last_unstale":"2024-09-10T10:03:59.737717+0000","last_undegraded":"2024-09-10T10:03:59.737717+0000","last_fullsized":"2024-09-10T10:03:59.737717+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:49.760077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022234+0000","last_change":"2024-09-10T10:03:58.748742+0000","last_active":"2024-09-10T10:04:00.022234+0000","last_peered":"2024-09-10T10:04:00.022234+0000","last_clean":"2024-09-10T10:04:00.022234+0000","last_became_active":"2024-09-10T10:03:58.748041+0000","last_became_peered":"2024-09-10T10:03:58.748041+0000","last_unstale":"2024-09-10T10:04:00.022234+0000","last_undegraded":"2024-09-10T10:04:00.022234+0000","last_fullsized":"2024-09-10T10:04:00.022234+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:24.709279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989233+0000","last_change":"2024-09-10T10:03:57.726476+0000","last_active":"2024-09-10T10:04:00.989233+0000","last_peered":"2024-09-10T10:04:00.989233+0000","last_clean":"2024-09-10T10:04:00.989233+0000","last_became_active":"2024-09-10T10:03:57.726149+0000","last_became_peered":"2024-09-10T10:03:57.726149+0000","last_unstale":"2024-09-10T10:04:00.989233+0000","last_undegraded":"2024-09-10T10:04:00.989233+0000","last_fullsized":"2024-09-10T10:04:00.989233+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:41.858327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.740956+0000","last_change":"2024-09-10T10:03:58.742589+0000","last_active":"2024-09-10T10:03:59.740956+0000","last_peered":"2024-09-10T10:03:59.740956+0000","last_clean":"2024-09-10T10:03:59.740956+0000","last_became_active":"2024-09-10T10:03:58.742319+0000","last_became_peered":"2024-09-10T10:03:58.742319+0000","last_unstale":"2024-09-10T10:03:59.740956+0000","last_undegraded":"2024-09-10T10:03:59.740956+0000","last_fullsized":"2024-09-10T10:03:59.740956+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:19.223193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605331+0000","last_change":"2024-09-10T10:03:57.734198+0000","last_active":"2024-09-10T10:04:00.605331+0000","last_peered":"2024-09-10T10:04:00.605331+0000","last_clean":"2024-09-10T10:04:00.605331+0000","last_became_active":"2024-09-10T10:03:57.733799+0000","last_became_peered":"2024-09-10T10:03:57.733799+0000","last_unstale":"2024-09-10T10:04:00.605331+0000","last_undegraded":"2024-09-10T10:04:00.605331+0000","last_fullsized":"2024-09-10T10:04:00.605331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:19.425659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.605314+0000","last_change":"2024-09-10T10:03:58.744898+0000","last_active":"2024-09-10T10:04:00.605314+0000","last_peered":"2024-09-10T10:04:00.605314+0000","last_clean":"2024-09-10T10:04:00.605314+0000","last_became_active":"2024-09-10T10:03:58.744626+0000","last_became_peered":"2024-09-10T10:03:58.744626+0000","last_unstale":"2024-09-10T10:04:00.605314+0000","last_undegraded":"2024-09-10T10:04:00.605314+0000","last_fullsized":"2024-09-10T10:04:00.605314+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:24.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"117'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.732875+0000","last_change":"2024-09-10T10:03:57.733131+0000","last_active":"2024-09-10T10:04:00.732875+0000","last_peered":"2024-09-10T10:04:00.732875+0000","last_clean":"2024-09-10T10:04:00.732875+0000","last_became_active":"2024-09-10T10:03:57.732834+0000","last_became_peered":"2024-09-10T10:03:57.732834+0000","last_unstale":"2024-09-10T10:04:00.732875+0000","last_undegraded":"2024-09-10T10:04:00.732875+0000","last_fullsized":"2024-09-10T10:04:00.732875+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:41.030732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357008+0000","last_change":"2024-09-10T10:03:58.758443+0000","last_active":"2024-09-10T10:04:00.357008+0000","last_peered":"2024-09-10T10:04:00.357008+0000","last_clean":"2024-09-10T10:04:00.357008+0000","last_became_active":"2024-09-10T10:03:58.754351+0000","last_became_peered":"2024-09-10T10:03:58.754351+0000","last_unstale":"2024-09-10T10:04:00.357008+0000","last_undegraded":"2024-09-10T10:04:00.357008+0000","last_fullsized":"2024-09-10T10:04:00.357008+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:26.889887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.357058+0000","last_change":"2024-09-10T10:03:57.733776+0000","last_active":"2024-09-10T10:04:00.357058+0000","last_peered":"2024-09-10T10:04:00.357058+0000","last_clean":"2024-09-10T10:04:00.357058+0000","last_became_active":"2024-09-10T10:03:57.730896+0000","last_became_peered":"2024-09-10T10:03:57.730896+0000","last_unstale":"2024-09-10T10:04:00.357058+0000","last_undegraded":"2024-09-10T10:04:00.357058+0000","last_fullsized":"2024-09-10T10:04:00.357058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:17.495422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.741060+0000","last_change":"2024-09-10T10:03:58.732754+0000","last_active":"2024-09-10T10:03:59.741060+0000","last_peered":"2024-09-10T10:03:59.741060+0000","last_clean":"2024-09-10T10:03:59.741060+0000","last_became_active":"2024-09-10T10:03:58.732469+0000","last_became_peered":"2024-09-10T10:03:58.732469+0000","last_unstale":"2024-09-10T10:03:59.741060+0000","last_undegraded":"2024-09-10T10:03:59.741060+0000","last_fullsized":"2024-09-10T10:03:59.741060+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:00.860201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989331+0000","last_change":"2024-09-10T10:03:57.730736+0000","last_active":"2024-09-10T10:04:00.989331+0000","last_peered":"2024-09-10T10:04:00.989331+0000","last_clean":"2024-09-10T10:04:00.989331+0000","last_became_active":"2024-09-10T10:03:57.730437+0000","last_became_peered":"2024-09-10T10:03:57.730437+0000","last_unstale":"2024-09-10T10:04:00.989331+0000","last_undegraded":"2024-09-10T10:04:00.989331+0000","last_fullsized":"2024-09-10T10:04:00.989331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:54.908884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022359+0000","last_change":"2024-09-10T10:03:58.757454+0000","last_active":"2024-09-10T10:04:00.022359+0000","last_peered":"2024-09-10T10:04:00.022359+0000","last_clean":"2024-09-10T10:04:00.022359+0000","last_became_active":"2024-09-10T10:03:58.757154+0000","last_became_peered":"2024-09-10T10:03:58.757154+0000","last_unstale":"2024-09-10T10:04:00.022359+0000","last_undegraded":"2024-09-10T10:04:00.022359+0000","last_fullsized":"2024-09-10T10:04:00.022359+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:17:02.778168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356844+0000","last_change":"2024-09-10T10:03:57.733419+0000","last_active":"2024-09-10T10:04:00.356844+0000","last_peered":"2024-09-10T10:04:00.356844+0000","last_clean":"2024-09-10T10:04:00.356844+0000","last_became_active":"2024-09-10T10:03:57.733145+0000","last_became_peered":"2024-09-10T10:03:57.733145+0000","last_unstale":"2024-09-10T10:04:00.356844+0000","last_undegraded":"2024-09-10T10:04:00.356844+0000","last_fullsized":"2024-09-10T10:04:00.356844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:20:57.726265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.572204+0000","last_change":"2024-09-10T10:03:58.749552+0000","last_active":"2024-09-10T10:04:00.572204+0000","last_peered":"2024-09-10T10:04:00.572204+0000","last_clean":"2024-09-10T10:04:00.572204+0000","last_became_active":"2024-09-10T10:03:58.749329+0000","last_became_peered":"2024-09-10T10:03:58.749329+0000","last_unstale":"2024-09-10T10:04:00.572204+0000","last_undegraded":"2024-09-10T10:04:00.572204+0000","last_fullsized":"2024-09-10T10:04:00.572204+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:08.214538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022141+0000","last_change":"2024-09-10T10:03:57.735891+0000","last_active":"2024-09-10T10:04:00.022141+0000","last_peered":"2024-09-10T10:04:00.022141+0000","last_clean":"2024-09-10T10:04:00.022141+0000","last_became_active":"2024-09-10T10:03:57.735484+0000","last_became_peered":"2024-09-10T10:03:57.735484+0000","last_unstale":"2024-09-10T10:04:00.022141+0000","last_undegraded":"2024-09-10T10:04:00.022141+0000","last_fullsized":"2024-09-10T10:04:00.022141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:59.854443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.738301+0000","last_change":"2024-09-10T10:03:58.747711+0000","last_active":"2024-09-10T10:03:59.738301+0000","last_peered":"2024-09-10T10:03:59.738301+0000","last_clean":"2024-09-10T10:03:59.738301+0000","last_became_active":"2024-09-10T10:03:58.747389+0000","last_became_peered":"2024-09-10T10:03:58.747389+0000","last_unstale":"2024-09-10T10:03:59.738301+0000","last_undegraded":"2024-09-10T10:03:59.738301+0000","last_fullsized":"2024-09-10T10:03:59.738301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:29.183472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:03:59.737819+0000","last_change":"2024-09-10T10:03:57.720632+0000","last_active":"2024-09-10T10:03:59.737819+0000","last_peered":"2024-09-10T10:03:59.737819+0000","last_clean":"2024-09-10T10:03:59.737819+0000","last_became_active":"2024-09-10T10:03:57.720348+0000","last_became_peered":"2024-09-10T10:03:57.720348+0000","last_unstale":"2024-09-10T10:03:59.737819+0000","last_undegraded":"2024-09-10T10:03:59.737819+0000","last_fullsized":"2024-09-10T10:03:59.737819+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:22.754697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.356738+0000","last_change":"2024-09-10T10:03:58.755159+0000","last_active":"2024-09-10T10:04:00.356738+0000","last_peered":"2024-09-10T10:04:00.356738+0000","last_clean":"2024-09-10T10:04:00.356738+0000","last_became_active":"2024-09-10T10:03:58.754915+0000","last_became_peered":"2024-09-10T10:03:58.754915+0000","last_unstale":"2024-09-10T10:04:00.356738+0000","last_undegraded":"2024-09-10T10:04:00.356738+0000","last_fullsized":"2024-09-10T10:04:00.356738+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:46.621125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.989332+0000","last_change":"2024-09-10T10:03:57.732231+0000","last_active":"2024-09-10T10:04:00.989332+0000","last_peered":"2024-09-10T10:04:00.989332+0000","last_clean":"2024-09-10T10:04:00.989332+0000","last_became_active":"2024-09-10T10:03:57.731815+0000","last_became_peered":"2024-09-10T10:03:57.731815+0000","last_unstale":"2024-09-10T10:04:00.989332+0000","last_undegraded":"2024-09-10T10:04:00.989332+0000","last_fullsized":"2024-09-10T10:04:00.989332+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:49.394741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022006+0000","last_change":"2024-09-10T10:03:58.742743+0000","last_active":"2024-09-10T10:04:00.022006+0000","last_peered":"2024-09-10T10:04:00.022006+0000","last_clean":"2024-09-10T10:04:00.022006+0000","last_became_active":"2024-09-10T10:03:58.741777+0000","last_became_peered":"2024-09-10T10:03:58.741777+0000","last_unstale":"2024-09-10T10:04:00.022006+0000","last_undegraded":"2024-09-10T10:04:00.022006+0000","last_fullsized":"2024-09-10T10:04:00.022006+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:57.702216+0000","last_clean_scrub_stamp":"2024-09-10T10:03:57.702216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:32.220934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.022318+0000","last_change":"2024-09-10T10:03:57.735434+0000","last_active":"2024-09-10T10:04:00.022318+0000","last_peered":"2024-09-10T10:04:00.022318+0000","last_clean":"2024-09-10T10:04:00.022318+0000","last_became_active":"2024-09-10T10:03:57.734831+0000","last_became_peered":"2024-09-10T10:03:57.734831+0000","last_unstale":"2024-09-10T10:04:00.022318+0000","last_undegraded":"2024-09-10T10:04:00.022318+0000","last_fullsized":"2024-09-10T10:04:00.022318+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:03:56.696458+0000","last_clean_scrub_stamp":"2024-09-10T10:03:56.696458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:10.823838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":382,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-10T10:04:00.021797+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:04:00.021797+0000","last_peered":"2024-09-10T10:04:00.021797+0000","last_clean":"2024-09-10T10:04:00.021797+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:04:00.021797+0000","last_undegraded":"2024-09-10T10:04:00.021797+0000","last_fullsized":"2024-09-10T10:04:00.021797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":1231063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1466368,"data_stored":640057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":638557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1458176,"data_stored":638515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542218,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1462272,"data_stored":638605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509513,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":639061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":638445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2048000,"data_stored":1228813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:04:03.187 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:04:03.188 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:04:03.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.362+0000 7f0459e00640 1 Processor -- start 2024-09-10T10:04:03.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.362+0000 7f0459e00640 1 -- start start 2024-09-10T10:04:03.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.362+0000 7f0459e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045415c560 0x7f045406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.362+0000 7f0459e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f045406f430 con 0x7f045406f010 2024-09-10T10:04:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.362+0000 7f0459e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0454155d50 con 0x7f0454155970 2024-09-10T10:04:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.362+0000 7f0459e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04540b4570 con 0x7f045415c560 2024-09-10T10:04:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0453400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f045406f010 0x7f04540ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43802/0 (socket says 172.21.15.110:43802) 2024-09-10T10:04:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0453400640 1 -- 172.21.15.110:0/2149780831 learned_addr learned my addr 172.21.15.110:0/2149780831 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:03.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0452000640 1 --2- 172.21.15.110:0/2149780831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045415c560 0x7f045406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0451600640 1 -- 172.21.15.110:0/2149780831 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136714833 0 0) 0x7f045406f430 con 0x7f045406f010 2024-09-10T10:04:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0451600640 1 -- 172.21.15.110:0/2149780831 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0438003530 con 0x7f045406f010 2024-09-10T10:04:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0451600640 1 -- 172.21.15.110:0/2149780831 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982632635 0 0) 0x7f0454155d50 con 0x7f0454155970 2024-09-10T10:04:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0451600640 1 -- 172.21.15.110:0/2149780831 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f045406f430 con 0x7f0454155970 2024-09-10T10:04:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0452000640 1 -- 172.21.15.110:0/2149780831 >> v1:172.21.15.145:6789/0 conn(0x7f0454155970 legacy=0x7f04540cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0452000640 1 -- 172.21.15.110:0/2149780831 >> v1:172.21.15.110:6789/0 conn(0x7f045406f010 legacy=0x7f04540ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0452000640 1 -- 172.21.15.110:0/2149780831 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04540b34d0 con 0x7f045415c560 2024-09-10T10:04:03.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0452000640 1 --2- 172.21.15.110:0/2149780831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045415c560 0x7f045406ead0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04400097f0 tx=0x7f044001c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=375ab7b6ec3c7bfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:03.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0451600640 1 -- 172.21.15.110:0/2149780831 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400ab620 con 0x7f045415c560 2024-09-10T10:04:03.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0451600640 1 -- 172.21.15.110:0/2149780831 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04400b0070 con 0x7f045415c560 2024-09-10T10:04:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.366+0000 7f0451600640 1 -- 172.21.15.110:0/2149780831 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400ab9c0 con 0x7f045415c560 2024-09-10T10:04:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- 172.21.15.110:0/2149780831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045415c560 msgr2=0x7f045406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 --2- 172.21.15.110:0/2149780831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045415c560 0x7f045406ead0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04400097f0 tx=0x7f044001c420 comp rx=0 tx=0).stop 2024-09-10T10:04:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- 172.21.15.110:0/2149780831 shutdown_connections 2024-09-10T10:04:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 --2- 172.21.15.110:0/2149780831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045415c560 0x7f045406ead0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- 172.21.15.110:0/2149780831 >> 172.21.15.110:0/2149780831 conn(0x7f045406bc50 msgr2=0x7f045406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:03.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- 172.21.15.110:0/2149780831 shutdown_connections 2024-09-10T10:04:03.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- 172.21.15.110:0/2149780831 wait complete. 2024-09-10T10:04:03.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 Processor -- start 2024-09-10T10:04:03.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- start start 2024-09-10T10:04:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045406f010 0x7f045415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454155970 0x7f0454072950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f045415c560 0x7f0454072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04540cf340 con 0x7f045415c560 2024-09-10T10:04:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04540b4050 con 0x7f0454155970 2024-09-10T10:04:03.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0459e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04540cf4c0 con 0x7f045406f010 2024-09-10T10:04:03.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0453400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f045415c560 0x7f0454072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0453400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f045415c560 0x7f0454072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50536/0 (socket says 172.21.15.110:50536) 2024-09-10T10:04:03.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0452a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045406f010 0x7f045415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0453400640 1 -- 172.21.15.110:0/2554743853 learned_addr learned my addr 172.21.15.110:0/2554743853 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:03.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.370+0000 7f0452000640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454155970 0x7f0454072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0453400640 1 -- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045406f010 msgr2=0x7f045415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0453400640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045406f010 0x7f045415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0453400640 1 -- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454155970 msgr2=0x7f0454072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0453400640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454155970 0x7f0454072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0453400640 1 -- 172.21.15.110:0/2554743853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045415cbf0 con 0x7f045415c560 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0453400640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f045415c560 0x7f0454072e90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f044c00cb40 tx=0x7f044c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f044b400640 1 -- 172.21.15.110:0/2554743853 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c017070 con 0x7f045415c560 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f044b400640 1 -- 172.21.15.110:0/2554743853 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c00acc0 con 0x7f045415c560 2024-09-10T10:04:03.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f044b400640 1 -- 172.21.15.110:0/2554743853 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c012640 con 0x7f045415c560 2024-09-10T10:04:03.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045415ced0 con 0x7f045415c560 2024-09-10T10:04:03.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045415d490 con 0x7f045415c560 2024-09-10T10:04:03.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f045415d690 con 0x7f045415c560 2024-09-10T10:04:03.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f044b400640 1 -- 172.21.15.110:0/2554743853 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f044c01a070 con 0x7f045415c560 2024-09-10T10:04:03.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f044b400640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0434067cb0 0x7f043406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f044b400640 1 -- 172.21.15.110:0/2554743853 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f044c086150 con 0x7f045415c560 2024-09-10T10:04:03.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.374+0000 7f0452a00640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0434067cb0 0x7f043406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.378+0000 7f0452a00640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0434067cb0 0x7f043406a170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f045415e8f0 tx=0x7f0444009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:03.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.382+0000 7f044b400640 1 -- 172.21.15.110:0/2554743853 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f044c0560d0 con 0x7f045415c560 2024-09-10T10:04:03.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.594+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f04540ce420 con 0x7f045415c560 2024-09-10T10:04:03.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f044b400640 1 -- 172.21.15.110:0/2554743853 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f044c05a3d0 con 0x7f045415c560 2024-09-10T10:04:03.602 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:04:03.602 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:04:03.602 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:04:03.602 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:04:03.603 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:04:03.603 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:04:03.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0434067cb0 msgr2=0x7f043406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0434067cb0 0x7f043406a170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f045415e8f0 tx=0x7f0444009040 comp rx=0 tx=0).stop 2024-09-10T10:04:03.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f045415c560 msgr2=0x7f0454072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f045415c560 0x7f0454072e90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f044c00cb40 tx=0x7f044c00a060 comp rx=0 tx=0).stop 2024-09-10T10:04:03.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 shutdown_connections 2024-09-10T10:04:03.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0434067cb0 0x7f043406a170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f045415c560 0x7f0454072e90 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454155970 0x7f0454072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 --2- 172.21.15.110:0/2554743853 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f045406f010 0x7f045415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 >> 172.21.15.110:0/2554743853 conn(0x7f045406bc50 msgr2=0x7f04540710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:03.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 shutdown_connections 2024-09-10T10:04:03.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.598+0000 7f0459e00640 1 -- 172.21.15.110:0/2554743853 wait complete. 2024-09-10T10:04:03.623 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:03.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.754+0000 7f2ece000640 1 Processor -- start 2024-09-10T10:04:03.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.754+0000 7f2ece000640 1 -- start start 2024-09-10T10:04:03.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ece000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2ec80ae840 0x7f2ec806e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ece000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2ec81621e0 0x7f2ec8162610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ece000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec815ec50 con 0x7f2ec80aec20 2024-09-10T10:04:03.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ece000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec815eef0 con 0x7f2ec81621e0 2024-09-10T10:04:03.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ece000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec815f1d0 con 0x7f2ec80ae840 2024-09-10T10:04:03.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2eccc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2ec80ae840 0x7f2ec806e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ec7e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f2ec80aec20 0x7f2ec80b08a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:53994/0 (socket says 172.21.15.110:53994) 2024-09-10T10:04:03.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ec7e00640 1 -- 172.21.15.110:0/1047547178 learned_addr learned my addr 172.21.15.110:0/1047547178 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:03.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ecd600640 1 --2- 172.21.15.110:0/1047547178 >> v2:172.21.15.110:3300/0 conn(0x7f2ec81621e0 0x7f2ec8162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2eccc00640 1 -- 172.21.15.110:0/1047547178 >> v2:172.21.15.110:3300/0 conn(0x7f2ec81621e0 msgr2=0x7f2ec8162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:03.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2eccc00640 1 --2- 172.21.15.110:0/1047547178 >> v2:172.21.15.110:3300/0 conn(0x7f2ec81621e0 0x7f2ec8162610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2eccc00640 1 -- 172.21.15.110:0/1047547178 >> v1:172.21.15.145:6789/0 conn(0x7f2ec80aec20 legacy=0x7f2ec80b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2eccc00640 1 -- 172.21.15.110:0/1047547178 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec815f4b0 con 0x7f2ec80ae840 2024-09-10T10:04:03.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.758+0000 7f2ec7400640 1 -- 172.21.15.110:0/1047547178 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1880710527 0 0) 0x7f2ec815ec50 con 0x7f2ec80aec20 2024-09-10T10:04:03.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2eccc00640 1 --2- 172.21.15.110:0/1047547178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 0x7f2ec806e140 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8075a50 tx=0x7f2ebc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=452c507c2c000565 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:03.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ec7400640 1 -- 172.21.15.110:0/1047547178 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0b0070 con 0x7f2ec80ae840 2024-09-10T10:04:03.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ec7400640 1 -- 172.21.15.110:0/1047547178 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ebc0ab600 con 0x7f2ec80ae840 2024-09-10T10:04:03.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ec7400640 1 -- 172.21.15.110:0/1047547178 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0ab9d0 con 0x7f2ec80ae840 2024-09-10T10:04:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 -- 172.21.15.110:0/1047547178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 msgr2=0x7f2ec806e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 --2- 172.21.15.110:0/1047547178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 0x7f2ec806e140 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8075a50 tx=0x7f2ebc01c480 comp rx=0 tx=0).stop 2024-09-10T10:04:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 -- 172.21.15.110:0/1047547178 shutdown_connections 2024-09-10T10:04:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 --2- 172.21.15.110:0/1047547178 >> v2:172.21.15.110:3300/0 conn(0x7f2ec81621e0 0x7f2ec8162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 --2- 172.21.15.110:0/1047547178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 0x7f2ec806e140 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 -- 172.21.15.110:0/1047547178 >> 172.21.15.110:0/1047547178 conn(0x7f2ec806cd60 msgr2=0x7f2ec806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 -- 172.21.15.110:0/1047547178 shutdown_connections 2024-09-10T10:04:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 -- 172.21.15.110:0/1047547178 wait complete. 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 Processor -- start 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.762+0000 7f2ece000640 1 -- start start 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 0x7f2ec81530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ec80aec20 0x7f2ec80cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ec81621e0 0x7f2ec80d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80af070 con 0x7f2ec81621e0 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80b0fb0 con 0x7f2ec80ae840 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec80af1f0 con 0x7f2ec80aec20 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ec81621e0 0x7f2ec80d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ec81621e0 0x7f2ec80d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50560/0 (socket says 172.21.15.110:50560) 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 -- 172.21.15.110:0/1546950385 learned_addr learned my addr 172.21.15.110:0/1546950385 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2eccc00640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 0x7f2ec81530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ec7e00640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ec80aec20 0x7f2ec80cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 -- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ec80aec20 msgr2=0x7f2ec80cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ec80aec20 0x7f2ec80cfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 -- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 msgr2=0x7f2ec81530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 0x7f2ec81530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 -- 172.21.15.110:0/1546950385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec8157930 con 0x7f2ec81621e0 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ecd600640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ec81621e0 0x7f2ec80d00e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000efc0 tx=0x7f2ec000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ec4c00640 1 -- 172.21.15.110:0/1546950385 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0018020 con 0x7f2ec81621e0 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ec4c00640 1 -- 172.21.15.110:0/1546950385 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec0014480 con 0x7f2ec81621e0 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec8157370 con 0x7f2ec81621e0 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ec4c00640 1 -- 172.21.15.110:0/1546950385 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0014870 con 0x7f2ec81621e0 2024-09-10T10:04:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec8167ed0 con 0x7f2ec81621e0 2024-09-10T10:04:03.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.766+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec806e140 con 0x7f2ec81621e0 2024-09-10T10:04:03.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.770+0000 7f2ec4c00640 1 -- 172.21.15.110:0/1546950385 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f2ec001b020 con 0x7f2ec81621e0 2024-09-10T10:04:03.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.770+0000 7f2ec4c00640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ea8067cb0 0x7f2ea806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:03.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.770+0000 7f2ec4c00640 1 -- 172.21.15.110:0/1546950385 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f2ec0087640 con 0x7f2ec81621e0 2024-09-10T10:04:03.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.770+0000 7f2eccc00640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ea8067cb0 0x7f2ea806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:03.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.770+0000 7f2eccc00640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ea8067cb0 0x7f2ea806a170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0ad000 tx=0x7f2ebc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:03.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.774+0000 7f2ec4c00640 1 -- 172.21.15.110:0/1546950385 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2ec0022070 con 0x7f2ec81621e0 2024-09-10T10:04:03.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.962+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec80b4020 con 0x7f2ec81621e0 2024-09-10T10:04:03.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.962+0000 7f2ec4c00640 1 -- 172.21.15.110:0/1546950385 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5237 (secure 0 0 0) 0x7f2ec0010040 con 0x7f2ec81621e0 2024-09-10T10:04:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:03.968 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":78,"btime":"2024-09-10T10:04:01:743659+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1172125083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1172125083},{"type":"v1","addr":"172.21.15.145:6834","nonce":1172125083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6194,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3529040969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3529040969},{"type":"v1","addr":"172.21.15.145:6835","nonce":3529040969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2459795528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2459795528},{"type":"v1","addr":"172.21.15.110:6835","nonce":2459795528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:58.096051+0000","modified":"2024-09-10T10:04:01.113268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6834/1757537079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1757537079},{"type":"v1","addr":"172.21.15.110:6834","nonce":1757537079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":9}]} 2024-09-10T10:04:03.968 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 78 2024-09-10T10:04:03.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ea8067cb0 msgr2=0x7f2ea806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ea8067cb0 0x7f2ea806a170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0ad000 tx=0x7f2ebc008040 comp rx=0 tx=0).stop 2024-09-10T10:04:03.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ec81621e0 msgr2=0x7f2ec80d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:03.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ec81621e0 0x7f2ec80d00e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000efc0 tx=0x7f2ec000c490 comp rx=0 tx=0).stop 2024-09-10T10:04:03.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 shutdown_connections 2024-09-10T10:04:03.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2ea8067cb0 0x7f2ea806a170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ec81621e0 0x7f2ec80d00e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ec80aec20 0x7f2ec80cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 --2- 172.21.15.110:0/1546950385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ec80ae840 0x7f2ec81530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:03.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 >> 172.21.15.110:0/1546950385 conn(0x7f2ec806cd60 msgr2=0x7f2ec80b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:03.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 shutdown_connections 2024-09-10T10:04:03.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:03.966+0000 7f2ece000640 1 -- 172.21.15.110:0/1546950385 wait complete. 2024-09-10T10:04:04.391 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:03:58.096051+0000', 'modified': '2024-09-10T10:04:01.113268+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6834/1757537079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6832', 'nonce': 1757537079}, {'type': 'v1', 'addr': '172.21.15.110:6834', 'nonce': 1757537079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-10T10:04:04.391 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:04:04.391 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:04:04.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f176c200640 1 Processor -- start 2024-09-10T10:04:04.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f176c200640 1 -- start start 2024-09-10T10:04:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f176c200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640cfe50 0x7f17640b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f176c200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f176415f9f0 0x7f1764072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f176c200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17640d0230 con 0x7f17640ae840 2024-09-10T10:04:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f176c200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f17640b4710 con 0x7f176415f9f0 2024-09-10T10:04:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f176c200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1764073230 con 0x7f17640cfe50 2024-09-10T10:04:04.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1769800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640cfe50 0x7f17640b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1769800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640cfe50 0x7f17640b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:50504/0 (socket says 172.21.15.110:50504) 2024-09-10T10:04:04.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1769800640 1 -- 172.21.15.110:0/4129141975 learned_addr learned my addr 172.21.15.110:0/4129141975 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:04.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1768e00640 1 --2- 172.21.15.110:0/4129141975 >> v2:172.21.15.145:3300/0 conn(0x7f176415f9f0 0x7f1764072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1763e00640 1 -- 172.21.15.110:0/4129141975 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3925308212 0 0) 0x7f17640d0230 con 0x7f17640ae840 2024-09-10T10:04:04.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1763e00640 1 -- 172.21.15.110:0/4129141975 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f174c003530 con 0x7f17640ae840 2024-09-10T10:04:04.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1769800640 1 -- 172.21.15.110:0/4129141975 >> v2:172.21.15.145:3300/0 conn(0x7f176415f9f0 msgr2=0x7f1764072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1769800640 1 --2- 172.21.15.110:0/4129141975 >> v2:172.21.15.145:3300/0 conn(0x7f176415f9f0 0x7f1764072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1769800640 1 -- 172.21.15.110:0/4129141975 >> v1:172.21.15.145:6789/0 conn(0x7f17640ae840 legacy=0x7f17640710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.526+0000 7f1769800640 1 -- 172.21.15.110:0/4129141975 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1764072f50 con 0x7f17640cfe50 2024-09-10T10:04:04.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f1769800640 1 --2- 172.21.15.110:0/4129141975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640cfe50 0x7f17640b5420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1758009e50 tx=0x7f175801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2d9c1610100935 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f1763e00640 1 -- 172.21.15.110:0/4129141975 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580a5850 con 0x7f17640cfe50 2024-09-10T10:04:04.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f1763e00640 1 -- 172.21.15.110:0/4129141975 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17580a59f0 con 0x7f17640cfe50 2024-09-10T10:04:04.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f1763e00640 1 -- 172.21.15.110:0/4129141975 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580a5da0 con 0x7f17640cfe50 2024-09-10T10:04:04.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f176c200640 1 -- 172.21.15.110:0/4129141975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640cfe50 msgr2=0x7f17640b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f176c200640 1 --2- 172.21.15.110:0/4129141975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640cfe50 0x7f17640b5420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1758009e50 tx=0x7f175801c970 comp rx=0 tx=0).stop 2024-09-10T10:04:04.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f176c200640 1 -- 172.21.15.110:0/4129141975 shutdown_connections 2024-09-10T10:04:04.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f176c200640 1 --2- 172.21.15.110:0/4129141975 >> v2:172.21.15.145:3300/0 conn(0x7f176415f9f0 0x7f1764072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f176c200640 1 --2- 172.21.15.110:0/4129141975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640cfe50 0x7f17640b5420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f176c200640 1 -- 172.21.15.110:0/4129141975 >> 172.21.15.110:0/4129141975 conn(0x7f1764062d10 msgr2=0x7f176406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:04.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f176c200640 1 -- 172.21.15.110:0/4129141975 shutdown_connections 2024-09-10T10:04:04.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.530+0000 7f176c200640 1 -- 172.21.15.110:0/4129141975 wait complete. 2024-09-10T10:04:04.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 Processor -- start 2024-09-10T10:04:04.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 -- start start 2024-09-10T10:04:04.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640ae840 0x7f1764147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17640cfe50 0x7f17641473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176415f9f0 0x7f1764157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f17640b5960 con 0x7f17640cfe50 2024-09-10T10:04:04.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17640b4f00 con 0x7f176415f9f0 2024-09-10T10:04:04.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f17640af080 con 0x7f17640ae840 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17640cfe50 0x7f17641473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17640cfe50 0x7f17641473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50568/0 (socket says 172.21.15.110:50568) 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 -- 172.21.15.110:0/160698355 learned_addr learned my addr 172.21.15.110:0/160698355 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1769800640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640ae840 0x7f1764147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176a200640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176415f9f0 0x7f1764157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 -- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640ae840 msgr2=0x7f1764147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640ae840 0x7f1764147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 -- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176415f9f0 msgr2=0x7f1764157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176415f9f0 0x7f1764157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 -- 172.21.15.110:0/160698355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17641482d0 con 0x7f17640cfe50 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176a200640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176415f9f0 0x7f1764157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:04.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1769800640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640ae840 0x7f1764147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:04.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1768e00640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17640cfe50 0x7f17641473b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f175000e840 tx=0x7f175000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1761600640 1 -- 172.21.15.110:0/160698355 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750017070 con 0x7f17640cfe50 2024-09-10T10:04:04.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1761600640 1 -- 172.21.15.110:0/160698355 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175000ee90 con 0x7f17640cfe50 2024-09-10T10:04:04.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f1761600640 1 -- 172.21.15.110:0/160698355 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175000cb60 con 0x7f17640cfe50 2024-09-10T10:04:04.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 -- 172.21.15.110:0/160698355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17641485b0 con 0x7f17640cfe50 2024-09-10T10:04:04.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.534+0000 7f176c200640 1 -- 172.21.15.110:0/160698355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1764148b70 con 0x7f17640cfe50 2024-09-10T10:04:04.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.538+0000 7f1761600640 1 -- 172.21.15.110:0/160698355 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f175001a070 con 0x7f17640cfe50 2024-09-10T10:04:04.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.538+0000 7f1761600640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1744067cb0 0x7f174406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.538+0000 7f1761600640 1 -- 172.21.15.110:0/160698355 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f17500865d0 con 0x7f17640cfe50 2024-09-10T10:04:04.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.538+0000 7f1769800640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1744067cb0 0x7f174406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.538+0000 7f176f538000 1 Processor -- start 2024-09-10T10:04:04.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.538+0000 7f176f538000 1 -- start start 2024-09-10T10:04:04.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.538+0000 7f1769800640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1744067cb0 0x7f174406a170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f17580ae000 tx=0x7f17580a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 0x55f483f73d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f74290 0x55f483f74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f483e5bb60 con 0x55f483fbdb20 2024-09-10T10:04:04.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f483dc2c70 con 0x55f483fbe440 2024-09-10T10:04:04.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55f483d68ed0 con 0x55f483f74290 2024-09-10T10:04:04.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50572/0 (socket says 172.21.15.110:50572) 2024-09-10T10:04:04.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 -- 172.21.15.110:0/974991300 learned_addr learned my addr 172.21.15.110:0/974991300 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:04.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1768e00640 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 0x55f483f73d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176a200640 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f74290 0x55f483f74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 -- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f74290 msgr2=0x55f483f74670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f74290 0x55f483f74670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 -- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 msgr2=0x55f483f73d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 0x55f483f73d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 -- 172.21.15.110:0/974991300 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f483d7ae50 con 0x55f483fbdb20 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176a200640 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f74290 0x55f483f74670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1769800640 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbdf00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f1758009c90 tx=0x7f1758004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c0d564f6f0b629 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1756000640 1 -- 172.21.15.110:0/974991300 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580c6070 con 0x55f483fbdb20 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1756000640 1 -- 172.21.15.110:0/974991300 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17580b6420 con 0x55f483fbdb20 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f1756000640 1 -- 172.21.15.110:0/974991300 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580b67f0 con 0x55f483fbdb20 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- 172.21.15.110:0/974991300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 msgr2=0x55f483fbdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbdf00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f1758009c90 tx=0x7f1758004ac0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- 172.21.15.110:0/974991300 shutdown_connections 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f74290 0x55f483f74670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 0x55f483f73d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 --2- 172.21.15.110:0/974991300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbdf00 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- 172.21.15.110:0/974991300 >> 172.21.15.110:0/974991300 conn(0x55f483f7a600 msgr2=0x55f483f72630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:04.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- 172.21.15.110:0/974991300 shutdown_connections 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- 172.21.15.110:0/974991300 wait complete. 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 -- start start 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.542+0000 7f176f538000 1 Processor -- start 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 0x55f483f78e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 0x55f483f7ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483f7f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f483e24b00 con 0x55f483f79390 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f483d9d360 con 0x55f483fbdb20 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55f483d9a2d0 con 0x55f483fbe440 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 10 client.0 _ll_register_callbacks cb 0x55f483f5f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483f7f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483f7f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50580/0 (socket says 172.21.15.110:50580) 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 -- 172.21.15.110:0/3330715167 learned_addr learned my addr 172.21.15.110:0/3330715167 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f1769800640 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 0x55f483f78e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f1768e00640 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 0x55f483f7ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 -- 172.21.15.110:0/3330715167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 msgr2=0x55f483f7ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 0x55f483f7ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 -- 172.21.15.110:0/3330715167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 msgr2=0x55f483f78e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 0x55f483f78e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 -- 172.21.15.110:0/3330715167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f483d69120 con 0x55f483f79390 2024-09-10T10:04:04.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176a200640 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483f7f230 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f175c007a80 tx=0x7f175c0012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a7e8bb78a2982f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 1 -- 172.21.15.110:0/3330715167 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c001d20 con 0x55f483f79390 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 1 -- 172.21.15.110:0/3330715167 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f175c014400 con 0x55f483f79390 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 -- 172.21.15.110:0/3330715167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17580b6420 con 0x55f483f79390 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 1 -- 172.21.15.110:0/3330715167 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f175c021050 con 0x55f483f79390 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 10 client.7431 fetch_fsmap learned FSMap version 78 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 -- 172.21.15.110:0/3330715167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55f483d62450 con 0x55f483f79390 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 1 -- 172.21.15.110:0/3330715167 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17580b6420 con 0x55f483f79390 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 1 -- 172.21.15.110:0/3330715167 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 78) ==== 3502+0+0 (secure 0 0 0) 0x7f175c024020 con 0x55f483f79390 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 10 client.7431 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T10:04:04.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 10 client.7431 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:04:04.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 20 client.7431 populate_metadata read hostname 'smithi110' 2024-09-10T10:04:04.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] conn(0x55f483fcc280 0x55f483fce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 4 client.7431 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:04.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 -- 172.21.15.110:0/3330715167 --> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f483d5cc00 con 0x55f483fcc280 2024-09-10T10:04:04.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f1769800640 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] conn(0x55f483fcc280 0x55f483fce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 10 client.7431 ms_handle_connect on v2:172.21.15.110:6832/1757537079 2024-09-10T10:04:04.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f1769800640 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] conn(0x55f483fcc280 0x55f483fce670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86448407df89e10 server_cookie=d844d11a3ae86f7d in_seq=0 out_seq=0 2024-09-10T10:04:04.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 1 -- 172.21.15.110:0/3330715167 <== mds.0 v2:172.21.15.110:6832/1757537079 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f483d5cc00 con 0x55f483fcc280 2024-09-10T10:04:04.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f174aa00640 10 client.7431 handle_command_reply: tid=0 2024-09-10T10:04:04.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.546+0000 7f176f538000 1 client.7431 shutdown 2024-09-10T10:04:04.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 -- 172.21.15.110:0/3330715167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 msgr2=0x55f483f7f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483f7f230 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f175c007a80 tx=0x7f175c0012e0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 -- 172.21.15.110:0/3330715167 shutdown_connections 2024-09-10T10:04:04.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] conn(0x55f483fcc280 0x55f483fce670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483f7f230 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 0x55f483f7ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 --2- 172.21.15.110:0/3330715167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 0x55f483f78e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f174aa00640 0 client.7431 ms_handle_reset on v2:172.21.15.110:6832/1757537079 2024-09-10T10:04:04.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 -- 172.21.15.110:0/3330715167 >> 172.21.15.110:0/3330715167 conn(0x55f483f74290 msgr2=0x55f483f72d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:04.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 -- 172.21.15.110:0/3330715167 shutdown_connections 2024-09-10T10:04:04.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 1 -- 172.21.15.110:0/3330715167 wait complete. 2024-09-10T10:04:04.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.550+0000 7f176f538000 20 client.7431 trim_cache size 0 max 16384 2024-09-10T10:04:04.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f176f538000 1 Processor -- start 2024-09-10T10:04:04.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f176f538000 1 -- start start 2024-09-10T10:04:04.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f176f538000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f79390 0x55f483fbccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f176f538000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f176f538000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 0x55f483fce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f176f538000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1764151840 con 0x55f483fbdb20 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f176f538000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1764150ba0 con 0x55f483fbe440 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f176f538000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f17641511f0 con 0x55f483f79390 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f1768e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f1768e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50582/0 (socket says 172.21.15.110:50582) 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.566+0000 7f1768e00640 1 -- 172.21.15.110:0/4034042916 learned_addr learned my addr 172.21.15.110:0/4034042916 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f79390 0x55f483fbccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176a200640 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 0x55f483fce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1768e00640 1 -- 172.21.15.110:0/4034042916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f79390 msgr2=0x55f483fbccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1768e00640 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f79390 0x55f483fbccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1768e00640 1 -- 172.21.15.110:0/4034042916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 msgr2=0x55f483fce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1768e00640 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 0x55f483fce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1768e00640 1 -- 172.21.15.110:0/4034042916 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f483d49fd0 con 0x55f483fbdb20 2024-09-10T10:04:04.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1768e00640 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbd200 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f1750006aa0 tx=0x7f1750086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322523e9001d5316 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1755600640 1 -- 172.21.15.110:0/4034042916 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750021040 con 0x55f483fbdb20 2024-09-10T10:04:04.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1755600640 1 -- 172.21.15.110:0/4034042916 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1750082070 con 0x55f483fbdb20 2024-09-10T10:04:04.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1755600640 1 -- 172.21.15.110:0/4034042916 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175006e1f0 con 0x55f483fbdb20 2024-09-10T10:04:04.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- 172.21.15.110:0/4034042916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 msgr2=0x55f483fbd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbd200 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f1750006aa0 tx=0x7f1750086040 comp rx=0 tx=0).stop 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- 172.21.15.110:0/4034042916 shutdown_connections 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbe440 0x55f483fce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483fbdb20 0x55f483fbd200 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 --2- 172.21.15.110:0/4034042916 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483f79390 0x55f483fbccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- 172.21.15.110:0/4034042916 >> 172.21.15.110:0/4034042916 conn(0x55f483f74290 msgr2=0x55f483f78650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- 172.21.15.110:0/4034042916 shutdown_connections 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- 172.21.15.110:0/4034042916 wait complete. 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- start start 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 Processor -- start 2024-09-10T10:04:04.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483fd1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 0x55f483fec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 0x55f483fecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f483dc1f80 con 0x55f483f79390 2024-09-10T10:04:04.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f483d7aca0 con 0x55f483fbdb20 2024-09-10T10:04:04.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x55f483dc72a0 con 0x55f483fbe440 2024-09-10T10:04:04.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176f538000 10 client.0 _ll_register_callbacks cb 0x7f175c007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:04.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483fd1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483fd1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50590/0 (socket says 172.21.15.110:50590) 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 -- 172.21.15.110:0/2106923731 learned_addr learned my addr 172.21.15.110:0/2106923731 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1768e00640 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 0x55f483fec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f176a200640 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 0x55f483fecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 -- 172.21.15.110:0/2106923731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 msgr2=0x55f483fecea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 0x55f483fecea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 -- 172.21.15.110:0/2106923731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 msgr2=0x55f483fec960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 0x55f483fec960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.570+0000 7f1769800640 1 -- 172.21.15.110:0/2106923731 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f483e45750 con 0x55f483f79390 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f1769800640 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483fd1590 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f175801c9b0 tx=0x7f175801b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbc538bd7342dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 1 -- 172.21.15.110:0/2106923731 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580c6040 con 0x55f483f79390 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 1 -- 172.21.15.110:0/2106923731 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f17580d7af0 con 0x55f483f79390 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 -- 172.21.15.110:0/2106923731 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1750082070 con 0x55f483f79390 2024-09-10T10:04:04.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 1 -- 172.21.15.110:0/2106923731 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f17580d7cf0 con 0x55f483f79390 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 10 client.7437 fetch_fsmap learned FSMap version 78 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 -- 172.21.15.110:0/2106923731 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55f483d50a30 con 0x55f483f79390 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 1 -- 172.21.15.110:0/2106923731 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1750082070 con 0x55f483f79390 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 1 -- 172.21.15.110:0/2106923731 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 78) ==== 3502+0+0 (secure 0 0 0) 0x7f1758008030 con 0x55f483f79390 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 10 client.7437 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 10 client.7437 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 20 client.7437 populate_metadata read hostname 'smithi110' 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] conn(0x55f483fed3e0 0x55f484030de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 4 client.7437 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 -- 172.21.15.110:0/2106923731 --> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] -- command(tid 0: {"prefix": "status"}) -- 0x55f483dd03a0 con 0x55f483fed3e0 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f1768e00640 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] conn(0x55f483fed3e0 0x55f484030de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 10 client.7437 ms_handle_connect on v2:172.21.15.110:6832/1757537079 2024-09-10T10:04:04.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f1768e00640 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] conn(0x55f483fed3e0 0x55f484030de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8810221a161f80a0 server_cookie=7fc4e0f6ae0937b4 in_seq=0 out_seq=0 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 1 -- 172.21.15.110:0/2106923731 <== mds.0 v2:172.21.15.110:6832/1757537079 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f483dd03a0 con 0x55f483fed3e0 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 10 client.7437 handle_command_reply: tid=0 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 client.7437 shutdown 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 -- 172.21.15.110:0/2106923731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 msgr2=0x55f483fd1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483fd1590 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f175801c9b0 tx=0x7f175801b1b0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 -- 172.21.15.110:0/2106923731 shutdown_connections 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.110:6832/1757537079,v1:172.21.15.110:6834/1757537079] conn(0x55f483fed3e0 0x55f484030de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x55f483fbe440 0x55f483fecea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f483fbdb20 0x55f483fec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f174aa00640 0 client.7437 ms_handle_reset on v2:172.21.15.110:6832/1757537079 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 --2- 172.21.15.110:0/2106923731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f483f79390 0x55f483fd1590 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 -- 172.21.15.110:0/2106923731 >> 172.21.15.110:0/2106923731 conn(0x55f483f74290 msgr2=0x55f483fbbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 -- 172.21.15.110:0/2106923731 shutdown_connections 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 1 -- 172.21.15.110:0/2106923731 wait complete. 2024-09-10T10:04:04.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176f538000 20 client.7437 trim_cache size 0 max 16384 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 7311, 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 5.8574494420000001, 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 78, 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 117, 2024-09-10T10:04:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 117, 2024-09-10T10:04:04.586 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 10.734706155 2024-09-10T10:04:04.586 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:04:04.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 -- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1744067cb0 msgr2=0x7f174406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1744067cb0 0x7f174406a170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f17580ae000 tx=0x7f17580a7070 comp rx=0 tx=0).stop 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 -- 172.21.15.110:0/160698355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17640cfe50 msgr2=0x7f17641473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17640cfe50 0x7f17641473b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f175000e840 tx=0x7f175000ec40 comp rx=0 tx=0).stop 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 -- 172.21.15.110:0/160698355 shutdown_connections 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1744067cb0 0x7f174406a170 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f176415f9f0 0x7f1764157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17640cfe50 0x7f17641473b0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 --2- 172.21.15.110:0/160698355 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f17640ae840 0x7f1764147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 -- 172.21.15.110:0/160698355 >> 172.21.15.110:0/160698355 conn(0x7f1764062d10 msgr2=0x7f1764152e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 -- 172.21.15.110:0/160698355 shutdown_connections 2024-09-10T10:04:04.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.574+0000 7f176c200640 1 -- 172.21.15.110:0/160698355 wait complete. 2024-09-10T10:04:04.599 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:04:04.701 INFO:tasks.ceph.mds.a.smithi110.stderr:2024-09-10T10:04:04.698+0000 7f1cf2200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:04:04.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.730+0000 7f3dc2600640 1 Processor -- start 2024-09-10T10:04:04.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.730+0000 7f3dc2600640 1 -- start start 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.730+0000 7f3dc2600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc0cd510 0x7f3dbc0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.730+0000 7f3dc2600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dbc072150 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.730+0000 7f3dc2600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dbc06d2d0 con 0x7f3dbc06e090 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.730+0000 7f3dc2600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3dbc0cdeb0 con 0x7f3dbc0cd510 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dbb400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3dbc0cfe30 0x7f3dbc071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43816/0 (socket says 172.21.15.110:43816) 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dbb400640 1 -- 172.21.15.110:0/2149419699 learned_addr learned my addr 172.21.15.110:0/2149419699 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dbaa00640 1 --2- 172.21.15.110:0/2149419699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc0cd510 0x7f3dbc0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dba000640 1 -- 172.21.15.110:0/2149419699 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407154779 0 0) 0x7f3dbc072150 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dba000640 1 -- 172.21.15.110:0/2149419699 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3da0003530 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dba000640 1 -- 172.21.15.110:0/2149419699 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4141307301 0 0) 0x7f3dbc06d2d0 con 0x7f3dbc06e090 2024-09-10T10:04:04.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dba000640 1 -- 172.21.15.110:0/2149419699 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dbc072150 con 0x7f3dbc06e090 2024-09-10T10:04:04.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dbaa00640 1 -- 172.21.15.110:0/2149419699 >> v1:172.21.15.145:6789/0 conn(0x7f3dbc06e090 legacy=0x7f3dbc06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dbaa00640 1 -- 172.21.15.110:0/2149419699 >> v1:172.21.15.110:6789/0 conn(0x7f3dbc0cfe30 legacy=0x7f3dbc071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dbaa00640 1 -- 172.21.15.110:0/2149419699 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dbc06d5b0 con 0x7f3dbc0cd510 2024-09-10T10:04:04.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.734+0000 7f3dbaa00640 1 --2- 172.21.15.110:0/2149419699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc0cd510 0x7f3dbc0cd970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097f0 tx=0x7f3da801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14cbfca06569172 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dba000640 1 -- 172.21.15.110:0/2149419699 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80b2070 con 0x7f3dbc0cd510 2024-09-10T10:04:04.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dba000640 1 -- 172.21.15.110:0/2149419699 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da80a9070 con 0x7f3dbc0cd510 2024-09-10T10:04:04.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dba000640 1 -- 172.21.15.110:0/2149419699 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80ad860 con 0x7f3dbc0cd510 2024-09-10T10:04:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- 172.21.15.110:0/2149419699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc0cd510 msgr2=0x7f3dbc0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 --2- 172.21.15.110:0/2149419699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc0cd510 0x7f3dbc0cd970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097f0 tx=0x7f3da801c450 comp rx=0 tx=0).stop 2024-09-10T10:04:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- 172.21.15.110:0/2149419699 shutdown_connections 2024-09-10T10:04:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 --2- 172.21.15.110:0/2149419699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc0cd510 0x7f3dbc0cd970 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- 172.21.15.110:0/2149419699 >> 172.21.15.110:0/2149419699 conn(0x7f3dbc06bc50 msgr2=0x7f3dbc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- 172.21.15.110:0/2149419699 shutdown_connections 2024-09-10T10:04:04.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- 172.21.15.110:0/2149419699 wait complete. 2024-09-10T10:04:04.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 Processor -- start 2024-09-10T10:04:04.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- start start 2024-09-10T10:04:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc06e090 0x7f3dbc155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dbc0cd510 0x7f3dbc151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dbc0cfe30 0x7f3dbc1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc15d9a0 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc15d640 con 0x7f3dbc0cd510 2024-09-10T10:04:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dc2600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3dbc16a190 con 0x7f3dbc06e090 2024-09-10T10:04:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.738+0000 7f3dbbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dbc0cfe30 0x7f3dbc1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dbc0cfe30 0x7f3dbc1526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50594/0 (socket says 172.21.15.110:50594) 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbbe00640 1 -- 172.21.15.110:0/3616439126 learned_addr learned my addr 172.21.15.110:0/3616439126 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbaa00640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dbc0cd510 0x7f3dbc151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbb400640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc06e090 0x7f3dbc155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbbe00640 1 -- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc06e090 msgr2=0x7f3dbc155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbbe00640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc06e090 0x7f3dbc155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbbe00640 1 -- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dbc0cd510 msgr2=0x7f3dbc151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbbe00640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dbc0cd510 0x7f3dbc151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbbe00640 1 -- 172.21.15.110:0/3616439126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dbc0b1e20 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dbbe00640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dbc0cfe30 0x7f3dbc1526f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c3c0 tx=0x7f3db400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3db3400640 1 -- 172.21.15.110:0/3616439126 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4017070 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3db3400640 1 -- 172.21.15.110:0/3616439126 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db40052d0 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3db3400640 1 -- 172.21.15.110:0/3616439126 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db40054c0 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dbc0b0770 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dbc0b0d30 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dbc0cd970 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3db3400640 1 -- 172.21.15.110:0/3616439126 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f3db401a070 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3db3400640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d9c067cb0 0x7f3d9c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:04.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.742+0000 7f3db3400640 1 -- 172.21.15.110:0/3616439126 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f3db40856b0 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.746+0000 7f3dbb400640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d9c067cb0 0x7f3d9c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:04.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.746+0000 7f3dbb400640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d9c067cb0 0x7f3d9c06a170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3dac001570 tx=0x7f3dac001a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.746+0000 7f3db3400640 1 -- 172.21.15.110:0/3616439126 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3db40555a0 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.938+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3dbc15fa30 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.938+0000 7f3db3400640 1 -- 172.21.15.110:0/3616439126 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) ==== 53+0+73 (secure 0 0 0) 0x7f3db40598a0 con 0x7f3dbc0cfe30 2024-09-10T10:04:04.944 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:04:04.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d9c067cb0 msgr2=0x7f3d9c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d9c067cb0 0x7f3d9c06a170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3dac001570 tx=0x7f3dac001a00 comp rx=0 tx=0).stop 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dbc0cfe30 msgr2=0x7f3dbc1526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dbc0cfe30 0x7f3dbc1526f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c3c0 tx=0x7f3db400c880 comp rx=0 tx=0).stop 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 shutdown_connections 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d9c067cb0 0x7f3d9c06a170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3dbc0cfe30 0x7f3dbc1526f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dbc0cd510 0x7f3dbc151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 --2- 172.21.15.110:0/3616439126 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3dbc06e090 0x7f3dbc155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 >> 172.21.15.110:0/3616439126 conn(0x7f3dbc06bc50 msgr2=0x7f3dbc06e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 shutdown_connections 2024-09-10T10:04:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:04.942+0000 7f3dc2600640 1 -- 172.21.15.110:0/3616439126 wait complete. 2024-09-10T10:04:04.964 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:04:04.964 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:04:04.965 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:04:04.965 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:04:04.965 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:04.965 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:04:04.965 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:04:04.965 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:04:04.965 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:04:04.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:04.966 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:04.975 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42130sec preferred_lft 42130sec 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:04.976 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:04:04.977 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:04.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:04.978 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:04:05.038 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:04:05.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:05.041 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:04:05.096 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:04:05.097 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:04:05.097 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:04:05.097 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:04:05.097 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:04:05.097 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:04:05.098 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:04:05.098 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:05.098 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:05.098 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:05.098 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:04:05.098 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:04:05.098 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:04:05.098 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:04:05.099 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:04:05.099 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:04:05.099 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:04:05.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:04:05.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:05.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:05.233 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:04:05.243 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:04:05.244 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:04:05.244 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:04:05.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:05.244 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:04:05.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:04:05.304 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:05.352 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:04:05.352 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:04:05.352 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:05.408 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:04:05.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:05.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:05.495 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:05.495 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:04:05.509 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:05.509 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:04:05.509 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:04:05.509 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:04:05.509 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:04:05.514 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:04:05.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:05.514 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:05.575 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:04:05.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:04:05.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:05.683 INFO:teuthology.orchestra.run.smithi110.stdout:client6967 2024-09-10T10:04:05.683 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:05.683 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:05.692 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:04:05.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.866+0000 7fe7dee00640 1 Processor -- start 2024-09-10T10:04:05.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.866+0000 7fe7dee00640 1 -- start start 2024-09-10T10:04:05.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7dee00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7d80cfe30 0x7fe7d8071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:05.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7dee00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7d80cd510 0x7fe7d80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:05.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7dee00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d806d2d0 con 0x7fe7d806e090 2024-09-10T10:04:05.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7dee00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d815d9a0 con 0x7fe7d80cd510 2024-09-10T10:04:05.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7dee00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d806d5b0 con 0x7fe7d80cfe30 2024-09-10T10:04:05.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7d7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7d80cd510 0x7fe7d80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:05.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7d7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7d80cd510 0x7fe7d80cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50596/0 (socket says 172.21.15.110:50596) 2024-09-10T10:04:05.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7d7400640 1 -- 172.21.15.110:0/3878562617 learned_addr learned my addr 172.21.15.110:0/3878562617 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:05.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7d7e00640 1 --2- 172.21.15.110:0/3878562617 >> v2:172.21.15.145:3300/0 conn(0x7fe7d80cfe30 0x7fe7d8071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:05.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7d7400640 1 -- 172.21.15.110:0/3878562617 >> v2:172.21.15.145:3300/0 conn(0x7fe7d80cfe30 msgr2=0x7fe7d8071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:05.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7d7400640 1 --2- 172.21.15.110:0/3878562617 >> v2:172.21.15.145:3300/0 conn(0x7fe7d80cfe30 0x7fe7d8071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:05.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7d7400640 1 -- 172.21.15.110:0/3878562617 >> v1:172.21.15.145:6789/0 conn(0x7fe7d806e090 legacy=0x7fe7d806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:05.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.870+0000 7fe7d7400640 1 -- 172.21.15.110:0/3878562617 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d806d890 con 0x7fe7d80cd510 2024-09-10T10:04:05.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7d7400640 1 --2- 172.21.15.110:0/3878562617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cd510 0x7fe7d80cd9a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ce120 tx=0x7fe7c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ec8608f6f923e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:05.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7d6a00640 1 -- 172.21.15.110:0/3878562617 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40b2070 con 0x7fe7d80cd510 2024-09-10T10:04:05.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7d6a00640 1 -- 172.21.15.110:0/3878562617 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c40a9070 con 0x7fe7d80cd510 2024-09-10T10:04:05.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7d6a00640 1 -- 172.21.15.110:0/3878562617 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40ad880 con 0x7fe7d80cd510 2024-09-10T10:04:05.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 -- 172.21.15.110:0/3878562617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cd510 msgr2=0x7fe7d80cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:05.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3878562617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cd510 0x7fe7d80cd9a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ce120 tx=0x7fe7c401c450 comp rx=0 tx=0).stop 2024-09-10T10:04:05.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 -- 172.21.15.110:0/3878562617 shutdown_connections 2024-09-10T10:04:05.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3878562617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cd510 0x7fe7d80cd9a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:05.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3878562617 >> v2:172.21.15.145:3300/0 conn(0x7fe7d80cfe30 0x7fe7d8071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:05.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 -- 172.21.15.110:0/3878562617 >> 172.21.15.110:0/3878562617 conn(0x7fe7d806bc50 msgr2=0x7fe7d806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:05.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 -- 172.21.15.110:0/3878562617 shutdown_connections 2024-09-10T10:04:05.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 -- 172.21.15.110:0/3878562617 wait complete. 2024-09-10T10:04:05.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 Processor -- start 2024-09-10T10:04:05.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.874+0000 7fe7dee00640 1 -- start start 2024-09-10T10:04:05.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7d806e090 0x7fe7d80b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:05.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7d80cd510 0x7fe7d8147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:05.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cfe30 0x7fe7d814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:05.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8072190 con 0x7fe7d80cfe30 2024-09-10T10:04:05.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d815d6c0 con 0x7fe7d806e090 2024-09-10T10:04:05.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d8169180 con 0x7fe7d80cd510 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cfe30 0x7fe7d814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cfe30 0x7fe7d814c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50610/0 (socket says 172.21.15.110:50610) 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 -- 172.21.15.110:0/3398734331 learned_addr learned my addr 172.21.15.110:0/3398734331 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7d7e00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7d806e090 0x7fe7d80b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7d7400640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7d80cd510 0x7fe7d8147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 -- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7d80cd510 msgr2=0x7fe7d8147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7d80cd510 0x7fe7d8147440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 -- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7d806e090 msgr2=0x7fe7d80b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7d806e090 0x7fe7d80b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 -- 172.21.15.110:0/3398734331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d80b3670 con 0x7fe7d80cfe30 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dce00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cfe30 0x7fe7d814c5e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000b410 tx=0x7fe7d000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:05.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7cfe00640 1 -- 172.21.15.110:0/3398734331 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0015020 con 0x7fe7d80cfe30 2024-09-10T10:04:05.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7cfe00640 1 -- 172.21.15.110:0/3398734331 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d0011870 con 0x7fe7d80cfe30 2024-09-10T10:04:05.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d80b3950 con 0x7fe7d80cfe30 2024-09-10T10:04:05.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7cfe00640 1 -- 172.21.15.110:0/3398734331 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0011c60 con 0x7fe7d80cfe30 2024-09-10T10:04:05.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d80b3f10 con 0x7fe7d80cfe30 2024-09-10T10:04:05.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d80b43e0 con 0x7fe7d80cfe30 2024-09-10T10:04:05.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.878+0000 7fe7cfe00640 1 -- 172.21.15.110:0/3398734331 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7fe7d0018020 con 0x7fe7d80cfe30 2024-09-10T10:04:05.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.882+0000 7fe7cfe00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7b8067ce0 0x7fe7b806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:05.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.882+0000 7fe7cfe00640 1 -- 172.21.15.110:0/3398734331 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7fe7d0085e00 con 0x7fe7d80cfe30 2024-09-10T10:04:05.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.882+0000 7fe7d7e00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7b8067ce0 0x7fe7b806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:05.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.882+0000 7fe7d7e00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7b8067ce0 0x7fe7b806a1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009fd0 tx=0x7fe7c8009800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:05.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:05.886+0000 7fe7cfe00640 1 -- 172.21.15.110:0/3398734331 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe7d001f030 con 0x7fe7d80cfe30 2024-09-10T10:04:06.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.074+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe7d815f7c0 con 0x7fe7d80cfe30 2024-09-10T10:04:06.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.074+0000 7fe7cfe00640 1 -- 172.21.15.110:0/3398734331 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) ==== 53+0+73 (secure 0 0 0) 0x7fe7d0055cf0 con 0x7fe7d80cfe30 2024-09-10T10:04:06.080 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:04:06.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7b8067ce0 msgr2=0x7fe7b806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:06.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7b8067ce0 0x7fe7b806a1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009fd0 tx=0x7fe7c8009800 comp rx=0 tx=0).stop 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cfe30 msgr2=0x7fe7d814c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cfe30 0x7fe7d814c5e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000b410 tx=0x7fe7d000b8d0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 shutdown_connections 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7b8067ce0 0x7fe7b806a1a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7d80cfe30 0x7fe7d814c5e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7d80cd510 0x7fe7d8147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 --2- 172.21.15.110:0/3398734331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7d806e090 0x7fe7d80b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 >> 172.21.15.110:0/3398734331 conn(0x7fe7d806bc50 msgr2=0x7fe7d815a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 shutdown_connections 2024-09-10T10:04:06.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.078+0000 7fe7dee00640 1 -- 172.21.15.110:0/3398734331 wait complete. 2024-09-10T10:04:06.102 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:04:06.102 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:04:06.102 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:04:06.102 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:04:06.102 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:06.102 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:04:06.102 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:04:06.103 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:04:06.103 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:04:06.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:06.103 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:04:06.113 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:04:06.113 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:04:06.113 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:04:06.113 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:06.113 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:04:06.113 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:06.113 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42125sec preferred_lft 42125sec 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:04:06.114 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:04:06.115 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:06.116 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:04:06.116 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:04:06.116 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:04:06.116 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:06.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:06.117 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:04:06.180 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:04:06.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:06.183 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:04:06.243 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:04:06.243 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:04:06.243 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:04:06.243 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:04:06.243 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:04:06.243 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:04:06.243 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:04:06.243 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:06.244 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:06.244 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:06.244 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:04:06.244 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:04:06.245 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:04:06.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:04:06.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:04:06.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:04:06.245 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:04:06.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:04:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:04:06.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:06.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:06.387 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:04:06.396 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:04:06.396 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:04:06.397 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:04:06.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:06.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:04:06.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:04:06.457 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:06.509 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:04:06.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:04:06.509 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:06.561 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:04:06.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:06.561 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:06.649 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:06.649 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:04:06.663 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:06.663 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:04:06.663 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:04:06.663 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:04:06.663 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:04:06.667 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:04:06.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:06.668 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:06.687 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:04:06.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:04:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:04:06.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:06.789 INFO:teuthology.orchestra.run.smithi145.stdout:client6976 2024-09-10T10:04:06.790 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:06.790 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:06.798 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:04:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.934+0000 7f8ef1200640 1 Processor -- start 2024-09-10T10:04:06.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.934+0000 7f8ef1200640 1 -- start start 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8ef1200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec073200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8ef1200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8eec073740 0x7f8eec15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8ef1200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eec0706a0 con 0x7f8eec15c820 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8ef1200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8eec073b70 con 0x7f8eec073740 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8ef1200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8eec0cffc0 con 0x7f8eec0d0410 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8eebe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8eec073740 0x7f8eec15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8eebe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8eec073740 0x7f8eec15cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50620/0 (socket says 172.21.15.110:50620) 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8eebe00640 1 -- 172.21.15.110:0/3690669336 learned_addr learned my addr 172.21.15.110:0/3690669336 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8eeaa00640 1 --2- 172.21.15.110:0/3690669336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8eebe00640 1 -- 172.21.15.110:0/3690669336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 msgr2=0x7f8eec073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8eebe00640 1 --2- 172.21.15.110:0/3690669336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec073200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8eebe00640 1 -- 172.21.15.110:0/3690669336 >> v1:172.21.15.110:6789/0 conn(0x7f8eec15c820 legacy=0x7f8eec0b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:06.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.938+0000 7f8eebe00640 1 -- 172.21.15.110:0/3690669336 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec0b5f70 con 0x7f8eec073740 2024-09-10T10:04:06.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8eebe00640 1 --2- 172.21.15.110:0/3690669336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec15cc00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009870 tx=0x7f8ee401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6d41c1163ccd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:06.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8eea000640 1 -- 172.21.15.110:0/3690669336 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40aa600 con 0x7f8eec073740 2024-09-10T10:04:06.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8eea000640 1 -- 172.21.15.110:0/3690669336 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee40af070 con 0x7f8eec073740 2024-09-10T10:04:06.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8eea000640 1 -- 172.21.15.110:0/3690669336 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40aa9c0 con 0x7f8eec073740 2024-09-10T10:04:06.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- 172.21.15.110:0/3690669336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 msgr2=0x7f8eec15cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:06.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 --2- 172.21.15.110:0/3690669336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec15cc00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009870 tx=0x7f8ee401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- 172.21.15.110:0/3690669336 shutdown_connections 2024-09-10T10:04:06.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 --2- 172.21.15.110:0/3690669336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec15cc00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 --2- 172.21.15.110:0/3690669336 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec073200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- 172.21.15.110:0/3690669336 >> 172.21.15.110:0/3690669336 conn(0x7f8eec0d2530 msgr2=0x7f8eec0d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:06.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- 172.21.15.110:0/3690669336 shutdown_connections 2024-09-10T10:04:06.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- 172.21.15.110:0/3690669336 wait complete. 2024-09-10T10:04:06.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 Processor -- start 2024-09-10T10:04:06.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- start start 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec06e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8eec15c820 0x7f8eec1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec0d07f0 con 0x7f8eec073740 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec073f50 con 0x7f8eec15c820 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8ef1200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8eec0cfbe0 con 0x7f8eec0d0410 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.942+0000 7f8eeb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec06e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50622/0 (socket says 172.21.15.110:50622) 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeb400640 1 -- 172.21.15.110:0/1023905985 learned_addr learned my addr 172.21.15.110:0/1023905985 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:06.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeaa00640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eebe00640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8eec15c820 0x7f8eec1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeb400640 1 -- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 msgr2=0x7f8eec143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeb400640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec143f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeb400640 1 -- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8eec15c820 msgr2=0x7f8eec1547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeb400640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8eec15c820 0x7f8eec1547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeb400640 1 -- 172.21.15.110:0/1023905985 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec154f20 con 0x7f8eec073740 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeaa00640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eebe00640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8eec15c820 0x7f8eec1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8eeb400640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec06e950 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c760 tx=0x7f8edc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ee3400640 1 -- 172.21.15.110:0/1023905985 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc016020 con 0x7f8eec073740 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ee3400640 1 -- 172.21.15.110:0/1023905985 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8edc006d60 con 0x7f8eec073740 2024-09-10T10:04:06.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec15dd80 con 0x7f8eec073740 2024-09-10T10:04:06.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ee3400640 1 -- 172.21.15.110:0/1023905985 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc003bc0 con 0x7f8eec073740 2024-09-10T10:04:06.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec15e340 con 0x7f8eec073740 2024-09-10T10:04:06.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eec15cc00 con 0x7f8eec073740 2024-09-10T10:04:06.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ee3400640 1 -- 172.21.15.110:0/1023905985 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f8edc00a850 con 0x7f8eec073740 2024-09-10T10:04:06.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ee3400640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ecc067cb0 0x7f8ecc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:06.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.946+0000 7f8ee3400640 1 -- 172.21.15.110:0/1023905985 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f8edc086030 con 0x7f8eec073740 2024-09-10T10:04:06.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.950+0000 7f8eeaa00640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ecc067cb0 0x7f8ecc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:06.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.950+0000 7f8eeaa00640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ecc067cb0 0x7f8ecc06a170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8eec1603b0 tx=0x7f8ed8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:06.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:06.950+0000 7f8ee3400640 1 -- 172.21.15.110:0/1023905985 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8edc020020 con 0x7f8eec073740 2024-09-10T10:04:07.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.142+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f8eec070ac0 con 0x7f8eec073740 2024-09-10T10:04:07.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ee3400640 1 -- 172.21.15.110:0/1023905985 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v79) ==== 136+0+0 (secure 0 0 0) 0x7f8edc056600 con 0x7f8eec073740 2024-09-10T10:04:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ecc067cb0 msgr2=0x7f8ecc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ecc067cb0 0x7f8ecc06a170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8eec1603b0 tx=0x7f8ed8008040 comp rx=0 tx=0).stop 2024-09-10T10:04:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 msgr2=0x7f8eec06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec06e950 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c760 tx=0x7f8edc00cc20 comp rx=0 tx=0).stop 2024-09-10T10:04:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 shutdown_connections 2024-09-10T10:04:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ecc067cb0 0x7f8ecc06a170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8eec15c820 0x7f8eec1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8eec0d0410 0x7f8eec143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.762+0000 7f8ef1200640 1 --2- 172.21.15.110:0/1023905985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8eec073740 0x7f8eec06e950 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.766+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 >> 172.21.15.110:0/1023905985 conn(0x7f8eec0d2530 msgr2=0x7f8eec0711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.766+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 shutdown_connections 2024-09-10T10:04:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.766+0000 7f8ef1200640 1 -- 172.21.15.110:0/1023905985 wait complete. 2024-09-10T10:04:07.788 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:04:07.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399fe00640 1 Processor -- start 2024-09-10T10:04:07.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399fe00640 1 -- start start 2024-09-10T10:04:07.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399fe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cfb90 0x7f399815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399fe00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f399815d4d0 con 0x7f39980b3f40 2024-09-10T10:04:07.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399fe00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f399815d7b0 con 0x7f39980cdcb0 2024-09-10T10:04:07.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399fe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f399806d130 con 0x7f39980cfb90 2024-09-10T10:04:07.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399ca00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f39980b3f40 0x7f39980cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43838/0 (socket says 172.21.15.110:43838) 2024-09-10T10:04:07.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399ca00640 1 -- 172.21.15.110:0/1499795822 learned_addr learned my addr 172.21.15.110:0/1499795822 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:07.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399d400640 1 --2- 172.21.15.110:0/1499795822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cfb90 0x7f399815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f3997e00640 1 -- 172.21.15.110:0/1499795822 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4000050349 0 0) 0x7f399815d4d0 con 0x7f39980b3f40 2024-09-10T10:04:07.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f3997e00640 1 -- 172.21.15.110:0/1499795822 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3980003530 con 0x7f39980b3f40 2024-09-10T10:04:07.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f3997e00640 1 -- 172.21.15.110:0/1499795822 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 461440724 0 0) 0x7f399815d7b0 con 0x7f39980cdcb0 2024-09-10T10:04:07.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f3997e00640 1 -- 172.21.15.110:0/1499795822 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f399815d4d0 con 0x7f39980cdcb0 2024-09-10T10:04:07.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399d400640 1 -- 172.21.15.110:0/1499795822 >> v1:172.21.15.145:6789/0 conn(0x7f39980cdcb0 legacy=0x7f399806db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399d400640 1 -- 172.21.15.110:0/1499795822 >> v1:172.21.15.110:6789/0 conn(0x7f39980b3f40 legacy=0x7f39980cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f3997e00640 1 -- 172.21.15.110:0/1499795822 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2499624584 0 0) 0x7f3980003530 con 0x7f39980b3f40 2024-09-10T10:04:07.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.922+0000 7f399d400640 1 -- 172.21.15.110:0/1499795822 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399806d380 con 0x7f39980cfb90 2024-09-10T10:04:07.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f399d400640 1 --2- 172.21.15.110:0/1499795822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cfb90 0x7f399815ca10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f398c009dc0 tx=0x7f398c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61afa94fa06c4bed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f3997e00640 1 -- 172.21.15.110:0/1499795822 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c0b2070 con 0x7f39980cfb90 2024-09-10T10:04:07.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f3997e00640 1 -- 172.21.15.110:0/1499795822 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398c0a57c0 con 0x7f39980cfb90 2024-09-10T10:04:07.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f3997e00640 1 -- 172.21.15.110:0/1499795822 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c0a5b90 con 0x7f39980cfb90 2024-09-10T10:04:07.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f399fe00640 1 -- 172.21.15.110:0/1499795822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cfb90 msgr2=0x7f399815ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f399fe00640 1 --2- 172.21.15.110:0/1499795822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cfb90 0x7f399815ca10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f398c009dc0 tx=0x7f398c01c900 comp rx=0 tx=0).stop 2024-09-10T10:04:07.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f399fe00640 1 -- 172.21.15.110:0/1499795822 shutdown_connections 2024-09-10T10:04:07.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f399fe00640 1 --2- 172.21.15.110:0/1499795822 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cfb90 0x7f399815ca10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f399fe00640 1 -- 172.21.15.110:0/1499795822 >> 172.21.15.110:0/1499795822 conn(0x7f3998062d10 msgr2=0x7f399806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:07.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f399fe00640 1 -- 172.21.15.110:0/1499795822 shutdown_connections 2024-09-10T10:04:07.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.926+0000 7f399fe00640 1 -- 172.21.15.110:0/1499795822 wait complete. 2024-09-10T10:04:07.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 Processor -- start 2024-09-10T10:04:07.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 -- start start 2024-09-10T10:04:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39980b3f40 0x7f3998152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cdcb0 0x7f39980ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39980cfb90 0x7f39980adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f399816a390 con 0x7f39980b3f40 2024-09-10T10:04:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f399806eaa0 con 0x7f39980cfb90 2024-09-10T10:04:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f399816a510 con 0x7f39980cdcb0 2024-09-10T10:04:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39980b3f40 0x7f3998152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39980b3f40 0x7f3998152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50628/0 (socket says 172.21.15.110:50628) 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 -- 172.21.15.110:0/2655334012 learned_addr learned my addr 172.21.15.110:0/2655334012 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399de00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39980cfb90 0x7f39980adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399ca00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cdcb0 0x7f39980ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 -- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cdcb0 msgr2=0x7f39980ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cdcb0 0x7f39980ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 -- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39980cfb90 msgr2=0x7f39980adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39980cfb90 0x7f39980adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 -- 172.21.15.110:0/2655334012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39980763f0 con 0x7f39980b3f40 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399d400640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39980b3f40 0x7f3998152930 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f398c01a530 tx=0x7f398c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f3995600640 1 -- 172.21.15.110:0/2655334012 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c0bd070 con 0x7f39980b3f40 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f3995600640 1 -- 172.21.15.110:0/2655334012 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398c0a5e50 con 0x7f39980b3f40 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399806be60 con 0x7f39980b3f40 2024-09-10T10:04:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f3995600640 1 -- 172.21.15.110:0/2655334012 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c0b2040 con 0x7f39980b3f40 2024-09-10T10:04:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39980ae7a0 con 0x7f39980b3f40 2024-09-10T10:04:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.930+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399815ca10 con 0x7f39980b3f40 2024-09-10T10:04:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.934+0000 7f3995600640 1 -- 172.21.15.110:0/2655334012 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f398c0ad870 con 0x7f39980b3f40 2024-09-10T10:04:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.934+0000 7f3995600640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3978067cb0 0x7f397806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.934+0000 7f3995600640 1 -- 172.21.15.110:0/2655334012 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f398c121380 con 0x7f39980b3f40 2024-09-10T10:04:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.934+0000 7f399ca00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3978067cb0 0x7f397806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.934+0000 7f399ca00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3978067cb0 0x7f397806a170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f39980ccad0 tx=0x7f3984009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:07.938+0000 7f3995600640 1 -- 172.21.15.110:0/2655334012 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f398c016580 con 0x7f39980b3f40 2024-09-10T10:04:08.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.118+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f39980b5660 con 0x7f3978067cb0 2024-09-10T10:04:08.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f3995600640 1 -- 172.21.15.110:0/2655334012 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f39980b5660 con 0x7f3978067cb0 2024-09-10T10:04:08.126 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:04:08.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:08.126 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:04:08.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:08.126 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:04:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3978067cb0 msgr2=0x7f397806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3978067cb0 0x7f397806a170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f39980ccad0 tx=0x7f3984009040 comp rx=0 tx=0).stop 2024-09-10T10:04:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39980b3f40 msgr2=0x7f3998152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39980b3f40 0x7f3998152930 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f398c01a530 tx=0x7f398c0196a0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 shutdown_connections 2024-09-10T10:04:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3978067cb0 0x7f397806a170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39980cfb90 0x7f39980adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39980cdcb0 0x7f39980ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 --2- 172.21.15.110:0/2655334012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39980b3f40 0x7f3998152930 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 >> 172.21.15.110:0/2655334012 conn(0x7f3998062d10 msgr2=0x7f399806db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 shutdown_connections 2024-09-10T10:04:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.122+0000 7f399fe00640 1 -- 172.21.15.110:0/2655334012 wait complete. 2024-09-10T10:04:08.147 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:04:08.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4f6200640 1 Processor -- start 2024-09-10T10:04:08.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4f6200640 1 -- start start 2024-09-10T10:04:08.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4f6200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f0145f30 0x7ff4f00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:08.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4f6200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff4f00cfd90 0x7ff4f00b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:08.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4f6200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f00b37c0 con 0x7ff4f00b29e0 2024-09-10T10:04:08.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4f6200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f00b2e20 con 0x7ff4f00cfd90 2024-09-10T10:04:08.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4f6200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f00b3aa0 con 0x7ff4f0145f30 2024-09-10T10:04:08.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4efe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff4f00cfd90 0x7ff4f00b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:08.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4ef400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f0145f30 0x7ff4f00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:08.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4efe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff4f00cfd90 0x7ff4f00b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:54448/0 (socket says 172.21.15.110:54448) 2024-09-10T10:04:08.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4ef400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f0145f30 0x7ff4f00b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:40956/0 (socket says 172.21.15.110:40956) 2024-09-10T10:04:08.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4eea00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ff4f00b29e0 0x7ff4f00cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:56484/0 (socket says 172.21.15.110:56484) 2024-09-10T10:04:08.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4efe00640 1 -- 172.21.15.110:0/1080686673 learned_addr learned my addr 172.21.15.110:0/1080686673 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:08.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4efe00640 1 -- 172.21.15.110:0/1080686673 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f0145f30 msgr2=0x7ff4f00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:08.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4efe00640 1 --2- 172.21.15.110:0/1080686673 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f0145f30 0x7ff4f00b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4efe00640 1 -- 172.21.15.110:0/1080686673 >> v1:172.21.15.145:6789/0 conn(0x7ff4f00b29e0 legacy=0x7ff4f00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:08.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.282+0000 7ff4efe00640 1 -- 172.21.15.110:0/1080686673 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f00b3d80 con 0x7ff4f00cfd90 2024-09-10T10:04:08.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4efe00640 1 --2- 172.21.15.110:0/1080686673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00cfd90 0x7ff4f00b4580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009e50 tx=0x7ff4e801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca999e732ca7012 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:08.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4ee000640 1 -- 172.21.15.110:0/1080686673 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80a5700 con 0x7ff4f00cfd90 2024-09-10T10:04:08.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4ee000640 1 -- 172.21.15.110:0/1080686673 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e80a58a0 con 0x7ff4f00cfd90 2024-09-10T10:04:08.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4ee000640 1 -- 172.21.15.110:0/1080686673 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80a5c70 con 0x7ff4f00cfd90 2024-09-10T10:04:08.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4f6200640 1 -- 172.21.15.110:0/1080686673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00cfd90 msgr2=0x7ff4f00b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:08.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4f6200640 1 --2- 172.21.15.110:0/1080686673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00cfd90 0x7ff4f00b4580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009e50 tx=0x7ff4e801ca30 comp rx=0 tx=0).stop 2024-09-10T10:04:08.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4f6200640 1 -- 172.21.15.110:0/1080686673 shutdown_connections 2024-09-10T10:04:08.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4f6200640 1 --2- 172.21.15.110:0/1080686673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00cfd90 0x7ff4f00b4580 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4f6200640 1 --2- 172.21.15.110:0/1080686673 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f0145f30 0x7ff4f00b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4f6200640 1 -- 172.21.15.110:0/1080686673 >> 172.21.15.110:0/1080686673 conn(0x7ff4f0062e40 msgr2=0x7ff4f006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:08.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4f6200640 1 -- 172.21.15.110:0/1080686673 shutdown_connections 2024-09-10T10:04:08.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.286+0000 7ff4f6200640 1 -- 172.21.15.110:0/1080686673 wait complete. 2024-09-10T10:04:08.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 Processor -- start 2024-09-10T10:04:08.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 -- start start 2024-09-10T10:04:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00b29e0 0x7ff4f013f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f00cfd90 0x7ff4f0140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4f0145f30 0x7ff4f0140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f00b56e0 con 0x7ff4f0145f30 2024-09-10T10:04:08.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f00b5560 con 0x7ff4f00b29e0 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f00b5860 con 0x7ff4f00cfd90 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4f0145f30 0x7ff4f0140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4f0145f30 0x7ff4f0140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48338/0 (socket says 172.21.15.110:48338) 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 -- 172.21.15.110:0/3172322012 learned_addr learned my addr 172.21.15.110:0/3172322012 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4ef400640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00b29e0 0x7ff4f013f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4eea00640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f00cfd90 0x7ff4f0140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 -- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f00cfd90 msgr2=0x7ff4f0140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f00cfd90 0x7ff4f0140500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 -- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00b29e0 msgr2=0x7ff4f013f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:08.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00b29e0 0x7ff4f013f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 -- 172.21.15.110:0/3172322012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f013dd50 con 0x7ff4f0145f30 2024-09-10T10:04:08.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4efe00640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4f0145f30 0x7ff4f0140a40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80af040 tx=0x7ff4e80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:08.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4e7400640 1 -- 172.21.15.110:0/3172322012 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80a5700 con 0x7ff4f0145f30 2024-09-10T10:04:08.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4e7400640 1 -- 172.21.15.110:0/3172322012 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e80be070 con 0x7ff4f0145f30 2024-09-10T10:04:08.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4e7400640 1 -- 172.21.15.110:0/3172322012 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8009510 con 0x7ff4f0145f30 2024-09-10T10:04:08.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f013e030 con 0x7ff4f0145f30 2024-09-10T10:04:08.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f013e5f0 con 0x7ff4f0145f30 2024-09-10T10:04:08.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.290+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f00b4580 con 0x7ff4f0145f30 2024-09-10T10:04:08.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.294+0000 7ff4e7400640 1 -- 172.21.15.110:0/3172322012 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7ff4e80b8ac0 con 0x7ff4f0145f30 2024-09-10T10:04:08.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.294+0000 7ff4e7400640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff4d0067cb0 0x7ff4d006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:08.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.294+0000 7ff4e7400640 1 -- 172.21.15.110:0/3172322012 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7ff4e8121590 con 0x7ff4f0145f30 2024-09-10T10:04:08.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.294+0000 7ff4ef400640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff4d0067cb0 0x7ff4d006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:08.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.294+0000 7ff4ef400640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff4d0067cb0 0x7ff4d006a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0142180 tx=0x7ff4e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:08.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.298+0000 7ff4e7400640 1 -- 172.21.15.110:0/3172322012 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff4e8016620 con 0x7ff4f0145f30 2024-09-10T10:04:08.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.454+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff4f0142e90 con 0x7ff4f0145f30 2024-09-10T10:04:08.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.454+0000 7ff4e7400640 1 -- 172.21.15.110:0/3172322012 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7ff4e80f1bc0 con 0x7ff4f0145f30 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff4d0067cb0 msgr2=0x7ff4d006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff4d0067cb0 0x7ff4d006a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0142180 tx=0x7ff4e0008040 comp rx=0 tx=0).stop 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4f0145f30 msgr2=0x7ff4f0140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4f0145f30 0x7ff4f0140a40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80af040 tx=0x7ff4e80197d0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 shutdown_connections 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff4d0067cb0 0x7ff4d006a170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4f0145f30 0x7ff4f0140a40 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4f00cfd90 0x7ff4f0140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 --2- 172.21.15.110:0/3172322012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4f00b29e0 0x7ff4f013f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 >> 172.21.15.110:0/3172322012 conn(0x7ff4f0062e40 msgr2=0x7ff4f00cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 shutdown_connections 2024-09-10T10:04:08.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:08.458+0000 7ff4f6200640 1 -- 172.21.15.110:0/3172322012 wait complete. 2024-09-10T10:04:08.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:08.481 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol47570/e8b6cc38-9278-4293-af4d-37db63293428) 2024-09-10T10:04:08.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:08.508 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol47570/e8b6cc38-9278-4293-af4d-37db63293428/.snap/fake) 2024-09-10T10:04:08.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:08.579 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T10:04:08.629 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:04:08.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:08.630 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol47570/e8b6cc38-9278-4293-af4d-37db63293428) 2024-09-10T10:04:08.696 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:08.696 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol47570/.meta 2024-09-10T10:04:08.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:08.760 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp4440/subvol65511/33198619-5600-4c43-8499-ac39bcb647be) 2024-09-10T10:04:08.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:08.823 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp4440/subvol65511/33198619-5600-4c43-8499-ac39bcb647be/.snap/fake) 2024-09-10T10:04:08.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:08.890 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T10:04:08.941 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:04:08.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:08.942 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp4440/subvol65511/33198619-5600-4c43-8499-ac39bcb647be) 2024-09-10T10:04:09.008 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:09.008 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp4440/subvol65511/.meta 2024-09-10T10:04:09.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:09.071 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol78472/86bf95ec-8594-4f0d-8f53-5c3e527725a0) 2024-09-10T10:04:09.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:09.133 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T10:04:09.185 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:04:09.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:09.185 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol78472/86bf95ec-8594-4f0d-8f53-5c3e527725a0) 2024-09-10T10:04:09.253 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:09.253 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol78472/.meta 2024-09-10T10:04:09.316 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol47570 2024-09-10T10:04:09.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.498+0000 7f905d200640 1 Processor -- start 2024-09-10T10:04:09.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.498+0000 7f905d200640 1 -- start start 2024-09-10T10:04:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.498+0000 7f905d200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90580b2bb0 0x7f90580cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.498+0000 7f905d200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f90580cfbc0 0x7f90580b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.498+0000 7f905d200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90580b37c0 con 0x7f9058145f30 2024-09-10T10:04:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.498+0000 7f905d200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f90580b3aa0 con 0x7f90580cfbc0 2024-09-10T10:04:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.498+0000 7f905d200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f90580b3d80 con 0x7f90580b2bb0 2024-09-10T10:04:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9056800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f90580cfbc0 0x7f90580b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9056800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f90580cfbc0 0x7f90580b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48340/0 (socket says 172.21.15.110:48340) 2024-09-10T10:04:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9056800640 1 -- 172.21.15.110:0/1679865795 learned_addr learned my addr 172.21.15.110:0/1679865795 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f904fe00640 1 --2- 172.21.15.110:0/1679865795 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90580b2bb0 0x7f90580cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9056800640 1 -- 172.21.15.110:0/1679865795 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90580b2bb0 msgr2=0x7f90580cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9056800640 1 --2- 172.21.15.110:0/1679865795 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90580b2bb0 0x7f90580cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9056800640 1 -- 172.21.15.110:0/1679865795 >> v1:172.21.15.145:6789/0 conn(0x7f9058145f30 legacy=0x7f90580b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9056800640 1 -- 172.21.15.110:0/1679865795 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9058146310 con 0x7f90580cfbc0 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9056800640 1 --2- 172.21.15.110:0/1679865795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580cfbc0 0x7f90580b4580 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9050009e50 tx=0x7f905001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0e9731ff99ba9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9055400640 1 -- 172.21.15.110:0/1679865795 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500a5700 con 0x7f90580cfbc0 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9055400640 1 -- 172.21.15.110:0/1679865795 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90500a58a0 con 0x7f90580cfbc0 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.502+0000 7f9055400640 1 -- 172.21.15.110:0/1679865795 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500a5c70 con 0x7f90580cfbc0 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- 172.21.15.110:0/1679865795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580cfbc0 msgr2=0x7f90580b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 --2- 172.21.15.110:0/1679865795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580cfbc0 0x7f90580b4580 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9050009e50 tx=0x7f905001ca30 comp rx=0 tx=0).stop 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- 172.21.15.110:0/1679865795 shutdown_connections 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 --2- 172.21.15.110:0/1679865795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580cfbc0 0x7f90580b4580 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 --2- 172.21.15.110:0/1679865795 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f90580b2bb0 0x7f90580cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- 172.21.15.110:0/1679865795 >> 172.21.15.110:0/1679865795 conn(0x7f9058062e40 msgr2=0x7f905806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:09.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- 172.21.15.110:0/1679865795 shutdown_connections 2024-09-10T10:04:09.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- 172.21.15.110:0/1679865795 wait complete. 2024-09-10T10:04:09.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 Processor -- start 2024-09-10T10:04:09.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- start start 2024-09-10T10:04:09.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580b2bb0 0x7f90581f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580cfbc0 0x7f90581f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9058145f30 0x7f905815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90580b56a0 con 0x7f90580b2bb0 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f90580b5520 con 0x7f90580cfbc0 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f905d200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f90580b5820 con 0x7f9058145f30 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9055e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580b2bb0 0x7f90581f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9055e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580b2bb0 0x7f90581f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48350/0 (socket says 172.21.15.110:48350) 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9055e00640 1 -- 172.21.15.110:0/4265231920 learned_addr learned my addr 172.21.15.110:0/4265231920 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f904fe00640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580cfbc0 0x7f90581f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9056800640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9058145f30 0x7f905815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9055e00640 1 -- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9058145f30 msgr2=0x7f905815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9055e00640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9058145f30 0x7f905815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9055e00640 1 -- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580cfbc0 msgr2=0x7f90581f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9055e00640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580cfbc0 0x7f90581f9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.506+0000 7f9055e00640 1 -- 172.21.15.110:0/4265231920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90581fa460 con 0x7f90580b2bb0 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f9055e00640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580b2bb0 0x7f90581f7150 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f904800c3c0 tx=0x7f904800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f904e000640 1 -- 172.21.15.110:0/4265231920 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048017070 con 0x7f90580b2bb0 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f904e000640 1 -- 172.21.15.110:0/4265231920 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90480052d0 con 0x7f90580b2bb0 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f904e000640 1 -- 172.21.15.110:0/4265231920 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048003840 con 0x7f90580b2bb0 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905815dbb0 con 0x7f90580b2bb0 2024-09-10T10:04:09.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905815e170 con 0x7f90580b2bb0 2024-09-10T10:04:09.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90580cf680 con 0x7f90580b2bb0 2024-09-10T10:04:09.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f904e000640 1 -- 172.21.15.110:0/4265231920 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f904801a070 con 0x7f90580b2bb0 2024-09-10T10:04:09.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f904e000640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9034067cb0 0x7f903406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f904e000640 1 -- 172.21.15.110:0/4265231920 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f9048085680 con 0x7f90580b2bb0 2024-09-10T10:04:09.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.510+0000 7f904fe00640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9034067cb0 0x7f903406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.514+0000 7f904fe00640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9034067cb0 0x7f903406a170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9058167160 tx=0x7f9040009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:09.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.514+0000 7f904e000640 1 -- 172.21.15.110:0/4265231920 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9048005480 con 0x7f90580b2bb0 2024-09-10T10:04:09.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.698+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol47570", "target": ["mon-mgr", ""]}) -- 0x7f90580b0860 con 0x7f9034067cb0 2024-09-10T10:04:09.711 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:09.710+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:09.711 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:09.710+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:09.711 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:09.710+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:09.712 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:09.710+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:09.712 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:09.710+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:09.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.746+0000 7f904e000640 1 -- 172.21.15.110:0/4265231920 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f90580b0860 con 0x7f9034067cb0 2024-09-10T10:04:09.752 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol47570/e8b6cc38-9278-4293-af4d-37db63293428 2024-09-10T10:04:09.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9034067cb0 msgr2=0x7f903406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9034067cb0 0x7f903406a170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9058167160 tx=0x7f9040009190 comp rx=0 tx=0).stop 2024-09-10T10:04:09.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580b2bb0 msgr2=0x7f90581f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580b2bb0 0x7f90581f7150 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f904800c3c0 tx=0x7f904800c880 comp rx=0 tx=0).stop 2024-09-10T10:04:09.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 shutdown_connections 2024-09-10T10:04:09.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9034067cb0 0x7f903406a170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9058145f30 0x7f905815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f90580cfbc0 0x7f90581f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 --2- 172.21.15.110:0/4265231920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90580b2bb0 0x7f90581f7150 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 >> 172.21.15.110:0/4265231920 conn(0x7f9058062e40 msgr2=0x7f9058071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:09.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 shutdown_connections 2024-09-10T10:04:09.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.750+0000 7f905d200640 1 -- 172.21.15.110:0/4265231920 wait complete. 2024-09-10T10:04:09.773 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol65511 subvol_grp4440 2024-09-10T10:04:09.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f0571600640 1 Processor -- start 2024-09-10T10:04:09.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f0571600640 1 -- start start 2024-09-10T10:04:09.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f0571600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c15c560 0x7f056c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f0571600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f056c06f010 0x7f056c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f0571600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056c06f430 con 0x7f056c155970 2024-09-10T10:04:09.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f0571600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f056c0b4570 con 0x7f056c06f010 2024-09-10T10:04:09.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f0571600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f056c155d50 con 0x7f056c15c560 2024-09-10T10:04:09.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056a000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c15c560 0x7f056c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056a000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c15c560 0x7f056c06ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:41000/0 (socket says 172.21.15.110:41000) 2024-09-10T10:04:09.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056a000640 1 -- 172.21.15.110:0/534417340 learned_addr learned my addr 172.21.15.110:0/534417340 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:09.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f056c06f010 0x7f056c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056aa00640 1 --1- 172.21.15.110:0/534417340 >> v1:172.21.15.145:6789/0 conn(0x7f056c155970 0x7f056c0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:56490/0 (socket says 172.21.15.110:56490) 2024-09-10T10:04:09.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056a000640 1 -- 172.21.15.110:0/534417340 >> v2:172.21.15.145:3300/0 conn(0x7f056c06f010 msgr2=0x7f056c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:09.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056a000640 1 --2- 172.21.15.110:0/534417340 >> v2:172.21.15.145:3300/0 conn(0x7f056c06f010 0x7f056c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056a000640 1 -- 172.21.15.110:0/534417340 >> v1:172.21.15.145:6789/0 conn(0x7f056c155970 legacy=0x7f056c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.910+0000 7f056a000640 1 -- 172.21.15.110:0/534417340 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c0b34d0 con 0x7f056c15c560 2024-09-10T10:04:09.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f056a000640 1 --2- 172.21.15.110:0/534417340 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c15c560 0x7f056c06ead0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0558009870 tx=0x7f055801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801a71a90fd0a4cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:09.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0569600640 1 -- 172.21.15.110:0/534417340 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580aa600 con 0x7f056c15c560 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0569600640 1 -- 172.21.15.110:0/534417340 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05580af070 con 0x7f056c15c560 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0569600640 1 -- 172.21.15.110:0/534417340 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580aa9c0 con 0x7f056c15c560 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0571600640 1 -- 172.21.15.110:0/534417340 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c15c560 msgr2=0x7f056c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0571600640 1 --2- 172.21.15.110:0/534417340 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c15c560 0x7f056c06ead0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0558009870 tx=0x7f055801c4a0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0571600640 1 -- 172.21.15.110:0/534417340 shutdown_connections 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0571600640 1 --2- 172.21.15.110:0/534417340 >> v2:172.21.15.145:3300/0 conn(0x7f056c06f010 0x7f056c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0571600640 1 --2- 172.21.15.110:0/534417340 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c15c560 0x7f056c06ead0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0571600640 1 -- 172.21.15.110:0/534417340 >> 172.21.15.110:0/534417340 conn(0x7f056c06bc50 msgr2=0x7f056c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0571600640 1 -- 172.21.15.110:0/534417340 shutdown_connections 2024-09-10T10:04:09.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.914+0000 7f0571600640 1 -- 172.21.15.110:0/534417340 wait complete. 2024-09-10T10:04:09.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 Processor -- start 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 -- start start 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f056c06f010 0x7f056c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c155970 0x7f056c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f056c15c560 0x7f056c15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f056c0cf170 con 0x7f056c15c560 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f056c0b4050 con 0x7f056c06f010 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f056c0cf2f0 con 0x7f056c155970 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f056c15c560 0x7f056c15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f056c15c560 0x7f056c15f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48362/0 (socket says 172.21.15.110:48362) 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 -- 172.21.15.110:0/1418104869 learned_addr learned my addr 172.21.15.110:0/1418104869 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:09.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056aa00640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f056c06f010 0x7f056c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056a000640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c155970 0x7f056c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 -- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c155970 msgr2=0x7f056c158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c155970 0x7f056c158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 -- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f056c06f010 msgr2=0x7f056c153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f056c06f010 0x7f056c153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 -- 172.21.15.110:0/1418104869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c15e0e0 con 0x7f056c15c560 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f056b400640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f056c15c560 0x7f056c15f2e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f056400efc0 tx=0x7f056400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0562a00640 1 -- 172.21.15.110:0/1418104869 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564018020 con 0x7f056c15c560 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0562a00640 1 -- 172.21.15.110:0/1418104869 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564014480 con 0x7f056c15c560 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056c15e3c0 con 0x7f056c15c560 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0562a00640 1 -- 172.21.15.110:0/1418104869 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564014870 con 0x7f056c15c560 2024-09-10T10:04:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.918+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056c15edc0 con 0x7f056c15c560 2024-09-10T10:04:09.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.922+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056c0ce420 con 0x7f056c15c560 2024-09-10T10:04:09.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.922+0000 7f0562a00640 1 -- 172.21.15.110:0/1418104869 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f056401b020 con 0x7f056c15c560 2024-09-10T10:04:09.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.922+0000 7f0562a00640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f054c067cb0 0x7f054c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:09.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.922+0000 7f0562a00640 1 -- 172.21.15.110:0/1418104869 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f0564086f70 con 0x7f056c15c560 2024-09-10T10:04:09.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.922+0000 7f056aa00640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f054c067cb0 0x7f054c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:09.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.922+0000 7f056aa00640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f054c067cb0 0x7f054c06a170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f055c000f70 tx=0x7f055c007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:09.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:09.926+0000 7f0562a00640 1 -- 172.21.15.110:0/1418104869 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0564022070 con 0x7f056c15c560 2024-09-10T10:04:10.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.110+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol65511", "group_name": "subvol_grp4440", "target": ["mon-mgr", ""]}) -- 0x7f056c0cf770 con 0x7f054c067cb0 2024-09-10T10:04:10.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.146+0000 7f0562a00640 1 -- 172.21.15.110:0/1418104869 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7f056c0cf770 con 0x7f054c067cb0 2024-09-10T10:04:10.151 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/subvol_grp4440/subvol65511/33198619-5600-4c43-8499-ac39bcb647be 2024-09-10T10:04:10.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f054c067cb0 msgr2=0x7f054c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f054c067cb0 0x7f054c06a170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f055c000f70 tx=0x7f055c007e90 comp rx=0 tx=0).stop 2024-09-10T10:04:10.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f056c15c560 msgr2=0x7f056c15f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f056c15c560 0x7f056c15f2e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f056400efc0 tx=0x7f056400c490 comp rx=0 tx=0).stop 2024-09-10T10:04:10.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 shutdown_connections 2024-09-10T10:04:10.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f054c067cb0 0x7f054c06a170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f056c15c560 0x7f056c15f2e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f056c155970 0x7f056c158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 --2- 172.21.15.110:0/1418104869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f056c06f010 0x7f056c153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 >> 172.21.15.110:0/1418104869 conn(0x7f056c06bc50 msgr2=0x7f056c071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:10.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 shutdown_connections 2024-09-10T10:04:10.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.150+0000 7f0571600640 1 -- 172.21.15.110:0/1418104869 wait complete. 2024-09-10T10:04:10.172 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol78472 2024-09-10T10:04:10.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa47ae00640 1 Processor -- start 2024-09-10T10:04:10.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa47ae00640 1 -- start start 2024-09-10T10:04:10.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa47ae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa474145f30 0x7fa4740b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa47ae00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4740cfd90 0x7fa4740b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa47ae00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4740b37c0 con 0x7fa4740b29e0 2024-09-10T10:04:10.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa47ae00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa4740b2e20 con 0x7fa4740cfd90 2024-09-10T10:04:10.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa47ae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4740b3aa0 con 0x7fa474145f30 2024-09-10T10:04:10.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa478e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4740cfd90 0x7fa4740b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa478e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa4740cfd90 0x7fa4740b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48366/0 (socket says 172.21.15.110:48366) 2024-09-10T10:04:10.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa478e00640 1 -- 172.21.15.110:0/3955699742 learned_addr learned my addr 172.21.15.110:0/3955699742 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:10.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa473e00640 1 --2- 172.21.15.110:0/3955699742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa474145f30 0x7fa4740b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa478e00640 1 -- 172.21.15.110:0/3955699742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa474145f30 msgr2=0x7fa4740b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:10.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa478e00640 1 --2- 172.21.15.110:0/3955699742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa474145f30 0x7fa4740b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa478e00640 1 -- 172.21.15.110:0/3955699742 >> v1:172.21.15.145:6789/0 conn(0x7fa4740b29e0 legacy=0x7fa4740cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:10.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa478e00640 1 -- 172.21.15.110:0/3955699742 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4740b3d80 con 0x7fa4740cfd90 2024-09-10T10:04:10.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.306+0000 7fa473e00640 1 --2- 172.21.15.110:0/3955699742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa474145f30 0x7fa4740b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:10.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa478e00640 1 --2- 172.21.15.110:0/3955699742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4740cfd90 0x7fa4740b4580 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009e50 tx=0x7fa46c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60814a928e8a3115 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:10.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa472a00640 1 -- 172.21.15.110:0/3955699742 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0a5700 con 0x7fa4740cfd90 2024-09-10T10:04:10.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa472a00640 1 -- 172.21.15.110:0/3955699742 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46c0a58a0 con 0x7fa4740cfd90 2024-09-10T10:04:10.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa472a00640 1 -- 172.21.15.110:0/3955699742 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0a5c70 con 0x7fa4740cfd90 2024-09-10T10:04:10.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa47ae00640 1 -- 172.21.15.110:0/3955699742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4740cfd90 msgr2=0x7fa4740b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa47ae00640 1 --2- 172.21.15.110:0/3955699742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4740cfd90 0x7fa4740b4580 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009e50 tx=0x7fa46c01ca30 comp rx=0 tx=0).stop 2024-09-10T10:04:10.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa47ae00640 1 -- 172.21.15.110:0/3955699742 shutdown_connections 2024-09-10T10:04:10.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa47ae00640 1 --2- 172.21.15.110:0/3955699742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4740cfd90 0x7fa4740b4580 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa47ae00640 1 --2- 172.21.15.110:0/3955699742 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa474145f30 0x7fa4740b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa47ae00640 1 -- 172.21.15.110:0/3955699742 >> 172.21.15.110:0/3955699742 conn(0x7fa474062e40 msgr2=0x7fa47406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:10.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa47ae00640 1 -- 172.21.15.110:0/3955699742 shutdown_connections 2024-09-10T10:04:10.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.310+0000 7fa47ae00640 1 -- 172.21.15.110:0/3955699742 wait complete. 2024-09-10T10:04:10.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 Processor -- start 2024-09-10T10:04:10.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 -- start start 2024-09-10T10:04:10.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4740b29e0 0x7fa4740aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4740cfd90 0x7fa4741f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa474145f30 0x7fa4741f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4740b56e0 con 0x7fa474145f30 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4740b5560 con 0x7fa4740b29e0 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa4740b5860 con 0x7fa4740cfd90 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa474145f30 0x7fa4741f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa474145f30 0x7fa4741f8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48382/0 (socket says 172.21.15.110:48382) 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 -- 172.21.15.110:0/1656746172 learned_addr learned my addr 172.21.15.110:0/1656746172 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa473400640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4740cfd90 0x7fa4741f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa473e00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4740b29e0 0x7fa4740aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 -- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4740cfd90 msgr2=0x7fa4741f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4740cfd90 0x7fa4741f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 -- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4740b29e0 msgr2=0x7fa4740aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4740b29e0 0x7fa4740aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 -- 172.21.15.110:0/1656746172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4740ab380 con 0x7fa474145f30 2024-09-10T10:04:10.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa478e00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa474145f30 0x7fa4741f8e00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0af040 tx=0x7fa46c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:10.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa46be00640 1 -- 172.21.15.110:0/1656746172 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0bc020 con 0x7fa474145f30 2024-09-10T10:04:10.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa46be00640 1 -- 172.21.15.110:0/1656746172 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46c0b8880 con 0x7fa474145f30 2024-09-10T10:04:10.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4740aa5d0 con 0x7fa474145f30 2024-09-10T10:04:10.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa46be00640 1 -- 172.21.15.110:0/1656746172 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c009510 con 0x7fa474145f30 2024-09-10T10:04:10.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.314+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47415bbf0 con 0x7fa474145f30 2024-09-10T10:04:10.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.318+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4740b4580 con 0x7fa474145f30 2024-09-10T10:04:10.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.318+0000 7fa46be00640 1 -- 172.21.15.110:0/1656746172 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7fa46c0bf020 con 0x7fa474145f30 2024-09-10T10:04:10.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.318+0000 7fa46be00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa454067cb0 0x7fa45406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.318+0000 7fa46be00640 1 -- 172.21.15.110:0/1656746172 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7fa46c121640 con 0x7fa474145f30 2024-09-10T10:04:10.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.318+0000 7fa473e00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa454067cb0 0x7fa45406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.318+0000 7fa473e00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa454067cb0 0x7fa45406a170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa464006fd0 tx=0x7fa464008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:10.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.322+0000 7fa46be00640 1 -- 172.21.15.110:0/1656746172 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa46c0c6030 con 0x7fa474145f30 2024-09-10T10:04:10.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.510+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol78472", "target": ["mon-mgr", ""]}) -- 0x7fa47415c3f0 con 0x7fa454067cb0 2024-09-10T10:04:10.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.538+0000 7fa46be00640 1 -- 172.21.15.110:0/1656746172 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7fa47415c3f0 con 0x7fa454067cb0 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout: "status": { 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "pending", 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout: "source": { 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout: "volume": "fake", 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout: "subvolume": "fake", 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot": "fake" 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:10.545 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:04:10.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa454067cb0 msgr2=0x7fa45406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa454067cb0 0x7fa45406a170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa464006fd0 tx=0x7fa464008040 comp rx=0 tx=0).stop 2024-09-10T10:04:10.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa474145f30 msgr2=0x7fa4741f8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa474145f30 0x7fa4741f8e00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0af040 tx=0x7fa46c0197d0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 shutdown_connections 2024-09-10T10:04:10.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa454067cb0 0x7fa45406a170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa474145f30 0x7fa4741f8e00 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4740cfd90 0x7fa4741f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 --2- 172.21.15.110:0/1656746172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4740b29e0 0x7fa4740aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 >> 172.21.15.110:0/1656746172 conn(0x7fa474062e40 msgr2=0x7fa4740cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:10.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 shutdown_connections 2024-09-10T10:04:10.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.542+0000 7fa47ae00640 1 -- 172.21.15.110:0/1656746172 wait complete. 2024-09-10T10:04:10.567 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol47570 fake 2024-09-10T10:04:10.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.702+0000 7f25f3000640 1 Processor -- start 2024-09-10T10:04:10.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.702+0000 7f25f3000640 1 -- start start 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25f3000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f25ec0cfe30 0x7f25ec071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25f3000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f25ec0cd510 0x7f25ec0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25f3000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 0x7f25ec06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25f3000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f25ec15d9a0 con 0x7f25ec0cfe30 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25f3000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f25ec06ea60 con 0x7f25ec0cd510 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25f3000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec06d2d0 con 0x7f25ec06e090 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f25ec0cfe30 0x7f25ec071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f25ec0cfe30 0x7f25ec071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48388/0 (socket says 172.21.15.110:48388) 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 -- 172.21.15.110:0/1159337747 learned_addr learned my addr 172.21.15.110:0/1159337747 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25f1000640 1 --2- 172.21.15.110:0/1159337747 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 0x7f25ec06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25eb400640 1 --2- 172.21.15.110:0/1159337747 >> v2:172.21.15.145:3300/0 conn(0x7f25ec0cd510 0x7f25ec0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 -- 172.21.15.110:0/1159337747 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 msgr2=0x7f25ec06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:10.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 --2- 172.21.15.110:0/1159337747 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 0x7f25ec06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 -- 172.21.15.110:0/1159337747 >> v2:172.21.15.145:3300/0 conn(0x7f25ec0cd510 msgr2=0x7f25ec0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:10.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 --2- 172.21.15.110:0/1159337747 >> v2:172.21.15.145:3300/0 conn(0x7f25ec0cd510 0x7f25ec0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 -- 172.21.15.110:0/1159337747 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec06d5b0 con 0x7f25ec0cfe30 2024-09-10T10:04:10.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.706+0000 7f25ebe00640 1 --2- 172.21.15.110:0/1159337747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cfe30 0x7f25ec071a40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0ce0f0 tx=0x7f25dc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573e19be7f214ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:10.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eaa00640 1 -- 172.21.15.110:0/1159337747 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc0a56f0 con 0x7f25ec0cfe30 2024-09-10T10:04:10.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eaa00640 1 -- 172.21.15.110:0/1159337747 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc0a5890 con 0x7f25ec0cfe30 2024-09-10T10:04:10.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eaa00640 1 -- 172.21.15.110:0/1159337747 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc0a5c60 con 0x7f25ec0cfe30 2024-09-10T10:04:10.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- 172.21.15.110:0/1159337747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cfe30 msgr2=0x7f25ec071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 --2- 172.21.15.110:0/1159337747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cfe30 0x7f25ec071a40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0ce0f0 tx=0x7f25dc01c960 comp rx=0 tx=0).stop 2024-09-10T10:04:10.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- 172.21.15.110:0/1159337747 shutdown_connections 2024-09-10T10:04:10.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 --2- 172.21.15.110:0/1159337747 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 0x7f25ec06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 --2- 172.21.15.110:0/1159337747 >> v2:172.21.15.145:3300/0 conn(0x7f25ec0cd510 0x7f25ec0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 --2- 172.21.15.110:0/1159337747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cfe30 0x7f25ec071a40 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- 172.21.15.110:0/1159337747 >> 172.21.15.110:0/1159337747 conn(0x7f25ec06bc50 msgr2=0x7f25ec06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:10.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- 172.21.15.110:0/1159337747 shutdown_connections 2024-09-10T10:04:10.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- 172.21.15.110:0/1159337747 wait complete. 2024-09-10T10:04:10.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 Processor -- start 2024-09-10T10:04:10.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- start start 2024-09-10T10:04:10.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 0x7f25ec0ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cd510 0x7f25ec0aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0cfe30 0x7f25ec0aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec072190 con 0x7f25ec0cd510 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec15d640 con 0x7f25ec0cfe30 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f3000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec160010 con 0x7f25ec06e090 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cd510 0x7f25ec0aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cd510 0x7f25ec0aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48396/0 (socket says 172.21.15.110:48396) 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eb400640 1 -- 172.21.15.110:0/593402144 learned_addr learned my addr 172.21.15.110:0/593402144 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25ebe00640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 0x7f25ec0ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25f1000640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0cfe30 0x7f25ec0aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eb400640 1 -- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 msgr2=0x7f25ec0ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eb400640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 0x7f25ec0ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eb400640 1 -- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0cfe30 msgr2=0x7f25ec0aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eb400640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0cfe30 0x7f25ec0aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.710+0000 7f25eb400640 1 -- 172.21.15.110:0/593402144 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec0acde0 con 0x7f25ec0cd510 2024-09-10T10:04:10.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25eb400640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cd510 0x7f25ec0aa430 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f25d800d800 tx=0x7f25d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:10.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25e3e00640 1 -- 172.21.15.110:0/593402144 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800b6c0 con 0x7f25ec0cd510 2024-09-10T10:04:10.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25e3e00640 1 -- 172.21.15.110:0/593402144 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d8019070 con 0x7f25ec0cd510 2024-09-10T10:04:10.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec0aef60 con 0x7f25ec0cd510 2024-09-10T10:04:10.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25e3e00640 1 -- 172.21.15.110:0/593402144 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800ba60 con 0x7f25ec0cd510 2024-09-10T10:04:10.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec0af240 con 0x7f25ec0cd510 2024-09-10T10:04:10.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ec0cc7f0 con 0x7f25ec0cd510 2024-09-10T10:04:10.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25e3e00640 1 -- 172.21.15.110:0/593402144 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f25d800bc00 con 0x7f25ec0cd510 2024-09-10T10:04:10.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25e3e00640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f25cc067cb0 0x7f25cc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:10.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25e3e00640 1 -- 172.21.15.110:0/593402144 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f25d8086860 con 0x7f25ec0cd510 2024-09-10T10:04:10.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.714+0000 7f25ebe00640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f25cc067cb0 0x7f25cc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:10.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.718+0000 7f25ebe00640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f25cc067cb0 0x7f25cc06a170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0ae000 tx=0x7f25dc0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:10.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.718+0000 7f25e3e00640 1 -- 172.21.15.110:0/593402144 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f25d8056df0 con 0x7f25ec0cd510 2024-09-10T10:04:10.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.906+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol47570", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f25ec0719c0 con 0x7f25cc067cb0 2024-09-10T10:04:10.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.954+0000 7f25e3e00640 1 -- 172.21.15.110:0/593402144 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25ec0719c0 con 0x7f25cc067cb0 2024-09-10T10:04:10.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f25cc067cb0 msgr2=0x7f25cc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f25cc067cb0 0x7f25cc06a170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0ae000 tx=0x7f25dc0a7070 comp rx=0 tx=0).stop 2024-09-10T10:04:10.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cd510 msgr2=0x7f25ec0aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cd510 0x7f25ec0aa430 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f25d800d800 tx=0x7f25d800dcc0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 shutdown_connections 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f25cc067cb0 0x7f25cc06a170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0cfe30 0x7f25ec0aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25ec0cd510 0x7f25ec0aa430 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 --2- 172.21.15.110:0/593402144 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f25ec06e090 0x7f25ec0ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 >> 172.21.15.110:0/593402144 conn(0x7f25ec06bc50 msgr2=0x7f25ec0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 shutdown_connections 2024-09-10T10:04:10.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:10.958+0000 7f25f3000640 1 -- 172.21.15.110:0/593402144 wait complete. 2024-09-10T10:04:10.980 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol65511 fake subvol_grp4440 2024-09-10T10:04:11.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c68800640 1 Processor -- start 2024-09-10T10:04:11.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c68800640 1 -- start start 2024-09-10T10:04:11.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c68800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6c6015c560 0x7f6c6006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c68800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c6006f430 con 0x7f6c60155970 2024-09-10T10:04:11.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c68800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c60155d50 con 0x7f6c6006f010 2024-09-10T10:04:11.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c68800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6c600b4570 con 0x7f6c6015c560 2024-09-10T10:04:11.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c65e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6c60155970 0x7f6c600cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57890/0 (socket says 172.21.15.110:57890) 2024-09-10T10:04:11.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c65e00640 1 -- 172.21.15.110:0/3945395932 learned_addr learned my addr 172.21.15.110:0/3945395932 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c65400640 1 --2- 172.21.15.110:0/3945395932 >> v2:172.21.15.145:3300/0 conn(0x7f6c6015c560 0x7f6c6006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1411650384 0 0) 0x7f6c6006f430 con 0x7f6c60155970 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c48003530 con 0x7f6c60155970 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386392676 0 0) 0x7f6c60155d50 con 0x7f6c6006f010 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c6006f430 con 0x7f6c6006f010 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3324424446 0 0) 0x7f6c48003530 con 0x7f6c60155970 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 >> v2:172.21.15.145:3300/0 conn(0x7f6c6015c560 msgr2=0x7f6c6006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c64a00640 1 --2- 172.21.15.110:0/3945395932 >> v2:172.21.15.145:3300/0 conn(0x7f6c6015c560 0x7f6c6006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.118+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 >> v1:172.21.15.145:6789/0 conn(0x7f6c6006f010 legacy=0x7f6c600ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c600b34d0 con 0x7f6c60155970 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f6c54001680 con 0x7f6c60155970 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 >> v1:172.21.15.110:6789/0 conn(0x7f6c60155970 legacy=0x7f6c600cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c48003ba0 0x7f6c48003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48004810 0x7f6c48004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c480051d0 0x7f6c4800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4800e2b0 con 0x7f6c48004810 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4800e430 con 0x7f6c48003ba0 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4800e630 con 0x7f6c480051d0 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65400640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48004810 0x7f6c48004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65e00640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c48003ba0 0x7f6c48003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c66800640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c480051d0 0x7f6c4800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65400640 1 -- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c480051d0 msgr2=0x7f6c4800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:11.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65400640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c480051d0 0x7f6c4800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65400640 1 -- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c48003ba0 msgr2=0x7f6c48003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:11.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65400640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c48003ba0 0x7f6c48003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65400640 1 -- 172.21.15.110:0/3945395932 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c48003530 con 0x7f6c48004810 2024-09-10T10:04:11.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c66800640 1 -- 172.21.15.110:0/3945395932 reap_dead start 2024-09-10T10:04:11.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65e00640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c48003ba0 0x7f6c48003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:11.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c65400640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48004810 0x7f6c48004c90 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009ae0 tx=0x7f6c5001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432cf7506d8eb8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:11.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500b2070 con 0x7f6c48004810 2024-09-10T10:04:11.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c500a5680 con 0x7f6c48004810 2024-09-10T10:04:11.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.122+0000 7f6c64a00640 1 -- 172.21.15.110:0/3945395932 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500096b0 con 0x7f6c48004810 2024-09-10T10:04:11.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- 172.21.15.110:0/3945395932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48004810 msgr2=0x7f6c48004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48004810 0x7f6c48004c90 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009ae0 tx=0x7f6c5001ca90 comp rx=0 tx=0).stop 2024-09-10T10:04:11.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- 172.21.15.110:0/3945395932 shutdown_connections 2024-09-10T10:04:11.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 --2- 172.21.15.110:0/3945395932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48004810 0x7f6c48004c90 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- 172.21.15.110:0/3945395932 >> 172.21.15.110:0/3945395932 conn(0x7f6c6006bc50 msgr2=0x7f6c6006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:11.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- 172.21.15.110:0/3945395932 shutdown_connections 2024-09-10T10:04:11.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- 172.21.15.110:0/3945395932 wait complete. 2024-09-10T10:04:11.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 Processor -- start 2024-09-10T10:04:11.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- start start 2024-09-10T10:04:11.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c48004810 0x7f6c6015e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c60153790 0x7f6c600ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c600aed80 0x7f6c6015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c600b3d70 con 0x7f6c60153790 2024-09-10T10:04:11.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c600b3a90 con 0x7f6c600aed80 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c68800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6c600b4050 con 0x7f6c48004810 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c60153790 0x7f6c600ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c60153790 0x7f6c600ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48428/0 (socket says 172.21.15.110:48428) 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65400640 1 -- 172.21.15.110:0/2403071664 learned_addr learned my addr 172.21.15.110:0/2403071664 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65e00640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c48004810 0x7f6c6015e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c66800640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c600aed80 0x7f6c6015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65400640 1 -- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c48004810 msgr2=0x7f6c6015e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65400640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c48004810 0x7f6c6015e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65400640 1 -- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c600aed80 msgr2=0x7f6c6015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65400640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c600aed80 0x7f6c6015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.126+0000 7f6c65400640 1 -- 172.21.15.110:0/2403071664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6015d630 con 0x7f6c60153790 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c65400640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c60153790 0x7f6c600ae840 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6c5001c8c0 tx=0x7f6c50019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:11.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c5a000640 1 -- 172.21.15.110:0/2403071664 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500b2040 con 0x7f6c60153790 2024-09-10T10:04:11.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c5a000640 1 -- 172.21.15.110:0/2403071664 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c50004020 con 0x7f6c60153790 2024-09-10T10:04:11.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c5a000640 1 -- 172.21.15.110:0/2403071664 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50004410 con 0x7f6c60153790 2024-09-10T10:04:11.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6015d910 con 0x7f6c60153790 2024-09-10T10:04:11.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c6015ded0 con 0x7f6c60153790 2024-09-10T10:04:11.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c6006c240 con 0x7f6c60153790 2024-09-10T10:04:11.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c5a000640 1 -- 172.21.15.110:0/2403071664 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f6c500bf030 con 0x7f6c60153790 2024-09-10T10:04:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c5a000640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c44067cb0 0x7f6c4406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c5a000640 1 -- 172.21.15.110:0/2403071664 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f6c50120190 con 0x7f6c60153790 2024-09-10T10:04:11.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.130+0000 7f6c65e00640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c44067cb0 0x7f6c4406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.134+0000 7f6c65e00640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c44067cb0 0x7f6c4406a170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c54001040 tx=0x7f6c54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:11.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.134+0000 7f6c5a000640 1 -- 172.21.15.110:0/2403071664 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6c500c6060 con 0x7f6c60153790 2024-09-10T10:04:11.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.322+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol65511", "snap_name": "fake", "group_name": "subvol_grp4440", "target": ["mon-mgr", ""]}) -- 0x7f6c600af160 con 0x7f6c44067cb0 2024-09-10T10:04:11.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c5a000640 1 -- 172.21.15.110:0/2403071664 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c600af160 con 0x7f6c44067cb0 2024-09-10T10:04:11.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c44067cb0 msgr2=0x7f6c4406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c44067cb0 0x7f6c4406a170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c54001040 tx=0x7f6c54009040 comp rx=0 tx=0).stop 2024-09-10T10:04:11.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c60153790 msgr2=0x7f6c600ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c60153790 0x7f6c600ae840 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6c5001c8c0 tx=0x7f6c50019830 comp rx=0 tx=0).stop 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 shutdown_connections 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c44067cb0 0x7f6c4406a170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c600aed80 0x7f6c6015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c60153790 0x7f6c600ae840 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 --2- 172.21.15.110:0/2403071664 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c48004810 0x7f6c6015e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 >> 172.21.15.110:0/2403071664 conn(0x7f6c6006bc50 msgr2=0x7f6c600b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 shutdown_connections 2024-09-10T10:04:11.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.366+0000 7f6c68800640 1 -- 172.21.15.110:0/2403071664 wait complete. 2024-09-10T10:04:11.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:11.391 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol47570/.meta) 2024-09-10T10:04:11.412 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:04:11.412 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T10:04:11.412 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:04:11.412 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/_nogroup/subvol47570/e8b6cc38-9278-4293-af4d-37db63293428 2024-09-10T10:04:11.412 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:04:11.412 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:11.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:11.415 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp4440/subvol65511/.meta) 2024-09-10T10:04:11.480 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:04:11.480 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T10:04:11.480 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:04:11.480 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/subvol_grp4440/subvol65511/33198619-5600-4c43-8499-ac39bcb647be 2024-09-10T10:04:11.480 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:04:11.480 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:11.482 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol47570 2024-09-10T10:04:11.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a11800640 1 Processor -- start 2024-09-10T10:04:11.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a11800640 1 -- start start 2024-09-10T10:04:11.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a11800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1a0c145f30 0x7f1a0c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a11800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c0b37c0 con 0x7f1a0c0b29e0 2024-09-10T10:04:11.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a11800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c0b3aa0 con 0x7f1a0c0cfd90 2024-09-10T10:04:11.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a11800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1a0c0b2e50 con 0x7f1a0c145f30 2024-09-10T10:04:11.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a0a000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1a0c0b29e0 0x7f1a0c0cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57896/0 (socket says 172.21.15.110:57896) 2024-09-10T10:04:11.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a0a000640 1 -- 172.21.15.110:0/2352936717 learned_addr learned my addr 172.21.15.110:0/2352936717 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a0aa00640 1 --2- 172.21.15.110:0/2352936717 >> v2:172.21.15.145:3300/0 conn(0x7f1a0c145f30 0x7f1a0c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2863818792 0 0) 0x7f1a0c0b37c0 con 0x7f1a0c0b29e0 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19f0003530 con 0x7f1a0c0b29e0 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 313435225 0 0) 0x7f19f0003530 con 0x7f1a0c0b29e0 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 >> v2:172.21.15.145:3300/0 conn(0x7f1a0c145f30 msgr2=0x7f1a0c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 --2- 172.21.15.110:0/2352936717 >> v2:172.21.15.145:3300/0 conn(0x7f1a0c145f30 0x7f1a0c0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 >> v1:172.21.15.145:6789/0 conn(0x7f1a0c0cfd90 legacy=0x7f1a0c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c0b3d80 con 0x7f1a0c0b29e0 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f19f80016a0 con 0x7f1a0c0b29e0 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 >> v1:172.21.15.110:6789/0 conn(0x7f1a0c0b29e0 legacy=0x7f1a0c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f19f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f0004810 0x7f19f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00051d0 0x7f19f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b2e50 con 0x7f19f00039c0 2024-09-10T10:04:11.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f19f000c070 con 0x7f19f0004810 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f19f000c270 con 0x7f19f00051d0 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a0aa00640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f19f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a0a000640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f0004810 0x7f19f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.658+0000 7f1a0b400640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00051d0 0x7f19f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a0aa00640 1 -- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00051d0 msgr2=0x7f19f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a0aa00640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00051d0 0x7f19f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a0aa00640 1 -- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f0004810 msgr2=0x7f19f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a0aa00640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f0004810 0x7f19f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a0aa00640 1 -- 172.21.15.110:0/2352936717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f0003530 con 0x7f19f00039c0 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a0a000640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f0004810 0x7f19f0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:11.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a0b400640 1 -- 172.21.15.110:0/2352936717 reap_dead start 2024-09-10T10:04:11.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a0aa00640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f19f0003de0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009ae0 tx=0x7f19fc01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7aac1e438976743 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:11.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0096b0 con 0x7f19f00039c0 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc0b0070 con 0x7f19f00039c0 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a09600640 1 -- 172.21.15.110:0/2352936717 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0ab6c0 con 0x7f19f00039c0 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a11800640 1 -- 172.21.15.110:0/2352936717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 msgr2=0x7f19f0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a11800640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f19f0003de0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009ae0 tx=0x7f19fc01ca90 comp rx=0 tx=0).stop 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a11800640 1 -- 172.21.15.110:0/2352936717 shutdown_connections 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a11800640 1 --2- 172.21.15.110:0/2352936717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f19f0003de0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a11800640 1 -- 172.21.15.110:0/2352936717 >> 172.21.15.110:0/2352936717 conn(0x7f1a0c062e40 msgr2=0x7f1a0c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a11800640 1 -- 172.21.15.110:0/2352936717 shutdown_connections 2024-09-10T10:04:11.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.662+0000 7f1a11800640 1 -- 172.21.15.110:0/2352936717 wait complete. 2024-09-10T10:04:11.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 Processor -- start 2024-09-10T10:04:11.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 -- start start 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f1a0c1f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a0c15bf20 0x7f1a0c0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a0c0aed80 0x7f1a0c13dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c160a50 con 0x7f19f00039c0 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0d0170 con 0x7f1a0c15bf20 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c160bd0 con 0x7f1a0c0aed80 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f1a0c1f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f1a0c1f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48436/0 (socket says 172.21.15.110:48436) 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 -- 172.21.15.110:0/2332263557 learned_addr learned my addr 172.21.15.110:0/2332263557 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0b400640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a0c0aed80 0x7f1a0c13dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0a000640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a0c15bf20 0x7f1a0c0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 -- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a0c0aed80 msgr2=0x7f1a0c13dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a0c0aed80 0x7f1a0c13dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 -- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a0c15bf20 msgr2=0x7f1a0c0ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a0c15bf20 0x7f1a0c0ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 -- 172.21.15.110:0/2332263557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c1f8bf0 con 0x7f19f00039c0 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0a000640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a0c15bf20 0x7f1a0c0ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0b400640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a0c0aed80 0x7f1a0c13dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a0aa00640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f1a0c1f86b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0ad040 tx=0x7f19f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a02a00640 1 -- 172.21.15.110:0/2332263557 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0c1070 con 0x7f19f00039c0 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a02a00640 1 -- 172.21.15.110:0/2332263557 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc0a7070 con 0x7f19f00039c0 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c140480 con 0x7f19f00039c0 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.666+0000 7f1a02a00640 1 -- 172.21.15.110:0/2332263557 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc006240 con 0x7f19f00039c0 2024-09-10T10:04:11.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.670+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c140a40 con 0x7f19f00039c0 2024-09-10T10:04:11.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.670+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0c0b4560 con 0x7f19f00039c0 2024-09-10T10:04:11.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.670+0000 7f1a02a00640 1 -- 172.21.15.110:0/2332263557 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 87802+0+0 (secure 0 0 0) 0x7f19fc0c4070 con 0x7f19f00039c0 2024-09-10T10:04:11.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.670+0000 7f1a02a00640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19ec067ce0 0x7f19ec06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.670+0000 7f1a02a00640 1 -- 172.21.15.110:0/2332263557 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f19fc120610 con 0x7f19f00039c0 2024-09-10T10:04:11.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.670+0000 7f1a0a000640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19ec067ce0 0x7f19ec06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.670+0000 7f1a0a000640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19ec067ce0 0x7f19ec06a1a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f19f8000c10 tx=0x7f19f8001e20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:11.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.674+0000 7f1a02a00640 1 -- 172.21.15.110:0/2332263557 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f19fc0167a0 con 0x7f19f00039c0 2024-09-10T10:04:11.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.782+0000 7f1a02a00640 1 -- 172.21.15.110:0/2332263557 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 19) ==== 87853+0+0 (secure 0 0 0) 0x7f19fc0ecba0 con 0x7f19f00039c0 2024-09-10T10:04:11.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.862+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol47570", "target": ["mon-mgr", ""]}) -- 0x7f1a0c071470 con 0x7f19ec067ce0 2024-09-10T10:04:11.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.898+0000 7f1a02a00640 1 -- 172.21.15.110:0/2332263557 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1a0c071470 con 0x7f19ec067ce0 2024-09-10T10:04:11.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19ec067ce0 msgr2=0x7f19ec06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19ec067ce0 0x7f19ec06a1a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f19f8000c10 tx=0x7f19f8001e20 comp rx=0 tx=0).stop 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 msgr2=0x7f1a0c1f86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f1a0c1f86b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0ad040 tx=0x7f19f0005840 comp rx=0 tx=0).stop 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 shutdown_connections 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19ec067ce0 0x7f19ec06a1a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1a0c0aed80 0x7f1a0c13dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1a0c15bf20 0x7f1a0c0ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 --2- 172.21.15.110:0/2332263557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00039c0 0x7f1a0c1f86b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 >> 172.21.15.110:0/2332263557 conn(0x7f1a0c062e40 msgr2=0x7f1a0c15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:11.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 shutdown_connections 2024-09-10T10:04:11.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:11.902+0000 7f1a11800640 1 -- 172.21.15.110:0/2332263557 wait complete. 2024-09-10T10:04:11.909 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.906+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.909 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.906+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.909 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.906+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.909 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.906+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.909 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.906+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.925 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.922+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.925 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.922+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.926 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.922+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.926 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.922+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.926 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:11.922+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:11.927 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol65511 subvol_grp4440 2024-09-10T10:04:12.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.078+0000 7f8164e00640 1 Processor -- start 2024-09-10T10:04:12.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.078+0000 7f8164e00640 1 -- start start 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.078+0000 7f8164e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f816015e700 0x7f816006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.078+0000 7f8164e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f816006ff60 0x7f81600af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.078+0000 7f8164e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81600703a0 con 0x7f81600729b0 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.078+0000 7f8164e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f816015eae0 con 0x7f816015e700 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.078+0000 7f8164e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f816006eec0 con 0x7f816006ff60 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f816015e700 0x7f816006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f816015e700 0x7f816006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48450/0 (socket says 172.21.15.110:48450) 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815de00640 1 -- 172.21.15.110:0/2894790955 learned_addr learned my addr 172.21.15.110:0/2894790955 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:12.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815d400640 1 --2- 172.21.15.110:0/2894790955 >> v2:172.21.15.145:3300/0 conn(0x7f816006ff60 0x7f81600af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815de00640 1 -- 172.21.15.110:0/2894790955 >> v2:172.21.15.145:3300/0 conn(0x7f816006ff60 msgr2=0x7f81600af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:12.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815de00640 1 --2- 172.21.15.110:0/2894790955 >> v2:172.21.15.145:3300/0 conn(0x7f816006ff60 0x7f81600af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815de00640 1 -- 172.21.15.110:0/2894790955 >> v1:172.21.15.145:6789/0 conn(0x7f81600729b0 legacy=0x7f8160072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:12.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815de00640 1 -- 172.21.15.110:0/2894790955 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816006f1a0 con 0x7f816015e700 2024-09-10T10:04:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815de00640 1 --2- 172.21.15.110:0/2894790955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f816015e700 0x7f816006e890 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f816006e040 tx=0x7f815001c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6184b49998c86d25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815ca00640 1 -- 172.21.15.110:0/2894790955 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500b0070 con 0x7f816015e700 2024-09-10T10:04:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815ca00640 1 -- 172.21.15.110:0/2894790955 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81500ab600 con 0x7f816015e700 2024-09-10T10:04:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.082+0000 7f815ca00640 1 -- 172.21.15.110:0/2894790955 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500ab9d0 con 0x7f816015e700 2024-09-10T10:04:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- 172.21.15.110:0/2894790955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f816015e700 msgr2=0x7f816006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 --2- 172.21.15.110:0/2894790955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f816015e700 0x7f816006e890 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f816006e040 tx=0x7f815001c2d0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- 172.21.15.110:0/2894790955 shutdown_connections 2024-09-10T10:04:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 --2- 172.21.15.110:0/2894790955 >> v2:172.21.15.145:3300/0 conn(0x7f816006ff60 0x7f81600af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 --2- 172.21.15.110:0/2894790955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f816015e700 0x7f816006e890 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- 172.21.15.110:0/2894790955 >> 172.21.15.110:0/2894790955 conn(0x7f816006cb90 msgr2=0x7f816006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- 172.21.15.110:0/2894790955 shutdown_connections 2024-09-10T10:04:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- 172.21.15.110:0/2894790955 wait complete. 2024-09-10T10:04:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 Processor -- start 2024-09-10T10:04:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- start start 2024-09-10T10:04:12.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f816006ff60 0x7f816013c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81600729b0 0x7f81600b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f816015e700 0x7f816014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81600af930 con 0x7f81600729b0 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81600af7b0 con 0x7f816015e700 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f8164e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81600b4660 con 0x7f816006ff60 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81600729b0 0x7f81600b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81600729b0 0x7f81600b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48462/0 (socket says 172.21.15.110:48462) 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815d400640 1 -- 172.21.15.110:0/3616340371 learned_addr learned my addr 172.21.15.110:0/3616340371 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815de00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f816006ff60 0x7f816013c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815e800640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f816015e700 0x7f816014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815d400640 1 -- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f816006ff60 msgr2=0x7f816013c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815d400640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f816006ff60 0x7f816013c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815d400640 1 -- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f816015e700 msgr2=0x7f816014d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815d400640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f816015e700 0x7f816014d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.086+0000 7f815d400640 1 -- 172.21.15.110:0/3616340371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816014fbe0 con 0x7f81600729b0 2024-09-10T10:04:12.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f815d400640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81600729b0 0x7f81600b5590 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d800 tx=0x7f814c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8156000640 1 -- 172.21.15.110:0/3616340371 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c00b660 con 0x7f81600729b0 2024-09-10T10:04:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8156000640 1 -- 172.21.15.110:0/3616340371 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f814c019070 con 0x7f81600729b0 2024-09-10T10:04:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8156000640 1 -- 172.21.15.110:0/3616340371 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c00ba00 con 0x7f81600729b0 2024-09-10T10:04:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81600d01c0 con 0x7f81600729b0 2024-09-10T10:04:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81600cda40 con 0x7f81600729b0 2024-09-10T10:04:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81600b51b0 con 0x7f81600729b0 2024-09-10T10:04:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8156000640 1 -- 172.21.15.110:0/3616340371 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) ==== 87853+0+0 (secure 0 0 0) 0x7f814c00bba0 con 0x7f81600729b0 2024-09-10T10:04:12.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8156000640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8140067d40 0x7f814006a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f8156000640 1 -- 172.21.15.110:0/3616340371 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f814c0866b0 con 0x7f81600729b0 2024-09-10T10:04:12.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.090+0000 7f815de00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8140067d40 0x7f814006a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.094+0000 7f815de00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8140067d40 0x7f814006a200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f81500ad000 tx=0x7f81500a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.098+0000 7f8156000640 1 -- 172.21.15.110:0/3616340371 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f814c056610 con 0x7f81600729b0 2024-09-10T10:04:12.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.278+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol65511", "group_name": "subvol_grp4440", "target": ["mon-mgr", ""]}) -- 0x7f81600b5ad0 con 0x7f8140067d40 2024-09-10T10:04:12.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8156000640 1 -- 172.21.15.110:0/3616340371 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f81600b5ad0 con 0x7f8140067d40 2024-09-10T10:04:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8140067d40 msgr2=0x7f814006a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8140067d40 0x7f814006a200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f81500ad000 tx=0x7f81500a6000 comp rx=0 tx=0).stop 2024-09-10T10:04:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81600729b0 msgr2=0x7f81600b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81600729b0 0x7f81600b5590 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d800 tx=0x7f814c00dcc0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 shutdown_connections 2024-09-10T10:04:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8140067d40 0x7f814006a200 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f816015e700 0x7f816014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81600729b0 0x7f81600b5590 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 --2- 172.21.15.110:0/3616340371 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f816006ff60 0x7f816013c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 >> 172.21.15.110:0/3616340371 conn(0x7f816006cb90 msgr2=0x7f81600b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:12.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 shutdown_connections 2024-09-10T10:04:12.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.322+0000 7f8164e00640 1 -- 172.21.15.110:0/3616340371 wait complete. 2024-09-10T10:04:12.349 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol78472 2024-09-10T10:04:12.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.482+0000 7fbc49000640 1 Processor -- start 2024-09-10T10:04:12.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.482+0000 7fbc49000640 1 -- start start 2024-09-10T10:04:12.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc49000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc44145f30 0x7fbc440b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc49000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440b29e0 0x7fbc440cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc49000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc440b37c0 con 0x7fbc440cfbc0 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc49000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbc440b2e20 con 0x7fbc44145f30 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc49000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbc440b3aa0 con 0x7fbc440b29e0 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc41e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc44145f30 0x7fbc440b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc41e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc44145f30 0x7fbc440b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48476/0 (socket says 172.21.15.110:48476) 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc41e00640 1 -- 172.21.15.110:0/3087458809 learned_addr learned my addr 172.21.15.110:0/3087458809 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc41400640 1 --2- 172.21.15.110:0/3087458809 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440b29e0 0x7fbc440cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc41e00640 1 -- 172.21.15.110:0/3087458809 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440b29e0 msgr2=0x7fbc440cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:12.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc41e00640 1 --2- 172.21.15.110:0/3087458809 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440b29e0 0x7fbc440cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc41e00640 1 -- 172.21.15.110:0/3087458809 >> v1:172.21.15.145:6789/0 conn(0x7fbc440cfbc0 legacy=0x7fbc44160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.486+0000 7fbc41e00640 1 -- 172.21.15.110:0/3087458809 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc440b3d80 con 0x7fbc44145f30 2024-09-10T10:04:12.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc41e00640 1 --2- 172.21.15.110:0/3087458809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44145f30 0x7fbc440b24a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009e50 tx=0x7fbc3401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f421770831998e83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc40a00640 1 -- 172.21.15.110:0/3087458809 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc340a5700 con 0x7fbc44145f30 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc40a00640 1 -- 172.21.15.110:0/3087458809 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc340a58a0 con 0x7fbc44145f30 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc40a00640 1 -- 172.21.15.110:0/3087458809 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc340a5c70 con 0x7fbc44145f30 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- 172.21.15.110:0/3087458809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44145f30 msgr2=0x7fbc440b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 --2- 172.21.15.110:0/3087458809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44145f30 0x7fbc440b24a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009e50 tx=0x7fbc3401ca30 comp rx=0 tx=0).stop 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- 172.21.15.110:0/3087458809 shutdown_connections 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 --2- 172.21.15.110:0/3087458809 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440b29e0 0x7fbc440cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 --2- 172.21.15.110:0/3087458809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44145f30 0x7fbc440b24a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- 172.21.15.110:0/3087458809 >> 172.21.15.110:0/3087458809 conn(0x7fbc44062e40 msgr2=0x7fbc4406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:12.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- 172.21.15.110:0/3087458809 shutdown_connections 2024-09-10T10:04:12.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- 172.21.15.110:0/3087458809 wait complete. 2024-09-10T10:04:12.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 Processor -- start 2024-09-10T10:04:12.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- start start 2024-09-10T10:04:12.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc440b29e0 0x7fbc4413e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440cfbc0 0x7fbc440ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44145f30 0x7fbc440aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc44169530 con 0x7fbc44145f30 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441693b0 con 0x7fbc440b29e0 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.490+0000 7fbc49000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbc441696b0 con 0x7fbc440cfbc0 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc440b29e0 0x7fbc4413e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440cfbc0 0x7fbc440ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc440b29e0 0x7fbc4413e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:54604/0 (socket says 172.21.15.110:54604) 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 -- 172.21.15.110:0/1775281811 learned_addr learned my addr 172.21.15.110:0/1775281811 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc42800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44145f30 0x7fbc440aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 -- 172.21.15.110:0/1775281811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440cfbc0 msgr2=0x7fbc440ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 --2- 172.21.15.110:0/1775281811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc440cfbc0 0x7fbc440ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 -- 172.21.15.110:0/1775281811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44145f30 msgr2=0x7fbc440aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 --2- 172.21.15.110:0/1775281811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44145f30 0x7fbc440aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 -- 172.21.15.110:0/1775281811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4413e960 con 0x7fbc440b29e0 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc41e00640 1 --2- 172.21.15.110:0/1775281811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc440b29e0 0x7fbc4413e0f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbc3401a660 tx=0x7fbc340197d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc3a000640 1 -- 172.21.15.110:0/1775281811 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34009510 con 0x7fbc440b29e0 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc3a000640 1 -- 172.21.15.110:0/1775281811 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc340b8470 con 0x7fbc440b29e0 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc49000640 1 -- 172.21.15.110:0/1775281811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4413ec40 con 0x7fbc440b29e0 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc3a000640 1 -- 172.21.15.110:0/1775281811 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc340b8860 con 0x7fbc440b29e0 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc49000640 1 -- 172.21.15.110:0/1775281811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc440b1970 con 0x7fbc440b29e0 2024-09-10T10:04:12.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc49000640 1 -- 172.21.15.110:0/1775281811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc440cf680 con 0x7fbc440b29e0 2024-09-10T10:04:12.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc3a000640 1 -- 172.21.15.110:0/1775281811 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 19) ==== 87853+0+0 (secure 0 0 0) 0x7fbc340c0020 con 0x7fbc440b29e0 2024-09-10T10:04:12.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.494+0000 7fbc3a000640 1 --2- 172.21.15.110:0/1775281811 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc24067d40 0x7fbc2406a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.498+0000 7fbc3a000640 1 -- 172.21.15.110:0/1775281811 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7fbc34120b30 con 0x7fbc440b29e0 2024-09-10T10:04:12.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.498+0000 7fbc41400640 1 --2- 172.21.15.110:0/1775281811 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc24067d40 0x7fbc2406a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.498+0000 7fbc41400640 1 --2- 172.21.15.110:0/1775281811 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc24067d40 0x7fbc2406a200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbc4415f610 tx=0x7fbc30009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.502+0000 7fbc3a000640 1 -- 172.21.15.110:0/1775281811 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbc340c7020 con 0x7fbc440b29e0 2024-09-10T10:04:12.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.686+0000 7fbc49000640 1 -- 172.21.15.110:0/1775281811 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol78472", "target": ["mon-mgr", ""]}) -- 0x7fbc441426c0 con 0x7fbc24067d40 2024-09-10T10:04:12.713 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:12.714+0000 7f95dda00640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol78472' is not ready for operation rm (use --force to override) 2024-09-10T10:04:12.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.710+0000 7fbc3a000640 1 -- 172.21.15.110:0/1775281811 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol78472' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7fbc441426c0 con 0x7fbc24067d40 2024-09-10T10:04:12.714 INFO:teuthology.orchestra.run.smithi110.stderr:Error EAGAIN: subvolume 'subvol78472' is not ready for operation rm (use --force to override) 2024-09-10T10:04:12.723 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-10T10:04:12.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:12.724 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol78472/.meta) 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:type = clone 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/_nogroup/subvol78472/86bf95ec-8594-4f0d-8f53-5c3e527725a0 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:state = pending 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:[source] 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:volume = fake 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:subvolume = fake 2024-09-10T10:04:12.745 INFO:teuthology.orchestra.run.smithi110.stdout:snapshot = fake 2024-09-10T10:04:12.747 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol78472 --force 2024-09-10T10:04:12.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f060b800640 1 Processor -- start 2024-09-10T10:04:12.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f060b800640 1 -- start start 2024-09-10T10:04:12.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f060b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cfe30 0x7f0604071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f060b800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f06040cd510 0x7f06040cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f060b800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060406d2d0 con 0x7f060406e090 2024-09-10T10:04:12.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f060b800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f06040cdeb0 con 0x7f06040cd510 2024-09-10T10:04:12.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f060b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f060415d9a0 con 0x7f06040cfe30 2024-09-10T10:04:12.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f0603e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f06040cd510 0x7f06040cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f0608e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cfe30 0x7f0604071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.922+0000 7f0609800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f060406e090 0x7f060406e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:56542/0 (socket says 172.21.15.110:56542) 2024-09-10T10:04:12.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0609800640 1 -- 172.21.15.110:0/260376631 learned_addr learned my addr 172.21.15.110:0/260376631 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:12.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0608e00640 1 -- 172.21.15.110:0/260376631 >> v2:172.21.15.145:3300/0 conn(0x7f06040cd510 msgr2=0x7f06040cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:12.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0608e00640 1 --2- 172.21.15.110:0/260376631 >> v2:172.21.15.145:3300/0 conn(0x7f06040cd510 0x7f06040cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0608e00640 1 -- 172.21.15.110:0/260376631 >> v1:172.21.15.145:6789/0 conn(0x7f060406e090 legacy=0x7f060406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0608e00640 1 -- 172.21.15.110:0/260376631 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060406d5b0 con 0x7f06040cfe30 2024-09-10T10:04:12.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0608e00640 1 --2- 172.21.15.110:0/260376631 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cfe30 0x7f0604071a40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009d90 tx=0x7f05f401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af9a2e52a85fb2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0603400640 1 -- 172.21.15.110:0/260376631 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f40b2070 con 0x7f06040cfe30 2024-09-10T10:04:12.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0603400640 1 -- 172.21.15.110:0/260376631 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f40a58b0 con 0x7f06040cfe30 2024-09-10T10:04:12.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.926+0000 7f0603400640 1 -- 172.21.15.110:0/260376631 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f40a5c80 con 0x7f06040cfe30 2024-09-10T10:04:12.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- 172.21.15.110:0/260376631 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cfe30 msgr2=0x7f0604071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 --2- 172.21.15.110:0/260376631 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cfe30 0x7f0604071a40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009d90 tx=0x7f05f401c8e0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- 172.21.15.110:0/260376631 shutdown_connections 2024-09-10T10:04:12.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 --2- 172.21.15.110:0/260376631 >> v2:172.21.15.145:3300/0 conn(0x7f06040cd510 0x7f06040cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 --2- 172.21.15.110:0/260376631 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cfe30 0x7f0604071a40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- 172.21.15.110:0/260376631 >> 172.21.15.110:0/260376631 conn(0x7f060406bc50 msgr2=0x7f060406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:12.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- 172.21.15.110:0/260376631 shutdown_connections 2024-09-10T10:04:12.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- 172.21.15.110:0/260376631 wait complete. 2024-09-10T10:04:12.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 Processor -- start 2024-09-10T10:04:12.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- start start 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060406e090 0x7f06041530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cd510 0x7f0604150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06040cfe30 0x7f06040ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0604072190 con 0x7f060406e090 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f060415d640 con 0x7f06040cfe30 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f060b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0604169180 con 0x7f06040cd510 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060406e090 0x7f06041530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060406e090 0x7f06041530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48486/0 (socket says 172.21.15.110:48486) 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 -- 172.21.15.110:0/2917065293 learned_addr learned my addr 172.21.15.110:0/2917065293 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0603e00640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cd510 0x7f0604150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0609800640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06040cfe30 0x7f06040ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 -- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cd510 msgr2=0x7f0604150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cd510 0x7f0604150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 -- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06040cfe30 msgr2=0x7f06040ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06040cfe30 0x7f06040ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 -- 172.21.15.110:0/2917065293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0604071ae0 con 0x7f060406e090 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0609800640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06040cfe30 0x7f06040ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.930+0000 7f0608e00640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060406e090 0x7f06041530f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f05f40b0040 tx=0x7f05f4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f0600c00640 1 -- 172.21.15.110:0/2917065293 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f40b2050 con 0x7f060406e090 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f0600c00640 1 -- 172.21.15.110:0/2917065293 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f40be070 con 0x7f060406e090 2024-09-10T10:04:12.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f0600c00640 1 -- 172.21.15.110:0/2917065293 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f40ada50 con 0x7f060406e090 2024-09-10T10:04:12.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0604157c70 con 0x7f060406e090 2024-09-10T10:04:12.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0604158230 con 0x7f060406e090 2024-09-10T10:04:12.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06040cd970 con 0x7f060406e090 2024-09-10T10:04:12.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f0600c00640 1 -- 172.21.15.110:0/2917065293 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) ==== 87853+0+0 (secure 0 0 0) 0x7f05f40a5730 con 0x7f060406e090 2024-09-10T10:04:12.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f0600c00640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05e4067d40 0x7f05e406a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:12.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f0600c00640 1 -- 172.21.15.110:0/2917065293 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f05f41213c0 con 0x7f060406e090 2024-09-10T10:04:12.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f0603e00640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05e4067d40 0x7f05e406a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:12.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.934+0000 7f0603e00640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05e4067d40 0x7f05e406a200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0604153040 tx=0x7f05f0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:12.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:12.938+0000 7f0600c00640 1 -- 172.21.15.110:0/2917065293 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f05f4016560 con 0x7f060406e090 2024-09-10T10:04:13.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.118+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol78472", "force": true, "target": ["mon-mgr", ""]}) -- 0x7f0604075560 con 0x7f05e4067d40 2024-09-10T10:04:13.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.146+0000 7f0600c00640 1 -- 172.21.15.110:0/2917065293 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0604075560 con 0x7f05e4067d40 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05e4067d40 msgr2=0x7f05e406a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05e4067d40 0x7f05e406a200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0604153040 tx=0x7f05f0009190 comp rx=0 tx=0).stop 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060406e090 msgr2=0x7f06041530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060406e090 0x7f06041530f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f05f40b0040 tx=0x7f05f4019680 comp rx=0 tx=0).stop 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 shutdown_connections 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f05e4067d40 0x7f05e406a200 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06040cfe30 0x7f06040ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f06040cd510 0x7f0604150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 --2- 172.21.15.110:0/2917065293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060406e090 0x7f06041530f0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 >> 172.21.15.110:0/2917065293 conn(0x7f060406bc50 msgr2=0x7f060415a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 shutdown_connections 2024-09-10T10:04:13.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.150+0000 7f060b800640 1 -- 172.21.15.110:0/2917065293 wait complete. 2024-09-10T10:04:13.174 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-10T10:04:13.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.306+0000 7fd089600640 1 Processor -- start 2024-09-10T10:04:13.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.306+0000 7fd089600640 1 -- start start 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd089600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd084145f30 0x7fd0840b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd089600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd0840cfd90 0x7fd0840b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd089600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0840b37c0 con 0x7fd0840b29e0 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd089600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd0840b2e50 con 0x7fd0840cfd90 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd089600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd0840d0170 con 0x7fd084145f30 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd082000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fd0840b29e0 0x7fd0840cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57904/0 (socket says 172.21.15.110:57904) 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd082000640 1 -- 172.21.15.110:0/3465608855 learned_addr learned my addr 172.21.15.110:0/3465608855 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd082a00640 1 --2- 172.21.15.110:0/3465608855 >> v2:172.21.15.145:3300/0 conn(0x7fd084145f30 0x7fd0840b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd083400640 1 --2- 172.21.15.110:0/3465608855 >> v2:172.21.15.110:3300/0 conn(0x7fd0840cfd90 0x7fd0840b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd081600640 1 -- 172.21.15.110:0/3465608855 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2174567176 0 0) 0x7fd0840b37c0 con 0x7fd0840b29e0 2024-09-10T10:04:13.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd081600640 1 -- 172.21.15.110:0/3465608855 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd068003530 con 0x7fd0840b29e0 2024-09-10T10:04:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd082a00640 1 -- 172.21.15.110:0/3465608855 >> v2:172.21.15.110:3300/0 conn(0x7fd0840cfd90 msgr2=0x7fd0840b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd082a00640 1 --2- 172.21.15.110:0/3465608855 >> v2:172.21.15.110:3300/0 conn(0x7fd0840cfd90 0x7fd0840b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd082a00640 1 -- 172.21.15.110:0/3465608855 >> v1:172.21.15.110:6789/0 conn(0x7fd0840b29e0 legacy=0x7fd0840cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd082a00640 1 -- 172.21.15.110:0/3465608855 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0840b3aa0 con 0x7fd084145f30 2024-09-10T10:04:13.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.310+0000 7fd083400640 1 --2- 172.21.15.110:0/3465608855 >> v2:172.21.15.110:3300/0 conn(0x7fd0840cfd90 0x7fd0840b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:13.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd082a00640 1 --2- 172.21.15.110:0/3465608855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd084145f30 0x7fd0840b24a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd0840b3000 tx=0x7fd07401c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5a60965d8d05ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd081600640 1 -- 172.21.15.110:0/3465608855 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0740b2070 con 0x7fd084145f30 2024-09-10T10:04:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd081600640 1 -- 172.21.15.110:0/3465608855 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0740a57f0 con 0x7fd084145f30 2024-09-10T10:04:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd081600640 1 -- 172.21.15.110:0/3465608855 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0740a5ba0 con 0x7fd084145f30 2024-09-10T10:04:13.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 -- 172.21.15.110:0/3465608855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd084145f30 msgr2=0x7fd0840b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 --2- 172.21.15.110:0/3465608855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd084145f30 0x7fd0840b24a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd0840b3000 tx=0x7fd07401c930 comp rx=0 tx=0).stop 2024-09-10T10:04:13.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 -- 172.21.15.110:0/3465608855 shutdown_connections 2024-09-10T10:04:13.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 --2- 172.21.15.110:0/3465608855 >> v2:172.21.15.110:3300/0 conn(0x7fd0840cfd90 0x7fd0840b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 --2- 172.21.15.110:0/3465608855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd084145f30 0x7fd0840b24a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 -- 172.21.15.110:0/3465608855 >> 172.21.15.110:0/3465608855 conn(0x7fd084062e40 msgr2=0x7fd08406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 -- 172.21.15.110:0/3465608855 shutdown_connections 2024-09-10T10:04:13.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 -- 172.21.15.110:0/3465608855 wait complete. 2024-09-10T10:04:13.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 Processor -- start 2024-09-10T10:04:13.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.314+0000 7fd089600640 1 -- start start 2024-09-10T10:04:13.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0840b29e0 0x7fd08413aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0840cfd90 0x7fd08413afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd084145f30 0x7fd084144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd0840b56a0 con 0x7fd084145f30 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd0840b5520 con 0x7fd0840b29e0 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd0840b5820 con 0x7fd0840cfd90 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd084145f30 0x7fd084144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd084145f30 0x7fd084144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48496/0 (socket says 172.21.15.110:48496) 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 -- 172.21.15.110:0/1993255603 learned_addr learned my addr 172.21.15.110:0/1993255603 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd082a00640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0840b29e0 0x7fd08413aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd082000640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0840cfd90 0x7fd08413afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 -- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0840cfd90 msgr2=0x7fd08413afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0840cfd90 0x7fd08413afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 -- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0840b29e0 msgr2=0x7fd08413aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0840b29e0 0x7fd08413aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 -- 172.21.15.110:0/1993255603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08415b840 con 0x7fd084145f30 2024-09-10T10:04:13.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd083400640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd084145f30 0x7fd084144ab0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00e930 tx=0x7fd07c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd07aa00640 1 -- 172.21.15.110:0/1993255603 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c017070 con 0x7fd084145f30 2024-09-10T10:04:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd07aa00640 1 -- 172.21.15.110:0/1993255603 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c004020 con 0x7fd084145f30 2024-09-10T10:04:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08415cb80 con 0x7fd084145f30 2024-09-10T10:04:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd07aa00640 1 -- 172.21.15.110:0/1993255603 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c004430 con 0x7fd084145f30 2024-09-10T10:04:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08415bdc0 con 0x7fd084145f30 2024-09-10T10:04:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.318+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0840b4580 con 0x7fd084145f30 2024-09-10T10:04:13.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.322+0000 7fd07aa00640 1 -- 172.21.15.110:0/1993255603 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) ==== 87853+0+0 (secure 0 0 0) 0x7fd07c00c940 con 0x7fd084145f30 2024-09-10T10:04:13.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.322+0000 7fd07aa00640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd064067d40 0x7fd06406a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.322+0000 7fd07aa00640 1 -- 172.21.15.110:0/1993255603 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7fd07c086230 con 0x7fd084145f30 2024-09-10T10:04:13.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.322+0000 7fd082a00640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd064067d40 0x7fd06406a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.322+0000 7fd082a00640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd064067d40 0x7fd06406a200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd0840b3000 tx=0x7fd0740a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.326+0000 7fd07aa00640 1 -- 172.21.15.110:0/1993255603 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd07c056090 con 0x7fd084145f30 2024-09-10T10:04:13.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.506+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fd0840ccfb0 con 0x7fd064067d40 2024-09-10T10:04:13.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.510+0000 7fd07aa00640 1 -- 172.21.15.110:0/1993255603 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fd0840ccfb0 con 0x7fd064067d40 2024-09-10T10:04:13.515 INFO:teuthology.orchestra.run.smithi110.stdout:[] 2024-09-10T10:04:13.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd064067d40 msgr2=0x7fd06406a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd064067d40 0x7fd06406a200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd0840b3000 tx=0x7fd0740a7070 comp rx=0 tx=0).stop 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd084145f30 msgr2=0x7fd084144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd084145f30 0x7fd084144ab0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00e930 tx=0x7fd07c00edf0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 shutdown_connections 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd064067d40 0x7fd06406a200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd084145f30 0x7fd084144ab0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0840cfd90 0x7fd08413afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 --2- 172.21.15.110:0/1993255603 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0840b29e0 0x7fd08413aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 >> 172.21.15.110:0/1993255603 conn(0x7fd084062e40 msgr2=0x7fd084142d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 shutdown_connections 2024-09-10T10:04:13.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13.514+0000 7fd089600640 1 -- 172.21.15.110:0/1993255603 wait complete. 2024-09-10T10:04:13.537 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T10:04:13.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:04:13.587 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-10T10:04:13.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:13.594 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:04:13.594 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:04:13.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:04:13.813 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:04:13.813 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:04:13.813 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:13.813 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:13.822 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:13.822 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:13.873 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:04:14.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.050+0000 7f642fc00640 1 Processor -- start 2024-09-10T10:04:14.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.050+0000 7f642fc00640 1 -- start start 2024-09-10T10:04:14.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642fc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f64280b4ce0 0x7f6428165690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642fc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f642815ae90 con 0x7f642815aab0 2024-09-10T10:04:14.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642fc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f642815b170 con 0x7f64280b4170 2024-09-10T10:04:14.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642fc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f642815b450 con 0x7f64280b4ce0 2024-09-10T10:04:14.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642d200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f642815aab0 0x7f6428072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57908/0 (socket says 172.21.15.110:57908) 2024-09-10T10:04:14.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642d200640 1 -- 172.21.15.110:0/984910563 learned_addr learned my addr 172.21.15.110:0/984910563 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:14.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642dc00640 1 --2- 172.21.15.110:0/984910563 >> v2:172.21.15.110:3300/0 conn(0x7f64280b4ce0 0x7f6428165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642dc00640 1 -- 172.21.15.110:0/984910563 >> v1:172.21.15.145:6789/0 conn(0x7f64280b4170 legacy=0x7f64280b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:14.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642dc00640 1 -- 172.21.15.110:0/984910563 >> v1:172.21.15.110:6789/0 conn(0x7f642815aab0 legacy=0x7f6428072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f642dc00640 1 -- 172.21.15.110:0/984910563 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642815b730 con 0x7f64280b4ce0 2024-09-10T10:04:14.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.054+0000 7f6427400640 1 -- 172.21.15.110:0/984910563 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138163671 0 0) 0x7f642815ae90 con 0x7f642815aab0 2024-09-10T10:04:14.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642dc00640 1 --2- 172.21.15.110:0/984910563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64280b4ce0 0x7f6428165690 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6420009d90 tx=0x7f642001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f8b624b31bfd20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:14.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f6427400640 1 -- 172.21.15.110:0/984910563 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200b2070 con 0x7f64280b4ce0 2024-09-10T10:04:14.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f6427400640 1 -- 172.21.15.110:0/984910563 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64200a58e0 con 0x7f64280b4ce0 2024-09-10T10:04:14.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f6427400640 1 -- 172.21.15.110:0/984910563 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200a5cb0 con 0x7f64280b4ce0 2024-09-10T10:04:14.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 -- 172.21.15.110:0/984910563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64280b4ce0 msgr2=0x7f6428165690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 --2- 172.21.15.110:0/984910563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64280b4ce0 0x7f6428165690 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6420009d90 tx=0x7f642001c930 comp rx=0 tx=0).stop 2024-09-10T10:04:14.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 -- 172.21.15.110:0/984910563 shutdown_connections 2024-09-10T10:04:14.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 --2- 172.21.15.110:0/984910563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64280b4ce0 0x7f6428165690 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 -- 172.21.15.110:0/984910563 >> 172.21.15.110:0/984910563 conn(0x7f6428062d10 msgr2=0x7f642806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:14.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 -- 172.21.15.110:0/984910563 shutdown_connections 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 -- 172.21.15.110:0/984910563 wait complete. 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 Processor -- start 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 -- start start 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64280b4170 0x7f64281fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64280b4ce0 0x7f64281509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f642815aab0 0x7f64280b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.058+0000 7f642fc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6428168b90 con 0x7f642815aab0 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642fc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6428168a10 con 0x7f64280b4170 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642fc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6428168d10 con 0x7f64280b4ce0 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f642815aab0 0x7f64280b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f642815aab0 0x7f64280b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48502/0 (socket says 172.21.15.110:48502) 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 -- 172.21.15.110:0/2146773511 learned_addr learned my addr 172.21.15.110:0/2146773511 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:14.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642d200640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64280b4170 0x7f64281fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f6427e00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64280b4ce0 0x7f64281509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 -- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64280b4ce0 msgr2=0x7f64281509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64280b4ce0 0x7f64281509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 -- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64280b4170 msgr2=0x7f64281fb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64280b4170 0x7f64281fb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 -- 172.21.15.110:0/2146773511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6428153a60 con 0x7f642815aab0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642dc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f642815aab0 0x7f64280b0610 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f642001a560 tx=0x7f64200196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f6424c00640 1 -- 172.21.15.110:0/2146773511 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200b2040 con 0x7f642815aab0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f6424c00640 1 -- 172.21.15.110:0/2146773511 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64200bf070 con 0x7f642815aab0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f6424c00640 1 -- 172.21.15.110:0/2146773511 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200ad8c0 con 0x7f642815aab0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64281590d0 con 0x7f642815aab0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64280c58c0 con 0x7f642815aab0 2024-09-10T10:04:14.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64280b45d0 con 0x7f642815aab0 2024-09-10T10:04:14.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.062+0000 7f6424c00640 1 -- 172.21.15.110:0/2146773511 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f64200ada60 con 0x7f642815aab0 2024-09-10T10:04:14.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.066+0000 7f6424c00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6408067e90 0x7f640806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.066+0000 7f6424c00640 1 -- 172.21.15.110:0/2146773511 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f6420121210 con 0x7f642815aab0 2024-09-10T10:04:14.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.066+0000 7f642d200640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6408067e90 0x7f640806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.066+0000 7f642d200640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6408067e90 0x7f640806a350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f641c000f70 tx=0x7f641c007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:14.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.070+0000 7f6424c00640 1 -- 172.21.15.110:0/2146773511 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f64200c8020 con 0x7f642815aab0 2024-09-10T10:04:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6428075560 con 0x7f642815aab0 2024-09-10T10:04:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f6424c00640 1 -- 172.21.15.110:0/2146773511 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) ==== 80+0+0 (secure 0 0 0) 0x7f6420016520 con 0x7f642815aab0 2024-09-10T10:04:14.242 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:04:14.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6408067e90 msgr2=0x7f640806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6408067e90 0x7f640806a350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f641c000f70 tx=0x7f641c007df0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f642815aab0 msgr2=0x7f64280b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f642815aab0 0x7f64280b0610 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f642001a560 tx=0x7f64200196d0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 shutdown_connections 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6408067e90 0x7f640806a350 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f642815aab0 0x7f64280b0610 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f64280b4ce0 0x7f64281509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 --2- 172.21.15.110:0/2146773511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64280b4170 0x7f64281fb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 >> 172.21.15.110:0/2146773511 conn(0x7f6428062d10 msgr2=0x7f64280b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 shutdown_connections 2024-09-10T10:04:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.238+0000 7f642fc00640 1 -- 172.21.15.110:0/2146773511 wait complete. 2024-09-10T10:04:14.262 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:04:14.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:14.263 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:14.321 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:04:14.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:14.322 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:04:14.339 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:04:14.339 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:14.340 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:14.350 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:14.350 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:14.401 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:04:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb981800640 1 Processor -- start 2024-09-10T10:04:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb981800640 1 -- start start 2024-09-10T10:04:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb981800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb97c0cfb90 0x7fb97c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb981800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb97c0b3f40 0x7fb97c0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb981800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0cff70 0x7fb97c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb981800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb97c06d130 con 0x7fb97c0b3f40 2024-09-10T10:04:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb981800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb97c15d4d0 con 0x7fb97c0cfb90 2024-09-10T10:04:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb981800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c15d7b0 con 0x7fb97c0cff70 2024-09-10T10:04:14.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb973e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb97c0b3f40 0x7fb97c0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb973e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb97c0b3f40 0x7fb97c0cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48508/0 (socket says 172.21.15.110:48508) 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb973e00640 1 -- 172.21.15.110:0/2100381764 learned_addr learned my addr 172.21.15.110:0/2100381764 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb97b400640 1 --2- 172.21.15.110:0/2100381764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0cff70 0x7fb97c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb97aa00640 1 --2- 172.21.15.110:0/2100381764 >> v2:172.21.15.145:3300/0 conn(0x7fb97c0cfb90 0x7fb97c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb973e00640 1 -- 172.21.15.110:0/2100381764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0cff70 msgr2=0x7fb97c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb973e00640 1 --2- 172.21.15.110:0/2100381764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0cff70 0x7fb97c0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb973e00640 1 -- 172.21.15.110:0/2100381764 >> v2:172.21.15.145:3300/0 conn(0x7fb97c0cfb90 msgr2=0x7fb97c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb973e00640 1 --2- 172.21.15.110:0/2100381764 >> v2:172.21.15.145:3300/0 conn(0x7fb97c0cfb90 0x7fb97c15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.534+0000 7fb973e00640 1 -- 172.21.15.110:0/2100381764 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c06d380 con 0x7fb97c0b3f40 2024-09-10T10:04:14.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb973e00640 1 --2- 172.21.15.110:0/2100381764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0b3f40 0x7fb97c0cd5a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb97c073450 tx=0x7fb96401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4772ad7ed5ef31f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:14.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb97a000640 1 -- 172.21.15.110:0/2100381764 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640b0070 con 0x7fb97c0b3f40 2024-09-10T10:04:14.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb97a000640 1 -- 172.21.15.110:0/2100381764 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9640ab600 con 0x7fb97c0b3f40 2024-09-10T10:04:14.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb97a000640 1 -- 172.21.15.110:0/2100381764 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640ab9d0 con 0x7fb97c0b3f40 2024-09-10T10:04:14.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 -- 172.21.15.110:0/2100381764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0b3f40 msgr2=0x7fb97c0cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 --2- 172.21.15.110:0/2100381764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0b3f40 0x7fb97c0cd5a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb97c073450 tx=0x7fb96401c490 comp rx=0 tx=0).stop 2024-09-10T10:04:14.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 -- 172.21.15.110:0/2100381764 shutdown_connections 2024-09-10T10:04:14.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 --2- 172.21.15.110:0/2100381764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0cff70 0x7fb97c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 --2- 172.21.15.110:0/2100381764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0b3f40 0x7fb97c0cd5a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 --2- 172.21.15.110:0/2100381764 >> v2:172.21.15.145:3300/0 conn(0x7fb97c0cfb90 0x7fb97c15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 -- 172.21.15.110:0/2100381764 >> 172.21.15.110:0/2100381764 conn(0x7fb97c062d10 msgr2=0x7fb97c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 -- 172.21.15.110:0/2100381764 shutdown_connections 2024-09-10T10:04:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.538+0000 7fb981800640 1 -- 172.21.15.110:0/2100381764 wait complete. 2024-09-10T10:04:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 Processor -- start 2024-09-10T10:04:14.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 -- start start 2024-09-10T10:04:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0b3f40 0x7fb97c1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0cfb90 0x7fb97c165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb97c0cff70 0x7fb97c165a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c160ba0 con 0x7fb97c0cfb90 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c160a20 con 0x7fb97c0cff70 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c160d20 con 0x7fb97c0b3f40 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0cfb90 0x7fb97c165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0cfb90 0x7fb97c165520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48520/0 (socket says 172.21.15.110:48520) 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 -- 172.21.15.110:0/2690746063 learned_addr learned my addr 172.21.15.110:0/2690746063 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb97b400640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb97c0cff70 0x7fb97c165a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb97aa00640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0b3f40 0x7fb97c1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 -- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0b3f40 msgr2=0x7fb97c1504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0b3f40 0x7fb97c1504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 -- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb97c0cff70 msgr2=0x7fb97c165a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb97c0cff70 0x7fb97c165a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 -- 172.21.15.110:0/2690746063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c150a10 con 0x7fb97c0cfb90 2024-09-10T10:04:14.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb973e00640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0cfb90 0x7fb97c165520 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb96401ab40 tx=0x7fb964019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:14.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb972a00640 1 -- 172.21.15.110:0/2690746063 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640b0040 con 0x7fb97c0cfb90 2024-09-10T10:04:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb972a00640 1 -- 172.21.15.110:0/2690746063 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9640b9070 con 0x7fb97c0cfb90 2024-09-10T10:04:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c1568d0 con 0x7fb97c0cfb90 2024-09-10T10:04:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb972a00640 1 -- 172.21.15.110:0/2690746063 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640b4880 con 0x7fb97c0cfb90 2024-09-10T10:04:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.542+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c071530 con 0x7fb97c0cfb90 2024-09-10T10:04:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.546+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97c15ca10 con 0x7fb97c0cfb90 2024-09-10T10:04:14.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.546+0000 7fb972a00640 1 -- 172.21.15.110:0/2690746063 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fb9640ab4c0 con 0x7fb97c0cfb90 2024-09-10T10:04:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.546+0000 7fb972a00640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb958067e60 0x7fb95806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.546+0000 7fb972a00640 1 -- 172.21.15.110:0/2690746063 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7fb964120390 con 0x7fb97c0cfb90 2024-09-10T10:04:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.546+0000 7fb97aa00640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb958067e60 0x7fb95806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:14.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.546+0000 7fb97aa00640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb958067e60 0x7fb95806a320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009ae0 tx=0x7fb96c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:14.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.550+0000 7fb972a00640 1 -- 172.21.15.110:0/2690746063 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb964016110 con 0x7fb97c0cfb90 2024-09-10T10:04:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb97c0b4ef0 con 0x7fb97c0cfb90 2024-09-10T10:04:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb972a00640 1 -- 172.21.15.110:0/2690746063 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) ==== 80+0+0 (secure 0 0 0) 0x7fb9640f0050 con 0x7fb97c0cfb90 2024-09-10T10:04:14.722 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb958067e60 msgr2=0x7fb95806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb958067e60 0x7fb95806a320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009ae0 tx=0x7fb96c009190 comp rx=0 tx=0).stop 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0cfb90 msgr2=0x7fb97c165520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0cfb90 0x7fb97c165520 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb96401ab40 tx=0x7fb964019230 comp rx=0 tx=0).stop 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 shutdown_connections 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb958067e60 0x7fb95806a320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb97c0cff70 0x7fb97c165a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb97c0cfb90 0x7fb97c165520 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 --2- 172.21.15.110:0/2690746063 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb97c0b3f40 0x7fb97c1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 >> 172.21.15.110:0/2690746063 conn(0x7fb97c062d10 msgr2=0x7fb97c0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 shutdown_connections 2024-09-10T10:04:14.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:14.718+0000 7fb981800640 1 -- 172.21.15.110:0/2690746063 wait complete. 2024-09-10T10:04:14.746 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:04:14.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:14.746 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:14.773 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:04:14.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:14.774 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:04:14.826 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:04:14.826 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:14.826 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:14.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:04:14.837 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:04:14.837 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:14.837 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:14.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:04:14.882 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:15.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf17800640 1 Processor -- start 2024-09-10T10:04:15.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf17800640 1 -- start start 2024-09-10T10:04:15.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf17800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf100ccb60 0x7faf10072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf17800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7faf10072670 0x7faf10140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf17800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf100ccf40 con 0x7faf10140a60 2024-09-10T10:04:15.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf17800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7faf10072ad0 con 0x7faf10072670 2024-09-10T10:04:15.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf17800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7faf1015c130 con 0x7faf100ccb60 2024-09-10T10:04:15.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf15800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7faf10140a60 0x7faf100710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57916/0 (socket says 172.21.15.110:57916) 2024-09-10T10:04:15.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf15800640 1 -- 172.21.15.110:0/3942185812 learned_addr learned my addr 172.21.15.110:0/3942185812 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0fe00640 1 --2- 172.21.15.110:0/3942185812 >> v2:172.21.15.145:3300/0 conn(0x7faf10072670 0x7faf10140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf14e00640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf100ccb60 0x7faf10072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68806048 0 0) 0x7faf100ccf40 con 0x7faf10140a60 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faef8005740 con 0x7faf10140a60 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 499098731 0 0) 0x7faef8005740 con 0x7faf10140a60 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf100ccb60 msgr2=0x7faf10072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf100ccb60 0x7faf10072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 >> v2:172.21.15.145:3300/0 conn(0x7faf10072670 msgr2=0x7faf10140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 --2- 172.21.15.110:0/3942185812 >> v2:172.21.15.145:3300/0 conn(0x7faf10072670 0x7faf10140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf101412a0 con 0x7faf10140a60 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7faf08001a00 con 0x7faf10140a60 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 >> v1:172.21.15.110:6789/0 conn(0x7faf10140a60 legacy=0x7faf100710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef8005bd0 0x7faef8005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf100ccb60 0x7faef8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faef80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faef800dbe0 con 0x7faef8006f60 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faef800dd60 con 0x7faef8005bd0 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7faef800dee0 con 0x7faf100ccb60 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf15800640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faef80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf14e00640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef8005bd0 0x7faef8005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf0fe00640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf100ccb60 0x7faef8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf15800640 1 -- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf100ccb60 msgr2=0x7faef8006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:15.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf15800640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf100ccb60 0x7faef8006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf15800640 1 -- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef8005bd0 msgr2=0x7faef8005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:15.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf15800640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef8005bd0 0x7faef8005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.014+0000 7faf15800640 1 -- 172.21.15.110:0/3942185812 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef8005740 con 0x7faef8006f60 2024-09-10T10:04:15.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf15800640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faef80073e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7faf0800a7d0 tx=0x7faf0801d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb2113cfe4b93c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0800a3f0 con 0x7faef8006f60 2024-09-10T10:04:15.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf080ad630 con 0x7faef8006f60 2024-09-10T10:04:15.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf0f400640 1 -- 172.21.15.110:0/3942185812 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080ada40 con 0x7faef8006f60 2024-09-10T10:04:15.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf17800640 1 -- 172.21.15.110:0/3942185812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 msgr2=0x7faef80073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf17800640 1 --2- 172.21.15.110:0/3942185812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faef80073e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7faf0800a7d0 tx=0x7faf0801d300 comp rx=0 tx=0).stop 2024-09-10T10:04:15.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf15800640 1 -- 172.21.15.110:0/3942185812 reap_dead start 2024-09-10T10:04:15.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf17800640 1 -- 172.21.15.110:0/3942185812 shutdown_connections 2024-09-10T10:04:15.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf17800640 1 -- 172.21.15.110:0/3942185812 >> 172.21.15.110:0/3942185812 conn(0x7faf1006bc50 msgr2=0x7faf1006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:15.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf17800640 1 -- 172.21.15.110:0/3942185812 shutdown_connections 2024-09-10T10:04:15.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.018+0000 7faf17800640 1 -- 172.21.15.110:0/3942185812 wait complete. 2024-09-10T10:04:15.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 Processor -- start 2024-09-10T10:04:15.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 -- start start 2024-09-10T10:04:15.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faf1013ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf1013c750 0x7faf1013f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf10135ee0 0x7faf10137170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf1015ed20 con 0x7faef8006f60 2024-09-10T10:04:15.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf1015ea40 con 0x7faf10135ee0 2024-09-10T10:04:15.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7faf1015f000 con 0x7faf1013c750 2024-09-10T10:04:15.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faf1013ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faf1013ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48544/0 (socket says 172.21.15.110:48544) 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 -- 172.21.15.110:0/1254358321 learned_addr learned my addr 172.21.15.110:0/1254358321 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf0fe00640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf1013c750 0x7faf1013f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf15800640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf10135ee0 0x7faf10137170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 -- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf1013c750 msgr2=0x7faf1013f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf1013c750 0x7faf1013f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 -- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf10135ee0 msgr2=0x7faf10137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf10135ee0 0x7faf10137170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 -- 172.21.15.110:0/1254358321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1013a5f0 con 0x7faef8006f60 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf15800640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf10135ee0 0x7faf10137170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf14e00640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faf1013ef80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7faf04012a50 tx=0x7faf04012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf0cc00640 1 -- 172.21.15.110:0/1254358321 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0400b040 con 0x7faef8006f60 2024-09-10T10:04:15.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf0cc00640 1 -- 172.21.15.110:0/1254358321 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf04017070 con 0x7faef8006f60 2024-09-10T10:04:15.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf0cc00640 1 -- 172.21.15.110:0/1254358321 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0401c470 con 0x7faef8006f60 2024-09-10T10:04:15.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1013a8d0 con 0x7faef8006f60 2024-09-10T10:04:15.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.022+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1013ae90 con 0x7faef8006f60 2024-09-10T10:04:15.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.026+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1013b3b0 con 0x7faef8006f60 2024-09-10T10:04:15.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.026+0000 7faf0cc00640 1 -- 172.21.15.110:0/1254358321 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7faf0401c610 con 0x7faef8006f60 2024-09-10T10:04:15.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.026+0000 7faf0cc00640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faef0067e60 0x7faef006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.026+0000 7faf0cc00640 1 -- 172.21.15.110:0/1254358321 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7faf04087370 con 0x7faef8006f60 2024-09-10T10:04:15.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.026+0000 7faf0fe00640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faef0067e60 0x7faef006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.026+0000 7faf0fe00640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faef0067e60 0x7faef006a320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf10158320 tx=0x7faefc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.030+0000 7faf0cc00640 1 -- 172.21.15.110:0/1254358321 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7faf1013b3b0 con 0x7faef8006f60 2024-09-10T10:04:15.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.218+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf1006c210 con 0x7faef8006f60 2024-09-10T10:04:15.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.218+0000 7faf0cc00640 1 -- 172.21.15.110:0/1254358321 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5237 (secure 0 0 0) 0x7faf04057630 con 0x7faef8006f60 2024-09-10T10:04:15.224 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:15.224 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":79,"btime":"2024-09-10T10:04:07:761751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1172125083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1172125083},{"type":"v1","addr":"172.21.15.145:6834","nonce":1172125083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6194,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3529040969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3529040969},{"type":"v1","addr":"172.21.15.145:6835","nonce":3529040969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2459795528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2459795528},{"type":"v1","addr":"172.21.15.110:6835","nonce":2459795528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:58.096051+0000","modified":"2024-09-10T10:04:01.113268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6834/1757537079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1757537079},{"type":"v1","addr":"172.21.15.110:6834","nonce":1757537079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":9}]} 2024-09-10T10:04:15.225 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 79 2024-09-10T10:04:15.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faef0067e60 msgr2=0x7faef006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faef0067e60 0x7faef006a320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf10158320 tx=0x7faefc008040 comp rx=0 tx=0).stop 2024-09-10T10:04:15.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 msgr2=0x7faf1013ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faf1013ef80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7faf04012a50 tx=0x7faf04012f10 comp rx=0 tx=0).stop 2024-09-10T10:04:15.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 shutdown_connections 2024-09-10T10:04:15.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7faef0067e60 0x7faef006a320 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf10135ee0 0x7faf10137170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7faf1013c750 0x7faf1013f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 --2- 172.21.15.110:0/1254358321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faef8006f60 0x7faf1013ef80 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 >> 172.21.15.110:0/1254358321 conn(0x7faf1006bc50 msgr2=0x7faf100710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:15.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 shutdown_connections 2024-09-10T10:04:15.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.222+0000 7faf17800640 1 -- 172.21.15.110:0/1254358321 wait complete. 2024-09-10T10:04:15.247 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:15.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.382+0000 7f4ccb800640 1 Processor -- start 2024-09-10T10:04:15.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.382+0000 7f4ccb800640 1 -- start start 2024-09-10T10:04:15.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4ccb800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4cc406f1e0 0x7f4cc4166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4ccb800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc4155d50 con 0x7f4cc4155970 2024-09-10T10:04:15.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4ccb800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc40b34d0 con 0x7f4cc415c560 2024-09-10T10:04:15.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4ccb800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4cc40b4570 con 0x7f4cc406f1e0 2024-09-10T10:04:15.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4cc9800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4cc406f1e0 0x7f4cc4166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4cc9800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4cc406f1e0 0x7f4cc4166020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48546/0 (socket says 172.21.15.110:48546) 2024-09-10T10:04:15.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4cc9800640 1 -- 172.21.15.110:0/2563914177 learned_addr learned my addr 172.21.15.110:0/2563914177 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:15.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4cc9800640 1 -- 172.21.15.110:0/2563914177 >> v1:172.21.15.145:6789/0 conn(0x7f4cc415c560 legacy=0x7f4cc406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:15.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4cc9800640 1 -- 172.21.15.110:0/2563914177 >> v1:172.21.15.110:6789/0 conn(0x7f4cc4155970 legacy=0x7f4cc40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.386+0000 7f4cc9800640 1 -- 172.21.15.110:0/2563914177 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc40b37b0 con 0x7f4cc406f1e0 2024-09-10T10:04:15.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4cc9800640 1 --2- 172.21.15.110:0/2563914177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc406f1e0 0x7f4cc4166020 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009e50 tx=0x7f4cbc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfe3972aee5e664 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4cc3400640 1 -- 172.21.15.110:0/2563914177 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0a5880 con 0x7f4cc406f1e0 2024-09-10T10:04:15.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4cc3400640 1 -- 172.21.15.110:0/2563914177 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cbc0a5a20 con 0x7f4cc406f1e0 2024-09-10T10:04:15.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4cc3400640 1 -- 172.21.15.110:0/2563914177 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0a5df0 con 0x7f4cc406f1e0 2024-09-10T10:04:15.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 -- 172.21.15.110:0/2563914177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc406f1e0 msgr2=0x7f4cc4166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 --2- 172.21.15.110:0/2563914177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc406f1e0 0x7f4cc4166020 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009e50 tx=0x7f4cbc01c9c0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 -- 172.21.15.110:0/2563914177 shutdown_connections 2024-09-10T10:04:15.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 --2- 172.21.15.110:0/2563914177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc406f1e0 0x7f4cc4166020 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 -- 172.21.15.110:0/2563914177 >> 172.21.15.110:0/2563914177 conn(0x7f4cc406bc50 msgr2=0x7f4cc406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:15.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 -- 172.21.15.110:0/2563914177 shutdown_connections 2024-09-10T10:04:15.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 -- 172.21.15.110:0/2563914177 wait complete. 2024-09-10T10:04:15.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 Processor -- start 2024-09-10T10:04:15.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.390+0000 7f4ccb800640 1 -- start start 2024-09-10T10:04:15.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cc406f1e0 0x7f4cc415f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cc4155970 0x7f4cc413e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc415c560 0x7f4cc4140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4168f00 con 0x7f4cc415c560 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4168d80 con 0x7f4cc4155970 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc4169080 con 0x7f4cc406f1e0 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc415c560 0x7f4cc4140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc415c560 0x7f4cc4140f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48562/0 (socket says 172.21.15.110:48562) 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 -- 172.21.15.110:0/1080745187 learned_addr learned my addr 172.21.15.110:0/1080745187 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc8e00640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cc406f1e0 0x7f4cc415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 -- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cc406f1e0 msgr2=0x7f4cc415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc3e00640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cc4155970 0x7f4cc413e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cc406f1e0 0x7f4cc415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 -- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cc4155970 msgr2=0x7f4cc413e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cc4155970 0x7f4cc413e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 -- 172.21.15.110:0/1080745187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc415fd70 con 0x7f4cc415c560 2024-09-10T10:04:15.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc8e00640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cc406f1e0 0x7f4cc415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:15.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc3e00640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cc4155970 0x7f4cc413e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:15.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc9800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc415c560 0x7f4cc4140f00 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0195d0 tx=0x7f4cbc019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc0c00640 1 -- 172.21.15.110:0/1080745187 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0b8880 con 0x7f4cc415c560 2024-09-10T10:04:15.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc0c00640 1 -- 172.21.15.110:0/1080745187 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cbc0be070 con 0x7f4cc415c560 2024-09-10T10:04:15.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc413fa40 con 0x7f4cc415c560 2024-09-10T10:04:15.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc0c00640 1 -- 172.21.15.110:0/1080745187 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0b8c60 con 0x7f4cc415c560 2024-09-10T10:04:15.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc4141cf0 con 0x7f4cc415c560 2024-09-10T10:04:15.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc40cc8c0 con 0x7f4cc415c560 2024-09-10T10:04:15.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.394+0000 7f4cc0c00640 1 -- 172.21.15.110:0/1080745187 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f4cbc0c3460 con 0x7f4cc415c560 2024-09-10T10:04:15.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.398+0000 7f4cc0c00640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4ca4067e90 0x7f4ca406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.398+0000 7f4cc0c00640 1 -- 172.21.15.110:0/1080745187 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f4cbc121b60 con 0x7f4cc415c560 2024-09-10T10:04:15.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.398+0000 7f4cc8e00640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4ca4067e90 0x7f4ca406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.398+0000 7f4cc8e00640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4ca4067e90 0x7f4ca406a350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40016f0 tx=0x7f4cb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.402+0000 7f4cc0c00640 1 -- 172.21.15.110:0/1080745187 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4cbc0165e0 con 0x7f4cc415c560 2024-09-10T10:04:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.594+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cc415f4e0 con 0x7f4cc415c560 2024-09-10T10:04:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.594+0000 7f4cc0c00640 1 -- 172.21.15.110:0/1080745187 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5237 (secure 0 0 0) 0x7f4cbc0f1f80 con 0x7f4cc415c560 2024-09-10T10:04:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:15.600 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":79,"btime":"2024-09-10T10:04:07:761751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1172125083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1172125083},{"type":"v1","addr":"172.21.15.145:6834","nonce":1172125083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6194,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3529040969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3529040969},{"type":"v1","addr":"172.21.15.145:6835","nonce":3529040969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2459795528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2459795528},{"type":"v1","addr":"172.21.15.110:6835","nonce":2459795528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:58.096051+0000","modified":"2024-09-10T10:04:01.113268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6834/1757537079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1757537079},{"type":"v1","addr":"172.21.15.110:6834","nonce":1757537079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":9}]} 2024-09-10T10:04:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 79 2024-09-10T10:04:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4ca4067e90 msgr2=0x7f4ca406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4ca4067e90 0x7f4ca406a350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40016f0 tx=0x7f4cb4009040 comp rx=0 tx=0).stop 2024-09-10T10:04:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc415c560 msgr2=0x7f4cc4140f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc415c560 0x7f4cc4140f00 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0195d0 tx=0x7f4cbc019760 comp rx=0 tx=0).stop 2024-09-10T10:04:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 shutdown_connections 2024-09-10T10:04:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4ca4067e90 0x7f4ca406a350 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cc415c560 0x7f4cc4140f00 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4cc4155970 0x7f4cc413e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 --2- 172.21.15.110:0/1080745187 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4cc406f1e0 0x7f4cc415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 >> 172.21.15.110:0/1080745187 conn(0x7f4cc406bc50 msgr2=0x7f4cc40aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 shutdown_connections 2024-09-10T10:04:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.598+0000 7f4ccb800640 1 -- 172.21.15.110:0/1080745187 wait complete. 2024-09-10T10:04:15.622 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:15.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39ca600640 1 Processor -- start 2024-09-10T10:04:15.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39ca600640 1 -- start start 2024-09-10T10:04:15.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39ca600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f39c4145b00 0x7f39c40cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39ca600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39c4071db0 con 0x7f39c40b5690 2024-09-10T10:04:15.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39ca600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39c40ce5f0 con 0x7f39c415d050 2024-09-10T10:04:15.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39ca600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f39c40ce8d0 con 0x7f39c4145b00 2024-09-10T10:04:15.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39c9200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f39c4145b00 0x7f39c40cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39c3e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f39c40b5690 0x7f39c415c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57922/0 (socket says 172.21.15.110:57922) 2024-09-10T10:04:15.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39c3e00640 1 -- 172.21.15.110:0/3626449904 learned_addr learned my addr 172.21.15.110:0/3626449904 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:15.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39c9200640 1 -- 172.21.15.110:0/3626449904 >> v1:172.21.15.145:6789/0 conn(0x7f39c415d050 legacy=0x7f39c4071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39c9200640 1 -- 172.21.15.110:0/3626449904 >> v1:172.21.15.110:6789/0 conn(0x7f39c40b5690 legacy=0x7f39c415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39c3400640 1 -- 172.21.15.110:0/3626449904 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3674088036 0 0) 0x7f39c4071db0 con 0x7f39c40b5690 2024-09-10T10:04:15.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.754+0000 7f39c9200640 1 -- 172.21.15.110:0/3626449904 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c40cebb0 con 0x7f39c4145b00 2024-09-10T10:04:15.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39c9200640 1 --2- 172.21.15.110:0/3626449904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c40cf0d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f39b40097c0 tx=0x7f39b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e0316be419e7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39c3400640 1 -- 172.21.15.110:0/3626449904 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b40b0070 con 0x7f39c4145b00 2024-09-10T10:04:15.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39c3400640 1 -- 172.21.15.110:0/3626449904 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b40ab600 con 0x7f39c4145b00 2024-09-10T10:04:15.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39c3400640 1 -- 172.21.15.110:0/3626449904 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b40ab9d0 con 0x7f39c4145b00 2024-09-10T10:04:15.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39ca600640 1 -- 172.21.15.110:0/3626449904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 msgr2=0x7f39c40cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39ca600640 1 --2- 172.21.15.110:0/3626449904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c40cf0d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f39b40097c0 tx=0x7f39b401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39ca600640 1 -- 172.21.15.110:0/3626449904 shutdown_connections 2024-09-10T10:04:15.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39ca600640 1 --2- 172.21.15.110:0/3626449904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c40cf0d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39ca600640 1 -- 172.21.15.110:0/3626449904 >> 172.21.15.110:0/3626449904 conn(0x7f39c406d0f0 msgr2=0x7f39c406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:15.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39ca600640 1 -- 172.21.15.110:0/3626449904 shutdown_connections 2024-09-10T10:04:15.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.758+0000 7f39ca600640 1 -- 172.21.15.110:0/3626449904 wait complete. 2024-09-10T10:04:15.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 Processor -- start 2024-09-10T10:04:15.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 -- start start 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39c40b5690 0x7f39c4144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c41423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39c415d050 0x7f39c415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4071b60 con 0x7f39c4145b00 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39c416a7b0 con 0x7f39c415d050 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f39c416a930 con 0x7f39c40b5690 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c41423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c41423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48582/0 (socket says 172.21.15.110:48582) 2024-09-10T10:04:15.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 -- 172.21.15.110:0/1245570792 learned_addr learned my addr 172.21.15.110:0/1245570792 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c9200640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39c40b5690 0x7f39c4144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c9c00640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39c415d050 0x7f39c415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 -- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39c40b5690 msgr2=0x7f39c4144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39c40b5690 0x7f39c4144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 -- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39c415d050 msgr2=0x7f39c415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39c415d050 0x7f39c415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 -- 172.21.15.110:0/1245570792 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c4142c40 con 0x7f39c4145b00 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c9c00640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39c415d050 0x7f39c415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c3e00640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c41423f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f39b000c4a0 tx=0x7f39b000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c0c00640 1 -- 172.21.15.110:0/1245570792 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0018070 con 0x7f39c4145b00 2024-09-10T10:04:15.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c0c00640 1 -- 172.21.15.110:0/1245570792 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b00074e0 con 0x7f39c4145b00 2024-09-10T10:04:15.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c4142f20 con 0x7f39c4145b00 2024-09-10T10:04:15.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39c0c00640 1 -- 172.21.15.110:0/1245570792 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0004a30 con 0x7f39c4145b00 2024-09-10T10:04:15.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.762+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c4142930 con 0x7f39c4145b00 2024-09-10T10:04:15.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.766+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c40cf0d0 con 0x7f39c4145b00 2024-09-10T10:04:15.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.766+0000 7f39c0c00640 1 -- 172.21.15.110:0/1245570792 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f39b001b070 con 0x7f39c4145b00 2024-09-10T10:04:15.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.766+0000 7f39c0c00640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f39a4067e60 0x7f39a406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:15.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.766+0000 7f39c0c00640 1 -- 172.21.15.110:0/1245570792 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f39b0085ad0 con 0x7f39c4145b00 2024-09-10T10:04:15.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.766+0000 7f39c9200640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f39a4067e60 0x7f39a406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:15.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.766+0000 7f39c9200640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f39a4067e60 0x7f39a406a320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39b40ad000 tx=0x7f39b40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:15.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.770+0000 7f39c0c00640 1 -- 172.21.15.110:0/1245570792 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f39b0055d00 con 0x7f39c4145b00 2024-09-10T10:04:15.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.930+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39c4142080 con 0x7f39c4145b00 2024-09-10T10:04:15.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.930+0000 7f39c0c00640 1 -- 172.21.15.110:0/1245570792 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16931 (secure 0 0 0) 0x7f39b0055600 con 0x7f39c4145b00 2024-09-10T10:04:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:15.936 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":118,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:14.788697+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:55.813835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:57.093856+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":118,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f39a4067e60 msgr2=0x7f39a406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f39a4067e60 0x7f39a406a320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39b40ad000 tx=0x7f39b40a6000 comp rx=0 tx=0).stop 2024-09-10T10:04:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 msgr2=0x7f39c41423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c41423f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f39b000c4a0 tx=0x7f39b000c960 comp rx=0 tx=0).stop 2024-09-10T10:04:15.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 shutdown_connections 2024-09-10T10:04:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f39a4067e60 0x7f39a406a320 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39c415d050 0x7f39c415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39c4145b00 0x7f39c41423f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 --2- 172.21.15.110:0/1245570792 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39c40b5690 0x7f39c4144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 >> 172.21.15.110:0/1245570792 conn(0x7f39c406d0f0 msgr2=0x7f39c40cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 shutdown_connections 2024-09-10T10:04:15.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:15.934+0000 7f39ca600640 1 -- 172.21.15.110:0/1245570792 wait complete. 2024-09-10T10:04:15.958 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:04:15.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:16.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7597c00640 1 Processor -- start 2024-09-10T10:04:16.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7597c00640 1 -- start start 2024-09-10T10:04:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7597c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f759015e710 0x7f75900b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7597c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75900d15a0 0x7f7590070e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7597c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75900714b0 con 0x7f7590070a30 2024-09-10T10:04:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7597c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f759006d6b0 con 0x7f759015e710 2024-09-10T10:04:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7597c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7590071790 con 0x7f75900d15a0 2024-09-10T10:04:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7595200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f759015e710 0x7f75900b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7595200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f759015e710 0x7f75900b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48590/0 (socket says 172.21.15.110:48590) 2024-09-10T10:04:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7595200640 1 -- 172.21.15.110:0/2614288905 learned_addr learned my addr 172.21.15.110:0/2614288905 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7595c00640 1 --2- 172.21.15.110:0/2614288905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75900d15a0 0x7f7590070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7595200640 1 -- 172.21.15.110:0/2614288905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75900d15a0 msgr2=0x7f7590070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7595200640 1 --2- 172.21.15.110:0/2614288905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75900d15a0 0x7f7590070e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7595200640 1 -- 172.21.15.110:0/2614288905 >> v1:172.21.15.145:6789/0 conn(0x7f7590070a30 legacy=0x7f75900d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.090+0000 7f7595200640 1 -- 172.21.15.110:0/2614288905 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7590071a70 con 0x7f759015e710 2024-09-10T10:04:16.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7595200640 1 --2- 172.21.15.110:0/2614288905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759015e710 0x7f75900b5590 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7580009e50 tx=0x7f758001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d9856fac2a4f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f758f400640 1 -- 172.21.15.110:0/2614288905 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800a58e0 con 0x7f759015e710 2024-09-10T10:04:16.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f758f400640 1 -- 172.21.15.110:0/2614288905 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75800a5a80 con 0x7f759015e710 2024-09-10T10:04:16.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f758f400640 1 -- 172.21.15.110:0/2614288905 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800a5e50 con 0x7f759015e710 2024-09-10T10:04:16.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 -- 172.21.15.110:0/2614288905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759015e710 msgr2=0x7f75900b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 --2- 172.21.15.110:0/2614288905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759015e710 0x7f75900b5590 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7580009e50 tx=0x7f758001ca30 comp rx=0 tx=0).stop 2024-09-10T10:04:16.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 -- 172.21.15.110:0/2614288905 shutdown_connections 2024-09-10T10:04:16.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 --2- 172.21.15.110:0/2614288905 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f75900d15a0 0x7f7590070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 --2- 172.21.15.110:0/2614288905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f759015e710 0x7f75900b5590 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 -- 172.21.15.110:0/2614288905 >> 172.21.15.110:0/2614288905 conn(0x7f75900b4290 msgr2=0x7f75900b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:16.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 -- 172.21.15.110:0/2614288905 shutdown_connections 2024-09-10T10:04:16.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 -- 172.21.15.110:0/2614288905 wait complete. 2024-09-10T10:04:16.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.094+0000 7f7597c00640 1 Processor -- start 2024-09-10T10:04:16.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 -- start start 2024-09-10T10:04:16.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7590070a30 0x7f759014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75900d15a0 0x7f75900cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f759015e710 0x7f7590151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f759006e4d0 con 0x7f75900d15a0 2024-09-10T10:04:16.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f759006e350 con 0x7f7590070a30 2024-09-10T10:04:16.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f759006e650 con 0x7f759015e710 2024-09-10T10:04:16.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75900d15a0 0x7f75900cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7595c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f759015e710 0x7f7590151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75900d15a0 0x7f75900cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48592/0 (socket says 172.21.15.110:48592) 2024-09-10T10:04:16.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 -- 172.21.15.110:0/1707144493 learned_addr learned my addr 172.21.15.110:0/1707144493 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:16.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7595200640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7590070a30 0x7f759014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 -- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f759015e710 msgr2=0x7f7590151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f759015e710 0x7f7590151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 -- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7590070a30 msgr2=0x7f759014f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7590070a30 0x7f759014f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 -- 172.21.15.110:0/1707144493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75900ce5f0 con 0x7f75900d15a0 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758fe00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75900d15a0 0x7f75900cdb30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f757c007fd0 tx=0x7f757c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758cc00640 1 -- 172.21.15.110:0/1707144493 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c017070 con 0x7f75900d15a0 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758cc00640 1 -- 172.21.15.110:0/1707144493 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c0052d0 con 0x7f75900d15a0 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75900ce8d0 con 0x7f75900d15a0 2024-09-10T10:04:16.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f758cc00640 1 -- 172.21.15.110:0/1707144493 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c003840 con 0x7f75900d15a0 2024-09-10T10:04:16.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75900cee90 con 0x7f75900d15a0 2024-09-10T10:04:16.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.098+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759006ecd0 con 0x7f75900d15a0 2024-09-10T10:04:16.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.102+0000 7f758cc00640 1 -- 172.21.15.110:0/1707144493 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f757c01a070 con 0x7f75900d15a0 2024-09-10T10:04:16.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.102+0000 7f758cc00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7570067e60 0x7f757006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.102+0000 7f758cc00640 1 -- 172.21.15.110:0/1707144493 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f757c085a40 con 0x7f75900d15a0 2024-09-10T10:04:16.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.102+0000 7f7595200640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7570067e60 0x7f757006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.102+0000 7f7595200640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7570067e60 0x7f757006a320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f75800ae000 tx=0x7f75800a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.106+0000 7f758cc00640 1 -- 172.21.15.110:0/1707144493 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f757c005480 con 0x7f75900d15a0 2024-09-10T10:04:16.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75900b4ef0 con 0x7f75900d15a0 2024-09-10T10:04:16.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f758cc00640 1 -- 172.21.15.110:0/1707144493 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5237 (secure 0 0 0) 0x7f757c0556b0 con 0x7f75900d15a0 2024-09-10T10:04:16.301 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:16.302 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":79,"btime":"2024-09-10T10:04:07:761751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1172125083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1172125083},{"type":"v1","addr":"172.21.15.145:6834","nonce":1172125083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6194,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3529040969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3529040969},{"type":"v1","addr":"172.21.15.145:6835","nonce":3529040969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2459795528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2459795528},{"type":"v1","addr":"172.21.15.110:6835","nonce":2459795528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:58.096051+0000","modified":"2024-09-10T10:04:01.113268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6834/1757537079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1757537079},{"type":"v1","addr":"172.21.15.110:6834","nonce":1757537079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":9}]} 2024-09-10T10:04:16.302 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 79 2024-09-10T10:04:16.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7570067e60 msgr2=0x7f757006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7570067e60 0x7f757006a320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f75800ae000 tx=0x7f75800a7070 comp rx=0 tx=0).stop 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75900d15a0 msgr2=0x7f75900cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75900d15a0 0x7f75900cdb30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f757c007fd0 tx=0x7f757c00c890 comp rx=0 tx=0).stop 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 shutdown_connections 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7570067e60 0x7f757006a320 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f759015e710 0x7f7590151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75900d15a0 0x7f75900cdb30 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 --2- 172.21.15.110:0/1707144493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7590070a30 0x7f759014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 >> 172.21.15.110:0/1707144493 conn(0x7f75900b4290 msgr2=0x7f759015ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 shutdown_connections 2024-09-10T10:04:16.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.298+0000 7f7597c00640 1 -- 172.21.15.110:0/1707144493 wait complete. 2024-09-10T10:04:16.322 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:16.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff363e00640 1 Processor -- start 2024-09-10T10:04:16.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff363e00640 1 -- start start 2024-09-10T10:04:16.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff363e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff35c15d4b0 0x7ff35c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff363e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff35c15c7b0 con 0x7ff35c0d1660 2024-09-10T10:04:16.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff363e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff35c0cfb30 con 0x7ff35c15c3d0 2024-09-10T10:04:16.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff363e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff35c15d920 con 0x7ff35c15d4b0 2024-09-10T10:04:16.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff360a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff35c15d4b0 0x7ff35c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff360a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff35c15d4b0 0x7ff35c0d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48608/0 (socket says 172.21.15.110:48608) 2024-09-10T10:04:16.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff360a00640 1 -- 172.21.15.110:0/1570977884 learned_addr learned my addr 172.21.15.110:0/1570977884 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:16.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff361e00640 1 --1- 172.21.15.110:0/1570977884 >> v1:172.21.15.110:6789/0 conn(0x7ff35c0d1660 0x7ff35c0d1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57928/0 (socket says 172.21.15.110:57928) 2024-09-10T10:04:16.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff360a00640 1 -- 172.21.15.110:0/1570977884 >> v1:172.21.15.145:6789/0 conn(0x7ff35c15c3d0 legacy=0x7ff35c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff360a00640 1 -- 172.21.15.110:0/1570977884 >> v1:172.21.15.110:6789/0 conn(0x7ff35c0d1660 legacy=0x7ff35c0d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.454+0000 7ff360a00640 1 -- 172.21.15.110:0/1570977884 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c0cfe10 con 0x7ff35c15d4b0 2024-09-10T10:04:16.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff360a00640 1 --2- 172.21.15.110:0/1570977884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15d4b0 0x7ff35c0d0bd0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff348009870 tx=0x7ff34801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918719e4334f88b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff35be00640 1 -- 172.21.15.110:0/1570977884 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3480a95f0 con 0x7ff35c15d4b0 2024-09-10T10:04:16.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff35be00640 1 -- 172.21.15.110:0/1570977884 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3480a9790 con 0x7ff35c15d4b0 2024-09-10T10:04:16.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff35be00640 1 -- 172.21.15.110:0/1570977884 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3480a9b60 con 0x7ff35c15d4b0 2024-09-10T10:04:16.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff363e00640 1 -- 172.21.15.110:0/1570977884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15d4b0 msgr2=0x7ff35c0d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff363e00640 1 --2- 172.21.15.110:0/1570977884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15d4b0 0x7ff35c0d0bd0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff348009870 tx=0x7ff34801c530 comp rx=0 tx=0).stop 2024-09-10T10:04:16.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff363e00640 1 -- 172.21.15.110:0/1570977884 shutdown_connections 2024-09-10T10:04:16.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff363e00640 1 --2- 172.21.15.110:0/1570977884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15d4b0 0x7ff35c0d0bd0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff363e00640 1 -- 172.21.15.110:0/1570977884 >> 172.21.15.110:0/1570977884 conn(0x7ff35c06cf60 msgr2=0x7ff35c06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:16.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff363e00640 1 -- 172.21.15.110:0/1570977884 shutdown_connections 2024-09-10T10:04:16.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.458+0000 7ff363e00640 1 -- 172.21.15.110:0/1570977884 wait complete. 2024-09-10T10:04:16.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 Processor -- start 2024-09-10T10:04:16.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 -- start start 2024-09-10T10:04:16.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff35c0d1660 0x7ff35c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15c3d0 0x7ff35c0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff35c15d4b0 0x7ff35c06e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c15e480 con 0x7ff35c15c3d0 2024-09-10T10:04:16.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c15e300 con 0x7ff35c15d4b0 2024-09-10T10:04:16.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff35c15e600 con 0x7ff35c0d1660 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15c3d0 0x7ff35c0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15c3d0 0x7ff35c0b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48610/0 (socket says 172.21.15.110:48610) 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 -- 172.21.15.110:0/1910042808 learned_addr learned my addr 172.21.15.110:0/1910042808 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff361e00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff35c15d4b0 0x7ff35c06e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff361400640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff35c0d1660 0x7ff35c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 -- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff35c0d1660 msgr2=0x7ff35c06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff35c0d1660 0x7ff35c06fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 -- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff35c15d4b0 msgr2=0x7ff35c06e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff35c15d4b0 0x7ff35c06e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 -- 172.21.15.110:0/1910042808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c06ed90 con 0x7ff35c15c3d0 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff361e00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff35c15d4b0 0x7ff35c06e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:16.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff360a00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15c3d0 0x7ff35c0b13b0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff3480ab040 tx=0x7ff34801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff359600640 1 -- 172.21.15.110:0/1910042808 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3480b1800 con 0x7ff35c15c3d0 2024-09-10T10:04:16.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff359600640 1 -- 172.21.15.110:0/1910042808 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3480b19c0 con 0x7ff35c15c3d0 2024-09-10T10:04:16.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff359600640 1 -- 172.21.15.110:0/1910042808 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3480b1db0 con 0x7ff35c15c3d0 2024-09-10T10:04:16.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35c06f070 con 0x7ff35c15c3d0 2024-09-10T10:04:16.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35c06f630 con 0x7ff35c15c3d0 2024-09-10T10:04:16.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.462+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35c0d0bd0 con 0x7ff35c15c3d0 2024-09-10T10:04:16.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.466+0000 7ff359600640 1 -- 172.21.15.110:0/1910042808 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7ff3480c2060 con 0x7ff35c15c3d0 2024-09-10T10:04:16.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.466+0000 7ff359600640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff33c067e90 0x7ff33c06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.466+0000 7ff359600640 1 -- 172.21.15.110:0/1910042808 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7ff348120f70 con 0x7ff35c15c3d0 2024-09-10T10:04:16.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.466+0000 7ff361400640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff33c067e90 0x7ff33c06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.466+0000 7ff361400640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff33c067e90 0x7ff33c06a350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff350000f70 tx=0x7ff350009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.470+0000 7ff359600640 1 -- 172.21.15.110:0/1910042808 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff3480b6070 con 0x7ff35c15c3d0 2024-09-10T10:04:16.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.658+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff35c070c00 con 0x7ff35c15c3d0 2024-09-10T10:04:16.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff359600640 1 -- 172.21.15.110:0/1910042808 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5237 (secure 0 0 0) 0x7ff3480ad020 con 0x7ff35c15c3d0 2024-09-10T10:04:16.665 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:16.665 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":79,"btime":"2024-09-10T10:04:07:761751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1172125083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1172125083},{"type":"v1","addr":"172.21.15.145:6834","nonce":1172125083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6194,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3529040969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3529040969},{"type":"v1","addr":"172.21.15.145:6835","nonce":3529040969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2459795528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2459795528},{"type":"v1","addr":"172.21.15.110:6835","nonce":2459795528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:03:58.096051+0000","modified":"2024-09-10T10:04:01.113268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6834/1757537079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1757537079},{"type":"v1","addr":"172.21.15.110:6834","nonce":1757537079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":9}]} 2024-09-10T10:04:16.666 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 79 2024-09-10T10:04:16.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff33c067e90 msgr2=0x7ff33c06a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff33c067e90 0x7ff33c06a350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff350000f70 tx=0x7ff350009040 comp rx=0 tx=0).stop 2024-09-10T10:04:16.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15c3d0 msgr2=0x7ff35c0b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15c3d0 0x7ff35c0b13b0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff3480ab040 tx=0x7ff34801ad70 comp rx=0 tx=0).stop 2024-09-10T10:04:16.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 shutdown_connections 2024-09-10T10:04:16.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff33c067e90 0x7ff33c06a350 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff35c15d4b0 0x7ff35c06e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff35c15c3d0 0x7ff35c0b13b0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 --2- 172.21.15.110:0/1910042808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff35c0d1660 0x7ff35c06fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 >> 172.21.15.110:0/1910042808 conn(0x7ff35c06cf60 msgr2=0x7ff35c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:16.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 shutdown_connections 2024-09-10T10:04:16.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.662+0000 7ff363e00640 1 -- 172.21.15.110:0/1910042808 wait complete. 2024-09-10T10:04:16.687 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:16.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.822+0000 7f3a52800640 1 Processor -- start 2024-09-10T10:04:16.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.822+0000 7f3a52800640 1 -- start start 2024-09-10T10:04:16.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a52800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3a4c072f70 0x7f3a4c073400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a52800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3a4c0cf2e0 0x7f3a4c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a52800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a4c0cd7e0 con 0x7f3a4c155680 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a52800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3a4c0cfc90 con 0x7f3a4c072f70 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a52800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3a4c0cdac0 con 0x7f3a4c0cf2e0 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a50a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3a4c072f70 0x7f3a4c073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a50a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3a4c072f70 0x7f3a4c073400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48612/0 (socket says 172.21.15.110:48612) 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a51400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3a4c155680 0x7f3a4c0ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57934/0 (socket says 172.21.15.110:57934) 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a50a00640 1 -- 172.21.15.110:0/2984769531 learned_addr learned my addr 172.21.15.110:0/2984769531 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a51e00640 1 --2- 172.21.15.110:0/2984769531 >> v2:172.21.15.145:3300/0 conn(0x7f3a4c0cf2e0 0x7f3a4c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a50a00640 1 -- 172.21.15.110:0/2984769531 >> v2:172.21.15.145:3300/0 conn(0x7f3a4c0cf2e0 msgr2=0x7f3a4c0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a50a00640 1 --2- 172.21.15.110:0/2984769531 >> v2:172.21.15.145:3300/0 conn(0x7f3a4c0cf2e0 0x7f3a4c0cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a50a00640 1 -- 172.21.15.110:0/2984769531 >> v1:172.21.15.110:6789/0 conn(0x7f3a4c155680 legacy=0x7f3a4c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a50a00640 1 -- 172.21.15.110:0/2984769531 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c072770 con 0x7f3a4c072f70 2024-09-10T10:04:16.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a50a00640 1 --2- 172.21.15.110:0/2984769531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c073400 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c073940 tx=0x7f3a3801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63df41df1d3fd626 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.826+0000 7f3a4be00640 1 -- 172.21.15.110:0/2984769531 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a380aa670 con 0x7f3a4c072f70 2024-09-10T10:04:16.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a4be00640 1 -- 172.21.15.110:0/2984769531 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a380af070 con 0x7f3a4c072f70 2024-09-10T10:04:16.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a4be00640 1 -- 172.21.15.110:0/2984769531 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a380aaa30 con 0x7f3a4c072f70 2024-09-10T10:04:16.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- 172.21.15.110:0/2984769531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 msgr2=0x7f3a4c073400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 --2- 172.21.15.110:0/2984769531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c073400 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c073940 tx=0x7f3a3801c4f0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- 172.21.15.110:0/2984769531 shutdown_connections 2024-09-10T10:04:16.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 --2- 172.21.15.110:0/2984769531 >> v2:172.21.15.145:3300/0 conn(0x7f3a4c0cf2e0 0x7f3a4c0cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 --2- 172.21.15.110:0/2984769531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c073400 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- 172.21.15.110:0/2984769531 >> 172.21.15.110:0/2984769531 conn(0x7f3a4c06d0f0 msgr2=0x7f3a4c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:16.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- 172.21.15.110:0/2984769531 shutdown_connections 2024-09-10T10:04:16.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- 172.21.15.110:0/2984769531 wait complete. 2024-09-10T10:04:16.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 Processor -- start 2024-09-10T10:04:16.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- start start 2024-09-10T10:04:16.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a4c0cf2e0 0x7f3a4c149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c155680 0x7f3a4c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c0cd430 con 0x7f3a4c072f70 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c155a60 con 0x7f3a4c155680 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a52800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4c157ed0 con 0x7f3a4c0cf2e0 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a51400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a51400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c145440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48616/0 (socket says 172.21.15.110:48616) 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a51400640 1 -- 172.21.15.110:0/2323382642 learned_addr learned my addr 172.21.15.110:0/2323382642 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a50a00640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a4c0cf2e0 0x7f3a4c149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.830+0000 7f3a51e00640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c155680 0x7f3a4c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a51400640 1 -- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a4c0cf2e0 msgr2=0x7f3a4c149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a51400640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a4c0cf2e0 0x7f3a4c149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a51400640 1 -- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c155680 msgr2=0x7f3a4c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a51400640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c155680 0x7f3a4c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a51400640 1 -- 172.21.15.110:0/2323382642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c155e30 con 0x7f3a4c072f70 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a50a00640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a4c0cf2e0 0x7f3a4c149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a51400640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c145440 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000c370 tx=0x7f3a4000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a49600640 1 -- 172.21.15.110:0/2323382642 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400074e0 con 0x7f3a4c072f70 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a49600640 1 -- 172.21.15.110:0/2323382642 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a40007680 con 0x7f3a4c072f70 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a49600640 1 -- 172.21.15.110:0/2323382642 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40004a30 con 0x7f3a4c072f70 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c156110 con 0x7f3a4c072f70 2024-09-10T10:04:16.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c1566d0 con 0x7f3a4c072f70 2024-09-10T10:04:16.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4c073400 con 0x7f3a4c072f70 2024-09-10T10:04:16.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a49600640 1 -- 172.21.15.110:0/2323382642 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f3a4001a020 con 0x7f3a4c072f70 2024-09-10T10:04:16.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a49600640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3a2c067e60 0x7f3a2c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:16.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.834+0000 7f3a49600640 1 -- 172.21.15.110:0/2323382642 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f3a40085e40 con 0x7f3a4c072f70 2024-09-10T10:04:16.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.838+0000 7f3a50a00640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3a2c067e60 0x7f3a2c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:16.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.838+0000 7f3a50a00640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3a2c067e60 0x7f3a2c06a320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a380ae000 tx=0x7f3a380a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:16.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:16.838+0000 7f3a49600640 1 -- 172.21.15.110:0/2323382642 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3a4008a070 con 0x7f3a4c072f70 2024-09-10T10:04:17.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a4c15b540 con 0x7f3a4c072f70 2024-09-10T10:04:17.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a49600640 1 -- 172.21.15.110:0/2323382642 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16985 (secure 0 0 0) 0x7f3a400560a0 con 0x7f3a4c072f70 2024-09-10T10:04:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:17.006 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":119,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:16.802205+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:03:55.813835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T10:03:57.093856+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":118,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.12","mappings":[{"from":2,"to":0}]},{"pgid":"19.30","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:17.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3a2c067e60 msgr2=0x7f3a2c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3a2c067e60 0x7f3a2c06a320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a380ae000 tx=0x7f3a380a7000 comp rx=0 tx=0).stop 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 msgr2=0x7f3a4c145440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c145440 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000c370 tx=0x7f3a4000c830 comp rx=0 tx=0).stop 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 shutdown_connections 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3a2c067e60 0x7f3a2c06a320 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a4c155680 0x7f3a4c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3a4c0cf2e0 0x7f3a4c149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 --2- 172.21.15.110:0/2323382642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a4c072f70 0x7f3a4c145440 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 >> 172.21.15.110:0/2323382642 conn(0x7f3a4c06d0f0 msgr2=0x7f3a4c15a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 shutdown_connections 2024-09-10T10:04:17.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.002+0000 7f3a52800640 1 -- 172.21.15.110:0/2323382642 wait complete. 2024-09-10T10:04:17.027 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:04:17.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.158+0000 7f219ea00640 1 Processor -- start 2024-09-10T10:04:17.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.158+0000 7f219ea00640 1 -- start start 2024-09-10T10:04:17.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.158+0000 7f219ea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cfb90 0x7f219815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.158+0000 7f219ea00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219815d4d0 con 0x7f21980cdcb0 2024-09-10T10:04:17.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.158+0000 7f219ea00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219815d7b0 con 0x7f21980b3f40 2024-09-10T10:04:17.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.158+0000 7f219ea00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f219806d130 con 0x7f21980cfb90 2024-09-10T10:04:17.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f219ca00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f21980cdcb0 0x7f219806db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57938/0 (socket says 172.21.15.110:57938) 2024-09-10T10:04:17.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f219ca00640 1 -- 172.21.15.110:0/2421686366 learned_addr learned my addr 172.21.15.110:0/2421686366 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:17.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2197e00640 1 --2- 172.21.15.110:0/2421686366 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cfb90 0x7f219815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2196a00640 1 -- 172.21.15.110:0/2421686366 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1240121221 0 0) 0x7f219815d4d0 con 0x7f21980cdcb0 2024-09-10T10:04:17.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2196a00640 1 -- 172.21.15.110:0/2421686366 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f217c003530 con 0x7f21980cdcb0 2024-09-10T10:04:17.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2196a00640 1 -- 172.21.15.110:0/2421686366 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145619729 0 0) 0x7f219815d7b0 con 0x7f21980b3f40 2024-09-10T10:04:17.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2196a00640 1 -- 172.21.15.110:0/2421686366 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f219815d4d0 con 0x7f21980b3f40 2024-09-10T10:04:17.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2197e00640 1 -- 172.21.15.110:0/2421686366 >> v1:172.21.15.145:6789/0 conn(0x7f21980b3f40 legacy=0x7f21980cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2197e00640 1 -- 172.21.15.110:0/2421686366 >> v1:172.21.15.110:6789/0 conn(0x7f21980cdcb0 legacy=0x7f219806db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2197e00640 1 -- 172.21.15.110:0/2421686366 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219806d380 con 0x7f21980cfb90 2024-09-10T10:04:17.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.162+0000 7f2197e00640 1 --2- 172.21.15.110:0/2421686366 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cfb90 0x7f219815ca10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2188009dc0 tx=0x7f218801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2362ff8be3a1390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f2196a00640 1 -- 172.21.15.110:0/2421686366 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21880b2070 con 0x7f21980cfb90 2024-09-10T10:04:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f2196a00640 1 -- 172.21.15.110:0/2421686366 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21880a57c0 con 0x7f21980cfb90 2024-09-10T10:04:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f2196a00640 1 -- 172.21.15.110:0/2421686366 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21880a5b70 con 0x7f21980cfb90 2024-09-10T10:04:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- 172.21.15.110:0/2421686366 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cfb90 msgr2=0x7f219815ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 --2- 172.21.15.110:0/2421686366 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cfb90 0x7f219815ca10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2188009dc0 tx=0x7f218801c900 comp rx=0 tx=0).stop 2024-09-10T10:04:17.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- 172.21.15.110:0/2421686366 shutdown_connections 2024-09-10T10:04:17.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 --2- 172.21.15.110:0/2421686366 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cfb90 0x7f219815ca10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- 172.21.15.110:0/2421686366 >> 172.21.15.110:0/2421686366 conn(0x7f2198062d10 msgr2=0x7f219806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:17.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- 172.21.15.110:0/2421686366 shutdown_connections 2024-09-10T10:04:17.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- 172.21.15.110:0/2421686366 wait complete. 2024-09-10T10:04:17.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 Processor -- start 2024-09-10T10:04:17.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- start start 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21980b3f40 0x7f2198072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cdcb0 0x7f219814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21980cfb90 0x7f21980ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f219816a390 con 0x7f21980b3f40 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f219806eaa0 con 0x7f21980cfb90 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ea00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f219816a510 con 0x7f21980cdcb0 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f2197e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21980b3f40 0x7f2198072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f2197e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21980b3f40 0x7f2198072b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48630/0 (socket says 172.21.15.110:48630) 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f2197e00640 1 -- 172.21.15.110:0/3323256518 learned_addr learned my addr 172.21.15.110:0/3323256518 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:17.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f219ca00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21980cfb90 0x7f21980ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.166+0000 7f2197400640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cdcb0 0x7f219814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f2197e00640 1 -- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cdcb0 msgr2=0x7f219814c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f2197e00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cdcb0 0x7f219814c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f2197e00640 1 -- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21980cfb90 msgr2=0x7f21980ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f2197e00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21980cfb90 0x7f21980ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f2197e00640 1 -- 172.21.15.110:0/3323256518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2198071650 con 0x7f21980b3f40 2024-09-10T10:04:17.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f2197e00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21980b3f40 0x7f2198072b20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f218801a530 tx=0x7f21880196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:17.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f218fe00640 1 -- 172.21.15.110:0/3323256518 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21880bd020 con 0x7f21980b3f40 2024-09-10T10:04:17.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f218fe00640 1 -- 172.21.15.110:0/3323256518 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21880a5e50 con 0x7f21980b3f40 2024-09-10T10:04:17.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2198071930 con 0x7f21980b3f40 2024-09-10T10:04:17.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f218fe00640 1 -- 172.21.15.110:0/3323256518 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21880b2040 con 0x7f21980b3f40 2024-09-10T10:04:17.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2198071ef0 con 0x7f21980b3f40 2024-09-10T10:04:17.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2198153b90 con 0x7f21980b3f40 2024-09-10T10:04:17.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f218fe00640 1 -- 172.21.15.110:0/3323256518 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f21880c0020 con 0x7f21980b3f40 2024-09-10T10:04:17.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f218fe00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2178067e90 0x7f217806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f218fe00640 1 -- 172.21.15.110:0/3323256518 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f2188121370 con 0x7f21980b3f40 2024-09-10T10:04:17.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.170+0000 7f2197400640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2178067e90 0x7f217806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.174+0000 7f2197400640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2178067e90 0x7f217806a350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2198157ed0 tx=0x7f2184009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:17.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.174+0000 7f218fe00640 1 -- 172.21.15.110:0/3323256518 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f21880c7070 con 0x7f21980b3f40 2024-09-10T10:04:17.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.366+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f219815ca10 con 0x7f21980b3f40 2024-09-10T10:04:17.370 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:04:17.366+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:04:17.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.366+0000 7f218fe00640 1 -- 172.21.15.110:0/3323256518 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) ==== 161+0+0 (secure 0 0 0) 0x7f2188016580 con 0x7f21980b3f40 2024-09-10T10:04:17.372 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:04:17.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2178067e90 msgr2=0x7f217806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2178067e90 0x7f217806a350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2198157ed0 tx=0x7f2184009040 comp rx=0 tx=0).stop 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21980b3f40 msgr2=0x7f2198072b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21980b3f40 0x7f2198072b20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f218801a530 tx=0x7f21880196a0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 shutdown_connections 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2178067e90 0x7f217806a350 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f21980cfb90 0x7f21980ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21980cdcb0 0x7f219814c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 --2- 172.21.15.110:0/3323256518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21980b3f40 0x7f2198072b20 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 >> 172.21.15.110:0/3323256518 conn(0x7f2198062d10 msgr2=0x7f219815f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 shutdown_connections 2024-09-10T10:04:17.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.370+0000 7f219ea00640 1 -- 172.21.15.110:0/3323256518 wait complete. 2024-09-10T10:04:17.397 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:04:17.442 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:04:17.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be8200640 1 Processor -- start 2024-09-10T10:04:17.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be8200640 1 -- start start 2024-09-10T10:04:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be8200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be015e9b0 0x7f2be006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be8200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2be00d2b70 0x7f2be0162a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be8200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be00b5970 con 0x7f2be00734d0 2024-09-10T10:04:17.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be8200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2be0162600 con 0x7f2be00d2b70 2024-09-10T10:04:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be8200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2be006e910 con 0x7f2be015e9b0 2024-09-10T10:04:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be4e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2be00734d0 0x7f2be00d2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:57962/0 (socket says 172.21.15.110:57962) 2024-09-10T10:04:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be4e00640 1 -- 172.21.15.110:0/1751378898 learned_addr learned my addr 172.21.15.110:0/1751378898 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:17.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be5800640 1 --2- 172.21.15.110:0/1751378898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be015e9b0 0x7f2be006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be6200640 1 --2- 172.21.15.110:0/1751378898 >> v2:172.21.15.145:3300/0 conn(0x7f2be00d2b70 0x7f2be0162a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2bdfe00640 1 -- 172.21.15.110:0/1751378898 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321653761 0 0) 0x7f2be00b5970 con 0x7f2be00734d0 2024-09-10T10:04:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2bdfe00640 1 -- 172.21.15.110:0/1751378898 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bc8003530 con 0x7f2be00734d0 2024-09-10T10:04:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be6200640 1 -- 172.21.15.110:0/1751378898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be015e9b0 msgr2=0x7f2be006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be6200640 1 --2- 172.21.15.110:0/1751378898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be015e9b0 0x7f2be006cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be6200640 1 -- 172.21.15.110:0/1751378898 >> v1:172.21.15.110:6789/0 conn(0x7f2be00734d0 legacy=0x7f2be00d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.542+0000 7f2be6200640 1 -- 172.21.15.110:0/1751378898 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be006ebf0 con 0x7f2be00d2b70 2024-09-10T10:04:17.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be6200640 1 --2- 172.21.15.110:0/1751378898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be00d2b70 0x7f2be0162a90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80097c0 tx=0x7f2bd801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb880cdcd3eb193 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:17.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2bdfe00640 1 -- 172.21.15.110:0/1751378898 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80b1070 con 0x7f2be00d2b70 2024-09-10T10:04:17.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2bdfe00640 1 -- 172.21.15.110:0/1751378898 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd80ac410 con 0x7f2be00d2b70 2024-09-10T10:04:17.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2bdfe00640 1 -- 172.21.15.110:0/1751378898 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80ac7c0 con 0x7f2be00d2b70 2024-09-10T10:04:17.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be8200640 1 -- 172.21.15.110:0/1751378898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be00d2b70 msgr2=0x7f2be0162a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be8200640 1 --2- 172.21.15.110:0/1751378898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be00d2b70 0x7f2be0162a90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80097c0 tx=0x7f2bd801c4a0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be8200640 1 -- 172.21.15.110:0/1751378898 shutdown_connections 2024-09-10T10:04:17.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be8200640 1 --2- 172.21.15.110:0/1751378898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be00d2b70 0x7f2be0162a90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be8200640 1 --2- 172.21.15.110:0/1751378898 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be015e9b0 0x7f2be006cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be8200640 1 -- 172.21.15.110:0/1751378898 >> 172.21.15.110:0/1751378898 conn(0x7f2be006f420 msgr2=0x7f2be006f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:17.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be8200640 1 -- 172.21.15.110:0/1751378898 shutdown_connections 2024-09-10T10:04:17.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.546+0000 7f2be8200640 1 -- 172.21.15.110:0/1751378898 wait complete. 2024-09-10T10:04:17.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 Processor -- start 2024-09-10T10:04:17.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 -- start start 2024-09-10T10:04:17.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be00734d0 0x7f2be0152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2be00d2b70 0x7f2be015f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be015e9b0 0x7f2be0148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0163560 con 0x7f2be00d2b70 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01633e0 con 0x7f2be015e9b0 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2be016bfa0 con 0x7f2be00734d0 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2be00d2b70 0x7f2be015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2be00d2b70 0x7f2be015f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48670/0 (socket says 172.21.15.110:48670) 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 -- 172.21.15.110:0/3668813332 learned_addr learned my addr 172.21.15.110:0/3668813332 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be6200640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be015e9b0 0x7f2be0148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be5800640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be00734d0 0x7f2be0152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 -- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be00734d0 msgr2=0x7f2be0152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be00734d0 0x7f2be0152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 -- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be015e9b0 msgr2=0x7f2be0148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:17.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be015e9b0 0x7f2be0148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:17.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 -- 172.21.15.110:0/3668813332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be0153560 con 0x7f2be00d2b70 2024-09-10T10:04:17.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be4e00640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2be00d2b70 0x7f2be015f920 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c420 tx=0x7f2bcc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:17.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2bdd600640 1 -- 172.21.15.110:0/3668813332 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc017070 con 0x7f2be00d2b70 2024-09-10T10:04:17.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2bdd600640 1 -- 172.21.15.110:0/3668813332 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bcc005d10 con 0x7f2be00d2b70 2024-09-10T10:04:17.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be0153840 con 0x7f2be00d2b70 2024-09-10T10:04:17.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2bdd600640 1 -- 172.21.15.110:0/3668813332 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc007310 con 0x7f2be00d2b70 2024-09-10T10:04:17.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.550+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be0149370 con 0x7f2be00d2b70 2024-09-10T10:04:17.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.554+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be0162a90 con 0x7f2be00d2b70 2024-09-10T10:04:17.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.554+0000 7f2bdd600640 1 -- 172.21.15.110:0/3668813332 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f2bcc01a070 con 0x7f2be00d2b70 2024-09-10T10:04:17.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.554+0000 7f2bdd600640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bc0067e60 0x7f2bc006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:17.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.554+0000 7f2bdd600640 1 -- 172.21.15.110:0/3668813332 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5909+0+0 (secure 0 0 0) 0x7f2bcc085b40 con 0x7f2be00d2b70 2024-09-10T10:04:17.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.554+0000 7f2be5800640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bc0067e60 0x7f2bc006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:17.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.554+0000 7f2be5800640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bc0067e60 0x7f2bc006a320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4011340 tx=0x7f2bd400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:17.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.558+0000 7f2bdd600640 1 -- 172.21.15.110:0/3668813332 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2bcc055800 con 0x7f2be00d2b70 2024-09-10T10:04:17.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:17.750+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2be0071760 con 0x7f2be00d2b70 2024-09-10T10:04:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.370+0000 7f2bdd600640 1 -- 172.21.15.110:0/3668813332 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) ==== 104+0+0 (secure 0 0 0) 0x7f2bcc059b00 con 0x7f2be00d2b70 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.370+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bc0067e60 msgr2=0x7f2bc006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.370+0000 7f2be8200640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bc0067e60 0x7f2bc006a320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4011340 tx=0x7f2bd400b040 comp rx=0 tx=0).stop 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.370+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2be00d2b70 msgr2=0x7f2be015f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.370+0000 7f2be8200640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2be00d2b70 0x7f2be015f920 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c420 tx=0x7f2bcc00c8e0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.370+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 shutdown_connections 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.374+0000 7f2be8200640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bc0067e60 0x7f2bc006a320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.374+0000 7f2be8200640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2be015e9b0 0x7f2be0148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.374+0000 7f2be8200640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2be00d2b70 0x7f2be015f920 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.374+0000 7f2be8200640 1 --2- 172.21.15.110:0/3668813332 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2be00734d0 0x7f2be0152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.374+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 >> 172.21.15.110:0/3668813332 conn(0x7f2be006f420 msgr2=0x7f2be01517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.374+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 shutdown_connections 2024-09-10T10:04:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.374+0000 7f2be8200640 1 -- 172.21.15.110:0/3668813332 wait complete. 2024-09-10T10:04:18.398 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:04:18.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.578+0000 7f0a19e00640 1 Processor -- start 2024-09-10T10:04:18.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.578+0000 7f0a19e00640 1 -- start start 2024-09-10T10:04:18.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a19e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0a1415c560 0x7f0a1406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a19e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1406f010 0x7f0a140ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a19e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a1406f430 con 0x7f0a14155970 2024-09-10T10:04:18.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a19e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0a140b4570 con 0x7f0a1415c560 2024-09-10T10:04:18.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a19e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0a14155d50 con 0x7f0a1406f010 2024-09-10T10:04:18.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a12000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0a1415c560 0x7f0a1406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a12a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0a14155970 0x7f0a140cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37724/0 (socket says 172.21.15.110:37724) 2024-09-10T10:04:18.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a12a00640 1 -- 172.21.15.110:0/952695829 learned_addr learned my addr 172.21.15.110:0/952695829 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:18.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a13400640 1 --2- 172.21.15.110:0/952695829 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1406f010 0x7f0a140ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a12000640 1 -- 172.21.15.110:0/952695829 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1406f010 msgr2=0x7f0a140ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:18.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a12000640 1 --2- 172.21.15.110:0/952695829 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1406f010 0x7f0a140ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a12000640 1 -- 172.21.15.110:0/952695829 >> v1:172.21.15.110:6789/0 conn(0x7f0a14155970 legacy=0x7f0a140cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.582+0000 7f0a12000640 1 -- 172.21.15.110:0/952695829 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a140b34d0 con 0x7f0a1415c560 2024-09-10T10:04:18.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a12000640 1 --2- 172.21.15.110:0/952695829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a1415c560 0x7f0a1406ead0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0a000097c0 tx=0x7f0a0001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e95d26761fb1fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:18.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a11600640 1 -- 172.21.15.110:0/952695829 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a000a7070 con 0x7f0a1415c560 2024-09-10T10:04:18.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a11600640 1 -- 172.21.15.110:0/952695829 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a000b0070 con 0x7f0a1415c560 2024-09-10T10:04:18.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a11600640 1 -- 172.21.15.110:0/952695829 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a000ab890 con 0x7f0a1415c560 2024-09-10T10:04:18.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 -- 172.21.15.110:0/952695829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a1415c560 msgr2=0x7f0a1406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 --2- 172.21.15.110:0/952695829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a1415c560 0x7f0a1406ead0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0a000097c0 tx=0x7f0a0001c470 comp rx=0 tx=0).stop 2024-09-10T10:04:18.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 -- 172.21.15.110:0/952695829 shutdown_connections 2024-09-10T10:04:18.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 --2- 172.21.15.110:0/952695829 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1406f010 0x7f0a140ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 --2- 172.21.15.110:0/952695829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a1415c560 0x7f0a1406ead0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 -- 172.21.15.110:0/952695829 >> 172.21.15.110:0/952695829 conn(0x7f0a1406bc50 msgr2=0x7f0a1406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:18.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 -- 172.21.15.110:0/952695829 shutdown_connections 2024-09-10T10:04:18.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 -- 172.21.15.110:0/952695829 wait complete. 2024-09-10T10:04:18.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 Processor -- start 2024-09-10T10:04:18.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 -- start start 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a1406f010 0x7f0a1414f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a14155970 0x7f0a14154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.586+0000 7f0a19e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1415c560 0x7f0a14154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a19e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a140cf170 con 0x7f0a14155970 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a19e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0a140b4050 con 0x7f0a1406f010 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a19e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0a140cf2f0 con 0x7f0a1415c560 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a14155970 0x7f0a14154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a14155970 0x7f0a14154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49586/0 (socket says 172.21.15.110:49586) 2024-09-10T10:04:18.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 -- 172.21.15.110:0/674736433 learned_addr learned my addr 172.21.15.110:0/674736433 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a13400640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1415c560 0x7f0a14154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 -- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1415c560 msgr2=0x7f0a14154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12a00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a1406f010 0x7f0a1414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1415c560 0x7f0a14154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 -- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a1406f010 msgr2=0x7f0a1414f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a1406f010 0x7f0a1414f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 -- 172.21.15.110:0/674736433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a140ab570 con 0x7f0a14155970 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a13400640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1415c560 0x7f0a14154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12a00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a1406f010 0x7f0a1414f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a12000640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a14155970 0x7f0a14154a40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0a0001a0a0 tx=0x7f0a0001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a0b400640 1 -- 172.21.15.110:0/674736433 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a000bf610 con 0x7f0a14155970 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a0b400640 1 -- 172.21.15.110:0/674736433 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a000bf7d0 con 0x7f0a14155970 2024-09-10T10:04:18.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a140ab850 con 0x7f0a14155970 2024-09-10T10:04:18.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a0b400640 1 -- 172.21.15.110:0/674736433 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a000bfbc0 con 0x7f0a14155970 2024-09-10T10:04:18.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a140abe10 con 0x7f0a14155970 2024-09-10T10:04:18.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a140adf20 con 0x7f0a14155970 2024-09-10T10:04:18.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.590+0000 7f0a0b400640 1 -- 172.21.15.110:0/674736433 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f0a000bd020 con 0x7f0a14155970 2024-09-10T10:04:18.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.594+0000 7f0a0b400640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f09f4067e60 0x7f09f406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:18.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.594+0000 7f0a0b400640 1 -- 172.21.15.110:0/674736433 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5909+0+0 (secure 0 0 0) 0x7f0a001211e0 con 0x7f0a14155970 2024-09-10T10:04:18.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.594+0000 7f0a12a00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f09f4067e60 0x7f09f406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:18.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.594+0000 7f0a12a00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f09f4067e60 0x7f09f406a320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000f70 tx=0x7f0a04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:18.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.598+0000 7f0a0b400640 1 -- 172.21.15.110:0/674736433 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0a00016090 con 0x7f0a14155970 2024-09-10T10:04:18.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:18.754+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0a140ce420 con 0x7f0a14155970 2024-09-10T10:04:19.264 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:04:19.268 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:04:19.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.378+0000 7f0a0b400640 1 -- 172.21.15.110:0/674736433 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) ==== 181+0+0 (secure 0 0 0) 0x7f0a000f1290 con 0x7f0a14155970 2024-09-10T10:04:19.382 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:04:19.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f09f4067e60 msgr2=0x7f09f406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f09f4067e60 0x7f09f406a320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000f70 tx=0x7f0a04009040 comp rx=0 tx=0).stop 2024-09-10T10:04:19.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a14155970 msgr2=0x7f0a14154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a14155970 0x7f0a14154a40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0a0001a0a0 tx=0x7f0a0001acb0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 shutdown_connections 2024-09-10T10:04:19.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f09f4067e60 0x7f09f406a320 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0a1415c560 0x7f0a14154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a14155970 0x7f0a14154a40 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 --2- 172.21.15.110:0/674736433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a1406f010 0x7f0a1414f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 >> 172.21.15.110:0/674736433 conn(0x7f0a1406bc50 msgr2=0x7f0a1415ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:19.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 shutdown_connections 2024-09-10T10:04:19.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.382+0000 7f0a19e00640 1 -- 172.21.15.110:0/674736433 wait complete. 2024-09-10T10:04:19.406 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:04:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa544a00640 1 Processor -- start 2024-09-10T10:04:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa544a00640 1 -- start start 2024-09-10T10:04:19.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa544a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa540155970 0x7fa5400cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa544a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa540155d50 con 0x7fa54006f1e0 2024-09-10T10:04:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa544a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5400b34d0 con 0x7fa54015c560 2024-09-10T10:04:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa544a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa5400b4570 con 0x7fa540155970 2024-09-10T10:04:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa53e800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa54006f1e0 0x7fa5400ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37732/0 (socket says 172.21.15.110:37732) 2024-09-10T10:04:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa53e800640 1 -- 172.21.15.110:0/1009529345 learned_addr learned my addr 172.21.15.110:0/1009529345 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa53de00640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa540155970 0x7fa5400cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2272302304 0 0) 0x7fa540155d50 con 0x7fa54006f1e0 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.542+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa524003530 con 0x7fa54006f1e0 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3601860752 0 0) 0x7fa524003530 con 0x7fa54006f1e0 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa540155970 msgr2=0x7fa5400cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa540155970 0x7fa5400cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 >> v1:172.21.15.145:6789/0 conn(0x7fa54015c560 legacy=0x7fa54006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5400b37b0 con 0x7fa54006f1e0 2024-09-10T10:04:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fa538001680 con 0x7fa54006f1e0 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 >> v1:172.21.15.110:6789/0 conn(0x7fa54006f1e0 legacy=0x7fa5400ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa540155970 0x7fa5240039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa524003f70 0x7fa524004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa5240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa52400bb60 con 0x7fa524004d50 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa52400bce0 con 0x7fa524003f70 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa52400bee0 con 0x7fa540155970 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53e800640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa5240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53de00640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa540155970 0x7fa5240039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53d400640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa524003f70 0x7fa524004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53e800640 1 -- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa540155970 msgr2=0x7fa5240039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53e800640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa540155970 0x7fa5240039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53e800640 1 -- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa524003f70 msgr2=0x7fa524004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53e800640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa524003f70 0x7fa524004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53e800640 1 -- 172.21.15.110:0/1009529345 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa524003530 con 0x7fa524004d50 2024-09-10T10:04:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.546+0000 7fa53e800640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa5240051d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c240 tx=0x7fa53801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9be3bfd09b299d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:19.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538007d20 con 0x7fa524004d50 2024-09-10T10:04:19.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa538005ae0 con 0x7fa524004d50 2024-09-10T10:04:19.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa53ca00640 1 -- 172.21.15.110:0/1009529345 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538007070 con 0x7fa524004d50 2024-09-10T10:04:19.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- 172.21.15.110:0/1009529345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 msgr2=0x7fa5240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 --2- 172.21.15.110:0/1009529345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa5240051d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c240 tx=0x7fa53801f3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa53e800640 1 -- 172.21.15.110:0/1009529345 reap_dead start 2024-09-10T10:04:19.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- 172.21.15.110:0/1009529345 shutdown_connections 2024-09-10T10:04:19.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- 172.21.15.110:0/1009529345 >> 172.21.15.110:0/1009529345 conn(0x7fa54006bc50 msgr2=0x7fa54006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:19.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- 172.21.15.110:0/1009529345 shutdown_connections 2024-09-10T10:04:19.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- 172.21.15.110:0/1009529345 wait complete. 2024-09-10T10:04:19.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 Processor -- start 2024-09-10T10:04:19.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- start start 2024-09-10T10:04:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa54006eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa54015f610 0x7fa54014e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa54014eed0 0x7fa54015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa5400b4050 con 0x7fa524004d50 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa5400b3d70 con 0x7fa54015f610 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa544a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa5400cc8c0 con 0x7fa54014eed0 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa53de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa54006eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa53de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa54006eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49614/0 (socket says 172.21.15.110:49614) 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.550+0000 7fa53de00640 1 -- 172.21.15.110:0/3039959280 learned_addr learned my addr 172.21.15.110:0/3039959280 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa53de00640 1 -- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa54014eed0 msgr2=0x7fa54015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa53de00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa54014eed0 0x7fa54015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa53de00640 1 -- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa54015f610 msgr2=0x7fa54014e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa53de00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa54015f610 0x7fa54014e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa53de00640 1 -- 172.21.15.110:0/3039959280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54015fa90 con 0x7fa524004d50 2024-09-10T10:04:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa53de00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa54006eb50 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa530011e30 tx=0x7fa53000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa536000640 1 -- 172.21.15.110:0/3039959280 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530015630 con 0x7fa524004d50 2024-09-10T10:04:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa536000640 1 -- 172.21.15.110:0/3039959280 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5300157d0 con 0x7fa524004d50 2024-09-10T10:04:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5401546b0 con 0x7fa524004d50 2024-09-10T10:04:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa536000640 1 -- 172.21.15.110:0/3039959280 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530015ba0 con 0x7fa524004d50 2024-09-10T10:04:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54015ee10 con 0x7fa524004d50 2024-09-10T10:04:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5400ccce0 con 0x7fa524004d50 2024-09-10T10:04:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa536000640 1 -- 172.21.15.110:0/3039959280 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fa53001d400 con 0x7fa524004d50 2024-09-10T10:04:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa536000640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa520067e90 0x7fa52006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.554+0000 7fa536000640 1 -- 172.21.15.110:0/3039959280 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5471+0+0 (secure 0 0 0) 0x7fa5300875e0 con 0x7fa524004d50 2024-09-10T10:04:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.558+0000 7fa53d400640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa520067e90 0x7fa52006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:19.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.558+0000 7fa53d400640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa520067e90 0x7fa52006a350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa540073210 tx=0x7fa52c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:19.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.558+0000 7fa536000640 1 -- 172.21.15.110:0/3039959280 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa530019030 con 0x7fa524004d50 2024-09-10T10:04:19.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:19.718+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5400d0070 con 0x7fa524004d50 2024-09-10T10:04:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.382+0000 7fa536000640 1 -- 172.21.15.110:0/3039959280 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) ==== 169+0+0 (secure 0 0 0) 0x7fa530010040 con 0x7fa524004d50 2024-09-10T10:04:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:04:20.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa520067e90 msgr2=0x7fa52006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa520067e90 0x7fa52006a350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa540073210 tx=0x7fa52c009040 comp rx=0 tx=0).stop 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 msgr2=0x7fa54006eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa54006eb50 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa530011e30 tx=0x7fa53000f580 comp rx=0 tx=0).stop 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 shutdown_connections 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa520067e90 0x7fa52006a350 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa54014eed0 0x7fa54015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa54015f610 0x7fa54014e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 --2- 172.21.15.110:0/3039959280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524004d50 0x7fa54006eb50 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 >> 172.21.15.110:0/3039959280 conn(0x7fa54006bc50 msgr2=0x7fa540150f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 shutdown_connections 2024-09-10T10:04:20.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.386+0000 7fa544a00640 1 -- 172.21.15.110:0/3039959280 wait complete. 2024-09-10T10:04:20.413 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:04:20.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d8800640 1 Processor -- start 2024-09-10T10:04:20.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d8800640 1 -- start start 2024-09-10T10:04:20.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d8800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d015b330 0x7fd0d006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d8800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd0d006e850 0x7fd0d00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d8800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d00b4960 con 0x7fd0d00ce4d0 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d8800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d00b5570 con 0x7fd0d006e850 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d8800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d00b4c40 con 0x7fd0d015b330 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d6800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd0d006e850 0x7fd0d00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d6800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd0d006e850 0x7fd0d00b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49620/0 (socket says 172.21.15.110:49620) 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d6800640 1 -- 172.21.15.110:0/2544566534 learned_addr learned my addr 172.21.15.110:0/2544566534 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d5e00640 1 --2- 172.21.15.110:0/2544566534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d015b330 0x7fd0d006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d6800640 1 -- 172.21.15.110:0/2544566534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d015b330 msgr2=0x7fd0d006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d6800640 1 --2- 172.21.15.110:0/2544566534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d015b330 0x7fd0d006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d6800640 1 -- 172.21.15.110:0/2544566534 >> v1:172.21.15.145:6789/0 conn(0x7fd0d00ce4d0 legacy=0x7fd0d006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d6800640 1 -- 172.21.15.110:0/2544566534 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d00b4f20 con 0x7fd0d006e850 2024-09-10T10:04:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.546+0000 7fd0d5e00640 1 --2- 172.21.15.110:0/2544566534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d015b330 0x7fd0d006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:20.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d6800640 1 --2- 172.21.15.110:0/2544566534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d006e850 0x7fd0d00b0860 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009e50 tx=0x7fd0cc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed76b69663fe089 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d4a00640 1 -- 172.21.15.110:0/2544566534 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0a5810 con 0x7fd0d006e850 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d4a00640 1 -- 172.21.15.110:0/2544566534 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0cc0a59b0 con 0x7fd0d006e850 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d4a00640 1 -- 172.21.15.110:0/2544566534 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0a5d80 con 0x7fd0d006e850 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d8800640 1 -- 172.21.15.110:0/2544566534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d006e850 msgr2=0x7fd0d00b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2544566534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d006e850 0x7fd0d00b0860 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009e50 tx=0x7fd0cc01ca30 comp rx=0 tx=0).stop 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d8800640 1 -- 172.21.15.110:0/2544566534 shutdown_connections 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2544566534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d006e850 0x7fd0d00b0860 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2544566534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d015b330 0x7fd0d006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d8800640 1 -- 172.21.15.110:0/2544566534 >> 172.21.15.110:0/2544566534 conn(0x7fd0d0062e40 msgr2=0x7fd0d006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d8800640 1 -- 172.21.15.110:0/2544566534 shutdown_connections 2024-09-10T10:04:20.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.550+0000 7fd0d8800640 1 -- 172.21.15.110:0/2544566534 wait complete. 2024-09-10T10:04:20.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 Processor -- start 2024-09-10T10:04:20.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 -- start start 2024-09-10T10:04:20.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0d006e850 0x7fd0d015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d015b330 0x7fd0d0156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d014c930 0x7fd0d00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01680b0 con 0x7fd0d015b330 2024-09-10T10:04:20.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d00b17c0 con 0x7fd0d006e850 2024-09-10T10:04:20.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d0168230 con 0x7fd0d014c930 2024-09-10T10:04:20.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d015b330 0x7fd0d0156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d015b330 0x7fd0d0156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49634/0 (socket says 172.21.15.110:49634) 2024-09-10T10:04:20.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 -- 172.21.15.110:0/2558285378 learned_addr learned my addr 172.21.15.110:0/2558285378 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:20.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5e00640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0d006e850 0x7fd0d015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d6800640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d014c930 0x7fd0d00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 -- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d014c930 msgr2=0x7fd0d00ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d014c930 0x7fd0d00ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 -- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0d006e850 msgr2=0x7fd0d015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0d006e850 0x7fd0d015b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 -- 172.21.15.110:0/2558285378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d015bff0 con 0x7fd0d015b330 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d5400640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d015b330 0x7fd0d0156fe0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000c430 tx=0x7fd0c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0ca000640 1 -- 172.21.15.110:0/2558285378 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00052d0 con 0x7fd0d015b330 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0ca000640 1 -- 172.21.15.110:0/2558285378 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c0005470 con 0x7fd0d015b330 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0ca000640 1 -- 172.21.15.110:0/2558285378 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0007260 con 0x7fd0d015b330 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d015c2d0 con 0x7fd0d015b330 2024-09-10T10:04:20.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.554+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d015c890 con 0x7fd0d015b330 2024-09-10T10:04:20.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.558+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d006c6a0 con 0x7fd0d015b330 2024-09-10T10:04:20.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.558+0000 7fd0ca000640 1 -- 172.21.15.110:0/2558285378 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fd0c0007400 con 0x7fd0d015b330 2024-09-10T10:04:20.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.558+0000 7fd0ca000640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd0b4067e60 0x7fd0b406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.558+0000 7fd0ca000640 1 -- 172.21.15.110:0/2558285378 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4999+0+0 (secure 0 0 0) 0x7fd0c00858d0 con 0x7fd0d015b330 2024-09-10T10:04:20.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.558+0000 7fd0d5e00640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd0b4067e60 0x7fd0b406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.558+0000 7fd0d5e00640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd0b4067e60 0x7fd0b406a320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006e90 tx=0x7fd0c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.562+0000 7fd0ca000640 1 -- 172.21.15.110:0/2558285378 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd0c00557d0 con 0x7fd0d015b330 2024-09-10T10:04:20.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.718+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd0d00cff10 con 0x7fd0d015b330 2024-09-10T10:04:20.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.722+0000 7fd0ca000640 1 -- 172.21.15.110:0/2558285378 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fd0c0059ad0 con 0x7fd0d015b330 2024-09-10T10:04:20.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd0b4067e60 msgr2=0x7fd0b406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd0b4067e60 0x7fd0b406a320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006e90 tx=0x7fd0c4008040 comp rx=0 tx=0).stop 2024-09-10T10:04:20.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d015b330 msgr2=0x7fd0d0156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d015b330 0x7fd0d0156fe0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000c430 tx=0x7fd0c000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 shutdown_connections 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd0b4067e60 0x7fd0b406a320 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd0d014c930 0x7fd0d00ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0d015b330 0x7fd0d0156fe0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 --2- 172.21.15.110:0/2558285378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0d006e850 0x7fd0d015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 >> 172.21.15.110:0/2558285378 conn(0x7fd0d0062e40 msgr2=0x7fd0d00cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 shutdown_connections 2024-09-10T10:04:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.726+0000 7fd0d8800640 1 -- 172.21.15.110:0/2558285378 wait complete. 2024-09-10T10:04:20.750 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-10T10:04:20.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.882+0000 7fad72400640 1 Processor -- start 2024-09-10T10:04:20.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.882+0000 7fad72400640 1 -- start start 2024-09-10T10:04:20.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.882+0000 7fad72400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fad6c15c960 0x7fad6c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.882+0000 7fad72400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c1603c0 0x7fad6c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.882+0000 7fad72400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6c0706a0 con 0x7fad6c0d04c0 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.882+0000 7fad72400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fad6c15cd40 con 0x7fad6c15c960 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.882+0000 7fad72400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c0b6370 con 0x7fad6c1603c0 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fad6c15c960 0x7fad6c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fad6c15c960 0x7fad6c0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49640/0 (socket says 172.21.15.110:49640) 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6b400640 1 -- 172.21.15.110:0/673264222 learned_addr learned my addr 172.21.15.110:0/673264222 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6aa00640 1 --2- 172.21.15.110:0/673264222 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c1603c0 0x7fad6c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6b400640 1 -- 172.21.15.110:0/673264222 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c1603c0 msgr2=0x7fad6c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6b400640 1 --2- 172.21.15.110:0/673264222 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c1603c0 0x7fad6c0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6b400640 1 -- 172.21.15.110:0/673264222 >> v1:172.21.15.145:6789/0 conn(0x7fad6c0d04c0 legacy=0x7fad6c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6b400640 1 -- 172.21.15.110:0/673264222 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c070a80 con 0x7fad6c15c960 2024-09-10T10:04:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6b400640 1 --2- 172.21.15.110:0/673264222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c15c960 0x7fad6c0b5e30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009870 tx=0x7fad5c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daab68a4ec2f9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6a000640 1 -- 172.21.15.110:0/673264222 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0b0070 con 0x7fad6c15c960 2024-09-10T10:04:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6a000640 1 -- 172.21.15.110:0/673264222 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad5c0ab600 con 0x7fad6c15c960 2024-09-10T10:04:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.886+0000 7fad6a000640 1 -- 172.21.15.110:0/673264222 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0ab9d0 con 0x7fad6c15c960 2024-09-10T10:04:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- 172.21.15.110:0/673264222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c15c960 msgr2=0x7fad6c0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 --2- 172.21.15.110:0/673264222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c15c960 0x7fad6c0b5e30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009870 tx=0x7fad5c01c3d0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- 172.21.15.110:0/673264222 shutdown_connections 2024-09-10T10:04:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 --2- 172.21.15.110:0/673264222 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c1603c0 0x7fad6c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 --2- 172.21.15.110:0/673264222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c15c960 0x7fad6c0b5e30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- 172.21.15.110:0/673264222 >> 172.21.15.110:0/673264222 conn(0x7fad6c0d25c0 msgr2=0x7fad6c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- 172.21.15.110:0/673264222 shutdown_connections 2024-09-10T10:04:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- 172.21.15.110:0/673264222 wait complete. 2024-09-10T10:04:20.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 Processor -- start 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- start start 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0d04c0 0x7fad6c1f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c15c960 0x7fad6c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c1603c0 0x7fad6c15ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c0d3150 con 0x7fad6c1603c0 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1611b0 con 0x7fad6c15c960 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad72400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c0d32d0 con 0x7fad6c0d04c0 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c15c960 0x7fad6c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6b400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0d04c0 0x7fad6c1f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c1603c0 0x7fad6c15ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c15c960 0x7fad6c15a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:59200/0 (socket says 172.21.15.110:59200) 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6aa00640 1 -- 172.21.15.110:0/1783796896 learned_addr learned my addr 172.21.15.110:0/1783796896 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6be00640 1 -- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0d04c0 msgr2=0x7fad6c1f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6be00640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0d04c0 0x7fad6c1f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6be00640 1 -- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c15c960 msgr2=0x7fad6c15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6be00640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c15c960 0x7fad6c15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.890+0000 7fad6be00640 1 -- 172.21.15.110:0/1783796896 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c06daa0 con 0x7fad6c1603c0 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad6be00640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c1603c0 0x7fad6c15ae10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fad6400be30 tx=0x7fad6400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad63400640 1 -- 172.21.15.110:0/1783796896 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64019070 con 0x7fad6c1603c0 2024-09-10T10:04:20.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad63400640 1 -- 172.21.15.110:0/1783796896 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad64014440 con 0x7fad6c1603c0 2024-09-10T10:04:20.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c06dd80 con 0x7fad6c1603c0 2024-09-10T10:04:20.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad63400640 1 -- 172.21.15.110:0/1783796896 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64014830 con 0x7fad6c1603c0 2024-09-10T10:04:20.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c06e060 con 0x7fad6c1603c0 2024-09-10T10:04:20.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c071a80 con 0x7fad6c1603c0 2024-09-10T10:04:20.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad63400640 1 -- 172.21.15.110:0/1783796896 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fad6401c070 con 0x7fad6c1603c0 2024-09-10T10:04:20.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad63400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c067e90 0x7fad4c06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad63400640 1 -- 172.21.15.110:0/1783796896 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4999+0+0 (secure 0 0 0) 0x7fad640861e0 con 0x7fad6c1603c0 2024-09-10T10:04:20.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.894+0000 7fad6b400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c067e90 0x7fad4c06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.898+0000 7fad6b400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c067e90 0x7fad4c06a350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0ad000 tx=0x7fad5c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:20.898+0000 7fad63400640 1 -- 172.21.15.110:0/1783796896 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fad64056170 con 0x7fad6c1603c0 2024-09-10T10:04:21.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.122+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7fad6c0b19d0 con 0x7fad6c1603c0 2024-09-10T10:04:21.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad63400640 1 -- 172.21.15.110:0/1783796896 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v327) ==== 148+0+0 (secure 0 0 0) 0x7fad6405a470 con 0x7fad6c1603c0 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c067e90 msgr2=0x7fad4c06a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c067e90 0x7fad4c06a350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0ad000 tx=0x7fad5c0a6000 comp rx=0 tx=0).stop 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c1603c0 msgr2=0x7fad6c15ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c1603c0 0x7fad6c15ae10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fad6400be30 tx=0x7fad6400bf20 comp rx=0 tx=0).stop 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 shutdown_connections 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c067e90 0x7fad4c06a350 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c1603c0 0x7fad6c15ae10 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c15c960 0x7fad6c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 --2- 172.21.15.110:0/1783796896 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0d04c0 0x7fad6c1f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 >> 172.21.15.110:0/1783796896 conn(0x7fad6c0d25c0 msgr2=0x7fad6c073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:21.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 shutdown_connections 2024-09-10T10:04:21.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.386+0000 7fad72400640 1 -- 172.21.15.110:0/1783796896 wait complete. 2024-09-10T10:04:21.412 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-10T10:04:21.412 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:04:21.412 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:21.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.542+0000 7f15c3e00640 1 Processor -- start 2024-09-10T10:04:21.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.542+0000 7f15c3e00640 1 -- start start 2024-09-10T10:04:21.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc15c560 0x7f15bc06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c3e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15bc06f430 con 0x7f15bc06f010 2024-09-10T10:04:21.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c3e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15bc155d50 con 0x7f15bc155970 2024-09-10T10:04:21.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c3e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc0b4570 con 0x7f15bc15c560 2024-09-10T10:04:21.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c1e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f15bc06f010 0x7f15bc0ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37738/0 (socket says 172.21.15.110:37738) 2024-09-10T10:04:21.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c1e00640 1 -- 172.21.15.110:0/1598380031 learned_addr learned my addr 172.21.15.110:0/1598380031 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:21.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c0a00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc15c560 0x7f15bc06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781372227 0 0) 0x7f15bc06f430 con 0x7f15bc06f010 2024-09-10T10:04:21.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15a4003530 con 0x7f15bc06f010 2024-09-10T10:04:21.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390888106 0 0) 0x7f15bc155d50 con 0x7f15bc155970 2024-09-10T10:04:21.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15bc06f430 con 0x7f15bc155970 2024-09-10T10:04:21.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1616904002 0 0) 0x7f15a4003530 con 0x7f15bc06f010 2024-09-10T10:04:21.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc15c560 msgr2=0x7f15bc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:21.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc15c560 0x7f15bc06ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 >> v1:172.21.15.145:6789/0 conn(0x7f15bc155970 legacy=0x7f15bc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc0b34d0 con 0x7f15bc06f010 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f15b4001680 con 0x7f15bc06f010 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 >> v1:172.21.15.110:6789/0 conn(0x7f15bc06f010 legacy=0x7f15bc0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc15c560 0x7f15a4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15a4004d50 0x7f15a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15a400bb60 con 0x7f15a40040e0 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15a400bce0 con 0x7f15a4004d50 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f15a400beb0 con 0x7f15bc15c560 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c0a00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c1400640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc15c560 0x7f15a4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c1e00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15a4004d50 0x7f15a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c0a00640 1 -- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc15c560 msgr2=0x7f15a4003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:21.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c0a00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc15c560 0x7f15a4003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c0a00640 1 -- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15a4004d50 msgr2=0x7f15a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:21.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c0a00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15a4004d50 0x7f15a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.546+0000 7f15c0a00640 1 -- 172.21.15.110:0/1598380031 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a4003530 con 0x7f15a40040e0 2024-09-10T10:04:21.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15c0a00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15a4004810 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009ae0 tx=0x7f15a801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e41ebd5fabf1bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:21.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a80b2070 con 0x7f15a40040e0 2024-09-10T10:04:21.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a80a5520 con 0x7f15a40040e0 2024-09-10T10:04:21.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15bbe00640 1 -- 172.21.15.110:0/1598380031 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a80096b0 con 0x7f15a40040e0 2024-09-10T10:04:21.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15c3e00640 1 -- 172.21.15.110:0/1598380031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 msgr2=0x7f15a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15c3e00640 1 --2- 172.21.15.110:0/1598380031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15a4004810 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009ae0 tx=0x7f15a801ca40 comp rx=0 tx=0).stop 2024-09-10T10:04:21.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15c1e00640 1 -- 172.21.15.110:0/1598380031 reap_dead start 2024-09-10T10:04:21.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15c3e00640 1 -- 172.21.15.110:0/1598380031 shutdown_connections 2024-09-10T10:04:21.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15c3e00640 1 -- 172.21.15.110:0/1598380031 >> 172.21.15.110:0/1598380031 conn(0x7f15bc06bc50 msgr2=0x7f15bc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:21.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15c3e00640 1 -- 172.21.15.110:0/1598380031 shutdown_connections 2024-09-10T10:04:21.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.550+0000 7f15c3e00640 1 -- 172.21.15.110:0/1598380031 wait complete. 2024-09-10T10:04:21.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 Processor -- start 2024-09-10T10:04:21.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 -- start start 2024-09-10T10:04:21.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15bc155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15bc1568e0 0x7f15bc15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc14cb00 0x7f15bc0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc0cf340 con 0x7f15a40040e0 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc0b4050 con 0x7f15bc1568e0 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc0cf4c0 con 0x7f15bc14cb00 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15bc155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15bc155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49660/0 (socket says 172.21.15.110:49660) 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 -- 172.21.15.110:0/2200365318 learned_addr learned my addr 172.21.15.110:0/2200365318 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c0a00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15bc1568e0 0x7f15bc15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1e00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc14cb00 0x7f15bc0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 -- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc14cb00 msgr2=0x7f15bc0710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc14cb00 0x7f15bc0710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 -- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15bc1568e0 msgr2=0x7f15bc15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15bc1568e0 0x7f15bc15dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 -- 172.21.15.110:0/2200365318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc15e100 con 0x7f15a40040e0 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c0a00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15bc1568e0 0x7f15bc15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:21.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c1400640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15bc155f70 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f15b000fb10 tx=0x7f15b000d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:21.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15b9600640 1 -- 172.21.15.110:0/2200365318 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0003b90 con 0x7f15a40040e0 2024-09-10T10:04:21.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15b9600640 1 -- 172.21.15.110:0/2200365318 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b0018070 con 0x7f15a40040e0 2024-09-10T10:04:21.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc157060 con 0x7f15a40040e0 2024-09-10T10:04:21.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15b9600640 1 -- 172.21.15.110:0/2200365318 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b000d950 con 0x7f15a40040e0 2024-09-10T10:04:21.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1505e0 con 0x7f15a40040e0 2024-09-10T10:04:21.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.554+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc0ce420 con 0x7f15a40040e0 2024-09-10T10:04:21.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.558+0000 7f15b9600640 1 -- 172.21.15.110:0/2200365318 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f15b000daf0 con 0x7f15a40040e0 2024-09-10T10:04:21.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.558+0000 7f15b9600640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f159c067e60 0x7f159c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.558+0000 7f15b9600640 1 -- 172.21.15.110:0/2200365318 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4999+0+0 (secure 0 0 0) 0x7f15b0086650 con 0x7f15a40040e0 2024-09-10T10:04:21.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.558+0000 7f15c0a00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f159c067e60 0x7f159c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.558+0000 7f15c0a00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f159c067e60 0x7f159c06a320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f15a80af000 tx=0x7f15a80046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:21.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.562+0000 7f15b9600640 1 -- 172.21.15.110:0/2200365318 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f15b0056c70 con 0x7f15a40040e0 2024-09-10T10:04:21.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.722+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15bc1604c0 con 0x7f15a40040e0 2024-09-10T10:04:21.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.722+0000 7f15b9600640 1 -- 172.21.15.110:0/2200365318 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+13169 (secure 0 0 0) 0x7f15b0056570 con 0x7f15a40040e0 2024-09-10T10:04:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:21.728 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":122,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:20.382949+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6834/1757537079":"2024-09-11T10:04:17.371300+0000","172.21.15.110:6832/1757537079":"2024-09-11T10:04:17.371300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f159c067e60 msgr2=0x7f159c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f159c067e60 0x7f159c06a320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f15a80af000 tx=0x7f15a80046d0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 msgr2=0x7f15bc155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15bc155f70 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f15b000fb10 tx=0x7f15b000d150 comp rx=0 tx=0).stop 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 shutdown_connections 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f159c067e60 0x7f159c06a320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15bc14cb00 0x7f15bc0710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15bc1568e0 0x7f15bc15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 --2- 172.21.15.110:0/2200365318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a40040e0 0x7f15bc155f70 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 >> 172.21.15.110:0/2200365318 conn(0x7f15bc06bc50 msgr2=0x7f15bc0cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:21.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 shutdown_connections 2024-09-10T10:04:21.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.726+0000 7f15c3e00640 1 -- 172.21.15.110:0/2200365318 wait complete. 2024-09-10T10:04:21.749 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:04:21.749 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:04:21.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd155200640 1 Processor -- start 2024-09-10T10:04:21.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd155200640 1 -- start start 2024-09-10T10:04:21.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd155200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd1500cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd155200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd1500b5f90 0x7fd1500b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd155200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd15006cfb0 con 0x7fd150070970 2024-09-10T10:04:21.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd155200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd15006cd60 con 0x7fd1500b5f90 2024-09-10T10:04:21.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd155200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd15006d290 con 0x7fd15015e520 2024-09-10T10:04:21.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd1500b5f90 0x7fd1500b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd1500cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd1500b5f90 0x7fd1500b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:59224/0 (socket says 172.21.15.110:59224) 2024-09-10T10:04:21.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14ea00640 1 -- 172.21.15.110:0/1625839465 learned_addr learned my addr 172.21.15.110:0/1625839465 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:21.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14ea00640 1 -- 172.21.15.110:0/1625839465 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 msgr2=0x7fd1500cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:21.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14ea00640 1 --2- 172.21.15.110:0/1625839465 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd1500cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14ea00640 1 -- 172.21.15.110:0/1625839465 >> v1:172.21.15.145:6789/0 conn(0x7fd150070970 legacy=0x7fd15006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14ea00640 1 -- 172.21.15.110:0/1625839465 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15006d570 con 0x7fd1500b5f90 2024-09-10T10:04:21.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.886+0000 7fd14f400640 1 --2- 172.21.15.110:0/1625839465 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd1500cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:21.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd14ea00640 1 --2- 172.21.15.110:0/1625839465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1500b5f90 0x7fd1500b63f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009870 tx=0x7fd13c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a33fcaa2d52a551 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:21.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd14e000640 1 -- 172.21.15.110:0/1625839465 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0b0070 con 0x7fd1500b5f90 2024-09-10T10:04:21.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd14e000640 1 -- 172.21.15.110:0/1625839465 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c0ab600 con 0x7fd1500b5f90 2024-09-10T10:04:21.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd14e000640 1 -- 172.21.15.110:0/1625839465 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0ab9d0 con 0x7fd1500b5f90 2024-09-10T10:04:21.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd155200640 1 -- 172.21.15.110:0/1625839465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1500b5f90 msgr2=0x7fd1500b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd155200640 1 --2- 172.21.15.110:0/1625839465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1500b5f90 0x7fd1500b63f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009870 tx=0x7fd13c01c4d0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd155200640 1 -- 172.21.15.110:0/1625839465 shutdown_connections 2024-09-10T10:04:21.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd155200640 1 --2- 172.21.15.110:0/1625839465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1500b5f90 0x7fd1500b63f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd155200640 1 --2- 172.21.15.110:0/1625839465 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd1500cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd155200640 1 -- 172.21.15.110:0/1625839465 >> 172.21.15.110:0/1625839465 conn(0x7fd1500b41f0 msgr2=0x7fd1500b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:21.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd155200640 1 -- 172.21.15.110:0/1625839465 shutdown_connections 2024-09-10T10:04:21.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.890+0000 7fd155200640 1 -- 172.21.15.110:0/1625839465 wait complete. 2024-09-10T10:04:21.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 Processor -- start 2024-09-10T10:04:21.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 -- start start 2024-09-10T10:04:21.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd150070970 0x7fd1500ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1500b5f90 0x7fd150155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd150156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd15016b4f0 con 0x7fd1500b5f90 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd15015eb10 con 0x7fd150070970 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd15016b670 con 0x7fd15015e520 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1500b5f90 0x7fd150155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1500b5f90 0x7fd150155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49664/0 (socket says 172.21.15.110:49664) 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 -- 172.21.15.110:0/547809158 learned_addr learned my addr 172.21.15.110:0/547809158 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14f400640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd150070970 0x7fd1500ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14fe00640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd150156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 -- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 msgr2=0x7fd150156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd150156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 -- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd150070970 msgr2=0x7fd1500ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd150070970 0x7fd1500ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 -- 172.21.15.110:0/547809158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd150154a70 con 0x7fd1500b5f90 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14fe00640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd150156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd14ea00640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1500b5f90 0x7fd150155b10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0ae040 tx=0x7fd13c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd147400640 1 -- 172.21.15.110:0/547809158 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0b0040 con 0x7fd1500b5f90 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd147400640 1 -- 172.21.15.110:0/547809158 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c0b4420 con 0x7fd1500b5f90 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd147400640 1 -- 172.21.15.110:0/547809158 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0b4810 con 0x7fd1500b5f90 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd150154d50 con 0x7fd1500b5f90 2024-09-10T10:04:21.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.894+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd150155310 con 0x7fd1500b5f90 2024-09-10T10:04:21.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.898+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1500b63f0 con 0x7fd1500b5f90 2024-09-10T10:04:21.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.898+0000 7fd147400640 1 -- 172.21.15.110:0/547809158 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fd13c0b49b0 con 0x7fd1500b5f90 2024-09-10T10:04:21.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.898+0000 7fd147400640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd130067e90 0x7fd13006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:21.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.898+0000 7fd147400640 1 -- 172.21.15.110:0/547809158 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4999+0+0 (secure 0 0 0) 0x7fd13c11fff0 con 0x7fd1500b5f90 2024-09-10T10:04:21.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.898+0000 7fd14f400640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd130067e90 0x7fd13006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:21.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.898+0000 7fd14f400640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd130067e90 0x7fd13006a350 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd15015ecb0 tx=0x7fd14000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:21.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:21.902+0000 7fd147400640 1 -- 172.21.15.110:0/547809158 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd13c016150 con 0x7fd1500b5f90 2024-09-10T10:04:22.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.058+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1500b1e40 con 0x7fd1500b5f90 2024-09-10T10:04:22.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.058+0000 7fd147400640 1 -- 172.21.15.110:0/547809158 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7fd13c0f04c0 con 0x7fd1500b5f90 2024-09-10T10:04:22.065 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd130067e90 msgr2=0x7fd13006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd130067e90 0x7fd13006a350 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd15015ecb0 tx=0x7fd14000b040 comp rx=0 tx=0).stop 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1500b5f90 msgr2=0x7fd150155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1500b5f90 0x7fd150155b10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0ae040 tx=0x7fd13c019270 comp rx=0 tx=0).stop 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 shutdown_connections 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd130067e90 0x7fd13006a350 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd15015e520 0x7fd150156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1500b5f90 0x7fd150155b10 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 --2- 172.21.15.110:0/547809158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd150070970 0x7fd1500ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 >> 172.21.15.110:0/547809158 conn(0x7fd1500b41f0 msgr2=0x7fd1500d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 shutdown_connections 2024-09-10T10:04:22.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.062+0000 7fd155200640 1 -- 172.21.15.110:0/547809158 wait complete. 2024-09-10T10:04:22.085 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-10T10:04:22.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.214+0000 7f08ec000640 1 Processor -- start 2024-09-10T10:04:22.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.214+0000 7f08ec000640 1 -- start start 2024-09-10T10:04:22.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08ec000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f08e40ccb60 0x7f08e4072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08ec000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 0x7f08e415b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08ec000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e40ccf40 con 0x7f08e4072670 2024-09-10T10:04:22.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08ec000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f08e4072ad0 con 0x7f08e40ccb60 2024-09-10T10:04:22.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08ec000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f08e41412a0 con 0x7f08e4140c30 2024-09-10T10:04:22.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08e9600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f08e40ccb60 0x7f08e4072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08ea000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 0x7f08e415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08e9600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f08e40ccb60 0x7f08e4072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:59244/0 (socket says 172.21.15.110:59244) 2024-09-10T10:04:22.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08e9600640 1 -- 172.21.15.110:0/170631471 learned_addr learned my addr 172.21.15.110:0/170631471 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:22.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08e9600640 1 -- 172.21.15.110:0/170631471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 msgr2=0x7f08e415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:22.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08e9600640 1 --2- 172.21.15.110:0/170631471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 0x7f08e415b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08e9600640 1 -- 172.21.15.110:0/170631471 >> v1:172.21.15.145:6789/0 conn(0x7f08e4072670 legacy=0x7f08e4140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.218+0000 7f08e9600640 1 -- 172.21.15.110:0/170631471 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e415e760 con 0x7f08e40ccb60 2024-09-10T10:04:22.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08e9600640 1 --2- 172.21.15.110:0/170631471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e40ccb60 0x7f08e4072130 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f08d800d150 tx=0x7f08d801ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14683a67bc0470a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:22.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08e3e00640 1 -- 172.21.15.110:0/170631471 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d80ad070 con 0x7f08e40ccb60 2024-09-10T10:04:22.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08e3e00640 1 -- 172.21.15.110:0/170631471 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d80a7bf0 con 0x7f08e40ccb60 2024-09-10T10:04:22.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08e3e00640 1 -- 172.21.15.110:0/170631471 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d80b1400 con 0x7f08e40ccb60 2024-09-10T10:04:22.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08ec000640 1 -- 172.21.15.110:0/170631471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e40ccb60 msgr2=0x7f08e4072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08ec000640 1 --2- 172.21.15.110:0/170631471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e40ccb60 0x7f08e4072130 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f08d800d150 tx=0x7f08d801ec30 comp rx=0 tx=0).stop 2024-09-10T10:04:22.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08ec000640 1 -- 172.21.15.110:0/170631471 shutdown_connections 2024-09-10T10:04:22.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08ec000640 1 --2- 172.21.15.110:0/170631471 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 0x7f08e415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08ec000640 1 --2- 172.21.15.110:0/170631471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e40ccb60 0x7f08e4072130 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08ec000640 1 -- 172.21.15.110:0/170631471 >> 172.21.15.110:0/170631471 conn(0x7f08e406bc50 msgr2=0x7f08e406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:22.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08ec000640 1 -- 172.21.15.110:0/170631471 shutdown_connections 2024-09-10T10:04:22.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.222+0000 7f08ec000640 1 -- 172.21.15.110:0/170631471 wait complete. 2024-09-10T10:04:22.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 Processor -- start 2024-09-10T10:04:22.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 -- start start 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e4072670 0x7f08e413e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e40ccb60 0x7f08e40b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 0x7f08e415c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08e415f520 con 0x7f08e40ccb60 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f08e415b660 con 0x7f08e4072670 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f08e40b0b50 con 0x7f08e4140c30 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e8c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e40ccb60 0x7f08e40b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e9600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e4072670 0x7f08e413e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e9600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e4072670 0x7f08e413e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:59252/0 (socket says 172.21.15.110:59252) 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e9600640 1 -- 172.21.15.110:0/1715191506 learned_addr learned my addr 172.21.15.110:0/1715191506 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:22.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ea000640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 0x7f08e415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e8c00640 1 -- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 msgr2=0x7f08e415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e8c00640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 0x7f08e415c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e8c00640 1 -- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e4072670 msgr2=0x7f08e413e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e8c00640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e4072670 0x7f08e413e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e8c00640 1 -- 172.21.15.110:0/1715191506 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e40b0d80 con 0x7f08e40ccb60 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e8c00640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e40ccb60 0x7f08e40b29f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f08d000c3f0 tx=0x7f08d000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e1600640 1 -- 172.21.15.110:0/1715191506 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d00052d0 con 0x7f08e40ccb60 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e1600640 1 -- 172.21.15.110:0/1715191506 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d0005470 con 0x7f08e40ccb60 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e40b1060 con 0x7f08e40ccb60 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08e1600640 1 -- 172.21.15.110:0/1715191506 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0007260 con 0x7f08e40ccb60 2024-09-10T10:04:22.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.226+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e40b1620 con 0x7f08e40ccb60 2024-09-10T10:04:22.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.230+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e40b1e00 con 0x7f08e40ccb60 2024-09-10T10:04:22.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.230+0000 7f08e1600640 1 -- 172.21.15.110:0/1715191506 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f08d0007400 con 0x7f08e40ccb60 2024-09-10T10:04:22.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.230+0000 7f08e1600640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08c4067e60 0x7f08c406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:22.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.230+0000 7f08e1600640 1 -- 172.21.15.110:0/1715191506 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4999+0+0 (secure 0 0 0) 0x7f08d00857c0 con 0x7f08e40ccb60 2024-09-10T10:04:22.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.230+0000 7f08e9600640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08c4067e60 0x7f08c406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:22.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.230+0000 7f08e9600640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08c4067e60 0x7f08c406a320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08d8000f90 tx=0x7f08d800a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:22.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.234+0000 7f08e1600640 1 -- 172.21.15.110:0/1715191506 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f08e40b1e00 con 0x7f08e40ccb60 2024-09-10T10:04:22.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:22.450+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f08e4072130 con 0x7f08e40ccb60 2024-09-10T10:04:23.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.390+0000 7f08e1600640 1 -- 172.21.15.110:0/1715191506 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v329) ==== 154+0+0 (secure 0 0 0) 0x7f08d0055670 con 0x7f08e40ccb60 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08c4067e60 msgr2=0x7f08c406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08c4067e60 0x7f08c406a320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08d8000f90 tx=0x7f08d800a040 comp rx=0 tx=0).stop 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e40ccb60 msgr2=0x7f08e40b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e40ccb60 0x7f08e40b29f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f08d000c3f0 tx=0x7f08d000c8b0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 shutdown_connections 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08c4067e60 0x7f08c406a320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f08e4140c30 0x7f08e415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08e40ccb60 0x7f08e40b29f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 --2- 172.21.15.110:0/1715191506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08e4072670 0x7f08e413e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 >> 172.21.15.110:0/1715191506 conn(0x7f08e406bc50 msgr2=0x7f08e415ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 shutdown_connections 2024-09-10T10:04:23.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.394+0000 7f08ec000640 1 -- 172.21.15.110:0/1715191506 wait complete. 2024-09-10T10:04:23.421 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:04:23.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb555800640 1 Processor -- start 2024-09-10T10:04:23.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb555800640 1 -- start start 2024-09-10T10:04:23.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb555800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb550145f30 0x7fb5500b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:23.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb555800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500cfd90 0x7fb5500b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:23.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb555800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5500b37c0 con 0x7fb5500b29e0 2024-09-10T10:04:23.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb555800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb5500b2e50 con 0x7fb550145f30 2024-09-10T10:04:23.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb555800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb5500b3aa0 con 0x7fb5500cfd90 2024-09-10T10:04:23.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb550145f30 0x7fb5500b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:23.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb550145f30 0x7fb5500b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:59258/0 (socket says 172.21.15.110:59258) 2024-09-10T10:04:23.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54ea00640 1 -- 172.21.15.110:0/3309087182 learned_addr learned my addr 172.21.15.110:0/3309087182 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:23.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54f400640 1 --2- 172.21.15.110:0/3309087182 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500cfd90 0x7fb5500b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:23.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54e000640 1 --1- 172.21.15.110:0/3309087182 >> v1:172.21.15.145:6789/0 conn(0x7fb5500b29e0 0x7fb5500cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:47468/0 (socket says 172.21.15.110:47468) 2024-09-10T10:04:23.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54ea00640 1 -- 172.21.15.110:0/3309087182 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500cfd90 msgr2=0x7fb5500b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:23.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54ea00640 1 --2- 172.21.15.110:0/3309087182 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500cfd90 0x7fb5500b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54ea00640 1 -- 172.21.15.110:0/3309087182 >> v1:172.21.15.145:6789/0 conn(0x7fb5500b29e0 legacy=0x7fb5500cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:23.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.558+0000 7fb54ea00640 1 -- 172.21.15.110:0/3309087182 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5500b3d80 con 0x7fb550145f30 2024-09-10T10:04:23.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb54ea00640 1 --2- 172.21.15.110:0/3309087182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 0x7fb5500b24a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb540009e50 tx=0x7fb54001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a970a0a19dfef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:23.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb54d600640 1 -- 172.21.15.110:0/3309087182 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400a5700 con 0x7fb550145f30 2024-09-10T10:04:23.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb54d600640 1 -- 172.21.15.110:0/3309087182 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5400a58a0 con 0x7fb550145f30 2024-09-10T10:04:23.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb54d600640 1 -- 172.21.15.110:0/3309087182 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400a5c70 con 0x7fb550145f30 2024-09-10T10:04:23.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb555800640 1 -- 172.21.15.110:0/3309087182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 msgr2=0x7fb5500b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:23.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb555800640 1 --2- 172.21.15.110:0/3309087182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 0x7fb5500b24a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb540009e50 tx=0x7fb54001ca30 comp rx=0 tx=0).stop 2024-09-10T10:04:23.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb555800640 1 -- 172.21.15.110:0/3309087182 shutdown_connections 2024-09-10T10:04:23.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb555800640 1 --2- 172.21.15.110:0/3309087182 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500cfd90 0x7fb5500b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb555800640 1 --2- 172.21.15.110:0/3309087182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 0x7fb5500b24a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb555800640 1 -- 172.21.15.110:0/3309087182 >> 172.21.15.110:0/3309087182 conn(0x7fb550062e40 msgr2=0x7fb55006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:23.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb555800640 1 -- 172.21.15.110:0/3309087182 shutdown_connections 2024-09-10T10:04:23.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.562+0000 7fb555800640 1 -- 172.21.15.110:0/3309087182 wait complete. 2024-09-10T10:04:23.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 Processor -- start 2024-09-10T10:04:23.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 -- start start 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500b29e0 0x7fb550140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5500cfd90 0x7fb55013f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 0x7fb55013f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5500b5660 con 0x7fb5500cfd90 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb5500b54e0 con 0x7fb550145f30 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb5500b57e0 con 0x7fb5500b29e0 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5500cfd90 0x7fb55013f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5500cfd90 0x7fb55013f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49686/0 (socket says 172.21.15.110:49686) 2024-09-10T10:04:23.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 -- 172.21.15.110:0/1828407726 learned_addr learned my addr 172.21.15.110:0/1828407726 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:23.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54ea00640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500b29e0 0x7fb550140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:23.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54f400640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 0x7fb55013f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:23.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 -- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500b29e0 msgr2=0x7fb550140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:23.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500b29e0 0x7fb550140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 -- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 msgr2=0x7fb55013f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:23.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 0x7fb55013f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 -- 172.21.15.110:0/1828407726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5500ab960 con 0x7fb5500cfd90 2024-09-10T10:04:23.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb54e000640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5500cfd90 0x7fb55013f0d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00c3a0 tx=0x7fb53c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:23.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb546a00640 1 -- 172.21.15.110:0/1828407726 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0074e0 con 0x7fb5500cfd90 2024-09-10T10:04:23.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb546a00640 1 -- 172.21.15.110:0/1828407726 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53c0076a0 con 0x7fb5500cfd90 2024-09-10T10:04:23.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb546a00640 1 -- 172.21.15.110:0/1828407726 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c004a30 con 0x7fb5500cfd90 2024-09-10T10:04:23.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5500aa8c0 con 0x7fb5500cfd90 2024-09-10T10:04:23.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5500aae80 con 0x7fb5500cfd90 2024-09-10T10:04:23.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.566+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5500b4560 con 0x7fb5500cfd90 2024-09-10T10:04:23.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.570+0000 7fb546a00640 1 -- 172.21.15.110:0/1828407726 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fb53c01a020 con 0x7fb5500cfd90 2024-09-10T10:04:23.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.570+0000 7fb546a00640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb530067e60 0x7fb53006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:23.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.570+0000 7fb546a00640 1 -- 172.21.15.110:0/1828407726 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4999+0+0 (secure 0 0 0) 0x7fb53c085a90 con 0x7fb5500cfd90 2024-09-10T10:04:23.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.570+0000 7fb54ea00640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb530067e60 0x7fb53006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:23.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.570+0000 7fb54ea00640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb530067e60 0x7fb53006a320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb5400ae000 tx=0x7fb5400a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:23.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.574+0000 7fb546a00640 1 -- 172.21.15.110:0/1828407726 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb53c055990 con 0x7fb5500cfd90 2024-09-10T10:04:23.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.734+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb55013e8b0 con 0x7fb5500cfd90 2024-09-10T10:04:23.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb546a00640 1 -- 172.21.15.110:0/1828407726 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fb53c05ae20 con 0x7fb5500cfd90 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb530067e60 msgr2=0x7fb53006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb530067e60 0x7fb53006a320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb5400ae000 tx=0x7fb5400a7070 comp rx=0 tx=0).stop 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5500cfd90 msgr2=0x7fb55013f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5500cfd90 0x7fb55013f0d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00c3a0 tx=0x7fb53c00c860 comp rx=0 tx=0).stop 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 shutdown_connections 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb530067e60 0x7fb53006a320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb550145f30 0x7fb55013f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5500cfd90 0x7fb55013f0d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 --2- 172.21.15.110:0/1828407726 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb5500b29e0 0x7fb550140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 >> 172.21.15.110:0/1828407726 conn(0x7fb550062e40 msgr2=0x7fb550071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:23.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 shutdown_connections 2024-09-10T10:04:23.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:23.738+0000 7fb555800640 1 -- 172.21.15.110:0/1828407726 wait complete. 2024-09-10T10:04:23.762 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:04:23.763 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:04:23.835 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:04:23.912 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:04:23.912 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:23.912 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:23.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:04:23.921 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:23.921 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:23.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:04:23.930 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:23.930 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:23.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:04:23.972 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:23.972 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:23.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:04:23.986 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:04:23.986 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:24.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff308e00640 1 Processor -- start 2024-09-10T10:04:24.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff308e00640 1 -- start start 2024-09-10T10:04:24.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff308e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3040cd510 0x7ff3040cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff308e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff30406e090 0x7ff30406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff308e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff304072150 con 0x7ff3040cfe30 2024-09-10T10:04:24.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff308e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff30415d9a0 con 0x7ff30406e090 2024-09-10T10:04:24.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff308e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff30406d2d0 con 0x7ff3040cd510 2024-09-10T10:04:24.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff302800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff30406e090 0x7ff30406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff302800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff30406e090 0x7ff30406e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49692/0 (socket says 172.21.15.110:49692) 2024-09-10T10:04:24.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.146+0000 7ff302800640 1 -- 172.21.15.110:0/514695601 learned_addr learned my addr 172.21.15.110:0/514695601 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:24.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff301400640 1 --2- 172.21.15.110:0/514695601 >> v2:172.21.15.145:3300/0 conn(0x7ff3040cd510 0x7ff3040cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff302800640 1 -- 172.21.15.110:0/514695601 >> v2:172.21.15.145:3300/0 conn(0x7ff3040cd510 msgr2=0x7ff3040cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:24.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff302800640 1 --2- 172.21.15.110:0/514695601 >> v2:172.21.15.145:3300/0 conn(0x7ff3040cd510 0x7ff3040cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff302800640 1 -- 172.21.15.110:0/514695601 >> v1:172.21.15.145:6789/0 conn(0x7ff3040cfe30 legacy=0x7ff304071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:24.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff302800640 1 -- 172.21.15.110:0/514695601 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30406d5b0 con 0x7ff30406e090 2024-09-10T10:04:24.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff302800640 1 --2- 172.21.15.110:0/514695601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff30406e4e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff3040ce0f0 tx=0x7ff2fc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dba5456513cdba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:24.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff300a00640 1 -- 172.21.15.110:0/514695601 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0b2070 con 0x7ff30406e090 2024-09-10T10:04:24.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff300a00640 1 -- 172.21.15.110:0/514695601 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc0a9070 con 0x7ff30406e090 2024-09-10T10:04:24.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff300a00640 1 -- 172.21.15.110:0/514695601 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0ad880 con 0x7ff30406e090 2024-09-10T10:04:24.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff308e00640 1 -- 172.21.15.110:0/514695601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 msgr2=0x7ff30406e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.150+0000 7ff308e00640 1 --2- 172.21.15.110:0/514695601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff30406e4e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff3040ce0f0 tx=0x7ff2fc01c450 comp rx=0 tx=0).stop 2024-09-10T10:04:24.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- 172.21.15.110:0/514695601 shutdown_connections 2024-09-10T10:04:24.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 --2- 172.21.15.110:0/514695601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff30406e4e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 --2- 172.21.15.110:0/514695601 >> v2:172.21.15.145:3300/0 conn(0x7ff3040cd510 0x7ff3040cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- 172.21.15.110:0/514695601 >> 172.21.15.110:0/514695601 conn(0x7ff30406bc50 msgr2=0x7ff30406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:24.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- 172.21.15.110:0/514695601 shutdown_connections 2024-09-10T10:04:24.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- 172.21.15.110:0/514695601 wait complete. 2024-09-10T10:04:24.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 Processor -- start 2024-09-10T10:04:24.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- start start 2024-09-10T10:04:24.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff304155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3040cd510 0x7ff3040b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3040cfe30 0x7ff3041561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff304169180 con 0x7ff30406e090 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff30415d6c0 con 0x7ff3040cd510 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff304169300 con 0x7ff3040cfe30 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff304155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff304155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49704/0 (socket says 172.21.15.110:49704) 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 -- 172.21.15.110:0/3967469078 learned_addr learned my addr 172.21.15.110:0/3967469078 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301400640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3040cd510 0x7ff3040b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff302800640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3040cfe30 0x7ff3041561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 -- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3040cfe30 msgr2=0x7ff3041561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3040cfe30 0x7ff3041561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 -- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3040cd510 msgr2=0x7ff3040b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3040cd510 0x7ff3040b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 -- 172.21.15.110:0/3967469078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3040b2520 con 0x7ff30406e090 2024-09-10T10:04:24.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301400640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3040cd510 0x7ff3040b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:24.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff301e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff304155290 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400ba50 tx=0x7ff2f400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:24.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff2fa000640 1 -- 172.21.15.110:0/3967469078 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4019070 con 0x7ff30406e090 2024-09-10T10:04:24.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff2fa000640 1 -- 172.21.15.110:0/3967469078 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f400cc20 con 0x7ff30406e090 2024-09-10T10:04:24.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff2fa000640 1 -- 172.21.15.110:0/3967469078 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4014680 con 0x7ff30406e090 2024-09-10T10:04:24.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.154+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3040b2800 con 0x7ff30406e090 2024-09-10T10:04:24.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.158+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3040b2dc0 con 0x7ff30406e090 2024-09-10T10:04:24.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.158+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30406e4e0 con 0x7ff30406e090 2024-09-10T10:04:24.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.158+0000 7ff2fa000640 1 -- 172.21.15.110:0/3967469078 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7ff2f401c070 con 0x7ff30406e090 2024-09-10T10:04:24.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.158+0000 7ff2fa000640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2e4067e60 0x7ff2e406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.158+0000 7ff2fa000640 1 -- 172.21.15.110:0/3967469078 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4999+0+0 (secure 0 0 0) 0x7ff2f4086110 con 0x7ff30406e090 2024-09-10T10:04:24.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.158+0000 7ff301400640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2e4067e60 0x7ff2e406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.162+0000 7ff301400640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2e4067e60 0x7ff2e406a320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000b9f0 tx=0x7ff2f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:24.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.162+0000 7ff2fa000640 1 -- 172.21.15.110:0/3967469078 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff2f4056010 con 0x7ff30406e090 2024-09-10T10:04:24.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff304075560 con 0x7ff30406e090 2024-09-10T10:04:24.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff2fa000640 1 -- 172.21.15.110:0/3967469078 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+3920 (secure 0 0 0) 0x7ff2f405a310 con 0x7ff30406e090 2024-09-10T10:04:24.358 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:24.359 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":81,"btime":"2024-09-10T10:04:18:373578+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6191,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1172125083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1172125083},{"type":"v1","addr":"172.21.15.145:6834","nonce":1172125083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6194,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3529040969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3529040969},{"type":"v1","addr":"172.21.15.145:6835","nonce":3529040969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2459795528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2459795528},{"type":"v1","addr":"172.21.15.110:6835","nonce":2459795528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/804838434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":804838434},{"type":"v1","addr":"172.21.15.110:6834","nonce":804838434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[]} 2024-09-10T10:04:24.359 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 81 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2e4067e60 msgr2=0x7ff2e406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2e4067e60 0x7ff2e406a320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000b9f0 tx=0x7ff2f0008040 comp rx=0 tx=0).stop 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 msgr2=0x7ff304155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff304155290 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400ba50 tx=0x7ff2f400bf10 comp rx=0 tx=0).stop 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 shutdown_connections 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff2e4067e60 0x7ff2e406a320 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3040cfe30 0x7ff3041561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3040cd510 0x7ff3040b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 --2- 172.21.15.110:0/3967469078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff30406e090 0x7ff304155290 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 >> 172.21.15.110:0/3967469078 conn(0x7ff30406bc50 msgr2=0x7ff3040ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 shutdown_connections 2024-09-10T10:04:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.354+0000 7ff308e00640 1 -- 172.21.15.110:0/3967469078 wait complete. 2024-09-10T10:04:24.380 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:04:24.380 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:04:24.381 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:04:24.381 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:04:24.381 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:04:24.381 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:04:24.381 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:04:24.381 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:04:24.381 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:04:24.381 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:04:24.382 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:04:24.382 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:04:24.382 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:04:24.382 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:04:24.382 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:04:24.382 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:04:24.394 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:04:24.394 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:04:24.395 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:04:24.395 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:04:24.398 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:04:24.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:04:24.399 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:04:24.399 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:04:24.439 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:04:24.439 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:24.441 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:04:24.441 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:24.447 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:04:24.447 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:24.449 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:04:24.449 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:24.450 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:04:24.525 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:04:24.526 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:04:24.536 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:04:24.537 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:04:24.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.590+0000 7f6e24200640 1 Processor -- start 2024-09-10T10:04:24.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.590+0000 7f6e24200640 1 -- start start 2024-09-10T10:04:24.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.590+0000 7f6e24200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6e1c0d20c0 0x7f6e1c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.590+0000 7f6e24200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1c0d04c0 con 0x7f6e1c15c580 2024-09-10T10:04:24.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.590+0000 7f6e24200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1c0d07a0 con 0x7f6e1c071150 2024-09-10T10:04:24.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.590+0000 7f6e24200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6e1c15ed90 con 0x7f6e1c0d20c0 2024-09-10T10:04:24.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.590+0000 7f6e21800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6e1c15c580 0x7f6e1c06fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37750/0 (socket says 172.21.15.110:37750) 2024-09-10T10:04:24.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.590+0000 7f6e21800640 1 -- 172.21.15.110:0/1353232678 learned_addr learned my addr 172.21.15.110:0/1353232678 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:24.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e20e00640 1 --2- 172.21.15.110:0/1353232678 >> v2:172.21.15.145:3300/0 conn(0x7f6e1c0d20c0 0x7f6e1c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e1be00640 1 -- 172.21.15.110:0/1353232678 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308042085 0 0) 0x7f6e1c0d04c0 con 0x7f6e1c15c580 2024-09-10T10:04:24.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e1be00640 1 -- 172.21.15.110:0/1353232678 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e04003530 con 0x7f6e1c15c580 2024-09-10T10:04:24.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e1be00640 1 -- 172.21.15.110:0/1353232678 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 844792524 0 0) 0x7f6e04003530 con 0x7f6e1c15c580 2024-09-10T10:04:24.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e20e00640 1 -- 172.21.15.110:0/1353232678 >> v1:172.21.15.145:6789/0 conn(0x7f6e1c071150 legacy=0x7f6e1c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e20e00640 1 -- 172.21.15.110:0/1353232678 >> v1:172.21.15.110:6789/0 conn(0x7f6e1c15c580 legacy=0x7f6e1c06fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e20e00640 1 -- 172.21.15.110:0/1353232678 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1c0d0a80 con 0x7f6e1c0d20c0 2024-09-10T10:04:24.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e20e00640 1 --2- 172.21.15.110:0/1353232678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 0x7f6e1c070c10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6e080097c0 tx=0x7f6e0801c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b08e3408cdbe67e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:24.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e1be00640 1 -- 172.21.15.110:0/1353232678 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080b0070 con 0x7f6e1c0d20c0 2024-09-10T10:04:24.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e1be00640 1 -- 172.21.15.110:0/1353232678 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e080ab400 con 0x7f6e1c0d20c0 2024-09-10T10:04:24.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.594+0000 7f6e1be00640 1 -- 172.21.15.110:0/1353232678 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10001680 con 0x7f6e1c0d20c0 2024-09-10T10:04:24.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- 172.21.15.110:0/1353232678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 msgr2=0x7f6e1c070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 --2- 172.21.15.110:0/1353232678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 0x7f6e1c070c10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6e080097c0 tx=0x7f6e0801c270 comp rx=0 tx=0).stop 2024-09-10T10:04:24.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- 172.21.15.110:0/1353232678 shutdown_connections 2024-09-10T10:04:24.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 --2- 172.21.15.110:0/1353232678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 0x7f6e1c070c10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- 172.21.15.110:0/1353232678 >> 172.21.15.110:0/1353232678 conn(0x7f6e1c0d2630 msgr2=0x7f6e1c0d2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:24.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- 172.21.15.110:0/1353232678 shutdown_connections 2024-09-10T10:04:24.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- 172.21.15.110:0/1353232678 wait complete. 2024-09-10T10:04:24.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 Processor -- start 2024-09-10T10:04:24.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- start start 2024-09-10T10:04:24.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e1c071150 0x7f6e1c0b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 0x7f6e1c14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e1c15c580 0x7f6e1c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c16a390 con 0x7f6e1c15c580 2024-09-10T10:04:24.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c16a210 con 0x7f6e1c0d20c0 2024-09-10T10:04:24.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e24200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1c16a510 con 0x7f6e1c071150 2024-09-10T10:04:24.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e1c15c580 0x7f6e1c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e1c15c580 0x7f6e1c0b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49726/0 (socket says 172.21.15.110:49726) 2024-09-10T10:04:24.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 -- 172.21.15.110:0/600741457 learned_addr learned my addr 172.21.15.110:0/600741457 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:24.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 -- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e1c071150 msgr2=0x7f6e1c0b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e21800640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e1c071150 0x7f6e1c0b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e20e00640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 0x7f6e1c14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e1c071150 0x7f6e1c0b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 -- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 msgr2=0x7f6e1c14f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 0x7f6e1c14f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 -- 172.21.15.110:0/600741457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1c1fb680 con 0x7f6e1c15c580 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e21800640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e1c071150 0x7f6e1c0b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.598+0000 7f6e22200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e1c15c580 0x7f6e1c0b2940 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400c8a0 tx=0x7f6e1400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e19600640 1 -- 172.21.15.110:0/600741457 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14017070 con 0x7f6e1c15c580 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e19600640 1 -- 172.21.15.110:0/600741457 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e14003db0 con 0x7f6e1c15c580 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1c1fb960 con 0x7f6e1c15c580 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e19600640 1 -- 172.21.15.110:0/600741457 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e140052b0 con 0x7f6e1c15c580 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1c0b5c20 con 0x7f6e1c15c580 2024-09-10T10:04:24.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1c070c10 con 0x7f6e1c15c580 2024-09-10T10:04:24.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e19600640 1 -- 172.21.15.110:0/600741457 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f6e1401a070 con 0x7f6e1c15c580 2024-09-10T10:04:24.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e19600640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dfc067e60 0x7f6dfc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:24.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e19600640 1 -- 172.21.15.110:0/600741457 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4999+0+0 (secure 0 0 0) 0x7f6e14085c80 con 0x7f6e1c15c580 2024-09-10T10:04:24.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.602+0000 7f6e21800640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dfc067e60 0x7f6dfc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:24.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.606+0000 7f6e21800640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dfc067e60 0x7f6dfc06a320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e10001ec0 tx=0x7f6e10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:24.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.606+0000 7f6e19600640 1 -- 172.21.15.110:0/600741457 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6e140561d0 con 0x7f6e1c15c580 2024-09-10T10:04:24.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:24.774+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6e1c1fbb30 con 0x7f6e1c15c580 2024-09-10T10:04:25.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.410+0000 7f6e19600640 1 -- 172.21.15.110:0/600741457 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v124) ==== 97+0+0 (secure 0 0 0) 0x7f6e14055ad0 con 0x7f6e1c15c580 2024-09-10T10:04:25.415 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:04:25.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dfc067e60 msgr2=0x7f6dfc06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dfc067e60 0x7f6dfc06a320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e10001ec0 tx=0x7f6e10009040 comp rx=0 tx=0).stop 2024-09-10T10:04:25.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e1c15c580 msgr2=0x7f6e1c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e1c15c580 0x7f6e1c0b2940 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400c8a0 tx=0x7f6e1400cd60 comp rx=0 tx=0).stop 2024-09-10T10:04:25.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 shutdown_connections 2024-09-10T10:04:25.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dfc067e60 0x7f6dfc06a320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e1c15c580 0x7f6e1c0b2940 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e1c0d20c0 0x7f6e1c14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 --2- 172.21.15.110:0/600741457 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6e1c071150 0x7f6e1c0b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 >> 172.21.15.110:0/600741457 conn(0x7f6e1c0d2630 msgr2=0x7f6e1c159240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 shutdown_connections 2024-09-10T10:04:25.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.414+0000 7f6e24200640 1 -- 172.21.15.110:0/600741457 wait complete. 2024-09-10T10:04:25.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:04:25.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.578+0000 7f62c6c00640 1 Processor -- start 2024-09-10T10:04:25.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.578+0000 7f62c6c00640 1 -- start start 2024-09-10T10:04:25.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62c6c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f62c015c560 0x7f62c006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62c6c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 0x7f62c00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62c6c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c006f430 con 0x7f62c0155970 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62c6c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f62c00b4570 con 0x7f62c015c560 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62c6c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f62c0155d50 con 0x7f62c006f010 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bfe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f62c0155970 0x7f62c00cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37758/0 (socket says 172.21.15.110:37758) 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bfe00640 1 -- 172.21.15.110:0/975975765 learned_addr learned my addr 172.21.15.110:0/975975765 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bf400640 1 --2- 172.21.15.110:0/975975765 >> v2:172.21.15.110:3300/0 conn(0x7f62c015c560 0x7f62c006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62c4c00640 1 --2- 172.21.15.110:0/975975765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 0x7f62c00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bea00640 1 -- 172.21.15.110:0/975975765 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660193100 0 0) 0x7f62c006f430 con 0x7f62c0155970 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bea00640 1 -- 172.21.15.110:0/975975765 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62a4003530 con 0x7f62c0155970 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bf400640 1 -- 172.21.15.110:0/975975765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 msgr2=0x7f62c00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bf400640 1 --2- 172.21.15.110:0/975975765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 0x7f62c00ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bf400640 1 -- 172.21.15.110:0/975975765 >> v1:172.21.15.110:6789/0 conn(0x7f62c0155970 legacy=0x7f62c00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.582+0000 7f62bf400640 1 -- 172.21.15.110:0/975975765 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c00b34d0 con 0x7f62c015c560 2024-09-10T10:04:25.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62bf400640 1 --2- 172.21.15.110:0/975975765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c006ead0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0097c0 tx=0x7f62ac01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df182af05af8f059 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62bea00640 1 -- 172.21.15.110:0/975975765 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0a7070 con 0x7f62c015c560 2024-09-10T10:04:25.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62bea00640 1 -- 172.21.15.110:0/975975765 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ac0b0070 con 0x7f62c015c560 2024-09-10T10:04:25.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62bea00640 1 -- 172.21.15.110:0/975975765 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0ab890 con 0x7f62c015c560 2024-09-10T10:04:25.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 -- 172.21.15.110:0/975975765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 msgr2=0x7f62c006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 --2- 172.21.15.110:0/975975765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c006ead0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0097c0 tx=0x7f62ac01c470 comp rx=0 tx=0).stop 2024-09-10T10:04:25.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 -- 172.21.15.110:0/975975765 shutdown_connections 2024-09-10T10:04:25.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 --2- 172.21.15.110:0/975975765 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 0x7f62c00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 --2- 172.21.15.110:0/975975765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c006ead0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 -- 172.21.15.110:0/975975765 >> 172.21.15.110:0/975975765 conn(0x7f62c006bc50 msgr2=0x7f62c006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 -- 172.21.15.110:0/975975765 shutdown_connections 2024-09-10T10:04:25.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 -- 172.21.15.110:0/975975765 wait complete. 2024-09-10T10:04:25.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 Processor -- start 2024-09-10T10:04:25.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.586+0000 7f62c6c00640 1 -- start start 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 0x7f62c0162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0155970 0x7f62c01458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c0155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62c00cf170 con 0x7f62c015c560 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62c00b4050 con 0x7f62c0155970 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f62c00cf2f0 con 0x7f62c006f010 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c0155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c0155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49746/0 (socket says 172.21.15.110:49746) 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 0x7f62c0162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 -- 172.21.15.110:0/167786717 learned_addr learned my addr 172.21.15.110:0/167786717 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62bf400640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0155970 0x7f62c01458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 -- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 msgr2=0x7f62c0162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 0x7f62c0162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 -- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0155970 msgr2=0x7f62c01458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0155970 0x7f62c01458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 -- 172.21.15.110:0/167786717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c0156410 con 0x7f62c015c560 2024-09-10T10:04:25.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c4c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c0155ed0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f62b800e960 tx=0x7f62b800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62b7e00640 1 -- 172.21.15.110:0/167786717 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8017070 con 0x7f62c015c560 2024-09-10T10:04:25.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62b7e00640 1 -- 172.21.15.110:0/167786717 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b800c9a0 con 0x7f62c015c560 2024-09-10T10:04:25.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c0142cc0 con 0x7f62c015c560 2024-09-10T10:04:25.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62b7e00640 1 -- 172.21.15.110:0/167786717 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b800cd70 con 0x7f62c015c560 2024-09-10T10:04:25.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c0143280 con 0x7f62c015c560 2024-09-10T10:04:25.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.590+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c00ce420 con 0x7f62c015c560 2024-09-10T10:04:25.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.594+0000 7f62b7e00640 1 -- 172.21.15.110:0/167786717 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f62b801a070 con 0x7f62c015c560 2024-09-10T10:04:25.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.594+0000 7f62b7e00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62a0067e60 0x7f62a006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.594+0000 7f62b7e00640 1 -- 172.21.15.110:0/167786717 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4913+0+0 (secure 0 0 0) 0x7f62b8086390 con 0x7f62c015c560 2024-09-10T10:04:25.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.594+0000 7f62bfe00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62a0067e60 0x7f62a006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.594+0000 7f62bfe00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62a0067e60 0x7f62a006a320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f62b0001590 tx=0x7f62b0000f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.598+0000 7f62b7e00640 1 -- 172.21.15.110:0/167786717 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f62b8056320 con 0x7f62c015c560 2024-09-10T10:04:25.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f62c015b690 con 0x7f62c015c560 2024-09-10T10:04:25.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62b7e00640 1 -- 172.21.15.110:0/167786717 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f62b805a620 con 0x7f62c015c560 2024-09-10T10:04:25.814 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:25.814 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:04:25.814 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:04:25.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.814 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:04:25.814 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:25.815 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.817 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:25.818 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.820 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:04:25.822 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:04:25.823 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:04:25.825 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:04:25.826 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:04:25.828 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:04:25.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62a0067e60 msgr2=0x7f62a006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62a0067e60 0x7f62a006a320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f62b0001590 tx=0x7f62b0000f50 comp rx=0 tx=0).stop 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 msgr2=0x7f62c0155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c0155ed0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f62b800e960 tx=0x7f62b800ee20 comp rx=0 tx=0).stop 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 shutdown_connections 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62a0067e60 0x7f62a006a320 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62c015c560 0x7f62c0155ed0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62c0155970 0x7f62c01458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 --2- 172.21.15.110:0/167786717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62c006f010 0x7f62c0162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 >> 172.21.15.110:0/167786717 conn(0x7f62c006bc50 msgr2=0x7f62c006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 shutdown_connections 2024-09-10T10:04:25.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.810+0000 7f62c6c00640 1 -- 172.21.15.110:0/167786717 wait complete. 2024-09-10T10:04:25.836 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:04:25.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.966+0000 7f466b200640 1 Processor -- start 2024-09-10T10:04:25.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.966+0000 7f466b200640 1 -- start start 2024-09-10T10:04:25.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f466b200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f46640ccb60 0x7f4664072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f466b200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46640ccf40 con 0x7f4664072670 2024-09-10T10:04:25.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f466b200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46641412a0 con 0x7f4664140c30 2024-09-10T10:04:25.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f466b200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4664141080 con 0x7f46640ccb60 2024-09-10T10:04:25.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f4663400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f4664072670 0x7f4664140520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37762/0 (socket says 172.21.15.110:37762) 2024-09-10T10:04:25.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f4663400640 1 -- 172.21.15.110:0/64568140 learned_addr learned my addr 172.21.15.110:0/64568140 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:25.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f4663e00640 1 --2- 172.21.15.110:0/64568140 >> v2:172.21.15.145:3300/0 conn(0x7f46640ccb60 0x7f4664072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f4662a00640 1 -- 172.21.15.110:0/64568140 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 360175830 0 0) 0x7f46640ccf40 con 0x7f4664072670 2024-09-10T10:04:25.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f4662a00640 1 -- 172.21.15.110:0/64568140 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4648003530 con 0x7f4664072670 2024-09-10T10:04:25.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f4663e00640 1 -- 172.21.15.110:0/64568140 >> v1:172.21.15.145:6789/0 conn(0x7f4664140c30 legacy=0x7f466415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f4663e00640 1 -- 172.21.15.110:0/64568140 >> v1:172.21.15.110:6789/0 conn(0x7f4664072670 legacy=0x7f4664140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.970+0000 7f4663e00640 1 -- 172.21.15.110:0/64568140 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466415e760 con 0x7f46640ccb60 2024-09-10T10:04:25.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f4663e00640 1 --2- 172.21.15.110:0/64568140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46640ccb60 0x7f4664072130 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f46540097c0 tx=0x7f465401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e18dbdc6bb2ca5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f4662a00640 1 -- 172.21.15.110:0/64568140 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46540b1070 con 0x7f46640ccb60 2024-09-10T10:04:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f4662a00640 1 -- 172.21.15.110:0/64568140 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46540ac5f0 con 0x7f46640ccb60 2024-09-10T10:04:25.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f4662a00640 1 -- 172.21.15.110:0/64568140 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46540ac9c0 con 0x7f46640ccb60 2024-09-10T10:04:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- 172.21.15.110:0/64568140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46640ccb60 msgr2=0x7f4664072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 --2- 172.21.15.110:0/64568140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46640ccb60 0x7f4664072130 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f46540097c0 tx=0x7f465401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- 172.21.15.110:0/64568140 shutdown_connections 2024-09-10T10:04:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 --2- 172.21.15.110:0/64568140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46640ccb60 0x7f4664072130 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- 172.21.15.110:0/64568140 >> 172.21.15.110:0/64568140 conn(0x7f466406bc50 msgr2=0x7f466406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- 172.21.15.110:0/64568140 shutdown_connections 2024-09-10T10:04:25.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- 172.21.15.110:0/64568140 wait complete. 2024-09-10T10:04:25.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 Processor -- start 2024-09-10T10:04:25.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- start start 2024-09-10T10:04:25.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4664072670 0x7f466415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46640ccb60 0x7f466415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4664140c30 0x7f466415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f466415f520 con 0x7f4664140c30 2024-09-10T10:04:25.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f466415b830 con 0x7f4664072670 2024-09-10T10:04:25.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.974+0000 7f466b200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4664158430 con 0x7f46640ccb60 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4664140c30 0x7f466415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4664140c30 0x7f466415c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49754/0 (socket says 172.21.15.110:49754) 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 -- 172.21.15.110:0/3162446034 learned_addr learned my addr 172.21.15.110:0/3162446034 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4663e00640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4664072670 0x7f466415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4663400640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46640ccb60 0x7f466415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 -- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46640ccb60 msgr2=0x7f466415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46640ccb60 0x7f466415c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 -- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4664072670 msgr2=0x7f466415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4664072670 0x7f466415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 -- 172.21.15.110:0/3162446034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46640b0ae0 con 0x7f4664140c30 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4663e00640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4664072670 0x7f466415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f4669200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4664140c30 0x7f466415c850 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c730 tx=0x7f465c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f465be00640 1 -- 172.21.15.110:0/3162446034 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c003b80 con 0x7f4664140c30 2024-09-10T10:04:25.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f465be00640 1 -- 172.21.15.110:0/3162446034 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465c003d20 con 0x7f4664140c30 2024-09-10T10:04:25.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46640b0dc0 con 0x7f4664140c30 2024-09-10T10:04:25.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f465be00640 1 -- 172.21.15.110:0/3162446034 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c00aa30 con 0x7f4664140c30 2024-09-10T10:04:25.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4664151a30 con 0x7f4664140c30 2024-09-10T10:04:25.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4664072130 con 0x7f4664140c30 2024-09-10T10:04:25.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.978+0000 7f465be00640 1 -- 172.21.15.110:0/3162446034 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f465c00abd0 con 0x7f4664140c30 2024-09-10T10:04:25.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.982+0000 7f465be00640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4644067e90 0x7f464406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.982+0000 7f465be00640 1 -- 172.21.15.110:0/3162446034 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4913+0+0 (secure 0 0 0) 0x7f465c0861e0 con 0x7f4664140c30 2024-09-10T10:04:25.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.982+0000 7f4663e00640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4644067e90 0x7f464406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.982+0000 7f4663e00640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4644067e90 0x7f464406a350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46540ae000 tx=0x7f46540a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:25.986+0000 7f465be00640 1 -- 172.21.15.110:0/3162446034 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f465c056810 con 0x7f4664140c30 2024-09-10T10:04:26.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.142+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f466406c1d0 con 0x7f4664140c30 2024-09-10T10:04:26.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.142+0000 7f465be00640 1 -- 172.21.15.110:0/3162446034 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v126) ==== 84+0+65 (secure 0 0 0) 0x7f465c056110 con 0x7f4664140c30 2024-09-10T10:04:26.148 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:26.148 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:04:26.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:26.148 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:04:26.148 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:04:26.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:26.148 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4644067e90 msgr2=0x7f464406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4644067e90 0x7f464406a350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46540ae000 tx=0x7f46540a7000 comp rx=0 tx=0).stop 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4664140c30 msgr2=0x7f466415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4664140c30 0x7f466415c850 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c730 tx=0x7f465c00cbf0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 shutdown_connections 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4644067e90 0x7f464406a350 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4664140c30 0x7f466415c850 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46640ccb60 0x7f466415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 --2- 172.21.15.110:0/3162446034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4664072670 0x7f466415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 >> 172.21.15.110:0/3162446034 conn(0x7f466406bc50 msgr2=0x7f46640ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:26.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 shutdown_connections 2024-09-10T10:04:26.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.146+0000 7f466b200640 1 -- 172.21.15.110:0/3162446034 wait complete. 2024-09-10T10:04:26.169 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:04:26.169 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:04:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.302+0000 7f0f78c00640 1 Processor -- start 2024-09-10T10:04:26.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.302+0000 7f0f78c00640 1 -- start start 2024-09-10T10:04:26.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f78c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0f740cd510 0x7f0f740cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:26.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f78c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:26.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f78c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f74072150 con 0x7f0f740cfe30 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f78c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7415d9a0 con 0x7f0f740cd510 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f78c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7406ea60 con 0x7f0f7406e090 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f71e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0f740cfe30 0x7f0f74071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37764/0 (socket says 172.21.15.110:37764) 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f71e00640 1 -- 172.21.15.110:0/1386028475 learned_addr learned my addr 172.21.15.110:0/1386028475 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f72800640 1 --2- 172.21.15.110:0/1386028475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f71400640 1 --2- 172.21.15.110:0/1386028475 >> v2:172.21.15.145:3300/0 conn(0x7f0f740cd510 0x7f0f740cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f70a00640 1 -- 172.21.15.110:0/1386028475 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764566214 0 0) 0x7f0f74072150 con 0x7f0f740cfe30 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f70a00640 1 -- 172.21.15.110:0/1386028475 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f58003530 con 0x7f0f740cfe30 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f72800640 1 -- 172.21.15.110:0/1386028475 >> v2:172.21.15.145:3300/0 conn(0x7f0f740cd510 msgr2=0x7f0f740cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f72800640 1 --2- 172.21.15.110:0/1386028475 >> v2:172.21.15.145:3300/0 conn(0x7f0f740cd510 0x7f0f740cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f72800640 1 -- 172.21.15.110:0/1386028475 >> v1:172.21.15.110:6789/0 conn(0x7f0f740cfe30 legacy=0x7f0f74071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.306+0000 7f0f72800640 1 -- 172.21.15.110:0/1386028475 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7406d2d0 con 0x7f0f7406e090 2024-09-10T10:04:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f72800640 1 --2- 172.21.15.110:0/1386028475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7406e520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009d90 tx=0x7f0f6c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ea2daa8d102a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f70a00640 1 -- 172.21.15.110:0/1386028475 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0b2070 con 0x7f0f7406e090 2024-09-10T10:04:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f70a00640 1 -- 172.21.15.110:0/1386028475 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6c0a58b0 con 0x7f0f7406e090 2024-09-10T10:04:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f70a00640 1 -- 172.21.15.110:0/1386028475 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0a5c80 con 0x7f0f7406e090 2024-09-10T10:04:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 -- 172.21.15.110:0/1386028475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 msgr2=0x7f0f7406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 --2- 172.21.15.110:0/1386028475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7406e520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009d90 tx=0x7f0f6c01c8e0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 -- 172.21.15.110:0/1386028475 shutdown_connections 2024-09-10T10:04:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 --2- 172.21.15.110:0/1386028475 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7406e520 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 --2- 172.21.15.110:0/1386028475 >> v2:172.21.15.145:3300/0 conn(0x7f0f740cd510 0x7f0f740cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 -- 172.21.15.110:0/1386028475 >> 172.21.15.110:0/1386028475 conn(0x7f0f7406bc50 msgr2=0x7f0f7406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 -- 172.21.15.110:0/1386028475 shutdown_connections 2024-09-10T10:04:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 -- 172.21.15.110:0/1386028475 wait complete. 2024-09-10T10:04:26.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 Processor -- start 2024-09-10T10:04:26.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.310+0000 7f0f78c00640 1 -- start start 2024-09-10T10:04:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f740cd510 0x7f0f740b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f740cfe30 0x7f0f74142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f74169180 con 0x7f0f740cd510 2024-09-10T10:04:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7415d6c0 con 0x7f0f740cfe30 2024-09-10T10:04:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0f74169300 con 0x7f0f7406e090 2024-09-10T10:04:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f740cd510 0x7f0f740b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f740cd510 0x7f0f740b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49762/0 (socket says 172.21.15.110:49762) 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 -- 172.21.15.110:0/2436595729 learned_addr learned my addr 172.21.15.110:0/2436595729 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71e00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f72800640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f740cfe30 0x7f0f74142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 -- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 msgr2=0x7f0f7415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 -- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f740cfe30 msgr2=0x7f0f74142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f740cfe30 0x7f0f74142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 -- 172.21.15.110:0/2436595729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f741516a0 con 0x7f0f740cd510 2024-09-10T10:04:26.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f71400640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f740cd510 0x7f0f740b0040 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e8a0 tx=0x7f0f6000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:26.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f6a000640 1 -- 172.21.15.110:0/2436595729 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60017070 con 0x7f0f740cd510 2024-09-10T10:04:26.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f6a000640 1 -- 172.21.15.110:0/2436595729 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6000c940 con 0x7f0f740cd510 2024-09-10T10:04:26.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f6a000640 1 -- 172.21.15.110:0/2436595729 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000ccf0 con 0x7f0f740cd510 2024-09-10T10:04:26.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f74153090 con 0x7f0f740cd510 2024-09-10T10:04:26.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f740b0580 con 0x7f0f740cd510 2024-09-10T10:04:26.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7406e520 con 0x7f0f740cd510 2024-09-10T10:04:26.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.314+0000 7f0f6a000640 1 -- 172.21.15.110:0/2436595729 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f0f6001a070 con 0x7f0f740cd510 2024-09-10T10:04:26.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.318+0000 7f0f6a000640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0f54067e60 0x7f0f5406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:26.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.318+0000 7f0f6a000640 1 -- 172.21.15.110:0/2436595729 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4913+0+0 (secure 0 0 0) 0x7f0f600862b0 con 0x7f0f740cd510 2024-09-10T10:04:26.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.318+0000 7f0f71e00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0f54067e60 0x7f0f5406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:26.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.318+0000 7f0f71e00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0f54067e60 0x7f0f5406a320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f64007fd0 tx=0x7f0f64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:26.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.322+0000 7f0f6a000640 1 -- 172.21.15.110:0/2436595729 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0f60056180 con 0x7f0f740cd510 2024-09-10T10:04:26.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:26.482+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0f740b35d0 con 0x7f0f740cd510 2024-09-10T10:04:27.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.422+0000 7f0f6a000640 1 -- 172.21.15.110:0/2436595729 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v127) ==== 138+0+0 (secure 0 0 0) 0x7f0f600860d0 con 0x7f0f740cd510 2024-09-10T10:04:27.429 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:04:27.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0f54067e60 msgr2=0x7f0f5406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0f54067e60 0x7f0f5406a320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f64007fd0 tx=0x7f0f64009040 comp rx=0 tx=0).stop 2024-09-10T10:04:27.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f740cd510 msgr2=0x7f0f740b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f740cd510 0x7f0f740b0040 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e8a0 tx=0x7f0f6000eca0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 shutdown_connections 2024-09-10T10:04:27.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0f54067e60 0x7f0f5406a320 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f740cfe30 0x7f0f74142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f740cd510 0x7f0f740b0040 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 --2- 172.21.15.110:0/2436595729 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0f7406e090 0x7f0f7415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 >> 172.21.15.110:0/2436595729 conn(0x7f0f7406bc50 msgr2=0x7f0f7406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 shutdown_connections 2024-09-10T10:04:27.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.426+0000 7f0f78c00640 1 -- 172.21.15.110:0/2436595729 wait complete. 2024-09-10T10:04:27.457 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:04:27.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f47a6c00640 1 Processor -- start 2024-09-10T10:04:27.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f47a6c00640 1 -- start start 2024-09-10T10:04:27.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f47a6c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f47a00cd510 0x7f47a00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f47a6c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47a0072150 con 0x7f47a006e090 2024-09-10T10:04:27.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f47a6c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47a006d2d0 con 0x7f47a00cfe30 2024-09-10T10:04:27.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f47a6c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f47a015d9a0 con 0x7f47a00cd510 2024-09-10T10:04:27.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f479f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f47a00cd510 0x7f47a00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f47a4c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f47a006e090 0x7f47a006e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37770/0 (socket says 172.21.15.110:37770) 2024-09-10T10:04:27.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f47a4c00640 1 -- 172.21.15.110:0/4102135528 learned_addr learned my addr 172.21.15.110:0/4102135528 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:27.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f479f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f47a00cd510 0x7f47a00cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49774/0 (socket says 172.21.15.110:49774) 2024-09-10T10:04:27.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f479f400640 1 -- 172.21.15.110:0/4102135528 >> v1:172.21.15.145:6789/0 conn(0x7f47a00cfe30 legacy=0x7f47a0071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f479f400640 1 -- 172.21.15.110:0/4102135528 >> v1:172.21.15.110:6789/0 conn(0x7f47a006e090 legacy=0x7f47a006e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.598+0000 7f479f400640 1 -- 172.21.15.110:0/4102135528 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a006d5b0 con 0x7f47a00cd510 2024-09-10T10:04:27.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f479f400640 1 --2- 172.21.15.110:0/4102135528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a00cd510 0x7f47a00cd9a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f478c0097c0 tx=0x7f478c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e7a1bbfeba5693 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f479ea00640 1 -- 172.21.15.110:0/4102135528 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c0b0070 con 0x7f47a00cd510 2024-09-10T10:04:27.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f479ea00640 1 -- 172.21.15.110:0/4102135528 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f478c0ab600 con 0x7f47a00cd510 2024-09-10T10:04:27.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f479ea00640 1 -- 172.21.15.110:0/4102135528 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c0ab9d0 con 0x7f47a00cd510 2024-09-10T10:04:27.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 -- 172.21.15.110:0/4102135528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a00cd510 msgr2=0x7f47a00cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 --2- 172.21.15.110:0/4102135528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a00cd510 0x7f47a00cd9a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f478c0097c0 tx=0x7f478c01c410 comp rx=0 tx=0).stop 2024-09-10T10:04:27.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 -- 172.21.15.110:0/4102135528 shutdown_connections 2024-09-10T10:04:27.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 --2- 172.21.15.110:0/4102135528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a00cd510 0x7f47a00cd9a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 -- 172.21.15.110:0/4102135528 >> 172.21.15.110:0/4102135528 conn(0x7f47a006bc50 msgr2=0x7f47a006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 -- 172.21.15.110:0/4102135528 shutdown_connections 2024-09-10T10:04:27.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 -- 172.21.15.110:0/4102135528 wait complete. 2024-09-10T10:04:27.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 Processor -- start 2024-09-10T10:04:27.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.602+0000 7f47a6c00640 1 -- start start 2024-09-10T10:04:27.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a006e090 0x7f47a01566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47a00cd510 0x7f47a00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47a00cfe30 0x7f47a00aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f47a016a190 con 0x7f47a006e090 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f47a015d700 con 0x7f47a00cd510 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f47a016a310 con 0x7f47a00cfe30 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a006e090 0x7f47a01566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a006e090 0x7f47a01566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49790/0 (socket says 172.21.15.110:49790) 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 -- 172.21.15.110:0/595476114 learned_addr learned my addr 172.21.15.110:0/595476114 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479f400640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47a00cd510 0x7f47a00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a4c00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47a00cfe30 0x7f47a00aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 -- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47a00cfe30 msgr2=0x7f47a00aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47a00cfe30 0x7f47a00aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 -- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47a00cd510 msgr2=0x7f47a00ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47a00cd510 0x7f47a00ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 -- 172.21.15.110:0/595476114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a00b4bb0 con 0x7f47a006e090 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f479fe00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a006e090 0x7f47a01566b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f479000c3c0 tx=0x7f479000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f4797e00640 1 -- 172.21.15.110:0/595476114 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790017070 con 0x7f47a006e090 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f4797e00640 1 -- 172.21.15.110:0/595476114 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47900052d0 con 0x7f47a006e090 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f4797e00640 1 -- 172.21.15.110:0/595476114 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790003840 con 0x7f47a006e090 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a00b4e90 con 0x7f47a006e090 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a00b5450 con 0x7f47a006e090 2024-09-10T10:04:27.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.606+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a00cd9a0 con 0x7f47a006e090 2024-09-10T10:04:27.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.610+0000 7f4797e00640 1 -- 172.21.15.110:0/595476114 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f479001a070 con 0x7f47a006e090 2024-09-10T10:04:27.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.610+0000 7f4797e00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4780067e90 0x7f478006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.610+0000 7f4797e00640 1 -- 172.21.15.110:0/595476114 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5283+0+0 (secure 0 0 0) 0x7f4790085bc0 con 0x7f47a006e090 2024-09-10T10:04:27.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.610+0000 7f479f400640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4780067e90 0x7f478006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.610+0000 7f479f400640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4780067e90 0x7f478006a350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f478c0ad000 tx=0x7f478c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.614+0000 7f4797e00640 1 -- 172.21.15.110:0/595476114 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4790021020 con 0x7f47a006e090 2024-09-10T10:04:27.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:27.774+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f47a00cccb0 con 0x7f47a006e090 2024-09-10T10:04:28.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.430+0000 7f4797e00640 1 -- 172.21.15.110:0/595476114 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v128) ==== 170+0+0 (secure 0 0 0) 0x7f4790005480 con 0x7f47a006e090 2024-09-10T10:04:28.436 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4780067e90 msgr2=0x7f478006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4780067e90 0x7f478006a350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f478c0ad000 tx=0x7f478c008040 comp rx=0 tx=0).stop 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a006e090 msgr2=0x7f47a01566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a006e090 0x7f47a01566b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f479000c3c0 tx=0x7f479000c880 comp rx=0 tx=0).stop 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 shutdown_connections 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4780067e90 0x7f478006a350 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f47a00cfe30 0x7f47a00aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47a00cd510 0x7f47a00ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 --2- 172.21.15.110:0/595476114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47a006e090 0x7f47a01566b0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 >> 172.21.15.110:0/595476114 conn(0x7f47a006bc50 msgr2=0x7f47a0150250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 shutdown_connections 2024-09-10T10:04:28.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.434+0000 7f47a6c00640 1 -- 172.21.15.110:0/595476114 wait complete. 2024-09-10T10:04:28.471 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:04:28.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f19600640 1 Processor -- start 2024-09-10T10:04:28.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f19600640 1 -- start start 2024-09-10T10:04:28.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f19600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f140ce4d0 0x7f6f1406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:28.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f19600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f140b4960 con 0x7f6f1415b330 2024-09-10T10:04:28.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f19600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f140b4c40 con 0x7f6f1406e850 2024-09-10T10:04:28.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f19600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6f140b5570 con 0x7f6f140ce4d0 2024-09-10T10:04:28.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f12a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6f1415b330 0x7f6f1406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55830/0 (socket says 172.21.15.110:55830) 2024-09-10T10:04:28.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f12a00640 1 -- 172.21.15.110:0/2219429897 learned_addr learned my addr 172.21.15.110:0/2219429897 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:28.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f12000640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f140ce4d0 0x7f6f1406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:28.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122319257 0 0) 0x7f6f140b4960 con 0x7f6f1415b330 2024-09-10T10:04:28.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ef8005740 con 0x7f6f1415b330 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 762919109 0 0) 0x7f6ef8005740 con 0x7f6f1415b330 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f140ce4d0 msgr2=0x7f6f1406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f140ce4d0 0x7f6f1406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 >> v1:172.21.15.145:6789/0 conn(0x7f6f1406e850 legacy=0x7f6f140b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f140b4f20 con 0x7f6f1415b330 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f6f04001a00 con 0x7f6f1415b330 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 >> v1:172.21.15.110:6789/0 conn(0x7f6f1415b330 legacy=0x7f6f1406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6ef8006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef8006a20 0x7f6ef8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ef80073e0 0x7f6ef800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f140b5570 con 0x7f6ef8006a20 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef800e120 con 0x7f6ef8005c00 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef800e2a0 con 0x7f6ef80073e0 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f12a00640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6ef8006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f12000640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef8006a20 0x7f6ef8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.606+0000 7f6f13400640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ef80073e0 0x7f6ef800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f12a00640 1 -- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ef80073e0 msgr2=0x7f6ef800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f12a00640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ef80073e0 0x7f6ef800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f12a00640 1 -- 172.21.15.110:0/2219429897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef8006a20 msgr2=0x7f6ef8006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f12a00640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef8006a20 0x7f6ef8006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f12a00640 1 -- 172.21.15.110:0/2219429897 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef8005740 con 0x7f6ef8005c00 2024-09-10T10:04:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f12000640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef8006a20 0x7f6ef8006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f12a00640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6ef8006020 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400a840 tx=0x7f6f0401fac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b005181781d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400a460 con 0x7f6ef8005c00 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f040aa710 con 0x7f6ef8005c00 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f11600640 1 -- 172.21.15.110:0/2219429897 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040aab00 con 0x7f6ef8005c00 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f19600640 1 -- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 msgr2=0x7f6ef8006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f19600640 1 --2- 172.21.15.110:0/2219429897 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6ef8006020 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400a840 tx=0x7f6f0401fac0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.610+0000 7f6f13400640 1 -- 172.21.15.110:0/2219429897 reap_dead start 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 -- 172.21.15.110:0/2219429897 shutdown_connections 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 -- 172.21.15.110:0/2219429897 >> 172.21.15.110:0/2219429897 conn(0x7f6f14062e40 msgr2=0x7f6f1406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 -- 172.21.15.110:0/2219429897 shutdown_connections 2024-09-10T10:04:28.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 -- 172.21.15.110:0/2219429897 wait complete. 2024-09-10T10:04:28.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 Processor -- start 2024-09-10T10:04:28.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 -- start start 2024-09-10T10:04:28.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6f1414b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f1414ac10 0x7f6f1414a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f14071060 0x7f6f140714f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f140b1c60 con 0x7f6f14071060 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1415b710 con 0x7f6ef8005c00 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f19600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6f140b1de0 con 0x7f6f1414ac10 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f14071060 0x7f6f140714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f12a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6f1414b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f14071060 0x7f6f140714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51904/0 (socket says 172.21.15.110:51904) 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f12a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6f1414b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:35520/0 (socket says 172.21.15.110:35520) 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 -- 172.21.15.110:0/433428121 learned_addr learned my addr 172.21.15.110:0/433428121 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f12000640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f1414ac10 0x7f6f1414a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 -- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f1414ac10 msgr2=0x7f6f1414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f1414ac10 0x7f6f1414a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 -- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 msgr2=0x7f6f1414b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6f1414b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 -- 172.21.15.110:0/433428121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1414c4b0 con 0x7f6f14071060 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.614+0000 7f6f13400640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f14071060 0x7f6f140714f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6f14070ed0 tx=0x7f6f0c00f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f0aa00640 1 -- 172.21.15.110:0/433428121 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f000096b0 con 0x7f6f14071060 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f0aa00640 1 -- 172.21.15.110:0/433428121 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0c018070 con 0x7f6f14071060 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1414c790 con 0x7f6f14071060 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f0aa00640 1 -- 172.21.15.110:0/433428121 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c003600 con 0x7f6f14071060 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1414bfc0 con 0x7f6f14071060 2024-09-10T10:04:28.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f140ce4d0 con 0x7f6f14071060 2024-09-10T10:04:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f0aa00640 1 -- 172.21.15.110:0/433428121 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f6f0c0037a0 con 0x7f6f14071060 2024-09-10T10:04:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f0aa00640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ef4067e60 0x7f6ef406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:28.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f0aa00640 1 -- 172.21.15.110:0/433428121 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5665+0+0 (secure 0 0 0) 0x7f6f0c086a50 con 0x7f6f14071060 2024-09-10T10:04:28.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.618+0000 7f6f12a00640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ef4067e60 0x7f6ef406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:28.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.622+0000 7f6f12a00640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ef4067e60 0x7f6ef406a320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f040b1000 tx=0x7f6f04007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.622+0000 7f6f0aa00640 1 -- 172.21.15.110:0/433428121 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6f0c056d80 con 0x7f6f14071060 2024-09-10T10:04:28.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:28.814+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6f1415fcc0 con 0x7f6f14071060 2024-09-10T10:04:29.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.434+0000 7f6f0aa00640 1 -- 172.21.15.110:0/433428121 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v84) ==== 174+0+0 (secure 0 0 0) 0x7f6f0c056680 con 0x7f6f14071060 2024-09-10T10:04:29.438 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 20 and data pool 21 2024-09-10T10:04:29.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ef4067e60 msgr2=0x7f6ef406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ef4067e60 0x7f6ef406a320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f040b1000 tx=0x7f6f04007040 comp rx=0 tx=0).stop 2024-09-10T10:04:29.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f14071060 msgr2=0x7f6f140714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f14071060 0x7f6f140714f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6f14070ed0 tx=0x7f6f0c00f8a0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 shutdown_connections 2024-09-10T10:04:29.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6ef4067e60 0x7f6ef406a320 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f14071060 0x7f6f140714f0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f1414ac10 0x7f6f1414a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 --2- 172.21.15.110:0/433428121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ef8005c00 0x7f6f1414b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 >> 172.21.15.110:0/433428121 conn(0x7f6f14062e40 msgr2=0x7f6f140cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:29.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 shutdown_connections 2024-09-10T10:04:29.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.438+0000 7f6f19600640 1 -- 172.21.15.110:0/433428121 wait complete. 2024-09-10T10:04:29.485 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:29.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fb2800640 1 Processor -- start 2024-09-10T10:04:29.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fb2800640 1 -- start start 2024-09-10T10:04:29.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fb2800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac06f1e0 0x7f5fac166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fb2800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fac155d50 con 0x7f5fac15c560 2024-09-10T10:04:29.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fb2800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fac0b34d0 con 0x7f5fac155970 2024-09-10T10:04:29.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fb2800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac0b4570 con 0x7f5fac06f1e0 2024-09-10T10:04:29.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faaa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5fac15c560 0x7f5fac06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55832/0 (socket says 172.21.15.110:55832) 2024-09-10T10:04:29.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faaa00640 1 -- 172.21.15.110:0/3365467287 learned_addr learned my addr 172.21.15.110:0/3365467287 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:29.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fabe00640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac06f1e0 0x7f5fac166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468939368 0 0) 0x7f5fac155d50 con 0x7f5fac15c560 2024-09-10T10:04:29.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f90003530 con 0x7f5fac15c560 2024-09-10T10:04:29.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186750912 0 0) 0x7f5fac0b34d0 con 0x7f5fac155970 2024-09-10T10:04:29.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fac155d50 con 0x7f5fac155970 2024-09-10T10:04:29.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2872142125 0 0) 0x7f5f90003530 con 0x7f5fac15c560 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac06f1e0 msgr2=0x7f5fac166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac06f1e0 0x7f5fac166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 >> v1:172.21.15.145:6789/0 conn(0x7f5fac155970 legacy=0x7f5fac0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac0b37b0 con 0x7f5fac15c560 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f5f980016a0 con 0x7f5fac15c560 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 >> v1:172.21.15.110:6789/0 conn(0x7f5fac15c560 legacy=0x7f5fac06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5f90003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f90004810 0x7f5f90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac06f1e0 0x7f5f900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9000ba80 con 0x7f5f90003b80 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9000bc00 con 0x7f5f90004810 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9000be00 con 0x7f5fac06f1e0 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fab400640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5f90003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fabe00640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac06f1e0 0x7f5f900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faaa00640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f90004810 0x7f5f90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fab400640 1 -- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac06f1e0 msgr2=0x7f5f900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fab400640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac06f1e0 0x7f5f900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fab400640 1 -- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f90004810 msgr2=0x7f5f90004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fab400640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f90004810 0x7f5f90004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5fab400640 1 -- 172.21.15.110:0/3365467287 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f90003530 con 0x7f5f90003b80 2024-09-10T10:04:29.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.618+0000 7f5faaa00640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f90004810 0x7f5f90004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:29.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5fab400640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5f90003f80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c007fd0 tx=0x7f5f9c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d080ded0375b388 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0b0070 con 0x7f5f90003b80 2024-09-10T10:04:29.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9c0050d0 con 0x7f5f90003b80 2024-09-10T10:04:29.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5faa000640 1 -- 172.21.15.110:0/3365467287 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c001520 con 0x7f5f90003b80 2024-09-10T10:04:29.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5fb2800640 1 -- 172.21.15.110:0/3365467287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 msgr2=0x7f5f90003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5fb2800640 1 --2- 172.21.15.110:0/3365467287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5f90003f80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c007fd0 tx=0x7f5f9c01c510 comp rx=0 tx=0).stop 2024-09-10T10:04:29.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5fabe00640 1 -- 172.21.15.110:0/3365467287 reap_dead start 2024-09-10T10:04:29.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5fb2800640 1 -- 172.21.15.110:0/3365467287 shutdown_connections 2024-09-10T10:04:29.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5fb2800640 1 -- 172.21.15.110:0/3365467287 >> 172.21.15.110:0/3365467287 conn(0x7f5fac06bc50 msgr2=0x7f5fac06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:29.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5fb2800640 1 -- 172.21.15.110:0/3365467287 shutdown_connections 2024-09-10T10:04:29.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.622+0000 7f5fb2800640 1 -- 172.21.15.110:0/3365467287 wait complete. 2024-09-10T10:04:29.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 Processor -- start 2024-09-10T10:04:29.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 -- start start 2024-09-10T10:04:29.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5fac06e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fac15ad60 0x7f5fac150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac154b20 0x7f5fac154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1669b0 con 0x7f5f90003b80 2024-09-10T10:04:29.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac166830 con 0x7f5fac15ad60 2024-09-10T10:04:29.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac166b30 con 0x7f5fac154b20 2024-09-10T10:04:29.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5fac06e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5fac06e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51924/0 (socket says 172.21.15.110:51924) 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 -- 172.21.15.110:0/947560213 learned_addr learned my addr 172.21.15.110:0/947560213 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fabe00640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac154b20 0x7f5fac154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5faaa00640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fac15ad60 0x7f5fac150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 -- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac154b20 msgr2=0x7f5fac154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac154b20 0x7f5fac154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 -- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fac15ad60 msgr2=0x7f5fac150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fac15ad60 0x7f5fac150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 -- 172.21.15.110:0/947560213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac072850 con 0x7f5f90003b80 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fab400640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5fac06e3d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0ae040 tx=0x7f5f90005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fa3400640 1 -- 172.21.15.110:0/947560213 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0b0040 con 0x7f5f90003b80 2024-09-10T10:04:29.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fa3400640 1 -- 172.21.15.110:0/947560213 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9c0067e0 con 0x7f5f90003b80 2024-09-10T10:04:29.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fa3400640 1 -- 172.21.15.110:0/947560213 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c006b90 con 0x7f5f90003b80 2024-09-10T10:04:29.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac159720 con 0x7f5f90003b80 2024-09-10T10:04:29.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.626+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac159ce0 con 0x7f5f90003b80 2024-09-10T10:04:29.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.630+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac0cc8c0 con 0x7f5f90003b80 2024-09-10T10:04:29.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.630+0000 7f5fa3400640 1 -- 172.21.15.110:0/947560213 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f5f9c007030 con 0x7f5f90003b80 2024-09-10T10:04:29.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.630+0000 7f5fa3400640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5f8c067e60 0x7f5f8c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.630+0000 7f5fa3400640 1 -- 172.21.15.110:0/947560213 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5765+0+0 (secure 0 0 0) 0x7f5f9c120ab0 con 0x7f5f90003b80 2024-09-10T10:04:29.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.630+0000 7f5faaa00640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5f8c067e60 0x7f5f8c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.630+0000 7f5faaa00640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5f8c067e60 0x7f5f8c06a320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5fac155710 tx=0x7f5f98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.634+0000 7f5fa3400640 1 -- 172.21.15.110:0/947560213 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5f9c016250 con 0x7f5f90003b80 2024-09-10T10:04:29.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.794+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fac1554c0 con 0x7f5f90003b80 2024-09-10T10:04:29.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fa3400640 1 -- 172.21.15.110:0/947560213 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16877 (secure 0 0 0) 0x7f5f9c0f0de0 con 0x7f5f90003b80 2024-09-10T10:04:29.801 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:29.801 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":129,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:28.819817+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:04:26.532408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T10:04:27.823557+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:29.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5f8c067e60 msgr2=0x7f5f8c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5f8c067e60 0x7f5f8c06a320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5fac155710 tx=0x7f5f98009040 comp rx=0 tx=0).stop 2024-09-10T10:04:29.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 msgr2=0x7f5fac06e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5fac06e3d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0ae040 tx=0x7f5f90005840 comp rx=0 tx=0).stop 2024-09-10T10:04:29.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 shutdown_connections 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5f8c067e60 0x7f5f8c06a320 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5fac154b20 0x7f5fac154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fac15ad60 0x7f5fac150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 --2- 172.21.15.110:0/947560213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f90003b80 0x7f5fac06e3d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 >> 172.21.15.110:0/947560213 conn(0x7f5fac06bc50 msgr2=0x7f5fac071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 shutdown_connections 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.798+0000 7f5fb2800640 1 -- 172.21.15.110:0/947560213 wait complete. 2024-09-10T10:04:29.822 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:29.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9f5400640 1 Processor -- start 2024-09-10T10:04:29.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9f5400640 1 -- start start 2024-09-10T10:04:29.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9f5400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cfe30 0x7fa9f0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9f5400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa9f006e7a0 0x7fa9f00cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9f5400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9f006d2d0 con 0x7fa9f00cd510 2024-09-10T10:04:29.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9f5400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa9f00cdeb0 con 0x7fa9f006e7a0 2024-09-10T10:04:29.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9f5400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f015d9a0 con 0x7fa9f00cfe30 2024-09-10T10:04:29.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9ef200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa9f006e7a0 0x7fa9f00cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9ef200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa9f006e7a0 0x7fa9f00cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51934/0 (socket says 172.21.15.110:51934) 2024-09-10T10:04:29.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9ef200640 1 -- 172.21.15.110:0/3560790699 learned_addr learned my addr 172.21.15.110:0/3560790699 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:29.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9ee800640 1 --2- 172.21.15.110:0/3560790699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cfe30 0x7fa9f0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9ef200640 1 -- 172.21.15.110:0/3560790699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cfe30 msgr2=0x7fa9f0071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:29.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9ef200640 1 --2- 172.21.15.110:0/3560790699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cfe30 0x7fa9f0071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9ef200640 1 -- 172.21.15.110:0/3560790699 >> v1:172.21.15.145:6789/0 conn(0x7fa9f00cd510 legacy=0x7fa9f006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:29.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.954+0000 7fa9ef200640 1 -- 172.21.15.110:0/3560790699 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f006d5b0 con 0x7fa9f006e7a0 2024-09-10T10:04:29.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9ef200640 1 --2- 172.21.15.110:0/3560790699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f006e7a0 0x7fa9f00cd970 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009e50 tx=0x7fa9e801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1edf122d14ba69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9ede00640 1 -- 172.21.15.110:0/3560790699 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80a56f0 con 0x7fa9f006e7a0 2024-09-10T10:04:29.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9ede00640 1 -- 172.21.15.110:0/3560790699 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e80a5890 con 0x7fa9f006e7a0 2024-09-10T10:04:29.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9ede00640 1 -- 172.21.15.110:0/3560790699 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80a5c60 con 0x7fa9f006e7a0 2024-09-10T10:04:29.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9f5400640 1 -- 172.21.15.110:0/3560790699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f006e7a0 msgr2=0x7fa9f00cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9f5400640 1 --2- 172.21.15.110:0/3560790699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f006e7a0 0x7fa9f00cd970 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009e50 tx=0x7fa9e801c960 comp rx=0 tx=0).stop 2024-09-10T10:04:29.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9f5400640 1 -- 172.21.15.110:0/3560790699 shutdown_connections 2024-09-10T10:04:29.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9f5400640 1 --2- 172.21.15.110:0/3560790699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f006e7a0 0x7fa9f00cd970 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9f5400640 1 --2- 172.21.15.110:0/3560790699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cfe30 0x7fa9f0071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9f5400640 1 -- 172.21.15.110:0/3560790699 >> 172.21.15.110:0/3560790699 conn(0x7fa9f006bc50 msgr2=0x7fa9f006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9f5400640 1 -- 172.21.15.110:0/3560790699 shutdown_connections 2024-09-10T10:04:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.958+0000 7fa9f5400640 1 -- 172.21.15.110:0/3560790699 wait complete. 2024-09-10T10:04:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 Processor -- start 2024-09-10T10:04:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 -- start start 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9f006e7a0 0x7fa9f01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cd510 0x7fa9f013ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f00cfe30 0x7fa9f0154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0072190 con 0x7fa9f00cfe30 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f015d640 con 0x7fa9f006e7a0 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f0169180 con 0x7fa9f00cd510 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f00cfe30 0x7fa9f0154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f00cfe30 0x7fa9f0154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51938/0 (socket says 172.21.15.110:51938) 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 -- 172.21.15.110:0/2882202235 learned_addr learned my addr 172.21.15.110:0/2882202235 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ee800640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9f006e7a0 0x7fa9f01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9e7e00640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cd510 0x7fa9f013ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 -- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cd510 msgr2=0x7fa9f013ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cd510 0x7fa9f013ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 -- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9f006e7a0 msgr2=0x7fa9f01565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9f006e7a0 0x7fa9f01565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 -- 172.21.15.110:0/2882202235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f0155530 con 0x7fa9f00cfe30 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9ef200640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f00cfe30 0x7fa9f0154ce0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa9e801b010 tx=0x7fa9e8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9e6a00640 1 -- 172.21.15.110:0/2882202235 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80bc020 con 0x7fa9f00cfe30 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9e6a00640 1 -- 172.21.15.110:0/2882202235 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e80b8670 con 0x7fa9f00cfe30 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9e6a00640 1 -- 172.21.15.110:0/2882202235 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80b8a20 con 0x7fa9f00cfe30 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f0155810 con 0x7fa9f00cfe30 2024-09-10T10:04:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.962+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f0155dd0 con 0x7fa9f00cfe30 2024-09-10T10:04:29.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.966+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f00cd970 con 0x7fa9f00cfe30 2024-09-10T10:04:29.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.966+0000 7fa9e6a00640 1 -- 172.21.15.110:0/2882202235 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fa9e80bf020 con 0x7fa9f00cfe30 2024-09-10T10:04:29.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.966+0000 7fa9e6a00640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa9cc067e60 0x7fa9cc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.966+0000 7fa9e6a00640 1 -- 172.21.15.110:0/2882202235 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5765+0+0 (secure 0 0 0) 0x7fa9e8121470 con 0x7fa9f00cfe30 2024-09-10T10:04:29.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.966+0000 7fa9ee800640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa9cc067e60 0x7fa9cc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.966+0000 7fa9ee800640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa9cc067e60 0x7fa9cc06a320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0006aa0 tx=0x7fa9e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:29.970+0000 7fa9e6a00640 1 -- 172.21.15.110:0/2882202235 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa9e80c6030 con 0x7fa9f00cfe30 2024-09-10T10:04:30.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.130+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9f0071390 con 0x7fa9f00cfe30 2024-09-10T10:04:30.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.130+0000 7fa9e6a00640 1 -- 172.21.15.110:0/2882202235 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16877 (secure 0 0 0) 0x7fa9e80165e0 con 0x7fa9f00cfe30 2024-09-10T10:04:30.135 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:30.135 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":129,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:28.819817+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:04:26.532408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T10:04:27.823557+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:30.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa9cc067e60 msgr2=0x7fa9cc06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa9cc067e60 0x7fa9cc06a320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0006aa0 tx=0x7fa9e0008040 comp rx=0 tx=0).stop 2024-09-10T10:04:30.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f00cfe30 msgr2=0x7fa9f0154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f00cfe30 0x7fa9f0154ce0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa9e801b010 tx=0x7fa9e8019700 comp rx=0 tx=0).stop 2024-09-10T10:04:30.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 shutdown_connections 2024-09-10T10:04:30.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa9cc067e60 0x7fa9cc06a320 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f00cfe30 0x7fa9f0154ce0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa9f00cd510 0x7fa9f013ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 --2- 172.21.15.110:0/2882202235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9f006e7a0 0x7fa9f01565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 >> 172.21.15.110:0/2882202235 conn(0x7fa9f006bc50 msgr2=0x7fa9f006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 shutdown_connections 2024-09-10T10:04:30.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.134+0000 7fa9f5400640 1 -- 172.21.15.110:0/2882202235 wait complete. 2024-09-10T10:04:30.157 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:04:30.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87af000640 1 Processor -- start 2024-09-10T10:04:30.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87af000640 1 -- start start 2024-09-10T10:04:30.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87af000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f87a8145f30 0x7f87a80b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87af000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f87a80cfd90 0x7f87a80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87af000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87a80b37c0 con 0x7f87a80b29e0 2024-09-10T10:04:30.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87af000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f87a80b2e50 con 0x7f87a80cfd90 2024-09-10T10:04:30.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87af000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f87a80d0170 con 0x7f87a8145f30 2024-09-10T10:04:30.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87a7e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f87a8145f30 0x7f87a80b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87ad000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f87a80cfd90 0x7f87a80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87ad000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f87a80cfd90 0x7f87a80b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51950/0 (socket says 172.21.15.110:51950) 2024-09-10T10:04:30.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87a7e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f87a8145f30 0x7f87a80b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:35544/0 (socket says 172.21.15.110:35544) 2024-09-10T10:04:30.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.290+0000 7f87ad000640 1 -- 172.21.15.110:0/61398130 learned_addr learned my addr 172.21.15.110:0/61398130 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:30.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87ad000640 1 -- 172.21.15.110:0/61398130 >> v2:172.21.15.145:3300/0 conn(0x7f87a8145f30 msgr2=0x7f87a80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:30.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87ad000640 1 --2- 172.21.15.110:0/61398130 >> v2:172.21.15.145:3300/0 conn(0x7f87a8145f30 0x7f87a80b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87ad000640 1 -- 172.21.15.110:0/61398130 >> v1:172.21.15.145:6789/0 conn(0x7f87a80b29e0 legacy=0x7f87a80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87ad000640 1 -- 172.21.15.110:0/61398130 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a80b3aa0 con 0x7f87a80cfd90 2024-09-10T10:04:30.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87ad000640 1 --2- 172.21.15.110:0/61398130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80cfd90 0x7f87a80b4580 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f87a80b3000 tx=0x7f87a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62592e6a10bd647 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87a6a00640 1 -- 172.21.15.110:0/61398130 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a00a56c0 con 0x7f87a80cfd90 2024-09-10T10:04:30.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87a6a00640 1 -- 172.21.15.110:0/61398130 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a00a5860 con 0x7f87a80cfd90 2024-09-10T10:04:30.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87a6a00640 1 -- 172.21.15.110:0/61398130 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a00a5c30 con 0x7f87a80cfd90 2024-09-10T10:04:30.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87af000640 1 -- 172.21.15.110:0/61398130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80cfd90 msgr2=0x7f87a80b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.294+0000 7f87af000640 1 --2- 172.21.15.110:0/61398130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80cfd90 0x7f87a80b4580 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f87a80b3000 tx=0x7f87a001c9f0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- 172.21.15.110:0/61398130 shutdown_connections 2024-09-10T10:04:30.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 --2- 172.21.15.110:0/61398130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80cfd90 0x7f87a80b4580 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 --2- 172.21.15.110:0/61398130 >> v2:172.21.15.145:3300/0 conn(0x7f87a8145f30 0x7f87a80b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- 172.21.15.110:0/61398130 >> 172.21.15.110:0/61398130 conn(0x7f87a8062e40 msgr2=0x7f87a806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- 172.21.15.110:0/61398130 shutdown_connections 2024-09-10T10:04:30.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- 172.21.15.110:0/61398130 wait complete. 2024-09-10T10:04:30.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 Processor -- start 2024-09-10T10:04:30.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- start start 2024-09-10T10:04:30.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80b29e0 0x7f87a815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87a80cfd90 0x7f87a80aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87a8145f30 0x7f87a80aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f87a80b56a0 con 0x7f87a80b29e0 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f87a80b5520 con 0x7f87a8145f30 2024-09-10T10:04:30.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f87a80b5820 con 0x7f87a80cfd90 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80b29e0 0x7f87a815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80b29e0 0x7f87a815e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51954/0 (socket says 172.21.15.110:51954) 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 -- 172.21.15.110:0/4247622479 learned_addr learned my addr 172.21.15.110:0/4247622479 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87ad000640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87a8145f30 0x7f87a80aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7400640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87a80cfd90 0x7f87a80aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 -- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87a80cfd90 msgr2=0x7f87a80aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87a80cfd90 0x7f87a80aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 -- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87a8145f30 msgr2=0x7f87a80aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87a8145f30 0x7f87a80aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 -- 172.21.15.110:0/4247622479 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a815ee20 con 0x7f87a80b29e0 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87ad000640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87a8145f30 0x7f87a80aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:30.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87a7e00640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80b29e0 0x7f87a815e5f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f879800e8a0 tx=0x7f879800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f879fe00640 1 -- 172.21.15.110:0/4247622479 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798017070 con 0x7f87a80b29e0 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f879fe00640 1 -- 172.21.15.110:0/4247622479 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879800c940 con 0x7f87a80b29e0 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f879fe00640 1 -- 172.21.15.110:0/4247622479 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879800ccf0 con 0x7f87a80b29e0 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.298+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a815f100 con 0x7f87a80b29e0 2024-09-10T10:04:30.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.302+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a815f6c0 con 0x7f87a80b29e0 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.302+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a80b4580 con 0x7f87a80b29e0 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.302+0000 7f879fe00640 1 -- 172.21.15.110:0/4247622479 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f879801a070 con 0x7f87a80b29e0 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.302+0000 7f879fe00640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8788067e60 0x7f878806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.302+0000 7f879fe00640 1 -- 172.21.15.110:0/4247622479 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5765+0+0 (secure 0 0 0) 0x7f8798086980 con 0x7f87a80b29e0 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.302+0000 7f87a7400640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8788067e60 0x7f878806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:30.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.302+0000 7f87a7400640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8788067e60 0x7f878806a320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87a813c7e0 tx=0x7f8794009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:30.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.306+0000 7f879fe00640 1 -- 172.21.15.110:0/4247622479 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8798056c60 con 0x7f87a80b29e0 2024-09-10T10:04:30.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:30.502+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f87a806ecd0 con 0x7f87a80b29e0 2024-09-10T10:04:31.418 INFO:tasks.ceph.mds.c.smithi110.stderr:2024-09-10T10:04:31.418+0000 7f8adfc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:04:31.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.458+0000 7f879fe00640 1 -- 172.21.15.110:0/4247622479 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v87) ==== 118+0+0 (secure 0 0 0) 0x7f8798056560 con 0x7f87a80b29e0 2024-09-10T10:04:31.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8788067e60 msgr2=0x7f878806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:31.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8788067e60 0x7f878806a320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87a813c7e0 tx=0x7f8794009040 comp rx=0 tx=0).stop 2024-09-10T10:04:31.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80b29e0 msgr2=0x7f87a815e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:31.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80b29e0 0x7f87a815e5f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f879800e8a0 tx=0x7f879800eca0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 shutdown_connections 2024-09-10T10:04:31.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8788067e60 0x7f878806a320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87a8145f30 0x7f87a80aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87a80cfd90 0x7f87a80aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 --2- 172.21.15.110:0/4247622479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87a80b29e0 0x7f87a815e5f0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 >> 172.21.15.110:0/4247622479 conn(0x7f87a8062e40 msgr2=0x7f87a815cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:31.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 shutdown_connections 2024-09-10T10:04:31.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.462+0000 7f87af000640 1 -- 172.21.15.110:0/4247622479 wait complete. 2024-09-10T10:04:31.489 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:04:31.489 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:04:31.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.626+0000 7fc62aa00640 1 Processor -- start 2024-09-10T10:04:31.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.626+0000 7fc62aa00640 1 -- start start 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc62aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc6240ce4d0 0x7fc62406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc62aa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc62406e850 0x7fc6240b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc62aa00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6240b4960 con 0x7fc62415b330 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc62aa00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc6240b5570 con 0x7fc6240ce4d0 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc62aa00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc6240b4c40 con 0x7fc62406e850 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc623400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc6240ce4d0 0x7fc62406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc623e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc62415b330 0x7fc62406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55838/0 (socket says 172.21.15.110:55838) 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc623e00640 1 -- 172.21.15.110:0/4279311532 learned_addr learned my addr 172.21.15.110:0/4279311532 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:31.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc628a00640 1 --2- 172.21.15.110:0/4279311532 >> v2:172.21.15.145:3300/0 conn(0x7fc62406e850 0x7fc6240b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:31.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc623400640 1 -- 172.21.15.110:0/4279311532 >> v2:172.21.15.145:3300/0 conn(0x7fc62406e850 msgr2=0x7fc6240b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:31.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc623400640 1 --2- 172.21.15.110:0/4279311532 >> v2:172.21.15.145:3300/0 conn(0x7fc62406e850 0x7fc6240b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc623400640 1 -- 172.21.15.110:0/4279311532 >> v1:172.21.15.110:6789/0 conn(0x7fc62415b330 legacy=0x7fc62406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:31.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc623400640 1 -- 172.21.15.110:0/4279311532 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6240b4f20 con 0x7fc6240ce4d0 2024-09-10T10:04:31.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc622a00640 1 -- 172.21.15.110:0/4279311532 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368269689 0 0) 0x7fc6240b4960 con 0x7fc62415b330 2024-09-10T10:04:31.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.630+0000 7fc628a00640 1 --2- 172.21.15.110:0/4279311532 >> v2:172.21.15.145:3300/0 conn(0x7fc62406e850 0x7fc6240b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:31.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc623400640 1 --2- 172.21.15.110:0/4279311532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6240ce4d0 0x7fc62406c6a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc6240633d0 tx=0x7fc61001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaf73ab10eab1d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:31.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc622a00640 1 -- 172.21.15.110:0/4279311532 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100b1070 con 0x7fc6240ce4d0 2024-09-10T10:04:31.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc622a00640 1 -- 172.21.15.110:0/4279311532 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6100ac5f0 con 0x7fc6240ce4d0 2024-09-10T10:04:31.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc622a00640 1 -- 172.21.15.110:0/4279311532 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100ac9c0 con 0x7fc6240ce4d0 2024-09-10T10:04:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 -- 172.21.15.110:0/4279311532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6240ce4d0 msgr2=0x7fc62406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4279311532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6240ce4d0 0x7fc62406c6a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc6240633d0 tx=0x7fc61001c480 comp rx=0 tx=0).stop 2024-09-10T10:04:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 -- 172.21.15.110:0/4279311532 shutdown_connections 2024-09-10T10:04:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4279311532 >> v2:172.21.15.145:3300/0 conn(0x7fc62406e850 0x7fc6240b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4279311532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6240ce4d0 0x7fc62406c6a0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 -- 172.21.15.110:0/4279311532 >> 172.21.15.110:0/4279311532 conn(0x7fc624062e40 msgr2=0x7fc62406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 -- 172.21.15.110:0/4279311532 shutdown_connections 2024-09-10T10:04:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 -- 172.21.15.110:0/4279311532 wait complete. 2024-09-10T10:04:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 Processor -- start 2024-09-10T10:04:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 -- start start 2024-09-10T10:04:31.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc62406e850 0x7fc6241f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.634+0000 7fc62aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc62415b330 0x7fc62415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc62aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc62414ba40 0x7fc62414bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc62aa00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241660a0 con 0x7fc62406e850 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc62aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6240b1800 con 0x7fc62414ba40 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc62aa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc624166220 con 0x7fc62415b330 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc62406e850 0x7fc6241f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc62406e850 0x7fc6241f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51972/0 (socket says 172.21.15.110:51972) 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 -- 172.21.15.110:0/4053074639 learned_addr learned my addr 172.21.15.110:0/4053074639 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 -- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc62415b330 msgr2=0x7fc62415bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc62415b330 0x7fc62415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 -- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc62414ba40 msgr2=0x7fc62414bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:31.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc62414ba40 0x7fc62414bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:31.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 -- 172.21.15.110:0/4053074639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62415c610 con 0x7fc62406e850 2024-09-10T10:04:31.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc623e00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc62406e850 0x7fc6241f5030 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c7c0 tx=0x7fc61400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:31.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc61be00640 1 -- 172.21.15.110:0/4053074639 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614018070 con 0x7fc62406e850 2024-09-10T10:04:31.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc61be00640 1 -- 172.21.15.110:0/4053074639 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc61400a640 con 0x7fc62406e850 2024-09-10T10:04:31.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc61be00640 1 -- 172.21.15.110:0/4053074639 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61400aa30 con 0x7fc62406e850 2024-09-10T10:04:31.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62415c8f0 con 0x7fc62406e850 2024-09-10T10:04:31.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62415ceb0 con 0x7fc62406e850 2024-09-10T10:04:31.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62406d6b0 con 0x7fc62406e850 2024-09-10T10:04:31.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.638+0000 7fc61be00640 1 -- 172.21.15.110:0/4053074639 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fc61400abd0 con 0x7fc62406e850 2024-09-10T10:04:31.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.642+0000 7fc61be00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc604067e90 0x7fc60406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:31.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.642+0000 7fc61be00640 1 -- 172.21.15.110:0/4053074639 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fc614085d00 con 0x7fc62406e850 2024-09-10T10:04:31.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.642+0000 7fc623400640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc604067e90 0x7fc60406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:31.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.642+0000 7fc623400640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc604067e90 0x7fc60406a350 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc6100ae000 tx=0x7fc610004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:31.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.646+0000 7fc61be00640 1 -- 172.21.15.110:0/4053074639 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc614055f50 con 0x7fc62406e850 2024-09-10T10:04:31.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:31.838+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc62406d3d0 con 0x7fc62406e850 2024-09-10T10:04:32.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.462+0000 7fc61be00640 1 -- 172.21.15.110:0/4053074639 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v88) ==== 122+0+0 (secure 0 0 0) 0x7fc614055850 con 0x7fc62406e850 2024-09-10T10:04:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc604067e90 msgr2=0x7fc60406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc604067e90 0x7fc60406a350 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc6100ae000 tx=0x7fc610004740 comp rx=0 tx=0).stop 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc62406e850 msgr2=0x7fc6241f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc62406e850 0x7fc6241f5030 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c7c0 tx=0x7fc61400cc80 comp rx=0 tx=0).stop 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 shutdown_connections 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc604067e90 0x7fc60406a350 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc62414ba40 0x7fc62414bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc62415b330 0x7fc62415bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 --2- 172.21.15.110:0/4053074639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc62406e850 0x7fc6241f5030 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 >> 172.21.15.110:0/4053074639 conn(0x7fc624062e40 msgr2=0x7fc6240cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 shutdown_connections 2024-09-10T10:04:32.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.466+0000 7fc62aa00640 1 -- 172.21.15.110:0/4053074639 wait complete. 2024-09-10T10:04:32.492 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:04:32.492 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:32.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fc1600640 1 Processor -- start 2024-09-10T10:04:32.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fc1600640 1 -- start start 2024-09-10T10:04:32.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fc1600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1fbc15c960 0x7f1fbc0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fc1600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fbc0706a0 con 0x7f1fbc1603c0 2024-09-10T10:04:32.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fc1600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fbc070a80 con 0x7f1fbc0d13e0 2024-09-10T10:04:32.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fc1600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1fbc15cd40 con 0x7f1fbc15c960 2024-09-10T10:04:32.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fba000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1fbc1603c0 0x7f1fbc0d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55846/0 (socket says 172.21.15.110:55846) 2024-09-10T10:04:32.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fba000640 1 -- 172.21.15.110:0/1817342624 learned_addr learned my addr 172.21.15.110:0/1817342624 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:32.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fbaa00640 1 --2- 172.21.15.110:0/1817342624 >> v2:172.21.15.145:3300/0 conn(0x7f1fbc15c960 0x7f1fbc0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645994235 0 0) 0x7f1fbc0706a0 con 0x7f1fbc1603c0 2024-09-10T10:04:32.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa0003530 con 0x7f1fbc1603c0 2024-09-10T10:04:32.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3894582058 0 0) 0x7f1fa0003530 con 0x7f1fbc1603c0 2024-09-10T10:04:32.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 >> v2:172.21.15.145:3300/0 conn(0x7f1fbc15c960 msgr2=0x7f1fbc0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:32.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 --2- 172.21.15.110:0/1817342624 >> v2:172.21.15.145:3300/0 conn(0x7f1fbc15c960 0x7f1fbc0b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 >> v1:172.21.15.145:6789/0 conn(0x7f1fbc0d13e0 legacy=0x7f1fbc070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc160910 con 0x7f1fbc1603c0 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f1fa8001680 con 0x7f1fbc1603c0 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 >> v1:172.21.15.110:6789/0 conn(0x7f1fbc1603c0 legacy=0x7f1fbc0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1fa00039c0 0x7f1fa0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa0004810 0x7f1fa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fa000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc15cd40 con 0x7f1fa00051d0 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa000c120 con 0x7f1fa0004810 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa000c320 con 0x7f1fa00039c0 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fbb400640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fa000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fbaa00640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1fa00039c0 0x7f1fa0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.626+0000 7f1fba000640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa0004810 0x7f1fa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fbb400640 1 -- 172.21.15.110:0/1817342624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1fa00039c0 msgr2=0x7f1fa0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:32.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fbb400640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1fa00039c0 0x7f1fa0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fbb400640 1 -- 172.21.15.110:0/1817342624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa0004810 msgr2=0x7f1fa0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:32.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fbb400640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa0004810 0x7f1fa0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fbb400640 1 -- 172.21.15.110:0/1817342624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa0003530 con 0x7f1fa00051d0 2024-09-10T10:04:32.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fbb400640 1 -- 172.21.15.110:0/1817342624 reap_dead start 2024-09-10T10:04:32.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fbb400640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fa000b9d0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4001f60 tx=0x7f1fb401eff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1bb4f42d9d85ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:32.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4001b60 con 0x7f1fa00051d0 2024-09-10T10:04:32.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb40a8c10 con 0x7f1fa00051d0 2024-09-10T10:04:32.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fb9600640 1 -- 172.21.15.110:0/1817342624 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4003980 con 0x7f1fa00051d0 2024-09-10T10:04:32.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fc1600640 1 -- 172.21.15.110:0/1817342624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 msgr2=0x7f1fa000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fc1600640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fa000b9d0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4001f60 tx=0x7f1fb401eff0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fc1600640 1 -- 172.21.15.110:0/1817342624 shutdown_connections 2024-09-10T10:04:32.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fc1600640 1 --2- 172.21.15.110:0/1817342624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fa000b9d0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fc1600640 1 -- 172.21.15.110:0/1817342624 >> 172.21.15.110:0/1817342624 conn(0x7f1fbc0d25c0 msgr2=0x7f1fbc0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:32.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fc1600640 1 -- 172.21.15.110:0/1817342624 shutdown_connections 2024-09-10T10:04:32.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.630+0000 7f1fc1600640 1 -- 172.21.15.110:0/1817342624 wait complete. 2024-09-10T10:04:32.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 Processor -- start 2024-09-10T10:04:32.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 -- start start 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fbc1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc1f3410 0x7f1fbc15f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1fbc15fa60 0x7f1fbc0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1611b0 con 0x7f1fa00051d0 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc160ed0 con 0x7f1fbc1f3410 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1fbc0d3150 con 0x7f1fbc15fa60 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fbc1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fbc1546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51996/0 (socket says 172.21.15.110:51996) 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 -- 172.21.15.110:0/3081803288 learned_addr learned my addr 172.21.15.110:0/3081803288 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 -- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1fbc15fa60 msgr2=0x7f1fbc0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1fbc15fa60 0x7f1fbc0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 -- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc1f3410 msgr2=0x7f1fbc15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fba000640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc1f3410 0x7f1fbc15f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc1f3410 0x7f1fbc15f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 -- 172.21.15.110:0/3081803288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc0c9c30 con 0x7f1fa00051d0 2024-09-10T10:04:32.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fba000640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc1f3410 0x7f1fbc15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:32.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fbaa00640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fbc1546b0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0119c0 tx=0x7f1fac011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:32.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fb2a00640 1 -- 172.21.15.110:0/3081803288 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00fa90 con 0x7f1fa00051d0 2024-09-10T10:04:32.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fb2a00640 1 -- 172.21.15.110:0/3081803288 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac00fc30 con 0x7f1fa00051d0 2024-09-10T10:04:32.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fb2a00640 1 -- 172.21.15.110:0/3081803288 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac014440 con 0x7f1fa00051d0 2024-09-10T10:04:32.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc15d060 con 0x7f1fa00051d0 2024-09-10T10:04:32.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.634+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc15d620 con 0x7f1fa00051d0 2024-09-10T10:04:32.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.638+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc15fe80 con 0x7f1fa00051d0 2024-09-10T10:04:32.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.638+0000 7f1fb2a00640 1 -- 172.21.15.110:0/3081803288 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f1fac012040 con 0x7f1fa00051d0 2024-09-10T10:04:32.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.638+0000 7f1fb2a00640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1f9c067e60 0x7f1f9c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.638+0000 7f1fb2a00640 1 -- 172.21.15.110:0/3081803288 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f1fac086cf0 con 0x7f1fa00051d0 2024-09-10T10:04:32.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.638+0000 7f1fba000640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1f9c067e60 0x7f1f9c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.638+0000 7f1fba000640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1f9c067e60 0x7f1f9c06a320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8000c10 tx=0x7f1fa8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:32.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.642+0000 7f1fb2a00640 1 -- 172.21.15.110:0/3081803288 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1fac019030 con 0x7f1fa00051d0 2024-09-10T10:04:32.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.830+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fbc0d04c0 con 0x7f1fa00051d0 2024-09-10T10:04:32.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.830+0000 7f1fb2a00640 1 -- 172.21.15.110:0/3081803288 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5239 (secure 0 0 0) 0x7f1fac056fd0 con 0x7f1fa00051d0 2024-09-10T10:04:32.836 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:32.836 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":88,"btime":"2024-09-10T10:04:32:463468+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3129750368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3129750368},{"type":"v1","addr":"172.21.15.145:6834","nonce":3129750368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3791950507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3791950507},{"type":"v1","addr":"172.21.15.145:6835","nonce":3791950507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2024540113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2024540113},{"type":"v1","addr":"172.21.15.110:6834","nonce":2024540113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:04:28.820193+0000","modified":"2024-09-10T10:04:31.842870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1881265533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1881265533},{"type":"v1","addr":"172.21.15.110:6835","nonce":1881265533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7686,"qdb_cluster":[7686]},"id":10}]} 2024-09-10T10:04:32.837 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 88 2024-09-10T10:04:32.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1f9c067e60 msgr2=0x7f1f9c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1f9c067e60 0x7f1f9c06a320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8000c10 tx=0x7f1fa8009040 comp rx=0 tx=0).stop 2024-09-10T10:04:32.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 msgr2=0x7f1fbc1546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fbc1546b0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0119c0 tx=0x7f1fac011e80 comp rx=0 tx=0).stop 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 shutdown_connections 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1f9c067e60 0x7f1f9c06a320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1fbc15fa60 0x7f1fbc0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fbc1f3410 0x7f1fbc15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 --2- 172.21.15.110:0/3081803288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fa00051d0 0x7f1fbc1546b0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 >> 172.21.15.110:0/3081803288 conn(0x7f1fbc0d25c0 msgr2=0x7f1fbc15baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 shutdown_connections 2024-09-10T10:04:32.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.834+0000 7f1fc1600640 1 -- 172.21.15.110:0/3081803288 wait complete. 2024-09-10T10:04:32.857 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:32.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.986+0000 7f1355600640 1 Processor -- start 2024-09-10T10:04:32.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.986+0000 7f1355600640 1 -- start start 2024-09-10T10:04:32.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f1355600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f13500cfe30 0x7f1350071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f1355600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cd510 0x7f13500cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f1355600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f135006e090 0x7f135006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f1355600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f13500cdeb0 con 0x7f135006e090 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f1355600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f135015d9a0 con 0x7f13500cfe30 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f1355600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f135006ea60 con 0x7f13500cd510 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f135006e090 0x7f135006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f135006e090 0x7f135006e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52004/0 (socket says 172.21.15.110:52004) 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 -- 172.21.15.110:0/4134624461 learned_addr learned my addr 172.21.15.110:0/4134624461 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134ea00640 1 --2- 172.21.15.110:0/4134624461 >> v2:172.21.15.145:3300/0 conn(0x7f13500cfe30 0x7f1350071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134e000640 1 --2- 172.21.15.110:0/4134624461 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cd510 0x7f13500cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 -- 172.21.15.110:0/4134624461 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cd510 msgr2=0x7f13500cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 --2- 172.21.15.110:0/4134624461 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cd510 0x7f13500cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 -- 172.21.15.110:0/4134624461 >> v2:172.21.15.145:3300/0 conn(0x7f13500cfe30 msgr2=0x7f1350071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 --2- 172.21.15.110:0/4134624461 >> v2:172.21.15.145:3300/0 conn(0x7f13500cfe30 0x7f1350071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 -- 172.21.15.110:0/4134624461 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135006d2d0 con 0x7f135006e090 2024-09-10T10:04:32.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.990+0000 7f134f400640 1 --2- 172.21.15.110:0/4134624461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f135006e090 0x7f135006e520 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f13500ce120 tx=0x7f134801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8539642ee107238 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:32.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f134d600640 1 -- 172.21.15.110:0/4134624461 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480b2070 con 0x7f135006e090 2024-09-10T10:04:32.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f134d600640 1 -- 172.21.15.110:0/4134624461 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13480a9070 con 0x7f135006e090 2024-09-10T10:04:32.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f134d600640 1 -- 172.21.15.110:0/4134624461 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480ad880 con 0x7f135006e090 2024-09-10T10:04:32.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- 172.21.15.110:0/4134624461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f135006e090 msgr2=0x7f135006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:32.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 --2- 172.21.15.110:0/4134624461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f135006e090 0x7f135006e520 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f13500ce120 tx=0x7f134801c490 comp rx=0 tx=0).stop 2024-09-10T10:04:32.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- 172.21.15.110:0/4134624461 shutdown_connections 2024-09-10T10:04:32.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 --2- 172.21.15.110:0/4134624461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f135006e090 0x7f135006e520 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 --2- 172.21.15.110:0/4134624461 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cd510 0x7f13500cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 --2- 172.21.15.110:0/4134624461 >> v2:172.21.15.145:3300/0 conn(0x7f13500cfe30 0x7f1350071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:32.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- 172.21.15.110:0/4134624461 >> 172.21.15.110:0/4134624461 conn(0x7f135006bc50 msgr2=0x7f135006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:32.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- 172.21.15.110:0/4134624461 shutdown_connections 2024-09-10T10:04:32.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- 172.21.15.110:0/4134624461 wait complete. 2024-09-10T10:04:32.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 Processor -- start 2024-09-10T10:04:32.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- start start 2024-09-10T10:04:33.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f135006e090 0x7f13500b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13500cd510 0x7f135013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cfe30 0x7f13500b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f135015d430 con 0x7f13500cd510 2024-09-10T10:04:33.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1350072190 con 0x7f135006e090 2024-09-10T10:04:33.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f1355600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f135015d5b0 con 0x7f13500cfe30 2024-09-10T10:04:33.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f134e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13500cd510 0x7f135013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f134e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13500cd510 0x7f135013d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52014/0 (socket says 172.21.15.110:52014) 2024-09-10T10:04:33.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f134e000640 1 -- 172.21.15.110:0/1594623921 learned_addr learned my addr 172.21.15.110:0/1594623921 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:33.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f134f400640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cfe30 0x7f13500b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.994+0000 7f134ea00640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f135006e090 0x7f13500b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f134e000640 1 -- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cfe30 msgr2=0x7f13500b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f134e000640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cfe30 0x7f13500b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f134e000640 1 -- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f135006e090 msgr2=0x7f13500b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f134e000640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f135006e090 0x7f13500b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f134e000640 1 -- 172.21.15.110:0/1594623921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135013ddc0 con 0x7f13500cd510 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f134e000640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13500cd510 0x7f135013d880 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f133c00d5f0 tx=0x7f133c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1346a00640 1 -- 172.21.15.110:0/1594623921 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c018020 con 0x7f13500cd510 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1346a00640 1 -- 172.21.15.110:0/1594623921 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c014620 con 0x7f13500cd510 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13500b1d40 con 0x7f13500cd510 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1346a00640 1 -- 172.21.15.110:0/1594623921 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c014a10 con 0x7f13500cd510 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13501543f0 con 0x7f13500cd510 2024-09-10T10:04:33.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13500cd970 con 0x7f13500cd510 2024-09-10T10:04:33.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1346a00640 1 -- 172.21.15.110:0/1594623921 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f133c01b020 con 0x7f13500cd510 2024-09-10T10:04:33.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1346a00640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1330067e90 0x7f133006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:32.998+0000 7f1346a00640 1 -- 172.21.15.110:0/1594623921 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f133c086aa0 con 0x7f13500cd510 2024-09-10T10:04:33.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.002+0000 7f134ea00640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1330067e90 0x7f133006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.002+0000 7f134ea00640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1330067e90 0x7f133006a350 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1340003b90 tx=0x7f13400098d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.002+0000 7f1346a00640 1 -- 172.21.15.110:0/1594623921 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f133c022020 con 0x7f13500cd510 2024-09-10T10:04:33.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.162+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f135006c440 con 0x7f13500cd510 2024-09-10T10:04:33.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.162+0000 7f1346a00640 1 -- 172.21.15.110:0/1594623921 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16864 (secure 0 0 0) 0x7f133c056730 con 0x7f13500cd510 2024-09-10T10:04:33.168 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:33.168 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":130,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:30.451627+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:04:26.532408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T10:04:27.823557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:33.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1330067e90 msgr2=0x7f133006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1330067e90 0x7f133006a350 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1340003b90 tx=0x7f13400098d0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13500cd510 msgr2=0x7f135013d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13500cd510 0x7f135013d880 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f133c00d5f0 tx=0x7f133c00dab0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 shutdown_connections 2024-09-10T10:04:33.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1330067e90 0x7f133006a350 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f13500cfe30 0x7f13500b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13500cd510 0x7f135013d880 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 --2- 172.21.15.110:0/1594623921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f135006e090 0x7f13500b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 >> 172.21.15.110:0/1594623921 conn(0x7f135006bc50 msgr2=0x7f13500b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 shutdown_connections 2024-09-10T10:04:33.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.166+0000 7f1355600640 1 -- 172.21.15.110:0/1594623921 wait complete. 2024-09-10T10:04:33.190 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:04:33.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:04:33.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0477e00640 1 Processor -- start 2024-09-10T10:04:33.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0477e00640 1 -- start start 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0477e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f047815e470 0x7f04780b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0477e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0478162020 0x7f04780731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0477e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f047806d020 con 0x7f0478073720 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0477e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f04780b6870 con 0x7f0478162020 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0477e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04780b5c60 con 0x7f047815e470 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0476000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0478162020 0x7f04780731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0476000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0478162020 0x7f04780731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52016/0 (socket says 172.21.15.110:52016) 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0476000640 1 -- 172.21.15.110:0/2699850401 learned_addr learned my addr 172.21.15.110:0/2699850401 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0476a00640 1 --2- 172.21.15.110:0/2699850401 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f047815e470 0x7f04780b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0476000640 1 -- 172.21.15.110:0/2699850401 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f047815e470 msgr2=0x7f04780b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0476000640 1 --2- 172.21.15.110:0/2699850401 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f047815e470 0x7f04780b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0476000640 1 -- 172.21.15.110:0/2699850401 >> v1:172.21.15.110:6789/0 conn(0x7f0478073720 legacy=0x7f0478070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.322+0000 7f0476000640 1 -- 172.21.15.110:0/2699850401 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04780b5f40 con 0x7f0478162020 2024-09-10T10:04:33.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0476000640 1 --2- 172.21.15.110:0/2699850401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f04780731e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0464009870 tx=0x7f046401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1e8f73303ffb85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0475600640 1 -- 172.21.15.110:0/2699850401 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04640b5070 con 0x7f0478162020 2024-09-10T10:04:33.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0475600640 1 -- 172.21.15.110:0/2699850401 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04640a8a50 con 0x7f0478162020 2024-09-10T10:04:33.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0475600640 1 -- 172.21.15.110:0/2699850401 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04640a8e20 con 0x7f0478162020 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0477e00640 1 -- 172.21.15.110:0/2699850401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 msgr2=0x7f04780731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0477e00640 1 --2- 172.21.15.110:0/2699850401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f04780731e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0464009870 tx=0x7f046401ed50 comp rx=0 tx=0).stop 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0477e00640 1 -- 172.21.15.110:0/2699850401 shutdown_connections 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0477e00640 1 --2- 172.21.15.110:0/2699850401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f04780731e0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0477e00640 1 --2- 172.21.15.110:0/2699850401 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f047815e470 0x7f04780b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0477e00640 1 -- 172.21.15.110:0/2699850401 >> 172.21.15.110:0/2699850401 conn(0x7f04780b42d0 msgr2=0x7f04780b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0477e00640 1 -- 172.21.15.110:0/2699850401 shutdown_connections 2024-09-10T10:04:33.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.326+0000 7f0477e00640 1 -- 172.21.15.110:0/2699850401 wait complete. 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 Processor -- start 2024-09-10T10:04:33.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 -- start start 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0478073720 0x7f047814f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f047815e470 0x7f047815f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f0478160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f047815e850 con 0x7f0478162020 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04780b6500 con 0x7f047815e470 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f047806e730 con 0x7f0478073720 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f0478160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f0478160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52026/0 (socket says 172.21.15.110:52026) 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 -- 172.21.15.110:0/1845072197 learned_addr learned my addr 172.21.15.110:0/1845072197 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0476a00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0478073720 0x7f047814f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0476000640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f047815e470 0x7f047815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 -- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0478073720 msgr2=0x7f047814f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0478073720 0x7f047814f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 -- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f047815e470 msgr2=0x7f047815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f047815e470 0x7f047815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 -- 172.21.15.110:0/1845072197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0478160b60 con 0x7f0478162020 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0476000640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f047815e470 0x7f047815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:33.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477400640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f0478160180 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f047000c430 tx=0x7f047000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f046ea00640 1 -- 172.21.15.110:0/1845072197 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470005220 con 0x7f0478162020 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f046ea00640 1 -- 172.21.15.110:0/1845072197 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04700053c0 con 0x7f0478162020 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f046ea00640 1 -- 172.21.15.110:0/1845072197 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470007270 con 0x7f0478162020 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047815e9d0 con 0x7f0478162020 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.330+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047815ef90 con 0x7f0478162020 2024-09-10T10:04:33.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.334+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04780731e0 con 0x7f0478162020 2024-09-10T10:04:33.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.334+0000 7f046ea00640 1 -- 172.21.15.110:0/1845072197 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f0470019020 con 0x7f0478162020 2024-09-10T10:04:33.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.334+0000 7f046ea00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0458067e60 0x7f045806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.334+0000 7f046ea00640 1 -- 172.21.15.110:0/1845072197 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f0470085a30 con 0x7f0478162020 2024-09-10T10:04:33.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.334+0000 7f0476a00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0458067e60 0x7f045806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.334+0000 7f0476a00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0458067e60 0x7f045806a320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f04680113e0 tx=0x7f046800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.338+0000 7f046ea00640 1 -- 172.21.15.110:0/1845072197 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0470055d10 con 0x7f0478162020 2024-09-10T10:04:33.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.494+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04780b22e0 con 0x7f0458067e60 2024-09-10T10:04:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.502+0000 7f046ea00640 1 -- 172.21.15.110:0/1845072197 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252453 (secure 0 0 0) 0x7f04780b22e0 con 0x7f0458067e60 2024-09-10T10:04:33.509 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:33.510 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:04:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0458067e60 msgr2=0x7f045806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0458067e60 0x7f045806a320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f04680113e0 tx=0x7f046800b040 comp rx=0 tx=0).stop 2024-09-10T10:04:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 msgr2=0x7f0478160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f0478160180 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f047000c430 tx=0x7f047000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 shutdown_connections 2024-09-10T10:04:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0458067e60 0x7f045806a320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0478162020 0x7f0478160180 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f047815e470 0x7f047815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 --2- 172.21.15.110:0/1845072197 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0478073720 0x7f047814f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 >> 172.21.15.110:0/1845072197 conn(0x7f04780b42d0 msgr2=0x7f0478159160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 shutdown_connections 2024-09-10T10:04:33.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.506+0000 7f0477e00640 1 -- 172.21.15.110:0/1845072197 wait complete. 2024-09-10T10:04:33.529 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T10:04:31.723412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271680,"kb_used_data":13836,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754703040,"statfs":{"total":773094113280,"available":772815912960,"internally_reserved":0,"allocated":14168064,"data_stored":6873136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":263835180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.291937"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.460051+0000","last_change":"2024-09-10T10:04:29.464188+0000","last_active":"2024-09-10T10:04:30.460051+0000","last_peered":"2024-09-10T10:04:30.460051+0000","last_clean":"2024-09-10T10:04:30.460051+0000","last_became_active":"2024-09-10T10:04:29.463940+0000","last_became_peered":"2024-09-10T10:04:29.463940+0000","last_unstale":"2024-09-10T10:04:30.460051+0000","last_undegraded":"2024-09-10T10:04:30.460051+0000","last_fullsized":"2024-09-10T10:04:30.460051+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:13.591982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.474929+0000","last_change":"2024-09-10T10:04:29.468463+0000","last_active":"2024-09-10T10:04:29.474929+0000","last_peered":"2024-09-10T10:04:29.474929+0000","last_clean":"2024-09-10T10:04:29.474929+0000","last_became_active":"2024-09-10T10:04:29.468204+0000","last_became_peered":"2024-09-10T10:04:29.468204+0000","last_unstale":"2024-09-10T10:04:29.474929+0000","last_undegraded":"2024-09-10T10:04:29.474929+0000","last_fullsized":"2024-09-10T10:04:29.474929+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:56.046353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465882+0000","last_change":"2024-09-10T10:04:29.467526+0000","last_active":"2024-09-10T10:04:30.465882+0000","last_peered":"2024-09-10T10:04:30.465882+0000","last_clean":"2024-09-10T10:04:30.465882+0000","last_became_active":"2024-09-10T10:04:29.467107+0000","last_became_peered":"2024-09-10T10:04:29.467107+0000","last_unstale":"2024-09-10T10:04:30.465882+0000","last_undegraded":"2024-09-10T10:04:30.465882+0000","last_fullsized":"2024-09-10T10:04:30.465882+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:24.790019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466346+0000","last_change":"2024-09-10T10:04:29.457932+0000","last_active":"2024-09-10T10:04:30.466346+0000","last_peered":"2024-09-10T10:04:30.466346+0000","last_clean":"2024-09-10T10:04:30.466346+0000","last_became_active":"2024-09-10T10:04:29.457490+0000","last_became_peered":"2024-09-10T10:04:29.457490+0000","last_unstale":"2024-09-10T10:04:30.466346+0000","last_undegraded":"2024-09-10T10:04:30.466346+0000","last_fullsized":"2024-09-10T10:04:30.466346+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:42.907084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473399+0000","last_change":"2024-09-10T10:04:29.466898+0000","last_active":"2024-09-10T10:04:29.473399+0000","last_peered":"2024-09-10T10:04:29.473399+0000","last_clean":"2024-09-10T10:04:29.473399+0000","last_became_active":"2024-09-10T10:04:29.466404+0000","last_became_peered":"2024-09-10T10:04:29.466404+0000","last_unstale":"2024-09-10T10:04:29.473399+0000","last_undegraded":"2024-09-10T10:04:29.473399+0000","last_fullsized":"2024-09-10T10:04:29.473399+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:10.518696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458924+0000","last_change":"2024-09-10T10:04:29.452539+0000","last_active":"2024-09-10T10:04:30.458924+0000","last_peered":"2024-09-10T10:04:30.458924+0000","last_clean":"2024-09-10T10:04:30.458924+0000","last_became_active":"2024-09-10T10:04:29.452281+0000","last_became_peered":"2024-09-10T10:04:29.452281+0000","last_unstale":"2024-09-10T10:04:30.458924+0000","last_undegraded":"2024-09-10T10:04:30.458924+0000","last_fullsized":"2024-09-10T10:04:30.458924+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:44:08.724445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.466631+0000","last_change":"2024-09-10T10:04:29.461438+0000","last_active":"2024-09-10T10:04:29.466631+0000","last_peered":"2024-09-10T10:04:29.466631+0000","last_clean":"2024-09-10T10:04:29.466631+0000","last_became_active":"2024-09-10T10:04:29.460885+0000","last_became_peered":"2024-09-10T10:04:29.460885+0000","last_unstale":"2024-09-10T10:04:29.466631+0000","last_undegraded":"2024-09-10T10:04:29.466631+0000","last_fullsized":"2024-09-10T10:04:29.466631+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:56.969006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458785+0000","last_change":"2024-09-10T10:04:29.459320+0000","last_active":"2024-09-10T10:04:30.458785+0000","last_peered":"2024-09-10T10:04:30.458785+0000","last_clean":"2024-09-10T10:04:30.458785+0000","last_became_active":"2024-09-10T10:04:29.459047+0000","last_became_peered":"2024-09-10T10:04:29.459047+0000","last_unstale":"2024-09-10T10:04:30.458785+0000","last_undegraded":"2024-09-10T10:04:30.458785+0000","last_fullsized":"2024-09-10T10:04:30.458785+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:31.277449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473145+0000","last_change":"2024-09-10T10:04:29.464993+0000","last_active":"2024-09-10T10:04:29.473145+0000","last_peered":"2024-09-10T10:04:29.473145+0000","last_clean":"2024-09-10T10:04:29.473145+0000","last_became_active":"2024-09-10T10:04:29.464723+0000","last_became_peered":"2024-09-10T10:04:29.464723+0000","last_unstale":"2024-09-10T10:04:29.473145+0000","last_undegraded":"2024-09-10T10:04:29.473145+0000","last_fullsized":"2024-09-10T10:04:29.473145+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:07.866522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452590+0000","last_change":"2024-09-10T10:04:29.464700+0000","last_active":"2024-09-10T10:04:31.452590+0000","last_peered":"2024-09-10T10:04:31.452590+0000","last_clean":"2024-09-10T10:04:31.452590+0000","last_became_active":"2024-09-10T10:04:29.464117+0000","last_became_peered":"2024-09-10T10:04:29.464117+0000","last_unstale":"2024-09-10T10:04:31.452590+0000","last_undegraded":"2024-09-10T10:04:31.452590+0000","last_fullsized":"2024-09-10T10:04:31.452590+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:57.538761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459010+0000","last_change":"2024-09-10T10:04:29.462235+0000","last_active":"2024-09-10T10:04:30.459010+0000","last_peered":"2024-09-10T10:04:30.459010+0000","last_clean":"2024-09-10T10:04:30.459010+0000","last_became_active":"2024-09-10T10:04:29.461957+0000","last_became_peered":"2024-09-10T10:04:29.461957+0000","last_unstale":"2024-09-10T10:04:30.459010+0000","last_undegraded":"2024-09-10T10:04:30.459010+0000","last_fullsized":"2024-09-10T10:04:30.459010+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:53.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452465+0000","last_change":"2024-09-10T10:04:29.466716+0000","last_active":"2024-09-10T10:04:31.452465+0000","last_peered":"2024-09-10T10:04:31.452465+0000","last_clean":"2024-09-10T10:04:31.452465+0000","last_became_active":"2024-09-10T10:04:29.466412+0000","last_became_peered":"2024-09-10T10:04:29.466412+0000","last_unstale":"2024-09-10T10:04:31.452465+0000","last_undegraded":"2024-09-10T10:04:31.452465+0000","last_fullsized":"2024-09-10T10:04:31.452465+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:31.742985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473376+0000","last_change":"2024-09-10T10:04:29.466689+0000","last_active":"2024-09-10T10:04:29.473376+0000","last_peered":"2024-09-10T10:04:29.473376+0000","last_clean":"2024-09-10T10:04:29.473376+0000","last_became_active":"2024-09-10T10:04:29.466052+0000","last_became_peered":"2024-09-10T10:04:29.466052+0000","last_unstale":"2024-09-10T10:04:29.473376+0000","last_undegraded":"2024-09-10T10:04:29.473376+0000","last_fullsized":"2024-09-10T10:04:29.473376+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:23.257966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464905+0000","last_change":"2024-09-10T10:04:29.469312+0000","last_active":"2024-09-10T10:04:30.464905+0000","last_peered":"2024-09-10T10:04:30.464905+0000","last_clean":"2024-09-10T10:04:30.464905+0000","last_became_active":"2024-09-10T10:04:29.469070+0000","last_became_peered":"2024-09-10T10:04:29.469070+0000","last_unstale":"2024-09-10T10:04:30.464905+0000","last_undegraded":"2024-09-10T10:04:30.464905+0000","last_fullsized":"2024-09-10T10:04:30.464905+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:49.759441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452806+0000","last_change":"2024-09-10T10:04:29.464373+0000","last_active":"2024-09-10T10:04:31.452806+0000","last_peered":"2024-09-10T10:04:31.452806+0000","last_clean":"2024-09-10T10:04:31.452806+0000","last_became_active":"2024-09-10T10:04:29.464066+0000","last_became_peered":"2024-09-10T10:04:29.464066+0000","last_unstale":"2024-09-10T10:04:31.452806+0000","last_undegraded":"2024-09-10T10:04:31.452806+0000","last_fullsized":"2024-09-10T10:04:31.452806+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:55.535508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473392+0000","last_change":"2024-09-10T10:04:29.461774+0000","last_active":"2024-09-10T10:04:29.473392+0000","last_peered":"2024-09-10T10:04:29.473392+0000","last_clean":"2024-09-10T10:04:29.473392+0000","last_became_active":"2024-09-10T10:04:29.461467+0000","last_became_peered":"2024-09-10T10:04:29.461467+0000","last_unstale":"2024-09-10T10:04:29.473392+0000","last_undegraded":"2024-09-10T10:04:29.473392+0000","last_fullsized":"2024-09-10T10:04:29.473392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:26.897928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.453118+0000","last_change":"2024-09-10T10:04:29.467282+0000","last_active":"2024-09-10T10:04:31.453118+0000","last_peered":"2024-09-10T10:04:31.453118+0000","last_clean":"2024-09-10T10:04:31.453118+0000","last_became_active":"2024-09-10T10:04:29.465209+0000","last_became_peered":"2024-09-10T10:04:29.465209+0000","last_unstale":"2024-09-10T10:04:31.453118+0000","last_undegraded":"2024-09-10T10:04:31.453118+0000","last_fullsized":"2024-09-10T10:04:31.453118+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.187069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465621+0000","last_change":"2024-09-10T10:04:29.466188+0000","last_active":"2024-09-10T10:04:30.465621+0000","last_peered":"2024-09-10T10:04:30.465621+0000","last_clean":"2024-09-10T10:04:30.465621+0000","last_became_active":"2024-09-10T10:04:29.465828+0000","last_became_peered":"2024-09-10T10:04:29.465828+0000","last_unstale":"2024-09-10T10:04:30.465621+0000","last_undegraded":"2024-09-10T10:04:30.465621+0000","last_fullsized":"2024-09-10T10:04:30.465621+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:35.992553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459406+0000","last_change":"2024-09-10T10:04:29.457367+0000","last_active":"2024-09-10T10:04:30.459406+0000","last_peered":"2024-09-10T10:04:30.459406+0000","last_clean":"2024-09-10T10:04:30.459406+0000","last_became_active":"2024-09-10T10:04:29.456630+0000","last_became_peered":"2024-09-10T10:04:29.456630+0000","last_unstale":"2024-09-10T10:04:30.459406+0000","last_undegraded":"2024-09-10T10:04:30.459406+0000","last_fullsized":"2024-09-10T10:04:30.459406+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:03.075439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.466019+0000","last_change":"2024-09-10T10:04:29.464635+0000","last_active":"2024-09-10T10:04:29.466019+0000","last_peered":"2024-09-10T10:04:29.466019+0000","last_clean":"2024-09-10T10:04:29.466019+0000","last_became_active":"2024-09-10T10:04:29.464376+0000","last_became_peered":"2024-09-10T10:04:29.464376+0000","last_unstale":"2024-09-10T10:04:29.466019+0000","last_undegraded":"2024-09-10T10:04:29.466019+0000","last_fullsized":"2024-09-10T10:04:29.466019+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:43.362603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452838+0000","last_change":"2024-09-10T10:04:29.466456+0000","last_active":"2024-09-10T10:04:31.452838+0000","last_peered":"2024-09-10T10:04:31.452838+0000","last_clean":"2024-09-10T10:04:31.452838+0000","last_became_active":"2024-09-10T10:04:29.464397+0000","last_became_peered":"2024-09-10T10:04:29.464397+0000","last_unstale":"2024-09-10T10:04:31.452838+0000","last_undegraded":"2024-09-10T10:04:31.452838+0000","last_fullsized":"2024-09-10T10:04:31.452838+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:51.427699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465256+0000","last_change":"2024-09-10T10:04:29.468290+0000","last_active":"2024-09-10T10:04:30.465256+0000","last_peered":"2024-09-10T10:04:30.465256+0000","last_clean":"2024-09-10T10:04:30.465256+0000","last_became_active":"2024-09-10T10:04:29.467960+0000","last_became_peered":"2024-09-10T10:04:29.467960+0000","last_unstale":"2024-09-10T10:04:30.465256+0000","last_undegraded":"2024-09-10T10:04:30.465256+0000","last_fullsized":"2024-09-10T10:04:30.465256+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:58.129704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464911+0000","last_change":"2024-09-10T10:04:29.459892+0000","last_active":"2024-09-10T10:04:30.464911+0000","last_peered":"2024-09-10T10:04:30.464911+0000","last_clean":"2024-09-10T10:04:30.464911+0000","last_became_active":"2024-09-10T10:04:29.459620+0000","last_became_peered":"2024-09-10T10:04:29.459620+0000","last_unstale":"2024-09-10T10:04:30.464911+0000","last_undegraded":"2024-09-10T10:04:30.464911+0000","last_fullsized":"2024-09-10T10:04:30.464911+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:41.164700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452231+0000","last_change":"2024-09-10T10:04:29.463636+0000","last_active":"2024-09-10T10:04:31.452231+0000","last_peered":"2024-09-10T10:04:31.452231+0000","last_clean":"2024-09-10T10:04:31.452231+0000","last_became_active":"2024-09-10T10:04:29.462988+0000","last_became_peered":"2024-09-10T10:04:29.462988+0000","last_unstale":"2024-09-10T10:04:31.452231+0000","last_undegraded":"2024-09-10T10:04:31.452231+0000","last_fullsized":"2024-09-10T10:04:31.452231+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:08.577949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.462549+0000","last_change":"2024-09-10T10:04:29.458577+0000","last_active":"2024-09-10T10:04:29.462549+0000","last_peered":"2024-09-10T10:04:29.462549+0000","last_clean":"2024-09-10T10:04:29.462549+0000","last_became_active":"2024-09-10T10:04:29.458258+0000","last_became_peered":"2024-09-10T10:04:29.458258+0000","last_unstale":"2024-09-10T10:04:29.462549+0000","last_undegraded":"2024-09-10T10:04:29.462549+0000","last_fullsized":"2024-09-10T10:04:29.462549+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:26.201496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459778+0000","last_change":"2024-09-10T10:04:29.454611+0000","last_active":"2024-09-10T10:04:30.459778+0000","last_peered":"2024-09-10T10:04:30.459778+0000","last_clean":"2024-09-10T10:04:30.459778+0000","last_became_active":"2024-09-10T10:04:29.453822+0000","last_became_peered":"2024-09-10T10:04:29.453822+0000","last_unstale":"2024-09-10T10:04:30.459778+0000","last_undegraded":"2024-09-10T10:04:30.459778+0000","last_fullsized":"2024-09-10T10:04:30.459778+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:42.308264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.460992+0000","last_change":"2024-09-10T10:04:29.455448+0000","last_active":"2024-09-10T10:04:29.460992+0000","last_peered":"2024-09-10T10:04:29.460992+0000","last_clean":"2024-09-10T10:04:29.460992+0000","last_became_active":"2024-09-10T10:04:29.455195+0000","last_became_peered":"2024-09-10T10:04:29.455195+0000","last_unstale":"2024-09-10T10:04:29.460992+0000","last_undegraded":"2024-09-10T10:04:29.460992+0000","last_fullsized":"2024-09-10T10:04:29.460992+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:20.918438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.455744+0000","last_change":"2024-09-10T10:04:29.453662+0000","last_active":"2024-09-10T10:04:29.455744+0000","last_peered":"2024-09-10T10:04:29.455744+0000","last_clean":"2024-09-10T10:04:29.455744+0000","last_became_active":"2024-09-10T10:04:29.453392+0000","last_became_peered":"2024-09-10T10:04:29.453392+0000","last_unstale":"2024-09-10T10:04:29.455744+0000","last_undegraded":"2024-09-10T10:04:29.455744+0000","last_fullsized":"2024-09-10T10:04:29.455744+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:50.143383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458865+0000","last_change":"2024-09-10T10:04:29.452708+0000","last_active":"2024-09-10T10:04:30.458865+0000","last_peered":"2024-09-10T10:04:30.458865+0000","last_clean":"2024-09-10T10:04:30.458865+0000","last_became_active":"2024-09-10T10:04:29.452457+0000","last_became_peered":"2024-09-10T10:04:29.452457+0000","last_unstale":"2024-09-10T10:04:30.458865+0000","last_undegraded":"2024-09-10T10:04:30.458865+0000","last_fullsized":"2024-09-10T10:04:30.458865+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:38.657060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465398+0000","last_change":"2024-09-10T10:04:29.458411+0000","last_active":"2024-09-10T10:04:30.465398+0000","last_peered":"2024-09-10T10:04:30.465398+0000","last_clean":"2024-09-10T10:04:30.465398+0000","last_became_active":"2024-09-10T10:04:29.457956+0000","last_became_peered":"2024-09-10T10:04:29.457956+0000","last_unstale":"2024-09-10T10:04:30.465398+0000","last_undegraded":"2024-09-10T10:04:30.465398+0000","last_fullsized":"2024-09-10T10:04:30.465398+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:35.992553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.472676+0000","last_change":"2024-09-10T10:04:29.464846+0000","last_active":"2024-09-10T10:04:29.472676+0000","last_peered":"2024-09-10T10:04:29.472676+0000","last_clean":"2024-09-10T10:04:29.472676+0000","last_became_active":"2024-09-10T10:04:29.463671+0000","last_became_peered":"2024-09-10T10:04:29.463671+0000","last_unstale":"2024-09-10T10:04:29.472676+0000","last_undegraded":"2024-09-10T10:04:29.472676+0000","last_fullsized":"2024-09-10T10:04:29.472676+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:06.887302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476719+0000","last_change":"2024-09-10T10:04:29.470329+0000","last_active":"2024-09-10T10:04:29.476719+0000","last_peered":"2024-09-10T10:04:29.476719+0000","last_clean":"2024-09-10T10:04:29.476719+0000","last_became_active":"2024-09-10T10:04:29.469845+0000","last_became_peered":"2024-09-10T10:04:29.469845+0000","last_unstale":"2024-09-10T10:04:29.476719+0000","last_undegraded":"2024-09-10T10:04:29.476719+0000","last_fullsized":"2024-09-10T10:04:29.476719+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:42.344713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452636+0000","last_change":"2024-09-10T10:04:29.467255+0000","last_active":"2024-09-10T10:04:31.452636+0000","last_peered":"2024-09-10T10:04:31.452636+0000","last_clean":"2024-09-10T10:04:31.452636+0000","last_became_active":"2024-09-10T10:04:29.464420+0000","last_became_peered":"2024-09-10T10:04:29.464420+0000","last_unstale":"2024-09-10T10:04:31.452636+0000","last_undegraded":"2024-09-10T10:04:31.452636+0000","last_fullsized":"2024-09-10T10:04:31.452636+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:23.890969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"129'4","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.453057+0000","last_change":"2024-09-10T10:04:28.466121+0000","last_active":"2024-09-10T10:04:31.453057+0000","last_peered":"2024-09-10T10:04:31.453057+0000","last_clean":"2024-09-10T10:04:31.453057+0000","last_became_active":"2024-09-10T10:04:28.465484+0000","last_became_peered":"2024-09-10T10:04:28.465484+0000","last_unstale":"2024-09-10T10:04:31.453057+0000","last_undegraded":"2024-09-10T10:04:31.453057+0000","last_fullsized":"2024-09-10T10:04:31.453057+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:56.442466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.474882+0000","last_change":"2024-09-10T10:04:29.469003+0000","last_active":"2024-09-10T10:04:29.474882+0000","last_peered":"2024-09-10T10:04:29.474882+0000","last_clean":"2024-09-10T10:04:29.474882+0000","last_became_active":"2024-09-10T10:04:29.468713+0000","last_became_peered":"2024-09-10T10:04:29.468713+0000","last_unstale":"2024-09-10T10:04:29.474882+0000","last_undegraded":"2024-09-10T10:04:29.474882+0000","last_fullsized":"2024-09-10T10:04:29.474882+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:14.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.474489+0000","last_change":"2024-09-10T10:04:28.461952+0000","last_active":"2024-09-10T10:04:29.474489+0000","last_peered":"2024-09-10T10:04:29.474489+0000","last_clean":"2024-09-10T10:04:29.474489+0000","last_became_active":"2024-09-10T10:04:28.461666+0000","last_became_peered":"2024-09-10T10:04:28.461666+0000","last_unstale":"2024-09-10T10:04:29.474489+0000","last_undegraded":"2024-09-10T10:04:29.474489+0000","last_fullsized":"2024-09-10T10:04:29.474489+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:29.591631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458968+0000","last_change":"2024-09-10T10:04:29.458011+0000","last_active":"2024-09-10T10:04:30.458968+0000","last_peered":"2024-09-10T10:04:30.458968+0000","last_clean":"2024-09-10T10:04:30.458968+0000","last_became_active":"2024-09-10T10:04:29.457744+0000","last_became_peered":"2024-09-10T10:04:29.457744+0000","last_unstale":"2024-09-10T10:04:30.458968+0000","last_undegraded":"2024-09-10T10:04:30.458968+0000","last_fullsized":"2024-09-10T10:04:30.458968+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:29.745244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465762+0000","last_change":"2024-09-10T10:04:28.455407+0000","last_active":"2024-09-10T10:04:30.465762+0000","last_peered":"2024-09-10T10:04:30.465762+0000","last_clean":"2024-09-10T10:04:30.465762+0000","last_became_active":"2024-09-10T10:04:28.454290+0000","last_became_peered":"2024-09-10T10:04:28.454290+0000","last_unstale":"2024-09-10T10:04:30.465762+0000","last_undegraded":"2024-09-10T10:04:30.465762+0000","last_fullsized":"2024-09-10T10:04:30.465762+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:42:04.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452432+0000","last_change":"2024-09-10T10:04:29.463729+0000","last_active":"2024-09-10T10:04:31.452432+0000","last_peered":"2024-09-10T10:04:31.452432+0000","last_clean":"2024-09-10T10:04:31.452432+0000","last_became_active":"2024-09-10T10:04:29.463384+0000","last_became_peered":"2024-09-10T10:04:29.463384+0000","last_unstale":"2024-09-10T10:04:31.452432+0000","last_undegraded":"2024-09-10T10:04:31.452432+0000","last_fullsized":"2024-09-10T10:04:31.452432+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:26.765169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466359+0000","last_change":"2024-09-10T10:04:28.454198+0000","last_active":"2024-09-10T10:04:30.466359+0000","last_peered":"2024-09-10T10:04:30.466359+0000","last_clean":"2024-09-10T10:04:30.466359+0000","last_became_active":"2024-09-10T10:04:28.453633+0000","last_became_peered":"2024-09-10T10:04:28.453633+0000","last_unstale":"2024-09-10T10:04:30.466359+0000","last_undegraded":"2024-09-10T10:04:30.466359+0000","last_fullsized":"2024-09-10T10:04:30.466359+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:52.586896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.470364+0000","last_change":"2024-09-10T10:04:29.463463+0000","last_active":"2024-09-10T10:04:29.470364+0000","last_peered":"2024-09-10T10:04:29.470364+0000","last_clean":"2024-09-10T10:04:29.470364+0000","last_became_active":"2024-09-10T10:04:29.463164+0000","last_became_peered":"2024-09-10T10:04:29.463164+0000","last_unstale":"2024-09-10T10:04:29.470364+0000","last_undegraded":"2024-09-10T10:04:29.470364+0000","last_fullsized":"2024-09-10T10:04:29.470364+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:49.834054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476816+0000","last_change":"2024-09-10T10:04:28.461411+0000","last_active":"2024-09-10T10:04:29.476816+0000","last_peered":"2024-09-10T10:04:29.476816+0000","last_clean":"2024-09-10T10:04:29.476816+0000","last_became_active":"2024-09-10T10:04:28.461163+0000","last_became_peered":"2024-09-10T10:04:28.461163+0000","last_unstale":"2024-09-10T10:04:29.476816+0000","last_undegraded":"2024-09-10T10:04:29.476816+0000","last_fullsized":"2024-09-10T10:04:29.476816+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:39.676002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466037+0000","last_change":"2024-09-10T10:04:29.463989+0000","last_active":"2024-09-10T10:04:30.466037+0000","last_peered":"2024-09-10T10:04:30.466037+0000","last_clean":"2024-09-10T10:04:30.466037+0000","last_became_active":"2024-09-10T10:04:29.461843+0000","last_became_peered":"2024-09-10T10:04:29.461843+0000","last_unstale":"2024-09-10T10:04:30.466037+0000","last_undegraded":"2024-09-10T10:04:30.466037+0000","last_fullsized":"2024-09-10T10:04:30.466037+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:12.119083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.475461+0000","last_change":"2024-09-10T10:04:28.462377+0000","last_active":"2024-09-10T10:04:29.475461+0000","last_peered":"2024-09-10T10:04:29.475461+0000","last_clean":"2024-09-10T10:04:29.475461+0000","last_became_active":"2024-09-10T10:04:28.462093+0000","last_became_peered":"2024-09-10T10:04:28.462093+0000","last_unstale":"2024-09-10T10:04:29.475461+0000","last_undegraded":"2024-09-10T10:04:29.475461+0000","last_fullsized":"2024-09-10T10:04:29.475461+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:48.948456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473166+0000","last_change":"2024-09-10T10:04:29.465476+0000","last_active":"2024-09-10T10:04:29.473166+0000","last_peered":"2024-09-10T10:04:29.473166+0000","last_clean":"2024-09-10T10:04:29.473166+0000","last_became_active":"2024-09-10T10:04:29.465214+0000","last_became_peered":"2024-09-10T10:04:29.465214+0000","last_unstale":"2024-09-10T10:04:29.473166+0000","last_undegraded":"2024-09-10T10:04:29.473166+0000","last_fullsized":"2024-09-10T10:04:29.473166+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:00.305047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.448235+0000","last_change":"2024-09-10T10:04:28.457092+0000","last_active":"2024-09-10T10:04:29.448235+0000","last_peered":"2024-09-10T10:04:29.448235+0000","last_clean":"2024-09-10T10:04:29.448235+0000","last_became_active":"2024-09-10T10:04:28.456832+0000","last_became_peered":"2024-09-10T10:04:28.456832+0000","last_unstale":"2024-09-10T10:04:29.448235+0000","last_undegraded":"2024-09-10T10:04:29.448235+0000","last_fullsized":"2024-09-10T10:04:29.448235+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:02.024448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.469957+0000","last_change":"2024-09-10T10:04:29.462420+0000","last_active":"2024-09-10T10:04:29.469957+0000","last_peered":"2024-09-10T10:04:29.469957+0000","last_clean":"2024-09-10T10:04:29.469957+0000","last_became_active":"2024-09-10T10:04:29.459776+0000","last_became_peered":"2024-09-10T10:04:29.459776+0000","last_unstale":"2024-09-10T10:04:29.469957+0000","last_undegraded":"2024-09-10T10:04:29.469957+0000","last_fullsized":"2024-09-10T10:04:29.469957+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:41.331025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464690+0000","last_change":"2024-09-10T10:04:28.461601+0000","last_active":"2024-09-10T10:04:30.464690+0000","last_peered":"2024-09-10T10:04:30.464690+0000","last_clean":"2024-09-10T10:04:30.464690+0000","last_became_active":"2024-09-10T10:04:28.461312+0000","last_became_peered":"2024-09-10T10:04:28.461312+0000","last_unstale":"2024-09-10T10:04:30.464690+0000","last_undegraded":"2024-09-10T10:04:30.464690+0000","last_fullsized":"2024-09-10T10:04:30.464690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:42.657191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.469924+0000","last_change":"2024-09-10T10:04:29.460205+0000","last_active":"2024-09-10T10:04:29.469924+0000","last_peered":"2024-09-10T10:04:29.469924+0000","last_clean":"2024-09-10T10:04:29.469924+0000","last_became_active":"2024-09-10T10:04:29.459879+0000","last_became_peered":"2024-09-10T10:04:29.459879+0000","last_unstale":"2024-09-10T10:04:29.469924+0000","last_undegraded":"2024-09-10T10:04:29.469924+0000","last_fullsized":"2024-09-10T10:04:29.469924+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:34.339057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.455255+0000","last_change":"2024-09-10T10:04:28.460579+0000","last_active":"2024-09-10T10:04:29.455255+0000","last_peered":"2024-09-10T10:04:29.455255+0000","last_clean":"2024-09-10T10:04:29.455255+0000","last_became_active":"2024-09-10T10:04:28.460359+0000","last_became_peered":"2024-09-10T10:04:28.460359+0000","last_unstale":"2024-09-10T10:04:29.455255+0000","last_undegraded":"2024-09-10T10:04:29.455255+0000","last_fullsized":"2024-09-10T10:04:29.455255+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:56.678975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452913+0000","last_change":"2024-09-10T10:04:29.466959+0000","last_active":"2024-09-10T10:04:31.452913+0000","last_peered":"2024-09-10T10:04:31.452913+0000","last_clean":"2024-09-10T10:04:31.452913+0000","last_became_active":"2024-09-10T10:04:29.464658+0000","last_became_peered":"2024-09-10T10:04:29.464658+0000","last_unstale":"2024-09-10T10:04:31.452913+0000","last_undegraded":"2024-09-10T10:04:31.452913+0000","last_fullsized":"2024-09-10T10:04:31.452913+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:32.680744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452901+0000","last_change":"2024-09-10T10:04:28.465320+0000","last_active":"2024-09-10T10:04:31.452901+0000","last_peered":"2024-09-10T10:04:31.452901+0000","last_clean":"2024-09-10T10:04:31.452901+0000","last_became_active":"2024-09-10T10:04:28.465003+0000","last_became_peered":"2024-09-10T10:04:28.465003+0000","last_unstale":"2024-09-10T10:04:31.452901+0000","last_undegraded":"2024-09-10T10:04:31.452901+0000","last_fullsized":"2024-09-10T10:04:31.452901+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:59.199971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.464153+0000","last_change":"2024-09-10T10:04:29.461078+0000","last_active":"2024-09-10T10:04:29.464153+0000","last_peered":"2024-09-10T10:04:29.464153+0000","last_clean":"2024-09-10T10:04:29.464153+0000","last_became_active":"2024-09-10T10:04:29.460813+0000","last_became_peered":"2024-09-10T10:04:29.460813+0000","last_unstale":"2024-09-10T10:04:29.464153+0000","last_undegraded":"2024-09-10T10:04:29.464153+0000","last_fullsized":"2024-09-10T10:04:29.464153+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:41.717402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"129'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.481178+0000","last_change":"2024-09-10T10:04:28.459446+0000","last_active":"2024-09-10T10:04:29.481178+0000","last_peered":"2024-09-10T10:04:29.481178+0000","last_clean":"2024-09-10T10:04:29.481178+0000","last_became_active":"2024-09-10T10:04:28.458699+0000","last_became_peered":"2024-09-10T10:04:28.458699+0000","last_unstale":"2024-09-10T10:04:29.481178+0000","last_undegraded":"2024-09-10T10:04:29.481178+0000","last_fullsized":"2024-09-10T10:04:29.481178+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:44.361820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465121+0000","last_change":"2024-09-10T10:04:29.460735+0000","last_active":"2024-09-10T10:04:30.465121+0000","last_peered":"2024-09-10T10:04:30.465121+0000","last_clean":"2024-09-10T10:04:30.465121+0000","last_became_active":"2024-09-10T10:04:29.460423+0000","last_became_peered":"2024-09-10T10:04:29.460423+0000","last_unstale":"2024-09-10T10:04:30.465121+0000","last_undegraded":"2024-09-10T10:04:30.465121+0000","last_fullsized":"2024-09-10T10:04:30.465121+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:58.129704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465606+0000","last_change":"2024-09-10T10:04:28.459138+0000","last_active":"2024-09-10T10:04:30.465606+0000","last_peered":"2024-09-10T10:04:30.465606+0000","last_clean":"2024-09-10T10:04:30.465606+0000","last_became_active":"2024-09-10T10:04:28.458713+0000","last_became_peered":"2024-09-10T10:04:28.458713+0000","last_unstale":"2024-09-10T10:04:30.465606+0000","last_undegraded":"2024-09-10T10:04:30.465606+0000","last_fullsized":"2024-09-10T10:04:30.465606+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:12.589348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.475156+0000","last_change":"2024-09-10T10:04:29.468305+0000","last_active":"2024-09-10T10:04:29.475156+0000","last_peered":"2024-09-10T10:04:29.475156+0000","last_clean":"2024-09-10T10:04:29.475156+0000","last_became_active":"2024-09-10T10:04:29.468043+0000","last_became_peered":"2024-09-10T10:04:29.468043+0000","last_unstale":"2024-09-10T10:04:29.475156+0000","last_undegraded":"2024-09-10T10:04:29.475156+0000","last_fullsized":"2024-09-10T10:04:29.475156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:18.611645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465248+0000","last_change":"2024-09-10T10:04:28.458371+0000","last_active":"2024-09-10T10:04:30.465248+0000","last_peered":"2024-09-10T10:04:30.465248+0000","last_clean":"2024-09-10T10:04:30.465248+0000","last_became_active":"2024-09-10T10:04:28.457616+0000","last_became_peered":"2024-09-10T10:04:28.457616+0000","last_unstale":"2024-09-10T10:04:30.465248+0000","last_undegraded":"2024-09-10T10:04:30.465248+0000","last_fullsized":"2024-09-10T10:04:30.465248+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:20.125762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473465+0000","last_change":"2024-09-10T10:04:29.465531+0000","last_active":"2024-09-10T10:04:29.473465+0000","last_peered":"2024-09-10T10:04:29.473465+0000","last_clean":"2024-09-10T10:04:29.473465+0000","last_became_active":"2024-09-10T10:04:29.465274+0000","last_became_peered":"2024-09-10T10:04:29.465274+0000","last_unstale":"2024-09-10T10:04:29.473465+0000","last_undegraded":"2024-09-10T10:04:29.473465+0000","last_fullsized":"2024-09-10T10:04:29.473465+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:44:36.800356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458792+0000","last_change":"2024-09-10T10:04:28.458728+0000","last_active":"2024-09-10T10:04:30.458792+0000","last_peered":"2024-09-10T10:04:30.458792+0000","last_clean":"2024-09-10T10:04:30.458792+0000","last_became_active":"2024-09-10T10:04:28.458451+0000","last_became_peered":"2024-09-10T10:04:28.458451+0000","last_unstale":"2024-09-10T10:04:30.458792+0000","last_undegraded":"2024-09-10T10:04:30.458792+0000","last_fullsized":"2024-09-10T10:04:30.458792+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:33.336423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464943+0000","last_change":"2024-09-10T10:04:29.451343+0000","last_active":"2024-09-10T10:04:30.464943+0000","last_peered":"2024-09-10T10:04:30.464943+0000","last_clean":"2024-09-10T10:04:30.464943+0000","last_became_active":"2024-09-10T10:04:29.451063+0000","last_became_peered":"2024-09-10T10:04:29.451063+0000","last_unstale":"2024-09-10T10:04:30.464943+0000","last_undegraded":"2024-09-10T10:04:30.464943+0000","last_fullsized":"2024-09-10T10:04:30.464943+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:42.701415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"129'8","reported_seq":20,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.478868+0000","last_change":"2024-09-10T10:04:28.454458+0000","last_active":"2024-09-10T10:04:29.478868+0000","last_peered":"2024-09-10T10:04:29.478868+0000","last_clean":"2024-09-10T10:04:29.478868+0000","last_became_active":"2024-09-10T10:04:28.454225+0000","last_became_peered":"2024-09-10T10:04:28.454225+0000","last_unstale":"2024-09-10T10:04:29.478868+0000","last_undegraded":"2024-09-10T10:04:29.478868+0000","last_fullsized":"2024-09-10T10:04:29.478868+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:48.509347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452788+0000","last_change":"2024-09-10T10:04:29.468431+0000","last_active":"2024-09-10T10:04:31.452788+0000","last_peered":"2024-09-10T10:04:31.452788+0000","last_clean":"2024-09-10T10:04:31.452788+0000","last_became_active":"2024-09-10T10:04:29.468146+0000","last_became_peered":"2024-09-10T10:04:29.468146+0000","last_unstale":"2024-09-10T10:04:31.452788+0000","last_undegraded":"2024-09-10T10:04:31.452788+0000","last_fullsized":"2024-09-10T10:04:31.452788+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:13.697427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.474961+0000","last_change":"2024-09-10T10:04:28.455053+0000","last_active":"2024-09-10T10:04:29.474961+0000","last_peered":"2024-09-10T10:04:29.474961+0000","last_clean":"2024-09-10T10:04:29.474961+0000","last_became_active":"2024-09-10T10:04:28.454671+0000","last_became_peered":"2024-09-10T10:04:28.454671+0000","last_unstale":"2024-09-10T10:04:29.474961+0000","last_undegraded":"2024-09-10T10:04:29.474961+0000","last_fullsized":"2024-09-10T10:04:29.474961+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:41.554859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476666+0000","last_change":"2024-09-10T10:04:29.467823+0000","last_active":"2024-09-10T10:04:29.476666+0000","last_peered":"2024-09-10T10:04:29.476666+0000","last_clean":"2024-09-10T10:04:29.476666+0000","last_became_active":"2024-09-10T10:04:29.467550+0000","last_became_peered":"2024-09-10T10:04:29.467550+0000","last_unstale":"2024-09-10T10:04:29.476666+0000","last_undegraded":"2024-09-10T10:04:29.476666+0000","last_fullsized":"2024-09-10T10:04:29.476666+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:25.245567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.475400+0000","last_change":"2024-09-10T10:04:28.455763+0000","last_active":"2024-09-10T10:04:29.475400+0000","last_peered":"2024-09-10T10:04:29.475400+0000","last_clean":"2024-09-10T10:04:29.475400+0000","last_became_active":"2024-09-10T10:04:28.455463+0000","last_became_peered":"2024-09-10T10:04:28.455463+0000","last_unstale":"2024-09-10T10:04:29.475400+0000","last_undegraded":"2024-09-10T10:04:29.475400+0000","last_fullsized":"2024-09-10T10:04:29.475400+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:56.020885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466574+0000","last_change":"2024-09-10T10:04:29.461820+0000","last_active":"2024-09-10T10:04:30.466574+0000","last_peered":"2024-09-10T10:04:30.466574+0000","last_clean":"2024-09-10T10:04:30.466574+0000","last_became_active":"2024-09-10T10:04:29.461442+0000","last_became_peered":"2024-09-10T10:04:29.461442+0000","last_unstale":"2024-09-10T10:04:30.466574+0000","last_undegraded":"2024-09-10T10:04:30.466574+0000","last_fullsized":"2024-09-10T10:04:30.466574+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:42.907084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.477832+0000","last_change":"2024-09-10T10:04:28.459930+0000","last_active":"2024-09-10T10:04:29.477832+0000","last_peered":"2024-09-10T10:04:29.477832+0000","last_clean":"2024-09-10T10:04:29.477832+0000","last_became_active":"2024-09-10T10:04:28.459654+0000","last_became_peered":"2024-09-10T10:04:28.459654+0000","last_unstale":"2024-09-10T10:04:29.477832+0000","last_undegraded":"2024-09-10T10:04:29.477832+0000","last_fullsized":"2024-09-10T10:04:29.477832+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:03.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459148+0000","last_change":"2024-09-10T10:04:29.453976+0000","last_active":"2024-09-10T10:04:30.459148+0000","last_peered":"2024-09-10T10:04:30.459148+0000","last_clean":"2024-09-10T10:04:30.459148+0000","last_became_active":"2024-09-10T10:04:29.453473+0000","last_became_peered":"2024-09-10T10:04:29.453473+0000","last_unstale":"2024-09-10T10:04:30.459148+0000","last_undegraded":"2024-09-10T10:04:30.459148+0000","last_fullsized":"2024-09-10T10:04:30.459148+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:42:05.539879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.451008+0000","last_change":"2024-09-10T10:04:28.462231+0000","last_active":"2024-09-10T10:04:29.451008+0000","last_peered":"2024-09-10T10:04:29.451008+0000","last_clean":"2024-09-10T10:04:29.451008+0000","last_became_active":"2024-09-10T10:04:28.461842+0000","last_became_peered":"2024-09-10T10:04:28.461842+0000","last_unstale":"2024-09-10T10:04:29.451008+0000","last_undegraded":"2024-09-10T10:04:29.451008+0000","last_fullsized":"2024-09-10T10:04:29.451008+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:25.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465550+0000","last_change":"2024-09-10T10:04:29.457948+0000","last_active":"2024-09-10T10:04:30.465550+0000","last_peered":"2024-09-10T10:04:30.465550+0000","last_clean":"2024-09-10T10:04:30.465550+0000","last_became_active":"2024-09-10T10:04:29.457596+0000","last_became_peered":"2024-09-10T10:04:29.457596+0000","last_unstale":"2024-09-10T10:04:30.465550+0000","last_undegraded":"2024-09-10T10:04:30.465550+0000","last_fullsized":"2024-09-10T10:04:30.465550+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:24.790019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476378+0000","last_change":"2024-09-10T10:04:28.450232+0000","last_active":"2024-09-10T10:04:29.476378+0000","last_peered":"2024-09-10T10:04:29.476378+0000","last_clean":"2024-09-10T10:04:29.476378+0000","last_became_active":"2024-09-10T10:04:28.449951+0000","last_became_peered":"2024-09-10T10:04:28.449951+0000","last_unstale":"2024-09-10T10:04:29.476378+0000","last_undegraded":"2024-09-10T10:04:29.476378+0000","last_fullsized":"2024-09-10T10:04:29.476378+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:56.442466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.465927+0000","last_change":"2024-09-10T10:04:29.460170+0000","last_active":"2024-09-10T10:04:29.465927+0000","last_peered":"2024-09-10T10:04:29.465927+0000","last_clean":"2024-09-10T10:04:29.465927+0000","last_became_active":"2024-09-10T10:04:29.459901+0000","last_became_peered":"2024-09-10T10:04:29.459901+0000","last_unstale":"2024-09-10T10:04:29.465927+0000","last_undegraded":"2024-09-10T10:04:29.465927+0000","last_fullsized":"2024-09-10T10:04:29.465927+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:56.576350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452481+0000","last_change":"2024-09-10T10:04:28.466991+0000","last_active":"2024-09-10T10:04:31.452481+0000","last_peered":"2024-09-10T10:04:31.452481+0000","last_clean":"2024-09-10T10:04:31.452481+0000","last_became_active":"2024-09-10T10:04:28.466642+0000","last_became_peered":"2024-09-10T10:04:28.466642+0000","last_unstale":"2024-09-10T10:04:31.452481+0000","last_undegraded":"2024-09-10T10:04:31.452481+0000","last_fullsized":"2024-09-10T10:04:31.452481+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:43.970259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465247+0000","last_change":"2024-09-10T10:04:29.458511+0000","last_active":"2024-09-10T10:04:30.465247+0000","last_peered":"2024-09-10T10:04:30.465247+0000","last_clean":"2024-09-10T10:04:30.465247+0000","last_became_active":"2024-09-10T10:04:29.458224+0000","last_became_peered":"2024-09-10T10:04:29.458224+0000","last_unstale":"2024-09-10T10:04:30.465247+0000","last_undegraded":"2024-09-10T10:04:30.465247+0000","last_fullsized":"2024-09-10T10:04:30.465247+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:42.701415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452251+0000","last_change":"2024-09-10T10:04:28.466001+0000","last_active":"2024-09-10T10:04:31.452251+0000","last_peered":"2024-09-10T10:04:31.452251+0000","last_clean":"2024-09-10T10:04:31.452251+0000","last_became_active":"2024-09-10T10:04:28.465770+0000","last_became_peered":"2024-09-10T10:04:28.465770+0000","last_unstale":"2024-09-10T10:04:31.452251+0000","last_undegraded":"2024-09-10T10:04:31.452251+0000","last_fullsized":"2024-09-10T10:04:31.452251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:43.483139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.466956+0000","last_change":"2024-09-10T10:04:29.461448+0000","last_active":"2024-09-10T10:04:29.466956+0000","last_peered":"2024-09-10T10:04:29.466956+0000","last_clean":"2024-09-10T10:04:29.466956+0000","last_became_active":"2024-09-10T10:04:29.461205+0000","last_became_peered":"2024-09-10T10:04:29.461205+0000","last_unstale":"2024-09-10T10:04:29.466956+0000","last_undegraded":"2024-09-10T10:04:29.466956+0000","last_fullsized":"2024-09-10T10:04:29.466956+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:57.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464735+0000","last_change":"2024-09-10T10:04:28.450080+0000","last_active":"2024-09-10T10:04:30.464735+0000","last_peered":"2024-09-10T10:04:30.464735+0000","last_clean":"2024-09-10T10:04:30.464735+0000","last_became_active":"2024-09-10T10:04:28.449798+0000","last_became_peered":"2024-09-10T10:04:28.449798+0000","last_unstale":"2024-09-10T10:04:30.464735+0000","last_undegraded":"2024-09-10T10:04:30.464735+0000","last_fullsized":"2024-09-10T10:04:30.464735+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:02.072805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473315+0000","last_change":"2024-09-10T10:04:29.467753+0000","last_active":"2024-09-10T10:04:29.473315+0000","last_peered":"2024-09-10T10:04:29.473315+0000","last_clean":"2024-09-10T10:04:29.473315+0000","last_became_active":"2024-09-10T10:04:29.467494+0000","last_became_peered":"2024-09-10T10:04:29.467494+0000","last_unstale":"2024-09-10T10:04:29.473315+0000","last_undegraded":"2024-09-10T10:04:29.473315+0000","last_fullsized":"2024-09-10T10:04:29.473315+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:42.419512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.453124+0000","last_change":"2024-09-10T10:04:28.465825+0000","last_active":"2024-09-10T10:04:31.453124+0000","last_peered":"2024-09-10T10:04:31.453124+0000","last_clean":"2024-09-10T10:04:31.453124+0000","last_became_active":"2024-09-10T10:04:28.465065+0000","last_became_peered":"2024-09-10T10:04:28.465065+0000","last_unstale":"2024-09-10T10:04:31.453124+0000","last_undegraded":"2024-09-10T10:04:31.453124+0000","last_fullsized":"2024-09-10T10:04:31.453124+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:57.361291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466522+0000","last_change":"2024-09-10T10:04:29.461069+0000","last_active":"2024-09-10T10:04:30.466522+0000","last_peered":"2024-09-10T10:04:30.466522+0000","last_clean":"2024-09-10T10:04:30.466522+0000","last_became_active":"2024-09-10T10:04:29.460771+0000","last_became_peered":"2024-09-10T10:04:29.460771+0000","last_unstale":"2024-09-10T10:04:30.466522+0000","last_undegraded":"2024-09-10T10:04:30.466522+0000","last_fullsized":"2024-09-10T10:04:30.466522+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:12.119083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"130'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.450757+0000","last_change":"2024-09-10T10:04:28.460104+0000","last_active":"2024-09-10T10:04:31.450757+0000","last_peered":"2024-09-10T10:04:31.450757+0000","last_clean":"2024-09-10T10:04:31.450757+0000","last_became_active":"2024-09-10T10:04:28.459863+0000","last_became_peered":"2024-09-10T10:04:28.459863+0000","last_unstale":"2024-09-10T10:04:31.450757+0000","last_undegraded":"2024-09-10T10:04:31.450757+0000","last_fullsized":"2024-09-10T10:04:31.450757+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:44:35.797722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476981+0000","last_change":"2024-09-10T10:04:29.472460+0000","last_active":"2024-09-10T10:04:29.476981+0000","last_peered":"2024-09-10T10:04:29.476981+0000","last_clean":"2024-09-10T10:04:29.476981+0000","last_became_active":"2024-09-10T10:04:29.472172+0000","last_became_peered":"2024-09-10T10:04:29.472172+0000","last_unstale":"2024-09-10T10:04:29.476981+0000","last_undegraded":"2024-09-10T10:04:29.476981+0000","last_fullsized":"2024-09-10T10:04:29.476981+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:52.826481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459493+0000","last_change":"2024-09-10T10:04:28.457883+0000","last_active":"2024-09-10T10:04:30.459493+0000","last_peered":"2024-09-10T10:04:30.459493+0000","last_clean":"2024-09-10T10:04:30.459493+0000","last_became_active":"2024-09-10T10:04:28.457646+0000","last_became_peered":"2024-09-10T10:04:28.457646+0000","last_unstale":"2024-09-10T10:04:30.459493+0000","last_undegraded":"2024-09-10T10:04:30.459493+0000","last_fullsized":"2024-09-10T10:04:30.459493+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:40.328391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459277+0000","last_change":"2024-09-10T10:04:29.457580+0000","last_active":"2024-09-10T10:04:30.459277+0000","last_peered":"2024-09-10T10:04:30.459277+0000","last_clean":"2024-09-10T10:04:30.459277+0000","last_became_active":"2024-09-10T10:04:29.457102+0000","last_became_peered":"2024-09-10T10:04:29.457102+0000","last_unstale":"2024-09-10T10:04:30.459277+0000","last_undegraded":"2024-09-10T10:04:30.459277+0000","last_fullsized":"2024-09-10T10:04:30.459277+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:43.659824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459020+0000","last_change":"2024-09-10T10:04:28.457002+0000","last_active":"2024-09-10T10:04:30.459020+0000","last_peered":"2024-09-10T10:04:30.459020+0000","last_clean":"2024-09-10T10:04:30.459020+0000","last_became_active":"2024-09-10T10:04:28.456756+0000","last_became_peered":"2024-09-10T10:04:28.456756+0000","last_unstale":"2024-09-10T10:04:30.459020+0000","last_undegraded":"2024-09-10T10:04:30.459020+0000","last_fullsized":"2024-09-10T10:04:30.459020+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:22.255332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.462515+0000","last_change":"2024-09-10T10:04:29.459026+0000","last_active":"2024-09-10T10:04:29.462515+0000","last_peered":"2024-09-10T10:04:29.462515+0000","last_clean":"2024-09-10T10:04:29.462515+0000","last_became_active":"2024-09-10T10:04:29.458760+0000","last_became_peered":"2024-09-10T10:04:29.458760+0000","last_unstale":"2024-09-10T10:04:29.462515+0000","last_undegraded":"2024-09-10T10:04:29.462515+0000","last_fullsized":"2024-09-10T10:04:29.462515+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:35.832409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.449719+0000","last_change":"2024-09-10T10:04:28.449910+0000","last_active":"2024-09-10T10:04:29.449719+0000","last_peered":"2024-09-10T10:04:29.449719+0000","last_clean":"2024-09-10T10:04:29.449719+0000","last_became_active":"2024-09-10T10:04:28.449106+0000","last_became_peered":"2024-09-10T10:04:28.449106+0000","last_unstale":"2024-09-10T10:04:29.449719+0000","last_undegraded":"2024-09-10T10:04:29.449719+0000","last_fullsized":"2024-09-10T10:04:29.449719+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:57.361291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.459494+0000","last_change":"2024-09-10T10:04:29.455183+0000","last_active":"2024-09-10T10:04:29.459494+0000","last_peered":"2024-09-10T10:04:29.459494+0000","last_clean":"2024-09-10T10:04:29.459494+0000","last_became_active":"2024-09-10T10:04:29.454786+0000","last_became_peered":"2024-09-10T10:04:29.454786+0000","last_unstale":"2024-09-10T10:04:29.459494+0000","last_undegraded":"2024-09-10T10:04:29.459494+0000","last_fullsized":"2024-09-10T10:04:29.459494+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:31.588098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452471+0000","last_change":"2024-09-10T10:04:28.465790+0000","last_active":"2024-09-10T10:04:31.452471+0000","last_peered":"2024-09-10T10:04:31.452471+0000","last_clean":"2024-09-10T10:04:31.452471+0000","last_became_active":"2024-09-10T10:04:28.465039+0000","last_became_peered":"2024-09-10T10:04:28.465039+0000","last_unstale":"2024-09-10T10:04:31.452471+0000","last_undegraded":"2024-09-10T10:04:31.452471+0000","last_fullsized":"2024-09-10T10:04:31.452471+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:21.797140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452788+0000","last_change":"2024-09-10T10:04:29.463877+0000","last_active":"2024-09-10T10:04:31.452788+0000","last_peered":"2024-09-10T10:04:31.452788+0000","last_clean":"2024-09-10T10:04:31.452788+0000","last_became_active":"2024-09-10T10:04:29.463366+0000","last_became_peered":"2024-09-10T10:04:29.463366+0000","last_unstale":"2024-09-10T10:04:31.452788+0000","last_undegraded":"2024-09-10T10:04:31.452788+0000","last_fullsized":"2024-09-10T10:04:31.452788+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:39.047386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465055+0000","last_change":"2024-09-10T10:04:28.460668+0000","last_active":"2024-09-10T10:04:30.465055+0000","last_peered":"2024-09-10T10:04:30.465055+0000","last_clean":"2024-09-10T10:04:30.465055+0000","last_became_active":"2024-09-10T10:04:28.460311+0000","last_became_peered":"2024-09-10T10:04:28.460311+0000","last_unstale":"2024-09-10T10:04:30.465055+0000","last_undegraded":"2024-09-10T10:04:30.465055+0000","last_fullsized":"2024-09-10T10:04:30.465055+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:42.657191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.461382+0000","last_change":"2024-09-10T10:04:29.455516+0000","last_active":"2024-09-10T10:04:29.461382+0000","last_peered":"2024-09-10T10:04:29.461382+0000","last_clean":"2024-09-10T10:04:29.461382+0000","last_became_active":"2024-09-10T10:04:29.455017+0000","last_became_peered":"2024-09-10T10:04:29.455017+0000","last_unstale":"2024-09-10T10:04:29.461382+0000","last_undegraded":"2024-09-10T10:04:29.461382+0000","last_fullsized":"2024-09-10T10:04:29.461382+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:43.076635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.450302+0000","last_change":"2024-09-10T10:04:28.462057+0000","last_active":"2024-09-10T10:04:29.450302+0000","last_peered":"2024-09-10T10:04:29.450302+0000","last_clean":"2024-09-10T10:04:29.450302+0000","last_became_active":"2024-09-10T10:04:28.461779+0000","last_became_peered":"2024-09-10T10:04:28.461779+0000","last_unstale":"2024-09-10T10:04:29.450302+0000","last_undegraded":"2024-09-10T10:04:29.450302+0000","last_fullsized":"2024-09-10T10:04:29.450302+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:19.830305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.466337+0000","last_change":"2024-09-10T10:04:29.461184+0000","last_active":"2024-09-10T10:04:29.466337+0000","last_peered":"2024-09-10T10:04:29.466337+0000","last_clean":"2024-09-10T10:04:29.466337+0000","last_became_active":"2024-09-10T10:04:29.460946+0000","last_became_peered":"2024-09-10T10:04:29.460946+0000","last_unstale":"2024-09-10T10:04:29.466337+0000","last_undegraded":"2024-09-10T10:04:29.466337+0000","last_fullsized":"2024-09-10T10:04:29.466337+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:52.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"129'3","reported_seq":17,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465975+0000","last_change":"2024-09-10T10:04:28.454622+0000","last_active":"2024-09-10T10:04:30.465975+0000","last_peered":"2024-09-10T10:04:30.465975+0000","last_clean":"2024-09-10T10:04:30.465975+0000","last_became_active":"2024-09-10T10:04:28.454320+0000","last_became_peered":"2024-09-10T10:04:28.454320+0000","last_unstale":"2024-09-10T10:04:30.465975+0000","last_undegraded":"2024-09-10T10:04:30.465975+0000","last_fullsized":"2024-09-10T10:04:30.465975+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:41.305631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":407,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452887+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:04:31.452887+0000","last_peered":"2024-09-10T10:04:31.452887+0000","last_clean":"2024-09-10T10:04:31.452887+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:04:31.452887+0000","last_undegraded":"2024-09-10T10:04:31.452887+0000","last_fullsized":"2024-09-10T10:04:31.452887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33348,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338492,"statfs":{"total":96636764160,"available":96602615808,"internally_reserved":0,"allocated":2211840,"data_stored":1299584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1634304,"data_stored":715066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509518,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1613824,"data_stored":708586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1622016,"data_stored":714920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542224,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1613824,"data_stored":708654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509519,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1626112,"data_stored":710246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509518,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1626112,"data_stored":715502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509519,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37516,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334324,"statfs":{"total":96636764160,"available":96598347776,"internally_reserved":0,"allocated":2220032,"data_stored":1300578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:04:33.533 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:04:33.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.706+0000 7f00a1400640 1 Processor -- start 2024-09-10T10:04:33.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.706+0000 7f00a1400640 1 -- start start 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f00a1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f009c15c560 0x7f009c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f00a1400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f009c06f010 0x7f009c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f00a1400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f009c06f430 con 0x7f009c155970 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f00a1400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f009c0b4570 con 0x7f009c15c560 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f00a1400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f009c155d50 con 0x7f009c06f010 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f009c15c560 0x7f009c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f009c15c560 0x7f009c06ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52038/0 (socket says 172.21.15.110:52038) 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f009a800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f009c155970 0x7f009c0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55858/0 (socket says 172.21.15.110:55858) 2024-09-10T10:04:33.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099e00640 1 -- 172.21.15.110:0/2786716489 learned_addr learned my addr 172.21.15.110:0/2786716489 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:33.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099e00640 1 -- 172.21.15.110:0/2786716489 >> v2:172.21.15.145:3300/0 conn(0x7f009c06f010 msgr2=0x7f009c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:33.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099e00640 1 --2- 172.21.15.110:0/2786716489 >> v2:172.21.15.145:3300/0 conn(0x7f009c06f010 0x7f009c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099e00640 1 -- 172.21.15.110:0/2786716489 >> v1:172.21.15.110:6789/0 conn(0x7f009c155970 legacy=0x7f009c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099400640 1 -- 172.21.15.110:0/2786716489 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489878370 0 0) 0x7f009c06f430 con 0x7f009c155970 2024-09-10T10:04:33.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099e00640 1 -- 172.21.15.110:0/2786716489 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c0b34d0 con 0x7f009c15c560 2024-09-10T10:04:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099e00640 1 --2- 172.21.15.110:0/2786716489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c15c560 0x7f009c06ead0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f009c0735e0 tx=0x7f008801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e563a4044bced73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099400640 1 -- 172.21.15.110:0/2786716489 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00880aa650 con 0x7f009c15c560 2024-09-10T10:04:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099400640 1 -- 172.21.15.110:0/2786716489 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00880af070 con 0x7f009c15c560 2024-09-10T10:04:33.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.710+0000 7f0099400640 1 -- 172.21.15.110:0/2786716489 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00880aaa10 con 0x7f009c15c560 2024-09-10T10:04:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- 172.21.15.110:0/2786716489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c15c560 msgr2=0x7f009c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 --2- 172.21.15.110:0/2786716489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c15c560 0x7f009c06ead0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f009c0735e0 tx=0x7f008801c4f0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- 172.21.15.110:0/2786716489 shutdown_connections 2024-09-10T10:04:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 --2- 172.21.15.110:0/2786716489 >> v2:172.21.15.145:3300/0 conn(0x7f009c06f010 0x7f009c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 --2- 172.21.15.110:0/2786716489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c15c560 0x7f009c06ead0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- 172.21.15.110:0/2786716489 >> 172.21.15.110:0/2786716489 conn(0x7f009c06bc50 msgr2=0x7f009c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- 172.21.15.110:0/2786716489 shutdown_connections 2024-09-10T10:04:33.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- 172.21.15.110:0/2786716489 wait complete. 2024-09-10T10:04:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 Processor -- start 2024-09-10T10:04:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- start start 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f009c06f010 0x7f009c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c155970 0x7f009c15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f009c15c560 0x7f009c1455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f009c0cf170 con 0x7f009c155970 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f009c0b4050 con 0x7f009c15c560 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f00a1400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f009c0cf2f0 con 0x7f009c06f010 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f0099e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c155970 0x7f009c15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f0099e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c155970 0x7f009c15ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52048/0 (socket says 172.21.15.110:52048) 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f0099e00640 1 -- 172.21.15.110:0/493782048 learned_addr learned my addr 172.21.15.110:0/493782048 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f009a800640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f009c06f010 0x7f009c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.714+0000 7f009b200640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f009c15c560 0x7f009c1455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0099e00640 1 -- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f009c06f010 msgr2=0x7f009c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0099e00640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f009c06f010 0x7f009c15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0099e00640 1 -- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f009c15c560 msgr2=0x7f009c1455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0099e00640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f009c15c560 0x7f009c1455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0099e00640 1 -- 172.21.15.110:0/493782048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c142c70 con 0x7f009c155970 2024-09-10T10:04:33.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f009a800640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f009c06f010 0x7f009c15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:33.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0099e00640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c155970 0x7f009c15ffd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f00880ac040 tx=0x7f008801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0092a00640 1 -- 172.21.15.110:0/493782048 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00880aae60 con 0x7f009c155970 2024-09-10T10:04:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0092a00640 1 -- 172.21.15.110:0/493782048 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00880a6040 con 0x7f009c155970 2024-09-10T10:04:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009c142f30 con 0x7f009c155970 2024-09-10T10:04:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0092a00640 1 -- 172.21.15.110:0/493782048 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00880b3a60 con 0x7f009c155970 2024-09-10T10:04:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009c1434f0 con 0x7f009c155970 2024-09-10T10:04:33.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009c069bb0 con 0x7f009c155970 2024-09-10T10:04:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0092a00640 1 -- 172.21.15.110:0/493782048 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f00880b3c00 con 0x7f009c155970 2024-09-10T10:04:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0092a00640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f007c067e60 0x7f007c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f0092a00640 1 -- 172.21.15.110:0/493782048 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f0088120c80 con 0x7f009c155970 2024-09-10T10:04:33.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.718+0000 7f009a800640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f007c067e60 0x7f007c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:33.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.722+0000 7f009a800640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f007c067e60 0x7f007c06a320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f008c007fd0 tx=0x7f008c007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:33.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.726+0000 7f0092a00640 1 -- 172.21.15.110:0/493782048 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f00880c3060 con 0x7f009c155970 2024-09-10T10:04:33.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.878+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f009c0cf680 con 0x7f007c067e60 2024-09-10T10:04:33.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f0092a00640 1 -- 172.21.15.110:0/493782048 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252453 (secure 0 0 0) 0x7f009c0cf680 con 0x7f007c067e60 2024-09-10T10:04:33.891 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:33.893 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f007c067e60 msgr2=0x7f007c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f007c067e60 0x7f007c06a320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f008c007fd0 tx=0x7f008c007c20 comp rx=0 tx=0).stop 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c155970 msgr2=0x7f009c15ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c155970 0x7f009c15ffd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f00880ac040 tx=0x7f008801ad30 comp rx=0 tx=0).stop 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 shutdown_connections 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f007c067e60 0x7f007c06a320 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f009c15c560 0x7f009c1455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f009c155970 0x7f009c15ffd0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 --2- 172.21.15.110:0/493782048 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f009c06f010 0x7f009c15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.886+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 >> 172.21.15.110:0/493782048 conn(0x7f009c06bc50 msgr2=0x7f009c0b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:33.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.890+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 shutdown_connections 2024-09-10T10:04:33.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:33.890+0000 7f00a1400640 1 -- 172.21.15.110:0/493782048 wait complete. 2024-09-10T10:04:33.912 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-10T10:04:33.723875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271680,"kb_used_data":13836,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754703040,"statfs":{"total":773094113280,"available":772815912960,"internally_reserved":0,"allocated":14168064,"data_stored":6873136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":263835180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001699"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.460051+0000","last_change":"2024-09-10T10:04:29.464188+0000","last_active":"2024-09-10T10:04:30.460051+0000","last_peered":"2024-09-10T10:04:30.460051+0000","last_clean":"2024-09-10T10:04:30.460051+0000","last_became_active":"2024-09-10T10:04:29.463940+0000","last_became_peered":"2024-09-10T10:04:29.463940+0000","last_unstale":"2024-09-10T10:04:30.460051+0000","last_undegraded":"2024-09-10T10:04:30.460051+0000","last_fullsized":"2024-09-10T10:04:30.460051+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:13.591982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.474929+0000","last_change":"2024-09-10T10:04:29.468463+0000","last_active":"2024-09-10T10:04:29.474929+0000","last_peered":"2024-09-10T10:04:29.474929+0000","last_clean":"2024-09-10T10:04:29.474929+0000","last_became_active":"2024-09-10T10:04:29.468204+0000","last_became_peered":"2024-09-10T10:04:29.468204+0000","last_unstale":"2024-09-10T10:04:29.474929+0000","last_undegraded":"2024-09-10T10:04:29.474929+0000","last_fullsized":"2024-09-10T10:04:29.474929+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:56.046353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465882+0000","last_change":"2024-09-10T10:04:29.467526+0000","last_active":"2024-09-10T10:04:30.465882+0000","last_peered":"2024-09-10T10:04:30.465882+0000","last_clean":"2024-09-10T10:04:30.465882+0000","last_became_active":"2024-09-10T10:04:29.467107+0000","last_became_peered":"2024-09-10T10:04:29.467107+0000","last_unstale":"2024-09-10T10:04:30.465882+0000","last_undegraded":"2024-09-10T10:04:30.465882+0000","last_fullsized":"2024-09-10T10:04:30.465882+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:24.790019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466346+0000","last_change":"2024-09-10T10:04:29.457932+0000","last_active":"2024-09-10T10:04:30.466346+0000","last_peered":"2024-09-10T10:04:30.466346+0000","last_clean":"2024-09-10T10:04:30.466346+0000","last_became_active":"2024-09-10T10:04:29.457490+0000","last_became_peered":"2024-09-10T10:04:29.457490+0000","last_unstale":"2024-09-10T10:04:30.466346+0000","last_undegraded":"2024-09-10T10:04:30.466346+0000","last_fullsized":"2024-09-10T10:04:30.466346+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:42.907084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473399+0000","last_change":"2024-09-10T10:04:29.466898+0000","last_active":"2024-09-10T10:04:29.473399+0000","last_peered":"2024-09-10T10:04:29.473399+0000","last_clean":"2024-09-10T10:04:29.473399+0000","last_became_active":"2024-09-10T10:04:29.466404+0000","last_became_peered":"2024-09-10T10:04:29.466404+0000","last_unstale":"2024-09-10T10:04:29.473399+0000","last_undegraded":"2024-09-10T10:04:29.473399+0000","last_fullsized":"2024-09-10T10:04:29.473399+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:10.518696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458924+0000","last_change":"2024-09-10T10:04:29.452539+0000","last_active":"2024-09-10T10:04:30.458924+0000","last_peered":"2024-09-10T10:04:30.458924+0000","last_clean":"2024-09-10T10:04:30.458924+0000","last_became_active":"2024-09-10T10:04:29.452281+0000","last_became_peered":"2024-09-10T10:04:29.452281+0000","last_unstale":"2024-09-10T10:04:30.458924+0000","last_undegraded":"2024-09-10T10:04:30.458924+0000","last_fullsized":"2024-09-10T10:04:30.458924+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:44:08.724445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.466631+0000","last_change":"2024-09-10T10:04:29.461438+0000","last_active":"2024-09-10T10:04:29.466631+0000","last_peered":"2024-09-10T10:04:29.466631+0000","last_clean":"2024-09-10T10:04:29.466631+0000","last_became_active":"2024-09-10T10:04:29.460885+0000","last_became_peered":"2024-09-10T10:04:29.460885+0000","last_unstale":"2024-09-10T10:04:29.466631+0000","last_undegraded":"2024-09-10T10:04:29.466631+0000","last_fullsized":"2024-09-10T10:04:29.466631+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:56.969006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458785+0000","last_change":"2024-09-10T10:04:29.459320+0000","last_active":"2024-09-10T10:04:30.458785+0000","last_peered":"2024-09-10T10:04:30.458785+0000","last_clean":"2024-09-10T10:04:30.458785+0000","last_became_active":"2024-09-10T10:04:29.459047+0000","last_became_peered":"2024-09-10T10:04:29.459047+0000","last_unstale":"2024-09-10T10:04:30.458785+0000","last_undegraded":"2024-09-10T10:04:30.458785+0000","last_fullsized":"2024-09-10T10:04:30.458785+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:31.277449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473145+0000","last_change":"2024-09-10T10:04:29.464993+0000","last_active":"2024-09-10T10:04:29.473145+0000","last_peered":"2024-09-10T10:04:29.473145+0000","last_clean":"2024-09-10T10:04:29.473145+0000","last_became_active":"2024-09-10T10:04:29.464723+0000","last_became_peered":"2024-09-10T10:04:29.464723+0000","last_unstale":"2024-09-10T10:04:29.473145+0000","last_undegraded":"2024-09-10T10:04:29.473145+0000","last_fullsized":"2024-09-10T10:04:29.473145+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:07.866522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452590+0000","last_change":"2024-09-10T10:04:29.464700+0000","last_active":"2024-09-10T10:04:31.452590+0000","last_peered":"2024-09-10T10:04:31.452590+0000","last_clean":"2024-09-10T10:04:31.452590+0000","last_became_active":"2024-09-10T10:04:29.464117+0000","last_became_peered":"2024-09-10T10:04:29.464117+0000","last_unstale":"2024-09-10T10:04:31.452590+0000","last_undegraded":"2024-09-10T10:04:31.452590+0000","last_fullsized":"2024-09-10T10:04:31.452590+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:57.538761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459010+0000","last_change":"2024-09-10T10:04:29.462235+0000","last_active":"2024-09-10T10:04:30.459010+0000","last_peered":"2024-09-10T10:04:30.459010+0000","last_clean":"2024-09-10T10:04:30.459010+0000","last_became_active":"2024-09-10T10:04:29.461957+0000","last_became_peered":"2024-09-10T10:04:29.461957+0000","last_unstale":"2024-09-10T10:04:30.459010+0000","last_undegraded":"2024-09-10T10:04:30.459010+0000","last_fullsized":"2024-09-10T10:04:30.459010+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:53.589530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452465+0000","last_change":"2024-09-10T10:04:29.466716+0000","last_active":"2024-09-10T10:04:31.452465+0000","last_peered":"2024-09-10T10:04:31.452465+0000","last_clean":"2024-09-10T10:04:31.452465+0000","last_became_active":"2024-09-10T10:04:29.466412+0000","last_became_peered":"2024-09-10T10:04:29.466412+0000","last_unstale":"2024-09-10T10:04:31.452465+0000","last_undegraded":"2024-09-10T10:04:31.452465+0000","last_fullsized":"2024-09-10T10:04:31.452465+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:31.742985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473376+0000","last_change":"2024-09-10T10:04:29.466689+0000","last_active":"2024-09-10T10:04:29.473376+0000","last_peered":"2024-09-10T10:04:29.473376+0000","last_clean":"2024-09-10T10:04:29.473376+0000","last_became_active":"2024-09-10T10:04:29.466052+0000","last_became_peered":"2024-09-10T10:04:29.466052+0000","last_unstale":"2024-09-10T10:04:29.473376+0000","last_undegraded":"2024-09-10T10:04:29.473376+0000","last_fullsized":"2024-09-10T10:04:29.473376+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:23.257966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464905+0000","last_change":"2024-09-10T10:04:29.469312+0000","last_active":"2024-09-10T10:04:30.464905+0000","last_peered":"2024-09-10T10:04:30.464905+0000","last_clean":"2024-09-10T10:04:30.464905+0000","last_became_active":"2024-09-10T10:04:29.469070+0000","last_became_peered":"2024-09-10T10:04:29.469070+0000","last_unstale":"2024-09-10T10:04:30.464905+0000","last_undegraded":"2024-09-10T10:04:30.464905+0000","last_fullsized":"2024-09-10T10:04:30.464905+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:49.759441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452806+0000","last_change":"2024-09-10T10:04:29.464373+0000","last_active":"2024-09-10T10:04:31.452806+0000","last_peered":"2024-09-10T10:04:31.452806+0000","last_clean":"2024-09-10T10:04:31.452806+0000","last_became_active":"2024-09-10T10:04:29.464066+0000","last_became_peered":"2024-09-10T10:04:29.464066+0000","last_unstale":"2024-09-10T10:04:31.452806+0000","last_undegraded":"2024-09-10T10:04:31.452806+0000","last_fullsized":"2024-09-10T10:04:31.452806+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:55.535508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473392+0000","last_change":"2024-09-10T10:04:29.461774+0000","last_active":"2024-09-10T10:04:29.473392+0000","last_peered":"2024-09-10T10:04:29.473392+0000","last_clean":"2024-09-10T10:04:29.473392+0000","last_became_active":"2024-09-10T10:04:29.461467+0000","last_became_peered":"2024-09-10T10:04:29.461467+0000","last_unstale":"2024-09-10T10:04:29.473392+0000","last_undegraded":"2024-09-10T10:04:29.473392+0000","last_fullsized":"2024-09-10T10:04:29.473392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:26.897928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.453118+0000","last_change":"2024-09-10T10:04:29.467282+0000","last_active":"2024-09-10T10:04:31.453118+0000","last_peered":"2024-09-10T10:04:31.453118+0000","last_clean":"2024-09-10T10:04:31.453118+0000","last_became_active":"2024-09-10T10:04:29.465209+0000","last_became_peered":"2024-09-10T10:04:29.465209+0000","last_unstale":"2024-09-10T10:04:31.453118+0000","last_undegraded":"2024-09-10T10:04:31.453118+0000","last_fullsized":"2024-09-10T10:04:31.453118+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:54.187069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465621+0000","last_change":"2024-09-10T10:04:29.466188+0000","last_active":"2024-09-10T10:04:30.465621+0000","last_peered":"2024-09-10T10:04:30.465621+0000","last_clean":"2024-09-10T10:04:30.465621+0000","last_became_active":"2024-09-10T10:04:29.465828+0000","last_became_peered":"2024-09-10T10:04:29.465828+0000","last_unstale":"2024-09-10T10:04:30.465621+0000","last_undegraded":"2024-09-10T10:04:30.465621+0000","last_fullsized":"2024-09-10T10:04:30.465621+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:35.992553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459406+0000","last_change":"2024-09-10T10:04:29.457367+0000","last_active":"2024-09-10T10:04:30.459406+0000","last_peered":"2024-09-10T10:04:30.459406+0000","last_clean":"2024-09-10T10:04:30.459406+0000","last_became_active":"2024-09-10T10:04:29.456630+0000","last_became_peered":"2024-09-10T10:04:29.456630+0000","last_unstale":"2024-09-10T10:04:30.459406+0000","last_undegraded":"2024-09-10T10:04:30.459406+0000","last_fullsized":"2024-09-10T10:04:30.459406+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:03.075439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.466019+0000","last_change":"2024-09-10T10:04:29.464635+0000","last_active":"2024-09-10T10:04:29.466019+0000","last_peered":"2024-09-10T10:04:29.466019+0000","last_clean":"2024-09-10T10:04:29.466019+0000","last_became_active":"2024-09-10T10:04:29.464376+0000","last_became_peered":"2024-09-10T10:04:29.464376+0000","last_unstale":"2024-09-10T10:04:29.466019+0000","last_undegraded":"2024-09-10T10:04:29.466019+0000","last_fullsized":"2024-09-10T10:04:29.466019+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:43.362603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452838+0000","last_change":"2024-09-10T10:04:29.466456+0000","last_active":"2024-09-10T10:04:31.452838+0000","last_peered":"2024-09-10T10:04:31.452838+0000","last_clean":"2024-09-10T10:04:31.452838+0000","last_became_active":"2024-09-10T10:04:29.464397+0000","last_became_peered":"2024-09-10T10:04:29.464397+0000","last_unstale":"2024-09-10T10:04:31.452838+0000","last_undegraded":"2024-09-10T10:04:31.452838+0000","last_fullsized":"2024-09-10T10:04:31.452838+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:51.427699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465256+0000","last_change":"2024-09-10T10:04:29.468290+0000","last_active":"2024-09-10T10:04:30.465256+0000","last_peered":"2024-09-10T10:04:30.465256+0000","last_clean":"2024-09-10T10:04:30.465256+0000","last_became_active":"2024-09-10T10:04:29.467960+0000","last_became_peered":"2024-09-10T10:04:29.467960+0000","last_unstale":"2024-09-10T10:04:30.465256+0000","last_undegraded":"2024-09-10T10:04:30.465256+0000","last_fullsized":"2024-09-10T10:04:30.465256+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:58.129704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464911+0000","last_change":"2024-09-10T10:04:29.459892+0000","last_active":"2024-09-10T10:04:30.464911+0000","last_peered":"2024-09-10T10:04:30.464911+0000","last_clean":"2024-09-10T10:04:30.464911+0000","last_became_active":"2024-09-10T10:04:29.459620+0000","last_became_peered":"2024-09-10T10:04:29.459620+0000","last_unstale":"2024-09-10T10:04:30.464911+0000","last_undegraded":"2024-09-10T10:04:30.464911+0000","last_fullsized":"2024-09-10T10:04:30.464911+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:41.164700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452231+0000","last_change":"2024-09-10T10:04:29.463636+0000","last_active":"2024-09-10T10:04:31.452231+0000","last_peered":"2024-09-10T10:04:31.452231+0000","last_clean":"2024-09-10T10:04:31.452231+0000","last_became_active":"2024-09-10T10:04:29.462988+0000","last_became_peered":"2024-09-10T10:04:29.462988+0000","last_unstale":"2024-09-10T10:04:31.452231+0000","last_undegraded":"2024-09-10T10:04:31.452231+0000","last_fullsized":"2024-09-10T10:04:31.452231+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:08.577949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.462549+0000","last_change":"2024-09-10T10:04:29.458577+0000","last_active":"2024-09-10T10:04:29.462549+0000","last_peered":"2024-09-10T10:04:29.462549+0000","last_clean":"2024-09-10T10:04:29.462549+0000","last_became_active":"2024-09-10T10:04:29.458258+0000","last_became_peered":"2024-09-10T10:04:29.458258+0000","last_unstale":"2024-09-10T10:04:29.462549+0000","last_undegraded":"2024-09-10T10:04:29.462549+0000","last_fullsized":"2024-09-10T10:04:29.462549+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:26.201496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459778+0000","last_change":"2024-09-10T10:04:29.454611+0000","last_active":"2024-09-10T10:04:30.459778+0000","last_peered":"2024-09-10T10:04:30.459778+0000","last_clean":"2024-09-10T10:04:30.459778+0000","last_became_active":"2024-09-10T10:04:29.453822+0000","last_became_peered":"2024-09-10T10:04:29.453822+0000","last_unstale":"2024-09-10T10:04:30.459778+0000","last_undegraded":"2024-09-10T10:04:30.459778+0000","last_fullsized":"2024-09-10T10:04:30.459778+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:42.308264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.460992+0000","last_change":"2024-09-10T10:04:29.455448+0000","last_active":"2024-09-10T10:04:29.460992+0000","last_peered":"2024-09-10T10:04:29.460992+0000","last_clean":"2024-09-10T10:04:29.460992+0000","last_became_active":"2024-09-10T10:04:29.455195+0000","last_became_peered":"2024-09-10T10:04:29.455195+0000","last_unstale":"2024-09-10T10:04:29.460992+0000","last_undegraded":"2024-09-10T10:04:29.460992+0000","last_fullsized":"2024-09-10T10:04:29.460992+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:20.918438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.455744+0000","last_change":"2024-09-10T10:04:29.453662+0000","last_active":"2024-09-10T10:04:29.455744+0000","last_peered":"2024-09-10T10:04:29.455744+0000","last_clean":"2024-09-10T10:04:29.455744+0000","last_became_active":"2024-09-10T10:04:29.453392+0000","last_became_peered":"2024-09-10T10:04:29.453392+0000","last_unstale":"2024-09-10T10:04:29.455744+0000","last_undegraded":"2024-09-10T10:04:29.455744+0000","last_fullsized":"2024-09-10T10:04:29.455744+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:50.143383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458865+0000","last_change":"2024-09-10T10:04:29.452708+0000","last_active":"2024-09-10T10:04:30.458865+0000","last_peered":"2024-09-10T10:04:30.458865+0000","last_clean":"2024-09-10T10:04:30.458865+0000","last_became_active":"2024-09-10T10:04:29.452457+0000","last_became_peered":"2024-09-10T10:04:29.452457+0000","last_unstale":"2024-09-10T10:04:30.458865+0000","last_undegraded":"2024-09-10T10:04:30.458865+0000","last_fullsized":"2024-09-10T10:04:30.458865+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:38.657060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465398+0000","last_change":"2024-09-10T10:04:29.458411+0000","last_active":"2024-09-10T10:04:30.465398+0000","last_peered":"2024-09-10T10:04:30.465398+0000","last_clean":"2024-09-10T10:04:30.465398+0000","last_became_active":"2024-09-10T10:04:29.457956+0000","last_became_peered":"2024-09-10T10:04:29.457956+0000","last_unstale":"2024-09-10T10:04:30.465398+0000","last_undegraded":"2024-09-10T10:04:30.465398+0000","last_fullsized":"2024-09-10T10:04:30.465398+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:35.992553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.472676+0000","last_change":"2024-09-10T10:04:29.464846+0000","last_active":"2024-09-10T10:04:29.472676+0000","last_peered":"2024-09-10T10:04:29.472676+0000","last_clean":"2024-09-10T10:04:29.472676+0000","last_became_active":"2024-09-10T10:04:29.463671+0000","last_became_peered":"2024-09-10T10:04:29.463671+0000","last_unstale":"2024-09-10T10:04:29.472676+0000","last_undegraded":"2024-09-10T10:04:29.472676+0000","last_fullsized":"2024-09-10T10:04:29.472676+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:06.887302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476719+0000","last_change":"2024-09-10T10:04:29.470329+0000","last_active":"2024-09-10T10:04:29.476719+0000","last_peered":"2024-09-10T10:04:29.476719+0000","last_clean":"2024-09-10T10:04:29.476719+0000","last_became_active":"2024-09-10T10:04:29.469845+0000","last_became_peered":"2024-09-10T10:04:29.469845+0000","last_unstale":"2024-09-10T10:04:29.476719+0000","last_undegraded":"2024-09-10T10:04:29.476719+0000","last_fullsized":"2024-09-10T10:04:29.476719+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:42.344713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452636+0000","last_change":"2024-09-10T10:04:29.467255+0000","last_active":"2024-09-10T10:04:31.452636+0000","last_peered":"2024-09-10T10:04:31.452636+0000","last_clean":"2024-09-10T10:04:31.452636+0000","last_became_active":"2024-09-10T10:04:29.464420+0000","last_became_peered":"2024-09-10T10:04:29.464420+0000","last_unstale":"2024-09-10T10:04:31.452636+0000","last_undegraded":"2024-09-10T10:04:31.452636+0000","last_fullsized":"2024-09-10T10:04:31.452636+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:23.890969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"129'4","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.453057+0000","last_change":"2024-09-10T10:04:28.466121+0000","last_active":"2024-09-10T10:04:31.453057+0000","last_peered":"2024-09-10T10:04:31.453057+0000","last_clean":"2024-09-10T10:04:31.453057+0000","last_became_active":"2024-09-10T10:04:28.465484+0000","last_became_peered":"2024-09-10T10:04:28.465484+0000","last_unstale":"2024-09-10T10:04:31.453057+0000","last_undegraded":"2024-09-10T10:04:31.453057+0000","last_fullsized":"2024-09-10T10:04:31.453057+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:56.442466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.474882+0000","last_change":"2024-09-10T10:04:29.469003+0000","last_active":"2024-09-10T10:04:29.474882+0000","last_peered":"2024-09-10T10:04:29.474882+0000","last_clean":"2024-09-10T10:04:29.474882+0000","last_became_active":"2024-09-10T10:04:29.468713+0000","last_became_peered":"2024-09-10T10:04:29.468713+0000","last_unstale":"2024-09-10T10:04:29.474882+0000","last_undegraded":"2024-09-10T10:04:29.474882+0000","last_fullsized":"2024-09-10T10:04:29.474882+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:14.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.474489+0000","last_change":"2024-09-10T10:04:28.461952+0000","last_active":"2024-09-10T10:04:29.474489+0000","last_peered":"2024-09-10T10:04:29.474489+0000","last_clean":"2024-09-10T10:04:29.474489+0000","last_became_active":"2024-09-10T10:04:28.461666+0000","last_became_peered":"2024-09-10T10:04:28.461666+0000","last_unstale":"2024-09-10T10:04:29.474489+0000","last_undegraded":"2024-09-10T10:04:29.474489+0000","last_fullsized":"2024-09-10T10:04:29.474489+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:29.591631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458968+0000","last_change":"2024-09-10T10:04:29.458011+0000","last_active":"2024-09-10T10:04:30.458968+0000","last_peered":"2024-09-10T10:04:30.458968+0000","last_clean":"2024-09-10T10:04:30.458968+0000","last_became_active":"2024-09-10T10:04:29.457744+0000","last_became_peered":"2024-09-10T10:04:29.457744+0000","last_unstale":"2024-09-10T10:04:30.458968+0000","last_undegraded":"2024-09-10T10:04:30.458968+0000","last_fullsized":"2024-09-10T10:04:30.458968+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:29.745244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465762+0000","last_change":"2024-09-10T10:04:28.455407+0000","last_active":"2024-09-10T10:04:30.465762+0000","last_peered":"2024-09-10T10:04:30.465762+0000","last_clean":"2024-09-10T10:04:30.465762+0000","last_became_active":"2024-09-10T10:04:28.454290+0000","last_became_peered":"2024-09-10T10:04:28.454290+0000","last_unstale":"2024-09-10T10:04:30.465762+0000","last_undegraded":"2024-09-10T10:04:30.465762+0000","last_fullsized":"2024-09-10T10:04:30.465762+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:42:04.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452432+0000","last_change":"2024-09-10T10:04:29.463729+0000","last_active":"2024-09-10T10:04:31.452432+0000","last_peered":"2024-09-10T10:04:31.452432+0000","last_clean":"2024-09-10T10:04:31.452432+0000","last_became_active":"2024-09-10T10:04:29.463384+0000","last_became_peered":"2024-09-10T10:04:29.463384+0000","last_unstale":"2024-09-10T10:04:31.452432+0000","last_undegraded":"2024-09-10T10:04:31.452432+0000","last_fullsized":"2024-09-10T10:04:31.452432+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:26.765169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466359+0000","last_change":"2024-09-10T10:04:28.454198+0000","last_active":"2024-09-10T10:04:30.466359+0000","last_peered":"2024-09-10T10:04:30.466359+0000","last_clean":"2024-09-10T10:04:30.466359+0000","last_became_active":"2024-09-10T10:04:28.453633+0000","last_became_peered":"2024-09-10T10:04:28.453633+0000","last_unstale":"2024-09-10T10:04:30.466359+0000","last_undegraded":"2024-09-10T10:04:30.466359+0000","last_fullsized":"2024-09-10T10:04:30.466359+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:52.586896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.470364+0000","last_change":"2024-09-10T10:04:29.463463+0000","last_active":"2024-09-10T10:04:29.470364+0000","last_peered":"2024-09-10T10:04:29.470364+0000","last_clean":"2024-09-10T10:04:29.470364+0000","last_became_active":"2024-09-10T10:04:29.463164+0000","last_became_peered":"2024-09-10T10:04:29.463164+0000","last_unstale":"2024-09-10T10:04:29.470364+0000","last_undegraded":"2024-09-10T10:04:29.470364+0000","last_fullsized":"2024-09-10T10:04:29.470364+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:49.834054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476816+0000","last_change":"2024-09-10T10:04:28.461411+0000","last_active":"2024-09-10T10:04:29.476816+0000","last_peered":"2024-09-10T10:04:29.476816+0000","last_clean":"2024-09-10T10:04:29.476816+0000","last_became_active":"2024-09-10T10:04:28.461163+0000","last_became_peered":"2024-09-10T10:04:28.461163+0000","last_unstale":"2024-09-10T10:04:29.476816+0000","last_undegraded":"2024-09-10T10:04:29.476816+0000","last_fullsized":"2024-09-10T10:04:29.476816+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:39.676002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466037+0000","last_change":"2024-09-10T10:04:29.463989+0000","last_active":"2024-09-10T10:04:30.466037+0000","last_peered":"2024-09-10T10:04:30.466037+0000","last_clean":"2024-09-10T10:04:30.466037+0000","last_became_active":"2024-09-10T10:04:29.461843+0000","last_became_peered":"2024-09-10T10:04:29.461843+0000","last_unstale":"2024-09-10T10:04:30.466037+0000","last_undegraded":"2024-09-10T10:04:30.466037+0000","last_fullsized":"2024-09-10T10:04:30.466037+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:12.119083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.475461+0000","last_change":"2024-09-10T10:04:28.462377+0000","last_active":"2024-09-10T10:04:29.475461+0000","last_peered":"2024-09-10T10:04:29.475461+0000","last_clean":"2024-09-10T10:04:29.475461+0000","last_became_active":"2024-09-10T10:04:28.462093+0000","last_became_peered":"2024-09-10T10:04:28.462093+0000","last_unstale":"2024-09-10T10:04:29.475461+0000","last_undegraded":"2024-09-10T10:04:29.475461+0000","last_fullsized":"2024-09-10T10:04:29.475461+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:48.948456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473166+0000","last_change":"2024-09-10T10:04:29.465476+0000","last_active":"2024-09-10T10:04:29.473166+0000","last_peered":"2024-09-10T10:04:29.473166+0000","last_clean":"2024-09-10T10:04:29.473166+0000","last_became_active":"2024-09-10T10:04:29.465214+0000","last_became_peered":"2024-09-10T10:04:29.465214+0000","last_unstale":"2024-09-10T10:04:29.473166+0000","last_undegraded":"2024-09-10T10:04:29.473166+0000","last_fullsized":"2024-09-10T10:04:29.473166+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:00.305047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.448235+0000","last_change":"2024-09-10T10:04:28.457092+0000","last_active":"2024-09-10T10:04:29.448235+0000","last_peered":"2024-09-10T10:04:29.448235+0000","last_clean":"2024-09-10T10:04:29.448235+0000","last_became_active":"2024-09-10T10:04:28.456832+0000","last_became_peered":"2024-09-10T10:04:28.456832+0000","last_unstale":"2024-09-10T10:04:29.448235+0000","last_undegraded":"2024-09-10T10:04:29.448235+0000","last_fullsized":"2024-09-10T10:04:29.448235+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:02.024448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.469957+0000","last_change":"2024-09-10T10:04:29.462420+0000","last_active":"2024-09-10T10:04:29.469957+0000","last_peered":"2024-09-10T10:04:29.469957+0000","last_clean":"2024-09-10T10:04:29.469957+0000","last_became_active":"2024-09-10T10:04:29.459776+0000","last_became_peered":"2024-09-10T10:04:29.459776+0000","last_unstale":"2024-09-10T10:04:29.469957+0000","last_undegraded":"2024-09-10T10:04:29.469957+0000","last_fullsized":"2024-09-10T10:04:29.469957+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:41.331025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464690+0000","last_change":"2024-09-10T10:04:28.461601+0000","last_active":"2024-09-10T10:04:30.464690+0000","last_peered":"2024-09-10T10:04:30.464690+0000","last_clean":"2024-09-10T10:04:30.464690+0000","last_became_active":"2024-09-10T10:04:28.461312+0000","last_became_peered":"2024-09-10T10:04:28.461312+0000","last_unstale":"2024-09-10T10:04:30.464690+0000","last_undegraded":"2024-09-10T10:04:30.464690+0000","last_fullsized":"2024-09-10T10:04:30.464690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:42.657191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.469924+0000","last_change":"2024-09-10T10:04:29.460205+0000","last_active":"2024-09-10T10:04:29.469924+0000","last_peered":"2024-09-10T10:04:29.469924+0000","last_clean":"2024-09-10T10:04:29.469924+0000","last_became_active":"2024-09-10T10:04:29.459879+0000","last_became_peered":"2024-09-10T10:04:29.459879+0000","last_unstale":"2024-09-10T10:04:29.469924+0000","last_undegraded":"2024-09-10T10:04:29.469924+0000","last_fullsized":"2024-09-10T10:04:29.469924+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:34.339057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.455255+0000","last_change":"2024-09-10T10:04:28.460579+0000","last_active":"2024-09-10T10:04:29.455255+0000","last_peered":"2024-09-10T10:04:29.455255+0000","last_clean":"2024-09-10T10:04:29.455255+0000","last_became_active":"2024-09-10T10:04:28.460359+0000","last_became_peered":"2024-09-10T10:04:28.460359+0000","last_unstale":"2024-09-10T10:04:29.455255+0000","last_undegraded":"2024-09-10T10:04:29.455255+0000","last_fullsized":"2024-09-10T10:04:29.455255+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:56.678975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452913+0000","last_change":"2024-09-10T10:04:29.466959+0000","last_active":"2024-09-10T10:04:31.452913+0000","last_peered":"2024-09-10T10:04:31.452913+0000","last_clean":"2024-09-10T10:04:31.452913+0000","last_became_active":"2024-09-10T10:04:29.464658+0000","last_became_peered":"2024-09-10T10:04:29.464658+0000","last_unstale":"2024-09-10T10:04:31.452913+0000","last_undegraded":"2024-09-10T10:04:31.452913+0000","last_fullsized":"2024-09-10T10:04:31.452913+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:32.680744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452901+0000","last_change":"2024-09-10T10:04:28.465320+0000","last_active":"2024-09-10T10:04:31.452901+0000","last_peered":"2024-09-10T10:04:31.452901+0000","last_clean":"2024-09-10T10:04:31.452901+0000","last_became_active":"2024-09-10T10:04:28.465003+0000","last_became_peered":"2024-09-10T10:04:28.465003+0000","last_unstale":"2024-09-10T10:04:31.452901+0000","last_undegraded":"2024-09-10T10:04:31.452901+0000","last_fullsized":"2024-09-10T10:04:31.452901+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:59.199971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.464153+0000","last_change":"2024-09-10T10:04:29.461078+0000","last_active":"2024-09-10T10:04:29.464153+0000","last_peered":"2024-09-10T10:04:29.464153+0000","last_clean":"2024-09-10T10:04:29.464153+0000","last_became_active":"2024-09-10T10:04:29.460813+0000","last_became_peered":"2024-09-10T10:04:29.460813+0000","last_unstale":"2024-09-10T10:04:29.464153+0000","last_undegraded":"2024-09-10T10:04:29.464153+0000","last_fullsized":"2024-09-10T10:04:29.464153+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:41.717402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"129'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.481178+0000","last_change":"2024-09-10T10:04:28.459446+0000","last_active":"2024-09-10T10:04:29.481178+0000","last_peered":"2024-09-10T10:04:29.481178+0000","last_clean":"2024-09-10T10:04:29.481178+0000","last_became_active":"2024-09-10T10:04:28.458699+0000","last_became_peered":"2024-09-10T10:04:28.458699+0000","last_unstale":"2024-09-10T10:04:29.481178+0000","last_undegraded":"2024-09-10T10:04:29.481178+0000","last_fullsized":"2024-09-10T10:04:29.481178+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:44.361820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465121+0000","last_change":"2024-09-10T10:04:29.460735+0000","last_active":"2024-09-10T10:04:30.465121+0000","last_peered":"2024-09-10T10:04:30.465121+0000","last_clean":"2024-09-10T10:04:30.465121+0000","last_became_active":"2024-09-10T10:04:29.460423+0000","last_became_peered":"2024-09-10T10:04:29.460423+0000","last_unstale":"2024-09-10T10:04:30.465121+0000","last_undegraded":"2024-09-10T10:04:30.465121+0000","last_fullsized":"2024-09-10T10:04:30.465121+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:58.129704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465606+0000","last_change":"2024-09-10T10:04:28.459138+0000","last_active":"2024-09-10T10:04:30.465606+0000","last_peered":"2024-09-10T10:04:30.465606+0000","last_clean":"2024-09-10T10:04:30.465606+0000","last_became_active":"2024-09-10T10:04:28.458713+0000","last_became_peered":"2024-09-10T10:04:28.458713+0000","last_unstale":"2024-09-10T10:04:30.465606+0000","last_undegraded":"2024-09-10T10:04:30.465606+0000","last_fullsized":"2024-09-10T10:04:30.465606+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:12.589348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.475156+0000","last_change":"2024-09-10T10:04:29.468305+0000","last_active":"2024-09-10T10:04:29.475156+0000","last_peered":"2024-09-10T10:04:29.475156+0000","last_clean":"2024-09-10T10:04:29.475156+0000","last_became_active":"2024-09-10T10:04:29.468043+0000","last_became_peered":"2024-09-10T10:04:29.468043+0000","last_unstale":"2024-09-10T10:04:29.475156+0000","last_undegraded":"2024-09-10T10:04:29.475156+0000","last_fullsized":"2024-09-10T10:04:29.475156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:18.611645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465248+0000","last_change":"2024-09-10T10:04:28.458371+0000","last_active":"2024-09-10T10:04:30.465248+0000","last_peered":"2024-09-10T10:04:30.465248+0000","last_clean":"2024-09-10T10:04:30.465248+0000","last_became_active":"2024-09-10T10:04:28.457616+0000","last_became_peered":"2024-09-10T10:04:28.457616+0000","last_unstale":"2024-09-10T10:04:30.465248+0000","last_undegraded":"2024-09-10T10:04:30.465248+0000","last_fullsized":"2024-09-10T10:04:30.465248+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:20.125762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473465+0000","last_change":"2024-09-10T10:04:29.465531+0000","last_active":"2024-09-10T10:04:29.473465+0000","last_peered":"2024-09-10T10:04:29.473465+0000","last_clean":"2024-09-10T10:04:29.473465+0000","last_became_active":"2024-09-10T10:04:29.465274+0000","last_became_peered":"2024-09-10T10:04:29.465274+0000","last_unstale":"2024-09-10T10:04:29.473465+0000","last_undegraded":"2024-09-10T10:04:29.473465+0000","last_fullsized":"2024-09-10T10:04:29.473465+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:44:36.800356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.458792+0000","last_change":"2024-09-10T10:04:28.458728+0000","last_active":"2024-09-10T10:04:30.458792+0000","last_peered":"2024-09-10T10:04:30.458792+0000","last_clean":"2024-09-10T10:04:30.458792+0000","last_became_active":"2024-09-10T10:04:28.458451+0000","last_became_peered":"2024-09-10T10:04:28.458451+0000","last_unstale":"2024-09-10T10:04:30.458792+0000","last_undegraded":"2024-09-10T10:04:30.458792+0000","last_fullsized":"2024-09-10T10:04:30.458792+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:33.336423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464943+0000","last_change":"2024-09-10T10:04:29.451343+0000","last_active":"2024-09-10T10:04:30.464943+0000","last_peered":"2024-09-10T10:04:30.464943+0000","last_clean":"2024-09-10T10:04:30.464943+0000","last_became_active":"2024-09-10T10:04:29.451063+0000","last_became_peered":"2024-09-10T10:04:29.451063+0000","last_unstale":"2024-09-10T10:04:30.464943+0000","last_undegraded":"2024-09-10T10:04:30.464943+0000","last_fullsized":"2024-09-10T10:04:30.464943+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:42.701415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"129'8","reported_seq":20,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.478868+0000","last_change":"2024-09-10T10:04:28.454458+0000","last_active":"2024-09-10T10:04:29.478868+0000","last_peered":"2024-09-10T10:04:29.478868+0000","last_clean":"2024-09-10T10:04:29.478868+0000","last_became_active":"2024-09-10T10:04:28.454225+0000","last_became_peered":"2024-09-10T10:04:28.454225+0000","last_unstale":"2024-09-10T10:04:29.478868+0000","last_undegraded":"2024-09-10T10:04:29.478868+0000","last_fullsized":"2024-09-10T10:04:29.478868+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:48.509347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452788+0000","last_change":"2024-09-10T10:04:29.468431+0000","last_active":"2024-09-10T10:04:31.452788+0000","last_peered":"2024-09-10T10:04:31.452788+0000","last_clean":"2024-09-10T10:04:31.452788+0000","last_became_active":"2024-09-10T10:04:29.468146+0000","last_became_peered":"2024-09-10T10:04:29.468146+0000","last_unstale":"2024-09-10T10:04:31.452788+0000","last_undegraded":"2024-09-10T10:04:31.452788+0000","last_fullsized":"2024-09-10T10:04:31.452788+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:13.697427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.474961+0000","last_change":"2024-09-10T10:04:28.455053+0000","last_active":"2024-09-10T10:04:29.474961+0000","last_peered":"2024-09-10T10:04:29.474961+0000","last_clean":"2024-09-10T10:04:29.474961+0000","last_became_active":"2024-09-10T10:04:28.454671+0000","last_became_peered":"2024-09-10T10:04:28.454671+0000","last_unstale":"2024-09-10T10:04:29.474961+0000","last_undegraded":"2024-09-10T10:04:29.474961+0000","last_fullsized":"2024-09-10T10:04:29.474961+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:41.554859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476666+0000","last_change":"2024-09-10T10:04:29.467823+0000","last_active":"2024-09-10T10:04:29.476666+0000","last_peered":"2024-09-10T10:04:29.476666+0000","last_clean":"2024-09-10T10:04:29.476666+0000","last_became_active":"2024-09-10T10:04:29.467550+0000","last_became_peered":"2024-09-10T10:04:29.467550+0000","last_unstale":"2024-09-10T10:04:29.476666+0000","last_undegraded":"2024-09-10T10:04:29.476666+0000","last_fullsized":"2024-09-10T10:04:29.476666+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:25.245567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.475400+0000","last_change":"2024-09-10T10:04:28.455763+0000","last_active":"2024-09-10T10:04:29.475400+0000","last_peered":"2024-09-10T10:04:29.475400+0000","last_clean":"2024-09-10T10:04:29.475400+0000","last_became_active":"2024-09-10T10:04:28.455463+0000","last_became_peered":"2024-09-10T10:04:28.455463+0000","last_unstale":"2024-09-10T10:04:29.475400+0000","last_undegraded":"2024-09-10T10:04:29.475400+0000","last_fullsized":"2024-09-10T10:04:29.475400+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:56.020885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466574+0000","last_change":"2024-09-10T10:04:29.461820+0000","last_active":"2024-09-10T10:04:30.466574+0000","last_peered":"2024-09-10T10:04:30.466574+0000","last_clean":"2024-09-10T10:04:30.466574+0000","last_became_active":"2024-09-10T10:04:29.461442+0000","last_became_peered":"2024-09-10T10:04:29.461442+0000","last_unstale":"2024-09-10T10:04:30.466574+0000","last_undegraded":"2024-09-10T10:04:30.466574+0000","last_fullsized":"2024-09-10T10:04:30.466574+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:42.907084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"129'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.477832+0000","last_change":"2024-09-10T10:04:28.459930+0000","last_active":"2024-09-10T10:04:29.477832+0000","last_peered":"2024-09-10T10:04:29.477832+0000","last_clean":"2024-09-10T10:04:29.477832+0000","last_became_active":"2024-09-10T10:04:28.459654+0000","last_became_peered":"2024-09-10T10:04:28.459654+0000","last_unstale":"2024-09-10T10:04:29.477832+0000","last_undegraded":"2024-09-10T10:04:29.477832+0000","last_fullsized":"2024-09-10T10:04:29.477832+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:03.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459148+0000","last_change":"2024-09-10T10:04:29.453976+0000","last_active":"2024-09-10T10:04:30.459148+0000","last_peered":"2024-09-10T10:04:30.459148+0000","last_clean":"2024-09-10T10:04:30.459148+0000","last_became_active":"2024-09-10T10:04:29.453473+0000","last_became_peered":"2024-09-10T10:04:29.453473+0000","last_unstale":"2024-09-10T10:04:30.459148+0000","last_undegraded":"2024-09-10T10:04:30.459148+0000","last_fullsized":"2024-09-10T10:04:30.459148+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:42:05.539879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.451008+0000","last_change":"2024-09-10T10:04:28.462231+0000","last_active":"2024-09-10T10:04:29.451008+0000","last_peered":"2024-09-10T10:04:29.451008+0000","last_clean":"2024-09-10T10:04:29.451008+0000","last_became_active":"2024-09-10T10:04:28.461842+0000","last_became_peered":"2024-09-10T10:04:28.461842+0000","last_unstale":"2024-09-10T10:04:29.451008+0000","last_undegraded":"2024-09-10T10:04:29.451008+0000","last_fullsized":"2024-09-10T10:04:29.451008+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:25.197719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465550+0000","last_change":"2024-09-10T10:04:29.457948+0000","last_active":"2024-09-10T10:04:30.465550+0000","last_peered":"2024-09-10T10:04:30.465550+0000","last_clean":"2024-09-10T10:04:30.465550+0000","last_became_active":"2024-09-10T10:04:29.457596+0000","last_became_peered":"2024-09-10T10:04:29.457596+0000","last_unstale":"2024-09-10T10:04:30.465550+0000","last_undegraded":"2024-09-10T10:04:30.465550+0000","last_fullsized":"2024-09-10T10:04:30.465550+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:24.790019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476378+0000","last_change":"2024-09-10T10:04:28.450232+0000","last_active":"2024-09-10T10:04:29.476378+0000","last_peered":"2024-09-10T10:04:29.476378+0000","last_clean":"2024-09-10T10:04:29.476378+0000","last_became_active":"2024-09-10T10:04:28.449951+0000","last_became_peered":"2024-09-10T10:04:28.449951+0000","last_unstale":"2024-09-10T10:04:29.476378+0000","last_undegraded":"2024-09-10T10:04:29.476378+0000","last_fullsized":"2024-09-10T10:04:29.476378+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:56.442466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.465927+0000","last_change":"2024-09-10T10:04:29.460170+0000","last_active":"2024-09-10T10:04:29.465927+0000","last_peered":"2024-09-10T10:04:29.465927+0000","last_clean":"2024-09-10T10:04:29.465927+0000","last_became_active":"2024-09-10T10:04:29.459901+0000","last_became_peered":"2024-09-10T10:04:29.459901+0000","last_unstale":"2024-09-10T10:04:29.465927+0000","last_undegraded":"2024-09-10T10:04:29.465927+0000","last_fullsized":"2024-09-10T10:04:29.465927+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:56.576350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452481+0000","last_change":"2024-09-10T10:04:28.466991+0000","last_active":"2024-09-10T10:04:31.452481+0000","last_peered":"2024-09-10T10:04:31.452481+0000","last_clean":"2024-09-10T10:04:31.452481+0000","last_became_active":"2024-09-10T10:04:28.466642+0000","last_became_peered":"2024-09-10T10:04:28.466642+0000","last_unstale":"2024-09-10T10:04:31.452481+0000","last_undegraded":"2024-09-10T10:04:31.452481+0000","last_fullsized":"2024-09-10T10:04:31.452481+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:43.970259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465247+0000","last_change":"2024-09-10T10:04:29.458511+0000","last_active":"2024-09-10T10:04:30.465247+0000","last_peered":"2024-09-10T10:04:30.465247+0000","last_clean":"2024-09-10T10:04:30.465247+0000","last_became_active":"2024-09-10T10:04:29.458224+0000","last_became_peered":"2024-09-10T10:04:29.458224+0000","last_unstale":"2024-09-10T10:04:30.465247+0000","last_undegraded":"2024-09-10T10:04:30.465247+0000","last_fullsized":"2024-09-10T10:04:30.465247+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:42.701415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452251+0000","last_change":"2024-09-10T10:04:28.466001+0000","last_active":"2024-09-10T10:04:31.452251+0000","last_peered":"2024-09-10T10:04:31.452251+0000","last_clean":"2024-09-10T10:04:31.452251+0000","last_became_active":"2024-09-10T10:04:28.465770+0000","last_became_peered":"2024-09-10T10:04:28.465770+0000","last_unstale":"2024-09-10T10:04:31.452251+0000","last_undegraded":"2024-09-10T10:04:31.452251+0000","last_fullsized":"2024-09-10T10:04:31.452251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:43.483139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.466956+0000","last_change":"2024-09-10T10:04:29.461448+0000","last_active":"2024-09-10T10:04:29.466956+0000","last_peered":"2024-09-10T10:04:29.466956+0000","last_clean":"2024-09-10T10:04:29.466956+0000","last_became_active":"2024-09-10T10:04:29.461205+0000","last_became_peered":"2024-09-10T10:04:29.461205+0000","last_unstale":"2024-09-10T10:04:29.466956+0000","last_undegraded":"2024-09-10T10:04:29.466956+0000","last_fullsized":"2024-09-10T10:04:29.466956+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:57.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.464735+0000","last_change":"2024-09-10T10:04:28.450080+0000","last_active":"2024-09-10T10:04:30.464735+0000","last_peered":"2024-09-10T10:04:30.464735+0000","last_clean":"2024-09-10T10:04:30.464735+0000","last_became_active":"2024-09-10T10:04:28.449798+0000","last_became_peered":"2024-09-10T10:04:28.449798+0000","last_unstale":"2024-09-10T10:04:30.464735+0000","last_undegraded":"2024-09-10T10:04:30.464735+0000","last_fullsized":"2024-09-10T10:04:30.464735+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:02.072805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.473315+0000","last_change":"2024-09-10T10:04:29.467753+0000","last_active":"2024-09-10T10:04:29.473315+0000","last_peered":"2024-09-10T10:04:29.473315+0000","last_clean":"2024-09-10T10:04:29.473315+0000","last_became_active":"2024-09-10T10:04:29.467494+0000","last_became_peered":"2024-09-10T10:04:29.467494+0000","last_unstale":"2024-09-10T10:04:29.473315+0000","last_undegraded":"2024-09-10T10:04:29.473315+0000","last_fullsized":"2024-09-10T10:04:29.473315+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:42.419512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.453124+0000","last_change":"2024-09-10T10:04:28.465825+0000","last_active":"2024-09-10T10:04:31.453124+0000","last_peered":"2024-09-10T10:04:31.453124+0000","last_clean":"2024-09-10T10:04:31.453124+0000","last_became_active":"2024-09-10T10:04:28.465065+0000","last_became_peered":"2024-09-10T10:04:28.465065+0000","last_unstale":"2024-09-10T10:04:31.453124+0000","last_undegraded":"2024-09-10T10:04:31.453124+0000","last_fullsized":"2024-09-10T10:04:31.453124+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:57.361291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.466522+0000","last_change":"2024-09-10T10:04:29.461069+0000","last_active":"2024-09-10T10:04:30.466522+0000","last_peered":"2024-09-10T10:04:30.466522+0000","last_clean":"2024-09-10T10:04:30.466522+0000","last_became_active":"2024-09-10T10:04:29.460771+0000","last_became_peered":"2024-09-10T10:04:29.460771+0000","last_unstale":"2024-09-10T10:04:30.466522+0000","last_undegraded":"2024-09-10T10:04:30.466522+0000","last_fullsized":"2024-09-10T10:04:30.466522+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:12.119083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"130'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.450757+0000","last_change":"2024-09-10T10:04:28.460104+0000","last_active":"2024-09-10T10:04:31.450757+0000","last_peered":"2024-09-10T10:04:31.450757+0000","last_clean":"2024-09-10T10:04:31.450757+0000","last_became_active":"2024-09-10T10:04:28.459863+0000","last_became_peered":"2024-09-10T10:04:28.459863+0000","last_unstale":"2024-09-10T10:04:31.450757+0000","last_undegraded":"2024-09-10T10:04:31.450757+0000","last_fullsized":"2024-09-10T10:04:31.450757+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:44:35.797722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.476981+0000","last_change":"2024-09-10T10:04:29.472460+0000","last_active":"2024-09-10T10:04:29.476981+0000","last_peered":"2024-09-10T10:04:29.476981+0000","last_clean":"2024-09-10T10:04:29.476981+0000","last_became_active":"2024-09-10T10:04:29.472172+0000","last_became_peered":"2024-09-10T10:04:29.472172+0000","last_unstale":"2024-09-10T10:04:29.476981+0000","last_undegraded":"2024-09-10T10:04:29.476981+0000","last_fullsized":"2024-09-10T10:04:29.476981+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:52.826481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459493+0000","last_change":"2024-09-10T10:04:28.457883+0000","last_active":"2024-09-10T10:04:30.459493+0000","last_peered":"2024-09-10T10:04:30.459493+0000","last_clean":"2024-09-10T10:04:30.459493+0000","last_became_active":"2024-09-10T10:04:28.457646+0000","last_became_peered":"2024-09-10T10:04:28.457646+0000","last_unstale":"2024-09-10T10:04:30.459493+0000","last_undegraded":"2024-09-10T10:04:30.459493+0000","last_fullsized":"2024-09-10T10:04:30.459493+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:40.328391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459277+0000","last_change":"2024-09-10T10:04:29.457580+0000","last_active":"2024-09-10T10:04:30.459277+0000","last_peered":"2024-09-10T10:04:30.459277+0000","last_clean":"2024-09-10T10:04:30.459277+0000","last_became_active":"2024-09-10T10:04:29.457102+0000","last_became_peered":"2024-09-10T10:04:29.457102+0000","last_unstale":"2024-09-10T10:04:30.459277+0000","last_undegraded":"2024-09-10T10:04:30.459277+0000","last_fullsized":"2024-09-10T10:04:30.459277+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:43.659824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.459020+0000","last_change":"2024-09-10T10:04:28.457002+0000","last_active":"2024-09-10T10:04:30.459020+0000","last_peered":"2024-09-10T10:04:30.459020+0000","last_clean":"2024-09-10T10:04:30.459020+0000","last_became_active":"2024-09-10T10:04:28.456756+0000","last_became_peered":"2024-09-10T10:04:28.456756+0000","last_unstale":"2024-09-10T10:04:30.459020+0000","last_undegraded":"2024-09-10T10:04:30.459020+0000","last_fullsized":"2024-09-10T10:04:30.459020+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:22.255332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.462515+0000","last_change":"2024-09-10T10:04:29.459026+0000","last_active":"2024-09-10T10:04:29.462515+0000","last_peered":"2024-09-10T10:04:29.462515+0000","last_clean":"2024-09-10T10:04:29.462515+0000","last_became_active":"2024-09-10T10:04:29.458760+0000","last_became_peered":"2024-09-10T10:04:29.458760+0000","last_unstale":"2024-09-10T10:04:29.462515+0000","last_undegraded":"2024-09-10T10:04:29.462515+0000","last_fullsized":"2024-09-10T10:04:29.462515+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:35.832409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.449719+0000","last_change":"2024-09-10T10:04:28.449910+0000","last_active":"2024-09-10T10:04:29.449719+0000","last_peered":"2024-09-10T10:04:29.449719+0000","last_clean":"2024-09-10T10:04:29.449719+0000","last_became_active":"2024-09-10T10:04:28.449106+0000","last_became_peered":"2024-09-10T10:04:28.449106+0000","last_unstale":"2024-09-10T10:04:29.449719+0000","last_undegraded":"2024-09-10T10:04:29.449719+0000","last_fullsized":"2024-09-10T10:04:29.449719+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:57.361291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.459494+0000","last_change":"2024-09-10T10:04:29.455183+0000","last_active":"2024-09-10T10:04:29.459494+0000","last_peered":"2024-09-10T10:04:29.459494+0000","last_clean":"2024-09-10T10:04:29.459494+0000","last_became_active":"2024-09-10T10:04:29.454786+0000","last_became_peered":"2024-09-10T10:04:29.454786+0000","last_unstale":"2024-09-10T10:04:29.459494+0000","last_undegraded":"2024-09-10T10:04:29.459494+0000","last_fullsized":"2024-09-10T10:04:29.459494+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:31.588098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452471+0000","last_change":"2024-09-10T10:04:28.465790+0000","last_active":"2024-09-10T10:04:31.452471+0000","last_peered":"2024-09-10T10:04:31.452471+0000","last_clean":"2024-09-10T10:04:31.452471+0000","last_became_active":"2024-09-10T10:04:28.465039+0000","last_became_peered":"2024-09-10T10:04:28.465039+0000","last_unstale":"2024-09-10T10:04:31.452471+0000","last_undegraded":"2024-09-10T10:04:31.452471+0000","last_fullsized":"2024-09-10T10:04:31.452471+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:21.797140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452788+0000","last_change":"2024-09-10T10:04:29.463877+0000","last_active":"2024-09-10T10:04:31.452788+0000","last_peered":"2024-09-10T10:04:31.452788+0000","last_clean":"2024-09-10T10:04:31.452788+0000","last_became_active":"2024-09-10T10:04:29.463366+0000","last_became_peered":"2024-09-10T10:04:29.463366+0000","last_unstale":"2024-09-10T10:04:31.452788+0000","last_undegraded":"2024-09-10T10:04:31.452788+0000","last_fullsized":"2024-09-10T10:04:31.452788+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:39.047386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465055+0000","last_change":"2024-09-10T10:04:28.460668+0000","last_active":"2024-09-10T10:04:30.465055+0000","last_peered":"2024-09-10T10:04:30.465055+0000","last_clean":"2024-09-10T10:04:30.465055+0000","last_became_active":"2024-09-10T10:04:28.460311+0000","last_became_peered":"2024-09-10T10:04:28.460311+0000","last_unstale":"2024-09-10T10:04:30.465055+0000","last_undegraded":"2024-09-10T10:04:30.465055+0000","last_fullsized":"2024-09-10T10:04:30.465055+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:42.657191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.461382+0000","last_change":"2024-09-10T10:04:29.455516+0000","last_active":"2024-09-10T10:04:29.461382+0000","last_peered":"2024-09-10T10:04:29.461382+0000","last_clean":"2024-09-10T10:04:29.461382+0000","last_became_active":"2024-09-10T10:04:29.455017+0000","last_became_peered":"2024-09-10T10:04:29.455017+0000","last_unstale":"2024-09-10T10:04:29.461382+0000","last_undegraded":"2024-09-10T10:04:29.461382+0000","last_fullsized":"2024-09-10T10:04:29.461382+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:43.076635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.450302+0000","last_change":"2024-09-10T10:04:28.462057+0000","last_active":"2024-09-10T10:04:29.450302+0000","last_peered":"2024-09-10T10:04:29.450302+0000","last_clean":"2024-09-10T10:04:29.450302+0000","last_became_active":"2024-09-10T10:04:28.461779+0000","last_became_peered":"2024-09-10T10:04:28.461779+0000","last_unstale":"2024-09-10T10:04:29.450302+0000","last_undegraded":"2024-09-10T10:04:29.450302+0000","last_fullsized":"2024-09-10T10:04:29.450302+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:19.830305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-10T10:04:29.466337+0000","last_change":"2024-09-10T10:04:29.461184+0000","last_active":"2024-09-10T10:04:29.466337+0000","last_peered":"2024-09-10T10:04:29.466337+0000","last_clean":"2024-09-10T10:04:29.466337+0000","last_became_active":"2024-09-10T10:04:29.460946+0000","last_became_peered":"2024-09-10T10:04:29.460946+0000","last_unstale":"2024-09-10T10:04:29.466337+0000","last_undegraded":"2024-09-10T10:04:29.466337+0000","last_fullsized":"2024-09-10T10:04:29.466337+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:28.430113+0000","last_clean_scrub_stamp":"2024-09-10T10:04:28.430113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:52.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"129'3","reported_seq":17,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:30.465975+0000","last_change":"2024-09-10T10:04:28.454622+0000","last_active":"2024-09-10T10:04:30.465975+0000","last_peered":"2024-09-10T10:04:30.465975+0000","last_clean":"2024-09-10T10:04:30.465975+0000","last_became_active":"2024-09-10T10:04:28.454320+0000","last_became_peered":"2024-09-10T10:04:28.454320+0000","last_unstale":"2024-09-10T10:04:30.465975+0000","last_undegraded":"2024-09-10T10:04:30.465975+0000","last_fullsized":"2024-09-10T10:04:30.465975+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:04:27.427479+0000","last_clean_scrub_stamp":"2024-09-10T10:04:27.427479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:41.305631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":407,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-10T10:04:31.452887+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:04:31.452887+0000","last_peered":"2024-09-10T10:04:31.452887+0000","last_clean":"2024-09-10T10:04:31.452887+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:04:31.452887+0000","last_undegraded":"2024-09-10T10:04:31.452887+0000","last_fullsized":"2024-09-10T10:04:31.452887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509518,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33348,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338492,"statfs":{"total":96636764160,"available":96602615808,"internally_reserved":0,"allocated":2211840,"data_stored":1299584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1634304,"data_stored":715066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509518,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1613824,"data_stored":708586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1622016,"data_stored":714920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542224,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1613824,"data_stored":708654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509519,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1626112,"data_stored":710246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509518,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1626112,"data_stored":715502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509519,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37516,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334324,"statfs":{"total":96636764160,"available":96598347776,"internally_reserved":0,"allocated":2220032,"data_stored":1300578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:04:33.916 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:04:33.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:04:34.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fb2e00640 1 Processor -- start 2024-09-10T10:04:34.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fb2e00640 1 -- start start 2024-09-10T10:04:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fb2e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6fac0cfe30 0x7f6fac071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fb2e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fac06d2d0 con 0x7f6fac06e7a0 2024-09-10T10:04:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fb2e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fac06d5b0 con 0x7f6fac0cd510 2024-09-10T10:04:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fb2e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6fac15d9a0 con 0x7f6fac0cfe30 2024-09-10T10:04:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fabe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6fac0cfe30 0x7f6fac071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fabe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6fac0cfe30 0x7f6fac071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52052/0 (socket says 172.21.15.110:52052) 2024-09-10T10:04:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fabe00640 1 -- 172.21.15.110:0/830014523 learned_addr learned my addr 172.21.15.110:0/830014523 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fb0e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6fac06e7a0 0x7f6fac15cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55868/0 (socket says 172.21.15.110:55868) 2024-09-10T10:04:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fabe00640 1 -- 172.21.15.110:0/830014523 >> v1:172.21.15.145:6789/0 conn(0x7f6fac0cd510 legacy=0x7f6fac06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fabe00640 1 -- 172.21.15.110:0/830014523 >> v1:172.21.15.110:6789/0 conn(0x7f6fac06e7a0 legacy=0x7f6fac15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.094+0000 7f6fabe00640 1 -- 172.21.15.110:0/830014523 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac06d890 con 0x7f6fac0cfe30 2024-09-10T10:04:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6fabe00640 1 --2- 172.21.15.110:0/830014523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cfe30 0x7f6fac071a40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009e50 tx=0x7f6f9c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc89c0f01d8ff7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6faaa00640 1 -- 172.21.15.110:0/830014523 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0a5730 con 0x7f6fac0cfe30 2024-09-10T10:04:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6faaa00640 1 -- 172.21.15.110:0/830014523 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9c0a58d0 con 0x7f6fac0cfe30 2024-09-10T10:04:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6faaa00640 1 -- 172.21.15.110:0/830014523 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0a5ca0 con 0x7f6fac0cfe30 2024-09-10T10:04:34.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6fb2e00640 1 -- 172.21.15.110:0/830014523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cfe30 msgr2=0x7f6fac071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6fb2e00640 1 --2- 172.21.15.110:0/830014523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cfe30 0x7f6fac071a40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009e50 tx=0x7f6f9c01c960 comp rx=0 tx=0).stop 2024-09-10T10:04:34.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6fb2e00640 1 -- 172.21.15.110:0/830014523 shutdown_connections 2024-09-10T10:04:34.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6fb2e00640 1 --2- 172.21.15.110:0/830014523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cfe30 0x7f6fac071a40 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6fb2e00640 1 -- 172.21.15.110:0/830014523 >> 172.21.15.110:0/830014523 conn(0x7f6fac06bc50 msgr2=0x7f6fac06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6fb2e00640 1 -- 172.21.15.110:0/830014523 shutdown_connections 2024-09-10T10:04:34.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.098+0000 7f6fb2e00640 1 -- 172.21.15.110:0/830014523 wait complete. 2024-09-10T10:04:34.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 Processor -- start 2024-09-10T10:04:34.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 -- start start 2024-09-10T10:04:34.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fac06e7a0 0x7f6fac158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cd510 0x7f6fac154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6fac0cfe30 0x7f6fac155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac15d740 con 0x7f6fac0cd510 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac072190 con 0x7f6fac06e7a0 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6fac16abb0 con 0x7f6fac0cfe30 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cd510 0x7f6fac154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cd510 0x7f6fac154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52066/0 (socket says 172.21.15.110:52066) 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 -- 172.21.15.110:0/3841278236 learned_addr learned my addr 172.21.15.110:0/3841278236 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fabe00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fac06e7a0 0x7f6fac158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb0e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6fac0cfe30 0x7f6fac155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 -- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6fac0cfe30 msgr2=0x7f6fac155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6fac0cfe30 0x7f6fac155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 -- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fac06e7a0 msgr2=0x7f6fac158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fac06e7a0 0x7f6fac158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 -- 172.21.15.110:0/3841278236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac1575a0 con 0x7f6fac0cd510 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fabe00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fac06e7a0 0x7f6fac158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb0e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6fac0cfe30 0x7f6fac155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fab400640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cd510 0x7f6fac154c10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800b410 tx=0x7f6f9800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fa3e00640 1 -- 172.21.15.110:0/3841278236 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98015030 con 0x7f6fac0cd510 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fa3e00640 1 -- 172.21.15.110:0/3841278236 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f98004020 con 0x7f6fac0cd510 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac157880 con 0x7f6fac0cd510 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fa3e00640 1 -- 172.21.15.110:0/3841278236 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98004450 con 0x7f6fac0cd510 2024-09-10T10:04:34.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac158120 con 0x7f6fac0cd510 2024-09-10T10:04:34.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.102+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fac157a50 con 0x7f6fac0cd510 2024-09-10T10:04:34.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.106+0000 7f6fa3e00640 1 -- 172.21.15.110:0/3841278236 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f6f98018070 con 0x7f6fac0cd510 2024-09-10T10:04:34.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.106+0000 7f6fa3e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f8c067e60 0x7f6f8c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.106+0000 7f6fa3e00640 1 -- 172.21.15.110:0/3841278236 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f6f98086440 con 0x7f6fac0cd510 2024-09-10T10:04:34.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.106+0000 7f6fabe00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f8c067e60 0x7f6f8c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.106+0000 7f6fabe00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f8c067e60 0x7f6f8c06a320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0ae000 tx=0x7f6f9c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.110+0000 7f6fa3e00640 1 -- 172.21.15.110:0/3841278236 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6f98056600 con 0x7f6fac0cd510 2024-09-10T10:04:34.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.322+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6fac158320 con 0x7f6fac0cd510 2024-09-10T10:04:34.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.322+0000 7f6fa3e00640 1 -- 172.21.15.110:0/3841278236 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f6fac158320 con 0x7f6fac0cd510 2024-09-10T10:04:34.329 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:04:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:04:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:04:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:04:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f8c067e60 msgr2=0x7f6f8c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f8c067e60 0x7f6f8c06a320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0ae000 tx=0x7f6f9c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cd510 msgr2=0x7f6fac154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cd510 0x7f6fac154c10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800b410 tx=0x7f6f9800b8d0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 shutdown_connections 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f8c067e60 0x7f6f8c06a320 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6fac0cfe30 0x7f6fac155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fac0cd510 0x7f6fac154c10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 --2- 172.21.15.110:0/3841278236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6fac06e7a0 0x7f6fac158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 >> 172.21.15.110:0/3841278236 conn(0x7f6fac06bc50 msgr2=0x7f6fac06e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 shutdown_connections 2024-09-10T10:04:34.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.326+0000 7f6fb2e00640 1 -- 172.21.15.110:0/3841278236 wait complete. 2024-09-10T10:04:34.350 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:34.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.482+0000 7f1834c00640 1 Processor -- start 2024-09-10T10:04:34.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f1834c00640 1 -- start start 2024-09-10T10:04:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f1834c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f183006ea50 0x7f183006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f1834c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18300b5f50 con 0x7f183015e8f0 2024-09-10T10:04:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f1834c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18300b6230 con 0x7f183006bd30 2024-09-10T10:04:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f1834c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f183006f3d0 con 0x7f183006ea50 2024-09-10T10:04:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f182ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f183006ea50 0x7f183006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f182f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f183015e8f0 0x7f18300b5490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55884/0 (socket says 172.21.15.110:55884) 2024-09-10T10:04:34.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f182f400640 1 -- 172.21.15.110:0/1794130027 learned_addr learned my addr 172.21.15.110:0/1794130027 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:34.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f182ea00640 1 -- 172.21.15.110:0/1794130027 >> v1:172.21.15.145:6789/0 conn(0x7f183006bd30 legacy=0x7f183006c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f182ea00640 1 -- 172.21.15.110:0/1794130027 >> v1:172.21.15.110:6789/0 conn(0x7f183015e8f0 legacy=0x7f18300b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.486+0000 7f182ea00640 1 -- 172.21.15.110:0/1794130027 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18300b6510 con 0x7f183006ea50 2024-09-10T10:04:34.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f182ea00640 1 --2- 172.21.15.110:0/1794130027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006ea50 0x7f183006ee90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f181c0097c0 tx=0x7f181c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e69f23c01d3ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f182e000640 1 -- 172.21.15.110:0/1794130027 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c0b0070 con 0x7f183006ea50 2024-09-10T10:04:34.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f182e000640 1 -- 172.21.15.110:0/1794130027 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181c0ab600 con 0x7f183006ea50 2024-09-10T10:04:34.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f182e000640 1 -- 172.21.15.110:0/1794130027 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c0ab9d0 con 0x7f183006ea50 2024-09-10T10:04:34.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 -- 172.21.15.110:0/1794130027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006ea50 msgr2=0x7f183006ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 --2- 172.21.15.110:0/1794130027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006ea50 0x7f183006ee90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f181c0097c0 tx=0x7f181c01c3d0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 -- 172.21.15.110:0/1794130027 shutdown_connections 2024-09-10T10:04:34.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 --2- 172.21.15.110:0/1794130027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006ea50 0x7f183006ee90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 -- 172.21.15.110:0/1794130027 >> 172.21.15.110:0/1794130027 conn(0x7f18300d1480 msgr2=0x7f18300d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 -- 172.21.15.110:0/1794130027 shutdown_connections 2024-09-10T10:04:34.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 -- 172.21.15.110:0/1794130027 wait complete. 2024-09-10T10:04:34.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 Processor -- start 2024-09-10T10:04:34.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.490+0000 7f1834c00640 1 -- start start 2024-09-10T10:04:34.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006bd30 0x7f1830150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f183006ea50 0x7f18301497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f183015e8f0 0x7f183015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f18300cddc0 con 0x7f183006bd30 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18300cdc40 con 0x7f183015e8f0 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f183016c340 con 0x7f183006ea50 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006bd30 0x7f1830150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006bd30 0x7f1830150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52088/0 (socket says 172.21.15.110:52088) 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 -- 172.21.15.110:0/3291328258 learned_addr learned my addr 172.21.15.110:0/3291328258 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182ea00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f183006ea50 0x7f18301497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182fe00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f183015e8f0 0x7f183015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 -- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f183006ea50 msgr2=0x7f18301497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f183006ea50 0x7f18301497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 -- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f183015e8f0 msgr2=0x7f183015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f183015e8f0 0x7f183015b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 -- 172.21.15.110:0/3291328258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183015a080 con 0x7f183006bd30 2024-09-10T10:04:34.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f182f400640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006bd30 0x7f1830150640 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f182000c370 tx=0x7f182000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1827400640 1 -- 172.21.15.110:0/3291328258 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820017070 con 0x7f183006bd30 2024-09-10T10:04:34.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1827400640 1 -- 172.21.15.110:0/3291328258 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1820005cf0 con 0x7f183006bd30 2024-09-10T10:04:34.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1827400640 1 -- 172.21.15.110:0/3291328258 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18200072f0 con 0x7f183006bd30 2024-09-10T10:04:34.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183015a360 con 0x7f183006bd30 2024-09-10T10:04:34.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183015a920 con 0x7f183006bd30 2024-09-10T10:04:34.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183006ee90 con 0x7f183006bd30 2024-09-10T10:04:34.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.494+0000 7f1827400640 1 -- 172.21.15.110:0/3291328258 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f182001a070 con 0x7f183006bd30 2024-09-10T10:04:34.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.498+0000 7f1827400640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1810067e60 0x7f181006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.498+0000 7f1827400640 1 -- 172.21.15.110:0/3291328258 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f1820085aa0 con 0x7f183006bd30 2024-09-10T10:04:34.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.498+0000 7f182ea00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1810067e60 0x7f181006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.498+0000 7f182ea00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1810067e60 0x7f181006a320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f181c0ad000 tx=0x7f181c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.502+0000 7f1827400640 1 -- 172.21.15.110:0/3291328258 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f18200556a0 con 0x7f183006bd30 2024-09-10T10:04:34.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.690+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1830155850 con 0x7f183006bd30 2024-09-10T10:04:34.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1827400640 1 -- 172.21.15.110:0/3291328258 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5239 (secure 0 0 0) 0x7f18200599a0 con 0x7f183006bd30 2024-09-10T10:04:34.697 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:34.697 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":88,"btime":"2024-09-10T10:04:32:463468+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3129750368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3129750368},{"type":"v1","addr":"172.21.15.145:6834","nonce":3129750368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3791950507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3791950507},{"type":"v1","addr":"172.21.15.145:6835","nonce":3791950507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2024540113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2024540113},{"type":"v1","addr":"172.21.15.110:6834","nonce":2024540113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:04:28.820193+0000","modified":"2024-09-10T10:04:31.842870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1881265533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1881265533},{"type":"v1","addr":"172.21.15.110:6835","nonce":1881265533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7686,"qdb_cluster":[7686]},"id":10}]} 2024-09-10T10:04:34.698 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 88 2024-09-10T10:04:34.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1810067e60 msgr2=0x7f181006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1810067e60 0x7f181006a320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f181c0ad000 tx=0x7f181c008040 comp rx=0 tx=0).stop 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006bd30 msgr2=0x7f1830150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006bd30 0x7f1830150640 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f182000c370 tx=0x7f182000c830 comp rx=0 tx=0).stop 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 shutdown_connections 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1810067e60 0x7f181006a320 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f183015e8f0 0x7f183015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f183006ea50 0x7f18301497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 --2- 172.21.15.110:0/3291328258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f183006bd30 0x7f1830150640 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 >> 172.21.15.110:0/3291328258 conn(0x7f18300d1480 msgr2=0x7f18300b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 shutdown_connections 2024-09-10T10:04:34.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:34.694+0000 7f1834c00640 1 -- 172.21.15.110:0/3291328258 wait complete. 2024-09-10T10:04:35.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:04:28.820193+0000', 'modified': '2024-09-10T10:04:31.842870+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7686}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7686': {'gid': 7686, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6835/1881265533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6833', 'nonce': 1881265533}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 1881265533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7686, 'qdb_cluster': [7686]} 2024-09-10T10:04:35.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:04:35.121 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:04:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba3600640 1 Processor -- start 2024-09-10T10:04:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba3600640 1 -- start start 2024-09-10T10:04:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba3600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7b9c0cfb90 0x7f7b9c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba3600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7b9c0cdcb0 0x7f7b9c06db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba3600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b9c15d4d0 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba3600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f7b9c06d130 con 0x7f7b9c0cfb90 2024-09-10T10:04:35.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba3600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f7b9c15d7b0 con 0x7f7b9c0cdcb0 2024-09-10T10:04:35.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba0c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7b9c0cfb90 0x7f7b9c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba0c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7b9c0cfb90 0x7f7b9c15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52102/0 (socket says 172.21.15.110:52102) 2024-09-10T10:04:35.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba0c00640 1 -- 172.21.15.110:0/1350918910 learned_addr learned my addr 172.21.15.110:0/1350918910 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:35.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba1600640 1 --2- 172.21.15.110:0/1350918910 >> v2:172.21.15.145:3300/0 conn(0x7f7b9c0cdcb0 0x7f7b9c06db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba0c00640 1 -- 172.21.15.110:0/1350918910 >> v2:172.21.15.145:3300/0 conn(0x7f7b9c0cdcb0 msgr2=0x7f7b9c06db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/1350918910 >> v2:172.21.15.145:3300/0 conn(0x7f7b9c0cdcb0 0x7f7b9c06db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba0c00640 1 -- 172.21.15.110:0/1350918910 >> v1:172.21.15.145:6789/0 conn(0x7f7b9c0b3f40 legacy=0x7f7b9c0cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.258+0000 7f7ba0c00640 1 -- 172.21.15.110:0/1350918910 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c06d380 con 0x7f7b9c0cfb90 2024-09-10T10:04:35.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/1350918910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0cfb90 0x7f7b9c15ca10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0ce170 tx=0x7f7b8c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12ffc7ca6197e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7b9be00640 1 -- 172.21.15.110:0/1350918910 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0b0070 con 0x7f7b9c0cfb90 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7b9be00640 1 -- 172.21.15.110:0/1350918910 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8c0ab600 con 0x7f7b9c0cfb90 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7b9be00640 1 -- 172.21.15.110:0/1350918910 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0ab9d0 con 0x7f7b9c0cfb90 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 -- 172.21.15.110:0/1350918910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0cfb90 msgr2=0x7f7b9c15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 --2- 172.21.15.110:0/1350918910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0cfb90 0x7f7b9c15ca10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0ce170 tx=0x7f7b8c01c490 comp rx=0 tx=0).stop 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 -- 172.21.15.110:0/1350918910 shutdown_connections 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 --2- 172.21.15.110:0/1350918910 >> v2:172.21.15.145:3300/0 conn(0x7f7b9c0cdcb0 0x7f7b9c06db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 --2- 172.21.15.110:0/1350918910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0cfb90 0x7f7b9c15ca10 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 -- 172.21.15.110:0/1350918910 >> 172.21.15.110:0/1350918910 conn(0x7f7b9c062d10 msgr2=0x7f7b9c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 -- 172.21.15.110:0/1350918910 shutdown_connections 2024-09-10T10:04:35.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 -- 172.21.15.110:0/1350918910 wait complete. 2024-09-10T10:04:35.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.262+0000 7f7ba3600640 1 Processor -- start 2024-09-10T10:04:35.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 -- start start 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0b3f40 0x7f7b9c165a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b9c0cdcb0 0x7f7b9c150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b9c0cfb90 0x7f7b9c150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c169170 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1692f0 con 0x7f7b9c0cfb90 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9c169470 con 0x7f7b9c0cdcb0 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0b3f40 0x7f7b9c165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0b3f40 0x7f7b9c165a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52108/0 (socket says 172.21.15.110:52108) 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 -- 172.21.15.110:0/970566922 learned_addr learned my addr 172.21.15.110:0/970566922 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 -- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b9c0cdcb0 msgr2=0x7f7b9c150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7b93e00640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b9c0cdcb0 0x7f7b9c150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba1600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b9c0cfb90 0x7f7b9c150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b9c0cdcb0 0x7f7b9c150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 -- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b9c0cfb90 msgr2=0x7f7b9c150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b9c0cfb90 0x7f7b9c150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 -- 172.21.15.110:0/970566922 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c069bb0 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7b93e00640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b9c0cdcb0 0x7f7b9c150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba1600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b9c0cfb90 0x7f7b9c150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0b3f40 0x7f7b9c165a30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0ae040 tx=0x7f7b8c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7b99600640 1 -- 172.21.15.110:0/970566922 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0b0040 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7b99600640 1 -- 172.21.15.110:0/970566922 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8c0b9070 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7b99600640 1 -- 172.21.15.110:0/970566922 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0b48a0 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 -- 172.21.15.110:0/970566922 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c163980 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.266+0000 7f7ba3600640 1 -- 172.21.15.110:0/970566922 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c163f40 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b99600640 1 -- 172.21.15.110:0/970566922 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f7b8c0b4a40 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b99600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b78067e60 0x7f7b7806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b99600640 1 -- 172.21.15.110:0/970566922 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f7b8c120670 con 0x7f7b9c0b3f40 2024-09-10T10:04:35.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba69e0000 1 Processor -- start 2024-09-10T10:04:35.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b78067e60 0x7f7b7806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba69e0000 1 -- start start 2024-09-10T10:04:35.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b78067e60 0x7f7b7806a320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0b4c70 tx=0x7f7b84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 0x56438daf3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf3690 0x56438daa9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438daa9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56438d98cb60 con 0x56438daf3690 2024-09-10T10:04:35.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56438d8f3c70 con 0x56438daf2d70 2024-09-10T10:04:35.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x56438d899ed0 con 0x56438daa96a0 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf3690 0x56438daa9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf3690 0x56438daa9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52112/0 (socket says 172.21.15.110:52112) 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 -- 172.21.15.110:0/1206759275 learned_addr learned my addr 172.21.15.110:0/1206759275 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 -- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 msgr2=0x56438daa9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 0x56438daf3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba1600640 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438daa9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438daa9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 -- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 msgr2=0x56438daf3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 0x56438daf3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 -- 172.21.15.110:0/1206759275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438d8abe50 con 0x56438daf3690 2024-09-10T10:04:35.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 0x56438daf3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:35.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7ba1600640 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438daa9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:35.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b93e00640 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf3690 0x56438daa9160 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400d9d0 tx=0x7f7b84010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb43518e1448cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b91600640 1 -- 172.21.15.110:0/1206759275 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84014db0 con 0x56438daf3690 2024-09-10T10:04:35.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b91600640 1 -- 172.21.15.110:0/1206759275 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8400de90 con 0x56438daf3690 2024-09-10T10:04:35.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.270+0000 7f7b91600640 1 -- 172.21.15.110:0/1206759275 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84007220 con 0x56438daf3690 2024-09-10T10:04:35.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- 172.21.15.110:0/1206759275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf3690 msgr2=0x56438daa9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf3690 0x56438daa9160 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400d9d0 tx=0x7f7b84010000 comp rx=0 tx=0).stop 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- 172.21.15.110:0/1206759275 shutdown_connections 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438daa9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf3690 0x56438daa9160 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/1206759275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 0x56438daf3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- 172.21.15.110:0/1206759275 >> 172.21.15.110:0/1206759275 conn(0x56438daab600 msgr2=0x56438daa3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- 172.21.15.110:0/1206759275 shutdown_connections 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- 172.21.15.110:0/1206759275 wait complete. 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- start start 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 Processor -- start 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daf2d70 0x56438daf8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf3690 0x56438daf93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438dadf490 0x56438dadf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56438d955b00 con 0x56438dadf490 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56438d8ce360 con 0x56438daf3690 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x56438d8cb2d0 con 0x56438daf2d70 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 10 client.0 _ll_register_callbacks cb 0x56438da90820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438dadf490 0x56438dadf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438dadf490 0x56438dadf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52122/0 (socket says 172.21.15.110:52122) 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 -- 172.21.15.110:0/4242032944 learned_addr learned my addr 172.21.15.110:0/4242032944 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7b93e00640 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf3690 0x56438daf93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daf2d70 0x56438daf8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 -- 172.21.15.110:0/4242032944 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daf2d70 msgr2=0x56438daf8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daf2d70 0x56438daf8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 -- 172.21.15.110:0/4242032944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf3690 msgr2=0x56438daf93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf3690 0x56438daf93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 -- 172.21.15.110:0/4242032944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438d89a120 con 0x56438dadf490 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba1600640 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438dadf490 0x56438dadf870 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7b94007da0 tx=0x7f7b94010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafdcce62d3a9f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7b8a000640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7b8a000640 1 -- 172.21.15.110:0/4242032944 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9401a020 con 0x56438dadf490 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7b8a000640 1 -- 172.21.15.110:0/4242032944 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f7b94010e20 con 0x56438dadf490 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- 172.21.15.110:0/4242032944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8400de90 con 0x56438dadf490 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7b8a000640 1 -- 172.21.15.110:0/4242032944 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7b94024050 con 0x56438dadf490 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 10 client.7824 fetch_fsmap learned FSMap version 88 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.274+0000 7f7ba69e0000 1 -- 172.21.15.110:0/4242032944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x56438d893450 con 0x56438dadf490 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7b8a000640 1 -- 172.21.15.110:0/4242032944 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8400de90 con 0x56438dadf490 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7b8a000640 1 -- 172.21.15.110:0/4242032944 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 88) ==== 3502+0+0 (secure 0 0 0) 0x7f7b94027020 con 0x56438dadf490 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 10 client.7824 fetch_fsmap finished waiting for FSMap version 88 2024-09-10T10:04:35.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 10 client.7824 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:04:35.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 20 client.7824 populate_metadata read hostname 'smithi110' 2024-09-10T10:04:35.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] conn(0x56438dae0280 0x56438dae0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 4 client.7824 mds_command: new command op to 7686 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:35.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 -- 172.21.15.110:0/4242032944 --> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56438d88dc00 con 0x56438dae0280 2024-09-10T10:04:35.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] conn(0x56438dae0280 0x56438dae0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] conn(0x56438dae0280 0x56438dae0660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8447acfdea400d01 server_cookie=4205e263988da595 in_seq=0 out_seq=0 2024-09-10T10:04:35.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7b8a000640 10 client.7824 ms_handle_connect on v2:172.21.15.110:6833/1881265533 2024-09-10T10:04:35.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7b8a000640 1 -- 172.21.15.110:0/4242032944 <== mds.0 v2:172.21.15.110:6833/1881265533 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56438d88dc00 con 0x56438dae0280 2024-09-10T10:04:35.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7b8a000640 10 client.7824 handle_command_reply: tid=0 2024-09-10T10:04:35.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 client.7824 shutdown 2024-09-10T10:04:35.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 -- 172.21.15.110:0/4242032944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438dadf490 msgr2=0x56438dadf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438dadf490 0x56438dadf870 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7b94007da0 tx=0x7f7b94010590 comp rx=0 tx=0).stop 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 -- 172.21.15.110:0/4242032944 shutdown_connections 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] conn(0x56438dae0280 0x56438dae0660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438dadf490 0x56438dadf870 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf3690 0x56438daf93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/4242032944 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daf2d70 0x56438daf8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7b8a000640 0 client.7824 ms_handle_reset on v2:172.21.15.110:6833/1881265533 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 -- 172.21.15.110:0/4242032944 >> 172.21.15.110:0/4242032944 conn(0x56438daa96a0 msgr2=0x56438daaa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 -- 172.21.15.110:0/4242032944 shutdown_connections 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 1 -- 172.21.15.110:0/4242032944 wait complete. 2024-09-10T10:04:35.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.278+0000 7f7ba69e0000 20 client.7824 trim_cache size 0 max 16384 2024-09-10T10:04:35.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 Processor -- start 2024-09-10T10:04:35.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- start start 2024-09-10T10:04:35.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438db09260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438dadf490 0x56438db097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf2d70 0x56438daf7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c0724b0 con 0x56438daf2d70 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c161dd0 con 0x56438dadf490 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9c162420 con 0x56438daa96a0 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf2d70 0x56438daf7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf2d70 0x56438daf7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52126/0 (socket says 172.21.15.110:52126) 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 -- 172.21.15.110:0/217769447 learned_addr learned my addr 172.21.15.110:0/217769447 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438db09260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 -- 172.21.15.110:0/217769447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 msgr2=0x56438db09260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438db09260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 -- 172.21.15.110:0/217769447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438dadf490 msgr2=0x56438db097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438dadf490 0x56438db097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 -- 172.21.15.110:0/217769447 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438d87afd0 con 0x56438daf2d70 2024-09-10T10:04:35.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438db09260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:35.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba1600640 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf2d70 0x56438daf7ca0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f7b94000f90 tx=0x7f7b94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b640e1fe62695734 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7b90c00640 1 -- 172.21.15.110:0/217769447 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9402c020 con 0x56438daf2d70 2024-09-10T10:04:35.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7b90c00640 1 -- 172.21.15.110:0/217769447 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b94010e20 con 0x56438daf2d70 2024-09-10T10:04:35.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7b90c00640 1 -- 172.21.15.110:0/217769447 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9401a050 con 0x56438daf2d70 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- 172.21.15.110:0/217769447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf2d70 msgr2=0x56438daf7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf2d70 0x56438daf7ca0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f7b94000f90 tx=0x7f7b94016000 comp rx=0 tx=0).stop 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- 172.21.15.110:0/217769447 shutdown_connections 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daf2d70 0x56438daf7ca0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438dadf490 0x56438db097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/217769447 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438daa96a0 0x56438db09260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- 172.21.15.110:0/217769447 >> 172.21.15.110:0/217769447 conn(0x56438dadee20 msgr2=0x56438daa3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- 172.21.15.110:0/217769447 shutdown_connections 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- 172.21.15.110:0/217769447 wait complete. 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 -- start start 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.298+0000 7f7ba69e0000 1 Processor -- start 2024-09-10T10:04:35.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daa96a0 0x56438db66710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438dadf490 0x56438db66c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 0x56438dafccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56438d8f2f80 con 0x56438daa96a0 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56438d8abca0 con 0x56438daf2d70 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x56438d8f82a0 con 0x56438dadf490 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 10 client.0 _ll_register_callbacks cb 0x7f7b94009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daa96a0 0x56438db66710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daa96a0 0x56438db66710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52142/0 (socket says 172.21.15.110:52142) 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 -- 172.21.15.110:0/830308674 learned_addr learned my addr 172.21.15.110:0/830308674 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 -- 172.21.15.110:0/830308674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438dadf490 msgr2=0x56438db66c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438dadf490 0x56438db66c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 -- 172.21.15.110:0/830308674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 msgr2=0x56438dafccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 0x56438dafccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 -- 172.21.15.110:0/830308674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438d976750 con 0x56438daa96a0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba0c00640 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daa96a0 0x56438db66710 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c01ca00 tx=0x7f7b8c0ec3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684ec611f6222a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 1 -- 172.21.15.110:0/830308674 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0f8d20 con 0x56438daa96a0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 1 -- 172.21.15.110:0/830308674 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f7b8c0ee440 con 0x56438daa96a0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- 172.21.15.110:0/830308674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b94010e20 con 0x56438daa96a0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 1 -- 172.21.15.110:0/830308674 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c0ec7b0 con 0x56438daa96a0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 10 client.7830 fetch_fsmap learned FSMap version 88 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- 172.21.15.110:0/830308674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x56438d881a30 con 0x56438daa96a0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 1 -- 172.21.15.110:0/830308674 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b94010e20 con 0x56438daa96a0 2024-09-10T10:04:35.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 1 -- 172.21.15.110:0/830308674 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 88) ==== 3502+0+0 (secure 0 0 0) 0x7f7b8c0f4120 con 0x56438daa96a0 2024-09-10T10:04:35.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 10 client.7830 fetch_fsmap finished waiting for FSMap version 88 2024-09-10T10:04:35.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 10 client.7830 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:04:35.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 20 client.7830 populate_metadata read hostname 'smithi110' 2024-09-10T10:04:35.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] conn(0x56438daf3690 0x56438dafd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 4 client.7830 mds_command: new command op to 7686 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:04:35.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- 172.21.15.110:0/830308674 --> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] -- command(tid 0: {"prefix": "status"}) -- 0x56438d9013a0 con 0x56438daf3690 2024-09-10T10:04:35.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b93e00640 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] conn(0x56438daf3690 0x56438dafd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b93e00640 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] conn(0x56438daf3690 0x56438dafd200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec9f7533ff3a70f server_cookie=838ac1630713e217 in_seq=0 out_seq=0 2024-09-10T10:04:35.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 10 client.7830 ms_handle_connect on v2:172.21.15.110:6833/1881265533 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 1 -- 172.21.15.110:0/830308674 <== mds.0 v2:172.21.15.110:6833/1881265533 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56438d9013a0 con 0x56438daf3690 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 10 client.7830 handle_command_reply: tid=0 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 client.7830 shutdown 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- 172.21.15.110:0/830308674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daa96a0 msgr2=0x56438db66710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daa96a0 0x56438db66710 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c01ca00 tx=0x7f7b8c0ec3e0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- 172.21.15.110:0/830308674 shutdown_connections 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.110:6833/1881265533,v1:172.21.15.110:6835/1881265533] conn(0x56438daf3690 0x56438dafd200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56438daf2d70 0x56438dafccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x56438dadf490 0x56438db66c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7b8a000640 0 client.7830 ms_handle_reset on v2:172.21.15.110:6833/1881265533 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 --2- 172.21.15.110:0/830308674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438daa96a0 0x56438db66710 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- 172.21.15.110:0/830308674 >> 172.21.15.110:0/830308674 conn(0x56438dadee20 msgr2=0x56438db08250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- 172.21.15.110:0/830308674 shutdown_connections 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 1 -- 172.21.15.110:0/830308674 wait complete. 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.302+0000 7f7ba69e0000 20 client.7830 trim_cache size 0 max 16384 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 7686, 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:04:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:04:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:04:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 5.8691641309999998, 2024-09-10T10:04:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 88, 2024-09-10T10:04:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 130, 2024-09-10T10:04:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 130, 2024-09-10T10:04:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 10.777012156 2024-09-10T10:04:35.314 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 -- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b78067e60 msgr2=0x7f7b7806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b78067e60 0x7f7b7806a320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0b4c70 tx=0x7f7b84009040 comp rx=0 tx=0).stop 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 -- 172.21.15.110:0/970566922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0b3f40 msgr2=0x7f7b9c165a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0b3f40 0x7f7b9c165a30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0ae040 tx=0x7f7b8c019230 comp rx=0 tx=0).stop 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 -- 172.21.15.110:0/970566922 shutdown_connections 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b78067e60 0x7f7b7806a320 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b9c0cfb90 0x7f7b9c150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b9c0cdcb0 0x7f7b9c150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 --2- 172.21.15.110:0/970566922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c0b3f40 0x7f7b9c165a30 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 -- 172.21.15.110:0/970566922 >> 172.21.15.110:0/970566922 conn(0x7f7b9c062d10 msgr2=0x7f7b9c1669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 -- 172.21.15.110:0/970566922 shutdown_connections 2024-09-10T10:04:35.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.306+0000 7f7ba3600640 1 -- 172.21.15.110:0/970566922 wait complete. 2024-09-10T10:04:35.327 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:04:35.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.458+0000 7f0917400640 1 Processor -- start 2024-09-10T10:04:35.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.458+0000 7f0917400640 1 -- start start 2024-09-10T10:04:35.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0917400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f091006f1e0 0x7f0910166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0917400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0910155d50 con 0x7f0910155970 2024-09-10T10:04:35.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0917400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09100b34d0 con 0x7f091015c560 2024-09-10T10:04:35.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0917400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f09100b4570 con 0x7f091006f1e0 2024-09-10T10:04:35.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0914a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0910155970 0x7f09100cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55888/0 (socket says 172.21.15.110:55888) 2024-09-10T10:04:35.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0914a00640 1 -- 172.21.15.110:0/742817918 learned_addr learned my addr 172.21.15.110:0/742817918 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:35.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0915400640 1 --2- 172.21.15.110:0/742817918 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f091006f1e0 0x7f0910166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f090f400640 1 -- 172.21.15.110:0/742817918 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661453375 0 0) 0x7f0910155d50 con 0x7f0910155970 2024-09-10T10:04:35.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f090f400640 1 -- 172.21.15.110:0/742817918 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08f4003530 con 0x7f0910155970 2024-09-10T10:04:35.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0915400640 1 -- 172.21.15.110:0/742817918 >> v1:172.21.15.145:6789/0 conn(0x7f091015c560 legacy=0x7f091006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0915400640 1 -- 172.21.15.110:0/742817918 >> v1:172.21.15.110:6789/0 conn(0x7f0910155970 legacy=0x7f09100cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.462+0000 7f0915400640 1 -- 172.21.15.110:0/742817918 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09100b37b0 con 0x7f091006f1e0 2024-09-10T10:04:35.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0915400640 1 --2- 172.21.15.110:0/742817918 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f091006f1e0 0x7f0910166020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f09080097f0 tx=0x7f090801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8a8f24f54d78416 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f090f400640 1 -- 172.21.15.110:0/742817918 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080aa610 con 0x7f091006f1e0 2024-09-10T10:04:35.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f090f400640 1 -- 172.21.15.110:0/742817918 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09080af070 con 0x7f091006f1e0 2024-09-10T10:04:35.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f090f400640 1 -- 172.21.15.110:0/742817918 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080aa9d0 con 0x7f091006f1e0 2024-09-10T10:04:35.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 -- 172.21.15.110:0/742817918 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f091006f1e0 msgr2=0x7f0910166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 --2- 172.21.15.110:0/742817918 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f091006f1e0 0x7f0910166020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f09080097f0 tx=0x7f090801c4b0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 -- 172.21.15.110:0/742817918 shutdown_connections 2024-09-10T10:04:35.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 --2- 172.21.15.110:0/742817918 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f091006f1e0 0x7f0910166020 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 -- 172.21.15.110:0/742817918 >> 172.21.15.110:0/742817918 conn(0x7f091006bc50 msgr2=0x7f091006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 -- 172.21.15.110:0/742817918 shutdown_connections 2024-09-10T10:04:35.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 -- 172.21.15.110:0/742817918 wait complete. 2024-09-10T10:04:35.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 Processor -- start 2024-09-10T10:04:35.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.466+0000 7f0917400640 1 -- start start 2024-09-10T10:04:35.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f091006f1e0 0x7f0910146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0910155970 0x7f091014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f091015c560 0x7f09100ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0910168f00 con 0x7f091015c560 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0910168d80 con 0x7f091006f1e0 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0910169080 con 0x7f0910155970 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f091015c560 0x7f09100ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f091015c560 0x7f09100ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52144/0 (socket says 172.21.15.110:52144) 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 -- 172.21.15.110:0/1613668888 learned_addr learned my addr 172.21.15.110:0/1613668888 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0914a00640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f091006f1e0 0x7f0910146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f090fe00640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0910155970 0x7f091014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 -- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0910155970 msgr2=0x7f091014fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0910155970 0x7f091014fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 -- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f091006f1e0 msgr2=0x7f0910146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f091006f1e0 0x7f0910146910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 -- 172.21.15.110:0/1613668888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091014aca0 con 0x7f091015c560 2024-09-10T10:04:35.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f090fe00640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0910155970 0x7f091014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:35.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0915400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f091015c560 0x7f09100ae840 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f09080097f0 tx=0x7f090801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f090cc00640 1 -- 172.21.15.110:0/1613668888 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080be610 con 0x7f091015c560 2024-09-10T10:04:35.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f090cc00640 1 -- 172.21.15.110:0/1613668888 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09080be7b0 con 0x7f091015c560 2024-09-10T10:04:35.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f090cc00640 1 -- 172.21.15.110:0/1613668888 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080beba0 con 0x7f091015c560 2024-09-10T10:04:35.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091014e7e0 con 0x7f091015c560 2024-09-10T10:04:35.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091014b220 con 0x7f091015c560 2024-09-10T10:04:35.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09100cc8c0 con 0x7f091015c560 2024-09-10T10:04:35.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.470+0000 7f090cc00640 1 -- 172.21.15.110:0/1613668888 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f09080bc020 con 0x7f091015c560 2024-09-10T10:04:35.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.474+0000 7f090cc00640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08f0067e60 0x7f08f006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.474+0000 7f090cc00640 1 -- 172.21.15.110:0/1613668888 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f0908120910 con 0x7f091015c560 2024-09-10T10:04:35.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.474+0000 7f0914a00640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08f0067e60 0x7f08f006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.474+0000 7f0914a00640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08f0067e60 0x7f08f006a320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0900005c10 tx=0x7f0900009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.478+0000 7f090cc00640 1 -- 172.21.15.110:0/1613668888 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f09080c5030 con 0x7f091015c560 2024-09-10T10:04:35.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.670+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0910073100 con 0x7f091015c560 2024-09-10T10:04:35.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.670+0000 7f090cc00640 1 -- 172.21.15.110:0/1613668888 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7f09080a7070 con 0x7f091015c560 2024-09-10T10:04:35.675 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.670+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08f0067e60 msgr2=0x7f08f006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.670+0000 7f0917400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08f0067e60 0x7f08f006a320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0900005c10 tx=0x7f0900009040 comp rx=0 tx=0).stop 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.670+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f091015c560 msgr2=0x7f09100ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.670+0000 7f0917400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f091015c560 0x7f09100ae840 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f09080097f0 tx=0x7f090801acf0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.674+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 shutdown_connections 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.674+0000 7f0917400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f08f0067e60 0x7f08f006a320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.674+0000 7f0917400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f091015c560 0x7f09100ae840 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.674+0000 7f0917400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0910155970 0x7f091014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.674+0000 7f0917400640 1 --2- 172.21.15.110:0/1613668888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f091006f1e0 0x7f0910146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.674+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 >> 172.21.15.110:0/1613668888 conn(0x7f091006bc50 msgr2=0x7f09100710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.674+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 shutdown_connections 2024-09-10T10:04:35.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35.674+0000 7f0917400640 1 -- 172.21.15.110:0/1613668888 wait complete. 2024-09-10T10:04:35.696 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:04:35.696 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:04:35.696 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:04:35.696 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:04:35.696 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:35.697 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:04:35.697 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:04:35.697 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:04:35.697 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:04:35.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:35.697 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:04:35.706 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:04:35.706 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:04:35.706 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:04:35.706 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:35.706 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:04:35.706 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42099sec preferred_lft 42099sec 2024-09-10T10:04:35.707 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:04:35.708 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:04:35.709 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:04:35.709 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:35.709 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:04:35.709 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:04:35.709 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:04:35.709 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:35.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:35.710 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:04:35.774 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:04:35.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:35.776 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:04:35.827 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:04:35.827 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:04:35.827 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:04:35.827 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:04:35.827 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:04:35.827 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:04:35.828 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:04:35.828 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:35.828 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:35.828 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:35.828 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:04:35.828 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:04:35.828 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:04:35.828 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:04:35.828 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:04:35.829 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:04:35.829 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:04:35.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:04:35.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:35.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:35.961 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:04:36.013 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:04:36.014 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:04:36.014 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:04:36.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:36.014 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:04:36.079 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:04:36.079 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:36.128 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:04:36.129 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:04:36.129 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:36.183 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:04:36.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:36.184 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:36.272 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:36.272 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:04:36.284 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:36.284 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:04:36.285 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:04:36.285 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:04:36.285 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:04:36.288 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:04:36.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:36.289 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:36.309 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:04:36.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:04:36.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:36.418 INFO:teuthology.orchestra.run.smithi110.stdout:client7839 2024-09-10T10:04:36.418 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:36.418 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:36.468 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:04:36.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.642+0000 7fbc6e600640 1 Processor -- start 2024-09-10T10:04:36.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.642+0000 7fbc6e600640 1 -- start start 2024-09-10T10:04:36.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc6e600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbc6806f1e0 0x7fbc68166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc6e600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc68155d50 con 0x7fbc6815c560 2024-09-10T10:04:36.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc6e600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc680b34d0 con 0x7fbc68155970 2024-09-10T10:04:36.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc6e600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbc680b4570 con 0x7fbc6806f1e0 2024-09-10T10:04:36.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc66a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fbc6815c560 0x7fbc6806ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55900/0 (socket says 172.21.15.110:55900) 2024-09-10T10:04:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc66a00640 1 -- 172.21.15.110:0/2835546592 learned_addr learned my addr 172.21.15.110:0/2835546592 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc67e00640 1 --2- 172.21.15.110:0/2835546592 >> v2:172.21.15.110:3300/0 conn(0x7fbc6806f1e0 0x7fbc68166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc67e00640 1 -- 172.21.15.110:0/2835546592 >> v1:172.21.15.145:6789/0 conn(0x7fbc68155970 legacy=0x7fbc680cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc67e00640 1 -- 172.21.15.110:0/2835546592 >> v1:172.21.15.110:6789/0 conn(0x7fbc6815c560 legacy=0x7fbc6806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.646+0000 7fbc67e00640 1 -- 172.21.15.110:0/2835546592 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc680b37b0 con 0x7fbc6806f1e0 2024-09-10T10:04:36.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc67e00640 1 --2- 172.21.15.110:0/2835546592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc6806f1e0 0x7fbc68166020 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbc600097c0 tx=0x7fbc6001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be919e4d9a1c7d37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc66000640 1 -- 172.21.15.110:0/2835546592 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600ab430 con 0x7fbc6806f1e0 2024-09-10T10:04:36.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc66000640 1 -- 172.21.15.110:0/2835546592 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc600b0070 con 0x7fbc6806f1e0 2024-09-10T10:04:36.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc66000640 1 -- 172.21.15.110:0/2835546592 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600ab7f0 con 0x7fbc6806f1e0 2024-09-10T10:04:36.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- 172.21.15.110:0/2835546592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc6806f1e0 msgr2=0x7fbc68166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 --2- 172.21.15.110:0/2835546592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc6806f1e0 0x7fbc68166020 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbc600097c0 tx=0x7fbc6001c430 comp rx=0 tx=0).stop 2024-09-10T10:04:36.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- 172.21.15.110:0/2835546592 shutdown_connections 2024-09-10T10:04:36.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 --2- 172.21.15.110:0/2835546592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc6806f1e0 0x7fbc68166020 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- 172.21.15.110:0/2835546592 >> 172.21.15.110:0/2835546592 conn(0x7fbc6806bc50 msgr2=0x7fbc6806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:36.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- 172.21.15.110:0/2835546592 shutdown_connections 2024-09-10T10:04:36.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- 172.21.15.110:0/2835546592 wait complete. 2024-09-10T10:04:36.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 Processor -- start 2024-09-10T10:04:36.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- start start 2024-09-10T10:04:36.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc6806f1e0 0x7fbc6814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc68155970 0x7fbc6814feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc6815c560 0x7fbc68151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc68168f00 con 0x7fbc68155970 2024-09-10T10:04:36.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc68168d80 con 0x7fbc6806f1e0 2024-09-10T10:04:36.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc6e600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbc68169080 con 0x7fbc6815c560 2024-09-10T10:04:36.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc66a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc68155970 0x7fbc6814feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc66a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc68155970 0x7fbc6814feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52166/0 (socket says 172.21.15.110:52166) 2024-09-10T10:04:36.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.650+0000 7fbc66a00640 1 -- 172.21.15.110:0/2105333829 learned_addr learned my addr 172.21.15.110:0/2105333829 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:36.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc67e00640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc6815c560 0x7fbc68151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc67400640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc6806f1e0 0x7fbc6814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc66a00640 1 -- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc6815c560 msgr2=0x7fbc68151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc66a00640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc6815c560 0x7fbc68151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc66a00640 1 -- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc6806f1e0 msgr2=0x7fbc6814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc66a00640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc6806f1e0 0x7fbc6814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc66a00640 1 -- 172.21.15.110:0/2105333829 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6814d120 con 0x7fbc68155970 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc67400640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc6806f1e0 0x7fbc6814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc66a00640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc68155970 0x7fbc6814feb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400c420 tx=0x7fbc5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc5f400640 1 -- 172.21.15.110:0/2105333829 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54017070 con 0x7fbc68155970 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc5f400640 1 -- 172.21.15.110:0/2105333829 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc54005cf0 con 0x7fbc68155970 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc5f400640 1 -- 172.21.15.110:0/2105333829 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc540072f0 con 0x7fbc68155970 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6814d400 con 0x7fbc68155970 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6814d9c0 con 0x7fbc68155970 2024-09-10T10:04:36.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6814dda0 con 0x7fbc68155970 2024-09-10T10:04:36.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc5f400640 1 -- 172.21.15.110:0/2105333829 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fbc5401a070 con 0x7fbc68155970 2024-09-10T10:04:36.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc5f400640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc48067e90 0x7fbc4806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:36.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.654+0000 7fbc5f400640 1 -- 172.21.15.110:0/2105333829 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fbc54085ac0 con 0x7fbc68155970 2024-09-10T10:04:36.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.658+0000 7fbc67400640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc48067e90 0x7fbc4806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:36.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.658+0000 7fbc67400640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc48067e90 0x7fbc4806a350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc58007e90 tx=0x7fbc580073a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:36.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.658+0000 7fbc5f400640 1 -- 172.21.15.110:0/2105333829 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbc6814dda0 con 0x7fbc68155970 2024-09-10T10:04:36.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.850+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbc680cc8c0 con 0x7fbc68155970 2024-09-10T10:04:36.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.850+0000 7fbc5f400640 1 -- 172.21.15.110:0/2105333829 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7fbc54055d10 con 0x7fbc68155970 2024-09-10T10:04:36.855 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:04:36.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc48067e90 msgr2=0x7fbc4806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc48067e90 0x7fbc4806a350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc58007e90 tx=0x7fbc580073a0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc68155970 msgr2=0x7fbc6814feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:36.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc68155970 0x7fbc6814feb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400c420 tx=0x7fbc5400c8e0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 shutdown_connections 2024-09-10T10:04:36.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbc48067e90 0x7fbc4806a350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbc6815c560 0x7fbc68151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc68155970 0x7fbc6814feb0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 --2- 172.21.15.110:0/2105333829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc6806f1e0 0x7fbc6814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:36.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 >> 172.21.15.110:0/2105333829 conn(0x7fbc6806bc50 msgr2=0x7fbc6815afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:36.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 shutdown_connections 2024-09-10T10:04:36.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:36.854+0000 7fbc6e600640 1 -- 172.21.15.110:0/2105333829 wait complete. 2024-09-10T10:04:36.876 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:04:36.876 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:04:36.876 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:04:36.877 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:04:36.877 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:36.877 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:04:36.877 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:04:36.877 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:04:36.877 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:04:36.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:36.878 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:04:36.888 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:04:36.889 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42094sec preferred_lft 42094sec 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:36.890 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:04:36.891 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:04:36.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:36.892 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:04:36.953 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:04:36.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:36.955 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:04:37.008 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:04:37.008 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:04:37.009 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:04:37.009 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:04:37.010 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:04:37.010 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:04:37.010 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:04:37.010 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:04:37.010 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:04:37.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:04:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:04:37.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:37.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:37.155 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:04:37.165 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:04:37.165 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:04:37.165 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:04:37.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:37.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:04:37.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:04:37.225 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:37.277 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:04:37.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:04:37.277 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:37.332 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:04:37.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:37.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:37.426 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:37.426 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:04:37.438 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:04:37.439 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:04:37.439 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:04:37.439 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:04:37.439 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:04:37.442 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:04:37.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:37.442 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:37.464 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:04:37.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:04:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:04:37.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:04:37.560 INFO:teuthology.orchestra.run.smithi145.stdout:client6545 2024-09-10T10:04:37.561 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:37.561 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:37.569 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:04:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.706+0000 7f8f4a400640 1 Processor -- start 2024-09-10T10:04:37.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.706+0000 7f8f4a400640 1 -- start start 2024-09-10T10:04:37.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f4a400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8f4406f1e0 0x7f8f44166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f4a400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f44155d50 con 0x7f8f44155970 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f4a400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f440b34d0 con 0x7f8f4415c560 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f4a400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8f440b4570 con 0x7f8f4406f1e0 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f43400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f8f44155970 0x7f8f440cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55910/0 (socket says 172.21.15.110:55910) 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f43400640 1 -- 172.21.15.110:0/1466239220 learned_addr learned my addr 172.21.15.110:0/1466239220 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f43e00640 1 --2- 172.21.15.110:0/1466239220 >> v2:172.21.15.145:3300/0 conn(0x7f8f4406f1e0 0x7f8f44166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f42a00640 1 -- 172.21.15.110:0/1466239220 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359493132 0 0) 0x7f8f44155d50 con 0x7f8f44155970 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f42a00640 1 -- 172.21.15.110:0/1466239220 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f28003530 con 0x7f8f44155970 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f43e00640 1 -- 172.21.15.110:0/1466239220 >> v1:172.21.15.145:6789/0 conn(0x7f8f4415c560 legacy=0x7f8f4406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:37.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f43e00640 1 -- 172.21.15.110:0/1466239220 >> v1:172.21.15.110:6789/0 conn(0x7f8f44155970 legacy=0x7f8f440cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:37.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.710+0000 7f8f43e00640 1 -- 172.21.15.110:0/1466239220 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f440b37b0 con 0x7f8f4406f1e0 2024-09-10T10:04:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f43e00640 1 --2- 172.21.15.110:0/1466239220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f4406f1e0 0x7f8f44166020 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009d90 tx=0x7f8f3c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=291af294abecc9e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f42a00640 1 -- 172.21.15.110:0/1466239220 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0b2070 con 0x7f8f4406f1e0 2024-09-10T10:04:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f42a00640 1 -- 172.21.15.110:0/1466239220 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f3c0a5a00 con 0x7f8f4406f1e0 2024-09-10T10:04:37.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f42a00640 1 -- 172.21.15.110:0/1466239220 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0a5dd0 con 0x7f8f4406f1e0 2024-09-10T10:04:37.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 -- 172.21.15.110:0/1466239220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f4406f1e0 msgr2=0x7f8f44166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:37.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 --2- 172.21.15.110:0/1466239220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f4406f1e0 0x7f8f44166020 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009d90 tx=0x7f8f3c01c940 comp rx=0 tx=0).stop 2024-09-10T10:04:37.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 -- 172.21.15.110:0/1466239220 shutdown_connections 2024-09-10T10:04:37.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 --2- 172.21.15.110:0/1466239220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f4406f1e0 0x7f8f44166020 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:37.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 -- 172.21.15.110:0/1466239220 >> 172.21.15.110:0/1466239220 conn(0x7f8f4406bc50 msgr2=0x7f8f4406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 -- 172.21.15.110:0/1466239220 shutdown_connections 2024-09-10T10:04:37.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 -- 172.21.15.110:0/1466239220 wait complete. 2024-09-10T10:04:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 Processor -- start 2024-09-10T10:04:37.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.714+0000 7f8f4a400640 1 -- start start 2024-09-10T10:04:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f4406f1e0 0x7f8f44148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f44155970 0x7f8f44154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8f4415c560 0x7f8f44154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f44168f00 con 0x7f8f4406f1e0 2024-09-10T10:04:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8f44168d80 con 0x7f8f44155970 2024-09-10T10:04:37.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8f44169080 con 0x7f8f4415c560 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f4406f1e0 0x7f8f44148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f4406f1e0 0x7f8f44148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52174/0 (socket says 172.21.15.110:52174) 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 -- 172.21.15.110:0/2000242861 learned_addr learned my addr 172.21.15.110:0/2000242861 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f3be00640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f44155970 0x7f8f44154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43e00640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8f4415c560 0x7f8f44154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 -- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8f4415c560 msgr2=0x7f8f44154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8f4415c560 0x7f8f44154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 -- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f44155970 msgr2=0x7f8f44154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f44155970 0x7f8f44154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 -- 172.21.15.110:0/2000242861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4414ce10 con 0x7f8f4406f1e0 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f43400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f4406f1e0 0x7f8f44148e80 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400c420 tx=0x7f8f3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f3b400640 1 -- 172.21.15.110:0/2000242861 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34017070 con 0x7f8f4406f1e0 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f3b400640 1 -- 172.21.15.110:0/2000242861 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f34005cf0 con 0x7f8f4406f1e0 2024-09-10T10:04:37.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f3b400640 1 -- 172.21.15.110:0/2000242861 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f340072d0 con 0x7f8f4406f1e0 2024-09-10T10:04:37.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f44156bc0 con 0x7f8f4406f1e0 2024-09-10T10:04:37.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4414d0b0 con 0x7f8f4406f1e0 2024-09-10T10:04:37.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f440cc8c0 con 0x7f8f4406f1e0 2024-09-10T10:04:37.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.718+0000 7f8f3b400640 1 -- 172.21.15.110:0/2000242861 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f8f3401a070 con 0x7f8f4406f1e0 2024-09-10T10:04:37.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.722+0000 7f8f3b400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8f20067e60 0x7f8f2006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:37.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.722+0000 7f8f3b400640 1 -- 172.21.15.110:0/2000242861 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f8f34085a60 con 0x7f8f4406f1e0 2024-09-10T10:04:37.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.722+0000 7f8f3be00640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8f20067e60 0x7f8f2006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:37.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.722+0000 7f8f3be00640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8f20067e60 0x7f8f2006a320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f440729b0 tx=0x7f8f2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:37.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.726+0000 7f8f3b400640 1 -- 172.21.15.110:0/2000242861 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8f340556f0 con 0x7f8f4406f1e0 2024-09-10T10:04:37.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:37.918+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f8f4414fba0 con 0x7f8f4406f1e0 2024-09-10T10:04:38.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.486+0000 7f8f3b400640 1 -- 172.21.15.110:0/2000242861 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v89) ==== 136+0+0 (secure 0 0 0) 0x7f8f340599f0 con 0x7f8f4406f1e0 2024-09-10T10:04:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8f20067e60 msgr2=0x7f8f2006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8f20067e60 0x7f8f2006a320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f440729b0 tx=0x7f8f2c009040 comp rx=0 tx=0).stop 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f4406f1e0 msgr2=0x7f8f44148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f4406f1e0 0x7f8f44148e80 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400c420 tx=0x7f8f3400c8e0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 shutdown_connections 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8f20067e60 0x7f8f2006a320 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8f4415c560 0x7f8f44154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f44155970 0x7f8f44154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 --2- 172.21.15.110:0/2000242861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f4406f1e0 0x7f8f44148e80 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 >> 172.21.15.110:0/2000242861 conn(0x7f8f4406bc50 msgr2=0x7f8f44074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 shutdown_connections 2024-09-10T10:04:38.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.490+0000 7f8f4a400640 1 -- 172.21.15.110:0/2000242861 wait complete. 2024-09-10T10:04:38.514 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:04:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.646+0000 7f4142e00640 1 Processor -- start 2024-09-10T10:04:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.646+0000 7f4142e00640 1 -- start start 2024-09-10T10:04:38.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f4142e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f413c0cdcb0 0x7f413c167160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:38.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f4142e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413c15d4d0 con 0x7f413c0b3f40 2024-09-10T10:04:38.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f4142e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413c15d7b0 con 0x7f413c0cfb90 2024-09-10T10:04:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f4142e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f413c06d130 con 0x7f413c0cdcb0 2024-09-10T10:04:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f413b400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f413c0b3f40 0x7f413c0cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47570/0 (socket says 172.21.15.110:47570) 2024-09-10T10:04:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f413b400640 1 -- 172.21.15.110:0/2466190694 learned_addr learned my addr 172.21.15.110:0/2466190694 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f4140e00640 1 --2- 172.21.15.110:0/2466190694 >> v2:172.21.15.110:3300/0 conn(0x7f413c0cdcb0 0x7f413c167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f4140e00640 1 -- 172.21.15.110:0/2466190694 >> v1:172.21.15.145:6789/0 conn(0x7f413c0cfb90 legacy=0x7f413c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f4140e00640 1 -- 172.21.15.110:0/2466190694 >> v1:172.21.15.110:6789/0 conn(0x7f413c0b3f40 legacy=0x7f413c0cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.650+0000 7f4140e00640 1 -- 172.21.15.110:0/2466190694 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c06d380 con 0x7f413c0cdcb0 2024-09-10T10:04:38.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4140e00640 1 --2- 172.21.15.110:0/2466190694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c167160 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f41340097c0 tx=0x7f413401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea906cbdb57c2aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:38.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f413aa00640 1 -- 172.21.15.110:0/2466190694 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41340b0070 con 0x7f413c0cdcb0 2024-09-10T10:04:38.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f413aa00640 1 -- 172.21.15.110:0/2466190694 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41340ab600 con 0x7f413c0cdcb0 2024-09-10T10:04:38.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f413aa00640 1 -- 172.21.15.110:0/2466190694 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41340ab9d0 con 0x7f413c0cdcb0 2024-09-10T10:04:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- 172.21.15.110:0/2466190694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 msgr2=0x7f413c167160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 --2- 172.21.15.110:0/2466190694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c167160 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f41340097c0 tx=0x7f413401c410 comp rx=0 tx=0).stop 2024-09-10T10:04:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- 172.21.15.110:0/2466190694 shutdown_connections 2024-09-10T10:04:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 --2- 172.21.15.110:0/2466190694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c167160 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- 172.21.15.110:0/2466190694 >> 172.21.15.110:0/2466190694 conn(0x7f413c062d10 msgr2=0x7f413c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- 172.21.15.110:0/2466190694 shutdown_connections 2024-09-10T10:04:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- 172.21.15.110:0/2466190694 wait complete. 2024-09-10T10:04:38.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 Processor -- start 2024-09-10T10:04:38.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- start start 2024-09-10T10:04:38.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0b3f40 0x7f413c14dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:38.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:38.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0cfb90 0x7f413c152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:38.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f413c16a250 con 0x7f413c0cdcb0 2024-09-10T10:04:38.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f413c16a0d0 con 0x7f413c0cfb90 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4142e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f413c16a3d0 con 0x7f413c0b3f40 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f413b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f413b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55000/0 (socket says 172.21.15.110:55000) 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f413b400640 1 -- 172.21.15.110:0/3007831456 learned_addr learned my addr 172.21.15.110:0/3007831456 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f413be00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0b3f40 0x7f413c14dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.654+0000 7f4140e00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0cfb90 0x7f413c152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f413b400640 1 -- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0b3f40 msgr2=0x7f413c14dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f413b400640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0b3f40 0x7f413c14dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f413b400640 1 -- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0cfb90 msgr2=0x7f413c152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f413b400640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0cfb90 0x7f413c152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f413b400640 1 -- 172.21.15.110:0/3007831456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c154fb0 con 0x7f413c0cdcb0 2024-09-10T10:04:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f413b400640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c150b10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f412800c4d0 tx=0x7f412800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4133e00640 1 -- 172.21.15.110:0/3007831456 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128017070 con 0x7f413c0cdcb0 2024-09-10T10:04:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4133e00640 1 -- 172.21.15.110:0/3007831456 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4128005cf0 con 0x7f413c0cdcb0 2024-09-10T10:04:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c151390 con 0x7f413c0cdcb0 2024-09-10T10:04:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4133e00640 1 -- 172.21.15.110:0/3007831456 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128007270 con 0x7f413c0cdcb0 2024-09-10T10:04:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c151950 con 0x7f413c0cdcb0 2024-09-10T10:04:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413c0cd5a0 con 0x7f413c0cdcb0 2024-09-10T10:04:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4133e00640 1 -- 172.21.15.110:0/3007831456 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f412801a070 con 0x7f413c0cdcb0 2024-09-10T10:04:38.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4133e00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f411c067e60 0x7f411c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:38.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f4133e00640 1 -- 172.21.15.110:0/3007831456 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f41280859c0 con 0x7f413c0cdcb0 2024-09-10T10:04:38.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.658+0000 7f413be00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f411c067e60 0x7f411c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:38.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.662+0000 7f413be00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f411c067e60 0x7f411c06a320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f412c001e00 tx=0x7f412c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:38.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.666+0000 7f4133e00640 1 -- 172.21.15.110:0/3007831456 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f413c154fb0 con 0x7f413c0cdcb0 2024-09-10T10:04:38.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.842+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f413c15e3a0 con 0x7f411c067e60 2024-09-10T10:04:38.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4133e00640 1 -- 172.21.15.110:0/3007831456 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f413c15e3a0 con 0x7f411c067e60 2024-09-10T10:04:38.850 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:04:38.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:38.850 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:04:38.850 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:38.850 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:04:38.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f411c067e60 msgr2=0x7f411c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f411c067e60 0x7f411c06a320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f412c001e00 tx=0x7f412c009040 comp rx=0 tx=0).stop 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 msgr2=0x7f413c150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c150b10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f412800c4d0 tx=0x7f412800c990 comp rx=0 tx=0).stop 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 shutdown_connections 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f411c067e60 0x7f411c06a320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c0cfb90 0x7f413c152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f413c0cdcb0 0x7f413c150b10 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 --2- 172.21.15.110:0/3007831456 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f413c0b3f40 0x7f413c14dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 >> 172.21.15.110:0/3007831456 conn(0x7f413c062d10 msgr2=0x7f413c14eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 shutdown_connections 2024-09-10T10:04:38.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:38.846+0000 7f4142e00640 1 -- 172.21.15.110:0/3007831456 wait complete. 2024-09-10T10:04:38.870 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:04:39.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.002+0000 7ff0b3e00640 1 Processor -- start 2024-09-10T10:04:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b3e00640 1 -- start start 2024-09-10T10:04:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40b2df0 0x7ff0b40c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b3e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff0b40c5e40 0x7ff0b40cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b3e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b415c9a0 con 0x7ff0b40cf5e0 2024-09-10T10:04:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b3e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b413f740 con 0x7ff0b40c5e40 2024-09-10T10:04:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b3e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b415cc80 con 0x7ff0b40b2df0 2024-09-10T10:04:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b3400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff0b40cf5e0 0x7ff0b4141a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47584/0 (socket says 172.21.15.110:47584) 2024-09-10T10:04:39.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b3400640 1 -- 172.21.15.110:0/1526493610 learned_addr learned my addr 172.21.15.110:0/1526493610 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:39.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b2a00640 1 --2- 172.21.15.110:0/1526493610 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40b2df0 0x7ff0b40c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b2000640 1 --2- 172.21.15.110:0/1526493610 >> v2:172.21.15.145:3300/0 conn(0x7ff0b40c5e40 0x7ff0b40cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b1600640 1 -- 172.21.15.110:0/1526493610 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3361537840 0 0) 0x7ff0b415c9a0 con 0x7ff0b40cf5e0 2024-09-10T10:04:39.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b1600640 1 -- 172.21.15.110:0/1526493610 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff098003530 con 0x7ff0b40cf5e0 2024-09-10T10:04:39.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b2a00640 1 -- 172.21.15.110:0/1526493610 >> v2:172.21.15.145:3300/0 conn(0x7ff0b40c5e40 msgr2=0x7ff0b40cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b2a00640 1 --2- 172.21.15.110:0/1526493610 >> v2:172.21.15.145:3300/0 conn(0x7ff0b40c5e40 0x7ff0b40cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b2a00640 1 -- 172.21.15.110:0/1526493610 >> v1:172.21.15.110:6789/0 conn(0x7ff0b40cf5e0 legacy=0x7ff0b4141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b2a00640 1 -- 172.21.15.110:0/1526493610 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b415cf60 con 0x7ff0b40b2df0 2024-09-10T10:04:39.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.006+0000 7ff0b2000640 1 --2- 172.21.15.110:0/1526493610 >> v2:172.21.15.145:3300/0 conn(0x7ff0b40c5e40 0x7ff0b40cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:39.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b2a00640 1 --2- 172.21.15.110:0/1526493610 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40b2df0 0x7ff0b40c5900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40097c0 tx=0x7ff0a401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aafe674c9c68be33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b1600640 1 -- 172.21.15.110:0/1526493610 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40b0070 con 0x7ff0b40b2df0 2024-09-10T10:04:39.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b1600640 1 -- 172.21.15.110:0/1526493610 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a40ab600 con 0x7ff0b40b2df0 2024-09-10T10:04:39.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b1600640 1 -- 172.21.15.110:0/1526493610 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40ab9b0 con 0x7ff0b40b2df0 2024-09-10T10:04:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b3e00640 1 -- 172.21.15.110:0/1526493610 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40b2df0 msgr2=0x7ff0b40c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/1526493610 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40b2df0 0x7ff0b40c5900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40097c0 tx=0x7ff0a401c4a0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b3e00640 1 -- 172.21.15.110:0/1526493610 shutdown_connections 2024-09-10T10:04:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/1526493610 >> v2:172.21.15.145:3300/0 conn(0x7ff0b40c5e40 0x7ff0b40cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/1526493610 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40b2df0 0x7ff0b40c5900 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b3e00640 1 -- 172.21.15.110:0/1526493610 >> 172.21.15.110:0/1526493610 conn(0x7ff0b406d710 msgr2=0x7ff0b40b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b3e00640 1 -- 172.21.15.110:0/1526493610 shutdown_connections 2024-09-10T10:04:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.010+0000 7ff0b3e00640 1 -- 172.21.15.110:0/1526493610 wait complete. 2024-09-10T10:04:39.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 Processor -- start 2024-09-10T10:04:39.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 -- start start 2024-09-10T10:04:39.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0b40b2df0 0x7ff0b40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40c5e40 0x7ff0b41593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0b40cf5e0 0x7ff0b41451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4169220 con 0x7ff0b40b2df0 2024-09-10T10:04:39.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b40b6160 con 0x7ff0b40cf5e0 2024-09-10T10:04:39.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b41693a0 con 0x7ff0b40c5e40 2024-09-10T10:04:39.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0b40b2df0 0x7ff0b40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0b40b2df0 0x7ff0b40b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55008/0 (socket says 172.21.15.110:55008) 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 -- 172.21.15.110:0/3701128239 learned_addr learned my addr 172.21.15.110:0/3701128239 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2000640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40c5e40 0x7ff0b41593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3400640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0b40cf5e0 0x7ff0b41451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 -- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40c5e40 msgr2=0x7ff0b41593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40c5e40 0x7ff0b41593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 -- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0b40cf5e0 msgr2=0x7ff0b41451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0b40cf5e0 0x7ff0b41451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 -- 172.21.15.110:0/3701128239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b4144740 con 0x7ff0b40b2df0 2024-09-10T10:04:39.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3400640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0b40cf5e0 0x7ff0b41451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:39.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b2a00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0b40b2df0 0x7ff0b40b4f80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40ae040 tx=0x7ff0a4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0aaa00640 1 -- 172.21.15.110:0/3701128239 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40b0040 con 0x7ff0b40b2df0 2024-09-10T10:04:39.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0aaa00640 1 -- 172.21.15.110:0/3701128239 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a40ab400 con 0x7ff0b40b2df0 2024-09-10T10:04:39.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b4142a60 con 0x7ff0b40b2df0 2024-09-10T10:04:39.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0aaa00640 1 -- 172.21.15.110:0/3701128239 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40ab9b0 con 0x7ff0b40b2df0 2024-09-10T10:04:39.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b41449e0 con 0x7ff0b40b2df0 2024-09-10T10:04:39.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.014+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b4141830 con 0x7ff0b40b2df0 2024-09-10T10:04:39.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.018+0000 7ff0aaa00640 1 -- 172.21.15.110:0/3701128239 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7ff0a40bd020 con 0x7ff0b40b2df0 2024-09-10T10:04:39.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.018+0000 7ff0aaa00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff094067e90 0x7ff09406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.018+0000 7ff0aaa00640 1 -- 172.21.15.110:0/3701128239 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7ff0a411ff60 con 0x7ff0b40b2df0 2024-09-10T10:04:39.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.018+0000 7ff0b2000640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff094067e90 0x7ff09406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.018+0000 7ff0b2000640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff094067e90 0x7ff09406a350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009580 tx=0x7ff0a0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.022+0000 7ff0aaa00640 1 -- 172.21.15.110:0/3701128239 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff0a4016090 con 0x7ff0b40b2df0 2024-09-10T10:04:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.178+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff0b4141d90 con 0x7ff0b40b2df0 2024-09-10T10:04:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.178+0000 7ff0aaa00640 1 -- 172.21.15.110:0/3701128239 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7ff0a40f0200 con 0x7ff0b40b2df0 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff094067e90 msgr2=0x7ff09406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff094067e90 0x7ff09406a350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009580 tx=0x7ff0a0009190 comp rx=0 tx=0).stop 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0b40b2df0 msgr2=0x7ff0b40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0b40b2df0 0x7ff0b40b4f80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40ae040 tx=0x7ff0a4019240 comp rx=0 tx=0).stop 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 shutdown_connections 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff094067e90 0x7ff09406a350 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0b40cf5e0 0x7ff0b41451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff0b40c5e40 0x7ff0b41593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 --2- 172.21.15.110:0/3701128239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0b40b2df0 0x7ff0b40b4f80 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 >> 172.21.15.110:0/3701128239 conn(0x7ff0b406d710 msgr2=0x7ff0b415e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 shutdown_connections 2024-09-10T10:04:39.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.182+0000 7ff0b3e00640 1 -- 172.21.15.110:0/3701128239 wait complete. 2024-09-10T10:04:39.205 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.338+0000 7f197de00640 1 Processor -- start 2024-09-10T10:04:39.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.338+0000 7f197de00640 1 -- start start 2024-09-10T10:04:39.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f197de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 0x7f197806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f197de00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f19780ce4d0 0x7f197806e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f197de00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f19780b1c60 0x7f197806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f197de00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f19780b5570 con 0x7f19780ce4d0 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f197de00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f19780b4960 con 0x7f19780b1c60 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f197de00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f19780b4c40 con 0x7f197815b330 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f19780ce4d0 0x7f197806e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f19780ce4d0 0x7f197806e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55010/0 (socket says 172.21.15.110:55010) 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 -- 172.21.15.110:0/3054779600 learned_addr learned my addr 172.21.15.110:0/3054779600 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1977400640 1 --2- 172.21.15.110:0/3054779600 >> v2:172.21.15.145:3300/0 conn(0x7f19780b1c60 0x7f197806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976a00640 1 --2- 172.21.15.110:0/3054779600 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 0x7f197806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 -- 172.21.15.110:0/3054779600 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 msgr2=0x7f197806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 --2- 172.21.15.110:0/3054779600 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 0x7f197806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 -- 172.21.15.110:0/3054779600 >> v2:172.21.15.145:3300/0 conn(0x7f19780b1c60 msgr2=0x7f197806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 --2- 172.21.15.110:0/3054779600 >> v2:172.21.15.145:3300/0 conn(0x7f19780b1c60 0x7f197806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 -- 172.21.15.110:0/3054779600 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19780b4f20 con 0x7f19780ce4d0 2024-09-10T10:04:39.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976a00640 1 --2- 172.21.15.110:0/3054779600 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 0x7f197806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:39.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1977400640 1 --2- 172.21.15.110:0/3054779600 >> v2:172.21.15.145:3300/0 conn(0x7f19780b1c60 0x7f197806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:39.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.342+0000 7f1976000640 1 --2- 172.21.15.110:0/3054779600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19780ce4d0 0x7f197806e140 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f19780b0650 tx=0x7f196401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae3ea5c06d18f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f1975600640 1 -- 172.21.15.110:0/3054779600 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640ab5f0 con 0x7f19780ce4d0 2024-09-10T10:04:39.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f1975600640 1 -- 172.21.15.110:0/3054779600 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19640b0070 con 0x7f19780ce4d0 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f1975600640 1 -- 172.21.15.110:0/3054779600 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19640ab9b0 con 0x7f19780ce4d0 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- 172.21.15.110:0/3054779600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19780ce4d0 msgr2=0x7f197806e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 --2- 172.21.15.110:0/3054779600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19780ce4d0 0x7f197806e140 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f19780b0650 tx=0x7f196401c520 comp rx=0 tx=0).stop 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- 172.21.15.110:0/3054779600 shutdown_connections 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 --2- 172.21.15.110:0/3054779600 >> v2:172.21.15.145:3300/0 conn(0x7f19780b1c60 0x7f197806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 --2- 172.21.15.110:0/3054779600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19780ce4d0 0x7f197806e140 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 --2- 172.21.15.110:0/3054779600 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 0x7f197806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- 172.21.15.110:0/3054779600 >> 172.21.15.110:0/3054779600 conn(0x7f1978062e40 msgr2=0x7f197806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- 172.21.15.110:0/3054779600 shutdown_connections 2024-09-10T10:04:39.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- 172.21.15.110:0/3054779600 wait complete. 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 Processor -- start 2024-09-10T10:04:39.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- start start 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19780b1c60 0x7f1978147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 0x7f197814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1978155440 0x7f1978072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f197815fae0 con 0x7f1978155440 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f197815f960 con 0x7f19780b1c60 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f197de00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f197815fc60 con 0x7f197815b330 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f1977400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1978155440 0x7f1978072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f1977400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1978155440 0x7f1978072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55020/0 (socket says 172.21.15.110:55020) 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.346+0000 7f1977400640 1 -- 172.21.15.110:0/2010710435 learned_addr learned my addr 172.21.15.110:0/2010710435 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f1977400640 1 -- 172.21.15.110:0/2010710435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 msgr2=0x7f197814c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f1977400640 1 --2- 172.21.15.110:0/2010710435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f197815b330 0x7f197814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f1977400640 1 -- 172.21.15.110:0/2010710435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19780b1c60 msgr2=0x7f1978147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f1977400640 1 --2- 172.21.15.110:0/2010710435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19780b1c60 0x7f1978147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f1977400640 1 -- 172.21.15.110:0/2010710435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197814cc70 con 0x7f1978155440 2024-09-10T10:04:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f1977400640 1 --2- 172.21.15.110:0/2010710435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1978155440 0x7f1978072400 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1970007db0 tx=0x7f197000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f196f400640 1 -- 172.21.15.110:0/2010710435 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197001a070 con 0x7f1978155440 2024-09-10T10:04:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f196f400640 1 -- 172.21.15.110:0/2010710435 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1970015400 con 0x7f1978155440 2024-09-10T10:04:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f197de00640 1 -- 172.21.15.110:0/2010710435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197814cf50 con 0x7f1978155440 2024-09-10T10:04:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f196f400640 1 -- 172.21.15.110:0/2010710435 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700157f0 con 0x7f1978155440 2024-09-10T10:04:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f197de00640 1 -- 172.21.15.110:0/2010710435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197814d510 con 0x7f1978155440 2024-09-10T10:04:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f197de00640 1 -- 172.21.15.110:0/2010710435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197806c6a0 con 0x7f1978155440 2024-09-10T10:04:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f196f400640 1 -- 172.21.15.110:0/2010710435 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f197001d070 con 0x7f1978155440 2024-09-10T10:04:39.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f196f400640 1 --2- 172.21.15.110:0/2010710435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1958067e60 0x7f195806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.350+0000 7f196f400640 1 -- 172.21.15.110:0/2010710435 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f1970086a80 con 0x7f1978155440 2024-09-10T10:04:39.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.354+0000 7f1976a00640 1 --2- 172.21.15.110:0/2010710435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1958067e60 0x7f195806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.354+0000 7f1976a00640 1 --2- 172.21.15.110:0/2010710435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1958067e60 0x7f195806a320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1968006e90 tx=0x7f1968008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.354+0000 7f196f400640 1 -- 172.21.15.110:0/2010710435 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1970010070 con 0x7f1978155440 2024-09-10T10:04:39.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.538+0000 7f197de00640 1 -- 172.21.15.110:0/2010710435 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f19780b29a0 con 0x7f1958067e60 2024-09-10T10:04:39.545 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:39.542+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:39.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.542+0000 7f196f400640 1 -- 172.21.15.110:0/2010710435 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f19780b29a0 con 0x7f1958067e60 2024-09-10T10:04:39.545 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:39.555 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:39.556 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-10T10:04:39.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00f6400640 1 Processor -- start 2024-09-10T10:04:39.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00f6400640 1 -- start start 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00f6400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f00f006e850 0x7f00f006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00f6400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f00b4960 con 0x7f00f00ce4d0 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00f6400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f00b4c40 con 0x7f00f015b330 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00f6400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f00f00b5570 con 0x7f00f006e850 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00eea00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f00f00ce4d0 0x7f00f00b0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47598/0 (socket says 172.21.15.110:47598) 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00eea00640 1 -- 172.21.15.110:0/1387002174 learned_addr learned my addr 172.21.15.110:0/1387002174 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00efe00640 1 --2- 172.21.15.110:0/1387002174 >> v2:172.21.15.145:3300/0 conn(0x7f00f006e850 0x7f00f006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744544377 0 0) 0x7f00f00b4960 con 0x7f00f00ce4d0 2024-09-10T10:04:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00d4005740 con 0x7f00f00ce4d0 2024-09-10T10:04:39.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2546261358 0 0) 0x7f00d4005740 con 0x7f00f00ce4d0 2024-09-10T10:04:39.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 >> v2:172.21.15.145:3300/0 conn(0x7f00f006e850 msgr2=0x7f00f006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 --2- 172.21.15.110:0/1387002174 >> v2:172.21.15.145:3300/0 conn(0x7f00f006e850 0x7f00f006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 >> v1:172.21.15.145:6789/0 conn(0x7f00f015b330 legacy=0x7f00f006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f00b4f20 con 0x7f00f00ce4d0 2024-09-10T10:04:39.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f00dc001680 con 0x7f00f00ce4d0 2024-09-10T10:04:39.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.690+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 >> v1:172.21.15.110:6789/0 conn(0x7f00f00ce4d0 legacy=0x7f00f00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d4005c20 0x7f00d4006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d4006a20 0x7f00d4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d40073e0 0x7f00d400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00b5570 con 0x7f00d4006a20 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f00d400e2e0 con 0x7f00d4005c20 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f00d400e4e0 con 0x7f00d40073e0 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00eea00640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d4006a20 0x7f00d4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00efe00640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d40073e0 0x7f00d400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ef400640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d4005c20 0x7f00d4006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00eea00640 1 -- 172.21.15.110:0/1387002174 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d40073e0 msgr2=0x7f00d400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00eea00640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d40073e0 0x7f00d400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00eea00640 1 -- 172.21.15.110:0/1387002174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d4005c20 msgr2=0x7f00d4006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00eea00640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d4005c20 0x7f00d4006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00eea00640 1 -- 172.21.15.110:0/1387002174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d4005740 con 0x7f00d4006a20 2024-09-10T10:04:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00efe00640 1 -- 172.21.15.110:0/1387002174 reap_dead start 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00eea00640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d4006a20 0x7f00d4006ea0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c110 tx=0x7f00dc01f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81fc059789f3705 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc007c30 con 0x7f00d4006a20 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc0ac020 con 0x7f00d4006a20 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.694+0000 7f00ee000640 1 -- 172.21.15.110:0/1387002174 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0058e0 con 0x7f00d4006a20 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- 172.21.15.110:0/1387002174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d4006a20 msgr2=0x7f00d4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d4006a20 0x7f00d4006ea0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c110 tx=0x7f00dc01f270 comp rx=0 tx=0).stop 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- 172.21.15.110:0/1387002174 shutdown_connections 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 --2- 172.21.15.110:0/1387002174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d4006a20 0x7f00d4006ea0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- 172.21.15.110:0/1387002174 >> 172.21.15.110:0/1387002174 conn(0x7f00f0062e40 msgr2=0x7f00f006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- 172.21.15.110:0/1387002174 shutdown_connections 2024-09-10T10:04:39.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- 172.21.15.110:0/1387002174 wait complete. 2024-09-10T10:04:39.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 Processor -- start 2024-09-10T10:04:39.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- start start 2024-09-10T10:04:39.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d4006a20 0x7f00f015d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00f006d500 0x7f00f006d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00f01f46e0 0x7f00f01f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00b1c60 con 0x7f00f006d500 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f00f015b710 con 0x7f00f01f46e0 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00f6400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f00f00b1de0 con 0x7f00d4006a20 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00eea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00f006d500 0x7f00f006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00eea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00f006d500 0x7f00f006d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55034/0 (socket says 172.21.15.110:55034) 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00eea00640 1 -- 172.21.15.110:0/603967504 learned_addr learned my addr 172.21.15.110:0/603967504 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00ef400640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d4006a20 0x7f00f015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00efe00640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00f01f46e0 0x7f00f01f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00eea00640 1 -- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d4006a20 msgr2=0x7f00f015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00eea00640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d4006a20 0x7f00f015d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00eea00640 1 -- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00f01f46e0 msgr2=0x7f00f01f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00eea00640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00f01f46e0 0x7f00f01f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00eea00640 1 -- 172.21.15.110:0/603967504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f015db50 con 0x7f00f006d500 2024-09-10T10:04:39.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.698+0000 7f00ef400640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f00d4006a20 0x7f00f015d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:39.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00eea00640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00f006d500 0x7f00f006d980 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0ae040 tx=0x7f00d4007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00e7400640 1 -- 172.21.15.110:0/603967504 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc001b70 con 0x7f00f006d500 2024-09-10T10:04:39.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00e7400640 1 -- 172.21.15.110:0/603967504 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc000ee0 con 0x7f00f006d500 2024-09-10T10:04:39.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00f6400640 1 -- 172.21.15.110:0/603967504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f015de30 con 0x7f00f006d500 2024-09-10T10:04:39.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00e7400640 1 -- 172.21.15.110:0/603967504 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc001680 con 0x7f00f006d500 2024-09-10T10:04:39.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00f6400640 1 -- 172.21.15.110:0/603967504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f015e3f0 con 0x7f00f006d500 2024-09-10T10:04:39.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00f6400640 1 -- 172.21.15.110:0/603967504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f006c6a0 con 0x7f00f006d500 2024-09-10T10:04:39.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00e7400640 1 -- 172.21.15.110:0/603967504 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f00dc0072f0 con 0x7f00f006d500 2024-09-10T10:04:39.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00e7400640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00d0067e90 0x7f00d006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00e7400640 1 -- 172.21.15.110:0/603967504 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f00dc123a00 con 0x7f00f006d500 2024-09-10T10:04:39.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.702+0000 7f00ef400640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00d0067e90 0x7f00d006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.706+0000 7f00ef400640 1 --2- 172.21.15.110:0/603967504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f00d0067e90 0x7f00d006a350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b17e0 tx=0x7f00e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.706+0000 7f00e7400640 1 -- 172.21.15.110:0/603967504 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f00dc018f80 con 0x7f00f006d500 2024-09-10T10:04:39.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.890+0000 7f00f6400640 1 -- 172.21.15.110:0/603967504 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f00f01567c0 con 0x7f00d0067e90 2024-09-10T10:04:39.894 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:39.894+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:39.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:39.890+0000 7f00e7400640 1 -- 172.21.15.110:0/603967504 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f00f01567c0 con 0x7f00d0067e90 2024-09-10T10:04:39.895 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:39.904 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:39.904 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-10T10:04:40.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a8e400640 1 Processor -- start 2024-09-10T10:04:40.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a8e400640 1 -- start start 2024-09-10T10:04:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a8e400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a8815b330 0x7f5a8806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a8e400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5a8806e850 0x7f5a880b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a8e400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a880b4960 con 0x7f5a880ce4d0 2024-09-10T10:04:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a8e400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5a880b5570 con 0x7f5a8806e850 2024-09-10T10:04:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a8e400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5a880b4c40 con 0x7f5a8815b330 2024-09-10T10:04:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5a880ce4d0 0x7f5a8806e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47612/0 (socket says 172.21.15.110:47612) 2024-09-10T10:04:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86a00640 1 -- 172.21.15.110:0/3326755138 learned_addr learned my addr 172.21.15.110:0/3326755138 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87400640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a8815b330 0x7f5a8806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87e00640 1 --2- 172.21.15.110:0/3326755138 >> v2:172.21.15.145:3300/0 conn(0x7f5a8806e850 0x7f5a880b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763007210 0 0) 0x7f5a880b4960 con 0x7f5a880ce4d0 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a6c003530 con 0x7f5a880ce4d0 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 923338464 0 0) 0x7f5a6c003530 con 0x7f5a880ce4d0 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a8815b330 msgr2=0x7f5a8806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a8815b330 0x7f5a8806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 >> v2:172.21.15.145:3300/0 conn(0x7f5a8806e850 msgr2=0x7f5a880b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 --2- 172.21.15.110:0/3326755138 >> v2:172.21.15.145:3300/0 conn(0x7f5a8806e850 0x7f5a880b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a880b4f20 con 0x7f5a880ce4d0 2024-09-10T10:04:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f5a74001680 con 0x7f5a880ce4d0 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 >> v1:172.21.15.110:6789/0 conn(0x7f5a880ce4d0 legacy=0x7f5a8806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a6c0039c0 0x7f5a6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a6c0043b0 0x7f5a6c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 0x7f5a6c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a880b5570 con 0x7f5a6c004f00 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a880b4c40 con 0x7f5a6c0043b0 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c00bf10 con 0x7f5a6c0039c0 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87e00640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 0x7f5a6c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a86a00640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a6c0043b0 0x7f5a6c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87400640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a6c0039c0 0x7f5a6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87e00640 1 -- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a6c0039c0 msgr2=0x7f5a6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:40.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87e00640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a6c0039c0 0x7f5a6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87e00640 1 -- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a6c0043b0 msgr2=0x7f5a6c004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:40.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87e00640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a6c0043b0 0x7f5a6c004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.042+0000 7f5a87e00640 1 -- 172.21.15.110:0/3326755138 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c003530 con 0x7f5a6c004f00 2024-09-10T10:04:40.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a87e00640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 0x7f5a6c00b9d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5a88070ed0 tx=0x7f5a8001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad10ccb19fae705 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800b2070 con 0x7f5a6c004f00 2024-09-10T10:04:40.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a800a5c30 con 0x7f5a6c004f00 2024-09-10T10:04:40.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a86000640 1 -- 172.21.15.110:0/3326755138 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80009c10 con 0x7f5a6c004f00 2024-09-10T10:04:40.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a8e400640 1 -- 172.21.15.110:0/3326755138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 msgr2=0x7f5a6c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a8e400640 1 --2- 172.21.15.110:0/3326755138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 0x7f5a6c00b9d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5a88070ed0 tx=0x7f5a8001cc20 comp rx=0 tx=0).stop 2024-09-10T10:04:40.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a87e00640 1 -- 172.21.15.110:0/3326755138 reap_dead start 2024-09-10T10:04:40.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a8e400640 1 -- 172.21.15.110:0/3326755138 shutdown_connections 2024-09-10T10:04:40.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a8e400640 1 -- 172.21.15.110:0/3326755138 >> 172.21.15.110:0/3326755138 conn(0x7f5a88062e40 msgr2=0x7f5a8806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:40.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a8e400640 1 -- 172.21.15.110:0/3326755138 shutdown_connections 2024-09-10T10:04:40.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.046+0000 7f5a8e400640 1 -- 172.21.15.110:0/3326755138 wait complete. 2024-09-10T10:04:40.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 Processor -- start 2024-09-10T10:04:40.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 -- start start 2024-09-10T10:04:40.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 0x7f5a881f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a88151610 0x7f5a880cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a88151a50 0x7f5a880cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a880b1c60 con 0x7f5a6c004f00 2024-09-10T10:04:40.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8815b710 con 0x7f5a88151610 2024-09-10T10:04:40.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5a880b1de0 con 0x7f5a88151a50 2024-09-10T10:04:40.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 0x7f5a881f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 0x7f5a881f5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55048/0 (socket says 172.21.15.110:55048) 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 -- 172.21.15.110:0/3628450909 learned_addr learned my addr 172.21.15.110:0/3628450909 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a86a00640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a88151610 0x7f5a880cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87e00640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a88151a50 0x7f5a880cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 -- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a88151a50 msgr2=0x7f5a880cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a88151a50 0x7f5a880cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 -- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a88151610 msgr2=0x7f5a880cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a88151610 0x7f5a880cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 -- 172.21.15.110:0/3628450909 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a880ad380 con 0x7f5a6c004f00 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87e00640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a88151a50 0x7f5a880cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a87400640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a6c004f00 0x7f5a881f5c40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5a78011890 tx=0x7f5a78011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a7f400640 1 -- 172.21.15.110:0/3628450909 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78019070 con 0x7f5a6c004f00 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a7f400640 1 -- 172.21.15.110:0/3628450909 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a78004240 con 0x7f5a6c004f00 2024-09-10T10:04:40.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a7f400640 1 -- 172.21.15.110:0/3628450909 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800f7a0 con 0x7f5a6c004f00 2024-09-10T10:04:40.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 -- 172.21.15.110:0/3628450909 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a880af180 con 0x7f5a6c004f00 2024-09-10T10:04:40.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.050+0000 7f5a8e400640 1 -- 172.21.15.110:0/3628450909 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a880ab990 con 0x7f5a6c004f00 2024-09-10T10:04:40.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.054+0000 7f5a8e400640 1 -- 172.21.15.110:0/3628450909 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8806c6a0 con 0x7f5a6c004f00 2024-09-10T10:04:40.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.054+0000 7f5a7f400640 1 -- 172.21.15.110:0/3628450909 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f5a7801c070 con 0x7f5a6c004f00 2024-09-10T10:04:40.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.054+0000 7f5a7f400640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a68067e60 0x7f5a6806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.054+0000 7f5a7f400640 1 -- 172.21.15.110:0/3628450909 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f5a78086aa0 con 0x7f5a6c004f00 2024-09-10T10:04:40.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.054+0000 7f5a86a00640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a68067e60 0x7f5a6806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.054+0000 7f5a86a00640 1 --2- 172.21.15.110:0/3628450909 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a68067e60 0x7f5a6806a320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5a88072920 tx=0x7f5a74007900 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.058+0000 7f5a7f400640 1 -- 172.21.15.110:0/3628450909 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5a880ad380 con 0x7f5a6c004f00 2024-09-10T10:04:40.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.242+0000 7f5a8e400640 1 -- 172.21.15.110:0/3628450909 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f5a8815e510 con 0x7f5a68067e60 2024-09-10T10:04:40.247 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:40.246+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:40.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.242+0000 7f5a7f400640 1 -- 172.21.15.110:0/3628450909 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f5a8815e510 con 0x7f5a68067e60 2024-09-10T10:04:40.247 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:40.257 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:40.257 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-10T10:04:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.390+0000 7f6a5ce00640 1 Processor -- start 2024-09-10T10:04:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.390+0000 7f6a5ce00640 1 -- start start 2024-09-10T10:04:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a5ce00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6a580ce4d0 0x7f6a5806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a5ce00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6a5806e850 0x7f6a580b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a5ce00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a580b4960 con 0x7f6a5815b330 2024-09-10T10:04:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a5ce00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6a580b5570 con 0x7f6a580ce4d0 2024-09-10T10:04:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a5ce00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6a580b4c40 con 0x7f6a5806e850 2024-09-10T10:04:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a55400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6a580ce4d0 0x7f6a5806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a55400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6a580ce4d0 0x7f6a5806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55060/0 (socket says 172.21.15.110:55060) 2024-09-10T10:04:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a55400640 1 -- 172.21.15.110:0/4174501765 learned_addr learned my addr 172.21.15.110:0/4174501765 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a56800640 1 --2- 172.21.15.110:0/4174501765 >> v2:172.21.15.145:3300/0 conn(0x7f6a5806e850 0x7f6a580b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a55400640 1 -- 172.21.15.110:0/4174501765 >> v2:172.21.15.145:3300/0 conn(0x7f6a5806e850 msgr2=0x7f6a580b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a55400640 1 --2- 172.21.15.110:0/4174501765 >> v2:172.21.15.145:3300/0 conn(0x7f6a5806e850 0x7f6a580b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a55400640 1 -- 172.21.15.110:0/4174501765 >> v1:172.21.15.145:6789/0 conn(0x7f6a5815b330 legacy=0x7f6a5806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.394+0000 7f6a55400640 1 -- 172.21.15.110:0/4174501765 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a580b4f20 con 0x7f6a580ce4d0 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a55400640 1 --2- 172.21.15.110:0/4174501765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a580ce4d0 0x7f6a5806c6a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6a580633d0 tx=0x7f6a4401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a33d47098b19832 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a54a00640 1 -- 172.21.15.110:0/4174501765 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440ab430 con 0x7f6a580ce4d0 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a54a00640 1 -- 172.21.15.110:0/4174501765 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a440b0070 con 0x7f6a580ce4d0 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a54a00640 1 -- 172.21.15.110:0/4174501765 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440ab7f0 con 0x7f6a580ce4d0 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- 172.21.15.110:0/4174501765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a580ce4d0 msgr2=0x7f6a5806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 --2- 172.21.15.110:0/4174501765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a580ce4d0 0x7f6a5806c6a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6a580633d0 tx=0x7f6a4401c520 comp rx=0 tx=0).stop 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- 172.21.15.110:0/4174501765 shutdown_connections 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 --2- 172.21.15.110:0/4174501765 >> v2:172.21.15.145:3300/0 conn(0x7f6a5806e850 0x7f6a580b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 --2- 172.21.15.110:0/4174501765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a580ce4d0 0x7f6a5806c6a0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- 172.21.15.110:0/4174501765 >> 172.21.15.110:0/4174501765 conn(0x7f6a58062e40 msgr2=0x7f6a5806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:40.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- 172.21.15.110:0/4174501765 shutdown_connections 2024-09-10T10:04:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- 172.21.15.110:0/4174501765 wait complete. 2024-09-10T10:04:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 Processor -- start 2024-09-10T10:04:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- start start 2024-09-10T10:04:40.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a5806e850 0x7f6a5814d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a5815b330 0x7f6a5814c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a5814cda0 0x7f6a58155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581680b0 con 0x7f6a5815b330 2024-09-10T10:04:40.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6a580b1800 con 0x7f6a5814cda0 2024-09-10T10:04:40.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.398+0000 7f6a5ce00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6a58168230 con 0x7f6a5806e850 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a5815b330 0x7f6a5814c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a5815b330 0x7f6a5814c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55074/0 (socket says 172.21.15.110:55074) 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 -- 172.21.15.110:0/2538119046 learned_addr learned my addr 172.21.15.110:0/2538119046 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55e00640 1 --2- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a5806e850 0x7f6a5814d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a56800640 1 --2- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a5814cda0 0x7f6a58155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 -- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a5806e850 msgr2=0x7f6a5814d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 --2- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a5806e850 0x7f6a5814d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 -- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a5814cda0 msgr2=0x7f6a58155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 --2- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a5814cda0 0x7f6a58155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 -- 172.21.15.110:0/2538119046 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a58150ef0 con 0x7f6a5815b330 2024-09-10T10:04:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a55400640 1 --2- 172.21.15.110:0/2538119046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a5815b330 0x7f6a5814c570 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6a44019130 tx=0x7f6a4401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a4e000640 1 -- 172.21.15.110:0/2538119046 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440be5f0 con 0x7f6a5815b330 2024-09-10T10:04:40.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a4e000640 1 -- 172.21.15.110:0/2538119046 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a440be790 con 0x7f6a5815b330 2024-09-10T10:04:40.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a5ce00640 1 -- 172.21.15.110:0/2538119046 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a581511d0 con 0x7f6a5815b330 2024-09-10T10:04:40.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a4e000640 1 -- 172.21.15.110:0/2538119046 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440beb40 con 0x7f6a5815b330 2024-09-10T10:04:40.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a5ce00640 1 -- 172.21.15.110:0/2538119046 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a58151790 con 0x7f6a5815b330 2024-09-10T10:04:40.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a5ce00640 1 -- 172.21.15.110:0/2538119046 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a580b0860 con 0x7f6a5815b330 2024-09-10T10:04:40.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.402+0000 7f6a4e000640 1 -- 172.21.15.110:0/2538119046 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f6a440b43f0 con 0x7f6a5815b330 2024-09-10T10:04:40.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.406+0000 7f6a4e000640 1 --2- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a38067e90 0x7f6a3806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.406+0000 7f6a4e000640 1 -- 172.21.15.110:0/2538119046 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f6a440a8070 con 0x7f6a5815b330 2024-09-10T10:04:40.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.406+0000 7f6a55e00640 1 --2- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a38067e90 0x7f6a3806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.406+0000 7f6a55e00640 1 --2- 172.21.15.110:0/2538119046 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a38067e90 0x7f6a3806a350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a48001770 tx=0x7f6a48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.410+0000 7f6a4e000640 1 -- 172.21.15.110:0/2538119046 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6a44016110 con 0x7f6a5815b330 2024-09-10T10:04:40.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.594+0000 7f6a5ce00640 1 -- 172.21.15.110:0/2538119046 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f6a580cd240 con 0x7f6a38067e90 2024-09-10T10:04:40.599 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:40.598+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:40.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.594+0000 7f6a4e000640 1 -- 172.21.15.110:0/2538119046 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f6a580cd240 con 0x7f6a38067e90 2024-09-10T10:04:40.600 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:40.609 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:40.609 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-10T10:04:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1992e00640 1 Processor -- start 2024-09-10T10:04:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1992e00640 1 -- start start 2024-09-10T10:04:40.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1992e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f198c0cfe30 0x7f198c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1992e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f198c06e7a0 0x7f198c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1992e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c06d2d0 con 0x7f198c0cd510 2024-09-10T10:04:40.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1992e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f198c15d9a0 con 0x7f198c06e7a0 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1992e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f198c06d5b0 con 0x7f198c0cfe30 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1990e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f198c06e7a0 0x7f198c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1990e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f198c06e7a0 0x7f198c0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55084/0 (socket says 172.21.15.110:55084) 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1990e00640 1 -- 172.21.15.110:0/319511963 learned_addr learned my addr 172.21.15.110:0/319511963 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f198be00640 1 --2- 172.21.15.110:0/319511963 >> v2:172.21.15.145:3300/0 conn(0x7f198c0cfe30 0x7f198c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1990e00640 1 -- 172.21.15.110:0/319511963 >> v2:172.21.15.145:3300/0 conn(0x7f198c0cfe30 msgr2=0x7f198c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1990e00640 1 --2- 172.21.15.110:0/319511963 >> v2:172.21.15.145:3300/0 conn(0x7f198c0cfe30 0x7f198c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1990e00640 1 -- 172.21.15.110:0/319511963 >> v1:172.21.15.145:6789/0 conn(0x7f198c0cd510 legacy=0x7f198c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:40.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f1990e00640 1 -- 172.21.15.110:0/319511963 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c06d890 con 0x7f198c06e7a0 2024-09-10T10:04:40.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.746+0000 7f198be00640 1 --2- 172.21.15.110:0/319511963 >> v2:172.21.15.145:3300/0 conn(0x7f198c0cfe30 0x7f198c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:40.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1990e00640 1 --2- 172.21.15.110:0/319511963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c06e7a0 0x7f198c0cd9a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f198c0ce120 tx=0x7f198401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a40026fb7fc93f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f198b400640 1 -- 172.21.15.110:0/319511963 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19840a56f0 con 0x7f198c06e7a0 2024-09-10T10:04:40.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f198b400640 1 -- 172.21.15.110:0/319511963 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19840a5890 con 0x7f198c06e7a0 2024-09-10T10:04:40.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f198b400640 1 -- 172.21.15.110:0/319511963 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19840a5c60 con 0x7f198c06e7a0 2024-09-10T10:04:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1992e00640 1 -- 172.21.15.110:0/319511963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c06e7a0 msgr2=0x7f198c0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1992e00640 1 --2- 172.21.15.110:0/319511963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c06e7a0 0x7f198c0cd9a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f198c0ce120 tx=0x7f198401c920 comp rx=0 tx=0).stop 2024-09-10T10:04:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1992e00640 1 -- 172.21.15.110:0/319511963 shutdown_connections 2024-09-10T10:04:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1992e00640 1 --2- 172.21.15.110:0/319511963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c06e7a0 0x7f198c0cd9a0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1992e00640 1 --2- 172.21.15.110:0/319511963 >> v2:172.21.15.145:3300/0 conn(0x7f198c0cfe30 0x7f198c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1992e00640 1 -- 172.21.15.110:0/319511963 >> 172.21.15.110:0/319511963 conn(0x7f198c06bc50 msgr2=0x7f198c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:40.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1992e00640 1 -- 172.21.15.110:0/319511963 shutdown_connections 2024-09-10T10:04:40.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.750+0000 7f1992e00640 1 -- 172.21.15.110:0/319511963 wait complete. 2024-09-10T10:04:40.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 Processor -- start 2024-09-10T10:04:40.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 -- start start 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c06e7a0 0x7f198c152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f198c0cd510 0x7f198c1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f198c0cfe30 0x7f198c152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f198c072190 con 0x7f198c06e7a0 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f198c15d6c0 con 0x7f198c0cfe30 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f198c169180 con 0x7f198c0cd510 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c06e7a0 0x7f198c152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c06e7a0 0x7f198c152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55090/0 (socket says 172.21.15.110:55090) 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 -- 172.21.15.110:0/363586220 learned_addr learned my addr 172.21.15.110:0/363586220 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:40.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1983e00640 1 --2- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f198c0cd510 0x7f198c1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1990e00640 1 --2- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f198c0cfe30 0x7f198c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 -- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f198c0cd510 msgr2=0x7f198c1595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 --2- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f198c0cd510 0x7f198c1595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 -- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f198c0cfe30 msgr2=0x7f198c152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 --2- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f198c0cfe30 0x7f198c152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 -- 172.21.15.110:0/363586220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c154050 con 0x7f198c06e7a0 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f198be00640 1 --2- 172.21.15.110:0/363586220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c06e7a0 0x7f198c152120 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c7f0 tx=0x7f197c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1988c00640 1 -- 172.21.15.110:0/363586220 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c015070 con 0x7f198c06e7a0 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1988c00640 1 -- 172.21.15.110:0/363586220 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c007820 con 0x7f198c06e7a0 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1988c00640 1 -- 172.21.15.110:0/363586220 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c007c10 con 0x7f198c06e7a0 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 -- 172.21.15.110:0/363586220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c154330 con 0x7f198c06e7a0 2024-09-10T10:04:40.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 -- 172.21.15.110:0/363586220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c1548f0 con 0x7f198c06e7a0 2024-09-10T10:04:40.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.754+0000 7f1992e00640 1 -- 172.21.15.110:0/363586220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c0008e0 con 0x7f198c06e7a0 2024-09-10T10:04:40.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.758+0000 7f1988c00640 1 -- 172.21.15.110:0/363586220 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f197c018070 con 0x7f198c06e7a0 2024-09-10T10:04:40.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.758+0000 7f1988c00640 1 --2- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1968067e60 0x7f196806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:40.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.758+0000 7f1988c00640 1 -- 172.21.15.110:0/363586220 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f197c086860 con 0x7f198c06e7a0 2024-09-10T10:04:40.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.758+0000 7f1983e00640 1 --2- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1968067e60 0x7f196806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:40.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.758+0000 7f1983e00640 1 --2- 172.21.15.110:0/363586220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1968067e60 0x7f196806a320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f198c0ccc20 tx=0x7f1974009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:40.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.762+0000 7f1988c00640 1 -- 172.21.15.110:0/363586220 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f197c056420 con 0x7f198c06e7a0 2024-09-10T10:04:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.946+0000 7f1992e00640 1 -- 172.21.15.110:0/363586220 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7f198c075560 con 0x7f1968067e60 2024-09-10T10:04:40.954 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:40.954+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:40.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:40.950+0000 7f1988c00640 1 -- 172.21.15.110:0/363586220 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f198c075560 con 0x7f1968067e60 2024-09-10T10:04:40.954 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:40.962 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:40.963 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-10T10:04:41.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.098+0000 7fe688000640 1 Processor -- start 2024-09-10T10:04:41.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.098+0000 7fe688000640 1 -- start start 2024-09-10T10:04:41.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.098+0000 7fe688000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe6800cfe30 0x7fe680071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.098+0000 7fe688000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe6800cd510 0x7fe6800cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.098+0000 7fe688000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68006d2d0 con 0x7fe68006e090 2024-09-10T10:04:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.098+0000 7fe688000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe68015d9a0 con 0x7fe6800cd510 2024-09-10T10:04:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.098+0000 7fe688000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe68006d5b0 con 0x7fe6800cfe30 2024-09-10T10:04:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe684c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe6800cd510 0x7fe6800cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe686000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe68006e090 0x7fe68006e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47622/0 (socket says 172.21.15.110:47622) 2024-09-10T10:04:41.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe686000640 1 -- 172.21.15.110:0/2651867377 learned_addr learned my addr 172.21.15.110:0/2651867377 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:41.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe685600640 1 --2- 172.21.15.110:0/2651867377 >> v2:172.21.15.145:3300/0 conn(0x7fe6800cfe30 0x7fe680071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe684c00640 1 -- 172.21.15.110:0/2651867377 >> v2:172.21.15.145:3300/0 conn(0x7fe6800cfe30 msgr2=0x7fe680071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe684c00640 1 --2- 172.21.15.110:0/2651867377 >> v2:172.21.15.145:3300/0 conn(0x7fe6800cfe30 0x7fe680071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe684c00640 1 -- 172.21.15.110:0/2651867377 >> v1:172.21.15.110:6789/0 conn(0x7fe68006e090 legacy=0x7fe68006e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe684c00640 1 -- 172.21.15.110:0/2651867377 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68006d890 con 0x7fe6800cd510 2024-09-10T10:04:41.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe67fe00640 1 -- 172.21.15.110:0/2651867377 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678797516 0 0) 0x7fe68006d2d0 con 0x7fe68006e090 2024-09-10T10:04:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe684c00640 1 --2- 172.21.15.110:0/2651867377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6800cd510 0x7fe6800cd9a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe6800ce120 tx=0x7fe66c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecfa0d027519c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe67fe00640 1 -- 172.21.15.110:0/2651867377 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0b0070 con 0x7fe6800cd510 2024-09-10T10:04:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe67fe00640 1 -- 172.21.15.110:0/2651867377 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66c0ab600 con 0x7fe6800cd510 2024-09-10T10:04:41.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.102+0000 7fe67fe00640 1 -- 172.21.15.110:0/2651867377 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0ab9d0 con 0x7fe6800cd510 2024-09-10T10:04:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- 172.21.15.110:0/2651867377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6800cd510 msgr2=0x7fe6800cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 --2- 172.21.15.110:0/2651867377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6800cd510 0x7fe6800cd9a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe6800ce120 tx=0x7fe66c01c3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- 172.21.15.110:0/2651867377 shutdown_connections 2024-09-10T10:04:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 --2- 172.21.15.110:0/2651867377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6800cd510 0x7fe6800cd9a0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 --2- 172.21.15.110:0/2651867377 >> v2:172.21.15.145:3300/0 conn(0x7fe6800cfe30 0x7fe680071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- 172.21.15.110:0/2651867377 >> 172.21.15.110:0/2651867377 conn(0x7fe68006bc50 msgr2=0x7fe68006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- 172.21.15.110:0/2651867377 shutdown_connections 2024-09-10T10:04:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- 172.21.15.110:0/2651867377 wait complete. 2024-09-10T10:04:41.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 Processor -- start 2024-09-10T10:04:41.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- start start 2024-09-10T10:04:41.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe68006e090 0x7fe680146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe6800cd510 0x7fe680146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6800cfe30 0x7fe6801551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe680072190 con 0x7fe68006e090 2024-09-10T10:04:41.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe68015d6c0 con 0x7fe6800cfe30 2024-09-10T10:04:41.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe688000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe680169180 con 0x7fe6800cd510 2024-09-10T10:04:41.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe685600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe68006e090 0x7fe680146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe685600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe68006e090 0x7fe680146910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55100/0 (socket says 172.21.15.110:55100) 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe685600640 1 -- 172.21.15.110:0/511333330 learned_addr learned my addr 172.21.15.110:0/511333330 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe684c00640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe6800cd510 0x7fe680146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe686000640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6800cfe30 0x7fe6801551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe685600640 1 -- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe6800cd510 msgr2=0x7fe680146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe685600640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe6800cd510 0x7fe680146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe685600640 1 -- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6800cfe30 msgr2=0x7fe6801551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe685600640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6800cfe30 0x7fe6801551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe685600640 1 -- 172.21.15.110:0/511333330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6800710f0 con 0x7fe68006e090 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe684c00640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe6800cd510 0x7fe680146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.106+0000 7fe686000640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6800cfe30 0x7fe6801551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe685600640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe68006e090 0x7fe680146910 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe674007e40 tx=0x7fe67400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe67d600640 1 -- 172.21.15.110:0/511333330 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674014620 con 0x7fe68006e090 2024-09-10T10:04:41.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe67d600640 1 -- 172.21.15.110:0/511333330 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe674019070 con 0x7fe68006e090 2024-09-10T10:04:41.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe67d600640 1 -- 172.21.15.110:0/511333330 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674014a00 con 0x7fe68006e090 2024-09-10T10:04:41.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe688000640 1 -- 172.21.15.110:0/511333330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe680156160 con 0x7fe68006e090 2024-09-10T10:04:41.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe688000640 1 -- 172.21.15.110:0/511333330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6800b0a90 con 0x7fe68006e090 2024-09-10T10:04:41.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe688000640 1 -- 172.21.15.110:0/511333330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68014dbe0 con 0x7fe68006e090 2024-09-10T10:04:41.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe67d600640 1 -- 172.21.15.110:0/511333330 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fe674014ba0 con 0x7fe68006e090 2024-09-10T10:04:41.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe67d600640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe660067e90 0x7fe66006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe67d600640 1 -- 172.21.15.110:0/511333330 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fe674087000 con 0x7fe68006e090 2024-09-10T10:04:41.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.110+0000 7fe684c00640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe660067e90 0x7fe66006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.114+0000 7fe684c00640 1 --2- 172.21.15.110:0/511333330 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe660067e90 0x7fe66006a350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0ad000 tx=0x7fe66c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.114+0000 7fe67d600640 1 -- 172.21.15.110:0/511333330 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe674057250 con 0x7fe68006e090 2024-09-10T10:04:41.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.302+0000 7fe688000640 1 -- 172.21.15.110:0/511333330 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7fe68014c5c0 con 0x7fe660067e90 2024-09-10T10:04:41.308 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:41.306+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:41.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.302+0000 7fe67d600640 1 -- 172.21.15.110:0/511333330 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe68014c5c0 con 0x7fe660067e90 2024-09-10T10:04:41.308 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:41.318 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:41.318 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-10T10:04:41.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffacd800640 1 Processor -- start 2024-09-10T10:04:41.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffacd800640 1 -- start start 2024-09-10T10:04:41.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffacd800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b5590 0x7ffac80cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffacd800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac80b37c0 con 0x7ffac80b2bb0 2024-09-10T10:04:41.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffacd800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac80b3aa0 con 0x7ffac8145f30 2024-09-10T10:04:41.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffacd800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffac80b3d80 con 0x7ffac80b5590 2024-09-10T10:04:41.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac6000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ffac80b2bb0 0x7ffac80cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47638/0 (socket says 172.21.15.110:47638) 2024-09-10T10:04:41.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac6000640 1 -- 172.21.15.110:0/1309839975 learned_addr learned my addr 172.21.15.110:0/1309839975 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:41.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac7400640 1 --2- 172.21.15.110:0/1309839975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b5590 0x7ffac80cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac5600640 1 -- 172.21.15.110:0/1309839975 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1373933966 0 0) 0x7ffac80b37c0 con 0x7ffac80b2bb0 2024-09-10T10:04:41.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac5600640 1 -- 172.21.15.110:0/1309839975 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaac003530 con 0x7ffac80b2bb0 2024-09-10T10:04:41.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac7400640 1 -- 172.21.15.110:0/1309839975 >> v1:172.21.15.145:6789/0 conn(0x7ffac8145f30 legacy=0x7ffac80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac5600640 1 -- 172.21.15.110:0/1309839975 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871885880 0 0) 0x7ffac80b3aa0 con 0x7ffac8145f30 2024-09-10T10:04:41.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac7400640 1 -- 172.21.15.110:0/1309839975 >> v1:172.21.15.110:6789/0 conn(0x7ffac80b2bb0 legacy=0x7ffac80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.454+0000 7ffac7400640 1 -- 172.21.15.110:0/1309839975 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac8146310 con 0x7ffac80b5590 2024-09-10T10:04:41.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffac7400640 1 --2- 172.21.15.110:0/1309839975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b5590 0x7ffac80cfd90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009dc0 tx=0x7ffac001c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d43099193467b804 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffac5600640 1 -- 172.21.15.110:0/1309839975 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac00b2070 con 0x7ffac80b5590 2024-09-10T10:04:41.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffac5600640 1 -- 172.21.15.110:0/1309839975 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac00a57c0 con 0x7ffac80b5590 2024-09-10T10:04:41.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffac5600640 1 -- 172.21.15.110:0/1309839975 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac00a5b70 con 0x7ffac80b5590 2024-09-10T10:04:41.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffacd800640 1 -- 172.21.15.110:0/1309839975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b5590 msgr2=0x7ffac80cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffacd800640 1 --2- 172.21.15.110:0/1309839975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b5590 0x7ffac80cfd90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009dc0 tx=0x7ffac001c900 comp rx=0 tx=0).stop 2024-09-10T10:04:41.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffacd800640 1 -- 172.21.15.110:0/1309839975 shutdown_connections 2024-09-10T10:04:41.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffacd800640 1 --2- 172.21.15.110:0/1309839975 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b5590 0x7ffac80cfd90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffacd800640 1 -- 172.21.15.110:0/1309839975 >> 172.21.15.110:0/1309839975 conn(0x7ffac8062e40 msgr2=0x7ffac806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffacd800640 1 -- 172.21.15.110:0/1309839975 shutdown_connections 2024-09-10T10:04:41.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.458+0000 7ffacd800640 1 -- 172.21.15.110:0/1309839975 wait complete. 2024-09-10T10:04:41.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 Processor -- start 2024-09-10T10:04:41.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 -- start start 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b2bb0 0x7ffac81627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac80b5590 0x7ffac815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffac8145f30 0x7ffac815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8169d40 con 0x7ffac80b5590 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8169bc0 con 0x7ffac8145f30 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffac8169ec0 con 0x7ffac80b2bb0 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac80b5590 0x7ffac815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac80b5590 0x7ffac815dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55116/0 (socket says 172.21.15.110:55116) 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 -- 172.21.15.110:0/1145384818 learned_addr learned my addr 172.21.15.110:0/1145384818 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:41.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6a00640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b2bb0 0x7ffac81627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac7400640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffac8145f30 0x7ffac815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 -- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b2bb0 msgr2=0x7ffac81627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffac80b2bb0 0x7ffac81627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 -- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffac8145f30 msgr2=0x7ffac815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffac8145f30 0x7ffac815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 -- 172.21.15.110:0/1145384818 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac8144f80 con 0x7ffac80b5590 2024-09-10T10:04:41.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac7400640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffac8145f30 0x7ffac815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:41.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffac6000640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac80b5590 0x7ffac815dce0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ffab400c3c0 tx=0x7ffab400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffabea00640 1 -- 172.21.15.110:0/1145384818 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4017070 con 0x7ffac80b5590 2024-09-10T10:04:41.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffabea00640 1 -- 172.21.15.110:0/1145384818 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab4005d10 con 0x7ffac80b5590 2024-09-10T10:04:41.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffabea00640 1 -- 172.21.15.110:0/1145384818 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4007310 con 0x7ffac80b5590 2024-09-10T10:04:41.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 -- 172.21.15.110:0/1145384818 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac8145260 con 0x7ffac80b5590 2024-09-10T10:04:41.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 -- 172.21.15.110:0/1145384818 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac8142520 con 0x7ffac80b5590 2024-09-10T10:04:41.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.462+0000 7ffacd800640 1 -- 172.21.15.110:0/1145384818 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac80b24a0 con 0x7ffac80b5590 2024-09-10T10:04:41.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.466+0000 7ffabea00640 1 -- 172.21.15.110:0/1145384818 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7ffab401a070 con 0x7ffac80b5590 2024-09-10T10:04:41.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.466+0000 7ffabea00640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffaa8067e60 0x7ffaa806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.466+0000 7ffabea00640 1 -- 172.21.15.110:0/1145384818 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7ffab4085aa0 con 0x7ffac80b5590 2024-09-10T10:04:41.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.466+0000 7ffac6a00640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffaa8067e60 0x7ffaa806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.466+0000 7ffac6a00640 1 --2- 172.21.15.110:0/1145384818 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffaa8067e60 0x7ffaa806a320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffab8007fd0 tx=0x7ffab8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.470+0000 7ffabea00640 1 -- 172.21.15.110:0/1145384818 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ffab4055760 con 0x7ffac80b5590 2024-09-10T10:04:41.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.654+0000 7ffacd800640 1 -- 172.21.15.110:0/1145384818 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7ffac80cd360 con 0x7ffaa8067e60 2024-09-10T10:04:41.659 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:41.658+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:41.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.654+0000 7ffabea00640 1 -- 172.21.15.110:0/1145384818 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ffac80cd360 con 0x7ffaa8067e60 2024-09-10T10:04:41.660 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:41.669 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:41.669 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T10:04:41.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2939c00640 1 Processor -- start 2024-09-10T10:04:41.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2939c00640 1 -- start start 2024-09-10T10:04:41.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2939c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293406f1e0 0x7f2934166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2939c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2934155d50 con 0x7f2934155970 2024-09-10T10:04:41.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2939c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29340b34d0 con 0x7f293415c560 2024-09-10T10:04:41.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2939c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f29340b4570 con 0x7f293406f1e0 2024-09-10T10:04:41.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2932a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2934155970 0x7f29340cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47652/0 (socket says 172.21.15.110:47652) 2024-09-10T10:04:41.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2932a00640 1 -- 172.21.15.110:0/3609102419 learned_addr learned my addr 172.21.15.110:0/3609102419 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:41.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2933400640 1 --2- 172.21.15.110:0/3609102419 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293406f1e0 0x7f2934166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2931600640 1 -- 172.21.15.110:0/3609102419 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511783429 0 0) 0x7f2934155d50 con 0x7f2934155970 2024-09-10T10:04:41.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2931600640 1 -- 172.21.15.110:0/3609102419 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2918003530 con 0x7f2934155970 2024-09-10T10:04:41.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2933400640 1 -- 172.21.15.110:0/3609102419 >> v1:172.21.15.145:6789/0 conn(0x7f293415c560 legacy=0x7f293406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2931600640 1 -- 172.21.15.110:0/3609102419 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477662774 0 0) 0x7f29340b34d0 con 0x7f293415c560 2024-09-10T10:04:41.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2933400640 1 -- 172.21.15.110:0/3609102419 >> v1:172.21.15.110:6789/0 conn(0x7f2934155970 legacy=0x7f29340cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.806+0000 7f2933400640 1 -- 172.21.15.110:0/3609102419 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29340b37b0 con 0x7f293406f1e0 2024-09-10T10:04:41.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2933400640 1 --2- 172.21.15.110:0/3609102419 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293406f1e0 0x7f2934166020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f292c009dc0 tx=0x7f292c01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb2f159533b50e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2931600640 1 -- 172.21.15.110:0/3609102419 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c0b2070 con 0x7f293406f1e0 2024-09-10T10:04:41.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2931600640 1 -- 172.21.15.110:0/3609102419 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c0a5a20 con 0x7f293406f1e0 2024-09-10T10:04:41.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2931600640 1 -- 172.21.15.110:0/3609102419 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c0a5dd0 con 0x7f293406f1e0 2024-09-10T10:04:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2939c00640 1 -- 172.21.15.110:0/3609102419 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293406f1e0 msgr2=0x7f2934166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2939c00640 1 --2- 172.21.15.110:0/3609102419 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293406f1e0 0x7f2934166020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f292c009dc0 tx=0x7f292c01c960 comp rx=0 tx=0).stop 2024-09-10T10:04:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2939c00640 1 -- 172.21.15.110:0/3609102419 shutdown_connections 2024-09-10T10:04:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2939c00640 1 --2- 172.21.15.110:0/3609102419 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293406f1e0 0x7f2934166020 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2939c00640 1 -- 172.21.15.110:0/3609102419 >> 172.21.15.110:0/3609102419 conn(0x7f293406bc50 msgr2=0x7f293406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2939c00640 1 -- 172.21.15.110:0/3609102419 shutdown_connections 2024-09-10T10:04:41.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.810+0000 7f2939c00640 1 -- 172.21.15.110:0/3609102419 wait complete. 2024-09-10T10:04:41.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 Processor -- start 2024-09-10T10:04:41.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 -- start start 2024-09-10T10:04:41.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f293406f1e0 0x7f293414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2934155970 0x7f2934147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293415c560 0x7f293414dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2934168f00 con 0x7f2934155970 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2934168d80 con 0x7f293406f1e0 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2934169080 con 0x7f293415c560 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2934155970 0x7f2934147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2934155970 0x7f2934147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55132/0 (socket says 172.21.15.110:55132) 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 -- 172.21.15.110:0/380401574 learned_addr learned my addr 172.21.15.110:0/380401574 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2933400640 1 --2- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293415c560 0x7f293414dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932a00640 1 --2- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f293406f1e0 0x7f293414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 -- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293415c560 msgr2=0x7f293414dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 --2- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f293415c560 0x7f293414dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 -- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f293406f1e0 msgr2=0x7f293414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 --2- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f293406f1e0 0x7f293414f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 -- 172.21.15.110:0/380401574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293414fde0 con 0x7f2934155970 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2932000640 1 --2- 172.21.15.110:0/380401574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2934155970 0x7f2934147d70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f292000c4b0 tx=0x7f292000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f292aa00640 1 -- 172.21.15.110:0/380401574 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920005480 con 0x7f2934155970 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f292aa00640 1 -- 172.21.15.110:0/380401574 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2920003600 con 0x7f2934155970 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f292aa00640 1 -- 172.21.15.110:0/380401574 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200039f0 con 0x7f2934155970 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 -- 172.21.15.110:0/380401574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2934158c10 con 0x7f2934155970 2024-09-10T10:04:41.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.814+0000 7f2939c00640 1 -- 172.21.15.110:0/380401574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2934148760 con 0x7f2934155970 2024-09-10T10:04:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.818+0000 7f2939c00640 1 -- 172.21.15.110:0/380401574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29340cc8c0 con 0x7f2934155970 2024-09-10T10:04:41.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.818+0000 7f292aa00640 1 -- 172.21.15.110:0/380401574 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f2920003b90 con 0x7f2934155970 2024-09-10T10:04:41.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.818+0000 7f292aa00640 1 --2- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2914067e60 0x7f291406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.818+0000 7f292aa00640 1 -- 172.21.15.110:0/380401574 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f2920086210 con 0x7f2934155970 2024-09-10T10:04:41.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.818+0000 7f2932a00640 1 --2- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2914067e60 0x7f291406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.818+0000 7f2932a00640 1 --2- 172.21.15.110:0/380401574 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2914067e60 0x7f291406a320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f293414def0 tx=0x7f2924000bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:41.822+0000 7f292aa00640 1 -- 172.21.15.110:0/380401574 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2920055ea0 con 0x7f2934155970 2024-09-10T10:04:42.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.006+0000 7f2939c00640 1 -- 172.21.15.110:0/380401574 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f29341589b0 con 0x7f2914067e60 2024-09-10T10:04:42.011 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:42.010+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.006+0000 7f292aa00640 1 -- 172.21.15.110:0/380401574 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f2924005860 con 0x7f2914067e60 2024-09-10T10:04:42.012 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:42.021 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:42.021 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T10:04:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.158+0000 7fef24e00640 1 Processor -- start 2024-09-10T10:04:42.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.158+0000 7fef24e00640 1 -- start start 2024-09-10T10:04:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef24e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef20147f00 0x7fef200ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef24e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fef200aed80 0x7fef2006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef24e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef200af1a0 con 0x7fef2006ec20 2024-09-10T10:04:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef24e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fef200b34d0 con 0x7fef200aed80 2024-09-10T10:04:42.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef24e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fef201482e0 con 0x7fef20147f00 2024-09-10T10:04:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1d400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fef200aed80 0x7fef2006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef20147f00 0x7fef200ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1d400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fef200aed80 0x7fef2006e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:36700/0 (socket says 172.21.15.110:36700) 2024-09-10T10:04:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1d400640 1 -- 172.21.15.110:0/1286572417 learned_addr learned my addr 172.21.15.110:0/1286572417 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:42.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1e800640 1 --1- 172.21.15.110:0/1286572417 >> v1:172.21.15.145:6789/0 conn(0x7fef2006ec20 0x7fef2006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:35166/0 (socket says 172.21.15.110:35166) 2024-09-10T10:04:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1d400640 1 -- 172.21.15.110:0/1286572417 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef20147f00 msgr2=0x7fef200ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1d400640 1 --2- 172.21.15.110:0/1286572417 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef20147f00 0x7fef200ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1d400640 1 -- 172.21.15.110:0/1286572417 >> v1:172.21.15.145:6789/0 conn(0x7fef2006ec20 legacy=0x7fef2006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.162+0000 7fef1d400640 1 -- 172.21.15.110:0/1286572417 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2015ee20 con 0x7fef200aed80 2024-09-10T10:04:42.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef1d400640 1 --2- 172.21.15.110:0/1286572417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef200aed80 0x7fef2006e6e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009870 tx=0x7fef0c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b460c302a7c56c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef1ca00640 1 -- 172.21.15.110:0/1286572417 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c0b0070 con 0x7fef200aed80 2024-09-10T10:04:42.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef1ca00640 1 -- 172.21.15.110:0/1286572417 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0c0ab600 con 0x7fef200aed80 2024-09-10T10:04:42.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef1ca00640 1 -- 172.21.15.110:0/1286572417 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c0ab9d0 con 0x7fef200aed80 2024-09-10T10:04:42.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 -- 172.21.15.110:0/1286572417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef200aed80 msgr2=0x7fef2006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 --2- 172.21.15.110:0/1286572417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef200aed80 0x7fef2006e6e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009870 tx=0x7fef0c01c560 comp rx=0 tx=0).stop 2024-09-10T10:04:42.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 -- 172.21.15.110:0/1286572417 shutdown_connections 2024-09-10T10:04:42.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 --2- 172.21.15.110:0/1286572417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef200aed80 0x7fef2006e6e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 --2- 172.21.15.110:0/1286572417 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef20147f00 0x7fef200ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 -- 172.21.15.110:0/1286572417 >> 172.21.15.110:0/1286572417 conn(0x7fef2006bc50 msgr2=0x7fef2006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:42.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 -- 172.21.15.110:0/1286572417 shutdown_connections 2024-09-10T10:04:42.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 -- 172.21.15.110:0/1286572417 wait complete. 2024-09-10T10:04:42.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 Processor -- start 2024-09-10T10:04:42.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.166+0000 7fef24e00640 1 -- start start 2024-09-10T10:04:42.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef2006ec20 0x7fef200cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef200aed80 0x7fef20145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef20147f00 0x7fef201462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef20073320 con 0x7fef20147f00 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fef2006f050 con 0x7fef200aed80 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fef200734a0 con 0x7fef2006ec20 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef20147f00 0x7fef201462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef200aed80 0x7fef20145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef20147f00 0x7fef201462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55148/0 (socket says 172.21.15.110:55148) 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef2006ec20 0x7fef200cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 -- 172.21.15.110:0/1302269987 learned_addr learned my addr 172.21.15.110:0/1302269987 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 -- 172.21.15.110:0/1302269987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef2006ec20 msgr2=0x7fef200cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 --2- 172.21.15.110:0/1302269987 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fef2006ec20 0x7fef200cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 -- 172.21.15.110:0/1302269987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef200aed80 msgr2=0x7fef20145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 --2- 172.21.15.110:0/1302269987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fef200aed80 0x7fef20145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 -- 172.21.15.110:0/1302269987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef200ce640 con 0x7fef20147f00 2024-09-10T10:04:42.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef1e800640 1 --2- 172.21.15.110:0/1302269987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef20147f00 0x7fef201462e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fef1800c3c0 tx=0x7fef1800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef16000640 1 -- 172.21.15.110:0/1302269987 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180074e0 con 0x7fef20147f00 2024-09-10T10:04:42.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef16000640 1 -- 172.21.15.110:0/1302269987 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef18007680 con 0x7fef20147f00 2024-09-10T10:04:42.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef16000640 1 -- 172.21.15.110:0/1302269987 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18004a10 con 0x7fef20147f00 2024-09-10T10:04:42.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 -- 172.21.15.110:0/1302269987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef200ce920 con 0x7fef20147f00 2024-09-10T10:04:42.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 -- 172.21.15.110:0/1302269987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef200ceee0 con 0x7fef20147f00 2024-09-10T10:04:42.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef24e00640 1 -- 172.21.15.110:0/1302269987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef2006e6e0 con 0x7fef20147f00 2024-09-10T10:04:42.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.170+0000 7fef16000640 1 -- 172.21.15.110:0/1302269987 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fef1801a020 con 0x7fef20147f00 2024-09-10T10:04:42.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.174+0000 7fef16000640 1 --2- 172.21.15.110:0/1302269987 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fef00067e60 0x7fef0006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.174+0000 7fef16000640 1 -- 172.21.15.110:0/1302269987 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fef18085d10 con 0x7fef20147f00 2024-09-10T10:04:42.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.174+0000 7fef1de00640 1 --2- 172.21.15.110:0/1302269987 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fef00067e60 0x7fef0006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.174+0000 7fef1de00640 1 --2- 172.21.15.110:0/1302269987 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fef00067e60 0x7fef0006a320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef100113e0 tx=0x7fef1000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.178+0000 7fef16000640 1 -- 172.21.15.110:0/1302269987 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fef180559a0 con 0x7fef20147f00 2024-09-10T10:04:42.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.366+0000 7fef24e00640 1 -- 172.21.15.110:0/1302269987 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fef2015e640 con 0x7fef00067e60 2024-09-10T10:04:42.371 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:42.370+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.366+0000 7fef16000640 1 -- 172.21.15.110:0/1302269987 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fef2015e640 con 0x7fef00067e60 2024-09-10T10:04:42.372 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:42.380 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:42.380 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T10:04:42.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b41200640 1 Processor -- start 2024-09-10T10:04:42.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b41200640 1 -- start start 2024-09-10T10:04:42.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b41200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7b3c145c70 0x7f7b3c073f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b41200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b3c15ccf0 con 0x7f7b3c0712f0 2024-09-10T10:04:42.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b41200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b3c071a80 con 0x7f7b3c0b5da0 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b41200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f7b3c15cab0 con 0x7f7b3c145c70 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3aa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f7b3c0712f0 0x7f7b3c0b5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47654/0 (socket says 172.21.15.110:47654) 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3aa00640 1 -- 172.21.15.110:0/40742615 learned_addr learned my addr 172.21.15.110:0/40742615 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3b400640 1 --2- 172.21.15.110:0/40742615 >> v2:172.21.15.145:3300/0 conn(0x7f7b3c145c70 0x7f7b3c073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3363577610 0 0) 0x7f7b3c15ccf0 con 0x7f7b3c0712f0 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b20003530 con 0x7f7b3c0712f0 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1935692779 0 0) 0x7f7b20003530 con 0x7f7b3c0712f0 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 >> v2:172.21.15.145:3300/0 conn(0x7f7b3c145c70 msgr2=0x7f7b3c073f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 --2- 172.21.15.110:0/40742615 >> v2:172.21.15.145:3300/0 conn(0x7f7b3c145c70 0x7f7b3c073f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 >> v1:172.21.15.145:6789/0 conn(0x7f7b3c0b5da0 legacy=0x7f7b3c15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c071d60 con 0x7f7b3c0712f0 2024-09-10T10:04:42.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f7b28001680 con 0x7f7b3c0712f0 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 >> v1:172.21.15.110:6789/0 conn(0x7f7b3c0712f0 legacy=0x7f7b3c0b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b200039c0 0x7f7b20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b20004810 0x7f7b20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b200051d0 0x7f7b2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c15cab0 con 0x7f7b200051d0 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2000c170 con 0x7f7b200039c0 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2000c370 con 0x7f7b20004810 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3be00640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b200051d0 0x7f7b2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3b400640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b200039c0 0x7f7b20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3aa00640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b20004810 0x7f7b20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3be00640 1 -- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b20004810 msgr2=0x7f7b20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3be00640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b20004810 0x7f7b20004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3be00640 1 -- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b200039c0 msgr2=0x7f7b20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:42.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3be00640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b200039c0 0x7f7b20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.518+0000 7f7b3be00640 1 -- 172.21.15.110:0/40742615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b20003530 con 0x7f7b200051d0 2024-09-10T10:04:42.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b3b400640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b200039c0 0x7f7b20003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:42.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b3aa00640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b20004810 0x7f7b20004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:42.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b3be00640 1 -- 172.21.15.110:0/40742615 reap_dead start 2024-09-10T10:04:42.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b3be00640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b200051d0 0x7f7b2000b9d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7b34001e40 tx=0x7f7b3401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a4dfc70a2d360e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34001a90 con 0x7f7b200051d0 2024-09-10T10:04:42.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b34003b80 con 0x7f7b200051d0 2024-09-10T10:04:42.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b3a000640 1 -- 172.21.15.110:0/40742615 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340050d0 con 0x7f7b200051d0 2024-09-10T10:04:42.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b41200640 1 -- 172.21.15.110:0/40742615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b200051d0 msgr2=0x7f7b2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b41200640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b200051d0 0x7f7b2000b9d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7b34001e40 tx=0x7f7b3401c760 comp rx=0 tx=0).stop 2024-09-10T10:04:42.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b41200640 1 -- 172.21.15.110:0/40742615 shutdown_connections 2024-09-10T10:04:42.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b41200640 1 --2- 172.21.15.110:0/40742615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b200051d0 0x7f7b2000b9d0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b41200640 1 -- 172.21.15.110:0/40742615 >> 172.21.15.110:0/40742615 conn(0x7f7b3c06d0f0 msgr2=0x7f7b3c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:42.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b41200640 1 -- 172.21.15.110:0/40742615 shutdown_connections 2024-09-10T10:04:42.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.522+0000 7f7b41200640 1 -- 172.21.15.110:0/40742615 wait complete. 2024-09-10T10:04:42.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 Processor -- start 2024-09-10T10:04:42.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 -- start start 2024-09-10T10:04:42.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b200051d0 0x7f7b3c0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b3c15c000 0x7f7b3c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b3c0c5da0 0x7f7b3c144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c146050 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c072040 con 0x7f7b3c15c000 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7b3c1461d0 con 0x7f7b200051d0 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b3c0c5da0 0x7f7b3c144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b3c0c5da0 0x7f7b3c144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55164/0 (socket says 172.21.15.110:55164) 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 -- 172.21.15.110:0/2053244740 learned_addr learned my addr 172.21.15.110:0/2053244740 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3b400640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b200051d0 0x7f7b3c0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3aa00640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b3c15c000 0x7f7b3c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 -- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b200051d0 msgr2=0x7f7b3c0aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7b200051d0 0x7f7b3c0aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 -- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b3c15c000 msgr2=0x7f7b3c0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b3c15c000 0x7f7b3c0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 -- 172.21.15.110:0/2053244740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c130ed0 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3aa00640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b3c15c000 0x7f7b3c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:42.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b3be00640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b3c0c5da0 0x7f7b3c144c40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f7b3401c590 tx=0x7f7b20005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b33400640 1 -- 172.21.15.110:0/2053244740 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34007dc0 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b33400640 1 -- 172.21.15.110:0/2053244740 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b34006d70 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 -- 172.21.15.110:0/2053244740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3c15c480 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b33400640 1 -- 172.21.15.110:0/2053244740 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34001a90 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.526+0000 7f7b41200640 1 -- 172.21.15.110:0/2053244740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b3c131450 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.530+0000 7f7b41200640 1 -- 172.21.15.110:0/2053244740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3c073f10 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.530+0000 7f7b33400640 1 -- 172.21.15.110:0/2053244740 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f7b34007270 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.530+0000 7f7b33400640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b1c067e90 0x7f7b1c06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.530+0000 7f7b33400640 1 -- 172.21.15.110:0/2053244740 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f7b341201f0 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.530+0000 7f7b3b400640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b1c067e90 0x7f7b1c06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.530+0000 7f7b3b400640 1 --2- 172.21.15.110:0/2053244740 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7b1c067e90 0x7f7b1c06a350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c006f90 tx=0x7f7b2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.534+0000 7f7b33400640 1 -- 172.21.15.110:0/2053244740 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7b34016470 con 0x7f7b3c0c5da0 2024-09-10T10:04:42.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.718+0000 7f7b41200640 1 -- 172.21.15.110:0/2053244740 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f7b3c1e4ad0 con 0x7f7b1c067e90 2024-09-10T10:04:42.724 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:42.722+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.718+0000 7f7b33400640 1 -- 172.21.15.110:0/2053244740 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7b3c1e4ad0 con 0x7f7b1c067e90 2024-09-10T10:04:42.724 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:42.733 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:42.733 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.870+0000 7fecd3000640 1 Processor -- start 2024-09-10T10:04:42.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.870+0000 7fecd3000640 1 -- start start 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.870+0000 7fecd3000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feccc0cd510 0x7feccc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7fecd3000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 0x7feccc06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7fecd3000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feccc072150 con 0x7feccc0cfe30 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7fecd3000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7feccc15d9a0 con 0x7feccc0cd510 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7fecd3000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feccc06ea60 con 0x7feccc06e090 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccb400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feccc0cd510 0x7feccc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccbe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7feccc0cfe30 0x7feccc071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47656/0 (socket says 172.21.15.110:47656) 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccbe00640 1 -- 172.21.15.110:0/2466545480 learned_addr learned my addr 172.21.15.110:0/2466545480 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:42.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7fecd1000640 1 --2- 172.21.15.110:0/2466545480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 0x7feccc06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccb400640 1 -- 172.21.15.110:0/2466545480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 msgr2=0x7feccc06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccb400640 1 --2- 172.21.15.110:0/2466545480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 0x7feccc06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccb400640 1 -- 172.21.15.110:0/2466545480 >> v1:172.21.15.110:6789/0 conn(0x7feccc0cfe30 legacy=0x7feccc071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccb400640 1 -- 172.21.15.110:0/2466545480 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc06d2d0 con 0x7feccc0cd510 2024-09-10T10:04:42.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7fecd1000640 1 --2- 172.21.15.110:0/2466545480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 0x7feccc06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccb400640 1 --2- 172.21.15.110:0/2466545480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feccc0cd510 0x7feccc0cd9a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fecb80097c0 tx=0x7fecb801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b854febe46efe8d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccaa00640 1 -- 172.21.15.110:0/2466545480 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80b0070 con 0x7feccc0cd510 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccaa00640 1 -- 172.21.15.110:0/2466545480 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb80ab600 con 0x7feccc0cd510 2024-09-10T10:04:42.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.874+0000 7feccaa00640 1 -- 172.21.15.110:0/2466545480 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80ab9d0 con 0x7feccc0cd510 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- 172.21.15.110:0/2466545480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feccc0cd510 msgr2=0x7feccc0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 --2- 172.21.15.110:0/2466545480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feccc0cd510 0x7feccc0cd9a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fecb80097c0 tx=0x7fecb801c410 comp rx=0 tx=0).stop 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- 172.21.15.110:0/2466545480 shutdown_connections 2024-09-10T10:04:42.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 --2- 172.21.15.110:0/2466545480 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 0x7feccc06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 --2- 172.21.15.110:0/2466545480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feccc0cd510 0x7feccc0cd9a0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- 172.21.15.110:0/2466545480 >> 172.21.15.110:0/2466545480 conn(0x7feccc06bc50 msgr2=0x7feccc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- 172.21.15.110:0/2466545480 shutdown_connections 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- 172.21.15.110:0/2466545480 wait complete. 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 Processor -- start 2024-09-10T10:04:42.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- start start 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 0x7feccc142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feccc0cd510 0x7feccc0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feccc0cfe30 0x7feccc15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feccc169180 con 0x7feccc0cd510 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feccc15d6c0 con 0x7feccc0cfe30 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd3000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feccc169300 con 0x7feccc06e090 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feccc0cd510 0x7feccc0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feccc0cd510 0x7feccc0b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55174/0 (socket says 172.21.15.110:55174) 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccb400640 1 -- 172.21.15.110:0/2269544517 learned_addr learned my addr 172.21.15.110:0/2269544517 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccbe00640 1 --2- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 0x7feccc142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7fecd1000640 1 --2- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feccc0cfe30 0x7feccc15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccb400640 1 -- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 msgr2=0x7feccc142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccb400640 1 --2- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feccc06e090 0x7feccc142e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccb400640 1 -- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feccc0cfe30 msgr2=0x7feccc15eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccb400640 1 --2- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feccc0cfe30 0x7feccc15eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.878+0000 7feccb400640 1 -- 172.21.15.110:0/2269544517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc073110 con 0x7feccc0cd510 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7feccb400640 1 --2- 172.21.15.110:0/2269544517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feccc0cd510 0x7feccc0b2970 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fecb801a040 tx=0x7fecb80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecc3e00640 1 -- 172.21.15.110:0/2269544517 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80b8070 con 0x7feccc0cd510 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecc3e00640 1 -- 172.21.15.110:0/2269544517 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb80b4630 con 0x7feccc0cd510 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecd3000640 1 -- 172.21.15.110:0/2269544517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc143800 con 0x7feccc0cd510 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecc3e00640 1 -- 172.21.15.110:0/2269544517 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80b0040 con 0x7feccc0cd510 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecd3000640 1 -- 172.21.15.110:0/2269544517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc14c550 con 0x7feccc0cd510 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecd3000640 1 -- 172.21.15.110:0/2269544517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc06e500 con 0x7feccc0cd510 2024-09-10T10:04:42.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecc3e00640 1 -- 172.21.15.110:0/2269544517 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fecb80bb070 con 0x7feccc0cd510 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecc3e00640 1 --2- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecac067e60 0x7fecac06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.882+0000 7fecc3e00640 1 -- 172.21.15.110:0/2269544517 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fecb8120590 con 0x7feccc0cd510 2024-09-10T10:04:42.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.886+0000 7feccbe00640 1 --2- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecac067e60 0x7fecac06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.886+0000 7feccbe00640 1 --2- 172.21.15.110:0/2269544517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecac067e60 0x7fecac06a320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0012f0 tx=0x7fecbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:42.890+0000 7fecc3e00640 1 -- 172.21.15.110:0/2269544517 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fecb8016090 con 0x7feccc0cd510 2024-09-10T10:04:43.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.070+0000 7fecd3000640 1 -- 172.21.15.110:0/2269544517 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7feccc06c160 con 0x7fecac067e60 2024-09-10T10:04:43.078 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:43.078+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.074+0000 7fecc3e00640 1 -- 172.21.15.110:0/2269544517 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7feccc06c160 con 0x7fecac067e60 2024-09-10T10:04:43.078 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:43.087 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:43.088 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T10:04:43.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.222+0000 7fc9f4000640 1 Processor -- start 2024-09-10T10:04:43.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.222+0000 7fc9f4000640 1 -- start start 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f4000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc9ec0cfe30 0x7fc9ec071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f4000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc9ec06e7a0 0x7fc9ec0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f4000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9ec06d2d0 con 0x7fc9ec0cd510 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f4000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc9ec15d9a0 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f4000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc9ec06d5b0 con 0x7fc9ec06e7a0 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f1600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc9ec0cfe30 0x7fc9ec071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f1600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc9ec0cfe30 0x7fc9ec071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55176/0 (socket says 172.21.15.110:55176) 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f1600640 1 -- 172.21.15.110:0/1022303644 learned_addr learned my addr 172.21.15.110:0/1022303644 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f2000640 1 --2- 172.21.15.110:0/1022303644 >> v2:172.21.15.145:3300/0 conn(0x7fc9ec06e7a0 0x7fc9ec0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f1600640 1 -- 172.21.15.110:0/1022303644 >> v2:172.21.15.145:3300/0 conn(0x7fc9ec06e7a0 msgr2=0x7fc9ec0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f1600640 1 --2- 172.21.15.110:0/1022303644 >> v2:172.21.15.145:3300/0 conn(0x7fc9ec06e7a0 0x7fc9ec0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f1600640 1 -- 172.21.15.110:0/1022303644 >> v1:172.21.15.145:6789/0 conn(0x7fc9ec0cd510 legacy=0x7fc9ec06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f1600640 1 -- 172.21.15.110:0/1022303644 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec06d890 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.226+0000 7fc9f2000640 1 --2- 172.21.15.110:0/1022303644 >> v2:172.21.15.145:3300/0 conn(0x7fc9ec06e7a0 0x7fc9ec0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:43.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f1600640 1 --2- 172.21.15.110:0/1022303644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9ec0cfe30 0x7fc9ec071a40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec06cb00 tx=0x7fc9e001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a73b23459a2bd76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9ebe00640 1 -- 172.21.15.110:0/1022303644 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00a5730 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9ebe00640 1 -- 172.21.15.110:0/1022303644 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e00a58d0 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9ebe00640 1 -- 172.21.15.110:0/1022303644 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00a5ca0 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- 172.21.15.110:0/1022303644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9ec0cfe30 msgr2=0x7fc9ec071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 --2- 172.21.15.110:0/1022303644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9ec0cfe30 0x7fc9ec071a40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec06cb00 tx=0x7fc9e001c960 comp rx=0 tx=0).stop 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- 172.21.15.110:0/1022303644 shutdown_connections 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 --2- 172.21.15.110:0/1022303644 >> v2:172.21.15.145:3300/0 conn(0x7fc9ec06e7a0 0x7fc9ec0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 --2- 172.21.15.110:0/1022303644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9ec0cfe30 0x7fc9ec071a40 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- 172.21.15.110:0/1022303644 >> 172.21.15.110:0/1022303644 conn(0x7fc9ec06bc50 msgr2=0x7fc9ec06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- 172.21.15.110:0/1022303644 shutdown_connections 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- 172.21.15.110:0/1022303644 wait complete. 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 Processor -- start 2024-09-10T10:04:43.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- start start 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9ec06e7a0 0x7fc9ec14a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9ec0cd510 0x7fc9ec155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9ec0cfe30 0x7fc9ec1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec072190 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec15d6c0 con 0x7fc9ec06e7a0 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f4000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ec169180 con 0x7fc9ec0cd510 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9ec0cfe30 0x7fc9ec1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f2000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9ec0cfe30 0x7fc9ec1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55180/0 (socket says 172.21.15.110:55180) 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f2000640 1 -- 172.21.15.110:0/4282335816 learned_addr learned my addr 172.21.15.110:0/4282335816 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.230+0000 7fc9f1600640 1 --2- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9ec06e7a0 0x7fc9ec14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f0c00640 1 --2- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9ec0cd510 0x7fc9ec155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f2000640 1 -- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9ec0cd510 msgr2=0x7fc9ec155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f2000640 1 --2- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc9ec0cd510 0x7fc9ec155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f2000640 1 -- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9ec06e7a0 msgr2=0x7fc9ec14a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f2000640 1 --2- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9ec06e7a0 0x7fc9ec14a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f2000640 1 -- 172.21.15.110:0/4282335816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec155160 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f2000640 1 --2- 172.21.15.110:0/4282335816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9ec0cfe30 0x7fc9ec1529c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400d800 tx=0x7fc9e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9e9600640 1 -- 172.21.15.110:0/4282335816 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e40096a0 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9e9600640 1 -- 172.21.15.110:0/4282335816 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e4019070 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9e9600640 1 -- 172.21.15.110:0/4282335816 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4009a80 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f4000640 1 -- 172.21.15.110:0/4282335816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ec14b3e0 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f4000640 1 -- 172.21.15.110:0/4282335816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ec1543a0 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9f4000640 1 -- 172.21.15.110:0/4282335816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ec15a290 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9e9600640 1 -- 172.21.15.110:0/4282335816 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fc9e4009c20 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9e9600640 1 --2- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc9cc067e60 0x7fc9cc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.234+0000 7fc9e9600640 1 -- 172.21.15.110:0/4282335816 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fc9e4086aa0 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.238+0000 7fc9f1600640 1 --2- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc9cc067e60 0x7fc9cc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.238+0000 7fc9f1600640 1 --2- 172.21.15.110:0/4282335816 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc9cc067e60 0x7fc9cc06a320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00ae000 tx=0x7fc9e00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.242+0000 7fc9e9600640 1 -- 172.21.15.110:0/4282335816 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc9e4056d80 con 0x7fc9ec0cfe30 2024-09-10T10:04:43.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.426+0000 7fc9f4000640 1 -- 172.21.15.110:0/4282335816 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fc9ec153e70 con 0x7fc9cc067e60 2024-09-10T10:04:43.432 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:43.430+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.430+0000 7fc9e9600640 1 -- 172.21.15.110:0/4282335816 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc9ec153e70 con 0x7fc9cc067e60 2024-09-10T10:04:43.433 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:43.441 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:43.441 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-10T10:04:43.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.578+0000 7f7742c00640 1 Processor -- start 2024-09-10T10:04:43.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.578+0000 7f7742c00640 1 -- start start 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7742c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f773c155970 0x7f773c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7742c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f773c06f1e0 0x7f773c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7742c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f773c155d50 con 0x7f773c15c560 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7742c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f773c0b4570 con 0x7f773c06f1e0 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7742c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f773c0b34d0 con 0x7f773c155970 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7740c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f773c06f1e0 0x7f773c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7740c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f773c06f1e0 0x7f773c0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55188/0 (socket says 172.21.15.110:55188) 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7740c00640 1 -- 172.21.15.110:0/884175001 learned_addr learned my addr 172.21.15.110:0/884175001 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f773be00640 1 --2- 172.21.15.110:0/884175001 >> v2:172.21.15.145:3300/0 conn(0x7f773c155970 0x7f773c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7740c00640 1 -- 172.21.15.110:0/884175001 >> v2:172.21.15.145:3300/0 conn(0x7f773c155970 msgr2=0x7f773c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7740c00640 1 --2- 172.21.15.110:0/884175001 >> v2:172.21.15.145:3300/0 conn(0x7f773c155970 0x7f773c0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7740c00640 1 -- 172.21.15.110:0/884175001 >> v1:172.21.15.145:6789/0 conn(0x7f773c15c560 legacy=0x7f773c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.582+0000 7f7740c00640 1 -- 172.21.15.110:0/884175001 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c0b37b0 con 0x7f773c06f1e0 2024-09-10T10:04:43.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7740c00640 1 --2- 172.21.15.110:0/884175001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f773c06f1e0 0x7f773c0ce420 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f773c06cc30 tx=0x7f773401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b962851e009ba005 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f773aa00640 1 -- 172.21.15.110:0/884175001 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340aa610 con 0x7f773c06f1e0 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f773aa00640 1 -- 172.21.15.110:0/884175001 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77340af070 con 0x7f773c06f1e0 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f773aa00640 1 -- 172.21.15.110:0/884175001 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340aa9d0 con 0x7f773c06f1e0 2024-09-10T10:04:43.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- 172.21.15.110:0/884175001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f773c06f1e0 msgr2=0x7f773c0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 --2- 172.21.15.110:0/884175001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f773c06f1e0 0x7f773c0ce420 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f773c06cc30 tx=0x7f773401c4b0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- 172.21.15.110:0/884175001 shutdown_connections 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 --2- 172.21.15.110:0/884175001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f773c06f1e0 0x7f773c0ce420 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 --2- 172.21.15.110:0/884175001 >> v2:172.21.15.145:3300/0 conn(0x7f773c155970 0x7f773c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- 172.21.15.110:0/884175001 >> 172.21.15.110:0/884175001 conn(0x7f773c06bc50 msgr2=0x7f773c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- 172.21.15.110:0/884175001 shutdown_connections 2024-09-10T10:04:43.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- 172.21.15.110:0/884175001 wait complete. 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 Processor -- start 2024-09-10T10:04:43.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- start start 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f773c06f1e0 0x7f773c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f773c155970 0x7f773c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f773c15c560 0x7f773c153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f773c0cf170 con 0x7f773c06f1e0 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f773c0cce00 con 0x7f773c155970 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7742c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f773c0cf2f0 con 0x7f773c15c560 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f773be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f773c06f1e0 0x7f773c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f773be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f773c06f1e0 0x7f773c152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55204/0 (socket says 172.21.15.110:55204) 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f773be00640 1 -- 172.21.15.110:0/3665649207 learned_addr learned my addr 172.21.15.110:0/3665649207 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f773b400640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f773c155970 0x7f773c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.586+0000 7f7740c00640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f773c15c560 0x7f773c153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f773be00640 1 -- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f773c15c560 msgr2=0x7f773c153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f773be00640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f773c15c560 0x7f773c153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f773be00640 1 -- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f773c155970 msgr2=0x7f773c14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f773be00640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f773c155970 0x7f773c14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f773be00640 1 -- 172.21.15.110:0/3665649207 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c1507d0 con 0x7f773c06f1e0 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7740c00640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f773c15c560 0x7f773c153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f773be00640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f773c06f1e0 0x7f773c152660 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f772c007f20 tx=0x7f772c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7733e00640 1 -- 172.21.15.110:0/3665649207 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c014620 con 0x7f773c06f1e0 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7733e00640 1 -- 172.21.15.110:0/3665649207 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772c019070 con 0x7f773c06f1e0 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7742c00640 1 -- 172.21.15.110:0/3665649207 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c150ab0 con 0x7f773c06f1e0 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7733e00640 1 -- 172.21.15.110:0/3665649207 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c014a00 con 0x7f773c06f1e0 2024-09-10T10:04:43.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7742c00640 1 -- 172.21.15.110:0/3665649207 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c151070 con 0x7f773c06f1e0 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7742c00640 1 -- 172.21.15.110:0/3665649207 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773c069bb0 con 0x7f773c06f1e0 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7733e00640 1 -- 172.21.15.110:0/3665649207 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f772c014ba0 con 0x7f773c06f1e0 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7733e00640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f771c067e60 0x7f771c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.590+0000 7f7733e00640 1 -- 172.21.15.110:0/3665649207 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f772c087000 con 0x7f773c06f1e0 2024-09-10T10:04:43.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.594+0000 7f773b400640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f771c067e60 0x7f771c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.594+0000 7f773b400640 1 --2- 172.21.15.110:0/3665649207 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f771c067e60 0x7f771c06a320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f773c152ba0 tx=0x7f7728009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.594+0000 7f7733e00640 1 -- 172.21.15.110:0/3665649207 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f772c057290 con 0x7f773c06f1e0 2024-09-10T10:04:43.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.778+0000 7f7742c00640 1 -- 172.21.15.110:0/3665649207 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f773c0cf680 con 0x7f771c067e60 2024-09-10T10:04:43.786 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:43.786+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.782+0000 7f7733e00640 1 -- 172.21.15.110:0/3665649207 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f773c0cf680 con 0x7f771c067e60 2024-09-10T10:04:43.787 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:43.795 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:43.796 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.930+0000 7ffb2a800640 1 Processor -- start 2024-09-10T10:04:43.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.930+0000 7ffb2a800640 1 -- start start 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb2a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffb240cd510 0x7ffb240cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb2a800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb24072150 con 0x7ffb240cfe30 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb2a800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb2406d2d0 con 0x7ffb2406e090 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb2a800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ffb2415d9a0 con 0x7ffb240cd510 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ffb240cfe30 0x7ffb24071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47666/0 (socket says 172.21.15.110:47666) 2024-09-10T10:04:43.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23400640 1 -- 172.21.15.110:0/1875214042 learned_addr learned my addr 172.21.15.110:0/1875214042 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22a00640 1 --2- 172.21.15.110:0/1875214042 >> v2:172.21.15.145:3300/0 conn(0x7ffb240cd510 0x7ffb240cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141263178 0 0) 0x7ffb24072150 con 0x7ffb240cfe30 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb08003530 con 0x7ffb240cfe30 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 535746529 0 0) 0x7ffb08003530 con 0x7ffb240cfe30 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 >> v2:172.21.15.145:3300/0 conn(0x7ffb240cd510 msgr2=0x7ffb240cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 --2- 172.21.15.110:0/1875214042 >> v2:172.21.15.145:3300/0 conn(0x7ffb240cd510 0x7ffb240cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 >> v1:172.21.15.145:6789/0 conn(0x7ffb2406e090 legacy=0x7ffb2406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2406d5b0 con 0x7ffb240cfe30 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7ffb14001680 con 0x7ffb240cfe30 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 >> v1:172.21.15.110:6789/0 conn(0x7ffb240cfe30 legacy=0x7ffb24071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb080039c0 0x7ffb08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb08004810 0x7ffb08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb080051d0 0x7ffb0800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2415d9a0 con 0x7ffb080051d0 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0800c170 con 0x7ffb08004810 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0800c370 con 0x7ffb080039c0 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22a00640 1 --2- 172.21.15.110:0/1875214042 >> v2:172.21.15.145:3300/0 conn(0x7ffb240cd510 0x7ffb240cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23e00640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb080051d0 0x7ffb0800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23400640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb080039c0 0x7ffb08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22a00640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb08004810 0x7ffb08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23e00640 1 -- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb080039c0 msgr2=0x7ffb08003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23e00640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb080039c0 0x7ffb08003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23e00640 1 -- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb08004810 msgr2=0x7ffb08004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23e00640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb08004810 0x7ffb08004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb23e00640 1 -- 172.21.15.110:0/1875214042 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb08003530 con 0x7ffb080051d0 2024-09-10T10:04:43.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.934+0000 7ffb22a00640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb08004810 0x7ffb08004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb23e00640 1 -- 172.21.15.110:0/1875214042 reap_dead start 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb23e00640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb080051d0 0x7ffb0800b9d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c001e70 tx=0x7ffb1c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7681ed6cbe4bf10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c001a90 con 0x7ffb080051d0 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1c003bc0 con 0x7ffb080051d0 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb22000640 1 -- 172.21.15.110:0/1875214042 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0050d0 con 0x7ffb080051d0 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb2a800640 1 -- 172.21.15.110:0/1875214042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb080051d0 msgr2=0x7ffb0800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb2a800640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb080051d0 0x7ffb0800b9d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c001e70 tx=0x7ffb1c01c780 comp rx=0 tx=0).stop 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb2a800640 1 -- 172.21.15.110:0/1875214042 shutdown_connections 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb2a800640 1 --2- 172.21.15.110:0/1875214042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb080051d0 0x7ffb0800b9d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb2a800640 1 -- 172.21.15.110:0/1875214042 >> 172.21.15.110:0/1875214042 conn(0x7ffb2406bc50 msgr2=0x7ffb2406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb2a800640 1 -- 172.21.15.110:0/1875214042 shutdown_connections 2024-09-10T10:04:43.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.938+0000 7ffb2a800640 1 -- 172.21.15.110:0/1875214042 wait complete. 2024-09-10T10:04:43.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 Processor -- start 2024-09-10T10:04:43.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 -- start start 2024-09-10T10:04:43.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb080051d0 0x7ffb241f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb240b2730 0x7ffb240b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb240ae840 0x7ffb240aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffb240cdb20 con 0x7ffb240ae840 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb240cd9a0 con 0x7ffb240b2730 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffb240cdca0 con 0x7ffb080051d0 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb240ae840 0x7ffb240aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb240ae840 0x7ffb240aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55222/0 (socket says 172.21.15.110:55222) 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 -- 172.21.15.110:0/216516434 learned_addr learned my addr 172.21.15.110:0/216516434 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb22a00640 1 --2- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb240b2730 0x7ffb240b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23400640 1 --2- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb080051d0 0x7ffb241f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 -- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb080051d0 msgr2=0x7ffb241f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 --2- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb080051d0 0x7ffb241f1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 -- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb240b2730 msgr2=0x7ffb240b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 --2- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb240b2730 0x7ffb240b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 -- 172.21.15.110:0/216516434 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb241f7670 con 0x7ffb240ae840 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb23e00640 1 --2- 172.21.15.110:0/216516434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb240ae840 0x7ffb240aecc0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c01c5b0 tx=0x7ffb08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb1b400640 1 -- 172.21.15.110:0/216516434 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c007dc0 con 0x7ffb240ae840 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb1b400640 1 -- 172.21.15.110:0/216516434 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1c0af070 con 0x7ffb240ae840 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb1b400640 1 -- 172.21.15.110:0/216516434 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c001a90 con 0x7ffb240ae840 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 -- 172.21.15.110:0/216516434 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb241f7950 con 0x7ffb240ae840 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 -- 172.21.15.110:0/216516434 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb241f7f10 con 0x7ffb240ae840 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.942+0000 7ffb2a800640 1 -- 172.21.15.110:0/216516434 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2406e4e0 con 0x7ffb240ae840 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.946+0000 7ffb1b400640 1 -- 172.21.15.110:0/216516434 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7ffb1c0b1030 con 0x7ffb240ae840 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.946+0000 7ffb1b400640 1 --2- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb04067e60 0x7ffb0406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.946+0000 7ffb1b400640 1 -- 172.21.15.110:0/216516434 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7ffb1c11f500 con 0x7ffb240ae840 2024-09-10T10:04:43.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.946+0000 7ffb23400640 1 --2- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb04067e60 0x7ffb0406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.946+0000 7ffb23400640 1 --2- 172.21.15.110:0/216516434 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb04067e60 0x7ffb0406a320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb14000c10 tx=0x7ffb14001160 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:43.950+0000 7ffb1b400640 1 -- 172.21.15.110:0/216516434 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ffb1c016490 con 0x7ffb240ae840 2024-09-10T10:04:44.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.138+0000 7ffb2a800640 1 -- 172.21.15.110:0/216516434 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7ffb240ce710 con 0x7ffb04067e60 2024-09-10T10:04:44.150 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:44.150+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:44.150 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:44.150+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:44.150 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:44.150+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:44.150 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:44.150+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:44.150 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:44.150+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:44.157 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:44.154+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.154+0000 7ffb1b400640 1 -- 172.21.15.110:0/216516434 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ffb240ce710 con 0x7ffb04067e60 2024-09-10T10:04:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:44.167 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:44.167 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-10T10:04:44.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.302+0000 7fc465800640 1 Processor -- start 2024-09-10T10:04:44.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.302+0000 7fc465800640 1 -- start start 2024-09-10T10:04:44.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc465800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc4600cf0a0 0x7fc4600cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc465800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc46013f9c0 0x7fc46013fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc465800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4600cfa00 con 0x7fc4600b2df0 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc465800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc46013f740 con 0x7fc46013f9c0 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc465800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc4600c6010 con 0x7fc4600cf0a0 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc464e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc46013f9c0 0x7fc46013fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc464e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc46013f9c0 0x7fc46013fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55238/0 (socket says 172.21.15.110:55238) 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc464e00640 1 -- 172.21.15.110:0/1095928333 learned_addr learned my addr 172.21.15.110:0/1095928333 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc45f400640 1 --2- 172.21.15.110:0/1095928333 >> v2:172.21.15.145:3300/0 conn(0x7fc4600cf0a0 0x7fc4600cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc464e00640 1 -- 172.21.15.110:0/1095928333 >> v2:172.21.15.145:3300/0 conn(0x7fc4600cf0a0 msgr2=0x7fc4600cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc464e00640 1 --2- 172.21.15.110:0/1095928333 >> v2:172.21.15.145:3300/0 conn(0x7fc4600cf0a0 0x7fc4600cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc464e00640 1 -- 172.21.15.110:0/1095928333 >> v1:172.21.15.110:6789/0 conn(0x7fc4600b2df0 legacy=0x7fc4600c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc464e00640 1 -- 172.21.15.110:0/1095928333 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46015c9a0 con 0x7fc46013f9c0 2024-09-10T10:04:44.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc45ea00640 1 -- 172.21.15.110:0/1095928333 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3385431776 0 0) 0x7fc4600cfa00 con 0x7fc4600b2df0 2024-09-10T10:04:44.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc464e00640 1 --2- 172.21.15.110:0/1095928333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc46013f9c0 0x7fc46013fe50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fc4600b5dc0 tx=0x7fc45801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8279f0969d5ee062 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.306+0000 7fc45ea00640 1 -- 172.21.15.110:0/1095928333 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580ab610 con 0x7fc46013f9c0 2024-09-10T10:04:44.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45ea00640 1 -- 172.21.15.110:0/1095928333 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4580b0070 con 0x7fc46013f9c0 2024-09-10T10:04:44.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45ea00640 1 -- 172.21.15.110:0/1095928333 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580ab9d0 con 0x7fc46013f9c0 2024-09-10T10:04:44.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- 172.21.15.110:0/1095928333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc46013f9c0 msgr2=0x7fc46013fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 --2- 172.21.15.110:0/1095928333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc46013f9c0 0x7fc46013fe50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fc4600b5dc0 tx=0x7fc45801c520 comp rx=0 tx=0).stop 2024-09-10T10:04:44.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- 172.21.15.110:0/1095928333 shutdown_connections 2024-09-10T10:04:44.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 --2- 172.21.15.110:0/1095928333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc46013f9c0 0x7fc46013fe50 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 --2- 172.21.15.110:0/1095928333 >> v2:172.21.15.145:3300/0 conn(0x7fc4600cf0a0 0x7fc4600cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- 172.21.15.110:0/1095928333 >> 172.21.15.110:0/1095928333 conn(0x7fc46006d710 msgr2=0x7fc4600b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:44.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- 172.21.15.110:0/1095928333 shutdown_connections 2024-09-10T10:04:44.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- 172.21.15.110:0/1095928333 wait complete. 2024-09-10T10:04:44.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 Processor -- start 2024-09-10T10:04:44.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- start start 2024-09-10T10:04:44.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4600b2df0 0x7fc46013c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4600cf0a0 0x7fc46015da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc46013f9c0 0x7fc46013d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4600b60d0 con 0x7fc4600cf0a0 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4600b5f50 con 0x7fc4600b2df0 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc465800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc460169010 con 0x7fc46013f9c0 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4600cf0a0 0x7fc46015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4600cf0a0 0x7fc46015da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55248/0 (socket says 172.21.15.110:55248) 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45f400640 1 -- 172.21.15.110:0/3250633622 learned_addr learned my addr 172.21.15.110:0/3250633622 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45fe00640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4600b2df0 0x7fc46013c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc464e00640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc46013f9c0 0x7fc46013d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45f400640 1 -- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc46013f9c0 msgr2=0x7fc46013d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45f400640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc46013f9c0 0x7fc46013d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45f400640 1 -- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4600b2df0 msgr2=0x7fc46013c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45f400640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4600b2df0 0x7fc46013c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45f400640 1 -- 172.21.15.110:0/3250633622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46015f3b0 con 0x7fc4600cf0a0 2024-09-10T10:04:44.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.310+0000 7fc45fe00640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4600b2df0 0x7fc46013c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:44.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc45f400640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4600cf0a0 0x7fc46015da30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00d5f0 tx=0x7fc44c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc457e00640 1 -- 172.21.15.110:0/3250633622 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0125f0 con 0x7fc4600cf0a0 2024-09-10T10:04:44.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc457e00640 1 -- 172.21.15.110:0/3250633622 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c019070 con 0x7fc4600cf0a0 2024-09-10T10:04:44.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc465800640 1 -- 172.21.15.110:0/3250633622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46015f690 con 0x7fc4600cf0a0 2024-09-10T10:04:44.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc457e00640 1 -- 172.21.15.110:0/3250633622 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0129b0 con 0x7fc4600cf0a0 2024-09-10T10:04:44.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc465800640 1 -- 172.21.15.110:0/3250633622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46015fc50 con 0x7fc4600cf0a0 2024-09-10T10:04:44.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc465800640 1 -- 172.21.15.110:0/3250633622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46013fe50 con 0x7fc4600cf0a0 2024-09-10T10:04:44.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc457e00640 1 -- 172.21.15.110:0/3250633622 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fc44c012b50 con 0x7fc4600cf0a0 2024-09-10T10:04:44.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc457e00640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc440067e60 0x7fc44006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc457e00640 1 -- 172.21.15.110:0/3250633622 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fc44c086b80 con 0x7fc4600cf0a0 2024-09-10T10:04:44.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.314+0000 7fc45fe00640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc440067e60 0x7fc44006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.318+0000 7fc45fe00640 1 --2- 172.21.15.110:0/3250633622 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc440067e60 0x7fc44006a320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc4500012f0 tx=0x7fc450009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.318+0000 7fc457e00640 1 -- 172.21.15.110:0/3250633622 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc44c056dd0 con 0x7fc4600cf0a0 2024-09-10T10:04:44.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.518+0000 7fc465800640 1 -- 172.21.15.110:0/3250633622 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fc46015f160 con 0x7fc440067e60 2024-09-10T10:04:44.523 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:44.522+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:44.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.518+0000 7fc457e00640 1 -- 172.21.15.110:0/3250633622 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc46015f160 con 0x7fc440067e60 2024-09-10T10:04:44.523 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:44.533 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:44.534 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-10T10:04:44.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.666+0000 7ff5ddc00640 1 Processor -- start 2024-09-10T10:04:44.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.666+0000 7ff5ddc00640 1 -- start start 2024-09-10T10:04:44.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5ddc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff5d8073780 0x7ff5d815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5ddc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d80d2c40 0x7ff5d80d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5ddc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5d8070b60 con 0x7ff5d815c580 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5ddc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff5d815dd30 con 0x7ff5d8073780 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5ddc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d8070e40 con 0x7ff5d80d2c40 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d6000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff5d8073780 0x7ff5d815d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d6a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff5d815c580 0x7ff5d80d2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47682/0 (socket says 172.21.15.110:47682) 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d6000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff5d8073780 0x7ff5d815d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55254/0 (socket says 172.21.15.110:55254) 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d6a00640 1 -- 172.21.15.110:0/2180985796 learned_addr learned my addr 172.21.15.110:0/2180985796 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d7400640 1 --2- 172.21.15.110:0/2180985796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d80d2c40 0x7ff5d80d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d6000640 1 -- 172.21.15.110:0/2180985796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d80d2c40 msgr2=0x7ff5d80d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d6000640 1 --2- 172.21.15.110:0/2180985796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d80d2c40 0x7ff5d80d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d6000640 1 -- 172.21.15.110:0/2180985796 >> v1:172.21.15.110:6789/0 conn(0x7ff5d815c580 legacy=0x7ff5d80d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.670+0000 7ff5d6000640 1 -- 172.21.15.110:0/2180985796 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d8071120 con 0x7ff5d8073780 2024-09-10T10:04:44.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5d6000640 1 --2- 172.21.15.110:0/2180985796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5d8073780 0x7ff5d815d7f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097c0 tx=0x7ff5c401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1ccbff50b80d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5d5600640 1 -- 172.21.15.110:0/2180985796 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40b1070 con 0x7ff5d8073780 2024-09-10T10:04:44.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5d5600640 1 -- 172.21.15.110:0/2180985796 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c40a8050 con 0x7ff5d8073780 2024-09-10T10:04:44.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5d5600640 1 -- 172.21.15.110:0/2180985796 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40ac860 con 0x7ff5d8073780 2024-09-10T10:04:44.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 -- 172.21.15.110:0/2180985796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5d8073780 msgr2=0x7ff5d815d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 --2- 172.21.15.110:0/2180985796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5d8073780 0x7ff5d815d7f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097c0 tx=0x7ff5c401c3e0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 -- 172.21.15.110:0/2180985796 shutdown_connections 2024-09-10T10:04:44.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 --2- 172.21.15.110:0/2180985796 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d80d2c40 0x7ff5d80d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 --2- 172.21.15.110:0/2180985796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5d8073780 0x7ff5d815d7f0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 -- 172.21.15.110:0/2180985796 >> 172.21.15.110:0/2180985796 conn(0x7ff5d806f420 msgr2=0x7ff5d806f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:44.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 -- 172.21.15.110:0/2180985796 shutdown_connections 2024-09-10T10:04:44.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 -- 172.21.15.110:0/2180985796 wait complete. 2024-09-10T10:04:44.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 Processor -- start 2024-09-10T10:04:44.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 -- start start 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5d8073780 0x7ff5d81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5d80d2c40 0x7ff5d806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.674+0000 7ff5ddc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d815c580 0x7ff5d80cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5ddc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d80d1770 con 0x7ff5d80d2c40 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5ddc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d80d15f0 con 0x7ff5d8073780 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5ddc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d80d18f0 con 0x7ff5d815c580 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5d80d2c40 0x7ff5d806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5d80d2c40 0x7ff5d806d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55268/0 (socket says 172.21.15.110:55268) 2024-09-10T10:04:44.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 -- 172.21.15.110:0/1241589945 learned_addr learned my addr 172.21.15.110:0/1241589945 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6a00640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5d8073780 0x7ff5d81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d7400640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d815c580 0x7ff5d80cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 -- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d815c580 msgr2=0x7ff5d80cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d815c580 0x7ff5d80cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 -- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5d8073780 msgr2=0x7ff5d81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5d8073780 0x7ff5d81531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 -- 172.21.15.110:0/1241589945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d806dea0 con 0x7ff5d80d2c40 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d7400640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5d815c580 0x7ff5d80cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6a00640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5d8073780 0x7ff5d81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5d6000640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5d80d2c40 0x7ff5d806d470 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff5c401a010 tx=0x7ff5c4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5cea00640 1 -- 172.21.15.110:0/1241589945 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40b9070 con 0x7ff5d80d2c40 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5cea00640 1 -- 172.21.15.110:0/1241589945 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c40b5410 con 0x7ff5d80d2c40 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5ddc00640 1 -- 172.21.15.110:0/1241589945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d80b6670 con 0x7ff5d80d2c40 2024-09-10T10:04:44.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5cea00640 1 -- 172.21.15.110:0/1241589945 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40b1040 con 0x7ff5d80d2c40 2024-09-10T10:04:44.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5ddc00640 1 -- 172.21.15.110:0/1241589945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d80b6c30 con 0x7ff5d80d2c40 2024-09-10T10:04:44.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5ddc00640 1 -- 172.21.15.110:0/1241589945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d815d7f0 con 0x7ff5d80d2c40 2024-09-10T10:04:44.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.678+0000 7ff5cea00640 1 -- 172.21.15.110:0/1241589945 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7ff5c40bc070 con 0x7ff5d80d2c40 2024-09-10T10:04:44.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.682+0000 7ff5cea00640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5b8067e60 0x7ff5b806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.682+0000 7ff5cea00640 1 -- 172.21.15.110:0/1241589945 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7ff5c4120350 con 0x7ff5d80d2c40 2024-09-10T10:04:44.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.682+0000 7ff5d6a00640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5b8067e60 0x7ff5b806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.682+0000 7ff5d6a00640 1 --2- 172.21.15.110:0/1241589945 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5b8067e60 0x7ff5b806a320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80012f0 tx=0x7ff5c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.686+0000 7ff5cea00640 1 -- 172.21.15.110:0/1241589945 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff5c4016090 con 0x7ff5d80d2c40 2024-09-10T10:04:44.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.862+0000 7ff5ddc00640 1 -- 172.21.15.110:0/1241589945 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7ff5d806dcf0 con 0x7ff5b8067e60 2024-09-10T10:04:44.869 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:44.870+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:44.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:44.866+0000 7ff5cea00640 1 -- 172.21.15.110:0/1241589945 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ff5d806dcf0 con 0x7ff5b8067e60 2024-09-10T10:04:44.870 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:44.878 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:44.878 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-10T10:04:45.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.010+0000 7f98ac200640 1 Processor -- start 2024-09-10T10:04:45.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98ac200640 1 -- start start 2024-09-10T10:04:45.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98ac200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f98a415b330 0x7f98a406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98ac200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f98a40ce4d0 0x7f98a406e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98ac200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 0x7f98a406e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98ac200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40b5570 con 0x7f98a415b330 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98ac200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40b4960 con 0x7f98a40ce4d0 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98ac200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f98a40b4c40 con 0x7f98a40b1c60 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a9800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f98a415b330 0x7f98a406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a9800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f98a415b330 0x7f98a406c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55280/0 (socket says 172.21.15.110:55280) 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a9800640 1 -- 172.21.15.110:0/4233628294 learned_addr learned my addr 172.21.15.110:0/4233628294 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98aa200640 1 --2- 172.21.15.110:0/4233628294 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 0x7f98a406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a8e00640 1 --2- 172.21.15.110:0/4233628294 >> v2:172.21.15.145:3300/0 conn(0x7f98a40ce4d0 0x7f98a406e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a9800640 1 -- 172.21.15.110:0/4233628294 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 msgr2=0x7f98a406e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a9800640 1 --2- 172.21.15.110:0/4233628294 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 0x7f98a406e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a9800640 1 -- 172.21.15.110:0/4233628294 >> v2:172.21.15.145:3300/0 conn(0x7f98a40ce4d0 msgr2=0x7f98a406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:45.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a9800640 1 --2- 172.21.15.110:0/4233628294 >> v2:172.21.15.145:3300/0 conn(0x7f98a40ce4d0 0x7f98a406e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.014+0000 7f98a9800640 1 -- 172.21.15.110:0/4233628294 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40b4f20 con 0x7f98a415b330 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98a9800640 1 --2- 172.21.15.110:0/4233628294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a415b330 0x7f98a406c6a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f98a415eea0 tx=0x7f989801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8103a008262d63b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98a3e00640 1 -- 172.21.15.110:0/4233628294 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98980a5780 con 0x7f98a415b330 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98a3e00640 1 -- 172.21.15.110:0/4233628294 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98980a5920 con 0x7f98a415b330 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98a3e00640 1 -- 172.21.15.110:0/4233628294 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98980a5cf0 con 0x7f98a415b330 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 -- 172.21.15.110:0/4233628294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a415b330 msgr2=0x7f98a406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 --2- 172.21.15.110:0/4233628294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a415b330 0x7f98a406c6a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f98a415eea0 tx=0x7f989801c9f0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 -- 172.21.15.110:0/4233628294 shutdown_connections 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 --2- 172.21.15.110:0/4233628294 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 0x7f98a406e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 --2- 172.21.15.110:0/4233628294 >> v2:172.21.15.145:3300/0 conn(0x7f98a40ce4d0 0x7f98a406e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 --2- 172.21.15.110:0/4233628294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a415b330 0x7f98a406c6a0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 -- 172.21.15.110:0/4233628294 >> 172.21.15.110:0/4233628294 conn(0x7f98a4062e40 msgr2=0x7f98a406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 -- 172.21.15.110:0/4233628294 shutdown_connections 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 -- 172.21.15.110:0/4233628294 wait complete. 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.018+0000 7f98ac200640 1 Processor -- start 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 -- start start 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 0x7f98a4157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a415b330 0x7f98a4161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a4161db0 0x7f98a41639f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98a415fb20 con 0x7f98a415b330 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98a415f9a0 con 0x7f98a4161db0 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f98a415fca0 con 0x7f98a40b1c60 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a415b330 0x7f98a4161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a415b330 0x7f98a4161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55294/0 (socket says 172.21.15.110:55294) 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 -- 172.21.15.110:0/1831033902 learned_addr learned my addr 172.21.15.110:0/1831033902 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98aa200640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a4161db0 0x7f98a41639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a9800640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 0x7f98a4157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 -- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 msgr2=0x7f98a4157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98a40b1c60 0x7f98a4157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 -- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a4161db0 msgr2=0x7f98a41639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a4161db0 0x7f98a41639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 -- 172.21.15.110:0/1831033902 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a415a040 con 0x7f98a415b330 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98aa200640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a4161db0 0x7f98a41639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a8e00640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a415b330 0x7f98a4161870 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f989000d5f0 tx=0x7f989000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a1600640 1 -- 172.21.15.110:0/1831033902 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900125f0 con 0x7f98a415b330 2024-09-10T10:04:45.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a1600640 1 -- 172.21.15.110:0/1831033902 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890019070 con 0x7f98a415b330 2024-09-10T10:04:45.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 -- 172.21.15.110:0/1831033902 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a41646d0 con 0x7f98a415b330 2024-09-10T10:04:45.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98a1600640 1 -- 172.21.15.110:0/1831033902 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900129b0 con 0x7f98a415b330 2024-09-10T10:04:45.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 -- 172.21.15.110:0/1831033902 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a4164c90 con 0x7f98a415b330 2024-09-10T10:04:45.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.022+0000 7f98ac200640 1 -- 172.21.15.110:0/1831033902 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a41621c0 con 0x7f98a415b330 2024-09-10T10:04:45.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.026+0000 7f98a1600640 1 -- 172.21.15.110:0/1831033902 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7f9890012b50 con 0x7f98a415b330 2024-09-10T10:04:45.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.026+0000 7f98a1600640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9884067e60 0x7f988406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.026+0000 7f98a1600640 1 -- 172.21.15.110:0/1831033902 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f9890086960 con 0x7f98a415b330 2024-09-10T10:04:45.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.026+0000 7f98a9800640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9884067e60 0x7f988406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.026+0000 7f98a9800640 1 --2- 172.21.15.110:0/1831033902 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9884067e60 0x7f988406a320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f98980ae000 tx=0x7f98980a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.030+0000 7f98a1600640 1 -- 172.21.15.110:0/1831033902 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f98a41621c0 con 0x7f98a415b330 2024-09-10T10:04:45.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.210+0000 7f98ac200640 1 -- 172.21.15.110:0/1831033902 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f98a406c6a0 con 0x7f9884067e60 2024-09-10T10:04:45.214 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:45.214+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:45.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.210+0000 7f98a1600640 1 -- 172.21.15.110:0/1831033902 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f98a406c6a0 con 0x7f9884067e60 2024-09-10T10:04:45.215 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:45.223 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:45.224 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-10T10:04:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1e6800640 1 Processor -- start 2024-09-10T10:04:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1e6800640 1 -- start start 2024-09-10T10:04:45.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1e6800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe1e015c580 0x7fe1e015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1e6800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe1e00d2c40 0x7fe1e00d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1e6800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e0070b60 con 0x7fe1e0073780 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1e6800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e015dd30 con 0x7fe1e015c580 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1e6800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e0070e40 con 0x7fe1e00d2c40 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1df400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe1e015c580 0x7fe1e015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1df400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe1e015c580 0x7fe1e015d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55302/0 (socket says 172.21.15.110:55302) 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1df400640 1 -- 172.21.15.110:0/613459842 learned_addr learned my addr 172.21.15.110:0/613459842 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1dfe00640 1 --2- 172.21.15.110:0/613459842 >> v2:172.21.15.145:3300/0 conn(0x7fe1e00d2c40 0x7fe1e00d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1df400640 1 -- 172.21.15.110:0/613459842 >> v2:172.21.15.145:3300/0 conn(0x7fe1e00d2c40 msgr2=0x7fe1e00d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1df400640 1 --2- 172.21.15.110:0/613459842 >> v2:172.21.15.145:3300/0 conn(0x7fe1e00d2c40 0x7fe1e00d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1df400640 1 -- 172.21.15.110:0/613459842 >> v1:172.21.15.145:6789/0 conn(0x7fe1e0073780 legacy=0x7fe1e00d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.358+0000 7fe1df400640 1 -- 172.21.15.110:0/613459842 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0071120 con 0x7fe1e015c580 2024-09-10T10:04:45.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1df400640 1 --2- 172.21.15.110:0/613459842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1e015c580 0x7fe1e015d7f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00cf290 tx=0x7fe1d001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a99d279672b608f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1de000640 1 -- 172.21.15.110:0/613459842 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a57b0 con 0x7fe1e015c580 2024-09-10T10:04:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1de000640 1 -- 172.21.15.110:0/613459842 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d00a5950 con 0x7fe1e015c580 2024-09-10T10:04:45.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1de000640 1 -- 172.21.15.110:0/613459842 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a5d20 con 0x7fe1e015c580 2024-09-10T10:04:45.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1e6800640 1 -- 172.21.15.110:0/613459842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1e015c580 msgr2=0x7fe1e015d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1e6800640 1 --2- 172.21.15.110:0/613459842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1e015c580 0x7fe1e015d7f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00cf290 tx=0x7fe1d001c8f0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1e6800640 1 -- 172.21.15.110:0/613459842 shutdown_connections 2024-09-10T10:04:45.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1e6800640 1 --2- 172.21.15.110:0/613459842 >> v2:172.21.15.145:3300/0 conn(0x7fe1e00d2c40 0x7fe1e00d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1e6800640 1 --2- 172.21.15.110:0/613459842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1e015c580 0x7fe1e015d7f0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1e6800640 1 -- 172.21.15.110:0/613459842 >> 172.21.15.110:0/613459842 conn(0x7fe1e006f420 msgr2=0x7fe1e006f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:45.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1e6800640 1 -- 172.21.15.110:0/613459842 shutdown_connections 2024-09-10T10:04:45.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.362+0000 7fe1e6800640 1 -- 172.21.15.110:0/613459842 wait complete. 2024-09-10T10:04:45.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 Processor -- start 2024-09-10T10:04:45.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 -- start start 2024-09-10T10:04:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1e0073780 0x7fe1e00b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1e00d2c40 0x7fe1e0154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1e015c580 0x7fe1e00b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e00d1770 con 0x7fe1e015c580 2024-09-10T10:04:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e00d15f0 con 0x7fe1e0073780 2024-09-10T10:04:45.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e00d18f0 con 0x7fe1e00d2c40 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1e015c580 0x7fe1e00b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1e015c580 0x7fe1e00b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55312/0 (socket says 172.21.15.110:55312) 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 -- 172.21.15.110:0/550755757 learned_addr learned my addr 172.21.15.110:0/550755757 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1df400640 1 --2- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1e0073780 0x7fe1e00b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dea00640 1 --2- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1e00d2c40 0x7fe1e0154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 -- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1e00d2c40 msgr2=0x7fe1e0154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 --2- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1e00d2c40 0x7fe1e0154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 -- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1e0073780 msgr2=0x7fe1e00b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 --2- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1e0073780 0x7fe1e00b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 -- 172.21.15.110:0/550755757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0155100 con 0x7fe1e015c580 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1dfe00640 1 --2- 172.21.15.110:0/550755757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1e015c580 0x7fe1e00b2970 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800d5d0 tx=0x7fe1d800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1d7400640 1 -- 172.21.15.110:0/550755757 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80125f0 con 0x7fe1e015c580 2024-09-10T10:04:45.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1d7400640 1 -- 172.21.15.110:0/550755757 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d8019070 con 0x7fe1e015c580 2024-09-10T10:04:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1d7400640 1 -- 172.21.15.110:0/550755757 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80129f0 con 0x7fe1e015c580 2024-09-10T10:04:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 -- 172.21.15.110:0/550755757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e006e9f0 con 0x7fe1e015c580 2024-09-10T10:04:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.366+0000 7fe1e6800640 1 -- 172.21.15.110:0/550755757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e006ce00 con 0x7fe1e015c580 2024-09-10T10:04:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.370+0000 7fe1e6800640 1 -- 172.21.15.110:0/550755757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e00d08a0 con 0x7fe1e015c580 2024-09-10T10:04:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.370+0000 7fe1d7400640 1 -- 172.21.15.110:0/550755757 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 87955+0+0 (secure 0 0 0) 0x7fe1d8012b90 con 0x7fe1e015c580 2024-09-10T10:04:45.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.370+0000 7fe1d7400640 1 --2- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe1c0067e60 0x7fe1c006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.370+0000 7fe1d7400640 1 -- 172.21.15.110:0/550755757 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fe1d8086af0 con 0x7fe1e015c580 2024-09-10T10:04:45.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.370+0000 7fe1df400640 1 --2- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe1c0067e60 0x7fe1c006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.370+0000 7fe1df400640 1 --2- 172.21.15.110:0/550755757 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe1c0067e60 0x7fe1c006a320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00ae000 tx=0x7fe1d00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.374+0000 7fe1d7400640 1 -- 172.21.15.110:0/550755757 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe1e0155100 con 0x7fe1e015c580 2024-09-10T10:04:45.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.514+0000 7fe1d7400640 1 -- 172.21.15.110:0/550755757 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fe1d8052cf0 con 0x7fe1e015c580 2024-09-10T10:04:45.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.562+0000 7fe1e6800640 1 -- 172.21.15.110:0/550755757 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fe1e00b2450 con 0x7fe1c0067e60 2024-09-10T10:04:45.569 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:45.566+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.566+0000 7fe1d7400640 1 -- 172.21.15.110:0/550755757 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe1e00b2450 con 0x7fe1c0067e60 2024-09-10T10:04:45.570 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:45.579 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:45.580 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-10T10:04:45.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.714+0000 7f6df8400640 1 Processor -- start 2024-09-10T10:04:45.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.714+0000 7f6df8400640 1 -- start start 2024-09-10T10:04:45.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df8400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 0x7f6df00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df8400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6df015c560 0x7f6df006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df8400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df0155d50 con 0x7f6df006f010 2024-09-10T10:04:45.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df8400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6df00b4570 con 0x7f6df015c560 2024-09-10T10:04:45.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df8400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6df006f450 con 0x7f6df0155970 2024-09-10T10:04:45.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df5000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6df015c560 0x7f6df006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df6400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6df006f010 0x7f6df015eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47686/0 (socket says 172.21.15.110:47686) 2024-09-10T10:04:45.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df5000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6df015c560 0x7f6df006ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55326/0 (socket says 172.21.15.110:55326) 2024-09-10T10:04:45.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df6400640 1 -- 172.21.15.110:0/856681461 learned_addr learned my addr 172.21.15.110:0/856681461 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:45.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df5a00640 1 --2- 172.21.15.110:0/856681461 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 0x7f6df00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df5000640 1 -- 172.21.15.110:0/856681461 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 msgr2=0x7f6df00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:45.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df5000640 1 --2- 172.21.15.110:0/856681461 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 0x7f6df00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df5000640 1 -- 172.21.15.110:0/856681461 >> v1:172.21.15.110:6789/0 conn(0x7f6df006f010 legacy=0x7f6df015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6df5000640 1 -- 172.21.15.110:0/856681461 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df00b34d0 con 0x7f6df015c560 2024-09-10T10:04:45.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.718+0000 7f6defe00640 1 -- 172.21.15.110:0/856681461 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 458227628 0 0) 0x7f6df0155d50 con 0x7f6df006f010 2024-09-10T10:04:45.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df5000640 1 --2- 172.21.15.110:0/856681461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6df015c560 0x7f6df006ead0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f6df006f5d0 tx=0x7f6ddc01eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6c09dcec495ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6defe00640 1 -- 172.21.15.110:0/856681461 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0b4070 con 0x7f6df015c560 2024-09-10T10:04:45.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6defe00640 1 -- 172.21.15.110:0/856681461 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ddc0a7b80 con 0x7f6df015c560 2024-09-10T10:04:45.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6defe00640 1 -- 172.21.15.110:0/856681461 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0af440 con 0x7f6df015c560 2024-09-10T10:04:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- 172.21.15.110:0/856681461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6df015c560 msgr2=0x7f6df006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 --2- 172.21.15.110:0/856681461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6df015c560 0x7f6df006ead0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f6df006f5d0 tx=0x7f6ddc01eab0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- 172.21.15.110:0/856681461 shutdown_connections 2024-09-10T10:04:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 --2- 172.21.15.110:0/856681461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6df015c560 0x7f6df006ead0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 --2- 172.21.15.110:0/856681461 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 0x7f6df00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- 172.21.15.110:0/856681461 >> 172.21.15.110:0/856681461 conn(0x7f6df006bc50 msgr2=0x7f6df006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- 172.21.15.110:0/856681461 shutdown_connections 2024-09-10T10:04:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- 172.21.15.110:0/856681461 wait complete. 2024-09-10T10:04:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 Processor -- start 2024-09-10T10:04:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- start start 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6df006f010 0x7f6df014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 0x7f6df0150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6df015c560 0x7f6df0150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6df00cce00 con 0x7f6df015c560 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6df00b4050 con 0x7f6df006f010 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.722+0000 7f6df8400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6df006d9c0 con 0x7f6df0155970 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6df015c560 0x7f6df0150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6df015c560 0x7f6df0150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55328/0 (socket says 172.21.15.110:55328) 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 -- 172.21.15.110:0/1196081598 learned_addr learned my addr 172.21.15.110:0/1196081598 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:45.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df5a00640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6df006f010 0x7f6df014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 -- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 msgr2=0x7f6df0150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df5000640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 0x7f6df0150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 0x7f6df0150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 -- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6df006f010 msgr2=0x7f6df014c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6df006f010 0x7f6df014c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 -- 172.21.15.110:0/1196081598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df0151720 con 0x7f6df015c560 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df5a00640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6df006f010 0x7f6df014c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df5000640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6df0155970 0x7f6df0150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df6400640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6df015c560 0x7f6df0150dc0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f6de800c4b0 tx=0x7f6de800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6ded600640 1 -- 172.21.15.110:0/1196081598 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8018070 con 0x7f6df015c560 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6ded600640 1 -- 172.21.15.110:0/1196081598 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de80074e0 con 0x7f6df015c560 2024-09-10T10:04:45.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6ded600640 1 -- 172.21.15.110:0/1196081598 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8004a30 con 0x7f6df015c560 2024-09-10T10:04:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df8400640 1 -- 172.21.15.110:0/1196081598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df0151440 con 0x7f6df015c560 2024-09-10T10:04:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df8400640 1 -- 172.21.15.110:0/1196081598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df0149f40 con 0x7f6df015c560 2024-09-10T10:04:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6df8400640 1 -- 172.21.15.110:0/1196081598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df00cc8c0 con 0x7f6df015c560 2024-09-10T10:04:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.726+0000 7f6ded600640 1 -- 172.21.15.110:0/1196081598 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f6de8004bd0 con 0x7f6df015c560 2024-09-10T10:04:45.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.730+0000 7f6ded600640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dd0067ef0 0x7f6dd006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:45.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.730+0000 7f6ded600640 1 -- 172.21.15.110:0/1196081598 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f6de8085ae0 con 0x7f6df015c560 2024-09-10T10:04:45.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.730+0000 7f6df5a00640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dd0067ef0 0x7f6dd006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:45.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.730+0000 7f6df5a00640 1 --2- 172.21.15.110:0/1196081598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6dd0067ef0 0x7f6dd006a3b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6de4007330 tx=0x7f6de400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:45.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.734+0000 7f6ded600640 1 -- 172.21.15.110:0/1196081598 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6de8055cf0 con 0x7f6df015c560 2024-09-10T10:04:45.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.914+0000 7f6df8400640 1 -- 172.21.15.110:0/1196081598 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7f6df0142030 con 0x7f6dd0067ef0 2024-09-10T10:04:45.918 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:45.918+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:45.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:45.914+0000 7f6ded600640 1 -- 172.21.15.110:0/1196081598 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f6de40039a0 con 0x7f6dd0067ef0 2024-09-10T10:04:45.919 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:45.927 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:45.928 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-10T10:04:46.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.058+0000 7f87ede00640 1 Processor -- start 2024-09-10T10:04:46.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.058+0000 7f87ede00640 1 -- start start 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87ede00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f87e8145f30 0x7f87e80b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87ede00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f87e80b29e0 0x7f87e80cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87ede00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80cfbc0 0x7f87e8160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87ede00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f87e80b2e50 con 0x7f87e8145f30 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87ede00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f87e80b37c0 con 0x7f87e80b29e0 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87ede00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f87e80b3aa0 con 0x7f87e80cfbc0 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f87e8145f30 0x7f87e80b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f87e8145f30 0x7f87e80b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55340/0 (socket says 172.21.15.110:55340) 2024-09-10T10:04:46.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6a00640 1 -- 172.21.15.110:0/4170332528 learned_addr learned my addr 172.21.15.110:0/4170332528 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:46.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e7400640 1 --2- 172.21.15.110:0/4170332528 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80cfbc0 0x7f87e8160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6000640 1 --2- 172.21.15.110:0/4170332528 >> v2:172.21.15.145:3300/0 conn(0x7f87e80b29e0 0x7f87e80cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6a00640 1 -- 172.21.15.110:0/4170332528 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80cfbc0 msgr2=0x7f87e8160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6a00640 1 --2- 172.21.15.110:0/4170332528 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80cfbc0 0x7f87e8160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6a00640 1 -- 172.21.15.110:0/4170332528 >> v2:172.21.15.145:3300/0 conn(0x7f87e80b29e0 msgr2=0x7f87e80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6a00640 1 --2- 172.21.15.110:0/4170332528 >> v2:172.21.15.145:3300/0 conn(0x7f87e80b29e0 0x7f87e80cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.062+0000 7f87e6a00640 1 -- 172.21.15.110:0/4170332528 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e80b3d80 con 0x7f87e8145f30 2024-09-10T10:04:46.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87e6a00640 1 --2- 172.21.15.110:0/4170332528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8145f30 0x7f87e80b24a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f87e806cb60 tx=0x7f87d801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46f975242b4c237 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87e5600640 1 -- 172.21.15.110:0/4170332528 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d80b0070 con 0x7f87e8145f30 2024-09-10T10:04:46.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87e5600640 1 -- 172.21.15.110:0/4170332528 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d80ab600 con 0x7f87e8145f30 2024-09-10T10:04:46.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87e5600640 1 -- 172.21.15.110:0/4170332528 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d80ab9d0 con 0x7f87e8145f30 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- 172.21.15.110:0/4170332528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8145f30 msgr2=0x7f87e80b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 --2- 172.21.15.110:0/4170332528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8145f30 0x7f87e80b24a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f87e806cb60 tx=0x7f87d801c260 comp rx=0 tx=0).stop 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- 172.21.15.110:0/4170332528 shutdown_connections 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 --2- 172.21.15.110:0/4170332528 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80cfbc0 0x7f87e8160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 --2- 172.21.15.110:0/4170332528 >> v2:172.21.15.145:3300/0 conn(0x7f87e80b29e0 0x7f87e80cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 --2- 172.21.15.110:0/4170332528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8145f30 0x7f87e80b24a0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- 172.21.15.110:0/4170332528 >> 172.21.15.110:0/4170332528 conn(0x7f87e8062e40 msgr2=0x7f87e806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- 172.21.15.110:0/4170332528 shutdown_connections 2024-09-10T10:04:46.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- 172.21.15.110:0/4170332528 wait complete. 2024-09-10T10:04:46.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 Processor -- start 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- start start 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80b29e0 0x7f87e815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e80cfbc0 0x7f87e8161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87e8145f30 0x7f87e815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80b4910 con 0x7f87e80cfbc0 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80b4790 con 0x7f87e8145f30 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87ede00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f87e80b4a90 con 0x7f87e80b29e0 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87e6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e80cfbc0 0x7f87e8161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87e6000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e80cfbc0 0x7f87e8161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55354/0 (socket says 172.21.15.110:55354) 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.066+0000 7f87e6000640 1 -- 172.21.15.110:0/3033543061 learned_addr learned my addr 172.21.15.110:0/3033543061 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87e6a00640 1 --2- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80b29e0 0x7f87e815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87e7400640 1 --2- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87e8145f30 0x7f87e815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87e6000640 1 -- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80b29e0 msgr2=0x7f87e815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87e6000640 1 --2- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f87e80b29e0 0x7f87e815c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87e6000640 1 -- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87e8145f30 msgr2=0x7f87e815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87e6000640 1 --2- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87e8145f30 0x7f87e815ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87e6000640 1 -- 172.21.15.110:0/3033543061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e806f3d0 con 0x7f87e80cfbc0 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87e6000640 1 --2- 172.21.15.110:0/3033543061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e80cfbc0 0x7f87e8161160 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f87d400d800 tx=0x7f87d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87df400640 1 -- 172.21.15.110:0/3033543061 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4018020 con 0x7f87e80cfbc0 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87df400640 1 -- 172.21.15.110:0/3033543061 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d40096a0 con 0x7f87e80cfbc0 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87ede00640 1 -- 172.21.15.110:0/3033543061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e8162170 con 0x7f87e80cfbc0 2024-09-10T10:04:46.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87df400640 1 -- 172.21.15.110:0/3033543061 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4009a50 con 0x7f87e80cfbc0 2024-09-10T10:04:46.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87ede00640 1 -- 172.21.15.110:0/3033543061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e8161bb0 con 0x7f87e80cfbc0 2024-09-10T10:04:46.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87ede00640 1 -- 172.21.15.110:0/3033543061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e8160030 con 0x7f87e80cfbc0 2024-09-10T10:04:46.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87df400640 1 -- 172.21.15.110:0/3033543061 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f87d401b020 con 0x7f87e80cfbc0 2024-09-10T10:04:46.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87df400640 1 --2- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f87c8067f20 0x7f87c806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.070+0000 7f87df400640 1 -- 172.21.15.110:0/3033543061 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f87d4086c10 con 0x7f87e80cfbc0 2024-09-10T10:04:46.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.074+0000 7f87e6a00640 1 --2- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f87c8067f20 0x7f87c806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.074+0000 7f87e6a00640 1 --2- 172.21.15.110:0/3033543061 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f87c8067f20 0x7f87c806a3e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f87d80ad000 tx=0x7f87d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.074+0000 7f87df400640 1 -- 172.21.15.110:0/3033543061 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f87d4022020 con 0x7f87e80cfbc0 2024-09-10T10:04:46.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.254+0000 7f87ede00640 1 -- 172.21.15.110:0/3033543061 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f87e8145380 con 0x7f87c8067f20 2024-09-10T10:04:46.260 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:46.258+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:46.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.258+0000 7f87df400640 1 -- 172.21.15.110:0/3033543061 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f87e8145380 con 0x7f87c8067f20 2024-09-10T10:04:46.261 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:46.269 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:46.270 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-10T10:04:46.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf6a000640 1 Processor -- start 2024-09-10T10:04:46.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf6a000640 1 -- start start 2024-09-10T10:04:46.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf6a000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbf640d1b40 0x7fbf64070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf6a000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6415e8b0 0x7fbf640b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf6a000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf640b4770 con 0x7fbf6406e290 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf6a000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbf640b79b0 con 0x7fbf640d1b40 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf6a000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbf640b4a50 con 0x7fbf6415e8b0 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf62a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbf640d1b40 0x7fbf64070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf62a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbf640d1b40 0x7fbf64070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55368/0 (socket says 172.21.15.110:55368) 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.406+0000 7fbf62a00640 1 -- 172.21.15.110:0/3392582895 learned_addr learned my addr 172.21.15.110:0/3392582895 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf62a00640 1 -- 172.21.15.110:0/3392582895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6415e8b0 msgr2=0x7fbf640b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf63e00640 1 --2- 172.21.15.110:0/3392582895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6415e8b0 0x7fbf640b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf62a00640 1 --2- 172.21.15.110:0/3392582895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6415e8b0 0x7fbf640b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf62a00640 1 -- 172.21.15.110:0/3392582895 >> v1:172.21.15.145:6789/0 conn(0x7fbf6406e290 legacy=0x7fbf6406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf62a00640 1 -- 172.21.15.110:0/3392582895 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf640b4d30 con 0x7fbf640d1b40 2024-09-10T10:04:46.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf63e00640 1 --2- 172.21.15.110:0/3392582895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6415e8b0 0x7fbf640b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:46.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf62a00640 1 --2- 172.21.15.110:0/3392582895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640d1b40 0x7fbf64070b40 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009870 tx=0x7fbf5001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12304f507a8d7314 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf62000640 1 -- 172.21.15.110:0/3392582895 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf500b1070 con 0x7fbf640d1b40 2024-09-10T10:04:46.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf62000640 1 -- 172.21.15.110:0/3392582895 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf500ac620 con 0x7fbf640d1b40 2024-09-10T10:04:46.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf62000640 1 -- 172.21.15.110:0/3392582895 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf500ac9f0 con 0x7fbf640d1b40 2024-09-10T10:04:46.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf6a000640 1 -- 172.21.15.110:0/3392582895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640d1b40 msgr2=0x7fbf64070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf6a000640 1 --2- 172.21.15.110:0/3392582895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640d1b40 0x7fbf64070b40 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009870 tx=0x7fbf5001c4f0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf6a000640 1 -- 172.21.15.110:0/3392582895 shutdown_connections 2024-09-10T10:04:46.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf6a000640 1 --2- 172.21.15.110:0/3392582895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6415e8b0 0x7fbf640b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf6a000640 1 --2- 172.21.15.110:0/3392582895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640d1b40 0x7fbf64070b40 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf6a000640 1 -- 172.21.15.110:0/3392582895 >> 172.21.15.110:0/3392582895 conn(0x7fbf6406f420 msgr2=0x7fbf6406f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf6a000640 1 -- 172.21.15.110:0/3392582895 shutdown_connections 2024-09-10T10:04:46.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.410+0000 7fbf6a000640 1 -- 172.21.15.110:0/3392582895 wait complete. 2024-09-10T10:04:46.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 Processor -- start 2024-09-10T10:04:46.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 -- start start 2024-09-10T10:04:46.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6406e290 0x7fbf6414a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf6415e8b0 0x7fbf6415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf6415f4a0 0x7fbf64155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6406d510 con 0x7fbf6415e8b0 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbf640b4550 con 0x7fbf6415f4a0 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6406d690 con 0x7fbf6406e290 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf6415e8b0 0x7fbf6415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf6415e8b0 0x7fbf6415a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55374/0 (socket says 172.21.15.110:55374) 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 -- 172.21.15.110:0/2540510986 learned_addr learned my addr 172.21.15.110:0/2540510986 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf63e00640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf6415f4a0 0x7fbf64155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 -- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6406e290 msgr2=0x7fbf6414a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf63400640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6406e290 0x7fbf6414a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6406e290 0x7fbf6414a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 -- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf6415f4a0 msgr2=0x7fbf64155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf6415f4a0 0x7fbf64155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 -- 172.21.15.110:0/2540510986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6415ac70 con 0x7fbf6415e8b0 2024-09-10T10:04:46.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf63e00640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf6415f4a0 0x7fbf64155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:46.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf63400640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf6406e290 0x7fbf6414a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:46.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf62a00640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf6415e8b0 0x7fbf6415a420 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbf5001cd90 tx=0x7fbf50019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf5b400640 1 -- 172.21.15.110:0/2540510986 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf500b1040 con 0x7fbf6415e8b0 2024-09-10T10:04:46.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf5b400640 1 -- 172.21.15.110:0/2540510986 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf50004020 con 0x7fbf6415e8b0 2024-09-10T10:04:46.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf5b400640 1 -- 172.21.15.110:0/2540510986 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf500043d0 con 0x7fbf6415e8b0 2024-09-10T10:04:46.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 -- 172.21.15.110:0/2540510986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6415af50 con 0x7fbf6415e8b0 2024-09-10T10:04:46.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.414+0000 7fbf6a000640 1 -- 172.21.15.110:0/2540510986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6415b510 con 0x7fbf6415e8b0 2024-09-10T10:04:46.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.418+0000 7fbf6a000640 1 -- 172.21.15.110:0/2540510986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf640b7470 con 0x7fbf6415e8b0 2024-09-10T10:04:46.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.418+0000 7fbf5b400640 1 -- 172.21.15.110:0/2540510986 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fbf500be060 con 0x7fbf6415e8b0 2024-09-10T10:04:46.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.418+0000 7fbf5b400640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf44067ef0 0x7fbf4406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.418+0000 7fbf5b400640 1 -- 172.21.15.110:0/2540510986 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fbf50120140 con 0x7fbf6415e8b0 2024-09-10T10:04:46.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.418+0000 7fbf63400640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf44067ef0 0x7fbf4406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.418+0000 7fbf63400640 1 --2- 172.21.15.110:0/2540510986 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf44067ef0 0x7fbf4406a3b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbf54007fd0 tx=0x7fbf54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.422+0000 7fbf5b400640 1 -- 172.21.15.110:0/2540510986 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbf50016110 con 0x7fbf6415e8b0 2024-09-10T10:04:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.606+0000 7fbf6a000640 1 -- 172.21.15.110:0/2540510986 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fbf6414a970 con 0x7fbf44067ef0 2024-09-10T10:04:46.611 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:46.610+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:46.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.606+0000 7fbf5b400640 1 -- 172.21.15.110:0/2540510986 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fbf6414a970 con 0x7fbf44067ef0 2024-09-10T10:04:46.611 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:46.619 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:46.620 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-10T10:04:46.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.754+0000 7f5010600640 1 Processor -- start 2024-09-10T10:04:46.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f5010600640 1 -- start start 2024-09-10T10:04:46.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f5010600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f50080cfe30 0x7f5008071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f5010600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f500806e7a0 0x7f50080cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f5010600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f500806d2d0 con 0x7f50080cd510 2024-09-10T10:04:46.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f5010600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f500815d9a0 con 0x7f500806e7a0 2024-09-10T10:04:46.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f5010600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f500806d5b0 con 0x7f50080cfe30 2024-09-10T10:04:46.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500e600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f500806e7a0 0x7f50080cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500e600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f500806e7a0 0x7f50080cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55390/0 (socket says 172.21.15.110:55390) 2024-09-10T10:04:46.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500e600640 1 -- 172.21.15.110:0/2923677414 learned_addr learned my addr 172.21.15.110:0/2923677414 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:46.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500d200640 1 --1- 172.21.15.110:0/2923677414 >> v1:172.21.15.110:6789/0 conn(0x7f50080cd510 0x7f500806e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:47692/0 (socket says 172.21.15.110:47692) 2024-09-10T10:04:46.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500dc00640 1 --2- 172.21.15.110:0/2923677414 >> v2:172.21.15.145:3300/0 conn(0x7f50080cfe30 0x7f5008071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500e600640 1 -- 172.21.15.110:0/2923677414 >> v2:172.21.15.145:3300/0 conn(0x7f50080cfe30 msgr2=0x7f5008071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500e600640 1 --2- 172.21.15.110:0/2923677414 >> v2:172.21.15.145:3300/0 conn(0x7f50080cfe30 0x7f5008071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500e600640 1 -- 172.21.15.110:0/2923677414 >> v1:172.21.15.110:6789/0 conn(0x7f50080cd510 legacy=0x7f500806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500e600640 1 -- 172.21.15.110:0/2923677414 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500806d890 con 0x7f500806e7a0 2024-09-10T10:04:46.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.758+0000 7f500dc00640 1 --2- 172.21.15.110:0/2923677414 >> v2:172.21.15.145:3300/0 conn(0x7f50080cfe30 0x7f5008071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:46.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f500e600640 1 --2- 172.21.15.110:0/2923677414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f500806e7a0 0x7f50080cd9a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f50080ce120 tx=0x7f500001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f289c7e79d230a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5007e00640 1 -- 172.21.15.110:0/2923677414 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000a5730 con 0x7f500806e7a0 2024-09-10T10:04:46.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5007e00640 1 -- 172.21.15.110:0/2923677414 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50000a58d0 con 0x7f500806e7a0 2024-09-10T10:04:46.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5007e00640 1 -- 172.21.15.110:0/2923677414 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000a5ca0 con 0x7f500806e7a0 2024-09-10T10:04:46.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 -- 172.21.15.110:0/2923677414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f500806e7a0 msgr2=0x7f50080cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 --2- 172.21.15.110:0/2923677414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f500806e7a0 0x7f50080cd9a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f50080ce120 tx=0x7f500001c960 comp rx=0 tx=0).stop 2024-09-10T10:04:46.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 -- 172.21.15.110:0/2923677414 shutdown_connections 2024-09-10T10:04:46.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 --2- 172.21.15.110:0/2923677414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f500806e7a0 0x7f50080cd9a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 --2- 172.21.15.110:0/2923677414 >> v2:172.21.15.145:3300/0 conn(0x7f50080cfe30 0x7f5008071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 -- 172.21.15.110:0/2923677414 >> 172.21.15.110:0/2923677414 conn(0x7f500806bc50 msgr2=0x7f500806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 -- 172.21.15.110:0/2923677414 shutdown_connections 2024-09-10T10:04:46.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 -- 172.21.15.110:0/2923677414 wait complete. 2024-09-10T10:04:46.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 Processor -- start 2024-09-10T10:04:46.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.762+0000 7f5010600640 1 -- start start 2024-09-10T10:04:46.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f500806e7a0 0x7f500814d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50080cd510 0x7f5008153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50080cfe30 0x7f5008152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5008072190 con 0x7f500806e7a0 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f500815d6c0 con 0x7f50080cd510 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5008169180 con 0x7f50080cfe30 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f500806e7a0 0x7f500814d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f500806e7a0 0x7f500814d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55398/0 (socket says 172.21.15.110:55398) 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 -- 172.21.15.110:0/4007590220 learned_addr learned my addr 172.21.15.110:0/4007590220 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500e600640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50080cfe30 0x7f5008152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500d200640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50080cd510 0x7f5008153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 -- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50080cfe30 msgr2=0x7f5008152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50080cfe30 0x7f5008152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 -- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50080cd510 msgr2=0x7f5008153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f50080cd510 0x7f5008153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 -- 172.21.15.110:0/4007590220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50080b4b80 con 0x7f500806e7a0 2024-09-10T10:04:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500e600640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f50080cfe30 0x7f5008152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f500dc00640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f500806e7a0 0x7f500814d280 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c820 tx=0x7f4ffc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5005600640 1 -- 172.21.15.110:0/4007590220 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc015070 con 0x7f500806e7a0 2024-09-10T10:04:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5005600640 1 -- 172.21.15.110:0/4007590220 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ffc007a30 con 0x7f500806e7a0 2024-09-10T10:04:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5005600640 1 -- 172.21.15.110:0/4007590220 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc007e40 con 0x7f500806e7a0 2024-09-10T10:04:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 -- 172.21.15.110:0/4007590220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50080b1b10 con 0x7f500806e7a0 2024-09-10T10:04:46.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 -- 172.21.15.110:0/4007590220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500814e8b0 con 0x7f500806e7a0 2024-09-10T10:04:46.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.766+0000 7f5010600640 1 -- 172.21.15.110:0/4007590220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50080af7b0 con 0x7f500806e7a0 2024-09-10T10:04:46.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.770+0000 7f5005600640 1 -- 172.21.15.110:0/4007590220 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f4ffc018070 con 0x7f500806e7a0 2024-09-10T10:04:46.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.770+0000 7f5005600640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4fe8067f20 0x7f4fe806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.770+0000 7f5005600640 1 -- 172.21.15.110:0/4007590220 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f4ffc0866e0 con 0x7f500806e7a0 2024-09-10T10:04:46.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.770+0000 7f500d200640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4fe8067f20 0x7f4fe806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.770+0000 7f500d200640 1 --2- 172.21.15.110:0/4007590220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4fe8067f20 0x7f4fe806a3e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f500815a520 tx=0x7f4ff4007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.774+0000 7f5005600640 1 -- 172.21.15.110:0/4007590220 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4ffc056870 con 0x7f500806e7a0 2024-09-10T10:04:46.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.954+0000 7f5010600640 1 -- 172.21.15.110:0/4007590220 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f500814e420 con 0x7f4fe8067f20 2024-09-10T10:04:46.960 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:46.958+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:46.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:46.958+0000 7f5005600640 1 -- 172.21.15.110:0/4007590220 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f500814e420 con 0x7f4fe8067f20 2024-09-10T10:04:46.961 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:46.970 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:46.971 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-10T10:04:47.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f04600640 1 Processor -- start 2024-09-10T10:04:47.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f04600640 1 -- start start 2024-09-10T10:04:47.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f04600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7efc0d1b40 0x7f7efc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f04600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 0x7f7efc0b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f04600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7efc0b4770 con 0x7f7efc06e290 2024-09-10T10:04:47.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f04600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f7efc0b79b0 con 0x7f7efc0d1b40 2024-09-10T10:04:47.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f04600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc0b4a50 con 0x7f7efc15e8b0 2024-09-10T10:04:47.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f01200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7efc0d1b40 0x7f7efc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f01200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7efc0d1b40 0x7f7efc070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55410/0 (socket says 172.21.15.110:55410) 2024-09-10T10:04:47.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f01200640 1 -- 172.21.15.110:0/3034963971 learned_addr learned my addr 172.21.15.110:0/3034963971 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:47.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f02600640 1 --2- 172.21.15.110:0/3034963971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 0x7f7efc0b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f01200640 1 -- 172.21.15.110:0/3034963971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 msgr2=0x7f7efc0b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:47.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f01200640 1 --2- 172.21.15.110:0/3034963971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 0x7f7efc0b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f01200640 1 -- 172.21.15.110:0/3034963971 >> v1:172.21.15.145:6789/0 conn(0x7f7efc06e290 legacy=0x7f7efc06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.106+0000 7f7f01200640 1 -- 172.21.15.110:0/3034963971 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc0b4d30 con 0x7f7efc0d1b40 2024-09-10T10:04:47.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f01200640 1 --2- 172.21.15.110:0/3034963971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7efc0d1b40 0x7f7efc070b40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009870 tx=0x7f7ee801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f527c60337d6d51e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:47.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7efbe00640 1 -- 172.21.15.110:0/3034963971 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80ab640 con 0x7f7efc0d1b40 2024-09-10T10:04:47.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7efbe00640 1 -- 172.21.15.110:0/3034963971 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee80b0070 con 0x7f7efc0d1b40 2024-09-10T10:04:47.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7efbe00640 1 -- 172.21.15.110:0/3034963971 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80aba00 con 0x7f7efc0d1b40 2024-09-10T10:04:47.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f04600640 1 -- 172.21.15.110:0/3034963971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7efc0d1b40 msgr2=0x7f7efc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f04600640 1 --2- 172.21.15.110:0/3034963971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7efc0d1b40 0x7f7efc070b40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009870 tx=0x7f7ee801c530 comp rx=0 tx=0).stop 2024-09-10T10:04:47.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f04600640 1 -- 172.21.15.110:0/3034963971 shutdown_connections 2024-09-10T10:04:47.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f04600640 1 --2- 172.21.15.110:0/3034963971 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 0x7f7efc0b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f04600640 1 --2- 172.21.15.110:0/3034963971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7efc0d1b40 0x7f7efc070b40 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f04600640 1 -- 172.21.15.110:0/3034963971 >> 172.21.15.110:0/3034963971 conn(0x7f7efc06f420 msgr2=0x7f7efc06f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:47.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f04600640 1 -- 172.21.15.110:0/3034963971 shutdown_connections 2024-09-10T10:04:47.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.110+0000 7f7f04600640 1 -- 172.21.15.110:0/3034963971 wait complete. 2024-09-10T10:04:47.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 Processor -- start 2024-09-10T10:04:47.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 -- start start 2024-09-10T10:04:47.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7efc06e290 0x7f7efc160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7efc0d1b40 0x7f7efc153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 0x7f7efc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc06d510 con 0x7f7efc06e290 2024-09-10T10:04:47.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc0b4550 con 0x7f7efc0d1b40 2024-09-10T10:04:47.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc06d690 con 0x7f7efc15e8b0 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7efc06e290 0x7f7efc160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7efc06e290 0x7f7efc160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55424/0 (socket says 172.21.15.110:55424) 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 -- 172.21.15.110:0/1865165202 learned_addr learned my addr 172.21.15.110:0/1865165202 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f02600640 1 --2- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 0x7f7efc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01200640 1 --2- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7efc0d1b40 0x7f7efc153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 -- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 msgr2=0x7f7efc0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 --2- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7efc15e8b0 0x7f7efc0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 -- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7efc0d1b40 msgr2=0x7f7efc153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 --2- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7efc0d1b40 0x7f7efc153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 -- 172.21.15.110:0/1865165202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc1547b0 con 0x7f7efc06e290 2024-09-10T10:04:47.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f01c00640 1 --2- 172.21.15.110:0/1865165202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7efc06e290 0x7f7efc160d70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000c790 tx=0x7f7ef000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:47.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7ef9600640 1 -- 172.21.15.110:0/1865165202 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0016020 con 0x7f7efc06e290 2024-09-10T10:04:47.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7ef9600640 1 -- 172.21.15.110:0/1865165202 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef0006d60 con 0x7f7efc06e290 2024-09-10T10:04:47.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7ef9600640 1 -- 172.21.15.110:0/1865165202 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0003bc0 con 0x7f7efc06e290 2024-09-10T10:04:47.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 -- 172.21.15.110:0/1865165202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc160290 con 0x7f7efc06e290 2024-09-10T10:04:47.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.114+0000 7f7f04600640 1 -- 172.21.15.110:0/1865165202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc160850 con 0x7f7efc06e290 2024-09-10T10:04:47.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.118+0000 7f7f04600640 1 -- 172.21.15.110:0/1865165202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7efc0b7470 con 0x7f7efc06e290 2024-09-10T10:04:47.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.118+0000 7f7ef9600640 1 -- 172.21.15.110:0/1865165202 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f7ef000a850 con 0x7f7efc06e290 2024-09-10T10:04:47.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.118+0000 7f7ef9600640 1 --2- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7edc067f20 0x7f7edc06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.118+0000 7f7ef9600640 1 -- 172.21.15.110:0/1865165202 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f7ef00863f0 con 0x7f7efc06e290 2024-09-10T10:04:47.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.118+0000 7f7f01200640 1 --2- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7edc067f20 0x7f7edc06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.118+0000 7f7f01200640 1 --2- 172.21.15.110:0/1865165202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7edc067f20 0x7f7edc06a3e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80af000 tx=0x7f7ee80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:47.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.122+0000 7f7ef9600640 1 -- 172.21.15.110:0/1865165202 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7ef0020020 con 0x7f7efc06e290 2024-09-10T10:04:47.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.302+0000 7f7f04600640 1 -- 172.21.15.110:0/1865165202 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f7efc154a90 con 0x7f7edc067f20 2024-09-10T10:04:47.309 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:04:47.306+0000 7f95dda00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T10:04:47.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.306+0000 7f7ef9600640 1 -- 172.21.15.110:0/1865165202 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7efc154a90 con 0x7f7edc067f20 2024-09-10T10:04:47.309 INFO:teuthology.orchestra.run.smithi110.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T10:04:47.318 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:04:47.319 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:04:47.319 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:04:47.419 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:04:47.508 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:04:47.508 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:04:47.508 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:47.508 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:47.517 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:47.517 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:47.573 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:04:47.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead9e00640 1 Processor -- start 2024-09-10T10:04:47.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead9e00640 1 -- start start 2024-09-10T10:04:47.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead9e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead40726e0 0x7fead40cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead9e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fead406efa0 0x7fead40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead9e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fead40b5430 0x7fead40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead9e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fead40b5880 con 0x7fead40b5430 2024-09-10T10:04:47.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead9e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fead40cee50 con 0x7fead406efa0 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead9e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fead406f3e0 con 0x7fead40726e0 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead3400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fead40b5430 0x7fead40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead3400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fead40b5430 0x7fead40b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55440/0 (socket says 172.21.15.110:55440) 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead3400640 1 -- 172.21.15.110:0/3994179998 learned_addr learned my addr 172.21.15.110:0/3994179998 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead2a00640 1 --2- 172.21.15.110:0/3994179998 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead40726e0 0x7fead40cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7feacbe00640 1 --2- 172.21.15.110:0/3994179998 >> v2:172.21.15.145:3300/0 conn(0x7fead406efa0 0x7fead40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead3400640 1 -- 172.21.15.110:0/3994179998 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead40726e0 msgr2=0x7fead40cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead3400640 1 --2- 172.21.15.110:0/3994179998 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead40726e0 0x7fead40cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead3400640 1 -- 172.21.15.110:0/3994179998 >> v2:172.21.15.145:3300/0 conn(0x7fead406efa0 msgr2=0x7fead40b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead3400640 1 --2- 172.21.15.110:0/3994179998 >> v2:172.21.15.145:3300/0 conn(0x7fead406efa0 0x7fead40b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead3400640 1 -- 172.21.15.110:0/3994179998 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead415fda0 con 0x7fead40b5430 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7fead2a00640 1 --2- 172.21.15.110:0/3994179998 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead40726e0 0x7fead40cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:47.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.750+0000 7feacbe00640 1 --2- 172.21.15.110:0/3994179998 >> v2:172.21.15.145:3300/0 conn(0x7fead406efa0 0x7fead40b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:47.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead3400640 1 --2- 172.21.15.110:0/3994179998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead40b34d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fead406d180 tx=0x7feacc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec372b88833d3364 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:47.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead2000640 1 -- 172.21.15.110:0/3994179998 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a5840 con 0x7fead40b5430 2024-09-10T10:04:47.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead2000640 1 -- 172.21.15.110:0/3994179998 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feacc0a59e0 con 0x7fead40b5430 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead2000640 1 -- 172.21.15.110:0/3994179998 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a5db0 con 0x7fead40b5430 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 -- 172.21.15.110:0/3994179998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 msgr2=0x7fead40b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 --2- 172.21.15.110:0/3994179998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead40b34d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fead406d180 tx=0x7feacc01c9c0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 -- 172.21.15.110:0/3994179998 shutdown_connections 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 --2- 172.21.15.110:0/3994179998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead40b34d0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 --2- 172.21.15.110:0/3994179998 >> v2:172.21.15.145:3300/0 conn(0x7fead406efa0 0x7fead40b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 --2- 172.21.15.110:0/3994179998 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead40726e0 0x7fead40cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 -- 172.21.15.110:0/3994179998 >> 172.21.15.110:0/3994179998 conn(0x7fead4062d10 msgr2=0x7fead406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 -- 172.21.15.110:0/3994179998 shutdown_connections 2024-09-10T10:04:47.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 -- 172.21.15.110:0/3994179998 wait complete. 2024-09-10T10:04:47.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 Processor -- start 2024-09-10T10:04:47.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.754+0000 7fead9e00640 1 -- start start 2024-09-10T10:04:47.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead406efa0 0x7fead41f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead40726e0 0x7fead41573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead41578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fead4072ac0 con 0x7fead40b5430 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fead4073270 con 0x7fead40726e0 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fead4072c40 con 0x7fead406efa0 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead41578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead41578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55456/0 (socket says 172.21.15.110:55456) 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 -- 172.21.15.110:0/4256502924 learned_addr learned my addr 172.21.15.110:0/4256502924 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7feacbe00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead40726e0 0x7fead41573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead2a00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead406efa0 0x7fead41f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 -- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead406efa0 msgr2=0x7fead41f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead406efa0 0x7fead41f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 -- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead40726e0 msgr2=0x7fead41573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead40726e0 0x7fead41573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 -- 172.21.15.110:0/4256502924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead415e420 con 0x7fead40b5430 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7feacbe00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead40726e0 0x7fead41573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead3400640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead41578f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7feacc0af040 tx=0x7feacc019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7feacb400640 1 -- 172.21.15.110:0/4256502924 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0bc020 con 0x7fead40b5430 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7feacb400640 1 -- 172.21.15.110:0/4256502924 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feacc0b8690 con 0x7fead40b5430 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7feacb400640 1 -- 172.21.15.110:0/4256502924 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0b8a80 con 0x7fead40b5430 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead41557f0 con 0x7fead40b5430 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead40b2410 con 0x7fead40b5430 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead415f950 con 0x7fead40b5430 2024-09-10T10:04:47.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.758+0000 7feacb400640 1 -- 172.21.15.110:0/4256502924 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7feacc0bf020 con 0x7fead40b5430 2024-09-10T10:04:47.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.762+0000 7feacb400640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feab0067f20 0x7feab006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.762+0000 7feacb400640 1 -- 172.21.15.110:0/4256502924 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7feacc1219b0 con 0x7fead40b5430 2024-09-10T10:04:47.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.762+0000 7fead2a00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feab0067f20 0x7feab006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.762+0000 7fead2a00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feab0067f20 0x7feab006a3e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feac400ed30 tx=0x7feac4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:47.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.766+0000 7feacb400640 1 -- 172.21.15.110:0/4256502924 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feacc0c6030 con 0x7fead40b5430 2024-09-10T10:04:47.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fead40b2640 con 0x7fead40b5430 2024-09-10T10:04:47.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7feacb400640 1 -- 172.21.15.110:0/4256502924 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v130) ==== 80+0+0 (secure 0 0 0) 0x7feacc0165e0 con 0x7fead40b5430 2024-09-10T10:04:47.938 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:04:47.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feab0067f20 msgr2=0x7feab006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feab0067f20 0x7feab006a3e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feac400ed30 tx=0x7feac4008040 comp rx=0 tx=0).stop 2024-09-10T10:04:47.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 msgr2=0x7fead41578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead41578f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7feacc0af040 tx=0x7feacc019760 comp rx=0 tx=0).stop 2024-09-10T10:04:47.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 shutdown_connections 2024-09-10T10:04:47.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feab0067f20 0x7feab006a3e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fead40b5430 0x7fead41578f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead40726e0 0x7fead41573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 --2- 172.21.15.110:0/4256502924 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fead406efa0 0x7fead41f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 >> 172.21.15.110:0/4256502924 conn(0x7fead4062d10 msgr2=0x7fead406d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:47.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 shutdown_connections 2024-09-10T10:04:47.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:47.934+0000 7fead9e00640 1 -- 172.21.15.110:0/4256502924 wait complete. 2024-09-10T10:04:47.959 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:04:47.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:47.959 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:04:47.985 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:04:47.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:47.986 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:04:48.041 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:04:48.041 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:48.041 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:48.050 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:48.051 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:48.101 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:04:48.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.230+0000 7f233bc00640 1 Processor -- start 2024-09-10T10:04:48.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f233bc00640 1 -- start start 2024-09-10T10:04:48.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f233bc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f23340b3f40 0x7f23340cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f233bc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f23340cff70 0x7f23340cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f233bc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f233415d4d0 con 0x7f23340cfb90 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f233bc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f233406d130 con 0x7f23340b3f40 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f233bc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f233415d7b0 con 0x7f23340cff70 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2339200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f23340cfb90 0x7f233415ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35446/0 (socket says 172.21.15.110:35446) 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2339200640 1 -- 172.21.15.110:0/2870863632 learned_addr learned my addr 172.21.15.110:0/2870863632 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2333e00640 1 --2- 172.21.15.110:0/2870863632 >> v2:172.21.15.110:3300/0 conn(0x7f23340b3f40 0x7f23340cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2339c00640 1 --2- 172.21.15.110:0/2870863632 >> v2:172.21.15.145:3300/0 conn(0x7f23340cff70 0x7f23340cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2333e00640 1 -- 172.21.15.110:0/2870863632 >> v2:172.21.15.145:3300/0 conn(0x7f23340cff70 msgr2=0x7f23340cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2333e00640 1 --2- 172.21.15.110:0/2870863632 >> v2:172.21.15.145:3300/0 conn(0x7f23340cff70 0x7f23340cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2333e00640 1 -- 172.21.15.110:0/2870863632 >> v1:172.21.15.110:6789/0 conn(0x7f23340cfb90 legacy=0x7f233415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2333400640 1 -- 172.21.15.110:0/2870863632 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2532839167 0 0) 0x7f233415d4d0 con 0x7f23340cfb90 2024-09-10T10:04:48.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.234+0000 7f2333e00640 1 -- 172.21.15.110:0/2870863632 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233406d380 con 0x7f23340b3f40 2024-09-10T10:04:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f2333e00640 1 --2- 172.21.15.110:0/2870863632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340b3f40 0x7f23340cd5a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f23340ce170 tx=0x7f232001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbb7ff0beabc147 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f2333400640 1 -- 172.21.15.110:0/2870863632 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200b0070 con 0x7f23340b3f40 2024-09-10T10:04:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f2333400640 1 -- 172.21.15.110:0/2870863632 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23200ab600 con 0x7f23340b3f40 2024-09-10T10:04:48.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f2333400640 1 -- 172.21.15.110:0/2870863632 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200ab9d0 con 0x7f23340b3f40 2024-09-10T10:04:48.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 -- 172.21.15.110:0/2870863632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340b3f40 msgr2=0x7f23340cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 --2- 172.21.15.110:0/2870863632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340b3f40 0x7f23340cd5a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f23340ce170 tx=0x7f232001c3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 -- 172.21.15.110:0/2870863632 shutdown_connections 2024-09-10T10:04:48.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 --2- 172.21.15.110:0/2870863632 >> v2:172.21.15.145:3300/0 conn(0x7f23340cff70 0x7f23340cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 --2- 172.21.15.110:0/2870863632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340b3f40 0x7f23340cd5a0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 -- 172.21.15.110:0/2870863632 >> 172.21.15.110:0/2870863632 conn(0x7f2334062d10 msgr2=0x7f233406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 -- 172.21.15.110:0/2870863632 shutdown_connections 2024-09-10T10:04:48.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 -- 172.21.15.110:0/2870863632 wait complete. 2024-09-10T10:04:48.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 Processor -- start 2024-09-10T10:04:48.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.238+0000 7f233bc00640 1 -- start start 2024-09-10T10:04:48.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23340b3f40 0x7f2334150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340cfb90 0x7f23341623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23340cff70 0x7f23340b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f233406e720 con 0x7f23340cfb90 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f233406e5a0 con 0x7f23340cff70 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f233406e8a0 con 0x7f23340b3f40 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340cfb90 0x7f23341623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340cfb90 0x7f23341623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34394/0 (socket says 172.21.15.110:34394) 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 -- 172.21.15.110:0/791449389 learned_addr learned my addr 172.21.15.110:0/791449389 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2339c00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23340cff70 0x7f23340b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2339200640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23340b3f40 0x7f2334150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 -- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23340b3f40 msgr2=0x7f2334150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23340b3f40 0x7f2334150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 -- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23340cff70 msgr2=0x7f23340b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23340cff70 0x7f23340b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 -- 172.21.15.110:0/791449389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2334162db0 con 0x7f23340cfb90 2024-09-10T10:04:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2339c00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23340cff70 0x7f23340b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:48.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2333e00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340cfb90 0x7f23341623a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f23200ae040 tx=0x7f2320019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2330c00640 1 -- 172.21.15.110:0/791449389 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200b8020 con 0x7f23340cfb90 2024-09-10T10:04:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2330c00640 1 -- 172.21.15.110:0/791449389 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23200b4410 con 0x7f23340cfb90 2024-09-10T10:04:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2330c00640 1 -- 172.21.15.110:0/791449389 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200b0040 con 0x7f23340cfb90 2024-09-10T10:04:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2334163090 con 0x7f23340cfb90 2024-09-10T10:04:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2334163650 con 0x7f23340cfb90 2024-09-10T10:04:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233415ca10 con 0x7f23340cfb90 2024-09-10T10:04:48.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.242+0000 7f2330c00640 1 -- 172.21.15.110:0/791449389 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f23200bb020 con 0x7f23340cfb90 2024-09-10T10:04:48.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.246+0000 7f2330c00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2314067ef0 0x7f231406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.246+0000 7f2330c00640 1 -- 172.21.15.110:0/791449389 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f23201204e0 con 0x7f23340cfb90 2024-09-10T10:04:48.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.246+0000 7f2339200640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2314067ef0 0x7f231406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.246+0000 7f2339200640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2314067ef0 0x7f231406a3b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2324007fd0 tx=0x7f2324007f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.250+0000 7f2330c00640 1 -- 172.21.15.110:0/791449389 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f23200c2030 con 0x7f23340cfb90 2024-09-10T10:04:48.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.418+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23340cdad0 con 0x7f23340cfb90 2024-09-10T10:04:48.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.418+0000 7f2330c00640 1 -- 172.21.15.110:0/791449389 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v130) ==== 80+0+0 (secure 0 0 0) 0x7f2320016070 con 0x7f23340cfb90 2024-09-10T10:04:48.423 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.418+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2314067ef0 msgr2=0x7f231406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.418+0000 7f233bc00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2314067ef0 0x7f231406a3b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2324007fd0 tx=0x7f2324007f00 comp rx=0 tx=0).stop 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.418+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340cfb90 msgr2=0x7f23341623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.418+0000 7f233bc00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340cfb90 0x7f23341623a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f23200ae040 tx=0x7f2320019190 comp rx=0 tx=0).stop 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.422+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 shutdown_connections 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.422+0000 7f233bc00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2314067ef0 0x7f231406a3b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.422+0000 7f233bc00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23340cff70 0x7f23340b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.422+0000 7f233bc00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340cfb90 0x7f23341623a0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.422+0000 7f233bc00640 1 --2- 172.21.15.110:0/791449389 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f23340b3f40 0x7f2334150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.422+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 >> 172.21.15.110:0/791449389 conn(0x7f2334062d10 msgr2=0x7f23340b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.422+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 shutdown_connections 2024-09-10T10:04:48.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.422+0000 7f233bc00640 1 -- 172.21.15.110:0/791449389 wait complete. 2024-09-10T10:04:48.444 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:04:48.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:48.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:04:48.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:04:48.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:04:48.472 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:04:48.522 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:04:48.522 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:48.522 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:48.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:04:48.531 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:04:48.531 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:48.531 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:48.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:04:48.574 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:48.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e8000640 1 Processor -- start 2024-09-10T10:04:48.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e8000640 1 -- start start 2024-09-10T10:04:48.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e8000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e0155970 0x7f63e00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e8000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f63e006f1e0 0x7f63e00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e8000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e0155d50 con 0x7f63e015c560 2024-09-10T10:04:48.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e8000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f63e00b4570 con 0x7f63e006f1e0 2024-09-10T10:04:48.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e8000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f63e00b34d0 con 0x7f63e0155970 2024-09-10T10:04:48.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e6000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f63e006f1e0 0x7f63e00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.706+0000 7f63e6000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f63e006f1e0 0x7f63e00ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34404/0 (socket says 172.21.15.110:34404) 2024-09-10T10:04:48.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63e6000640 1 -- 172.21.15.110:0/2107227573 learned_addr learned my addr 172.21.15.110:0/2107227573 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:48.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63e5600640 1 --2- 172.21.15.110:0/2107227573 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e0155970 0x7f63e00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63e6000640 1 -- 172.21.15.110:0/2107227573 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e0155970 msgr2=0x7f63e00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:48.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63e6000640 1 --2- 172.21.15.110:0/2107227573 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e0155970 0x7f63e00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63e6000640 1 -- 172.21.15.110:0/2107227573 >> v1:172.21.15.145:6789/0 conn(0x7f63e015c560 legacy=0x7f63e006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63e6000640 1 -- 172.21.15.110:0/2107227573 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e00b37b0 con 0x7f63e006f1e0 2024-09-10T10:04:48.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63e6000640 1 --2- 172.21.15.110:0/2107227573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e006f1e0 0x7f63e00ce420 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f63d800bb10 tx=0x7f63d801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2929d15a4c733ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63dfe00640 1 -- 172.21.15.110:0/2107227573 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b4070 con 0x7f63e006f1e0 2024-09-10T10:04:48.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63dfe00640 1 -- 172.21.15.110:0/2107227573 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d80a7c40 con 0x7f63e006f1e0 2024-09-10T10:04:48.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.710+0000 7f63dfe00640 1 -- 172.21.15.110:0/2107227573 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80af440 con 0x7f63e006f1e0 2024-09-10T10:04:48.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- 172.21.15.110:0/2107227573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e006f1e0 msgr2=0x7f63e00ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 --2- 172.21.15.110:0/2107227573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e006f1e0 0x7f63e00ce420 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f63d800bb10 tx=0x7f63d801eb70 comp rx=0 tx=0).stop 2024-09-10T10:04:48.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- 172.21.15.110:0/2107227573 shutdown_connections 2024-09-10T10:04:48.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 --2- 172.21.15.110:0/2107227573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e006f1e0 0x7f63e00ce420 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 --2- 172.21.15.110:0/2107227573 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e0155970 0x7f63e00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- 172.21.15.110:0/2107227573 >> 172.21.15.110:0/2107227573 conn(0x7f63e006bc50 msgr2=0x7f63e006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- 172.21.15.110:0/2107227573 shutdown_connections 2024-09-10T10:04:48.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- 172.21.15.110:0/2107227573 wait complete. 2024-09-10T10:04:48.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 Processor -- start 2024-09-10T10:04:48.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- start start 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63e006f1e0 0x7f63e0155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e0155970 0x7f63e0147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e015c560 0x7f63e0154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00cf170 con 0x7f63e0155970 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00cce00 con 0x7f63e006f1e0 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e8000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f63e00cf2f0 con 0x7f63e015c560 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e0155970 0x7f63e0147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e0155970 0x7f63e0147ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34418/0 (socket says 172.21.15.110:34418) 2024-09-10T10:04:48.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e4c00640 1 -- 172.21.15.110:0/4130178760 learned_addr learned my addr 172.21.15.110:0/4130178760 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e5600640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63e006f1e0 0x7f63e0155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e6000640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e015c560 0x7f63e0154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e4c00640 1 -- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e015c560 msgr2=0x7f63e0154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e4c00640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e015c560 0x7f63e0154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e4c00640 1 -- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63e006f1e0 msgr2=0x7f63e0155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e4c00640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63e006f1e0 0x7f63e0155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e4c00640 1 -- 172.21.15.110:0/4130178760 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e0156530 con 0x7f63e0155970 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.714+0000 7f63e5600640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63e006f1e0 0x7f63e0155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63e4c00640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e0155970 0x7f63e0147ce0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00c340 tx=0x7f63cc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63dd600640 1 -- 172.21.15.110:0/4130178760 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0052d0 con 0x7f63e0155970 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63dd600640 1 -- 172.21.15.110:0/4130178760 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc005470 con 0x7f63e0155970 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63dd600640 1 -- 172.21.15.110:0/4130178760 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc007260 con 0x7f63e0155970 2024-09-10T10:04:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e0156810 con 0x7f63e0155970 2024-09-10T10:04:48.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e0156dd0 con 0x7f63e0155970 2024-09-10T10:04:48.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e00ce420 con 0x7f63e0155970 2024-09-10T10:04:48.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63dd600640 1 -- 172.21.15.110:0/4130178760 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f63cc007400 con 0x7f63e0155970 2024-09-10T10:04:48.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63dd600640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f63c0067ef0 0x7f63c006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63dd600640 1 -- 172.21.15.110:0/4130178760 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f63cc085b80 con 0x7f63e0155970 2024-09-10T10:04:48.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.718+0000 7f63e5600640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f63c0067ef0 0x7f63c006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.722+0000 7f63e5600640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f63c0067ef0 0x7f63c006a3b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f63d400e740 tx=0x7f63d400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.722+0000 7f63dd600640 1 -- 172.21.15.110:0/4130178760 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f63cc055740 con 0x7f63e0155970 2024-09-10T10:04:48.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e006c3b0 con 0x7f63e0155970 2024-09-10T10:04:48.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63dd600640 1 -- 172.21.15.110:0/4130178760 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5239 (secure 0 0 0) 0x7f63cc059a40 con 0x7f63e0155970 2024-09-10T10:04:48.918 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:48.919 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":89,"btime":"2024-09-10T10:04:38:484563+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3129750368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3129750368},{"type":"v1","addr":"172.21.15.145:6834","nonce":3129750368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3791950507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3791950507},{"type":"v1","addr":"172.21.15.145:6835","nonce":3791950507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2024540113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2024540113},{"type":"v1","addr":"172.21.15.110:6834","nonce":2024540113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:04:28.820193+0000","modified":"2024-09-10T10:04:31.842870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1881265533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1881265533},{"type":"v1","addr":"172.21.15.110:6835","nonce":1881265533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7686,"qdb_cluster":[7686]},"id":10}]} 2024-09-10T10:04:48.919 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 89 2024-09-10T10:04:48.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f63c0067ef0 msgr2=0x7f63c006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f63c0067ef0 0x7f63c006a3b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f63d400e740 tx=0x7f63d400b040 comp rx=0 tx=0).stop 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e0155970 msgr2=0x7f63e0147ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e0155970 0x7f63e0147ce0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00c340 tx=0x7f63cc00c800 comp rx=0 tx=0).stop 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 shutdown_connections 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f63c0067ef0 0x7f63c006a3b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f63e015c560 0x7f63e0154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63e0155970 0x7f63e0147ce0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 --2- 172.21.15.110:0/4130178760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63e006f1e0 0x7f63e0155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 >> 172.21.15.110:0/4130178760 conn(0x7f63e006bc50 msgr2=0x7f63e00ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.914+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 shutdown_connections 2024-09-10T10:04:48.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:48.918+0000 7f63e8000640 1 -- 172.21.15.110:0/4130178760 wait complete. 2024-09-10T10:04:48.941 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:49.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.070+0000 7f36eac00640 1 Processor -- start 2024-09-10T10:04:49.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.070+0000 7f36eac00640 1 -- start start 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36eac00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f36e4155970 0x7f36e40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36eac00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f36e406f1e0 0x7f36e40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36eac00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e4155d50 con 0x7f36e415c560 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36eac00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f36e40b4570 con 0x7f36e406f1e0 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36eac00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f36e40b34d0 con 0x7f36e4155970 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e8c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f36e406f1e0 0x7f36e40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e3400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f36e415c560 0x7f36e406ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35454/0 (socket says 172.21.15.110:35454) 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e8c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f36e406f1e0 0x7f36e40ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34422/0 (socket says 172.21.15.110:34422) 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e3400640 1 -- 172.21.15.110:0/2535527692 learned_addr learned my addr 172.21.15.110:0/2535527692 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e3e00640 1 --2- 172.21.15.110:0/2535527692 >> v2:172.21.15.145:3300/0 conn(0x7f36e4155970 0x7f36e40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e8c00640 1 -- 172.21.15.110:0/2535527692 >> v2:172.21.15.145:3300/0 conn(0x7f36e4155970 msgr2=0x7f36e40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e8c00640 1 --2- 172.21.15.110:0/2535527692 >> v2:172.21.15.145:3300/0 conn(0x7f36e4155970 0x7f36e40cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e8c00640 1 -- 172.21.15.110:0/2535527692 >> v1:172.21.15.110:6789/0 conn(0x7f36e415c560 legacy=0x7f36e406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.074+0000 7f36e8c00640 1 -- 172.21.15.110:0/2535527692 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e40b37b0 con 0x7f36e406f1e0 2024-09-10T10:04:49.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36e8c00640 1 --2- 172.21.15.110:0/2535527692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e406f1e0 0x7f36e40ce420 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f36e406cc30 tx=0x7f36dc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f883e68d56669a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36e2a00640 1 -- 172.21.15.110:0/2535527692 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0ab650 con 0x7f36e406f1e0 2024-09-10T10:04:49.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36e2a00640 1 -- 172.21.15.110:0/2535527692 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36dc0b0070 con 0x7f36e406f1e0 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36e2a00640 1 -- 172.21.15.110:0/2535527692 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0aba10 con 0x7f36e406f1e0 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- 172.21.15.110:0/2535527692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e406f1e0 msgr2=0x7f36e40ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 --2- 172.21.15.110:0/2535527692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e406f1e0 0x7f36e40ce420 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f36e406cc30 tx=0x7f36dc01c450 comp rx=0 tx=0).stop 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- 172.21.15.110:0/2535527692 shutdown_connections 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 --2- 172.21.15.110:0/2535527692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e406f1e0 0x7f36e40ce420 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 --2- 172.21.15.110:0/2535527692 >> v2:172.21.15.145:3300/0 conn(0x7f36e4155970 0x7f36e40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- 172.21.15.110:0/2535527692 >> 172.21.15.110:0/2535527692 conn(0x7f36e406bc50 msgr2=0x7f36e406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- 172.21.15.110:0/2535527692 shutdown_connections 2024-09-10T10:04:49.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- 172.21.15.110:0/2535527692 wait complete. 2024-09-10T10:04:49.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 Processor -- start 2024-09-10T10:04:49.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- start start 2024-09-10T10:04:49.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36e406f1e0 0x7f36e40aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36e4155970 0x7f36e40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e415c560 0x7f36e41529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36e40cf170 con 0x7f36e415c560 2024-09-10T10:04:49.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36e40cce00 con 0x7f36e406f1e0 2024-09-10T10:04:49.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.078+0000 7f36eac00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f36e40cf2f0 con 0x7f36e4155970 2024-09-10T10:04:49.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e415c560 0x7f36e41529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e415c560 0x7f36e41529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34436/0 (socket says 172.21.15.110:34436) 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 -- 172.21.15.110:0/293943984 learned_addr learned my addr 172.21.15.110:0/293943984 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e3400640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36e4155970 0x7f36e40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e3e00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36e406f1e0 0x7f36e40aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 -- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36e4155970 msgr2=0x7f36e40723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36e4155970 0x7f36e40723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 -- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36e406f1e0 msgr2=0x7f36e40aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36e406f1e0 0x7f36e40aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 -- 172.21.15.110:0/293943984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e40ae190 con 0x7f36e415c560 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36e8c00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e415c560 0x7f36e41529d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f36dc019060 tx=0x7f36dc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36dbe00640 1 -- 172.21.15.110:0/293943984 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0bf610 con 0x7f36e415c560 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36dbe00640 1 -- 172.21.15.110:0/293943984 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36dc0bf7b0 con 0x7f36e415c560 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36dbe00640 1 -- 172.21.15.110:0/293943984 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0bfb80 con 0x7f36e415c560 2024-09-10T10:04:49.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e40ae470 con 0x7f36e415c560 2024-09-10T10:04:49.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e40addb0 con 0x7f36e415c560 2024-09-10T10:04:49.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e4069bb0 con 0x7f36e415c560 2024-09-10T10:04:49.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36dbe00640 1 -- 172.21.15.110:0/293943984 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f36dc0bd020 con 0x7f36e415c560 2024-09-10T10:04:49.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.082+0000 7f36dbe00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f36c4067ef0 0x7f36c406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.086+0000 7f36dbe00640 1 -- 172.21.15.110:0/293943984 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f36dc120cc0 con 0x7f36e415c560 2024-09-10T10:04:49.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.086+0000 7f36e3e00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f36c4067ef0 0x7f36c406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.086+0000 7f36e3e00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f36c4067ef0 0x7f36c406a3b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f36d4003b90 tx=0x7f36d40098d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.090+0000 7f36dbe00640 1 -- 172.21.15.110:0/293943984 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f36dc0c6070 con 0x7f36e415c560 2024-09-10T10:04:49.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.274+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36e40cfe40 con 0x7f36e415c560 2024-09-10T10:04:49.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.274+0000 7f36dbe00640 1 -- 172.21.15.110:0/293943984 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5239 (secure 0 0 0) 0x7f36dc016070 con 0x7f36e415c560 2024-09-10T10:04:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:49.281 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":89,"btime":"2024-09-10T10:04:38:484563+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3129750368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3129750368},{"type":"v1","addr":"172.21.15.145:6834","nonce":3129750368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3791950507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3791950507},{"type":"v1","addr":"172.21.15.145:6835","nonce":3791950507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2024540113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2024540113},{"type":"v1","addr":"172.21.15.110:6834","nonce":2024540113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:04:28.820193+0000","modified":"2024-09-10T10:04:31.842870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1881265533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1881265533},{"type":"v1","addr":"172.21.15.110:6835","nonce":1881265533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7686,"qdb_cluster":[7686]},"id":10}]} 2024-09-10T10:04:49.281 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 89 2024-09-10T10:04:49.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f36c4067ef0 msgr2=0x7f36c406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f36c4067ef0 0x7f36c406a3b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f36d4003b90 tx=0x7f36d40098d0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e415c560 msgr2=0x7f36e41529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e415c560 0x7f36e41529d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f36dc019060 tx=0x7f36dc01ac90 comp rx=0 tx=0).stop 2024-09-10T10:04:49.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 shutdown_connections 2024-09-10T10:04:49.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f36c4067ef0 0x7f36c406a3b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e415c560 0x7f36e41529d0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f36e4155970 0x7f36e40723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 --2- 172.21.15.110:0/293943984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36e406f1e0 0x7f36e40aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 >> 172.21.15.110:0/293943984 conn(0x7f36e406bc50 msgr2=0x7f36e40b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 shutdown_connections 2024-09-10T10:04:49.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.278+0000 7f36eac00640 1 -- 172.21.15.110:0/293943984 wait complete. 2024-09-10T10:04:49.301 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:49.321 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:04:49.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:04:49.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d5be00640 1 Processor -- start 2024-09-10T10:04:49.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d5be00640 1 -- start start 2024-09-10T10:04:49.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d5be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3d5406c360 0x7f3d540d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d5be00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d540d0ad0 con 0x7f3d540d06a0 2024-09-10T10:04:49.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d5be00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d540b3dd0 con 0x7f3d5415c6d0 2024-09-10T10:04:49.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d5be00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3d5406e6f0 con 0x7f3d5406c360 2024-09-10T10:04:49.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d59e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3d540d06a0 0x7f3d5406f140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35466/0 (socket says 172.21.15.110:35466) 2024-09-10T10:04:49.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d59e00640 1 -- 172.21.15.110:0/1196975263 learned_addr learned my addr 172.21.15.110:0/1196975263 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:49.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d58a00640 1 --2- 172.21.15.110:0/1196975263 >> v2:172.21.15.145:3300/0 conn(0x7f3d5406c360 0x7f3d540d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d53e00640 1 -- 172.21.15.110:0/1196975263 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2988704277 0 0) 0x7f3d540d0ad0 con 0x7f3d540d06a0 2024-09-10T10:04:49.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d53e00640 1 -- 172.21.15.110:0/1196975263 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d3c003530 con 0x7f3d540d06a0 2024-09-10T10:04:49.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d58a00640 1 -- 172.21.15.110:0/1196975263 >> v1:172.21.15.145:6789/0 conn(0x7f3d5415c6d0 legacy=0x7f3d5406bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d58a00640 1 -- 172.21.15.110:0/1196975263 >> v1:172.21.15.110:6789/0 conn(0x7f3d540d06a0 legacy=0x7f3d5406f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.434+0000 7f3d58a00640 1 -- 172.21.15.110:0/1196975263 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d540d0d40 con 0x7f3d5406c360 2024-09-10T10:04:49.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d58a00640 1 --2- 172.21.15.110:0/1196975263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 0x7f3d540d0160 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3d400097c0 tx=0x7f3d4001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79fd5bafe9295640 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d53e00640 1 -- 172.21.15.110:0/1196975263 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400b0070 con 0x7f3d5406c360 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d53e00640 1 -- 172.21.15.110:0/1196975263 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d400ab600 con 0x7f3d5406c360 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d53e00640 1 -- 172.21.15.110:0/1196975263 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400ab9b0 con 0x7f3d5406c360 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d5be00640 1 -- 172.21.15.110:0/1196975263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 msgr2=0x7f3d540d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d5be00640 1 --2- 172.21.15.110:0/1196975263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 0x7f3d540d0160 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3d400097c0 tx=0x7f3d4001c380 comp rx=0 tx=0).stop 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d5be00640 1 -- 172.21.15.110:0/1196975263 shutdown_connections 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d5be00640 1 --2- 172.21.15.110:0/1196975263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 0x7f3d540d0160 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d5be00640 1 -- 172.21.15.110:0/1196975263 >> 172.21.15.110:0/1196975263 conn(0x7f3d5406d360 msgr2=0x7f3d540b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d5be00640 1 -- 172.21.15.110:0/1196975263 shutdown_connections 2024-09-10T10:04:49.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.438+0000 7f3d5be00640 1 -- 172.21.15.110:0/1196975263 wait complete. 2024-09-10T10:04:49.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 Processor -- start 2024-09-10T10:04:49.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 -- start start 2024-09-10T10:04:49.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 0x7f3d5414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d540d06a0 0x7f3d5414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d5415c6d0 0x7f3d5414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d540701e0 con 0x7f3d540d06a0 2024-09-10T10:04:49.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54070060 con 0x7f3d5406c360 2024-09-10T10:04:49.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3d54070360 con 0x7f3d5415c6d0 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d540d06a0 0x7f3d5414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d540d06a0 0x7f3d5414e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34440/0 (socket says 172.21.15.110:34440) 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 -- 172.21.15.110:0/2033540517 learned_addr learned my addr 172.21.15.110:0/2033540517 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d59400640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 0x7f3d5414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d59e00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d5415c6d0 0x7f3d5414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 -- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d5415c6d0 msgr2=0x7f3d5414e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d5415c6d0 0x7f3d5414e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 -- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 msgr2=0x7f3d5414abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 0x7f3d5414abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 -- 172.21.15.110:0/2033540517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5414c1e0 con 0x7f3d540d06a0 2024-09-10T10:04:49.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d59400640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 0x7f3d5414abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d59e00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d5415c6d0 0x7f3d5414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d58a00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d540d06a0 0x7f3d5414e2d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3d400097c0 tx=0x7f3d4001abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d51600640 1 -- 172.21.15.110:0/2033540517 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400b0070 con 0x7f3d540d06a0 2024-09-10T10:04:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d51600640 1 -- 172.21.15.110:0/2033540517 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d400ba070 con 0x7f3d540d06a0 2024-09-10T10:04:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5414c4c0 con 0x7f3d540d06a0 2024-09-10T10:04:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d51600640 1 -- 172.21.15.110:0/2033540517 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400abbc0 con 0x7f3d540d06a0 2024-09-10T10:04:49.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5414ca80 con 0x7f3d540d06a0 2024-09-10T10:04:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.442+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d540b0f30 con 0x7f3d540d06a0 2024-09-10T10:04:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.446+0000 7f3d51600640 1 -- 172.21.15.110:0/2033540517 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f3d40003620 con 0x7f3d540d06a0 2024-09-10T10:04:49.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.446+0000 7f3d51600640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d34067ef0 0x7f3d3406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.446+0000 7f3d51600640 1 -- 172.21.15.110:0/2033540517 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f3d401208f0 con 0x7f3d540d06a0 2024-09-10T10:04:49.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.446+0000 7f3d59400640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d34067ef0 0x7f3d3406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.446+0000 7f3d59400640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d34067ef0 0x7f3d3406a3b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3d5414b850 tx=0x7f3d48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.450+0000 7f3d51600640 1 -- 172.21.15.110:0/2033540517 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3d40016090 con 0x7f3d540d06a0 2024-09-10T10:04:49.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.610+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d540cd890 con 0x7f3d540d06a0 2024-09-10T10:04:49.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.610+0000 7f3d51600640 1 -- 172.21.15.110:0/2033540517 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16864 (secure 0 0 0) 0x7f3d400f0b60 con 0x7f3d540d06a0 2024-09-10T10:04:49.616 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:49.616 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":130,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:30.451627+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:04:26.532408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T10:04:27.823557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:49.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d34067ef0 msgr2=0x7f3d3406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d34067ef0 0x7f3d3406a3b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3d5414b850 tx=0x7f3d48009040 comp rx=0 tx=0).stop 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d540d06a0 msgr2=0x7f3d5414e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d540d06a0 0x7f3d5414e2d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3d400097c0 tx=0x7f3d4001abc0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 shutdown_connections 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3d34067ef0 0x7f3d3406a3b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3d5415c6d0 0x7f3d5414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d540d06a0 0x7f3d5414e2d0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 --2- 172.21.15.110:0/2033540517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d5406c360 0x7f3d5414abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 >> 172.21.15.110:0/2033540517 conn(0x7f3d5406d360 msgr2=0x7f3d5406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 shutdown_connections 2024-09-10T10:04:49.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.614+0000 7f3d5be00640 1 -- 172.21.15.110:0/2033540517 wait complete. 2024-09-10T10:04:49.636 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:04:49.636 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.810+0000 7fc6fe800640 1 Processor -- start 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.810+0000 7fc6fe800640 1 -- start start 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6fe800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc6f8155970 0x7fc6f80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6fe800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc6f815c560 0x7fc6f806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6fe800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f806f430 con 0x7fc6f806f010 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6fe800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f80b4570 con 0x7fc6f815c560 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6fe800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f8155d50 con 0x7fc6f8155970 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc6f815c560 0x7fc6f806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f7e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc6f806f010 0x7fc6f815eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35474/0 (socket says 172.21.15.110:35474) 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc6f815c560 0x7fc6f806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34452/0 (socket says 172.21.15.110:34452) 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f7e00640 1 -- 172.21.15.110:0/1451541920 learned_addr learned my addr 172.21.15.110:0/1451541920 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f7400640 1 --2- 172.21.15.110:0/1451541920 >> v2:172.21.15.145:3300/0 conn(0x7fc6f8155970 0x7fc6f80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f6a00640 1 -- 172.21.15.110:0/1451541920 >> v2:172.21.15.145:3300/0 conn(0x7fc6f8155970 msgr2=0x7fc6f80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f6a00640 1 --2- 172.21.15.110:0/1451541920 >> v2:172.21.15.145:3300/0 conn(0x7fc6f8155970 0x7fc6f80cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f6a00640 1 -- 172.21.15.110:0/1451541920 >> v1:172.21.15.110:6789/0 conn(0x7fc6f806f010 legacy=0x7fc6f815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.814+0000 7fc6f6a00640 1 -- 172.21.15.110:0/1451541920 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f80b34d0 con 0x7fc6f815c560 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6f6a00640 1 --2- 172.21.15.110:0/1451541920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f815c560 0x7fc6f806ead0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc6f806cc30 tx=0x7fc6e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd602de3955c395 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6f6000640 1 -- 172.21.15.110:0/1451541920 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40ab650 con 0x7fc6f815c560 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6f6000640 1 -- 172.21.15.110:0/1451541920 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e40b0070 con 0x7fc6f815c560 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6f6000640 1 -- 172.21.15.110:0/1451541920 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40aba10 con 0x7fc6f815c560 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- 172.21.15.110:0/1451541920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f815c560 msgr2=0x7fc6f806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 --2- 172.21.15.110:0/1451541920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f815c560 0x7fc6f806ead0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc6f806cc30 tx=0x7fc6e401c450 comp rx=0 tx=0).stop 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- 172.21.15.110:0/1451541920 shutdown_connections 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 --2- 172.21.15.110:0/1451541920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f815c560 0x7fc6f806ead0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 --2- 172.21.15.110:0/1451541920 >> v2:172.21.15.145:3300/0 conn(0x7fc6f8155970 0x7fc6f80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- 172.21.15.110:0/1451541920 >> 172.21.15.110:0/1451541920 conn(0x7fc6f806bc50 msgr2=0x7fc6f806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- 172.21.15.110:0/1451541920 shutdown_connections 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- 172.21.15.110:0/1451541920 wait complete. 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 Processor -- start 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- start start 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f806f010 0x7fc6f815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f8155970 0x7fc6f8072710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc6f815c560 0x7fc6f8072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f80cce00 con 0x7fc6f8155970 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f80b4050 con 0x7fc6f806f010 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6fe800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f8168030 con 0x7fc6f815c560 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6f6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f8155970 0x7fc6f8072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6f6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f8155970 0x7fc6f8072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34464/0 (socket says 172.21.15.110:34464) 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.818+0000 7fc6f6a00640 1 -- 172.21.15.110:0/3977053590 learned_addr learned my addr 172.21.15.110:0/3977053590 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6f7e00640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc6f815c560 0x7fc6f8072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6f7400640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f806f010 0x7fc6f815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6f6a00640 1 -- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc6f815c560 msgr2=0x7fc6f8072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6f6a00640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc6f815c560 0x7fc6f8072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6f6a00640 1 -- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f806f010 msgr2=0x7fc6f815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6f6a00640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f806f010 0x7fc6f815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6f6a00640 1 -- 172.21.15.110:0/3977053590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f8154680 con 0x7fc6f8155970 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6f6a00640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f8155970 0x7fc6f8072710 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40ad040 tx=0x7fc6e401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6ef400640 1 -- 172.21.15.110:0/3977053590 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40bf610 con 0x7fc6f8155970 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6ef400640 1 -- 172.21.15.110:0/3977053590 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e40bf7d0 con 0x7fc6f8155970 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6ef400640 1 -- 172.21.15.110:0/3977053590 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40bfbc0 con 0x7fc6f8155970 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f8152ee0 con 0x7fc6f8155970 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f81534a0 con 0x7fc6f8155970 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f80cc8c0 con 0x7fc6f8155970 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6ef400640 1 -- 172.21.15.110:0/3977053590 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fc6e40bd020 con 0x7fc6f8155970 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6ef400640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc6d8067ef0 0x7fc6d806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.822+0000 7fc6ef400640 1 -- 172.21.15.110:0/3977053590 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7fc6e4120d10 con 0x7fc6f8155970 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.826+0000 7fc6f7400640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc6d8067ef0 0x7fc6d806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.826+0000 7fc6f7400640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc6d8067ef0 0x7fc6d806a3b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8007a60 tx=0x7fc6e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:49.826+0000 7fc6ef400640 1 -- 172.21.15.110:0/3977053590 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc6e40c6070 con 0x7fc6f8155970 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.014+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6f80750c0 con 0x7fc6f8155970 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6ef400640 1 -- 172.21.15.110:0/3977053590 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5239 (secure 0 0 0) 0x7fc6e4016070 con 0x7fc6f8155970 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":89,"btime":"2024-09-10T10:04:38:484563+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3129750368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3129750368},{"type":"v1","addr":"172.21.15.145:6834","nonce":3129750368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3791950507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3791950507},{"type":"v1","addr":"172.21.15.145:6835","nonce":3791950507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2024540113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2024540113},{"type":"v1","addr":"172.21.15.110:6834","nonce":2024540113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:04:28.820193+0000","modified":"2024-09-10T10:04:31.842870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1881265533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1881265533},{"type":"v1","addr":"172.21.15.110:6835","nonce":1881265533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7686,"qdb_cluster":[7686]},"id":10}]} 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 89 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc6d8067ef0 msgr2=0x7fc6d806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc6d8067ef0 0x7fc6d806a3b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8007a60 tx=0x7fc6e8008040 comp rx=0 tx=0).stop 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f8155970 msgr2=0x7fc6f8072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f8155970 0x7fc6f8072710 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40ad040 tx=0x7fc6e401ac90 comp rx=0 tx=0).stop 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 shutdown_connections 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc6d8067ef0 0x7fc6d806a3b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc6f815c560 0x7fc6f8072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6f8155970 0x7fc6f8072710 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 --2- 172.21.15.110:0/3977053590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6f806f010 0x7fc6f815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 >> 172.21.15.110:0/3977053590 conn(0x7fc6f806bc50 msgr2=0x7fc6f8154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 shutdown_connections 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.018+0000 7fc6fe800640 1 -- 172.21.15.110:0/3977053590 wait complete. 2024-09-10T10:04:50.042 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e5e400640 1 Processor -- start 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e5e400640 1 -- start start 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e5e400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e5815c960 0x7f9e580b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e5e400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e580d13e0 0x7f9e58070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e5e400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e580706a0 con 0x7f9e581603c0 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e5e400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5815cd40 con 0x7f9e5815c960 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e5e400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e580b6370 con 0x7f9e580d13e0 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e580d13e0 0x7f9e58070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e5815c960 0x7f9e580b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e5815c960 0x7f9e580b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:53148/0 (socket says 172.21.15.110:53148) 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57400640 1 -- 172.21.15.110:0/453633907 learned_addr learned my addr 172.21.15.110:0/453633907 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e580d13e0 0x7f9e58070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:47570/0 (socket says 172.21.15.110:47570) 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e56000640 1 -- 172.21.15.110:0/453633907 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242122729 0 0) 0x7f9e580706a0 con 0x7f9e581603c0 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57400640 1 -- 172.21.15.110:0/453633907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e580d13e0 msgr2=0x7f9e58070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57400640 1 --2- 172.21.15.110:0/453633907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e580d13e0 0x7f9e58070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57400640 1 -- 172.21.15.110:0/453633907 >> v1:172.21.15.145:6789/0 conn(0x7f9e581603c0 legacy=0x7f9e580d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57400640 1 -- 172.21.15.110:0/453633907 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58070a80 con 0x7f9e5815c960 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.174+0000 7f9e57e00640 1 --2- 172.21.15.110:0/453633907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e580d13e0 0x7f9e58070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e57400640 1 --2- 172.21.15.110:0/453633907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e5815c960 0x7f9e580b5e30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9e580b64f0 tx=0x7f9e4801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=890df2cfc14ab69b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e56000640 1 -- 172.21.15.110:0/453633907 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e480ab5f0 con 0x7f9e5815c960 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e56000640 1 -- 172.21.15.110:0/453633907 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e480b0070 con 0x7f9e5815c960 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e56000640 1 -- 172.21.15.110:0/453633907 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e480ab9b0 con 0x7f9e5815c960 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e5e400640 1 -- 172.21.15.110:0/453633907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e5815c960 msgr2=0x7f9e580b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e5e400640 1 --2- 172.21.15.110:0/453633907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e5815c960 0x7f9e580b5e30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9e580b64f0 tx=0x7f9e4801c3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e5e400640 1 -- 172.21.15.110:0/453633907 shutdown_connections 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e5e400640 1 --2- 172.21.15.110:0/453633907 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e580d13e0 0x7f9e58070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e5e400640 1 --2- 172.21.15.110:0/453633907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e5815c960 0x7f9e580b5e30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e5e400640 1 -- 172.21.15.110:0/453633907 >> 172.21.15.110:0/453633907 conn(0x7f9e580d25c0 msgr2=0x7f9e580d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e5e400640 1 -- 172.21.15.110:0/453633907 shutdown_connections 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.178+0000 7f9e5e400640 1 -- 172.21.15.110:0/453633907 wait complete. 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 Processor -- start 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 -- start start 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e580d13e0 0x7f9e58153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e5815c960 0x7f9e58166550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e581603c0 0x7f9e58166a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e580d3150 con 0x7f9e580d13e0 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581611b0 con 0x7f9e5815c960 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e580d32d0 con 0x7f9e581603c0 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e580d13e0 0x7f9e58153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e580d13e0 0x7f9e58153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34474/0 (socket says 172.21.15.110:34474) 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 -- 172.21.15.110:0/1362487225 learned_addr learned my addr 172.21.15.110:0/1362487225 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 -- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e581603c0 msgr2=0x7f9e58166a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e581603c0 0x7f9e58166a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 -- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e5815c960 msgr2=0x7f9e58166550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e5815c960 0x7f9e58166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 -- 172.21.15.110:0/1362487225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58150260 con 0x7f9e580d13e0 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e57400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e580d13e0 0x7f9e58153df0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f9e4801cc90 tx=0x7f9e4801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e4f400640 1 -- 172.21.15.110:0/1362487225 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e480ba070 con 0x7f9e580d13e0 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e4f400640 1 -- 172.21.15.110:0/1362487225 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e48004220 con 0x7f9e580d13e0 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e58156cc0 con 0x7f9e580d13e0 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e4f400640 1 -- 172.21.15.110:0/1362487225 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e480ab7c0 con 0x7f9e580d13e0 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581507e0 con 0x7f9e580d13e0 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.182+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e58070f60 con 0x7f9e580d13e0 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.186+0000 7f9e4f400640 1 -- 172.21.15.110:0/1362487225 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f9e480ab960 con 0x7f9e580d13e0 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.186+0000 7f9e4f400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e38067f20 0x7f9e3806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.186+0000 7f9e4f400640 1 -- 172.21.15.110:0/1362487225 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f9e48120560 con 0x7f9e580d13e0 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.186+0000 7f9e56a00640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e38067f20 0x7f9e3806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.186+0000 7f9e56a00640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e38067f20 0x7f9e3806a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e44007fd0 tx=0x7f9e44007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.190+0000 7f9e4f400640 1 -- 172.21.15.110:0/1362487225 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9e48016130 con 0x7f9e580d13e0 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e580d34d0 con 0x7f9e580d13e0 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e4f400640 1 -- 172.21.15.110:0/1362487225 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5239 (secure 0 0 0) 0x7f9e480b3e20 con 0x7f9e580d13e0 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":89,"btime":"2024-09-10T10:04:38:484563+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3129750368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3129750368},{"type":"v1","addr":"172.21.15.145:6834","nonce":3129750368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3791950507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3791950507},{"type":"v1","addr":"172.21.15.145:6835","nonce":3791950507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2024540113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2024540113},{"type":"v1","addr":"172.21.15.110:6834","nonce":2024540113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:04:28.820193+0000","modified":"2024-09-10T10:04:31.842870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7686":{"gid":7686,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6835/1881265533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1881265533},{"type":"v1","addr":"172.21.15.110:6835","nonce":1881265533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7686,"qdb_cluster":[7686]},"id":10}]} 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 89 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e38067f20 msgr2=0x7f9e3806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e38067f20 0x7f9e3806a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e44007fd0 tx=0x7f9e44007940 comp rx=0 tx=0).stop 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e580d13e0 msgr2=0x7f9e58153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e580d13e0 0x7f9e58153df0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f9e4801cc90 tx=0x7f9e4801ac30 comp rx=0 tx=0).stop 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 shutdown_connections 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9e38067f20 0x7f9e3806a3e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e581603c0 0x7f9e58166a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e5815c960 0x7f9e58166550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 --2- 172.21.15.110:0/1362487225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e580d13e0 0x7f9e58153df0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 >> 172.21.15.110:0/1362487225 conn(0x7f9e580d25c0 msgr2=0x7f9e58073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 shutdown_connections 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.378+0000 7f9e5e400640 1 -- 172.21.15.110:0/1362487225 wait complete. 2024-09-10T10:04:50.404 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.534+0000 7f3b04e00640 1 Processor -- start 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3b04e00640 1 -- start start 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3b04e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3b000cfe30 0x7f3b00071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3b04e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0006d2d0 con 0x7f3b000cd510 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3b04e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0006d5b0 con 0x7f3b0006e7a0 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3b04e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0015d9a0 con 0x7f3b000cfe30 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3af7e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3b000cd510 0x7f3b0006e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35482/0 (socket says 172.21.15.110:35482) 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3af7e00640 1 -- 172.21.15.110:0/3493740895 learned_addr learned my addr 172.21.15.110:0/3493740895 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3afde00640 1 --2- 172.21.15.110:0/3493740895 >> v2:172.21.15.145:3300/0 conn(0x7f3b000cfe30 0x7f3b00071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3afde00640 1 -- 172.21.15.110:0/3493740895 >> v1:172.21.15.145:6789/0 conn(0x7f3b0006e7a0 legacy=0x7f3b0015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3afde00640 1 -- 172.21.15.110:0/3493740895 >> v1:172.21.15.110:6789/0 conn(0x7f3b000cd510 legacy=0x7f3b0006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.538+0000 7f3afde00640 1 -- 172.21.15.110:0/3493740895 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0006d890 con 0x7f3b000cfe30 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3afde00640 1 --2- 172.21.15.110:0/3493740895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 0x7f3b00071a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3af00097c0 tx=0x7f3af001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cc9338e478c3877 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3afd400640 1 -- 172.21.15.110:0/3493740895 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00b0070 con 0x7f3b000cfe30 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3afd400640 1 -- 172.21.15.110:0/3493740895 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af00ab600 con 0x7f3b000cfe30 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3afd400640 1 -- 172.21.15.110:0/3493740895 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00ab9d0 con 0x7f3b000cfe30 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3b04e00640 1 -- 172.21.15.110:0/3493740895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 msgr2=0x7f3b00071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3b04e00640 1 --2- 172.21.15.110:0/3493740895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 0x7f3b00071a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3af00097c0 tx=0x7f3af001c410 comp rx=0 tx=0).stop 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3b04e00640 1 -- 172.21.15.110:0/3493740895 shutdown_connections 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3b04e00640 1 --2- 172.21.15.110:0/3493740895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 0x7f3b00071a40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3b04e00640 1 -- 172.21.15.110:0/3493740895 >> 172.21.15.110:0/3493740895 conn(0x7f3b0006bc50 msgr2=0x7f3b0006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.542+0000 7f3b04e00640 1 -- 172.21.15.110:0/3493740895 shutdown_connections 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 -- 172.21.15.110:0/3493740895 wait complete. 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 Processor -- start 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 -- start start 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b0006e7a0 0x7f3b000b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b000cd510 0x7f3b0014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 0x7f3b00155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0015d740 con 0x7f3b000cd510 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00072190 con 0x7f3b000cfe30 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3b04e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0016abb0 con 0x7f3b0006e7a0 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b000cd510 0x7f3b0014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b000cd510 0x7f3b0014edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34488/0 (socket says 172.21.15.110:34488) 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 -- 172.21.15.110:0/260335380 learned_addr learned my addr 172.21.15.110:0/260335380 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3afde00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b0006e7a0 0x7f3b000b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3afe800640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 0x7f3b00155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 -- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b0006e7a0 msgr2=0x7f3b000b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b0006e7a0 0x7f3b000b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 -- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 msgr2=0x7f3b00155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 0x7f3b00155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 -- 172.21.15.110:0/260335380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b000afb90 con 0x7f3b000cd510 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3afe800640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 0x7f3b00155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.546+0000 7f3af7e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b000cd510 0x7f3b0014edc0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c500 tx=0x7f3ae800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.550+0000 7f3af6000640 1 -- 172.21.15.110:0/260335380 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8017070 con 0x7f3b000cd510 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.550+0000 7f3af6000640 1 -- 172.21.15.110:0/260335380 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae8005cf0 con 0x7f3b000cd510 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.550+0000 7f3af6000640 1 -- 172.21.15.110:0/260335380 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80072f0 con 0x7f3b000cd510 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.550+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b000afe70 con 0x7f3b000cd510 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.550+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b000b2570 con 0x7f3b000cd510 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.550+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b000b0100 con 0x7f3b000cd510 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.550+0000 7f3af6000640 1 -- 172.21.15.110:0/260335380 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f3ae801a070 con 0x7f3b000cd510 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.550+0000 7f3af6000640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3adc067ef0 0x7f3adc06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.554+0000 7f3af6000640 1 -- 172.21.15.110:0/260335380 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f3ae8085aa0 con 0x7f3b000cd510 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.554+0000 7f3afde00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3adc067ef0 0x7f3adc06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.554+0000 7f3afde00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3adc067ef0 0x7f3adc06a3b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3af00ad000 tx=0x7f3af0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.558+0000 7f3af6000640 1 -- 172.21.15.110:0/260335380 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3ae80556a0 con 0x7f3b000cd510 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.714+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b000719c0 con 0x7f3b000cd510 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3af6000640 1 -- 172.21.15.110:0/260335380 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16864 (secure 0 0 0) 0x7f3ae80599a0 con 0x7f3b000cd510 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":130,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:30.451627+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:04:26.532408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T10:04:27.823557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3adc067ef0 msgr2=0x7f3adc06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3adc067ef0 0x7f3adc06a3b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3af00ad000 tx=0x7f3af0008040 comp rx=0 tx=0).stop 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b000cd510 msgr2=0x7f3b0014edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b000cd510 0x7f3b0014edc0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c500 tx=0x7f3ae800c9c0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 shutdown_connections 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3adc067ef0 0x7f3adc06a3b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b000cfe30 0x7f3b00155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b000cd510 0x7f3b0014edc0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 --2- 172.21.15.110:0/260335380 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3b0006e7a0 0x7f3b000b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 >> 172.21.15.110:0/260335380 conn(0x7f3b0006bc50 msgr2=0x7f3b0015b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 shutdown_connections 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.718+0000 7f3b04e00640 1 -- 172.21.15.110:0/260335380 wait complete. 2024-09-10T10:04:50.744 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.874+0000 7f204b200640 1 Processor -- start 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.874+0000 7f204b200640 1 -- start start 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f204b200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f20440cfb90 0x7f204415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f204b200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440b3f40 0x7f20440cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f204b200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f204415d4d0 con 0x7f20440cff70 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f204b200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f204406d130 con 0x7f20440cfb90 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f204b200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f204415d7b0 con 0x7f20440b3f40 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2049200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f20440cff70 0x7f2044160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35498/0 (socket says 172.21.15.110:35498) 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2049200640 1 -- 172.21.15.110:0/3056236812 learned_addr learned my addr 172.21.15.110:0/3056236812 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2043400640 1 --2- 172.21.15.110:0/3056236812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440b3f40 0x7f20440cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2043e00640 1 --2- 172.21.15.110:0/3056236812 >> v2:172.21.15.145:3300/0 conn(0x7f20440cfb90 0x7f204415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2042a00640 1 -- 172.21.15.110:0/3056236812 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1856927964 0 0) 0x7f204415d4d0 con 0x7f20440cff70 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2042a00640 1 -- 172.21.15.110:0/3056236812 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2028003530 con 0x7f20440cff70 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2043400640 1 -- 172.21.15.110:0/3056236812 >> v2:172.21.15.145:3300/0 conn(0x7f20440cfb90 msgr2=0x7f204415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2043400640 1 --2- 172.21.15.110:0/3056236812 >> v2:172.21.15.145:3300/0 conn(0x7f20440cfb90 0x7f204415ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2043400640 1 -- 172.21.15.110:0/3056236812 >> v1:172.21.15.110:6789/0 conn(0x7f20440cff70 legacy=0x7f2044160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.878+0000 7f2043400640 1 -- 172.21.15.110:0/3056236812 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204406d380 con 0x7f20440b3f40 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f2043400640 1 --2- 172.21.15.110:0/3056236812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440b3f40 0x7f20440cd5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f20300097c0 tx=0x7f203001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4231ebc8ae8386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f2042a00640 1 -- 172.21.15.110:0/3056236812 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300b0070 con 0x7f20440b3f40 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f2042a00640 1 -- 172.21.15.110:0/3056236812 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20300ab600 con 0x7f20440b3f40 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f2042a00640 1 -- 172.21.15.110:0/3056236812 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300ab9d0 con 0x7f20440b3f40 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- 172.21.15.110:0/3056236812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440b3f40 msgr2=0x7f20440cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 --2- 172.21.15.110:0/3056236812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440b3f40 0x7f20440cd5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f20300097c0 tx=0x7f203001c410 comp rx=0 tx=0).stop 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- 172.21.15.110:0/3056236812 shutdown_connections 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 --2- 172.21.15.110:0/3056236812 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440b3f40 0x7f20440cd5a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 --2- 172.21.15.110:0/3056236812 >> v2:172.21.15.145:3300/0 conn(0x7f20440cfb90 0x7f204415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- 172.21.15.110:0/3056236812 >> 172.21.15.110:0/3056236812 conn(0x7f2044062d10 msgr2=0x7f204406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- 172.21.15.110:0/3056236812 shutdown_connections 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- 172.21.15.110:0/3056236812 wait complete. 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 Processor -- start 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- start start 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20440b3f40 0x7f2044165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20440cfb90 0x7f2044165a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440cff70 0x7f20441561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2044169500 con 0x7f20440b3f40 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2044169380 con 0x7f20440cfb90 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.882+0000 7f204b200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2044169680 con 0x7f20440cff70 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20440b3f40 0x7f2044165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20440b3f40 0x7f2044165520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34500/0 (socket says 172.21.15.110:34500) 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 -- 172.21.15.110:0/3332491493 learned_addr learned my addr 172.21.15.110:0/3332491493 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043400640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20440cfb90 0x7f2044165a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2049200640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440cff70 0x7f20441561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 -- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440cff70 msgr2=0x7f20441561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440cff70 0x7f20441561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 -- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20440cfb90 msgr2=0x7f2044165a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20440cfb90 0x7f2044165a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 -- 172.21.15.110:0/3332491493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2044152990 con 0x7f20440b3f40 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2049200640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440cff70 0x7f20441561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f2043e00640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20440b3f40 0x7f2044165520 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2034009db0 tx=0x7f203400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f203be00640 1 -- 172.21.15.110:0/3332491493 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034018020 con 0x7f20440b3f40 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f203be00640 1 -- 172.21.15.110:0/3332491493 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2034014480 con 0x7f20440b3f40 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2044152c70 con 0x7f20440b3f40 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f203be00640 1 -- 172.21.15.110:0/3332491493 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034014850 con 0x7f20440b3f40 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20441566e0 con 0x7f20440b3f40 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20440008e0 con 0x7f20440b3f40 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f203be00640 1 -- 172.21.15.110:0/3332491493 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f203401b020 con 0x7f20440b3f40 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f203be00640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2024067f20 0x7f202406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.886+0000 7f203be00640 1 -- 172.21.15.110:0/3332491493 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5765+0+0 (secure 0 0 0) 0x7f20340872c0 con 0x7f20440b3f40 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.890+0000 7f2043400640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2024067f20 0x7f202406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.890+0000 7f2043400640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2024067f20 0x7f202406a3e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20300ad000 tx=0x7f2030008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:50.894+0000 7f203be00640 1 -- 172.21.15.110:0/3332491493 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2034022070 con 0x7f20440b3f40 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.082+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f20440cd5a0 con 0x7f20440b3f40 2024-09-10T10:04:51.088 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:04:51.082+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.086+0000 7f203be00640 1 -- 172.21.15.110:0/3332491493 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7f2034010040 con 0x7f20440b3f40 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2024067f20 msgr2=0x7f202406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2024067f20 0x7f202406a3e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20300ad000 tx=0x7f2030008040 comp rx=0 tx=0).stop 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20440b3f40 msgr2=0x7f2044165520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20440b3f40 0x7f2044165520 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2034009db0 tx=0x7f203400c490 comp rx=0 tx=0).stop 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 shutdown_connections 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2024067f20 0x7f202406a3e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f20440cff70 0x7f20441561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20440cfb90 0x7f2044165a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 --2- 172.21.15.110:0/3332491493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20440b3f40 0x7f2044165520 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 >> 172.21.15.110:0/3332491493 conn(0x7f2044062d10 msgr2=0x7f20440b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 shutdown_connections 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.090+0000 7f204b200640 1 -- 172.21.15.110:0/3332491493 wait complete. 2024-09-10T10:04:51.099 INFO:tasks.ceph.mds.c.smithi110.stderr: -2857> 2024-09-10T10:04:31.418+0000 7f8adfc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:04:51.114 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:04:51.145 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:04:51.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac6400640 1 Processor -- start 2024-09-10T10:04:51.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac6400640 1 -- start start 2024-09-10T10:04:51.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac6400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feac006ea80 0x7feac015f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:51.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac6400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7feac00b45f0 0x7feac00b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac6400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac0162e80 con 0x7feac0073250 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac6400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7feac00b4f60 con 0x7feac006ea80 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac6400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7feac01628c0 con 0x7feac00b45f0 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac5000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feac006ea80 0x7feac015f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac5000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feac006ea80 0x7feac015f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34522/0 (socket says 172.21.15.110:34522) 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac5000640 1 -- 172.21.15.110:0/3077719371 learned_addr learned my addr 172.21.15.110:0/3077719371 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feabfe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7feac0073250 0x7feac0161fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35520/0 (socket says 172.21.15.110:35520) 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac5a00640 1 --2- 172.21.15.110:0/3077719371 >> v2:172.21.15.145:3300/0 conn(0x7feac00b45f0 0x7feac00b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac5000640 1 -- 172.21.15.110:0/3077719371 >> v2:172.21.15.145:3300/0 conn(0x7feac00b45f0 msgr2=0x7feac00b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac5000640 1 --2- 172.21.15.110:0/3077719371 >> v2:172.21.15.145:3300/0 conn(0x7feac00b45f0 0x7feac00b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac5000640 1 -- 172.21.15.110:0/3077719371 >> v1:172.21.15.110:6789/0 conn(0x7feac0073250 legacy=0x7feac0161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:51.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.250+0000 7feac5000640 1 -- 172.21.15.110:0/3077719371 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac0162ba0 con 0x7feac006ea80 2024-09-10T10:04:51.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac5000640 1 --2- 172.21.15.110:0/3077719371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac006ea80 0x7feac015f370 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7feac015c5a0 tx=0x7feab001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46f452c78927a39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:51.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feabf400640 1 -- 172.21.15.110:0/3077719371 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab00b2070 con 0x7feac006ea80 2024-09-10T10:04:51.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feabf400640 1 -- 172.21.15.110:0/3077719371 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab00ad680 con 0x7feac006ea80 2024-09-10T10:04:51.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feabf400640 1 -- 172.21.15.110:0/3077719371 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab00ada50 con 0x7feac006ea80 2024-09-10T10:04:51.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac6400640 1 -- 172.21.15.110:0/3077719371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac006ea80 msgr2=0x7feac015f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:51.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac6400640 1 --2- 172.21.15.110:0/3077719371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac006ea80 0x7feac015f370 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7feac015c5a0 tx=0x7feab001c490 comp rx=0 tx=0).stop 2024-09-10T10:04:51.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac6400640 1 -- 172.21.15.110:0/3077719371 shutdown_connections 2024-09-10T10:04:51.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac6400640 1 --2- 172.21.15.110:0/3077719371 >> v2:172.21.15.145:3300/0 conn(0x7feac00b45f0 0x7feac00b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac6400640 1 --2- 172.21.15.110:0/3077719371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac006ea80 0x7feac015f370 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac6400640 1 -- 172.21.15.110:0/3077719371 >> 172.21.15.110:0/3077719371 conn(0x7feac00b7be0 msgr2=0x7feac00b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:51.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac6400640 1 -- 172.21.15.110:0/3077719371 shutdown_connections 2024-09-10T10:04:51.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.254+0000 7feac6400640 1 -- 172.21.15.110:0/3077719371 wait complete. 2024-09-10T10:04:51.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 Processor -- start 2024-09-10T10:04:51.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 -- start start 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feac006ea80 0x7feac0154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac0073250 0x7feac01fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feac00b45f0 0x7feac01fb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feac00b7130 con 0x7feac0073250 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feac00b6fb0 con 0x7feac006ea80 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feac00b72b0 con 0x7feac00b45f0 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac0073250 0x7feac01fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac0073250 0x7feac01fae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34532/0 (socket says 172.21.15.110:34532) 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac5000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feac006ea80 0x7feac0154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 -- 172.21.15.110:0/3042378892 learned_addr learned my addr 172.21.15.110:0/3042378892 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac5a00640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feac00b45f0 0x7feac01fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:51.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 -- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feac00b45f0 msgr2=0x7feac01fb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:51.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feac00b45f0 0x7feac01fb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 -- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feac006ea80 msgr2=0x7feac0154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:51.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feac006ea80 0x7feac0154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 -- 172.21.15.110:0/3042378892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac0165ab0 con 0x7feac0073250 2024-09-10T10:04:51.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac5000640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feac006ea80 0x7feac0154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:51.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabfe00640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac0073250 0x7feac01fae10 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c370 tx=0x7feaac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:51.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabcc00640 1 -- 172.21.15.110:0/3042378892 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac0074e0 con 0x7feac0073250 2024-09-10T10:04:51.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabcc00640 1 -- 172.21.15.110:0/3042378892 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac0076a0 con 0x7feac0073250 2024-09-10T10:04:51.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac0165d90 con 0x7feac0073250 2024-09-10T10:04:51.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feabcc00640 1 -- 172.21.15.110:0/3042378892 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac004a10 con 0x7feac0073250 2024-09-10T10:04:51.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac0166350 con 0x7feac0073250 2024-09-10T10:04:51.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.258+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac015f370 con 0x7feac0073250 2024-09-10T10:04:51.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.262+0000 7feabcc00640 1 -- 172.21.15.110:0/3042378892 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7feaac01a020 con 0x7feac0073250 2024-09-10T10:04:51.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.262+0000 7feabcc00640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feaa0067ef0 0x7feaa006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:51.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.262+0000 7feabcc00640 1 -- 172.21.15.110:0/3042378892 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5851+0+0 (secure 0 0 0) 0x7feaac085e70 con 0x7feac0073250 2024-09-10T10:04:51.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.262+0000 7feac5000640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feaa0067ef0 0x7feaa006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:51.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.262+0000 7feac5000640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feaa0067ef0 0x7feaa006a3b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feab00af000 tx=0x7feab00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:51.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.266+0000 7feabcc00640 1 -- 172.21.15.110:0/3042378892 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feaac0559e0 con 0x7feac0073250 2024-09-10T10:04:51.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:51.462+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feac00b3f60 con 0x7feac0073250 2024-09-10T10:04:52.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.086+0000 7feabcc00640 1 -- 172.21.15.110:0/3042378892 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) ==== 104+0+0 (secure 0 0 0) 0x7feaac059ce0 con 0x7feac0073250 2024-09-10T10:04:52.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feaa0067ef0 msgr2=0x7feaa006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feaa0067ef0 0x7feaa006a3b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feab00af000 tx=0x7feab00a7000 comp rx=0 tx=0).stop 2024-09-10T10:04:52.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac0073250 msgr2=0x7feac01fae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac0073250 0x7feac01fae10 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c370 tx=0x7feaac00c830 comp rx=0 tx=0).stop 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 shutdown_connections 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feaa0067ef0 0x7feaa006a3b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feac00b45f0 0x7feac01fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feac0073250 0x7feac01fae10 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 --2- 172.21.15.110:0/3042378892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feac006ea80 0x7feac0154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 >> 172.21.15.110:0/3042378892 conn(0x7feac00b7be0 msgr2=0x7feac015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 shutdown_connections 2024-09-10T10:04:52.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.090+0000 7feac6400640 1 -- 172.21.15.110:0/3042378892 wait complete. 2024-09-10T10:04:52.116 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:04:52.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.290+0000 7f9977600640 1 Processor -- start 2024-09-10T10:04:52.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.290+0000 7f9977600640 1 -- start start 2024-09-10T10:04:52.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.290+0000 7f9977600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f997015c560 0x7f997006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:52.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.290+0000 7f9977600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997006f430 con 0x7f997006f010 2024-09-10T10:04:52.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f9977600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9970155d50 con 0x7f9970155970 2024-09-10T10:04:52.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f9977600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f99700b4570 con 0x7f997015c560 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f9975600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f997006f010 0x7f99700ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35524/0 (socket says 172.21.15.110:35524) 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f9975600640 1 -- 172.21.15.110:0/264474217 learned_addr learned my addr 172.21.15.110:0/264474217 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996fe00640 1 --2- 172.21.15.110:0/264474217 >> v2:172.21.15.145:3300/0 conn(0x7f997015c560 0x7f997006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996f400640 1 -- 172.21.15.110:0/264474217 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3008846501 0 0) 0x7f997006f430 con 0x7f997006f010 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996f400640 1 -- 172.21.15.110:0/264474217 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9954003530 con 0x7f997006f010 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996f400640 1 -- 172.21.15.110:0/264474217 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994726123 0 0) 0x7f9970155d50 con 0x7f9970155970 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996f400640 1 -- 172.21.15.110:0/264474217 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f997006f430 con 0x7f9970155970 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996f400640 1 -- 172.21.15.110:0/264474217 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1254104065 0 0) 0x7f9954003530 con 0x7f997006f010 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996fe00640 1 -- 172.21.15.110:0/264474217 >> v1:172.21.15.145:6789/0 conn(0x7f9970155970 legacy=0x7f99700cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996fe00640 1 -- 172.21.15.110:0/264474217 >> v1:172.21.15.110:6789/0 conn(0x7f997006f010 legacy=0x7f99700ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.294+0000 7f996fe00640 1 -- 172.21.15.110:0/264474217 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99700b34d0 con 0x7f997015c560 2024-09-10T10:04:52.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f996fe00640 1 --2- 172.21.15.110:0/264474217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f997015c560 0x7f997006ead0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f995c0097c0 tx=0x7f995c01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a1a5fe7aae8f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:52.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f996f400640 1 -- 172.21.15.110:0/264474217 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c0b1070 con 0x7f997015c560 2024-09-10T10:04:52.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f996f400640 1 -- 172.21.15.110:0/264474217 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c0ac440 con 0x7f997015c560 2024-09-10T10:04:52.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f996f400640 1 -- 172.21.15.110:0/264474217 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968001a00 con 0x7f997015c560 2024-09-10T10:04:52.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- 172.21.15.110:0/264474217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f997015c560 msgr2=0x7f997006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 --2- 172.21.15.110:0/264474217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f997015c560 0x7f997006ead0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f995c0097c0 tx=0x7f995c01c390 comp rx=0 tx=0).stop 2024-09-10T10:04:52.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- 172.21.15.110:0/264474217 shutdown_connections 2024-09-10T10:04:52.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 --2- 172.21.15.110:0/264474217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f997015c560 0x7f997006ead0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- 172.21.15.110:0/264474217 >> 172.21.15.110:0/264474217 conn(0x7f997006bc50 msgr2=0x7f997006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:52.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- 172.21.15.110:0/264474217 shutdown_connections 2024-09-10T10:04:52.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- 172.21.15.110:0/264474217 wait complete. 2024-09-10T10:04:52.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 Processor -- start 2024-09-10T10:04:52.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- start start 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f997006f010 0x7f9970154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9970155970 0x7f997014bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f997015c560 0x7f9970144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99700cf340 con 0x7f997006f010 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99700b4050 con 0x7f997015c560 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9977600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f99700cf4c0 con 0x7f9970155970 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9974c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f997006f010 0x7f9970154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9974c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f997006f010 0x7f9970154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34540/0 (socket says 172.21.15.110:34540) 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.298+0000 7f9974c00640 1 -- 172.21.15.110:0/2942245065 learned_addr learned my addr 172.21.15.110:0/2942245065 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:52.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f996fe00640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9970155970 0x7f997014bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9974c00640 1 -- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9970155970 msgr2=0x7f997014bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9974c00640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9970155970 0x7f997014bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9974c00640 1 -- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f997015c560 msgr2=0x7f9970144f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9974c00640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f997015c560 0x7f9970144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9974c00640 1 -- 172.21.15.110:0/2942245065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997014c210 con 0x7f997006f010 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9974c00640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f997006f010 0x7f9970154410 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f996000c4c0 tx=0x7f996000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f996cc00640 1 -- 172.21.15.110:0/2942245065 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600052d0 con 0x7f997006f010 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997014c4b0 con 0x7f997006f010 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f996cc00640 1 -- 172.21.15.110:0/2942245065 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960005470 con 0x7f997006f010 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f996cc00640 1 -- 172.21.15.110:0/2942245065 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960012870 con 0x7f997006f010 2024-09-10T10:04:52.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997014ca70 con 0x7f997006f010 2024-09-10T10:04:52.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99700ce420 con 0x7f997006f010 2024-09-10T10:04:52.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f996cc00640 1 -- 172.21.15.110:0/2942245065 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f9960012a10 con 0x7f997006f010 2024-09-10T10:04:52.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f996cc00640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9950067ef0 0x7f995006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:52.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f996cc00640 1 -- 172.21.15.110:0/2942245065 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5851+0+0 (secure 0 0 0) 0x7f9960086630 con 0x7f997006f010 2024-09-10T10:04:52.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.302+0000 7f996fe00640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9950067ef0 0x7f995006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:52.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.306+0000 7f996fe00640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9950067ef0 0x7f995006a3b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f995c0ae000 tx=0x7f995c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:52.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.306+0000 7f996cc00640 1 -- 172.21.15.110:0/2942245065 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f99600566e0 con 0x7f997006f010 2024-09-10T10:04:52.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:52.466+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f99701454a0 con 0x7f997006f010 2024-09-10T10:04:53.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.090+0000 7f996cc00640 1 -- 172.21.15.110:0/2942245065 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v132) ==== 181+0+0 (secure 0 0 0) 0x7f9960055fe0 con 0x7f997006f010 2024-09-10T10:04:53.094 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:04:53.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.090+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9950067ef0 msgr2=0x7f995006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:53.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.090+0000 7f9977600640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9950067ef0 0x7f995006a3b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f995c0ae000 tx=0x7f995c008040 comp rx=0 tx=0).stop 2024-09-10T10:04:53.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f997006f010 msgr2=0x7f9970154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:53.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f997006f010 0x7f9970154410 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f996000c4c0 tx=0x7f996000c980 comp rx=0 tx=0).stop 2024-09-10T10:04:53.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 shutdown_connections 2024-09-10T10:04:53.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9950067ef0 0x7f995006a3b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f997015c560 0x7f9970144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9970155970 0x7f997014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 --2- 172.21.15.110:0/2942245065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f997006f010 0x7f9970154410 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 >> 172.21.15.110:0/2942245065 conn(0x7f997006bc50 msgr2=0x7f997006e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:53.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 shutdown_connections 2024-09-10T10:04:53.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.094+0000 7f9977600640 1 -- 172.21.15.110:0/2942245065 wait complete. 2024-09-10T10:04:53.120 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:04:53.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.254+0000 7fea26000640 1 Processor -- start 2024-09-10T10:04:53.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.254+0000 7fea26000640 1 -- start start 2024-09-10T10:04:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea26000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea26000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fea200719b0 0x7fea200d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:53.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea26000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea200cf680 con 0x7fea200b41f0 2024-09-10T10:04:53.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea26000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fea20070a50 con 0x7fea200719b0 2024-09-10T10:04:53.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea26000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fea20160110 con 0x7fea200b1a60 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea1fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fea200719b0 0x7fea200d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea24c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea1fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fea200719b0 0x7fea200d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:53260/0 (socket says 172.21.15.110:53260) 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea1fe00640 1 -- 172.21.15.110:0/2138091104 learned_addr learned my addr 172.21.15.110:0/2138091104 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea24c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:47670/0 (socket says 172.21.15.110:47670) 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea25600640 1 --1- 172.21.15.110:0/2138091104 >> v1:172.21.15.145:6789/0 conn(0x7fea200b41f0 0x7fea2006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:49374/0 (socket says 172.21.15.110:49374) 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea1fe00640 1 -- 172.21.15.110:0/2138091104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 msgr2=0x7fea200cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea1fe00640 1 --2- 172.21.15.110:0/2138091104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea1fe00640 1 -- 172.21.15.110:0/2138091104 >> v1:172.21.15.145:6789/0 conn(0x7fea200b41f0 legacy=0x7fea2006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:53.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.258+0000 7fea1fe00640 1 -- 172.21.15.110:0/2138091104 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea20070f80 con 0x7fea200719b0 2024-09-10T10:04:53.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea1fe00640 1 --2- 172.21.15.110:0/2138091104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200719b0 0x7fea200d10a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009870 tx=0x7fea0c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4307ce705ee95b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:53.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea1f400640 1 -- 172.21.15.110:0/2138091104 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0aa600 con 0x7fea200719b0 2024-09-10T10:04:53.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea1f400640 1 -- 172.21.15.110:0/2138091104 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c0af070 con 0x7fea200719b0 2024-09-10T10:04:53.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea1f400640 1 -- 172.21.15.110:0/2138091104 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0aa9c0 con 0x7fea200719b0 2024-09-10T10:04:53.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 -- 172.21.15.110:0/2138091104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200719b0 msgr2=0x7fea200d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:53.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 --2- 172.21.15.110:0/2138091104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200719b0 0x7fea200d10a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009870 tx=0x7fea0c01c4a0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 -- 172.21.15.110:0/2138091104 shutdown_connections 2024-09-10T10:04:53.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 --2- 172.21.15.110:0/2138091104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200719b0 0x7fea200d10a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 --2- 172.21.15.110:0/2138091104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 -- 172.21.15.110:0/2138091104 >> 172.21.15.110:0/2138091104 conn(0x7fea2006f440 msgr2=0x7fea2006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:53.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 -- 172.21.15.110:0/2138091104 shutdown_connections 2024-09-10T10:04:53.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 -- 172.21.15.110:0/2138091104 wait complete. 2024-09-10T10:04:53.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 Processor -- start 2024-09-10T10:04:53.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.262+0000 7fea26000640 1 -- start start 2024-09-10T10:04:53.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea200719b0 0x7fea20156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200b41f0 0x7fea2015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea200b4a10 con 0x7fea200719b0 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fea2015fe30 con 0x7fea200b41f0 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fea20156a10 con 0x7fea200b1a60 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea200719b0 0x7fea20156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea200719b0 0x7fea20156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34556/0 (socket says 172.21.15.110:34556) 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 -- 172.21.15.110:0/2609865397 learned_addr learned my addr 172.21.15.110:0/2609865397 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea1fe00640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 -- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 msgr2=0x7fea200cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:53.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea25600640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200b41f0 0x7fea2015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 -- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200b41f0 msgr2=0x7fea2015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200b41f0 0x7fea2015cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 -- 172.21.15.110:0/2609865397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea200ce400 con 0x7fea200719b0 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea1fe00640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea25600640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200b41f0 0x7fea2015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea24c00640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea200719b0 0x7fea20156ea0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fea10009db0 tx=0x7fea1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea1cc00640 1 -- 172.21.15.110:0/2609865397 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10018020 con 0x7fea200719b0 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea201559b0 con 0x7fea200719b0 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea1cc00640 1 -- 172.21.15.110:0/2609865397 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea10014480 con 0x7fea200719b0 2024-09-10T10:04:53.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea1cc00640 1 -- 172.21.15.110:0/2609865397 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10014850 con 0x7fea200719b0 2024-09-10T10:04:53.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201566a0 con 0x7fea200719b0 2024-09-10T10:04:53.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.266+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2006def0 con 0x7fea200719b0 2024-09-10T10:04:53.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.270+0000 7fea1cc00640 1 -- 172.21.15.110:0/2609865397 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fea1001b020 con 0x7fea200719b0 2024-09-10T10:04:53.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.270+0000 7fea1cc00640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fea00067ef0 0x7fea0006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:53.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.270+0000 7fea1cc00640 1 -- 172.21.15.110:0/2609865397 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5413+0+0 (secure 0 0 0) 0x7fea100871f0 con 0x7fea200719b0 2024-09-10T10:04:53.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.270+0000 7fea1fe00640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fea00067ef0 0x7fea0006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:53.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.270+0000 7fea1fe00640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fea00067ef0 0x7fea0006a3b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0ae000 tx=0x7fea0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:53.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.274+0000 7fea1cc00640 1 -- 172.21.15.110:0/2609865397 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fea10022070 con 0x7fea200719b0 2024-09-10T10:04:53.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:53.430+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fea20158570 con 0x7fea200719b0 2024-09-10T10:04:54.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea1cc00640 1 -- 172.21.15.110:0/2609865397 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v133) ==== 169+0+0 (secure 0 0 0) 0x7fea10010040 con 0x7fea200719b0 2024-09-10T10:04:54.102 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:04:54.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fea00067ef0 msgr2=0x7fea0006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fea00067ef0 0x7fea0006a3b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0ae000 tx=0x7fea0c008040 comp rx=0 tx=0).stop 2024-09-10T10:04:54.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea200719b0 msgr2=0x7fea20156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea200719b0 0x7fea20156ea0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fea10009db0 tx=0x7fea1000c490 comp rx=0 tx=0).stop 2024-09-10T10:04:54.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 shutdown_connections 2024-09-10T10:04:54.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fea00067ef0 0x7fea0006a3b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea200b41f0 0x7fea2015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fea200b1a60 0x7fea200cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 --2- 172.21.15.110:0/2609865397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea200719b0 0x7fea20156ea0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.098+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 >> 172.21.15.110:0/2609865397 conn(0x7fea2006f440 msgr2=0x7fea2006e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:54.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.102+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 shutdown_connections 2024-09-10T10:04:54.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.102+0000 7fea26000640 1 -- 172.21.15.110:0/2609865397 wait complete. 2024-09-10T10:04:54.131 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d0c800640 1 Processor -- start 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d0c800640 1 -- start start 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d0c800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1d040b5000 0x7f1d0406e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d0c800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d040b29e0 con 0x7f1d0406e680 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d0c800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d040b35b0 con 0x7f1d04158530 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d0c800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1d0406ead0 con 0x7f1d040b5000 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d0a800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1d0406e680 0x7f1d040af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35532/0 (socket says 172.21.15.110:35532) 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d0a800640 1 -- 172.21.15.110:0/98142851 learned_addr learned my addr 172.21.15.110:0/98142851 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d09400640 1 --2- 172.21.15.110:0/98142851 >> v2:172.21.15.145:3300/0 conn(0x7f1d040b5000 0x7f1d0406e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d08a00640 1 -- 172.21.15.110:0/98142851 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292915145 0 0) 0x7f1d040b29e0 con 0x7f1d0406e680 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.270+0000 7f1d08a00640 1 -- 172.21.15.110:0/98142851 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cec003530 con 0x7f1d0406e680 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.274+0000 7f1d08a00640 1 -- 172.21.15.110:0/98142851 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2580256733 0 0) 0x7f1cec003530 con 0x7f1d0406e680 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.274+0000 7f1d09400640 1 -- 172.21.15.110:0/98142851 >> v1:172.21.15.145:6789/0 conn(0x7f1d04158530 legacy=0x7f1d040b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.274+0000 7f1d09400640 1 -- 172.21.15.110:0/98142851 >> v1:172.21.15.110:6789/0 conn(0x7f1d0406e680 legacy=0x7f1d040af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.274+0000 7f1d09400640 1 -- 172.21.15.110:0/98142851 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d040b3890 con 0x7f1d040b5000 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.274+0000 7f1d09400640 1 --2- 172.21.15.110:0/98142851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d040b5000 0x7f1d0406e140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40097c0 tx=0x7f1cf401d380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3caf6cc5cf5f84c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.274+0000 7f1d08a00640 1 -- 172.21.15.110:0/98142851 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40b1070 con 0x7f1d040b5000 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.274+0000 7f1d08a00640 1 -- 172.21.15.110:0/98142851 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf40ac400 con 0x7f1d040b5000 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.274+0000 7f1d08a00640 1 -- 172.21.15.110:0/98142851 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00001a20 con 0x7f1d040b5000 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- 172.21.15.110:0/98142851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d040b5000 msgr2=0x7f1d0406e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 --2- 172.21.15.110:0/98142851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d040b5000 0x7f1d0406e140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40097c0 tx=0x7f1cf401d380 comp rx=0 tx=0).stop 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- 172.21.15.110:0/98142851 shutdown_connections 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 --2- 172.21.15.110:0/98142851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d040b5000 0x7f1d0406e140 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- 172.21.15.110:0/98142851 >> 172.21.15.110:0/98142851 conn(0x7f1d04062e40 msgr2=0x7f1d0406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- 172.21.15.110:0/98142851 shutdown_connections 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- 172.21.15.110:0/98142851 wait complete. 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 Processor -- start 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- start start 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d0406e680 0x7f1d041540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d04158530 0x7f1d0414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d0414fb90 0x7f1d04156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d040b0040 con 0x7f1d0414fb90 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1d040afec0 con 0x7f1d04158530 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0c800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1d040b01c0 con 0x7f1d0406e680 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d0414fb90 0x7f1d04156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d0414fb90 0x7f1d04156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34566/0 (socket says 172.21.15.110:34566) 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d0a800640 1 -- 172.21.15.110:0/2184768706 learned_addr learned my addr 172.21.15.110:0/2184768706 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d09400640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d04158530 0x7f1d0414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.278+0000 7f1d09e00640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d0406e680 0x7f1d041540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0a800640 1 -- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d0406e680 msgr2=0x7f1d041540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0a800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d0406e680 0x7f1d041540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0a800640 1 -- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d04158530 msgr2=0x7f1d0414f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0a800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d04158530 0x7f1d0414f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0a800640 1 -- 172.21.15.110:0/2184768706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0414f8c0 con 0x7f1d0414fb90 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d09e00640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d0406e680 0x7f1d041540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0a800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d0414fb90 0x7f1d04156f70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000ca70 tx=0x7f1d0000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1cfe000640 1 -- 172.21.15.110:0/2184768706 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00018070 con 0x7f1d0414fb90 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1cfe000640 1 -- 172.21.15.110:0/2184768706 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d0000a6a0 con 0x7f1d0414fb90 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1cfe000640 1 -- 172.21.15.110:0/2184768706 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000aa90 con 0x7f1d0414fb90 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d041574b0 con 0x7f1d0414fb90 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0415df90 con 0x7f1d0414fb90 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d040d0210 con 0x7f1d0414fb90 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1cfe000640 1 -- 172.21.15.110:0/2184768706 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f1d0000ac30 con 0x7f1d0414fb90 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1cfe000640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1ce8067ef0 0x7f1ce806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1cfe000640 1 -- 172.21.15.110:0/2184768706 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4999+0+0 (secure 0 0 0) 0x7f1d00085c80 con 0x7f1d0414fb90 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.282+0000 7f1d09e00640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1ce8067ef0 0x7f1ce806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.286+0000 7f1d09e00640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1ce8067ef0 0x7f1ce806a3b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8000f70 tx=0x7f1cf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.286+0000 7f1cfe000640 1 -- 172.21.15.110:0/2184768706 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1d00056100 con 0x7f1d0414fb90 2024-09-10T10:04:54.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.446+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1d040af460 con 0x7f1d0414fb90 2024-09-10T10:04:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1cfe000640 1 -- 172.21.15.110:0/2184768706 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f1d00055a00 con 0x7f1d0414fb90 2024-09-10T10:04:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1ce8067ef0 msgr2=0x7f1ce806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1ce8067ef0 0x7f1ce806a3b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8000f70 tx=0x7f1cf8009040 comp rx=0 tx=0).stop 2024-09-10T10:04:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d0414fb90 msgr2=0x7f1d04156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d0414fb90 0x7f1d04156f70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000ca70 tx=0x7f1d0000cf30 comp rx=0 tx=0).stop 2024-09-10T10:04:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 shutdown_connections 2024-09-10T10:04:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1ce8067ef0 0x7f1ce806a3b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d0414fb90 0x7f1d04156f70 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d04158530 0x7f1d0414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 --2- 172.21.15.110:0/2184768706 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d0406e680 0x7f1d041540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 >> 172.21.15.110:0/2184768706 conn(0x7f1d04062e40 msgr2=0x7f1d04165df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 shutdown_connections 2024-09-10T10:04:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.450+0000 7f1d0c800640 1 -- 172.21.15.110:0/2184768706 wait complete. 2024-09-10T10:04:54.478 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-10T10:04:54.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feeca800640 1 Processor -- start 2024-09-10T10:04:54.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feeca800640 1 -- start start 2024-09-10T10:04:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feeca800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec40aef50 0x7feec406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feeca800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec41482e0 con 0x7feec406ec20 2024-09-10T10:04:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feeca800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec415ee20 con 0x7feec4147f00 2024-09-10T10:04:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feeca800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feec40b34d0 con 0x7feec40aef50 2024-09-10T10:04:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec3e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7feec406ec20 0x7feec4072310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35538/0 (socket says 172.21.15.110:35538) 2024-09-10T10:04:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec3e00640 1 -- 172.21.15.110:0/2467064811 learned_addr learned my addr 172.21.15.110:0/2467064811 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2a00640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec40aef50 0x7feec406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736505255 0 0) 0x7feec41482e0 con 0x7feec406ec20 2024-09-10T10:04:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feea8003530 con 0x7feec406ec20 2024-09-10T10:04:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2469455517 0 0) 0x7feea8003530 con 0x7feec406ec20 2024-09-10T10:04:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec40aef50 msgr2=0x7feec406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec40aef50 0x7feec406e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 >> v1:172.21.15.145:6789/0 conn(0x7feec4147f00 legacy=0x7feec40ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec415f100 con 0x7feec406ec20 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7feebc001a00 con 0x7feec406ec20 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 >> v1:172.21.15.110:6789/0 conn(0x7feec406ec20 legacy=0x7feec4072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec40aef50 0x7feea80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feea8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feea8004d50 0x7feea80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feea800ba00 con 0x7feea8003f30 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feea800bbb0 con 0x7feea8004d50 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.610+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feea800bdb0 con 0x7feec40aef50 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2a00640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feea8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec3400640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec40aef50 0x7feea80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec3e00640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feea8004d50 0x7feea80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2a00640 1 -- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec40aef50 msgr2=0x7feea80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:54.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2a00640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec40aef50 0x7feea80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2a00640 1 -- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feea8004d50 msgr2=0x7feea80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:54.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2a00640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feea8004d50 0x7feea80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2a00640 1 -- 172.21.15.110:0/2467064811 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea8003530 con 0x7feea8003f30 2024-09-10T10:04:54.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2a00640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feea8004810 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7feec40b51f0 tx=0x7feeb001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fda7d53434bc007 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:54.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00b2070 con 0x7feea8003f30 2024-09-10T10:04:54.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb00a5520 con 0x7feea8003f30 2024-09-10T10:04:54.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feec2000640 1 -- 172.21.15.110:0/2467064811 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00096b0 con 0x7feea8003f30 2024-09-10T10:04:54.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feeca800640 1 -- 172.21.15.110:0/2467064811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 msgr2=0x7feea8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.614+0000 7feeca800640 1 --2- 172.21.15.110:0/2467064811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feea8004810 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7feec40b51f0 tx=0x7feeb001c930 comp rx=0 tx=0).stop 2024-09-10T10:04:54.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3e00640 1 -- 172.21.15.110:0/2467064811 reap_dead start 2024-09-10T10:04:54.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 -- 172.21.15.110:0/2467064811 shutdown_connections 2024-09-10T10:04:54.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 -- 172.21.15.110:0/2467064811 >> 172.21.15.110:0/2467064811 conn(0x7feec406bc50 msgr2=0x7feec406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:54.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 -- 172.21.15.110:0/2467064811 shutdown_connections 2024-09-10T10:04:54.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 -- 172.21.15.110:0/2467064811 wait complete. 2024-09-10T10:04:54.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 Processor -- start 2024-09-10T10:04:54.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 -- start start 2024-09-10T10:04:54.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feec415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec412e040 0x7feec40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec41611e0 0x7feec4161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feec406f0b0 con 0x7feea8003f30 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feec415f6c0 con 0x7feec412e040 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feeca800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feec406e6e0 con 0x7feec41611e0 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feec415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feec415bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34578/0 (socket says 172.21.15.110:34578) 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 -- 172.21.15.110:0/696686409 learned_addr learned my addr 172.21.15.110:0/696686409 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec2a00640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec412e040 0x7feec40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3e00640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec41611e0 0x7feec4161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 -- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec41611e0 msgr2=0x7feec4161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec41611e0 0x7feec4161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 -- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec412e040 msgr2=0x7feec40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec412e040 0x7feec40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 -- 172.21.15.110:0/696686409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec40cd100 con 0x7feea8003f30 2024-09-10T10:04:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec2a00640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec412e040 0x7feec40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feec3400640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feec415bf20 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7feeb40079b0 tx=0x7feeb400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.618+0000 7feebb400640 1 -- 172.21.15.110:0/696686409 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4019020 con 0x7feea8003f30 2024-09-10T10:04:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feebb400640 1 -- 172.21.15.110:0/696686409 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb400fcb0 con 0x7feea8003f30 2024-09-10T10:04:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feebb400640 1 -- 172.21.15.110:0/696686409 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4005310 con 0x7feea8003f30 2024-09-10T10:04:54.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec40cd3e0 con 0x7feea8003f30 2024-09-10T10:04:54.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec40cd9a0 con 0x7feea8003f30 2024-09-10T10:04:54.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec415ee20 con 0x7feea8003f30 2024-09-10T10:04:54.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feebb400640 1 -- 172.21.15.110:0/696686409 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7feeb401c020 con 0x7feea8003f30 2024-09-10T10:04:54.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feebb400640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feea4067ef0 0x7feea406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:54.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feebb400640 1 -- 172.21.15.110:0/696686409 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4999+0+0 (secure 0 0 0) 0x7feeb4089c70 con 0x7feea8003f30 2024-09-10T10:04:54.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.622+0000 7feec2a00640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feea4067ef0 0x7feea406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:54.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.626+0000 7feec2a00640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feea4067ef0 0x7feea406a3b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feeb00af000 tx=0x7feeb00046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:54.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.626+0000 7feebb400640 1 -- 172.21.15.110:0/696686409 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feeb4023020 con 0x7feea8003f30 2024-09-10T10:04:54.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:54.838+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7feec40cce30 con 0x7feea8003f30 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.098+0000 7feebb400640 1 -- 172.21.15.110:0/696686409 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v360) ==== 151+0+0 (secure 0 0 0) 0x7feeb4013330 con 0x7feea8003f30 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feea4067ef0 msgr2=0x7feea406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feea4067ef0 0x7feea406a3b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feeb00af000 tx=0x7feeb00046d0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 msgr2=0x7feec415bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feec415bf20 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7feeb40079b0 tx=0x7feeb400f5a0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 shutdown_connections 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feea4067ef0 0x7feea406a3b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feec41611e0 0x7feec4161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec412e040 0x7feec40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 --2- 172.21.15.110:0/696686409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feea8003f30 0x7feec415bf20 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 >> 172.21.15.110:0/696686409 conn(0x7feec406bc50 msgr2=0x7feec413c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:55.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 shutdown_connections 2024-09-10T10:04:55.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.102+0000 7feeca800640 1 -- 172.21.15.110:0/696686409 wait complete. 2024-09-10T10:04:55.125 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-10T10:04:55.125 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-10T10:04:55.125 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:04:55.126 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3227600640 1 Processor -- start 2024-09-10T10:04:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3227600640 1 -- start start 2024-09-10T10:04:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3227600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e4a0 0x7f3220070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3227600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f322015e880 0x7f32200b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3227600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f32200d1f40 0x7f322006e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3227600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f32200d1d10 con 0x7f322015e880 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3227600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3220071510 con 0x7f32200d1f40 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3227600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f32200717f0 con 0x7f322015e4a0 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f321fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f322015e880 0x7f32200b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f321fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f322015e880 0x7f32200b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34594/0 (socket says 172.21.15.110:34594) 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f321fe00640 1 -- 172.21.15.110:0/846747921 learned_addr learned my addr 172.21.15.110:0/846747921 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3225600640 1 --2- 172.21.15.110:0/846747921 >> v2:172.21.15.145:3300/0 conn(0x7f32200d1f40 0x7f322006e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3224c00640 1 --2- 172.21.15.110:0/846747921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e4a0 0x7f3220070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f321fe00640 1 -- 172.21.15.110:0/846747921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e4a0 msgr2=0x7f3220070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f321fe00640 1 --2- 172.21.15.110:0/846747921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e4a0 0x7f3220070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f321fe00640 1 -- 172.21.15.110:0/846747921 >> v2:172.21.15.145:3300/0 conn(0x7f32200d1f40 msgr2=0x7f322006e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f321fe00640 1 --2- 172.21.15.110:0/846747921 >> v2:172.21.15.145:3300/0 conn(0x7f32200d1f40 0x7f322006e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f321fe00640 1 -- 172.21.15.110:0/846747921 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3220071ad0 con 0x7f322015e880 2024-09-10T10:04:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.258+0000 7f3225600640 1 --2- 172.21.15.110:0/846747921 >> v2:172.21.15.145:3300/0 conn(0x7f32200d1f40 0x7f322006e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:55.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f321fe00640 1 --2- 172.21.15.110:0/846747921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f322015e880 0x7f32200b75c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f32200b5a30 tx=0x7f320c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f822a1af72c35be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f321f400640 1 -- 172.21.15.110:0/846747921 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0b1070 con 0x7f322015e880 2024-09-10T10:04:55.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f321f400640 1 -- 172.21.15.110:0/846747921 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c0ac620 con 0x7f322015e880 2024-09-10T10:04:55.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f321f400640 1 -- 172.21.15.110:0/846747921 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0ac9f0 con 0x7f322015e880 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 -- 172.21.15.110:0/846747921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f322015e880 msgr2=0x7f32200b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 --2- 172.21.15.110:0/846747921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f322015e880 0x7f32200b75c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f32200b5a30 tx=0x7f320c01c4f0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 -- 172.21.15.110:0/846747921 shutdown_connections 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 --2- 172.21.15.110:0/846747921 >> v2:172.21.15.145:3300/0 conn(0x7f32200d1f40 0x7f322006e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 --2- 172.21.15.110:0/846747921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f322015e880 0x7f32200b75c0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 --2- 172.21.15.110:0/846747921 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e4a0 0x7f3220070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 -- 172.21.15.110:0/846747921 >> 172.21.15.110:0/846747921 conn(0x7f322006f400 msgr2=0x7f322006f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 -- 172.21.15.110:0/846747921 shutdown_connections 2024-09-10T10:04:55.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.262+0000 7f3227600640 1 -- 172.21.15.110:0/846747921 wait complete. 2024-09-10T10:04:55.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 Processor -- start 2024-09-10T10:04:55.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 -- start start 2024-09-10T10:04:55.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32200d1f40 0x7f32200b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f322015e4a0 0x7f3220155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e880 0x7f322015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f322006d6f0 con 0x7f32200d1f40 2024-09-10T10:04:55.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f322006d570 con 0x7f322015e4a0 2024-09-10T10:04:55.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f322006d870 con 0x7f322015e880 2024-09-10T10:04:55.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32200d1f40 0x7f32200b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32200d1f40 0x7f32200b23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34596/0 (socket says 172.21.15.110:34596) 2024-09-10T10:04:55.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 -- 172.21.15.110:0/2478129343 learned_addr learned my addr 172.21.15.110:0/2478129343 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:55.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f321fe00640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f322015e4a0 0x7f3220155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3225600640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e880 0x7f322015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 -- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e880 msgr2=0x7f322015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e880 0x7f322015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 -- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f322015e4a0 msgr2=0x7f3220155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f322015e4a0 0x7f3220155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 -- 172.21.15.110:0/2478129343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322015cf10 con 0x7f32200d1f40 2024-09-10T10:04:55.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3224c00640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32200d1f40 0x7f32200b23d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f321000d800 tx=0x7f321000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f321cc00640 1 -- 172.21.15.110:0/2478129343 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321000b6c0 con 0x7f32200d1f40 2024-09-10T10:04:55.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f321cc00640 1 -- 172.21.15.110:0/2478129343 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3210019070 con 0x7f32200d1f40 2024-09-10T10:04:55.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322015d1f0 con 0x7f32200d1f40 2024-09-10T10:04:55.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f321cc00640 1 -- 172.21.15.110:0/2478129343 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321000bae0 con 0x7f32200d1f40 2024-09-10T10:04:55.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.266+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322015d7b0 con 0x7f32200d1f40 2024-09-10T10:04:55.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.270+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3220070ba0 con 0x7f32200d1f40 2024-09-10T10:04:55.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.270+0000 7f321cc00640 1 -- 172.21.15.110:0/2478129343 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f321000bc80 con 0x7f32200d1f40 2024-09-10T10:04:55.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.270+0000 7f321cc00640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3200067ef0 0x7f320006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.270+0000 7f321cc00640 1 -- 172.21.15.110:0/2478129343 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4999+0+0 (secure 0 0 0) 0x7f3210086a20 con 0x7f32200d1f40 2024-09-10T10:04:55.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.270+0000 7f321fe00640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3200067ef0 0x7f320006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.270+0000 7f321fe00640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3200067ef0 0x7f320006a3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f320c0ae000 tx=0x7f320c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.274+0000 7f321cc00640 1 -- 172.21.15.110:0/2478129343 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3210056ea0 con 0x7f32200d1f40 2024-09-10T10:04:55.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.434+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32200d1090 con 0x7f32200d1f40 2024-09-10T10:04:55.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.434+0000 7f321cc00640 1 -- 172.21.15.110:0/2478129343 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+13169 (secure 0 0 0) 0x7f32100567a0 con 0x7f32200d1f40 2024-09-10T10:04:55.440 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:55.440 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":133,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:04:54.095654+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6835/1881265533":"2024-09-11T10:04:51.088802+0000","172.21.15.110:6833/1881265533":"2024-09-11T10:04:51.088802+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3200067ef0 msgr2=0x7f320006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3200067ef0 0x7f320006a3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f320c0ae000 tx=0x7f320c0a7000 comp rx=0 tx=0).stop 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32200d1f40 msgr2=0x7f32200b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32200d1f40 0x7f32200b23d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f321000d800 tx=0x7f321000dcc0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 shutdown_connections 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3200067ef0 0x7f320006a3b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f322015e880 0x7f322015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f322015e4a0 0x7f3220155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 --2- 172.21.15.110:0/2478129343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32200d1f40 0x7f32200b23d0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 >> 172.21.15.110:0/2478129343 conn(0x7f322006f400 msgr2=0x7f322015ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 shutdown_connections 2024-09-10T10:04:55.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.438+0000 7f3227600640 1 -- 172.21.15.110:0/2478129343 wait complete. 2024-09-10T10:04:55.461 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:04:55.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:04:55.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.594+0000 7fc3f7000640 1 Processor -- start 2024-09-10T10:04:55.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.594+0000 7fc3f7000640 1 -- start start 2024-09-10T10:04:55.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f7000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc3f0073250 0x7fc3f0161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f7000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f0162e80 con 0x7fc3f006ea80 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f7000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f01628c0 con 0x7fc3f00b45f0 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f7000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f0162ba0 con 0x7fc3f0073250 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f5200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc3f0073250 0x7fc3f0161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f5200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc3f0073250 0x7fc3f0161fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34610/0 (socket says 172.21.15.110:34610) 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f5c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc3f006ea80 0x7fc3f015f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35554/0 (socket says 172.21.15.110:35554) 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f5200640 1 -- 172.21.15.110:0/4225246637 learned_addr learned my addr 172.21.15.110:0/4225246637 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f5200640 1 -- 172.21.15.110:0/4225246637 >> v1:172.21.15.145:6789/0 conn(0x7fc3f00b45f0 legacy=0x7fc3f00b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f5200640 1 -- 172.21.15.110:0/4225246637 >> v1:172.21.15.110:6789/0 conn(0x7fc3f006ea80 legacy=0x7fc3f015f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.598+0000 7fc3f5200640 1 -- 172.21.15.110:0/4225246637 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f015eb70 con 0x7fc3f0073250 2024-09-10T10:04:55.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f5200640 1 --2- 172.21.15.110:0/4225246637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f0073250 0x7fc3f0161fc0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009870 tx=0x7fc3dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dd8a1b65b65581 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3efe00640 1 -- 172.21.15.110:0/4225246637 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0b1070 con 0x7fc3f0073250 2024-09-10T10:04:55.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3efe00640 1 -- 172.21.15.110:0/4225246637 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3dc0ac600 con 0x7fc3f0073250 2024-09-10T10:04:55.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3efe00640 1 -- 172.21.15.110:0/4225246637 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0ac9d0 con 0x7fc3f0073250 2024-09-10T10:04:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- 172.21.15.110:0/4225246637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f0073250 msgr2=0x7fc3f0161fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 --2- 172.21.15.110:0/4225246637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f0073250 0x7fc3f0161fc0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009870 tx=0x7fc3dc01c490 comp rx=0 tx=0).stop 2024-09-10T10:04:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- 172.21.15.110:0/4225246637 shutdown_connections 2024-09-10T10:04:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 --2- 172.21.15.110:0/4225246637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f0073250 0x7fc3f0161fc0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- 172.21.15.110:0/4225246637 >> 172.21.15.110:0/4225246637 conn(0x7fc3f00b7be0 msgr2=0x7fc3f00b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- 172.21.15.110:0/4225246637 shutdown_connections 2024-09-10T10:04:55.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- 172.21.15.110:0/4225246637 wait complete. 2024-09-10T10:04:55.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 Processor -- start 2024-09-10T10:04:55.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- start start 2024-09-10T10:04:55.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f006ea80 0x7fc3f0164060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3f0073250 0x7fc3f01645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3f00b45f0 0x7fc3f0157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0162680 con 0x7fc3f006ea80 2024-09-10T10:04:55.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0162500 con 0x7fc3f00b45f0 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f7000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f016c930 con 0x7fc3f0073250 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f006ea80 0x7fc3f0164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f006ea80 0x7fc3f0164060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34624/0 (socket says 172.21.15.110:34624) 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.602+0000 7fc3f5c00640 1 -- 172.21.15.110:0/985744438 learned_addr learned my addr 172.21.15.110:0/985744438 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f6600640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3f00b45f0 0x7fc3f0157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f5200640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3f0073250 0x7fc3f01645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f5c00640 1 -- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3f0073250 msgr2=0x7fc3f01645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f5c00640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3f0073250 0x7fc3f01645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f5c00640 1 -- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3f00b45f0 msgr2=0x7fc3f0157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f5c00640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3f00b45f0 0x7fc3f0157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f5c00640 1 -- 172.21.15.110:0/985744438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f0155af0 con 0x7fc3f006ea80 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f5200640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3f0073250 0x7fc3f01645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f6600640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3f00b45f0 0x7fc3f0157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:55.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f5c00640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f006ea80 0x7fc3f0164060 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c370 tx=0x7fc3e400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3ed600640 1 -- 172.21.15.110:0/985744438 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40074e0 con 0x7fc3f006ea80 2024-09-10T10:04:55.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3ed600640 1 -- 172.21.15.110:0/985744438 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e40076a0 con 0x7fc3f006ea80 2024-09-10T10:04:55.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f0155d90 con 0x7fc3f006ea80 2024-09-10T10:04:55.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3ed600640 1 -- 172.21.15.110:0/985744438 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4004a10 con 0x7fc3f006ea80 2024-09-10T10:04:55.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f0156350 con 0x7fc3f006ea80 2024-09-10T10:04:55.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f0161fc0 con 0x7fc3f006ea80 2024-09-10T10:04:55.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3ed600640 1 -- 172.21.15.110:0/985744438 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fc3e401a020 con 0x7fc3f006ea80 2024-09-10T10:04:55.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3ed600640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc3d0067ef0 0x7fc3d006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.606+0000 7fc3ed600640 1 -- 172.21.15.110:0/985744438 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4999+0+0 (secure 0 0 0) 0x7fc3e4085c80 con 0x7fc3f006ea80 2024-09-10T10:04:55.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.610+0000 7fc3f5200640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc3d0067ef0 0x7fc3d006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.610+0000 7fc3f5200640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc3d0067ef0 0x7fc3d006a3b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0ae000 tx=0x7fc3dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.610+0000 7fc3ed600640 1 -- 172.21.15.110:0/985744438 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc3e4056190 con 0x7fc3f006ea80 2024-09-10T10:04:55.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.770+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc3f0161860 con 0x7fc3f006ea80 2024-09-10T10:04:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.770+0000 7fc3ed600640 1 -- 172.21.15.110:0/985744438 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) ==== 93+0+10 (secure 0 0 0) 0x7fc3e4055a90 con 0x7fc3f006ea80 2024-09-10T10:04:55.775 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:04:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc3d0067ef0 msgr2=0x7fc3d006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc3d0067ef0 0x7fc3d006a3b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0ae000 tx=0x7fc3dc008040 comp rx=0 tx=0).stop 2024-09-10T10:04:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f006ea80 msgr2=0x7fc3f0164060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f006ea80 0x7fc3f0164060 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c370 tx=0x7fc3e400c830 comp rx=0 tx=0).stop 2024-09-10T10:04:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 shutdown_connections 2024-09-10T10:04:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc3d0067ef0 0x7fc3d006a3b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc3f00b45f0 0x7fc3f0157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc3f0073250 0x7fc3f01645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 --2- 172.21.15.110:0/985744438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f006ea80 0x7fc3f0164060 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 >> 172.21.15.110:0/985744438 conn(0x7fc3f00b7be0 msgr2=0x7fc3f006cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 shutdown_connections 2024-09-10T10:04:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.774+0000 7fc3f7000640 1 -- 172.21.15.110:0/985744438 wait complete. 2024-09-10T10:04:55.796 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-10T10:04:55.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.926+0000 7fc0a2800640 1 Processor -- start 2024-09-10T10:04:55.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.926+0000 7fc0a2800640 1 -- start start 2024-09-10T10:04:55.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc0a2800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c15c560 0x7fc09c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc0a2800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc09c06f010 0x7fc09c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc0a2800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc09c06f430 con 0x7fc09c155970 2024-09-10T10:04:55.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc0a2800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc09c0b4570 con 0x7fc09c06f010 2024-09-10T10:04:55.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc0a2800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc09c155d50 con 0x7fc09c15c560 2024-09-10T10:04:55.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c15c560 0x7fc09c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc09c06f010 0x7fc09c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c15c560 0x7fc09c06ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:47752/0 (socket says 172.21.15.110:47752) 2024-09-10T10:04:55.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09b400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc09c155970 0x7fc09c0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:49428/0 (socket says 172.21.15.110:49428) 2024-09-10T10:04:55.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09aa00640 1 -- 172.21.15.110:0/4042402357 learned_addr learned my addr 172.21.15.110:0/4042402357 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:55.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09aa00640 1 -- 172.21.15.110:0/4042402357 >> v2:172.21.15.145:3300/0 conn(0x7fc09c06f010 msgr2=0x7fc09c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:55.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09aa00640 1 --2- 172.21.15.110:0/4042402357 >> v2:172.21.15.145:3300/0 conn(0x7fc09c06f010 0x7fc09c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09aa00640 1 -- 172.21.15.110:0/4042402357 >> v1:172.21.15.145:6789/0 conn(0x7fc09c155970 legacy=0x7fc09c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.930+0000 7fc09aa00640 1 -- 172.21.15.110:0/4042402357 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c0b34d0 con 0x7fc09c15c560 2024-09-10T10:04:55.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc09aa00640 1 --2- 172.21.15.110:0/4042402357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c15c560 0x7fc09c06ead0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc088009870 tx=0x7fc08801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e5b5641c38c7eef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc09a000640 1 -- 172.21.15.110:0/4042402357 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0880aa600 con 0x7fc09c15c560 2024-09-10T10:04:55.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc09a000640 1 -- 172.21.15.110:0/4042402357 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0880af070 con 0x7fc09c15c560 2024-09-10T10:04:55.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc09a000640 1 -- 172.21.15.110:0/4042402357 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0880aa9c0 con 0x7fc09c15c560 2024-09-10T10:04:55.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 -- 172.21.15.110:0/4042402357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c15c560 msgr2=0x7fc09c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 --2- 172.21.15.110:0/4042402357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c15c560 0x7fc09c06ead0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc088009870 tx=0x7fc08801c4a0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 -- 172.21.15.110:0/4042402357 shutdown_connections 2024-09-10T10:04:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 --2- 172.21.15.110:0/4042402357 >> v2:172.21.15.145:3300/0 conn(0x7fc09c06f010 0x7fc09c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 --2- 172.21.15.110:0/4042402357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c15c560 0x7fc09c06ead0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 -- 172.21.15.110:0/4042402357 >> 172.21.15.110:0/4042402357 conn(0x7fc09c06bc50 msgr2=0x7fc09c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 -- 172.21.15.110:0/4042402357 shutdown_connections 2024-09-10T10:04:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 -- 172.21.15.110:0/4042402357 wait complete. 2024-09-10T10:04:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.934+0000 7fc0a2800640 1 Processor -- start 2024-09-10T10:04:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 -- start start 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc09c06f010 0x7fc09c13d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c155970 0x7fc09c1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c15c560 0x7fc09c1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c0cf170 con 0x7fc09c06f010 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c0b4050 con 0x7fc09c15c560 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc09c0cf2f0 con 0x7fc09c155970 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc09c06f010 0x7fc09c13d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc09c06f010 0x7fc09c13d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34638/0 (socket says 172.21.15.110:34638) 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 -- 172.21.15.110:0/292338596 learned_addr learned my addr 172.21.15.110:0/292338596 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09be00640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c15c560 0x7fc09c1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09aa00640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c155970 0x7fc09c1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 -- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c155970 msgr2=0x7fc09c1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c155970 0x7fc09c1588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 -- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c15c560 msgr2=0x7fc09c1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c15c560 0x7fc09c1547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 -- 172.21.15.110:0/292338596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c13d930 con 0x7fc09c06f010 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc09b400640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc09c06f010 0x7fc09c13d3f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00c7c0 tx=0x7fc08c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc093400640 1 -- 172.21.15.110:0/292338596 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c017070 con 0x7fc09c06f010 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc093400640 1 -- 172.21.15.110:0/292338596 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc08c00a640 con 0x7fc09c06f010 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc093400640 1 -- 172.21.15.110:0/292338596 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00a9f0 con 0x7fc09c06f010 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09c13de10 con 0x7fc09c06f010 2024-09-10T10:04:55.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09c13e3d0 con 0x7fc09c06f010 2024-09-10T10:04:55.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.938+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc09c0ce420 con 0x7fc09c06f010 2024-09-10T10:04:55.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.942+0000 7fc093400640 1 -- 172.21.15.110:0/292338596 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fc08c01a070 con 0x7fc09c06f010 2024-09-10T10:04:55.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.942+0000 7fc093400640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc07c067ef0 0x7fc07c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:55.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.942+0000 7fc093400640 1 -- 172.21.15.110:0/292338596 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4999+0+0 (secure 0 0 0) 0x7fc08c085ac0 con 0x7fc09c06f010 2024-09-10T10:04:55.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.942+0000 7fc09aa00640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc07c067ef0 0x7fc07c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:55.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.942+0000 7fc09aa00640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc07c067ef0 0x7fc07c06a3b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ae000 tx=0x7fc088008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:55.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:55.946+0000 7fc093400640 1 -- 172.21.15.110:0/292338596 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc08c0558f0 con 0x7fc09c06f010 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:56.158+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7fc09c1652b0 con 0x7fc09c06f010 2024-09-10T10:04:57.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.106+0000 7fc093400640 1 -- 172.21.15.110:0/292338596 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v362) ==== 152+0+0 (secure 0 0 0) 0x7fc08c059bf0 con 0x7fc09c06f010 2024-09-10T10:04:57.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc07c067ef0 msgr2=0x7fc07c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc07c067ef0 0x7fc07c06a3b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ae000 tx=0x7fc088008040 comp rx=0 tx=0).stop 2024-09-10T10:04:57.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc09c06f010 msgr2=0x7fc09c13d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc09c06f010 0x7fc09c13d3f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00c7c0 tx=0x7fc08c00cc80 comp rx=0 tx=0).stop 2024-09-10T10:04:57.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 shutdown_connections 2024-09-10T10:04:57.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc07c067ef0 0x7fc07c06a3b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc09c15c560 0x7fc09c1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc09c155970 0x7fc09c1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 --2- 172.21.15.110:0/292338596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc09c06f010 0x7fc09c13d3f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 >> 172.21.15.110:0/292338596 conn(0x7fc09c06bc50 msgr2=0x7fc09c0b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:57.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 shutdown_connections 2024-09-10T10:04:57.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.110+0000 7fc0a2800640 1 -- 172.21.15.110:0/292338596 wait complete. 2024-09-10T10:04:57.136 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:04:57.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa14bc00640 1 Processor -- start 2024-09-10T10:04:57.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa14bc00640 1 -- start start 2024-09-10T10:04:57.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa14bc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa14406ec70 0x7fa14406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa14bc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa144072690 con 0x7fa14415f010 2024-09-10T10:04:57.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa14bc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa144072970 con 0x7fa1440b4c80 2024-09-10T10:04:57.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa14bc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa144072c50 con 0x7fa14406ec70 2024-09-10T10:04:57.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa13be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa14406ec70 0x7fa14406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa149200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa14415f010 0x7fa14415f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35566/0 (socket says 172.21.15.110:35566) 2024-09-10T10:04:57.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa13be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa14406ec70 0x7fa14406f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34648/0 (socket says 172.21.15.110:34648) 2024-09-10T10:04:57.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa149200640 1 -- 172.21.15.110:0/1704463038 learned_addr learned my addr 172.21.15.110:0/1704463038 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:57.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa13be00640 1 -- 172.21.15.110:0/1704463038 >> v1:172.21.15.145:6789/0 conn(0x7fa1440b4c80 legacy=0x7fa1440b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:57.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa13be00640 1 -- 172.21.15.110:0/1704463038 >> v1:172.21.15.110:6789/0 conn(0x7fa14415f010 legacy=0x7fa14415f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa13be00640 1 -- 172.21.15.110:0/1704463038 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa144072f30 con 0x7fa14406ec70 2024-09-10T10:04:57.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.274+0000 7fa143e00640 1 -- 172.21.15.110:0/1704463038 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307067092 0 0) 0x7fa144072690 con 0x7fa14415f010 2024-09-10T10:04:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa13be00640 1 --2- 172.21.15.110:0/1704463038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa14406ec70 0x7fa14406f100 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0097c0 tx=0x7fa12c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a41daf3ca3c845 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa143e00640 1 -- 172.21.15.110:0/1704463038 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0b0070 con 0x7fa14406ec70 2024-09-10T10:04:57.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa143e00640 1 -- 172.21.15.110:0/1704463038 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12c0ab600 con 0x7fa14406ec70 2024-09-10T10:04:57.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa143e00640 1 -- 172.21.15.110:0/1704463038 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0ab9d0 con 0x7fa14406ec70 2024-09-10T10:04:57.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa14bc00640 1 -- 172.21.15.110:0/1704463038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa14406ec70 msgr2=0x7fa14406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa14bc00640 1 --2- 172.21.15.110:0/1704463038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa14406ec70 0x7fa14406f100 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0097c0 tx=0x7fa12c01c3d0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa14bc00640 1 -- 172.21.15.110:0/1704463038 shutdown_connections 2024-09-10T10:04:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa14bc00640 1 --2- 172.21.15.110:0/1704463038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa14406ec70 0x7fa14406f100 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa14bc00640 1 -- 172.21.15.110:0/1704463038 >> 172.21.15.110:0/1704463038 conn(0x7fa14406ca70 msgr2=0x7fa14406ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa14bc00640 1 -- 172.21.15.110:0/1704463038 shutdown_connections 2024-09-10T10:04:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.278+0000 7fa14bc00640 1 -- 172.21.15.110:0/1704463038 wait complete. 2024-09-10T10:04:57.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 Processor -- start 2024-09-10T10:04:57.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 -- start start 2024-09-10T10:04:57.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14406ec70 0x7fa14414eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1440b4c80 0x7fa1440cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14415f010 0x7fa1440cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa144072450 con 0x7fa1440b4c80 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa1440722d0 con 0x7fa14415f010 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa14416c600 con 0x7fa14406ec70 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1440b4c80 0x7fa1440cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1440b4c80 0x7fa1440cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34664/0 (socket says 172.21.15.110:34664) 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 -- 172.21.15.110:0/996181657 learned_addr learned my addr 172.21.15.110:0/996181657 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa149200640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14406ec70 0x7fa14414eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa149c00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14415f010 0x7fa1440cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 -- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14406ec70 msgr2=0x7fa14414eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14406ec70 0x7fa14414eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 -- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14415f010 msgr2=0x7fa1440cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14415f010 0x7fa1440cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 -- 172.21.15.110:0/996181657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14415b2a0 con 0x7fa1440b4c80 2024-09-10T10:04:57.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa13be00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1440b4c80 0x7fa1440cffc0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0097c0 tx=0x7fa12c01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:57.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa141600640 1 -- 172.21.15.110:0/996181657 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0b0040 con 0x7fa1440b4c80 2024-09-10T10:04:57.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa141600640 1 -- 172.21.15.110:0/996181657 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12c004030 con 0x7fa1440b4c80 2024-09-10T10:04:57.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa141600640 1 -- 172.21.15.110:0/996181657 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0043e0 con 0x7fa1440b4c80 2024-09-10T10:04:57.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa144156d90 con 0x7fa1440b4c80 2024-09-10T10:04:57.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa144157350 con 0x7fa1440b4c80 2024-09-10T10:04:57.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.282+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa144071ea0 con 0x7fa1440b4c80 2024-09-10T10:04:57.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.286+0000 7fa141600640 1 -- 172.21.15.110:0/996181657 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fa12c003620 con 0x7fa1440b4c80 2024-09-10T10:04:57.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.286+0000 7fa141600640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa120067ef0 0x7fa12006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.286+0000 7fa141600640 1 -- 172.21.15.110:0/996181657 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4999+0+0 (secure 0 0 0) 0x7fa12c120ce0 con 0x7fa1440b4c80 2024-09-10T10:04:57.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.286+0000 7fa149200640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa120067ef0 0x7fa12006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.286+0000 7fa149200640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa120067ef0 0x7fa12006a3b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa134000f70 tx=0x7fa134007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:57.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.290+0000 7fa141600640 1 -- 172.21.15.110:0/996181657 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa12c016050 con 0x7fa1440b4c80 2024-09-10T10:04:57.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.446+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa1440627b0 con 0x7fa1440b4c80 2024-09-10T10:04:57.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.450+0000 7fa141600640 1 -- 172.21.15.110:0/996181657 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fa12c0f0da0 con 0x7fa1440b4c80 2024-09-10T10:04:57.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa120067ef0 msgr2=0x7fa12006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa120067ef0 0x7fa12006a3b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa134000f70 tx=0x7fa134007df0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1440b4c80 msgr2=0x7fa1440cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1440b4c80 0x7fa1440cffc0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0097c0 tx=0x7fa12c01ac10 comp rx=0 tx=0).stop 2024-09-10T10:04:57.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 shutdown_connections 2024-09-10T10:04:57.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa120067ef0 0x7fa12006a3b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa14415f010 0x7fa1440cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1440b4c80 0x7fa1440cffc0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 --2- 172.21.15.110:0/996181657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa14406ec70 0x7fa14414eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 >> 172.21.15.110:0/996181657 conn(0x7fa14406ca70 msgr2=0x7fa14415b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:57.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 shutdown_connections 2024-09-10T10:04:57.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.454+0000 7fa14bc00640 1 -- 172.21.15.110:0/996181657 wait complete. 2024-09-10T10:04:57.477 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:04:57.477 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:04:57.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:04:57.668 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:04:57.668 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:57.668 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:57.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:04:57.677 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:57.677 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:57.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:04:57.686 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:04:57.686 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:57.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:04:57.728 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:04:57.728 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:04:57.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:04:57.737 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:04:57.737 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:04:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.902+0000 7f711a200640 1 Processor -- start 2024-09-10T10:04:57.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.902+0000 7f711a200640 1 -- start start 2024-09-10T10:04:57.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f711a200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7114145b00 0x7f71140cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f711a200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 0x7f71140b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f711a200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7114071db0 con 0x7f71140b5690 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f711a200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f71140b6010 con 0x7f7114145b00 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f711a200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f71140ce5f0 con 0x7f711415d050 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7118e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7114145b00 0x7f71140cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7113e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f71140b5690 0x7f711415c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:35576/0 (socket says 172.21.15.110:35576) 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7113e00640 1 -- 172.21.15.110:0/3825580964 learned_addr learned my addr 172.21.15.110:0/3825580964 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7119800640 1 --2- 172.21.15.110:0/3825580964 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 0x7f71140b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7118e00640 1 -- 172.21.15.110:0/3825580964 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 msgr2=0x7f71140b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7118e00640 1 --2- 172.21.15.110:0/3825580964 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 0x7f71140b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7118e00640 1 -- 172.21.15.110:0/3825580964 >> v1:172.21.15.110:6789/0 conn(0x7f71140b5690 legacy=0x7f711415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7118e00640 1 -- 172.21.15.110:0/3825580964 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71140ce8d0 con 0x7f7114145b00 2024-09-10T10:04:57.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.906+0000 7f7113400640 1 -- 172.21.15.110:0/3825580964 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801478517 0 0) 0x7f7114071db0 con 0x7f71140b5690 2024-09-10T10:04:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f7118e00640 1 --2- 172.21.15.110:0/3825580964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7114145b00 0x7f71140cf0d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f71140b6190 tx=0x7f710401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582ebda96891f08c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f7113400640 1 -- 172.21.15.110:0/3825580964 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040b2070 con 0x7f7114145b00 2024-09-10T10:04:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f7113400640 1 -- 172.21.15.110:0/3825580964 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71040a5810 con 0x7f7114145b00 2024-09-10T10:04:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f7113400640 1 -- 172.21.15.110:0/3825580964 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040a5be0 con 0x7f7114145b00 2024-09-10T10:04:57.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- 172.21.15.110:0/3825580964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7114145b00 msgr2=0x7f71140cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 --2- 172.21.15.110:0/3825580964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7114145b00 0x7f71140cf0d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f71140b6190 tx=0x7f710401c950 comp rx=0 tx=0).stop 2024-09-10T10:04:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- 172.21.15.110:0/3825580964 shutdown_connections 2024-09-10T10:04:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 --2- 172.21.15.110:0/3825580964 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 0x7f71140b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 --2- 172.21.15.110:0/3825580964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7114145b00 0x7f71140cf0d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- 172.21.15.110:0/3825580964 >> 172.21.15.110:0/3825580964 conn(0x7f711406d0f0 msgr2=0x7f711406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- 172.21.15.110:0/3825580964 shutdown_connections 2024-09-10T10:04:57.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- 172.21.15.110:0/3825580964 wait complete. 2024-09-10T10:04:57.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 Processor -- start 2024-09-10T10:04:57.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- start start 2024-09-10T10:04:57.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71140b5690 0x7f711415b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7114145b00 0x7f711413c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 0x7f711415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71140719d0 con 0x7f71140b5690 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7114071850 con 0x7f7114145b00 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f711a200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7114071b50 con 0x7f711415d050 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f7118e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71140b5690 0x7f711415b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f7118e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71140b5690 0x7f711415b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34674/0 (socket says 172.21.15.110:34674) 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.910+0000 7f7118e00640 1 -- 172.21.15.110:0/3429893366 learned_addr learned my addr 172.21.15.110:0/3429893366 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7118e00640 1 -- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 msgr2=0x7f711415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7118e00640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 0x7f711415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7118e00640 1 -- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7114145b00 msgr2=0x7f711413c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7118e00640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7114145b00 0x7f711413c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7118e00640 1 -- 172.21.15.110:0/3429893366 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711413cd00 con 0x7f71140b5690 2024-09-10T10:04:57.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7118e00640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71140b5690 0x7f711415b590 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f71040b0040 tx=0x7f71040196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:57.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7110c00640 1 -- 172.21.15.110:0/3429893366 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040b2040 con 0x7f71140b5690 2024-09-10T10:04:57.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7110c00640 1 -- 172.21.15.110:0/3429893366 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7104003ba0 con 0x7f71140b5690 2024-09-10T10:04:57.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7110c00640 1 -- 172.21.15.110:0/3429893366 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104004020 con 0x7f71140b5690 2024-09-10T10:04:57.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711413e160 con 0x7f71140b5690 2024-09-10T10:04:57.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711413e6e0 con 0x7f71140b5690 2024-09-10T10:04:57.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71140cf0d0 con 0x7f71140b5690 2024-09-10T10:04:57.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7110c00640 1 -- 172.21.15.110:0/3429893366 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f71040c2060 con 0x7f71140b5690 2024-09-10T10:04:57.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7110c00640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4067ef0 0x7f70f406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:57.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.914+0000 7f7110c00640 1 -- 172.21.15.110:0/3429893366 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4999+0+0 (secure 0 0 0) 0x7f7104120a00 con 0x7f71140b5690 2024-09-10T10:04:57.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.918+0000 7f7113e00640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4067ef0 0x7f70f406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:57.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.918+0000 7f7113e00640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4067ef0 0x7f70f406a3b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f71000011f0 tx=0x7f7100007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:57.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:57.918+0000 7f7110c00640 1 -- 172.21.15.110:0/3429893366 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f711413cd00 con 0x7f71140b5690 2024-09-10T10:04:58.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71140b5b30 con 0x7f71140b5690 2024-09-10T10:04:58.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f7110c00640 1 -- 172.21.15.110:0/3429893366 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+3923 (secure 0 0 0) 0x7f7104016540 con 0x7f71140b5690 2024-09-10T10:04:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:58.114 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":91,"btime":"2024-09-10T10:04:52:091213+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3129750368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":3129750368},{"type":"v1","addr":"172.21.15.145:6834","nonce":3129750368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3791950507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3791950507},{"type":"v1","addr":"172.21.15.145:6835","nonce":3791950507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2024540113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2024540113},{"type":"v1","addr":"172.21.15.110:6834","nonce":2024540113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":8088,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2347005851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2347005851},{"type":"v1","addr":"172.21.15.110:6835","nonce":2347005851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[]} 2024-09-10T10:04:58.114 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 91 2024-09-10T10:04:58.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4067ef0 msgr2=0x7f70f406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:58.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4067ef0 0x7f70f406a3b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f71000011f0 tx=0x7f7100007930 comp rx=0 tx=0).stop 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71140b5690 msgr2=0x7f711415b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71140b5690 0x7f711415b590 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f71040b0040 tx=0x7f71040196f0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 shutdown_connections 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4067ef0 0x7f70f406a3b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711415d050 0x7f711415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7114145b00 0x7f711413c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 --2- 172.21.15.110:0/3429893366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71140b5690 0x7f711415b590 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 >> 172.21.15.110:0/3429893366 conn(0x7f711406d0f0 msgr2=0x7f711415e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 shutdown_connections 2024-09-10T10:04:58.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.110+0000 7f711a200640 1 -- 172.21.15.110:0/3429893366 wait complete. 2024-09-10T10:04:58.136 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:04:58.136 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:04:58.136 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:04:58.136 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:04:58.137 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:04:58.137 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:04:58.137 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:04:58.137 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:04:58.137 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:04:58.137 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:04:58.138 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:04:58.138 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:04:58.138 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:04:58.138 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:04:58.138 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:04:58.138 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:04:58.148 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:04:58.148 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:04:58.150 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:04:58.150 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:04:58.155 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:04:58.155 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:04:58.156 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:04:58.156 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:04:58.194 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:04:58.195 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:58.196 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:04:58.197 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:58.199 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:04:58.199 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:58.201 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:04:58.201 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:58.201 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:04:58.281 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:04:58.282 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:04:58.289 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:04:58.290 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:04:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff224a00640 1 Processor -- start 2024-09-10T10:04:58.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff224a00640 1 -- start start 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff224a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff2200b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff224a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff2200ae840 0x7ff2200aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff224a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2200d0230 con 0x7ff22015f9f0 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff224a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff2200b4710 con 0x7ff2200ae840 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff224a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff220072d10 con 0x7ff2200cfe50 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff21de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff2200b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff21e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff2200ae840 0x7ff2200aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff21de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff2200b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:35642/0 (socket says 172.21.15.110:35642) 2024-09-10T10:04:58.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff21de00640 1 -- 172.21.15.110:0/4084149159 learned_addr learned my addr 172.21.15.110:0/4084149159 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:58.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff21de00640 1 -- 172.21.15.110:0/4084149159 >> v2:172.21.15.145:3300/0 conn(0x7ff2200ae840 msgr2=0x7ff2200aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:58.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.342+0000 7ff21de00640 1 --2- 172.21.15.110:0/4084149159 >> v2:172.21.15.145:3300/0 conn(0x7ff2200ae840 0x7ff2200aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff21de00640 1 -- 172.21.15.110:0/4084149159 >> v1:172.21.15.145:6789/0 conn(0x7ff22015f9f0 legacy=0x7ff220072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:58.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff21de00640 1 -- 172.21.15.110:0/4084149159 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff220072f50 con 0x7ff2200cfe50 2024-09-10T10:04:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff21de00640 1 --2- 172.21.15.110:0/4084149159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff2200b5420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff210009e50 tx=0x7ff21001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=139ba91152832953 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff21ca00640 1 -- 172.21.15.110:0/4084149159 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2100a5660 con 0x7ff2200cfe50 2024-09-10T10:04:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff21ca00640 1 -- 172.21.15.110:0/4084149159 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2100a5800 con 0x7ff2200cfe50 2024-09-10T10:04:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff21ca00640 1 -- 172.21.15.110:0/4084149159 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2100a5bd0 con 0x7ff2200cfe50 2024-09-10T10:04:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff224a00640 1 -- 172.21.15.110:0/4084149159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 msgr2=0x7ff2200b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:58.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff224a00640 1 --2- 172.21.15.110:0/4084149159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff2200b5420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff210009e50 tx=0x7ff21001c9a0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff224a00640 1 -- 172.21.15.110:0/4084149159 shutdown_connections 2024-09-10T10:04:58.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff224a00640 1 --2- 172.21.15.110:0/4084149159 >> v2:172.21.15.145:3300/0 conn(0x7ff2200ae840 0x7ff2200aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff224a00640 1 --2- 172.21.15.110:0/4084149159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff2200b5420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff224a00640 1 -- 172.21.15.110:0/4084149159 >> 172.21.15.110:0/4084149159 conn(0x7ff220062d10 msgr2=0x7ff22006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:58.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff224a00640 1 -- 172.21.15.110:0/4084149159 shutdown_connections 2024-09-10T10:04:58.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.346+0000 7ff224a00640 1 -- 172.21.15.110:0/4084149159 wait complete. 2024-09-10T10:04:58.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 Processor -- start 2024-09-10T10:04:58.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 -- start start 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2200ae840 0x7ff22014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff22014c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff22015f9f0 0x7ff22014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200718c0 con 0x7ff2200ae840 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200b5960 con 0x7ff22015f9f0 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff220071a40 con 0x7ff2200cfe50 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2200ae840 0x7ff22014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2200ae840 0x7ff22014c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47474/0 (socket says 172.21.15.110:47474) 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 -- 172.21.15.110:0/4284567926 learned_addr learned my addr 172.21.15.110:0/4284567926 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21d400640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff22014c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:58.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21e800640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff22015f9f0 0x7ff22014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 -- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 msgr2=0x7ff22014c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff22014c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 -- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff22015f9f0 msgr2=0x7ff22014eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff22015f9f0 0x7ff22014eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 -- 172.21.15.110:0/4284567926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22014ad50 con 0x7ff2200ae840 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21e800640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff22015f9f0 0x7ff22014eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:58.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff21de00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2200ae840 0x7ff22014c2b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7ff2100af040 tx=0x7ff210019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff216000640 1 -- 172.21.15.110:0/4284567926 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2100bc020 con 0x7ff2200ae840 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff216000640 1 -- 172.21.15.110:0/4284567926 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2100b8690 con 0x7ff2200ae840 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff216000640 1 -- 172.21.15.110:0/4284567926 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210009510 con 0x7ff2200ae840 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22014b030 con 0x7ff2200ae840 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.350+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22014b5f0 con 0x7ff2200ae840 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.354+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2200aecf0 con 0x7ff2200ae840 2024-09-10T10:04:58.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.354+0000 7ff216000640 1 -- 172.21.15.110:0/4284567926 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7ff2100bf020 con 0x7ff2200ae840 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.354+0000 7ff216000640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff200067f20 0x7ff20006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:58.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.354+0000 7ff216000640 1 -- 172.21.15.110:0/4284567926 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4999+0+0 (secure 0 0 0) 0x7ff210121570 con 0x7ff2200ae840 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.354+0000 7ff21d400640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff200067f20 0x7ff20006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:58.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.354+0000 7ff21d400640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff200067f20 0x7ff20006a3e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff220150620 tx=0x7ff20c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.358+0000 7ff216000640 1 -- 172.21.15.110:0/4284567926 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff2100c6030 con 0x7ff2200ae840 2024-09-10T10:04:58.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:58.534+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff220156c20 con 0x7ff2200ae840 2024-09-10T10:04:59.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.474+0000 7ff216000640 1 -- 172.21.15.110:0/4284567926 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v138) ==== 97+0+0 (secure 0 0 0) 0x7ff210016620 con 0x7ff2200ae840 2024-09-10T10:04:59.479 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:04:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff200067f20 msgr2=0x7ff20006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff200067f20 0x7ff20006a3e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff220150620 tx=0x7ff20c009040 comp rx=0 tx=0).stop 2024-09-10T10:04:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2200ae840 msgr2=0x7ff22014c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2200ae840 0x7ff22014c2b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7ff2100af040 tx=0x7ff210019740 comp rx=0 tx=0).stop 2024-09-10T10:04:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 shutdown_connections 2024-09-10T10:04:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff200067f20 0x7ff20006a3e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff22015f9f0 0x7ff22014eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff2200cfe50 0x7ff22014c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 --2- 172.21.15.110:0/4284567926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2200ae840 0x7ff22014c2b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 >> 172.21.15.110:0/4284567926 conn(0x7ff220062d10 msgr2=0x7ff2200b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 shutdown_connections 2024-09-10T10:04:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.478+0000 7ff224a00640 1 -- 172.21.15.110:0/4284567926 wait complete. 2024-09-10T10:04:59.504 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:04:59.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.638+0000 7fd19ee00640 1 Processor -- start 2024-09-10T10:04:59.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.638+0000 7fd19ee00640 1 -- start start 2024-09-10T10:04:59.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd19ee00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd1980cd510 0x7fd1980cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:59.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd19ee00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 0x7fd19806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:59.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd19ee00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd198072150 con 0x7fd1980cfe30 2024-09-10T10:04:59.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd19ee00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd19815d9a0 con 0x7fd1980cd510 2024-09-10T10:04:59.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd19ee00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd19806ea60 con 0x7fd19806e090 2024-09-10T10:04:59.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd197400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd1980cd510 0x7fd1980cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:59.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd197400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd1980cd510 0x7fd1980cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47488/0 (socket says 172.21.15.110:47488) 2024-09-10T10:04:59.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd197400640 1 -- 172.21.15.110:0/398434553 learned_addr learned my addr 172.21.15.110:0/398434553 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:59.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd19ce00640 1 --2- 172.21.15.110:0/398434553 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 0x7fd19806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:59.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd197400640 1 -- 172.21.15.110:0/398434553 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 msgr2=0x7fd19806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:59.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd197400640 1 --2- 172.21.15.110:0/398434553 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 0x7fd19806e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd197400640 1 -- 172.21.15.110:0/398434553 >> v1:172.21.15.145:6789/0 conn(0x7fd1980cfe30 legacy=0x7fd198071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:59.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.642+0000 7fd197400640 1 -- 172.21.15.110:0/398434553 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19806d2d0 con 0x7fd1980cd510 2024-09-10T10:04:59.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd197400640 1 --2- 172.21.15.110:0/398434553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cd510 0x7fd1980cd9a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd184009870 tx=0x7fd18401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe0b4b1136c26b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:59.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd196a00640 1 -- 172.21.15.110:0/398434553 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1840b1070 con 0x7fd1980cd510 2024-09-10T10:04:59.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd196a00640 1 -- 172.21.15.110:0/398434553 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1840a8050 con 0x7fd1980cd510 2024-09-10T10:04:59.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd196a00640 1 -- 172.21.15.110:0/398434553 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1840ac860 con 0x7fd1980cd510 2024-09-10T10:04:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- 172.21.15.110:0/398434553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cd510 msgr2=0x7fd1980cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 --2- 172.21.15.110:0/398434553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cd510 0x7fd1980cd9a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd184009870 tx=0x7fd18401c4d0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- 172.21.15.110:0/398434553 shutdown_connections 2024-09-10T10:04:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 --2- 172.21.15.110:0/398434553 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 0x7fd19806e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 --2- 172.21.15.110:0/398434553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cd510 0x7fd1980cd9a0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- 172.21.15.110:0/398434553 >> 172.21.15.110:0/398434553 conn(0x7fd19806bc50 msgr2=0x7fd19806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- 172.21.15.110:0/398434553 shutdown_connections 2024-09-10T10:04:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- 172.21.15.110:0/398434553 wait complete. 2024-09-10T10:04:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 Processor -- start 2024-09-10T10:04:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- start start 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 0x7fd198148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1980cd510 0x7fd198159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cfe30 0x7fd19814d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd198169180 con 0x7fd1980cfe30 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd19815d6c0 con 0x7fd1980cd510 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ee00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd198169300 con 0x7fd19806e090 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.646+0000 7fd19ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cfe30 0x7fd19814d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cfe30 0x7fd19814d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47502/0 (socket says 172.21.15.110:47502) 2024-09-10T10:04:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ce00640 1 -- 172.21.15.110:0/2656750435 learned_addr learned my addr 172.21.15.110:0/2656750435 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:04:59.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd197400640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1980cd510 0x7fd198159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:59.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd197e00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 0x7fd198148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:59.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ce00640 1 -- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 msgr2=0x7fd198148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:59.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ce00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 0x7fd198148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ce00640 1 -- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1980cd510 msgr2=0x7fd198159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:59.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ce00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1980cd510 0x7fd198159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ce00640 1 -- 172.21.15.110:0/2656750435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd198151270 con 0x7fd1980cfe30 2024-09-10T10:04:59.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ce00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cfe30 0x7fd19814d0c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d560 tx=0x7fd19000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:59.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd18fe00640 1 -- 172.21.15.110:0/2656750435 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900125f0 con 0x7fd1980cfe30 2024-09-10T10:04:59.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd18fe00640 1 -- 172.21.15.110:0/2656750435 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd190019070 con 0x7fd1980cfe30 2024-09-10T10:04:59.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd198159810 con 0x7fd1980cfe30 2024-09-10T10:04:59.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd18fe00640 1 -- 172.21.15.110:0/2656750435 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900129f0 con 0x7fd1980cfe30 2024-09-10T10:04:59.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd198159dd0 con 0x7fd1980cfe30 2024-09-10T10:04:59.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19806e500 con 0x7fd1980cfe30 2024-09-10T10:04:59.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd18fe00640 1 -- 172.21.15.110:0/2656750435 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fd190012b90 con 0x7fd1980cfe30 2024-09-10T10:04:59.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd18fe00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd178067ef0 0x7fd17806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:59.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.650+0000 7fd18fe00640 1 -- 172.21.15.110:0/2656750435 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4913+0+0 (secure 0 0 0) 0x7fd190086820 con 0x7fd1980cfe30 2024-09-10T10:04:59.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.654+0000 7fd197e00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd178067ef0 0x7fd17806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:59.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.654+0000 7fd197e00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd178067ef0 0x7fd17806a3b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd198150f90 tx=0x7fd188009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:59.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.654+0000 7fd18fe00640 1 -- 172.21.15.110:0/2656750435 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd1900566e0 con 0x7fd1980cfe30 2024-09-10T10:04:59.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.866+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd19806c0d0 con 0x7fd1980cfe30 2024-09-10T10:04:59.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.866+0000 7fd18fe00640 1 -- 172.21.15.110:0/2656750435 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7fd19005a9e0 con 0x7fd1980cfe30 2024-09-10T10:04:59.871 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:04:59.873 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:59.874 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:04:59.875 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.876 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:04:59.877 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:04:59.879 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:04:59.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.881 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:04:59.882 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:04:59.884 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:04:59.885 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:04:59.885 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.885 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:04:59.885 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:04:59.885 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:04:59.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd178067ef0 msgr2=0x7fd17806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd178067ef0 0x7fd17806a3b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd198150f90 tx=0x7fd188009040 comp rx=0 tx=0).stop 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cfe30 msgr2=0x7fd19814d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cfe30 0x7fd19814d0c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d560 tx=0x7fd19000da20 comp rx=0 tx=0).stop 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 shutdown_connections 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd178067ef0 0x7fd17806a3b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1980cfe30 0x7fd19814d0c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1980cd510 0x7fd198159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 --2- 172.21.15.110:0/2656750435 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd19806e090 0x7fd198148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 >> 172.21.15.110:0/2656750435 conn(0x7fd19806bc50 msgr2=0x7fd19815b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 shutdown_connections 2024-09-10T10:04:59.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:04:59.870+0000 7fd19ee00640 1 -- 172.21.15.110:0/2656750435 wait complete. 2024-09-10T10:04:59.893 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f389b600640 1 Processor -- start 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f389b600640 1 -- start start 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f389b600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894161f90 0x7f38940cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f389b600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38940706a0 con 0x7f3894162680 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f389b600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3894163060 con 0x7f38940739a0 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f389b600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3894163340 con 0x7f3894161f90 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3893e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3894162680 0x7f3894073290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53160/0 (socket says 172.21.15.110:53160) 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3893e00640 1 -- 172.21.15.110:0/653920135 learned_addr learned my addr 172.21.15.110:0/653920135 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3898c00640 1 --2- 172.21.15.110:0/653920135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894161f90 0x7f38940cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3893400640 1 -- 172.21.15.110:0/653920135 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566775917 0 0) 0x7f38940706a0 con 0x7f3894162680 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3893400640 1 -- 172.21.15.110:0/653920135 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f387c003530 con 0x7f3894162680 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3893400640 1 -- 172.21.15.110:0/653920135 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973325193 0 0) 0x7f3894163060 con 0x7f38940739a0 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3893400640 1 -- 172.21.15.110:0/653920135 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38940706a0 con 0x7f38940739a0 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3898c00640 1 -- 172.21.15.110:0/653920135 >> v1:172.21.15.145:6789/0 conn(0x7f38940739a0 legacy=0x7f389406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3898c00640 1 -- 172.21.15.110:0/653920135 >> v1:172.21.15.110:6789/0 conn(0x7f3894162680 legacy=0x7f3894073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.038+0000 7f3898c00640 1 -- 172.21.15.110:0/653920135 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38940cf130 con 0x7f3894161f90 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f3898c00640 1 --2- 172.21.15.110:0/653920135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894161f90 0x7f38940cf930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38880097f0 tx=0x7f388801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7f16f4857e8eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f3893400640 1 -- 172.21.15.110:0/653920135 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880b0070 con 0x7f3894161f90 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f3893400640 1 -- 172.21.15.110:0/653920135 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38880ab600 con 0x7f3894161f90 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f3893400640 1 -- 172.21.15.110:0/653920135 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880ab9b0 con 0x7f3894161f90 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f389b600640 1 -- 172.21.15.110:0/653920135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894161f90 msgr2=0x7f38940cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f389b600640 1 --2- 172.21.15.110:0/653920135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894161f90 0x7f38940cf930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38880097f0 tx=0x7f388801c4e0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f389b600640 1 -- 172.21.15.110:0/653920135 shutdown_connections 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f389b600640 1 --2- 172.21.15.110:0/653920135 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894161f90 0x7f38940cf930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f389b600640 1 -- 172.21.15.110:0/653920135 >> 172.21.15.110:0/653920135 conn(0x7f38940d25c0 msgr2=0x7f38940d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f389b600640 1 -- 172.21.15.110:0/653920135 shutdown_connections 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.042+0000 7f389b600640 1 -- 172.21.15.110:0/653920135 wait complete. 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f389b600640 1 Processor -- start 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f389b600640 1 -- start start 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f389b600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38940739a0 0x7f3894143b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f389b600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3894161f90 0x7f38941408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f389b600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894162680 0x7f38940cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f389b600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f38940d0080 con 0x7f38940739a0 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f389b600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38940741b0 con 0x7f3894161f90 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f389b600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f389406e050 con 0x7f3894162680 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3898c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38940739a0 0x7f3894143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3898c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38940739a0 0x7f3894143b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47504/0 (socket says 172.21.15.110:47504) 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3898c00640 1 -- 172.21.15.110:0/3833868159 learned_addr learned my addr 172.21.15.110:0/3833868159 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3893e00640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3894161f90 0x7f38941408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3898c00640 1 -- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894162680 msgr2=0x7f38940cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3899600640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894162680 0x7f38940cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3898c00640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894162680 0x7f38940cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3898c00640 1 -- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3894161f90 msgr2=0x7f38941408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3898c00640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3894161f90 0x7f38941408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3898c00640 1 -- 172.21.15.110:0/3833868159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3894140e30 con 0x7f38940739a0 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.046+0000 7f3893e00640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3894161f90 0x7f38941408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3898c00640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38940739a0 0x7f3894143b60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c10 tx=0x7f388801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3890c00640 1 -- 172.21.15.110:0/3833868159 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880b0050 con 0x7f38940739a0 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3890c00640 1 -- 172.21.15.110:0/3833868159 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38880b1040 con 0x7f38940739a0 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3890c00640 1 -- 172.21.15.110:0/3833868159 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888006e20 con 0x7f38940739a0 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f389b600640 1 -- 172.21.15.110:0/3833868159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38940cd2e0 con 0x7f38940739a0 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f389b600640 1 -- 172.21.15.110:0/3833868159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38940cbe50 con 0x7f38940739a0 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3886000640 1 -- 172.21.15.110:0/3833868159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38940cf930 con 0x7f38940739a0 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3890c00640 1 -- 172.21.15.110:0/3833868159 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f38880be030 con 0x7f38940739a0 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3890c00640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3874067ef0 0x7f387406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3890c00640 1 -- 172.21.15.110:0/3833868159 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4913+0+0 (secure 0 0 0) 0x7f388811ffb0 con 0x7f38940739a0 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.050+0000 7f3893e00640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3874067ef0 0x7f387406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.054+0000 7f3893e00640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3874067ef0 0x7f387406a3b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38941fb6a0 tx=0x7f3880009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.054+0000 7f3890c00640 1 -- 172.21.15.110:0/3833868159 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f38880c5060 con 0x7f38940739a0 2024-09-10T10:05:00.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 -- 172.21.15.110:0/3833868159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f38940d0a10 con 0x7f38940739a0 2024-09-10T10:05:00.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3890c00640 1 -- 172.21.15.110:0/3833868159 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v138) ==== 84+0+65 (secure 0 0 0) 0x7f38880160d0 con 0x7f38940739a0 2024-09-10T10:05:00.222 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:00.222 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:05:00.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:00.222 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:05:00.222 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:05:00.222 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:00.222 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:05:00.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 -- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3874067ef0 msgr2=0x7f387406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3874067ef0 0x7f387406a3b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38941fb6a0 tx=0x7f3880009040 comp rx=0 tx=0).stop 2024-09-10T10:05:00.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 -- 172.21.15.110:0/3833868159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38940739a0 msgr2=0x7f3894143b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38940739a0 0x7f3894143b60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c10 tx=0x7f388801ad20 comp rx=0 tx=0).stop 2024-09-10T10:05:00.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 -- 172.21.15.110:0/3833868159 shutdown_connections 2024-09-10T10:05:00.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3874067ef0 0x7f387406a3b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3894162680 0x7f38940cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3894161f90 0x7f38941408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 --2- 172.21.15.110:0/3833868159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38940739a0 0x7f3894143b60 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 -- 172.21.15.110:0/3833868159 >> 172.21.15.110:0/3833868159 conn(0x7f38940d25c0 msgr2=0x7f38940b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:00.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 -- 172.21.15.110:0/3833868159 shutdown_connections 2024-09-10T10:05:00.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.218+0000 7f3886000640 1 -- 172.21.15.110:0/3833868159 wait complete. 2024-09-10T10:05:00.245 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:05:00.245 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:05:00.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4489c00640 1 Processor -- start 2024-09-10T10:05:00.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4489c00640 1 -- start start 2024-09-10T10:05:00.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4489c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4484155970 0x7f44840cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4489c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f448415c560 0x7f448406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4489c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f448406f430 con 0x7f448406f010 2024-09-10T10:05:00.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4489c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f44840b4570 con 0x7f4484155970 2024-09-10T10:05:00.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4489c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4484155d50 con 0x7f448415c560 2024-09-10T10:05:00.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4483400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f448406f010 0x7f448415eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53172/0 (socket says 172.21.15.110:53172) 2024-09-10T10:05:00.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4483400640 1 -- 172.21.15.110:0/4182181175 learned_addr learned my addr 172.21.15.110:0/4182181175 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:00.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4482a00640 1 --2- 172.21.15.110:0/4182181175 >> v2:172.21.15.145:3300/0 conn(0x7f4484155970 0x7f44840cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4482000640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f448415c560 0x7f448406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3920934988 0 0) 0x7f448406f430 con 0x7f448406f010 2024-09-10T10:05:00.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4468003530 con 0x7f448406f010 2024-09-10T10:05:00.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2954004606 0 0) 0x7f4468003530 con 0x7f448406f010 2024-09-10T10:05:00.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f448415c560 msgr2=0x7f448406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:00.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f448415c560 0x7f448406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 >> v2:172.21.15.145:3300/0 conn(0x7f4484155970 msgr2=0x7f44840cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 --2- 172.21.15.110:0/4182181175 >> v2:172.21.15.145:3300/0 conn(0x7f4484155970 0x7f44840cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44840b34d0 con 0x7f448406f010 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f447c001a00 con 0x7f448406f010 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.382+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 >> v1:172.21.15.110:6789/0 conn(0x7f448406f010 legacy=0x7f448415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f448415c560 0x7f44680039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4468004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4468004d50 0x7f44680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4484155d50 con 0x7f4468003f40 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f446800b9d0 con 0x7f4468004d50 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f446800bba0 con 0x7f448415c560 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4482000640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4468004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4482a00640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f448415c560 0x7f44680039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4483400640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4468004d50 0x7f44680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4482000640 1 -- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f448415c560 msgr2=0x7f44680039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:00.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4482000640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f448415c560 0x7f44680039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4482000640 1 -- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4468004d50 msgr2=0x7f44680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:00.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4482000640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4468004d50 0x7f44680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4482000640 1 -- 172.21.15.110:0/4182181175 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4468003530 con 0x7f4468003f40 2024-09-10T10:05:00.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4483400640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4468004d50 0x7f44680051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:00.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4482000640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4468004810 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f4470009670 tx=0x7f447001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58ac1841b25d746 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:00.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700b4070 con 0x7f4468003f40 2024-09-10T10:05:00.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44700a7830 con 0x7f4468003f40 2024-09-10T10:05:00.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.386+0000 7f4481600640 1 -- 172.21.15.110:0/4182181175 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700096b0 con 0x7f4468003f40 2024-09-10T10:05:00.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- 172.21.15.110:0/4182181175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 msgr2=0x7f4468004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 --2- 172.21.15.110:0/4182181175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4468004810 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f4470009670 tx=0x7f447001ec50 comp rx=0 tx=0).stop 2024-09-10T10:05:00.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4483400640 1 -- 172.21.15.110:0/4182181175 reap_dead start 2024-09-10T10:05:00.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- 172.21.15.110:0/4182181175 shutdown_connections 2024-09-10T10:05:00.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- 172.21.15.110:0/4182181175 >> 172.21.15.110:0/4182181175 conn(0x7f448406bc50 msgr2=0x7f448406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:00.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- 172.21.15.110:0/4182181175 shutdown_connections 2024-09-10T10:05:00.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- 172.21.15.110:0/4182181175 wait complete. 2024-09-10T10:05:00.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 Processor -- start 2024-09-10T10:05:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- start start 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4484148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4484154580 0x7f4484149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4484156d30 0x7f44840ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44840cce00 con 0x7f4468003f40 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f44840b4050 con 0x7f4484156d30 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4489c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f448406d9c0 con 0x7f4484154580 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4484148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4484148f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47524/0 (socket says 172.21.15.110:47524) 2024-09-10T10:05:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 -- 172.21.15.110:0/1937500763 learned_addr learned my addr 172.21.15.110:0/1937500763 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 -- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4484154580 msgr2=0x7f4484149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4484154580 0x7f4484149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 -- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4484156d30 msgr2=0x7f44840ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4484156d30 0x7f44840ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 -- 172.21.15.110:0/1937500763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4484157980 con 0x7f4468003f40 2024-09-10T10:05:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f4482a00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4484148f10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f4474011660 tx=0x7f4474011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:00.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.390+0000 7f447aa00640 1 -- 172.21.15.110:0/1937500763 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447400f9c0 con 0x7f4468003f40 2024-09-10T10:05:00.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f447aa00640 1 -- 172.21.15.110:0/1937500763 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f447400fb60 con 0x7f4468003f40 2024-09-10T10:05:00.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44841576a0 con 0x7f4468003f40 2024-09-10T10:05:00.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f447aa00640 1 -- 172.21.15.110:0/1937500763 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474015460 con 0x7f4468003f40 2024-09-10T10:05:00.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448415dc80 con 0x7f4468003f40 2024-09-10T10:05:00.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44840cc8c0 con 0x7f4468003f40 2024-09-10T10:05:00.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f447aa00640 1 -- 172.21.15.110:0/1937500763 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f4474015600 con 0x7f4468003f40 2024-09-10T10:05:00.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f447aa00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4464067f20 0x7f446406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:00.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f447aa00640 1 -- 172.21.15.110:0/1937500763 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4913+0+0 (secure 0 0 0) 0x7f4474086f20 con 0x7f4468003f40 2024-09-10T10:05:00.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.394+0000 7f4482000640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4464067f20 0x7f446406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:00.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.398+0000 7f4482000640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4464067f20 0x7f446406a3e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f44700b1000 tx=0x7f4470004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:00.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.398+0000 7f447aa00640 1 -- 172.21.15.110:0/1937500763 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4484157980 con 0x7f4468003f40 2024-09-10T10:05:00.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:00.558+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f44840b2ab0 con 0x7f4468003f40 2024-09-10T10:05:01.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.478+0000 7f447aa00640 1 -- 172.21.15.110:0/1937500763 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v139) ==== 138+0+0 (secure 0 0 0) 0x7f4474019030 con 0x7f4468003f40 2024-09-10T10:05:01.483 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:05:01.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4464067f20 msgr2=0x7f446406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:01.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4464067f20 0x7f446406a3e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f44700b1000 tx=0x7f4470004740 comp rx=0 tx=0).stop 2024-09-10T10:05:01.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 msgr2=0x7f4484148f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:01.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4484148f10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f4474011660 tx=0x7f4474011b20 comp rx=0 tx=0).stop 2024-09-10T10:05:01.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 shutdown_connections 2024-09-10T10:05:01.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4464067f20 0x7f446406a3e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4484156d30 0x7f44840ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4484154580 0x7f4484149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 --2- 172.21.15.110:0/1937500763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4468003f40 0x7f4484148f10 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 >> 172.21.15.110:0/1937500763 conn(0x7f448406bc50 msgr2=0x7f448415bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:01.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 shutdown_connections 2024-09-10T10:05:01.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.482+0000 7f4489c00640 1 -- 172.21.15.110:0/1937500763 wait complete. 2024-09-10T10:05:01.514 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0aea200640 1 Processor -- start 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0aea200640 1 -- start start 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0aea200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 0x7f0ae40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0aea200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae4155d50 con 0x7f0ae415c560 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0aea200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae40b34d0 con 0x7f0ae406f1e0 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0aea200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40b4570 con 0x7f0ae4155970 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0ae415c560 0x7f0ae406ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53178/0 (socket says 172.21.15.110:53178) 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2a00640 1 -- 172.21.15.110:0/3935707799 learned_addr learned my addr 172.21.15.110:0/3935707799 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae3400640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 0x7f0ae40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642857821 0 0) 0x7f0ae4155d50 con 0x7f0ae415c560 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ac8003530 con 0x7f0ae415c560 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3946416605 0 0) 0x7f0ae40b34d0 con 0x7f0ae406f1e0 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ae4155d50 con 0x7f0ae406f1e0 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3485249353 0 0) 0x7f0ac8003530 con 0x7f0ae415c560 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 msgr2=0x7f0ae40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2000640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 0x7f0ae40cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.658+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 >> v1:172.21.15.145:6789/0 conn(0x7f0ae406f1e0 legacy=0x7f0ae40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae40b37b0 con 0x7f0ae415c560 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f0ad0001680 con 0x7f0ae415c560 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3400640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 0x7f0ae40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 >> v1:172.21.15.110:6789/0 conn(0x7f0ae415c560 legacy=0x7f0ae406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ac8003b80 0x7f0ac8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 0x7f0ac8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8004d50 0x7f0ac80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac800ba80 con 0x7f0ac8003b80 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac800bc30 con 0x7f0ac8004d50 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac800be30 con 0x7f0ae4155970 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3400640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ac8003b80 0x7f0ac8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2a00640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 0x7f0ac8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3e00640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8004d50 0x7f0ac80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3400640 1 -- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 msgr2=0x7f0ac8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3400640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae4155970 0x7f0ac8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3400640 1 -- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8004d50 msgr2=0x7f0ac80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3400640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8004d50 0x7f0ac80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3400640 1 -- 172.21.15.110:0/3935707799 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac8003530 con 0x7f0ac8003b80 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae3400640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ac8003b80 0x7f0ac8003f80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e350 tx=0x7f0ad401cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e3d7754920d76e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40b2070 con 0x7f0ac8003b80 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad40a5dc0 con 0x7f0ac8003b80 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.662+0000 7f0ae2000640 1 -- 172.21.15.110:0/3935707799 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4009de0 con 0x7f0ac8003b80 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- 172.21.15.110:0/3935707799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ac8003b80 msgr2=0x7f0ac8003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 --2- 172.21.15.110:0/3935707799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ac8003b80 0x7f0ac8003f80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e350 tx=0x7f0ad401cdd0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae3e00640 1 -- 172.21.15.110:0/3935707799 reap_dead start 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- 172.21.15.110:0/3935707799 shutdown_connections 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- 172.21.15.110:0/3935707799 >> 172.21.15.110:0/3935707799 conn(0x7f0ae406bc50 msgr2=0x7f0ae406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- 172.21.15.110:0/3935707799 shutdown_connections 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- 172.21.15.110:0/3935707799 wait complete. 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 Processor -- start 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- start start 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8003b80 0x7f0ae415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae4152f50 0x7f0ae4141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae41533d0 0x7f0ae41420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40cce50 con 0x7f0ae4152f50 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40cccd0 con 0x7f0ac8003b80 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0aea200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40cf340 con 0x7f0ae41533d0 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae4152f50 0x7f0ae4141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae4152f50 0x7f0ae4141b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47534/0 (socket says 172.21.15.110:47534) 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae2a00640 1 -- 172.21.15.110:0/562821338 learned_addr learned my addr 172.21.15.110:0/562821338 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae3400640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8003b80 0x7f0ae415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae2a00640 1 -- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae41533d0 msgr2=0x7f0ae41420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae2a00640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae41533d0 0x7f0ae41420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae2a00640 1 -- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8003b80 msgr2=0x7f0ae415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae2a00640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8003b80 0x7f0ae415f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae2a00640 1 -- 172.21.15.110:0/562821338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae41e82e0 con 0x7f0ae4152f50 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.666+0000 7f0ae3400640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8003b80 0x7f0ae415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0ae2a00640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae4152f50 0x7f0ae4141b80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00017f0 tx=0x7f0ad00074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0adb400640 1 -- 172.21.15.110:0/562821338 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000f7b0 con 0x7f0ae4152f50 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0adb400640 1 -- 172.21.15.110:0/562821338 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad0018070 con 0x7f0ae4152f50 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0adb400640 1 -- 172.21.15.110:0/562821338 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000fb70 con 0x7f0ae4152f50 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae413f6c0 con 0x7f0ae4152f50 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae413fc80 con 0x7f0ae4152f50 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae40ce420 con 0x7f0ae4152f50 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0adb400640 1 -- 172.21.15.110:0/562821338 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f0ad001d460 con 0x7f0ae4152f50 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0adb400640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0ac4067ef0 0x7f0ac406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.670+0000 7f0adb400640 1 -- 172.21.15.110:0/562821338 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5283+0+0 (secure 0 0 0) 0x7f0ad00867e0 con 0x7f0ae4152f50 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.674+0000 7f0ae3400640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0ac4067ef0 0x7f0ac406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.674+0000 7f0ae3400640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0ac4067ef0 0x7f0ac406a3b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40af000 tx=0x7f0ad4004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.678+0000 7f0adb400640 1 -- 172.21.15.110:0/562821338 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0ad0056bd0 con 0x7f0ae4152f50 2024-09-10T10:05:01.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:01.834+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0ae406c470 con 0x7f0ae4152f50 2024-09-10T10:05:02.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0adb400640 1 -- 172.21.15.110:0/562821338 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v140) ==== 170+0+0 (secure 0 0 0) 0x7f0ad00564d0 con 0x7f0ae4152f50 2024-09-10T10:05:02.490 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:05:02.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0ac4067ef0 msgr2=0x7f0ac406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:02.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0ac4067ef0 0x7f0ac406a3b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40af000 tx=0x7f0ad4004ac0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae4152f50 msgr2=0x7f0ae4141b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae4152f50 0x7f0ae4141b80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00017f0 tx=0x7f0ad00074f0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 shutdown_connections 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0ac4067ef0 0x7f0ac406a3b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0ae41533d0 0x7f0ae41420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ae4152f50 0x7f0ae4141b80 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 --2- 172.21.15.110:0/562821338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac8003b80 0x7f0ae415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 >> 172.21.15.110:0/562821338 conn(0x7f0ae406bc50 msgr2=0x7f0ae4072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 shutdown_connections 2024-09-10T10:05:02.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.486+0000 7f0aea200640 1 -- 172.21.15.110:0/562821338 wait complete. 2024-09-10T10:05:02.532 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:05:02.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.666+0000 7fa8f3200640 1 Processor -- start 2024-09-10T10:05:02.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.666+0000 7fa8f3200640 1 -- start start 2024-09-10T10:05:02.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f3200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8f40b6260 0x7fa8f415ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:02.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f3200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa8f40b6640 0x7fa8f40b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f3200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f415bde0 0x7fa8f415c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f3200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f415c790 con 0x7fa8f40b6640 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f3200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f406d770 con 0x7fa8f40b6260 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f3200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f415feb0 con 0x7fa8f415bde0 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa8f40b6640 0x7fa8f40b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa8f40b6640 0x7fa8f40b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47540/0 (socket says 172.21.15.110:47540) 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1400640 1 -- 172.21.15.110:0/990692782 learned_addr learned my addr 172.21.15.110:0/990692782 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1e00640 1 --2- 172.21.15.110:0/990692782 >> v2:172.21.15.145:3300/0 conn(0x7fa8f40b6260 0x7fa8f415ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f2800640 1 --2- 172.21.15.110:0/990692782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f415bde0 0x7fa8f415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1400640 1 -- 172.21.15.110:0/990692782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f415bde0 msgr2=0x7fa8f415c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1400640 1 --2- 172.21.15.110:0/990692782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f415bde0 0x7fa8f415c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1400640 1 -- 172.21.15.110:0/990692782 >> v2:172.21.15.145:3300/0 conn(0x7fa8f40b6260 msgr2=0x7fa8f415ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1400640 1 --2- 172.21.15.110:0/990692782 >> v2:172.21.15.145:3300/0 conn(0x7fa8f40b6260 0x7fa8f415ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.670+0000 7fa8f1400640 1 -- 172.21.15.110:0/990692782 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f415f340 con 0x7fa8f40b6640 2024-09-10T10:05:02.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f1400640 1 --2- 172.21.15.110:0/990692782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f40b56f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa8f406f060 tx=0x7fa8e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5440502d43a0146 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f0a00640 1 -- 172.21.15.110:0/990692782 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00aa5f0 con 0x7fa8f40b6640 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f0a00640 1 -- 172.21.15.110:0/990692782 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e00af070 con 0x7fa8f40b6640 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f0a00640 1 -- 172.21.15.110:0/990692782 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00aa9b0 con 0x7fa8f40b6640 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 -- 172.21.15.110:0/990692782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 msgr2=0x7fa8f40b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 --2- 172.21.15.110:0/990692782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f40b56f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa8f406f060 tx=0x7fa8e001c4d0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 -- 172.21.15.110:0/990692782 shutdown_connections 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 --2- 172.21.15.110:0/990692782 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f415bde0 0x7fa8f415c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 --2- 172.21.15.110:0/990692782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f40b56f0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 --2- 172.21.15.110:0/990692782 >> v2:172.21.15.145:3300/0 conn(0x7fa8f40b6260 0x7fa8f415ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 -- 172.21.15.110:0/990692782 >> 172.21.15.110:0/990692782 conn(0x7fa8f40626e0 msgr2=0x7fa8f4062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:02.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 -- 172.21.15.110:0/990692782 shutdown_connections 2024-09-10T10:05:02.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 -- 172.21.15.110:0/990692782 wait complete. 2024-09-10T10:05:02.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 Processor -- start 2024-09-10T10:05:02.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 -- start start 2024-09-10T10:05:02.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f40b6260 0x7fa8f4148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:02.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f4158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:02.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.674+0000 7fa8f3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8f415bde0 0x7fa8f406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:02.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f3200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4073a10 con 0x7fa8f40b6640 2024-09-10T10:05:02.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f415fbe0 con 0x7fa8f415bde0 2024-09-10T10:05:02.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f3200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f4073b90 con 0x7fa8f40b6260 2024-09-10T10:05:02.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f4158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:02.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f4158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47544/0 (socket says 172.21.15.110:47544) 2024-09-10T10:05:02.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 -- 172.21.15.110:0/2493076701 learned_addr learned my addr 172.21.15.110:0/2493076701 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1e00640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f40b6260 0x7fa8f4148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f2800640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8f415bde0 0x7fa8f406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 -- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f40b6260 msgr2=0x7fa8f4148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f40b6260 0x7fa8f4148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 -- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8f415bde0 msgr2=0x7fa8f406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8f415bde0 0x7fa8f406d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 -- 172.21.15.110:0/2493076701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f4149e80 con 0x7fa8f40b6640 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f1400640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f4158c00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa8e001ab80 tx=0x7fa8e0019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8ea000640 1 -- 172.21.15.110:0/2493076701 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0003af0 con 0x7fa8f40b6640 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8ea000640 1 -- 172.21.15.110:0/2493076701 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e0003c90 con 0x7fa8f40b6640 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8ea000640 1 -- 172.21.15.110:0/2493076701 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00b3630 con 0x7fa8f40b6640 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f414a160 con 0x7fa8f40b6640 2024-09-10T10:05:02.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f4148f90 con 0x7fa8f40b6640 2024-09-10T10:05:02.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f4149c00 con 0x7fa8f40b6640 2024-09-10T10:05:02.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.678+0000 7fa8ea000640 1 -- 172.21.15.110:0/2493076701 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fa8e00b37d0 con 0x7fa8f40b6640 2024-09-10T10:05:02.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.682+0000 7fa8ea000640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa8d4067ef0 0x7fa8d406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:02.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.682+0000 7fa8ea000640 1 -- 172.21.15.110:0/2493076701 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5665+0+0 (secure 0 0 0) 0x7fa8e0120c90 con 0x7fa8f40b6640 2024-09-10T10:05:02.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.682+0000 7fa8f1e00640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa8d4067ef0 0x7fa8d406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:02.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.682+0000 7fa8f1e00640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa8d4067ef0 0x7fa8d406a3b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40ce990 tx=0x7fa8e4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:02.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.686+0000 7fa8ea000640 1 -- 172.21.15.110:0/2493076701 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa8f4149c00 con 0x7fa8f40b6640 2024-09-10T10:05:02.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:02.874+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa8f406e6b0 con 0x7fa8f40b6640 2024-09-10T10:05:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.486+0000 7fa8ea000640 1 -- 172.21.15.110:0/2493076701 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v93) ==== 174+0+0 (secure 0 0 0) 0x7fa8e00160f0 con 0x7fa8f40b6640 2024-09-10T10:05:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 22 and data pool 23 2024-09-10T10:05:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa8d4067ef0 msgr2=0x7fa8d406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa8d4067ef0 0x7fa8d406a3b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40ce990 tx=0x7fa8e4009190 comp rx=0 tx=0).stop 2024-09-10T10:05:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 msgr2=0x7fa8f4158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f4158c00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa8e001ab80 tx=0x7fa8e0019270 comp rx=0 tx=0).stop 2024-09-10T10:05:03.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 shutdown_connections 2024-09-10T10:05:03.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa8d4067ef0 0x7fa8d406a3b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8f415bde0 0x7fa8f406d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8f40b6640 0x7fa8f4158c00 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 --2- 172.21.15.110:0/2493076701 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa8f40b6260 0x7fa8f4148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.490+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 >> 172.21.15.110:0/2493076701 conn(0x7fa8f40626e0 msgr2=0x7fa8f4062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:03.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.494+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 shutdown_connections 2024-09-10T10:05:03.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.494+0000 7fa8f3200640 1 -- 172.21.15.110:0/2493076701 wait complete. 2024-09-10T10:05:03.537 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:03.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0d7800640 1 Processor -- start 2024-09-10T10:05:03.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0d7800640 1 -- start start 2024-09-10T10:05:03.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0d7800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d0070a40 0x7fb0d00d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:03.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0d7800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d00d1140 con 0x7fb0d00d1d70 2024-09-10T10:05:03.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0d7800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d015ed10 con 0x7fb0d0070660 2024-09-10T10:05:03.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0d7800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d015fad0 con 0x7fb0d0070a40 2024-09-10T10:05:03.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0d5800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb0d00d1d70 0x7fb0d00d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53190/0 (socket says 172.21.15.110:53190) 2024-09-10T10:05:03.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0d5800640 1 -- 172.21.15.110:0/2561441234 learned_addr learned my addr 172.21.15.110:0/2561441234 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:03.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cfe00640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d0070a40 0x7fb0d00d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:03.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2715950373 0 0) 0x7fb0d00d1140 con 0x7fb0d00d1d70 2024-09-10T10:05:03.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0b4003530 con 0x7fb0d00d1d70 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1373210415 0 0) 0x7fb0d015ed10 con 0x7fb0d0070660 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0d00d1140 con 0x7fb0d0070660 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1752629340 0 0) 0x7fb0b4003530 con 0x7fb0d00d1d70 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d0070a40 msgr2=0x7fb0d00d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d0070a40 0x7fb0d00d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 >> v1:172.21.15.145:6789/0 conn(0x7fb0d0070660 legacy=0x7fb0d00d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d015eff0 con 0x7fb0d00d1d70 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fb0c8001a20 con 0x7fb0d00d1d70 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 >> v1:172.21.15.110:6789/0 conn(0x7fb0d00d1d70 legacy=0x7fb0d00d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4003b80 0x7fb0b4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d0070a40 0x7fb0b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.670+0000 7fb0cf400640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0b4004d50 0x7fb0b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:03.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d015fad0 con 0x7fb0b4004d50 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b400ba80 con 0x7fb0b4003b80 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b400bc80 con 0x7fb0d0070a40 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0d5800640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0b4004d50 0x7fb0b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0d4e00640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4003b80 0x7fb0b4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0cfe00640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d0070a40 0x7fb0b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0d5800640 1 -- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d0070a40 msgr2=0x7fb0b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0d5800640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d0070a40 0x7fb0b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0d5800640 1 -- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4003b80 msgr2=0x7fb0b4003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0d5800640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4003b80 0x7fb0b4003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0d5800640 1 -- 172.21.15.110:0/2561441234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b4003530 con 0x7fb0b4004d50 2024-09-10T10:05:03.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0d5800640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0b4004d50 0x7fb0b40051d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800a7d0 tx=0x7fb0c801fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c68e4c7e390b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:03.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800a430 con 0x7fb0b4004d50 2024-09-10T10:05:03.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c80aa670 con 0x7fb0b4004d50 2024-09-10T10:05:03.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.674+0000 7fb0cf400640 1 -- 172.21.15.110:0/2561441234 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80aaa60 con 0x7fb0b4004d50 2024-09-10T10:05:03.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- 172.21.15.110:0/2561441234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0b4004d50 msgr2=0x7fb0b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 --2- 172.21.15.110:0/2561441234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0b4004d50 0x7fb0b40051d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800a7d0 tx=0x7fb0c801fa20 comp rx=0 tx=0).stop 2024-09-10T10:05:03.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d5800640 1 -- 172.21.15.110:0/2561441234 reap_dead start 2024-09-10T10:05:03.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- 172.21.15.110:0/2561441234 shutdown_connections 2024-09-10T10:05:03.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- 172.21.15.110:0/2561441234 >> 172.21.15.110:0/2561441234 conn(0x7fb0d006cf60 msgr2=0x7fb0d006d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:03.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- 172.21.15.110:0/2561441234 shutdown_connections 2024-09-10T10:05:03.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- 172.21.15.110:0/2561441234 wait complete. 2024-09-10T10:05:03.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 Processor -- start 2024-09-10T10:05:03.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- start start 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4004d50 0x7fb0d01584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d0156630 0x7fb0d0154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d015af40 0x7fb0d015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d00d04f0 con 0x7fb0d0156630 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d015f640 con 0x7fb0b4004d50 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d7800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d00d0670 con 0x7fb0d015af40 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d0156630 0x7fb0d0154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d0156630 0x7fb0d0154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47552/0 (socket says 172.21.15.110:47552) 2024-09-10T10:05:03.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 -- 172.21.15.110:0/3582767033 learned_addr learned my addr 172.21.15.110:0/3582767033 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:03.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d4e00640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4004d50 0x7fb0d01584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:03.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0d5800640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d015af40 0x7fb0d015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:03.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 -- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d015af40 msgr2=0x7fb0d015b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d015af40 0x7fb0d015b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 -- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4004d50 msgr2=0x7fb0d01584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4004d50 0x7fb0d01584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 -- 172.21.15.110:0/3582767033 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d0156b10 con 0x7fb0d0156630 2024-09-10T10:05:03.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.678+0000 7fb0cfe00640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d0156630 0x7fb0d0154650 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0119d0 tx=0x7fb0bc011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:03.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0ccc00640 1 -- 172.21.15.110:0/3582767033 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc00fa70 con 0x7fb0d0156630 2024-09-10T10:05:03.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0ccc00640 1 -- 172.21.15.110:0/3582767033 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0bc00fc10 con 0x7fb0d0156630 2024-09-10T10:05:03.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0ccc00640 1 -- 172.21.15.110:0/3582767033 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc014440 con 0x7fb0d0156630 2024-09-10T10:05:03.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d0156df0 con 0x7fb0d0156630 2024-09-10T10:05:03.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d01573b0 con 0x7fb0d0156630 2024-09-10T10:05:03.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d00d1660 con 0x7fb0d0156630 2024-09-10T10:05:03.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0ccc00640 1 -- 172.21.15.110:0/3582767033 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fb0bc012040 con 0x7fb0d0156630 2024-09-10T10:05:03.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0ccc00640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb0b0067ef0 0x7fb0b006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:03.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0ccc00640 1 -- 172.21.15.110:0/3582767033 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5765+0+0 (secure 0 0 0) 0x7fb0bc086df0 con 0x7fb0d0156630 2024-09-10T10:05:03.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.682+0000 7fb0d4e00640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb0b0067ef0 0x7fb0b006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:03.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.686+0000 7fb0d4e00640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb0b0067ef0 0x7fb0b006a3b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb0d01f7670 tx=0x7fb0c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:03.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.686+0000 7fb0ccc00640 1 -- 172.21.15.110:0/3582767033 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb0bc019030 con 0x7fb0d0156630 2024-09-10T10:05:03.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.846+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0d00b4b40 con 0x7fb0d0156630 2024-09-10T10:05:03.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.846+0000 7fb0ccc00640 1 -- 172.21.15.110:0/3582767033 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16869 (secure 0 0 0) 0x7fb0bc057030 con 0x7fb0d0156630 2024-09-10T10:05:03.853 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:03.853 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":141,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:02.880406+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:00.607363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:01.886645+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb0b0067ef0 msgr2=0x7fb0b006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb0b0067ef0 0x7fb0b006a3b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb0d01f7670 tx=0x7fb0c0009040 comp rx=0 tx=0).stop 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d0156630 msgr2=0x7fb0d0154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d0156630 0x7fb0d0154650 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0119d0 tx=0x7fb0bc011dd0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 shutdown_connections 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb0b0067ef0 0x7fb0b006a3b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb0d015af40 0x7fb0d015b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d0156630 0x7fb0d0154650 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 --2- 172.21.15.110:0/3582767033 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0b4004d50 0x7fb0d01584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 >> 172.21.15.110:0/3582767033 conn(0x7fb0d006cf60 msgr2=0x7fb0d00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 shutdown_connections 2024-09-10T10:05:03.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:03.850+0000 7fb0d7800640 1 -- 172.21.15.110:0/3582767033 wait complete. 2024-09-10T10:05:03.874 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:04.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd868200640 1 Processor -- start 2024-09-10T10:05:04.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd868200640 1 -- start start 2024-09-10T10:05:04.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd868200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8600713d0 0x7fd8600b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd868200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8600d0e40 con 0x7fd8600cefb0 2024-09-10T10:05:04.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd868200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8600b7960 con 0x7fd860070f50 2024-09-10T10:05:04.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd868200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd8600d10e0 con 0x7fd8600713d0 2024-09-10T10:05:04.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd865800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fd8600cefb0 0x7fd8600cf390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53200/0 (socket says 172.21.15.110:53200) 2024-09-10T10:05:04.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd865800640 1 -- 172.21.15.110:0/1835369385 learned_addr learned my addr 172.21.15.110:0/1835369385 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2149549611 0 0) 0x7fd8600d0e40 con 0x7fd8600cefb0 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd848003530 con 0x7fd8600cefb0 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd866200640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8600713d0 0x7fd8600b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4181168247 0 0) 0x7fd848003530 con 0x7fd8600cefb0 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8600713d0 msgr2=0x7fd8600b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8600713d0 0x7fd8600b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 >> v1:172.21.15.145:6789/0 conn(0x7fd860070f50 legacy=0x7fd8600b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8600d13c0 con 0x7fd8600cefb0 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fd854001a00 con 0x7fd8600cefb0 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.006+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 >> v1:172.21.15.110:6789/0 conn(0x7fd8600cefb0 legacy=0x7fd8600cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8480039f0 0x7fd848003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd848004810 0x7fd848004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8600713d0 0x7fd848005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8600d10e0 con 0x7fd848004810 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800bb00 con 0x7fd8480039f0 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd84800bd00 con 0x7fd8600713d0 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd864e00640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd848004810 0x7fd848004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd865800640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8480039f0 0x7fd848003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd866200640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8600713d0 0x7fd848005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd864e00640 1 -- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8600713d0 msgr2=0x7fd848005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd864e00640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8600713d0 0x7fd848005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd864e00640 1 -- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8480039f0 msgr2=0x7fd848003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd864e00640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8480039f0 0x7fd848003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd864e00640 1 -- 172.21.15.110:0/1835369385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd848003530 con 0x7fd848004810 2024-09-10T10:05:04.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd864e00640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd848004810 0x7fd848004c90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd84c001780 tx=0x7fd84c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd77066b2cf0b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:04.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0013a0 con 0x7fd848004810 2024-09-10T10:05:04.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84c0052f0 con 0x7fd848004810 2024-09-10T10:05:04.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.010+0000 7fd85fe00640 1 -- 172.21.15.110:0/1835369385 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c003840 con 0x7fd848004810 2024-09-10T10:05:04.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- 172.21.15.110:0/1835369385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd848004810 msgr2=0x7fd848004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 --2- 172.21.15.110:0/1835369385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd848004810 0x7fd848004c90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd84c001780 tx=0x7fd84c01c3f0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 -- 172.21.15.110:0/1835369385 reap_dead start 2024-09-10T10:05:04.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- 172.21.15.110:0/1835369385 shutdown_connections 2024-09-10T10:05:04.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- 172.21.15.110:0/1835369385 >> 172.21.15.110:0/1835369385 conn(0x7fd8600b4290 msgr2=0x7fd8600b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:04.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- 172.21.15.110:0/1835369385 shutdown_connections 2024-09-10T10:05:04.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- 172.21.15.110:0/1835369385 wait complete. 2024-09-10T10:05:04.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 Processor -- start 2024-09-10T10:05:04.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- start start 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd848004810 0x7fd860149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd86015a150 0x7fd86015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd86014ac00 0x7fd86014b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601682b0 con 0x7fd86014ac00 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd86015cbe0 con 0x7fd86015a150 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd868200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd860168430 con 0x7fd848004810 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd86014ac00 0x7fd86014b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd86014ac00 0x7fd86014b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47572/0 (socket says 172.21.15.110:47572) 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 -- 172.21.15.110:0/2647866709 learned_addr learned my addr 172.21.15.110:0/2647866709 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd864e00640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd86015a150 0x7fd86015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd865800640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd848004810 0x7fd860149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 -- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd848004810 msgr2=0x7fd860149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd848004810 0x7fd860149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 -- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd86015a150 msgr2=0x7fd86015a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd86015a150 0x7fd86015a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd866200640 1 -- 172.21.15.110:0/2647866709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86015abd0 con 0x7fd86014ac00 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd864e00640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd86015a150 0x7fd86015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.014+0000 7fd865800640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd848004810 0x7fd860149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd866200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd86014ac00 0x7fd86014b080 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd848004790 tx=0x7fd858011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd85d600640 1 -- 172.21.15.110:0/2647866709 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858004240 con 0x7fd86014ac00 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd85d600640 1 -- 172.21.15.110:0/2647866709 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85801b070 con 0x7fd86014ac00 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd85d600640 1 -- 172.21.15.110:0/2647866709 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85800f710 con 0x7fd86014ac00 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86014f1a0 con 0x7fd86014ac00 2024-09-10T10:05:04.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86014f760 con 0x7fd86014ac00 2024-09-10T10:05:04.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8600b5b10 con 0x7fd86014ac00 2024-09-10T10:05:04.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd85d600640 1 -- 172.21.15.110:0/2647866709 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fd858012040 con 0x7fd86014ac00 2024-09-10T10:05:04.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd85d600640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd840067f20 0x7fd84006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd85d600640 1 -- 172.21.15.110:0/2647866709 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5765+0+0 (secure 0 0 0) 0x7fd858086670 con 0x7fd86014ac00 2024-09-10T10:05:04.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.018+0000 7fd865800640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd840067f20 0x7fd84006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.022+0000 7fd865800640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd840067f20 0x7fd84006a3e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd854004c30 tx=0x7fd854006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:04.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.022+0000 7fd85d600640 1 -- 172.21.15.110:0/2647866709 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd858056830 con 0x7fd86014ac00 2024-09-10T10:05:04.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.182+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8600b3520 con 0x7fd86014ac00 2024-09-10T10:05:04.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.182+0000 7fd85d600640 1 -- 172.21.15.110:0/2647866709 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16869 (secure 0 0 0) 0x7fd858056130 con 0x7fd86014ac00 2024-09-10T10:05:04.189 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:04.189 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":141,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:02.880406+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:00.607363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:01.886645+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:04.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd840067f20 msgr2=0x7fd84006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd840067f20 0x7fd84006a3e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd854004c30 tx=0x7fd854006040 comp rx=0 tx=0).stop 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd86014ac00 msgr2=0x7fd86014b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd86014ac00 0x7fd86014b080 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd848004790 tx=0x7fd858011cf0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 shutdown_connections 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd840067f20 0x7fd84006a3e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd86014ac00 0x7fd86014b080 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd86015a150 0x7fd86015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 --2- 172.21.15.110:0/2647866709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd848004810 0x7fd860149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 >> 172.21.15.110:0/2647866709 conn(0x7fd8600b4290 msgr2=0x7fd8600b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 shutdown_connections 2024-09-10T10:05:04.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.186+0000 7fd868200640 1 -- 172.21.15.110:0/2647866709 wait complete. 2024-09-10T10:05:04.209 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:05:04.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.342+0000 7ff351e00640 1 Processor -- start 2024-09-10T10:05:04.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.342+0000 7ff351e00640 1 -- start start 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 0x7ff34c070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff34c148000 0x7ff34c06c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff34c0ae840 0x7ff34c0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff34c06c570 con 0x7ff34c0ae840 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff34c0b2690 con 0x7ff34c148000 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c0b2970 con 0x7ff34c06bc50 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff34c0ae840 0x7ff34c0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff34c0ae840 0x7ff34c0aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47574/0 (socket says 172.21.15.110:47574) 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351400640 1 -- 172.21.15.110:0/1021179065 learned_addr learned my addr 172.21.15.110:0/1021179065 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff350a00640 1 --2- 172.21.15.110:0/1021179065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 0x7ff34c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff34be00640 1 --2- 172.21.15.110:0/1021179065 >> v2:172.21.15.145:3300/0 conn(0x7ff34c148000 0x7ff34c06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351400640 1 -- 172.21.15.110:0/1021179065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 msgr2=0x7ff34c070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:04.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351400640 1 --2- 172.21.15.110:0/1021179065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 0x7ff34c070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351400640 1 -- 172.21.15.110:0/1021179065 >> v2:172.21.15.145:3300/0 conn(0x7ff34c148000 msgr2=0x7ff34c06c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:04.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351400640 1 --2- 172.21.15.110:0/1021179065 >> v2:172.21.15.145:3300/0 conn(0x7ff34c148000 0x7ff34c06c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff351400640 1 -- 172.21.15.110:0/1021179065 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0b2c50 con 0x7ff34c0ae840 2024-09-10T10:05:04.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.346+0000 7ff34be00640 1 --2- 172.21.15.110:0/1021179065 >> v2:172.21.15.145:3300/0 conn(0x7ff34c148000 0x7ff34c06c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:04.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351400640 1 --2- 172.21.15.110:0/1021179065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c0aecf0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff34c074060 tx=0x7ff34401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5495c67117d88d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:04.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff34b400640 1 -- 172.21.15.110:0/1021179065 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3440a5660 con 0x7ff34c0ae840 2024-09-10T10:05:04.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff34b400640 1 -- 172.21.15.110:0/1021179065 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3440a5800 con 0x7ff34c0ae840 2024-09-10T10:05:04.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff34b400640 1 -- 172.21.15.110:0/1021179065 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3440a5bd0 con 0x7ff34c0ae840 2024-09-10T10:05:04.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 -- 172.21.15.110:0/1021179065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 msgr2=0x7ff34c0aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 --2- 172.21.15.110:0/1021179065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c0aecf0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff34c074060 tx=0x7ff34401c8d0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 -- 172.21.15.110:0/1021179065 shutdown_connections 2024-09-10T10:05:04.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 --2- 172.21.15.110:0/1021179065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c0aecf0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 --2- 172.21.15.110:0/1021179065 >> v2:172.21.15.145:3300/0 conn(0x7ff34c148000 0x7ff34c06c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 --2- 172.21.15.110:0/1021179065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 0x7ff34c070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 -- 172.21.15.110:0/1021179065 >> 172.21.15.110:0/1021179065 conn(0x7ff34c06f7e0 msgr2=0x7ff34c06fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:04.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 -- 172.21.15.110:0/1021179065 shutdown_connections 2024-09-10T10:05:04.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 -- 172.21.15.110:0/1021179065 wait complete. 2024-09-10T10:05:04.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 Processor -- start 2024-09-10T10:05:04.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 -- start start 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.350+0000 7ff351e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 0x7ff34c15ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c140a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff34c148000 0x7ff34c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c0b5e40 con 0x7ff34c0ae840 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c0b5cc0 con 0x7ff34c148000 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c0b5fc0 con 0x7ff34c06bc50 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c140a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c140a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47580/0 (socket says 172.21.15.110:47580) 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 -- 172.21.15.110:0/357746221 learned_addr learned my addr 172.21.15.110:0/357746221 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff350a00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 0x7ff34c15ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351400640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff34c148000 0x7ff34c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 -- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 msgr2=0x7ff34c15ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 0x7ff34c15ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 -- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff34c148000 msgr2=0x7ff34c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff34c148000 0x7ff34c15d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 -- 172.21.15.110:0/357746221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c1667f0 con 0x7ff34c0ae840 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff34be00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c140a20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff33800d7d0 tx=0x7ff33800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff348c00640 1 -- 172.21.15.110:0/357746221 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338018020 con 0x7ff34c0ae840 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff348c00640 1 -- 172.21.15.110:0/357746221 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3380096a0 con 0x7ff34c0ae840 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff348c00640 1 -- 172.21.15.110:0/357746221 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338009a90 con 0x7ff34c0ae840 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c166a90 con 0x7ff34c0ae840 2024-09-10T10:05:04.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c167050 con 0x7ff34c0ae840 2024-09-10T10:05:04.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c0aecf0 con 0x7ff34c0ae840 2024-09-10T10:05:04.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff348c00640 1 -- 172.21.15.110:0/357746221 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7ff33801b020 con 0x7ff34c0ae840 2024-09-10T10:05:04.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.354+0000 7ff348c00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff32c067ef0 0x7ff32c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:04.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.358+0000 7ff348c00640 1 -- 172.21.15.110:0/357746221 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5765+0+0 (secure 0 0 0) 0x7ff338086d60 con 0x7ff34c0ae840 2024-09-10T10:05:04.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.358+0000 7ff350a00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff32c067ef0 0x7ff32c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:04.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.358+0000 7ff350a00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff32c067ef0 0x7ff32c06a3b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009a70 tx=0x7ff33c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:04.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.362+0000 7ff348c00640 1 -- 172.21.15.110:0/357746221 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff338022020 con 0x7ff34c0ae840 2024-09-10T10:05:04.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:04.554+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff34c0d1280 con 0x7ff34c0ae840 2024-09-10T10:05:05.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff348c00640 1 -- 172.21.15.110:0/357746221 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v96) ==== 118+0+0 (secure 0 0 0) 0x7ff338056f70 con 0x7ff34c0ae840 2024-09-10T10:05:05.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff32c067ef0 msgr2=0x7ff32c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff32c067ef0 0x7ff32c06a3b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009a70 tx=0x7ff33c008040 comp rx=0 tx=0).stop 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 msgr2=0x7ff34c140a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c140a20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff33800d7d0 tx=0x7ff33800dc90 comp rx=0 tx=0).stop 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 shutdown_connections 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff32c067ef0 0x7ff32c06a3b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff34c148000 0x7ff34c15d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff34c0ae840 0x7ff34c140a20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 --2- 172.21.15.110:0/357746221 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff34c06bc50 0x7ff34c15ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 >> 172.21.15.110:0/357746221 conn(0x7ff34c06f7e0 msgr2=0x7ff34c074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:05.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 shutdown_connections 2024-09-10T10:05:05.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.518+0000 7ff351e00640 1 -- 172.21.15.110:0/357746221 wait complete. 2024-09-10T10:05:05.546 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:05:05.546 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:05:05.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.682+0000 7f9bcd600640 1 Processor -- start 2024-09-10T10:05:05.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.682+0000 7f9bcd600640 1 -- start start 2024-09-10T10:05:05.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bcd600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9bc80cf0a0 0x7f9bc80cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bcd600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9bc813f9c0 0x7f9bc813fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bcd600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc80cfa00 con 0x7f9bc80b2df0 2024-09-10T10:05:05.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bcd600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc813f740 con 0x7f9bc80cf0a0 2024-09-10T10:05:05.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bcd600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc80c6010 con 0x7f9bc813f9c0 2024-09-10T10:05:05.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc7e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9bc80b2df0 0x7f9bc80c5900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53210/0 (socket says 172.21.15.110:53210) 2024-09-10T10:05:05.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc7e00640 1 -- 172.21.15.110:0/2050561145 learned_addr learned my addr 172.21.15.110:0/2050561145 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:05.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc7400640 1 --2- 172.21.15.110:0/2050561145 >> v2:172.21.15.110:3300/0 conn(0x7f9bc80cf0a0 0x7f9bc80cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bccc00640 1 --2- 172.21.15.110:0/2050561145 >> v2:172.21.15.145:3300/0 conn(0x7f9bc813f9c0 0x7f9bc813fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc6a00640 1 -- 172.21.15.110:0/2050561145 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3319894672 0 0) 0x7f9bc80cfa00 con 0x7f9bc80b2df0 2024-09-10T10:05:05.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc6a00640 1 -- 172.21.15.110:0/2050561145 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bac003530 con 0x7f9bc80b2df0 2024-09-10T10:05:05.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc7400640 1 -- 172.21.15.110:0/2050561145 >> v2:172.21.15.145:3300/0 conn(0x7f9bc813f9c0 msgr2=0x7f9bc813fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:05.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc7400640 1 --2- 172.21.15.110:0/2050561145 >> v2:172.21.15.145:3300/0 conn(0x7f9bc813f9c0 0x7f9bc813fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc7400640 1 -- 172.21.15.110:0/2050561145 >> v1:172.21.15.110:6789/0 conn(0x7f9bc80b2df0 legacy=0x7f9bc80c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.686+0000 7f9bc7400640 1 -- 172.21.15.110:0/2050561145 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc815c9a0 con 0x7f9bc80cf0a0 2024-09-10T10:05:05.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bc7400640 1 --2- 172.21.15.110:0/2050561145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80cf0a0 0x7f9bc80cf4c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9bc815fdc0 tx=0x7f9bb401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d488566937143a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:05.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bc6a00640 1 -- 172.21.15.110:0/2050561145 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40b0070 con 0x7f9bc80cf0a0 2024-09-10T10:05:05.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bc6a00640 1 -- 172.21.15.110:0/2050561145 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb40ab600 con 0x7f9bc80cf0a0 2024-09-10T10:05:05.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bc6a00640 1 -- 172.21.15.110:0/2050561145 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40ab9b0 con 0x7f9bc80cf0a0 2024-09-10T10:05:05.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- 172.21.15.110:0/2050561145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80cf0a0 msgr2=0x7f9bc80cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 --2- 172.21.15.110:0/2050561145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80cf0a0 0x7f9bc80cf4c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9bc815fdc0 tx=0x7f9bb401c480 comp rx=0 tx=0).stop 2024-09-10T10:05:05.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- 172.21.15.110:0/2050561145 shutdown_connections 2024-09-10T10:05:05.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 --2- 172.21.15.110:0/2050561145 >> v2:172.21.15.145:3300/0 conn(0x7f9bc813f9c0 0x7f9bc813fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 --2- 172.21.15.110:0/2050561145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80cf0a0 0x7f9bc80cf4c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- 172.21.15.110:0/2050561145 >> 172.21.15.110:0/2050561145 conn(0x7f9bc806d710 msgr2=0x7f9bc80b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:05.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- 172.21.15.110:0/2050561145 shutdown_connections 2024-09-10T10:05:05.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- 172.21.15.110:0/2050561145 wait complete. 2024-09-10T10:05:05.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 Processor -- start 2024-09-10T10:05:05.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- start start 2024-09-10T10:05:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80b2df0 0x7f9bc8071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc80cf0a0 0x7f9bc8158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc813f9c0 0x7f9bc8139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc80b60d0 con 0x7f9bc80b2df0 2024-09-10T10:05:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc80b5f50 con 0x7f9bc813f9c0 2024-09-10T10:05:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bcd600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc8169010 con 0x7f9bc80cf0a0 2024-09-10T10:05:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80b2df0 0x7f9bc8071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80b2df0 0x7f9bc8071400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47608/0 (socket says 172.21.15.110:47608) 2024-09-10T10:05:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.690+0000 7f9bc7e00640 1 -- 172.21.15.110:0/246336390 learned_addr learned my addr 172.21.15.110:0/246336390 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:05.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bc7e00640 1 -- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc80cf0a0 msgr2=0x7f9bc8158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bc7e00640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc80cf0a0 0x7f9bc8158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bc7e00640 1 -- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc813f9c0 msgr2=0x7f9bc8139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bccc00640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc813f9c0 0x7f9bc8139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bc7e00640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc813f9c0 0x7f9bc8139770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bc7e00640 1 -- 172.21.15.110:0/246336390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc8138a20 con 0x7f9bc80b2df0 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bccc00640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc813f9c0 0x7f9bc8139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bc7e00640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80b2df0 0x7f9bc8071400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800c3c0 tx=0x7f9bb800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bbfe00640 1 -- 172.21.15.110:0/246336390 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8017070 con 0x7f9bc80b2df0 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bbfe00640 1 -- 172.21.15.110:0/246336390 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb8005cf0 con 0x7f9bc80b2df0 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bbfe00640 1 -- 172.21.15.110:0/246336390 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb80072f0 con 0x7f9bc80b2df0 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc8138c90 con 0x7f9bc80b2df0 2024-09-10T10:05:05.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc8139250 con 0x7f9bc80b2df0 2024-09-10T10:05:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc813fe30 con 0x7f9bc80b2df0 2024-09-10T10:05:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bbfe00640 1 -- 172.21.15.110:0/246336390 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f9bb801a070 con 0x7f9bc80b2df0 2024-09-10T10:05:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.694+0000 7f9bbfe00640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9ba8067ef0 0x7f9ba806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.698+0000 7f9bbfe00640 1 -- 172.21.15.110:0/246336390 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f9bb8085c90 con 0x7f9bc80b2df0 2024-09-10T10:05:05.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.698+0000 7f9bc7400640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9ba8067ef0 0x7f9ba806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.698+0000 7f9bc7400640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9ba8067ef0 0x7f9ba806a3b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40ad000 tx=0x7f9bb4004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:05.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.702+0000 7f9bbfe00640 1 -- 172.21.15.110:0/246336390 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9bb8055e10 con 0x7f9bc80b2df0 2024-09-10T10:05:05.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:05.890+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9bc8141830 con 0x7f9bc80b2df0 2024-09-10T10:05:06.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.522+0000 7f9bbfe00640 1 -- 172.21.15.110:0/246336390 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v97) ==== 122+0+0 (secure 0 0 0) 0x7f9bb8055710 con 0x7f9bc80b2df0 2024-09-10T10:05:06.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9ba8067ef0 msgr2=0x7f9ba806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9ba8067ef0 0x7f9ba806a3b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40ad000 tx=0x7f9bb4004740 comp rx=0 tx=0).stop 2024-09-10T10:05:06.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80b2df0 msgr2=0x7f9bc8071400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80b2df0 0x7f9bc8071400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800c3c0 tx=0x7f9bb800c880 comp rx=0 tx=0).stop 2024-09-10T10:05:06.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 shutdown_connections 2024-09-10T10:05:06.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9ba8067ef0 0x7f9ba806a3b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9bc813f9c0 0x7f9bc8139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9bc80cf0a0 0x7f9bc8158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 --2- 172.21.15.110:0/246336390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc80b2df0 0x7f9bc8071400 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 >> 172.21.15.110:0/246336390 conn(0x7f9bc806d710 msgr2=0x7f9bc80b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:06.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 shutdown_connections 2024-09-10T10:05:06.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.526+0000 7f9bcd600640 1 -- 172.21.15.110:0/246336390 wait complete. 2024-09-10T10:05:06.551 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:05:06.551 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:06.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8be00640 1 Processor -- start 2024-09-10T10:05:06.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8be00640 1 -- start start 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcb8c155680 0x7fcb8c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcb8c0cf2e0 0x7fcb8c0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8be00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8c0cd7e0 con 0x7fcb8c0cd260 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8be00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c0cfc70 con 0x7fcb8c155680 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8be00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c0cdac0 con 0x7fcb8c0cf2e0 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcb8c155680 0x7fcb8c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcb8c155680 0x7fcb8c0ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47614/0 (socket says 172.21.15.110:47614) 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.682+0000 7fcb8aa00640 1 -- 172.21.15.110:0/2378312642 learned_addr learned my addr 172.21.15.110:0/2378312642 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb8b400640 1 --2- 172.21.15.110:0/2378312642 >> v2:172.21.15.145:3300/0 conn(0x7fcb8c0cf2e0 0x7fcb8c0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb8aa00640 1 -- 172.21.15.110:0/2378312642 >> v2:172.21.15.145:3300/0 conn(0x7fcb8c0cf2e0 msgr2=0x7fcb8c0cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb8aa00640 1 --2- 172.21.15.110:0/2378312642 >> v2:172.21.15.145:3300/0 conn(0x7fcb8c0cf2e0 0x7fcb8c0cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb8aa00640 1 -- 172.21.15.110:0/2378312642 >> v1:172.21.15.145:6789/0 conn(0x7fcb8c0cd260 legacy=0x7fcb8c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb8aa00640 1 -- 172.21.15.110:0/2378312642 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c072770 con 0x7fcb8c155680 2024-09-10T10:05:06.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb8aa00640 1 --2- 172.21.15.110:0/2378312642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c0ccd20 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c071c80 tx=0x7fcb7c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1261c06d55b9fca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:06.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb89600640 1 -- 172.21.15.110:0/2378312642 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0aa670 con 0x7fcb8c155680 2024-09-10T10:05:06.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb89600640 1 -- 172.21.15.110:0/2378312642 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb7c0af070 con 0x7fcb8c155680 2024-09-10T10:05:06.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.686+0000 7fcb89600640 1 -- 172.21.15.110:0/2378312642 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0aaa30 con 0x7fcb8c155680 2024-09-10T10:05:06.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- 172.21.15.110:0/2378312642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 msgr2=0x7fcb8c0ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 --2- 172.21.15.110:0/2378312642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c0ccd20 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c071c80 tx=0x7fcb7c01c4f0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- 172.21.15.110:0/2378312642 shutdown_connections 2024-09-10T10:05:06.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 --2- 172.21.15.110:0/2378312642 >> v2:172.21.15.145:3300/0 conn(0x7fcb8c0cf2e0 0x7fcb8c0cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 --2- 172.21.15.110:0/2378312642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c0ccd20 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- 172.21.15.110:0/2378312642 >> 172.21.15.110:0/2378312642 conn(0x7fcb8c06d0f0 msgr2=0x7fcb8c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:06.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- 172.21.15.110:0/2378312642 shutdown_connections 2024-09-10T10:05:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- 172.21.15.110:0/2378312642 wait complete. 2024-09-10T10:05:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 Processor -- start 2024-09-10T10:05:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- start start 2024-09-10T10:05:06.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb8c0cd260 0x7fcb8c162040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb8c0cf2e0 0x7fcb8c15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c155be0 con 0x7fcb8c155680 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c155a60 con 0x7fcb8c0cf2e0 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8be00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8c073680 con 0x7fcb8c0cd260 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47622/0 (socket says 172.21.15.110:47622) 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8b400640 1 -- 172.21.15.110:0/1728404115 learned_addr learned my addr 172.21.15.110:0/1728404115 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8aa00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb8c0cd260 0x7fcb8c162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8a000640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb8c0cf2e0 0x7fcb8c15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8b400640 1 -- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb8c0cd260 msgr2=0x7fcb8c162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8b400640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb8c0cd260 0x7fcb8c162040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8b400640 1 -- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb8c0cf2e0 msgr2=0x7fcb8c15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8b400640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb8c0cf2e0 0x7fcb8c15d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.690+0000 7fcb8b400640 1 -- 172.21.15.110:0/1728404115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c151020 con 0x7fcb8c155680 2024-09-10T10:05:06.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb8b400640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c073980 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400efc0 tx=0x7fcb8400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:06.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb82a00640 1 -- 172.21.15.110:0/1728404115 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84018020 con 0x7fcb8c155680 2024-09-10T10:05:06.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb82a00640 1 -- 172.21.15.110:0/1728404115 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb84014480 con 0x7fcb8c155680 2024-09-10T10:05:06.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb82a00640 1 -- 172.21.15.110:0/1728404115 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84014870 con 0x7fcb8c155680 2024-09-10T10:05:06.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c151300 con 0x7fcb8c155680 2024-09-10T10:05:06.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c1518c0 con 0x7fcb8c155680 2024-09-10T10:05:06.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c15b540 con 0x7fcb8c155680 2024-09-10T10:05:06.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb82a00640 1 -- 172.21.15.110:0/1728404115 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fcb8401b020 con 0x7fcb8c155680 2024-09-10T10:05:06.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb82a00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcb6c067ef0 0x7fcb6c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.694+0000 7fcb82a00640 1 -- 172.21.15.110:0/1728404115 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fcb840871b0 con 0x7fcb8c155680 2024-09-10T10:05:06.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.698+0000 7fcb8aa00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcb6c067ef0 0x7fcb6c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.698+0000 7fcb8aa00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcb6c067ef0 0x7fcb6c06a3b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0ae000 tx=0x7fcb7c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:06.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.698+0000 7fcb82a00640 1 -- 172.21.15.110:0/1728404115 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcb84022070 con 0x7fcb8c155680 2024-09-10T10:05:06.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb8c1529e0 con 0x7fcb8c155680 2024-09-10T10:05:06.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb82a00640 1 -- 172.21.15.110:0/1728404115 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5239 (secure 0 0 0) 0x7fcb84010040 con 0x7fcb8c155680 2024-09-10T10:05:06.895 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:06.895 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":97,"btime":"2024-09-10T10:05:06:522341+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2050751760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2050751760},{"type":"v1","addr":"172.21.15.110:6835","nonce":2050751760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6743,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/4094794865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4094794865},{"type":"v1","addr":"172.21.15.145:6834","nonce":4094794865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2883879805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2883879805},{"type":"v1","addr":"172.21.15.110:6834","nonce":2883879805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:02.880808+0000","modified":"2024-09-10T10:05:05.895746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1325843861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1325843861},{"type":"v1","addr":"172.21.15.145:6835","nonce":1325843861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8178,"qdb_cluster":[8178]},"id":11}]} 2024-09-10T10:05:06.897 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 97 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcb6c067ef0 msgr2=0x7fcb6c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcb6c067ef0 0x7fcb6c06a3b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0ae000 tx=0x7fcb7c0a7000 comp rx=0 tx=0).stop 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 msgr2=0x7fcb8c073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c073980 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400efc0 tx=0x7fcb8400c490 comp rx=0 tx=0).stop 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 shutdown_connections 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcb6c067ef0 0x7fcb6c06a3b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb8c155680 0x7fcb8c073980 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb8c0cf2e0 0x7fcb8c15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 --2- 172.21.15.110:0/1728404115 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcb8c0cd260 0x7fcb8c162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 >> 172.21.15.110:0/1728404115 conn(0x7fcb8c06d0f0 msgr2=0x7fcb8c159460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 shutdown_connections 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:06.890+0000 7fcb8be00640 1 -- 172.21.15.110:0/1728404115 wait complete. 2024-09-10T10:05:06.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:07.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.046+0000 7f2b57400640 1 Processor -- start 2024-09-10T10:05:07.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.046+0000 7f2b57400640 1 -- start start 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b57400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2b5015c560 0x7f2b5006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b57400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b5006f430 con 0x7f2b5006f010 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b57400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b50155d50 con 0x7f2b50155970 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b57400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2b500b4570 con 0x7f2b5015c560 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b4fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2b5015c560 0x7f2b5006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b55400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2b5006f010 0x7f2b500ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53220/0 (socket says 172.21.15.110:53220) 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b55400640 1 -- 172.21.15.110:0/837211666 learned_addr learned my addr 172.21.15.110:0/837211666 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b4fe00640 1 -- 172.21.15.110:0/837211666 >> v1:172.21.15.145:6789/0 conn(0x7f2b50155970 legacy=0x7f2b500cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:07.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b4fe00640 1 -- 172.21.15.110:0/837211666 >> v1:172.21.15.110:6789/0 conn(0x7f2b5006f010 legacy=0x7f2b500ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.050+0000 7f2b4fe00640 1 -- 172.21.15.110:0/837211666 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b500b34d0 con 0x7f2b5015c560 2024-09-10T10:05:07.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b4fe00640 1 --2- 172.21.15.110:0/837211666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b5015c560 0x7f2b5006ead0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2b380097c0 tx=0x7f2b3801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568e13673900f1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b4f400640 1 -- 172.21.15.110:0/837211666 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380ab430 con 0x7f2b5015c560 2024-09-10T10:05:07.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b4f400640 1 -- 172.21.15.110:0/837211666 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b380b0070 con 0x7f2b5015c560 2024-09-10T10:05:07.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b4f400640 1 -- 172.21.15.110:0/837211666 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380ab7f0 con 0x7f2b5015c560 2024-09-10T10:05:07.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 -- 172.21.15.110:0/837211666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b5015c560 msgr2=0x7f2b5006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 --2- 172.21.15.110:0/837211666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b5015c560 0x7f2b5006ead0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2b380097c0 tx=0x7f2b3801c430 comp rx=0 tx=0).stop 2024-09-10T10:05:07.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 -- 172.21.15.110:0/837211666 shutdown_connections 2024-09-10T10:05:07.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 --2- 172.21.15.110:0/837211666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b5015c560 0x7f2b5006ead0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 -- 172.21.15.110:0/837211666 >> 172.21.15.110:0/837211666 conn(0x7f2b5006bc50 msgr2=0x7f2b5006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:07.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 -- 172.21.15.110:0/837211666 shutdown_connections 2024-09-10T10:05:07.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 -- 172.21.15.110:0/837211666 wait complete. 2024-09-10T10:05:07.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 Processor -- start 2024-09-10T10:05:07.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 -- start start 2024-09-10T10:05:07.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.054+0000 7f2b57400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b5006f010 0x7f2b5013d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b57400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b50155970 0x7f2b500adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b57400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b5015c560 0x7f2b50154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b57400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500cf340 con 0x7f2b50155970 2024-09-10T10:05:07.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b57400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500b4050 con 0x7f2b5006f010 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b57400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2b500cf4c0 con 0x7f2b5015c560 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b50155970 0x7f2b500adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b50155970 0x7f2b500adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47650/0 (socket says 172.21.15.110:47650) 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 -- 172.21.15.110:0/690871270 learned_addr learned my addr 172.21.15.110:0/690871270 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b55400640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b5015c560 0x7f2b50154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b54a00640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b5006f010 0x7f2b5013d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 -- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b5015c560 msgr2=0x7f2b50154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b5015c560 0x7f2b50154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 -- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b5006f010 msgr2=0x7f2b5013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b5006f010 0x7f2b5013d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 -- 172.21.15.110:0/690871270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5013d7c0 con 0x7f2b50155970 2024-09-10T10:05:07.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4fe00640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b50155970 0x7f2b500adc90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2b38019040 tx=0x7f2b3801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4cc00640 1 -- 172.21.15.110:0/690871270 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380bf610 con 0x7f2b50155970 2024-09-10T10:05:07.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4cc00640 1 -- 172.21.15.110:0/690871270 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b380bf7d0 con 0x7f2b50155970 2024-09-10T10:05:07.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b500ae710 con 0x7f2b50155970 2024-09-10T10:05:07.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4cc00640 1 -- 172.21.15.110:0/690871270 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380bfbc0 con 0x7f2b50155970 2024-09-10T10:05:07.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b500aecd0 con 0x7f2b50155970 2024-09-10T10:05:07.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b5006ead0 con 0x7f2b50155970 2024-09-10T10:05:07.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.058+0000 7f2b4cc00640 1 -- 172.21.15.110:0/690871270 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f2b380bd020 con 0x7f2b50155970 2024-09-10T10:05:07.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.062+0000 7f2b4cc00640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2b2c067ef0 0x7f2b2c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.062+0000 7f2b4cc00640 1 -- 172.21.15.110:0/690871270 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f2b380a8030 con 0x7f2b50155970 2024-09-10T10:05:07.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.062+0000 7f2b54a00640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2b2c067ef0 0x7f2b2c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.062+0000 7f2b54a00640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2b2c067ef0 0x7f2b2c06a3b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b50166360 tx=0x7f2b40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.066+0000 7f2b4cc00640 1 -- 172.21.15.110:0/690871270 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2b380c4060 con 0x7f2b50155970 2024-09-10T10:05:07.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.222+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b500ccaa0 con 0x7f2b50155970 2024-09-10T10:05:07.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.222+0000 7f2b4cc00640 1 -- 172.21.15.110:0/690871270 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16856 (secure 0 0 0) 0x7f2b38016050 con 0x7f2b50155970 2024-09-10T10:05:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:07.229 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":142,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:04.492566+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:00.607363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:01.886645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2b2c067ef0 msgr2=0x7f2b2c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2b2c067ef0 0x7f2b2c06a3b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b50166360 tx=0x7f2b40009040 comp rx=0 tx=0).stop 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b50155970 msgr2=0x7f2b500adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b50155970 0x7f2b500adc90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2b38019040 tx=0x7f2b3801ac70 comp rx=0 tx=0).stop 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 shutdown_connections 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2b2c067ef0 0x7f2b2c06a3b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b5015c560 0x7f2b50154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b50155970 0x7f2b500adc90 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 --2- 172.21.15.110:0/690871270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b5006f010 0x7f2b5013d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 >> 172.21.15.110:0/690871270 conn(0x7f2b5006bc50 msgr2=0x7f2b5006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 shutdown_connections 2024-09-10T10:05:07.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.226+0000 7f2b57400640 1 -- 172.21.15.110:0/690871270 wait complete. 2024-09-10T10:05:07.249 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:05:07.250 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:05:07.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.378+0000 7fe7d3800640 1 Processor -- start 2024-09-10T10:05:07.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.378+0000 7fe7d3800640 1 -- start start 2024-09-10T10:05:07.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d3800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7cc155970 0x7fe7cc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d3800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 0x7fe7cc0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d3800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7cc155d50 con 0x7fe7cc15c560 2024-09-10T10:05:07.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d3800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe7cc0b4570 con 0x7fe7cc155970 2024-09-10T10:05:07.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d3800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc0b34d0 con 0x7fe7cc06f1e0 2024-09-10T10:05:07.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d0e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7cc155970 0x7fe7cc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d0e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7cc155970 0x7fe7cc0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:37918/0 (socket says 172.21.15.110:37918) 2024-09-10T10:05:07.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d0e00640 1 -- 172.21.15.110:0/3282758375 learned_addr learned my addr 172.21.15.110:0/3282758375 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:07.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d1800640 1 --2- 172.21.15.110:0/3282758375 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 0x7fe7cc0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7cbe00640 1 --1- 172.21.15.110:0/3282758375 >> v1:172.21.15.145:6789/0 conn(0x7fe7cc15c560 0x7fe7cc06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:52822/0 (socket says 172.21.15.110:52822) 2024-09-10T10:05:07.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d0e00640 1 -- 172.21.15.110:0/3282758375 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 msgr2=0x7fe7cc0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:07.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d0e00640 1 --2- 172.21.15.110:0/3282758375 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 0x7fe7cc0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d0e00640 1 -- 172.21.15.110:0/3282758375 >> v1:172.21.15.145:6789/0 conn(0x7fe7cc15c560 legacy=0x7fe7cc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7d0e00640 1 -- 172.21.15.110:0/3282758375 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc0b37b0 con 0x7fe7cc155970 2024-09-10T10:05:07.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.382+0000 7fe7cb400640 1 -- 172.21.15.110:0/3282758375 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2554935374 0 0) 0x7fe7cc155d50 con 0x7fe7cc15c560 2024-09-10T10:05:07.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d0e00640 1 --2- 172.21.15.110:0/3282758375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc155970 0x7fe7cc0cc8c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00bb10 tx=0x7fe7bc01ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49bd1fe217fd2f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7cb400640 1 -- 172.21.15.110:0/3282758375 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0b4070 con 0x7fe7cc155970 2024-09-10T10:05:07.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7cb400640 1 -- 172.21.15.110:0/3282758375 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7bc0a7cd0 con 0x7fe7cc155970 2024-09-10T10:05:07.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7cb400640 1 -- 172.21.15.110:0/3282758375 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0af660 con 0x7fe7cc155970 2024-09-10T10:05:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d3800640 1 -- 172.21.15.110:0/3282758375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc155970 msgr2=0x7fe7cc0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d3800640 1 --2- 172.21.15.110:0/3282758375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc155970 0x7fe7cc0cc8c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00bb10 tx=0x7fe7bc01ec00 comp rx=0 tx=0).stop 2024-09-10T10:05:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d3800640 1 -- 172.21.15.110:0/3282758375 shutdown_connections 2024-09-10T10:05:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d3800640 1 --2- 172.21.15.110:0/3282758375 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 0x7fe7cc0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d3800640 1 --2- 172.21.15.110:0/3282758375 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc155970 0x7fe7cc0cc8c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d3800640 1 -- 172.21.15.110:0/3282758375 >> 172.21.15.110:0/3282758375 conn(0x7fe7cc06bc50 msgr2=0x7fe7cc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d3800640 1 -- 172.21.15.110:0/3282758375 shutdown_connections 2024-09-10T10:05:07.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.386+0000 7fe7d3800640 1 -- 172.21.15.110:0/3282758375 wait complete. 2024-09-10T10:05:07.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 Processor -- start 2024-09-10T10:05:07.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 -- start start 2024-09-10T10:05:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 0x7fe7cc15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc155970 0x7fe7cc143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c560 0x7fe7cc1f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc0cf170 con 0x7fe7cc155970 2024-09-10T10:05:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc0cce00 con 0x7fe7cc15c560 2024-09-10T10:05:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc0cf2f0 con 0x7fe7cc06f1e0 2024-09-10T10:05:07.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc155970 0x7fe7cc143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc155970 0x7fe7cc143c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47656/0 (socket says 172.21.15.110:47656) 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 -- 172.21.15.110:0/255782628 learned_addr learned my addr 172.21.15.110:0/255782628 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d1800640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c560 0x7fe7cc1f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d0e00640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 0x7fe7cc15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 -- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 msgr2=0x7fe7cc15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 0x7fe7cc15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 -- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c560 msgr2=0x7fe7cc1f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c560 0x7fe7cc1f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 -- 172.21.15.110:0/255782628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc1579b0 con 0x7fe7cc155970 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7cbe00640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc155970 0x7fe7cc143c00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c450 tx=0x7fe7b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7c8c00640 1 -- 172.21.15.110:0/255782628 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8018070 con 0x7fe7cc155970 2024-09-10T10:05:07.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7c8c00640 1 -- 172.21.15.110:0/255782628 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b80074a0 con 0x7fe7cc155970 2024-09-10T10:05:07.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7c8c00640 1 -- 172.21.15.110:0/255782628 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8004a30 con 0x7fe7cc155970 2024-09-10T10:05:07.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc157c90 con 0x7fe7cc155970 2024-09-10T10:05:07.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.390+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc158250 con 0x7fe7cc155970 2024-09-10T10:05:07.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.394+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc0ce420 con 0x7fe7cc155970 2024-09-10T10:05:07.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.394+0000 7fe7c8c00640 1 -- 172.21.15.110:0/255782628 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fe7b8004bd0 con 0x7fe7cc155970 2024-09-10T10:05:07.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.394+0000 7fe7c8c00640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac067ef0 0x7fe7ac06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.394+0000 7fe7c8c00640 1 -- 172.21.15.110:0/255782628 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fe7b80859b0 con 0x7fe7cc155970 2024-09-10T10:05:07.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.394+0000 7fe7d0e00640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac067ef0 0x7fe7ac06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.394+0000 7fe7d0e00640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac067ef0 0x7fe7ac06a3b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0b1000 tx=0x7fe7bc0a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.398+0000 7fe7c8c00640 1 -- 172.21.15.110:0/255782628 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe7b80554e0 con 0x7fe7cc155970 2024-09-10T10:05:07.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.554+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7cc158690 con 0x7fe7ac067ef0 2024-09-10T10:05:07.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.558+0000 7fe7c8c00640 1 -- 172.21.15.110:0/255782628 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240294 (secure 0 0 0) 0x7fe7cc158690 con 0x7fe7ac067ef0 2024-09-10T10:05:07.567 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:07.568 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:05:07.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac067ef0 msgr2=0x7fe7ac06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac067ef0 0x7fe7ac06a3b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0b1000 tx=0x7fe7bc0a9070 comp rx=0 tx=0).stop 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc155970 msgr2=0x7fe7cc143c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc155970 0x7fe7cc143c00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c450 tx=0x7fe7b800c910 comp rx=0 tx=0).stop 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 shutdown_connections 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac067ef0 0x7fe7ac06a3b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c560 0x7fe7cc1f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc155970 0x7fe7cc143c00 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 --2- 172.21.15.110:0/255782628 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc06f1e0 0x7fe7cc15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 >> 172.21.15.110:0/255782628 conn(0x7fe7cc06bc50 msgr2=0x7fe7cc0ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 shutdown_connections 2024-09-10T10:05:07.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.562+0000 7fe7d3800640 1 -- 172.21.15.110:0/255782628 wait complete. 2024-09-10T10:05:07.587 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-10T10:05:05.732324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276716,"kb_used_data":14712,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754698004,"statfs":{"total":773094113280,"available":772810756096,"internally_reserved":0,"allocated":15065088,"data_stored":7273956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":268095020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.246462"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.493614+0000","last_change":"2024-09-10T10:05:01.493614+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.493614+0000","last_undegraded":"2024-09-10T10:05:01.493614+0000","last_fullsized":"2024-09-10T10:05:01.493614+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.489534+0000","last_change":"2024-09-10T10:05:01.489534+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.489534+0000","last_undegraded":"2024-09-10T10:05:01.489534+0000","last_fullsized":"2024-09-10T10:05:01.489534+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.488711+0000","last_change":"2024-09-10T10:05:01.488711+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.488711+0000","last_undegraded":"2024-09-10T10:05:01.488711+0000","last_fullsized":"2024-09-10T10:05:01.488711+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.488751+0000","last_change":"2024-09-10T10:05:01.488751+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.488751+0000","last_undegraded":"2024-09-10T10:05:01.488751+0000","last_fullsized":"2024-09-10T10:05:01.488751+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.492563+0000","last_change":"2024-09-10T10:05:01.492563+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.492563+0000","last_undegraded":"2024-09-10T10:05:01.492563+0000","last_fullsized":"2024-09-10T10:05:01.492563+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.488637+0000","last_change":"2024-09-10T10:05:01.488637+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.488637+0000","last_undegraded":"2024-09-10T10:05:01.488637+0000","last_fullsized":"2024-09-10T10:05:01.488637+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.488569+0000","last_change":"2024-09-10T10:05:01.488569+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.488569+0000","last_undegraded":"2024-09-10T10:05:01.488569+0000","last_fullsized":"2024-09-10T10:05:01.488569+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.491561+0000","last_change":"2024-09-10T10:05:01.491561+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.491561+0000","last_undegraded":"2024-09-10T10:05:01.491561+0000","last_fullsized":"2024-09-10T10:05:01.491561+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.493175+0000","last_change":"2024-09-10T10:05:01.493175+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.493175+0000","last_undegraded":"2024-09-10T10:05:01.493175+0000","last_fullsized":"2024-09-10T10:05:01.493175+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.493135+0000","last_change":"2024-09-10T10:05:01.493135+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.493135+0000","last_undegraded":"2024-09-10T10:05:01.493135+0000","last_fullsized":"2024-09-10T10:05:01.493135+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.489649+0000","last_change":"2024-09-10T10:05:01.489649+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.489649+0000","last_undegraded":"2024-09-10T10:05:01.489649+0000","last_fullsized":"2024-09-10T10:05:01.489649+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.488246+0000","last_change":"2024-09-10T10:05:01.488246+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.488246+0000","last_undegraded":"2024-09-10T10:05:01.488246+0000","last_fullsized":"2024-09-10T10:05:01.488246+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.490310+0000","last_change":"2024-09-10T10:05:01.490310+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.490310+0000","last_undegraded":"2024-09-10T10:05:01.490310+0000","last_fullsized":"2024-09-10T10:05:01.490310+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.488256+0000","last_change":"2024-09-10T10:05:01.488256+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.488256+0000","last_undegraded":"2024-09-10T10:05:01.488256+0000","last_fullsized":"2024-09-10T10:05:01.488256+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.490188+0000","last_change":"2024-09-10T10:05:01.490188+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.490188+0000","last_undegraded":"2024-09-10T10:05:01.490188+0000","last_fullsized":"2024-09-10T10:05:01.490188+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.492507+0000","last_change":"2024-09-10T10:05:01.492507+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.492507+0000","last_undegraded":"2024-09-10T10:05:01.492507+0000","last_fullsized":"2024-09-10T10:05:01.492507+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.493342+0000","last_change":"2024-09-10T10:05:01.493342+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.493342+0000","last_undegraded":"2024-09-10T10:05:01.493342+0000","last_fullsized":"2024-09-10T10:05:01.493342+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.493203+0000","last_change":"2024-09-10T10:05:01.493203+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.493203+0000","last_undegraded":"2024-09-10T10:05:01.493203+0000","last_fullsized":"2024-09-10T10:05:01.493203+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.493140+0000","last_change":"2024-09-10T10:05:01.493140+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.493140+0000","last_undegraded":"2024-09-10T10:05:01.493140+0000","last_fullsized":"2024-09-10T10:05:01.493140+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.487689+0000","last_change":"2024-09-10T10:05:01.487689+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.487689+0000","last_undegraded":"2024-09-10T10:05:01.487689+0000","last_fullsized":"2024-09-10T10:05:01.487689+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.493112+0000","last_change":"2024-09-10T10:05:01.493112+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.493112+0000","last_undegraded":"2024-09-10T10:05:01.493112+0000","last_fullsized":"2024-09-10T10:05:01.493112+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.487704+0000","last_change":"2024-09-10T10:05:01.487704+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.487704+0000","last_undegraded":"2024-09-10T10:05:01.487704+0000","last_fullsized":"2024-09-10T10:05:01.487704+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.493153+0000","last_change":"2024-09-10T10:05:01.493153+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.493153+0000","last_undegraded":"2024-09-10T10:05:01.493153+0000","last_fullsized":"2024-09-10T10:05:01.493153+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.490394+0000","last_change":"2024-09-10T10:05:01.490394+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.490394+0000","last_undegraded":"2024-09-10T10:05:01.490394+0000","last_fullsized":"2024-09-10T10:05:01.490394+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.492542+0000","last_change":"2024-09-10T10:05:01.492542+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.492542+0000","last_undegraded":"2024-09-10T10:05:01.492542+0000","last_fullsized":"2024-09-10T10:05:01.492542+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.488156+0000","last_change":"2024-09-10T10:05:01.488156+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.488156+0000","last_undegraded":"2024-09-10T10:05:01.488156+0000","last_fullsized":"2024-09-10T10:05:01.488156+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.490984+0000","last_change":"2024-09-10T10:05:01.490984+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.490984+0000","last_undegraded":"2024-09-10T10:05:01.490984+0000","last_fullsized":"2024-09-10T10:05:01.490984+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.489246+0000","last_change":"2024-09-10T10:05:01.489246+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.489246+0000","last_undegraded":"2024-09-10T10:05:01.489246+0000","last_fullsized":"2024-09-10T10:05:01.489246+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.487637+0000","last_change":"2024-09-10T10:05:01.487637+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.487637+0000","last_undegraded":"2024-09-10T10:05:01.487637+0000","last_fullsized":"2024-09-10T10:05:01.487637+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.488023+0000","last_change":"2024-09-10T10:05:01.488023+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.488023+0000","last_undegraded":"2024-09-10T10:05:01.488023+0000","last_fullsized":"2024-09-10T10:05:01.488023+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.489261+0000","last_change":"2024-09-10T10:05:01.489261+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.489261+0000","last_undegraded":"2024-09-10T10:05:01.489261+0000","last_fullsized":"2024-09-10T10:05:01.489261+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:05:02.484247+0000","last_change":"2024-09-10T10:05:02.484247+0000","last_active":"2024-09-10T10:05:02.484247+0000","last_peered":"2024-09-10T10:05:02.484247+0000","last_clean":"2024-09-10T10:05:02.484247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:02.484247+0000","last_undegraded":"2024-09-10T10:05:02.484247+0000","last_fullsized":"2024-09-10T10:05:02.484247+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-10T10:05:01.489240+0000","last_change":"2024-09-10T10:05:01.489240+0000","last_active":"2024-09-10T10:05:01.481493+0000","last_peered":"2024-09-10T10:05:01.481493+0000","last_clean":"2024-09-10T10:05:01.481493+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:01.489240+0000","last_undegraded":"2024-09-10T10:05:01.489240+0000","last_fullsized":"2024-09-10T10:05:01.489240+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":423,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-10T10:05:01.494432+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:05:01.494432+0000","last_peered":"2024-09-10T10:05:01.494432+0000","last_clean":"2024-09-10T10:05:01.494432+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:05:01.494432+0000","last_undegraded":"2024-09-10T10:05:01.494432+0000","last_fullsized":"2024-09-10T10:05:01.494432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1349084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1740800,"data_stored":764589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509524,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1728512,"data_stored":758716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1740800,"data_stored":764589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542230,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":758716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":758716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509524,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32892,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338948,"statfs":{"total":96636764160,"available":96603082752,"internally_reserved":0,"allocated":1740800,"data_stored":764589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37636,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334204,"statfs":{"total":96636764160,"available":96598224896,"internally_reserved":0,"allocated":2334720,"data_stored":1354957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:05:07.592 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:05:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.758+0000 7ffa8ac00640 1 Processor -- start 2024-09-10T10:05:07.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.758+0000 7ffa8ac00640 1 -- start start 2024-09-10T10:05:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa8ac00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa840ccb60 0x7ffa84072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa8ac00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ffa84140c30 0x7ffa8415b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa8ac00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa840ccf40 con 0x7ffa84072670 2024-09-10T10:05:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa8ac00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ffa84072ad0 con 0x7ffa84140c30 2024-09-10T10:05:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa8ac00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffa841412a0 con 0x7ffa840ccb60 2024-09-10T10:05:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa88c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ffa84140c30 0x7ffa8415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa88c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ffa84140c30 0x7ffa8415b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47664/0 (socket says 172.21.15.110:47664) 2024-09-10T10:05:07.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa88c00640 1 -- 172.21.15.110:0/3350059935 learned_addr learned my addr 172.21.15.110:0/3350059935 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:07.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa88c00640 1 -- 172.21.15.110:0/3350059935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa840ccb60 msgr2=0x7ffa84072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:07.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa83e00640 1 --2- 172.21.15.110:0/3350059935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa840ccb60 0x7ffa84072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa88c00640 1 --2- 172.21.15.110:0/3350059935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa840ccb60 0x7ffa84072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa88c00640 1 -- 172.21.15.110:0/3350059935 >> v1:172.21.15.145:6789/0 conn(0x7ffa84072670 legacy=0x7ffa84140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:07.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa88c00640 1 -- 172.21.15.110:0/3350059935 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8415e760 con 0x7ffa84140c30 2024-09-10T10:05:07.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.762+0000 7ffa83e00640 1 --2- 172.21.15.110:0/3350059935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa840ccb60 0x7ffa84072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa88c00640 1 --2- 172.21.15.110:0/3350059935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa8415b120 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009870 tx=0x7ffa7c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff9c9489034ec8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa82a00640 1 -- 172.21.15.110:0/3350059935 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0a6040 con 0x7ffa84140c30 2024-09-10T10:05:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa82a00640 1 -- 172.21.15.110:0/3350059935 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c0af070 con 0x7ffa84140c30 2024-09-10T10:05:07.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa82a00640 1 -- 172.21.15.110:0/3350059935 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0aa860 con 0x7ffa84140c30 2024-09-10T10:05:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- 172.21.15.110:0/3350059935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 msgr2=0x7ffa8415b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/3350059935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa8415b120 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009870 tx=0x7ffa7c01c520 comp rx=0 tx=0).stop 2024-09-10T10:05:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- 172.21.15.110:0/3350059935 shutdown_connections 2024-09-10T10:05:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/3350059935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa8415b120 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/3350059935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa840ccb60 0x7ffa84072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- 172.21.15.110:0/3350059935 >> 172.21.15.110:0/3350059935 conn(0x7ffa8406bc50 msgr2=0x7ffa8406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- 172.21.15.110:0/3350059935 shutdown_connections 2024-09-10T10:05:07.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- 172.21.15.110:0/3350059935 wait complete. 2024-09-10T10:05:07.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 Processor -- start 2024-09-10T10:05:07.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- start start 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa84072670 0x7ffa840b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa840ccb60 0x7ffa8415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa84154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8415f520 con 0x7ffa84140c30 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8415be70 con 0x7ffa840ccb60 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.766+0000 7ffa8ac00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840b06a0 con 0x7ffa84072670 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa84154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa84154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47674/0 (socket says 172.21.15.110:47674) 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 -- 172.21.15.110:0/626058291 learned_addr learned my addr 172.21.15.110:0/626058291 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa83e00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa84072670 0x7ffa840b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa83400640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa840ccb60 0x7ffa8415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 -- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa84072670 msgr2=0x7ffa840b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa84072670 0x7ffa840b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 -- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa840ccb60 msgr2=0x7ffa8415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa840ccb60 0x7ffa8415dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 -- 172.21.15.110:0/626058291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa841539f0 con 0x7ffa84140c30 2024-09-10T10:05:07.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa88c00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa84154a70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c01abd0 tx=0x7ffa7c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa7be00640 1 -- 172.21.15.110:0/626058291 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0bd3f0 con 0x7ffa84140c30 2024-09-10T10:05:07.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa7be00640 1 -- 172.21.15.110:0/626058291 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c0bd590 con 0x7ffa84140c30 2024-09-10T10:05:07.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa7be00640 1 -- 172.21.15.110:0/626058291 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0bd960 con 0x7ffa84140c30 2024-09-10T10:05:07.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa84153fb0 con 0x7ffa84140c30 2024-09-10T10:05:07.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa84154570 con 0x7ffa84140c30 2024-09-10T10:05:07.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa84072130 con 0x7ffa84140c30 2024-09-10T10:05:07.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa7be00640 1 -- 172.21.15.110:0/626058291 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7ffa7c0bdb00 con 0x7ffa84140c30 2024-09-10T10:05:07.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.770+0000 7ffa7be00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa64067ef0 0x7ffa6406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:07.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.774+0000 7ffa7be00640 1 -- 172.21.15.110:0/626058291 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7ffa7c120be0 con 0x7ffa84140c30 2024-09-10T10:05:07.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.774+0000 7ffa83e00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa64067ef0 0x7ffa6406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:07.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.774+0000 7ffa83e00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa64067ef0 0x7ffa6406a3b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400d620 tx=0x7ffa74007810 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:07.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.778+0000 7ffa7be00640 1 -- 172.21.15.110:0/626058291 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ffa7c016110 con 0x7ffa84140c30 2024-09-10T10:05:07.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.930+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa84072130 con 0x7ffa64067ef0 2024-09-10T10:05:07.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.938+0000 7ffa7be00640 1 -- 172.21.15.110:0/626058291 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252483 (secure 0 0 0) 0x7ffa84072130 con 0x7ffa64067ef0 2024-09-10T10:05:07.943 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:07.945 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:05:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.938+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa64067ef0 msgr2=0x7ffa6406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.938+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa64067ef0 0x7ffa6406a3b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400d620 tx=0x7ffa74007810 comp rx=0 tx=0).stop 2024-09-10T10:05:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.938+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 msgr2=0x7ffa84154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.938+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa84154a70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c01abd0 tx=0x7ffa7c01ad60 comp rx=0 tx=0).stop 2024-09-10T10:05:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.942+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 shutdown_connections 2024-09-10T10:05:07.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.942+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa64067ef0 0x7ffa6406a3b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.942+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa84140c30 0x7ffa84154a70 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.942+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa840ccb60 0x7ffa8415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.942+0000 7ffa8ac00640 1 --2- 172.21.15.110:0/626058291 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa84072670 0x7ffa840b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.942+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 >> 172.21.15.110:0/626058291 conn(0x7ffa8406bc50 msgr2=0x7ffa8406d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.942+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 shutdown_connections 2024-09-10T10:05:07.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:07.942+0000 7ffa8ac00640 1 -- 172.21.15.110:0/626058291 wait complete. 2024-09-10T10:05:07.963 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-10T10:05:07.732794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277128,"kb_used_data":15112,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754697592,"statfs":{"total":773094113280,"available":772810334208,"internally_reserved":0,"allocated":15474688,"data_stored":7458412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":268095020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001617"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501440+0000","last_change":"2024-09-10T10:05:03.535470+0000","last_active":"2024-09-10T10:05:04.501440+0000","last_peered":"2024-09-10T10:05:04.501440+0000","last_clean":"2024-09-10T10:05:04.501440+0000","last_became_active":"2024-09-10T10:05:03.534680+0000","last_became_peered":"2024-09-10T10:05:03.534680+0000","last_unstale":"2024-09-10T10:05:04.501440+0000","last_undegraded":"2024-09-10T10:05:04.501440+0000","last_fullsized":"2024-09-10T10:05:04.501440+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:33.401814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501698+0000","last_change":"2024-09-10T10:05:03.528499+0000","last_active":"2024-09-10T10:05:04.501698+0000","last_peered":"2024-09-10T10:05:04.501698+0000","last_clean":"2024-09-10T10:05:04.501698+0000","last_became_active":"2024-09-10T10:05:03.528235+0000","last_became_peered":"2024-09-10T10:05:03.528235+0000","last_unstale":"2024-09-10T10:05:04.501698+0000","last_undegraded":"2024-09-10T10:05:04.501698+0000","last_fullsized":"2024-09-10T10:05:04.501698+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:51.320167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235711+0000","last_change":"2024-09-10T10:05:03.515206+0000","last_active":"2024-09-10T10:05:05.235711+0000","last_peered":"2024-09-10T10:05:05.235711+0000","last_clean":"2024-09-10T10:05:05.235711+0000","last_became_active":"2024-09-10T10:05:03.514921+0000","last_became_peered":"2024-09-10T10:05:03.514921+0000","last_unstale":"2024-09-10T10:05:05.235711+0000","last_undegraded":"2024-09-10T10:05:05.235711+0000","last_fullsized":"2024-09-10T10:05:05.235711+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:18.539906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235880+0000","last_change":"2024-09-10T10:05:03.525844+0000","last_active":"2024-09-10T10:05:05.235880+0000","last_peered":"2024-09-10T10:05:05.235880+0000","last_clean":"2024-09-10T10:05:05.235880+0000","last_became_active":"2024-09-10T10:05:03.525349+0000","last_became_peered":"2024-09-10T10:05:03.525349+0000","last_unstale":"2024-09-10T10:05:05.235880+0000","last_undegraded":"2024-09-10T10:05:05.235880+0000","last_fullsized":"2024-09-10T10:05:05.235880+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:47.430761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501754+0000","last_change":"2024-09-10T10:05:03.520641+0000","last_active":"2024-09-10T10:05:04.501754+0000","last_peered":"2024-09-10T10:05:04.501754+0000","last_clean":"2024-09-10T10:05:04.501754+0000","last_became_active":"2024-09-10T10:05:03.520164+0000","last_became_peered":"2024-09-10T10:05:03.520164+0000","last_unstale":"2024-09-10T10:05:04.501754+0000","last_undegraded":"2024-09-10T10:05:04.501754+0000","last_fullsized":"2024-09-10T10:05:04.501754+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:00.819303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236816+0000","last_change":"2024-09-10T10:05:03.511711+0000","last_active":"2024-09-10T10:05:05.236816+0000","last_peered":"2024-09-10T10:05:05.236816+0000","last_clean":"2024-09-10T10:05:05.236816+0000","last_became_active":"2024-09-10T10:05:03.511438+0000","last_became_peered":"2024-09-10T10:05:03.511438+0000","last_unstale":"2024-09-10T10:05:05.236816+0000","last_undegraded":"2024-09-10T10:05:05.236816+0000","last_fullsized":"2024-09-10T10:05:05.236816+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:23.813575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235385+0000","last_change":"2024-09-10T10:05:03.511444+0000","last_active":"2024-09-10T10:05:05.235385+0000","last_peered":"2024-09-10T10:05:05.235385+0000","last_clean":"2024-09-10T10:05:05.235385+0000","last_became_active":"2024-09-10T10:05:03.511171+0000","last_became_peered":"2024-09-10T10:05:03.511171+0000","last_unstale":"2024-09-10T10:05:05.235385+0000","last_undegraded":"2024-09-10T10:05:05.235385+0000","last_fullsized":"2024-09-10T10:05:05.235385+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:49.581933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.502352+0000","last_change":"2024-09-10T10:05:03.518544+0000","last_active":"2024-09-10T10:05:04.502352+0000","last_peered":"2024-09-10T10:05:04.502352+0000","last_clean":"2024-09-10T10:05:04.502352+0000","last_became_active":"2024-09-10T10:05:03.517101+0000","last_became_peered":"2024-09-10T10:05:03.517101+0000","last_unstale":"2024-09-10T10:05:04.502352+0000","last_undegraded":"2024-09-10T10:05:04.502352+0000","last_fullsized":"2024-09-10T10:05:04.502352+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:32:29.589641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.502029+0000","last_change":"2024-09-10T10:05:03.532881+0000","last_active":"2024-09-10T10:05:04.502029+0000","last_peered":"2024-09-10T10:05:04.502029+0000","last_clean":"2024-09-10T10:05:04.502029+0000","last_became_active":"2024-09-10T10:05:03.531946+0000","last_became_peered":"2024-09-10T10:05:03.531946+0000","last_unstale":"2024-09-10T10:05:04.502029+0000","last_undegraded":"2024-09-10T10:05:04.502029+0000","last_fullsized":"2024-09-10T10:05:04.502029+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:13.101520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501945+0000","last_change":"2024-09-10T10:05:03.518525+0000","last_active":"2024-09-10T10:05:04.501945+0000","last_peered":"2024-09-10T10:05:04.501945+0000","last_clean":"2024-09-10T10:05:04.501945+0000","last_became_active":"2024-09-10T10:05:03.518222+0000","last_became_peered":"2024-09-10T10:05:03.518222+0000","last_unstale":"2024-09-10T10:05:04.501945+0000","last_undegraded":"2024-09-10T10:05:04.501945+0000","last_fullsized":"2024-09-10T10:05:04.501945+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:31.592895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501950+0000","last_change":"2024-09-10T10:05:03.520779+0000","last_active":"2024-09-10T10:05:04.501950+0000","last_peered":"2024-09-10T10:05:04.501950+0000","last_clean":"2024-09-10T10:05:04.501950+0000","last_became_active":"2024-09-10T10:05:03.520490+0000","last_became_peered":"2024-09-10T10:05:03.520490+0000","last_unstale":"2024-09-10T10:05:04.501950+0000","last_undegraded":"2024-09-10T10:05:04.501950+0000","last_fullsized":"2024-09-10T10:05:04.501950+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:57.945103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501694+0000","last_change":"2024-09-10T10:05:03.532291+0000","last_active":"2024-09-10T10:05:04.501694+0000","last_peered":"2024-09-10T10:05:04.501694+0000","last_clean":"2024-09-10T10:05:04.501694+0000","last_became_active":"2024-09-10T10:05:03.531950+0000","last_became_peered":"2024-09-10T10:05:03.531950+0000","last_unstale":"2024-09-10T10:05:04.501694+0000","last_undegraded":"2024-09-10T10:05:04.501694+0000","last_fullsized":"2024-09-10T10:05:04.501694+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:42.632083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235424+0000","last_change":"2024-09-10T10:05:03.511255+0000","last_active":"2024-09-10T10:05:05.235424+0000","last_peered":"2024-09-10T10:05:05.235424+0000","last_clean":"2024-09-10T10:05:05.235424+0000","last_became_active":"2024-09-10T10:05:03.510358+0000","last_became_peered":"2024-09-10T10:05:03.510358+0000","last_unstale":"2024-09-10T10:05:05.235424+0000","last_undegraded":"2024-09-10T10:05:05.235424+0000","last_fullsized":"2024-09-10T10:05:05.235424+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:31.694510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235293+0000","last_change":"2024-09-10T10:05:03.528483+0000","last_active":"2024-09-10T10:05:05.235293+0000","last_peered":"2024-09-10T10:05:05.235293+0000","last_clean":"2024-09-10T10:05:05.235293+0000","last_became_active":"2024-09-10T10:05:03.528205+0000","last_became_peered":"2024-09-10T10:05:03.528205+0000","last_unstale":"2024-09-10T10:05:05.235293+0000","last_undegraded":"2024-09-10T10:05:05.235293+0000","last_fullsized":"2024-09-10T10:05:05.235293+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:01.135828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501303+0000","last_change":"2024-09-10T10:05:03.532844+0000","last_active":"2024-09-10T10:05:04.501303+0000","last_peered":"2024-09-10T10:05:04.501303+0000","last_clean":"2024-09-10T10:05:04.501303+0000","last_became_active":"2024-09-10T10:05:03.532518+0000","last_became_peered":"2024-09-10T10:05:03.532518+0000","last_unstale":"2024-09-10T10:05:04.501303+0000","last_undegraded":"2024-09-10T10:05:04.501303+0000","last_fullsized":"2024-09-10T10:05:04.501303+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:30.070230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.234908+0000","last_change":"2024-09-10T10:05:03.530926+0000","last_active":"2024-09-10T10:05:05.234908+0000","last_peered":"2024-09-10T10:05:05.234908+0000","last_clean":"2024-09-10T10:05:05.234908+0000","last_became_active":"2024-09-10T10:05:03.530646+0000","last_became_peered":"2024-09-10T10:05:03.530646+0000","last_unstale":"2024-09-10T10:05:05.234908+0000","last_undegraded":"2024-09-10T10:05:05.234908+0000","last_fullsized":"2024-09-10T10:05:05.234908+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:21.043019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236786+0000","last_change":"2024-09-10T10:05:03.517449+0000","last_active":"2024-09-10T10:05:05.236786+0000","last_peered":"2024-09-10T10:05:05.236786+0000","last_clean":"2024-09-10T10:05:05.236786+0000","last_became_active":"2024-09-10T10:05:03.517205+0000","last_became_peered":"2024-09-10T10:05:03.517205+0000","last_unstale":"2024-09-10T10:05:05.236786+0000","last_undegraded":"2024-09-10T10:05:05.236786+0000","last_fullsized":"2024-09-10T10:05:05.236786+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:37:15.218834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.496962+0000","last_change":"2024-09-10T10:05:03.525118+0000","last_active":"2024-09-10T10:05:04.496962+0000","last_peered":"2024-09-10T10:05:04.496962+0000","last_clean":"2024-09-10T10:05:04.496962+0000","last_became_active":"2024-09-10T10:05:03.522653+0000","last_became_peered":"2024-09-10T10:05:03.522653+0000","last_unstale":"2024-09-10T10:05:04.496962+0000","last_undegraded":"2024-09-10T10:05:04.496962+0000","last_fullsized":"2024-09-10T10:05:04.496962+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:31.273574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236931+0000","last_change":"2024-09-10T10:05:03.516876+0000","last_active":"2024-09-10T10:05:05.236931+0000","last_peered":"2024-09-10T10:05:05.236931+0000","last_clean":"2024-09-10T10:05:05.236931+0000","last_became_active":"2024-09-10T10:05:03.516430+0000","last_became_peered":"2024-09-10T10:05:03.516430+0000","last_unstale":"2024-09-10T10:05:05.236931+0000","last_undegraded":"2024-09-10T10:05:05.236931+0000","last_fullsized":"2024-09-10T10:05:05.236931+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:32.183838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235543+0000","last_change":"2024-09-10T10:05:03.530911+0000","last_active":"2024-09-10T10:05:05.235543+0000","last_peered":"2024-09-10T10:05:05.235543+0000","last_clean":"2024-09-10T10:05:05.235543+0000","last_became_active":"2024-09-10T10:05:03.530636+0000","last_became_peered":"2024-09-10T10:05:03.530636+0000","last_unstale":"2024-09-10T10:05:05.235543+0000","last_undegraded":"2024-09-10T10:05:05.235543+0000","last_fullsized":"2024-09-10T10:05:05.235543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:19.027027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.237028+0000","last_change":"2024-09-10T10:05:03.520916+0000","last_active":"2024-09-10T10:05:05.237028+0000","last_peered":"2024-09-10T10:05:05.237028+0000","last_clean":"2024-09-10T10:05:05.237028+0000","last_became_active":"2024-09-10T10:05:03.520615+0000","last_became_peered":"2024-09-10T10:05:03.520615+0000","last_unstale":"2024-09-10T10:05:05.237028+0000","last_undegraded":"2024-09-10T10:05:05.237028+0000","last_fullsized":"2024-09-10T10:05:05.237028+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:10.046687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501745+0000","last_change":"2024-09-10T10:05:03.511954+0000","last_active":"2024-09-10T10:05:04.501745+0000","last_peered":"2024-09-10T10:05:04.501745+0000","last_clean":"2024-09-10T10:05:04.501745+0000","last_became_active":"2024-09-10T10:05:03.511585+0000","last_became_peered":"2024-09-10T10:05:03.511585+0000","last_unstale":"2024-09-10T10:05:04.501745+0000","last_undegraded":"2024-09-10T10:05:04.501745+0000","last_fullsized":"2024-09-10T10:05:04.501745+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:05.797119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236342+0000","last_change":"2024-09-10T10:05:03.522927+0000","last_active":"2024-09-10T10:05:05.236342+0000","last_peered":"2024-09-10T10:05:05.236342+0000","last_clean":"2024-09-10T10:05:05.236342+0000","last_became_active":"2024-09-10T10:05:03.522626+0000","last_became_peered":"2024-09-10T10:05:03.522626+0000","last_unstale":"2024-09-10T10:05:05.236342+0000","last_undegraded":"2024-09-10T10:05:05.236342+0000","last_fullsized":"2024-09-10T10:05:05.236342+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:00.696672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.502151+0000","last_change":"2024-09-10T10:05:03.511760+0000","last_active":"2024-09-10T10:05:04.502151+0000","last_peered":"2024-09-10T10:05:04.502151+0000","last_clean":"2024-09-10T10:05:04.502151+0000","last_became_active":"2024-09-10T10:05:03.511435+0000","last_became_peered":"2024-09-10T10:05:03.511435+0000","last_unstale":"2024-09-10T10:05:04.502151+0000","last_undegraded":"2024-09-10T10:05:04.502151+0000","last_fullsized":"2024-09-10T10:05:04.502151+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:47.751561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235862+0000","last_change":"2024-09-10T10:05:03.517614+0000","last_active":"2024-09-10T10:05:05.235862+0000","last_peered":"2024-09-10T10:05:05.235862+0000","last_clean":"2024-09-10T10:05:05.235862+0000","last_became_active":"2024-09-10T10:05:03.516645+0000","last_became_peered":"2024-09-10T10:05:03.516645+0000","last_unstale":"2024-09-10T10:05:05.235862+0000","last_undegraded":"2024-09-10T10:05:05.235862+0000","last_fullsized":"2024-09-10T10:05:05.235862+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:34.256738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235341+0000","last_change":"2024-09-10T10:05:03.516255+0000","last_active":"2024-09-10T10:05:05.235341+0000","last_peered":"2024-09-10T10:05:05.235341+0000","last_clean":"2024-09-10T10:05:05.235341+0000","last_became_active":"2024-09-10T10:05:03.511570+0000","last_became_peered":"2024-09-10T10:05:03.511570+0000","last_unstale":"2024-09-10T10:05:05.235341+0000","last_undegraded":"2024-09-10T10:05:05.235341+0000","last_fullsized":"2024-09-10T10:05:05.235341+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:09.576301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235894+0000","last_change":"2024-09-10T10:05:03.506931+0000","last_active":"2024-09-10T10:05:05.235894+0000","last_peered":"2024-09-10T10:05:05.235894+0000","last_clean":"2024-09-10T10:05:05.235894+0000","last_became_active":"2024-09-10T10:05:03.506644+0000","last_became_peered":"2024-09-10T10:05:03.506644+0000","last_unstale":"2024-09-10T10:05:05.235894+0000","last_undegraded":"2024-09-10T10:05:05.235894+0000","last_fullsized":"2024-09-10T10:05:05.235894+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:56.853907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236110+0000","last_change":"2024-09-10T10:05:03.526650+0000","last_active":"2024-09-10T10:05:05.236110+0000","last_peered":"2024-09-10T10:05:05.236110+0000","last_clean":"2024-09-10T10:05:05.236110+0000","last_became_active":"2024-09-10T10:05:03.526338+0000","last_became_peered":"2024-09-10T10:05:03.526338+0000","last_unstale":"2024-09-10T10:05:05.236110+0000","last_undegraded":"2024-09-10T10:05:05.236110+0000","last_fullsized":"2024-09-10T10:05:05.236110+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:43.757727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235489+0000","last_change":"2024-09-10T10:05:03.505789+0000","last_active":"2024-09-10T10:05:05.235489+0000","last_peered":"2024-09-10T10:05:05.235489+0000","last_clean":"2024-09-10T10:05:05.235489+0000","last_became_active":"2024-09-10T10:05:03.505108+0000","last_became_peered":"2024-09-10T10:05:03.505108+0000","last_unstale":"2024-09-10T10:05:05.235489+0000","last_undegraded":"2024-09-10T10:05:05.235489+0000","last_fullsized":"2024-09-10T10:05:05.235489+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:41.430140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236049+0000","last_change":"2024-09-10T10:05:03.529710+0000","last_active":"2024-09-10T10:05:05.236049+0000","last_peered":"2024-09-10T10:05:05.236049+0000","last_clean":"2024-09-10T10:05:05.236049+0000","last_became_active":"2024-09-10T10:05:03.529104+0000","last_became_peered":"2024-09-10T10:05:03.529104+0000","last_unstale":"2024-09-10T10:05:05.236049+0000","last_undegraded":"2024-09-10T10:05:05.236049+0000","last_fullsized":"2024-09-10T10:05:05.236049+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:05.797119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235988+0000","last_change":"2024-09-10T10:05:03.527166+0000","last_active":"2024-09-10T10:05:05.235988+0000","last_peered":"2024-09-10T10:05:05.235988+0000","last_clean":"2024-09-10T10:05:05.235988+0000","last_became_active":"2024-09-10T10:05:03.526027+0000","last_became_peered":"2024-09-10T10:05:03.526027+0000","last_unstale":"2024-09-10T10:05:05.235988+0000","last_undegraded":"2024-09-10T10:05:05.235988+0000","last_fullsized":"2024-09-10T10:05:05.235988+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:00.819303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.496573+0000","last_change":"2024-09-10T10:05:03.522228+0000","last_active":"2024-09-10T10:05:04.496573+0000","last_peered":"2024-09-10T10:05:04.496573+0000","last_clean":"2024-09-10T10:05:04.496573+0000","last_became_active":"2024-09-10T10:05:03.521896+0000","last_became_peered":"2024-09-10T10:05:03.521896+0000","last_unstale":"2024-09-10T10:05:04.496573+0000","last_undegraded":"2024-09-10T10:05:04.496573+0000","last_fullsized":"2024-09-10T10:05:04.496573+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:20.628275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235421+0000","last_change":"2024-09-10T10:05:03.516526+0000","last_active":"2024-09-10T10:05:05.235421+0000","last_peered":"2024-09-10T10:05:05.235421+0000","last_clean":"2024-09-10T10:05:05.235421+0000","last_became_active":"2024-09-10T10:05:03.510990+0000","last_became_peered":"2024-09-10T10:05:03.510990+0000","last_unstale":"2024-09-10T10:05:05.235421+0000","last_undegraded":"2024-09-10T10:05:05.235421+0000","last_fullsized":"2024-09-10T10:05:05.235421+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:50.839453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"141'4","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501340+0000","last_change":"2024-09-10T10:05:02.516325+0000","last_active":"2024-09-10T10:05:04.501340+0000","last_peered":"2024-09-10T10:05:04.501340+0000","last_clean":"2024-09-10T10:05:04.501340+0000","last_became_active":"2024-09-10T10:05:02.516025+0000","last_became_peered":"2024-09-10T10:05:02.516025+0000","last_unstale":"2024-09-10T10:05:04.501340+0000","last_undegraded":"2024-09-10T10:05:04.501340+0000","last_fullsized":"2024-09-10T10:05:04.501340+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:06.501245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236571+0000","last_change":"2024-09-10T10:05:03.527339+0000","last_active":"2024-09-10T10:05:05.236571+0000","last_peered":"2024-09-10T10:05:05.236571+0000","last_clean":"2024-09-10T10:05:05.236571+0000","last_became_active":"2024-09-10T10:05:03.527040+0000","last_became_peered":"2024-09-10T10:05:03.527040+0000","last_unstale":"2024-09-10T10:05:05.236571+0000","last_undegraded":"2024-09-10T10:05:05.236571+0000","last_fullsized":"2024-09-10T10:05:05.236571+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:18:57.945103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236329+0000","last_change":"2024-09-10T10:05:02.505735+0000","last_active":"2024-09-10T10:05:05.236329+0000","last_peered":"2024-09-10T10:05:05.236329+0000","last_clean":"2024-09-10T10:05:05.236329+0000","last_became_active":"2024-09-10T10:05:02.502459+0000","last_became_peered":"2024-09-10T10:05:02.502459+0000","last_unstale":"2024-09-10T10:05:05.236329+0000","last_undegraded":"2024-09-10T10:05:05.236329+0000","last_fullsized":"2024-09-10T10:05:05.236329+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:30.998140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236717+0000","last_change":"2024-09-10T10:05:03.535391+0000","last_active":"2024-09-10T10:05:05.236717+0000","last_peered":"2024-09-10T10:05:05.236717+0000","last_clean":"2024-09-10T10:05:05.236717+0000","last_became_active":"2024-09-10T10:05:03.535137+0000","last_became_peered":"2024-09-10T10:05:03.535137+0000","last_unstale":"2024-09-10T10:05:05.236717+0000","last_undegraded":"2024-09-10T10:05:05.236717+0000","last_fullsized":"2024-09-10T10:05:05.236717+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:03.854637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235733+0000","last_change":"2024-09-10T10:05:02.517691+0000","last_active":"2024-09-10T10:05:05.235733+0000","last_peered":"2024-09-10T10:05:05.235733+0000","last_clean":"2024-09-10T10:05:05.235733+0000","last_became_active":"2024-09-10T10:05:02.517539+0000","last_became_peered":"2024-09-10T10:05:02.517539+0000","last_unstale":"2024-09-10T10:05:05.235733+0000","last_undegraded":"2024-09-10T10:05:05.235733+0000","last_fullsized":"2024-09-10T10:05:05.235733+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:00.133074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235895+0000","last_change":"2024-09-10T10:05:03.514261+0000","last_active":"2024-09-10T10:05:05.235895+0000","last_peered":"2024-09-10T10:05:05.235895+0000","last_clean":"2024-09-10T10:05:05.235895+0000","last_became_active":"2024-09-10T10:05:03.512670+0000","last_became_peered":"2024-09-10T10:05:03.512670+0000","last_unstale":"2024-09-10T10:05:05.235895+0000","last_undegraded":"2024-09-10T10:05:05.235895+0000","last_fullsized":"2024-09-10T10:05:05.235895+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:31.499234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236167+0000","last_change":"2024-09-10T10:05:02.511801+0000","last_active":"2024-09-10T10:05:05.236167+0000","last_peered":"2024-09-10T10:05:05.236167+0000","last_clean":"2024-09-10T10:05:05.236167+0000","last_became_active":"2024-09-10T10:05:02.511430+0000","last_became_peered":"2024-09-10T10:05:02.511430+0000","last_unstale":"2024-09-10T10:05:05.236167+0000","last_undegraded":"2024-09-10T10:05:05.236167+0000","last_fullsized":"2024-09-10T10:05:05.236167+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:55.851153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235741+0000","last_change":"2024-09-10T10:05:03.520229+0000","last_active":"2024-09-10T10:05:05.235741+0000","last_peered":"2024-09-10T10:05:05.235741+0000","last_clean":"2024-09-10T10:05:05.235741+0000","last_became_active":"2024-09-10T10:05:03.519879+0000","last_became_peered":"2024-09-10T10:05:03.519879+0000","last_unstale":"2024-09-10T10:05:05.235741+0000","last_undegraded":"2024-09-10T10:05:05.235741+0000","last_fullsized":"2024-09-10T10:05:05.235741+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:34.627404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235497+0000","last_change":"2024-09-10T10:05:02.514053+0000","last_active":"2024-09-10T10:05:05.235497+0000","last_peered":"2024-09-10T10:05:05.235497+0000","last_clean":"2024-09-10T10:05:05.235497+0000","last_became_active":"2024-09-10T10:05:02.513755+0000","last_became_peered":"2024-09-10T10:05:02.513755+0000","last_unstale":"2024-09-10T10:05:05.235497+0000","last_undegraded":"2024-09-10T10:05:05.235497+0000","last_fullsized":"2024-09-10T10:05:05.235497+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:08.307135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235549+0000","last_change":"2024-09-10T10:05:03.508314+0000","last_active":"2024-09-10T10:05:05.235549+0000","last_peered":"2024-09-10T10:05:05.235549+0000","last_clean":"2024-09-10T10:05:05.235549+0000","last_became_active":"2024-09-10T10:05:03.508057+0000","last_became_peered":"2024-09-10T10:05:03.508057+0000","last_unstale":"2024-09-10T10:05:05.235549+0000","last_undegraded":"2024-09-10T10:05:05.235549+0000","last_fullsized":"2024-09-10T10:05:05.235549+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:32.000894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.234826+0000","last_change":"2024-09-10T10:05:02.514751+0000","last_active":"2024-09-10T10:05:05.234826+0000","last_peered":"2024-09-10T10:05:05.234826+0000","last_clean":"2024-09-10T10:05:05.234826+0000","last_became_active":"2024-09-10T10:05:02.514176+0000","last_became_peered":"2024-09-10T10:05:02.514176+0000","last_unstale":"2024-09-10T10:05:05.234826+0000","last_undegraded":"2024-09-10T10:05:05.234826+0000","last_fullsized":"2024-09-10T10:05:05.234826+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:16.709161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.234974+0000","last_change":"2024-09-10T10:05:03.530779+0000","last_active":"2024-09-10T10:05:05.234974+0000","last_peered":"2024-09-10T10:05:05.234974+0000","last_clean":"2024-09-10T10:05:05.234974+0000","last_became_active":"2024-09-10T10:05:03.530486+0000","last_became_peered":"2024-09-10T10:05:03.530486+0000","last_unstale":"2024-09-10T10:05:05.234974+0000","last_undegraded":"2024-09-10T10:05:05.234974+0000","last_fullsized":"2024-09-10T10:05:05.234974+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:24:31.023140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.502248+0000","last_change":"2024-09-10T10:05:02.511844+0000","last_active":"2024-09-10T10:05:04.502248+0000","last_peered":"2024-09-10T10:05:04.502248+0000","last_clean":"2024-09-10T10:05:04.502248+0000","last_became_active":"2024-09-10T10:05:02.511456+0000","last_became_peered":"2024-09-10T10:05:02.511456+0000","last_unstale":"2024-09-10T10:05:04.502248+0000","last_undegraded":"2024-09-10T10:05:04.502248+0000","last_fullsized":"2024-09-10T10:05:04.502248+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:30.496480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236558+0000","last_change":"2024-09-10T10:05:03.527553+0000","last_active":"2024-09-10T10:05:05.236558+0000","last_peered":"2024-09-10T10:05:05.236558+0000","last_clean":"2024-09-10T10:05:05.236558+0000","last_became_active":"2024-09-10T10:05:03.527281+0000","last_became_peered":"2024-09-10T10:05:03.527281+0000","last_unstale":"2024-09-10T10:05:05.236558+0000","last_undegraded":"2024-09-10T10:05:05.236558+0000","last_fullsized":"2024-09-10T10:05:05.236558+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:09.167072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235072+0000","last_change":"2024-09-10T10:05:02.513042+0000","last_active":"2024-09-10T10:05:05.235072+0000","last_peered":"2024-09-10T10:05:05.235072+0000","last_clean":"2024-09-10T10:05:05.235072+0000","last_became_active":"2024-09-10T10:05:02.512696+0000","last_became_peered":"2024-09-10T10:05:02.512696+0000","last_unstale":"2024-09-10T10:05:05.235072+0000","last_undegraded":"2024-09-10T10:05:05.235072+0000","last_fullsized":"2024-09-10T10:05:05.235072+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:27.238449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.497286+0000","last_change":"2024-09-10T10:05:03.510377+0000","last_active":"2024-09-10T10:05:04.497286+0000","last_peered":"2024-09-10T10:05:04.497286+0000","last_clean":"2024-09-10T10:05:04.497286+0000","last_became_active":"2024-09-10T10:05:03.510132+0000","last_became_peered":"2024-09-10T10:05:03.510132+0000","last_unstale":"2024-09-10T10:05:04.497286+0000","last_undegraded":"2024-09-10T10:05:04.497286+0000","last_fullsized":"2024-09-10T10:05:04.497286+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:00.601880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236830+0000","last_change":"2024-09-10T10:05:02.516801+0000","last_active":"2024-09-10T10:05:05.236830+0000","last_peered":"2024-09-10T10:05:05.236830+0000","last_clean":"2024-09-10T10:05:05.236830+0000","last_became_active":"2024-09-10T10:05:02.516457+0000","last_became_peered":"2024-09-10T10:05:02.516457+0000","last_unstale":"2024-09-10T10:05:05.236830+0000","last_undegraded":"2024-09-10T10:05:05.236830+0000","last_fullsized":"2024-09-10T10:05:05.236830+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:28.488556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.496933+0000","last_change":"2024-09-10T10:05:03.509397+0000","last_active":"2024-09-10T10:05:04.496933+0000","last_peered":"2024-09-10T10:05:04.496933+0000","last_clean":"2024-09-10T10:05:04.496933+0000","last_became_active":"2024-09-10T10:05:03.509139+0000","last_became_peered":"2024-09-10T10:05:03.509139+0000","last_unstale":"2024-09-10T10:05:04.496933+0000","last_undegraded":"2024-09-10T10:05:04.496933+0000","last_fullsized":"2024-09-10T10:05:04.496933+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:39.702041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"141'3","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236921+0000","last_change":"2024-09-10T10:05:02.511942+0000","last_active":"2024-09-10T10:05:05.236921+0000","last_peered":"2024-09-10T10:05:05.236921+0000","last_clean":"2024-09-10T10:05:05.236921+0000","last_became_active":"2024-09-10T10:05:02.511594+0000","last_became_peered":"2024-09-10T10:05:02.511594+0000","last_unstale":"2024-09-10T10:05:05.236921+0000","last_undegraded":"2024-09-10T10:05:05.236921+0000","last_fullsized":"2024-09-10T10:05:05.236921+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:04.328829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236560+0000","last_change":"2024-09-10T10:05:03.520651+0000","last_active":"2024-09-10T10:05:05.236560+0000","last_peered":"2024-09-10T10:05:05.236560+0000","last_clean":"2024-09-10T10:05:05.236560+0000","last_became_active":"2024-09-10T10:05:03.519383+0000","last_became_peered":"2024-09-10T10:05:03.519383+0000","last_unstale":"2024-09-10T10:05:05.236560+0000","last_undegraded":"2024-09-10T10:05:05.236560+0000","last_fullsized":"2024-09-10T10:05:05.236560+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:13.101520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"141'6","reported_seq":20,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236171+0000","last_change":"2024-09-10T10:05:02.509563+0000","last_active":"2024-09-10T10:05:05.236171+0000","last_peered":"2024-09-10T10:05:05.236171+0000","last_clean":"2024-09-10T10:05:05.236171+0000","last_became_active":"2024-09-10T10:05:02.509255+0000","last_became_peered":"2024-09-10T10:05:02.509255+0000","last_unstale":"2024-09-10T10:05:05.236171+0000","last_undegraded":"2024-09-10T10:05:05.236171+0000","last_fullsized":"2024-09-10T10:05:05.236171+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:46.428007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.497114+0000","last_change":"2024-09-10T10:05:03.521175+0000","last_active":"2024-09-10T10:05:04.497114+0000","last_peered":"2024-09-10T10:05:04.497114+0000","last_clean":"2024-09-10T10:05:04.497114+0000","last_became_active":"2024-09-10T10:05:03.520927+0000","last_became_peered":"2024-09-10T10:05:03.520927+0000","last_unstale":"2024-09-10T10:05:04.497114+0000","last_undegraded":"2024-09-10T10:05:04.497114+0000","last_fullsized":"2024-09-10T10:05:04.497114+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:17:12.815837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235460+0000","last_change":"2024-09-10T10:05:02.513417+0000","last_active":"2024-09-10T10:05:05.235460+0000","last_peered":"2024-09-10T10:05:05.235460+0000","last_clean":"2024-09-10T10:05:05.235460+0000","last_became_active":"2024-09-10T10:05:02.512368+0000","last_became_peered":"2024-09-10T10:05:02.512368+0000","last_unstale":"2024-09-10T10:05:05.235460+0000","last_undegraded":"2024-09-10T10:05:05.235460+0000","last_fullsized":"2024-09-10T10:05:05.235460+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:30.691756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235459+0000","last_change":"2024-09-10T10:05:03.517078+0000","last_active":"2024-09-10T10:05:05.235459+0000","last_peered":"2024-09-10T10:05:05.235459+0000","last_clean":"2024-09-10T10:05:05.235459+0000","last_became_active":"2024-09-10T10:05:03.516749+0000","last_became_peered":"2024-09-10T10:05:03.516749+0000","last_unstale":"2024-09-10T10:05:05.235459+0000","last_undegraded":"2024-09-10T10:05:05.235459+0000","last_fullsized":"2024-09-10T10:05:05.235459+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:54.522836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.496813+0000","last_change":"2024-09-10T10:05:02.516029+0000","last_active":"2024-09-10T10:05:04.496813+0000","last_peered":"2024-09-10T10:05:04.496813+0000","last_clean":"2024-09-10T10:05:04.496813+0000","last_became_active":"2024-09-10T10:05:02.515188+0000","last_became_peered":"2024-09-10T10:05:02.515188+0000","last_unstale":"2024-09-10T10:05:04.496813+0000","last_undegraded":"2024-09-10T10:05:04.496813+0000","last_fullsized":"2024-09-10T10:05:04.496813+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:07.000846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.234928+0000","last_change":"2024-09-10T10:05:03.505311+0000","last_active":"2024-09-10T10:05:05.234928+0000","last_peered":"2024-09-10T10:05:05.234928+0000","last_clean":"2024-09-10T10:05:05.234928+0000","last_became_active":"2024-09-10T10:05:03.504133+0000","last_became_peered":"2024-09-10T10:05:03.504133+0000","last_unstale":"2024-09-10T10:05:05.234928+0000","last_undegraded":"2024-09-10T10:05:05.234928+0000","last_fullsized":"2024-09-10T10:05:05.234928+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:26.101433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235107+0000","last_change":"2024-09-10T10:05:02.516009+0000","last_active":"2024-09-10T10:05:05.235107+0000","last_peered":"2024-09-10T10:05:05.235107+0000","last_clean":"2024-09-10T10:05:05.235107+0000","last_became_active":"2024-09-10T10:05:02.515715+0000","last_became_peered":"2024-09-10T10:05:02.515715+0000","last_unstale":"2024-09-10T10:05:05.235107+0000","last_undegraded":"2024-09-10T10:05:05.235107+0000","last_fullsized":"2024-09-10T10:05:05.235107+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:13.861825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235186+0000","last_change":"2024-09-10T10:05:03.519098+0000","last_active":"2024-09-10T10:05:05.235186+0000","last_peered":"2024-09-10T10:05:05.235186+0000","last_clean":"2024-09-10T10:05:05.235186+0000","last_became_active":"2024-09-10T10:05:03.517792+0000","last_became_peered":"2024-09-10T10:05:03.517792+0000","last_unstale":"2024-09-10T10:05:05.235186+0000","last_undegraded":"2024-09-10T10:05:05.235186+0000","last_fullsized":"2024-09-10T10:05:05.235186+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:14.864579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"141'8","reported_seq":22,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235948+0000","last_change":"2024-09-10T10:05:02.510158+0000","last_active":"2024-09-10T10:05:05.235948+0000","last_peered":"2024-09-10T10:05:05.235948+0000","last_clean":"2024-09-10T10:05:05.235948+0000","last_became_active":"2024-09-10T10:05:02.509880+0000","last_became_peered":"2024-09-10T10:05:02.509880+0000","last_unstale":"2024-09-10T10:05:05.235948+0000","last_undegraded":"2024-09-10T10:05:05.235948+0000","last_fullsized":"2024-09-10T10:05:05.235948+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:23.194763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236224+0000","last_change":"2024-09-10T10:05:03.514908+0000","last_active":"2024-09-10T10:05:05.236224+0000","last_peered":"2024-09-10T10:05:05.236224+0000","last_clean":"2024-09-10T10:05:05.236224+0000","last_became_active":"2024-09-10T10:05:03.514635+0000","last_became_peered":"2024-09-10T10:05:03.514635+0000","last_unstale":"2024-09-10T10:05:05.236224+0000","last_undegraded":"2024-09-10T10:05:05.236224+0000","last_fullsized":"2024-09-10T10:05:05.236224+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.130353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235952+0000","last_change":"2024-09-10T10:05:02.516028+0000","last_active":"2024-09-10T10:05:05.235952+0000","last_peered":"2024-09-10T10:05:05.235952+0000","last_clean":"2024-09-10T10:05:05.235952+0000","last_became_active":"2024-09-10T10:05:02.515756+0000","last_became_peered":"2024-09-10T10:05:02.515756+0000","last_unstale":"2024-09-10T10:05:05.235952+0000","last_undegraded":"2024-09-10T10:05:05.235952+0000","last_fullsized":"2024-09-10T10:05:05.235952+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:37:14.216080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.497370+0000","last_change":"2024-09-10T10:05:03.517418+0000","last_active":"2024-09-10T10:05:04.497370+0000","last_peered":"2024-09-10T10:05:04.497370+0000","last_clean":"2024-09-10T10:05:04.497370+0000","last_became_active":"2024-09-10T10:05:03.517157+0000","last_became_peered":"2024-09-10T10:05:03.517157+0000","last_unstale":"2024-09-10T10:05:04.497370+0000","last_undegraded":"2024-09-10T10:05:04.497370+0000","last_fullsized":"2024-09-10T10:05:04.497370+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:53.263918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235707+0000","last_change":"2024-09-10T10:05:02.513472+0000","last_active":"2024-09-10T10:05:05.235707+0000","last_peered":"2024-09-10T10:05:05.235707+0000","last_clean":"2024-09-10T10:05:05.235707+0000","last_became_active":"2024-09-10T10:05:02.513227+0000","last_became_peered":"2024-09-10T10:05:02.513227+0000","last_unstale":"2024-09-10T10:05:05.235707+0000","last_undegraded":"2024-09-10T10:05:05.235707+0000","last_fullsized":"2024-09-10T10:05:05.235707+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:16:02.851883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236145+0000","last_change":"2024-09-10T10:05:03.512439+0000","last_active":"2024-09-10T10:05:05.236145+0000","last_peered":"2024-09-10T10:05:05.236145+0000","last_clean":"2024-09-10T10:05:05.236145+0000","last_became_active":"2024-09-10T10:05:03.512183+0000","last_became_peered":"2024-09-10T10:05:03.512183+0000","last_unstale":"2024-09-10T10:05:05.236145+0000","last_undegraded":"2024-09-10T10:05:05.236145+0000","last_fullsized":"2024-09-10T10:05:05.236145+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:46.272177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501117+0000","last_change":"2024-09-10T10:05:02.518210+0000","last_active":"2024-09-10T10:05:04.501117+0000","last_peered":"2024-09-10T10:05:04.501117+0000","last_clean":"2024-09-10T10:05:04.501117+0000","last_became_active":"2024-09-10T10:05:02.517926+0000","last_became_peered":"2024-09-10T10:05:02.517926+0000","last_unstale":"2024-09-10T10:05:04.501117+0000","last_undegraded":"2024-09-10T10:05:04.501117+0000","last_fullsized":"2024-09-10T10:05:04.501117+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:13.772487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235239+0000","last_change":"2024-09-10T10:05:03.516452+0000","last_active":"2024-09-10T10:05:05.235239+0000","last_peered":"2024-09-10T10:05:05.235239+0000","last_clean":"2024-09-10T10:05:05.235239+0000","last_became_active":"2024-09-10T10:05:03.516190+0000","last_became_peered":"2024-09-10T10:05:03.516190+0000","last_unstale":"2024-09-10T10:05:05.235239+0000","last_undegraded":"2024-09-10T10:05:05.235239+0000","last_fullsized":"2024-09-10T10:05:05.235239+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:21.322826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501332+0000","last_change":"2024-09-10T10:05:02.512708+0000","last_active":"2024-09-10T10:05:04.501332+0000","last_peered":"2024-09-10T10:05:04.501332+0000","last_clean":"2024-09-10T10:05:04.501332+0000","last_became_active":"2024-09-10T10:05:02.512420+0000","last_became_peered":"2024-09-10T10:05:02.512420+0000","last_unstale":"2024-09-10T10:05:04.501332+0000","last_undegraded":"2024-09-10T10:05:04.501332+0000","last_fullsized":"2024-09-10T10:05:04.501332+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:22.001452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235225+0000","last_change":"2024-09-10T10:05:03.514082+0000","last_active":"2024-09-10T10:05:05.235225+0000","last_peered":"2024-09-10T10:05:05.235225+0000","last_clean":"2024-09-10T10:05:05.235225+0000","last_became_active":"2024-09-10T10:05:03.512970+0000","last_became_peered":"2024-09-10T10:05:03.512970+0000","last_unstale":"2024-09-10T10:05:05.235225+0000","last_undegraded":"2024-09-10T10:05:05.235225+0000","last_fullsized":"2024-09-10T10:05:05.235225+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:21.467563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235576+0000","last_change":"2024-09-10T10:05:02.515379+0000","last_active":"2024-09-10T10:05:05.235576+0000","last_peered":"2024-09-10T10:05:05.235576+0000","last_clean":"2024-09-10T10:05:05.235576+0000","last_became_active":"2024-09-10T10:05:02.515119+0000","last_became_peered":"2024-09-10T10:05:02.515119+0000","last_unstale":"2024-09-10T10:05:05.235576+0000","last_undegraded":"2024-09-10T10:05:05.235576+0000","last_fullsized":"2024-09-10T10:05:05.235576+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:18.024273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235629+0000","last_change":"2024-09-10T10:05:03.528570+0000","last_active":"2024-09-10T10:05:05.235629+0000","last_peered":"2024-09-10T10:05:05.235629+0000","last_clean":"2024-09-10T10:05:05.235629+0000","last_became_active":"2024-09-10T10:05:03.528252+0000","last_became_peered":"2024-09-10T10:05:03.528252+0000","last_unstale":"2024-09-10T10:05:05.235629+0000","last_undegraded":"2024-09-10T10:05:05.235629+0000","last_fullsized":"2024-09-10T10:05:05.235629+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:30.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501338+0000","last_change":"2024-09-10T10:05:02.516052+0000","last_active":"2024-09-10T10:05:04.501338+0000","last_peered":"2024-09-10T10:05:04.501338+0000","last_clean":"2024-09-10T10:05:04.501338+0000","last_became_active":"2024-09-10T10:05:02.515755+0000","last_became_peered":"2024-09-10T10:05:02.515755+0000","last_unstale":"2024-09-10T10:05:04.501338+0000","last_undegraded":"2024-09-10T10:05:04.501338+0000","last_fullsized":"2024-09-10T10:05:04.501338+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:18.688580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501282+0000","last_change":"2024-09-10T10:05:03.536297+0000","last_active":"2024-09-10T10:05:04.501282+0000","last_peered":"2024-09-10T10:05:04.501282+0000","last_clean":"2024-09-10T10:05:04.501282+0000","last_became_active":"2024-09-10T10:05:03.536020+0000","last_became_peered":"2024-09-10T10:05:03.536020+0000","last_unstale":"2024-09-10T10:05:04.501282+0000","last_undegraded":"2024-09-10T10:05:04.501282+0000","last_fullsized":"2024-09-10T10:05:04.501282+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:56.619065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235232+0000","last_change":"2024-09-10T10:05:02.510748+0000","last_active":"2024-09-10T10:05:05.235232+0000","last_peered":"2024-09-10T10:05:05.235232+0000","last_clean":"2024-09-10T10:05:05.235232+0000","last_became_active":"2024-09-10T10:05:02.510495+0000","last_became_peered":"2024-09-10T10:05:02.510495+0000","last_unstale":"2024-09-10T10:05:05.235232+0000","last_undegraded":"2024-09-10T10:05:05.235232+0000","last_fullsized":"2024-09-10T10:05:05.235232+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:20.464809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236603+0000","last_change":"2024-09-10T10:05:03.522596+0000","last_active":"2024-09-10T10:05:05.236603+0000","last_peered":"2024-09-10T10:05:05.236603+0000","last_clean":"2024-09-10T10:05:05.236603+0000","last_became_active":"2024-09-10T10:05:03.522271+0000","last_became_peered":"2024-09-10T10:05:03.522271+0000","last_unstale":"2024-09-10T10:05:05.236603+0000","last_undegraded":"2024-09-10T10:05:05.236603+0000","last_fullsized":"2024-09-10T10:05:05.236603+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:09.309889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501601+0000","last_change":"2024-09-10T10:05:02.516732+0000","last_active":"2024-09-10T10:05:04.501601+0000","last_peered":"2024-09-10T10:05:04.501601+0000","last_clean":"2024-09-10T10:05:04.501601+0000","last_became_active":"2024-09-10T10:05:02.515602+0000","last_became_peered":"2024-09-10T10:05:02.515602+0000","last_unstale":"2024-09-10T10:05:04.501601+0000","last_undegraded":"2024-09-10T10:05:04.501601+0000","last_fullsized":"2024-09-10T10:05:04.501601+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:42.881135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.497200+0000","last_change":"2024-09-10T10:05:03.510553+0000","last_active":"2024-09-10T10:05:04.497200+0000","last_peered":"2024-09-10T10:05:04.497200+0000","last_clean":"2024-09-10T10:05:04.497200+0000","last_became_active":"2024-09-10T10:05:03.510295+0000","last_became_peered":"2024-09-10T10:05:03.510295+0000","last_unstale":"2024-09-10T10:05:04.497200+0000","last_undegraded":"2024-09-10T10:05:04.497200+0000","last_fullsized":"2024-09-10T10:05:04.497200+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:10:51.938408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.497438+0000","last_change":"2024-09-10T10:05:02.500925+0000","last_active":"2024-09-10T10:05:04.497438+0000","last_peered":"2024-09-10T10:05:04.497438+0000","last_clean":"2024-09-10T10:05:04.497438+0000","last_became_active":"2024-09-10T10:05:02.500602+0000","last_became_peered":"2024-09-10T10:05:02.500602+0000","last_unstale":"2024-09-10T10:05:04.497438+0000","last_undegraded":"2024-09-10T10:05:04.497438+0000","last_fullsized":"2024-09-10T10:05:04.497438+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:47.511293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235770+0000","last_change":"2024-09-10T10:05:03.524538+0000","last_active":"2024-09-10T10:05:05.235770+0000","last_peered":"2024-09-10T10:05:05.235770+0000","last_clean":"2024-09-10T10:05:05.235770+0000","last_became_active":"2024-09-10T10:05:03.524189+0000","last_became_peered":"2024-09-10T10:05:03.524189+0000","last_unstale":"2024-09-10T10:05:05.235770+0000","last_undegraded":"2024-09-10T10:05:05.235770+0000","last_fullsized":"2024-09-10T10:05:05.235770+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:09.168922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"142'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235899+0000","last_change":"2024-09-10T10:05:02.518318+0000","last_active":"2024-09-10T10:05:05.235899+0000","last_peered":"2024-09-10T10:05:05.235899+0000","last_clean":"2024-09-10T10:05:05.235899+0000","last_became_active":"2024-09-10T10:05:02.517943+0000","last_became_peered":"2024-09-10T10:05:02.517943+0000","last_unstale":"2024-09-10T10:05:05.235899+0000","last_undegraded":"2024-09-10T10:05:05.235899+0000","last_fullsized":"2024-09-10T10:05:05.235899+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:22.810821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236828+0000","last_change":"2024-09-10T10:05:03.513797+0000","last_active":"2024-09-10T10:05:05.236828+0000","last_peered":"2024-09-10T10:05:05.236828+0000","last_clean":"2024-09-10T10:05:05.236828+0000","last_became_active":"2024-09-10T10:05:03.513411+0000","last_became_peered":"2024-09-10T10:05:03.513411+0000","last_unstale":"2024-09-10T10:05:05.236828+0000","last_undegraded":"2024-09-10T10:05:05.236828+0000","last_fullsized":"2024-09-10T10:05:05.236828+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:16.755549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501896+0000","last_change":"2024-09-10T10:05:02.510761+0000","last_active":"2024-09-10T10:05:04.501896+0000","last_peered":"2024-09-10T10:05:04.501896+0000","last_clean":"2024-09-10T10:05:04.501896+0000","last_became_active":"2024-09-10T10:05:02.510424+0000","last_became_peered":"2024-09-10T10:05:02.510424+0000","last_unstale":"2024-09-10T10:05:04.501896+0000","last_undegraded":"2024-09-10T10:05:04.501896+0000","last_fullsized":"2024-09-10T10:05:04.501896+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:31.415304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.496871+0000","last_change":"2024-09-10T10:05:03.529360+0000","last_active":"2024-09-10T10:05:04.496871+0000","last_peered":"2024-09-10T10:05:04.496871+0000","last_clean":"2024-09-10T10:05:04.496871+0000","last_became_active":"2024-09-10T10:05:03.528524+0000","last_became_peered":"2024-09-10T10:05:03.528524+0000","last_unstale":"2024-09-10T10:05:04.496871+0000","last_undegraded":"2024-09-10T10:05:04.496871+0000","last_fullsized":"2024-09-10T10:05:04.496871+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:48:40.262409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.496861+0000","last_change":"2024-09-10T10:05:02.507652+0000","last_active":"2024-09-10T10:05:04.496861+0000","last_peered":"2024-09-10T10:05:04.496861+0000","last_clean":"2024-09-10T10:05:04.496861+0000","last_became_active":"2024-09-10T10:05:02.507337+0000","last_became_peered":"2024-09-10T10:05:02.507337+0000","last_unstale":"2024-09-10T10:05:04.496861+0000","last_undegraded":"2024-09-10T10:05:04.496861+0000","last_fullsized":"2024-09-10T10:05:04.496861+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:49.343094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.496735+0000","last_change":"2024-09-10T10:05:03.522074+0000","last_active":"2024-09-10T10:05:04.496735+0000","last_peered":"2024-09-10T10:05:04.496735+0000","last_clean":"2024-09-10T10:05:04.496735+0000","last_became_active":"2024-09-10T10:05:03.521757+0000","last_became_peered":"2024-09-10T10:05:03.521757+0000","last_unstale":"2024-09-10T10:05:04.496735+0000","last_undegraded":"2024-09-10T10:05:04.496735+0000","last_fullsized":"2024-09-10T10:05:04.496735+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:24:28.447434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236161+0000","last_change":"2024-09-10T10:05:02.513851+0000","last_active":"2024-09-10T10:05:05.236161+0000","last_peered":"2024-09-10T10:05:05.236161+0000","last_clean":"2024-09-10T10:05:05.236161+0000","last_became_active":"2024-09-10T10:05:02.513023+0000","last_became_peered":"2024-09-10T10:05:02.513023+0000","last_unstale":"2024-09-10T10:05:05.236161+0000","last_undegraded":"2024-09-10T10:05:05.236161+0000","last_fullsized":"2024-09-10T10:05:05.236161+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:24:30.020386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236159+0000","last_change":"2024-09-10T10:05:03.520668+0000","last_active":"2024-09-10T10:05:05.236159+0000","last_peered":"2024-09-10T10:05:05.236159+0000","last_clean":"2024-09-10T10:05:05.236159+0000","last_became_active":"2024-09-10T10:05:03.520074+0000","last_became_peered":"2024-09-10T10:05:03.520074+0000","last_unstale":"2024-09-10T10:05:05.236159+0000","last_undegraded":"2024-09-10T10:05:05.236159+0000","last_fullsized":"2024-09-10T10:05:05.236159+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:31.592895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.234884+0000","last_change":"2024-09-10T10:05:02.514912+0000","last_active":"2024-09-10T10:05:05.234884+0000","last_peered":"2024-09-10T10:05:05.234884+0000","last_clean":"2024-09-10T10:05:05.234884+0000","last_became_active":"2024-09-10T10:05:02.514564+0000","last_became_peered":"2024-09-10T10:05:02.514564+0000","last_unstale":"2024-09-10T10:05:05.234884+0000","last_undegraded":"2024-09-10T10:05:05.234884+0000","last_fullsized":"2024-09-10T10:05:05.234884+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:40.427386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236026+0000","last_change":"2024-09-10T10:05:03.530786+0000","last_active":"2024-09-10T10:05:05.236026+0000","last_peered":"2024-09-10T10:05:05.236026+0000","last_clean":"2024-09-10T10:05:05.236026+0000","last_became_active":"2024-09-10T10:05:03.530388+0000","last_became_peered":"2024-09-10T10:05:03.530388+0000","last_unstale":"2024-09-10T10:05:05.236026+0000","last_undegraded":"2024-09-10T10:05:05.236026+0000","last_fullsized":"2024-09-10T10:05:05.236026+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:19.691334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.502351+0000","last_change":"2024-09-10T10:05:02.512066+0000","last_active":"2024-09-10T10:05:04.502351+0000","last_peered":"2024-09-10T10:05:04.502351+0000","last_clean":"2024-09-10T10:05:04.502351+0000","last_became_active":"2024-09-10T10:05:02.511801+0000","last_became_peered":"2024-09-10T10:05:02.511801+0000","last_unstale":"2024-09-10T10:05:04.502351+0000","last_undegraded":"2024-09-10T10:05:04.502351+0000","last_fullsized":"2024-09-10T10:05:04.502351+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:23.194763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.235334+0000","last_change":"2024-09-10T10:05:03.525632+0000","last_active":"2024-09-10T10:05:05.235334+0000","last_peered":"2024-09-10T10:05:05.235334+0000","last_clean":"2024-09-10T10:05:05.235334+0000","last_became_active":"2024-09-10T10:05:03.524718+0000","last_became_peered":"2024-09-10T10:05:03.524718+0000","last_unstale":"2024-09-10T10:05:05.235334+0000","last_undegraded":"2024-09-10T10:05:05.235334+0000","last_fullsized":"2024-09-10T10:05:05.235334+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:17.711915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236017+0000","last_change":"2024-09-10T10:05:02.503531+0000","last_active":"2024-09-10T10:05:05.236017+0000","last_peered":"2024-09-10T10:05:05.236017+0000","last_clean":"2024-09-10T10:05:05.236017+0000","last_became_active":"2024-09-10T10:05:02.503255+0000","last_became_peered":"2024-09-10T10:05:02.503255+0000","last_unstale":"2024-09-10T10:05:05.236017+0000","last_undegraded":"2024-09-10T10:05:05.236017+0000","last_fullsized":"2024-09-10T10:05:05.236017+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:29.627730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.497182+0000","last_change":"2024-09-10T10:05:03.519503+0000","last_active":"2024-09-10T10:05:04.497182+0000","last_peered":"2024-09-10T10:05:04.497182+0000","last_clean":"2024-09-10T10:05:04.497182+0000","last_became_active":"2024-09-10T10:05:03.519225+0000","last_became_peered":"2024-09-10T10:05:03.519225+0000","last_unstale":"2024-09-10T10:05:04.497182+0000","last_undegraded":"2024-09-10T10:05:04.497182+0000","last_fullsized":"2024-09-10T10:05:04.497182+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:02.484247+0000","last_clean_scrub_stamp":"2024-09-10T10:05:02.484247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:56.259333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:05.236475+0000","last_change":"2024-09-10T10:05:02.509574+0000","last_active":"2024-09-10T10:05:05.236475+0000","last_peered":"2024-09-10T10:05:05.236475+0000","last_clean":"2024-09-10T10:05:05.236475+0000","last_became_active":"2024-09-10T10:05:02.509283+0000","last_became_peered":"2024-09-10T10:05:02.509283+0000","last_unstale":"2024-09-10T10:05:05.236475+0000","last_undegraded":"2024-09-10T10:05:05.236475+0000","last_fullsized":"2024-09-10T10:05:05.236475+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:01.481493+0000","last_clean_scrub_stamp":"2024-09-10T10:05:01.481493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:25.098679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":429,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T10:05:04.501611+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:05:04.501611+0000","last_peered":"2024-09-10T10:05:04.501611+0000","last_clean":"2024-09-10T10:05:04.501611+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:05:04.501611+0000","last_undegraded":"2024-09-10T10:05:04.501611+0000","last_fullsized":"2024-09-10T10:05:04.501611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33536,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338304,"statfs":{"total":96636764160,"available":96602423296,"internally_reserved":0,"allocated":2396160,"data_stored":1376829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1781760,"data_stored":784129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1785856,"data_stored":784711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1794048,"data_stored":785789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542231,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37096,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334744,"statfs":{"total":96636764160,"available":96598777856,"internally_reserved":0,"allocated":1781760,"data_stored":784141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37092,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334748,"statfs":{"total":96636764160,"available":96598781952,"internally_reserved":0,"allocated":1781760,"data_stored":784197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1781760,"data_stored":784141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37664,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334176,"statfs":{"total":96636764160,"available":96598196224,"internally_reserved":0,"allocated":2371584,"data_stored":1374475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:05:07.967 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:05:07.967 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:05:08.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc95e000640 1 Processor -- start 2024-09-10T10:05:08.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc95e000640 1 -- start start 2024-09-10T10:05:08.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc95e000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc9580cd510 0x7fc9580cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc95e000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc95806e090 0x7fc95806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc95e000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc958072150 con 0x7fc9580cfe30 2024-09-10T10:05:08.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc95e000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc95815d9a0 con 0x7fc95806e090 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc95e000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc95806d2d0 con 0x7fc9580cd510 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc957400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc9580cfe30 0x7fc958071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53230/0 (socket says 172.21.15.110:53230) 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc957400640 1 -- 172.21.15.110:0/2980935991 learned_addr learned my addr 172.21.15.110:0/2980935991 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.146+0000 7fc957e00640 1 --2- 172.21.15.110:0/2980935991 >> v2:172.21.15.110:3300/0 conn(0x7fc95806e090 0x7fc95806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc956a00640 1 --2- 172.21.15.110:0/2980935991 >> v2:172.21.15.145:3300/0 conn(0x7fc9580cd510 0x7fc9580cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc956000640 1 -- 172.21.15.110:0/2980935991 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111380735 0 0) 0x7fc958072150 con 0x7fc9580cfe30 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc956000640 1 -- 172.21.15.110:0/2980935991 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc93c003530 con 0x7fc9580cfe30 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc957e00640 1 -- 172.21.15.110:0/2980935991 >> v2:172.21.15.145:3300/0 conn(0x7fc9580cd510 msgr2=0x7fc9580cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc957e00640 1 --2- 172.21.15.110:0/2980935991 >> v2:172.21.15.145:3300/0 conn(0x7fc9580cd510 0x7fc9580cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc957e00640 1 -- 172.21.15.110:0/2980935991 >> v1:172.21.15.110:6789/0 conn(0x7fc9580cfe30 legacy=0x7fc958071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc957e00640 1 -- 172.21.15.110:0/2980935991 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95806d5b0 con 0x7fc95806e090 2024-09-10T10:05:08.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc957e00640 1 --2- 172.21.15.110:0/2980935991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc95806e090 0x7fc95806e4e0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc9580ce0f0 tx=0x7fc95001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedf86960e7fc294 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:08.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc956000640 1 -- 172.21.15.110:0/2980935991 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9500b2070 con 0x7fc95806e090 2024-09-10T10:05:08.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc956000640 1 -- 172.21.15.110:0/2980935991 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9500a5870 con 0x7fc95806e090 2024-09-10T10:05:08.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.150+0000 7fc956000640 1 -- 172.21.15.110:0/2980935991 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9500a5c40 con 0x7fc95806e090 2024-09-10T10:05:08.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- 172.21.15.110:0/2980935991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc95806e090 msgr2=0x7fc95806e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 --2- 172.21.15.110:0/2980935991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc95806e090 0x7fc95806e4e0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc9580ce0f0 tx=0x7fc95001c8a0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- 172.21.15.110:0/2980935991 shutdown_connections 2024-09-10T10:05:08.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 --2- 172.21.15.110:0/2980935991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc95806e090 0x7fc95806e4e0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 --2- 172.21.15.110:0/2980935991 >> v2:172.21.15.145:3300/0 conn(0x7fc9580cd510 0x7fc9580cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- 172.21.15.110:0/2980935991 >> 172.21.15.110:0/2980935991 conn(0x7fc95806bc50 msgr2=0x7fc95806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:08.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- 172.21.15.110:0/2980935991 shutdown_connections 2024-09-10T10:05:08.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- 172.21.15.110:0/2980935991 wait complete. 2024-09-10T10:05:08.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 Processor -- start 2024-09-10T10:05:08.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- start start 2024-09-10T10:05:08.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc95806e090 0x7fc9581562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9580cd510 0x7fc9581514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9580cfe30 0x7fc95814b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc958169180 con 0x7fc9580cd510 2024-09-10T10:05:08.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc95815d6c0 con 0x7fc9580cfe30 2024-09-10T10:05:08.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc95e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc958169300 con 0x7fc95806e090 2024-09-10T10:05:08.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc956a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9580cd510 0x7fc9581514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc956a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9580cd510 0x7fc9581514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47678/0 (socket says 172.21.15.110:47678) 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc956a00640 1 -- 172.21.15.110:0/3983714895 learned_addr learned my addr 172.21.15.110:0/3983714895 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc957400640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc95806e090 0x7fc9581562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc957e00640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9580cfe30 0x7fc95814b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc956a00640 1 -- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc95806e090 msgr2=0x7fc9581562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc956a00640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc95806e090 0x7fc9581562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc956a00640 1 -- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9580cfe30 msgr2=0x7fc95814b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc956a00640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9580cfe30 0x7fc95814b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.154+0000 7fc956a00640 1 -- 172.21.15.110:0/3983714895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc958156820 con 0x7fc9580cd510 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc956a00640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9580cd510 0x7fc9581514f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc94400e8a0 tx=0x7fc94400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc94f400640 1 -- 172.21.15.110:0/3983714895 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944017070 con 0x7fc9580cd510 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc94f400640 1 -- 172.21.15.110:0/3983714895 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94400c940 con 0x7fc9580cd510 2024-09-10T10:05:08.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9581429f0 con 0x7fc9580cd510 2024-09-10T10:05:08.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc94f400640 1 -- 172.21.15.110:0/3983714895 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94400ccf0 con 0x7fc9580cd510 2024-09-10T10:05:08.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9580b2c40 con 0x7fc9580cd510 2024-09-10T10:05:08.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9580b2f80 con 0x7fc9580cd510 2024-09-10T10:05:08.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc94f400640 1 -- 172.21.15.110:0/3983714895 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fc94401a070 con 0x7fc9580cd510 2024-09-10T10:05:08.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc94f400640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc938067ef0 0x7fc93806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc94f400640 1 -- 172.21.15.110:0/3983714895 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fc944086650 con 0x7fc9580cd510 2024-09-10T10:05:08.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.158+0000 7fc957400640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc938067ef0 0x7fc93806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.162+0000 7fc957400640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc938067ef0 0x7fc93806a3b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc948005890 tx=0x7fc948009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:08.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.162+0000 7fc94f400640 1 -- 172.21.15.110:0/3983714895 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc9580b2f80 con 0x7fc9580cd510 2024-09-10T10:05:08.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.378+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc95806e4e0 con 0x7fc9580cd510 2024-09-10T10:05:08.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.378+0000 7fc94f400640 1 -- 172.21.15.110:0/3983714895 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7fc944056910 con 0x7fc9580cd510 2024-09-10T10:05:08.384 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:05:08.384 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:05:08.384 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:05:08.384 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:05:08.384 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:05:08.384 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc938067ef0 msgr2=0x7fc93806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc938067ef0 0x7fc93806a3b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc948005890 tx=0x7fc948009040 comp rx=0 tx=0).stop 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9580cd510 msgr2=0x7fc9581514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9580cd510 0x7fc9581514f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc94400e8a0 tx=0x7fc94400eca0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 shutdown_connections 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc938067ef0 0x7fc93806a3b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc9580cfe30 0x7fc95814b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9580cd510 0x7fc9581514f0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 --2- 172.21.15.110:0/3983714895 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc95806e090 0x7fc9581562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 >> 172.21.15.110:0/3983714895 conn(0x7fc95806bc50 msgr2=0x7fc9581434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 shutdown_connections 2024-09-10T10:05:08.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.382+0000 7fc95e000640 1 -- 172.21.15.110:0/3983714895 wait complete. 2024-09-10T10:05:08.406 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc75000640 1 Processor -- start 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc75000640 1 -- start start 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc75000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700b1c60 0x7fdc70142320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc75000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdc700731e0 0x7fdc70073620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc75000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7015c790 con 0x7fdc7015fd80 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc75000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdc70073b60 con 0x7fdc700731e0 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc75000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdc700b2040 con 0x7fdc700b1c60 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc67e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fdc7015fd80 0x7fdc7013f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59918/0 (socket says 172.21.15.110:59918) 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc67e00640 1 -- 172.21.15.110:0/3736469842 learned_addr learned my addr 172.21.15.110:0/3736469842 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc6f400640 1 --2- 172.21.15.110:0/3736469842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700b1c60 0x7fdc70142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.538+0000 7fdc6fe00640 1 --2- 172.21.15.110:0/3736469842 >> v2:172.21.15.145:3300/0 conn(0x7fdc700731e0 0x7fdc70073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6ea00640 1 -- 172.21.15.110:0/3736469842 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385580726 0 0) 0x7fdc58001120 con 0x7fdc7015fd80 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6ea00640 1 -- 172.21.15.110:0/3736469842 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc54003530 con 0x7fdc7015fd80 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6f400640 1 -- 172.21.15.110:0/3736469842 >> v2:172.21.15.145:3300/0 conn(0x7fdc700731e0 msgr2=0x7fdc70073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6f400640 1 --2- 172.21.15.110:0/3736469842 >> v2:172.21.15.145:3300/0 conn(0x7fdc700731e0 0x7fdc70073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6f400640 1 -- 172.21.15.110:0/3736469842 >> v1:172.21.15.110:6789/0 conn(0x7fdc7015fd80 legacy=0x7fdc7013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6f400640 1 -- 172.21.15.110:0/3736469842 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc70142860 con 0x7fdc700b1c60 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6fe00640 1 --2- 172.21.15.110:0/3736469842 >> v2:172.21.15.145:3300/0 conn(0x7fdc700731e0 0x7fdc70073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6f400640 1 --2- 172.21.15.110:0/3736469842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700b1c60 0x7fdc70142320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc600097c0 tx=0x7fdc6001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c44610ae96d6f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6ea00640 1 -- 172.21.15.110:0/3736469842 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc600b0070 con 0x7fdc700b1c60 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6ea00640 1 -- 172.21.15.110:0/3736469842 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc600ab600 con 0x7fdc700b1c60 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.542+0000 7fdc6ea00640 1 -- 172.21.15.110:0/3736469842 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc600ab9b0 con 0x7fdc700b1c60 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- 172.21.15.110:0/3736469842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700b1c60 msgr2=0x7fdc70142320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 --2- 172.21.15.110:0/3736469842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700b1c60 0x7fdc70142320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc600097c0 tx=0x7fdc6001c4a0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- 172.21.15.110:0/3736469842 shutdown_connections 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 --2- 172.21.15.110:0/3736469842 >> v2:172.21.15.145:3300/0 conn(0x7fdc700731e0 0x7fdc70073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 --2- 172.21.15.110:0/3736469842 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700b1c60 0x7fdc70142320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- 172.21.15.110:0/3736469842 >> 172.21.15.110:0/3736469842 conn(0x7fdc700b42d0 msgr2=0x7fdc700b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- 172.21.15.110:0/3736469842 shutdown_connections 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- 172.21.15.110:0/3736469842 wait complete. 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 Processor -- start 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- start start 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700731e0 0x7fdc701560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc700b1c60 0x7fdc7015e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc7015fd80 0x7fdc70141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7015d460 con 0x7fdc700b1c60 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70160b50 con 0x7fdc7015fd80 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc75000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7015d5e0 con 0x7fdc700731e0 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc700b1c60 0x7fdc7015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc700b1c60 0x7fdc7015e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37636/0 (socket says 172.21.15.110:37636) 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 -- 172.21.15.110:0/869120695 learned_addr learned my addr 172.21.15.110:0/869120695 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc6f400640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700731e0 0x7fdc701560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc6fe00640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc7015fd80 0x7fdc70141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 -- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700731e0 msgr2=0x7fdc701560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700731e0 0x7fdc701560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 -- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc7015fd80 msgr2=0x7fdc70141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc7015fd80 0x7fdc70141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 -- 172.21.15.110:0/869120695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7013fa80 con 0x7fdc700b1c60 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67e00640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc700b1c60 0x7fdc7015e960 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c730 tx=0x7fdc5800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.546+0000 7fdc67400640 1 -- 172.21.15.110:0/869120695 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58017070 con 0x7fdc700b1c60 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc67400640 1 -- 172.21.15.110:0/869120695 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc580058c0 con 0x7fdc700b1c60 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc67400640 1 -- 172.21.15.110:0/869120695 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800ce60 con 0x7fdc700b1c60 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc70139f80 con 0x7fdc700b1c60 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7013a540 con 0x7fdc700b1c60 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc70073620 con 0x7fdc700b1c60 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc67400640 1 -- 172.21.15.110:0/869120695 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fdc5801a070 con 0x7fdc700b1c60 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc67400640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc4c067ef0 0x7fdc4c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc67400640 1 -- 172.21.15.110:0/869120695 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fdc58085a60 con 0x7fdc700b1c60 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.550+0000 7fdc6f400640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc4c067ef0 0x7fdc4c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.554+0000 7fdc6f400640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc4c067ef0 0x7fdc4c06a3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc600ad000 tx=0x7fdc600a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.554+0000 7fdc67400640 1 -- 172.21.15.110:0/869120695 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdc7015c790 con 0x7fdc700b1c60 2024-09-10T10:05:08.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.745+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc7013b7c0 con 0x7fdc700b1c60 2024-09-10T10:05:08.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.745+0000 7fdc67400640 1 -- 172.21.15.110:0/869120695 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5239 (secure 0 0 0) 0x7fdc58055620 con 0x7fdc700b1c60 2024-09-10T10:05:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:08.751 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":97,"btime":"2024-09-10T10:05:06:522341+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2050751760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2050751760},{"type":"v1","addr":"172.21.15.110:6835","nonce":2050751760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6743,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/4094794865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4094794865},{"type":"v1","addr":"172.21.15.145:6834","nonce":4094794865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2883879805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2883879805},{"type":"v1","addr":"172.21.15.110:6834","nonce":2883879805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:02.880808+0000","modified":"2024-09-10T10:05:05.895746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1325843861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1325843861},{"type":"v1","addr":"172.21.15.145:6835","nonce":1325843861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8178,"qdb_cluster":[8178]},"id":11}]} 2024-09-10T10:05:08.751 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 97 2024-09-10T10:05:08.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc4c067ef0 msgr2=0x7fdc4c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc4c067ef0 0x7fdc4c06a3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc600ad000 tx=0x7fdc600a6000 comp rx=0 tx=0).stop 2024-09-10T10:05:08.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc700b1c60 msgr2=0x7fdc7015e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:08.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc700b1c60 0x7fdc7015e960 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c730 tx=0x7fdc5800cb30 comp rx=0 tx=0).stop 2024-09-10T10:05:08.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 shutdown_connections 2024-09-10T10:05:08.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdc4c067ef0 0x7fdc4c06a3b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdc7015fd80 0x7fdc70141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc700b1c60 0x7fdc7015e960 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 --2- 172.21.15.110:0/869120695 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdc700731e0 0x7fdc701560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:08.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 >> 172.21.15.110:0/869120695 conn(0x7fdc700b42d0 msgr2=0x7fdc7013e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:08.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 shutdown_connections 2024-09-10T10:05:08.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:08.749+0000 7fdc75000640 1 -- 172.21.15.110:0/869120695 wait complete. 2024-09-10T10:05:09.127 INFO:tasks.ceph.mds.b.smithi145.stderr:2024-09-10T10:05:09.126+0000 7f7e69200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:05:09.175 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:05:02.880808+0000', 'modified': '2024-09-10T10:05:05.895746+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8178}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8178': {'gid': 8178, 'name': 'b', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.145:6835/1325843861', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6833', 'nonce': 1325843861}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 1325843861}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8178, 'qdb_cluster': [8178]} 2024-09-10T10:05:09.176 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:05:09.176 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T10:05:09.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.305+0000 7fa724c00640 1 Processor -- start 2024-09-10T10:05:09.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.305+0000 7fa724c00640 1 -- start start 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa724c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa7200cfb90 0x7fa72015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa724c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7200cdcb0 0x7fa72006db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa724c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa72015d4d0 con 0x7fa7200b3f40 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa724c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa72006d130 con 0x7fa7200cfb90 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa724c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa72015d7b0 con 0x7fa7200cdcb0 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71d400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa7200b3f40 0x7fa7200cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59924/0 (socket says 172.21.15.110:59924) 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71d400640 1 -- 172.21.15.110:0/798819777 learned_addr learned my addr 172.21.15.110:0/798819777 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71de00640 1 --2- 172.21.15.110:0/798819777 >> v2:172.21.15.145:3300/0 conn(0x7fa7200cfb90 0x7fa72015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71e800640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7200cdcb0 0x7fa72006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3641654560 0 0) 0x7fa72015d4d0 con 0x7fa7200b3f40 2024-09-10T10:05:09.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa704003530 con 0x7fa7200b3f40 2024-09-10T10:05:09.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3573275721 0 0) 0x7fa704003530 con 0x7fa7200b3f40 2024-09-10T10:05:09.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7200cdcb0 msgr2=0x7fa72006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7200cdcb0 0x7fa72006db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 >> v2:172.21.15.145:3300/0 conn(0x7fa7200cfb90 msgr2=0x7fa72015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 --2- 172.21.15.110:0/798819777 >> v2:172.21.15.145:3300/0 conn(0x7fa7200cfb90 0x7fa72015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72006d380 con 0x7fa7200b3f40 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fa70c001680 con 0x7fa7200b3f40 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 >> v1:172.21.15.110:6789/0 conn(0x7fa7200b3f40 legacy=0x7fa7200cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7200cdcb0 0x7fa7040039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa704003f90 0x7fa704004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704004d50 0x7fa7040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa72006d130 con 0x7fa704003f90 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa72015d7b0 con 0x7fa704004d50 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa70400ba80 con 0x7fa7200cdcb0 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71de00640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7200cdcb0 0x7fa7040039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71d400640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa704003f90 0x7fa704004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71e800640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704004d50 0x7fa7040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71d400640 1 -- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7200cdcb0 msgr2=0x7fa7040039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71d400640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7200cdcb0 0x7fa7040039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71d400640 1 -- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704004d50 msgr2=0x7fa7040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71d400640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704004d50 0x7fa7040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71d400640 1 -- 172.21.15.110:0/798819777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa704003530 con 0x7fa704003f90 2024-09-10T10:05:09.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.309+0000 7fa71e800640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704004d50 0x7fa7040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:09.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa71d400640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa704003f90 0x7fa704004810 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa7200b5bc0 tx=0x7fa70c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe6344a8550f4d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c007c90 con 0x7fa704003f90 2024-09-10T10:05:09.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c0a9020 con 0x7fa704003f90 2024-09-10T10:05:09.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa71ca00640 1 -- 172.21.15.110:0/798819777 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c005ae0 con 0x7fa704003f90 2024-09-10T10:05:09.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa724c00640 1 -- 172.21.15.110:0/798819777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa704003f90 msgr2=0x7fa704004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa724c00640 1 --2- 172.21.15.110:0/798819777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa704003f90 0x7fa704004810 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa7200b5bc0 tx=0x7fa70c01cbc0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa71e800640 1 -- 172.21.15.110:0/798819777 reap_dead start 2024-09-10T10:05:09.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa724c00640 1 -- 172.21.15.110:0/798819777 shutdown_connections 2024-09-10T10:05:09.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa724c00640 1 -- 172.21.15.110:0/798819777 >> 172.21.15.110:0/798819777 conn(0x7fa720062d10 msgr2=0x7fa72006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa724c00640 1 -- 172.21.15.110:0/798819777 shutdown_connections 2024-09-10T10:05:09.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa724c00640 1 -- 172.21.15.110:0/798819777 wait complete. 2024-09-10T10:05:09.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.313+0000 7fa724c00640 1 Processor -- start 2024-09-10T10:05:09.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 -- start start 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704003f90 0x7fa720145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7201586d0 0x7fa720158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7201f3800 0x7fa7201f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa72006e720 con 0x7fa7201f3800 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa72006e5a0 con 0x7fa704003f90 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa72006e8a0 con 0x7fa7201586d0 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7201f3800 0x7fa7201f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7201f3800 0x7fa7201f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37648/0 (socket says 172.21.15.110:37648) 2024-09-10T10:05:09.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 -- 172.21.15.110:0/2842739547 learned_addr learned my addr 172.21.15.110:0/2842739547 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71d400640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7201586d0 0x7fa720158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71de00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704003f90 0x7fa720145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 -- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7201586d0 msgr2=0x7fa720158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7201586d0 0x7fa720158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 -- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704003f90 msgr2=0x7fa720145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704003f90 0x7fa720145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 -- 172.21.15.110:0/2842739547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7201540b0 con 0x7fa7201f3800 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71d400640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7201586d0 0x7fa720158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa71e800640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7201f3800 0x7fa7201f3c80 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa7200ccd10 tx=0x7fa718011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa716000640 1 -- 172.21.15.110:0/2842739547 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71801d070 con 0x7fa7201f3800 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa716000640 1 -- 172.21.15.110:0/2842739547 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa718004250 con 0x7fa7201f3800 2024-09-10T10:05:09.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa716000640 1 -- 172.21.15.110:0/2842739547 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718015730 con 0x7fa7201f3800 2024-09-10T10:05:09.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 -- 172.21.15.110:0/2842739547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa720154390 con 0x7fa7201f3800 2024-09-10T10:05:09.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.317+0000 7fa724c00640 1 -- 172.21.15.110:0/2842739547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa720154950 con 0x7fa7201f3800 2024-09-10T10:05:09.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa716000640 1 -- 172.21.15.110:0/2842739547 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fa718020070 con 0x7fa7201f3800 2024-09-10T10:05:09.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa716000640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa700067f20 0x7fa70006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa716000640 1 -- 172.21.15.110:0/2842739547 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fa718085fe0 con 0x7fa7201f3800 2024-09-10T10:05:09.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa727fa4000 1 Processor -- start 2024-09-10T10:05:09.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa727fa4000 1 -- start start 2024-09-10T10:05:09.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa700067f20 0x7fa70006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa700067f20 0x7fa70006a3e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa710014510 tx=0x7fa71000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58eddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58e93d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e94290 0x564b58e94670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa727fa4000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564b58d7bb60 con 0x564b58eddb20 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa727fa4000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564b58ce2c70 con 0x564b58e94290 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa727fa4000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x564b58c88ed0 con 0x564b58ede440 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58eddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58eddf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37652/0 (socket says 172.21.15.110:37652) 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 -- 172.21.15.110:0/2290362722 learned_addr learned my addr 172.21.15.110:0/2290362722 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71d400640 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58e93d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 -- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 msgr2=0x564b58e93d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71e800640 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e94290 0x564b58e94670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58e93d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 -- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e94290 msgr2=0x564b58e94670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e94290 0x564b58e94670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 -- 172.21.15.110:0/2290362722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b58c9ae50 con 0x564b58eddb20 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71d400640 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58e93d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71e800640 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e94290 0x564b58e94670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.321+0000 7fa71de00640 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58eddf00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa710016810 tx=0x7fa710017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c14455f9d5da336 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa70aa00640 1 -- 172.21.15.110:0/2290362722 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710026070 con 0x564b58eddb20 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa70aa00640 1 -- 172.21.15.110:0/2290362722 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa710015b10 con 0x564b58eddb20 2024-09-10T10:05:09.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa70aa00640 1 -- 172.21.15.110:0/2290362722 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710022420 con 0x564b58eddb20 2024-09-10T10:05:09.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- 172.21.15.110:0/2290362722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 msgr2=0x564b58eddf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58eddf00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa710016810 tx=0x7fa710017000 comp rx=0 tx=0).stop 2024-09-10T10:05:09.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- 172.21.15.110:0/2290362722 shutdown_connections 2024-09-10T10:05:09.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e94290 0x564b58e94670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58e93d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2290362722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58eddf00 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- 172.21.15.110:0/2290362722 >> 172.21.15.110:0/2290362722 conn(0x564b58e9a600 msgr2=0x564b58e92630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- 172.21.15.110:0/2290362722 shutdown_connections 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- 172.21.15.110:0/2290362722 wait complete. 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- start start 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 Processor -- start 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 0x564b58e98df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58ede440 0x564b58e9ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58e99330 0x564b58e9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564b58d44b00 con 0x564b58ede440 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564b58cbd360 con 0x564b58eddb20 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x564b58cba2d0 con 0x564b58e99330 2024-09-10T10:05:09.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 10 client.0 _ll_register_callbacks cb 0x564b58e7f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58ede440 0x564b58e9ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58ede440 0x564b58e9ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37660/0 (socket says 172.21.15.110:37660) 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 -- 172.21.15.110:0/501410281 learned_addr learned my addr 172.21.15.110:0/501410281 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71de00640 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 0x564b58e98df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71e800640 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58e99330 0x564b58e9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 -- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58e99330 msgr2=0x564b58e9f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58e99330 0x564b58e9f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 -- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 msgr2=0x564b58e98df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 0x564b58e98df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 -- 172.21.15.110:0/501410281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b58c89120 con 0x564b58ede440 2024-09-10T10:05:09.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71e800640 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58e99330 0x564b58e9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa71d400640 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58ede440 0x564b58e9ecf0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0ab040 tx=0x7fa70c019960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce05daf28cc429 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa6ff400640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa6ff400640 1 -- 172.21.15.110:0/501410281 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0bc060 con 0x564b58ede440 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa6ff400640 1 -- 172.21.15.110:0/501410281 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fa70c004a00 con 0x564b58ede440 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- 172.21.15.110:0/501410281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa710015b10 con 0x564b58ede440 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa6ff400640 1 -- 172.21.15.110:0/501410281 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fa70c0c2050 con 0x564b58ede440 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 10 client.8289 fetch_fsmap learned FSMap version 97 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.325+0000 7fa727fa4000 1 -- 172.21.15.110:0/501410281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x564b58c82450 con 0x564b58ede440 2024-09-10T10:05:09.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa6ff400640 1 -- 172.21.15.110:0/501410281 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa710015b10 con 0x564b58ede440 2024-09-10T10:05:09.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa6ff400640 1 -- 172.21.15.110:0/501410281 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 97) ==== 3502+0+0 (secure 0 0 0) 0x7fa70c0c5020 con 0x564b58ede440 2024-09-10T10:05:09.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 10 client.8289 fetch_fsmap finished waiting for FSMap version 97 2024-09-10T10:05:09.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 10 client.8289 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:05:09.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 20 client.8289 populate_metadata read hostname 'smithi110' 2024-09-10T10:05:09.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] conn(0x564b58eec280 0x564b58eee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 4 client.8289 mds_command: new command op to 8178 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:05:09.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 -- 172.21.15.110:0/501410281 --> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b58c7cc00 con 0x564b58eec280 2024-09-10T10:05:09.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa71de00640 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] conn(0x564b58eec280 0x564b58eee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa71de00640 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] conn(0x564b58eec280 0x564b58eee670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e1e1652ba5910e server_cookie=71309e5c191896f1 in_seq=0 out_seq=0 2024-09-10T10:05:09.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa6ff400640 10 client.8289 ms_handle_connect on v2:172.21.15.145:6833/1325843861 2024-09-10T10:05:09.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa6ff400640 1 -- 172.21.15.110:0/501410281 <== mds.0 v2:172.21.15.145:6833/1325843861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b58c7cc00 con 0x564b58eec280 2024-09-10T10:05:09.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa6ff400640 10 client.8289 handle_command_reply: tid=0 2024-09-10T10:05:09.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 client.8289 shutdown 2024-09-10T10:05:09.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 -- 172.21.15.110:0/501410281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58ede440 msgr2=0x564b58e9ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58ede440 0x564b58e9ecf0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0ab040 tx=0x7fa70c019960 comp rx=0 tx=0).stop 2024-09-10T10:05:09.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 -- 172.21.15.110:0/501410281 shutdown_connections 2024-09-10T10:05:09.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] conn(0x564b58eec280 0x564b58eee670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58e99330 0x564b58e9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58ede440 0x564b58e9ecf0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa6ff400640 0 client.8289 ms_handle_reset on v2:172.21.15.145:6833/1325843861 2024-09-10T10:05:09.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 --2- 172.21.15.110:0/501410281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 0x564b58e98df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 -- 172.21.15.110:0/501410281 >> 172.21.15.110:0/501410281 conn(0x564b58e94290 msgr2=0x564b58e92d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 -- 172.21.15.110:0/501410281 shutdown_connections 2024-09-10T10:05:09.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 1 -- 172.21.15.110:0/501410281 wait complete. 2024-09-10T10:05:09.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.329+0000 7fa727fa4000 20 client.8289 trim_cache size 0 max 16384 2024-09-10T10:05:09.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 Processor -- start 2024-09-10T10:05:09.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- start start 2024-09-10T10:05:09.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58e99330 0x564b58edccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 0x564b58edd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58eee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201ff1e0 con 0x564b58e99330 2024-09-10T10:05:09.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa720157820 con 0x564b58eddb20 2024-09-10T10:05:09.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa720157e70 con 0x564b58ede440 2024-09-10T10:05:09.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58e99330 0x564b58edccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58e99330 0x564b58edccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37676/0 (socket says 172.21.15.110:37676) 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 -- 172.21.15.110:0/748253394 learned_addr learned my addr 172.21.15.110:0/748253394 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71d400640 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 0x564b58edd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71e800640 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58eee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 -- 172.21.15.110:0/748253394 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 msgr2=0x564b58eee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58eee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 -- 172.21.15.110:0/748253394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 msgr2=0x564b58edd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 0x564b58edd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 -- 172.21.15.110:0/748253394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b58ede270 con 0x564b58e99330 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa71de00640 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58e99330 0x564b58edccc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa710017fd0 tx=0x7fa71001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0437c545c24a474 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa70a000640 1 -- 172.21.15.110:0/748253394 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710026040 con 0x564b58e99330 2024-09-10T10:05:09.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa70a000640 1 -- 172.21.15.110:0/748253394 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa710022420 con 0x564b58e99330 2024-09-10T10:05:09.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa70a000640 1 -- 172.21.15.110:0/748253394 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71001a040 con 0x564b58e99330 2024-09-10T10:05:09.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- 172.21.15.110:0/748253394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58e99330 msgr2=0x564b58edccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58e99330 0x564b58edccc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa710017fd0 tx=0x7fa71001c000 comp rx=0 tx=0).stop 2024-09-10T10:05:09.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- 172.21.15.110:0/748253394 shutdown_connections 2024-09-10T10:05:09.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58eee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58eddb20 0x564b58edd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 --2- 172.21.15.110:0/748253394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58e99330 0x564b58edccc0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- 172.21.15.110:0/748253394 >> 172.21.15.110:0/748253394 conn(0x564b58e94290 msgr2=0x564b58e985f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- 172.21.15.110:0/748253394 shutdown_connections 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.349+0000 7fa727fa4000 1 -- 172.21.15.110:0/748253394 wait complete. 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 -- start start 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 Processor -- start 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e99330 0x564b58ef1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58f15170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58f156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564b58ce1f80 con 0x564b58eddb20 2024-09-10T10:05:09.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564b58c9aca0 con 0x564b58e99330 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x564b58ce72a0 con 0x564b58ede440 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 10 client.0 _ll_register_callbacks cb 0x7fa704003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58f15170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58f15170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37680/0 (socket says 172.21.15.110:37680) 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 -- 172.21.15.110:0/2316856837 learned_addr learned my addr 172.21.15.110:0/2316856837 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71de00640 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e99330 0x564b58ef1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71e800640 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58f156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 -- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 msgr2=0x564b58f156b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58f156b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 -- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e99330 msgr2=0x564b58ef1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e99330 0x564b58ef1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 -- 172.21.15.110:0/2316856837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b58c69fd0 con 0x564b58eddb20 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71d400640 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58f15170 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa70c01c630 tx=0x7fa70c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c923868a5220e0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa6ff400640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa6ff400640 1 -- 172.21.15.110:0/2316856837 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0070d0 con 0x564b58eddb20 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa6ff400640 1 -- 172.21.15.110:0/2316856837 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fa70c0a56a0 con 0x564b58eddb20 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 -- 172.21.15.110:0/2316856837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa710022420 con 0x564b58eddb20 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa6ff400640 1 -- 172.21.15.110:0/2316856837 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fa70c0cd050 con 0x564b58eddb20 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 10 client.8295 fetch_fsmap learned FSMap version 97 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 -- 172.21.15.110:0/2316856837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x564b58d65750 con 0x564b58eddb20 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa6ff400640 1 -- 172.21.15.110:0/2316856837 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa710022420 con 0x564b58eddb20 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa6ff400640 1 -- 172.21.15.110:0/2316856837 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 97) ==== 3502+0+0 (secure 0 0 0) 0x7fa70c0d0020 con 0x564b58eddb20 2024-09-10T10:05:09.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 10 client.8295 fetch_fsmap finished waiting for FSMap version 97 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 10 client.8295 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 20 client.8295 populate_metadata read hostname 'smithi110' 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] conn(0x564b58f15bf0 0x564b58f59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 4 client.8295 mds_command: new command op to 8178 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa727fa4000 1 -- 172.21.15.110:0/2316856837 --> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] -- command(tid 0: {"prefix": "status"}) -- 0x564b58cf03a0 con 0x564b58f15bf0 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.353+0000 7fa71de00640 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] conn(0x564b58f15bf0 0x564b58f59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa6ff400640 10 client.8295 ms_handle_connect on v2:172.21.15.145:6833/1325843861 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa71de00640 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] conn(0x564b58f15bf0 0x564b58f59830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d12294ab296422 server_cookie=d7b21f36ff7998a3 in_seq=0 out_seq=0 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa6ff400640 1 -- 172.21.15.110:0/2316856837 <== mds.0 v2:172.21.15.145:6833/1325843861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x564b58cf03a0 con 0x564b58f15bf0 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa6ff400640 10 client.8295 handle_command_reply: tid=0 2024-09-10T10:05:09.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 client.8295 shutdown 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 -- 172.21.15.110:0/2316856837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 msgr2=0x564b58f15170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58f15170 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa70c01c630 tx=0x7fa70c01cbc0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 -- 172.21.15.110:0/2316856837 shutdown_connections 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:6833/1325843861,v1:172.21.15.145:6835/1325843861] conn(0x564b58f15bf0 0x564b58f59830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x564b58ede440 0x564b58f156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564b58eddb20 0x564b58f15170 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa6ff400640 0 client.8295 ms_handle_reset on v2:172.21.15.145:6833/1325843861 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 --2- 172.21.15.110:0/2316856837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564b58e99330 0x564b58ef1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 -- 172.21.15.110:0/2316856837 >> 172.21.15.110:0/2316856837 conn(0x564b58e94290 msgr2=0x564b58edbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 -- 172.21.15.110:0/2316856837 shutdown_connections 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 1 -- 172.21.15.110:0/2316856837 wait complete. 2024-09-10T10:05:09.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa727fa4000 20 client.8295 trim_cache size 0 max 16384 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 8178, 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 5.8683629980000003, 2024-09-10T10:05:09.368 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 97, 2024-09-10T10:05:09.369 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 142, 2024-09-10T10:05:09.369 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 142, 2024-09-10T10:05:09.369 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 11.068208265999999 2024-09-10T10:05:09.369 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:05:09.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 -- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa700067f20 msgr2=0x7fa70006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa700067f20 0x7fa70006a3e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa710014510 tx=0x7fa71000b040 comp rx=0 tx=0).stop 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 -- 172.21.15.110:0/2842739547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7201f3800 msgr2=0x7fa7201f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7201f3800 0x7fa7201f3c80 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa7200ccd10 tx=0x7fa718011920 comp rx=0 tx=0).stop 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 -- 172.21.15.110:0/2842739547 shutdown_connections 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa700067f20 0x7fa70006a3e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7201f3800 0x7fa7201f3c80 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7201586d0 0x7fa720158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 --2- 172.21.15.110:0/2842739547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa704003f90 0x7fa720145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 -- 172.21.15.110:0/2842739547 >> 172.21.15.110:0/2842739547 conn(0x7fa720062d10 msgr2=0x7fa72006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 -- 172.21.15.110:0/2842739547 shutdown_connections 2024-09-10T10:05:09.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.357+0000 7fa724c00640 1 -- 172.21.15.110:0/2842739547 wait complete. 2024-09-10T10:05:09.382 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:05:09.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb05d400640 1 Processor -- start 2024-09-10T10:05:09.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb05d400640 1 -- start start 2024-09-10T10:05:09.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb05d400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb058155970 0x7fb0580cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb05d400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb058155d50 con 0x7fb05815c560 2024-09-10T10:05:09.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb05d400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0580b34d0 con 0x7fb05806f1e0 2024-09-10T10:05:09.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb05d400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0580b4570 con 0x7fb058155970 2024-09-10T10:05:09.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb055e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb05815c560 0x7fb05806ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59938/0 (socket says 172.21.15.110:59938) 2024-09-10T10:05:09.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb055e00640 1 -- 172.21.15.110:0/3781024546 learned_addr learned my addr 172.21.15.110:0/3781024546 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:09.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb056800640 1 --2- 172.21.15.110:0/3781024546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb058155970 0x7fb0580cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb055400640 1 -- 172.21.15.110:0/3781024546 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196844883 0 0) 0x7fb058155d50 con 0x7fb05815c560 2024-09-10T10:05:09.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb055400640 1 -- 172.21.15.110:0/3781024546 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb03c003530 con 0x7fb05815c560 2024-09-10T10:05:09.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb055400640 1 -- 172.21.15.110:0/3781024546 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 517097467 0 0) 0x7fb0580b34d0 con 0x7fb05806f1e0 2024-09-10T10:05:09.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb055400640 1 -- 172.21.15.110:0/3781024546 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb058155d50 con 0x7fb05806f1e0 2024-09-10T10:05:09.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb056800640 1 -- 172.21.15.110:0/3781024546 >> v1:172.21.15.145:6789/0 conn(0x7fb05806f1e0 legacy=0x7fb0580ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb056800640 1 -- 172.21.15.110:0/3781024546 >> v1:172.21.15.110:6789/0 conn(0x7fb05815c560 legacy=0x7fb05806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.513+0000 7fb056800640 1 -- 172.21.15.110:0/3781024546 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0580b37b0 con 0x7fb058155970 2024-09-10T10:05:09.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb056800640 1 --2- 172.21.15.110:0/3781024546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb058155970 0x7fb0580cc8c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb0480097f0 tx=0x7fb04801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1dac3709b4277c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb055400640 1 -- 172.21.15.110:0/3781024546 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0480ab620 con 0x7fb058155970 2024-09-10T10:05:09.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb055400640 1 -- 172.21.15.110:0/3781024546 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0480b0070 con 0x7fb058155970 2024-09-10T10:05:09.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb055400640 1 -- 172.21.15.110:0/3781024546 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0480ab9c0 con 0x7fb058155970 2024-09-10T10:05:09.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb05d400640 1 -- 172.21.15.110:0/3781024546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb058155970 msgr2=0x7fb0580cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb05d400640 1 --2- 172.21.15.110:0/3781024546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb058155970 0x7fb0580cc8c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb0480097f0 tx=0x7fb04801c420 comp rx=0 tx=0).stop 2024-09-10T10:05:09.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb05d400640 1 -- 172.21.15.110:0/3781024546 shutdown_connections 2024-09-10T10:05:09.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb05d400640 1 --2- 172.21.15.110:0/3781024546 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb058155970 0x7fb0580cc8c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb05d400640 1 -- 172.21.15.110:0/3781024546 >> 172.21.15.110:0/3781024546 conn(0x7fb05806bc50 msgr2=0x7fb05806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb05d400640 1 -- 172.21.15.110:0/3781024546 shutdown_connections 2024-09-10T10:05:09.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.517+0000 7fb05d400640 1 -- 172.21.15.110:0/3781024546 wait complete. 2024-09-10T10:05:09.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 Processor -- start 2024-09-10T10:05:09.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 -- start start 2024-09-10T10:05:09.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb05806f1e0 0x7fb05814a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb058155970 0x7fb05814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb05815c560 0x7fb058152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0580cf340 con 0x7fb05806f1e0 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0580cce00 con 0x7fb058155970 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb0580cf4c0 con 0x7fb05815c560 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb05806f1e0 0x7fb05814a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb05806f1e0 0x7fb05814a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37688/0 (socket says 172.21.15.110:37688) 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 -- 172.21.15.110:0/1780161714 learned_addr learned my addr 172.21.15.110:0/1780161714 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb055e00640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb058155970 0x7fb05814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb057200640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb05815c560 0x7fb058152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 -- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb05815c560 msgr2=0x7fb058152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb05815c560 0x7fb058152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 -- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb058155970 msgr2=0x7fb05814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb058155970 0x7fb05814f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 -- 172.21.15.110:0/1780161714 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05814aaa0 con 0x7fb05806f1e0 2024-09-10T10:05:09.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb057200640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb05815c560 0x7fb058152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:09.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb056800640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb05806f1e0 0x7fb05814a560 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fb0480097f0 tx=0x7fb04801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb04ea00640 1 -- 172.21.15.110:0/1780161714 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0480abb20 con 0x7fb05806f1e0 2024-09-10T10:05:09.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb04ea00640 1 -- 172.21.15.110:0/1780161714 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0480abcc0 con 0x7fb05806f1e0 2024-09-10T10:05:09.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb04ea00640 1 -- 172.21.15.110:0/1780161714 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0480bf610 con 0x7fb05806f1e0 2024-09-10T10:05:09.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.521+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb058148720 con 0x7fb05806f1e0 2024-09-10T10:05:09.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.525+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb058150220 con 0x7fb05806f1e0 2024-09-10T10:05:09.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.525+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0580cc8c0 con 0x7fb05806f1e0 2024-09-10T10:05:09.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.525+0000 7fb04ea00640 1 -- 172.21.15.110:0/1780161714 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fb0480bd020 con 0x7fb05806f1e0 2024-09-10T10:05:09.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.525+0000 7fb04ea00640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb038067ef0 0x7fb03806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:09.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.525+0000 7fb04ea00640 1 -- 172.21.15.110:0/1780161714 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fb048120bf0 con 0x7fb05806f1e0 2024-09-10T10:05:09.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.525+0000 7fb055e00640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb038067ef0 0x7fb03806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:09.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.525+0000 7fb055e00640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb038067ef0 0x7fb03806a3b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb058166650 tx=0x7fb044009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:09.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.529+0000 7fb04ea00640 1 -- 172.21.15.110:0/1780161714 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb0480160d0 con 0x7fb05806f1e0 2024-09-10T10:05:09.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.721+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb05815c140 con 0x7fb05806f1e0 2024-09-10T10:05:09.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.721+0000 7fb04ea00640 1 -- 172.21.15.110:0/1780161714 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) ==== 53+0+73 (secure 0 0 0) 0x7fb0480f0dc0 con 0x7fb05806f1e0 2024-09-10T10:05:09.728 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:05:09.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb038067ef0 msgr2=0x7fb03806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb038067ef0 0x7fb03806a3b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb058166650 tx=0x7fb044009040 comp rx=0 tx=0).stop 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb05806f1e0 msgr2=0x7fb05814a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb05806f1e0 0x7fb05814a560 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fb0480097f0 tx=0x7fb04801ac60 comp rx=0 tx=0).stop 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 shutdown_connections 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb038067ef0 0x7fb03806a3b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb05815c560 0x7fb058152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb058155970 0x7fb05814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 --2- 172.21.15.110:0/1780161714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb05806f1e0 0x7fb05814a560 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 >> 172.21.15.110:0/1780161714 conn(0x7fb05806bc50 msgr2=0x7fb058151150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 shutdown_connections 2024-09-10T10:05:09.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09.725+0000 7fb05d400640 1 -- 172.21.15.110:0/1780161714 wait complete. 2024-09-10T10:05:09.749 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:05:09.749 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:05:09.749 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:05:09.749 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:05:09.750 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:09.750 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:05:09.750 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:05:09.750 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:05:09.750 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:05:09.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:09.750 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:05:09.759 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:05:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:05:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:05:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:05:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:09.759 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:05:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42065sec preferred_lft 42065sec 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:05:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:09.762 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:05:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:05:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:05:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:09.762 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:05:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:05:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:05:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:09.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:09.763 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:05:09.821 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:05:09.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:09.824 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:09.876 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:05:09.877 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:05:09.877 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:05:09.877 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:05:09.877 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:05:09.877 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:05:09.877 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:05:09.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:05:10.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:10.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:10.009 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:05:10.018 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:05:10.018 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:05:10.019 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:05:10.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:10.019 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:05:10.079 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:05:10.079 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:10.132 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:05:10.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:05:10.132 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:10.183 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:05:10.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:10.184 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:10.271 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:10.271 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:05:10.285 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:10.286 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:05:10.286 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:05:10.286 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:05:10.286 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:05:10.290 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:05:10.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:10.290 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:10.310 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:05:10.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:05:10.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:10.418 INFO:teuthology.orchestra.run.smithi110.stdout:client6836 2024-09-10T10:05:10.418 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:10.419 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:10.427 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:05:10.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.601+0000 7fdd49000640 1 Processor -- start 2024-09-10T10:05:10.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.601+0000 7fdd49000640 1 -- start start 2024-09-10T10:05:10.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.601+0000 7fdd49000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdd44155970 0x7fdd440cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:10.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.601+0000 7fdd49000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdd4415c560 0x7fdd4406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:10.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.601+0000 7fdd49000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd4406f430 con 0x7fdd4406f010 2024-09-10T10:05:10.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.601+0000 7fdd49000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdd440b4570 con 0x7fdd44155970 2024-09-10T10:05:10.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.601+0000 7fdd49000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdd44155d50 con 0x7fdd4415c560 2024-09-10T10:05:10.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdd44155970 0x7fdd440cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:10.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdd44155970 0x7fdd440cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37690/0 (socket says 172.21.15.110:37690) 2024-09-10T10:05:10.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41e00640 1 -- 172.21.15.110:0/1329249291 learned_addr learned my addr 172.21.15.110:0/1329249291 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:10.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41400640 1 --2- 172.21.15.110:0/1329249291 >> v2:172.21.15.145:3300/0 conn(0x7fdd4415c560 0x7fdd4406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:10.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41e00640 1 -- 172.21.15.110:0/1329249291 >> v2:172.21.15.145:3300/0 conn(0x7fdd4415c560 msgr2=0x7fdd4406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:10.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41e00640 1 --2- 172.21.15.110:0/1329249291 >> v2:172.21.15.145:3300/0 conn(0x7fdd4415c560 0x7fdd4406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41e00640 1 -- 172.21.15.110:0/1329249291 >> v1:172.21.15.110:6789/0 conn(0x7fdd4406f010 legacy=0x7fdd4415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:10.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41e00640 1 -- 172.21.15.110:0/1329249291 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd440b34d0 con 0x7fdd44155970 2024-09-10T10:05:10.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41400640 1 --2- 172.21.15.110:0/1329249291 >> v2:172.21.15.145:3300/0 conn(0x7fdd4415c560 0x7fdd4406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:10.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd41e00640 1 --2- 172.21.15.110:0/1329249291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd44155970 0x7fdd440cc8c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fdd4406d930 tx=0x7fdd3401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6ea1be8ac0a0f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:10.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd40a00640 1 -- 172.21.15.110:0/1329249291 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd340a5880 con 0x7fdd44155970 2024-09-10T10:05:10.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd40a00640 1 -- 172.21.15.110:0/1329249291 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd340a5a20 con 0x7fdd44155970 2024-09-10T10:05:10.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.605+0000 7fdd40a00640 1 -- 172.21.15.110:0/1329249291 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd340a5df0 con 0x7fdd44155970 2024-09-10T10:05:10.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- 172.21.15.110:0/1329249291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd44155970 msgr2=0x7fdd440cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:10.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 --2- 172.21.15.110:0/1329249291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd44155970 0x7fdd440cc8c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fdd4406d930 tx=0x7fdd3401c9c0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- 172.21.15.110:0/1329249291 shutdown_connections 2024-09-10T10:05:10.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 --2- 172.21.15.110:0/1329249291 >> v2:172.21.15.145:3300/0 conn(0x7fdd4415c560 0x7fdd4406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 --2- 172.21.15.110:0/1329249291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd44155970 0x7fdd440cc8c0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- 172.21.15.110:0/1329249291 >> 172.21.15.110:0/1329249291 conn(0x7fdd4406bc50 msgr2=0x7fdd4406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:10.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- 172.21.15.110:0/1329249291 shutdown_connections 2024-09-10T10:05:10.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- 172.21.15.110:0/1329249291 wait complete. 2024-09-10T10:05:10.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 Processor -- start 2024-09-10T10:05:10.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- start start 2024-09-10T10:05:10.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd4406f010 0x7fdd44151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:10.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd44155970 0x7fdd440723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:10.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd4415c560 0x7fdd44072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:10.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd440cce00 con 0x7fdd4406f010 2024-09-10T10:05:10.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdd440b4050 con 0x7fdd44155970 2024-09-10T10:05:10.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd49000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdd44168030 con 0x7fdd4415c560 2024-09-10T10:05:10.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd4406f010 0x7fdd44151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd4406f010 0x7fdd44151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37702/0 (socket says 172.21.15.110:37702) 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41e00640 1 -- 172.21.15.110:0/2775330220 learned_addr learned my addr 172.21.15.110:0/2775330220 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd42800640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd4415c560 0x7fdd44072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41400640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd44155970 0x7fdd440723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41e00640 1 -- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd4415c560 msgr2=0x7fdd44072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41e00640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd4415c560 0x7fdd44072920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41e00640 1 -- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd44155970 msgr2=0x7fdd440723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41e00640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd44155970 0x7fdd440723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.609+0000 7fdd41e00640 1 -- 172.21.15.110:0/2775330220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd44148e80 con 0x7fdd4406f010 2024-09-10T10:05:10.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd41e00640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd4406f010 0x7fdd44151890 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fdd3401a5f0 tx=0x7fdd34019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:10.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd3a000640 1 -- 172.21.15.110:0/2775330220 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd340b8670 con 0x7fdd4406f010 2024-09-10T10:05:10.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd3a000640 1 -- 172.21.15.110:0/2775330220 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd340be070 con 0x7fdd4406f010 2024-09-10T10:05:10.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd44071340 con 0x7fdd4406f010 2024-09-10T10:05:10.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd3a000640 1 -- 172.21.15.110:0/2775330220 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd340b8a90 con 0x7fdd4406f010 2024-09-10T10:05:10.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd44071900 con 0x7fdd4406f010 2024-09-10T10:05:10.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd440cc8c0 con 0x7fdd4406f010 2024-09-10T10:05:10.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd3a000640 1 -- 172.21.15.110:0/2775330220 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fdd340b8c30 con 0x7fdd4406f010 2024-09-10T10:05:10.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd3a000640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdd24067ef0 0x7fdd2406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:10.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd3a000640 1 -- 172.21.15.110:0/2775330220 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fdd34121c00 con 0x7fdd4406f010 2024-09-10T10:05:10.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.613+0000 7fdd41400640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdd24067ef0 0x7fdd2406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:10.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.617+0000 7fdd41400640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdd24067ef0 0x7fdd2406a3b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd441543f0 tx=0x7fdd30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:10.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.621+0000 7fdd3a000640 1 -- 172.21.15.110:0/2775330220 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdd340165e0 con 0x7fdd4406f010 2024-09-10T10:05:10.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.813+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdd440b1a70 con 0x7fdd4406f010 2024-09-10T10:05:10.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.813+0000 7fdd3a000640 1 -- 172.21.15.110:0/2775330220 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) ==== 53+0+73 (secure 0 0 0) 0x7fdd340f1730 con 0x7fdd4406f010 2024-09-10T10:05:10.820 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdd24067ef0 msgr2=0x7fdd2406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdd24067ef0 0x7fdd2406a3b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd441543f0 tx=0x7fdd30008040 comp rx=0 tx=0).stop 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd4406f010 msgr2=0x7fdd44151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd4406f010 0x7fdd44151890 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fdd3401a5f0 tx=0x7fdd34019760 comp rx=0 tx=0).stop 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 shutdown_connections 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdd24067ef0 0x7fdd2406a3b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd4415c560 0x7fdd44072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd44155970 0x7fdd440723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 --2- 172.21.15.110:0/2775330220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd4406f010 0x7fdd44151890 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:10.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 >> 172.21.15.110:0/2775330220 conn(0x7fdd4406bc50 msgr2=0x7fdd440b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:10.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 shutdown_connections 2024-09-10T10:05:10.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:10.817+0000 7fdd49000640 1 -- 172.21.15.110:0/2775330220 wait complete. 2024-09-10T10:05:10.840 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:05:10.840 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:05:10.840 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:05:10.840 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:05:10.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:10.841 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:05:10.841 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:05:10.841 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:05:10.841 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:05:10.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:10.841 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:05:10.851 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:05:10.852 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42060sec preferred_lft 42060sec 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:10.853 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:05:10.854 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:10.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:10.855 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:05:10.920 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:05:10.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:10.923 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:05:10.977 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:05:10.977 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:05:10.977 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:05:10.977 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:05:10.977 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:05:10.977 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:05:10.977 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:05:10.977 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:10.978 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:10.978 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:10.978 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:05:10.978 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:05:10.978 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:05:10.978 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:05:10.978 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:05:10.978 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:05:10.978 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:05:11.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:05:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:05:11.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:05:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:11.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:11.127 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:05:11.136 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:05:11.136 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:05:11.137 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:05:11.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:11.137 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:05:11.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:05:11.204 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:11.253 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:05:11.253 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:05:11.253 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:11.307 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:05:11.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:11.308 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:11.401 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:11.401 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:05:11.414 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:11.414 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:05:11.414 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:05:11.414 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:05:11.414 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:05:11.419 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:05:11.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:11.419 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:11.438 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:05:11.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:05:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:05:11.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:05:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:11.539 INFO:teuthology.orchestra.run.smithi145.stdout:client7705 2024-09-10T10:05:11.540 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:11.540 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:11.550 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:05:11.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.685+0000 7fcfa3200640 1 Processor -- start 2024-09-10T10:05:11.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.685+0000 7fcfa3200640 1 -- start start 2024-09-10T10:05:11.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa3200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa3200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcfa40b5bf0 0x7fcfa40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa3200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa40b3b40 con 0x7fcfa40cf2e0 2024-09-10T10:05:11.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa3200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa40b3910 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa3200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa40cfc60 con 0x7fcfa415b170 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa1e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa1e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:42348/0 (socket says 172.21.15.110:42348) 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa1e00640 1 -- 172.21.15.110:0/4205134676 learned_addr learned my addr 172.21.15.110:0/4205134676 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa2800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcfa40b5bf0 0x7fcfa40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcf99400640 1 --1- 172.21.15.110:0/4205134676 >> v1:172.21.15.145:6789/0 conn(0x7fcfa40cf2e0 0x7fcfa415ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:45608/0 (socket says 172.21.15.110:45608) 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa1e00640 1 -- 172.21.15.110:0/4205134676 >> v2:172.21.15.145:3300/0 conn(0x7fcfa40b5bf0 msgr2=0x7fcfa40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa1e00640 1 --2- 172.21.15.110:0/4205134676 >> v2:172.21.15.145:3300/0 conn(0x7fcfa40b5bf0 0x7fcfa40cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa1e00640 1 -- 172.21.15.110:0/4205134676 >> v1:172.21.15.145:6789/0 conn(0x7fcfa40cf2e0 legacy=0x7fcfa415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:11.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa1e00640 1 -- 172.21.15.110:0/4205134676 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa40b3e20 con 0x7fcfa415b170 2024-09-10T10:05:11.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa1400640 1 -- 172.21.15.110:0/4205134676 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4065962539 0 0) 0x7fcfa40b3b40 con 0x7fcfa40cf2e0 2024-09-10T10:05:11.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.689+0000 7fcfa2800640 1 --2- 172.21.15.110:0/4205134676 >> v2:172.21.15.145:3300/0 conn(0x7fcfa40b5bf0 0x7fcfa40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:11.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa1e00640 1 --2- 172.21.15.110:0/4205134676 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4073390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400bb10 tx=0x7fcf9401ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bebba98b2a09985 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:11.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa1400640 1 -- 172.21.15.110:0/4205134676 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf940b4070 con 0x7fcfa415b170 2024-09-10T10:05:11.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa1400640 1 -- 172.21.15.110:0/4205134676 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf940a7bf0 con 0x7fcfa415b170 2024-09-10T10:05:11.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa1400640 1 -- 172.21.15.110:0/4205134676 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf940af440 con 0x7fcfa415b170 2024-09-10T10:05:11.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 -- 172.21.15.110:0/4205134676 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 msgr2=0x7fcfa4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:11.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 --2- 172.21.15.110:0/4205134676 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4073390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400bb10 tx=0x7fcf9401ec00 comp rx=0 tx=0).stop 2024-09-10T10:05:11.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 -- 172.21.15.110:0/4205134676 shutdown_connections 2024-09-10T10:05:11.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 --2- 172.21.15.110:0/4205134676 >> v2:172.21.15.145:3300/0 conn(0x7fcfa40b5bf0 0x7fcfa40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 --2- 172.21.15.110:0/4205134676 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4073390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 -- 172.21.15.110:0/4205134676 >> 172.21.15.110:0/4205134676 conn(0x7fcfa406d0f0 msgr2=0x7fcfa406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:11.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 -- 172.21.15.110:0/4205134676 shutdown_connections 2024-09-10T10:05:11.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 -- 172.21.15.110:0/4205134676 wait complete. 2024-09-10T10:05:11.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 Processor -- start 2024-09-10T10:05:11.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.693+0000 7fcfa3200640 1 -- start start 2024-09-10T10:05:11.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfa40b5bf0 0x7fcfa41f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa40cf2e0 0x7fcfa4146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4072680 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa415b550 con 0x7fcfa40cf2e0 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa4072800 con 0x7fcfa415b170 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfa40b5bf0 0x7fcfa41f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfa40b5bf0 0x7fcfa41f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37706/0 (socket says 172.21.15.110:37706) 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 -- 172.21.15.110:0/1476704430 learned_addr learned my addr 172.21.15.110:0/1476704430 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcf99400640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa40cf2e0 0x7fcfa4146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa2800640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 -- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 msgr2=0x7fcfa4156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 -- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa40cf2e0 msgr2=0x7fcfa4146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:11.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa40cf2e0 0x7fcfa4146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:11.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 -- 172.21.15.110:0/1476704430 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa4147150 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa2800640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:11.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa1e00640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfa40b5bf0 0x7fcfa41f7180 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400bb10 tx=0x7fcf9401d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:11.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcf9aa00640 1 -- 172.21.15.110:0/1476704430 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf940b4040 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcf9aa00640 1 -- 172.21.15.110:0/1476704430 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf940afbf0 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa4151660 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcf9aa00640 1 -- 172.21.15.110:0/1476704430 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94004020 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa4158780 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa40739f0 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.697+0000 7fcf9aa00640 1 -- 172.21.15.110:0/1476704430 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fcf940c1060 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.701+0000 7fcf9aa00640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf80067ef0 0x7fcf8006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:11.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.701+0000 7fcf9aa00640 1 -- 172.21.15.110:0/1476704430 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fcf94120b20 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.701+0000 7fcf99400640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf80067ef0 0x7fcf8006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:11.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.701+0000 7fcf99400640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf80067ef0 0x7fcf8006a3b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40cdc50 tx=0x7fcf8c007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:11.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.705+0000 7fcf9aa00640 1 -- 172.21.15.110:0/1476704430 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcf94018820 con 0x7fcfa40b5bf0 2024-09-10T10:05:11.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:11.897+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fcfa40b0ff0 con 0x7fcfa40b5bf0 2024-09-10T10:05:12.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcf9aa00640 1 -- 172.21.15.110:0/1476704430 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v98) ==== 136+0+0 (secure 0 0 0) 0x7fcf940f0cf0 con 0x7fcfa40b5bf0 2024-09-10T10:05:12.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf80067ef0 msgr2=0x7fcf8006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf80067ef0 0x7fcf8006a3b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40cdc50 tx=0x7fcf8c007d60 comp rx=0 tx=0).stop 2024-09-10T10:05:12.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfa40b5bf0 msgr2=0x7fcfa41f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfa40b5bf0 0x7fcfa41f7180 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400bb10 tx=0x7fcf9401d440 comp rx=0 tx=0).stop 2024-09-10T10:05:12.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 shutdown_connections 2024-09-10T10:05:12.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf80067ef0 0x7fcf8006a3b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfa415b170 0x7fcfa4156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfa40cf2e0 0x7fcfa4146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 --2- 172.21.15.110:0/1476704430 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfa40b5bf0 0x7fcfa41f7180 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 >> 172.21.15.110:0/1476704430 conn(0x7fcfa406d0f0 msgr2=0x7fcfa40b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:12.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 shutdown_connections 2024-09-10T10:05:12.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.549+0000 7fcfa3200640 1 -- 172.21.15.110:0/1476704430 wait complete. 2024-09-10T10:05:12.577 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:05:12.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.709+0000 7fa96b800640 1 Processor -- start 2024-09-10T10:05:12.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.709+0000 7fa96b800640 1 -- start start 2024-09-10T10:05:12.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa96b800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa9640b5600 0x7fa9640cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:12.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa96b800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96406f410 con 0x7fa9640726e0 2024-09-10T10:05:12.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa96b800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96415fda0 con 0x7fa96406efa0 2024-09-10T10:05:12.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa96b800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa9640cee50 con 0x7fa9640b5600 2024-09-10T10:05:12.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa968e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa9640726e0 0x7fa9640b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59946/0 (socket says 172.21.15.110:59946) 2024-09-10T10:05:12.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa968e00640 1 -- 172.21.15.110:0/3371181256 learned_addr learned my addr 172.21.15.110:0/3371181256 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:12.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa963e00640 1 --2- 172.21.15.110:0/3371181256 >> v2:172.21.15.145:3300/0 conn(0x7fa9640b5600 0x7fa9640cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:12.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa963400640 1 -- 172.21.15.110:0/3371181256 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873881900 0 0) 0x7fa9540014a0 con 0x7fa9640726e0 2024-09-10T10:05:12.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa963400640 1 -- 172.21.15.110:0/3371181256 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa948005740 con 0x7fa9640726e0 2024-09-10T10:05:12.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa963e00640 1 -- 172.21.15.110:0/3371181256 >> v1:172.21.15.145:6789/0 conn(0x7fa96406efa0 legacy=0x7fa9640b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa963e00640 1 -- 172.21.15.110:0/3371181256 >> v1:172.21.15.110:6789/0 conn(0x7fa9640726e0 legacy=0x7fa9640b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.713+0000 7fa963e00640 1 -- 172.21.15.110:0/3371181256 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9640ce520 con 0x7fa9640b5600 2024-09-10T10:05:12.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa963e00640 1 --2- 172.21.15.110:0/3371181256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640b5600 0x7fa9640cf2e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa9500097c0 tx=0x7fa95001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d132b75b6c9582e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:12.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa963400640 1 -- 172.21.15.110:0/3371181256 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500b0070 con 0x7fa9640b5600 2024-09-10T10:05:12.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa963400640 1 -- 172.21.15.110:0/3371181256 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9500ab600 con 0x7fa9640b5600 2024-09-10T10:05:12.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa963400640 1 -- 172.21.15.110:0/3371181256 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500ab9b0 con 0x7fa9640b5600 2024-09-10T10:05:12.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 -- 172.21.15.110:0/3371181256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640b5600 msgr2=0x7fa9640cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 --2- 172.21.15.110:0/3371181256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640b5600 0x7fa9640cf2e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa9500097c0 tx=0x7fa95001c470 comp rx=0 tx=0).stop 2024-09-10T10:05:12.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 -- 172.21.15.110:0/3371181256 shutdown_connections 2024-09-10T10:05:12.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 --2- 172.21.15.110:0/3371181256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640b5600 0x7fa9640cf2e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 -- 172.21.15.110:0/3371181256 >> 172.21.15.110:0/3371181256 conn(0x7fa964062d10 msgr2=0x7fa96406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:12.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 -- 172.21.15.110:0/3371181256 shutdown_connections 2024-09-10T10:05:12.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 -- 172.21.15.110:0/3371181256 wait complete. 2024-09-10T10:05:12.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 Processor -- start 2024-09-10T10:05:12.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.717+0000 7fa96b800640 1 -- start start 2024-09-10T10:05:12.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa96406efa0 0x7fa9640b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:12.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640726e0 0x7fa96415f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:12.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9640b5600 0x7fa9640b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640b4170 con 0x7fa9640b5600 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640b3ff0 con 0x7fa9640726e0 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa964072ac0 con 0x7fa96406efa0 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9640b5600 0x7fa9640b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9640b5600 0x7fa9640b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37714/0 (socket says 172.21.15.110:37714) 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 -- 172.21.15.110:0/3400107128 learned_addr learned my addr 172.21.15.110:0/3400107128 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa968e00640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa96406efa0 0x7fa9640b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa963e00640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640726e0 0x7fa96415f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 -- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa96406efa0 msgr2=0x7fa9640b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa96406efa0 0x7fa9640b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 -- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640726e0 msgr2=0x7fa96415f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640726e0 0x7fa96415f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 -- 172.21.15.110:0/3400107128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9640afcf0 con 0x7fa9640b5600 2024-09-10T10:05:12.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa963e00640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640726e0 0x7fa96415f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:12.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa969800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9640b5600 0x7fa9640b0120 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00c430 tx=0x7fa95c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:12.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa960c00640 1 -- 172.21.15.110:0/3400107128 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c005220 con 0x7fa9640b5600 2024-09-10T10:05:12.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa960c00640 1 -- 172.21.15.110:0/3400107128 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95c0053c0 con 0x7fa9640b5600 2024-09-10T10:05:12.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa960c00640 1 -- 172.21.15.110:0/3400107128 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c007270 con 0x7fa9640b5600 2024-09-10T10:05:12.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9641fb300 con 0x7fa9640b5600 2024-09-10T10:05:12.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9641fb8c0 con 0x7fa9640b5600 2024-09-10T10:05:12.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96415f950 con 0x7fa9640b5600 2024-09-10T10:05:12.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.721+0000 7fa960c00640 1 -- 172.21.15.110:0/3400107128 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fa95c019020 con 0x7fa9640b5600 2024-09-10T10:05:12.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.725+0000 7fa960c00640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa944067ef0 0x7fa94406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:12.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.725+0000 7fa960c00640 1 -- 172.21.15.110:0/3400107128 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fa95c085c40 con 0x7fa9640b5600 2024-09-10T10:05:12.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.725+0000 7fa968e00640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa944067ef0 0x7fa94406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:12.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.725+0000 7fa968e00640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa944067ef0 0x7fa94406a3b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa954001640 tx=0x7fa954009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:12.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.729+0000 7fa960c00640 1 -- 172.21.15.110:0/3400107128 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa95c055dc0 con 0x7fa9640b5600 2024-09-10T10:05:12.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.909+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa96415e980 con 0x7fa944067ef0 2024-09-10T10:05:12.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.909+0000 7fa960c00640 1 -- 172.21.15.110:0/3400107128 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa96415e980 con 0x7fa944067ef0 2024-09-10T10:05:12.917 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:05:12.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:12.917 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:05:12.917 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:12.917 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:05:12.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa944067ef0 msgr2=0x7fa94406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa944067ef0 0x7fa94406a3b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa954001640 tx=0x7fa954009040 comp rx=0 tx=0).stop 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9640b5600 msgr2=0x7fa9640b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9640b5600 0x7fa9640b0120 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00c430 tx=0x7fa95c00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 shutdown_connections 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa944067ef0 0x7fa94406a3b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9640b5600 0x7fa9640b0120 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9640726e0 0x7fa96415f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 --2- 172.21.15.110:0/3400107128 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa96406efa0 0x7fa9640b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 >> 172.21.15.110:0/3400107128 conn(0x7fa964062d10 msgr2=0x7fa9640b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 shutdown_connections 2024-09-10T10:05:12.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:12.913+0000 7fa96b800640 1 -- 172.21.15.110:0/3400107128 wait complete. 2024-09-10T10:05:12.938 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:05:13.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.069+0000 7fa30e800640 1 Processor -- start 2024-09-10T10:05:13.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa30e800640 1 -- start start 2024-09-10T10:05:13.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa30e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa3080ce4d0 0x7fa30806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa30e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa30806e850 0x7fa3080b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa30e800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3080b4960 con 0x7fa30815b330 2024-09-10T10:05:13.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa30e800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa3080b5570 con 0x7fa3080ce4d0 2024-09-10T10:05:13.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa30e800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa3080b4c40 con 0x7fa30806e850 2024-09-10T10:05:13.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa306a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa3080ce4d0 0x7fa30806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa306a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa3080ce4d0 0x7fa30806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37728/0 (socket says 172.21.15.110:37728) 2024-09-10T10:05:13.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa306a00640 1 -- 172.21.15.110:0/3023601884 learned_addr learned my addr 172.21.15.110:0/3023601884 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:13.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa307e00640 1 --2- 172.21.15.110:0/3023601884 >> v2:172.21.15.145:3300/0 conn(0x7fa30806e850 0x7fa3080b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa306a00640 1 -- 172.21.15.110:0/3023601884 >> v2:172.21.15.145:3300/0 conn(0x7fa30806e850 msgr2=0x7fa3080b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:13.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa306a00640 1 --2- 172.21.15.110:0/3023601884 >> v2:172.21.15.145:3300/0 conn(0x7fa30806e850 0x7fa3080b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa306a00640 1 -- 172.21.15.110:0/3023601884 >> v1:172.21.15.145:6789/0 conn(0x7fa30815b330 legacy=0x7fa30806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:13.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa306a00640 1 -- 172.21.15.110:0/3023601884 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3080b4f20 con 0x7fa3080ce4d0 2024-09-10T10:05:13.080 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.073+0000 7fa307e00640 1 --2- 172.21.15.110:0/3023601884 >> v2:172.21.15.145:3300/0 conn(0x7fa30806e850 0x7fa3080b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:13.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa306a00640 1 --2- 172.21.15.110:0/3023601884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3080ce4d0 0x7fa30806c6a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa3080633d0 tx=0x7fa2f401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ab5f2e373eb428 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.081 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa306000640 1 -- 172.21.15.110:0/3023601884 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40ab5f0 con 0x7fa3080ce4d0 2024-09-10T10:05:13.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa306000640 1 -- 172.21.15.110:0/3023601884 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f40b0070 con 0x7fa3080ce4d0 2024-09-10T10:05:13.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa306000640 1 -- 172.21.15.110:0/3023601884 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40ab9b0 con 0x7fa3080ce4d0 2024-09-10T10:05:13.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 -- 172.21.15.110:0/3023601884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3080ce4d0 msgr2=0x7fa30806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 --2- 172.21.15.110:0/3023601884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3080ce4d0 0x7fa30806c6a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa3080633d0 tx=0x7fa2f401c4e0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 -- 172.21.15.110:0/3023601884 shutdown_connections 2024-09-10T10:05:13.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 --2- 172.21.15.110:0/3023601884 >> v2:172.21.15.145:3300/0 conn(0x7fa30806e850 0x7fa3080b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 --2- 172.21.15.110:0/3023601884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3080ce4d0 0x7fa30806c6a0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 -- 172.21.15.110:0/3023601884 >> 172.21.15.110:0/3023601884 conn(0x7fa308062e40 msgr2=0x7fa30806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 -- 172.21.15.110:0/3023601884 shutdown_connections 2024-09-10T10:05:13.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 -- 172.21.15.110:0/3023601884 wait complete. 2024-09-10T10:05:13.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 Processor -- start 2024-09-10T10:05:13.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.077+0000 7fa30e800640 1 -- start start 2024-09-10T10:05:13.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa30806e850 0x7fa3080ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30815b330 0x7fa308071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3080715a0 0x7fa308159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081680b0 con 0x7fa30815b330 2024-09-10T10:05:13.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3080b1800 con 0x7fa3080715a0 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa308168230 con 0x7fa30806e850 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30815b330 0x7fa308071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30815b330 0x7fa308071060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37738/0 (socket says 172.21.15.110:37738) 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 -- 172.21.15.110:0/988040540 learned_addr learned my addr 172.21.15.110:0/988040540 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa307e00640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3080715a0 0x7fa308159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa307400640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa30806e850 0x7fa3080ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 -- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa30806e850 msgr2=0x7fa3080ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa30806e850 0x7fa3080ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 -- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3080715a0 msgr2=0x7fa308159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3080715a0 0x7fa308159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 -- 172.21.15.110:0/988040540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3080cd060 con 0x7fa30815b330 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa307400640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa30806e850 0x7fa3080ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:13.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa307e00640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3080715a0 0x7fa308159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:13.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa306a00640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30815b330 0x7fa308071060 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40ad040 tx=0x7fa2f401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa2ff400640 1 -- 172.21.15.110:0/988040540 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40be5f0 con 0x7fa30815b330 2024-09-10T10:05:13.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa2ff400640 1 -- 172.21.15.110:0/988040540 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f40be790 con 0x7fa30815b330 2024-09-10T10:05:13.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa2ff400640 1 -- 172.21.15.110:0/988040540 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40beb80 con 0x7fa30815b330 2024-09-10T10:05:13.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30815a6b0 con 0x7fa30815b330 2024-09-10T10:05:13.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa308142c60 con 0x7fa30815b330 2024-09-10T10:05:13.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3080719b0 con 0x7fa30815b330 2024-09-10T10:05:13.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.081+0000 7fa2ff400640 1 -- 172.21.15.110:0/988040540 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fa2f40b43f0 con 0x7fa30815b330 2024-09-10T10:05:13.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.085+0000 7fa2ff400640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa2e8067f20 0x7fa2e806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.085+0000 7fa2ff400640 1 -- 172.21.15.110:0/988040540 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fa2f4121020 con 0x7fa30815b330 2024-09-10T10:05:13.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.085+0000 7fa307400640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa2e8067f20 0x7fa2e806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.085+0000 7fa307400640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa2e8067f20 0x7fa2e806a3e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8007e90 tx=0x7fa2f80073a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.089+0000 7fa2ff400640 1 -- 172.21.15.110:0/988040540 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa3080719b0 con 0x7fa30815b330 2024-09-10T10:05:13.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa3080cd240 con 0x7fa30815b330 2024-09-10T10:05:13.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa2ff400640 1 -- 172.21.15.110:0/988040540 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fa3080cd060 con 0x7fa30815b330 2024-09-10T10:05:13.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa2e8067f20 msgr2=0x7fa2e806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa2e8067f20 0x7fa2e806a3e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8007e90 tx=0x7fa2f80073a0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30815b330 msgr2=0x7fa308071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30815b330 0x7fa308071060 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40ad040 tx=0x7fa2f401ad20 comp rx=0 tx=0).stop 2024-09-10T10:05:13.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 shutdown_connections 2024-09-10T10:05:13.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa2e8067f20 0x7fa2e806a3e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3080715a0 0x7fa308159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa30815b330 0x7fa308071060 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 --2- 172.21.15.110:0/988040540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa30806e850 0x7fa3080ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 >> 172.21.15.110:0/988040540 conn(0x7fa308062e40 msgr2=0x7fa30806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 shutdown_connections 2024-09-10T10:05:13.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.249+0000 7fa30e800640 1 -- 172.21.15.110:0/988040540 wait complete. 2024-09-10T10:05:13.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:13.275 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol5770) 2024-09-10T10:05:13.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:13.303 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp84850/subvol36291) 2024-09-10T10:05:13.367 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol5770 2024-09-10T10:05:13.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24ade00640 1 Processor -- start 2024-09-10T10:05:13.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24ade00640 1 -- start start 2024-09-10T10:05:13.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24ade00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f24a80cf0f0 0x7f24a80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24ade00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a80cd800 con 0x7f24a80722b0 2024-09-10T10:05:13.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24ade00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a80cdae0 con 0x7f24a8155850 2024-09-10T10:05:13.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24ade00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f24a80ce410 con 0x7f24a80cf0f0 2024-09-10T10:05:13.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24ad400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f24a80722b0 0x7f24a8072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59952/0 (socket says 172.21.15.110:59952) 2024-09-10T10:05:13.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24ad400640 1 -- 172.21.15.110:0/3409988394 learned_addr learned my addr 172.21.15.110:0/3409988394 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:13.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24a7e00640 1 --2- 172.21.15.110:0/3409988394 >> v2:172.21.15.145:3300/0 conn(0x7f24a80cf0f0 0x7f24a80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24a7400640 1 -- 172.21.15.110:0/3409988394 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890687085 0 0) 0x7f24a80cd800 con 0x7f24a80722b0 2024-09-10T10:05:13.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24a7400640 1 -- 172.21.15.110:0/3409988394 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f248c003530 con 0x7f24a80722b0 2024-09-10T10:05:13.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24a7400640 1 -- 172.21.15.110:0/3409988394 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3692126353 0 0) 0x7f248c003530 con 0x7f24a80722b0 2024-09-10T10:05:13.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24a7e00640 1 -- 172.21.15.110:0/3409988394 >> v1:172.21.15.145:6789/0 conn(0x7f24a8155850 legacy=0x7f24a80ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24a7e00640 1 -- 172.21.15.110:0/3409988394 >> v1:172.21.15.110:6789/0 conn(0x7f24a80722b0 legacy=0x7f24a8072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.541+0000 7f24a7e00640 1 -- 172.21.15.110:0/3409988394 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a80cddc0 con 0x7f24a80cf0f0 2024-09-10T10:05:13.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24a7e00640 1 --2- 172.21.15.110:0/3409988394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 0x7f24a80cf550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f24940097c0 tx=0x7f249401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7312b9f1d892543c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24a7400640 1 -- 172.21.15.110:0/3409988394 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24940b0070 con 0x7f24a80cf0f0 2024-09-10T10:05:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24a7400640 1 -- 172.21.15.110:0/3409988394 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24940ab400 con 0x7f24a80cf0f0 2024-09-10T10:05:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24a7400640 1 -- 172.21.15.110:0/3409988394 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0001a00 con 0x7f24a80cf0f0 2024-09-10T10:05:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24ade00640 1 -- 172.21.15.110:0/3409988394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 msgr2=0x7f24a80cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24ade00640 1 --2- 172.21.15.110:0/3409988394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 0x7f24a80cf550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f24940097c0 tx=0x7f249401c3c0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24ade00640 1 -- 172.21.15.110:0/3409988394 shutdown_connections 2024-09-10T10:05:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24ade00640 1 --2- 172.21.15.110:0/3409988394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 0x7f24a80cf550 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24ade00640 1 -- 172.21.15.110:0/3409988394 >> 172.21.15.110:0/3409988394 conn(0x7f24a806d710 msgr2=0x7f24a80b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24ade00640 1 -- 172.21.15.110:0/3409988394 shutdown_connections 2024-09-10T10:05:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.545+0000 7f24ade00640 1 -- 172.21.15.110:0/3409988394 wait complete. 2024-09-10T10:05:13.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ade00640 1 Processor -- start 2024-09-10T10:05:13.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ade00640 1 -- start start 2024-09-10T10:05:13.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ade00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a80722b0 0x7f24a8144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ade00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 0x7f24a8159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ade00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24a8155850 0x7f24a8159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ade00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8169010 con 0x7f24a80722b0 2024-09-10T10:05:13.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ade00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8168e90 con 0x7f24a80cf0f0 2024-09-10T10:05:13.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ade00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f24a8169190 con 0x7f24a8155850 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a80722b0 0x7f24a8144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a80722b0 0x7f24a8144580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37740/0 (socket says 172.21.15.110:37740) 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 -- 172.21.15.110:0/1180190392 learned_addr learned my addr 172.21.15.110:0/1180190392 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24a7e00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 0x7f24a8159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24ad400640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24a8155850 0x7f24a8159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 -- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24a8155850 msgr2=0x7f24a8159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24a8155850 0x7f24a8159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 -- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 msgr2=0x7f24a8159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 0x7f24a8159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 -- 172.21.15.110:0/1180190392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a815ac50 con 0x7f24a80722b0 2024-09-10T10:05:13.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24aca00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a80722b0 0x7f24a8144580 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f249800c530 tx=0x7f249800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.549+0000 7f24a4c00640 1 -- 172.21.15.110:0/1180190392 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498016020 con 0x7f24a80722b0 2024-09-10T10:05:13.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24a4c00640 1 -- 172.21.15.110:0/1180190392 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2498005220 con 0x7f24a80722b0 2024-09-10T10:05:13.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24a4c00640 1 -- 172.21.15.110:0/1180190392 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498007030 con 0x7f24a80722b0 2024-09-10T10:05:13.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a81450b0 con 0x7f24a80722b0 2024-09-10T10:05:13.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a8159e90 con 0x7f24a80722b0 2024-09-10T10:05:13.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a80cd800 con 0x7f24a80722b0 2024-09-10T10:05:13.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24a4c00640 1 -- 172.21.15.110:0/1180190392 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f24980071d0 con 0x7f24a80722b0 2024-09-10T10:05:13.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24a4c00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2488067ef0 0x7f248806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24a4c00640 1 -- 172.21.15.110:0/1180190392 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f24980860d0 con 0x7f24a80722b0 2024-09-10T10:05:13.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24a7e00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2488067ef0 0x7f248806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.553+0000 7f24a7e00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2488067ef0 0x7f248806a3b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f24940ad000 tx=0x7f24940a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.557+0000 7f24a4c00640 1 -- 172.21.15.110:0/1180190392 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2498020020 con 0x7f24a80722b0 2024-09-10T10:05:13.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.741+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol5770", "target": ["mon-mgr", ""]}) -- 0x7f24a814ba40 con 0x7f2488067ef0 2024-09-10T10:05:13.754 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:13.753+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:13.754 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:13.753+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:13.754 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:13.753+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:13.755 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:13.753+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:13.755 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:13.753+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:13.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24a4c00640 1 -- 172.21.15.110:0/1180190392 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+29 (secure 0 0 0) 0x7f24a814ba40 con 0x7f2488067ef0 2024-09-10T10:05:13.785 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol5770 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2488067ef0 msgr2=0x7f248806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2488067ef0 0x7f248806a3b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f24940ad000 tx=0x7f24940a6000 comp rx=0 tx=0).stop 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a80722b0 msgr2=0x7f24a8144580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a80722b0 0x7f24a8144580 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f249800c530 tx=0x7f249800c9f0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 shutdown_connections 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2488067ef0 0x7f248806a3b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f24a8155850 0x7f24a8159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f24a80cf0f0 0x7f24a8159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 --2- 172.21.15.110:0/1180190392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a80722b0 0x7f24a8144580 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 >> 172.21.15.110:0/1180190392 conn(0x7f24a806d710 msgr2=0x7f24a80ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 shutdown_connections 2024-09-10T10:05:13.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.781+0000 7f24ade00640 1 -- 172.21.15.110:0/1180190392 wait complete. 2024-09-10T10:05:13.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol36291 subvol_grp84850 2024-09-10T10:05:13.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35cc000640 1 Processor -- start 2024-09-10T10:05:13.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35cc000640 1 -- start start 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35cc000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f35c415c560 0x7f35c406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35cc000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f35c406f010 0x7f35c40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35cc000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c406f430 con 0x7f35c4155970 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35cc000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f35c40b4570 con 0x7f35c406f010 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35cc000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f35c4155d50 con 0x7f35c415c560 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35ca000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f35c406f010 0x7f35c40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35ca000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f35c406f010 0x7f35c40ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37756/0 (socket says 172.21.15.110:37756) 2024-09-10T10:05:13.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35ca000640 1 -- 172.21.15.110:0/711651287 learned_addr learned my addr 172.21.15.110:0/711651287 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:13.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35ca000640 1 -- 172.21.15.110:0/711651287 >> v2:172.21.15.145:3300/0 conn(0x7f35c415c560 msgr2=0x7f35c406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:13.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35ca000640 1 --2- 172.21.15.110:0/711651287 >> v2:172.21.15.145:3300/0 conn(0x7f35c415c560 0x7f35c406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35ca000640 1 -- 172.21.15.110:0/711651287 >> v1:172.21.15.145:6789/0 conn(0x7f35c4155970 legacy=0x7f35c40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:13.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.941+0000 7f35ca000640 1 -- 172.21.15.110:0/711651287 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c40b34d0 con 0x7f35c406f010 2024-09-10T10:05:13.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35ca000640 1 --2- 172.21.15.110:0/711651287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c406f010 0x7f35c40ce420 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f35c406d930 tx=0x7f35bc01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4bd9d31c6809a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35c3e00640 1 -- 172.21.15.110:0/711651287 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc0a5840 con 0x7f35c406f010 2024-09-10T10:05:13.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35c3e00640 1 -- 172.21.15.110:0/711651287 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35bc0a59e0 con 0x7f35c406f010 2024-09-10T10:05:13.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35c3e00640 1 -- 172.21.15.110:0/711651287 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc0a5db0 con 0x7f35c406f010 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35cc000640 1 -- 172.21.15.110:0/711651287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c406f010 msgr2=0x7f35c40ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35cc000640 1 --2- 172.21.15.110:0/711651287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c406f010 0x7f35c40ce420 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f35c406d930 tx=0x7f35bc01c980 comp rx=0 tx=0).stop 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35cc000640 1 -- 172.21.15.110:0/711651287 shutdown_connections 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35cc000640 1 --2- 172.21.15.110:0/711651287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c406f010 0x7f35c40ce420 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35cc000640 1 --2- 172.21.15.110:0/711651287 >> v2:172.21.15.145:3300/0 conn(0x7f35c415c560 0x7f35c406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35cc000640 1 -- 172.21.15.110:0/711651287 >> 172.21.15.110:0/711651287 conn(0x7f35c406bc50 msgr2=0x7f35c406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35cc000640 1 -- 172.21.15.110:0/711651287 shutdown_connections 2024-09-10T10:05:13.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.945+0000 7f35cc000640 1 -- 172.21.15.110:0/711651287 wait complete. 2024-09-10T10:05:13.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 Processor -- start 2024-09-10T10:05:13.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 -- start start 2024-09-10T10:05:13.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35c406f010 0x7f35c4153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c4155970 0x7f35c40aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35c415c560 0x7f35c4141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35c40cf170 con 0x7f35c4155970 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f35c40b4050 con 0x7f35c406f010 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f35c40cf2f0 con 0x7f35c415c560 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c4155970 0x7f35c40aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c4155970 0x7f35c40aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37758/0 (socket says 172.21.15.110:37758) 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 -- 172.21.15.110:0/1010419263 learned_addr learned my addr 172.21.15.110:0/1010419263 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c9600640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35c406f010 0x7f35c4153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35ca000640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35c415c560 0x7f35c4141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 -- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35c415c560 msgr2=0x7f35c4141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35c415c560 0x7f35c4141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 -- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35c406f010 msgr2=0x7f35c4153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35c406f010 0x7f35c4153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 -- 172.21.15.110:0/1010419263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c4153a90 con 0x7f35c4155970 2024-09-10T10:05:13.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c8c00640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c4155970 0x7f35c40aeaa0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f35b000b410 tx=0x7f35b000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c1600640 1 -- 172.21.15.110:0/1010419263 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0015020 con 0x7f35c4155970 2024-09-10T10:05:13.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c1600640 1 -- 172.21.15.110:0/1010419263 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b0011830 con 0x7f35c4155970 2024-09-10T10:05:13.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c4153d70 con 0x7f35c4155970 2024-09-10T10:05:13.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35c1600640 1 -- 172.21.15.110:0/1010419263 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0011c20 con 0x7f35c4155970 2024-09-10T10:05:13.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c4154330 con 0x7f35c4155970 2024-09-10T10:05:13.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.949+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c4069bb0 con 0x7f35c4155970 2024-09-10T10:05:13.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.953+0000 7f35c1600640 1 -- 172.21.15.110:0/1010419263 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7f35b0018020 con 0x7f35c4155970 2024-09-10T10:05:13.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.953+0000 7f35c1600640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35a4067f20 0x7f35a406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.953+0000 7f35c1600640 1 -- 172.21.15.110:0/1010419263 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f35b0086680 con 0x7f35c4155970 2024-09-10T10:05:13.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.953+0000 7f35c9600640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35a4067f20 0x7f35a406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.953+0000 7f35c9600640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35a4067f20 0x7f35a406a3e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f35b8007fd0 tx=0x7f35b8007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:13.957+0000 7f35c1600640 1 -- 172.21.15.110:0/1010419263 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f35b001f020 con 0x7f35c4155970 2024-09-10T10:05:14.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.137+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol36291", "group_name": "subvol_grp84850", "target": ["mon-mgr", ""]}) -- 0x7f35c40cf680 con 0x7f35a4067f20 2024-09-10T10:05:14.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.157+0000 7f35c1600640 1 -- 172.21.15.110:0/1010419263 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f35c40cf680 con 0x7f35a4067f20 2024-09-10T10:05:14.164 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/subvol_grp84850/subvol36291 2024-09-10T10:05:14.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35a4067f20 msgr2=0x7f35a406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35a4067f20 0x7f35a406a3e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f35b8007fd0 tx=0x7f35b8007e40 comp rx=0 tx=0).stop 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c4155970 msgr2=0x7f35c40aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c4155970 0x7f35c40aeaa0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f35b000b410 tx=0x7f35b000b8d0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 shutdown_connections 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35a4067f20 0x7f35a406a3e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35c415c560 0x7f35c4141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35c4155970 0x7f35c40aeaa0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 --2- 172.21.15.110:0/1010419263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35c406f010 0x7f35c4153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 >> 172.21.15.110:0/1010419263 conn(0x7f35c406bc50 msgr2=0x7f35c406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:14.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 shutdown_connections 2024-09-10T10:05:14.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.161+0000 7f35cc000640 1 -- 172.21.15.110:0/1010419263 wait complete. 2024-09-10T10:05:14.186 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol5770 2024-09-10T10:05:14.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde12c00640 1 Processor -- start 2024-09-10T10:05:14.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde12c00640 1 -- start start 2024-09-10T10:05:14.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde12c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fde0c15b330 0x7fde0c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde12c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fde0c06e850 0x7fde0c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde12c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde0c0b4960 con 0x7fde0c0ce4d0 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde12c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fde0c0b5570 con 0x7fde0c06e850 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde12c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fde0c0b4c40 con 0x7fde0c15b330 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde10c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fde0c06e850 0x7fde0c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde10c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fde0c06e850 0x7fde0c0b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37768/0 (socket says 172.21.15.110:37768) 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde10c00640 1 -- 172.21.15.110:0/3347917316 learned_addr learned my addr 172.21.15.110:0/3347917316 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde03e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fde0c0ce4d0 0x7fde0c06e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59962/0 (socket says 172.21.15.110:59962) 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde0be00640 1 --2- 172.21.15.110:0/3347917316 >> v2:172.21.15.145:3300/0 conn(0x7fde0c15b330 0x7fde0c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde10c00640 1 -- 172.21.15.110:0/3347917316 >> v2:172.21.15.145:3300/0 conn(0x7fde0c15b330 msgr2=0x7fde0c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde10c00640 1 --2- 172.21.15.110:0/3347917316 >> v2:172.21.15.145:3300/0 conn(0x7fde0c15b330 0x7fde0c06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde10c00640 1 -- 172.21.15.110:0/3347917316 >> v1:172.21.15.110:6789/0 conn(0x7fde0c0ce4d0 legacy=0x7fde0c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.321+0000 7fde10c00640 1 -- 172.21.15.110:0/3347917316 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c0b4f20 con 0x7fde0c06e850 2024-09-10T10:05:14.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde10c00640 1 --2- 172.21.15.110:0/3347917316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0b0860 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fde0c06eca0 tx=0x7fde0401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ed13e59db6ea6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:14.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde0b400640 1 -- 172.21.15.110:0/3347917316 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde040a57d0 con 0x7fde0c06e850 2024-09-10T10:05:14.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde0b400640 1 -- 172.21.15.110:0/3347917316 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde040a5970 con 0x7fde0c06e850 2024-09-10T10:05:14.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde0b400640 1 -- 172.21.15.110:0/3347917316 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde040a5d40 con 0x7fde0c06e850 2024-09-10T10:05:14.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 -- 172.21.15.110:0/3347917316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 msgr2=0x7fde0c0b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 --2- 172.21.15.110:0/3347917316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0b0860 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fde0c06eca0 tx=0x7fde0401c9f0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 -- 172.21.15.110:0/3347917316 shutdown_connections 2024-09-10T10:05:14.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 --2- 172.21.15.110:0/3347917316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0b0860 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 --2- 172.21.15.110:0/3347917316 >> v2:172.21.15.145:3300/0 conn(0x7fde0c15b330 0x7fde0c06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 -- 172.21.15.110:0/3347917316 >> 172.21.15.110:0/3347917316 conn(0x7fde0c062e40 msgr2=0x7fde0c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:14.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 -- 172.21.15.110:0/3347917316 shutdown_connections 2024-09-10T10:05:14.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 -- 172.21.15.110:0/3347917316 wait complete. 2024-09-10T10:05:14.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.325+0000 7fde12c00640 1 Processor -- start 2024-09-10T10:05:14.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 -- start start 2024-09-10T10:05:14.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde0c15b330 0x7fde0c154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c14f300 0x7fde0c142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1680b0 con 0x7fde0c06e850 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c0b1800 con 0x7fde0c14f300 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c168230 con 0x7fde0c15b330 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37782/0 (socket says 172.21.15.110:37782) 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 -- 172.21.15.110:0/1561865357 learned_addr learned my addr 172.21.15.110:0/1561865357 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde03e00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde0c15b330 0x7fde0c154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde10c00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c14f300 0x7fde0c142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 -- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde0c15b330 msgr2=0x7fde0c154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde0c15b330 0x7fde0c154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 -- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c14f300 msgr2=0x7fde0c142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c14f300 0x7fde0c142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 -- 172.21.15.110:0/1561865357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c1562c0 con 0x7fde0c06e850 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde0be00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0ceef0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d620 tx=0x7fddfc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:14.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde08c00640 1 -- 172.21.15.110:0/1561865357 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00b620 con 0x7fde0c06e850 2024-09-10T10:05:14.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde08c00640 1 -- 172.21.15.110:0/1561865357 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddfc019070 con 0x7fde0c06e850 2024-09-10T10:05:14.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c15c3c0 con 0x7fde0c06e850 2024-09-10T10:05:14.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde08c00640 1 -- 172.21.15.110:0/1561865357 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00b9e0 con 0x7fde0c06e850 2024-09-10T10:05:14.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c155fe0 con 0x7fde0c06e850 2024-09-10T10:05:14.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.329+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0c0b0860 con 0x7fde0c06e850 2024-09-10T10:05:14.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.333+0000 7fde08c00640 1 -- 172.21.15.110:0/1561865357 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fddfc00bb80 con 0x7fde0c06e850 2024-09-10T10:05:14.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.333+0000 7fde08c00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdde8067ef0 0x7fdde806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.333+0000 7fde08c00640 1 -- 172.21.15.110:0/1561865357 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fddfc086c10 con 0x7fde0c06e850 2024-09-10T10:05:14.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.333+0000 7fde03e00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdde8067ef0 0x7fdde806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.333+0000 7fde03e00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdde8067ef0 0x7fdde806a3b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0cf8d0 tx=0x7fddf4007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:14.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.337+0000 7fde08c00640 1 -- 172.21.15.110:0/1561865357 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fddfc056e20 con 0x7fde0c06e850 2024-09-10T10:05:14.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.521+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol5770", "target": ["mon-mgr", ""]}) -- 0x7fde0c0cd590 con 0x7fdde8067ef0 2024-09-10T10:05:14.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.525+0000 7fde08c00640 1 -- 172.21.15.110:0/1561865357 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+29 (secure 0 0 0) 0x7fde0c0cd590 con 0x7fdde8067ef0 2024-09-10T10:05:14.533 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol5770 2024-09-10T10:05:14.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdde8067ef0 msgr2=0x7fdde806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdde8067ef0 0x7fdde806a3b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0cf8d0 tx=0x7fddf4007cd0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 msgr2=0x7fde0c0ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0ceef0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d620 tx=0x7fddfc00dae0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 shutdown_connections 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdde8067ef0 0x7fdde806a3b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c14f300 0x7fde0c142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fde0c15b330 0x7fde0c154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 --2- 172.21.15.110:0/1561865357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde0c06e850 0x7fde0c0ceef0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 >> 172.21.15.110:0/1561865357 conn(0x7fde0c062e40 msgr2=0x7fde0c0ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 shutdown_connections 2024-09-10T10:05:14.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.529+0000 7fde12c00640 1 -- 172.21.15.110:0/1561865357 wait complete. 2024-09-10T10:05:14.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:14.557 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/7f6a99a2741882bcdb162a577b457850.meta) 2024-09-10T10:05:14.584 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:05:14.584 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T10:05:14.584 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:05:14.584 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/_nogroup/subvol5770 2024-09-10T10:05:14.584 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:05:14.584 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:14.586 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol36291 subvol_grp84850 2024-09-10T10:05:14.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.765+0000 7ffb3c800640 1 Processor -- start 2024-09-10T10:05:14.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.765+0000 7ffb3c800640 1 -- start start 2024-09-10T10:05:14.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb3c800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ffb340cfe30 0x7ffb34071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb3c800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffb3406e7a0 0x7ffb340cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb3c800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb3406d2d0 con 0x7ffb340cd510 2024-09-10T10:05:14.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb3c800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ffb3415d9a0 con 0x7ffb340cfe30 2024-09-10T10:05:14.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb3c800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ffb3406d5b0 con 0x7ffb3406e7a0 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ffb340cfe30 0x7ffb34071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ffb340cfe30 0x7ffb34071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37790/0 (socket says 172.21.15.110:37790) 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39e00640 1 -- 172.21.15.110:0/544046935 learned_addr learned my addr 172.21.15.110:0/544046935 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39400640 1 --1- 172.21.15.110:0/544046935 >> v1:172.21.15.110:6789/0 conn(0x7ffb340cd510 0x7ffb3406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59966/0 (socket says 172.21.15.110:59966) 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb3a800640 1 --2- 172.21.15.110:0/544046935 >> v2:172.21.15.145:3300/0 conn(0x7ffb3406e7a0 0x7ffb340cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39e00640 1 -- 172.21.15.110:0/544046935 >> v2:172.21.15.145:3300/0 conn(0x7ffb3406e7a0 msgr2=0x7ffb340cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39e00640 1 --2- 172.21.15.110:0/544046935 >> v2:172.21.15.145:3300/0 conn(0x7ffb3406e7a0 0x7ffb340cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39e00640 1 -- 172.21.15.110:0/544046935 >> v1:172.21.15.110:6789/0 conn(0x7ffb340cd510 legacy=0x7ffb3406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb38a00640 1 -- 172.21.15.110:0/544046935 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206037260 0 0) 0x7ffb3406d2d0 con 0x7ffb340cd510 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39e00640 1 -- 172.21.15.110:0/544046935 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3406d890 con 0x7ffb340cfe30 2024-09-10T10:05:14.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb3a800640 1 --2- 172.21.15.110:0/544046935 >> v2:172.21.15.145:3300/0 conn(0x7ffb3406e7a0 0x7ffb340cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:14.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.769+0000 7ffb39e00640 1 --2- 172.21.15.110:0/544046935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb34071a40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ffb3406cb00 tx=0x7ffb2801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6f7be182a0925b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:14.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb38a00640 1 -- 172.21.15.110:0/544046935 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb280b1070 con 0x7ffb340cfe30 2024-09-10T10:05:14.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb38a00640 1 -- 172.21.15.110:0/544046935 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb280ac600 con 0x7ffb340cfe30 2024-09-10T10:05:14.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb38a00640 1 -- 172.21.15.110:0/544046935 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb280ac9d0 con 0x7ffb340cfe30 2024-09-10T10:05:14.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- 172.21.15.110:0/544046935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 msgr2=0x7ffb34071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 --2- 172.21.15.110:0/544046935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb34071a40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7ffb3406cb00 tx=0x7ffb2801c490 comp rx=0 tx=0).stop 2024-09-10T10:05:14.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- 172.21.15.110:0/544046935 shutdown_connections 2024-09-10T10:05:14.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 --2- 172.21.15.110:0/544046935 >> v2:172.21.15.145:3300/0 conn(0x7ffb3406e7a0 0x7ffb340cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 --2- 172.21.15.110:0/544046935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb34071a40 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- 172.21.15.110:0/544046935 >> 172.21.15.110:0/544046935 conn(0x7ffb3406bc50 msgr2=0x7ffb3406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:14.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- 172.21.15.110:0/544046935 shutdown_connections 2024-09-10T10:05:14.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- 172.21.15.110:0/544046935 wait complete. 2024-09-10T10:05:14.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 Processor -- start 2024-09-10T10:05:14.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- start start 2024-09-10T10:05:14.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb3406e7a0 0x7ffb3414b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb340cd510 0x7ffb340b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb340b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffb34072190 con 0x7ffb340cfe30 2024-09-10T10:05:14.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3415d6c0 con 0x7ffb340cd510 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3c800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffb34169180 con 0x7ffb3406e7a0 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb340b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb340b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37804/0 (socket says 172.21.15.110:37804) 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb3a800640 1 -- 172.21.15.110:0/3885758704 learned_addr learned my addr 172.21.15.110:0/3885758704 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb39e00640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb3406e7a0 0x7ffb3414b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.773+0000 7ffb39400640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb340cd510 0x7ffb340b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3a800640 1 -- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb3406e7a0 msgr2=0x7ffb3414b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3a800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb3406e7a0 0x7ffb3414b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3a800640 1 -- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb340cd510 msgr2=0x7ffb340b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3a800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb340cd510 0x7ffb340b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3a800640 1 -- 172.21.15.110:0/3885758704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb341599b0 con 0x7ffb340cfe30 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb39e00640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb3406e7a0 0x7ffb3414b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:14.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3a800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb340b1830 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000cd50 tx=0x7ffb3000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:14.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb2e000640 1 -- 172.21.15.110:0/3885758704 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000f680 con 0x7ffb340cfe30 2024-09-10T10:05:14.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb2e000640 1 -- 172.21.15.110:0/3885758704 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3000f840 con 0x7ffb340cfe30 2024-09-10T10:05:14.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb2e000640 1 -- 172.21.15.110:0/3885758704 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000fc30 con 0x7ffb340cfe30 2024-09-10T10:05:14.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3414c4a0 con 0x7ffb340cfe30 2024-09-10T10:05:14.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb3414ca60 con 0x7ffb340cfe30 2024-09-10T10:05:14.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3414cc60 con 0x7ffb340cfe30 2024-09-10T10:05:14.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb2e000640 1 -- 172.21.15.110:0/3885758704 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7ffb3000e3e0 con 0x7ffb340cfe30 2024-09-10T10:05:14.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb2e000640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb18067ef0 0x7ffb1806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:14.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb2e000640 1 -- 172.21.15.110:0/3885758704 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7ffb300875e0 con 0x7ffb340cfe30 2024-09-10T10:05:14.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.777+0000 7ffb39e00640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb18067ef0 0x7ffb1806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:14.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.781+0000 7ffb39e00640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb18067ef0 0x7ffb1806a3b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb280ae000 tx=0x7ffb280a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:14.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.781+0000 7ffb2e000640 1 -- 172.21.15.110:0/3885758704 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ffb3414cc60 con 0x7ffb340cfe30 2024-09-10T10:05:14.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.965+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol36291", "group_name": "subvol_grp84850", "target": ["mon-mgr", ""]}) -- 0x7ffb34159c90 con 0x7ffb18067ef0 2024-09-10T10:05:14.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb2e000640 1 -- 172.21.15.110:0/3885758704 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffb34159c90 con 0x7ffb18067ef0 2024-09-10T10:05:14.976 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/subvol_grp84850/subvol36291 2024-09-10T10:05:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb18067ef0 msgr2=0x7ffb1806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb18067ef0 0x7ffb1806a3b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb280ae000 tx=0x7ffb280a7000 comp rx=0 tx=0).stop 2024-09-10T10:05:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 msgr2=0x7ffb340b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb340b1830 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000cd50 tx=0x7ffb3000a650 comp rx=0 tx=0).stop 2024-09-10T10:05:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 shutdown_connections 2024-09-10T10:05:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffb18067ef0 0x7ffb1806a3b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb340cfe30 0x7ffb340b1830 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffb340cd510 0x7ffb340b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 --2- 172.21.15.110:0/3885758704 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffb3406e7a0 0x7ffb3414b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 >> 172.21.15.110:0/3885758704 conn(0x7ffb3406bc50 msgr2=0x7ffb341563a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 shutdown_connections 2024-09-10T10:05:14.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:14.973+0000 7ffb3c800640 1 -- 172.21.15.110:0/3885758704 wait complete. 2024-09-10T10:05:14.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:14.998 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/d3047248e32daa181424db401a129b54.meta) 2024-09-10T10:05:15.021 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:05:15.021 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T10:05:15.021 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:05:15.022 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/subvol_grp84850/subvol36291 2024-09-10T10:05:15.022 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:05:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:15.024 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol5770 2024-09-10T10:05:15.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda28e00640 1 Processor -- start 2024-09-10T10:05:15.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda28e00640 1 -- start start 2024-09-10T10:05:15.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda28e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fda2415b330 0x7fda2406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda28e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda240b4960 con 0x7fda240ce4d0 2024-09-10T10:05:15.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda28e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda240b4c40 con 0x7fda2406e850 2024-09-10T10:05:15.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda28e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fda240b5570 con 0x7fda2415b330 2024-09-10T10:05:15.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda21e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fda2415b330 0x7fda2406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda21e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fda2415b330 0x7fda2406c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37814/0 (socket says 172.21.15.110:37814) 2024-09-10T10:05:15.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda21400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fda240ce4d0 0x7fda2406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59970/0 (socket says 172.21.15.110:59970) 2024-09-10T10:05:15.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda21e00640 1 -- 172.21.15.110:0/3583381272 learned_addr learned my addr 172.21.15.110:0/3583381272 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:15.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda21e00640 1 -- 172.21.15.110:0/3583381272 >> v1:172.21.15.145:6789/0 conn(0x7fda2406e850 legacy=0x7fda240b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda21e00640 1 -- 172.21.15.110:0/3583381272 >> v1:172.21.15.110:6789/0 conn(0x7fda240ce4d0 legacy=0x7fda2406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda21e00640 1 -- 172.21.15.110:0/3583381272 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda240b4f20 con 0x7fda2415b330 2024-09-10T10:05:15.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.197+0000 7fda20a00640 1 -- 172.21.15.110:0/3583381272 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483249646 0 0) 0x7fda240b4960 con 0x7fda240ce4d0 2024-09-10T10:05:15.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda21e00640 1 --2- 172.21.15.110:0/3583381272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda2415b330 0x7fda2406c6a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fda1400bb10 tx=0x7fda1401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2393433b23f947d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:15.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda20a00640 1 -- 172.21.15.110:0/3583381272 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140b4070 con 0x7fda2415b330 2024-09-10T10:05:15.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda20a00640 1 -- 172.21.15.110:0/3583381272 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda140a7c10 con 0x7fda2415b330 2024-09-10T10:05:15.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda20a00640 1 -- 172.21.15.110:0/3583381272 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140af440 con 0x7fda2415b330 2024-09-10T10:05:15.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda28e00640 1 -- 172.21.15.110:0/3583381272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda2415b330 msgr2=0x7fda2406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda28e00640 1 --2- 172.21.15.110:0/3583381272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda2415b330 0x7fda2406c6a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fda1400bb10 tx=0x7fda1401ec30 comp rx=0 tx=0).stop 2024-09-10T10:05:15.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda28e00640 1 -- 172.21.15.110:0/3583381272 shutdown_connections 2024-09-10T10:05:15.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda28e00640 1 --2- 172.21.15.110:0/3583381272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda2415b330 0x7fda2406c6a0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda28e00640 1 -- 172.21.15.110:0/3583381272 >> 172.21.15.110:0/3583381272 conn(0x7fda24062e40 msgr2=0x7fda2406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:15.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda28e00640 1 -- 172.21.15.110:0/3583381272 shutdown_connections 2024-09-10T10:05:15.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.201+0000 7fda28e00640 1 -- 172.21.15.110:0/3583381272 wait complete. 2024-09-10T10:05:15.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 Processor -- start 2024-09-10T10:05:15.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 -- start start 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2406e850 0x7fda240ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda2415b330 0x7fda240aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda24149280 0x7fda2414cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fda240b3b00 con 0x7fda24149280 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fda240b3980 con 0x7fda2406e850 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fda240b3c80 con 0x7fda2415b330 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda24149280 0x7fda2414cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda21e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2406e850 0x7fda240ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda21400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda2415b330 0x7fda240aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda24149280 0x7fda2414cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37826/0 (socket says 172.21.15.110:37826) 2024-09-10T10:05:15.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 -- 172.21.15.110:0/37592391 learned_addr learned my addr 172.21.15.110:0/37592391 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 -- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda2415b330 msgr2=0x7fda240aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda2415b330 0x7fda240aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 -- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2406e850 msgr2=0x7fda240ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2406e850 0x7fda240ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 -- 172.21.15.110:0/37592391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2415bce0 con 0x7fda24149280 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda22800640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda24149280 0x7fda2414cd30 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00c700 tx=0x7fda1c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda1a000640 1 -- 172.21.15.110:0/37592391 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c003b80 con 0x7fda24149280 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda1a000640 1 -- 172.21.15.110:0/37592391 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1c003d20 con 0x7fda24149280 2024-09-10T10:05:15.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2415bfc0 con 0x7fda24149280 2024-09-10T10:05:15.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda1a000640 1 -- 172.21.15.110:0/37592391 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00aa30 con 0x7fda24149280 2024-09-10T10:05:15.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.205+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2415c580 con 0x7fda24149280 2024-09-10T10:05:15.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.209+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2414af20 con 0x7fda24149280 2024-09-10T10:05:15.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.209+0000 7fda1a000640 1 -- 172.21.15.110:0/37592391 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 88006+0+0 (secure 0 0 0) 0x7fda1c00abd0 con 0x7fda24149280 2024-09-10T10:05:15.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.209+0000 7fda1a000640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda04067f20 0x7fda0406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.209+0000 7fda1a000640 1 -- 172.21.15.110:0/37592391 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fda1c086530 con 0x7fda24149280 2024-09-10T10:05:15.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.209+0000 7fda21e00640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda04067f20 0x7fda0406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.209+0000 7fda21e00640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda04067f20 0x7fda0406a3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda140b1000 tx=0x7fda140a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:15.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.213+0000 7fda1a000640 1 -- 172.21.15.110:0/37592391 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fda2414af20 con 0x7fda24149280 2024-09-10T10:05:15.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.401+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol5770", "target": ["mon-mgr", ""]}) -- 0x7fda24149ae0 con 0x7fda04067f20 2024-09-10T10:05:15.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda1a000640 1 -- 172.21.15.110:0/37592391 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda140a7bb0 con 0x7fda04067f20 2024-09-10T10:05:15.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda04067f20 msgr2=0x7fda0406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda04067f20 0x7fda0406a3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda140b1000 tx=0x7fda140a9070 comp rx=0 tx=0).stop 2024-09-10T10:05:15.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda24149280 msgr2=0x7fda2414cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda24149280 0x7fda2414cd30 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00c700 tx=0x7fda1c00cbc0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 shutdown_connections 2024-09-10T10:05:15.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda04067f20 0x7fda0406a3e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda24149280 0x7fda2414cd30 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda2415b330 0x7fda240aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 --2- 172.21.15.110:0/37592391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda2406e850 0x7fda240ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 >> 172.21.15.110:0/37592391 conn(0x7fda24062e40 msgr2=0x7fda24072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:15.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 shutdown_connections 2024-09-10T10:05:15.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.433+0000 7fda28e00640 1 -- 172.21.15.110:0/37592391 wait complete. 2024-09-10T10:05:15.442 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.441+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.443 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.441+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.443 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.441+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.443 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.441+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.443 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.441+0000 7f95df800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.457 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol36291 subvol_grp84850 2024-09-10T10:05:15.490 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.457+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.490 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.457+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.490 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.457+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.490 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.457+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.490 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:15.457+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:15.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.625+0000 7fe5ba400640 1 Processor -- start 2024-09-10T10:05:15.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.625+0000 7fe5ba400640 1 -- start start 2024-09-10T10:05:15.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5ba400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe5b4155970 0x7fe5b40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5ba400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b4155d50 con 0x7fe5b406f1e0 2024-09-10T10:05:15.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5ba400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b40b34d0 con 0x7fe5b415c560 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5ba400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe5b40b4570 con 0x7fe5b4155970 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b3e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe5b406f1e0 0x7fe5b40ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59978/0 (socket says 172.21.15.110:59978) 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b3e00640 1 -- 172.21.15.110:0/3206440808 learned_addr learned my addr 172.21.15.110:0/3206440808 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b3400640 1 --2- 172.21.15.110:0/3206440808 >> v2:172.21.15.145:3300/0 conn(0x7fe5b4155970 0x7fe5b40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b2000640 1 -- 172.21.15.110:0/3206440808 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394624384 0 0) 0x7fe5b4155d50 con 0x7fe5b406f1e0 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b2000640 1 -- 172.21.15.110:0/3206440808 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe598003530 con 0x7fe5b406f1e0 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b2000640 1 -- 172.21.15.110:0/3206440808 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133287851 0 0) 0x7fe5b40b34d0 con 0x7fe5b415c560 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b2000640 1 -- 172.21.15.110:0/3206440808 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5b4155d50 con 0x7fe5b415c560 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b3400640 1 -- 172.21.15.110:0/3206440808 >> v1:172.21.15.145:6789/0 conn(0x7fe5b415c560 legacy=0x7fe5b406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b3400640 1 -- 172.21.15.110:0/3206440808 >> v1:172.21.15.110:6789/0 conn(0x7fe5b406f1e0 legacy=0x7fe5b40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.629+0000 7fe5b3400640 1 -- 172.21.15.110:0/3206440808 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b40b37b0 con 0x7fe5b4155970 2024-09-10T10:05:15.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5b3400640 1 --2- 172.21.15.110:0/3206440808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 0x7fe5b40cc8c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40097c0 tx=0x7fe5a401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f83fd2cc6bd59e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:15.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5b2000640 1 -- 172.21.15.110:0/3206440808 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40a7070 con 0x7fe5b4155970 2024-09-10T10:05:15.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5b2000640 1 -- 172.21.15.110:0/3206440808 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a40b0070 con 0x7fe5b4155970 2024-09-10T10:05:15.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5b2000640 1 -- 172.21.15.110:0/3206440808 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40ab870 con 0x7fe5b4155970 2024-09-10T10:05:15.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 -- 172.21.15.110:0/3206440808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 msgr2=0x7fe5b40cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 --2- 172.21.15.110:0/3206440808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 0x7fe5b40cc8c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40097c0 tx=0x7fe5a401c470 comp rx=0 tx=0).stop 2024-09-10T10:05:15.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 -- 172.21.15.110:0/3206440808 shutdown_connections 2024-09-10T10:05:15.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 --2- 172.21.15.110:0/3206440808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 0x7fe5b40cc8c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 -- 172.21.15.110:0/3206440808 >> 172.21.15.110:0/3206440808 conn(0x7fe5b406bc50 msgr2=0x7fe5b406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:15.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 -- 172.21.15.110:0/3206440808 shutdown_connections 2024-09-10T10:05:15.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 -- 172.21.15.110:0/3206440808 wait complete. 2024-09-10T10:05:15.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 Processor -- start 2024-09-10T10:05:15.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.633+0000 7fe5ba400640 1 -- start start 2024-09-10T10:05:15.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5b406f1e0 0x7fe5b4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 0x7fe5b415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5b415c560 0x7fe5b40ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b40cf340 con 0x7fe5b406f1e0 2024-09-10T10:05:15.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b40cce00 con 0x7fe5b4155970 2024-09-10T10:05:15.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b40cf4c0 con 0x7fe5b415c560 2024-09-10T10:05:15.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5b406f1e0 0x7fe5b4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5b406f1e0 0x7fe5b4143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37838/0 (socket says 172.21.15.110:37838) 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 -- 172.21.15.110:0/1109141234 learned_addr learned my addr 172.21.15.110:0/1109141234 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3e00640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5b415c560 0x7fe5b40ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b2a00640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 0x7fe5b415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 -- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5b415c560 msgr2=0x7fe5b40ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5b415c560 0x7fe5b40ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 -- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 msgr2=0x7fe5b415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 0x7fe5b415ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 -- 172.21.15.110:0/1109141234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b415cee0 con 0x7fe5b406f1e0 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b2a00640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 0x7fe5b415ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3e00640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5b415c560 0x7fe5b40ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:15.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5b3400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5b406f1e0 0x7fe5b4143d30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4019080 tx=0x7fe5a401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:15.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ab400640 1 -- 172.21.15.110:0/1109141234 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40bf610 con 0x7fe5b406f1e0 2024-09-10T10:05:15.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ab400640 1 -- 172.21.15.110:0/1109141234 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a40abe50 con 0x7fe5b406f1e0 2024-09-10T10:05:15.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ab400640 1 -- 172.21.15.110:0/1109141234 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40bf9b0 con 0x7fe5b406f1e0 2024-09-10T10:05:15.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b415d1c0 con 0x7fe5b406f1e0 2024-09-10T10:05:15.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b415d780 con 0x7fe5b406f1e0 2024-09-10T10:05:15.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.637+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b40cc8c0 con 0x7fe5b406f1e0 2024-09-10T10:05:15.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.641+0000 7fe5ab400640 1 -- 172.21.15.110:0/1109141234 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fe5a40bd020 con 0x7fe5b406f1e0 2024-09-10T10:05:15.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.641+0000 7fe5ab400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5940680d0 0x7fe59406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:15.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.641+0000 7fe5ab400640 1 -- 172.21.15.110:0/1109141234 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fe5a4120c50 con 0x7fe5b406f1e0 2024-09-10T10:05:15.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.641+0000 7fe5b2a00640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5940680d0 0x7fe59406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:15.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.641+0000 7fe5b2a00640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5940680d0 0x7fe59406a590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe5b41fa8f0 tx=0x7fe5a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:15.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.645+0000 7fe5ab400640 1 -- 172.21.15.110:0/1109141234 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe5a4016090 con 0x7fe5b406f1e0 2024-09-10T10:05:15.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.825+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol36291", "group_name": "subvol_grp84850", "target": ["mon-mgr", ""]}) -- 0x7fe5b4166df0 con 0x7fe5940680d0 2024-09-10T10:05:15.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.853+0000 7fe5ab400640 1 -- 172.21.15.110:0/1109141234 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5b4166df0 con 0x7fe5940680d0 2024-09-10T10:05:15.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5940680d0 msgr2=0x7fe59406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5940680d0 0x7fe59406a590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe5b41fa8f0 tx=0x7fe5a0009040 comp rx=0 tx=0).stop 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5b406f1e0 msgr2=0x7fe5b4143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5b406f1e0 0x7fe5b4143d30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4019080 tx=0x7fe5a401acb0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 shutdown_connections 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe5940680d0 0x7fe59406a590 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe5b415c560 0x7fe5b40ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5b4155970 0x7fe5b415ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 --2- 172.21.15.110:0/1109141234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5b406f1e0 0x7fe5b4143d30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 >> 172.21.15.110:0/1109141234 conn(0x7fe5b406bc50 msgr2=0x7fe5b40710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 shutdown_connections 2024-09-10T10:05:15.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15.857+0000 7fe5ba400640 1 -- 172.21.15.110:0/1109141234 wait complete. 2024-09-10T10:05:15.880 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T10:05:15.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:05:15.934 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-10T10:05:15.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:15.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp84850 2024-09-10T10:05:16.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95c5200640 1 Processor -- start 2024-09-10T10:05:16.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95c5200640 1 -- start start 2024-09-10T10:05:16.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95c5200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f95c00ce4d0 0x7f95c006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95c5200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c006e850 0x7f95c00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95c5200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c00b4960 con 0x7f95c015b330 2024-09-10T10:05:16.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95c5200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f95c00b5570 con 0x7f95c00ce4d0 2024-09-10T10:05:16.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95c5200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f95c00b4c40 con 0x7f95c006e850 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95be800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c006e850 0x7f95c00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95be800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c006e850 0x7f95c00b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:42432/0 (socket says 172.21.15.110:42432) 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95be800640 1 -- 172.21.15.110:0/3035957614 learned_addr learned my addr 172.21.15.110:0/3035957614 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95bd400640 1 --2- 172.21.15.110:0/3035957614 >> v2:172.21.15.145:3300/0 conn(0x7f95c00ce4d0 0x7f95c006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95be800640 1 -- 172.21.15.110:0/3035957614 >> v2:172.21.15.145:3300/0 conn(0x7f95c00ce4d0 msgr2=0x7f95c006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95be800640 1 --2- 172.21.15.110:0/3035957614 >> v2:172.21.15.145:3300/0 conn(0x7f95c00ce4d0 0x7f95c006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95be800640 1 -- 172.21.15.110:0/3035957614 >> v1:172.21.15.145:6789/0 conn(0x7f95c015b330 legacy=0x7f95c006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95be800640 1 -- 172.21.15.110:0/3035957614 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c00b4f20 con 0x7f95c006e850 2024-09-10T10:05:16.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.121+0000 7f95bd400640 1 --2- 172.21.15.110:0/3035957614 >> v2:172.21.15.145:3300/0 conn(0x7f95c00ce4d0 0x7f95c006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95be800640 1 --2- 172.21.15.110:0/3035957614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c006e850 0x7f95c00b0860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f95b800bb10 tx=0x7f95b801ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d2f98932ae85b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95bca00640 1 -- 172.21.15.110:0/3035957614 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b80b4070 con 0x7f95c006e850 2024-09-10T10:05:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95bca00640 1 -- 172.21.15.110:0/3035957614 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b80a7c10 con 0x7f95c006e850 2024-09-10T10:05:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95bca00640 1 -- 172.21.15.110:0/3035957614 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b80af440 con 0x7f95c006e850 2024-09-10T10:05:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95c5200640 1 -- 172.21.15.110:0/3035957614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c006e850 msgr2=0x7f95c00b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95c5200640 1 --2- 172.21.15.110:0/3035957614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c006e850 0x7f95c00b0860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f95b800bb10 tx=0x7f95b801ec30 comp rx=0 tx=0).stop 2024-09-10T10:05:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95c5200640 1 -- 172.21.15.110:0/3035957614 shutdown_connections 2024-09-10T10:05:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95c5200640 1 --2- 172.21.15.110:0/3035957614 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c006e850 0x7f95c00b0860 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95c5200640 1 --2- 172.21.15.110:0/3035957614 >> v2:172.21.15.145:3300/0 conn(0x7f95c00ce4d0 0x7f95c006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95c5200640 1 -- 172.21.15.110:0/3035957614 >> 172.21.15.110:0/3035957614 conn(0x7f95c0062e40 msgr2=0x7f95c006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95c5200640 1 -- 172.21.15.110:0/3035957614 shutdown_connections 2024-09-10T10:05:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.125+0000 7f95c5200640 1 -- 172.21.15.110:0/3035957614 wait complete. 2024-09-10T10:05:16.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 Processor -- start 2024-09-10T10:05:16.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 -- start start 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95c006e850 0x7f95c01f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c015b330 0x7f95c01fa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95c0150810 0x7f95c0164480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95c00b36b0 con 0x7f95c0150810 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f95c00b1800 con 0x7f95c006e850 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f95c00b3830 con 0x7f95c015b330 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95c0150810 0x7f95c0164480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95c0150810 0x7f95c0164480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37854/0 (socket says 172.21.15.110:37854) 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 -- 172.21.15.110:0/3357166316 learned_addr learned my addr 172.21.15.110:0/3357166316 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95bde00640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95c006e850 0x7f95c01f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95bd400640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c015b330 0x7f95c01fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 -- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c015b330 msgr2=0x7f95c01fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c015b330 0x7f95c01fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 -- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95c006e850 msgr2=0x7f95c01f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95c006e850 0x7f95c01f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 -- 172.21.15.110:0/3357166316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c01511f0 con 0x7f95c0150810 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95be800640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95c0150810 0x7f95c0164480 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f95b800b2c0 tx=0x7f95b801b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95b6000640 1 -- 172.21.15.110:0/3357166316 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b80b4070 con 0x7f95c0150810 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95b6000640 1 -- 172.21.15.110:0/3357166316 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b80b6070 con 0x7f95c0150810 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c014d2a0 con 0x7f95c0150810 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95b6000640 1 -- 172.21.15.110:0/3357166316 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b80bb470 con 0x7f95c0150810 2024-09-10T10:05:16.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c014d860 con 0x7f95c0150810 2024-09-10T10:05:16.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.129+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c006c6a0 con 0x7f95c0150810 2024-09-10T10:05:16.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.133+0000 7f95b6000640 1 -- 172.21.15.110:0/3357166316 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f95b80c43f0 con 0x7f95c0150810 2024-09-10T10:05:16.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.133+0000 7f95b6000640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f95a00680d0 0x7f95a006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.133+0000 7f95b6000640 1 -- 172.21.15.110:0/3357166316 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f95b8121300 con 0x7f95c0150810 2024-09-10T10:05:16.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.133+0000 7f95bde00640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f95a00680d0 0x7f95a006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.133+0000 7f95bde00640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f95a00680d0 0x7f95a006a590 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f95c014efc0 tx=0x7f95b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:16.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.137+0000 7f95b6000640 1 -- 172.21.15.110:0/3357166316 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f95b8018820 con 0x7f95c0150810 2024-09-10T10:05:16.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.313+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp84850", "target": ["mon-mgr", ""]}) -- 0x7f95c0072890 con 0x7f95a00680d0 2024-09-10T10:05:16.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95b6000640 1 -- 172.21.15.110:0/3357166316 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95c0072890 con 0x7f95a00680d0 2024-09-10T10:05:16.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f95a00680d0 msgr2=0x7f95a006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f95a00680d0 0x7f95a006a590 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f95c014efc0 tx=0x7f95b0009040 comp rx=0 tx=0).stop 2024-09-10T10:05:16.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95c0150810 msgr2=0x7f95c0164480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95c0150810 0x7f95c0164480 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f95b800b2c0 tx=0x7f95b801b9d0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 shutdown_connections 2024-09-10T10:05:16.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f95a00680d0 0x7f95a006a590 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95c0150810 0x7f95c0164480 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f95c015b330 0x7f95c01fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 --2- 172.21.15.110:0/3357166316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f95c006e850 0x7f95c01f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 >> 172.21.15.110:0/3357166316 conn(0x7f95c0062e40 msgr2=0x7f95c015e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:16.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 shutdown_connections 2024-09-10T10:05:16.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.333+0000 7f95c5200640 1 -- 172.21.15.110:0/3357166316 wait complete. 2024-09-10T10:05:16.356 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:05:16.357 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:05:16.435 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:05:16.519 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:05:16.519 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:05:16.520 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:16.520 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:16.529 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:16.529 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:16.581 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:05:16.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26c00640 1 Processor -- start 2024-09-10T10:05:16.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26c00640 1 -- start start 2024-09-10T10:05:16.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6a2006d530 0x7f6a200cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2006da80 con 0x7f6a2015c360 2024-09-10T10:05:16.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2006dd60 con 0x7f6a2006eb20 2024-09-10T10:05:16.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6a200b5b00 con 0x7f6a2006d530 2024-09-10T10:05:16.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a25800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6a2015c360 0x7f6a200b53f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:59992/0 (socket says 172.21.15.110:59992) 2024-09-10T10:05:16.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a25800640 1 -- 172.21.15.110:0/1235148824 learned_addr learned my addr 172.21.15.110:0/1235148824 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:16.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a24e00640 1 --2- 172.21.15.110:0/1235148824 >> v2:172.21.15.145:3300/0 conn(0x7f6a2006d530 0x7f6a200cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3055834736 0 0) 0x7f6a2006da80 con 0x7f6a2015c360 2024-09-10T10:05:16.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a08003530 con 0x7f6a2015c360 2024-09-10T10:05:16.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2587129714 0 0) 0x7f6a08003530 con 0x7f6a2015c360 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 >> v2:172.21.15.145:3300/0 conn(0x7f6a2006d530 msgr2=0x7f6a200cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 --2- 172.21.15.110:0/1235148824 >> v2:172.21.15.145:3300/0 conn(0x7f6a2006d530 0x7f6a200cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 >> v1:172.21.15.145:6789/0 conn(0x7f6a2006eb20 legacy=0x7f6a2015c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2006e040 con 0x7f6a2015c360 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f6a14001a00 con 0x7f6a2015c360 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 >> v1:172.21.15.110:6789/0 conn(0x7f6a2015c360 legacy=0x7f6a200b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a08003a10 0x7f6a08003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a08004810 0x7f6a08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a080051d0 0x7f6a0800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0800c0a0 con 0x7f6a080051d0 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0800c250 con 0x7f6a08004810 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0800c450 con 0x7f6a08003a10 2024-09-10T10:05:16.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26200640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a080051d0 0x7f6a0800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a25800640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a08003a10 0x7f6a08003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a24e00640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a08004810 0x7f6a08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26200640 1 -- 172.21.15.110:0/1235148824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a08003a10 msgr2=0x7f6a08003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:16.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26200640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a08003a10 0x7f6a08003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26200640 1 -- 172.21.15.110:0/1235148824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a08004810 msgr2=0x7f6a08004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:16.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26200640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a08004810 0x7f6a08004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.757+0000 7f6a26200640 1 -- 172.21.15.110:0/1235148824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a08003530 con 0x7f6a080051d0 2024-09-10T10:05:16.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26200640 1 -- 172.21.15.110:0/1235148824 reap_dead start 2024-09-10T10:05:16.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26200640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a080051d0 0x7f6a0800b9d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f6a18001780 tx=0x7f6a1801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e25808d87a04409 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:16.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180013a0 con 0x7f6a080051d0 2024-09-10T10:05:16.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a180a9070 con 0x7f6a080051d0 2024-09-10T10:05:16.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a1fe00640 1 -- 172.21.15.110:0/1235148824 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18003600 con 0x7f6a080051d0 2024-09-10T10:05:16.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26c00640 1 -- 172.21.15.110:0/1235148824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a080051d0 msgr2=0x7f6a0800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26c00640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a080051d0 0x7f6a0800b9d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f6a18001780 tx=0x7f6a1801c420 comp rx=0 tx=0).stop 2024-09-10T10:05:16.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26c00640 1 -- 172.21.15.110:0/1235148824 shutdown_connections 2024-09-10T10:05:16.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26c00640 1 --2- 172.21.15.110:0/1235148824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a080051d0 0x7f6a0800b9d0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26c00640 1 -- 172.21.15.110:0/1235148824 >> 172.21.15.110:0/1235148824 conn(0x7f6a2006c050 msgr2=0x7f6a2006c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:16.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26c00640 1 -- 172.21.15.110:0/1235148824 shutdown_connections 2024-09-10T10:05:16.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.761+0000 7f6a26c00640 1 -- 172.21.15.110:0/1235148824 wait complete. 2024-09-10T10:05:16.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 Processor -- start 2024-09-10T10:05:16.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 -- start start 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a080051d0 0x7f6a20153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a20142e40 0x7f6a20156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a201435e0 0x7f6a2015e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a200b53f0 con 0x7f6a201435e0 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2006e600 con 0x7f6a080051d0 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6a200b5570 con 0x7f6a20142e40 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a201435e0 0x7f6a2015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a201435e0 0x7f6a2015e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37872/0 (socket says 172.21.15.110:37872) 2024-09-10T10:05:16.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 -- 172.21.15.110:0/1301645127 learned_addr learned my addr 172.21.15.110:0/1301645127 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:16.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a25800640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a080051d0 0x7f6a20153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a24e00640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a20142e40 0x7f6a20156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 -- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a20142e40 msgr2=0x7f6a20156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a20142e40 0x7f6a20156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 -- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a080051d0 msgr2=0x7f6a20153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a080051d0 0x7f6a20153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 -- 172.21.15.110:0/1301645127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a201432c0 con 0x7f6a201435e0 2024-09-10T10:05:16.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a24e00640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a20142e40 0x7f6a20156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:16.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26200640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a201435e0 0x7f6a2015e910 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6a1801ccc0 tx=0x7f6a08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:16.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a1d600640 1 -- 172.21.15.110:0/1301645127 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180bb070 con 0x7f6a201435e0 2024-09-10T10:05:16.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a1d600640 1 -- 172.21.15.110:0/1301645127 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a180050d0 con 0x7f6a201435e0 2024-09-10T10:05:16.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a201518e0 con 0x7f6a201435e0 2024-09-10T10:05:16.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a1d600640 1 -- 172.21.15.110:0/1301645127 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18007a40 con 0x7f6a201435e0 2024-09-10T10:05:16.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a20151ea0 con 0x7f6a201435e0 2024-09-10T10:05:16.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.765+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2006da40 con 0x7f6a201435e0 2024-09-10T10:05:16.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.769+0000 7f6a1d600640 1 -- 172.21.15.110:0/1301645127 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f6a180be070 con 0x7f6a201435e0 2024-09-10T10:05:16.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.769+0000 7f6a1d600640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a000680d0 0x7f6a0006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.769+0000 7f6a1d600640 1 -- 172.21.15.110:0/1301645127 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f6a1811ff40 con 0x7f6a201435e0 2024-09-10T10:05:16.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.769+0000 7f6a25800640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a000680d0 0x7f6a0006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.769+0000 7f6a25800640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a000680d0 0x7f6a0006a590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6a14000f90 tx=0x7f6a14006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:16.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.773+0000 7f6a1d600640 1 -- 172.21.15.110:0/1301645127 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6a18016130 con 0x7f6a201435e0 2024-09-10T10:05:16.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6a2015a2c0 con 0x7f6a201435e0 2024-09-10T10:05:16.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a1d600640 1 -- 172.21.15.110:0/1301645127 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v142) ==== 80+0+0 (secure 0 0 0) 0x7f6a180efd60 con 0x7f6a201435e0 2024-09-10T10:05:16.946 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:05:16.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a000680d0 msgr2=0x7f6a0006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a000680d0 0x7f6a0006a590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6a14000f90 tx=0x7f6a14006040 comp rx=0 tx=0).stop 2024-09-10T10:05:16.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a201435e0 msgr2=0x7f6a2015e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a201435e0 0x7f6a2015e910 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6a1801ccc0 tx=0x7f6a08005840 comp rx=0 tx=0).stop 2024-09-10T10:05:16.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 shutdown_connections 2024-09-10T10:05:16.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6a000680d0 0x7f6a0006a590 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a201435e0 0x7f6a2015e910 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6a20142e40 0x7f6a20156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 --2- 172.21.15.110:0/1301645127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6a080051d0 0x7f6a20153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 >> 172.21.15.110:0/1301645127 conn(0x7f6a2006c050 msgr2=0x7f6a2015a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:16.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 shutdown_connections 2024-09-10T10:05:16.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:16.941+0000 7f6a26c00640 1 -- 172.21.15.110:0/1301645127 wait complete. 2024-09-10T10:05:16.967 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:05:16.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:16.968 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:16.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:05:16.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:16.996 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:05:17.044 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:05:17.044 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:17.045 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:17.054 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:17.054 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:17.105 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:05:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f4846600640 1 Processor -- start 2024-09-10T10:05:17.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f4846600640 1 -- start start 2024-09-10T10:05:17.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f4846600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48400cfb90 0x7f484015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f4846600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400b3f40 0x7f48400cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f4846600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f484015d4d0 con 0x7f48400cff70 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f4846600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f484006d130 con 0x7f48400cfb90 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f4846600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f484015d7b0 con 0x7f48400b3f40 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48400cfb90 0x7f484015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48400cfb90 0x7f484015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37886/0 (socket says 172.21.15.110:37886) 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483f400640 1 -- 172.21.15.110:0/3814175274 learned_addr learned my addr 172.21.15.110:0/3814175274 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483ea00640 1 --2- 172.21.15.110:0/3814175274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400b3f40 0x7f48400cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483f400640 1 -- 172.21.15.110:0/3814175274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400b3f40 msgr2=0x7f48400cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483f400640 1 --2- 172.21.15.110:0/3814175274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400b3f40 0x7f48400cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483f400640 1 -- 172.21.15.110:0/3814175274 >> v1:172.21.15.110:6789/0 conn(0x7f48400cff70 legacy=0x7f4840160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483f400640 1 -- 172.21.15.110:0/3814175274 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484006d380 con 0x7f48400cfb90 2024-09-10T10:05:17.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.237+0000 7f483ea00640 1 --2- 172.21.15.110:0/3814175274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400b3f40 0x7f48400cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:17.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f483f400640 1 --2- 172.21.15.110:0/3814175274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400cfb90 0x7f484015ca10 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4830009e50 tx=0x7f483001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe8054533c6e8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:17.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f483e000640 1 -- 172.21.15.110:0/3814175274 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300a5660 con 0x7f48400cfb90 2024-09-10T10:05:17.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f483e000640 1 -- 172.21.15.110:0/3814175274 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48300a5800 con 0x7f48400cfb90 2024-09-10T10:05:17.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f483e000640 1 -- 172.21.15.110:0/3814175274 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300a5bd0 con 0x7f48400cfb90 2024-09-10T10:05:17.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 -- 172.21.15.110:0/3814175274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400cfb90 msgr2=0x7f484015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 --2- 172.21.15.110:0/3814175274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400cfb90 0x7f484015ca10 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4830009e50 tx=0x7f483001c9a0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 -- 172.21.15.110:0/3814175274 shutdown_connections 2024-09-10T10:05:17.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 --2- 172.21.15.110:0/3814175274 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400b3f40 0x7f48400cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 --2- 172.21.15.110:0/3814175274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400cfb90 0x7f484015ca10 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 -- 172.21.15.110:0/3814175274 >> 172.21.15.110:0/3814175274 conn(0x7f4840062d10 msgr2=0x7f484006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:17.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 -- 172.21.15.110:0/3814175274 shutdown_connections 2024-09-10T10:05:17.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 -- 172.21.15.110:0/3814175274 wait complete. 2024-09-10T10:05:17.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 Processor -- start 2024-09-10T10:05:17.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.241+0000 7f4846600640 1 -- start start 2024-09-10T10:05:17.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b3f40 0x7f4840151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48400cfb90 0x7f4840164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400cff70 0x7f4840155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4840169500 con 0x7f48400b3f40 2024-09-10T10:05:17.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4840169380 con 0x7f48400cfb90 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4840169680 con 0x7f48400cff70 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b3f40 0x7f4840151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b3f40 0x7f4840151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37892/0 (socket says 172.21.15.110:37892) 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 -- 172.21.15.110:0/516737472 learned_addr learned my addr 172.21.15.110:0/516737472 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483ea00640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48400cfb90 0x7f4840164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483fe00640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400cff70 0x7f4840155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 -- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400cff70 msgr2=0x7f4840155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400cff70 0x7f4840155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 -- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48400cfb90 msgr2=0x7f4840164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48400cfb90 0x7f4840164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 -- 172.21.15.110:0/516737472 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4840161360 con 0x7f48400b3f40 2024-09-10T10:05:17.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483ea00640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48400cfb90 0x7f4840164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:17.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f483f400640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b3f40 0x7f4840151830 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f48300af040 tx=0x7f4830019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:17.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4837400640 1 -- 172.21.15.110:0/516737472 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300bc020 con 0x7f48400b3f40 2024-09-10T10:05:17.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4837400640 1 -- 172.21.15.110:0/516737472 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48300b8880 con 0x7f48400b3f40 2024-09-10T10:05:17.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4840161640 con 0x7f48400b3f40 2024-09-10T10:05:17.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4837400640 1 -- 172.21.15.110:0/516737472 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830009510 con 0x7f48400b3f40 2024-09-10T10:05:17.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4840152040 con 0x7f48400b3f40 2024-09-10T10:05:17.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48400cd5a0 con 0x7f48400b3f40 2024-09-10T10:05:17.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.245+0000 7f4837400640 1 -- 172.21.15.110:0/516737472 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f48300bf020 con 0x7f48400b3f40 2024-09-10T10:05:17.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.249+0000 7f4837400640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48200680a0 0x7f482006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.249+0000 7f4837400640 1 -- 172.21.15.110:0/516737472 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f4830121db0 con 0x7f48400b3f40 2024-09-10T10:05:17.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.249+0000 7f483ea00640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48200680a0 0x7f482006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.249+0000 7f483ea00640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48200680a0 0x7f482006a560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4840166ea0 tx=0x7f482c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:17.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.253+0000 7f4837400640 1 -- 172.21.15.110:0/516737472 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f48300c6030 con 0x7f48400b3f40 2024-09-10T10:05:17.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.421+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f48401610c0 con 0x7f48400b3f40 2024-09-10T10:05:17.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.421+0000 7f4837400640 1 -- 172.21.15.110:0/516737472 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v142) ==== 80+0+0 (secure 0 0 0) 0x7f48400cdef0 con 0x7f48400b3f40 2024-09-10T10:05:17.427 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48200680a0 msgr2=0x7f482006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48200680a0 0x7f482006a560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4840166ea0 tx=0x7f482c008040 comp rx=0 tx=0).stop 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b3f40 msgr2=0x7f4840151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b3f40 0x7f4840151830 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f48300af040 tx=0x7f4830019740 comp rx=0 tx=0).stop 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 shutdown_connections 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f48200680a0 0x7f482006a560 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48400cff70 0x7f4840155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48400cfb90 0x7f4840164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 --2- 172.21.15.110:0/516737472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48400b3f40 0x7f4840151830 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 >> 172.21.15.110:0/516737472 conn(0x7f4840062d10 msgr2=0x7f4840072510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:17.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 shutdown_connections 2024-09-10T10:05:17.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.425+0000 7f4846600640 1 -- 172.21.15.110:0/516737472 wait complete. 2024-09-10T10:05:17.449 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:05:17.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:17.449 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:17.477 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:05:17.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:17.478 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:05:17.533 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:05:17.534 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:17.534 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:17.542 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:05:17.542 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:05:17.542 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:17.542 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:17.585 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:05:17.585 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:17.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 Processor -- start 2024-09-10T10:05:17.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- start start 2024-09-10T10:05:17.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 0x7f29f00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f00d37f0 con 0x7f29f00d30a0 2024-09-10T10:05:17.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f00d3ad0 con 0x7f29f006e350 2024-09-10T10:05:17.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f29f00d4400 con 0x7f29f0161bb0 2024-09-10T10:05:17.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f4e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f29f00d30a0 0x7f29f006dc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:60006/0 (socket says 172.21.15.110:60006) 2024-09-10T10:05:17.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f4e00640 1 -- 172.21.15.110:0/2046302840 learned_addr learned my addr 172.21.15.110:0/2046302840 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:17.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 0x7f29f00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197938366 0 0) 0x7f29f00d37f0 con 0x7f29f00d30a0 2024-09-10T10:05:17.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29d8003530 con 0x7f29f00d30a0 2024-09-10T10:05:17.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 784717563 0 0) 0x7f29d8003530 con 0x7f29f00d30a0 2024-09-10T10:05:17.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 msgr2=0x7f29f00cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 0x7f29f00cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 >> v1:172.21.15.145:6789/0 conn(0x7f29f006e350 legacy=0x7f29f00b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f00d3db0 con 0x7f29f00d30a0 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f29dc001680 con 0x7f29f00d30a0 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 0x7f29f00cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 >> v1:172.21.15.110:6789/0 conn(0x7f29f00d30a0 legacy=0x7f29f006dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29d80039f0 0x7f29d8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d8004810 0x7f29d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 0x7f29d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00d4400 con 0x7f29d80039f0 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f29d800bb10 con 0x7f29d8004810 2024-09-10T10:05:17.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f29d800bd10 con 0x7f29f0161bb0 2024-09-10T10:05:17.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29d80039f0 0x7f29d8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f4e00640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d8004810 0x7f29d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6200640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 0x7f29d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 -- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 msgr2=0x7f29d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:17.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f0161bb0 0x7f29d80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 -- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d8004810 msgr2=0x7f29d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:17.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d8004810 0x7f29d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 -- 172.21.15.110:0/2046302840 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d8003530 con 0x7f29d80039f0 2024-09-10T10:05:17.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f5800640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29d80039f0 0x7f29d8003e10 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009c80 tx=0x7f29e401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1302febc0760f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:17.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e40b2070 con 0x7f29d80039f0 2024-09-10T10:05:17.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e40a5840 con 0x7f29d80039f0 2024-09-10T10:05:17.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29efe00640 1 -- 172.21.15.110:0/2046302840 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4009880 con 0x7f29d80039f0 2024-09-10T10:05:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- 172.21.15.110:0/2046302840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29d80039f0 msgr2=0x7f29d8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 --2- 172.21.15.110:0/2046302840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29d80039f0 0x7f29d8003e10 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009c80 tx=0x7f29e401cb80 comp rx=0 tx=0).stop 2024-09-10T10:05:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6200640 1 -- 172.21.15.110:0/2046302840 reap_dead start 2024-09-10T10:05:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- 172.21.15.110:0/2046302840 shutdown_connections 2024-09-10T10:05:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- 172.21.15.110:0/2046302840 >> 172.21.15.110:0/2046302840 conn(0x7f29f00d4a50 msgr2=0x7f29f00d4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- 172.21.15.110:0/2046302840 shutdown_connections 2024-09-10T10:05:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.721+0000 7f29f6c00640 1 -- 172.21.15.110:0/2046302840 wait complete. 2024-09-10T10:05:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 Processor -- start 2024-09-10T10:05:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 -- start start 2024-09-10T10:05:17.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d80039f0 0x7f29f015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f01f5250 0x7f29f00b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f01f56d0 0x7f29f00b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0161f90 con 0x7f29f01f56d0 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00d34c0 con 0x7f29d80039f0 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f29f0162110 con 0x7f29f01f5250 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f01f56d0 0x7f29f00b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f01f56d0 0x7f29f00b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37908/0 (socket says 172.21.15.110:37908) 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 -- 172.21.15.110:0/247039041 learned_addr learned my addr 172.21.15.110:0/247039041 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f4e00640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f01f5250 0x7f29f00b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f5800640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d80039f0 0x7f29f015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 -- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f01f5250 msgr2=0x7f29f00b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f01f5250 0x7f29f00b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 -- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d80039f0 msgr2=0x7f29f015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d80039f0 0x7f29f015fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 -- 172.21.15.110:0/247039041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f00add70 con 0x7f29f01f56d0 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f5800640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d80039f0 0x7f29f015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6200640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f01f56d0 0x7f29f00b08b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f29f00cec20 tx=0x7f29e800e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29ed600640 1 -- 172.21.15.110:0/247039041 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8003b90 con 0x7f29f01f56d0 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29ed600640 1 -- 172.21.15.110:0/247039041 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e8018070 con 0x7f29f01f56d0 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29ed600640 1 -- 172.21.15.110:0/247039041 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80050d0 con 0x7f29f01f56d0 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f00ae030 con 0x7f29f01f56d0 2024-09-10T10:05:17.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.725+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f00ae5f0 con 0x7f29f01f56d0 2024-09-10T10:05:17.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.729+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f00cf850 con 0x7f29f01f56d0 2024-09-10T10:05:17.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.729+0000 7f29ed600640 1 -- 172.21.15.110:0/247039041 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f29e8005270 con 0x7f29f01f56d0 2024-09-10T10:05:17.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.729+0000 7f29ed600640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29d00680a0 0x7f29d006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:17.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.729+0000 7f29ed600640 1 -- 172.21.15.110:0/247039041 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f29e80842d0 con 0x7f29f01f56d0 2024-09-10T10:05:17.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.729+0000 7f29f5800640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29d00680a0 0x7f29d006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:17.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.729+0000 7f29f5800640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29d00680a0 0x7f29d006a560 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009c80 tx=0x7f29e40046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:17.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.733+0000 7f29ed600640 1 -- 172.21.15.110:0/247039041 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f29e8056700 con 0x7f29f01f56d0 2024-09-10T10:05:17.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.921+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29f00d4f00 con 0x7f29f01f56d0 2024-09-10T10:05:17.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.921+0000 7f29ed600640 1 -- 172.21.15.110:0/247039041 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5239 (secure 0 0 0) 0x7f29e8056000 con 0x7f29f01f56d0 2024-09-10T10:05:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:17.928 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":98,"btime":"2024-09-10T10:05:12:545008+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2050751760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2050751760},{"type":"v1","addr":"172.21.15.110:6835","nonce":2050751760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6743,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/4094794865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4094794865},{"type":"v1","addr":"172.21.15.145:6834","nonce":4094794865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2883879805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2883879805},{"type":"v1","addr":"172.21.15.110:6834","nonce":2883879805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:02.880808+0000","modified":"2024-09-10T10:05:05.895746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1325843861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1325843861},{"type":"v1","addr":"172.21.15.145:6835","nonce":1325843861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8178,"qdb_cluster":[8178]},"id":11}]} 2024-09-10T10:05:17.929 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 98 2024-09-10T10:05:17.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29d00680a0 msgr2=0x7f29d006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29d00680a0 0x7f29d006a560 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009c80 tx=0x7f29e40046d0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f01f56d0 msgr2=0x7f29f00b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:17.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f01f56d0 0x7f29f00b08b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f29f00cec20 tx=0x7f29e800e9e0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 shutdown_connections 2024-09-10T10:05:17.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f29d00680a0 0x7f29d006a560 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f01f56d0 0x7f29f00b08b0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f29f01f5250 0x7f29f00b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 --2- 172.21.15.110:0/247039041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29d80039f0 0x7f29f015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:17.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 >> 172.21.15.110:0/247039041 conn(0x7f29f00d4a50 msgr2=0x7f29f00b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:17.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 shutdown_connections 2024-09-10T10:05:17.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:17.925+0000 7f29f6c00640 1 -- 172.21.15.110:0/247039041 wait complete. 2024-09-10T10:05:17.950 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:18.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0fe00640 1 Processor -- start 2024-09-10T10:05:18.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0fe00640 1 -- start start 2024-09-10T10:05:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8d080ccb60 0x7f8d08072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0fe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08072670 0x7f8d08140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0fe00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d080ccf40 con 0x7f8d08140a60 2024-09-10T10:05:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0fe00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8d08072ad0 con 0x7f8d080ccb60 2024-09-10T10:05:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0fe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0815c130 con 0x7f8d08072670 2024-09-10T10:05:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0d400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8d080ccb60 0x7f8d08072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0d400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8d080ccb60 0x7f8d08072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37914/0 (socket says 172.21.15.110:37914) 2024-09-10T10:05:18.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0d400640 1 -- 172.21.15.110:0/3148014538 learned_addr learned my addr 172.21.15.110:0/3148014538 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:18.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0d400640 1 -- 172.21.15.110:0/3148014538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08072670 msgr2=0x7f8d08140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:18.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0ca00640 1 --2- 172.21.15.110:0/3148014538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08072670 0x7f8d08140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0d400640 1 --2- 172.21.15.110:0/3148014538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08072670 0x7f8d08140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0d400640 1 -- 172.21.15.110:0/3148014538 >> v1:172.21.15.145:6789/0 conn(0x7f8d08140a60 legacy=0x7f8d080710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0d400640 1 -- 172.21.15.110:0/3148014538 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d081412a0 con 0x7f8d080ccb60 2024-09-10T10:05:18.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.081+0000 7f8d0ca00640 1 --2- 172.21.15.110:0/3148014538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08072670 0x7f8d08140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:18.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0d400640 1 --2- 172.21.15.110:0/3148014538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d08072130 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009870 tx=0x7f8cfc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c18ff1c29f9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d07e00640 1 -- 172.21.15.110:0/3148014538 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0b1070 con 0x7f8d080ccb60 2024-09-10T10:05:18.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d07e00640 1 -- 172.21.15.110:0/3148014538 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc0ac630 con 0x7f8d080ccb60 2024-09-10T10:05:18.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d07e00640 1 -- 172.21.15.110:0/3148014538 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0a8050 con 0x7f8d080ccb60 2024-09-10T10:05:18.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- 172.21.15.110:0/3148014538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 msgr2=0x7f8d08072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/3148014538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d08072130 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009870 tx=0x7f8cfc01c4f0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- 172.21.15.110:0/3148014538 shutdown_connections 2024-09-10T10:05:18.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/3148014538 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08072670 0x7f8d08140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/3148014538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d08072130 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- 172.21.15.110:0/3148014538 >> 172.21.15.110:0/3148014538 conn(0x7f8d0806bc50 msgr2=0x7f8d0806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:18.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- 172.21.15.110:0/3148014538 shutdown_connections 2024-09-10T10:05:18.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- 172.21.15.110:0/3148014538 wait complete. 2024-09-10T10:05:18.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 Processor -- start 2024-09-10T10:05:18.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- start start 2024-09-10T10:05:18.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d08072670 0x7f8d080b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d080b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08140a60 0x7f8d080b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0815f520 con 0x7f8d080ccb60 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0815f000 con 0x7f8d08072670 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8d08159e10 con 0x7f8d08140a60 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d080b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d080b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37920/0 (socket says 172.21.15.110:37920) 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 -- 172.21.15.110:0/4044069478 learned_addr learned my addr 172.21.15.110:0/4044069478 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0de00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08140a60 0x7f8d080b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0d400640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d08072670 0x7f8d080b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 -- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08140a60 msgr2=0x7f8d080b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08140a60 0x7f8d080b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 -- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d08072670 msgr2=0x7f8d080b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d08072670 0x7f8d080b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 -- 172.21.15.110:0/4044069478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d080b0ed0 con 0x7f8d080ccb60 2024-09-10T10:05:18.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0ca00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d080b0360 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400c7f0 tx=0x7f8cf400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d05600640 1 -- 172.21.15.110:0/4044069478 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4007a30 con 0x7f8d080ccb60 2024-09-10T10:05:18.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d05600640 1 -- 172.21.15.110:0/4044069478 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf4007bd0 con 0x7f8d080ccb60 2024-09-10T10:05:18.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d05600640 1 -- 172.21.15.110:0/4044069478 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40113f0 con 0x7f8d080ccb60 2024-09-10T10:05:18.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d080b11b0 con 0x7f8d080ccb60 2024-09-10T10:05:18.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d080b1770 con 0x7f8d080ccb60 2024-09-10T10:05:18.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.089+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d08140ef0 con 0x7f8d080ccb60 2024-09-10T10:05:18.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.097+0000 7f8d05600640 1 -- 172.21.15.110:0/4044069478 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f8cf4011590 con 0x7f8d080ccb60 2024-09-10T10:05:18.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.097+0000 7f8d05600640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ce80680a0 0x7f8ce806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.097+0000 7f8d05600640 1 -- 172.21.15.110:0/4044069478 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f8cf4087210 con 0x7f8d080ccb60 2024-09-10T10:05:18.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.097+0000 7f8d0d400640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ce80680a0 0x7f8ce806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.097+0000 7f8d0d400640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ce80680a0 0x7f8ce806a560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0ae000 tx=0x7f8cfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.101+0000 7f8d05600640 1 -- 172.21.15.110:0/4044069478 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8cf4015020 con 0x7f8d080ccb60 2024-09-10T10:05:18.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.289+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d080b3a00 con 0x7f8d080ccb60 2024-09-10T10:05:18.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.289+0000 7f8d05600640 1 -- 172.21.15.110:0/4044069478 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5239 (secure 0 0 0) 0x7f8cf4056590 con 0x7f8d080ccb60 2024-09-10T10:05:18.292 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:18.292 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":98,"btime":"2024-09-10T10:05:12:545008+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2050751760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2050751760},{"type":"v1","addr":"172.21.15.110:6835","nonce":2050751760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6743,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/4094794865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4094794865},{"type":"v1","addr":"172.21.15.145:6834","nonce":4094794865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2883879805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2883879805},{"type":"v1","addr":"172.21.15.110:6834","nonce":2883879805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:02.880808+0000","modified":"2024-09-10T10:05:05.895746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1325843861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1325843861},{"type":"v1","addr":"172.21.15.145:6835","nonce":1325843861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8178,"qdb_cluster":[8178]},"id":11}]} 2024-09-10T10:05:18.293 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 98 2024-09-10T10:05:18.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ce80680a0 msgr2=0x7f8ce806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ce80680a0 0x7f8ce806a560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0ae000 tx=0x7f8cfc009040 comp rx=0 tx=0).stop 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 msgr2=0x7f8d080b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d080b0360 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400c7f0 tx=0x7f8cf400ccb0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 shutdown_connections 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8ce80680a0 0x7f8ce806a560 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8d08140a60 0x7f8d080b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d080ccb60 0x7f8d080b0360 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 --2- 172.21.15.110:0/4044069478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d08072670 0x7f8d080b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 >> 172.21.15.110:0/4044069478 conn(0x7f8d0806bc50 msgr2=0x7f8d08153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 shutdown_connections 2024-09-10T10:05:18.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.293+0000 7f8d0fe00640 1 -- 172.21.15.110:0/4044069478 wait complete. 2024-09-10T10:05:18.314 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:18.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff546200640 1 Processor -- start 2024-09-10T10:05:18.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff546200640 1 -- start start 2024-09-10T10:05:18.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff546200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff540145b00 0x7ff5400cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff546200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5400b5690 0x7ff5400b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff546200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff540071db0 con 0x7ff54015c940 2024-09-10T10:05:18.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff546200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff5400b6010 con 0x7ff540145b00 2024-09-10T10:05:18.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff546200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5400ce5f0 con 0x7ff5400b5690 2024-09-10T10:05:18.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff544e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff540145b00 0x7ff5400cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff544e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff540145b00 0x7ff5400cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:42852/0 (socket says 172.21.15.110:42852) 2024-09-10T10:05:18.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff544e00640 1 -- 172.21.15.110:0/1781948492 learned_addr learned my addr 172.21.15.110:0/1781948492 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:18.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff53fe00640 1 --2- 172.21.15.110:0/1781948492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5400b5690 0x7ff5400b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff544e00640 1 -- 172.21.15.110:0/1781948492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5400b5690 msgr2=0x7ff5400b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:18.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff544e00640 1 --2- 172.21.15.110:0/1781948492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5400b5690 0x7ff5400b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff544e00640 1 -- 172.21.15.110:0/1781948492 >> v1:172.21.15.145:6789/0 conn(0x7ff54015c940 legacy=0x7ff54015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.449+0000 7ff544e00640 1 -- 172.21.15.110:0/1781948492 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5400ce8d0 con 0x7ff540145b00 2024-09-10T10:05:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff544e00640 1 --2- 172.21.15.110:0/1781948492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff540145b00 0x7ff5400cf0d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff530009e50 tx=0x7ff53001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a796c7aa99e647e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff53f400640 1 -- 172.21.15.110:0/1781948492 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5300a5700 con 0x7ff540145b00 2024-09-10T10:05:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff53f400640 1 -- 172.21.15.110:0/1781948492 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5300a58a0 con 0x7ff540145b00 2024-09-10T10:05:18.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff53f400640 1 -- 172.21.15.110:0/1781948492 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5300a5c70 con 0x7ff540145b00 2024-09-10T10:05:18.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff546200640 1 -- 172.21.15.110:0/1781948492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff540145b00 msgr2=0x7ff5400cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff546200640 1 --2- 172.21.15.110:0/1781948492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff540145b00 0x7ff5400cf0d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff530009e50 tx=0x7ff53001ca30 comp rx=0 tx=0).stop 2024-09-10T10:05:18.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff546200640 1 -- 172.21.15.110:0/1781948492 shutdown_connections 2024-09-10T10:05:18.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff546200640 1 --2- 172.21.15.110:0/1781948492 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5400b5690 0x7ff5400b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff546200640 1 --2- 172.21.15.110:0/1781948492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff540145b00 0x7ff5400cf0d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff546200640 1 -- 172.21.15.110:0/1781948492 >> 172.21.15.110:0/1781948492 conn(0x7ff54006d0f0 msgr2=0x7ff54006d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:18.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff546200640 1 -- 172.21.15.110:0/1781948492 shutdown_connections 2024-09-10T10:05:18.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.453+0000 7ff546200640 1 -- 172.21.15.110:0/1781948492 wait complete. 2024-09-10T10:05:18.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 Processor -- start 2024-09-10T10:05:18.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 -- start start 2024-09-10T10:05:18.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5400b5690 0x7ff54015b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff540145b00 0x7ff5400cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54015c940 0x7ff5400af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff540071850 con 0x7ff540145b00 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5400719d0 con 0x7ff5400b5690 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff540071b50 con 0x7ff54015c940 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff540145b00 0x7ff5400cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff540145b00 0x7ff5400cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50204/0 (socket says 172.21.15.110:50204) 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 -- 172.21.15.110:0/4078032059 learned_addr learned my addr 172.21.15.110:0/4078032059 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff544e00640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5400b5690 0x7ff54015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff545800640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54015c940 0x7ff5400af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 -- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54015c940 msgr2=0x7ff5400af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54015c940 0x7ff5400af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 -- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5400b5690 msgr2=0x7ff54015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5400b5690 0x7ff54015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 -- 172.21.15.110:0/4078032059 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5400b4da0 con 0x7ff540145b00 2024-09-10T10:05:18.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53fe00640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff540145b00 0x7ff5400cd730 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00d800 tx=0x7ff52c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53cc00640 1 -- 172.21.15.110:0/4078032059 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c00b6c0 con 0x7ff540145b00 2024-09-10T10:05:18.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53cc00640 1 -- 172.21.15.110:0/4078032059 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff52c019070 con 0x7ff540145b00 2024-09-10T10:05:18.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5400b5080 con 0x7ff540145b00 2024-09-10T10:05:18.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff53cc00640 1 -- 172.21.15.110:0/4078032059 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c00ba60 con 0x7ff540145b00 2024-09-10T10:05:18.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.457+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54015d5f0 con 0x7ff540145b00 2024-09-10T10:05:18.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.461+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5400b5ad0 con 0x7ff540145b00 2024-09-10T10:05:18.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.461+0000 7ff53cc00640 1 -- 172.21.15.110:0/4078032059 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7ff52c00bc00 con 0x7ff540145b00 2024-09-10T10:05:18.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.461+0000 7ff53cc00640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5200680a0 0x7ff52006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.461+0000 7ff53cc00640 1 -- 172.21.15.110:0/4078032059 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7ff52c087370 con 0x7ff540145b00 2024-09-10T10:05:18.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.461+0000 7ff544e00640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5200680a0 0x7ff52006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.461+0000 7ff544e00640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5200680a0 0x7ff52006a560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff5300ae000 tx=0x7ff5300a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.465+0000 7ff53cc00640 1 -- 172.21.15.110:0/4078032059 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff52c056800 con 0x7ff540145b00 2024-09-10T10:05:18.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.629+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5400b0d30 con 0x7ff540145b00 2024-09-10T10:05:18.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.629+0000 7ff53cc00640 1 -- 172.21.15.110:0/4078032059 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16856 (secure 0 0 0) 0x7ff52c05ab00 con 0x7ff540145b00 2024-09-10T10:05:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:18.632 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":142,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:04.492566+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:00.607363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:01.886645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:18.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5200680a0 msgr2=0x7ff52006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5200680a0 0x7ff52006a560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff5300ae000 tx=0x7ff5300a7070 comp rx=0 tx=0).stop 2024-09-10T10:05:18.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff540145b00 msgr2=0x7ff5400cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff540145b00 0x7ff5400cd730 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00d800 tx=0x7ff52c00dcc0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 shutdown_connections 2024-09-10T10:05:18.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5200680a0 0x7ff52006a560 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff54015c940 0x7ff5400af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff540145b00 0x7ff5400cd730 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 --2- 172.21.15.110:0/4078032059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5400b5690 0x7ff54015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 >> 172.21.15.110:0/4078032059 conn(0x7ff54006d0f0 msgr2=0x7ff540143020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:18.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 shutdown_connections 2024-09-10T10:05:18.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.633+0000 7ff546200640 1 -- 172.21.15.110:0/4078032059 wait complete. 2024-09-10T10:05:18.653 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:05:18.653 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:18.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7d2600640 1 Processor -- start 2024-09-10T10:05:18.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7d2600640 1 -- start start 2024-09-10T10:05:18.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7d2600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7cc15c960 0x7fe7cc0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7d2600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc0d13e0 0x7fe7cc070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7d2600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7cc0706a0 con 0x7fe7cc1603c0 2024-09-10T10:05:18.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7d2600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe7cc15cd40 con 0x7fe7cc15c960 2024-09-10T10:05:18.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7d2600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc0b6370 con 0x7fe7cc0d13e0 2024-09-10T10:05:18.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7cb400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7cc15c960 0x7fe7cc0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7cb400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7cc15c960 0x7fe7cc0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:42864/0 (socket says 172.21.15.110:42864) 2024-09-10T10:05:18.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7cb400640 1 -- 172.21.15.110:0/952430430 learned_addr learned my addr 172.21.15.110:0/952430430 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:18.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7cbe00640 1 --2- 172.21.15.110:0/952430430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc0d13e0 0x7fe7cc070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7cb400640 1 -- 172.21.15.110:0/952430430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc0d13e0 msgr2=0x7fe7cc070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:18.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7cb400640 1 --2- 172.21.15.110:0/952430430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc0d13e0 0x7fe7cc070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7cb400640 1 -- 172.21.15.110:0/952430430 >> v1:172.21.15.145:6789/0 conn(0x7fe7cc1603c0 legacy=0x7fe7cc0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.789+0000 7fe7cb400640 1 -- 172.21.15.110:0/952430430 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc070a80 con 0x7fe7cc15c960 2024-09-10T10:05:18.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7cb400640 1 --2- 172.21.15.110:0/952430430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c960 0x7fe7cc0b5e30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0b64f0 tx=0x7fe7bc01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360f91c51818acb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7ca000640 1 -- 172.21.15.110:0/952430430 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0ab5f0 con 0x7fe7cc15c960 2024-09-10T10:05:18.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7ca000640 1 -- 172.21.15.110:0/952430430 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7bc0b0070 con 0x7fe7cc15c960 2024-09-10T10:05:18.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7ca000640 1 -- 172.21.15.110:0/952430430 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0ab9b0 con 0x7fe7cc15c960 2024-09-10T10:05:18.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7d2600640 1 -- 172.21.15.110:0/952430430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c960 msgr2=0x7fe7cc0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7d2600640 1 --2- 172.21.15.110:0/952430430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c960 0x7fe7cc0b5e30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0b64f0 tx=0x7fe7bc01c3f0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7d2600640 1 -- 172.21.15.110:0/952430430 shutdown_connections 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7d2600640 1 --2- 172.21.15.110:0/952430430 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc0d13e0 0x7fe7cc070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7d2600640 1 --2- 172.21.15.110:0/952430430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc15c960 0x7fe7cc0b5e30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7d2600640 1 -- 172.21.15.110:0/952430430 >> 172.21.15.110:0/952430430 conn(0x7fe7cc0d25c0 msgr2=0x7fe7cc0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7d2600640 1 -- 172.21.15.110:0/952430430 shutdown_connections 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.793+0000 7fe7d2600640 1 -- 172.21.15.110:0/952430430 wait complete. 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 Processor -- start 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 -- start start 2024-09-10T10:05:18.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc0d13e0 0x7fe7cc154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc15c960 0x7fe7cc14b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc1603c0 0x7fe7cc15b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc0d3150 con 0x7fe7cc1603c0 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1611b0 con 0x7fe7cc0d13e0 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc0d32d0 con 0x7fe7cc15c960 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc1603c0 0x7fe7cc15b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc1603c0 0x7fe7cc15b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50218/0 (socket says 172.21.15.110:50218) 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 -- 172.21.15.110:0/3349308055 learned_addr learned my addr 172.21.15.110:0/3349308055 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cb400640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc0d13e0 0x7fe7cc154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7caa00640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc15c960 0x7fe7cc14b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 -- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc15c960 msgr2=0x7fe7cc14b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc15c960 0x7fe7cc14b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 -- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc0d13e0 msgr2=0x7fe7cc154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc0d13e0 0x7fe7cc154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 -- 172.21.15.110:0/3349308055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc15e200 con 0x7fe7cc1603c0 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cb400640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc0d13e0 0x7fe7cc154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7cbe00640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc1603c0 0x7fe7cc15b4e0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009dd0 tx=0x7fe7c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7c3400640 1 -- 172.21.15.110:0/3349308055 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4018020 con 0x7fe7cc1603c0 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7c3400640 1 -- 172.21.15.110:0/3349308055 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c4014480 con 0x7fe7cc1603c0 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc152f30 con 0x7fe7cc1603c0 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7c3400640 1 -- 172.21.15.110:0/3349308055 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4014870 con 0x7fe7cc1603c0 2024-09-10T10:05:18.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc1534f0 con 0x7fe7cc1603c0 2024-09-10T10:05:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.797+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc070f60 con 0x7fe7cc1603c0 2024-09-10T10:05:18.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.801+0000 7fe7c3400640 1 -- 172.21.15.110:0/3349308055 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fe7c401b020 con 0x7fe7cc1603c0 2024-09-10T10:05:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.801+0000 7fe7c3400640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac0680a0 0x7fe7ac06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.801+0000 7fe7c3400640 1 -- 172.21.15.110:0/3349308055 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7fe7c40876b0 con 0x7fe7cc1603c0 2024-09-10T10:05:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.801+0000 7fe7cb400640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac0680a0 0x7fe7ac06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.801+0000 7fe7cb400640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac0680a0 0x7fe7ac06a560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0af000 tx=0x7fe7bc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.805+0000 7fe7c3400640 1 -- 172.21.15.110:0/3349308055 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe7c4022070 con 0x7fe7cc1603c0 2024-09-10T10:05:18.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.993+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7cc0d34d0 con 0x7fe7cc1603c0 2024-09-10T10:05:18.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.993+0000 7fe7c3400640 1 -- 172.21.15.110:0/3349308055 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5239 (secure 0 0 0) 0x7fe7c4010040 con 0x7fe7cc1603c0 2024-09-10T10:05:18.996 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:18.996 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":98,"btime":"2024-09-10T10:05:12:545008+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2050751760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2050751760},{"type":"v1","addr":"172.21.15.110:6835","nonce":2050751760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6743,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/4094794865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4094794865},{"type":"v1","addr":"172.21.15.145:6834","nonce":4094794865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2883879805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2883879805},{"type":"v1","addr":"172.21.15.110:6834","nonce":2883879805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:02.880808+0000","modified":"2024-09-10T10:05:05.895746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1325843861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1325843861},{"type":"v1","addr":"172.21.15.145:6835","nonce":1325843861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8178,"qdb_cluster":[8178]},"id":11}]} 2024-09-10T10:05:18.997 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 98 2024-09-10T10:05:18.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac0680a0 msgr2=0x7fe7ac06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac0680a0 0x7fe7ac06a560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0af000 tx=0x7fe7bc0a7000 comp rx=0 tx=0).stop 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc1603c0 msgr2=0x7fe7cc15b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc1603c0 0x7fe7cc15b4e0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009dd0 tx=0x7fe7c400c490 comp rx=0 tx=0).stop 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 shutdown_connections 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe7ac0680a0 0x7fe7ac06a560 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc1603c0 0x7fe7cc15b4e0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7cc15c960 0x7fe7cc14b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 --2- 172.21.15.110:0/3349308055 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7cc0d13e0 0x7fe7cc154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 >> 172.21.15.110:0/3349308055 conn(0x7fe7cc0d25c0 msgr2=0x7fe7cc166a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 shutdown_connections 2024-09-10T10:05:18.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:18.997+0000 7fe7d2600640 1 -- 172.21.15.110:0/3349308055 wait complete. 2024-09-10T10:05:19.017 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:19.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4aa00640 1 Processor -- start 2024-09-10T10:05:19.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4aa00640 1 -- start start 2024-09-10T10:05:19.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4aa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3f4415e620 0x7f3f440cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4aa00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4406d060 con 0x7f3f440b54f0 2024-09-10T10:05:19.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4aa00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3f440d1e90 con 0x7f3f4415e620 2024-09-10T10:05:19.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4aa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4406ce10 con 0x7f3f440b6710 2024-09-10T10:05:19.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f48c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3f440b54f0 0x7f3f440b6000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55912/0 (socket says 172.21.15.110:55912) 2024-09-10T10:05:19.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f48c00640 1 -- 172.21.15.110:0/2773916421 learned_addr learned my addr 172.21.15.110:0/2773916421 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:19.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f49600640 1 --2- 172.21.15.110:0/2773916421 >> v2:172.21.15.145:3300/0 conn(0x7f3f4415e620 0x7f3f440cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4a000640 1 --2- 172.21.15.110:0/2773916421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f43e00640 1 -- 172.21.15.110:0/2773916421 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552651452 0 0) 0x7f3f4406d060 con 0x7f3f440b54f0 2024-09-10T10:05:19.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f43e00640 1 -- 172.21.15.110:0/2773916421 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f2c003530 con 0x7f3f440b54f0 2024-09-10T10:05:19.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4a000640 1 -- 172.21.15.110:0/2773916421 >> v2:172.21.15.145:3300/0 conn(0x7f3f4415e620 msgr2=0x7f3f440cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:19.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4a000640 1 --2- 172.21.15.110:0/2773916421 >> v2:172.21.15.145:3300/0 conn(0x7f3f4415e620 0x7f3f440cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4a000640 1 -- 172.21.15.110:0/2773916421 >> v1:172.21.15.110:6789/0 conn(0x7f3f440b54f0 legacy=0x7f3f440b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f4a000640 1 -- 172.21.15.110:0/2773916421 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4406d340 con 0x7f3f440b6710 2024-09-10T10:05:19.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.153+0000 7f3f49600640 1 --2- 172.21.15.110:0/2773916421 >> v2:172.21.15.145:3300/0 conn(0x7f3f4415e620 0x7f3f440cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:19.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4a000640 1 --2- 172.21.15.110:0/2773916421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4406e100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009d90 tx=0x7f3f3c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc65b3410022e23a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f43e00640 1 -- 172.21.15.110:0/2773916421 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0b2070 con 0x7f3f440b6710 2024-09-10T10:05:19.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f43e00640 1 -- 172.21.15.110:0/2773916421 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0a57a0 con 0x7f3f440b6710 2024-09-10T10:05:19.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f43e00640 1 -- 172.21.15.110:0/2773916421 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0a5b50 con 0x7f3f440b6710 2024-09-10T10:05:19.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4aa00640 1 -- 172.21.15.110:0/2773916421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 msgr2=0x7f3f4406e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/2773916421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4406e100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009d90 tx=0x7f3f3c01c8e0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4aa00640 1 -- 172.21.15.110:0/2773916421 shutdown_connections 2024-09-10T10:05:19.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/2773916421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4406e100 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/2773916421 >> v2:172.21.15.145:3300/0 conn(0x7f3f4415e620 0x7f3f440cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4aa00640 1 -- 172.21.15.110:0/2773916421 >> 172.21.15.110:0/2773916421 conn(0x7f3f4406f440 msgr2=0x7f3f4406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:19.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4aa00640 1 -- 172.21.15.110:0/2773916421 shutdown_connections 2024-09-10T10:05:19.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.157+0000 7f3f4aa00640 1 -- 172.21.15.110:0/2773916421 wait complete. 2024-09-10T10:05:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 Processor -- start 2024-09-10T10:05:19.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 -- start start 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f440b54f0 0x7f3f440b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f4415e620 0x7f3f44156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4416b470 con 0x7f3f440b54f0 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4416b2f0 con 0x7f3f4415e620 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4416b5f0 con 0x7f3f440b6710 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f440b54f0 0x7f3f440b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f440b54f0 0x7f3f440b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50220/0 (socket says 172.21.15.110:50220) 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 -- 172.21.15.110:0/3169138717 learned_addr learned my addr 172.21.15.110:0/3169138717 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:19.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4a000640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f4415e620 0x7f3f44156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f48c00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 -- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 msgr2=0x7f3f4415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 -- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f4415e620 msgr2=0x7f3f44156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f4415e620 0x7f3f44156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 -- 172.21.15.110:0/3169138717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f44159830 con 0x7f3f440b54f0 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f49600640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f440b54f0 0x7f3f440b2a80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800ebe0 tx=0x7f3f3800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f41600640 1 -- 172.21.15.110:0/3169138717 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38017070 con 0x7f3f440b54f0 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f41600640 1 -- 172.21.15.110:0/3169138717 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3800cc30 con 0x7f3f440b54f0 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f44159b10 con 0x7f3f440b54f0 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f41600640 1 -- 172.21.15.110:0/3169138717 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38013470 con 0x7f3f440b54f0 2024-09-10T10:05:19.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4415a0d0 con 0x7f3f440b54f0 2024-09-10T10:05:19.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.161+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f44071210 con 0x7f3f440b54f0 2024-09-10T10:05:19.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.165+0000 7f3f41600640 1 -- 172.21.15.110:0/3169138717 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f3f3801a070 con 0x7f3f440b54f0 2024-09-10T10:05:19.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.165+0000 7f3f41600640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f240680a0 0x7f3f2406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.165+0000 7f3f41600640 1 -- 172.21.15.110:0/3169138717 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f3f380871a0 con 0x7f3f440b54f0 2024-09-10T10:05:19.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.165+0000 7f3f48c00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f240680a0 0x7f3f2406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.165+0000 7f3f48c00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f240680a0 0x7f3f2406a560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f4415aca0 tx=0x7f3f30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.169+0000 7f3f41600640 1 -- 172.21.15.110:0/3169138717 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3f380566c0 con 0x7f3f440b54f0 2024-09-10T10:05:19.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.357+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f4415a4b0 con 0x7f3f440b54f0 2024-09-10T10:05:19.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.357+0000 7f3f41600640 1 -- 172.21.15.110:0/3169138717 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5239 (secure 0 0 0) 0x7f3f3805a9c0 con 0x7f3f440b54f0 2024-09-10T10:05:19.360 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:19.360 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":98,"btime":"2024-09-10T10:05:12:545008+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2050751760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2050751760},{"type":"v1","addr":"172.21.15.110:6835","nonce":2050751760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6743,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/4094794865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4094794865},{"type":"v1","addr":"172.21.15.145:6834","nonce":4094794865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2883879805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2883879805},{"type":"v1","addr":"172.21.15.110:6834","nonce":2883879805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:02.880808+0000","modified":"2024-09-10T10:05:05.895746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1325843861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1325843861},{"type":"v1","addr":"172.21.15.145:6835","nonce":1325843861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8178,"qdb_cluster":[8178]},"id":11}]} 2024-09-10T10:05:19.361 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 98 2024-09-10T10:05:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f240680a0 msgr2=0x7f3f2406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f240680a0 0x7f3f2406a560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f4415aca0 tx=0x7f3f30009040 comp rx=0 tx=0).stop 2024-09-10T10:05:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f440b54f0 msgr2=0x7f3f440b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f440b54f0 0x7f3f440b2a80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800ebe0 tx=0x7f3f3800c520 comp rx=0 tx=0).stop 2024-09-10T10:05:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 shutdown_connections 2024-09-10T10:05:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f240680a0 0x7f3f2406a560 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f4415e620 0x7f3f44156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f440b6710 0x7f3f4415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 --2- 172.21.15.110:0/3169138717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f440b54f0 0x7f3f440b2a80 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 >> 172.21.15.110:0/3169138717 conn(0x7f3f4406f440 msgr2=0x7f3f440b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 shutdown_connections 2024-09-10T10:05:19.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.361+0000 7f3f4aa00640 1 -- 172.21.15.110:0/3169138717 wait complete. 2024-09-10T10:05:19.381 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:19.422 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:05:19.431 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:05:19.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76d1000640 1 Processor -- start 2024-09-10T10:05:19.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76d1000640 1 -- start start 2024-09-10T10:05:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76d1000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f76cc0b32b0 0x7f76cc0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76d1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f76cc06e140 0x7f76cc06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76d1000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76cc155b60 con 0x7f76cc15b8b0 2024-09-10T10:05:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76d1000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f76cc0b3bd0 con 0x7f76cc06e140 2024-09-10T10:05:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76d1000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f76cc15c600 con 0x7f76cc0b32b0 2024-09-10T10:05:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cbe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f76cc06e140 0x7f76cc06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cbe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f76cc06e140 0x7f76cc06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50234/0 (socket says 172.21.15.110:50234) 2024-09-10T10:05:19.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cbe00640 1 -- 172.21.15.110:0/570219905 learned_addr learned my addr 172.21.15.110:0/570219905 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cb400640 1 --2- 172.21.15.110:0/570219905 >> v2:172.21.15.145:3300/0 conn(0x7f76cc0b32b0 0x7f76cc0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cbe00640 1 -- 172.21.15.110:0/570219905 >> v2:172.21.15.145:3300/0 conn(0x7f76cc0b32b0 msgr2=0x7f76cc0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cbe00640 1 --2- 172.21.15.110:0/570219905 >> v2:172.21.15.145:3300/0 conn(0x7f76cc0b32b0 0x7f76cc0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cbe00640 1 -- 172.21.15.110:0/570219905 >> v1:172.21.15.145:6789/0 conn(0x7f76cc15b8b0 legacy=0x7f76cc15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cbe00640 1 -- 172.21.15.110:0/570219905 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc0b4660 con 0x7f76cc06e140 2024-09-10T10:05:19.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.521+0000 7f76cb400640 1 --2- 172.21.15.110:0/570219905 >> v2:172.21.15.145:3300/0 conn(0x7f76cc0b32b0 0x7f76cc0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:19.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76cbe00640 1 --2- 172.21.15.110:0/570219905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc06e140 0x7f76cc06e5d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0733b0 tx=0x7f76c401c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3161b7a9473b8973 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76ca000640 1 -- 172.21.15.110:0/570219905 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40a5580 con 0x7f76cc06e140 2024-09-10T10:05:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76ca000640 1 -- 172.21.15.110:0/570219905 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c40a5720 con 0x7f76cc06e140 2024-09-10T10:05:19.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76ca000640 1 -- 172.21.15.110:0/570219905 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40a5af0 con 0x7f76cc06e140 2024-09-10T10:05:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76d1000640 1 -- 172.21.15.110:0/570219905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc06e140 msgr2=0x7f76cc06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76d1000640 1 --2- 172.21.15.110:0/570219905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc06e140 0x7f76cc06e5d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0733b0 tx=0x7f76c401c7d0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76d1000640 1 -- 172.21.15.110:0/570219905 shutdown_connections 2024-09-10T10:05:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76d1000640 1 --2- 172.21.15.110:0/570219905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc06e140 0x7f76cc06e5d0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76d1000640 1 --2- 172.21.15.110:0/570219905 >> v2:172.21.15.145:3300/0 conn(0x7f76cc0b32b0 0x7f76cc0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76d1000640 1 -- 172.21.15.110:0/570219905 >> 172.21.15.110:0/570219905 conn(0x7f76cc06cd60 msgr2=0x7f76cc06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76d1000640 1 -- 172.21.15.110:0/570219905 shutdown_connections 2024-09-10T10:05:19.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.525+0000 7f76d1000640 1 -- 172.21.15.110:0/570219905 wait complete. 2024-09-10T10:05:19.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 Processor -- start 2024-09-10T10:05:19.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 -- start start 2024-09-10T10:05:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76cc06e140 0x7f76cc155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76cc0b32b0 0x7f76cc143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc15b8b0 0x7f76cc153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc0b55a0 con 0x7f76cc15b8b0 2024-09-10T10:05:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc0b5420 con 0x7f76cc06e140 2024-09-10T10:05:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f76cc0b62e0 con 0x7f76cc0b32b0 2024-09-10T10:05:19.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc15b8b0 0x7f76cc153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc15b8b0 0x7f76cc153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50250/0 (socket says 172.21.15.110:50250) 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 -- 172.21.15.110:0/2709092781 learned_addr learned my addr 172.21.15.110:0/2709092781 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cb400640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76cc06e140 0x7f76cc155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76caa00640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76cc0b32b0 0x7f76cc143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 -- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76cc0b32b0 msgr2=0x7f76cc143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76cc0b32b0 0x7f76cc143f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 -- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76cc06e140 msgr2=0x7f76cc155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76cc06e140 0x7f76cc155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 -- 172.21.15.110:0/2709092781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc15b5e0 con 0x7f76cc15b8b0 2024-09-10T10:05:19.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76cbe00640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc15b8b0 0x7f76cc153370 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f76c401ae80 tx=0x7f76c4019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76c3400640 1 -- 172.21.15.110:0/2709092781 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4009510 con 0x7f76cc15b8b0 2024-09-10T10:05:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76c3400640 1 -- 172.21.15.110:0/2709092781 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c40be070 con 0x7f76cc15b8b0 2024-09-10T10:05:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76c3400640 1 -- 172.21.15.110:0/2709092781 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40b88b0 con 0x7f76cc15b8b0 2024-09-10T10:05:19.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76cc1527b0 con 0x7f76cc15b8b0 2024-09-10T10:05:19.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.529+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76cc158750 con 0x7f76cc15b8b0 2024-09-10T10:05:19.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.533+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76cc06ef20 con 0x7f76cc15b8b0 2024-09-10T10:05:19.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.533+0000 7f76c3400640 1 -- 172.21.15.110:0/2709092781 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f76c40b8a50 con 0x7f76cc15b8b0 2024-09-10T10:05:19.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.533+0000 7f76c3400640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f76ac0680a0 0x7f76ac06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.533+0000 7f76c3400640 1 -- 172.21.15.110:0/2709092781 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f76c411f580 con 0x7f76cc15b8b0 2024-09-10T10:05:19.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.533+0000 7f76cb400640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f76ac0680a0 0x7f76ac06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.533+0000 7f76cb400640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f76ac0680a0 0x7f76ac06a560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00ab60 tx=0x7f76bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.537+0000 7f76c3400640 1 -- 172.21.15.110:0/2709092781 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f76cc155b60 con 0x7f76cc15b8b0 2024-09-10T10:05:19.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.697+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76cc15bdb0 con 0x7f76cc15b8b0 2024-09-10T10:05:19.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76c3400640 1 -- 172.21.15.110:0/2709092781 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16856 (secure 0 0 0) 0x7f76c40f1540 con 0x7f76cc15b8b0 2024-09-10T10:05:19.701 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:19.701 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":142,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:04.492566+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:00.607363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:01.886645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:19.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f76ac0680a0 msgr2=0x7f76ac06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f76ac0680a0 0x7f76ac06a560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00ab60 tx=0x7f76bc008040 comp rx=0 tx=0).stop 2024-09-10T10:05:19.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc15b8b0 msgr2=0x7f76cc153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc15b8b0 0x7f76cc153370 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f76c401ae80 tx=0x7f76c4019570 comp rx=0 tx=0).stop 2024-09-10T10:05:19.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 shutdown_connections 2024-09-10T10:05:19.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f76ac0680a0 0x7f76ac06a560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76cc15b8b0 0x7f76cc153370 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f76cc0b32b0 0x7f76cc143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 --2- 172.21.15.110:0/2709092781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76cc06e140 0x7f76cc155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 >> 172.21.15.110:0/2709092781 conn(0x7f76cc06cd60 msgr2=0x7f76cc0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:19.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 shutdown_connections 2024-09-10T10:05:19.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.701+0000 7f76d1000640 1 -- 172.21.15.110:0/2709092781 wait complete. 2024-09-10T10:05:19.723 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:05:19.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.897+0000 7f03a8200640 1 Processor -- start 2024-09-10T10:05:19.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.897+0000 7f03a8200640 1 -- start start 2024-09-10T10:05:19.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a8200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a00b3f40 0x7f03a00cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a8200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a015d4d0 con 0x7f03a00cff70 2024-09-10T10:05:19.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a8200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a015d7b0 con 0x7f03a00cfb90 2024-09-10T10:05:19.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a8200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f03a006d130 con 0x7f03a00b3f40 2024-09-10T10:05:19.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a6200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f03a00cff70 0x7f03a006db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55914/0 (socket says 172.21.15.110:55914) 2024-09-10T10:05:19.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a6200640 1 -- 172.21.15.110:0/3051874337 learned_addr learned my addr 172.21.15.110:0/3051874337 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:19.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a4e00640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a00b3f40 0x7f03a00cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 173457248 0 0) 0x7f03a015d4d0 con 0x7f03a00cff70 2024-09-10T10:05:19.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0388003530 con 0x7f03a00cff70 2024-09-10T10:05:19.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 933248161 0 0) 0x7f0388003530 con 0x7f03a00cff70 2024-09-10T10:05:19.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a00b3f40 msgr2=0x7f03a00cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a00b3f40 0x7f03a00cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 >> v1:172.21.15.145:6789/0 conn(0x7f03a00cfb90 legacy=0x7f03a015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a006d380 con 0x7f03a00cff70 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f0398001680 con 0x7f03a00cff70 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 >> v1:172.21.15.110:6789/0 conn(0x7f03a00cff70 legacy=0x7f03a006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03880039f0 0x7f0388003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0388004810 0x7f0388004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a00b3f40 0x7f0388005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03a006d130 con 0x7f03880039f0 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f038800bb00 con 0x7f0388004810 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f038800bd00 con 0x7f03a00b3f40 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a5800640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03880039f0 0x7f0388003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a6200640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a00b3f40 0x7f0388005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a4e00640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0388004810 0x7f0388004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a5800640 1 -- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a00b3f40 msgr2=0x7f0388005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:19.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a5800640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a00b3f40 0x7f0388005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a5800640 1 -- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0388004810 msgr2=0x7f0388004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:19.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a5800640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0388004810 0x7f0388004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.901+0000 7f03a5800640 1 -- 172.21.15.110:0/3051874337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0388003530 con 0x7f03880039f0 2024-09-10T10:05:19.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f03a5800640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03880039f0 0x7f0388003e10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0394001e70 tx=0x7f039401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5f0a3d8bb7d299 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940af070 con 0x7f03880039f0 2024-09-10T10:05:19.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0394003b80 con 0x7f03880039f0 2024-09-10T10:05:19.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f039fe00640 1 -- 172.21.15.110:0/3051874337 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394001a90 con 0x7f03880039f0 2024-09-10T10:05:19.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f03a8200640 1 -- 172.21.15.110:0/3051874337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03880039f0 msgr2=0x7f0388003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f03a8200640 1 --2- 172.21.15.110:0/3051874337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03880039f0 0x7f0388003e10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0394001e70 tx=0x7f039401c780 comp rx=0 tx=0).stop 2024-09-10T10:05:19.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f03a6200640 1 -- 172.21.15.110:0/3051874337 reap_dead start 2024-09-10T10:05:19.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f03a8200640 1 -- 172.21.15.110:0/3051874337 shutdown_connections 2024-09-10T10:05:19.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f03a8200640 1 -- 172.21.15.110:0/3051874337 >> 172.21.15.110:0/3051874337 conn(0x7f03a0062d10 msgr2=0x7f03a006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:19.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f03a8200640 1 -- 172.21.15.110:0/3051874337 shutdown_connections 2024-09-10T10:05:19.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.905+0000 7f03a8200640 1 -- 172.21.15.110:0/3051874337 wait complete. 2024-09-10T10:05:19.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 Processor -- start 2024-09-10T10:05:19.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 -- start start 2024-09-10T10:05:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03880039f0 0x7f03a01f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a014d9b0 0x7f03a014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0160fa0 0x7f03a0161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03a00cd930 con 0x7f03a0160fa0 2024-09-10T10:05:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03a00cd7b0 con 0x7f03880039f0 2024-09-10T10:05:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f03a00cdab0 con 0x7f03a014d9b0 2024-09-10T10:05:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0160fa0 0x7f03a0161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0160fa0 0x7f03a0161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50262/0 (socket says 172.21.15.110:50262) 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 -- 172.21.15.110:0/850360299 learned_addr learned my addr 172.21.15.110:0/850360299 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a5800640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03880039f0 0x7f03a01f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a4e00640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a014d9b0 0x7f03a014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 -- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a014d9b0 msgr2=0x7f03a014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a014d9b0 0x7f03a014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 -- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03880039f0 msgr2=0x7f03a01f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03880039f0 0x7f03a01f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 -- 172.21.15.110:0/850360299 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a01f8230 con 0x7f03a0160fa0 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a5800640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03880039f0 0x7f03a01f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a6200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0160fa0 0x7f03a0161450 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f03a00cf000 tx=0x7f0398007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f039d600640 1 -- 172.21.15.110:0/850360299 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039800f6a0 con 0x7f03a0160fa0 2024-09-10T10:05:19.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f039d600640 1 -- 172.21.15.110:0/850360299 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0398018070 con 0x7f03a0160fa0 2024-09-10T10:05:19.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f039d600640 1 -- 172.21.15.110:0/850360299 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039800fae0 con 0x7f03a0160fa0 2024-09-10T10:05:19.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a00b2aa0 con 0x7f03a0160fa0 2024-09-10T10:05:19.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a014d1b0 con 0x7f03a0160fa0 2024-09-10T10:05:19.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.909+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a015ca10 con 0x7f03a0160fa0 2024-09-10T10:05:19.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.913+0000 7f039d600640 1 -- 172.21.15.110:0/850360299 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f039800fc80 con 0x7f03a0160fa0 2024-09-10T10:05:19.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.913+0000 7f039d600640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f03800680d0 0x7f038006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:19.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.913+0000 7f039d600640 1 -- 172.21.15.110:0/850360299 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5765+0+0 (secure 0 0 0) 0x7f0398086be0 con 0x7f03a0160fa0 2024-09-10T10:05:19.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.913+0000 7f03a5800640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f03800680d0 0x7f038006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:19.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.913+0000 7f03a5800640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f03800680d0 0x7f038006a590 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f03940ac000 tx=0x7f0394001520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:19.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:19.917+0000 7f039d600640 1 -- 172.21.15.110:0/850360299 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f03980566c0 con 0x7f03a0160fa0 2024-09-10T10:05:20.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.105+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f03a0071c60 con 0x7f03a0160fa0 2024-09-10T10:05:20.108 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:05:20.105+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:20.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.109+0000 7f039d600640 1 -- 172.21.15.110:0/850360299 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) ==== 161+0+0 (secure 0 0 0) 0x7f0398055fc0 con 0x7f03a0160fa0 2024-09-10T10:05:20.110 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:05:20.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f03800680d0 msgr2=0x7f038006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f03800680d0 0x7f038006a590 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f03940ac000 tx=0x7f0394001520 comp rx=0 tx=0).stop 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0160fa0 msgr2=0x7f03a0161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0160fa0 0x7f03a0161450 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f03a00cf000 tx=0x7f0398007430 comp rx=0 tx=0).stop 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 shutdown_connections 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f03800680d0 0x7f038006a590 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0160fa0 0x7f03a0161450 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f03a014d9b0 0x7f03a014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 --2- 172.21.15.110:0/850360299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03880039f0 0x7f03a01f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 >> 172.21.15.110:0/850360299 conn(0x7f03a0062d10 msgr2=0x7f03a006da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 shutdown_connections 2024-09-10T10:05:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.113+0000 7f03a8200640 1 -- 172.21.15.110:0/850360299 wait complete. 2024-09-10T10:05:20.133 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:05:20.181 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:05:20.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.269+0000 7f017d600640 1 Processor -- start 2024-09-10T10:05:20.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.269+0000 7f017d600640 1 -- start start 2024-09-10T10:05:20.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f017d600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f01780cf0f0 0x7f01780cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f017d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f0178072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f017d600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01780cd800 con 0x7f0178155850 2024-09-10T10:05:20.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f017d600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f01780ce410 con 0x7f01780cf0f0 2024-09-10T10:05:20.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f017d600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f01780cfa90 con 0x7f01780722b0 2024-09-10T10:05:20.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f0177400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f01780cf0f0 0x7f01780cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f0177400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f01780cf0f0 0x7f01780cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50274/0 (socket says 172.21.15.110:50274) 2024-09-10T10:05:20.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f0177400640 1 -- 172.21.15.110:0/2417479786 learned_addr learned my addr 172.21.15.110:0/2417479786 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:20.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f017cc00640 1 --2- 172.21.15.110:0/2417479786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f0178072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f0177400640 1 -- 172.21.15.110:0/2417479786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 msgr2=0x7f0178072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:20.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f0177400640 1 --2- 172.21.15.110:0/2417479786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f0178072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f0177400640 1 -- 172.21.15.110:0/2417479786 >> v1:172.21.15.145:6789/0 conn(0x7f0178155850 legacy=0x7f01780ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f0177400640 1 -- 172.21.15.110:0/2417479786 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01780cdae0 con 0x7f01780cf0f0 2024-09-10T10:05:20.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.273+0000 7f017cc00640 1 --2- 172.21.15.110:0/2417479786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f0178072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:20.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f0177400640 1 --2- 172.21.15.110:0/2417479786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f01780cf550 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0164009870 tx=0x7f016401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aee09c4b7cf0fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:20.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f0176a00640 1 -- 172.21.15.110:0/2417479786 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640ab650 con 0x7f01780cf0f0 2024-09-10T10:05:20.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f0176a00640 1 -- 172.21.15.110:0/2417479786 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01640b0070 con 0x7f01780cf0f0 2024-09-10T10:05:20.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f0176a00640 1 -- 172.21.15.110:0/2417479786 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640aba10 con 0x7f01780cf0f0 2024-09-10T10:05:20.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 -- 172.21.15.110:0/2417479786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 msgr2=0x7f01780cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 --2- 172.21.15.110:0/2417479786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f01780cf550 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0164009870 tx=0x7f016401c560 comp rx=0 tx=0).stop 2024-09-10T10:05:20.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 -- 172.21.15.110:0/2417479786 shutdown_connections 2024-09-10T10:05:20.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 --2- 172.21.15.110:0/2417479786 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f0178072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 --2- 172.21.15.110:0/2417479786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f01780cf550 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 -- 172.21.15.110:0/2417479786 >> 172.21.15.110:0/2417479786 conn(0x7f017806d710 msgr2=0x7f01780b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:20.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 -- 172.21.15.110:0/2417479786 shutdown_connections 2024-09-10T10:05:20.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 -- 172.21.15.110:0/2417479786 wait complete. 2024-09-10T10:05:20.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 Processor -- start 2024-09-10T10:05:20.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.277+0000 7f017d600640 1 -- start start 2024-09-10T10:05:20.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f017814f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f017814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0178155850 0x7f017815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f01780b62b0 con 0x7f01780cf0f0 2024-09-10T10:05:20.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01780b6130 con 0x7f0178155850 2024-09-10T10:05:20.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0178167e80 con 0x7f01780722b0 2024-09-10T10:05:20.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f017814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f017814dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50278/0 (socket says 172.21.15.110:50278) 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 -- 172.21.15.110:0/95470155 learned_addr learned my addr 172.21.15.110:0/95470155 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177e00640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f017814f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017cc00640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0178155850 0x7f017815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 -- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 msgr2=0x7f017814f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f017814f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 -- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0178155850 msgr2=0x7f017815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0178155850 0x7f017815ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 -- 172.21.15.110:0/95470155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017814fc70 con 0x7f01780cf0f0 2024-09-10T10:05:20.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f0177400640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f017814dff0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f016401ac10 tx=0x7f016401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:20.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f016fe00640 1 -- 172.21.15.110:0/95470155 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640be5f0 con 0x7f01780cf0f0 2024-09-10T10:05:20.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f016fe00640 1 -- 172.21.15.110:0/95470155 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01640be7b0 con 0x7f01780cf0f0 2024-09-10T10:05:20.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f016fe00640 1 -- 172.21.15.110:0/95470155 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640beba0 con 0x7f01780cf0f0 2024-09-10T10:05:20.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017814ff50 con 0x7f01780cf0f0 2024-09-10T10:05:20.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0178150510 con 0x7f01780cf0f0 2024-09-10T10:05:20.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0178072740 con 0x7f01780cf0f0 2024-09-10T10:05:20.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.281+0000 7f016fe00640 1 -- 172.21.15.110:0/95470155 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f0178073490 con 0x7f01780cf0f0 2024-09-10T10:05:20.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.285+0000 7f016fe00640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f01580680d0 0x7f015806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.285+0000 7f016fe00640 1 -- 172.21.15.110:0/95470155 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5851+0+0 (secure 0 0 0) 0x7f0164120f00 con 0x7f01780cf0f0 2024-09-10T10:05:20.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.285+0000 7f0177e00640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f01580680d0 0x7f015806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.285+0000 7f0177e00640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f01580680d0 0x7f015806a590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0168001810 tx=0x7f0168009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:20.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.289+0000 7f016fe00640 1 -- 172.21.15.110:0/95470155 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0164016150 con 0x7f01780cf0f0 2024-09-10T10:05:20.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:20.481+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f017814e530 con 0x7f01780cf0f0 2024-09-10T10:05:21.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.109+0000 7f016fe00640 1 -- 172.21.15.110:0/95470155 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) ==== 104+0+0 (secure 0 0 0) 0x7f01640f06c0 con 0x7f01780cf0f0 2024-09-10T10:05:21.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f01580680d0 msgr2=0x7f015806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f01580680d0 0x7f015806a590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0168001810 tx=0x7f0168009040 comp rx=0 tx=0).stop 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 msgr2=0x7f017814dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f017814dff0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f016401ac10 tx=0x7f016401ada0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 shutdown_connections 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f01580680d0 0x7f015806a590 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0178155850 0x7f017815ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01780cf0f0 0x7f017814dff0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 --2- 172.21.15.110:0/95470155 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f01780722b0 0x7f017814f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 >> 172.21.15.110:0/95470155 conn(0x7f017806d710 msgr2=0x7f0178155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 shutdown_connections 2024-09-10T10:05:21.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.113+0000 7f017d600640 1 -- 172.21.15.110:0/95470155 wait complete. 2024-09-10T10:05:21.138 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:05:21.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca8a00640 1 Processor -- start 2024-09-10T10:05:21.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca8a00640 1 -- start start 2024-09-10T10:05:21.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca8a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca40cfe30 0x7f4ca4071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca8a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4ca40cd510 0x7f4ca40cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca8a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca406d2d0 con 0x7f4ca406e090 2024-09-10T10:05:21.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca8a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca40cdeb0 con 0x7f4ca40cd510 2024-09-10T10:05:21.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca8a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca415d9a0 con 0x7f4ca40cfe30 2024-09-10T10:05:21.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca2800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f4ca406e090 0x7f4ca406e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55926/0 (socket says 172.21.15.110:55926) 2024-09-10T10:05:21.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca2800640 1 -- 172.21.15.110:0/1353000388 learned_addr learned my addr 172.21.15.110:0/1353000388 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:21.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca1e00640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca40cfe30 0x7f4ca4071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca1400640 1 --2- 172.21.15.110:0/1353000388 >> v2:172.21.15.145:3300/0 conn(0x7f4ca40cd510 0x7f4ca40cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541829150 0 0) 0x7f4ca406d2d0 con 0x7f4ca406e090 2024-09-10T10:05:21.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.281+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c88003530 con 0x7f4ca406e090 2024-09-10T10:05:21.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2699339916 0 0) 0x7f4c88003530 con 0x7f4ca406e090 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca40cfe30 msgr2=0x7f4ca4071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca40cfe30 0x7f4ca4071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 >> v2:172.21.15.145:3300/0 conn(0x7f4ca40cd510 msgr2=0x7f4ca40cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 --2- 172.21.15.110:0/1353000388 >> v2:172.21.15.145:3300/0 conn(0x7f4ca40cd510 0x7f4ca40cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca406d5b0 con 0x7f4ca406e090 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f4c9c001a00 con 0x7f4ca406e090 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca1400640 1 --2- 172.21.15.110:0/1353000388 >> v2:172.21.15.145:3300/0 conn(0x7f4ca40cd510 0x7f4ca40cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 >> v1:172.21.15.110:6789/0 conn(0x7f4ca406e090 legacy=0x7f4ca406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca40cfe30 0x7f4c880039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c88003f40 0x7f4c88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c88004d50 0x7f4c880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca415d9a0 con 0x7f4c88004d50 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8800ba80 con 0x7f4c88003f40 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8800bc50 con 0x7f4ca40cfe30 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca2800640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c88004d50 0x7f4c880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca1e00640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca40cfe30 0x7f4c880039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca1400640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c88003f40 0x7f4c88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca2800640 1 -- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca40cfe30 msgr2=0x7f4c880039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca2800640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca40cfe30 0x7f4c880039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca2800640 1 -- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c88003f40 msgr2=0x7f4c88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca2800640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c88003f40 0x7f4c88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca2800640 1 -- 172.21.15.110:0/1353000388 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c88003530 con 0x7f4c88004d50 2024-09-10T10:05:21.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.285+0000 7f4ca2800640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c88004d50 0x7f4c880051d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00a8d0 tx=0x7f4c9c01fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d245cd0082d7d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:21.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c00a4c0 con 0x7f4c88004d50 2024-09-10T10:05:21.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c0ad020 con 0x7f4c88004d50 2024-09-10T10:05:21.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca0a00640 1 -- 172.21.15.110:0/1353000388 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0a9740 con 0x7f4c88004d50 2024-09-10T10:05:21.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- 172.21.15.110:0/1353000388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c88004d50 msgr2=0x7f4c880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 --2- 172.21.15.110:0/1353000388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c88004d50 0x7f4c880051d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00a8d0 tx=0x7f4c9c01fb20 comp rx=0 tx=0).stop 2024-09-10T10:05:21.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca2800640 1 -- 172.21.15.110:0/1353000388 reap_dead start 2024-09-10T10:05:21.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- 172.21.15.110:0/1353000388 shutdown_connections 2024-09-10T10:05:21.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- 172.21.15.110:0/1353000388 >> 172.21.15.110:0/1353000388 conn(0x7f4ca406bc50 msgr2=0x7f4ca406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:21.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- 172.21.15.110:0/1353000388 shutdown_connections 2024-09-10T10:05:21.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- 172.21.15.110:0/1353000388 wait complete. 2024-09-10T10:05:21.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 Processor -- start 2024-09-10T10:05:21.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- start start 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca414c970 0x7f4ca415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ca415acd0 0x7f4ca40c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca40c9d90 0x7f4ca40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4071bc0 con 0x7f4ca40c9d90 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4071a40 con 0x7f4ca415acd0 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca8a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca4071d40 con 0x7f4ca414c970 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca40c9d90 0x7f4ca40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca2800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca40c9d90 0x7f4ca40b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50290/0 (socket says 172.21.15.110:50290) 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca2800640 1 -- 172.21.15.110:0/3325790202 learned_addr learned my addr 172.21.15.110:0/3325790202 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca1e00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca414c970 0x7f4ca415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.289+0000 7f4ca1400640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ca415acd0 0x7f4ca40c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca2800640 1 -- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca414c970 msgr2=0x7f4ca415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca2800640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca414c970 0x7f4ca415a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca2800640 1 -- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ca415acd0 msgr2=0x7f4ca40c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca2800640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ca415acd0 0x7f4ca40c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca2800640 1 -- 172.21.15.110:0/3325790202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca40b3850 con 0x7f4ca40c9d90 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca1400640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ca415acd0 0x7f4ca40c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca2800640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca40c9d90 0x7f4ca40b4610 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00a870 tx=0x7f4c88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4c9a000640 1 -- 172.21.15.110:0/3325790202 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c002020 con 0x7f4ca40c9d90 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4c9a000640 1 -- 172.21.15.110:0/3325790202 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c0b6070 con 0x7f4ca40c9d90 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4c9a000640 1 -- 172.21.15.110:0/3325790202 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0a9960 con 0x7f4ca40c9d90 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca40b3b30 con 0x7f4ca40c9d90 2024-09-10T10:05:21.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca40b4b50 con 0x7f4ca40c9d90 2024-09-10T10:05:21.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca4069bb0 con 0x7f4ca40c9d90 2024-09-10T10:05:21.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4c9a000640 1 -- 172.21.15.110:0/3325790202 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f4c9c0a9b00 con 0x7f4ca40c9d90 2024-09-10T10:05:21.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4c9a000640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4c840680d0 0x7f4c8406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:21.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4c9a000640 1 -- 172.21.15.110:0/3325790202 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5851+0+0 (secure 0 0 0) 0x7f4c9c1234c0 con 0x7f4ca40c9d90 2024-09-10T10:05:21.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.293+0000 7f4ca1e00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4c840680d0 0x7f4c8406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:21.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.297+0000 7f4ca1e00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4c840680d0 0x7f4c8406a590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4150ab0 tx=0x7f4c94009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:21.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.297+0000 7f4c9a000640 1 -- 172.21.15.110:0/3325790202 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4c9c019830 con 0x7f4ca40c9d90 2024-09-10T10:05:21.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:21.461+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4ca4072120 con 0x7f4ca40c9d90 2024-09-10T10:05:22.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.113+0000 7f4c9a000640 1 -- 172.21.15.110:0/3325790202 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) ==== 181+0+0 (secure 0 0 0) 0x7f4c9c0f36b0 con 0x7f4ca40c9d90 2024-09-10T10:05:22.114 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4c840680d0 msgr2=0x7f4c8406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4c840680d0 0x7f4c8406a590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4150ab0 tx=0x7f4c94009190 comp rx=0 tx=0).stop 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca40c9d90 msgr2=0x7f4ca40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca40c9d90 0x7f4ca40b4610 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00a870 tx=0x7f4c88005840 comp rx=0 tx=0).stop 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 shutdown_connections 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4c840680d0 0x7f4c8406a590 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca40c9d90 0x7f4ca40b4610 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ca415acd0 0x7f4ca40c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 --2- 172.21.15.110:0/3325790202 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4ca414c970 0x7f4ca415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 >> 172.21.15.110:0/3325790202 conn(0x7f4ca406bc50 msgr2=0x7f4ca4072340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:22.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 shutdown_connections 2024-09-10T10:05:22.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.117+0000 7f4ca8a00640 1 -- 172.21.15.110:0/3325790202 wait complete. 2024-09-10T10:05:22.142 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:05:22.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.289+0000 7fdd07200640 1 Processor -- start 2024-09-10T10:05:22.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.289+0000 7fdd07200640 1 -- start start 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd07200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdd000cd510 0x7fdd000cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd07200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd0006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd07200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd00072150 con 0x7fdd000cfe30 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd07200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdd0015d9a0 con 0x7fdd000cd510 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd07200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0006ea60 con 0x7fdd0006e090 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdcffe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fdd000cfe30 0x7fdd00071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55930/0 (socket says 172.21.15.110:55930) 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdcffe00640 1 -- 172.21.15.110:0/1267389725 learned_addr learned my addr 172.21.15.110:0/1267389725 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd05200640 1 --2- 172.21.15.110:0/1267389725 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd0006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdcff400640 1 --2- 172.21.15.110:0/1267389725 >> v2:172.21.15.145:3300/0 conn(0x7fdd000cd510 0x7fdd000cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdcfea00640 1 -- 172.21.15.110:0/1267389725 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2481016570 0 0) 0x7fdd00072150 con 0x7fdd000cfe30 2024-09-10T10:05:22.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdcfea00640 1 -- 172.21.15.110:0/1267389725 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdce4003530 con 0x7fdd000cfe30 2024-09-10T10:05:22.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd05200640 1 -- 172.21.15.110:0/1267389725 >> v2:172.21.15.145:3300/0 conn(0x7fdd000cd510 msgr2=0x7fdd000cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd05200640 1 --2- 172.21.15.110:0/1267389725 >> v2:172.21.15.145:3300/0 conn(0x7fdd000cd510 0x7fdd000cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd05200640 1 -- 172.21.15.110:0/1267389725 >> v1:172.21.15.110:6789/0 conn(0x7fdd000cfe30 legacy=0x7fdd00071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdd05200640 1 -- 172.21.15.110:0/1267389725 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0006d2d0 con 0x7fdd0006e090 2024-09-10T10:05:22.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.293+0000 7fdcff400640 1 --2- 172.21.15.110:0/1267389725 >> v2:172.21.15.145:3300/0 conn(0x7fdd000cd510 0x7fdd000cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:22.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd05200640 1 --2- 172.21.15.110:0/1267389725 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd0006e520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e58f09bc8d7cf10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdcfea00640 1 -- 172.21.15.110:0/1267389725 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80b2070 con 0x7fdd0006e090 2024-09-10T10:05:22.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdcfea00640 1 -- 172.21.15.110:0/1267389725 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf80a58b0 con 0x7fdd0006e090 2024-09-10T10:05:22.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdcfea00640 1 -- 172.21.15.110:0/1267389725 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80a5c80 con 0x7fdd0006e090 2024-09-10T10:05:22.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- 172.21.15.110:0/1267389725 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 msgr2=0x7fdd0006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 --2- 172.21.15.110:0/1267389725 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd0006e520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf801c8e0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- 172.21.15.110:0/1267389725 shutdown_connections 2024-09-10T10:05:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 --2- 172.21.15.110:0/1267389725 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd0006e520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 --2- 172.21.15.110:0/1267389725 >> v2:172.21.15.145:3300/0 conn(0x7fdd000cd510 0x7fdd000cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- 172.21.15.110:0/1267389725 >> 172.21.15.110:0/1267389725 conn(0x7fdd0006bc50 msgr2=0x7fdd0006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- 172.21.15.110:0/1267389725 shutdown_connections 2024-09-10T10:05:22.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- 172.21.15.110:0/1267389725 wait complete. 2024-09-10T10:05:22.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 Processor -- start 2024-09-10T10:05:22.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- start start 2024-09-10T10:05:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd000b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd000cd510 0x7fdd00143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd000cfe30 0x7fdd0015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00169180 con 0x7fdd000cd510 2024-09-10T10:05:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0015d6c0 con 0x7fdd000cfe30 2024-09-10T10:05:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.297+0000 7fdd07200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdd00169300 con 0x7fdd0006e090 2024-09-10T10:05:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd000cd510 0x7fdd00143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcffe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd000b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd000cd510 0x7fdd00143490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50300/0 (socket says 172.21.15.110:50300) 2024-09-10T10:05:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 -- 172.21.15.110:0/3661141072 learned_addr learned my addr 172.21.15.110:0/3661141072 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdd05200640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd000cfe30 0x7fdd0015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 -- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 msgr2=0x7fdd000b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd000b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 -- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd000cfe30 msgr2=0x7fdd0015bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd000cfe30 0x7fdd0015bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 -- 172.21.15.110:0/3661141072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00151360 con 0x7fdd000cd510 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcff400640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd000cd510 0x7fdd00143490 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e980 tx=0x7fdcec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcf7e00640 1 -- 172.21.15.110:0/3661141072 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec017070 con 0x7fdd000cd510 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcf7e00640 1 -- 172.21.15.110:0/3661141072 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec00ca00 con 0x7fdd000cd510 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcf7e00640 1 -- 172.21.15.110:0/3661141072 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00cdd0 con 0x7fdd000cd510 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd000b0c50 con 0x7fdd000cd510 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd001518e0 con 0x7fdd000cd510 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0006e520 con 0x7fdd000cd510 2024-09-10T10:05:22.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcf7e00640 1 -- 172.21.15.110:0/3661141072 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fdcec01a070 con 0x7fdd000cd510 2024-09-10T10:05:22.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcf7e00640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdce00680a0 0x7fdce006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.301+0000 7fdcf7e00640 1 -- 172.21.15.110:0/3661141072 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5413+0+0 (secure 0 0 0) 0x7fdcec086d40 con 0x7fdd000cd510 2024-09-10T10:05:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.305+0000 7fdcffe00640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdce00680a0 0x7fdce006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.305+0000 7fdcffe00640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdce00680a0 0x7fdce006a560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0007fd0 tx=0x7fdcf0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.305+0000 7fdcf7e00640 1 -- 172.21.15.110:0/3661141072 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdcec0562a0 con 0x7fdd000cd510 2024-09-10T10:05:22.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:22.465+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdd00150f50 con 0x7fdd000cd510 2024-09-10T10:05:23.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.113+0000 7fdcf7e00640 1 -- 172.21.15.110:0/3661141072 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) ==== 169+0+0 (secure 0 0 0) 0x7fdcec05a5a0 con 0x7fdd000cd510 2024-09-10T10:05:23.116 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:05:23.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdce00680a0 msgr2=0x7fdce006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdce00680a0 0x7fdce006a560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0007fd0 tx=0x7fdcf0009040 comp rx=0 tx=0).stop 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd000cd510 msgr2=0x7fdd00143490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd000cd510 0x7fdd00143490 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e980 tx=0x7fdcec00ee40 comp rx=0 tx=0).stop 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 shutdown_connections 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdce00680a0 0x7fdce006a560 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd000cfe30 0x7fdd0015bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd000cd510 0x7fdd00143490 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 --2- 172.21.15.110:0/3661141072 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdd0006e090 0x7fdd000b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.117+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 >> 172.21.15.110:0/3661141072 conn(0x7fdd0006bc50 msgr2=0x7fdd0006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.121+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 shutdown_connections 2024-09-10T10:05:23.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.121+0000 7fdd07200640 1 -- 172.21.15.110:0/3661141072 wait complete. 2024-09-10T10:05:23.147 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:05:23.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37bc400640 1 Processor -- start 2024-09-10T10:05:23.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37bc400640 1 -- start start 2024-09-10T10:05:23.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37bc400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f37b415b330 0x7f37b406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37bc400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b40b4960 con 0x7f37b40ce4d0 2024-09-10T10:05:23.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37bc400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b40b4c40 con 0x7f37b406e850 2024-09-10T10:05:23.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37bc400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f37b40b5570 con 0x7f37b415b330 2024-09-10T10:05:23.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f37b40ce4d0 0x7f37b406e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55942/0 (socket says 172.21.15.110:55942) 2024-09-10T10:05:23.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9000640 1 -- 172.21.15.110:0/2470998396 learned_addr learned my addr 172.21.15.110:0/2470998396 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9a00640 1 --2- 172.21.15.110:0/2470998396 >> v2:172.21.15.145:3300/0 conn(0x7f37b415b330 0x7f37b406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2958515195 0 0) 0x7f37b40b4960 con 0x7f37b40ce4d0 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f379c003530 con 0x7f37b40ce4d0 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 886860757 0 0) 0x7f379c003530 con 0x7f37b40ce4d0 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 >> v2:172.21.15.145:3300/0 conn(0x7f37b415b330 msgr2=0x7f37b406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 --2- 172.21.15.110:0/2470998396 >> v2:172.21.15.145:3300/0 conn(0x7f37b415b330 0x7f37b406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 >> v1:172.21.15.145:6789/0 conn(0x7f37b406e850 legacy=0x7f37b40b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b40b4f20 con 0x7f37b40ce4d0 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f37a0001680 con 0x7f37b40ce4d0 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 >> v1:172.21.15.110:6789/0 conn(0x7f37b40ce4d0 legacy=0x7f37b406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f379c0039c0 0x7f379c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f379c004810 0x7f379c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0051d0 0x7f379c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37b40b5570 con 0x7f379c0039c0 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f379c00c070 con 0x7f379c0051d0 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f379c00c270 con 0x7f379c004810 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9a00640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f379c0039c0 0x7f379c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9000640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f379c004810 0x7f379c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37ba400640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0051d0 0x7f379c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9a00640 1 -- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f379c004810 msgr2=0x7f379c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9a00640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f379c004810 0x7f379c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9a00640 1 -- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0051d0 msgr2=0x7f379c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9a00640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0051d0 0x7f379c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37b9a00640 1 -- 172.21.15.110:0/2470998396 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c003530 con 0x7f379c0039c0 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37ba400640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0051d0 0x7f379c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.289+0000 7f37ba400640 1 -- 172.21.15.110:0/2470998396 reap_dead start 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37b9a00640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f379c0039c0 0x7f379c003de0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009fc0 tx=0x7f37a801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d345e6bfd74c5cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80b2070 con 0x7f379c0039c0 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a80a5b80 con 0x7f379c0039c0 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37b3e00640 1 -- 172.21.15.110:0/2470998396 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8009c10 con 0x7f379c0039c0 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37bc400640 1 -- 172.21.15.110:0/2470998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f379c0039c0 msgr2=0x7f379c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37bc400640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f379c0039c0 0x7f379c003de0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009fc0 tx=0x7f37a801cc80 comp rx=0 tx=0).stop 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37bc400640 1 -- 172.21.15.110:0/2470998396 shutdown_connections 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37bc400640 1 --2- 172.21.15.110:0/2470998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f379c0039c0 0x7f379c003de0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37bc400640 1 -- 172.21.15.110:0/2470998396 >> 172.21.15.110:0/2470998396 conn(0x7f37b4062e40 msgr2=0x7f37b406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37bc400640 1 -- 172.21.15.110:0/2470998396 shutdown_connections 2024-09-10T10:05:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.293+0000 7f37bc400640 1 -- 172.21.15.110:0/2470998396 wait complete. 2024-09-10T10:05:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 Processor -- start 2024-09-10T10:05:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 -- start start 2024-09-10T10:05:23.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0039c0 0x7f37b4157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37b414f310 0x7f37b414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b415dd00 0x7f37b40ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37b406e350 con 0x7f37b415dd00 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f37b406cab0 con 0x7f379c0039c0 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f37b406e4d0 con 0x7f37b414f310 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b415dd00 0x7f37b40ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b415dd00 0x7f37b40ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50318/0 (socket says 172.21.15.110:50318) 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 -- 172.21.15.110:0/4050528559 learned_addr learned my addr 172.21.15.110:0/4050528559 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37b9000640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37b414f310 0x7f37b414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37b9a00640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0039c0 0x7f37b4157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 -- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37b414f310 msgr2=0x7f37b414ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37b414f310 0x7f37b414ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 -- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0039c0 msgr2=0x7f37b4157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0039c0 0x7f37b4157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 -- 172.21.15.110:0/4050528559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b415e5e0 con 0x7f37b415dd00 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37ba400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b415dd00 0x7f37b40ae870 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f37b40b0670 tx=0x7f37ac00e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37b1600640 1 -- 172.21.15.110:0/4050528559 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac019070 con 0x7f37b415dd00 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37b1600640 1 -- 172.21.15.110:0/4050528559 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ac0050d0 con 0x7f37b415dd00 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37b1600640 1 -- 172.21.15.110:0/4050528559 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac003600 con 0x7f37b415dd00 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b415e8c0 con 0x7f37b415dd00 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.297+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b4150c00 con 0x7f37b415dd00 2024-09-10T10:05:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.301+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b415f750 con 0x7f37b415dd00 2024-09-10T10:05:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.301+0000 7f37b1600640 1 -- 172.21.15.110:0/4050528559 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f37ac01c070 con 0x7f37b415dd00 2024-09-10T10:05:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.301+0000 7f37b1600640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37940680a0 0x7f379406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.301+0000 7f37b1600640 1 -- 172.21.15.110:0/4050528559 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4999+0+0 (secure 0 0 0) 0x7f37ac085ac0 con 0x7f37b415dd00 2024-09-10T10:05:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.301+0000 7f37b9a00640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37940680a0 0x7f379406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.301+0000 7f37b9a00640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37940680a0 0x7f379406a560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f37a80af000 tx=0x7f37a8004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:23.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.305+0000 7f37b1600640 1 -- 172.21.15.110:0/4050528559 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f37ac055d40 con 0x7f37b415dd00 2024-09-10T10:05:23.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.473+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f37b4156700 con 0x7f37b415dd00 2024-09-10T10:05:23.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37b1600640 1 -- 172.21.15.110:0/4050528559 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f37ac055640 con 0x7f37b415dd00 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37940680a0 msgr2=0x7f379406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37940680a0 0x7f379406a560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f37a80af000 tx=0x7f37a8004ac0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b415dd00 msgr2=0x7f37b40ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b415dd00 0x7f37b40ae870 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f37b40b0670 tx=0x7f37ac00e610 comp rx=0 tx=0).stop 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 shutdown_connections 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37940680a0 0x7f379406a560 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b415dd00 0x7f37b40ae870 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37b414f310 0x7f37b414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 --2- 172.21.15.110:0/4050528559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f379c0039c0 0x7f37b4157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 >> 172.21.15.110:0/4050528559 conn(0x7f37b4062e40 msgr2=0x7f37b4156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:23.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 shutdown_connections 2024-09-10T10:05:23.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.477+0000 7f37bc400640 1 -- 172.21.15.110:0/4050528559 wait complete. 2024-09-10T10:05:23.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-10T10:05:23.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.633+0000 7f2754800640 1 Processor -- start 2024-09-10T10:05:23.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2754800640 1 -- start start 2024-09-10T10:05:23.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2754800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c155970 0x7f274c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2754800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f274c06f1e0 0x7f274c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2754800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274c155d50 con 0x7f274c15c560 2024-09-10T10:05:23.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2754800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f274c0b4570 con 0x7f274c06f1e0 2024-09-10T10:05:23.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2754800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0b34d0 con 0x7f274c155970 2024-09-10T10:05:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2751e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c155970 0x7f274c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2752800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f274c06f1e0 0x7f274c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2751e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c155970 0x7f274c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:39188/0 (socket says 172.21.15.110:39188) 2024-09-10T10:05:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2751e00640 1 -- 172.21.15.110:0/3142136585 learned_addr learned my addr 172.21.15.110:0/3142136585 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2752800640 1 -- 172.21.15.110:0/3142136585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c155970 msgr2=0x7f274c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2752800640 1 --2- 172.21.15.110:0/3142136585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c155970 0x7f274c0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2752800640 1 -- 172.21.15.110:0/3142136585 >> v1:172.21.15.145:6789/0 conn(0x7f274c15c560 legacy=0x7f274c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.637+0000 7f2752800640 1 -- 172.21.15.110:0/3142136585 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c0b37b0 con 0x7f274c06f1e0 2024-09-10T10:05:23.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2752800640 1 --2- 172.21.15.110:0/3142136585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 0x7f274c0ce420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2748009d90 tx=0x7f274801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa5bad2b483c52c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:23.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2750a00640 1 -- 172.21.15.110:0/3142136585 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27480b2070 con 0x7f274c06f1e0 2024-09-10T10:05:23.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2750a00640 1 -- 172.21.15.110:0/3142136585 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27480a5970 con 0x7f274c06f1e0 2024-09-10T10:05:23.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2750a00640 1 -- 172.21.15.110:0/3142136585 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27480a5d40 con 0x7f274c06f1e0 2024-09-10T10:05:23.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2754800640 1 -- 172.21.15.110:0/3142136585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 msgr2=0x7f274c0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2754800640 1 --2- 172.21.15.110:0/3142136585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 0x7f274c0ce420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2748009d90 tx=0x7f274801c8b0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2754800640 1 -- 172.21.15.110:0/3142136585 shutdown_connections 2024-09-10T10:05:23.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2754800640 1 --2- 172.21.15.110:0/3142136585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 0x7f274c0ce420 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2754800640 1 --2- 172.21.15.110:0/3142136585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c155970 0x7f274c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2754800640 1 -- 172.21.15.110:0/3142136585 >> 172.21.15.110:0/3142136585 conn(0x7f274c06bc50 msgr2=0x7f274c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:23.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2754800640 1 -- 172.21.15.110:0/3142136585 shutdown_connections 2024-09-10T10:05:23.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.641+0000 7f2754800640 1 -- 172.21.15.110:0/3142136585 wait complete. 2024-09-10T10:05:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 Processor -- start 2024-09-10T10:05:23.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 -- start start 2024-09-10T10:05:23.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 0x7f274c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f274c155970 0x7f274c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c15c560 0x7f274c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0cf170 con 0x7f274c155970 2024-09-10T10:05:23.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0cce00 con 0x7f274c06f1e0 2024-09-10T10:05:23.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0cf2f0 con 0x7f274c15c560 2024-09-10T10:05:23.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f274c155970 0x7f274c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f274c155970 0x7f274c0aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50320/0 (socket says 172.21.15.110:50320) 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 -- 172.21.15.110:0/2933390044 learned_addr learned my addr 172.21.15.110:0/2933390044 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751e00640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 0x7f274c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2752800640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c15c560 0x7f274c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 -- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c15c560 msgr2=0x7f274c14e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c15c560 0x7f274c14e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 -- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 msgr2=0x7f274c155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 0x7f274c155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 -- 172.21.15.110:0/2933390044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c14ff40 con 0x7f274c155970 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2751400640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f274c155970 0x7f274c0aeaa0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c340 tx=0x7f273c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2746000640 1 -- 172.21.15.110:0/2933390044 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c0052d0 con 0x7f274c155970 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2746000640 1 -- 172.21.15.110:0/2933390044 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c005470 con 0x7f274c155970 2024-09-10T10:05:23.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2746000640 1 -- 172.21.15.110:0/2933390044 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c007260 con 0x7f274c155970 2024-09-10T10:05:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c150200 con 0x7f274c155970 2024-09-10T10:05:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c1507c0 con 0x7f274c155970 2024-09-10T10:05:23.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.645+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c0ce420 con 0x7f274c155970 2024-09-10T10:05:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.649+0000 7f2746000640 1 -- 172.21.15.110:0/2933390044 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f273c007400 con 0x7f274c155970 2024-09-10T10:05:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.649+0000 7f2746000640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f27300680a0 0x7f273006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.649+0000 7f2746000640 1 -- 172.21.15.110:0/2933390044 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4999+0+0 (secure 0 0 0) 0x7f273c085ff0 con 0x7f274c155970 2024-09-10T10:05:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.649+0000 7f2751e00640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f27300680a0 0x7f273006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:23.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.649+0000 7f2751e00640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f27300680a0 0x7f273006a560 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2740009b50 tx=0x7f2740009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:23.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.653+0000 7f2746000640 1 -- 172.21.15.110:0/2933390044 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f273c055810 con 0x7f274c155970 2024-09-10T10:05:23.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:23.869+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f274c1511f0 con 0x7f274c155970 2024-09-10T10:05:24.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.125+0000 7f2746000640 1 -- 172.21.15.110:0/2933390044 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v388) ==== 149+0+0 (secure 0 0 0) 0x7f273c059b10 con 0x7f274c155970 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.125+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f27300680a0 msgr2=0x7f273006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.125+0000 7f2754800640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f27300680a0 0x7f273006a560 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2740009b50 tx=0x7f2740009240 comp rx=0 tx=0).stop 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.125+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f274c155970 msgr2=0x7f274c0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.125+0000 7f2754800640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f274c155970 0x7f274c0aeaa0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f273c00c340 tx=0x7f273c00c800 comp rx=0 tx=0).stop 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.129+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 shutdown_connections 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.129+0000 7f2754800640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f27300680a0 0x7f273006a560 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.129+0000 7f2754800640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f274c15c560 0x7f274c14e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.129+0000 7f2754800640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f274c155970 0x7f274c0aeaa0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.129+0000 7f2754800640 1 --2- 172.21.15.110:0/2933390044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f274c06f1e0 0x7f274c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.129+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 >> 172.21.15.110:0/2933390044 conn(0x7f274c06bc50 msgr2=0x7f274c0cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:24.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.129+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 shutdown_connections 2024-09-10T10:05:24.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.129+0000 7f2754800640 1 -- 172.21.15.110:0/2933390044 wait complete. 2024-09-10T10:05:24.149 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-10T10:05:24.150 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-10T10:05:24.150 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:05:24.150 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:24.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.285+0000 7fdb5a600640 1 Processor -- start 2024-09-10T10:05:24.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.285+0000 7fdb5a600640 1 -- start start 2024-09-10T10:05:24.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb5a600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5406f1e0 0x7fdb54166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb5a600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb54155d50 con 0x7fdb5415c560 2024-09-10T10:05:24.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb5a600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb540b34d0 con 0x7fdb54155970 2024-09-10T10:05:24.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb5a600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdb540b4570 con 0x7fdb5406f1e0 2024-09-10T10:05:24.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fdb5415c560 0x7fdb5406ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55946/0 (socket says 172.21.15.110:55946) 2024-09-10T10:05:24.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52a00640 1 -- 172.21.15.110:0/210057246 learned_addr learned my addr 172.21.15.110:0/210057246 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb53e00640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5406f1e0 0x7fdb54166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278008299 0 0) 0x7fdb54155d50 con 0x7fdb5415c560 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb38005740 con 0x7fdb5415c560 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 935282774 0 0) 0x7fdb38005740 con 0x7fdb5415c560 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5406f1e0 msgr2=0x7fdb54166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5406f1e0 0x7fdb54166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 >> v1:172.21.15.145:6789/0 conn(0x7fdb54155970 legacy=0x7fdb540cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb540b37b0 con 0x7fdb5415c560 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fdb400016a0 con 0x7fdb5415c560 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 >> v1:172.21.15.110:6789/0 conn(0x7fdb5415c560 legacy=0x7fdb5406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb38006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb38006a20 0x7fdb38006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb380073e0 0x7fdb3800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3800e1d0 con 0x7fdb38005c00 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3800e380 con 0x7fdb380073e0 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3800e580 con 0x7fdb38006a20 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb53400640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb38006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb53e00640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb380073e0 0x7fdb3800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52a00640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb38006a20 0x7fdb38006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb53400640 1 -- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb38006a20 msgr2=0x7fdb38006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb53400640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb38006a20 0x7fdb38006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb53400640 1 -- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb380073e0 msgr2=0x7fdb3800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb53400640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb380073e0 0x7fdb3800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb53400640 1 -- 172.21.15.110:0/210057246 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb38005740 con 0x7fdb38005c00 2024-09-10T10:05:24.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.289+0000 7fdb52a00640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb38006a20 0x7fdb38006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:24.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb53400640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb38006020 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fdb44001e40 tx=0x7fdb4401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b502f232d5fc00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44001a90 con 0x7fdb38005c00 2024-09-10T10:05:24.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb44003b80 con 0x7fdb38005c00 2024-09-10T10:05:24.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb52000640 1 -- 172.21.15.110:0/210057246 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44005080 con 0x7fdb38005c00 2024-09-10T10:05:24.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb5a600640 1 -- 172.21.15.110:0/210057246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 msgr2=0x7fdb38006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb5a600640 1 --2- 172.21.15.110:0/210057246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb38006020 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fdb44001e40 tx=0x7fdb4401c730 comp rx=0 tx=0).stop 2024-09-10T10:05:24.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb53e00640 1 -- 172.21.15.110:0/210057246 reap_dead start 2024-09-10T10:05:24.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb5a600640 1 -- 172.21.15.110:0/210057246 shutdown_connections 2024-09-10T10:05:24.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb5a600640 1 -- 172.21.15.110:0/210057246 >> 172.21.15.110:0/210057246 conn(0x7fdb5406bc50 msgr2=0x7fdb5406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:24.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb5a600640 1 -- 172.21.15.110:0/210057246 shutdown_connections 2024-09-10T10:05:24.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.293+0000 7fdb5a600640 1 -- 172.21.15.110:0/210057246 wait complete. 2024-09-10T10:05:24.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 Processor -- start 2024-09-10T10:05:24.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 -- start start 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb541456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb5415f840 0x7fdb5414de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5414e7a0 0x7fdb540b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406e350 con 0x7fdb38005c00 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406e1d0 con 0x7fdb5415f840 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5406e4d0 con 0x7fdb5414e7a0 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb541456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb541456d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50340/0 (socket says 172.21.15.110:50340) 2024-09-10T10:05:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 -- 172.21.15.110:0/473224162 learned_addr learned my addr 172.21.15.110:0/473224162 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb52a00640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb5415f840 0x7fdb5414de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53e00640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5414e7a0 0x7fdb540b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 -- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5414e7a0 msgr2=0x7fdb540b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5414e7a0 0x7fdb540b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 -- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb5415f840 msgr2=0x7fdb5414de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb5415f840 0x7fdb5414de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 -- 172.21.15.110:0/473224162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5414e380 con 0x7fdb38005c00 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb53400640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb541456d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdb44098120 tx=0x7fdb38007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb4b400640 1 -- 172.21.15.110:0/473224162 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44007dc0 con 0x7fdb38005c00 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb4b400640 1 -- 172.21.15.110:0/473224162 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb44001260 con 0x7fdb38005c00 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb4b400640 1 -- 172.21.15.110:0/473224162 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44005710 con 0x7fdb38005c00 2024-09-10T10:05:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5414f0d0 con 0x7fdb38005c00 2024-09-10T10:05:24.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5414d640 con 0x7fdb38005c00 2024-09-10T10:05:24.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.297+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5406ead0 con 0x7fdb38005c00 2024-09-10T10:05:24.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.301+0000 7fdb4b400640 1 -- 172.21.15.110:0/473224162 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fdb440b1040 con 0x7fdb38005c00 2024-09-10T10:05:24.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.301+0000 7fdb4b400640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdb340680d0 0x7fdb3406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.301+0000 7fdb4b400640 1 -- 172.21.15.110:0/473224162 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4999+0+0 (secure 0 0 0) 0x7fdb44121240 con 0x7fdb38005c00 2024-09-10T10:05:24.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.301+0000 7fdb52a00640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdb340680d0 0x7fdb3406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.301+0000 7fdb52a00640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdb340680d0 0x7fdb3406a590 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb5415fc60 tx=0x7fdb40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.305+0000 7fdb4b400640 1 -- 172.21.15.110:0/473224162 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdb440b8030 con 0x7fdb38005c00 2024-09-10T10:05:24.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.461+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb540cd740 con 0x7fdb38005c00 2024-09-10T10:05:24.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb4b400640 1 -- 172.21.15.110:0/473224162 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+13169 (secure 0 0 0) 0x7fdb440ad020 con 0x7fdb38005c00 2024-09-10T10:05:24.465 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:24.465 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":145,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:23.115435+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:6835/1325843861":"2024-09-11T10:05:20.109505+0000","172.21.15.145:6833/1325843861":"2024-09-11T10:05:20.109505+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:24.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdb340680d0 msgr2=0x7fdb3406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdb340680d0 0x7fdb3406a590 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb5415fc60 tx=0x7fdb40009040 comp rx=0 tx=0).stop 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 msgr2=0x7fdb541456d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb541456d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdb44098120 tx=0x7fdb38007a50 comp rx=0 tx=0).stop 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 shutdown_connections 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdb340680d0 0x7fdb3406a590 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdb5414e7a0 0x7fdb540b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb5415f840 0x7fdb5414de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 --2- 172.21.15.110:0/473224162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38005c00 0x7fdb541456d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 >> 172.21.15.110:0/473224162 conn(0x7fdb5406bc50 msgr2=0x7fdb54154920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 shutdown_connections 2024-09-10T10:05:24.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.465+0000 7fdb5a600640 1 -- 172.21.15.110:0/473224162 wait complete. 2024-09-10T10:05:24.488 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:05:24.488 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:05:24.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaaec00640 1 Processor -- start 2024-09-10T10:05:24.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaaec00640 1 -- start start 2024-09-10T10:05:24.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaaec00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbaa80ce4d0 0x7fbaa806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaaec00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fbaa806e850 0x7fbaa80b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaaec00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa80b4960 con 0x7fbaa815b330 2024-09-10T10:05:24.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaaec00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa80b5570 con 0x7fbaa80ce4d0 2024-09-10T10:05:24.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaaec00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa80b4c40 con 0x7fbaa806e850 2024-09-10T10:05:24.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaa7e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fbaa815b330 0x7fbaa806e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55950/0 (socket says 172.21.15.110:55950) 2024-09-10T10:05:24.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaa7e00640 1 -- 172.21.15.110:0/3417680496 learned_addr learned my addr 172.21.15.110:0/3417680496 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:24.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaa7400640 1 --2- 172.21.15.110:0/3417680496 >> v2:172.21.15.110:3300/0 conn(0x7fbaa80ce4d0 0x7fbaa806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaacc00640 1 --2- 172.21.15.110:0/3417680496 >> v2:172.21.15.145:3300/0 conn(0x7fbaa806e850 0x7fbaa80b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaa7400640 1 -- 172.21.15.110:0/3417680496 >> v2:172.21.15.145:3300/0 conn(0x7fbaa806e850 msgr2=0x7fbaa80b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:24.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaa7400640 1 --2- 172.21.15.110:0/3417680496 >> v2:172.21.15.145:3300/0 conn(0x7fbaa806e850 0x7fbaa80b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaa7400640 1 -- 172.21.15.110:0/3417680496 >> v1:172.21.15.110:6789/0 conn(0x7fbaa815b330 legacy=0x7fbaa806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaa7400640 1 -- 172.21.15.110:0/3417680496 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa80b4f20 con 0x7fbaa80ce4d0 2024-09-10T10:05:24.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.625+0000 7fbaacc00640 1 --2- 172.21.15.110:0/3417680496 >> v2:172.21.15.145:3300/0 conn(0x7fbaa806e850 0x7fbaa80b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:24.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaa7400640 1 --2- 172.21.15.110:0/3417680496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa80ce4d0 0x7fbaa806c6a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80633d0 tx=0x7fba9401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d710f2835702d19b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaa6a00640 1 -- 172.21.15.110:0/3417680496 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940b1070 con 0x7fbaa80ce4d0 2024-09-10T10:05:24.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaa6a00640 1 -- 172.21.15.110:0/3417680496 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba940ac5f0 con 0x7fbaa80ce4d0 2024-09-10T10:05:24.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaa6a00640 1 -- 172.21.15.110:0/3417680496 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940ac9c0 con 0x7fbaa80ce4d0 2024-09-10T10:05:24.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaaec00640 1 -- 172.21.15.110:0/3417680496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa80ce4d0 msgr2=0x7fbaa806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaaec00640 1 --2- 172.21.15.110:0/3417680496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa80ce4d0 0x7fbaa806c6a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80633d0 tx=0x7fba9401c480 comp rx=0 tx=0).stop 2024-09-10T10:05:24.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaaec00640 1 -- 172.21.15.110:0/3417680496 shutdown_connections 2024-09-10T10:05:24.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaaec00640 1 --2- 172.21.15.110:0/3417680496 >> v2:172.21.15.145:3300/0 conn(0x7fbaa806e850 0x7fbaa80b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaaec00640 1 --2- 172.21.15.110:0/3417680496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa80ce4d0 0x7fbaa806c6a0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaaec00640 1 -- 172.21.15.110:0/3417680496 >> 172.21.15.110:0/3417680496 conn(0x7fbaa8062e40 msgr2=0x7fbaa806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:24.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaaec00640 1 -- 172.21.15.110:0/3417680496 shutdown_connections 2024-09-10T10:05:24.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.629+0000 7fbaaec00640 1 -- 172.21.15.110:0/3417680496 wait complete. 2024-09-10T10:05:24.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 Processor -- start 2024-09-10T10:05:24.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 -- start start 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa806e850 0x7fbaa8160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa815b330 0x7fbaa81559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbaa8155450 0x7fbaa815db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81680b0 con 0x7fbaa815b330 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80b1800 con 0x7fbaa806e850 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa8168230 con 0x7fbaa8155450 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa815b330 0x7fbaa81559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa815b330 0x7fbaa81559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50354/0 (socket says 172.21.15.110:50354) 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 -- 172.21.15.110:0/537033721 learned_addr learned my addr 172.21.15.110:0/537033721 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 -- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbaa8155450 msgr2=0x7fbaa815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbaa8155450 0x7fbaa815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 -- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa806e850 msgr2=0x7fbaa8160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa806e850 0x7fbaa8160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 -- 172.21.15.110:0/537033721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa8160650 con 0x7fbaa815b330 2024-09-10T10:05:24.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaa7400640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa815b330 0x7fbaa81559e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fba9401c960 tx=0x7fba94019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fba9fe00640 1 -- 172.21.15.110:0/537033721 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940b1040 con 0x7fbaa815b330 2024-09-10T10:05:24.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fba9fe00640 1 -- 172.21.15.110:0/537033721 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba940ba070 con 0x7fbaa815b330 2024-09-10T10:05:24.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa8158ca0 con 0x7fbaa815b330 2024-09-10T10:05:24.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fba9fe00640 1 -- 172.21.15.110:0/537033721 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940b5670 con 0x7fbaa815b330 2024-09-10T10:05:24.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa8159260 con 0x7fbaa815b330 2024-09-10T10:05:24.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.633+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa80b0860 con 0x7fbaa815b330 2024-09-10T10:05:24.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.637+0000 7fba9fe00640 1 -- 172.21.15.110:0/537033721 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fba940b5810 con 0x7fbaa815b330 2024-09-10T10:05:24.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.637+0000 7fba9fe00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba880680a0 0x7fba8806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.637+0000 7fba9fe00640 1 -- 172.21.15.110:0/537033721 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4999+0+0 (secure 0 0 0) 0x7fba941207f0 con 0x7fbaa815b330 2024-09-10T10:05:24.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.637+0000 7fbaa7e00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba880680a0 0x7fba8806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.637+0000 7fbaa7e00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba880680a0 0x7fba8806a560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fba98000f70 tx=0x7fba98007a90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.641+0000 7fba9fe00640 1 -- 172.21.15.110:0/537033721 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fba940a9070 con 0x7fbaa815b330 2024-09-10T10:05:24.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.797+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbaa806e2f0 con 0x7fbaa815b330 2024-09-10T10:05:24.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.797+0000 7fba9fe00640 1 -- 172.21.15.110:0/537033721 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7fba94016070 con 0x7fbaa815b330 2024-09-10T10:05:24.800 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba880680a0 msgr2=0x7fba8806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba880680a0 0x7fba8806a560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fba98000f70 tx=0x7fba98007a90 comp rx=0 tx=0).stop 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa815b330 msgr2=0x7fbaa81559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa815b330 0x7fbaa81559e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fba9401c960 tx=0x7fba94019220 comp rx=0 tx=0).stop 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 shutdown_connections 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba880680a0 0x7fba8806a560 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbaa8155450 0x7fbaa815db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa815b330 0x7fbaa81559e0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 --2- 172.21.15.110:0/537033721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbaa806e850 0x7fbaa8160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 >> 172.21.15.110:0/537033721 conn(0x7fbaa8062e40 msgr2=0x7fbaa80cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 shutdown_connections 2024-09-10T10:05:24.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.801+0000 7fbaaec00640 1 -- 172.21.15.110:0/537033721 wait complete. 2024-09-10T10:05:24.821 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-10T10:05:24.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.953+0000 7ff1a1000640 1 Processor -- start 2024-09-10T10:05:24.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.953+0000 7ff1a1000640 1 -- start start 2024-09-10T10:05:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff1a1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff1a1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff19c15c560 0x7ff19c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff1a1000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff19c155d50 con 0x7ff19c06f010 2024-09-10T10:05:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff1a1000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff19c0b4570 con 0x7ff19c15c560 2024-09-10T10:05:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff1a1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff19c06f450 con 0x7ff19c155970 2024-09-10T10:05:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff19c15c560 0x7ff19c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff19c15c560 0x7ff19c06ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50364/0 (socket says 172.21.15.110:50364) 2024-09-10T10:05:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199400640 1 -- 172.21.15.110:0/3814920550 learned_addr learned my addr 172.21.15.110:0/3814920550 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199e00640 1 --2- 172.21.15.110:0/3814920550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199400640 1 -- 172.21.15.110:0/3814920550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 msgr2=0x7ff19c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:24.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199400640 1 --2- 172.21.15.110:0/3814920550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199400640 1 -- 172.21.15.110:0/3814920550 >> v1:172.21.15.145:6789/0 conn(0x7ff19c06f010 legacy=0x7ff19c15eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:24.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199400640 1 -- 172.21.15.110:0/3814920550 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c0b34d0 con 0x7ff19c15c560 2024-09-10T10:05:24.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.957+0000 7ff199e00640 1 --2- 172.21.15.110:0/3814920550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:24.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff199400640 1 --2- 172.21.15.110:0/3814920550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c06ead0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff19c06f5d0 tx=0x7ff18801eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d066472713fcec87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff198a00640 1 -- 172.21.15.110:0/3814920550 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880b4070 con 0x7ff19c15c560 2024-09-10T10:05:24.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff198a00640 1 -- 172.21.15.110:0/3814920550 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1880a7ba0 con 0x7ff19c15c560 2024-09-10T10:05:24.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff198a00640 1 -- 172.21.15.110:0/3814920550 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880af440 con 0x7ff19c15c560 2024-09-10T10:05:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 -- 172.21.15.110:0/3814920550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 msgr2=0x7ff19c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 --2- 172.21.15.110:0/3814920550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c06ead0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff19c06f5d0 tx=0x7ff18801eb10 comp rx=0 tx=0).stop 2024-09-10T10:05:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 -- 172.21.15.110:0/3814920550 shutdown_connections 2024-09-10T10:05:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 --2- 172.21.15.110:0/3814920550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c06ead0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 --2- 172.21.15.110:0/3814920550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 -- 172.21.15.110:0/3814920550 >> 172.21.15.110:0/3814920550 conn(0x7ff19c06bc50 msgr2=0x7ff19c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:24.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 -- 172.21.15.110:0/3814920550 shutdown_connections 2024-09-10T10:05:24.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 -- 172.21.15.110:0/3814920550 wait complete. 2024-09-10T10:05:24.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 Processor -- start 2024-09-10T10:05:24.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.961+0000 7ff1a1000640 1 -- start start 2024-09-10T10:05:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff19c06f010 0x7ff19c151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c0cce00 con 0x7ff19c15c560 2024-09-10T10:05:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c0b4050 con 0x7ff19c06f010 2024-09-10T10:05:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff19c06d9c0 con 0x7ff19c155970 2024-09-10T10:05:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50376/0 (socket says 172.21.15.110:50376) 2024-09-10T10:05:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff199e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff19c06f010 0x7ff19c151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 -- 172.21.15.110:0/2285754541 learned_addr learned my addr 172.21.15.110:0/2285754541 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff199400640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 -- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 msgr2=0x7ff19c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 -- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff19c06f010 msgr2=0x7ff19c151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff19c06f010 0x7ff19c151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 -- 172.21.15.110:0/2285754541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c15ade0 con 0x7ff19c15c560 2024-09-10T10:05:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff19a800640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c156640 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff19400c3c0 tx=0x7ff19400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff192000640 1 -- 172.21.15.110:0/2285754541 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194011790 con 0x7ff19c15c560 2024-09-10T10:05:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff192000640 1 -- 172.21.15.110:0/2285754541 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff194017070 con 0x7ff19c15c560 2024-09-10T10:05:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c150b50 con 0x7ff19c15c560 2024-09-10T10:05:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff192000640 1 -- 172.21.15.110:0/2285754541 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194011b70 con 0x7ff19c15c560 2024-09-10T10:05:24.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c151110 con 0x7ff19c15c560 2024-09-10T10:05:24.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19c0cc8c0 con 0x7ff19c15c560 2024-09-10T10:05:24.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.965+0000 7ff192000640 1 -- 172.21.15.110:0/2285754541 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7ff194011d10 con 0x7ff19c15c560 2024-09-10T10:05:24.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.969+0000 7ff192000640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff17c0680a0 0x7ff17c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.969+0000 7ff192000640 1 -- 172.21.15.110:0/2285754541 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4999+0+0 (secure 0 0 0) 0x7ff194086860 con 0x7ff19c15c560 2024-09-10T10:05:24.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.969+0000 7ff199e00640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff17c0680a0 0x7ff17c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.969+0000 7ff199e00640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff17c0680a0 0x7ff17c06a560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00d620 tx=0x7ff18c007810 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:24.973+0000 7ff192000640 1 -- 172.21.15.110:0/2285754541 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff19c15ade0 con 0x7ff19c15c560 2024-09-10T10:05:25.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:25.185+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7ff19c075560 con 0x7ff19c15c560 2024-09-10T10:05:26.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff192000640 1 -- 172.21.15.110:0/2285754541 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v390) ==== 148+0+0 (secure 0 0 0) 0x7ff194056080 con 0x7ff19c15c560 2024-09-10T10:05:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff17c0680a0 msgr2=0x7ff17c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff17c0680a0 0x7ff17c06a560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00d620 tx=0x7ff18c007810 comp rx=0 tx=0).stop 2024-09-10T10:05:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 msgr2=0x7ff19c156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c156640 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff19400c3c0 tx=0x7ff19400c880 comp rx=0 tx=0).stop 2024-09-10T10:05:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 shutdown_connections 2024-09-10T10:05:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff17c0680a0 0x7ff17c06a560 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff19c15c560 0x7ff19c156640 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff19c155970 0x7ff19c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 --2- 172.21.15.110:0/2285754541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff19c06f010 0x7ff19c151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 >> 172.21.15.110:0/2285754541 conn(0x7ff19c06bc50 msgr2=0x7ff19c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:26.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 shutdown_connections 2024-09-10T10:05:26.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.133+0000 7ff1a1000640 1 -- 172.21.15.110:0/2285754541 wait complete. 2024-09-10T10:05:26.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:05:26.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4796000640 1 Processor -- start 2024-09-10T10:05:26.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4796000640 1 -- start start 2024-09-10T10:05:26.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4796000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 0x7f479006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4796000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f47900ce4d0 0x7f479006e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4796000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47900b1870 con 0x7f47900b1c60 2024-09-10T10:05:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4796000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f47900b5570 con 0x7f47900ce4d0 2024-09-10T10:05:26.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4796000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f47900b4960 con 0x7f479015b330 2024-09-10T10:05:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f478f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 0x7f479006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4786a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f47900ce4d0 0x7f479006e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4786a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f47900ce4d0 0x7f479006e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:43010/0 (socket says 172.21.15.110:43010) 2024-09-10T10:05:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4786a00640 1 -- 172.21.15.110:0/1252455904 learned_addr learned my addr 172.21.15.110:0/1252455904 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4786a00640 1 -- 172.21.15.110:0/1252455904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 msgr2=0x7f479006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4786a00640 1 --2- 172.21.15.110:0/1252455904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 0x7f479006c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4786a00640 1 -- 172.21.15.110:0/1252455904 >> v1:172.21.15.145:6789/0 conn(0x7f47900b1c60 legacy=0x7f479015ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.301+0000 7f4786a00640 1 -- 172.21.15.110:0/1252455904 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47900b4c40 con 0x7f47900ce4d0 2024-09-10T10:05:26.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4786a00640 1 --2- 172.21.15.110:0/1252455904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900ce4d0 0x7f479006e140 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4778009870 tx=0x7f477801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c113ba64e82849b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f478ea00640 1 -- 172.21.15.110:0/1252455904 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47780b2070 con 0x7f47900ce4d0 2024-09-10T10:05:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f478ea00640 1 -- 172.21.15.110:0/1252455904 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47780a9070 con 0x7f47900ce4d0 2024-09-10T10:05:26.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f478ea00640 1 -- 172.21.15.110:0/1252455904 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47780ad880 con 0x7f47900ce4d0 2024-09-10T10:05:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4796000640 1 -- 172.21.15.110:0/1252455904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900ce4d0 msgr2=0x7f479006e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4796000640 1 --2- 172.21.15.110:0/1252455904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900ce4d0 0x7f479006e140 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4778009870 tx=0x7f477801c560 comp rx=0 tx=0).stop 2024-09-10T10:05:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4796000640 1 -- 172.21.15.110:0/1252455904 shutdown_connections 2024-09-10T10:05:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4796000640 1 --2- 172.21.15.110:0/1252455904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900ce4d0 0x7f479006e140 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4796000640 1 --2- 172.21.15.110:0/1252455904 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 0x7f479006c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4796000640 1 -- 172.21.15.110:0/1252455904 >> 172.21.15.110:0/1252455904 conn(0x7f4790062e40 msgr2=0x7f479006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4796000640 1 -- 172.21.15.110:0/1252455904 shutdown_connections 2024-09-10T10:05:26.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.305+0000 7f4796000640 1 -- 172.21.15.110:0/1252455904 wait complete. 2024-09-10T10:05:26.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 Processor -- start 2024-09-10T10:05:26.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 -- start start 2024-09-10T10:05:26.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900b1c60 0x7f479014d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 0x7f4790071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4790071c80 0x7f4790148460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f47900b3830 con 0x7f4790071c80 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f47900b36b0 con 0x7f47900b1c60 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f47900b39b0 con 0x7f479015b330 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4790071c80 0x7f4790148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4790071c80 0x7f4790148460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50378/0 (socket says 172.21.15.110:50378) 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 -- 172.21.15.110:0/164376356 learned_addr learned my addr 172.21.15.110:0/164376356 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478f400640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900b1c60 0x7f479014d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4786a00640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 0x7f4790071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 -- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 msgr2=0x7f4790071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 0x7f4790071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 -- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900b1c60 msgr2=0x7f479014d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900b1c60 0x7f479014d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 -- 172.21.15.110:0/164376356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479014f3e0 con 0x7f4790071c80 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f478fe00640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4790071c80 0x7f4790148460 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f478800c7a0 tx=0x7f478800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4787e00640 1 -- 172.21.15.110:0/164376356 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788003b80 con 0x7f4790071c80 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4787e00640 1 -- 172.21.15.110:0/164376356 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4788003d20 con 0x7f4790071c80 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479014f660 con 0x7f4790071c80 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4787e00640 1 -- 172.21.15.110:0/164376356 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478800aa30 con 0x7f4790071c80 2024-09-10T10:05:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.309+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479014fc20 con 0x7f4790071c80 2024-09-10T10:05:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.313+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4790150000 con 0x7f4790071c80 2024-09-10T10:05:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.313+0000 7f4787e00640 1 -- 172.21.15.110:0/164376356 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f478800abd0 con 0x7f4790071c80 2024-09-10T10:05:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.313+0000 7f4787e00640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f476c0680a0 0x7f476c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.313+0000 7f4787e00640 1 -- 172.21.15.110:0/164376356 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4999+0+0 (secure 0 0 0) 0x7f47880869a0 con 0x7f4790071c80 2024-09-10T10:05:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.313+0000 7f478f400640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f476c0680a0 0x7f476c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.313+0000 7f478f400640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f476c0680a0 0x7f476c06a560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4780009ac0 tx=0x7f4780009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:26.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.317+0000 7f4787e00640 1 -- 172.21.15.110:0/164376356 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4790150000 con 0x7f4790071c80 2024-09-10T10:05:26.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.477+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f479015f0a0 con 0x7f4790071c80 2024-09-10T10:05:26.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4787e00640 1 -- 172.21.15.110:0/164376356 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f479014f3e0 con 0x7f4790071c80 2024-09-10T10:05:26.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f476c0680a0 msgr2=0x7f476c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f476c0680a0 0x7f476c06a560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4780009ac0 tx=0x7f4780009190 comp rx=0 tx=0).stop 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4790071c80 msgr2=0x7f4790148460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4790071c80 0x7f4790148460 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f478800c7a0 tx=0x7f478800cc60 comp rx=0 tx=0).stop 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 shutdown_connections 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f476c0680a0 0x7f476c06a560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4790071c80 0x7f4790148460 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f479015b330 0x7f4790071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 --2- 172.21.15.110:0/164376356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47900b1c60 0x7f479014d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 >> 172.21.15.110:0/164376356 conn(0x7f4790062e40 msgr2=0x7f4790153d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 shutdown_connections 2024-09-10T10:05:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.481+0000 7f4796000640 1 -- 172.21.15.110:0/164376356 wait complete. 2024-09-10T10:05:26.504 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:05:26.504 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:05:26.599 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:05:26.703 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:05:26.704 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:26.704 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:26.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:05:26.713 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:26.713 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:26.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:05:26.722 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:26.723 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:26.764 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:05:26.764 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:26.764 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:26.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:05:26.773 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:05:26.773 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:26.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.941+0000 7f437f200640 1 Processor -- start 2024-09-10T10:05:26.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.941+0000 7f437f200640 1 -- start start 2024-09-10T10:05:26.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f437f200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f43780cfe30 0x7f4378071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f437f200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437806d2d0 con 0x7f43780cd510 2024-09-10T10:05:26.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f437f200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437806d5b0 con 0x7f437806e7a0 2024-09-10T10:05:26.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f437f200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f437815d9a0 con 0x7f43780cfe30 2024-09-10T10:05:26.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f43780cd510 0x7f437806e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55958/0 (socket says 172.21.15.110:55958) 2024-09-10T10:05:26.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377400640 1 -- 172.21.15.110:0/1502197409 learned_addr learned my addr 172.21.15.110:0/1502197409 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:26.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377e00640 1 --2- 172.21.15.110:0/1502197409 >> v2:172.21.15.145:3300/0 conn(0x7f43780cfe30 0x7f4378071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272457106 0 0) 0x7f437806d2d0 con 0x7f43780cd510 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f435c003530 con 0x7f43780cd510 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2420605708 0 0) 0x7f435c003530 con 0x7f43780cd510 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 >> v2:172.21.15.145:3300/0 conn(0x7f43780cfe30 msgr2=0x7f4378071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 --2- 172.21.15.110:0/1502197409 >> v2:172.21.15.145:3300/0 conn(0x7f43780cfe30 0x7f4378071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 >> v1:172.21.15.145:6789/0 conn(0x7f437806e7a0 legacy=0x7f437815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437806d890 con 0x7f43780cd510 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f4364001680 con 0x7f43780cd510 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 >> v1:172.21.15.110:6789/0 conn(0x7f43780cd510 legacy=0x7f437806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f435c0039c0 0x7f435c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f435c004810 0x7f435c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c0051d0 0x7f435c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f437815d9a0 con 0x7f435c004810 2024-09-10T10:05:26.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f435c00c170 con 0x7f435c0039c0 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f435c00c370 con 0x7f435c0051d0 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377400640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f435c004810 0x7f435c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377e00640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f435c0039c0 0x7f435c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f437d200640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c0051d0 0x7f435c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377400640 1 -- 172.21.15.110:0/1502197409 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c0051d0 msgr2=0x7f435c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377400640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c0051d0 0x7f435c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377400640 1 -- 172.21.15.110:0/1502197409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f435c0039c0 msgr2=0x7f435c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377400640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f435c0039c0 0x7f435c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f4377400640 1 -- 172.21.15.110:0/1502197409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c003530 con 0x7f435c004810 2024-09-10T10:05:26.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.945+0000 7f437d200640 1 -- 172.21.15.110:0/1502197409 reap_dead start 2024-09-10T10:05:26.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f4377400640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f435c004810 0x7f435c004c90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f436400c210 tx=0x7f436401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa4479d842518aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364007d40 con 0x7f435c004810 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43640a9070 con 0x7f435c004810 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f4376a00640 1 -- 172.21.15.110:0/1502197409 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43640a5dc0 con 0x7f435c004810 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 -- 172.21.15.110:0/1502197409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f435c004810 msgr2=0x7f435c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f435c004810 0x7f435c004c90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f436400c210 tx=0x7f436401cca0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 -- 172.21.15.110:0/1502197409 shutdown_connections 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 --2- 172.21.15.110:0/1502197409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f435c004810 0x7f435c004c90 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 -- 172.21.15.110:0/1502197409 >> 172.21.15.110:0/1502197409 conn(0x7f437806bc50 msgr2=0x7f437806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 -- 172.21.15.110:0/1502197409 shutdown_connections 2024-09-10T10:05:26.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 -- 172.21.15.110:0/1502197409 wait complete. 2024-09-10T10:05:26.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 Processor -- start 2024-09-10T10:05:26.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.949+0000 7f437f200640 1 -- start start 2024-09-10T10:05:26.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c004810 0x7f4378149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43781f8800 0x7f437814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437814ccf0 0x7f437814d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f437815cfb0 con 0x7f43781f8800 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f437815ce30 con 0x7f437814ccf0 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f437815d130 con 0x7f435c004810 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43781f8800 0x7f437814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43781f8800 0x7f437814dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50396/0 (socket says 172.21.15.110:50396) 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 -- 172.21.15.110:0/1235047910 learned_addr learned my addr 172.21.15.110:0/1235047910 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377e00640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c004810 0x7f4378149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437d200640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437814ccf0 0x7f437814d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 -- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c004810 msgr2=0x7f4378149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c004810 0x7f4378149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 -- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437814ccf0 msgr2=0x7f437814d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437814ccf0 0x7f437814d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 -- 172.21.15.110:0/1235047910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437814a340 con 0x7f43781f8800 2024-09-10T10:05:26.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377e00640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c004810 0x7f4378149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:26.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437d200640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437814ccf0 0x7f437814d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:26.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f4377400640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43781f8800 0x7f437814dcd0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f436401cad0 tx=0x7f435c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:26.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f436fe00640 1 -- 172.21.15.110:0/1235047910 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43640bc070 con 0x7f43781f8800 2024-09-10T10:05:26.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f436fe00640 1 -- 172.21.15.110:0/1235047910 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4364000ee0 con 0x7f43781f8800 2024-09-10T10:05:26.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437814e5d0 con 0x7f43781f8800 2024-09-10T10:05:26.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f436fe00640 1 -- 172.21.15.110:0/1235047910 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364007380 con 0x7f43781f8800 2024-09-10T10:05:26.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437814eb90 con 0x7f43781f8800 2024-09-10T10:05:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437815d4c0 con 0x7f43781f8800 2024-09-10T10:05:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.953+0000 7f436fe00640 1 -- 172.21.15.110:0/1235047910 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f43640bf070 con 0x7f43781f8800 2024-09-10T10:05:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.957+0000 7f436fe00640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f43580680d0 0x7f435806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.957+0000 7f436fe00640 1 -- 172.21.15.110:0/1235047910 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4999+0+0 (secure 0 0 0) 0x7f436411ffd0 con 0x7f43781f8800 2024-09-10T10:05:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.957+0000 7f4377e00640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f43580680d0 0x7f435806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:26.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.957+0000 7f4377e00640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f43580680d0 0x7f435806a590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f43780b5220 tx=0x7f436800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:26.961+0000 7f436fe00640 1 -- 172.21.15.110:0/1235047910 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f43640169b0 con 0x7f43781f8800 2024-09-10T10:05:27.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.149+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437815dbf0 con 0x7f43781f8800 2024-09-10T10:05:27.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.149+0000 7f436fe00640 1 -- 172.21.15.110:0/1235047910 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+3925 (secure 0 0 0) 0x7f43640f0250 con 0x7f43781f8800 2024-09-10T10:05:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:27.151 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":100,"btime":"2024-09-10T10:05:21:111615+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2050751760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":2050751760},{"type":"v1","addr":"172.21.15.110:6835","nonce":2050751760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6743,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/4094794865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":4094794865},{"type":"v1","addr":"172.21.15.145:6834","nonce":4094794865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6938,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1673505911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1673505911},{"type":"v1","addr":"172.21.15.145:6835","nonce":1673505911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2883879805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2883879805},{"type":"v1","addr":"172.21.15.110:6834","nonce":2883879805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[]} 2024-09-10T10:05:27.151 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 100 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f43580680d0 msgr2=0x7f435806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f43580680d0 0x7f435806a590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f43780b5220 tx=0x7f436800b040 comp rx=0 tx=0).stop 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43781f8800 msgr2=0x7f437814dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43781f8800 0x7f437814dcd0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f436401cad0 tx=0x7f435c005840 comp rx=0 tx=0).stop 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 shutdown_connections 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f43580680d0 0x7f435806a590 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f437814ccf0 0x7f437814d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43781f8800 0x7f437814dcd0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 --2- 172.21.15.110:0/1235047910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f435c004810 0x7f4378149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 >> 172.21.15.110:0/1235047910 conn(0x7f437806bc50 msgr2=0x7f4378072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 shutdown_connections 2024-09-10T10:05:27.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.153+0000 7f437f200640 1 -- 172.21.15.110:0/1235047910 wait complete. 2024-09-10T10:05:27.171 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:05:27.172 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:05:27.172 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:05:27.172 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:05:27.172 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:05:27.172 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:05:27.172 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:05:27.173 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:05:27.173 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:05:27.173 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:05:27.173 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:05:27.173 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:05:27.173 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:05:27.173 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:05:27.174 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:05:27.174 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:05:27.186 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:05:27.186 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:05:27.187 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:05:27.187 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:05:27.188 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:05:27.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:05:27.189 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:05:27.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:05:27.235 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:05:27.235 DEBUG:teuthology.parallel:result is None 2024-09-10T10:05:27.236 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:05:27.236 DEBUG:teuthology.parallel:result is None 2024-09-10T10:05:27.238 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:05:27.239 DEBUG:teuthology.parallel:result is None 2024-09-10T10:05:27.240 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:05:27.241 DEBUG:teuthology.parallel:result is None 2024-09-10T10:05:27.241 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:05:27.321 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:05:27.321 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:05:27.329 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:05:27.329 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:05:27.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedfe00640 1 Processor -- start 2024-09-10T10:05:27.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedfe00640 1 -- start start 2024-09-10T10:05:27.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fded815c570 0x7fded80d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedfe00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded80d2d10 con 0x7fded80d1650 2024-09-10T10:05:27.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedfe00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded80d2ff0 con 0x7fded8073910 2024-09-10T10:05:27.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedfe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fded80d0630 con 0x7fded815c570 2024-09-10T10:05:27.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedca00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fded80d1650 0x7fded8073200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:55964/0 (socket says 172.21.15.110:55964) 2024-09-10T10:05:27.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedca00640 1 -- 172.21.15.110:0/257942700 learned_addr learned my addr 172.21.15.110:0/257942700 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:27.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fdedd400640 1 --2- 172.21.15.110:0/257942700 >> v2:172.21.15.145:3300/0 conn(0x7fded815c570 0x7fded80d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450893518 0 0) 0x7fded80d2d10 con 0x7fded80d1650 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec0003530 con 0x7fded80d1650 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1745936816 0 0) 0x7fdec0003530 con 0x7fded80d1650 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 >> v2:172.21.15.145:3300/0 conn(0x7fded815c570 msgr2=0x7fded80d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fded7e00640 1 --2- 172.21.15.110:0/257942700 >> v2:172.21.15.145:3300/0 conn(0x7fded815c570 0x7fded80d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.385+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 >> v1:172.21.15.145:6789/0 conn(0x7fded8073910 legacy=0x7fded80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded80d0910 con 0x7fded80d1650 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fdec4001680 con 0x7fded80d1650 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 >> v1:172.21.15.110:6789/0 conn(0x7fded80d1650 legacy=0x7fded8073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00039c0 0x7fdec0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdec00043c0 0x7fdec0004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdec0005090 0x7fdec000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fded80d0630 con 0x7fdec00043c0 2024-09-10T10:05:27.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdec000bff0 con 0x7fdec0005090 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdec000c1f0 con 0x7fdec00039c0 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedca00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdec00043c0 0x7fdec0004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedd400640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00039c0 0x7fdec0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedde00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdec0005090 0x7fdec000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedca00640 1 -- 172.21.15.110:0/257942700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00039c0 msgr2=0x7fdec0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedca00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00039c0 0x7fdec0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedca00640 1 -- 172.21.15.110:0/257942700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdec0005090 msgr2=0x7fdec000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedca00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdec0005090 0x7fdec000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedca00640 1 -- 172.21.15.110:0/257942700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec0003530 con 0x7fdec00043c0 2024-09-10T10:05:27.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedde00640 1 -- 172.21.15.110:0/257942700 reap_dead start 2024-09-10T10:05:27.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fdedca00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdec00043c0 0x7fdec0004890 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c240 tx=0x7fdec401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704e3dfc8057dbee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:27.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.389+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4007d40 con 0x7fdec00043c0 2024-09-10T10:05:27.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec40a9070 con 0x7fdec00043c0 2024-09-10T10:05:27.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fded7e00640 1 -- 172.21.15.110:0/257942700 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4005d80 con 0x7fdec00043c0 2024-09-10T10:05:27.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- 172.21.15.110:0/257942700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdec00043c0 msgr2=0x7fdec0004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdec00043c0 0x7fdec0004890 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c240 tx=0x7fdec401cc50 comp rx=0 tx=0).stop 2024-09-10T10:05:27.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- 172.21.15.110:0/257942700 shutdown_connections 2024-09-10T10:05:27.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 --2- 172.21.15.110:0/257942700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdec00043c0 0x7fdec0004890 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- 172.21.15.110:0/257942700 >> 172.21.15.110:0/257942700 conn(0x7fded806f440 msgr2=0x7fded806f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:27.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- 172.21.15.110:0/257942700 shutdown_connections 2024-09-10T10:05:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- 172.21.15.110:0/257942700 wait complete. 2024-09-10T10:05:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 Processor -- start 2024-09-10T10:05:27.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- start start 2024-09-10T10:05:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00043c0 0x7fded815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fded8157330 0x7fded81577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fded81548b0 0x7fded8154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fded815cad0 con 0x7fded8157330 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fded815c950 con 0x7fded81548b0 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedfe00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fded80d1110 con 0x7fdec00043c0 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fded8157330 0x7fded81577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fded8157330 0x7fded81577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50436/0 (socket says 172.21.15.110:50436) 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedca00640 1 -- 172.21.15.110:0/302599566 learned_addr learned my addr 172.21.15.110:0/302599566 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedd400640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00043c0 0x7fded815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.393+0000 7fdedde00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fded81548b0 0x7fded8154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedd400640 1 -- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fded81548b0 msgr2=0x7fded8154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedd400640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fded81548b0 0x7fded8154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedd400640 1 -- 172.21.15.110:0/302599566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fded8157330 msgr2=0x7fded81577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedd400640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fded8157330 0x7fded81577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedd400640 1 -- 172.21.15.110:0/302599566 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded8155ee0 con 0x7fdec00043c0 2024-09-10T10:05:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedca00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fded8157330 0x7fded81577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedd400640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00043c0 0x7fded815ac30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fded80d34c0 tx=0x7fdecc011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fded5600640 1 -- 172.21.15.110:0/302599566 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc019070 con 0x7fdec00043c0 2024-09-10T10:05:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fded5600640 1 -- 172.21.15.110:0/302599566 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdecc004240 con 0x7fdec00043c0 2024-09-10T10:05:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fded5600640 1 -- 172.21.15.110:0/302599566 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00f7c0 con 0x7fdec00043c0 2024-09-10T10:05:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded81561c0 con 0x7fdec00043c0 2024-09-10T10:05:27.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fded8156780 con 0x7fdec00043c0 2024-09-10T10:05:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded80d2ff0 con 0x7fdec00043c0 2024-09-10T10:05:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fded5600640 1 -- 172.21.15.110:0/302599566 <== mon.2 v2:172.21.15.145:3301/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fdecc01c070 con 0x7fdec00043c0 2024-09-10T10:05:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fded5600640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdeb80680a0 0x7fdeb806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.397+0000 7fded5600640 1 -- 172.21.15.110:0/302599566 <== mon.2 v2:172.21.15.145:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 4999+0+0 (secure 0 0 0) 0x7fdecc086f70 con 0x7fdec00043c0 2024-09-10T10:05:27.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.401+0000 7fdedca00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdeb80680a0 0x7fdeb806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.401+0000 7fdedca00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdeb80680a0 0x7fdeb806a560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c240 tx=0x7fdec40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:27.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.405+0000 7fded5600640 1 -- 172.21.15.110:0/302599566 <== mon.2 v2:172.21.15.145:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdecc056790 con 0x7fdec00043c0 2024-09-10T10:05:27.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:27.573+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fded80b6d10 con 0x7fdec00043c0 2024-09-10T10:05:28.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.157+0000 7fded5600640 1 -- 172.21.15.110:0/302599566 <== mon.2 v2:172.21.15.145:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v147) ==== 97+0+0 (secure 0 0 0) 0x7fdecc05aa90 con 0x7fdec00043c0 2024-09-10T10:05:28.159 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:05:28.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdeb80680a0 msgr2=0x7fdeb806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdeb80680a0 0x7fdeb806a560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c240 tx=0x7fdec40a7070 comp rx=0 tx=0).stop 2024-09-10T10:05:28.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00043c0 msgr2=0x7fded815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00043c0 0x7fded815ac30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fded80d34c0 tx=0x7fdecc011cf0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 shutdown_connections 2024-09-10T10:05:28.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdeb80680a0 0x7fdeb806a560 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fded81548b0 0x7fded8154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fded8157330 0x7fded81577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 --2- 172.21.15.110:0/302599566 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdec00043c0 0x7fded815ac30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 >> 172.21.15.110:0/302599566 conn(0x7fded806f440 msgr2=0x7fded80cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:28.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 shutdown_connections 2024-09-10T10:05:28.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.161+0000 7fdedfe00640 1 -- 172.21.15.110:0/302599566 wait complete. 2024-09-10T10:05:28.184 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:05:28.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69e5000640 1 Processor -- start 2024-09-10T10:05:28.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69e5000640 1 -- start start 2024-09-10T10:05:28.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69e5000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69e00cfe30 0x7f69e0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69e5000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cd510 0x7f69e00cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69e5000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e006d2d0 con 0x7f69e006e090 2024-09-10T10:05:28.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69e5000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f69e00cdeb0 con 0x7f69e00cfe30 2024-09-10T10:05:28.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69e5000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f69e015d9a0 con 0x7f69e00cd510 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dd400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cd510 0x7f69e00cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dd400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cd510 0x7f69e00cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:49818/0 (socket says 172.21.15.110:49818) 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dd400640 1 -- 172.21.15.110:0/2972490191 learned_addr learned my addr 172.21.15.110:0/2972490191 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69de800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f69e006e090 0x7f69e006e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:39804/0 (socket says 172.21.15.110:39804) 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dde00640 1 --2- 172.21.15.110:0/2972490191 >> v2:172.21.15.145:3300/0 conn(0x7f69e00cfe30 0x7f69e0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dca00640 1 -- 172.21.15.110:0/2972490191 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1175645783 0 0) 0x7f69e006d2d0 con 0x7f69e006e090 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dca00640 1 -- 172.21.15.110:0/2972490191 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69c4003530 con 0x7f69e006e090 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dde00640 1 -- 172.21.15.110:0/2972490191 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cd510 msgr2=0x7f69e00cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dde00640 1 --2- 172.21.15.110:0/2972490191 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cd510 0x7f69e00cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dde00640 1 -- 172.21.15.110:0/2972490191 >> v1:172.21.15.145:6789/0 conn(0x7f69e006e090 legacy=0x7f69e006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dde00640 1 -- 172.21.15.110:0/2972490191 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e006d5b0 con 0x7f69e00cfe30 2024-09-10T10:05:28.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.321+0000 7f69dd400640 1 --2- 172.21.15.110:0/2972490191 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cd510 0x7f69e00cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:28.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69dde00640 1 --2- 172.21.15.110:0/2972490191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cfe30 0x7f69e0071a40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f69d00097c0 tx=0x7f69d001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=251e6bb84beaf48f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:28.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69dca00640 1 -- 172.21.15.110:0/2972490191 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d00b0070 con 0x7f69e00cfe30 2024-09-10T10:05:28.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69dca00640 1 -- 172.21.15.110:0/2972490191 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d00ab600 con 0x7f69e00cfe30 2024-09-10T10:05:28.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69dca00640 1 -- 172.21.15.110:0/2972490191 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d00ab9d0 con 0x7f69e00cfe30 2024-09-10T10:05:28.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69e5000640 1 -- 172.21.15.110:0/2972490191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cfe30 msgr2=0x7f69e0071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69e5000640 1 --2- 172.21.15.110:0/2972490191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cfe30 0x7f69e0071a40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f69d00097c0 tx=0x7f69d001c410 comp rx=0 tx=0).stop 2024-09-10T10:05:28.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69e5000640 1 -- 172.21.15.110:0/2972490191 shutdown_connections 2024-09-10T10:05:28.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69e5000640 1 --2- 172.21.15.110:0/2972490191 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cd510 0x7f69e00cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69e5000640 1 --2- 172.21.15.110:0/2972490191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cfe30 0x7f69e0071a40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69e5000640 1 -- 172.21.15.110:0/2972490191 >> 172.21.15.110:0/2972490191 conn(0x7f69e006bc50 msgr2=0x7f69e006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:28.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69e5000640 1 -- 172.21.15.110:0/2972490191 shutdown_connections 2024-09-10T10:05:28.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.325+0000 7f69e5000640 1 -- 172.21.15.110:0/2972490191 wait complete. 2024-09-10T10:05:28.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 Processor -- start 2024-09-10T10:05:28.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 -- start start 2024-09-10T10:05:28.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69e006e090 0x7f69e0158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cd510 0x7f69e01faae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cfe30 0x7f69e01fb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0072190 con 0x7f69e006e090 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f69e015d640 con 0x7f69e00cd510 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f69e0169180 con 0x7f69e00cfe30 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69e006e090 0x7f69e0158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69e006e090 0x7f69e0158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34092/0 (socket says 172.21.15.110:34092) 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 -- 172.21.15.110:0/387561353 learned_addr learned my addr 172.21.15.110:0/387561353 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dd400640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cd510 0x7f69e01faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69de800640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cfe30 0x7f69e01fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 -- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cfe30 msgr2=0x7f69e01fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cfe30 0x7f69e01fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 -- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cd510 msgr2=0x7f69e01faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cd510 0x7f69e01faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 -- 172.21.15.110:0/387561353 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e015b600 con 0x7f69e006e090 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69de800640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cfe30 0x7f69e01fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69dde00640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69e006e090 0x7f69e0158fe0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f69d001a040 tx=0x7f69d001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69d6000640 1 -- 172.21.15.110:0/387561353 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d00b0050 con 0x7f69e006e090 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69d6000640 1 -- 172.21.15.110:0/387561353 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d00bc070 con 0x7f69e006e090 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69d6000640 1 -- 172.21.15.110:0/387561353 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0006e20 con 0x7f69e006e090 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e015b8e0 con 0x7f69e006e090 2024-09-10T10:05:28.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e0157490 con 0x7f69e006e090 2024-09-10T10:05:28.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.329+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e00cd970 con 0x7f69e006e090 2024-09-10T10:05:28.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.333+0000 7f69d6000640 1 -- 172.21.15.110:0/387561353 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f69d00b5830 con 0x7f69e006e090 2024-09-10T10:05:28.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.333+0000 7f69d6000640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69c00680a0 0x7f69c006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.333+0000 7f69d6000640 1 -- 172.21.15.110:0/387561353 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4913+0+0 (secure 0 0 0) 0x7f69d0120e10 con 0x7f69e006e090 2024-09-10T10:05:28.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.333+0000 7f69dd400640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69c00680a0 0x7f69c006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.333+0000 7f69dd400640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69c00680a0 0x7f69c006a560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f69e006d490 tx=0x7f69cc009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:28.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.337+0000 7f69d6000640 1 -- 172.21.15.110:0/387561353 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f69d0016090 con 0x7f69e006e090 2024-09-10T10:05:28.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.549+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f69e0071d60 con 0x7f69e006e090 2024-09-10T10:05:28.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.549+0000 7f69d6000640 1 -- 172.21.15.110:0/387561353 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f69d00f0f40 con 0x7f69e006e090 2024-09-10T10:05:28.551 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:28.551 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:05:28.551 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.552 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.553 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.556 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:05:28.558 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.560 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:05:28.561 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.562 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.566 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:05:28.566 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:05:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69c00680a0 msgr2=0x7f69c006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69c00680a0 0x7f69c006a560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f69e006d490 tx=0x7f69cc009240 comp rx=0 tx=0).stop 2024-09-10T10:05:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69e006e090 msgr2=0x7f69e0158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69e006e090 0x7f69e0158fe0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f69d001a040 tx=0x7f69d001ac50 comp rx=0 tx=0).stop 2024-09-10T10:05:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 shutdown_connections 2024-09-10T10:05:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f69c00680a0 0x7f69c006a560 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f69e00cfe30 0x7f69e01fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69e00cd510 0x7f69e01faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 --2- 172.21.15.110:0/387561353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69e006e090 0x7f69e0158fe0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 >> 172.21.15.110:0/387561353 conn(0x7f69e006bc50 msgr2=0x7f69e015e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:28.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 shutdown_connections 2024-09-10T10:05:28.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.553+0000 7f69e5000640 1 -- 172.21.15.110:0/387561353 wait complete. 2024-09-10T10:05:28.572 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:05:28.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfef600640 1 Processor -- start 2024-09-10T10:05:28.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfef600640 1 -- start start 2024-09-10T10:05:28.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfef600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcfe815c960 0x7fcfe80b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfef600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe80706a0 con 0x7fcfe80d13e0 2024-09-10T10:05:28.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfef600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe8070a80 con 0x7fcfe81603c0 2024-09-10T10:05:28.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfef600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe815cd40 con 0x7fcfe815c960 2024-09-10T10:05:28.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfecc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcfe815c960 0x7fcfe80b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfecc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcfe815c960 0x7fcfe80b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34098/0 (socket says 172.21.15.110:34098) 2024-09-10T10:05:28.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfecc00640 1 -- 172.21.15.110:0/3345974050 learned_addr learned my addr 172.21.15.110:0/3345974050 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:28.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfecc00640 1 -- 172.21.15.110:0/3345974050 >> v1:172.21.15.145:6789/0 conn(0x7fcfe81603c0 legacy=0x7fcfe80d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:28.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfecc00640 1 -- 172.21.15.110:0/3345974050 >> v1:172.21.15.110:6789/0 conn(0x7fcfe80d13e0 legacy=0x7fcfe8070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfe7400640 1 -- 172.21.15.110:0/3345974050 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141916091 0 0) 0x7fcfe80706a0 con 0x7fcfe80d13e0 2024-09-10T10:05:28.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.709+0000 7fcfecc00640 1 -- 172.21.15.110:0/3345974050 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe8160910 con 0x7fcfe815c960 2024-09-10T10:05:28.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfecc00640 1 --2- 172.21.15.110:0/3345974050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe80b5e30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009870 tx=0x7fcfdc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314aa486f1c559e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:28.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfe7400640 1 -- 172.21.15.110:0/3345974050 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0b0070 con 0x7fcfe815c960 2024-09-10T10:05:28.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfe7400640 1 -- 172.21.15.110:0/3345974050 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfdc0ab600 con 0x7fcfe815c960 2024-09-10T10:05:28.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfe7400640 1 -- 172.21.15.110:0/3345974050 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0ab9d0 con 0x7fcfe815c960 2024-09-10T10:05:28.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 -- 172.21.15.110:0/3345974050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 msgr2=0x7fcfe80b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 --2- 172.21.15.110:0/3345974050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe80b5e30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009870 tx=0x7fcfdc01c400 comp rx=0 tx=0).stop 2024-09-10T10:05:28.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 -- 172.21.15.110:0/3345974050 shutdown_connections 2024-09-10T10:05:28.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 --2- 172.21.15.110:0/3345974050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe80b5e30 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 -- 172.21.15.110:0/3345974050 >> 172.21.15.110:0/3345974050 conn(0x7fcfe80d25c0 msgr2=0x7fcfe80d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:28.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 -- 172.21.15.110:0/3345974050 shutdown_connections 2024-09-10T10:05:28.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 -- 172.21.15.110:0/3345974050 wait complete. 2024-09-10T10:05:28.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 Processor -- start 2024-09-10T10:05:28.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.713+0000 7fcfef600640 1 -- start start 2024-09-10T10:05:28.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfe80d13e0 0x7fcfe815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe8150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfe81603c0 0x7fcfe8157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80d32d0 con 0x7fcfe815c960 2024-09-10T10:05:28.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80d3150 con 0x7fcfe80d13e0 2024-09-10T10:05:28.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe8071a80 con 0x7fcfe81603c0 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe8150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe8150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34104/0 (socket says 172.21.15.110:34104) 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 -- 172.21.15.110:0/2019146630 learned_addr learned my addr 172.21.15.110:0/2019146630 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 -- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfe81603c0 msgr2=0x7fcfe8157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfe81603c0 0x7fcfe8157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 -- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfe80d13e0 msgr2=0x7fcfe815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfe80d13e0 0x7fcfe815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 -- 172.21.15.110:0/2019146630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe815bb00 con 0x7fcfe815c960 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe7e00640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe8150120 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b3e0 tx=0x7fcfd400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe4c00640 1 -- 172.21.15.110:0/2019146630 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4015020 con 0x7fcfe815c960 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe4c00640 1 -- 172.21.15.110:0/2019146630 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd4011830 con 0x7fcfe815c960 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe4c00640 1 -- 172.21.15.110:0/2019146630 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4011c20 con 0x7fcfe815c960 2024-09-10T10:05:28.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe8151930 con 0x7fcfe815c960 2024-09-10T10:05:28.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe8167630 con 0x7fcfe815c960 2024-09-10T10:05:28.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe80b5e30 con 0x7fcfe815c960 2024-09-10T10:05:28.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.717+0000 7fcfe4c00640 1 -- 172.21.15.110:0/2019146630 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fcfd4018020 con 0x7fcfe815c960 2024-09-10T10:05:28.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.721+0000 7fcfe4c00640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfc80680d0 0x7fcfc806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:28.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.721+0000 7fcfe4c00640 1 -- 172.21.15.110:0/2019146630 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4913+0+0 (secure 0 0 0) 0x7fcfd4086820 con 0x7fcfe815c960 2024-09-10T10:05:28.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.721+0000 7fcfecc00640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfc80680d0 0x7fcfc806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:28.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.721+0000 7fcfecc00640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfc80680d0 0x7fcfc806a590 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0ad000 tx=0x7fcfdc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:28.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.725+0000 7fcfe4c00640 1 -- 172.21.15.110:0/2019146630 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcfd401f030 con 0x7fcfe815c960 2024-09-10T10:05:28.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.881+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcfe80d34d0 con 0x7fcfe815c960 2024-09-10T10:05:28.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.881+0000 7fcfe4c00640 1 -- 172.21.15.110:0/2019146630 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v149) ==== 84+0+65 (secure 0 0 0) 0x7fcfd40172c0 con 0x7fcfe815c960 2024-09-10T10:05:28.884 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:28.884 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:05:28.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:28.884 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:05:28.884 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:05:28.885 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:28.885 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfc80680d0 msgr2=0x7fcfc806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfc80680d0 0x7fcfc806a590 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0ad000 tx=0x7fcfdc008040 comp rx=0 tx=0).stop 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 msgr2=0x7fcfe8150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe8150120 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b3e0 tx=0x7fcfd400b8a0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 shutdown_connections 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcfc80680d0 0x7fcfc806a590 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcfe81603c0 0x7fcfe8157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe815c960 0x7fcfe8150120 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 --2- 172.21.15.110:0/2019146630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfe80d13e0 0x7fcfe815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 >> 172.21.15.110:0/2019146630 conn(0x7fcfe80d25c0 msgr2=0x7fcfe81566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 shutdown_connections 2024-09-10T10:05:28.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:28.885+0000 7fcfef600640 1 -- 172.21.15.110:0/2019146630 wait complete. 2024-09-10T10:05:28.904 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:05:28.905 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:05:29.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.041+0000 7f8c8aa00640 1 Processor -- start 2024-09-10T10:05:29.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.041+0000 7f8c8aa00640 1 -- start start 2024-09-10T10:05:29.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c8aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8c8415e620 0x7f8c840cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c8aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b54f0 0x7f8c840b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c8aa00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8406d060 con 0x7f8c840b5930 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c8aa00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f8c840d1e90 con 0x7f8c8415e620 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c8aa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8406ce10 con 0x7f8c840b54f0 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c89600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8c8415e620 0x7f8c840cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c89600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8c8415e620 0x7f8c840cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34108/0 (socket says 172.21.15.110:34108) 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c89600640 1 -- 172.21.15.110:0/1807109076 learned_addr learned my addr 172.21.15.110:0/1807109076 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c88c00640 1 --2- 172.21.15.110:0/1807109076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b54f0 0x7f8c840b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c89600640 1 -- 172.21.15.110:0/1807109076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b54f0 msgr2=0x7f8c840b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c89600640 1 --2- 172.21.15.110:0/1807109076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b54f0 0x7f8c840b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c89600640 1 -- 172.21.15.110:0/1807109076 >> v1:172.21.15.145:6789/0 conn(0x7f8c840b5930 legacy=0x7f8c8406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:29.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.045+0000 7f8c89600640 1 -- 172.21.15.110:0/1807109076 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8406d340 con 0x7f8c8415e620 2024-09-10T10:05:29.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c89600640 1 --2- 172.21.15.110:0/1807109076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c8415e620 0x7f8c840cf1b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009870 tx=0x7f8c7801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e433908bd9e2e709 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:29.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c83e00640 1 -- 172.21.15.110:0/1807109076 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c780b0070 con 0x7f8c8415e620 2024-09-10T10:05:29.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c83e00640 1 -- 172.21.15.110:0/1807109076 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c780ab600 con 0x7f8c8415e620 2024-09-10T10:05:29.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c83e00640 1 -- 172.21.15.110:0/1807109076 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c780ab9d0 con 0x7f8c8415e620 2024-09-10T10:05:29.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- 172.21.15.110:0/1807109076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c8415e620 msgr2=0x7f8c840cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:29.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/1807109076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c8415e620 0x7f8c840cf1b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009870 tx=0x7f8c7801c490 comp rx=0 tx=0).stop 2024-09-10T10:05:29.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- 172.21.15.110:0/1807109076 shutdown_connections 2024-09-10T10:05:29.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/1807109076 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b54f0 0x7f8c840b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:29.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/1807109076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c8415e620 0x7f8c840cf1b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:29.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- 172.21.15.110:0/1807109076 >> 172.21.15.110:0/1807109076 conn(0x7f8c8406f440 msgr2=0x7f8c8406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:29.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- 172.21.15.110:0/1807109076 shutdown_connections 2024-09-10T10:05:29.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- 172.21.15.110:0/1807109076 wait complete. 2024-09-10T10:05:29.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 Processor -- start 2024-09-10T10:05:29.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- start start 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c840b54f0 0x7f8c8413ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b5930 0x7f8c841512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c8415e620 0x7f8c84155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8416b2f0 con 0x7f8c840b54f0 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c840b6950 con 0x7f8c8415e620 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c8aa00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8416b470 con 0x7f8c840b5930 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.049+0000 7f8c89600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c840b54f0 0x7f8c8413ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c89600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c840b54f0 0x7f8c8413ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34116/0 (socket says 172.21.15.110:34116) 2024-09-10T10:05:29.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c89600640 1 -- 172.21.15.110:0/742455585 learned_addr learned my addr 172.21.15.110:0/742455585 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c88c00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b5930 0x7f8c841512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c8a000640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c8415e620 0x7f8c84155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c89600640 1 -- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b5930 msgr2=0x7f8c841512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c89600640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b5930 0x7f8c841512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c89600640 1 -- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c8415e620 msgr2=0x7f8c84155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c89600640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c8415e620 0x7f8c84155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c89600640 1 -- 172.21.15.110:0/742455585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c84151d70 con 0x7f8c840b54f0 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c88c00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b5930 0x7f8c841512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c8a000640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c8415e620 0x7f8c84155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:29.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c89600640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c840b54f0 0x7f8c8413ddd0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8c7801ab40 tx=0x7f8c7801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:29.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c81600640 1 -- 172.21.15.110:0/742455585 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c780b0040 con 0x7f8c840b54f0 2024-09-10T10:05:29.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c81600640 1 -- 172.21.15.110:0/742455585 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c780bc070 con 0x7f8c840b54f0 2024-09-10T10:05:29.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c84152050 con 0x7f8c840b54f0 2024-09-10T10:05:29.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c81600640 1 -- 172.21.15.110:0/742455585 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c780b6880 con 0x7f8c840b54f0 2024-09-10T10:05:29.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c84152610 con 0x7f8c840b54f0 2024-09-10T10:05:29.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c840b6000 con 0x7f8c840b54f0 2024-09-10T10:05:29.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.053+0000 7f8c81600640 1 -- 172.21.15.110:0/742455585 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f8c780ab480 con 0x7f8c840b54f0 2024-09-10T10:05:29.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.057+0000 7f8c81600640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8c640680d0 0x7f8c6406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:29.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.057+0000 7f8c81600640 1 -- 172.21.15.110:0/742455585 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4913+0+0 (secure 0 0 0) 0x7f8c78120d00 con 0x7f8c840b54f0 2024-09-10T10:05:29.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.057+0000 7f8c88c00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8c640680d0 0x7f8c6406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:29.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.057+0000 7f8c88c00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8c640680d0 0x7f8c6406a590 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8c84155f30 tx=0x7f8c70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:29.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.061+0000 7f8c81600640 1 -- 172.21.15.110:0/742455585 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8c78016110 con 0x7f8c840b54f0 2024-09-10T10:05:29.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:29.221+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8c84069570 con 0x7f8c840b54f0 2024-09-10T10:05:30.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.157+0000 7f8c81600640 1 -- 172.21.15.110:0/742455585 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v150) ==== 138+0+0 (secure 0 0 0) 0x7f8c780f0ec0 con 0x7f8c840b54f0 2024-09-10T10:05:30.157 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:05:30.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.157+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8c640680d0 msgr2=0x7f8c6406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:30.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.157+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8c640680d0 0x7f8c6406a590 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8c84155f30 tx=0x7f8c70008040 comp rx=0 tx=0).stop 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.157+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c840b54f0 msgr2=0x7f8c8413ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.157+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c840b54f0 0x7f8c8413ddd0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8c7801ab40 tx=0x7f8c7801acd0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.161+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 shutdown_connections 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.161+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8c640680d0 0x7f8c6406a590 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.161+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c8415e620 0x7f8c84155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.161+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8c840b5930 0x7f8c841512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.161+0000 7f8c8aa00640 1 --2- 172.21.15.110:0/742455585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c840b54f0 0x7f8c8413ddd0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.161+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 >> 172.21.15.110:0/742455585 conn(0x7f8c8406f440 msgr2=0x7f8c840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.161+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 shutdown_connections 2024-09-10T10:05:30.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.161+0000 7f8c8aa00640 1 -- 172.21.15.110:0/742455585 wait complete. 2024-09-10T10:05:30.188 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:05:30.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d8e200640 1 Processor -- start 2024-09-10T10:05:30.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d8e200640 1 -- start start 2024-09-10T10:05:30.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d8e200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4d88155970 0x7f4d880cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d8e200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4d8806f1e0 0x7f4d880ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d8e200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d88155d50 con 0x7f4d8815c560 2024-09-10T10:05:30.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d8e200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f4d880b4570 con 0x7f4d88155970 2024-09-10T10:05:30.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d8e200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4d880b34d0 con 0x7f4d8806f1e0 2024-09-10T10:05:30.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d87400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4d88155970 0x7f4d880cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.333+0000 7f4d87400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f4d88155970 0x7f4d880cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34120/0 (socket says 172.21.15.110:34120) 2024-09-10T10:05:30.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d87400640 1 -- 172.21.15.110:0/154647098 learned_addr learned my addr 172.21.15.110:0/154647098 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:30.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d87400640 1 -- 172.21.15.110:0/154647098 >> v2:172.21.15.145:3300/0 conn(0x7f4d8806f1e0 msgr2=0x7f4d880ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:30.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d87400640 1 --2- 172.21.15.110:0/154647098 >> v2:172.21.15.145:3300/0 conn(0x7f4d8806f1e0 0x7f4d880ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d87400640 1 -- 172.21.15.110:0/154647098 >> v1:172.21.15.145:6789/0 conn(0x7f4d8815c560 legacy=0x7f4d8806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:30.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d87400640 1 -- 172.21.15.110:0/154647098 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d880b37b0 con 0x7f4d88155970 2024-09-10T10:05:30.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d87400640 1 --2- 172.21.15.110:0/154647098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d88155970 0x7f4d880cc8c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4d880735e0 tx=0x7f4d7801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3d598fd890f345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:30.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d86000640 1 -- 172.21.15.110:0/154647098 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d780aa610 con 0x7f4d88155970 2024-09-10T10:05:30.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d86000640 1 -- 172.21.15.110:0/154647098 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d780af070 con 0x7f4d88155970 2024-09-10T10:05:30.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d86000640 1 -- 172.21.15.110:0/154647098 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d780aa9d0 con 0x7f4d88155970 2024-09-10T10:05:30.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d8e200640 1 -- 172.21.15.110:0/154647098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d88155970 msgr2=0x7f4d880cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:30.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.337+0000 7f4d8e200640 1 --2- 172.21.15.110:0/154647098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d88155970 0x7f4d880cc8c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4d880735e0 tx=0x7f4d7801c4b0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 -- 172.21.15.110:0/154647098 shutdown_connections 2024-09-10T10:05:30.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 --2- 172.21.15.110:0/154647098 >> v2:172.21.15.145:3300/0 conn(0x7f4d8806f1e0 0x7f4d880ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 --2- 172.21.15.110:0/154647098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d88155970 0x7f4d880cc8c0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 -- 172.21.15.110:0/154647098 >> 172.21.15.110:0/154647098 conn(0x7f4d8806bc50 msgr2=0x7f4d8806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:30.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 -- 172.21.15.110:0/154647098 shutdown_connections 2024-09-10T10:05:30.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 -- 172.21.15.110:0/154647098 wait complete. 2024-09-10T10:05:30.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 Processor -- start 2024-09-10T10:05:30.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 -- start start 2024-09-10T10:05:30.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8806f1e0 0x7f4d880aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d88155970 0x7f4d88147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d8815c560 0x7f4d88147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d880cf170 con 0x7f4d8815c560 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4d880cce00 con 0x7f4d8806f1e0 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d8e200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4d880cf2f0 con 0x7f4d88155970 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d8815c560 0x7f4d88147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d8815c560 0x7f4d88147eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34126/0 (socket says 172.21.15.110:34126) 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 -- 172.21.15.110:0/1620842613 learned_addr learned my addr 172.21.15.110:0/1620842613 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87400640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8806f1e0 0x7f4d880aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d86a00640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d88155970 0x7f4d88147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 -- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d88155970 msgr2=0x7f4d88147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d88155970 0x7f4d88147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 -- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8806f1e0 msgr2=0x7f4d880aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8806f1e0 0x7f4d880aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 -- 172.21.15.110:0/1620842613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d88159100 con 0x7f4d8815c560 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d86a00640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d88155970 0x7f4d88147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.341+0000 7f4d87e00640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d8815c560 0x7f4d88147eb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000b9a0 tx=0x7f4d8000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d7f400640 1 -- 172.21.15.110:0/1620842613 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80019070 con 0x7f4d8815c560 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d7f400640 1 -- 172.21.15.110:0/1620842613 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8000cbc0 con 0x7f4d8815c560 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d7f400640 1 -- 172.21.15.110:0/1620842613 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80014440 con 0x7f4d8815c560 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d881593e0 con 0x7f4d8815c560 2024-09-10T10:05:30.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d881599a0 con 0x7f4d8815c560 2024-09-10T10:05:30.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d880ce420 con 0x7f4d8815c560 2024-09-10T10:05:30.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d7f400640 1 -- 172.21.15.110:0/1620842613 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f4d8001c070 con 0x7f4d8815c560 2024-09-10T10:05:30.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d7f400640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4d680680a0 0x7f4d6806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d7f400640 1 -- 172.21.15.110:0/1620842613 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5283+0+0 (secure 0 0 0) 0x7f4d80086e90 con 0x7f4d8815c560 2024-09-10T10:05:30.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.345+0000 7f4d87400640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4d680680a0 0x7f4d6806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.349+0000 7f4d87400640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4d680680a0 0x7f4d6806a560 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4d780ae000 tx=0x7f4d780a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:30.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.349+0000 7f4d7f400640 1 -- 172.21.15.110:0/1620842613 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4d80023020 con 0x7f4d8815c560 2024-09-10T10:05:30.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:30.509+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4d8806c560 con 0x7f4d8815c560 2024-09-10T10:05:31.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d7f400640 1 -- 172.21.15.110:0/1620842613 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v151) ==== 170+0+0 (secure 0 0 0) 0x7f4d80056610 con 0x7f4d8815c560 2024-09-10T10:05:31.161 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:05:31.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4d680680a0 msgr2=0x7f4d6806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:31.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4d680680a0 0x7f4d6806a560 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4d780ae000 tx=0x7f4d780a7000 comp rx=0 tx=0).stop 2024-09-10T10:05:31.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d8815c560 msgr2=0x7f4d88147eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:31.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d8815c560 0x7f4d88147eb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000b9a0 tx=0x7f4d8000be60 comp rx=0 tx=0).stop 2024-09-10T10:05:31.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 shutdown_connections 2024-09-10T10:05:31.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4d680680a0 0x7f4d6806a560 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d8815c560 0x7f4d88147eb0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f4d88155970 0x7f4d88147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 --2- 172.21.15.110:0/1620842613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d8806f1e0 0x7f4d880aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.161+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 >> 172.21.15.110:0/1620842613 conn(0x7f4d8806bc50 msgr2=0x7f4d88154710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:31.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.165+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 shutdown_connections 2024-09-10T10:05:31.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.165+0000 7f4d8e200640 1 -- 172.21.15.110:0/1620842613 wait complete. 2024-09-10T10:05:31.203 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:05:31.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.341+0000 7fba3f200640 1 Processor -- start 2024-09-10T10:05:31.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.341+0000 7fba3f200640 1 -- start start 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.341+0000 7fba3f200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba4015ea40 0x7fba400b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.341+0000 7fba3f200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fba400cf580 0x7fba400d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.341+0000 7fba3f200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba400b34d0 con 0x7fba400b3980 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.341+0000 7fba3f200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fba400b37b0 con 0x7fba400cf580 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.341+0000 7fba3f200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fba400d2db0 con 0x7fba4015ea40 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fba400cf580 0x7fba400d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fba400cf580 0x7fba400d3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34138/0 (socket says 172.21.15.110:34138) 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3e800640 1 -- 172.21.15.110:0/1418007129 learned_addr learned my addr 172.21.15.110:0/1418007129 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3de00640 1 --2- 172.21.15.110:0/1418007129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba4015ea40 0x7fba400b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3e800640 1 -- 172.21.15.110:0/1418007129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba4015ea40 msgr2=0x7fba400b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:31.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3e800640 1 --2- 172.21.15.110:0/1418007129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba4015ea40 0x7fba400b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3e800640 1 -- 172.21.15.110:0/1418007129 >> v1:172.21.15.145:6789/0 conn(0x7fba400b3980 legacy=0x7fba400cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:31.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3e800640 1 -- 172.21.15.110:0/1418007129 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba400cfad0 con 0x7fba400cf580 2024-09-10T10:05:31.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3e800640 1 --2- 172.21.15.110:0/1418007129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba400cf580 0x7fba400d3240 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fba38009870 tx=0x7fba3801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae57c694d0006618 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:31.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3ca00640 1 -- 172.21.15.110:0/1418007129 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba380b1070 con 0x7fba400cf580 2024-09-10T10:05:31.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3ca00640 1 -- 172.21.15.110:0/1418007129 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba380a8050 con 0x7fba400cf580 2024-09-10T10:05:31.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.345+0000 7fba3ca00640 1 -- 172.21.15.110:0/1418007129 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba380ac860 con 0x7fba400cf580 2024-09-10T10:05:31.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- 172.21.15.110:0/1418007129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba400cf580 msgr2=0x7fba400d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:31.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 --2- 172.21.15.110:0/1418007129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba400cf580 0x7fba400d3240 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fba38009870 tx=0x7fba3801c4d0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- 172.21.15.110:0/1418007129 shutdown_connections 2024-09-10T10:05:31.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 --2- 172.21.15.110:0/1418007129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba400cf580 0x7fba400d3240 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 --2- 172.21.15.110:0/1418007129 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba4015ea40 0x7fba400b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- 172.21.15.110:0/1418007129 >> 172.21.15.110:0/1418007129 conn(0x7fba400b6120 msgr2=0x7fba400b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:31.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- 172.21.15.110:0/1418007129 shutdown_connections 2024-09-10T10:05:31.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- 172.21.15.110:0/1418007129 wait complete. 2024-09-10T10:05:31.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 Processor -- start 2024-09-10T10:05:31.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- start start 2024-09-10T10:05:31.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba400b3980 0x7fba401f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba400cf580 0x7fba400c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba4015ea40 0x7fba400cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fba4016b4d0 con 0x7fba4015ea40 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba400ba5d0 con 0x7fba400cf580 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3f200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fba4016b650 con 0x7fba400b3980 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba4015ea40 0x7fba400cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba4015ea40 0x7fba400cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34154/0 (socket says 172.21.15.110:34154) 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 -- 172.21.15.110:0/678783242 learned_addr learned my addr 172.21.15.110:0/678783242 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3de00640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba400b3980 0x7fba401f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3d400640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba400cf580 0x7fba400c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 -- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba400b3980 msgr2=0x7fba401f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:31.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba400b3980 0x7fba401f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 -- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba400cf580 msgr2=0x7fba400c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba400cf580 0x7fba400c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 -- 172.21.15.110:0/678783242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba40158640 con 0x7fba4015ea40 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.349+0000 7fba3e800640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba4015ea40 0x7fba400cab10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fba380af040 tx=0x7fba38019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba36000640 1 -- 172.21.15.110:0/678783242 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba380b1040 con 0x7fba4015ea40 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba36000640 1 -- 172.21.15.110:0/678783242 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba380ba070 con 0x7fba4015ea40 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba401f7450 con 0x7fba4015ea40 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba36000640 1 -- 172.21.15.110:0/678783242 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba380b5870 con 0x7fba4015ea40 2024-09-10T10:05:31.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba401f7a10 con 0x7fba4015ea40 2024-09-10T10:05:31.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba400d3240 con 0x7fba4015ea40 2024-09-10T10:05:31.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba36000640 1 -- 172.21.15.110:0/678783242 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fba380b5a10 con 0x7fba4015ea40 2024-09-10T10:05:31.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba36000640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba200680a0 0x7fba2006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:31.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba36000640 1 -- 172.21.15.110:0/678783242 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5665+0+0 (secure 0 0 0) 0x7fba381217c0 con 0x7fba4015ea40 2024-09-10T10:05:31.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.353+0000 7fba3de00640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba200680a0 0x7fba2006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:31.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.357+0000 7fba3de00640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba200680a0 0x7fba2006a560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fba30009f60 tx=0x7fba3000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.357+0000 7fba36000640 1 -- 172.21.15.110:0/678783242 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fba38016150 con 0x7fba4015ea40 2024-09-10T10:05:31.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:31.549+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fba40069490 con 0x7fba4015ea40 2024-09-10T10:05:32.189 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 24 and data pool 25 2024-09-10T10:05:32.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.185+0000 7fba36000640 1 -- 172.21.15.110:0/678783242 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v103) ==== 174+0+0 (secure 0 0 0) 0x7fba380f18b0 con 0x7fba4015ea40 2024-09-10T10:05:32.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba200680a0 msgr2=0x7fba2006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba200680a0 0x7fba2006a560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fba30009f60 tx=0x7fba3000b040 comp rx=0 tx=0).stop 2024-09-10T10:05:32.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba4015ea40 msgr2=0x7fba400cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba4015ea40 0x7fba400cab10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fba380af040 tx=0x7fba38019270 comp rx=0 tx=0).stop 2024-09-10T10:05:32.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 shutdown_connections 2024-09-10T10:05:32.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba200680a0 0x7fba2006a560 secure :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fba30009f60 tx=0x7fba3000b040 comp rx=0 tx=0).stop 2024-09-10T10:05:32.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba4015ea40 0x7fba400cab10 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba400cf580 0x7fba400c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 --2- 172.21.15.110:0/678783242 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba400b3980 0x7fba401f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 >> 172.21.15.110:0/678783242 conn(0x7fba400b6120 msgr2=0x7fba400d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:32.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.193+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 shutdown_connections 2024-09-10T10:05:32.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.197+0000 7fba3f200640 1 -- 172.21.15.110:0/678783242 wait complete. 2024-09-10T10:05:32.237 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:32.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.373+0000 7f68b5600640 1 Processor -- start 2024-09-10T10:05:32.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.373+0000 7f68b5600640 1 -- start start 2024-09-10T10:05:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68b5600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f68b015c560 0x7f68b006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68b5600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b006f010 0x7f68b00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68b5600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68b006f430 con 0x7f68b0155970 2024-09-10T10:05:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68b5600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f68b00b4570 con 0x7f68b015c560 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68b5600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f68b0155d50 con 0x7f68b006f010 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68ae000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f68b015c560 0x7f68b006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68ae000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f68b015c560 0x7f68b006ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34168/0 (socket says 172.21.15.110:34168) 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68ae000640 1 -- 172.21.15.110:0/3846512183 learned_addr learned my addr 172.21.15.110:0/3846512183 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68af400640 1 --2- 172.21.15.110:0/3846512183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b006f010 0x7f68b00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68ae000640 1 -- 172.21.15.110:0/3846512183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b006f010 msgr2=0x7f68b00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68ae000640 1 --2- 172.21.15.110:0/3846512183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b006f010 0x7f68b00ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68ae000640 1 -- 172.21.15.110:0/3846512183 >> v1:172.21.15.110:6789/0 conn(0x7f68b0155970 legacy=0x7f68b00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.377+0000 7f68ae000640 1 -- 172.21.15.110:0/3846512183 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b00b34d0 con 0x7f68b015c560 2024-09-10T10:05:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68ae000640 1 --2- 172.21.15.110:0/3846512183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b015c560 0x7f68b006ead0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f689c009870 tx=0x7f689c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19816edc9b26c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68ad600640 1 -- 172.21.15.110:0/3846512183 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0a95f0 con 0x7f68b015c560 2024-09-10T10:05:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68ad600640 1 -- 172.21.15.110:0/3846512183 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689c0a9790 con 0x7f68b015c560 2024-09-10T10:05:32.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68ad600640 1 -- 172.21.15.110:0/3846512183 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0a9b60 con 0x7f68b015c560 2024-09-10T10:05:32.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- 172.21.15.110:0/3846512183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b015c560 msgr2=0x7f68b006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 --2- 172.21.15.110:0/3846512183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b015c560 0x7f68b006ead0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f689c009870 tx=0x7f689c01c530 comp rx=0 tx=0).stop 2024-09-10T10:05:32.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- 172.21.15.110:0/3846512183 shutdown_connections 2024-09-10T10:05:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 --2- 172.21.15.110:0/3846512183 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b006f010 0x7f68b00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 --2- 172.21.15.110:0/3846512183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b015c560 0x7f68b006ead0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- 172.21.15.110:0/3846512183 >> 172.21.15.110:0/3846512183 conn(0x7f68b006bc50 msgr2=0x7f68b006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- 172.21.15.110:0/3846512183 shutdown_connections 2024-09-10T10:05:32.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- 172.21.15.110:0/3846512183 wait complete. 2024-09-10T10:05:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 Processor -- start 2024-09-10T10:05:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- start start 2024-09-10T10:05:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b006f010 0x7f68b0148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b0155970 0x7f68b0152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b015c560 0x7f68b0152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68b00cf170 con 0x7f68b0155970 2024-09-10T10:05:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68b00b4050 con 0x7f68b006f010 2024-09-10T10:05:32.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68b5600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f68b00cf2f0 con 0x7f68b015c560 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68ae000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b0155970 0x7f68b0152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68ae000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b0155970 0x7f68b0152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34172/0 (socket says 172.21.15.110:34172) 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68ae000640 1 -- 172.21.15.110:0/586586146 learned_addr learned my addr 172.21.15.110:0/586586146 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68af400640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b015c560 0x7f68b0152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.381+0000 7f68aea00640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b006f010 0x7f68b0148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68ae000640 1 -- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b015c560 msgr2=0x7f68b0152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68ae000640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b015c560 0x7f68b0152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68ae000640 1 -- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b006f010 msgr2=0x7f68b0148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68ae000640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b006f010 0x7f68b0148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68ae000640 1 -- 172.21.15.110:0/586586146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b014db00 con 0x7f68b0155970 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68aea00640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b006f010 0x7f68b0148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68ae000640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b0155970 0x7f68b0152900 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f689c01abe0 tx=0x7f689c0192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68a6a00640 1 -- 172.21.15.110:0/586586146 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0b1800 con 0x7f68b0155970 2024-09-10T10:05:32.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68a6a00640 1 -- 172.21.15.110:0/586586146 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689c0b19a0 con 0x7f68b0155970 2024-09-10T10:05:32.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68a6a00640 1 -- 172.21.15.110:0/586586146 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0b1d70 con 0x7f68b0155970 2024-09-10T10:05:32.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b014dde0 con 0x7f68b0155970 2024-09-10T10:05:32.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b014e3a0 con 0x7f68b0155970 2024-09-10T10:05:32.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b00ce420 con 0x7f68b0155970 2024-09-10T10:05:32.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68a6a00640 1 -- 172.21.15.110:0/586586146 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f689c0b93e0 con 0x7f68b0155970 2024-09-10T10:05:32.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68a6a00640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68900680a0 0x7f689006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.385+0000 7f68a6a00640 1 -- 172.21.15.110:0/586586146 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5765+0+0 (secure 0 0 0) 0x7f689c120f40 con 0x7f68b0155970 2024-09-10T10:05:32.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.389+0000 7f68aea00640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68900680a0 0x7f689006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.389+0000 7f68aea00640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68900680a0 0x7f689006a560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f68b01539b0 tx=0x7f68a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:32.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.393+0000 7f68a6a00640 1 -- 172.21.15.110:0/586586146 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f689c0c1030 con 0x7f68b0155970 2024-09-10T10:05:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.549+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68b0165de0 con 0x7f68b0155970 2024-09-10T10:05:32.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.549+0000 7f68a6a00640 1 -- 172.21.15.110:0/586586146 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+16877 (secure 0 0 0) 0x7f689c0ad020 con 0x7f68b0155970 2024-09-10T10:05:32.551 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:32.551 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":152,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:31.551038+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:29.268211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:30.557102+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:32.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68900680a0 msgr2=0x7f689006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68900680a0 0x7f689006a560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f68b01539b0 tx=0x7f68a0009040 comp rx=0 tx=0).stop 2024-09-10T10:05:32.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b0155970 msgr2=0x7f68b0152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b0155970 0x7f68b0152900 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f689c01abe0 tx=0x7f689c0192d0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 shutdown_connections 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f68900680a0 0x7f689006a560 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f68b015c560 0x7f68b0152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b0155970 0x7f68b0152900 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 --2- 172.21.15.110:0/586586146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b006f010 0x7f68b0148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 >> 172.21.15.110:0/586586146 conn(0x7f68b006bc50 msgr2=0x7f68b006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 shutdown_connections 2024-09-10T10:05:32.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.553+0000 7f68b5600640 1 -- 172.21.15.110:0/586586146 wait complete. 2024-09-10T10:05:32.572 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:32.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.705+0000 7f52de600640 1 Processor -- start 2024-09-10T10:05:32.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.705+0000 7f52de600640 1 -- start start 2024-09-10T10:05:32.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52de600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d8155970 0x7f52d80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52de600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f52d806f1e0 0x7f52d80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52de600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d8155d50 con 0x7f52d815c560 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52de600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f52d80b4570 con 0x7f52d806f1e0 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52de600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52d80b34d0 con 0x7f52d8155970 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f52d806f1e0 0x7f52d80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f52d806f1e0 0x7f52d80ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34180/0 (socket says 172.21.15.110:34180) 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7e00640 1 -- 172.21.15.110:0/1246823849 learned_addr learned my addr 172.21.15.110:0/1246823849 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7400640 1 --2- 172.21.15.110:0/1246823849 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d8155970 0x7f52d80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7e00640 1 -- 172.21.15.110:0/1246823849 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d8155970 msgr2=0x7f52d80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7e00640 1 --2- 172.21.15.110:0/1246823849 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d8155970 0x7f52d80cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7e00640 1 -- 172.21.15.110:0/1246823849 >> v1:172.21.15.145:6789/0 conn(0x7f52d815c560 legacy=0x7f52d806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7e00640 1 -- 172.21.15.110:0/1246823849 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d80b37b0 con 0x7f52d806f1e0 2024-09-10T10:05:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.709+0000 7f52d7400640 1 --2- 172.21.15.110:0/1246823849 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d8155970 0x7f52d80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52d7e00640 1 --2- 172.21.15.110:0/1246823849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d806f1e0 0x7f52d80ce420 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009870 tx=0x7f52d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af96e8277a6e1338 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:32.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52d6a00640 1 -- 172.21.15.110:0/1246823849 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d00ab620 con 0x7f52d806f1e0 2024-09-10T10:05:32.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52d6a00640 1 -- 172.21.15.110:0/1246823849 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d00b0070 con 0x7f52d806f1e0 2024-09-10T10:05:32.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52d6a00640 1 -- 172.21.15.110:0/1246823849 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d00ab9e0 con 0x7f52d806f1e0 2024-09-10T10:05:32.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- 172.21.15.110:0/1246823849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d806f1e0 msgr2=0x7f52d80ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 --2- 172.21.15.110:0/1246823849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d806f1e0 0x7f52d80ce420 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009870 tx=0x7f52d001c460 comp rx=0 tx=0).stop 2024-09-10T10:05:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- 172.21.15.110:0/1246823849 shutdown_connections 2024-09-10T10:05:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 --2- 172.21.15.110:0/1246823849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d806f1e0 0x7f52d80ce420 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 --2- 172.21.15.110:0/1246823849 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d8155970 0x7f52d80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- 172.21.15.110:0/1246823849 >> 172.21.15.110:0/1246823849 conn(0x7f52d806bc50 msgr2=0x7f52d806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- 172.21.15.110:0/1246823849 shutdown_connections 2024-09-10T10:05:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- 172.21.15.110:0/1246823849 wait complete. 2024-09-10T10:05:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 Processor -- start 2024-09-10T10:05:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- start start 2024-09-10T10:05:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d806f1e0 0x7f52d81517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8155970 0x7f52d814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52d815c560 0x7f52d814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52d80cf170 con 0x7f52d8155970 2024-09-10T10:05:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52d80cce00 con 0x7f52d815c560 2024-09-10T10:05:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52de600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f52d80cf2f0 con 0x7f52d806f1e0 2024-09-10T10:05:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52cea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8155970 0x7f52d814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52cea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8155970 0x7f52d814d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34194/0 (socket says 172.21.15.110:34194) 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52cea00640 1 -- 172.21.15.110:0/1787646807 learned_addr learned my addr 172.21.15.110:0/1787646807 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52d7400640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d806f1e0 0x7f52d81517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.713+0000 7f52d7e00640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52d815c560 0x7f52d814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cea00640 1 -- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d806f1e0 msgr2=0x7f52d81517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cea00640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d806f1e0 0x7f52d81517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cea00640 1 -- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52d815c560 msgr2=0x7f52d814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cea00640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52d815c560 0x7f52d814dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cea00640 1 -- 172.21.15.110:0/1787646807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d8151dc0 con 0x7f52d8155970 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cea00640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8155970 0x7f52d814d860 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b9c0 tx=0x7f52c000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cfe00640 1 -- 172.21.15.110:0/1787646807 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c000cba0 con 0x7f52d8155970 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cfe00640 1 -- 172.21.15.110:0/1787646807 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c000cd40 con 0x7f52d8155970 2024-09-10T10:05:32.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d81520a0 con 0x7f52d8155970 2024-09-10T10:05:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cfe00640 1 -- 172.21.15.110:0/1787646807 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0003bc0 con 0x7f52d8155970 2024-09-10T10:05:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d8152660 con 0x7f52d8155970 2024-09-10T10:05:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d80ce420 con 0x7f52d8155970 2024-09-10T10:05:32.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cfe00640 1 -- 172.21.15.110:0/1787646807 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f52c0006d10 con 0x7f52d8155970 2024-09-10T10:05:32.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cfe00640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f52b40680a0 0x7f52b406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52cfe00640 1 -- 172.21.15.110:0/1787646807 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5765+0+0 (secure 0 0 0) 0x7f52c0086b30 con 0x7f52d8155970 2024-09-10T10:05:32.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.717+0000 7f52d7400640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f52b40680a0 0x7f52b406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.721+0000 7f52d7400640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f52b40680a0 0x7f52b406a560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ab60 tx=0x7f52c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:32.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.721+0000 7f52cfe00640 1 -- 172.21.15.110:0/1787646807 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f52c0056050 con 0x7f52d8155970 2024-09-10T10:05:32.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.881+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52d8075150 con 0x7f52d8155970 2024-09-10T10:05:32.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52cfe00640 1 -- 172.21.15.110:0/1787646807 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+16877 (secure 0 0 0) 0x7f52c005a350 con 0x7f52d8155970 2024-09-10T10:05:32.885 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:32.885 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":152,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:31.551038+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:29.268211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:30.557102+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:32.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f52b40680a0 msgr2=0x7f52b406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f52b40680a0 0x7f52b406a560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ab60 tx=0x7f52c8008040 comp rx=0 tx=0).stop 2024-09-10T10:05:32.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8155970 msgr2=0x7f52d814d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8155970 0x7f52d814d860 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b9c0 tx=0x7f52c000be80 comp rx=0 tx=0).stop 2024-09-10T10:05:32.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 shutdown_connections 2024-09-10T10:05:32.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f52b40680a0 0x7f52b406a560 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52d815c560 0x7f52d814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8155970 0x7f52d814d860 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 --2- 172.21.15.110:0/1787646807 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f52d806f1e0 0x7f52d81517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 >> 172.21.15.110:0/1787646807 conn(0x7f52d806bc50 msgr2=0x7f52d80cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:32.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 shutdown_connections 2024-09-10T10:05:32.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:32.885+0000 7f52de600640 1 -- 172.21.15.110:0/1787646807 wait complete. 2024-09-10T10:05:32.907 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:05:33.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.041+0000 7f72a1800640 1 Processor -- start 2024-09-10T10:05:33.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.041+0000 7f72a1800640 1 -- start start 2024-09-10T10:05:33.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f72a1800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c0ce4d0 0x7f729c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:33.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f72a1800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f729c06e850 0x7f729c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f72a1800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f729c0b4960 con 0x7f729c15b330 2024-09-10T10:05:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f72a1800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f729c0b5570 con 0x7f729c06e850 2024-09-10T10:05:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f72a1800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f729c0b4c40 con 0x7f729c0ce4d0 2024-09-10T10:05:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f729b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f729c06e850 0x7f729c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f729b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f729c06e850 0x7f729c0b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34210/0 (socket says 172.21.15.110:34210) 2024-09-10T10:05:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f729b400640 1 -- 172.21.15.110:0/3425882351 learned_addr learned my addr 172.21.15.110:0/3425882351 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:33.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f729a000640 1 --2- 172.21.15.110:0/3425882351 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c0ce4d0 0x7f729c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f729b400640 1 -- 172.21.15.110:0/3425882351 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c0ce4d0 msgr2=0x7f729c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f729b400640 1 --2- 172.21.15.110:0/3425882351 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c0ce4d0 0x7f729c06c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f729b400640 1 -- 172.21.15.110:0/3425882351 >> v1:172.21.15.145:6789/0 conn(0x7f729c15b330 legacy=0x7f729c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:33.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.045+0000 7f729b400640 1 -- 172.21.15.110:0/3425882351 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c0b4f20 con 0x7f729c06e850 2024-09-10T10:05:33.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f729b400640 1 --2- 172.21.15.110:0/3425882351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c06e850 0x7f729c0b0860 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7294009870 tx=0x7f729401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29143534b711ec1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:33.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f7299600640 1 -- 172.21.15.110:0/3425882351 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940a7070 con 0x7f729c06e850 2024-09-10T10:05:33.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f7299600640 1 -- 172.21.15.110:0/3425882351 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72940b0070 con 0x7f729c06e850 2024-09-10T10:05:33.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f7299600640 1 -- 172.21.15.110:0/3425882351 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940ab890 con 0x7f729c06e850 2024-09-10T10:05:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 -- 172.21.15.110:0/3425882351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c06e850 msgr2=0x7f729c0b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 --2- 172.21.15.110:0/3425882351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c06e850 0x7f729c0b0860 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7294009870 tx=0x7f729401c560 comp rx=0 tx=0).stop 2024-09-10T10:05:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 -- 172.21.15.110:0/3425882351 shutdown_connections 2024-09-10T10:05:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 --2- 172.21.15.110:0/3425882351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c06e850 0x7f729c0b0860 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 --2- 172.21.15.110:0/3425882351 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c0ce4d0 0x7f729c06c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 -- 172.21.15.110:0/3425882351 >> 172.21.15.110:0/3425882351 conn(0x7f729c062e40 msgr2=0x7f729c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 -- 172.21.15.110:0/3425882351 shutdown_connections 2024-09-10T10:05:33.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 -- 172.21.15.110:0/3425882351 wait complete. 2024-09-10T10:05:33.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 Processor -- start 2024-09-10T10:05:33.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.049+0000 7f72a1800640 1 -- start start 2024-09-10T10:05:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f729c06e850 0x7f729c0725b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c15b330 0x7f729c141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c15ccb0 0x7f729c158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1680b0 con 0x7f729c15ccb0 2024-09-10T10:05:33.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0b17c0 con 0x7f729c06e850 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f729c168230 con 0x7f729c15b330 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c15ccb0 0x7f729c158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c15ccb0 0x7f729c158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34224/0 (socket says 172.21.15.110:34224) 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 -- 172.21.15.110:0/522001163 learned_addr learned my addr 172.21.15.110:0/522001163 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729a000640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c15b330 0x7f729c141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729aa00640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f729c06e850 0x7f729c0725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 -- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c15b330 msgr2=0x7f729c141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c15b330 0x7f729c141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 -- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f729c06e850 msgr2=0x7f729c0725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f729c06e850 0x7f729c0725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 -- 172.21.15.110:0/522001163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c15abb0 con 0x7f729c15ccb0 2024-09-10T10:05:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f729b400640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c15ccb0 0x7f729c158890 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f729401a190 tx=0x7f729401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f7292a00640 1 -- 172.21.15.110:0/522001163 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940be5f0 con 0x7f729c15ccb0 2024-09-10T10:05:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f7292a00640 1 -- 172.21.15.110:0/522001163 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72940be790 con 0x7f729c15ccb0 2024-09-10T10:05:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f7292a00640 1 -- 172.21.15.110:0/522001163 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940beb80 con 0x7f729c15ccb0 2024-09-10T10:05:33.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c06da40 con 0x7f729c15ccb0 2024-09-10T10:05:33.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c15bef0 con 0x7f729c15ccb0 2024-09-10T10:05:33.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729c06c6a0 con 0x7f729c15ccb0 2024-09-10T10:05:33.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.053+0000 7f7292a00640 1 -- 172.21.15.110:0/522001163 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f72940b43f0 con 0x7f729c15ccb0 2024-09-10T10:05:33.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.057+0000 7f7292a00640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f727c0680a0 0x7f727c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:33.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.057+0000 7f7292a00640 1 -- 172.21.15.110:0/522001163 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5765+0+0 (secure 0 0 0) 0x7f729411ee30 con 0x7f729c15ccb0 2024-09-10T10:05:33.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.057+0000 7f729aa00640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f727c0680a0 0x7f727c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:33.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.057+0000 7f729aa00640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f727c0680a0 0x7f727c06a560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f729c159430 tx=0x7f728c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:33.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.061+0000 7f7292a00640 1 -- 172.21.15.110:0/522001163 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7294016150 con 0x7f729c15ccb0 2024-09-10T10:05:33.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:33.257+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f729c063600 con 0x7f729c15ccb0 2024-09-10T10:05:34.157 INFO:tasks.ceph.mds.b.smithi145.stderr:2024-09-10T10:05:34.157+0000 7f3f84e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:05:34.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.197+0000 7f7292a00640 1 -- 172.21.15.110:0/522001163 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v105) ==== 118+0+0 (secure 0 0 0) 0x7f72940f0c10 con 0x7f729c15ccb0 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f727c0680a0 msgr2=0x7f727c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f727c0680a0 0x7f727c06a560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f729c159430 tx=0x7f728c009040 comp rx=0 tx=0).stop 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c15ccb0 msgr2=0x7f729c158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c15ccb0 0x7f729c158890 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f729401a190 tx=0x7f729401ada0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 shutdown_connections 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f727c0680a0 0x7f727c06a560 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f729c15ccb0 0x7f729c158890 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f729c15b330 0x7f729c141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 --2- 172.21.15.110:0/522001163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f729c06e850 0x7f729c0725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 >> 172.21.15.110:0/522001163 conn(0x7f729c062e40 msgr2=0x7f729c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:34.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 shutdown_connections 2024-09-10T10:05:34.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.201+0000 7f72a1800640 1 -- 172.21.15.110:0/522001163 wait complete. 2024-09-10T10:05:34.225 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:05:34.225 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:05:34.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.365+0000 7fad73c00640 1 Processor -- start 2024-09-10T10:05:34.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.365+0000 7fad73c00640 1 -- start start 2024-09-10T10:05:34.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad73c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fad6c145f30 0x7fad6c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:34.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad73c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0cfd90 0x7fad6c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:34.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad73c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6c0b37c0 con 0x7fad6c0b29e0 2024-09-10T10:05:34.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad73c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fad6c0b2e50 con 0x7fad6c145f30 2024-09-10T10:05:34.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad73c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c0b3aa0 con 0x7fad6c0cfd90 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad71c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0cfd90 0x7fad6c0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad6be00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fad6c0b29e0 0x7fad6c0cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:39884/0 (socket says 172.21.15.110:39884) 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad6be00640 1 -- 172.21.15.110:0/1409089977 learned_addr learned my addr 172.21.15.110:0/1409089977 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad71200640 1 --2- 172.21.15.110:0/1409089977 >> v2:172.21.15.145:3300/0 conn(0x7fad6c145f30 0x7fad6c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad6b400640 1 -- 172.21.15.110:0/1409089977 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797797852 0 0) 0x7fad6c0b37c0 con 0x7fad6c0b29e0 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad6b400640 1 -- 172.21.15.110:0/1409089977 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad50003530 con 0x7fad6c0b29e0 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad71c00640 1 -- 172.21.15.110:0/1409089977 >> v2:172.21.15.145:3300/0 conn(0x7fad6c145f30 msgr2=0x7fad6c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad71c00640 1 --2- 172.21.15.110:0/1409089977 >> v2:172.21.15.145:3300/0 conn(0x7fad6c145f30 0x7fad6c0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad71c00640 1 -- 172.21.15.110:0/1409089977 >> v1:172.21.15.145:6789/0 conn(0x7fad6c0b29e0 legacy=0x7fad6c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad71c00640 1 -- 172.21.15.110:0/1409089977 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c0b3d80 con 0x7fad6c0cfd90 2024-09-10T10:05:34.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.369+0000 7fad71200640 1 --2- 172.21.15.110:0/1409089977 >> v2:172.21.15.145:3300/0 conn(0x7fad6c145f30 0x7fad6c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:34.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad71c00640 1 --2- 172.21.15.110:0/1409089977 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0cfd90 0x7fad6c0b4580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad64009d90 tx=0x7fad6401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c32c11abf7f60b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:34.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad6b400640 1 -- 172.21.15.110:0/1409089977 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640b2070 con 0x7fad6c0cfd90 2024-09-10T10:05:34.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad6b400640 1 -- 172.21.15.110:0/1409089977 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad640a5830 con 0x7fad6c0cfd90 2024-09-10T10:05:34.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad6b400640 1 -- 172.21.15.110:0/1409089977 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640a5be0 con 0x7fad6c0cfd90 2024-09-10T10:05:34.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 -- 172.21.15.110:0/1409089977 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0cfd90 msgr2=0x7fad6c0b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:34.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 --2- 172.21.15.110:0/1409089977 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0cfd90 0x7fad6c0b4580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad64009d90 tx=0x7fad6401c970 comp rx=0 tx=0).stop 2024-09-10T10:05:34.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 -- 172.21.15.110:0/1409089977 shutdown_connections 2024-09-10T10:05:34.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 --2- 172.21.15.110:0/1409089977 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0cfd90 0x7fad6c0b4580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 --2- 172.21.15.110:0/1409089977 >> v2:172.21.15.145:3300/0 conn(0x7fad6c145f30 0x7fad6c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 -- 172.21.15.110:0/1409089977 >> 172.21.15.110:0/1409089977 conn(0x7fad6c062e40 msgr2=0x7fad6c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:34.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 -- 172.21.15.110:0/1409089977 shutdown_connections 2024-09-10T10:05:34.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 -- 172.21.15.110:0/1409089977 wait complete. 2024-09-10T10:05:34.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.373+0000 7fad73c00640 1 Processor -- start 2024-09-10T10:05:34.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 -- start start 2024-09-10T10:05:34.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0b29e0 0x7fad6c15bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:34.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c0cfd90 0x7fad6c15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:34.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c145f30 0x7fad6c142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:34.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c0b5660 con 0x7fad6c145f30 2024-09-10T10:05:34.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c0b54e0 con 0x7fad6c0cfd90 2024-09-10T10:05:34.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c0b57e0 con 0x7fad6c0b29e0 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c145f30 0x7fad6c142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c145f30 0x7fad6c142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34236/0 (socket says 172.21.15.110:34236) 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 -- 172.21.15.110:0/3256791950 learned_addr learned my addr 172.21.15.110:0/3256791950 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad6be00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c0cfd90 0x7fad6c15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71200640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0b29e0 0x7fad6c15bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 -- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0b29e0 msgr2=0x7fad6c15bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0b29e0 0x7fad6c15bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 -- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c0cfd90 msgr2=0x7fad6c15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c0cfd90 0x7fad6c15b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 -- 172.21.15.110:0/3256791950 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c15eeb0 con 0x7fad6c145f30 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad6be00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c0cfd90 0x7fad6c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71200640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0b29e0 0x7fad6c15bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:34.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad71c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c145f30 0x7fad6c142020 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fad6401a5a0 tx=0x7fad64019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:34.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad68c00640 1 -- 172.21.15.110:0/3256791950 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640b2050 con 0x7fad6c145f30 2024-09-10T10:05:34.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad68c00640 1 -- 172.21.15.110:0/3256791950 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad640be070 con 0x7fad6c145f30 2024-09-10T10:05:34.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c15f190 con 0x7fad6c145f30 2024-09-10T10:05:34.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad68c00640 1 -- 172.21.15.110:0/3256791950 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64009450 con 0x7fad6c145f30 2024-09-10T10:05:34.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c15b8e0 con 0x7fad6c145f30 2024-09-10T10:05:34.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.377+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c0b37c0 con 0x7fad6c145f30 2024-09-10T10:05:34.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.381+0000 7fad68c00640 1 -- 172.21.15.110:0/3256791950 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fad64004240 con 0x7fad6c145f30 2024-09-10T10:05:34.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.381+0000 7fad68c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c0680a0 0x7fad4c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:34.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.381+0000 7fad68c00640 1 -- 172.21.15.110:0/3256791950 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7fad64121d40 con 0x7fad6c145f30 2024-09-10T10:05:34.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.381+0000 7fad71200640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c0680a0 0x7fad4c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:34.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.381+0000 7fad71200640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c0680a0 0x7fad4c06a560 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fad6c141750 tx=0x7fad5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:34.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.385+0000 7fad68c00640 1 -- 172.21.15.110:0/3256791950 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fad64016560 con 0x7fad6c145f30 2024-09-10T10:05:34.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:34.577+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fad6c13ef10 con 0x7fad6c145f30 2024-09-10T10:05:35.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.197+0000 7fad68c00640 1 -- 172.21.15.110:0/3256791950 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v106) ==== 122+0+0 (secure 0 0 0) 0x7fad640f1cf0 con 0x7fad6c145f30 2024-09-10T10:05:35.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c0680a0 msgr2=0x7fad4c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c0680a0 0x7fad4c06a560 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fad6c141750 tx=0x7fad5c008040 comp rx=0 tx=0).stop 2024-09-10T10:05:35.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c145f30 msgr2=0x7fad6c142020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c145f30 0x7fad6c142020 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fad6401a5a0 tx=0x7fad64019710 comp rx=0 tx=0).stop 2024-09-10T10:05:35.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 shutdown_connections 2024-09-10T10:05:35.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fad4c0680a0 0x7fad4c06a560 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad6c145f30 0x7fad6c142020 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad6c0cfd90 0x7fad6c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 --2- 172.21.15.110:0/3256791950 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fad6c0b29e0 0x7fad6c15bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 >> 172.21.15.110:0/3256791950 conn(0x7fad6c062e40 msgr2=0x7fad6c13f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:35.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 shutdown_connections 2024-09-10T10:05:35.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.201+0000 7fad73c00640 1 -- 172.21.15.110:0/3256791950 wait complete. 2024-09-10T10:05:35.223 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:05:35.224 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:35.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c3d200640 1 Processor -- start 2024-09-10T10:05:35.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c3d200640 1 -- start start 2024-09-10T10:05:35.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c3d200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6c38145b00 0x7f6c380cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c3d200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6c3815d050 0x7f6c380b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c3d200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c38071db0 con 0x7f6c380b5690 2024-09-10T10:05:35.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c3d200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6c380b6010 con 0x7f6c3815d050 2024-09-10T10:05:35.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c3d200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6c380ce5f0 con 0x7f6c38145b00 2024-09-10T10:05:35.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c37e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6c3815d050 0x7f6c380b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c37e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6c3815d050 0x7f6c380b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34250/0 (socket says 172.21.15.110:34250) 2024-09-10T10:05:35.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c37e00640 1 -- 172.21.15.110:0/2298545512 learned_addr learned my addr 172.21.15.110:0/2298545512 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:35.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c37400640 1 --2- 172.21.15.110:0/2298545512 >> v2:172.21.15.145:3300/0 conn(0x7f6c38145b00 0x7f6c380cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c37e00640 1 -- 172.21.15.110:0/2298545512 >> v2:172.21.15.145:3300/0 conn(0x7f6c38145b00 msgr2=0x7f6c380cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:35.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c37e00640 1 --2- 172.21.15.110:0/2298545512 >> v2:172.21.15.145:3300/0 conn(0x7f6c38145b00 0x7f6c380cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c37e00640 1 -- 172.21.15.110:0/2298545512 >> v1:172.21.15.145:6789/0 conn(0x7f6c380b5690 legacy=0x7f6c3815c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.361+0000 7f6c37e00640 1 -- 172.21.15.110:0/2298545512 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c380ce8d0 con 0x7f6c3815d050 2024-09-10T10:05:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c37e00640 1 --2- 172.21.15.110:0/2298545512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c380b5ad0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b6640 tx=0x7f6c3001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adb54fe5fea3ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:35.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c36000640 1 -- 172.21.15.110:0/2298545512 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300a56c0 con 0x7f6c3815d050 2024-09-10T10:05:35.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c36000640 1 -- 172.21.15.110:0/2298545512 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c300a5860 con 0x7f6c3815d050 2024-09-10T10:05:35.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c36000640 1 -- 172.21.15.110:0/2298545512 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300a5c30 con 0x7f6c3815d050 2024-09-10T10:05:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c3d200640 1 -- 172.21.15.110:0/2298545512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 msgr2=0x7f6c380b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c3d200640 1 --2- 172.21.15.110:0/2298545512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c380b5ad0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b6640 tx=0x7f6c3001c9f0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c3d200640 1 -- 172.21.15.110:0/2298545512 shutdown_connections 2024-09-10T10:05:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c3d200640 1 --2- 172.21.15.110:0/2298545512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c380b5ad0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c3d200640 1 --2- 172.21.15.110:0/2298545512 >> v2:172.21.15.145:3300/0 conn(0x7f6c38145b00 0x7f6c380cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c3d200640 1 -- 172.21.15.110:0/2298545512 >> 172.21.15.110:0/2298545512 conn(0x7f6c3806d0f0 msgr2=0x7f6c3806d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c3d200640 1 -- 172.21.15.110:0/2298545512 shutdown_connections 2024-09-10T10:05:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.365+0000 7f6c3d200640 1 -- 172.21.15.110:0/2298545512 wait complete. 2024-09-10T10:05:35.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 Processor -- start 2024-09-10T10:05:35.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 -- start start 2024-09-10T10:05:35.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c380b5690 0x7f6c381413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c38145b00 0x7f6c38163950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c38140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c380719d0 con 0x7f6c3815d050 2024-09-10T10:05:35.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38071850 con 0x7f6c38145b00 2024-09-10T10:05:35.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6c38071b50 con 0x7f6c380b5690 2024-09-10T10:05:35.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c38140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c38140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34264/0 (socket says 172.21.15.110:34264) 2024-09-10T10:05:35.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 -- 172.21.15.110:0/1853383764 learned_addr learned my addr 172.21.15.110:0/1853383764 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:35.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c36a00640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c38145b00 0x7f6c38163950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37400640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c380b5690 0x7f6c381413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 -- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c380b5690 msgr2=0x7f6c381413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c380b5690 0x7f6c381413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 -- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c38145b00 msgr2=0x7f6c38163950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c38145b00 0x7f6c38163950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 -- 172.21.15.110:0/1853383764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c38141ac0 con 0x7f6c3815d050 2024-09-10T10:05:35.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c36a00640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c38145b00 0x7f6c38163950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:35.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c37e00640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c38140cb0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6c30019600 tx=0x7f6c30019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:35.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c2f400640 1 -- 172.21.15.110:0/1853383764 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300b8670 con 0x7f6c3815d050 2024-09-10T10:05:35.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c2f400640 1 -- 172.21.15.110:0/1853383764 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c300be070 con 0x7f6c3815d050 2024-09-10T10:05:35.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c38141da0 con 0x7f6c3815d050 2024-09-10T10:05:35.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c2f400640 1 -- 172.21.15.110:0/1853383764 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300b8a30 con 0x7f6c3815d050 2024-09-10T10:05:35.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.369+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c38142360 con 0x7f6c3815d050 2024-09-10T10:05:35.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.373+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c380cf0d0 con 0x7f6c3815d050 2024-09-10T10:05:35.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.373+0000 7f6c2f400640 1 -- 172.21.15.110:0/1853383764 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f6c300b8bd0 con 0x7f6c3815d050 2024-09-10T10:05:35.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.373+0000 7f6c2f400640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c180680d0 0x7f6c1806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.373+0000 7f6c2f400640 1 -- 172.21.15.110:0/1853383764 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f6c3011f0e0 con 0x7f6c3815d050 2024-09-10T10:05:35.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.373+0000 7f6c37400640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c180680d0 0x7f6c1806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.373+0000 7f6c37400640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c180680d0 0x7f6c1806a590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800b9d0 tx=0x7f6c28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:35.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.377+0000 7f6c2f400640 1 -- 172.21.15.110:0/1853383764 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6c300c7020 con 0x7f6c3815d050 2024-09-10T10:05:35.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c381430c0 con 0x7f6c3815d050 2024-09-10T10:05:35.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c2f400640 1 -- 172.21.15.110:0/1853383764 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5245 (secure 0 0 0) 0x7f6c300165e0 con 0x7f6c3815d050 2024-09-10T10:05:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:35.571 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":106,"btime":"2024-09-10T10:05:35:199222+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3244721906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3244721906},{"type":"v1","addr":"172.21.15.145:6834","nonce":3244721906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/1342717567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1342717567},{"type":"v1","addr":"172.21.15.110:6834","nonce":1342717567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1495976904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1495976904},{"type":"v1","addr":"172.21.15.110:6835","nonce":1495976904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:31.551366+0000","modified":"2024-09-10T10:05:34.580103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1869854747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1869854747},{"type":"v1","addr":"172.21.15.145:6835","nonce":1869854747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":12}]} 2024-09-10T10:05:35.571 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 106 2024-09-10T10:05:35.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c180680d0 msgr2=0x7f6c1806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c180680d0 0x7f6c1806a590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800b9d0 tx=0x7f6c28008040 comp rx=0 tx=0).stop 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 msgr2=0x7f6c38140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c38140cb0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6c30019600 tx=0x7f6c30019790 comp rx=0 tx=0).stop 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 shutdown_connections 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6c180680d0 0x7f6c1806a590 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c3815d050 0x7f6c38140cb0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c38145b00 0x7f6c38163950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 --2- 172.21.15.110:0/1853383764 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c380b5690 0x7f6c381413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.569+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 >> 172.21.15.110:0/1853383764 conn(0x7f6c3806d0f0 msgr2=0x7f6c380b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.573+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 shutdown_connections 2024-09-10T10:05:35.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.573+0000 7f6c3d200640 1 -- 172.21.15.110:0/1853383764 wait complete. 2024-09-10T10:05:35.591 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:35.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.725+0000 7efea6400640 1 Processor -- start 2024-09-10T10:05:35.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.725+0000 7efea6400640 1 -- start start 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efea6400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efea015c560 0x7efea006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efea6400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efea006f010 0x7efea00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efea6400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efea006f430 con 0x7efea0155970 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efea6400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7efea00b4570 con 0x7efea015c560 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efea6400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7efea0155d50 con 0x7efea006f010 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9ea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7efea015c560 0x7efea006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7efea0155970 0x7efea00cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:48920/0 (socket says 172.21.15.110:48920) 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9f400640 1 -- 172.21.15.110:0/133293420 learned_addr learned my addr 172.21.15.110:0/133293420 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:35.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9fe00640 1 --2- 172.21.15.110:0/133293420 >> v2:172.21.15.145:3300/0 conn(0x7efea006f010 0x7efea00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9ea00640 1 -- 172.21.15.110:0/133293420 >> v2:172.21.15.145:3300/0 conn(0x7efea006f010 msgr2=0x7efea00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:35.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9ea00640 1 --2- 172.21.15.110:0/133293420 >> v2:172.21.15.145:3300/0 conn(0x7efea006f010 0x7efea00ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9ea00640 1 -- 172.21.15.110:0/133293420 >> v1:172.21.15.110:6789/0 conn(0x7efea0155970 legacy=0x7efea00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9ea00640 1 -- 172.21.15.110:0/133293420 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea00b34d0 con 0x7efea015c560 2024-09-10T10:05:35.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.729+0000 7efe9ea00640 1 --2- 172.21.15.110:0/133293420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea006ead0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7efea00735e0 tx=0x7efe8c01e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa02f639c8f4798 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:35.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efe9e000640 1 -- 172.21.15.110:0/133293420 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0a8630 con 0x7efea015c560 2024-09-10T10:05:35.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efe9e000640 1 -- 172.21.15.110:0/133293420 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8c0a87d0 con 0x7efea015c560 2024-09-10T10:05:35.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efe9e000640 1 -- 172.21.15.110:0/133293420 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0a8ba0 con 0x7efea015c560 2024-09-10T10:05:35.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- 172.21.15.110:0/133293420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 msgr2=0x7efea006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 --2- 172.21.15.110:0/133293420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea006ead0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7efea00735e0 tx=0x7efe8c01e660 comp rx=0 tx=0).stop 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- 172.21.15.110:0/133293420 shutdown_connections 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 --2- 172.21.15.110:0/133293420 >> v2:172.21.15.145:3300/0 conn(0x7efea006f010 0x7efea00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 --2- 172.21.15.110:0/133293420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea006ead0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- 172.21.15.110:0/133293420 >> 172.21.15.110:0/133293420 conn(0x7efea006bc50 msgr2=0x7efea006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- 172.21.15.110:0/133293420 shutdown_connections 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- 172.21.15.110:0/133293420 wait complete. 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 Processor -- start 2024-09-10T10:05:35.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- start start 2024-09-10T10:05:35.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea006f010 0x7efea0164b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea0155970 0x7efea01542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efea00cf170 con 0x7efea015c560 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efea00b4050 con 0x7efea0155970 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efea6400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7efea00cf2f0 con 0x7efea006f010 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efe9fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efe9fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea015ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34276/0 (socket says 172.21.15.110:34276) 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efe9fe00640 1 -- 172.21.15.110:0/2740920638 learned_addr learned my addr 172.21.15.110:0/2740920638 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efe9f400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea006f010 0x7efea0164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.733+0000 7efe9ea00640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea0155970 0x7efea01542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe9fe00640 1 -- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea006f010 msgr2=0x7efea0164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe9fe00640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea006f010 0x7efea0164b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe9fe00640 1 -- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea0155970 msgr2=0x7efea01542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe9fe00640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea0155970 0x7efea01542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe9fe00640 1 -- 172.21.15.110:0/2740920638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea015d150 con 0x7efea015c560 2024-09-10T10:05:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe9fe00640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea015ca50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7efe9800efc0 tx=0x7efe9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe97400640 1 -- 172.21.15.110:0/2740920638 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98018020 con 0x7efea015c560 2024-09-10T10:05:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe97400640 1 -- 172.21.15.110:0/2740920638 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe98014480 con 0x7efea015c560 2024-09-10T10:05:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe97400640 1 -- 172.21.15.110:0/2740920638 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98014870 con 0x7efea015c560 2024-09-10T10:05:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea015d430 con 0x7efea015c560 2024-09-10T10:05:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea015d9f0 con 0x7efea015c560 2024-09-10T10:05:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea0069bb0 con 0x7efea015c560 2024-09-10T10:05:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe97400640 1 -- 172.21.15.110:0/2740920638 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7efe9801b020 con 0x7efea015c560 2024-09-10T10:05:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe97400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efe800680a0 0x7efe8006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.737+0000 7efe97400640 1 -- 172.21.15.110:0/2740920638 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7efe980876a0 con 0x7efea015c560 2024-09-10T10:05:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.741+0000 7efe9f400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efe800680a0 0x7efe8006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:35.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.741+0000 7efe9f400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efe800680a0 0x7efe8006a560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe90000f70 tx=0x7efe90007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:35.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.745+0000 7efe97400640 1 -- 172.21.15.110:0/2740920638 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efe98022070 con 0x7efea015c560 2024-09-10T10:05:35.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.901+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efea00cf680 con 0x7efea015c560 2024-09-10T10:05:35.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.901+0000 7efe97400640 1 -- 172.21.15.110:0/2740920638 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16864 (secure 0 0 0) 0x7efe98010040 con 0x7efea015c560 2024-09-10T10:05:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:35.904 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":153,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:33.186967+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:29.268211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:30.557102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:35.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efe800680a0 msgr2=0x7efe8006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efe800680a0 0x7efe8006a560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe90000f70 tx=0x7efe90007e90 comp rx=0 tx=0).stop 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 msgr2=0x7efea015ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea015ca50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7efe9800efc0 tx=0x7efe9800c490 comp rx=0 tx=0).stop 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 shutdown_connections 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7efe800680a0 0x7efe8006a560 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efea015c560 0x7efea015ca50 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efea0155970 0x7efea01542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 --2- 172.21.15.110:0/2740920638 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7efea006f010 0x7efea0164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 >> 172.21.15.110:0/2740920638 conn(0x7efea006bc50 msgr2=0x7efea00b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 shutdown_connections 2024-09-10T10:05:35.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:35.905+0000 7efea6400640 1 -- 172.21.15.110:0/2740920638 wait complete. 2024-09-10T10:05:35.925 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:05:35.925 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:05:36.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2148a00640 1 Processor -- start 2024-09-10T10:05:36.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2148a00640 1 -- start start 2024-09-10T10:05:36.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2148a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f214415e8f0 0x7f21440b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2148a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f214406f160 0x7f214406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2148a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21440b5f50 con 0x7f21440b59d0 2024-09-10T10:05:36.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2148a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f21440b6230 con 0x7f214415e8f0 2024-09-10T10:05:36.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2148a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f21440b6510 con 0x7f214406f160 2024-09-10T10:05:36.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2141e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f21440b59d0 0x7f214406ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:48936/0 (socket says 172.21.15.110:48936) 2024-09-10T10:05:36.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2141e00640 1 -- 172.21.15.110:0/775619407 learned_addr learned my addr 172.21.15.110:0/775619407 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:36.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2142800640 1 --2- 172.21.15.110:0/775619407 >> v2:172.21.15.145:3300/0 conn(0x7f214415e8f0 0x7f21440b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2143200640 1 --2- 172.21.15.110:0/775619407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f214406f160 0x7f214406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2141400640 1 -- 172.21.15.110:0/775619407 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4089578068 0 0) 0x7f21440b5f50 con 0x7f21440b59d0 2024-09-10T10:05:36.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2141400640 1 -- 172.21.15.110:0/775619407 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2128003530 con 0x7f21440b59d0 2024-09-10T10:05:36.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2142800640 1 -- 172.21.15.110:0/775619407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f214406f160 msgr2=0x7f214406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:36.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2142800640 1 --2- 172.21.15.110:0/775619407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f214406f160 0x7f214406bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2142800640 1 -- 172.21.15.110:0/775619407 >> v1:172.21.15.110:6789/0 conn(0x7f21440b59d0 legacy=0x7f214406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.061+0000 7f2142800640 1 -- 172.21.15.110:0/775619407 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21440b67f0 con 0x7f214415e8f0 2024-09-10T10:05:36.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2142800640 1 --2- 172.21.15.110:0/775619407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214415e8f0 0x7f21440b5490 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f21340097c0 tx=0x7f213401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c063f6aec42c663 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2141400640 1 -- 172.21.15.110:0/775619407 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340b0070 con 0x7f214415e8f0 2024-09-10T10:05:36.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2141400640 1 -- 172.21.15.110:0/775619407 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21340ab600 con 0x7f214415e8f0 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2141400640 1 -- 172.21.15.110:0/775619407 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340ab9b0 con 0x7f214415e8f0 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 -- 172.21.15.110:0/775619407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214415e8f0 msgr2=0x7f21440b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 --2- 172.21.15.110:0/775619407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214415e8f0 0x7f21440b5490 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f21340097c0 tx=0x7f213401c410 comp rx=0 tx=0).stop 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 -- 172.21.15.110:0/775619407 shutdown_connections 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 --2- 172.21.15.110:0/775619407 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f214406f160 0x7f214406bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 --2- 172.21.15.110:0/775619407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214415e8f0 0x7f21440b5490 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 -- 172.21.15.110:0/775619407 >> 172.21.15.110:0/775619407 conn(0x7f21440d1480 msgr2=0x7f21440d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 -- 172.21.15.110:0/775619407 shutdown_connections 2024-09-10T10:05:36.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 -- 172.21.15.110:0/775619407 wait complete. 2024-09-10T10:05:36.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.065+0000 7f2148a00640 1 Processor -- start 2024-09-10T10:05:36.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 -- start start 2024-09-10T10:05:36.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214406f160 0x7f21441490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21440b59d0 0x7f214415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f214415e8f0 0x7f21440cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f214406ce10 con 0x7f214415e8f0 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f214406cc90 con 0x7f214406f160 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f214406cf90 con 0x7f21440b59d0 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f214415e8f0 0x7f21440cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f214415e8f0 0x7f21440cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34284/0 (socket says 172.21.15.110:34284) 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 -- 172.21.15.110:0/2467306245 learned_addr learned my addr 172.21.15.110:0/2467306245 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 -- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21440b59d0 msgr2=0x7f214415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21440b59d0 0x7f214415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 -- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214406f160 msgr2=0x7f21441490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214406f160 0x7f21441490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 -- 172.21.15.110:0/2467306245 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214414d980 con 0x7f214415e8f0 2024-09-10T10:05:36.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2143200640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f214415e8f0 0x7f21440cae00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f213c009e30 tx=0x7f213c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f213aa00640 1 -- 172.21.15.110:0/2467306245 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c018020 con 0x7f214415e8f0 2024-09-10T10:05:36.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f213aa00640 1 -- 172.21.15.110:0/2467306245 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213c014480 con 0x7f214415e8f0 2024-09-10T10:05:36.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f213aa00640 1 -- 172.21.15.110:0/2467306245 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c014870 con 0x7f214415e8f0 2024-09-10T10:05:36.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214414dc60 con 0x7f214415e8f0 2024-09-10T10:05:36.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214414e220 con 0x7f214415e8f0 2024-09-10T10:05:36.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.069+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214406bd30 con 0x7f214415e8f0 2024-09-10T10:05:36.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.073+0000 7f213aa00640 1 -- 172.21.15.110:0/2467306245 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f213c01b020 con 0x7f214415e8f0 2024-09-10T10:05:36.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.073+0000 7f213aa00640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21240680a0 0x7f212406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.073+0000 7f213aa00640 1 -- 172.21.15.110:0/2467306245 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f213c087bb0 con 0x7f214415e8f0 2024-09-10T10:05:36.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.073+0000 7f2142800640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21240680a0 0x7f212406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.073+0000 7f2142800640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21240680a0 0x7f212406a560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f21340ad000 tx=0x7f2134008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.077+0000 7f213aa00640 1 -- 172.21.15.110:0/2467306245 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f213c022030 con 0x7f214415e8f0 2024-09-10T10:05:36.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.229+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21440cfcc0 con 0x7f21240680a0 2024-09-10T10:05:36.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.237+0000 7f213aa00640 1 -- 172.21.15.110:0/2467306245 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247088 (secure 0 0 0) 0x7f21440cfcc0 con 0x7f21240680a0 2024-09-10T10:05:36.242 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:36.243 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21240680a0 msgr2=0x7f212406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21240680a0 0x7f212406a560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f21340ad000 tx=0x7f2134008040 comp rx=0 tx=0).stop 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f214415e8f0 msgr2=0x7f21440cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f214415e8f0 0x7f21440cae00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f213c009e30 tx=0x7f213c00c490 comp rx=0 tx=0).stop 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 shutdown_connections 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21240680a0 0x7f212406a560 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f214415e8f0 0x7f21440cae00 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f21440b59d0 0x7f214415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 --2- 172.21.15.110:0/2467306245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f214406f160 0x7f21441490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 >> 172.21.15.110:0/2467306245 conn(0x7f21440d1480 msgr2=0x7f21440d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 shutdown_connections 2024-09-10T10:05:36.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.241+0000 7f2148a00640 1 -- 172.21.15.110:0/2467306245 wait complete. 2024-09-10T10:05:36.263 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-10T10:05:35.740163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282112,"kb_used_data":15952,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692608,"statfs":{"total":773094113280,"available":772805230592,"internally_reserved":0,"allocated":16334848,"data_stored":7845962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.578008"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178157+0000","last_change":"2024-09-10T10:05:31.178157+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178157+0000","last_undegraded":"2024-09-10T10:05:31.178157+0000","last_fullsized":"2024-09-10T10:05:31.178157+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173328+0000","last_change":"2024-09-10T10:05:31.173328+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173328+0000","last_undegraded":"2024-09-10T10:05:31.173328+0000","last_fullsized":"2024-09-10T10:05:31.173328+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179146+0000","last_change":"2024-09-10T10:05:31.179146+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179146+0000","last_undegraded":"2024-09-10T10:05:31.179146+0000","last_fullsized":"2024-09-10T10:05:31.179146+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173862+0000","last_change":"2024-09-10T10:05:31.173862+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173862+0000","last_undegraded":"2024-09-10T10:05:31.173862+0000","last_fullsized":"2024-09-10T10:05:31.173862+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172850+0000","last_change":"2024-09-10T10:05:31.172850+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172850+0000","last_undegraded":"2024-09-10T10:05:31.172850+0000","last_fullsized":"2024-09-10T10:05:31.172850+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173582+0000","last_change":"2024-09-10T10:05:31.173582+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173582+0000","last_undegraded":"2024-09-10T10:05:31.173582+0000","last_fullsized":"2024-09-10T10:05:31.173582+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171259+0000","last_change":"2024-09-10T10:05:31.171259+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171259+0000","last_undegraded":"2024-09-10T10:05:31.171259+0000","last_fullsized":"2024-09-10T10:05:31.171259+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171214+0000","last_change":"2024-09-10T10:05:31.171214+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171214+0000","last_undegraded":"2024-09-10T10:05:31.171214+0000","last_fullsized":"2024-09-10T10:05:31.171214+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180484+0000","last_change":"2024-09-10T10:05:31.180484+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180484+0000","last_undegraded":"2024-09-10T10:05:31.180484+0000","last_fullsized":"2024-09-10T10:05:31.180484+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.187473+0000","last_change":"2024-09-10T10:05:31.187473+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.187473+0000","last_undegraded":"2024-09-10T10:05:31.187473+0000","last_fullsized":"2024-09-10T10:05:31.187473+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170730+0000","last_change":"2024-09-10T10:05:31.170730+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170730+0000","last_undegraded":"2024-09-10T10:05:31.170730+0000","last_fullsized":"2024-09-10T10:05:31.170730+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173444+0000","last_change":"2024-09-10T10:05:31.173444+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173444+0000","last_undegraded":"2024-09-10T10:05:31.173444+0000","last_fullsized":"2024-09-10T10:05:31.173444+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173147+0000","last_change":"2024-09-10T10:05:31.173147+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173147+0000","last_undegraded":"2024-09-10T10:05:31.173147+0000","last_fullsized":"2024-09-10T10:05:31.173147+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178165+0000","last_change":"2024-09-10T10:05:31.178165+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178165+0000","last_undegraded":"2024-09-10T10:05:31.178165+0000","last_fullsized":"2024-09-10T10:05:31.178165+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172248+0000","last_change":"2024-09-10T10:05:31.172248+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172248+0000","last_undegraded":"2024-09-10T10:05:31.172248+0000","last_fullsized":"2024-09-10T10:05:31.172248+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170807+0000","last_change":"2024-09-10T10:05:31.170807+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170807+0000","last_undegraded":"2024-09-10T10:05:31.170807+0000","last_fullsized":"2024-09-10T10:05:31.170807+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173631+0000","last_change":"2024-09-10T10:05:31.173631+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173631+0000","last_undegraded":"2024-09-10T10:05:31.173631+0000","last_fullsized":"2024-09-10T10:05:31.173631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181226+0000","last_change":"2024-09-10T10:05:31.181226+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181226+0000","last_undegraded":"2024-09-10T10:05:31.181226+0000","last_fullsized":"2024-09-10T10:05:31.181226+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180425+0000","last_change":"2024-09-10T10:05:31.180425+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180425+0000","last_undegraded":"2024-09-10T10:05:31.180425+0000","last_fullsized":"2024-09-10T10:05:31.180425+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.175359+0000","last_change":"2024-09-10T10:05:31.175359+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.175359+0000","last_undegraded":"2024-09-10T10:05:31.175359+0000","last_fullsized":"2024-09-10T10:05:31.175359+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172406+0000","last_change":"2024-09-10T10:05:31.172406+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172406+0000","last_undegraded":"2024-09-10T10:05:31.172406+0000","last_fullsized":"2024-09-10T10:05:31.172406+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181999+0000","last_change":"2024-09-10T10:05:31.181999+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181999+0000","last_undegraded":"2024-09-10T10:05:31.181999+0000","last_fullsized":"2024-09-10T10:05:31.181999+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177007+0000","last_change":"2024-09-10T10:05:31.177007+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177007+0000","last_undegraded":"2024-09-10T10:05:31.177007+0000","last_fullsized":"2024-09-10T10:05:31.177007+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170320+0000","last_change":"2024-09-10T10:05:31.170320+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170320+0000","last_undegraded":"2024-09-10T10:05:31.170320+0000","last_fullsized":"2024-09-10T10:05:31.170320+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172778+0000","last_change":"2024-09-10T10:05:31.172778+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172778+0000","last_undegraded":"2024-09-10T10:05:31.172778+0000","last_fullsized":"2024-09-10T10:05:31.172778+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171873+0000","last_change":"2024-09-10T10:05:31.171873+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171873+0000","last_undegraded":"2024-09-10T10:05:31.171873+0000","last_fullsized":"2024-09-10T10:05:31.171873+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177404+0000","last_change":"2024-09-10T10:05:31.177404+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177404+0000","last_undegraded":"2024-09-10T10:05:31.177404+0000","last_fullsized":"2024-09-10T10:05:31.177404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174881+0000","last_change":"2024-09-10T10:05:31.174881+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174881+0000","last_undegraded":"2024-09-10T10:05:31.174881+0000","last_fullsized":"2024-09-10T10:05:31.174881+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172702+0000","last_change":"2024-09-10T10:05:31.172702+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172702+0000","last_undegraded":"2024-09-10T10:05:31.172702+0000","last_fullsized":"2024-09-10T10:05:31.172702+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177678+0000","last_change":"2024-09-10T10:05:31.177678+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177678+0000","last_undegraded":"2024-09-10T10:05:31.177678+0000","last_fullsized":"2024-09-10T10:05:31.177678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173678+0000","last_change":"2024-09-10T10:05:31.173678+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173678+0000","last_undegraded":"2024-09-10T10:05:31.173678+0000","last_fullsized":"2024-09-10T10:05:31.173678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172528+0000","last_change":"2024-09-10T10:05:31.172528+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172528+0000","last_undegraded":"2024-09-10T10:05:31.172528+0000","last_fullsized":"2024-09-10T10:05:31.172528+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170496+0000","last_change":"2024-09-10T10:05:31.170496+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170496+0000","last_undegraded":"2024-09-10T10:05:31.170496+0000","last_fullsized":"2024-09-10T10:05:31.170496+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193482+0000","last_change":"2024-09-10T10:05:31.187728+0000","last_active":"2024-09-10T10:05:31.193482+0000","last_peered":"2024-09-10T10:05:31.193482+0000","last_clean":"2024-09-10T10:05:31.193482+0000","last_became_active":"2024-09-10T10:05:31.186044+0000","last_became_peered":"2024-09-10T10:05:31.186044+0000","last_unstale":"2024-09-10T10:05:31.193482+0000","last_undegraded":"2024-09-10T10:05:31.193482+0000","last_fullsized":"2024-09-10T10:05:31.193482+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:28.803480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179647+0000","last_change":"2024-09-10T10:05:31.179647+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179647+0000","last_undegraded":"2024-09-10T10:05:31.179647+0000","last_fullsized":"2024-09-10T10:05:31.179647+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.189032+0000","last_change":"2024-09-10T10:05:31.187161+0000","last_active":"2024-09-10T10:05:31.189032+0000","last_peered":"2024-09-10T10:05:31.189032+0000","last_clean":"2024-09-10T10:05:31.189032+0000","last_became_active":"2024-09-10T10:05:31.186727+0000","last_became_peered":"2024-09-10T10:05:31.186727+0000","last_unstale":"2024-09-10T10:05:31.189032+0000","last_undegraded":"2024-09-10T10:05:31.189032+0000","last_fullsized":"2024-09-10T10:05:31.189032+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:15.098413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174762+0000","last_change":"2024-09-10T10:05:31.174762+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174762+0000","last_undegraded":"2024-09-10T10:05:31.174762+0000","last_fullsized":"2024-09-10T10:05:31.174762+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197736+0000","last_change":"2024-09-10T10:05:31.196039+0000","last_active":"2024-09-10T10:05:31.197736+0000","last_peered":"2024-09-10T10:05:31.197736+0000","last_clean":"2024-09-10T10:05:31.197736+0000","last_became_active":"2024-09-10T10:05:31.195762+0000","last_became_peered":"2024-09-10T10:05:31.195762+0000","last_unstale":"2024-09-10T10:05:31.197736+0000","last_undegraded":"2024-09-10T10:05:31.197736+0000","last_fullsized":"2024-09-10T10:05:31.197736+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:08.377153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177235+0000","last_change":"2024-09-10T10:05:31.177235+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177235+0000","last_undegraded":"2024-09-10T10:05:31.177235+0000","last_fullsized":"2024-09-10T10:05:31.177235+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190648+0000","last_change":"2024-09-10T10:05:31.184956+0000","last_active":"2024-09-10T10:05:31.190648+0000","last_peered":"2024-09-10T10:05:31.190648+0000","last_clean":"2024-09-10T10:05:31.190648+0000","last_became_active":"2024-09-10T10:05:31.184628+0000","last_became_peered":"2024-09-10T10:05:31.184628+0000","last_unstale":"2024-09-10T10:05:31.190648+0000","last_undegraded":"2024-09-10T10:05:31.190648+0000","last_fullsized":"2024-09-10T10:05:31.190648+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:20.844260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181360+0000","last_change":"2024-09-10T10:05:31.181360+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181360+0000","last_undegraded":"2024-09-10T10:05:31.181360+0000","last_fullsized":"2024-09-10T10:05:31.181360+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193807+0000","last_change":"2024-09-10T10:05:31.189563+0000","last_active":"2024-09-10T10:05:31.193807+0000","last_peered":"2024-09-10T10:05:31.193807+0000","last_clean":"2024-09-10T10:05:31.193807+0000","last_became_active":"2024-09-10T10:05:31.189169+0000","last_became_peered":"2024-09-10T10:05:31.189169+0000","last_unstale":"2024-09-10T10:05:31.193807+0000","last_undegraded":"2024-09-10T10:05:31.193807+0000","last_fullsized":"2024-09-10T10:05:31.193807+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:46.694678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171144+0000","last_change":"2024-09-10T10:05:31.171144+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171144+0000","last_undegraded":"2024-09-10T10:05:31.171144+0000","last_fullsized":"2024-09-10T10:05:31.171144+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190551+0000","last_change":"2024-09-10T10:05:31.184111+0000","last_active":"2024-09-10T10:05:31.190551+0000","last_peered":"2024-09-10T10:05:31.190551+0000","last_clean":"2024-09-10T10:05:31.190551+0000","last_became_active":"2024-09-10T10:05:31.183673+0000","last_became_peered":"2024-09-10T10:05:31.183673+0000","last_unstale":"2024-09-10T10:05:31.190551+0000","last_undegraded":"2024-09-10T10:05:31.190551+0000","last_fullsized":"2024-09-10T10:05:31.190551+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:21.245568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170694+0000","last_change":"2024-09-10T10:05:31.170694+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170694+0000","last_undegraded":"2024-09-10T10:05:31.170694+0000","last_fullsized":"2024-09-10T10:05:31.170694+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194428+0000","last_change":"2024-09-10T10:05:31.193233+0000","last_active":"2024-09-10T10:05:31.194428+0000","last_peered":"2024-09-10T10:05:31.194428+0000","last_clean":"2024-09-10T10:05:31.194428+0000","last_became_active":"2024-09-10T10:05:31.192971+0000","last_became_peered":"2024-09-10T10:05:31.192971+0000","last_unstale":"2024-09-10T10:05:31.194428+0000","last_undegraded":"2024-09-10T10:05:31.194428+0000","last_fullsized":"2024-09-10T10:05:31.194428+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:04.862858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178681+0000","last_change":"2024-09-10T10:05:31.178681+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178681+0000","last_undegraded":"2024-09-10T10:05:31.178681+0000","last_fullsized":"2024-09-10T10:05:31.178681+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190231+0000","last_change":"2024-09-10T10:05:31.185108+0000","last_active":"2024-09-10T10:05:31.190231+0000","last_peered":"2024-09-10T10:05:31.190231+0000","last_clean":"2024-09-10T10:05:31.190231+0000","last_became_active":"2024-09-10T10:05:31.184889+0000","last_became_peered":"2024-09-10T10:05:31.184889+0000","last_unstale":"2024-09-10T10:05:31.190231+0000","last_undegraded":"2024-09-10T10:05:31.190231+0000","last_fullsized":"2024-09-10T10:05:31.190231+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:57.982434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170177+0000","last_change":"2024-09-10T10:05:31.170177+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170177+0000","last_undegraded":"2024-09-10T10:05:31.170177+0000","last_fullsized":"2024-09-10T10:05:31.170177+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194581+0000","last_change":"2024-09-10T10:05:31.190056+0000","last_active":"2024-09-10T10:05:31.194581+0000","last_peered":"2024-09-10T10:05:31.194581+0000","last_clean":"2024-09-10T10:05:31.194581+0000","last_became_active":"2024-09-10T10:05:31.189791+0000","last_became_peered":"2024-09-10T10:05:31.189791+0000","last_unstale":"2024-09-10T10:05:31.194581+0000","last_undegraded":"2024-09-10T10:05:31.194581+0000","last_fullsized":"2024-09-10T10:05:31.194581+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:24.521559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174289+0000","last_change":"2024-09-10T10:05:31.174289+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174289+0000","last_undegraded":"2024-09-10T10:05:31.174289+0000","last_fullsized":"2024-09-10T10:05:31.174289+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193922+0000","last_change":"2024-09-10T10:05:31.188150+0000","last_active":"2024-09-10T10:05:31.193922+0000","last_peered":"2024-09-10T10:05:31.193922+0000","last_clean":"2024-09-10T10:05:31.193922+0000","last_became_active":"2024-09-10T10:05:31.185519+0000","last_became_peered":"2024-09-10T10:05:31.185519+0000","last_unstale":"2024-09-10T10:05:31.193922+0000","last_undegraded":"2024-09-10T10:05:31.193922+0000","last_fullsized":"2024-09-10T10:05:31.193922+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:45.379567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179119+0000","last_change":"2024-09-10T10:05:31.179119+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179119+0000","last_undegraded":"2024-09-10T10:05:31.179119+0000","last_fullsized":"2024-09-10T10:05:31.179119+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197722+0000","last_change":"2024-09-10T10:05:31.194843+0000","last_active":"2024-09-10T10:05:31.197722+0000","last_peered":"2024-09-10T10:05:31.197722+0000","last_clean":"2024-09-10T10:05:31.197722+0000","last_became_active":"2024-09-10T10:05:31.194168+0000","last_became_peered":"2024-09-10T10:05:31.194168+0000","last_unstale":"2024-09-10T10:05:31.197722+0000","last_undegraded":"2024-09-10T10:05:31.197722+0000","last_fullsized":"2024-09-10T10:05:31.197722+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:55.908854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173030+0000","last_change":"2024-09-10T10:05:31.173030+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173030+0000","last_undegraded":"2024-09-10T10:05:31.173030+0000","last_fullsized":"2024-09-10T10:05:31.173030+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192401+0000","last_change":"2024-09-10T10:05:31.189711+0000","last_active":"2024-09-10T10:05:31.192401+0000","last_peered":"2024-09-10T10:05:31.192401+0000","last_clean":"2024-09-10T10:05:31.192401+0000","last_became_active":"2024-09-10T10:05:31.189471+0000","last_became_peered":"2024-09-10T10:05:31.189471+0000","last_unstale":"2024-09-10T10:05:31.192401+0000","last_undegraded":"2024-09-10T10:05:31.192401+0000","last_fullsized":"2024-09-10T10:05:31.192401+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:17.765192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173658+0000","last_change":"2024-09-10T10:05:31.173658+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173658+0000","last_undegraded":"2024-09-10T10:05:31.173658+0000","last_fullsized":"2024-09-10T10:05:31.173658+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.196348+0000","last_change":"2024-09-10T10:05:31.195254+0000","last_active":"2024-09-10T10:05:31.196348+0000","last_peered":"2024-09-10T10:05:31.196348+0000","last_clean":"2024-09-10T10:05:31.196348+0000","last_became_active":"2024-09-10T10:05:31.194909+0000","last_became_peered":"2024-09-10T10:05:31.194909+0000","last_unstale":"2024-09-10T10:05:31.196348+0000","last_undegraded":"2024-09-10T10:05:31.196348+0000","last_fullsized":"2024-09-10T10:05:31.196348+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:24.286717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179062+0000","last_change":"2024-09-10T10:05:31.179062+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179062+0000","last_undegraded":"2024-09-10T10:05:31.179062+0000","last_fullsized":"2024-09-10T10:05:31.179062+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.184862+0000","last_change":"2024-09-10T10:05:31.176701+0000","last_active":"2024-09-10T10:05:31.184862+0000","last_peered":"2024-09-10T10:05:31.184862+0000","last_clean":"2024-09-10T10:05:31.184862+0000","last_became_active":"2024-09-10T10:05:31.176425+0000","last_became_peered":"2024-09-10T10:05:31.176425+0000","last_unstale":"2024-09-10T10:05:31.184862+0000","last_undegraded":"2024-09-10T10:05:31.184862+0000","last_fullsized":"2024-09-10T10:05:31.184862+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:25.572927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.176025+0000","last_change":"2024-09-10T10:05:31.176025+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.176025+0000","last_undegraded":"2024-09-10T10:05:31.176025+0000","last_fullsized":"2024-09-10T10:05:31.176025+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194415+0000","last_change":"2024-09-10T10:05:31.193087+0000","last_active":"2024-09-10T10:05:31.194415+0000","last_peered":"2024-09-10T10:05:31.194415+0000","last_clean":"2024-09-10T10:05:31.194415+0000","last_became_active":"2024-09-10T10:05:31.192788+0000","last_became_peered":"2024-09-10T10:05:31.192788+0000","last_unstale":"2024-09-10T10:05:31.194415+0000","last_undegraded":"2024-09-10T10:05:31.194415+0000","last_fullsized":"2024-09-10T10:05:31.194415+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:14.387457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181741+0000","last_change":"2024-09-10T10:05:31.181741+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181741+0000","last_undegraded":"2024-09-10T10:05:31.181741+0000","last_fullsized":"2024-09-10T10:05:31.181741+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193842+0000","last_change":"2024-09-10T10:05:31.189469+0000","last_active":"2024-09-10T10:05:31.193842+0000","last_peered":"2024-09-10T10:05:31.193842+0000","last_clean":"2024-09-10T10:05:31.193842+0000","last_became_active":"2024-09-10T10:05:31.189175+0000","last_became_peered":"2024-09-10T10:05:31.189175+0000","last_unstale":"2024-09-10T10:05:31.193842+0000","last_undegraded":"2024-09-10T10:05:31.193842+0000","last_fullsized":"2024-09-10T10:05:31.193842+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:09.097792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179763+0000","last_change":"2024-09-10T10:05:31.179763+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179763+0000","last_undegraded":"2024-09-10T10:05:31.179763+0000","last_fullsized":"2024-09-10T10:05:31.179763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.184460+0000","last_change":"2024-09-10T10:05:31.176322+0000","last_active":"2024-09-10T10:05:31.184460+0000","last_peered":"2024-09-10T10:05:31.184460+0000","last_clean":"2024-09-10T10:05:31.184460+0000","last_became_active":"2024-09-10T10:05:31.175852+0000","last_became_peered":"2024-09-10T10:05:31.175852+0000","last_unstale":"2024-09-10T10:05:31.184460+0000","last_undegraded":"2024-09-10T10:05:31.184460+0000","last_fullsized":"2024-09-10T10:05:31.184460+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:54.926756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177666+0000","last_change":"2024-09-10T10:05:31.177666+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177666+0000","last_undegraded":"2024-09-10T10:05:31.177666+0000","last_fullsized":"2024-09-10T10:05:31.177666+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194461+0000","last_change":"2024-09-10T10:05:31.193098+0000","last_active":"2024-09-10T10:05:31.194461+0000","last_peered":"2024-09-10T10:05:31.194461+0000","last_clean":"2024-09-10T10:05:31.194461+0000","last_became_active":"2024-09-10T10:05:31.192779+0000","last_became_peered":"2024-09-10T10:05:31.192779+0000","last_unstale":"2024-09-10T10:05:31.194461+0000","last_undegraded":"2024-09-10T10:05:31.194461+0000","last_fullsized":"2024-09-10T10:05:31.194461+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:46.842387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173084+0000","last_change":"2024-09-10T10:05:31.173084+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173084+0000","last_undegraded":"2024-09-10T10:05:31.173084+0000","last_fullsized":"2024-09-10T10:05:31.173084+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194424+0000","last_change":"2024-09-10T10:05:31.193283+0000","last_active":"2024-09-10T10:05:31.194424+0000","last_peered":"2024-09-10T10:05:31.194424+0000","last_clean":"2024-09-10T10:05:31.194424+0000","last_became_active":"2024-09-10T10:05:31.192470+0000","last_became_peered":"2024-09-10T10:05:31.192470+0000","last_unstale":"2024-09-10T10:05:31.194424+0000","last_undegraded":"2024-09-10T10:05:31.194424+0000","last_fullsized":"2024-09-10T10:05:31.194424+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:50.910428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180942+0000","last_change":"2024-09-10T10:05:31.180942+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180942+0000","last_undegraded":"2024-09-10T10:05:31.180942+0000","last_fullsized":"2024-09-10T10:05:31.180942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197139+0000","last_change":"2024-09-10T10:05:31.194554+0000","last_active":"2024-09-10T10:05:31.197139+0000","last_peered":"2024-09-10T10:05:31.197139+0000","last_clean":"2024-09-10T10:05:31.197139+0000","last_became_active":"2024-09-10T10:05:31.194265+0000","last_became_peered":"2024-09-10T10:05:31.194265+0000","last_unstale":"2024-09-10T10:05:31.197139+0000","last_undegraded":"2024-09-10T10:05:31.197139+0000","last_fullsized":"2024-09-10T10:05:31.197139+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:34.402529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170703+0000","last_change":"2024-09-10T10:05:31.170703+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170703+0000","last_undegraded":"2024-09-10T10:05:31.170703+0000","last_fullsized":"2024-09-10T10:05:31.170703+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194117+0000","last_change":"2024-09-10T10:05:31.188898+0000","last_active":"2024-09-10T10:05:31.194117+0000","last_peered":"2024-09-10T10:05:31.194117+0000","last_clean":"2024-09-10T10:05:31.194117+0000","last_became_active":"2024-09-10T10:05:31.186075+0000","last_became_peered":"2024-09-10T10:05:31.186075+0000","last_unstale":"2024-09-10T10:05:31.194117+0000","last_undegraded":"2024-09-10T10:05:31.194117+0000","last_fullsized":"2024-09-10T10:05:31.194117+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:59.362162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173203+0000","last_change":"2024-09-10T10:05:31.173203+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173203+0000","last_undegraded":"2024-09-10T10:05:31.173203+0000","last_fullsized":"2024-09-10T10:05:31.173203+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.196060+0000","last_change":"2024-09-10T10:05:31.187311+0000","last_active":"2024-09-10T10:05:31.196060+0000","last_peered":"2024-09-10T10:05:31.196060+0000","last_clean":"2024-09-10T10:05:31.196060+0000","last_became_active":"2024-09-10T10:05:31.186827+0000","last_became_peered":"2024-09-10T10:05:31.186827+0000","last_unstale":"2024-09-10T10:05:31.196060+0000","last_undegraded":"2024-09-10T10:05:31.196060+0000","last_fullsized":"2024-09-10T10:05:31.196060+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:32:57.257293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174497+0000","last_change":"2024-09-10T10:05:31.174497+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174497+0000","last_undegraded":"2024-09-10T10:05:31.174497+0000","last_fullsized":"2024-09-10T10:05:31.174497+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.189401+0000","last_change":"2024-09-10T10:05:31.187284+0000","last_active":"2024-09-10T10:05:31.189401+0000","last_peered":"2024-09-10T10:05:31.189401+0000","last_clean":"2024-09-10T10:05:31.189401+0000","last_became_active":"2024-09-10T10:05:31.186766+0000","last_became_peered":"2024-09-10T10:05:31.186766+0000","last_unstale":"2024-09-10T10:05:31.189401+0000","last_undegraded":"2024-09-10T10:05:31.189401+0000","last_fullsized":"2024-09-10T10:05:31.189401+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:11.425378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172755+0000","last_change":"2024-09-10T10:05:31.172755+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172755+0000","last_undegraded":"2024-09-10T10:05:31.172755+0000","last_fullsized":"2024-09-10T10:05:31.172755+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192180+0000","last_change":"2024-09-10T10:05:31.188601+0000","last_active":"2024-09-10T10:05:31.192180+0000","last_peered":"2024-09-10T10:05:31.192180+0000","last_clean":"2024-09-10T10:05:31.192180+0000","last_became_active":"2024-09-10T10:05:31.188289+0000","last_became_peered":"2024-09-10T10:05:31.188289+0000","last_unstale":"2024-09-10T10:05:31.192180+0000","last_undegraded":"2024-09-10T10:05:31.192180+0000","last_fullsized":"2024-09-10T10:05:31.192180+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:07.295402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178608+0000","last_change":"2024-09-10T10:05:31.178608+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178608+0000","last_undegraded":"2024-09-10T10:05:31.178608+0000","last_fullsized":"2024-09-10T10:05:31.178608+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.198112+0000","last_change":"2024-09-10T10:05:31.196820+0000","last_active":"2024-09-10T10:05:31.198112+0000","last_peered":"2024-09-10T10:05:31.198112+0000","last_clean":"2024-09-10T10:05:31.198112+0000","last_became_active":"2024-09-10T10:05:31.196557+0000","last_became_peered":"2024-09-10T10:05:31.196557+0000","last_unstale":"2024-09-10T10:05:31.198112+0000","last_undegraded":"2024-09-10T10:05:31.198112+0000","last_fullsized":"2024-09-10T10:05:31.198112+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:01.069466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170129+0000","last_change":"2024-09-10T10:05:31.170129+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170129+0000","last_undegraded":"2024-09-10T10:05:31.170129+0000","last_fullsized":"2024-09-10T10:05:31.170129+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.198182+0000","last_change":"2024-09-10T10:05:31.197033+0000","last_active":"2024-09-10T10:05:31.198182+0000","last_peered":"2024-09-10T10:05:31.198182+0000","last_clean":"2024-09-10T10:05:31.198182+0000","last_became_active":"2024-09-10T10:05:31.196739+0000","last_became_peered":"2024-09-10T10:05:31.196739+0000","last_unstale":"2024-09-10T10:05:31.198182+0000","last_undegraded":"2024-09-10T10:05:31.198182+0000","last_fullsized":"2024-09-10T10:05:31.198182+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:23.494858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170949+0000","last_change":"2024-09-10T10:05:31.170949+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170949+0000","last_undegraded":"2024-09-10T10:05:31.170949+0000","last_fullsized":"2024-09-10T10:05:31.170949+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.187377+0000","last_change":"2024-09-10T10:05:31.179485+0000","last_active":"2024-09-10T10:05:31.187377+0000","last_peered":"2024-09-10T10:05:31.187377+0000","last_clean":"2024-09-10T10:05:31.187377+0000","last_became_active":"2024-09-10T10:05:31.178050+0000","last_became_peered":"2024-09-10T10:05:31.178050+0000","last_unstale":"2024-09-10T10:05:31.187377+0000","last_undegraded":"2024-09-10T10:05:31.187377+0000","last_fullsized":"2024-09-10T10:05:31.187377+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:17.213280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179275+0000","last_change":"2024-09-10T10:05:31.179275+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179275+0000","last_undegraded":"2024-09-10T10:05:31.179275+0000","last_fullsized":"2024-09-10T10:05:31.179275+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194166+0000","last_change":"2024-09-10T10:05:31.193385+0000","last_active":"2024-09-10T10:05:31.194166+0000","last_peered":"2024-09-10T10:05:31.194166+0000","last_clean":"2024-09-10T10:05:31.194166+0000","last_became_active":"2024-09-10T10:05:31.193085+0000","last_became_peered":"2024-09-10T10:05:31.193085+0000","last_unstale":"2024-09-10T10:05:31.194166+0000","last_undegraded":"2024-09-10T10:05:31.194166+0000","last_fullsized":"2024-09-10T10:05:31.194166+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:13.939828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178450+0000","last_change":"2024-09-10T10:05:31.178450+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178450+0000","last_undegraded":"2024-09-10T10:05:31.178450+0000","last_fullsized":"2024-09-10T10:05:31.178450+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.188996+0000","last_change":"2024-09-10T10:05:31.186964+0000","last_active":"2024-09-10T10:05:31.188996+0000","last_peered":"2024-09-10T10:05:31.188996+0000","last_clean":"2024-09-10T10:05:31.188996+0000","last_became_active":"2024-09-10T10:05:31.186613+0000","last_became_peered":"2024-09-10T10:05:31.186613+0000","last_unstale":"2024-09-10T10:05:31.188996+0000","last_undegraded":"2024-09-10T10:05:31.188996+0000","last_fullsized":"2024-09-10T10:05:31.188996+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:59.668546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172624+0000","last_change":"2024-09-10T10:05:31.172624+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172624+0000","last_undegraded":"2024-09-10T10:05:31.172624+0000","last_fullsized":"2024-09-10T10:05:31.172624+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190738+0000","last_change":"2024-09-10T10:05:31.187417+0000","last_active":"2024-09-10T10:05:31.190738+0000","last_peered":"2024-09-10T10:05:31.190738+0000","last_clean":"2024-09-10T10:05:31.190738+0000","last_became_active":"2024-09-10T10:05:31.187030+0000","last_became_peered":"2024-09-10T10:05:31.187030+0000","last_unstale":"2024-09-10T10:05:31.190738+0000","last_undegraded":"2024-09-10T10:05:31.190738+0000","last_fullsized":"2024-09-10T10:05:31.190738+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:13.626786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171896+0000","last_change":"2024-09-10T10:05:31.171896+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171896+0000","last_undegraded":"2024-09-10T10:05:31.171896+0000","last_fullsized":"2024-09-10T10:05:31.171896+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197321+0000","last_change":"2024-09-10T10:05:31.194388+0000","last_active":"2024-09-10T10:05:31.197321+0000","last_peered":"2024-09-10T10:05:31.197321+0000","last_clean":"2024-09-10T10:05:31.197321+0000","last_became_active":"2024-09-10T10:05:31.193967+0000","last_became_peered":"2024-09-10T10:05:31.193967+0000","last_unstale":"2024-09-10T10:05:31.197321+0000","last_undegraded":"2024-09-10T10:05:31.197321+0000","last_fullsized":"2024-09-10T10:05:31.197321+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:53.149484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.175422+0000","last_change":"2024-09-10T10:05:31.175422+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.175422+0000","last_undegraded":"2024-09-10T10:05:31.175422+0000","last_fullsized":"2024-09-10T10:05:31.175422+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192494+0000","last_change":"2024-09-10T10:05:31.191205+0000","last_active":"2024-09-10T10:05:31.192494+0000","last_peered":"2024-09-10T10:05:31.192494+0000","last_clean":"2024-09-10T10:05:31.192494+0000","last_became_active":"2024-09-10T10:05:31.190885+0000","last_became_peered":"2024-09-10T10:05:31.190885+0000","last_unstale":"2024-09-10T10:05:31.192494+0000","last_undegraded":"2024-09-10T10:05:31.192494+0000","last_fullsized":"2024-09-10T10:05:31.192494+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:58.487180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":444,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.180502+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:05:31.180502+0000","last_peered":"2024-09-10T10:05:31.180502+0000","last_clean":"2024-09-10T10:05:31.180502+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:05:31.180502+0000","last_undegraded":"2024-09-10T10:05:31.180502+0000","last_fullsized":"2024-09-10T10:05:31.180502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2490368,"data_stored":1425104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509530,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542236,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37188,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334652,"statfs":{"total":96636764160,"available":96598683648,"internally_reserved":0,"allocated":1884160,"data_stored":828405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37188,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334652,"statfs":{"total":96636764160,"available":96598683648,"internally_reserved":0,"allocated":1884160,"data_stored":828405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37204,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2490368,"data_stored":1425104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:05:36.268 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:05:36.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.441+0000 7f70d9c00640 1 Processor -- start 2024-09-10T10:05:36.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.441+0000 7f70d9c00640 1 -- start start 2024-09-10T10:05:36.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d9c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f70d415c560 0x7f70d406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d9c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d406f430 con 0x7f70d4155970 2024-09-10T10:05:36.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d9c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d4155d50 con 0x7f70d406f010 2024-09-10T10:05:36.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d9c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f70d40b4570 con 0x7f70d415c560 2024-09-10T10:05:36.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d2a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f70d4155970 0x7f70d40cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:48948/0 (socket says 172.21.15.110:48948) 2024-09-10T10:05:36.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d2a00640 1 -- 172.21.15.110:0/783267444 learned_addr learned my addr 172.21.15.110:0/783267444 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:36.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d2000640 1 --2- 172.21.15.110:0/783267444 >> v2:172.21.15.145:3300/0 conn(0x7f70d415c560 0x7f70d406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d1600640 1 -- 172.21.15.110:0/783267444 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1834514723 0 0) 0x7f70d406f430 con 0x7f70d4155970 2024-09-10T10:05:36.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d1600640 1 -- 172.21.15.110:0/783267444 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70b8003530 con 0x7f70d4155970 2024-09-10T10:05:36.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d2000640 1 -- 172.21.15.110:0/783267444 >> v1:172.21.15.145:6789/0 conn(0x7f70d406f010 legacy=0x7f70d40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d2000640 1 -- 172.21.15.110:0/783267444 >> v1:172.21.15.110:6789/0 conn(0x7f70d4155970 legacy=0x7f70d40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.445+0000 7f70d2000640 1 -- 172.21.15.110:0/783267444 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d40b34d0 con 0x7f70d415c560 2024-09-10T10:05:36.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d2000640 1 --2- 172.21.15.110:0/783267444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 0x7f70d406ead0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f70c00097c0 tx=0x7f70c001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=549e2706e3b13a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d1600640 1 -- 172.21.15.110:0/783267444 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00a7070 con 0x7f70d415c560 2024-09-10T10:05:36.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d1600640 1 -- 172.21.15.110:0/783267444 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c00b0070 con 0x7f70d415c560 2024-09-10T10:05:36.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d1600640 1 -- 172.21.15.110:0/783267444 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00ab890 con 0x7f70d415c560 2024-09-10T10:05:36.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 -- 172.21.15.110:0/783267444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 msgr2=0x7f70d406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 --2- 172.21.15.110:0/783267444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 0x7f70d406ead0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f70c00097c0 tx=0x7f70c001c470 comp rx=0 tx=0).stop 2024-09-10T10:05:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 -- 172.21.15.110:0/783267444 shutdown_connections 2024-09-10T10:05:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 --2- 172.21.15.110:0/783267444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 0x7f70d406ead0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 -- 172.21.15.110:0/783267444 >> 172.21.15.110:0/783267444 conn(0x7f70d406bc50 msgr2=0x7f70d406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 -- 172.21.15.110:0/783267444 shutdown_connections 2024-09-10T10:05:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 -- 172.21.15.110:0/783267444 wait complete. 2024-09-10T10:05:36.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 Processor -- start 2024-09-10T10:05:36.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.449+0000 7f70d9c00640 1 -- start start 2024-09-10T10:05:36.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d406f010 0x7f70d414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d4155970 0x7f70d415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 0x7f70d4154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f70d40cf340 con 0x7f70d406f010 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f70d40b4050 con 0x7f70d415c560 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f70d40cf4c0 con 0x7f70d4155970 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d406f010 0x7f70d414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d406f010 0x7f70d414a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34296/0 (socket says 172.21.15.110:34296) 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 -- 172.21.15.110:0/1883964868 learned_addr learned my addr 172.21.15.110:0/1883964868 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2000640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d4155970 0x7f70d415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d3400640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 0x7f70d4154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 -- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d4155970 msgr2=0x7f70d415dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d4155970 0x7f70d415dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 -- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 msgr2=0x7f70d4154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 0x7f70d4154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 -- 172.21.15.110:0/1883964868 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d4150500 con 0x7f70d406f010 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2000640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d4155970 0x7f70d415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d2a00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d406f010 0x7f70d414a370 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f70c400cbc0 tx=0x7f70c400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70caa00640 1 -- 172.21.15.110:0/1883964868 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4017070 con 0x7f70d406f010 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70caa00640 1 -- 172.21.15.110:0/1883964868 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c400ad40 con 0x7f70d406f010 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70caa00640 1 -- 172.21.15.110:0/1883964868 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4012600 con 0x7f70d406f010 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d4150220 con 0x7f70d406f010 2024-09-10T10:05:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d415d7e0 con 0x7f70d406f010 2024-09-10T10:05:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d406ead0 con 0x7f70d406f010 2024-09-10T10:05:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.453+0000 7f70caa00640 1 -- 172.21.15.110:0/1883964868 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f70c401a070 con 0x7f70d406f010 2024-09-10T10:05:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.457+0000 7f70caa00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70b40680a0 0x7f70b406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.457+0000 7f70caa00640 1 -- 172.21.15.110:0/1883964868 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f70c4086cc0 con 0x7f70d406f010 2024-09-10T10:05:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.457+0000 7f70d2000640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70b40680a0 0x7f70b406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.457+0000 7f70d2000640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70b40680a0 0x7f70b406a560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f70c00af000 tx=0x7f70c00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.461+0000 7f70caa00640 1 -- 172.21.15.110:0/1883964868 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f70c40561e0 con 0x7f70d406f010 2024-09-10T10:05:36.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.613+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70d41663d0 con 0x7f70b40680a0 2024-09-10T10:05:36.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.621+0000 7f70caa00640 1 -- 172.21.15.110:0/1883964868 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247088 (secure 0 0 0) 0x7f70d41663d0 con 0x7f70b40680a0 2024-09-10T10:05:36.624 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:05:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70b40680a0 msgr2=0x7f70b406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70b40680a0 0x7f70b406a560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f70c00af000 tx=0x7f70c00a7000 comp rx=0 tx=0).stop 2024-09-10T10:05:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d406f010 msgr2=0x7f70d414a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d406f010 0x7f70d414a370 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f70c400cbc0 tx=0x7f70c400a430 comp rx=0 tx=0).stop 2024-09-10T10:05:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 shutdown_connections 2024-09-10T10:05:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70b40680a0 0x7f70b406a560 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70d415c560 0x7f70d4154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f70d4155970 0x7f70d415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 --2- 172.21.15.110:0/1883964868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70d406f010 0x7f70d414a370 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 >> 172.21.15.110:0/1883964868 conn(0x7f70d406bc50 msgr2=0x7f70d40ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 shutdown_connections 2024-09-10T10:05:36.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.625+0000 7f70d9c00640 1 -- 172.21.15.110:0/1883964868 wait complete. 2024-09-10T10:05:36.643 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-10T10:05:35.740163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282112,"kb_used_data":15952,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692608,"statfs":{"total":773094113280,"available":772805230592,"internally_reserved":0,"allocated":16334848,"data_stored":7845962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.578008"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178157+0000","last_change":"2024-09-10T10:05:31.178157+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178157+0000","last_undegraded":"2024-09-10T10:05:31.178157+0000","last_fullsized":"2024-09-10T10:05:31.178157+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173328+0000","last_change":"2024-09-10T10:05:31.173328+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173328+0000","last_undegraded":"2024-09-10T10:05:31.173328+0000","last_fullsized":"2024-09-10T10:05:31.173328+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179146+0000","last_change":"2024-09-10T10:05:31.179146+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179146+0000","last_undegraded":"2024-09-10T10:05:31.179146+0000","last_fullsized":"2024-09-10T10:05:31.179146+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173862+0000","last_change":"2024-09-10T10:05:31.173862+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173862+0000","last_undegraded":"2024-09-10T10:05:31.173862+0000","last_fullsized":"2024-09-10T10:05:31.173862+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172850+0000","last_change":"2024-09-10T10:05:31.172850+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172850+0000","last_undegraded":"2024-09-10T10:05:31.172850+0000","last_fullsized":"2024-09-10T10:05:31.172850+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173582+0000","last_change":"2024-09-10T10:05:31.173582+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173582+0000","last_undegraded":"2024-09-10T10:05:31.173582+0000","last_fullsized":"2024-09-10T10:05:31.173582+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171259+0000","last_change":"2024-09-10T10:05:31.171259+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171259+0000","last_undegraded":"2024-09-10T10:05:31.171259+0000","last_fullsized":"2024-09-10T10:05:31.171259+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171214+0000","last_change":"2024-09-10T10:05:31.171214+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171214+0000","last_undegraded":"2024-09-10T10:05:31.171214+0000","last_fullsized":"2024-09-10T10:05:31.171214+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180484+0000","last_change":"2024-09-10T10:05:31.180484+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180484+0000","last_undegraded":"2024-09-10T10:05:31.180484+0000","last_fullsized":"2024-09-10T10:05:31.180484+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.187473+0000","last_change":"2024-09-10T10:05:31.187473+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.187473+0000","last_undegraded":"2024-09-10T10:05:31.187473+0000","last_fullsized":"2024-09-10T10:05:31.187473+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170730+0000","last_change":"2024-09-10T10:05:31.170730+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170730+0000","last_undegraded":"2024-09-10T10:05:31.170730+0000","last_fullsized":"2024-09-10T10:05:31.170730+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173444+0000","last_change":"2024-09-10T10:05:31.173444+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173444+0000","last_undegraded":"2024-09-10T10:05:31.173444+0000","last_fullsized":"2024-09-10T10:05:31.173444+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173147+0000","last_change":"2024-09-10T10:05:31.173147+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173147+0000","last_undegraded":"2024-09-10T10:05:31.173147+0000","last_fullsized":"2024-09-10T10:05:31.173147+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178165+0000","last_change":"2024-09-10T10:05:31.178165+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178165+0000","last_undegraded":"2024-09-10T10:05:31.178165+0000","last_fullsized":"2024-09-10T10:05:31.178165+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172248+0000","last_change":"2024-09-10T10:05:31.172248+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172248+0000","last_undegraded":"2024-09-10T10:05:31.172248+0000","last_fullsized":"2024-09-10T10:05:31.172248+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170807+0000","last_change":"2024-09-10T10:05:31.170807+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170807+0000","last_undegraded":"2024-09-10T10:05:31.170807+0000","last_fullsized":"2024-09-10T10:05:31.170807+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173631+0000","last_change":"2024-09-10T10:05:31.173631+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173631+0000","last_undegraded":"2024-09-10T10:05:31.173631+0000","last_fullsized":"2024-09-10T10:05:31.173631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181226+0000","last_change":"2024-09-10T10:05:31.181226+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181226+0000","last_undegraded":"2024-09-10T10:05:31.181226+0000","last_fullsized":"2024-09-10T10:05:31.181226+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180425+0000","last_change":"2024-09-10T10:05:31.180425+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180425+0000","last_undegraded":"2024-09-10T10:05:31.180425+0000","last_fullsized":"2024-09-10T10:05:31.180425+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.175359+0000","last_change":"2024-09-10T10:05:31.175359+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.175359+0000","last_undegraded":"2024-09-10T10:05:31.175359+0000","last_fullsized":"2024-09-10T10:05:31.175359+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172406+0000","last_change":"2024-09-10T10:05:31.172406+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172406+0000","last_undegraded":"2024-09-10T10:05:31.172406+0000","last_fullsized":"2024-09-10T10:05:31.172406+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181999+0000","last_change":"2024-09-10T10:05:31.181999+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181999+0000","last_undegraded":"2024-09-10T10:05:31.181999+0000","last_fullsized":"2024-09-10T10:05:31.181999+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177007+0000","last_change":"2024-09-10T10:05:31.177007+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177007+0000","last_undegraded":"2024-09-10T10:05:31.177007+0000","last_fullsized":"2024-09-10T10:05:31.177007+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170320+0000","last_change":"2024-09-10T10:05:31.170320+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170320+0000","last_undegraded":"2024-09-10T10:05:31.170320+0000","last_fullsized":"2024-09-10T10:05:31.170320+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172778+0000","last_change":"2024-09-10T10:05:31.172778+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172778+0000","last_undegraded":"2024-09-10T10:05:31.172778+0000","last_fullsized":"2024-09-10T10:05:31.172778+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171873+0000","last_change":"2024-09-10T10:05:31.171873+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171873+0000","last_undegraded":"2024-09-10T10:05:31.171873+0000","last_fullsized":"2024-09-10T10:05:31.171873+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177404+0000","last_change":"2024-09-10T10:05:31.177404+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177404+0000","last_undegraded":"2024-09-10T10:05:31.177404+0000","last_fullsized":"2024-09-10T10:05:31.177404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174881+0000","last_change":"2024-09-10T10:05:31.174881+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174881+0000","last_undegraded":"2024-09-10T10:05:31.174881+0000","last_fullsized":"2024-09-10T10:05:31.174881+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172702+0000","last_change":"2024-09-10T10:05:31.172702+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172702+0000","last_undegraded":"2024-09-10T10:05:31.172702+0000","last_fullsized":"2024-09-10T10:05:31.172702+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177678+0000","last_change":"2024-09-10T10:05:31.177678+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177678+0000","last_undegraded":"2024-09-10T10:05:31.177678+0000","last_fullsized":"2024-09-10T10:05:31.177678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173678+0000","last_change":"2024-09-10T10:05:31.173678+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173678+0000","last_undegraded":"2024-09-10T10:05:31.173678+0000","last_fullsized":"2024-09-10T10:05:31.173678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172528+0000","last_change":"2024-09-10T10:05:31.172528+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172528+0000","last_undegraded":"2024-09-10T10:05:31.172528+0000","last_fullsized":"2024-09-10T10:05:31.172528+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170496+0000","last_change":"2024-09-10T10:05:31.170496+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170496+0000","last_undegraded":"2024-09-10T10:05:31.170496+0000","last_fullsized":"2024-09-10T10:05:31.170496+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193482+0000","last_change":"2024-09-10T10:05:31.187728+0000","last_active":"2024-09-10T10:05:31.193482+0000","last_peered":"2024-09-10T10:05:31.193482+0000","last_clean":"2024-09-10T10:05:31.193482+0000","last_became_active":"2024-09-10T10:05:31.186044+0000","last_became_peered":"2024-09-10T10:05:31.186044+0000","last_unstale":"2024-09-10T10:05:31.193482+0000","last_undegraded":"2024-09-10T10:05:31.193482+0000","last_fullsized":"2024-09-10T10:05:31.193482+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:28.803480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179647+0000","last_change":"2024-09-10T10:05:31.179647+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179647+0000","last_undegraded":"2024-09-10T10:05:31.179647+0000","last_fullsized":"2024-09-10T10:05:31.179647+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.189032+0000","last_change":"2024-09-10T10:05:31.187161+0000","last_active":"2024-09-10T10:05:31.189032+0000","last_peered":"2024-09-10T10:05:31.189032+0000","last_clean":"2024-09-10T10:05:31.189032+0000","last_became_active":"2024-09-10T10:05:31.186727+0000","last_became_peered":"2024-09-10T10:05:31.186727+0000","last_unstale":"2024-09-10T10:05:31.189032+0000","last_undegraded":"2024-09-10T10:05:31.189032+0000","last_fullsized":"2024-09-10T10:05:31.189032+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:15.098413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174762+0000","last_change":"2024-09-10T10:05:31.174762+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174762+0000","last_undegraded":"2024-09-10T10:05:31.174762+0000","last_fullsized":"2024-09-10T10:05:31.174762+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197736+0000","last_change":"2024-09-10T10:05:31.196039+0000","last_active":"2024-09-10T10:05:31.197736+0000","last_peered":"2024-09-10T10:05:31.197736+0000","last_clean":"2024-09-10T10:05:31.197736+0000","last_became_active":"2024-09-10T10:05:31.195762+0000","last_became_peered":"2024-09-10T10:05:31.195762+0000","last_unstale":"2024-09-10T10:05:31.197736+0000","last_undegraded":"2024-09-10T10:05:31.197736+0000","last_fullsized":"2024-09-10T10:05:31.197736+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:08.377153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177235+0000","last_change":"2024-09-10T10:05:31.177235+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177235+0000","last_undegraded":"2024-09-10T10:05:31.177235+0000","last_fullsized":"2024-09-10T10:05:31.177235+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190648+0000","last_change":"2024-09-10T10:05:31.184956+0000","last_active":"2024-09-10T10:05:31.190648+0000","last_peered":"2024-09-10T10:05:31.190648+0000","last_clean":"2024-09-10T10:05:31.190648+0000","last_became_active":"2024-09-10T10:05:31.184628+0000","last_became_peered":"2024-09-10T10:05:31.184628+0000","last_unstale":"2024-09-10T10:05:31.190648+0000","last_undegraded":"2024-09-10T10:05:31.190648+0000","last_fullsized":"2024-09-10T10:05:31.190648+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:20.844260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181360+0000","last_change":"2024-09-10T10:05:31.181360+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181360+0000","last_undegraded":"2024-09-10T10:05:31.181360+0000","last_fullsized":"2024-09-10T10:05:31.181360+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193807+0000","last_change":"2024-09-10T10:05:31.189563+0000","last_active":"2024-09-10T10:05:31.193807+0000","last_peered":"2024-09-10T10:05:31.193807+0000","last_clean":"2024-09-10T10:05:31.193807+0000","last_became_active":"2024-09-10T10:05:31.189169+0000","last_became_peered":"2024-09-10T10:05:31.189169+0000","last_unstale":"2024-09-10T10:05:31.193807+0000","last_undegraded":"2024-09-10T10:05:31.193807+0000","last_fullsized":"2024-09-10T10:05:31.193807+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:46.694678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171144+0000","last_change":"2024-09-10T10:05:31.171144+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171144+0000","last_undegraded":"2024-09-10T10:05:31.171144+0000","last_fullsized":"2024-09-10T10:05:31.171144+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190551+0000","last_change":"2024-09-10T10:05:31.184111+0000","last_active":"2024-09-10T10:05:31.190551+0000","last_peered":"2024-09-10T10:05:31.190551+0000","last_clean":"2024-09-10T10:05:31.190551+0000","last_became_active":"2024-09-10T10:05:31.183673+0000","last_became_peered":"2024-09-10T10:05:31.183673+0000","last_unstale":"2024-09-10T10:05:31.190551+0000","last_undegraded":"2024-09-10T10:05:31.190551+0000","last_fullsized":"2024-09-10T10:05:31.190551+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:21.245568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170694+0000","last_change":"2024-09-10T10:05:31.170694+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170694+0000","last_undegraded":"2024-09-10T10:05:31.170694+0000","last_fullsized":"2024-09-10T10:05:31.170694+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194428+0000","last_change":"2024-09-10T10:05:31.193233+0000","last_active":"2024-09-10T10:05:31.194428+0000","last_peered":"2024-09-10T10:05:31.194428+0000","last_clean":"2024-09-10T10:05:31.194428+0000","last_became_active":"2024-09-10T10:05:31.192971+0000","last_became_peered":"2024-09-10T10:05:31.192971+0000","last_unstale":"2024-09-10T10:05:31.194428+0000","last_undegraded":"2024-09-10T10:05:31.194428+0000","last_fullsized":"2024-09-10T10:05:31.194428+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:04.862858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178681+0000","last_change":"2024-09-10T10:05:31.178681+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178681+0000","last_undegraded":"2024-09-10T10:05:31.178681+0000","last_fullsized":"2024-09-10T10:05:31.178681+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190231+0000","last_change":"2024-09-10T10:05:31.185108+0000","last_active":"2024-09-10T10:05:31.190231+0000","last_peered":"2024-09-10T10:05:31.190231+0000","last_clean":"2024-09-10T10:05:31.190231+0000","last_became_active":"2024-09-10T10:05:31.184889+0000","last_became_peered":"2024-09-10T10:05:31.184889+0000","last_unstale":"2024-09-10T10:05:31.190231+0000","last_undegraded":"2024-09-10T10:05:31.190231+0000","last_fullsized":"2024-09-10T10:05:31.190231+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:57.982434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170177+0000","last_change":"2024-09-10T10:05:31.170177+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170177+0000","last_undegraded":"2024-09-10T10:05:31.170177+0000","last_fullsized":"2024-09-10T10:05:31.170177+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194581+0000","last_change":"2024-09-10T10:05:31.190056+0000","last_active":"2024-09-10T10:05:31.194581+0000","last_peered":"2024-09-10T10:05:31.194581+0000","last_clean":"2024-09-10T10:05:31.194581+0000","last_became_active":"2024-09-10T10:05:31.189791+0000","last_became_peered":"2024-09-10T10:05:31.189791+0000","last_unstale":"2024-09-10T10:05:31.194581+0000","last_undegraded":"2024-09-10T10:05:31.194581+0000","last_fullsized":"2024-09-10T10:05:31.194581+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:24.521559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174289+0000","last_change":"2024-09-10T10:05:31.174289+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174289+0000","last_undegraded":"2024-09-10T10:05:31.174289+0000","last_fullsized":"2024-09-10T10:05:31.174289+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193922+0000","last_change":"2024-09-10T10:05:31.188150+0000","last_active":"2024-09-10T10:05:31.193922+0000","last_peered":"2024-09-10T10:05:31.193922+0000","last_clean":"2024-09-10T10:05:31.193922+0000","last_became_active":"2024-09-10T10:05:31.185519+0000","last_became_peered":"2024-09-10T10:05:31.185519+0000","last_unstale":"2024-09-10T10:05:31.193922+0000","last_undegraded":"2024-09-10T10:05:31.193922+0000","last_fullsized":"2024-09-10T10:05:31.193922+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:45.379567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179119+0000","last_change":"2024-09-10T10:05:31.179119+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179119+0000","last_undegraded":"2024-09-10T10:05:31.179119+0000","last_fullsized":"2024-09-10T10:05:31.179119+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197722+0000","last_change":"2024-09-10T10:05:31.194843+0000","last_active":"2024-09-10T10:05:31.197722+0000","last_peered":"2024-09-10T10:05:31.197722+0000","last_clean":"2024-09-10T10:05:31.197722+0000","last_became_active":"2024-09-10T10:05:31.194168+0000","last_became_peered":"2024-09-10T10:05:31.194168+0000","last_unstale":"2024-09-10T10:05:31.197722+0000","last_undegraded":"2024-09-10T10:05:31.197722+0000","last_fullsized":"2024-09-10T10:05:31.197722+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:55.908854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173030+0000","last_change":"2024-09-10T10:05:31.173030+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173030+0000","last_undegraded":"2024-09-10T10:05:31.173030+0000","last_fullsized":"2024-09-10T10:05:31.173030+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192401+0000","last_change":"2024-09-10T10:05:31.189711+0000","last_active":"2024-09-10T10:05:31.192401+0000","last_peered":"2024-09-10T10:05:31.192401+0000","last_clean":"2024-09-10T10:05:31.192401+0000","last_became_active":"2024-09-10T10:05:31.189471+0000","last_became_peered":"2024-09-10T10:05:31.189471+0000","last_unstale":"2024-09-10T10:05:31.192401+0000","last_undegraded":"2024-09-10T10:05:31.192401+0000","last_fullsized":"2024-09-10T10:05:31.192401+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:17.765192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173658+0000","last_change":"2024-09-10T10:05:31.173658+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173658+0000","last_undegraded":"2024-09-10T10:05:31.173658+0000","last_fullsized":"2024-09-10T10:05:31.173658+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.196348+0000","last_change":"2024-09-10T10:05:31.195254+0000","last_active":"2024-09-10T10:05:31.196348+0000","last_peered":"2024-09-10T10:05:31.196348+0000","last_clean":"2024-09-10T10:05:31.196348+0000","last_became_active":"2024-09-10T10:05:31.194909+0000","last_became_peered":"2024-09-10T10:05:31.194909+0000","last_unstale":"2024-09-10T10:05:31.196348+0000","last_undegraded":"2024-09-10T10:05:31.196348+0000","last_fullsized":"2024-09-10T10:05:31.196348+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:24.286717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179062+0000","last_change":"2024-09-10T10:05:31.179062+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179062+0000","last_undegraded":"2024-09-10T10:05:31.179062+0000","last_fullsized":"2024-09-10T10:05:31.179062+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.184862+0000","last_change":"2024-09-10T10:05:31.176701+0000","last_active":"2024-09-10T10:05:31.184862+0000","last_peered":"2024-09-10T10:05:31.184862+0000","last_clean":"2024-09-10T10:05:31.184862+0000","last_became_active":"2024-09-10T10:05:31.176425+0000","last_became_peered":"2024-09-10T10:05:31.176425+0000","last_unstale":"2024-09-10T10:05:31.184862+0000","last_undegraded":"2024-09-10T10:05:31.184862+0000","last_fullsized":"2024-09-10T10:05:31.184862+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:25.572927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.176025+0000","last_change":"2024-09-10T10:05:31.176025+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.176025+0000","last_undegraded":"2024-09-10T10:05:31.176025+0000","last_fullsized":"2024-09-10T10:05:31.176025+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194415+0000","last_change":"2024-09-10T10:05:31.193087+0000","last_active":"2024-09-10T10:05:31.194415+0000","last_peered":"2024-09-10T10:05:31.194415+0000","last_clean":"2024-09-10T10:05:31.194415+0000","last_became_active":"2024-09-10T10:05:31.192788+0000","last_became_peered":"2024-09-10T10:05:31.192788+0000","last_unstale":"2024-09-10T10:05:31.194415+0000","last_undegraded":"2024-09-10T10:05:31.194415+0000","last_fullsized":"2024-09-10T10:05:31.194415+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:14.387457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181741+0000","last_change":"2024-09-10T10:05:31.181741+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181741+0000","last_undegraded":"2024-09-10T10:05:31.181741+0000","last_fullsized":"2024-09-10T10:05:31.181741+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193842+0000","last_change":"2024-09-10T10:05:31.189469+0000","last_active":"2024-09-10T10:05:31.193842+0000","last_peered":"2024-09-10T10:05:31.193842+0000","last_clean":"2024-09-10T10:05:31.193842+0000","last_became_active":"2024-09-10T10:05:31.189175+0000","last_became_peered":"2024-09-10T10:05:31.189175+0000","last_unstale":"2024-09-10T10:05:31.193842+0000","last_undegraded":"2024-09-10T10:05:31.193842+0000","last_fullsized":"2024-09-10T10:05:31.193842+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:09.097792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179763+0000","last_change":"2024-09-10T10:05:31.179763+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179763+0000","last_undegraded":"2024-09-10T10:05:31.179763+0000","last_fullsized":"2024-09-10T10:05:31.179763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.184460+0000","last_change":"2024-09-10T10:05:31.176322+0000","last_active":"2024-09-10T10:05:31.184460+0000","last_peered":"2024-09-10T10:05:31.184460+0000","last_clean":"2024-09-10T10:05:31.184460+0000","last_became_active":"2024-09-10T10:05:31.175852+0000","last_became_peered":"2024-09-10T10:05:31.175852+0000","last_unstale":"2024-09-10T10:05:31.184460+0000","last_undegraded":"2024-09-10T10:05:31.184460+0000","last_fullsized":"2024-09-10T10:05:31.184460+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:54.926756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177666+0000","last_change":"2024-09-10T10:05:31.177666+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177666+0000","last_undegraded":"2024-09-10T10:05:31.177666+0000","last_fullsized":"2024-09-10T10:05:31.177666+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194461+0000","last_change":"2024-09-10T10:05:31.193098+0000","last_active":"2024-09-10T10:05:31.194461+0000","last_peered":"2024-09-10T10:05:31.194461+0000","last_clean":"2024-09-10T10:05:31.194461+0000","last_became_active":"2024-09-10T10:05:31.192779+0000","last_became_peered":"2024-09-10T10:05:31.192779+0000","last_unstale":"2024-09-10T10:05:31.194461+0000","last_undegraded":"2024-09-10T10:05:31.194461+0000","last_fullsized":"2024-09-10T10:05:31.194461+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:46.842387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173084+0000","last_change":"2024-09-10T10:05:31.173084+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173084+0000","last_undegraded":"2024-09-10T10:05:31.173084+0000","last_fullsized":"2024-09-10T10:05:31.173084+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194424+0000","last_change":"2024-09-10T10:05:31.193283+0000","last_active":"2024-09-10T10:05:31.194424+0000","last_peered":"2024-09-10T10:05:31.194424+0000","last_clean":"2024-09-10T10:05:31.194424+0000","last_became_active":"2024-09-10T10:05:31.192470+0000","last_became_peered":"2024-09-10T10:05:31.192470+0000","last_unstale":"2024-09-10T10:05:31.194424+0000","last_undegraded":"2024-09-10T10:05:31.194424+0000","last_fullsized":"2024-09-10T10:05:31.194424+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:50.910428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180942+0000","last_change":"2024-09-10T10:05:31.180942+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180942+0000","last_undegraded":"2024-09-10T10:05:31.180942+0000","last_fullsized":"2024-09-10T10:05:31.180942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197139+0000","last_change":"2024-09-10T10:05:31.194554+0000","last_active":"2024-09-10T10:05:31.197139+0000","last_peered":"2024-09-10T10:05:31.197139+0000","last_clean":"2024-09-10T10:05:31.197139+0000","last_became_active":"2024-09-10T10:05:31.194265+0000","last_became_peered":"2024-09-10T10:05:31.194265+0000","last_unstale":"2024-09-10T10:05:31.197139+0000","last_undegraded":"2024-09-10T10:05:31.197139+0000","last_fullsized":"2024-09-10T10:05:31.197139+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:34.402529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170703+0000","last_change":"2024-09-10T10:05:31.170703+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170703+0000","last_undegraded":"2024-09-10T10:05:31.170703+0000","last_fullsized":"2024-09-10T10:05:31.170703+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194117+0000","last_change":"2024-09-10T10:05:31.188898+0000","last_active":"2024-09-10T10:05:31.194117+0000","last_peered":"2024-09-10T10:05:31.194117+0000","last_clean":"2024-09-10T10:05:31.194117+0000","last_became_active":"2024-09-10T10:05:31.186075+0000","last_became_peered":"2024-09-10T10:05:31.186075+0000","last_unstale":"2024-09-10T10:05:31.194117+0000","last_undegraded":"2024-09-10T10:05:31.194117+0000","last_fullsized":"2024-09-10T10:05:31.194117+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:59.362162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173203+0000","last_change":"2024-09-10T10:05:31.173203+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173203+0000","last_undegraded":"2024-09-10T10:05:31.173203+0000","last_fullsized":"2024-09-10T10:05:31.173203+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.196060+0000","last_change":"2024-09-10T10:05:31.187311+0000","last_active":"2024-09-10T10:05:31.196060+0000","last_peered":"2024-09-10T10:05:31.196060+0000","last_clean":"2024-09-10T10:05:31.196060+0000","last_became_active":"2024-09-10T10:05:31.186827+0000","last_became_peered":"2024-09-10T10:05:31.186827+0000","last_unstale":"2024-09-10T10:05:31.196060+0000","last_undegraded":"2024-09-10T10:05:31.196060+0000","last_fullsized":"2024-09-10T10:05:31.196060+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:32:57.257293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174497+0000","last_change":"2024-09-10T10:05:31.174497+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174497+0000","last_undegraded":"2024-09-10T10:05:31.174497+0000","last_fullsized":"2024-09-10T10:05:31.174497+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.189401+0000","last_change":"2024-09-10T10:05:31.187284+0000","last_active":"2024-09-10T10:05:31.189401+0000","last_peered":"2024-09-10T10:05:31.189401+0000","last_clean":"2024-09-10T10:05:31.189401+0000","last_became_active":"2024-09-10T10:05:31.186766+0000","last_became_peered":"2024-09-10T10:05:31.186766+0000","last_unstale":"2024-09-10T10:05:31.189401+0000","last_undegraded":"2024-09-10T10:05:31.189401+0000","last_fullsized":"2024-09-10T10:05:31.189401+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:11.425378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172755+0000","last_change":"2024-09-10T10:05:31.172755+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172755+0000","last_undegraded":"2024-09-10T10:05:31.172755+0000","last_fullsized":"2024-09-10T10:05:31.172755+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192180+0000","last_change":"2024-09-10T10:05:31.188601+0000","last_active":"2024-09-10T10:05:31.192180+0000","last_peered":"2024-09-10T10:05:31.192180+0000","last_clean":"2024-09-10T10:05:31.192180+0000","last_became_active":"2024-09-10T10:05:31.188289+0000","last_became_peered":"2024-09-10T10:05:31.188289+0000","last_unstale":"2024-09-10T10:05:31.192180+0000","last_undegraded":"2024-09-10T10:05:31.192180+0000","last_fullsized":"2024-09-10T10:05:31.192180+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:07.295402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178608+0000","last_change":"2024-09-10T10:05:31.178608+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178608+0000","last_undegraded":"2024-09-10T10:05:31.178608+0000","last_fullsized":"2024-09-10T10:05:31.178608+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.198112+0000","last_change":"2024-09-10T10:05:31.196820+0000","last_active":"2024-09-10T10:05:31.198112+0000","last_peered":"2024-09-10T10:05:31.198112+0000","last_clean":"2024-09-10T10:05:31.198112+0000","last_became_active":"2024-09-10T10:05:31.196557+0000","last_became_peered":"2024-09-10T10:05:31.196557+0000","last_unstale":"2024-09-10T10:05:31.198112+0000","last_undegraded":"2024-09-10T10:05:31.198112+0000","last_fullsized":"2024-09-10T10:05:31.198112+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:01.069466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170129+0000","last_change":"2024-09-10T10:05:31.170129+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170129+0000","last_undegraded":"2024-09-10T10:05:31.170129+0000","last_fullsized":"2024-09-10T10:05:31.170129+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.198182+0000","last_change":"2024-09-10T10:05:31.197033+0000","last_active":"2024-09-10T10:05:31.198182+0000","last_peered":"2024-09-10T10:05:31.198182+0000","last_clean":"2024-09-10T10:05:31.198182+0000","last_became_active":"2024-09-10T10:05:31.196739+0000","last_became_peered":"2024-09-10T10:05:31.196739+0000","last_unstale":"2024-09-10T10:05:31.198182+0000","last_undegraded":"2024-09-10T10:05:31.198182+0000","last_fullsized":"2024-09-10T10:05:31.198182+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:23.494858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170949+0000","last_change":"2024-09-10T10:05:31.170949+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170949+0000","last_undegraded":"2024-09-10T10:05:31.170949+0000","last_fullsized":"2024-09-10T10:05:31.170949+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.187377+0000","last_change":"2024-09-10T10:05:31.179485+0000","last_active":"2024-09-10T10:05:31.187377+0000","last_peered":"2024-09-10T10:05:31.187377+0000","last_clean":"2024-09-10T10:05:31.187377+0000","last_became_active":"2024-09-10T10:05:31.178050+0000","last_became_peered":"2024-09-10T10:05:31.178050+0000","last_unstale":"2024-09-10T10:05:31.187377+0000","last_undegraded":"2024-09-10T10:05:31.187377+0000","last_fullsized":"2024-09-10T10:05:31.187377+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:17.213280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179275+0000","last_change":"2024-09-10T10:05:31.179275+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179275+0000","last_undegraded":"2024-09-10T10:05:31.179275+0000","last_fullsized":"2024-09-10T10:05:31.179275+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194166+0000","last_change":"2024-09-10T10:05:31.193385+0000","last_active":"2024-09-10T10:05:31.194166+0000","last_peered":"2024-09-10T10:05:31.194166+0000","last_clean":"2024-09-10T10:05:31.194166+0000","last_became_active":"2024-09-10T10:05:31.193085+0000","last_became_peered":"2024-09-10T10:05:31.193085+0000","last_unstale":"2024-09-10T10:05:31.194166+0000","last_undegraded":"2024-09-10T10:05:31.194166+0000","last_fullsized":"2024-09-10T10:05:31.194166+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:13.939828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178450+0000","last_change":"2024-09-10T10:05:31.178450+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178450+0000","last_undegraded":"2024-09-10T10:05:31.178450+0000","last_fullsized":"2024-09-10T10:05:31.178450+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.188996+0000","last_change":"2024-09-10T10:05:31.186964+0000","last_active":"2024-09-10T10:05:31.188996+0000","last_peered":"2024-09-10T10:05:31.188996+0000","last_clean":"2024-09-10T10:05:31.188996+0000","last_became_active":"2024-09-10T10:05:31.186613+0000","last_became_peered":"2024-09-10T10:05:31.186613+0000","last_unstale":"2024-09-10T10:05:31.188996+0000","last_undegraded":"2024-09-10T10:05:31.188996+0000","last_fullsized":"2024-09-10T10:05:31.188996+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:59.668546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172624+0000","last_change":"2024-09-10T10:05:31.172624+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172624+0000","last_undegraded":"2024-09-10T10:05:31.172624+0000","last_fullsized":"2024-09-10T10:05:31.172624+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190738+0000","last_change":"2024-09-10T10:05:31.187417+0000","last_active":"2024-09-10T10:05:31.190738+0000","last_peered":"2024-09-10T10:05:31.190738+0000","last_clean":"2024-09-10T10:05:31.190738+0000","last_became_active":"2024-09-10T10:05:31.187030+0000","last_became_peered":"2024-09-10T10:05:31.187030+0000","last_unstale":"2024-09-10T10:05:31.190738+0000","last_undegraded":"2024-09-10T10:05:31.190738+0000","last_fullsized":"2024-09-10T10:05:31.190738+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:13.626786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171896+0000","last_change":"2024-09-10T10:05:31.171896+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171896+0000","last_undegraded":"2024-09-10T10:05:31.171896+0000","last_fullsized":"2024-09-10T10:05:31.171896+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197321+0000","last_change":"2024-09-10T10:05:31.194388+0000","last_active":"2024-09-10T10:05:31.197321+0000","last_peered":"2024-09-10T10:05:31.197321+0000","last_clean":"2024-09-10T10:05:31.197321+0000","last_became_active":"2024-09-10T10:05:31.193967+0000","last_became_peered":"2024-09-10T10:05:31.193967+0000","last_unstale":"2024-09-10T10:05:31.197321+0000","last_undegraded":"2024-09-10T10:05:31.197321+0000","last_fullsized":"2024-09-10T10:05:31.197321+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:53.149484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.175422+0000","last_change":"2024-09-10T10:05:31.175422+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.175422+0000","last_undegraded":"2024-09-10T10:05:31.175422+0000","last_fullsized":"2024-09-10T10:05:31.175422+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192494+0000","last_change":"2024-09-10T10:05:31.191205+0000","last_active":"2024-09-10T10:05:31.192494+0000","last_peered":"2024-09-10T10:05:31.192494+0000","last_clean":"2024-09-10T10:05:31.192494+0000","last_became_active":"2024-09-10T10:05:31.190885+0000","last_became_peered":"2024-09-10T10:05:31.190885+0000","last_unstale":"2024-09-10T10:05:31.192494+0000","last_undegraded":"2024-09-10T10:05:31.192494+0000","last_fullsized":"2024-09-10T10:05:31.192494+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:58.487180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":444,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.180502+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:05:31.180502+0000","last_peered":"2024-09-10T10:05:31.180502+0000","last_clean":"2024-09-10T10:05:31.180502+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:05:31.180502+0000","last_undegraded":"2024-09-10T10:05:31.180502+0000","last_fullsized":"2024-09-10T10:05:31.180502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2490368,"data_stored":1425104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509530,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542236,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37188,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334652,"statfs":{"total":96636764160,"available":96598683648,"internally_reserved":0,"allocated":1884160,"data_stored":828405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37188,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334652,"statfs":{"total":96636764160,"available":96598683648,"internally_reserved":0,"allocated":1884160,"data_stored":828405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37204,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2490368,"data_stored":1425104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:05:36.647 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:05:36.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81fc000640 1 Processor -- start 2024-09-10T10:05:36.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81fc000640 1 -- start start 2024-09-10T10:05:36.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81fc000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f81f415c560 0x7f81f406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81fc000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f406f010 0x7f81f40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81fc000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f406f430 con 0x7f81f4155970 2024-09-10T10:05:36.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81fc000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f81f40b4570 con 0x7f81f415c560 2024-09-10T10:05:36.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81fc000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81f4155d50 con 0x7f81f406f010 2024-09-10T10:05:36.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81f9600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f81f4155970 0x7f81f40cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:48958/0 (socket says 172.21.15.110:48958) 2024-09-10T10:05:36.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.817+0000 7f81f9600640 1 -- 172.21.15.110:0/2981700813 learned_addr learned my addr 172.21.15.110:0/2981700813 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:36.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81fa000640 1 --2- 172.21.15.110:0/2981700813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f406f010 0x7f81f40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81f8c00640 1 --2- 172.21.15.110:0/2981700813 >> v2:172.21.15.145:3300/0 conn(0x7f81f415c560 0x7f81f406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81f3e00640 1 -- 172.21.15.110:0/2981700813 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931224550 0 0) 0x7f81f406f430 con 0x7f81f4155970 2024-09-10T10:05:36.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81f3e00640 1 -- 172.21.15.110:0/2981700813 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81dc003530 con 0x7f81f4155970 2024-09-10T10:05:36.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81f3e00640 1 -- 172.21.15.110:0/2981700813 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1970353380 0 0) 0x7f81dc003530 con 0x7f81f4155970 2024-09-10T10:05:36.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81fa000640 1 -- 172.21.15.110:0/2981700813 >> v2:172.21.15.145:3300/0 conn(0x7f81f415c560 msgr2=0x7f81f406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:36.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81fa000640 1 --2- 172.21.15.110:0/2981700813 >> v2:172.21.15.145:3300/0 conn(0x7f81f415c560 0x7f81f406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81fa000640 1 -- 172.21.15.110:0/2981700813 >> v1:172.21.15.110:6789/0 conn(0x7f81f4155970 legacy=0x7f81f40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81fa000640 1 -- 172.21.15.110:0/2981700813 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f40b34d0 con 0x7f81f406f010 2024-09-10T10:05:36.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.821+0000 7f81fa000640 1 --2- 172.21.15.110:0/2981700813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f406f010 0x7f81f40ce420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0097c0 tx=0x7f81ec01c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3ef5196ba0ae96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81f3e00640 1 -- 172.21.15.110:0/2981700813 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0b0070 con 0x7f81f406f010 2024-09-10T10:05:36.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81f3e00640 1 -- 172.21.15.110:0/2981700813 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec0ab400 con 0x7f81f406f010 2024-09-10T10:05:36.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81f3e00640 1 -- 172.21.15.110:0/2981700813 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8001a20 con 0x7f81f406f010 2024-09-10T10:05:36.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- 172.21.15.110:0/2981700813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f406f010 msgr2=0x7f81f40ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 --2- 172.21.15.110:0/2981700813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f406f010 0x7f81f40ce420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0097c0 tx=0x7f81ec01c390 comp rx=0 tx=0).stop 2024-09-10T10:05:36.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- 172.21.15.110:0/2981700813 shutdown_connections 2024-09-10T10:05:36.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 --2- 172.21.15.110:0/2981700813 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f406f010 0x7f81f40ce420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 --2- 172.21.15.110:0/2981700813 >> v2:172.21.15.145:3300/0 conn(0x7f81f415c560 0x7f81f406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- 172.21.15.110:0/2981700813 >> 172.21.15.110:0/2981700813 conn(0x7f81f406bc50 msgr2=0x7f81f406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:36.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- 172.21.15.110:0/2981700813 shutdown_connections 2024-09-10T10:05:36.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- 172.21.15.110:0/2981700813 wait complete. 2024-09-10T10:05:36.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 Processor -- start 2024-09-10T10:05:36.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- start start 2024-09-10T10:05:36.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81f406f010 0x7f81f4168be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f4155970 0x7f81f414ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f415c560 0x7f81f41494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81f40cf170 con 0x7f81f406f010 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81f40b4050 con 0x7f81f4155970 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81fc000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81f40cf2f0 con 0x7f81f415c560 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81f9600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81f406f010 0x7f81f4168be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81f9600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81f406f010 0x7f81f4168be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34308/0 (socket says 172.21.15.110:34308) 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.825+0000 7f81f9600640 1 -- 172.21.15.110:0/4262426214 learned_addr learned my addr 172.21.15.110:0/4262426214 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81fa000640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f415c560 0x7f81f41494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f8c00640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f4155970 0x7f81f414ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f9600640 1 -- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f415c560 msgr2=0x7f81f41494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f9600640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f415c560 0x7f81f41494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f9600640 1 -- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f4155970 msgr2=0x7f81f414ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:36.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f9600640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f4155970 0x7f81f414ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f9600640 1 -- 172.21.15.110:0/4262426214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f4149e80 con 0x7f81f406f010 2024-09-10T10:05:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f9600640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81f406f010 0x7f81f4168be0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f81e800caa0 tx=0x7f81e800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f1600640 1 -- 172.21.15.110:0/4262426214 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8018070 con 0x7f81f406f010 2024-09-10T10:05:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f1600640 1 -- 172.21.15.110:0/4262426214 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e800a730 con 0x7f81f406f010 2024-09-10T10:05:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f1600640 1 -- 172.21.15.110:0/4262426214 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e800ab20 con 0x7f81f406f010 2024-09-10T10:05:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f414a160 con 0x7f81f406f010 2024-09-10T10:05:36.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f4152630 con 0x7f81f406f010 2024-09-10T10:05:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f40ce420 con 0x7f81f406f010 2024-09-10T10:05:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f1600640 1 -- 172.21.15.110:0/4262426214 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f81e800acc0 con 0x7f81f406f010 2024-09-10T10:05:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f1600640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81d40680d0 0x7f81d406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f1600640 1 -- 172.21.15.110:0/4262426214 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f81e80866c0 con 0x7f81f406f010 2024-09-10T10:05:36.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.829+0000 7f81f8c00640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81d40680d0 0x7f81d406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:36.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.833+0000 7f81f8c00640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81d40680d0 0x7f81d406a590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f81f415b300 tx=0x7f81e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:36.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.833+0000 7f81f1600640 1 -- 172.21.15.110:0/4262426214 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f81e8056270 con 0x7f81f406f010 2024-09-10T10:05:36.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.989+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81f414b180 con 0x7f81d40680d0 2024-09-10T10:05:36.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.993+0000 7f81f1600640 1 -- 172.21.15.110:0/4262426214 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247088 (secure 0 0 0) 0x7f81e00118b0 con 0x7f81d40680d0 2024-09-10T10:05:36.998 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:37.000 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81d40680d0 msgr2=0x7f81d406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81d40680d0 0x7f81d406a590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f81f415b300 tx=0x7f81e0008040 comp rx=0 tx=0).stop 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81f406f010 msgr2=0x7f81f4168be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81f406f010 0x7f81f4168be0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f81e800caa0 tx=0x7f81e800cf60 comp rx=0 tx=0).stop 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 shutdown_connections 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81d40680d0 0x7f81d406a590 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81f415c560 0x7f81f41494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81f4155970 0x7f81f414ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 --2- 172.21.15.110:0/4262426214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81f406f010 0x7f81f4168be0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 >> 172.21.15.110:0/4262426214 conn(0x7f81f406bc50 msgr2=0x7f81f406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:37.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 shutdown_connections 2024-09-10T10:05:37.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:36.997+0000 7f81fc000640 1 -- 172.21.15.110:0/4262426214 wait complete. 2024-09-10T10:05:37.017 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-10T10:05:35.740163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282112,"kb_used_data":15952,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692608,"statfs":{"total":773094113280,"available":772805230592,"internally_reserved":0,"allocated":16334848,"data_stored":7845962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.578008"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178157+0000","last_change":"2024-09-10T10:05:31.178157+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178157+0000","last_undegraded":"2024-09-10T10:05:31.178157+0000","last_fullsized":"2024-09-10T10:05:31.178157+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173328+0000","last_change":"2024-09-10T10:05:31.173328+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173328+0000","last_undegraded":"2024-09-10T10:05:31.173328+0000","last_fullsized":"2024-09-10T10:05:31.173328+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179146+0000","last_change":"2024-09-10T10:05:31.179146+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179146+0000","last_undegraded":"2024-09-10T10:05:31.179146+0000","last_fullsized":"2024-09-10T10:05:31.179146+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173862+0000","last_change":"2024-09-10T10:05:31.173862+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173862+0000","last_undegraded":"2024-09-10T10:05:31.173862+0000","last_fullsized":"2024-09-10T10:05:31.173862+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172850+0000","last_change":"2024-09-10T10:05:31.172850+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172850+0000","last_undegraded":"2024-09-10T10:05:31.172850+0000","last_fullsized":"2024-09-10T10:05:31.172850+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173582+0000","last_change":"2024-09-10T10:05:31.173582+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173582+0000","last_undegraded":"2024-09-10T10:05:31.173582+0000","last_fullsized":"2024-09-10T10:05:31.173582+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171259+0000","last_change":"2024-09-10T10:05:31.171259+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171259+0000","last_undegraded":"2024-09-10T10:05:31.171259+0000","last_fullsized":"2024-09-10T10:05:31.171259+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171214+0000","last_change":"2024-09-10T10:05:31.171214+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171214+0000","last_undegraded":"2024-09-10T10:05:31.171214+0000","last_fullsized":"2024-09-10T10:05:31.171214+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180484+0000","last_change":"2024-09-10T10:05:31.180484+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180484+0000","last_undegraded":"2024-09-10T10:05:31.180484+0000","last_fullsized":"2024-09-10T10:05:31.180484+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.187473+0000","last_change":"2024-09-10T10:05:31.187473+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.187473+0000","last_undegraded":"2024-09-10T10:05:31.187473+0000","last_fullsized":"2024-09-10T10:05:31.187473+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170730+0000","last_change":"2024-09-10T10:05:31.170730+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170730+0000","last_undegraded":"2024-09-10T10:05:31.170730+0000","last_fullsized":"2024-09-10T10:05:31.170730+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173444+0000","last_change":"2024-09-10T10:05:31.173444+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173444+0000","last_undegraded":"2024-09-10T10:05:31.173444+0000","last_fullsized":"2024-09-10T10:05:31.173444+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173147+0000","last_change":"2024-09-10T10:05:31.173147+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173147+0000","last_undegraded":"2024-09-10T10:05:31.173147+0000","last_fullsized":"2024-09-10T10:05:31.173147+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178165+0000","last_change":"2024-09-10T10:05:31.178165+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178165+0000","last_undegraded":"2024-09-10T10:05:31.178165+0000","last_fullsized":"2024-09-10T10:05:31.178165+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172248+0000","last_change":"2024-09-10T10:05:31.172248+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172248+0000","last_undegraded":"2024-09-10T10:05:31.172248+0000","last_fullsized":"2024-09-10T10:05:31.172248+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170807+0000","last_change":"2024-09-10T10:05:31.170807+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170807+0000","last_undegraded":"2024-09-10T10:05:31.170807+0000","last_fullsized":"2024-09-10T10:05:31.170807+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173631+0000","last_change":"2024-09-10T10:05:31.173631+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173631+0000","last_undegraded":"2024-09-10T10:05:31.173631+0000","last_fullsized":"2024-09-10T10:05:31.173631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181226+0000","last_change":"2024-09-10T10:05:31.181226+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181226+0000","last_undegraded":"2024-09-10T10:05:31.181226+0000","last_fullsized":"2024-09-10T10:05:31.181226+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180425+0000","last_change":"2024-09-10T10:05:31.180425+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180425+0000","last_undegraded":"2024-09-10T10:05:31.180425+0000","last_fullsized":"2024-09-10T10:05:31.180425+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.175359+0000","last_change":"2024-09-10T10:05:31.175359+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.175359+0000","last_undegraded":"2024-09-10T10:05:31.175359+0000","last_fullsized":"2024-09-10T10:05:31.175359+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172406+0000","last_change":"2024-09-10T10:05:31.172406+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172406+0000","last_undegraded":"2024-09-10T10:05:31.172406+0000","last_fullsized":"2024-09-10T10:05:31.172406+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181999+0000","last_change":"2024-09-10T10:05:31.181999+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181999+0000","last_undegraded":"2024-09-10T10:05:31.181999+0000","last_fullsized":"2024-09-10T10:05:31.181999+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177007+0000","last_change":"2024-09-10T10:05:31.177007+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177007+0000","last_undegraded":"2024-09-10T10:05:31.177007+0000","last_fullsized":"2024-09-10T10:05:31.177007+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170320+0000","last_change":"2024-09-10T10:05:31.170320+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170320+0000","last_undegraded":"2024-09-10T10:05:31.170320+0000","last_fullsized":"2024-09-10T10:05:31.170320+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172778+0000","last_change":"2024-09-10T10:05:31.172778+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172778+0000","last_undegraded":"2024-09-10T10:05:31.172778+0000","last_fullsized":"2024-09-10T10:05:31.172778+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171873+0000","last_change":"2024-09-10T10:05:31.171873+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171873+0000","last_undegraded":"2024-09-10T10:05:31.171873+0000","last_fullsized":"2024-09-10T10:05:31.171873+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177404+0000","last_change":"2024-09-10T10:05:31.177404+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177404+0000","last_undegraded":"2024-09-10T10:05:31.177404+0000","last_fullsized":"2024-09-10T10:05:31.177404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174881+0000","last_change":"2024-09-10T10:05:31.174881+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174881+0000","last_undegraded":"2024-09-10T10:05:31.174881+0000","last_fullsized":"2024-09-10T10:05:31.174881+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172702+0000","last_change":"2024-09-10T10:05:31.172702+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172702+0000","last_undegraded":"2024-09-10T10:05:31.172702+0000","last_fullsized":"2024-09-10T10:05:31.172702+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177678+0000","last_change":"2024-09-10T10:05:31.177678+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177678+0000","last_undegraded":"2024-09-10T10:05:31.177678+0000","last_fullsized":"2024-09-10T10:05:31.177678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173678+0000","last_change":"2024-09-10T10:05:31.173678+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173678+0000","last_undegraded":"2024-09-10T10:05:31.173678+0000","last_fullsized":"2024-09-10T10:05:31.173678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172528+0000","last_change":"2024-09-10T10:05:31.172528+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172528+0000","last_undegraded":"2024-09-10T10:05:31.172528+0000","last_fullsized":"2024-09-10T10:05:31.172528+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170496+0000","last_change":"2024-09-10T10:05:31.170496+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170496+0000","last_undegraded":"2024-09-10T10:05:31.170496+0000","last_fullsized":"2024-09-10T10:05:31.170496+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193482+0000","last_change":"2024-09-10T10:05:31.187728+0000","last_active":"2024-09-10T10:05:31.193482+0000","last_peered":"2024-09-10T10:05:31.193482+0000","last_clean":"2024-09-10T10:05:31.193482+0000","last_became_active":"2024-09-10T10:05:31.186044+0000","last_became_peered":"2024-09-10T10:05:31.186044+0000","last_unstale":"2024-09-10T10:05:31.193482+0000","last_undegraded":"2024-09-10T10:05:31.193482+0000","last_fullsized":"2024-09-10T10:05:31.193482+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:28.803480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179647+0000","last_change":"2024-09-10T10:05:31.179647+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179647+0000","last_undegraded":"2024-09-10T10:05:31.179647+0000","last_fullsized":"2024-09-10T10:05:31.179647+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.189032+0000","last_change":"2024-09-10T10:05:31.187161+0000","last_active":"2024-09-10T10:05:31.189032+0000","last_peered":"2024-09-10T10:05:31.189032+0000","last_clean":"2024-09-10T10:05:31.189032+0000","last_became_active":"2024-09-10T10:05:31.186727+0000","last_became_peered":"2024-09-10T10:05:31.186727+0000","last_unstale":"2024-09-10T10:05:31.189032+0000","last_undegraded":"2024-09-10T10:05:31.189032+0000","last_fullsized":"2024-09-10T10:05:31.189032+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:15.098413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174762+0000","last_change":"2024-09-10T10:05:31.174762+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174762+0000","last_undegraded":"2024-09-10T10:05:31.174762+0000","last_fullsized":"2024-09-10T10:05:31.174762+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197736+0000","last_change":"2024-09-10T10:05:31.196039+0000","last_active":"2024-09-10T10:05:31.197736+0000","last_peered":"2024-09-10T10:05:31.197736+0000","last_clean":"2024-09-10T10:05:31.197736+0000","last_became_active":"2024-09-10T10:05:31.195762+0000","last_became_peered":"2024-09-10T10:05:31.195762+0000","last_unstale":"2024-09-10T10:05:31.197736+0000","last_undegraded":"2024-09-10T10:05:31.197736+0000","last_fullsized":"2024-09-10T10:05:31.197736+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:08.377153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177235+0000","last_change":"2024-09-10T10:05:31.177235+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177235+0000","last_undegraded":"2024-09-10T10:05:31.177235+0000","last_fullsized":"2024-09-10T10:05:31.177235+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190648+0000","last_change":"2024-09-10T10:05:31.184956+0000","last_active":"2024-09-10T10:05:31.190648+0000","last_peered":"2024-09-10T10:05:31.190648+0000","last_clean":"2024-09-10T10:05:31.190648+0000","last_became_active":"2024-09-10T10:05:31.184628+0000","last_became_peered":"2024-09-10T10:05:31.184628+0000","last_unstale":"2024-09-10T10:05:31.190648+0000","last_undegraded":"2024-09-10T10:05:31.190648+0000","last_fullsized":"2024-09-10T10:05:31.190648+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:20.844260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181360+0000","last_change":"2024-09-10T10:05:31.181360+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181360+0000","last_undegraded":"2024-09-10T10:05:31.181360+0000","last_fullsized":"2024-09-10T10:05:31.181360+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193807+0000","last_change":"2024-09-10T10:05:31.189563+0000","last_active":"2024-09-10T10:05:31.193807+0000","last_peered":"2024-09-10T10:05:31.193807+0000","last_clean":"2024-09-10T10:05:31.193807+0000","last_became_active":"2024-09-10T10:05:31.189169+0000","last_became_peered":"2024-09-10T10:05:31.189169+0000","last_unstale":"2024-09-10T10:05:31.193807+0000","last_undegraded":"2024-09-10T10:05:31.193807+0000","last_fullsized":"2024-09-10T10:05:31.193807+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:46.694678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171144+0000","last_change":"2024-09-10T10:05:31.171144+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171144+0000","last_undegraded":"2024-09-10T10:05:31.171144+0000","last_fullsized":"2024-09-10T10:05:31.171144+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190551+0000","last_change":"2024-09-10T10:05:31.184111+0000","last_active":"2024-09-10T10:05:31.190551+0000","last_peered":"2024-09-10T10:05:31.190551+0000","last_clean":"2024-09-10T10:05:31.190551+0000","last_became_active":"2024-09-10T10:05:31.183673+0000","last_became_peered":"2024-09-10T10:05:31.183673+0000","last_unstale":"2024-09-10T10:05:31.190551+0000","last_undegraded":"2024-09-10T10:05:31.190551+0000","last_fullsized":"2024-09-10T10:05:31.190551+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:21.245568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170694+0000","last_change":"2024-09-10T10:05:31.170694+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170694+0000","last_undegraded":"2024-09-10T10:05:31.170694+0000","last_fullsized":"2024-09-10T10:05:31.170694+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194428+0000","last_change":"2024-09-10T10:05:31.193233+0000","last_active":"2024-09-10T10:05:31.194428+0000","last_peered":"2024-09-10T10:05:31.194428+0000","last_clean":"2024-09-10T10:05:31.194428+0000","last_became_active":"2024-09-10T10:05:31.192971+0000","last_became_peered":"2024-09-10T10:05:31.192971+0000","last_unstale":"2024-09-10T10:05:31.194428+0000","last_undegraded":"2024-09-10T10:05:31.194428+0000","last_fullsized":"2024-09-10T10:05:31.194428+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:04.862858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178681+0000","last_change":"2024-09-10T10:05:31.178681+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178681+0000","last_undegraded":"2024-09-10T10:05:31.178681+0000","last_fullsized":"2024-09-10T10:05:31.178681+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190231+0000","last_change":"2024-09-10T10:05:31.185108+0000","last_active":"2024-09-10T10:05:31.190231+0000","last_peered":"2024-09-10T10:05:31.190231+0000","last_clean":"2024-09-10T10:05:31.190231+0000","last_became_active":"2024-09-10T10:05:31.184889+0000","last_became_peered":"2024-09-10T10:05:31.184889+0000","last_unstale":"2024-09-10T10:05:31.190231+0000","last_undegraded":"2024-09-10T10:05:31.190231+0000","last_fullsized":"2024-09-10T10:05:31.190231+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:57.982434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170177+0000","last_change":"2024-09-10T10:05:31.170177+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170177+0000","last_undegraded":"2024-09-10T10:05:31.170177+0000","last_fullsized":"2024-09-10T10:05:31.170177+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194581+0000","last_change":"2024-09-10T10:05:31.190056+0000","last_active":"2024-09-10T10:05:31.194581+0000","last_peered":"2024-09-10T10:05:31.194581+0000","last_clean":"2024-09-10T10:05:31.194581+0000","last_became_active":"2024-09-10T10:05:31.189791+0000","last_became_peered":"2024-09-10T10:05:31.189791+0000","last_unstale":"2024-09-10T10:05:31.194581+0000","last_undegraded":"2024-09-10T10:05:31.194581+0000","last_fullsized":"2024-09-10T10:05:31.194581+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:24.521559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174289+0000","last_change":"2024-09-10T10:05:31.174289+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174289+0000","last_undegraded":"2024-09-10T10:05:31.174289+0000","last_fullsized":"2024-09-10T10:05:31.174289+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193922+0000","last_change":"2024-09-10T10:05:31.188150+0000","last_active":"2024-09-10T10:05:31.193922+0000","last_peered":"2024-09-10T10:05:31.193922+0000","last_clean":"2024-09-10T10:05:31.193922+0000","last_became_active":"2024-09-10T10:05:31.185519+0000","last_became_peered":"2024-09-10T10:05:31.185519+0000","last_unstale":"2024-09-10T10:05:31.193922+0000","last_undegraded":"2024-09-10T10:05:31.193922+0000","last_fullsized":"2024-09-10T10:05:31.193922+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:45.379567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179119+0000","last_change":"2024-09-10T10:05:31.179119+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179119+0000","last_undegraded":"2024-09-10T10:05:31.179119+0000","last_fullsized":"2024-09-10T10:05:31.179119+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197722+0000","last_change":"2024-09-10T10:05:31.194843+0000","last_active":"2024-09-10T10:05:31.197722+0000","last_peered":"2024-09-10T10:05:31.197722+0000","last_clean":"2024-09-10T10:05:31.197722+0000","last_became_active":"2024-09-10T10:05:31.194168+0000","last_became_peered":"2024-09-10T10:05:31.194168+0000","last_unstale":"2024-09-10T10:05:31.197722+0000","last_undegraded":"2024-09-10T10:05:31.197722+0000","last_fullsized":"2024-09-10T10:05:31.197722+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:55.908854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173030+0000","last_change":"2024-09-10T10:05:31.173030+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173030+0000","last_undegraded":"2024-09-10T10:05:31.173030+0000","last_fullsized":"2024-09-10T10:05:31.173030+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192401+0000","last_change":"2024-09-10T10:05:31.189711+0000","last_active":"2024-09-10T10:05:31.192401+0000","last_peered":"2024-09-10T10:05:31.192401+0000","last_clean":"2024-09-10T10:05:31.192401+0000","last_became_active":"2024-09-10T10:05:31.189471+0000","last_became_peered":"2024-09-10T10:05:31.189471+0000","last_unstale":"2024-09-10T10:05:31.192401+0000","last_undegraded":"2024-09-10T10:05:31.192401+0000","last_fullsized":"2024-09-10T10:05:31.192401+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:17.765192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173658+0000","last_change":"2024-09-10T10:05:31.173658+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173658+0000","last_undegraded":"2024-09-10T10:05:31.173658+0000","last_fullsized":"2024-09-10T10:05:31.173658+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.196348+0000","last_change":"2024-09-10T10:05:31.195254+0000","last_active":"2024-09-10T10:05:31.196348+0000","last_peered":"2024-09-10T10:05:31.196348+0000","last_clean":"2024-09-10T10:05:31.196348+0000","last_became_active":"2024-09-10T10:05:31.194909+0000","last_became_peered":"2024-09-10T10:05:31.194909+0000","last_unstale":"2024-09-10T10:05:31.196348+0000","last_undegraded":"2024-09-10T10:05:31.196348+0000","last_fullsized":"2024-09-10T10:05:31.196348+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:24.286717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179062+0000","last_change":"2024-09-10T10:05:31.179062+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179062+0000","last_undegraded":"2024-09-10T10:05:31.179062+0000","last_fullsized":"2024-09-10T10:05:31.179062+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.184862+0000","last_change":"2024-09-10T10:05:31.176701+0000","last_active":"2024-09-10T10:05:31.184862+0000","last_peered":"2024-09-10T10:05:31.184862+0000","last_clean":"2024-09-10T10:05:31.184862+0000","last_became_active":"2024-09-10T10:05:31.176425+0000","last_became_peered":"2024-09-10T10:05:31.176425+0000","last_unstale":"2024-09-10T10:05:31.184862+0000","last_undegraded":"2024-09-10T10:05:31.184862+0000","last_fullsized":"2024-09-10T10:05:31.184862+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:25.572927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.176025+0000","last_change":"2024-09-10T10:05:31.176025+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.176025+0000","last_undegraded":"2024-09-10T10:05:31.176025+0000","last_fullsized":"2024-09-10T10:05:31.176025+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194415+0000","last_change":"2024-09-10T10:05:31.193087+0000","last_active":"2024-09-10T10:05:31.194415+0000","last_peered":"2024-09-10T10:05:31.194415+0000","last_clean":"2024-09-10T10:05:31.194415+0000","last_became_active":"2024-09-10T10:05:31.192788+0000","last_became_peered":"2024-09-10T10:05:31.192788+0000","last_unstale":"2024-09-10T10:05:31.194415+0000","last_undegraded":"2024-09-10T10:05:31.194415+0000","last_fullsized":"2024-09-10T10:05:31.194415+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:14.387457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.181741+0000","last_change":"2024-09-10T10:05:31.181741+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.181741+0000","last_undegraded":"2024-09-10T10:05:31.181741+0000","last_fullsized":"2024-09-10T10:05:31.181741+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.193842+0000","last_change":"2024-09-10T10:05:31.189469+0000","last_active":"2024-09-10T10:05:31.193842+0000","last_peered":"2024-09-10T10:05:31.193842+0000","last_clean":"2024-09-10T10:05:31.193842+0000","last_became_active":"2024-09-10T10:05:31.189175+0000","last_became_peered":"2024-09-10T10:05:31.189175+0000","last_unstale":"2024-09-10T10:05:31.193842+0000","last_undegraded":"2024-09-10T10:05:31.193842+0000","last_fullsized":"2024-09-10T10:05:31.193842+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:18:09.097792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179763+0000","last_change":"2024-09-10T10:05:31.179763+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179763+0000","last_undegraded":"2024-09-10T10:05:31.179763+0000","last_fullsized":"2024-09-10T10:05:31.179763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.184460+0000","last_change":"2024-09-10T10:05:31.176322+0000","last_active":"2024-09-10T10:05:31.184460+0000","last_peered":"2024-09-10T10:05:31.184460+0000","last_clean":"2024-09-10T10:05:31.184460+0000","last_became_active":"2024-09-10T10:05:31.175852+0000","last_became_peered":"2024-09-10T10:05:31.175852+0000","last_unstale":"2024-09-10T10:05:31.184460+0000","last_undegraded":"2024-09-10T10:05:31.184460+0000","last_fullsized":"2024-09-10T10:05:31.184460+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:54.926756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.177666+0000","last_change":"2024-09-10T10:05:31.177666+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.177666+0000","last_undegraded":"2024-09-10T10:05:31.177666+0000","last_fullsized":"2024-09-10T10:05:31.177666+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194461+0000","last_change":"2024-09-10T10:05:31.193098+0000","last_active":"2024-09-10T10:05:31.194461+0000","last_peered":"2024-09-10T10:05:31.194461+0000","last_clean":"2024-09-10T10:05:31.194461+0000","last_became_active":"2024-09-10T10:05:31.192779+0000","last_became_peered":"2024-09-10T10:05:31.192779+0000","last_unstale":"2024-09-10T10:05:31.194461+0000","last_undegraded":"2024-09-10T10:05:31.194461+0000","last_fullsized":"2024-09-10T10:05:31.194461+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:46.842387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173084+0000","last_change":"2024-09-10T10:05:31.173084+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173084+0000","last_undegraded":"2024-09-10T10:05:31.173084+0000","last_fullsized":"2024-09-10T10:05:31.173084+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194424+0000","last_change":"2024-09-10T10:05:31.193283+0000","last_active":"2024-09-10T10:05:31.194424+0000","last_peered":"2024-09-10T10:05:31.194424+0000","last_clean":"2024-09-10T10:05:31.194424+0000","last_became_active":"2024-09-10T10:05:31.192470+0000","last_became_peered":"2024-09-10T10:05:31.192470+0000","last_unstale":"2024-09-10T10:05:31.194424+0000","last_undegraded":"2024-09-10T10:05:31.194424+0000","last_fullsized":"2024-09-10T10:05:31.194424+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:50.910428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.180942+0000","last_change":"2024-09-10T10:05:31.180942+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.180942+0000","last_undegraded":"2024-09-10T10:05:31.180942+0000","last_fullsized":"2024-09-10T10:05:31.180942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197139+0000","last_change":"2024-09-10T10:05:31.194554+0000","last_active":"2024-09-10T10:05:31.197139+0000","last_peered":"2024-09-10T10:05:31.197139+0000","last_clean":"2024-09-10T10:05:31.197139+0000","last_became_active":"2024-09-10T10:05:31.194265+0000","last_became_peered":"2024-09-10T10:05:31.194265+0000","last_unstale":"2024-09-10T10:05:31.197139+0000","last_undegraded":"2024-09-10T10:05:31.197139+0000","last_fullsized":"2024-09-10T10:05:31.197139+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:34.402529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170703+0000","last_change":"2024-09-10T10:05:31.170703+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170703+0000","last_undegraded":"2024-09-10T10:05:31.170703+0000","last_fullsized":"2024-09-10T10:05:31.170703+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194117+0000","last_change":"2024-09-10T10:05:31.188898+0000","last_active":"2024-09-10T10:05:31.194117+0000","last_peered":"2024-09-10T10:05:31.194117+0000","last_clean":"2024-09-10T10:05:31.194117+0000","last_became_active":"2024-09-10T10:05:31.186075+0000","last_became_peered":"2024-09-10T10:05:31.186075+0000","last_unstale":"2024-09-10T10:05:31.194117+0000","last_undegraded":"2024-09-10T10:05:31.194117+0000","last_fullsized":"2024-09-10T10:05:31.194117+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:59.362162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.173203+0000","last_change":"2024-09-10T10:05:31.173203+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.173203+0000","last_undegraded":"2024-09-10T10:05:31.173203+0000","last_fullsized":"2024-09-10T10:05:31.173203+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.196060+0000","last_change":"2024-09-10T10:05:31.187311+0000","last_active":"2024-09-10T10:05:31.196060+0000","last_peered":"2024-09-10T10:05:31.196060+0000","last_clean":"2024-09-10T10:05:31.196060+0000","last_became_active":"2024-09-10T10:05:31.186827+0000","last_became_peered":"2024-09-10T10:05:31.186827+0000","last_unstale":"2024-09-10T10:05:31.196060+0000","last_undegraded":"2024-09-10T10:05:31.196060+0000","last_fullsized":"2024-09-10T10:05:31.196060+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:32:57.257293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.174497+0000","last_change":"2024-09-10T10:05:31.174497+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.174497+0000","last_undegraded":"2024-09-10T10:05:31.174497+0000","last_fullsized":"2024-09-10T10:05:31.174497+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.189401+0000","last_change":"2024-09-10T10:05:31.187284+0000","last_active":"2024-09-10T10:05:31.189401+0000","last_peered":"2024-09-10T10:05:31.189401+0000","last_clean":"2024-09-10T10:05:31.189401+0000","last_became_active":"2024-09-10T10:05:31.186766+0000","last_became_peered":"2024-09-10T10:05:31.186766+0000","last_unstale":"2024-09-10T10:05:31.189401+0000","last_undegraded":"2024-09-10T10:05:31.189401+0000","last_fullsized":"2024-09-10T10:05:31.189401+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:11.425378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172755+0000","last_change":"2024-09-10T10:05:31.172755+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172755+0000","last_undegraded":"2024-09-10T10:05:31.172755+0000","last_fullsized":"2024-09-10T10:05:31.172755+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192180+0000","last_change":"2024-09-10T10:05:31.188601+0000","last_active":"2024-09-10T10:05:31.192180+0000","last_peered":"2024-09-10T10:05:31.192180+0000","last_clean":"2024-09-10T10:05:31.192180+0000","last_became_active":"2024-09-10T10:05:31.188289+0000","last_became_peered":"2024-09-10T10:05:31.188289+0000","last_unstale":"2024-09-10T10:05:31.192180+0000","last_undegraded":"2024-09-10T10:05:31.192180+0000","last_fullsized":"2024-09-10T10:05:31.192180+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:07.295402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178608+0000","last_change":"2024-09-10T10:05:31.178608+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178608+0000","last_undegraded":"2024-09-10T10:05:31.178608+0000","last_fullsized":"2024-09-10T10:05:31.178608+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.198112+0000","last_change":"2024-09-10T10:05:31.196820+0000","last_active":"2024-09-10T10:05:31.198112+0000","last_peered":"2024-09-10T10:05:31.198112+0000","last_clean":"2024-09-10T10:05:31.198112+0000","last_became_active":"2024-09-10T10:05:31.196557+0000","last_became_peered":"2024-09-10T10:05:31.196557+0000","last_unstale":"2024-09-10T10:05:31.198112+0000","last_undegraded":"2024-09-10T10:05:31.198112+0000","last_fullsized":"2024-09-10T10:05:31.198112+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:01.069466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170129+0000","last_change":"2024-09-10T10:05:31.170129+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170129+0000","last_undegraded":"2024-09-10T10:05:31.170129+0000","last_fullsized":"2024-09-10T10:05:31.170129+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.198182+0000","last_change":"2024-09-10T10:05:31.197033+0000","last_active":"2024-09-10T10:05:31.198182+0000","last_peered":"2024-09-10T10:05:31.198182+0000","last_clean":"2024-09-10T10:05:31.198182+0000","last_became_active":"2024-09-10T10:05:31.196739+0000","last_became_peered":"2024-09-10T10:05:31.196739+0000","last_unstale":"2024-09-10T10:05:31.198182+0000","last_undegraded":"2024-09-10T10:05:31.198182+0000","last_fullsized":"2024-09-10T10:05:31.198182+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:23.494858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.170949+0000","last_change":"2024-09-10T10:05:31.170949+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.170949+0000","last_undegraded":"2024-09-10T10:05:31.170949+0000","last_fullsized":"2024-09-10T10:05:31.170949+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.187377+0000","last_change":"2024-09-10T10:05:31.179485+0000","last_active":"2024-09-10T10:05:31.187377+0000","last_peered":"2024-09-10T10:05:31.187377+0000","last_clean":"2024-09-10T10:05:31.187377+0000","last_became_active":"2024-09-10T10:05:31.178050+0000","last_became_peered":"2024-09-10T10:05:31.178050+0000","last_unstale":"2024-09-10T10:05:31.187377+0000","last_undegraded":"2024-09-10T10:05:31.187377+0000","last_fullsized":"2024-09-10T10:05:31.187377+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:17.213280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.179275+0000","last_change":"2024-09-10T10:05:31.179275+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.179275+0000","last_undegraded":"2024-09-10T10:05:31.179275+0000","last_fullsized":"2024-09-10T10:05:31.179275+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.194166+0000","last_change":"2024-09-10T10:05:31.193385+0000","last_active":"2024-09-10T10:05:31.194166+0000","last_peered":"2024-09-10T10:05:31.194166+0000","last_clean":"2024-09-10T10:05:31.194166+0000","last_became_active":"2024-09-10T10:05:31.193085+0000","last_became_peered":"2024-09-10T10:05:31.193085+0000","last_unstale":"2024-09-10T10:05:31.194166+0000","last_undegraded":"2024-09-10T10:05:31.194166+0000","last_fullsized":"2024-09-10T10:05:31.194166+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:13.939828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.178450+0000","last_change":"2024-09-10T10:05:31.178450+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.178450+0000","last_undegraded":"2024-09-10T10:05:31.178450+0000","last_fullsized":"2024-09-10T10:05:31.178450+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.188996+0000","last_change":"2024-09-10T10:05:31.186964+0000","last_active":"2024-09-10T10:05:31.188996+0000","last_peered":"2024-09-10T10:05:31.188996+0000","last_clean":"2024-09-10T10:05:31.188996+0000","last_became_active":"2024-09-10T10:05:31.186613+0000","last_became_peered":"2024-09-10T10:05:31.186613+0000","last_unstale":"2024-09-10T10:05:31.188996+0000","last_undegraded":"2024-09-10T10:05:31.188996+0000","last_fullsized":"2024-09-10T10:05:31.188996+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:59.668546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.172624+0000","last_change":"2024-09-10T10:05:31.172624+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.172624+0000","last_undegraded":"2024-09-10T10:05:31.172624+0000","last_fullsized":"2024-09-10T10:05:31.172624+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.190738+0000","last_change":"2024-09-10T10:05:31.187417+0000","last_active":"2024-09-10T10:05:31.190738+0000","last_peered":"2024-09-10T10:05:31.190738+0000","last_clean":"2024-09-10T10:05:31.190738+0000","last_became_active":"2024-09-10T10:05:31.187030+0000","last_became_peered":"2024-09-10T10:05:31.187030+0000","last_unstale":"2024-09-10T10:05:31.190738+0000","last_undegraded":"2024-09-10T10:05:31.190738+0000","last_fullsized":"2024-09-10T10:05:31.190738+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:13.626786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.171896+0000","last_change":"2024-09-10T10:05:31.171896+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.171896+0000","last_undegraded":"2024-09-10T10:05:31.171896+0000","last_fullsized":"2024-09-10T10:05:31.171896+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.197321+0000","last_change":"2024-09-10T10:05:31.194388+0000","last_active":"2024-09-10T10:05:31.197321+0000","last_peered":"2024-09-10T10:05:31.197321+0000","last_clean":"2024-09-10T10:05:31.197321+0000","last_became_active":"2024-09-10T10:05:31.193967+0000","last_became_peered":"2024-09-10T10:05:31.193967+0000","last_unstale":"2024-09-10T10:05:31.197321+0000","last_undegraded":"2024-09-10T10:05:31.197321+0000","last_fullsized":"2024-09-10T10:05:31.197321+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:53.149484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":151,"state":"creating+peering","last_fresh":"2024-09-10T10:05:31.175422+0000","last_change":"2024-09-10T10:05:31.175422+0000","last_active":"2024-09-10T10:05:31.158515+0000","last_peered":"2024-09-10T10:05:31.158515+0000","last_clean":"2024-09-10T10:05:31.158515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:05:31.175422+0000","last_undegraded":"2024-09-10T10:05:31.175422+0000","last_fullsized":"2024-09-10T10:05:31.175422+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.192494+0000","last_change":"2024-09-10T10:05:31.191205+0000","last_active":"2024-09-10T10:05:31.192494+0000","last_peered":"2024-09-10T10:05:31.192494+0000","last_clean":"2024-09-10T10:05:31.192494+0000","last_became_active":"2024-09-10T10:05:31.190885+0000","last_became_peered":"2024-09-10T10:05:31.190885+0000","last_unstale":"2024-09-10T10:05:31.192494+0000","last_undegraded":"2024-09-10T10:05:31.192494+0000","last_fullsized":"2024-09-10T10:05:31.192494+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:58.487180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":444,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T10:05:31.180502+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:05:31.180502+0000","last_peered":"2024-09-10T10:05:31.180502+0000","last_clean":"2024-09-10T10:05:31.180502+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:05:31.180502+0000","last_undegraded":"2024-09-10T10:05:31.180502+0000","last_fullsized":"2024-09-10T10:05:31.180502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2490368,"data_stored":1425104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509530,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542236,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37188,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334652,"statfs":{"total":96636764160,"available":96598683648,"internally_reserved":0,"allocated":1884160,"data_stored":828405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37188,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334652,"statfs":{"total":96636764160,"available":96598683648,"internally_reserved":0,"allocated":1884160,"data_stored":828405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37204,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":1896448,"data_stored":834736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2490368,"data_stored":1425104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:05:37.021 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3f is not active+clean 2024-09-10T10:05:37.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.178157+0000', 'last_change': '2024-09-10T10:05:31.178157+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.178157+0000', 'last_undegraded': '2024-09-10T10:05:31.178157+0000', 'last_fullsized': '2024-09-10T10:05:31.178157+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3e is not active+clean 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173328+0000', 'last_change': '2024-09-10T10:05:31.173328+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173328+0000', 'last_undegraded': '2024-09-10T10:05:31.173328+0000', 'last_fullsized': '2024-09-10T10:05:31.173328+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3d is not active+clean 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.179146+0000', 'last_change': '2024-09-10T10:05:31.179146+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.179146+0000', 'last_undegraded': '2024-09-10T10:05:31.179146+0000', 'last_fullsized': '2024-09-10T10:05:31.179146+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3c is not active+clean 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173862+0000', 'last_change': '2024-09-10T10:05:31.173862+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173862+0000', 'last_undegraded': '2024-09-10T10:05:31.173862+0000', 'last_fullsized': '2024-09-10T10:05:31.173862+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3b is not active+clean 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.172850+0000', 'last_change': '2024-09-10T10:05:31.172850+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.172850+0000', 'last_undegraded': '2024-09-10T10:05:31.172850+0000', 'last_fullsized': '2024-09-10T10:05:31.172850+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3a is not active+clean 2024-09-10T10:05:37.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173582+0000', 'last_change': '2024-09-10T10:05:31.173582+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173582+0000', 'last_undegraded': '2024-09-10T10:05:31.173582+0000', 'last_fullsized': '2024-09-10T10:05:31.173582+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:PG 25.39 is not active+clean 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.171259+0000', 'last_change': '2024-09-10T10:05:31.171259+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.171259+0000', 'last_undegraded': '2024-09-10T10:05:31.171259+0000', 'last_fullsized': '2024-09-10T10:05:31.171259+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:PG 25.38 is not active+clean 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.171214+0000', 'last_change': '2024-09-10T10:05:31.171214+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.171214+0000', 'last_undegraded': '2024-09-10T10:05:31.171214+0000', 'last_fullsized': '2024-09-10T10:05:31.171214+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:PG 25.37 is not active+clean 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.180484+0000', 'last_change': '2024-09-10T10:05:31.180484+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.180484+0000', 'last_undegraded': '2024-09-10T10:05:31.180484+0000', 'last_fullsized': '2024-09-10T10:05:31.180484+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:PG 25.36 is not active+clean 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.187473+0000', 'last_change': '2024-09-10T10:05:31.187473+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.187473+0000', 'last_undegraded': '2024-09-10T10:05:31.187473+0000', 'last_fullsized': '2024-09-10T10:05:31.187473+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:PG 25.35 is not active+clean 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170730+0000', 'last_change': '2024-09-10T10:05:31.170730+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170730+0000', 'last_undegraded': '2024-09-10T10:05:31.170730+0000', 'last_fullsized': '2024-09-10T10:05:31.170730+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:PG 25.34 is not active+clean 2024-09-10T10:05:37.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173444+0000', 'last_change': '2024-09-10T10:05:31.173444+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173444+0000', 'last_undegraded': '2024-09-10T10:05:31.173444+0000', 'last_fullsized': '2024-09-10T10:05:31.173444+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:PG 25.33 is not active+clean 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173147+0000', 'last_change': '2024-09-10T10:05:31.173147+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173147+0000', 'last_undegraded': '2024-09-10T10:05:31.173147+0000', 'last_fullsized': '2024-09-10T10:05:31.173147+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:PG 25.32 is not active+clean 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.178165+0000', 'last_change': '2024-09-10T10:05:31.178165+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.178165+0000', 'last_undegraded': '2024-09-10T10:05:31.178165+0000', 'last_fullsized': '2024-09-10T10:05:31.178165+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:PG 25.31 is not active+clean 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.172248+0000', 'last_change': '2024-09-10T10:05:31.172248+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.172248+0000', 'last_undegraded': '2024-09-10T10:05:31.172248+0000', 'last_fullsized': '2024-09-10T10:05:31.172248+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:PG 25.30 is not active+clean 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170807+0000', 'last_change': '2024-09-10T10:05:31.170807+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170807+0000', 'last_undegraded': '2024-09-10T10:05:31.170807+0000', 'last_fullsized': '2024-09-10T10:05:31.170807+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2f is not active+clean 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173631+0000', 'last_change': '2024-09-10T10:05:31.173631+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173631+0000', 'last_undegraded': '2024-09-10T10:05:31.173631+0000', 'last_fullsized': '2024-09-10T10:05:31.173631+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2e is not active+clean 2024-09-10T10:05:37.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.181226+0000', 'last_change': '2024-09-10T10:05:31.181226+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.181226+0000', 'last_undegraded': '2024-09-10T10:05:31.181226+0000', 'last_fullsized': '2024-09-10T10:05:31.181226+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2d is not active+clean 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.180425+0000', 'last_change': '2024-09-10T10:05:31.180425+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.180425+0000', 'last_undegraded': '2024-09-10T10:05:31.180425+0000', 'last_fullsized': '2024-09-10T10:05:31.180425+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2c is not active+clean 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.175359+0000', 'last_change': '2024-09-10T10:05:31.175359+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.175359+0000', 'last_undegraded': '2024-09-10T10:05:31.175359+0000', 'last_fullsized': '2024-09-10T10:05:31.175359+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2b is not active+clean 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.172406+0000', 'last_change': '2024-09-10T10:05:31.172406+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.172406+0000', 'last_undegraded': '2024-09-10T10:05:31.172406+0000', 'last_fullsized': '2024-09-10T10:05:31.172406+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2a is not active+clean 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.181999+0000', 'last_change': '2024-09-10T10:05:31.181999+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.181999+0000', 'last_undegraded': '2024-09-10T10:05:31.181999+0000', 'last_fullsized': '2024-09-10T10:05:31.181999+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:PG 25.29 is not active+clean 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.177007+0000', 'last_change': '2024-09-10T10:05:31.177007+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.177007+0000', 'last_undegraded': '2024-09-10T10:05:31.177007+0000', 'last_fullsized': '2024-09-10T10:05:31.177007+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:05:37.025 INFO:tasks.ceph.ceph_manager.ceph:PG 25.28 is not active+clean 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170320+0000', 'last_change': '2024-09-10T10:05:31.170320+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170320+0000', 'last_undegraded': '2024-09-10T10:05:31.170320+0000', 'last_fullsized': '2024-09-10T10:05:31.170320+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:PG 25.27 is not active+clean 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.172778+0000', 'last_change': '2024-09-10T10:05:31.172778+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.172778+0000', 'last_undegraded': '2024-09-10T10:05:31.172778+0000', 'last_fullsized': '2024-09-10T10:05:31.172778+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:PG 25.26 is not active+clean 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.171873+0000', 'last_change': '2024-09-10T10:05:31.171873+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.171873+0000', 'last_undegraded': '2024-09-10T10:05:31.171873+0000', 'last_fullsized': '2024-09-10T10:05:31.171873+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:PG 25.25 is not active+clean 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.177404+0000', 'last_change': '2024-09-10T10:05:31.177404+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.177404+0000', 'last_undegraded': '2024-09-10T10:05:31.177404+0000', 'last_fullsized': '2024-09-10T10:05:31.177404+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:PG 25.24 is not active+clean 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.174881+0000', 'last_change': '2024-09-10T10:05:31.174881+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.174881+0000', 'last_undegraded': '2024-09-10T10:05:31.174881+0000', 'last_fullsized': '2024-09-10T10:05:31.174881+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:PG 25.23 is not active+clean 2024-09-10T10:05:37.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.172702+0000', 'last_change': '2024-09-10T10:05:31.172702+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.172702+0000', 'last_undegraded': '2024-09-10T10:05:31.172702+0000', 'last_fullsized': '2024-09-10T10:05:31.172702+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:PG 25.22 is not active+clean 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.177678+0000', 'last_change': '2024-09-10T10:05:31.177678+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.177678+0000', 'last_undegraded': '2024-09-10T10:05:31.177678+0000', 'last_fullsized': '2024-09-10T10:05:31.177678+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:PG 25.21 is not active+clean 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173678+0000', 'last_change': '2024-09-10T10:05:31.173678+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173678+0000', 'last_undegraded': '2024-09-10T10:05:31.173678+0000', 'last_fullsized': '2024-09-10T10:05:31.173678+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:PG 25.20 is not active+clean 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.172528+0000', 'last_change': '2024-09-10T10:05:31.172528+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.172528+0000', 'last_undegraded': '2024-09-10T10:05:31.172528+0000', 'last_fullsized': '2024-09-10T10:05:31.172528+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1e is not active+clean 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170496+0000', 'last_change': '2024-09-10T10:05:31.170496+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170496+0000', 'last_undegraded': '2024-09-10T10:05:31.170496+0000', 'last_fullsized': '2024-09-10T10:05:31.170496+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1f is not active+clean 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.179647+0000', 'last_change': '2024-09-10T10:05:31.179647+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.179647+0000', 'last_undegraded': '2024-09-10T10:05:31.179647+0000', 'last_fullsized': '2024-09-10T10:05:31.179647+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1c is not active+clean 2024-09-10T10:05:37.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.174762+0000', 'last_change': '2024-09-10T10:05:31.174762+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.174762+0000', 'last_undegraded': '2024-09-10T10:05:31.174762+0000', 'last_fullsized': '2024-09-10T10:05:31.174762+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1d is not active+clean 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.177235+0000', 'last_change': '2024-09-10T10:05:31.177235+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.177235+0000', 'last_undegraded': '2024-09-10T10:05:31.177235+0000', 'last_fullsized': '2024-09-10T10:05:31.177235+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1a is not active+clean 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.181360+0000', 'last_change': '2024-09-10T10:05:31.181360+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.181360+0000', 'last_undegraded': '2024-09-10T10:05:31.181360+0000', 'last_fullsized': '2024-09-10T10:05:31.181360+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1b is not active+clean 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.171144+0000', 'last_change': '2024-09-10T10:05:31.171144+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.171144+0000', 'last_undegraded': '2024-09-10T10:05:31.171144+0000', 'last_fullsized': '2024-09-10T10:05:31.171144+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:PG 25.19 is not active+clean 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170694+0000', 'last_change': '2024-09-10T10:05:31.170694+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170694+0000', 'last_undegraded': '2024-09-10T10:05:31.170694+0000', 'last_fullsized': '2024-09-10T10:05:31.170694+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:PG 25.16 is not active+clean 2024-09-10T10:05:37.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.178681+0000', 'last_change': '2024-09-10T10:05:31.178681+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.178681+0000', 'last_undegraded': '2024-09-10T10:05:31.178681+0000', 'last_fullsized': '2024-09-10T10:05:31.178681+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:PG 25.17 is not active+clean 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170177+0000', 'last_change': '2024-09-10T10:05:31.170177+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170177+0000', 'last_undegraded': '2024-09-10T10:05:31.170177+0000', 'last_fullsized': '2024-09-10T10:05:31.170177+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:PG 25.14 is not active+clean 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.174289+0000', 'last_change': '2024-09-10T10:05:31.174289+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.174289+0000', 'last_undegraded': '2024-09-10T10:05:31.174289+0000', 'last_fullsized': '2024-09-10T10:05:31.174289+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:PG 25.15 is not active+clean 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.179119+0000', 'last_change': '2024-09-10T10:05:31.179119+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.179119+0000', 'last_undegraded': '2024-09-10T10:05:31.179119+0000', 'last_fullsized': '2024-09-10T10:05:31.179119+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:PG 25.12 is not active+clean 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173030+0000', 'last_change': '2024-09-10T10:05:31.173030+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173030+0000', 'last_undegraded': '2024-09-10T10:05:31.173030+0000', 'last_fullsized': '2024-09-10T10:05:31.173030+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:PG 25.13 is not active+clean 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173658+0000', 'last_change': '2024-09-10T10:05:31.173658+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173658+0000', 'last_undegraded': '2024-09-10T10:05:31.173658+0000', 'last_fullsized': '2024-09-10T10:05:31.173658+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:PG 25.10 is not active+clean 2024-09-10T10:05:37.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.179062+0000', 'last_change': '2024-09-10T10:05:31.179062+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.179062+0000', 'last_undegraded': '2024-09-10T10:05:31.179062+0000', 'last_fullsized': '2024-09-10T10:05:31.179062+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:PG 25.11 is not active+clean 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.176025+0000', 'last_change': '2024-09-10T10:05:31.176025+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.176025+0000', 'last_undegraded': '2024-09-10T10:05:31.176025+0000', 'last_fullsized': '2024-09-10T10:05:31.176025+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:PG 25.e is not active+clean 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.181741+0000', 'last_change': '2024-09-10T10:05:31.181741+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.181741+0000', 'last_undegraded': '2024-09-10T10:05:31.181741+0000', 'last_fullsized': '2024-09-10T10:05:31.181741+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:PG 25.f is not active+clean 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.179763+0000', 'last_change': '2024-09-10T10:05:31.179763+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.179763+0000', 'last_undegraded': '2024-09-10T10:05:31.179763+0000', 'last_fullsized': '2024-09-10T10:05:31.179763+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:PG 25.c is not active+clean 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.177666+0000', 'last_change': '2024-09-10T10:05:31.177666+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.177666+0000', 'last_undegraded': '2024-09-10T10:05:31.177666+0000', 'last_fullsized': '2024-09-10T10:05:31.177666+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:PG 25.d is not active+clean 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173084+0000', 'last_change': '2024-09-10T10:05:31.173084+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173084+0000', 'last_undegraded': '2024-09-10T10:05:31.173084+0000', 'last_fullsized': '2024-09-10T10:05:31.173084+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:PG 25.a is not active+clean 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.180942+0000', 'last_change': '2024-09-10T10:05:31.180942+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.180942+0000', 'last_undegraded': '2024-09-10T10:05:31.180942+0000', 'last_fullsized': '2024-09-10T10:05:31.180942+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.030 INFO:tasks.ceph.ceph_manager.ceph:PG 25.b is not active+clean 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170703+0000', 'last_change': '2024-09-10T10:05:31.170703+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170703+0000', 'last_undegraded': '2024-09-10T10:05:31.170703+0000', 'last_fullsized': '2024-09-10T10:05:31.170703+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:PG 25.8 is not active+clean 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.173203+0000', 'last_change': '2024-09-10T10:05:31.173203+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.173203+0000', 'last_undegraded': '2024-09-10T10:05:31.173203+0000', 'last_fullsized': '2024-09-10T10:05:31.173203+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:PG 25.9 is not active+clean 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.174497+0000', 'last_change': '2024-09-10T10:05:31.174497+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.174497+0000', 'last_undegraded': '2024-09-10T10:05:31.174497+0000', 'last_fullsized': '2024-09-10T10:05:31.174497+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:PG 25.6 is not active+clean 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.172755+0000', 'last_change': '2024-09-10T10:05:31.172755+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.172755+0000', 'last_undegraded': '2024-09-10T10:05:31.172755+0000', 'last_fullsized': '2024-09-10T10:05:31.172755+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:PG 25.7 is not active+clean 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.178608+0000', 'last_change': '2024-09-10T10:05:31.178608+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.178608+0000', 'last_undegraded': '2024-09-10T10:05:31.178608+0000', 'last_fullsized': '2024-09-10T10:05:31.178608+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:PG 25.4 is not active+clean 2024-09-10T10:05:37.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170129+0000', 'last_change': '2024-09-10T10:05:31.170129+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170129+0000', 'last_undegraded': '2024-09-10T10:05:31.170129+0000', 'last_fullsized': '2024-09-10T10:05:31.170129+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:PG 25.5 is not active+clean 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.170949+0000', 'last_change': '2024-09-10T10:05:31.170949+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.170949+0000', 'last_undegraded': '2024-09-10T10:05:31.170949+0000', 'last_fullsized': '2024-09-10T10:05:31.170949+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2 is not active+clean 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.179275+0000', 'last_change': '2024-09-10T10:05:31.179275+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.179275+0000', 'last_undegraded': '2024-09-10T10:05:31.179275+0000', 'last_fullsized': '2024-09-10T10:05:31.179275+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3 is not active+clean 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.178450+0000', 'last_change': '2024-09-10T10:05:31.178450+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.178450+0000', 'last_undegraded': '2024-09-10T10:05:31.178450+0000', 'last_fullsized': '2024-09-10T10:05:31.178450+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:PG 25.0 is not active+clean 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.172624+0000', 'last_change': '2024-09-10T10:05:31.172624+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.172624+0000', 'last_undegraded': '2024-09-10T10:05:31.172624+0000', 'last_fullsized': '2024-09-10T10:05:31.172624+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1 is not active+clean 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.171896+0000', 'last_change': '2024-09-10T10:05:31.171896+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.171896+0000', 'last_undegraded': '2024-09-10T10:05:31.171896+0000', 'last_fullsized': '2024-09-10T10:05:31.171896+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:05:37.032 INFO:tasks.ceph.ceph_manager.ceph:PG 25.18 is not active+clean 2024-09-10T10:05:37.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 151, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:05:31.175422+0000', 'last_change': '2024-09-10T10:05:31.175422+0000', 'last_active': '2024-09-10T10:05:31.158515+0000', 'last_peered': '2024-09-10T10:05:31.158515+0000', 'last_clean': '2024-09-10T10:05:31.158515+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:05:31.175422+0000', 'last_undegraded': '2024-09-10T10:05:31.175422+0000', 'last_fullsized': '2024-09-10T10:05:31.175422+0000', 'mapping_epoch': 151, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 151, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'last_clean_scrub_stamp': '2024-09-10T10:05:31.158515+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:05:37.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T10:05:37.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7bbc00640 1 Processor -- start 2024-09-10T10:05:37.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7bbc00640 1 -- start start 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7bbc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b406eb30 0x7fb7b40d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7bbc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb7b4163000 0x7fb7b4071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7bbc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb7b4073290 0x7fb7b4073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7bbc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b415ee90 con 0x7fb7b4163000 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7bbc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40710d0 con 0x7fb7b4073290 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7bbc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b40d1370 con 0x7fb7b406eb30 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb7b4163000 0x7fb7b4071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb7b4163000 0x7fb7b4071560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34316/0 (socket says 172.21.15.110:34316) 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b3e00640 1 -- 172.21.15.110:0/3448198298 learned_addr learned my addr 172.21.15.110:0/3448198298 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b9200640 1 --2- 172.21.15.110:0/3448198298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b406eb30 0x7fb7b40d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b9c00640 1 --2- 172.21.15.110:0/3448198298 >> v2:172.21.15.145:3300/0 conn(0x7fb7b4073290 0x7fb7b4073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b3e00640 1 -- 172.21.15.110:0/3448198298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b406eb30 msgr2=0x7fb7b40d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:37.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b3e00640 1 --2- 172.21.15.110:0/3448198298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b406eb30 0x7fb7b40d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b3e00640 1 -- 172.21.15.110:0/3448198298 >> v2:172.21.15.145:3300/0 conn(0x7fb7b4073290 msgr2=0x7fb7b4073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:37.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b3e00640 1 --2- 172.21.15.110:0/3448198298 >> v2:172.21.15.145:3300/0 conn(0x7fb7b4073290 0x7fb7b4073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.197+0000 7fb7b3e00640 1 -- 172.21.15.110:0/3448198298 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b40706a0 con 0x7fb7b4163000 2024-09-10T10:05:37.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7b3e00640 1 --2- 172.21.15.110:0/3448198298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 0x7fb7b4071560 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4063650 tx=0x7fb7a001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ab62dc8d3469 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:37.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7b3400640 1 -- 172.21.15.110:0/3448198298 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00b0070 con 0x7fb7b4163000 2024-09-10T10:05:37.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7b3400640 1 -- 172.21.15.110:0/3448198298 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a00ab600 con 0x7fb7b4163000 2024-09-10T10:05:37.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7b3400640 1 -- 172.21.15.110:0/3448198298 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00ab9d0 con 0x7fb7b4163000 2024-09-10T10:05:37.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 -- 172.21.15.110:0/3448198298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 msgr2=0x7fb7b4071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/3448198298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 0x7fb7b4071560 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4063650 tx=0x7fb7a001c490 comp rx=0 tx=0).stop 2024-09-10T10:05:37.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 -- 172.21.15.110:0/3448198298 shutdown_connections 2024-09-10T10:05:37.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/3448198298 >> v2:172.21.15.145:3300/0 conn(0x7fb7b4073290 0x7fb7b4073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/3448198298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 0x7fb7b4071560 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/3448198298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b406eb30 0x7fb7b40d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 -- 172.21.15.110:0/3448198298 >> 172.21.15.110:0/3448198298 conn(0x7fb7b40d25c0 msgr2=0x7fb7b40d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:37.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 -- 172.21.15.110:0/3448198298 shutdown_connections 2024-09-10T10:05:37.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.201+0000 7fb7bbc00640 1 -- 172.21.15.110:0/3448198298 wait complete. 2024-09-10T10:05:37.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 Processor -- start 2024-09-10T10:05:37.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 -- start start 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b406eb30 0x7fb7b41552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b4073290 0x7fb7b4156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 0x7fb7b415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4071aa0 con 0x7fb7b4163000 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4163440 con 0x7fb7b406eb30 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b4071c20 con 0x7fb7b4073290 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 0x7fb7b415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b406eb30 0x7fb7b41552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b406eb30 0x7fb7b41552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:51836/0 (socket says 172.21.15.110:51836) 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9200640 1 -- 172.21.15.110:0/2951030564 learned_addr learned my addr 172.21.15.110:0/2951030564 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b4073290 0x7fb7b4156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9c00640 1 -- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b4073290 msgr2=0x7fb7b4156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9c00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b4073290 0x7fb7b4156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9c00640 1 -- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b406eb30 msgr2=0x7fb7b41552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9c00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b406eb30 0x7fb7b41552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9c00640 1 -- 172.21.15.110:0/2951030564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b415df10 con 0x7fb7b4163000 2024-09-10T10:05:37.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b9c00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 0x7fb7b415f670 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00efc0 tx=0x7fb7ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b0c00640 1 -- 172.21.15.110:0/2951030564 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac018020 con 0x7fb7b4163000 2024-09-10T10:05:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b0c00640 1 -- 172.21.15.110:0/2951030564 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ac014480 con 0x7fb7b4163000 2024-09-10T10:05:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b415d670 con 0x7fb7b4163000 2024-09-10T10:05:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7b0c00640 1 -- 172.21.15.110:0/2951030564 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac014870 con 0x7fb7b4163000 2024-09-10T10:05:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b415dc30 con 0x7fb7b4163000 2024-09-10T10:05:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.205+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b406dd60 con 0x7fb7b4163000 2024-09-10T10:05:37.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.209+0000 7fb7b0c00640 1 -- 172.21.15.110:0/2951030564 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fb7ac01b020 con 0x7fb7b4163000 2024-09-10T10:05:37.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.209+0000 7fb7b0c00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7940680a0 0x7fb79406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.209+0000 7fb7b0c00640 1 -- 172.21.15.110:0/2951030564 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7fb7ac087970 con 0x7fb7b4163000 2024-09-10T10:05:37.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.209+0000 7fb7b9200640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7940680a0 0x7fb79406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.209+0000 7fb7b9200640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7940680a0 0x7fb79406a560 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4162430 tx=0x7fb7a4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:37.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.213+0000 7fb7b0c00640 1 -- 172.21.15.110:0/2951030564 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb7ac022030 con 0x7fb7b4163000 2024-09-10T10:05:37.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.425+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb7b40b1c40 con 0x7fb7b4163000 2024-09-10T10:05:37.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.425+0000 7fb7b0c00640 1 -- 172.21.15.110:0/2951030564 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1500 (secure 0 0 0) 0x7fb7ac010040 con 0x7fb7b4163000 2024-09-10T10:05:37.428 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:37.428 INFO:teuthology.orchestra.run.smithi110.stdout:{"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":459,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":153,"num_osds":8,"num_up_osds":8,"osd_up_since":1725962292,"num_in_osds":8,"osd_in_since":1725962280,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":288882688,"bytes_avail":772805230592,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":106,"btime":"2024-09-10T10:05:35:199222+0000","id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"b","status":"up:active","gid":8526}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T10:04:51.728881+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7940680a0 msgr2=0x7fb79406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7940680a0 0x7fb79406a560 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4162430 tx=0x7fb7a4009190 comp rx=0 tx=0).stop 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 msgr2=0x7fb7b415f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 0x7fb7b415f670 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00efc0 tx=0x7fb7ac00c490 comp rx=0 tx=0).stop 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 shutdown_connections 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7940680a0 0x7fb79406a560 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7b4163000 0x7fb7b415f670 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7b4073290 0x7fb7b4156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 --2- 172.21.15.110:0/2951030564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b406eb30 0x7fb7b41552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 >> 172.21.15.110:0/2951030564 conn(0x7fb7b40d25c0 msgr2=0x7fb7b40b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 shutdown_connections 2024-09-10T10:05:37.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.429+0000 7fb7bbc00640 1 -- 172.21.15.110:0/2951030564 wait complete. 2024-09-10T10:05:37.450 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T10:05:37.450 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:05:37.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85eea00640 1 Processor -- start 2024-09-10T10:05:37.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85eea00640 1 -- start start 2024-09-10T10:05:37.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85eea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f85e80cd510 0x7f85e80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85eea00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e8072150 con 0x7f85e806e090 2024-09-10T10:05:37.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85eea00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e806d2d0 con 0x7f85e80cfe30 2024-09-10T10:05:37.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85eea00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f85e815d9a0 con 0x7f85e80cd510 2024-09-10T10:05:37.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85eca00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f85e806e090 0x7f85e806e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:48972/0 (socket says 172.21.15.110:48972) 2024-09-10T10:05:37.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85eca00640 1 -- 172.21.15.110:0/2502089690 learned_addr learned my addr 172.21.15.110:0/2502089690 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:37.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85e7400640 1 --2- 172.21.15.110:0/2502089690 >> v2:172.21.15.145:3300/0 conn(0x7f85e80cd510 0x7f85e80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3728367369 0 0) 0x7f85e8072150 con 0x7f85e806e090 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85cc003530 con 0x7f85e806e090 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 86544916 0 0) 0x7f85e806d2d0 con 0x7f85e80cfe30 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85e8072150 con 0x7f85e80cfe30 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3411466484 0 0) 0x7f85cc003530 con 0x7f85e806e090 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 >> v2:172.21.15.145:3300/0 conn(0x7f85e80cd510 msgr2=0x7f85e80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.585+0000 7f85e6a00640 1 --2- 172.21.15.110:0/2502089690 >> v2:172.21.15.145:3300/0 conn(0x7f85e80cd510 0x7f85e80cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 >> v1:172.21.15.145:6789/0 conn(0x7f85e80cfe30 legacy=0x7f85e8071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e806d5b0 con 0x7f85e806e090 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f85e0001680 con 0x7f85e806e090 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 >> v1:172.21.15.110:6789/0 conn(0x7f85e806e090 legacy=0x7f85e806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85cc003b50 0x7f85cc003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc004810 0x7f85cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85cc0051d0 0x7f85cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85e815d9a0 con 0x7f85cc003b50 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc00c100 con 0x7f85cc0051d0 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f85cc00c300 con 0x7f85cc004810 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7e00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85cc003b50 0x7f85cc003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7400640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc004810 0x7f85cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85eca00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85cc0051d0 0x7f85cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7e00640 1 -- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc004810 msgr2=0x7f85cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7e00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc004810 0x7f85cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7e00640 1 -- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85cc0051d0 msgr2=0x7f85cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7e00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85cc0051d0 0x7f85cc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7e00640 1 -- 172.21.15.110:0/2502089690 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc003530 con 0x7f85cc003b50 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85eca00640 1 -- 172.21.15.110:0/2502089690 reap_dead start 2024-09-10T10:05:37.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7400640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc004810 0x7f85cc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:37.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e7e00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85cc003b50 0x7f85cc003f50 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f85d8001fd0 tx=0x7f85d801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae527d269a19af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:37.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8001c20 con 0x7f85cc003b50 2024-09-10T10:05:37.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d8003ba0 con 0x7f85cc003b50 2024-09-10T10:05:37.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.589+0000 7f85e6a00640 1 -- 172.21.15.110:0/2502089690 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80050d0 con 0x7f85cc003b50 2024-09-10T10:05:37.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- 172.21.15.110:0/2502089690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85cc003b50 msgr2=0x7f85cc003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85cc003b50 0x7f85cc003f50 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f85d8001fd0 tx=0x7f85d801c8e0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- 172.21.15.110:0/2502089690 shutdown_connections 2024-09-10T10:05:37.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 --2- 172.21.15.110:0/2502089690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85cc003b50 0x7f85cc003f50 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- 172.21.15.110:0/2502089690 >> 172.21.15.110:0/2502089690 conn(0x7f85e806bc50 msgr2=0x7f85e806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:37.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- 172.21.15.110:0/2502089690 shutdown_connections 2024-09-10T10:05:37.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- 172.21.15.110:0/2502089690 wait complete. 2024-09-10T10:05:37.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 Processor -- start 2024-09-10T10:05:37.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- start start 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc003b50 0x7f85e815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e80ac7c0 0x7f85e80acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e80b2610 0x7f85e80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85e815d5b0 con 0x7f85e80ac7c0 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85e815d430 con 0x7f85e80b2610 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eea00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f85e815d730 con 0x7f85cc003b50 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e80ac7c0 0x7f85e80acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e80ac7c0 0x7f85e80acc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34338/0 (socket says 172.21.15.110:34338) 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7400640 1 -- 172.21.15.110:0/3854403085 learned_addr learned my addr 172.21.15.110:0/3854403085 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7e00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc003b50 0x7f85e815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85eca00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e80b2610 0x7f85e80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7400640 1 -- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc003b50 msgr2=0x7f85e815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7400640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc003b50 0x7f85e815c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7400640 1 -- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e80b2610 msgr2=0x7f85e80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7400640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e80b2610 0x7f85e80b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.593+0000 7f85e7400640 1 -- 172.21.15.110:0/3854403085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e815c810 con 0x7f85e80ac7c0 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85e7400640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e80ac7c0 0x7f85e80acc20 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f85d4011540 tx=0x7f85d4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85dfe00640 1 -- 172.21.15.110:0/3854403085 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4004220 con 0x7f85e80ac7c0 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85dfe00640 1 -- 172.21.15.110:0/3854403085 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d40043c0 con 0x7f85e80ac7c0 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85dfe00640 1 -- 172.21.15.110:0/3854403085 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4015890 con 0x7f85e80ac7c0 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e81572f0 con 0x7f85e80ac7c0 2024-09-10T10:05:37.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e80ad610 con 0x7f85e80ac7c0 2024-09-10T10:05:37.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e806e470 con 0x7f85e80ac7c0 2024-09-10T10:05:37.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85dfe00640 1 -- 172.21.15.110:0/3854403085 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f85d4015a30 con 0x7f85e80ac7c0 2024-09-10T10:05:37.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85dfe00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f85c80680d0 0x7f85c806a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:37.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85dfe00640 1 -- 172.21.15.110:0/3854403085 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f85d4087110 con 0x7f85e80ac7c0 2024-09-10T10:05:37.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.597+0000 7f85e7e00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f85c80680d0 0x7f85c806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:37.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.601+0000 7f85e7e00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f85c80680d0 0x7f85c806a590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f85d80ad000 tx=0x7f85d80a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:37.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.601+0000 7f85dfe00640 1 -- 172.21.15.110:0/3854403085 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f85d4019030 con 0x7f85e80ac7c0 2024-09-10T10:05:37.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.757+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85e8072ff0 con 0x7f85c80680d0 2024-09-10T10:05:37.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85dfe00640 1 -- 172.21.15.110:0/3854403085 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f85e8072ff0 con 0x7f85c80680d0 2024-09-10T10:05:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:37.769 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f85c80680d0 msgr2=0x7f85c806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f85c80680d0 0x7f85c806a590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f85d80ad000 tx=0x7f85d80a7070 comp rx=0 tx=0).stop 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e80ac7c0 msgr2=0x7f85e80acc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e80ac7c0 0x7f85e80acc20 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f85d4011540 tx=0x7f85d4011a00 comp rx=0 tx=0).stop 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 shutdown_connections 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f85c80680d0 0x7f85c806a590 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85e80b2610 0x7f85e80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e80ac7c0 0x7f85e80acc20 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 --2- 172.21.15.110:0/3854403085 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f85cc003b50 0x7f85e815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 >> 172.21.15.110:0/3854403085 conn(0x7f85e806bc50 msgr2=0x7f85e8153b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 shutdown_connections 2024-09-10T10:05:37.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:37.765+0000 7f85eea00640 1 -- 172.21.15.110:0/3854403085 wait complete. 2024-09-10T10:05:37.788 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-10T10:05:37.740580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282388,"kb_used_data":16232,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692332,"statfs":{"total":773094113280,"available":772804947968,"internally_reserved":0,"allocated":16621568,"data_stored":7968940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.571877"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371175+0000","last_change":"2024-09-10T10:05:32.206947+0000","last_active":"2024-09-10T10:05:35.371175+0000","last_peered":"2024-09-10T10:05:35.371175+0000","last_clean":"2024-09-10T10:05:35.371175+0000","last_became_active":"2024-09-10T10:05:32.206692+0000","last_became_peered":"2024-09-10T10:05:32.206692+0000","last_unstale":"2024-09-10T10:05:35.371175+0000","last_undegraded":"2024-09-10T10:05:35.371175+0000","last_fullsized":"2024-09-10T10:05:35.371175+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:37.239701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198862+0000","last_change":"2024-09-10T10:05:32.195002+0000","last_active":"2024-09-10T10:05:33.198862+0000","last_peered":"2024-09-10T10:05:33.198862+0000","last_clean":"2024-09-10T10:05:33.198862+0000","last_became_active":"2024-09-10T10:05:32.194741+0000","last_became_peered":"2024-09-10T10:05:32.194741+0000","last_unstale":"2024-09-10T10:05:33.198862+0000","last_undegraded":"2024-09-10T10:05:33.198862+0000","last_fullsized":"2024-09-10T10:05:33.198862+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:24.501475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202551+0000","last_change":"2024-09-10T10:05:32.208004+0000","last_active":"2024-09-10T10:05:33.202551+0000","last_peered":"2024-09-10T10:05:33.202551+0000","last_clean":"2024-09-10T10:05:33.202551+0000","last_became_active":"2024-09-10T10:05:32.207735+0000","last_became_peered":"2024-09-10T10:05:32.207735+0000","last_unstale":"2024-09-10T10:05:33.202551+0000","last_undegraded":"2024-09-10T10:05:33.202551+0000","last_fullsized":"2024-09-10T10:05:33.202551+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:03.301673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198476+0000","last_change":"2024-09-10T10:05:32.199447+0000","last_active":"2024-09-10T10:05:33.198476+0000","last_peered":"2024-09-10T10:05:33.198476+0000","last_clean":"2024-09-10T10:05:33.198476+0000","last_became_active":"2024-09-10T10:05:32.199160+0000","last_became_peered":"2024-09-10T10:05:32.199160+0000","last_unstale":"2024-09-10T10:05:33.198476+0000","last_undegraded":"2024-09-10T10:05:33.198476+0000","last_fullsized":"2024-09-10T10:05:33.198476+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:58.744499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519304+0000","last_change":"2024-09-10T10:05:32.199666+0000","last_active":"2024-09-10T10:05:34.519304+0000","last_peered":"2024-09-10T10:05:34.519304+0000","last_clean":"2024-09-10T10:05:34.519304+0000","last_became_active":"2024-09-10T10:05:32.199156+0000","last_became_peered":"2024-09-10T10:05:32.199156+0000","last_unstale":"2024-09-10T10:05:34.519304+0000","last_undegraded":"2024-09-10T10:05:34.519304+0000","last_fullsized":"2024-09-10T10:05:34.519304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:32:58.263910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202462+0000","last_change":"2024-09-10T10:05:32.211878+0000","last_active":"2024-09-10T10:05:33.202462+0000","last_peered":"2024-09-10T10:05:33.202462+0000","last_clean":"2024-09-10T10:05:33.202462+0000","last_became_active":"2024-09-10T10:05:32.206847+0000","last_became_peered":"2024-09-10T10:05:32.206847+0000","last_unstale":"2024-09-10T10:05:33.202462+0000","last_undegraded":"2024-09-10T10:05:33.202462+0000","last_fullsized":"2024-09-10T10:05:33.202462+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:59.529682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197813+0000","last_change":"2024-09-10T10:05:32.187394+0000","last_active":"2024-09-10T10:05:33.197813+0000","last_peered":"2024-09-10T10:05:33.197813+0000","last_clean":"2024-09-10T10:05:33.197813+0000","last_became_active":"2024-09-10T10:05:32.186764+0000","last_became_peered":"2024-09-10T10:05:32.186764+0000","last_unstale":"2024-09-10T10:05:33.197813+0000","last_undegraded":"2024-09-10T10:05:33.197813+0000","last_fullsized":"2024-09-10T10:05:33.197813+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:03.301673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369009+0000","last_change":"2024-09-10T10:05:32.194103+0000","last_active":"2024-09-10T10:05:35.369009+0000","last_peered":"2024-09-10T10:05:35.369009+0000","last_clean":"2024-09-10T10:05:35.369009+0000","last_became_active":"2024-09-10T10:05:32.193419+0000","last_became_peered":"2024-09-10T10:05:32.193419+0000","last_unstale":"2024-09-10T10:05:35.369009+0000","last_undegraded":"2024-09-10T10:05:35.369009+0000","last_fullsized":"2024-09-10T10:05:35.369009+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:32.835603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386393+0000","last_change":"2024-09-10T10:05:32.205318+0000","last_active":"2024-09-10T10:05:34.386393+0000","last_peered":"2024-09-10T10:05:34.386393+0000","last_clean":"2024-09-10T10:05:34.386393+0000","last_became_active":"2024-09-10T10:05:32.205061+0000","last_became_peered":"2024-09-10T10:05:32.205061+0000","last_unstale":"2024-09-10T10:05:34.386393+0000","last_undegraded":"2024-09-10T10:05:34.386393+0000","last_fullsized":"2024-09-10T10:05:34.386393+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:58.989051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195297+0000","last_change":"2024-09-10T10:05:32.209313+0000","last_active":"2024-09-10T10:05:33.195297+0000","last_peered":"2024-09-10T10:05:33.195297+0000","last_clean":"2024-09-10T10:05:33.195297+0000","last_became_active":"2024-09-10T10:05:32.208815+0000","last_became_peered":"2024-09-10T10:05:32.208815+0000","last_unstale":"2024-09-10T10:05:33.195297+0000","last_undegraded":"2024-09-10T10:05:33.195297+0000","last_fullsized":"2024-09-10T10:05:33.195297+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:19.137072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198422+0000","last_change":"2024-09-10T10:05:32.202439+0000","last_active":"2024-09-10T10:05:33.198422+0000","last_peered":"2024-09-10T10:05:33.198422+0000","last_clean":"2024-09-10T10:05:33.198422+0000","last_became_active":"2024-09-10T10:05:32.202133+0000","last_became_peered":"2024-09-10T10:05:32.202133+0000","last_unstale":"2024-09-10T10:05:33.198422+0000","last_undegraded":"2024-09-10T10:05:33.198422+0000","last_fullsized":"2024-09-10T10:05:33.198422+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:12.558158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198626+0000","last_change":"2024-09-10T10:05:32.208477+0000","last_active":"2024-09-10T10:05:33.198626+0000","last_peered":"2024-09-10T10:05:33.198626+0000","last_clean":"2024-09-10T10:05:33.198626+0000","last_became_active":"2024-09-10T10:05:32.206369+0000","last_became_peered":"2024-09-10T10:05:32.206369+0000","last_unstale":"2024-09-10T10:05:33.198626+0000","last_undegraded":"2024-09-10T10:05:33.198626+0000","last_fullsized":"2024-09-10T10:05:33.198626+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:02.076083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197890+0000","last_change":"2024-09-10T10:05:32.188306+0000","last_active":"2024-09-10T10:05:33.197890+0000","last_peered":"2024-09-10T10:05:33.197890+0000","last_clean":"2024-09-10T10:05:33.197890+0000","last_became_active":"2024-09-10T10:05:32.187978+0000","last_became_peered":"2024-09-10T10:05:32.187978+0000","last_unstale":"2024-09-10T10:05:33.197890+0000","last_undegraded":"2024-09-10T10:05:33.197890+0000","last_fullsized":"2024-09-10T10:05:33.197890+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:31.639104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386026+0000","last_change":"2024-09-10T10:05:32.200311+0000","last_active":"2024-09-10T10:05:34.386026+0000","last_peered":"2024-09-10T10:05:34.386026+0000","last_clean":"2024-09-10T10:05:34.386026+0000","last_became_active":"2024-09-10T10:05:32.199524+0000","last_became_peered":"2024-09-10T10:05:32.199524+0000","last_unstale":"2024-09-10T10:05:34.386026+0000","last_undegraded":"2024-09-10T10:05:34.386026+0000","last_fullsized":"2024-09-10T10:05:34.386026+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:35.861386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198042+0000","last_change":"2024-09-10T10:05:32.196485+0000","last_active":"2024-09-10T10:05:33.198042+0000","last_peered":"2024-09-10T10:05:33.198042+0000","last_clean":"2024-09-10T10:05:33.198042+0000","last_became_active":"2024-09-10T10:05:32.196122+0000","last_became_peered":"2024-09-10T10:05:32.196122+0000","last_unstale":"2024-09-10T10:05:33.198042+0000","last_undegraded":"2024-09-10T10:05:33.198042+0000","last_fullsized":"2024-09-10T10:05:33.198042+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:02.295728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369050+0000","last_change":"2024-09-10T10:05:32.194339+0000","last_active":"2024-09-10T10:05:35.369050+0000","last_peered":"2024-09-10T10:05:35.369050+0000","last_clean":"2024-09-10T10:05:35.369050+0000","last_became_active":"2024-09-10T10:05:32.193818+0000","last_became_peered":"2024-09-10T10:05:32.193818+0000","last_unstale":"2024-09-10T10:05:35.369050+0000","last_undegraded":"2024-09-10T10:05:35.369050+0000","last_fullsized":"2024-09-10T10:05:35.369050+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:18.060307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519404+0000","last_change":"2024-09-10T10:05:32.206204+0000","last_active":"2024-09-10T10:05:34.519404+0000","last_peered":"2024-09-10T10:05:34.519404+0000","last_clean":"2024-09-10T10:05:34.519404+0000","last_became_active":"2024-09-10T10:05:32.205359+0000","last_became_peered":"2024-09-10T10:05:32.205359+0000","last_unstale":"2024-09-10T10:05:34.519404+0000","last_undegraded":"2024-09-10T10:05:34.519404+0000","last_fullsized":"2024-09-10T10:05:34.519404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:35.409146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371140+0000","last_change":"2024-09-10T10:05:32.210783+0000","last_active":"2024-09-10T10:05:35.371140+0000","last_peered":"2024-09-10T10:05:35.371140+0000","last_clean":"2024-09-10T10:05:35.371140+0000","last_became_active":"2024-09-10T10:05:32.209419+0000","last_became_peered":"2024-09-10T10:05:32.209419+0000","last_unstale":"2024-09-10T10:05:35.371140+0000","last_undegraded":"2024-09-10T10:05:35.371140+0000","last_fullsized":"2024-09-10T10:05:35.371140+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:40.349695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194927+0000","last_change":"2024-09-10T10:05:32.200603+0000","last_active":"2024-09-10T10:05:33.194927+0000","last_peered":"2024-09-10T10:05:33.194927+0000","last_clean":"2024-09-10T10:05:33.194927+0000","last_became_active":"2024-09-10T10:05:32.200267+0000","last_became_peered":"2024-09-10T10:05:32.200267+0000","last_unstale":"2024-09-10T10:05:33.194927+0000","last_undegraded":"2024-09-10T10:05:33.194927+0000","last_fullsized":"2024-09-10T10:05:33.194927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:14.117340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519233+0000","last_change":"2024-09-10T10:05:32.207078+0000","last_active":"2024-09-10T10:05:34.519233+0000","last_peered":"2024-09-10T10:05:34.519233+0000","last_clean":"2024-09-10T10:05:34.519233+0000","last_became_active":"2024-09-10T10:05:32.205823+0000","last_became_peered":"2024-09-10T10:05:32.205823+0000","last_unstale":"2024-09-10T10:05:34.519233+0000","last_undegraded":"2024-09-10T10:05:34.519233+0000","last_fullsized":"2024-09-10T10:05:34.519233+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:41.775789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519459+0000","last_change":"2024-09-10T10:05:32.199430+0000","last_active":"2024-09-10T10:05:34.519459+0000","last_peered":"2024-09-10T10:05:34.519459+0000","last_clean":"2024-09-10T10:05:34.519459+0000","last_became_active":"2024-09-10T10:05:32.198629+0000","last_became_peered":"2024-09-10T10:05:32.198629+0000","last_unstale":"2024-09-10T10:05:34.519459+0000","last_undegraded":"2024-09-10T10:05:34.519459+0000","last_fullsized":"2024-09-10T10:05:34.519459+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:56.915472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194697+0000","last_change":"2024-09-10T10:05:32.194587+0000","last_active":"2024-09-10T10:05:33.194697+0000","last_peered":"2024-09-10T10:05:33.194697+0000","last_clean":"2024-09-10T10:05:33.194697+0000","last_became_active":"2024-09-10T10:05:32.193892+0000","last_became_peered":"2024-09-10T10:05:32.193892+0000","last_unstale":"2024-09-10T10:05:33.194697+0000","last_undegraded":"2024-09-10T10:05:33.194697+0000","last_fullsized":"2024-09-10T10:05:33.194697+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:34.240642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202372+0000","last_change":"2024-09-10T10:05:32.215026+0000","last_active":"2024-09-10T10:05:33.202372+0000","last_peered":"2024-09-10T10:05:33.202372+0000","last_clean":"2024-09-10T10:05:33.202372+0000","last_became_active":"2024-09-10T10:05:32.214758+0000","last_became_peered":"2024-09-10T10:05:32.214758+0000","last_unstale":"2024-09-10T10:05:33.202372+0000","last_undegraded":"2024-09-10T10:05:33.202372+0000","last_fullsized":"2024-09-10T10:05:33.202372+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:18.256201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369178+0000","last_change":"2024-09-10T10:05:32.215277+0000","last_active":"2024-09-10T10:05:35.369178+0000","last_peered":"2024-09-10T10:05:35.369178+0000","last_clean":"2024-09-10T10:05:35.369178+0000","last_became_active":"2024-09-10T10:05:32.214987+0000","last_became_peered":"2024-09-10T10:05:32.214987+0000","last_unstale":"2024-09-10T10:05:35.369178+0000","last_undegraded":"2024-09-10T10:05:35.369178+0000","last_fullsized":"2024-09-10T10:05:35.369178+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:06.383388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198913+0000","last_change":"2024-09-10T10:05:32.197493+0000","last_active":"2024-09-10T10:05:33.198913+0000","last_peered":"2024-09-10T10:05:33.198913+0000","last_clean":"2024-09-10T10:05:33.198913+0000","last_became_active":"2024-09-10T10:05:32.197225+0000","last_became_peered":"2024-09-10T10:05:32.197225+0000","last_unstale":"2024-09-10T10:05:33.198913+0000","last_undegraded":"2024-09-10T10:05:33.198913+0000","last_fullsized":"2024-09-10T10:05:33.198913+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:03.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198434+0000","last_change":"2024-09-10T10:05:32.196836+0000","last_active":"2024-09-10T10:05:33.198434+0000","last_peered":"2024-09-10T10:05:33.198434+0000","last_clean":"2024-09-10T10:05:33.198434+0000","last_became_active":"2024-09-10T10:05:32.194797+0000","last_became_peered":"2024-09-10T10:05:32.194797+0000","last_unstale":"2024-09-10T10:05:33.198434+0000","last_undegraded":"2024-09-10T10:05:33.198434+0000","last_fullsized":"2024-09-10T10:05:33.198434+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:31.639104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202693+0000","last_change":"2024-09-10T10:05:32.215811+0000","last_active":"2024-09-10T10:05:33.202693+0000","last_peered":"2024-09-10T10:05:33.202693+0000","last_clean":"2024-09-10T10:05:33.202693+0000","last_became_active":"2024-09-10T10:05:32.208360+0000","last_became_peered":"2024-09-10T10:05:32.208360+0000","last_unstale":"2024-09-10T10:05:33.202693+0000","last_undegraded":"2024-09-10T10:05:33.202693+0000","last_fullsized":"2024-09-10T10:05:33.202693+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:37.843191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519239+0000","last_change":"2024-09-10T10:05:32.196207+0000","last_active":"2024-09-10T10:05:34.519239+0000","last_peered":"2024-09-10T10:05:34.519239+0000","last_clean":"2024-09-10T10:05:34.519239+0000","last_became_active":"2024-09-10T10:05:32.195918+0000","last_became_peered":"2024-09-10T10:05:32.195918+0000","last_unstale":"2024-09-10T10:05:34.519239+0000","last_undegraded":"2024-09-10T10:05:34.519239+0000","last_fullsized":"2024-09-10T10:05:34.519239+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:16.425830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198325+0000","last_change":"2024-09-10T10:05:32.200164+0000","last_active":"2024-09-10T10:05:33.198325+0000","last_peered":"2024-09-10T10:05:33.198325+0000","last_clean":"2024-09-10T10:05:33.198325+0000","last_became_active":"2024-09-10T10:05:32.199910+0000","last_became_peered":"2024-09-10T10:05:32.199910+0000","last_unstale":"2024-09-10T10:05:33.198325+0000","last_undegraded":"2024-09-10T10:05:33.198325+0000","last_fullsized":"2024-09-10T10:05:33.198325+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:57.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386020+0000","last_change":"2024-09-10T10:05:32.201674+0000","last_active":"2024-09-10T10:05:34.386020+0000","last_peered":"2024-09-10T10:05:34.386020+0000","last_clean":"2024-09-10T10:05:34.386020+0000","last_became_active":"2024-09-10T10:05:32.199143+0000","last_became_peered":"2024-09-10T10:05:32.199143+0000","last_unstale":"2024-09-10T10:05:34.386020+0000","last_undegraded":"2024-09-10T10:05:34.386020+0000","last_fullsized":"2024-09-10T10:05:34.386020+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:04.619410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202424+0000","last_change":"2024-09-10T10:05:32.210480+0000","last_active":"2024-09-10T10:05:33.202424+0000","last_peered":"2024-09-10T10:05:33.202424+0000","last_clean":"2024-09-10T10:05:33.202424+0000","last_became_active":"2024-09-10T10:05:32.210213+0000","last_became_peered":"2024-09-10T10:05:32.210213+0000","last_unstale":"2024-09-10T10:05:33.202424+0000","last_undegraded":"2024-09-10T10:05:33.202424+0000","last_fullsized":"2024-09-10T10:05:33.202424+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:23.197105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519049+0000","last_change":"2024-09-10T10:05:32.197240+0000","last_active":"2024-09-10T10:05:34.519049+0000","last_peered":"2024-09-10T10:05:34.519049+0000","last_clean":"2024-09-10T10:05:34.519049+0000","last_became_active":"2024-09-10T10:05:32.194745+0000","last_became_peered":"2024-09-10T10:05:32.194745+0000","last_unstale":"2024-09-10T10:05:34.519049+0000","last_undegraded":"2024-09-10T10:05:34.519049+0000","last_fullsized":"2024-09-10T10:05:34.519049+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:34.471388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369402+0000","last_change":"2024-09-10T10:05:32.199710+0000","last_active":"2024-09-10T10:05:35.369402+0000","last_peered":"2024-09-10T10:05:35.369402+0000","last_clean":"2024-09-10T10:05:35.369402+0000","last_became_active":"2024-09-10T10:05:32.199388+0000","last_became_peered":"2024-09-10T10:05:32.199388+0000","last_unstale":"2024-09-10T10:05:35.369402+0000","last_undegraded":"2024-09-10T10:05:35.369402+0000","last_fullsized":"2024-09-10T10:05:35.369402+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:08:55.894584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"152'4","reported_seq":18,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519428+0000","last_change":"2024-09-10T10:05:31.187728+0000","last_active":"2024-09-10T10:05:34.519428+0000","last_peered":"2024-09-10T10:05:34.519428+0000","last_clean":"2024-09-10T10:05:34.519428+0000","last_became_active":"2024-09-10T10:05:31.186044+0000","last_became_peered":"2024-09-10T10:05:31.186044+0000","last_unstale":"2024-09-10T10:05:34.519428+0000","last_undegraded":"2024-09-10T10:05:34.519428+0000","last_fullsized":"2024-09-10T10:05:34.519428+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:47.086239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195512+0000","last_change":"2024-09-10T10:05:32.200253+0000","last_active":"2024-09-10T10:05:33.195512+0000","last_peered":"2024-09-10T10:05:33.195512+0000","last_clean":"2024-09-10T10:05:33.195512+0000","last_became_active":"2024-09-10T10:05:32.199931+0000","last_became_peered":"2024-09-10T10:05:32.199931+0000","last_unstale":"2024-09-10T10:05:33.195512+0000","last_undegraded":"2024-09-10T10:05:33.195512+0000","last_fullsized":"2024-09-10T10:05:33.195512+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:13.682778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386389+0000","last_change":"2024-09-10T10:05:31.187161+0000","last_active":"2024-09-10T10:05:34.386389+0000","last_peered":"2024-09-10T10:05:34.386389+0000","last_clean":"2024-09-10T10:05:34.386389+0000","last_became_active":"2024-09-10T10:05:31.186727+0000","last_became_peered":"2024-09-10T10:05:31.186727+0000","last_unstale":"2024-09-10T10:05:34.386389+0000","last_undegraded":"2024-09-10T10:05:34.386389+0000","last_fullsized":"2024-09-10T10:05:34.386389+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:33.464770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198361+0000","last_change":"2024-09-10T10:05:32.202596+0000","last_active":"2024-09-10T10:05:33.198361+0000","last_peered":"2024-09-10T10:05:33.198361+0000","last_clean":"2024-09-10T10:05:33.198361+0000","last_became_active":"2024-09-10T10:05:32.200011+0000","last_became_peered":"2024-09-10T10:05:32.200011+0000","last_unstale":"2024-09-10T10:05:33.198361+0000","last_undegraded":"2024-09-10T10:05:33.198361+0000","last_fullsized":"2024-09-10T10:05:33.198361+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:36.178267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202643+0000","last_change":"2024-09-10T10:05:31.196039+0000","last_active":"2024-09-10T10:05:33.202643+0000","last_peered":"2024-09-10T10:05:33.202643+0000","last_clean":"2024-09-10T10:05:33.202643+0000","last_became_active":"2024-09-10T10:05:31.195762+0000","last_became_peered":"2024-09-10T10:05:31.195762+0000","last_unstale":"2024-09-10T10:05:33.202643+0000","last_undegraded":"2024-09-10T10:05:33.202643+0000","last_fullsized":"2024-09-10T10:05:33.202643+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:24.919933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386358+0000","last_change":"2024-09-10T10:05:32.198986+0000","last_active":"2024-09-10T10:05:34.386358+0000","last_peered":"2024-09-10T10:05:34.386358+0000","last_clean":"2024-09-10T10:05:34.386358+0000","last_became_active":"2024-09-10T10:05:32.198643+0000","last_became_peered":"2024-09-10T10:05:32.198643+0000","last_unstale":"2024-09-10T10:05:34.386358+0000","last_undegraded":"2024-09-10T10:05:34.386358+0000","last_fullsized":"2024-09-10T10:05:34.386358+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:25.926551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198711+0000","last_change":"2024-09-10T10:05:31.184956+0000","last_active":"2024-09-10T10:05:33.198711+0000","last_peered":"2024-09-10T10:05:33.198711+0000","last_clean":"2024-09-10T10:05:33.198711+0000","last_became_active":"2024-09-10T10:05:31.184628+0000","last_became_peered":"2024-09-10T10:05:31.184628+0000","last_unstale":"2024-09-10T10:05:33.198711+0000","last_undegraded":"2024-09-10T10:05:33.198711+0000","last_fullsized":"2024-09-10T10:05:33.198711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:48.990477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194844+0000","last_change":"2024-09-10T10:05:32.194160+0000","last_active":"2024-09-10T10:05:33.194844+0000","last_peered":"2024-09-10T10:05:33.194844+0000","last_clean":"2024-09-10T10:05:33.194844+0000","last_became_active":"2024-09-10T10:05:32.193487+0000","last_became_peered":"2024-09-10T10:05:32.193487+0000","last_unstale":"2024-09-10T10:05:33.194844+0000","last_undegraded":"2024-09-10T10:05:33.194844+0000","last_fullsized":"2024-09-10T10:05:33.194844+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:19.992141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519463+0000","last_change":"2024-09-10T10:05:31.189563+0000","last_active":"2024-09-10T10:05:34.519463+0000","last_peered":"2024-09-10T10:05:34.519463+0000","last_clean":"2024-09-10T10:05:34.519463+0000","last_became_active":"2024-09-10T10:05:31.189169+0000","last_became_peered":"2024-09-10T10:05:31.189169+0000","last_unstale":"2024-09-10T10:05:34.519463+0000","last_undegraded":"2024-09-10T10:05:34.519463+0000","last_fullsized":"2024-09-10T10:05:34.519463+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:00.085710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369330+0000","last_change":"2024-09-10T10:05:32.199253+0000","last_active":"2024-09-10T10:05:35.369330+0000","last_peered":"2024-09-10T10:05:35.369330+0000","last_clean":"2024-09-10T10:05:35.369330+0000","last_became_active":"2024-09-10T10:05:32.198913+0000","last_became_peered":"2024-09-10T10:05:32.198913+0000","last_unstale":"2024-09-10T10:05:35.369330+0000","last_undegraded":"2024-09-10T10:05:35.369330+0000","last_fullsized":"2024-09-10T10:05:35.369330+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:16.177921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369126+0000","last_change":"2024-09-10T10:05:31.184111+0000","last_active":"2024-09-10T10:05:35.369126+0000","last_peered":"2024-09-10T10:05:35.369126+0000","last_clean":"2024-09-10T10:05:35.369126+0000","last_became_active":"2024-09-10T10:05:31.183673+0000","last_became_peered":"2024-09-10T10:05:31.183673+0000","last_unstale":"2024-09-10T10:05:35.369126+0000","last_undegraded":"2024-09-10T10:05:35.369126+0000","last_fullsized":"2024-09-10T10:05:35.369126+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:44.121547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197821+0000","last_change":"2024-09-10T10:05:32.207266+0000","last_active":"2024-09-10T10:05:33.197821+0000","last_peered":"2024-09-10T10:05:33.197821+0000","last_clean":"2024-09-10T10:05:33.197821+0000","last_became_active":"2024-09-10T10:05:32.205319+0000","last_became_peered":"2024-09-10T10:05:32.205319+0000","last_unstale":"2024-09-10T10:05:33.197821+0000","last_undegraded":"2024-09-10T10:05:33.197821+0000","last_fullsized":"2024-09-10T10:05:33.197821+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:37.843191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370732+0000","last_change":"2024-09-10T10:05:31.193233+0000","last_active":"2024-09-10T10:05:35.370732+0000","last_peered":"2024-09-10T10:05:35.370732+0000","last_clean":"2024-09-10T10:05:35.370732+0000","last_became_active":"2024-09-10T10:05:31.192971+0000","last_became_peered":"2024-09-10T10:05:31.192971+0000","last_unstale":"2024-09-10T10:05:35.370732+0000","last_undegraded":"2024-09-10T10:05:35.370732+0000","last_fullsized":"2024-09-10T10:05:35.370732+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:06.426027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202832+0000","last_change":"2024-09-10T10:05:32.207268+0000","last_active":"2024-09-10T10:05:33.202832+0000","last_peered":"2024-09-10T10:05:33.202832+0000","last_clean":"2024-09-10T10:05:33.202832+0000","last_became_active":"2024-09-10T10:05:32.206685+0000","last_became_peered":"2024-09-10T10:05:32.206685+0000","last_unstale":"2024-09-10T10:05:33.202832+0000","last_undegraded":"2024-09-10T10:05:33.202832+0000","last_fullsized":"2024-09-10T10:05:33.202832+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:48.365603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197806+0000","last_change":"2024-09-10T10:05:31.185108+0000","last_active":"2024-09-10T10:05:33.197806+0000","last_peered":"2024-09-10T10:05:33.197806+0000","last_clean":"2024-09-10T10:05:33.197806+0000","last_became_active":"2024-09-10T10:05:31.184889+0000","last_became_peered":"2024-09-10T10:05:31.184889+0000","last_unstale":"2024-09-10T10:05:33.197806+0000","last_undegraded":"2024-09-10T10:05:33.197806+0000","last_fullsized":"2024-09-10T10:05:33.197806+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:56.692445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369405+0000","last_change":"2024-09-10T10:05:32.198420+0000","last_active":"2024-09-10T10:05:35.369405+0000","last_peered":"2024-09-10T10:05:35.369405+0000","last_clean":"2024-09-10T10:05:35.369405+0000","last_became_active":"2024-09-10T10:05:32.198099+0000","last_became_peered":"2024-09-10T10:05:32.198099+0000","last_unstale":"2024-09-10T10:05:35.369405+0000","last_undegraded":"2024-09-10T10:05:35.369405+0000","last_fullsized":"2024-09-10T10:05:35.369405+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:57.163059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519363+0000","last_change":"2024-09-10T10:05:31.190056+0000","last_active":"2024-09-10T10:05:34.519363+0000","last_peered":"2024-09-10T10:05:34.519363+0000","last_clean":"2024-09-10T10:05:34.519363+0000","last_became_active":"2024-09-10T10:05:31.189791+0000","last_became_peered":"2024-09-10T10:05:31.189791+0000","last_unstale":"2024-09-10T10:05:34.519363+0000","last_undegraded":"2024-09-10T10:05:34.519363+0000","last_fullsized":"2024-09-10T10:05:34.519363+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:46.207558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198377+0000","last_change":"2024-09-10T10:05:32.206830+0000","last_active":"2024-09-10T10:05:33.198377+0000","last_peered":"2024-09-10T10:05:33.198377+0000","last_clean":"2024-09-10T10:05:33.198377+0000","last_became_active":"2024-09-10T10:05:32.206552+0000","last_became_peered":"2024-09-10T10:05:32.206552+0000","last_unstale":"2024-09-10T10:05:33.198377+0000","last_undegraded":"2024-09-10T10:05:33.198377+0000","last_fullsized":"2024-09-10T10:05:33.198377+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:51.678475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"152'3","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519202+0000","last_change":"2024-09-10T10:05:31.188150+0000","last_active":"2024-09-10T10:05:34.519202+0000","last_peered":"2024-09-10T10:05:34.519202+0000","last_clean":"2024-09-10T10:05:34.519202+0000","last_became_active":"2024-09-10T10:05:31.185519+0000","last_became_peered":"2024-09-10T10:05:31.185519+0000","last_unstale":"2024-09-10T10:05:34.519202+0000","last_undegraded":"2024-09-10T10:05:34.519202+0000","last_fullsized":"2024-09-10T10:05:34.519202+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:59.166885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195098+0000","last_change":"2024-09-10T10:05:32.195869+0000","last_active":"2024-09-10T10:05:33.195098+0000","last_peered":"2024-09-10T10:05:33.195098+0000","last_clean":"2024-09-10T10:05:33.195098+0000","last_became_active":"2024-09-10T10:05:32.195597+0000","last_became_peered":"2024-09-10T10:05:32.195597+0000","last_unstale":"2024-09-10T10:05:33.195098+0000","last_undegraded":"2024-09-10T10:05:33.195098+0000","last_fullsized":"2024-09-10T10:05:33.195098+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:32.283156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"152'6","reported_seq":20,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202554+0000","last_change":"2024-09-10T10:05:31.194843+0000","last_active":"2024-09-10T10:05:33.202554+0000","last_peered":"2024-09-10T10:05:33.202554+0000","last_clean":"2024-09-10T10:05:33.202554+0000","last_became_active":"2024-09-10T10:05:31.194168+0000","last_became_peered":"2024-09-10T10:05:31.194168+0000","last_unstale":"2024-09-10T10:05:33.202554+0000","last_undegraded":"2024-09-10T10:05:33.202554+0000","last_fullsized":"2024-09-10T10:05:33.202554+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:57.982434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202381+0000","last_change":"2024-09-10T10:05:32.207407+0000","last_active":"2024-09-10T10:05:33.202381+0000","last_peered":"2024-09-10T10:05:33.202381+0000","last_clean":"2024-09-10T10:05:33.202381+0000","last_became_active":"2024-09-10T10:05:32.206481+0000","last_became_peered":"2024-09-10T10:05:32.206481+0000","last_unstale":"2024-09-10T10:05:33.202381+0000","last_undegraded":"2024-09-10T10:05:33.202381+0000","last_fullsized":"2024-09-10T10:05:33.202381+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:19.513722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371351+0000","last_change":"2024-09-10T10:05:31.189711+0000","last_active":"2024-09-10T10:05:35.371351+0000","last_peered":"2024-09-10T10:05:35.371351+0000","last_clean":"2024-09-10T10:05:35.371351+0000","last_became_active":"2024-09-10T10:05:31.189471+0000","last_became_peered":"2024-09-10T10:05:31.189471+0000","last_unstale":"2024-09-10T10:05:35.371351+0000","last_undegraded":"2024-09-10T10:05:35.371351+0000","last_fullsized":"2024-09-10T10:05:35.371351+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:13.110723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198087+0000","last_change":"2024-09-10T10:05:32.192330+0000","last_active":"2024-09-10T10:05:33.198087+0000","last_peered":"2024-09-10T10:05:33.198087+0000","last_clean":"2024-09-10T10:05:33.198087+0000","last_became_active":"2024-09-10T10:05:32.192064+0000","last_became_peered":"2024-09-10T10:05:32.192064+0000","last_unstale":"2024-09-10T10:05:33.198087+0000","last_undegraded":"2024-09-10T10:05:33.198087+0000","last_fullsized":"2024-09-10T10:05:33.198087+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:48.365603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194922+0000","last_change":"2024-09-10T10:05:31.195254+0000","last_active":"2024-09-10T10:05:33.194922+0000","last_peered":"2024-09-10T10:05:33.194922+0000","last_clean":"2024-09-10T10:05:33.194922+0000","last_became_active":"2024-09-10T10:05:31.194909+0000","last_became_peered":"2024-09-10T10:05:31.194909+0000","last_unstale":"2024-09-10T10:05:33.194922+0000","last_undegraded":"2024-09-10T10:05:33.194922+0000","last_fullsized":"2024-09-10T10:05:33.194922+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:34.530341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.385981+0000","last_change":"2024-09-10T10:05:32.198620+0000","last_active":"2024-09-10T10:05:34.385981+0000","last_peered":"2024-09-10T10:05:34.385981+0000","last_clean":"2024-09-10T10:05:34.385981+0000","last_became_active":"2024-09-10T10:05:32.197165+0000","last_became_peered":"2024-09-10T10:05:32.197165+0000","last_unstale":"2024-09-10T10:05:34.385981+0000","last_undegraded":"2024-09-10T10:05:34.385981+0000","last_fullsized":"2024-09-10T10:05:34.385981+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:09.383770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.368912+0000","last_change":"2024-09-10T10:05:31.176701+0000","last_active":"2024-09-10T10:05:35.368912+0000","last_peered":"2024-09-10T10:05:35.368912+0000","last_clean":"2024-09-10T10:05:35.368912+0000","last_became_active":"2024-09-10T10:05:31.176425+0000","last_became_peered":"2024-09-10T10:05:31.176425+0000","last_unstale":"2024-09-10T10:05:35.368912+0000","last_undegraded":"2024-09-10T10:05:35.368912+0000","last_fullsized":"2024-09-10T10:05:35.368912+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:37.745872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194579+0000","last_change":"2024-09-10T10:05:32.190495+0000","last_active":"2024-09-10T10:05:33.194579+0000","last_peered":"2024-09-10T10:05:33.194579+0000","last_clean":"2024-09-10T10:05:33.194579+0000","last_became_active":"2024-09-10T10:05:32.189347+0000","last_became_peered":"2024-09-10T10:05:32.189347+0000","last_unstale":"2024-09-10T10:05:33.194579+0000","last_undegraded":"2024-09-10T10:05:33.194579+0000","last_fullsized":"2024-09-10T10:05:33.194579+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:07.432644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"152'8","reported_seq":22,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370869+0000","last_change":"2024-09-10T10:05:31.193087+0000","last_active":"2024-09-10T10:05:35.370869+0000","last_peered":"2024-09-10T10:05:35.370869+0000","last_clean":"2024-09-10T10:05:35.370869+0000","last_became_active":"2024-09-10T10:05:31.192788+0000","last_became_peered":"2024-09-10T10:05:31.192788+0000","last_unstale":"2024-09-10T10:05:35.370869+0000","last_undegraded":"2024-09-10T10:05:35.370869+0000","last_fullsized":"2024-09-10T10:05:35.370869+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:33.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195323+0000","last_change":"2024-09-10T10:05:32.187973+0000","last_active":"2024-09-10T10:05:33.195323+0000","last_peered":"2024-09-10T10:05:33.195323+0000","last_clean":"2024-09-10T10:05:33.195323+0000","last_became_active":"2024-09-10T10:05:32.187672+0000","last_became_peered":"2024-09-10T10:05:32.187672+0000","last_unstale":"2024-09-10T10:05:33.195323+0000","last_undegraded":"2024-09-10T10:05:33.195323+0000","last_fullsized":"2024-09-10T10:05:33.195323+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:11.967689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519472+0000","last_change":"2024-09-10T10:05:31.189469+0000","last_active":"2024-09-10T10:05:34.519472+0000","last_peered":"2024-09-10T10:05:34.519472+0000","last_clean":"2024-09-10T10:05:34.519472+0000","last_became_active":"2024-09-10T10:05:31.189175+0000","last_became_peered":"2024-09-10T10:05:31.189175+0000","last_unstale":"2024-09-10T10:05:34.519472+0000","last_undegraded":"2024-09-10T10:05:34.519472+0000","last_fullsized":"2024-09-10T10:05:34.519472+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:51.865168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386581+0000","last_change":"2024-09-10T10:05:32.205086+0000","last_active":"2024-09-10T10:05:34.386581+0000","last_peered":"2024-09-10T10:05:34.386581+0000","last_clean":"2024-09-10T10:05:34.386581+0000","last_became_active":"2024-09-10T10:05:32.204687+0000","last_became_peered":"2024-09-10T10:05:32.204687+0000","last_unstale":"2024-09-10T10:05:34.386581+0000","last_undegraded":"2024-09-10T10:05:34.386581+0000","last_fullsized":"2024-09-10T10:05:34.386581+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:57.699063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369353+0000","last_change":"2024-09-10T10:05:31.176322+0000","last_active":"2024-09-10T10:05:35.369353+0000","last_peered":"2024-09-10T10:05:35.369353+0000","last_clean":"2024-09-10T10:05:35.369353+0000","last_became_active":"2024-09-10T10:05:31.175852+0000","last_became_peered":"2024-09-10T10:05:31.175852+0000","last_unstale":"2024-09-10T10:05:35.369353+0000","last_undegraded":"2024-09-10T10:05:35.369353+0000","last_fullsized":"2024-09-10T10:05:35.369353+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:18.556500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386471+0000","last_change":"2024-09-10T10:05:32.210506+0000","last_active":"2024-09-10T10:05:34.386471+0000","last_peered":"2024-09-10T10:05:34.386471+0000","last_clean":"2024-09-10T10:05:34.386471+0000","last_became_active":"2024-09-10T10:05:32.210029+0000","last_became_peered":"2024-09-10T10:05:32.210029+0000","last_unstale":"2024-09-10T10:05:34.386471+0000","last_undegraded":"2024-09-10T10:05:34.386471+0000","last_fullsized":"2024-09-10T10:05:34.386471+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:51.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371111+0000","last_change":"2024-09-10T10:05:31.193098+0000","last_active":"2024-09-10T10:05:35.371111+0000","last_peered":"2024-09-10T10:05:35.371111+0000","last_clean":"2024-09-10T10:05:35.371111+0000","last_became_active":"2024-09-10T10:05:31.192779+0000","last_became_peered":"2024-09-10T10:05:31.192779+0000","last_unstale":"2024-09-10T10:05:35.371111+0000","last_undegraded":"2024-09-10T10:05:35.371111+0000","last_fullsized":"2024-09-10T10:05:35.371111+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:50.441112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198508+0000","last_change":"2024-09-10T10:05:32.200610+0000","last_active":"2024-09-10T10:05:33.198508+0000","last_peered":"2024-09-10T10:05:33.198508+0000","last_clean":"2024-09-10T10:05:33.198508+0000","last_became_active":"2024-09-10T10:05:32.198412+0000","last_became_peered":"2024-09-10T10:05:32.198412+0000","last_unstale":"2024-09-10T10:05:33.198508+0000","last_undegraded":"2024-09-10T10:05:33.198508+0000","last_fullsized":"2024-09-10T10:05:33.198508+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:25.293334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198525+0000","last_change":"2024-09-10T10:05:31.193283+0000","last_active":"2024-09-10T10:05:33.198525+0000","last_peered":"2024-09-10T10:05:33.198525+0000","last_clean":"2024-09-10T10:05:33.198525+0000","last_became_active":"2024-09-10T10:05:31.192470+0000","last_became_peered":"2024-09-10T10:05:31.192470+0000","last_unstale":"2024-09-10T10:05:33.198525+0000","last_undegraded":"2024-09-10T10:05:33.198525+0000","last_fullsized":"2024-09-10T10:05:33.198525+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:26.997421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194954+0000","last_change":"2024-09-10T10:05:32.198896+0000","last_active":"2024-09-10T10:05:33.194954+0000","last_peered":"2024-09-10T10:05:33.194954+0000","last_clean":"2024-09-10T10:05:33.194954+0000","last_became_active":"2024-09-10T10:05:32.198582+0000","last_became_peered":"2024-09-10T10:05:32.198582+0000","last_unstale":"2024-09-10T10:05:33.194954+0000","last_undegraded":"2024-09-10T10:05:33.194954+0000","last_fullsized":"2024-09-10T10:05:33.194954+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:51.447729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202606+0000","last_change":"2024-09-10T10:05:31.194554+0000","last_active":"2024-09-10T10:05:33.202606+0000","last_peered":"2024-09-10T10:05:33.202606+0000","last_clean":"2024-09-10T10:05:33.202606+0000","last_became_active":"2024-09-10T10:05:31.194265+0000","last_became_peered":"2024-09-10T10:05:31.194265+0000","last_unstale":"2024-09-10T10:05:33.202606+0000","last_undegraded":"2024-09-10T10:05:33.202606+0000","last_fullsized":"2024-09-10T10:05:33.202606+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:03.612793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198134+0000","last_change":"2024-09-10T10:05:32.195045+0000","last_active":"2024-09-10T10:05:33.198134+0000","last_peered":"2024-09-10T10:05:33.198134+0000","last_clean":"2024-09-10T10:05:33.198134+0000","last_became_active":"2024-09-10T10:05:32.194735+0000","last_became_peered":"2024-09-10T10:05:32.194735+0000","last_unstale":"2024-09-10T10:05:33.198134+0000","last_undegraded":"2024-09-10T10:05:33.198134+0000","last_fullsized":"2024-09-10T10:05:33.198134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:43.449510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519177+0000","last_change":"2024-09-10T10:05:31.188898+0000","last_active":"2024-09-10T10:05:34.519177+0000","last_peered":"2024-09-10T10:05:34.519177+0000","last_clean":"2024-09-10T10:05:34.519177+0000","last_became_active":"2024-09-10T10:05:31.186075+0000","last_became_peered":"2024-09-10T10:05:31.186075+0000","last_unstale":"2024-09-10T10:05:34.519177+0000","last_undegraded":"2024-09-10T10:05:34.519177+0000","last_fullsized":"2024-09-10T10:05:34.519177+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:01.924390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197595+0000","last_change":"2024-09-10T10:05:32.198142+0000","last_active":"2024-09-10T10:05:33.197595+0000","last_peered":"2024-09-10T10:05:33.197595+0000","last_clean":"2024-09-10T10:05:33.197595+0000","last_became_active":"2024-09-10T10:05:32.197887+0000","last_became_peered":"2024-09-10T10:05:32.197887+0000","last_unstale":"2024-09-10T10:05:33.197595+0000","last_undegraded":"2024-09-10T10:05:33.197595+0000","last_fullsized":"2024-09-10T10:05:33.197595+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:12.558158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202472+0000","last_change":"2024-09-10T10:05:31.187311+0000","last_active":"2024-09-10T10:05:33.202472+0000","last_peered":"2024-09-10T10:05:33.202472+0000","last_clean":"2024-09-10T10:05:33.202472+0000","last_became_active":"2024-09-10T10:05:31.186827+0000","last_became_peered":"2024-09-10T10:05:31.186827+0000","last_unstale":"2024-09-10T10:05:33.202472+0000","last_undegraded":"2024-09-10T10:05:33.202472+0000","last_fullsized":"2024-09-10T10:05:33.202472+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:50.910428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519067+0000","last_change":"2024-09-10T10:05:32.209255+0000","last_active":"2024-09-10T10:05:34.519067+0000","last_peered":"2024-09-10T10:05:34.519067+0000","last_clean":"2024-09-10T10:05:34.519067+0000","last_became_active":"2024-09-10T10:05:32.209021+0000","last_became_peered":"2024-09-10T10:05:32.209021+0000","last_unstale":"2024-09-10T10:05:34.519067+0000","last_undegraded":"2024-09-10T10:05:34.519067+0000","last_fullsized":"2024-09-10T10:05:34.519067+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:00.267164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386022+0000","last_change":"2024-09-10T10:05:31.187284+0000","last_active":"2024-09-10T10:05:34.386022+0000","last_peered":"2024-09-10T10:05:34.386022+0000","last_clean":"2024-09-10T10:05:34.386022+0000","last_became_active":"2024-09-10T10:05:31.186766+0000","last_became_peered":"2024-09-10T10:05:31.186766+0000","last_unstale":"2024-09-10T10:05:34.386022+0000","last_undegraded":"2024-09-10T10:05:34.386022+0000","last_fullsized":"2024-09-10T10:05:34.386022+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:18.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519346+0000","last_change":"2024-09-10T10:05:32.210555+0000","last_active":"2024-09-10T10:05:34.519346+0000","last_peered":"2024-09-10T10:05:34.519346+0000","last_clean":"2024-09-10T10:05:34.519346+0000","last_became_active":"2024-09-10T10:05:32.210243+0000","last_became_peered":"2024-09-10T10:05:32.210243+0000","last_unstale":"2024-09-10T10:05:34.519346+0000","last_undegraded":"2024-09-10T10:05:34.519346+0000","last_fullsized":"2024-09-10T10:05:34.519346+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:54.156101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371182+0000","last_change":"2024-09-10T10:05:31.188601+0000","last_active":"2024-09-10T10:05:35.371182+0000","last_peered":"2024-09-10T10:05:35.371182+0000","last_clean":"2024-09-10T10:05:35.371182+0000","last_became_active":"2024-09-10T10:05:31.188289+0000","last_became_peered":"2024-09-10T10:05:31.188289+0000","last_unstale":"2024-09-10T10:05:35.371182+0000","last_undegraded":"2024-09-10T10:05:35.371182+0000","last_fullsized":"2024-09-10T10:05:35.371182+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:31.276538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195571+0000","last_change":"2024-09-10T10:05:32.197359+0000","last_active":"2024-09-10T10:05:33.195571+0000","last_peered":"2024-09-10T10:05:33.195571+0000","last_clean":"2024-09-10T10:05:33.195571+0000","last_became_active":"2024-09-10T10:05:32.197045+0000","last_became_peered":"2024-09-10T10:05:32.197045+0000","last_unstale":"2024-09-10T10:05:33.195571+0000","last_undegraded":"2024-09-10T10:05:33.195571+0000","last_fullsized":"2024-09-10T10:05:33.195571+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:19.617660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"153'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195419+0000","last_change":"2024-09-10T10:05:31.196820+0000","last_active":"2024-09-10T10:05:33.195419+0000","last_peered":"2024-09-10T10:05:33.195419+0000","last_clean":"2024-09-10T10:05:33.195419+0000","last_became_active":"2024-09-10T10:05:31.196557+0000","last_became_peered":"2024-09-10T10:05:31.196557+0000","last_unstale":"2024-09-10T10:05:33.195419+0000","last_undegraded":"2024-09-10T10:05:33.195419+0000","last_fullsized":"2024-09-10T10:05:33.195419+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:28.197728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369302+0000","last_change":"2024-09-10T10:05:32.215523+0000","last_active":"2024-09-10T10:05:35.369302+0000","last_peered":"2024-09-10T10:05:35.369302+0000","last_clean":"2024-09-10T10:05:35.369302+0000","last_became_active":"2024-09-10T10:05:32.215239+0000","last_became_peered":"2024-09-10T10:05:32.215239+0000","last_unstale":"2024-09-10T10:05:35.369302+0000","last_undegraded":"2024-09-10T10:05:35.369302+0000","last_fullsized":"2024-09-10T10:05:35.369302+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:22.289745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195118+0000","last_change":"2024-09-10T10:05:31.197033+0000","last_active":"2024-09-10T10:05:33.195118+0000","last_peered":"2024-09-10T10:05:33.195118+0000","last_clean":"2024-09-10T10:05:33.195118+0000","last_became_active":"2024-09-10T10:05:31.196739+0000","last_became_peered":"2024-09-10T10:05:31.196739+0000","last_unstale":"2024-09-10T10:05:33.195118+0000","last_undegraded":"2024-09-10T10:05:33.195118+0000","last_fullsized":"2024-09-10T10:05:33.195118+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:02.511972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369913+0000","last_change":"2024-09-10T10:05:32.199610+0000","last_active":"2024-09-10T10:05:35.369913+0000","last_peered":"2024-09-10T10:05:35.369913+0000","last_clean":"2024-09-10T10:05:35.369913+0000","last_became_active":"2024-09-10T10:05:32.199339+0000","last_became_peered":"2024-09-10T10:05:32.199339+0000","last_unstale":"2024-09-10T10:05:35.369913+0000","last_undegraded":"2024-09-10T10:05:35.369913+0000","last_fullsized":"2024-09-10T10:05:35.369913+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:31.653256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369395+0000","last_change":"2024-09-10T10:05:31.179485+0000","last_active":"2024-09-10T10:05:35.369395+0000","last_peered":"2024-09-10T10:05:35.369395+0000","last_clean":"2024-09-10T10:05:35.369395+0000","last_became_active":"2024-09-10T10:05:31.178050+0000","last_became_peered":"2024-09-10T10:05:31.178050+0000","last_unstale":"2024-09-10T10:05:35.369395+0000","last_undegraded":"2024-09-10T10:05:35.369395+0000","last_fullsized":"2024-09-10T10:05:35.369395+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:54.607588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370973+0000","last_change":"2024-09-10T10:05:32.203836+0000","last_active":"2024-09-10T10:05:35.370973+0000","last_peered":"2024-09-10T10:05:35.370973+0000","last_clean":"2024-09-10T10:05:35.370973+0000","last_became_active":"2024-09-10T10:05:32.203166+0000","last_became_peered":"2024-09-10T10:05:32.203166+0000","last_unstale":"2024-09-10T10:05:35.370973+0000","last_undegraded":"2024-09-10T10:05:35.370973+0000","last_fullsized":"2024-09-10T10:05:35.370973+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:43.104902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386182+0000","last_change":"2024-09-10T10:05:31.193385+0000","last_active":"2024-09-10T10:05:34.386182+0000","last_peered":"2024-09-10T10:05:34.386182+0000","last_clean":"2024-09-10T10:05:34.386182+0000","last_became_active":"2024-09-10T10:05:31.193085+0000","last_became_peered":"2024-09-10T10:05:31.193085+0000","last_unstale":"2024-09-10T10:05:34.386182+0000","last_undegraded":"2024-09-10T10:05:34.386182+0000","last_fullsized":"2024-09-10T10:05:34.386182+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:10.299735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371137+0000","last_change":"2024-09-10T10:05:32.211776+0000","last_active":"2024-09-10T10:05:35.371137+0000","last_peered":"2024-09-10T10:05:35.371137+0000","last_clean":"2024-09-10T10:05:35.371137+0000","last_became_active":"2024-09-10T10:05:32.210887+0000","last_became_peered":"2024-09-10T10:05:32.210887+0000","last_unstale":"2024-09-10T10:05:35.371137+0000","last_undegraded":"2024-09-10T10:05:35.371137+0000","last_fullsized":"2024-09-10T10:05:35.371137+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:47.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386316+0000","last_change":"2024-09-10T10:05:31.186964+0000","last_active":"2024-09-10T10:05:34.386316+0000","last_peered":"2024-09-10T10:05:34.386316+0000","last_clean":"2024-09-10T10:05:34.386316+0000","last_became_active":"2024-09-10T10:05:31.186613+0000","last_became_peered":"2024-09-10T10:05:31.186613+0000","last_unstale":"2024-09-10T10:05:34.386316+0000","last_undegraded":"2024-09-10T10:05:34.386316+0000","last_fullsized":"2024-09-10T10:05:34.386316+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:28.486954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197580+0000","last_change":"2024-09-10T10:05:32.216197+0000","last_active":"2024-09-10T10:05:33.197580+0000","last_peered":"2024-09-10T10:05:33.197580+0000","last_clean":"2024-09-10T10:05:33.197580+0000","last_became_active":"2024-09-10T10:05:32.215922+0000","last_became_peered":"2024-09-10T10:05:32.215922+0000","last_unstale":"2024-09-10T10:05:33.197580+0000","last_undegraded":"2024-09-10T10:05:33.197580+0000","last_fullsized":"2024-09-10T10:05:33.197580+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:51.678475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198145+0000","last_change":"2024-09-10T10:05:31.187417+0000","last_active":"2024-09-10T10:05:33.198145+0000","last_peered":"2024-09-10T10:05:33.198145+0000","last_clean":"2024-09-10T10:05:33.198145+0000","last_became_active":"2024-09-10T10:05:31.187030+0000","last_became_peered":"2024-09-10T10:05:31.187030+0000","last_unstale":"2024-09-10T10:05:33.198145+0000","last_undegraded":"2024-09-10T10:05:33.198145+0000","last_fullsized":"2024-09-10T10:05:33.198145+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:48.710671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519141+0000","last_change":"2024-09-10T10:05:32.193628+0000","last_active":"2024-09-10T10:05:34.519141+0000","last_peered":"2024-09-10T10:05:34.519141+0000","last_clean":"2024-09-10T10:05:34.519141+0000","last_became_active":"2024-09-10T10:05:32.193327+0000","last_became_peered":"2024-09-10T10:05:32.193327+0000","last_unstale":"2024-09-10T10:05:34.519141+0000","last_undegraded":"2024-09-10T10:05:34.519141+0000","last_fullsized":"2024-09-10T10:05:34.519141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.619372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202360+0000","last_change":"2024-09-10T10:05:31.194388+0000","last_active":"2024-09-10T10:05:33.202360+0000","last_peered":"2024-09-10T10:05:33.202360+0000","last_clean":"2024-09-10T10:05:33.202360+0000","last_became_active":"2024-09-10T10:05:31.193967+0000","last_became_peered":"2024-09-10T10:05:31.193967+0000","last_unstale":"2024-09-10T10:05:33.202360+0000","last_undegraded":"2024-09-10T10:05:33.202360+0000","last_fullsized":"2024-09-10T10:05:33.202360+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:34.854768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194489+0000","last_change":"2024-09-10T10:05:32.186936+0000","last_active":"2024-09-10T10:05:33.194489+0000","last_peered":"2024-09-10T10:05:33.194489+0000","last_clean":"2024-09-10T10:05:33.194489+0000","last_became_active":"2024-09-10T10:05:32.186582+0000","last_became_peered":"2024-09-10T10:05:32.186582+0000","last_unstale":"2024-09-10T10:05:33.194489+0000","last_undegraded":"2024-09-10T10:05:33.194489+0000","last_fullsized":"2024-09-10T10:05:33.194489+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:47.849004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370868+0000","last_change":"2024-09-10T10:05:31.191205+0000","last_active":"2024-09-10T10:05:35.370868+0000","last_peered":"2024-09-10T10:05:35.370868+0000","last_clean":"2024-09-10T10:05:35.370868+0000","last_became_active":"2024-09-10T10:05:31.190885+0000","last_became_peered":"2024-09-10T10:05:31.190885+0000","last_unstale":"2024-09-10T10:05:35.370868+0000","last_undegraded":"2024-09-10T10:05:35.370868+0000","last_fullsized":"2024-09-10T10:05:35.370868+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:18.985524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":448,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370902+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:05:35.370902+0000","last_peered":"2024-09-10T10:05:35.370902+0000","last_clean":"2024-09-10T10:05:35.370902+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:05:35.370902+0000","last_undegraded":"2024-09-10T10:05:35.370902+0000","last_fullsized":"2024-09-10T10:05:35.370902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2523136,"data_stored":1438395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1921024,"data_stored":847923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1921024,"data_stored":847923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":850117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542237,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1933312,"data_stored":848117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1933312,"data_stored":848617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1933312,"data_stored":849535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509532,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37812,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334028,"statfs":{"total":96636764160,"available":96598044672,"internally_reserved":0,"allocated":2519040,"data_stored":1438313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:05:40.793 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:05:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3750400640 1 Processor -- start 2024-09-10T10:05:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3750400640 1 -- start start 2024-09-10T10:05:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3750400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f37480b4bd0 0x7f374806e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3750400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374806d770 con 0x7f37480721c0 2024-09-10T10:05:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3750400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374806da50 con 0x7f3748163a60 2024-09-10T10:05:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3750400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f374806dd30 con 0x7f37480b4bd0 2024-09-10T10:05:40.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f374e400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f37480721c0 0x7f374815fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37312/0 (socket says 172.21.15.110:37312) 2024-09-10T10:05:40.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f374e400640 1 -- 172.21.15.110:0/481502094 learned_addr learned my addr 172.21.15.110:0/481502094 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:40.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f374da00640 1 --2- 172.21.15.110:0/481502094 >> v2:172.21.15.145:3300/0 conn(0x7f37480b4bd0 0x7f374806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:40.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3747e00640 1 -- 172.21.15.110:0/481502094 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404681934 0 0) 0x7f374806d770 con 0x7f37480721c0 2024-09-10T10:05:40.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3747e00640 1 -- 172.21.15.110:0/481502094 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3730003530 con 0x7f37480721c0 2024-09-10T10:05:40.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3747e00640 1 -- 172.21.15.110:0/481502094 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591757699 0 0) 0x7f374806da50 con 0x7f3748163a60 2024-09-10T10:05:40.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f3747e00640 1 -- 172.21.15.110:0/481502094 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f374806d770 con 0x7f3748163a60 2024-09-10T10:05:40.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f374da00640 1 -- 172.21.15.110:0/481502094 >> v1:172.21.15.145:6789/0 conn(0x7f3748163a60 legacy=0x7f3748071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:40.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f374da00640 1 -- 172.21.15.110:0/481502094 >> v1:172.21.15.110:6789/0 conn(0x7f37480721c0 legacy=0x7f374815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:40.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.941+0000 7f374da00640 1 -- 172.21.15.110:0/481502094 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374806e010 con 0x7f37480b4bd0 2024-09-10T10:05:40.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f374da00640 1 --2- 172.21.15.110:0/481502094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37480b4bd0 0x7f374806e810 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f373c0097c0 tx=0x7f373c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4d333adb8827581 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:40.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3747e00640 1 -- 172.21.15.110:0/481502094 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0aa5f0 con 0x7f37480b4bd0 2024-09-10T10:05:40.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3747e00640 1 -- 172.21.15.110:0/481502094 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373c0af070 con 0x7f37480b4bd0 2024-09-10T10:05:40.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3747e00640 1 -- 172.21.15.110:0/481502094 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0aa9b0 con 0x7f37480b4bd0 2024-09-10T10:05:40.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3750400640 1 -- 172.21.15.110:0/481502094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37480b4bd0 msgr2=0x7f374806e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:40.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3750400640 1 --2- 172.21.15.110:0/481502094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37480b4bd0 0x7f374806e810 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f373c0097c0 tx=0x7f373c01c470 comp rx=0 tx=0).stop 2024-09-10T10:05:40.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3750400640 1 -- 172.21.15.110:0/481502094 shutdown_connections 2024-09-10T10:05:40.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3750400640 1 --2- 172.21.15.110:0/481502094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37480b4bd0 0x7f374806e810 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:40.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3750400640 1 -- 172.21.15.110:0/481502094 >> 172.21.15.110:0/481502094 conn(0x7f374806bc50 msgr2=0x7f374806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3750400640 1 -- 172.21.15.110:0/481502094 shutdown_connections 2024-09-10T10:05:40.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.945+0000 7f3750400640 1 -- 172.21.15.110:0/481502094 wait complete. 2024-09-10T10:05:40.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 Processor -- start 2024-09-10T10:05:40.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 -- start start 2024-09-10T10:05:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37480721c0 0x7f374814bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37480b4bd0 0x7f374814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3748163a60 0x7f37481ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:40.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37480b5130 con 0x7f37480b4bd0 2024-09-10T10:05:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f37480b4fb0 con 0x7f3748163a60 2024-09-10T10:05:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3748163e70 con 0x7f37480721c0 2024-09-10T10:05:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37480b4bd0 0x7f374814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37480b4bd0 0x7f374814c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39756/0 (socket says 172.21.15.110:39756) 2024-09-10T10:05:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 -- 172.21.15.110:0/1642220342 learned_addr learned my addr 172.21.15.110:0/1642220342 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374da00640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37480721c0 0x7f374814bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:40.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374e400640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3748163a60 0x7f37481ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 -- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37480721c0 msgr2=0x7f374814bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37480721c0 0x7f374814bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 -- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3748163a60 msgr2=0x7f37481ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:40.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3748163a60 0x7f37481ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 -- 172.21.15.110:0/1642220342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374815e480 con 0x7f37480b4bd0 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f374d000640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37480b4bd0 0x7f374814c1a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f373400c460 tx=0x7f373400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3745600640 1 -- 172.21.15.110:0/1642220342 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734017070 con 0x7f37480b4bd0 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3745600640 1 -- 172.21.15.110:0/1642220342 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37340052f0 con 0x7f37480b4bd0 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3745600640 1 -- 172.21.15.110:0/1642220342 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734003840 con 0x7f37480b4bd0 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374815e760 con 0x7f37480b4bd0 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374815ed20 con 0x7f37480b4bd0 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.949+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3748161a40 con 0x7f37480b4bd0 2024-09-10T10:05:40.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.953+0000 7f3745600640 1 -- 172.21.15.110:0/1642220342 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f373401a070 con 0x7f37480b4bd0 2024-09-10T10:05:40.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.953+0000 7f3745600640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37280680a0 0x7f372806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:40.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.953+0000 7f3745600640 1 -- 172.21.15.110:0/1642220342 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f37340861f0 con 0x7f37480b4bd0 2024-09-10T10:05:40.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.953+0000 7f374da00640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37280680a0 0x7f372806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:40.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.953+0000 7f374da00640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37280680a0 0x7f372806a560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f373c0ae000 tx=0x7f373c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:40.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:40.957+0000 7f3745600640 1 -- 172.21.15.110:0/1642220342 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3734055710 con 0x7f37480b4bd0 2024-09-10T10:05:41.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.113+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f374815ef20 con 0x7f37280680a0 2024-09-10T10:05:41.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3745600640 1 -- 172.21.15.110:0/1642220342 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f374815ef20 con 0x7f37280680a0 2024-09-10T10:05:41.241 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:41.242 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37280680a0 msgr2=0x7f372806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37280680a0 0x7f372806a560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f373c0ae000 tx=0x7f373c0a7000 comp rx=0 tx=0).stop 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37480b4bd0 msgr2=0x7f374814c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37480b4bd0 0x7f374814c1a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f373400c460 tx=0x7f373400c920 comp rx=0 tx=0).stop 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 shutdown_connections 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37280680a0 0x7f372806a560 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3748163a60 0x7f37481ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37480b4bd0 0x7f374814c1a0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 --2- 172.21.15.110:0/1642220342 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37480721c0 0x7f374814bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.121+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 >> 172.21.15.110:0/1642220342 conn(0x7f374806bc50 msgr2=0x7f3748071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.125+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 shutdown_connections 2024-09-10T10:05:41.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.125+0000 7f3750400640 1 -- 172.21.15.110:0/1642220342 wait complete. 2024-09-10T10:05:41.245 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-10T10:05:39.741339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282388,"kb_used_data":16232,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692332,"statfs":{"total":773094113280,"available":772804947968,"internally_reserved":0,"allocated":16621568,"data_stored":7968940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002434"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371175+0000","last_change":"2024-09-10T10:05:32.206947+0000","last_active":"2024-09-10T10:05:35.371175+0000","last_peered":"2024-09-10T10:05:35.371175+0000","last_clean":"2024-09-10T10:05:35.371175+0000","last_became_active":"2024-09-10T10:05:32.206692+0000","last_became_peered":"2024-09-10T10:05:32.206692+0000","last_unstale":"2024-09-10T10:05:35.371175+0000","last_undegraded":"2024-09-10T10:05:35.371175+0000","last_fullsized":"2024-09-10T10:05:35.371175+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:37.239701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198862+0000","last_change":"2024-09-10T10:05:32.195002+0000","last_active":"2024-09-10T10:05:33.198862+0000","last_peered":"2024-09-10T10:05:33.198862+0000","last_clean":"2024-09-10T10:05:33.198862+0000","last_became_active":"2024-09-10T10:05:32.194741+0000","last_became_peered":"2024-09-10T10:05:32.194741+0000","last_unstale":"2024-09-10T10:05:33.198862+0000","last_undegraded":"2024-09-10T10:05:33.198862+0000","last_fullsized":"2024-09-10T10:05:33.198862+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:24.501475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202551+0000","last_change":"2024-09-10T10:05:32.208004+0000","last_active":"2024-09-10T10:05:33.202551+0000","last_peered":"2024-09-10T10:05:33.202551+0000","last_clean":"2024-09-10T10:05:33.202551+0000","last_became_active":"2024-09-10T10:05:32.207735+0000","last_became_peered":"2024-09-10T10:05:32.207735+0000","last_unstale":"2024-09-10T10:05:33.202551+0000","last_undegraded":"2024-09-10T10:05:33.202551+0000","last_fullsized":"2024-09-10T10:05:33.202551+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:03.301673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198476+0000","last_change":"2024-09-10T10:05:32.199447+0000","last_active":"2024-09-10T10:05:33.198476+0000","last_peered":"2024-09-10T10:05:33.198476+0000","last_clean":"2024-09-10T10:05:33.198476+0000","last_became_active":"2024-09-10T10:05:32.199160+0000","last_became_peered":"2024-09-10T10:05:32.199160+0000","last_unstale":"2024-09-10T10:05:33.198476+0000","last_undegraded":"2024-09-10T10:05:33.198476+0000","last_fullsized":"2024-09-10T10:05:33.198476+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:58.744499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519304+0000","last_change":"2024-09-10T10:05:32.199666+0000","last_active":"2024-09-10T10:05:34.519304+0000","last_peered":"2024-09-10T10:05:34.519304+0000","last_clean":"2024-09-10T10:05:34.519304+0000","last_became_active":"2024-09-10T10:05:32.199156+0000","last_became_peered":"2024-09-10T10:05:32.199156+0000","last_unstale":"2024-09-10T10:05:34.519304+0000","last_undegraded":"2024-09-10T10:05:34.519304+0000","last_fullsized":"2024-09-10T10:05:34.519304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:32:58.263910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202462+0000","last_change":"2024-09-10T10:05:32.211878+0000","last_active":"2024-09-10T10:05:33.202462+0000","last_peered":"2024-09-10T10:05:33.202462+0000","last_clean":"2024-09-10T10:05:33.202462+0000","last_became_active":"2024-09-10T10:05:32.206847+0000","last_became_peered":"2024-09-10T10:05:32.206847+0000","last_unstale":"2024-09-10T10:05:33.202462+0000","last_undegraded":"2024-09-10T10:05:33.202462+0000","last_fullsized":"2024-09-10T10:05:33.202462+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:59.529682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197813+0000","last_change":"2024-09-10T10:05:32.187394+0000","last_active":"2024-09-10T10:05:33.197813+0000","last_peered":"2024-09-10T10:05:33.197813+0000","last_clean":"2024-09-10T10:05:33.197813+0000","last_became_active":"2024-09-10T10:05:32.186764+0000","last_became_peered":"2024-09-10T10:05:32.186764+0000","last_unstale":"2024-09-10T10:05:33.197813+0000","last_undegraded":"2024-09-10T10:05:33.197813+0000","last_fullsized":"2024-09-10T10:05:33.197813+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:03.301673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369009+0000","last_change":"2024-09-10T10:05:32.194103+0000","last_active":"2024-09-10T10:05:35.369009+0000","last_peered":"2024-09-10T10:05:35.369009+0000","last_clean":"2024-09-10T10:05:35.369009+0000","last_became_active":"2024-09-10T10:05:32.193419+0000","last_became_peered":"2024-09-10T10:05:32.193419+0000","last_unstale":"2024-09-10T10:05:35.369009+0000","last_undegraded":"2024-09-10T10:05:35.369009+0000","last_fullsized":"2024-09-10T10:05:35.369009+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:32.835603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386393+0000","last_change":"2024-09-10T10:05:32.205318+0000","last_active":"2024-09-10T10:05:34.386393+0000","last_peered":"2024-09-10T10:05:34.386393+0000","last_clean":"2024-09-10T10:05:34.386393+0000","last_became_active":"2024-09-10T10:05:32.205061+0000","last_became_peered":"2024-09-10T10:05:32.205061+0000","last_unstale":"2024-09-10T10:05:34.386393+0000","last_undegraded":"2024-09-10T10:05:34.386393+0000","last_fullsized":"2024-09-10T10:05:34.386393+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:58.989051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195297+0000","last_change":"2024-09-10T10:05:32.209313+0000","last_active":"2024-09-10T10:05:33.195297+0000","last_peered":"2024-09-10T10:05:33.195297+0000","last_clean":"2024-09-10T10:05:33.195297+0000","last_became_active":"2024-09-10T10:05:32.208815+0000","last_became_peered":"2024-09-10T10:05:32.208815+0000","last_unstale":"2024-09-10T10:05:33.195297+0000","last_undegraded":"2024-09-10T10:05:33.195297+0000","last_fullsized":"2024-09-10T10:05:33.195297+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:19.137072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198422+0000","last_change":"2024-09-10T10:05:32.202439+0000","last_active":"2024-09-10T10:05:33.198422+0000","last_peered":"2024-09-10T10:05:33.198422+0000","last_clean":"2024-09-10T10:05:33.198422+0000","last_became_active":"2024-09-10T10:05:32.202133+0000","last_became_peered":"2024-09-10T10:05:32.202133+0000","last_unstale":"2024-09-10T10:05:33.198422+0000","last_undegraded":"2024-09-10T10:05:33.198422+0000","last_fullsized":"2024-09-10T10:05:33.198422+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:12.558158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198626+0000","last_change":"2024-09-10T10:05:32.208477+0000","last_active":"2024-09-10T10:05:33.198626+0000","last_peered":"2024-09-10T10:05:33.198626+0000","last_clean":"2024-09-10T10:05:33.198626+0000","last_became_active":"2024-09-10T10:05:32.206369+0000","last_became_peered":"2024-09-10T10:05:32.206369+0000","last_unstale":"2024-09-10T10:05:33.198626+0000","last_undegraded":"2024-09-10T10:05:33.198626+0000","last_fullsized":"2024-09-10T10:05:33.198626+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:02.076083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197890+0000","last_change":"2024-09-10T10:05:32.188306+0000","last_active":"2024-09-10T10:05:33.197890+0000","last_peered":"2024-09-10T10:05:33.197890+0000","last_clean":"2024-09-10T10:05:33.197890+0000","last_became_active":"2024-09-10T10:05:32.187978+0000","last_became_peered":"2024-09-10T10:05:32.187978+0000","last_unstale":"2024-09-10T10:05:33.197890+0000","last_undegraded":"2024-09-10T10:05:33.197890+0000","last_fullsized":"2024-09-10T10:05:33.197890+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:31.639104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386026+0000","last_change":"2024-09-10T10:05:32.200311+0000","last_active":"2024-09-10T10:05:34.386026+0000","last_peered":"2024-09-10T10:05:34.386026+0000","last_clean":"2024-09-10T10:05:34.386026+0000","last_became_active":"2024-09-10T10:05:32.199524+0000","last_became_peered":"2024-09-10T10:05:32.199524+0000","last_unstale":"2024-09-10T10:05:34.386026+0000","last_undegraded":"2024-09-10T10:05:34.386026+0000","last_fullsized":"2024-09-10T10:05:34.386026+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:35.861386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198042+0000","last_change":"2024-09-10T10:05:32.196485+0000","last_active":"2024-09-10T10:05:33.198042+0000","last_peered":"2024-09-10T10:05:33.198042+0000","last_clean":"2024-09-10T10:05:33.198042+0000","last_became_active":"2024-09-10T10:05:32.196122+0000","last_became_peered":"2024-09-10T10:05:32.196122+0000","last_unstale":"2024-09-10T10:05:33.198042+0000","last_undegraded":"2024-09-10T10:05:33.198042+0000","last_fullsized":"2024-09-10T10:05:33.198042+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:02.295728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369050+0000","last_change":"2024-09-10T10:05:32.194339+0000","last_active":"2024-09-10T10:05:35.369050+0000","last_peered":"2024-09-10T10:05:35.369050+0000","last_clean":"2024-09-10T10:05:35.369050+0000","last_became_active":"2024-09-10T10:05:32.193818+0000","last_became_peered":"2024-09-10T10:05:32.193818+0000","last_unstale":"2024-09-10T10:05:35.369050+0000","last_undegraded":"2024-09-10T10:05:35.369050+0000","last_fullsized":"2024-09-10T10:05:35.369050+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:18.060307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519404+0000","last_change":"2024-09-10T10:05:32.206204+0000","last_active":"2024-09-10T10:05:34.519404+0000","last_peered":"2024-09-10T10:05:34.519404+0000","last_clean":"2024-09-10T10:05:34.519404+0000","last_became_active":"2024-09-10T10:05:32.205359+0000","last_became_peered":"2024-09-10T10:05:32.205359+0000","last_unstale":"2024-09-10T10:05:34.519404+0000","last_undegraded":"2024-09-10T10:05:34.519404+0000","last_fullsized":"2024-09-10T10:05:34.519404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:35.409146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371140+0000","last_change":"2024-09-10T10:05:32.210783+0000","last_active":"2024-09-10T10:05:35.371140+0000","last_peered":"2024-09-10T10:05:35.371140+0000","last_clean":"2024-09-10T10:05:35.371140+0000","last_became_active":"2024-09-10T10:05:32.209419+0000","last_became_peered":"2024-09-10T10:05:32.209419+0000","last_unstale":"2024-09-10T10:05:35.371140+0000","last_undegraded":"2024-09-10T10:05:35.371140+0000","last_fullsized":"2024-09-10T10:05:35.371140+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:40.349695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194927+0000","last_change":"2024-09-10T10:05:32.200603+0000","last_active":"2024-09-10T10:05:33.194927+0000","last_peered":"2024-09-10T10:05:33.194927+0000","last_clean":"2024-09-10T10:05:33.194927+0000","last_became_active":"2024-09-10T10:05:32.200267+0000","last_became_peered":"2024-09-10T10:05:32.200267+0000","last_unstale":"2024-09-10T10:05:33.194927+0000","last_undegraded":"2024-09-10T10:05:33.194927+0000","last_fullsized":"2024-09-10T10:05:33.194927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:14.117340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519233+0000","last_change":"2024-09-10T10:05:32.207078+0000","last_active":"2024-09-10T10:05:34.519233+0000","last_peered":"2024-09-10T10:05:34.519233+0000","last_clean":"2024-09-10T10:05:34.519233+0000","last_became_active":"2024-09-10T10:05:32.205823+0000","last_became_peered":"2024-09-10T10:05:32.205823+0000","last_unstale":"2024-09-10T10:05:34.519233+0000","last_undegraded":"2024-09-10T10:05:34.519233+0000","last_fullsized":"2024-09-10T10:05:34.519233+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:41.775789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519459+0000","last_change":"2024-09-10T10:05:32.199430+0000","last_active":"2024-09-10T10:05:34.519459+0000","last_peered":"2024-09-10T10:05:34.519459+0000","last_clean":"2024-09-10T10:05:34.519459+0000","last_became_active":"2024-09-10T10:05:32.198629+0000","last_became_peered":"2024-09-10T10:05:32.198629+0000","last_unstale":"2024-09-10T10:05:34.519459+0000","last_undegraded":"2024-09-10T10:05:34.519459+0000","last_fullsized":"2024-09-10T10:05:34.519459+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:56.915472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194697+0000","last_change":"2024-09-10T10:05:32.194587+0000","last_active":"2024-09-10T10:05:33.194697+0000","last_peered":"2024-09-10T10:05:33.194697+0000","last_clean":"2024-09-10T10:05:33.194697+0000","last_became_active":"2024-09-10T10:05:32.193892+0000","last_became_peered":"2024-09-10T10:05:32.193892+0000","last_unstale":"2024-09-10T10:05:33.194697+0000","last_undegraded":"2024-09-10T10:05:33.194697+0000","last_fullsized":"2024-09-10T10:05:33.194697+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:34.240642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202372+0000","last_change":"2024-09-10T10:05:32.215026+0000","last_active":"2024-09-10T10:05:33.202372+0000","last_peered":"2024-09-10T10:05:33.202372+0000","last_clean":"2024-09-10T10:05:33.202372+0000","last_became_active":"2024-09-10T10:05:32.214758+0000","last_became_peered":"2024-09-10T10:05:32.214758+0000","last_unstale":"2024-09-10T10:05:33.202372+0000","last_undegraded":"2024-09-10T10:05:33.202372+0000","last_fullsized":"2024-09-10T10:05:33.202372+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:18.256201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369178+0000","last_change":"2024-09-10T10:05:32.215277+0000","last_active":"2024-09-10T10:05:35.369178+0000","last_peered":"2024-09-10T10:05:35.369178+0000","last_clean":"2024-09-10T10:05:35.369178+0000","last_became_active":"2024-09-10T10:05:32.214987+0000","last_became_peered":"2024-09-10T10:05:32.214987+0000","last_unstale":"2024-09-10T10:05:35.369178+0000","last_undegraded":"2024-09-10T10:05:35.369178+0000","last_fullsized":"2024-09-10T10:05:35.369178+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:06.383388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198913+0000","last_change":"2024-09-10T10:05:32.197493+0000","last_active":"2024-09-10T10:05:33.198913+0000","last_peered":"2024-09-10T10:05:33.198913+0000","last_clean":"2024-09-10T10:05:33.198913+0000","last_became_active":"2024-09-10T10:05:32.197225+0000","last_became_peered":"2024-09-10T10:05:32.197225+0000","last_unstale":"2024-09-10T10:05:33.198913+0000","last_undegraded":"2024-09-10T10:05:33.198913+0000","last_fullsized":"2024-09-10T10:05:33.198913+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:03.518589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198434+0000","last_change":"2024-09-10T10:05:32.196836+0000","last_active":"2024-09-10T10:05:33.198434+0000","last_peered":"2024-09-10T10:05:33.198434+0000","last_clean":"2024-09-10T10:05:33.198434+0000","last_became_active":"2024-09-10T10:05:32.194797+0000","last_became_peered":"2024-09-10T10:05:32.194797+0000","last_unstale":"2024-09-10T10:05:33.198434+0000","last_undegraded":"2024-09-10T10:05:33.198434+0000","last_fullsized":"2024-09-10T10:05:33.198434+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:38:31.639104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202693+0000","last_change":"2024-09-10T10:05:32.215811+0000","last_active":"2024-09-10T10:05:33.202693+0000","last_peered":"2024-09-10T10:05:33.202693+0000","last_clean":"2024-09-10T10:05:33.202693+0000","last_became_active":"2024-09-10T10:05:32.208360+0000","last_became_peered":"2024-09-10T10:05:32.208360+0000","last_unstale":"2024-09-10T10:05:33.202693+0000","last_undegraded":"2024-09-10T10:05:33.202693+0000","last_fullsized":"2024-09-10T10:05:33.202693+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:37.843191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519239+0000","last_change":"2024-09-10T10:05:32.196207+0000","last_active":"2024-09-10T10:05:34.519239+0000","last_peered":"2024-09-10T10:05:34.519239+0000","last_clean":"2024-09-10T10:05:34.519239+0000","last_became_active":"2024-09-10T10:05:32.195918+0000","last_became_peered":"2024-09-10T10:05:32.195918+0000","last_unstale":"2024-09-10T10:05:34.519239+0000","last_undegraded":"2024-09-10T10:05:34.519239+0000","last_fullsized":"2024-09-10T10:05:34.519239+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:16.425830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198325+0000","last_change":"2024-09-10T10:05:32.200164+0000","last_active":"2024-09-10T10:05:33.198325+0000","last_peered":"2024-09-10T10:05:33.198325+0000","last_clean":"2024-09-10T10:05:33.198325+0000","last_became_active":"2024-09-10T10:05:32.199910+0000","last_became_peered":"2024-09-10T10:05:32.199910+0000","last_unstale":"2024-09-10T10:05:33.198325+0000","last_undegraded":"2024-09-10T10:05:33.198325+0000","last_fullsized":"2024-09-10T10:05:33.198325+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:57.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386020+0000","last_change":"2024-09-10T10:05:32.201674+0000","last_active":"2024-09-10T10:05:34.386020+0000","last_peered":"2024-09-10T10:05:34.386020+0000","last_clean":"2024-09-10T10:05:34.386020+0000","last_became_active":"2024-09-10T10:05:32.199143+0000","last_became_peered":"2024-09-10T10:05:32.199143+0000","last_unstale":"2024-09-10T10:05:34.386020+0000","last_undegraded":"2024-09-10T10:05:34.386020+0000","last_fullsized":"2024-09-10T10:05:34.386020+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:04.619410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202424+0000","last_change":"2024-09-10T10:05:32.210480+0000","last_active":"2024-09-10T10:05:33.202424+0000","last_peered":"2024-09-10T10:05:33.202424+0000","last_clean":"2024-09-10T10:05:33.202424+0000","last_became_active":"2024-09-10T10:05:32.210213+0000","last_became_peered":"2024-09-10T10:05:32.210213+0000","last_unstale":"2024-09-10T10:05:33.202424+0000","last_undegraded":"2024-09-10T10:05:33.202424+0000","last_fullsized":"2024-09-10T10:05:33.202424+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:23.197105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519049+0000","last_change":"2024-09-10T10:05:32.197240+0000","last_active":"2024-09-10T10:05:34.519049+0000","last_peered":"2024-09-10T10:05:34.519049+0000","last_clean":"2024-09-10T10:05:34.519049+0000","last_became_active":"2024-09-10T10:05:32.194745+0000","last_became_peered":"2024-09-10T10:05:32.194745+0000","last_unstale":"2024-09-10T10:05:34.519049+0000","last_undegraded":"2024-09-10T10:05:34.519049+0000","last_fullsized":"2024-09-10T10:05:34.519049+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:34.471388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369402+0000","last_change":"2024-09-10T10:05:32.199710+0000","last_active":"2024-09-10T10:05:35.369402+0000","last_peered":"2024-09-10T10:05:35.369402+0000","last_clean":"2024-09-10T10:05:35.369402+0000","last_became_active":"2024-09-10T10:05:32.199388+0000","last_became_peered":"2024-09-10T10:05:32.199388+0000","last_unstale":"2024-09-10T10:05:35.369402+0000","last_undegraded":"2024-09-10T10:05:35.369402+0000","last_fullsized":"2024-09-10T10:05:35.369402+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:08:55.894584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"152'4","reported_seq":18,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519428+0000","last_change":"2024-09-10T10:05:31.187728+0000","last_active":"2024-09-10T10:05:34.519428+0000","last_peered":"2024-09-10T10:05:34.519428+0000","last_clean":"2024-09-10T10:05:34.519428+0000","last_became_active":"2024-09-10T10:05:31.186044+0000","last_became_peered":"2024-09-10T10:05:31.186044+0000","last_unstale":"2024-09-10T10:05:34.519428+0000","last_undegraded":"2024-09-10T10:05:34.519428+0000","last_fullsized":"2024-09-10T10:05:34.519428+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:24:47.086239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195512+0000","last_change":"2024-09-10T10:05:32.200253+0000","last_active":"2024-09-10T10:05:33.195512+0000","last_peered":"2024-09-10T10:05:33.195512+0000","last_clean":"2024-09-10T10:05:33.195512+0000","last_became_active":"2024-09-10T10:05:32.199931+0000","last_became_peered":"2024-09-10T10:05:32.199931+0000","last_unstale":"2024-09-10T10:05:33.195512+0000","last_undegraded":"2024-09-10T10:05:33.195512+0000","last_fullsized":"2024-09-10T10:05:33.195512+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:13.682778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386389+0000","last_change":"2024-09-10T10:05:31.187161+0000","last_active":"2024-09-10T10:05:34.386389+0000","last_peered":"2024-09-10T10:05:34.386389+0000","last_clean":"2024-09-10T10:05:34.386389+0000","last_became_active":"2024-09-10T10:05:31.186727+0000","last_became_peered":"2024-09-10T10:05:31.186727+0000","last_unstale":"2024-09-10T10:05:34.386389+0000","last_undegraded":"2024-09-10T10:05:34.386389+0000","last_fullsized":"2024-09-10T10:05:34.386389+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:33.464770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198361+0000","last_change":"2024-09-10T10:05:32.202596+0000","last_active":"2024-09-10T10:05:33.198361+0000","last_peered":"2024-09-10T10:05:33.198361+0000","last_clean":"2024-09-10T10:05:33.198361+0000","last_became_active":"2024-09-10T10:05:32.200011+0000","last_became_peered":"2024-09-10T10:05:32.200011+0000","last_unstale":"2024-09-10T10:05:33.198361+0000","last_undegraded":"2024-09-10T10:05:33.198361+0000","last_fullsized":"2024-09-10T10:05:33.198361+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:36.178267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202643+0000","last_change":"2024-09-10T10:05:31.196039+0000","last_active":"2024-09-10T10:05:33.202643+0000","last_peered":"2024-09-10T10:05:33.202643+0000","last_clean":"2024-09-10T10:05:33.202643+0000","last_became_active":"2024-09-10T10:05:31.195762+0000","last_became_peered":"2024-09-10T10:05:31.195762+0000","last_unstale":"2024-09-10T10:05:33.202643+0000","last_undegraded":"2024-09-10T10:05:33.202643+0000","last_fullsized":"2024-09-10T10:05:33.202643+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:24.919933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386358+0000","last_change":"2024-09-10T10:05:32.198986+0000","last_active":"2024-09-10T10:05:34.386358+0000","last_peered":"2024-09-10T10:05:34.386358+0000","last_clean":"2024-09-10T10:05:34.386358+0000","last_became_active":"2024-09-10T10:05:32.198643+0000","last_became_peered":"2024-09-10T10:05:32.198643+0000","last_unstale":"2024-09-10T10:05:34.386358+0000","last_undegraded":"2024-09-10T10:05:34.386358+0000","last_fullsized":"2024-09-10T10:05:34.386358+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:25.926551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198711+0000","last_change":"2024-09-10T10:05:31.184956+0000","last_active":"2024-09-10T10:05:33.198711+0000","last_peered":"2024-09-10T10:05:33.198711+0000","last_clean":"2024-09-10T10:05:33.198711+0000","last_became_active":"2024-09-10T10:05:31.184628+0000","last_became_peered":"2024-09-10T10:05:31.184628+0000","last_unstale":"2024-09-10T10:05:33.198711+0000","last_undegraded":"2024-09-10T10:05:33.198711+0000","last_fullsized":"2024-09-10T10:05:33.198711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:48.990477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194844+0000","last_change":"2024-09-10T10:05:32.194160+0000","last_active":"2024-09-10T10:05:33.194844+0000","last_peered":"2024-09-10T10:05:33.194844+0000","last_clean":"2024-09-10T10:05:33.194844+0000","last_became_active":"2024-09-10T10:05:32.193487+0000","last_became_peered":"2024-09-10T10:05:32.193487+0000","last_unstale":"2024-09-10T10:05:33.194844+0000","last_undegraded":"2024-09-10T10:05:33.194844+0000","last_fullsized":"2024-09-10T10:05:33.194844+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:19.992141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519463+0000","last_change":"2024-09-10T10:05:31.189563+0000","last_active":"2024-09-10T10:05:34.519463+0000","last_peered":"2024-09-10T10:05:34.519463+0000","last_clean":"2024-09-10T10:05:34.519463+0000","last_became_active":"2024-09-10T10:05:31.189169+0000","last_became_peered":"2024-09-10T10:05:31.189169+0000","last_unstale":"2024-09-10T10:05:34.519463+0000","last_undegraded":"2024-09-10T10:05:34.519463+0000","last_fullsized":"2024-09-10T10:05:34.519463+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:00.085710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369330+0000","last_change":"2024-09-10T10:05:32.199253+0000","last_active":"2024-09-10T10:05:35.369330+0000","last_peered":"2024-09-10T10:05:35.369330+0000","last_clean":"2024-09-10T10:05:35.369330+0000","last_became_active":"2024-09-10T10:05:32.198913+0000","last_became_peered":"2024-09-10T10:05:32.198913+0000","last_unstale":"2024-09-10T10:05:35.369330+0000","last_undegraded":"2024-09-10T10:05:35.369330+0000","last_fullsized":"2024-09-10T10:05:35.369330+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:16.177921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369126+0000","last_change":"2024-09-10T10:05:31.184111+0000","last_active":"2024-09-10T10:05:35.369126+0000","last_peered":"2024-09-10T10:05:35.369126+0000","last_clean":"2024-09-10T10:05:35.369126+0000","last_became_active":"2024-09-10T10:05:31.183673+0000","last_became_peered":"2024-09-10T10:05:31.183673+0000","last_unstale":"2024-09-10T10:05:35.369126+0000","last_undegraded":"2024-09-10T10:05:35.369126+0000","last_fullsized":"2024-09-10T10:05:35.369126+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:44.121547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197821+0000","last_change":"2024-09-10T10:05:32.207266+0000","last_active":"2024-09-10T10:05:33.197821+0000","last_peered":"2024-09-10T10:05:33.197821+0000","last_clean":"2024-09-10T10:05:33.197821+0000","last_became_active":"2024-09-10T10:05:32.205319+0000","last_became_peered":"2024-09-10T10:05:32.205319+0000","last_unstale":"2024-09-10T10:05:33.197821+0000","last_undegraded":"2024-09-10T10:05:33.197821+0000","last_fullsized":"2024-09-10T10:05:33.197821+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:37.843191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370732+0000","last_change":"2024-09-10T10:05:31.193233+0000","last_active":"2024-09-10T10:05:35.370732+0000","last_peered":"2024-09-10T10:05:35.370732+0000","last_clean":"2024-09-10T10:05:35.370732+0000","last_became_active":"2024-09-10T10:05:31.192971+0000","last_became_peered":"2024-09-10T10:05:31.192971+0000","last_unstale":"2024-09-10T10:05:35.370732+0000","last_undegraded":"2024-09-10T10:05:35.370732+0000","last_fullsized":"2024-09-10T10:05:35.370732+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:06.426027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202832+0000","last_change":"2024-09-10T10:05:32.207268+0000","last_active":"2024-09-10T10:05:33.202832+0000","last_peered":"2024-09-10T10:05:33.202832+0000","last_clean":"2024-09-10T10:05:33.202832+0000","last_became_active":"2024-09-10T10:05:32.206685+0000","last_became_peered":"2024-09-10T10:05:32.206685+0000","last_unstale":"2024-09-10T10:05:33.202832+0000","last_undegraded":"2024-09-10T10:05:33.202832+0000","last_fullsized":"2024-09-10T10:05:33.202832+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:48.365603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197806+0000","last_change":"2024-09-10T10:05:31.185108+0000","last_active":"2024-09-10T10:05:33.197806+0000","last_peered":"2024-09-10T10:05:33.197806+0000","last_clean":"2024-09-10T10:05:33.197806+0000","last_became_active":"2024-09-10T10:05:31.184889+0000","last_became_peered":"2024-09-10T10:05:31.184889+0000","last_unstale":"2024-09-10T10:05:33.197806+0000","last_undegraded":"2024-09-10T10:05:33.197806+0000","last_fullsized":"2024-09-10T10:05:33.197806+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:56.692445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369405+0000","last_change":"2024-09-10T10:05:32.198420+0000","last_active":"2024-09-10T10:05:35.369405+0000","last_peered":"2024-09-10T10:05:35.369405+0000","last_clean":"2024-09-10T10:05:35.369405+0000","last_became_active":"2024-09-10T10:05:32.198099+0000","last_became_peered":"2024-09-10T10:05:32.198099+0000","last_unstale":"2024-09-10T10:05:35.369405+0000","last_undegraded":"2024-09-10T10:05:35.369405+0000","last_fullsized":"2024-09-10T10:05:35.369405+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:57.163059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519363+0000","last_change":"2024-09-10T10:05:31.190056+0000","last_active":"2024-09-10T10:05:34.519363+0000","last_peered":"2024-09-10T10:05:34.519363+0000","last_clean":"2024-09-10T10:05:34.519363+0000","last_became_active":"2024-09-10T10:05:31.189791+0000","last_became_peered":"2024-09-10T10:05:31.189791+0000","last_unstale":"2024-09-10T10:05:34.519363+0000","last_undegraded":"2024-09-10T10:05:34.519363+0000","last_fullsized":"2024-09-10T10:05:34.519363+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:46.207558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198377+0000","last_change":"2024-09-10T10:05:32.206830+0000","last_active":"2024-09-10T10:05:33.198377+0000","last_peered":"2024-09-10T10:05:33.198377+0000","last_clean":"2024-09-10T10:05:33.198377+0000","last_became_active":"2024-09-10T10:05:32.206552+0000","last_became_peered":"2024-09-10T10:05:32.206552+0000","last_unstale":"2024-09-10T10:05:33.198377+0000","last_undegraded":"2024-09-10T10:05:33.198377+0000","last_fullsized":"2024-09-10T10:05:33.198377+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:51.678475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"152'3","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519202+0000","last_change":"2024-09-10T10:05:31.188150+0000","last_active":"2024-09-10T10:05:34.519202+0000","last_peered":"2024-09-10T10:05:34.519202+0000","last_clean":"2024-09-10T10:05:34.519202+0000","last_became_active":"2024-09-10T10:05:31.185519+0000","last_became_peered":"2024-09-10T10:05:31.185519+0000","last_unstale":"2024-09-10T10:05:34.519202+0000","last_undegraded":"2024-09-10T10:05:34.519202+0000","last_fullsized":"2024-09-10T10:05:34.519202+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:59.166885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195098+0000","last_change":"2024-09-10T10:05:32.195869+0000","last_active":"2024-09-10T10:05:33.195098+0000","last_peered":"2024-09-10T10:05:33.195098+0000","last_clean":"2024-09-10T10:05:33.195098+0000","last_became_active":"2024-09-10T10:05:32.195597+0000","last_became_peered":"2024-09-10T10:05:32.195597+0000","last_unstale":"2024-09-10T10:05:33.195098+0000","last_undegraded":"2024-09-10T10:05:33.195098+0000","last_fullsized":"2024-09-10T10:05:33.195098+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:32.283156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"152'6","reported_seq":20,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202554+0000","last_change":"2024-09-10T10:05:31.194843+0000","last_active":"2024-09-10T10:05:33.202554+0000","last_peered":"2024-09-10T10:05:33.202554+0000","last_clean":"2024-09-10T10:05:33.202554+0000","last_became_active":"2024-09-10T10:05:31.194168+0000","last_became_peered":"2024-09-10T10:05:31.194168+0000","last_unstale":"2024-09-10T10:05:33.202554+0000","last_undegraded":"2024-09-10T10:05:33.202554+0000","last_fullsized":"2024-09-10T10:05:33.202554+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:57.982434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202381+0000","last_change":"2024-09-10T10:05:32.207407+0000","last_active":"2024-09-10T10:05:33.202381+0000","last_peered":"2024-09-10T10:05:33.202381+0000","last_clean":"2024-09-10T10:05:33.202381+0000","last_became_active":"2024-09-10T10:05:32.206481+0000","last_became_peered":"2024-09-10T10:05:32.206481+0000","last_unstale":"2024-09-10T10:05:33.202381+0000","last_undegraded":"2024-09-10T10:05:33.202381+0000","last_fullsized":"2024-09-10T10:05:33.202381+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:19.513722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371351+0000","last_change":"2024-09-10T10:05:31.189711+0000","last_active":"2024-09-10T10:05:35.371351+0000","last_peered":"2024-09-10T10:05:35.371351+0000","last_clean":"2024-09-10T10:05:35.371351+0000","last_became_active":"2024-09-10T10:05:31.189471+0000","last_became_peered":"2024-09-10T10:05:31.189471+0000","last_unstale":"2024-09-10T10:05:35.371351+0000","last_undegraded":"2024-09-10T10:05:35.371351+0000","last_fullsized":"2024-09-10T10:05:35.371351+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:13.110723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198087+0000","last_change":"2024-09-10T10:05:32.192330+0000","last_active":"2024-09-10T10:05:33.198087+0000","last_peered":"2024-09-10T10:05:33.198087+0000","last_clean":"2024-09-10T10:05:33.198087+0000","last_became_active":"2024-09-10T10:05:32.192064+0000","last_became_peered":"2024-09-10T10:05:32.192064+0000","last_unstale":"2024-09-10T10:05:33.198087+0000","last_undegraded":"2024-09-10T10:05:33.198087+0000","last_fullsized":"2024-09-10T10:05:33.198087+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:48.365603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194922+0000","last_change":"2024-09-10T10:05:31.195254+0000","last_active":"2024-09-10T10:05:33.194922+0000","last_peered":"2024-09-10T10:05:33.194922+0000","last_clean":"2024-09-10T10:05:33.194922+0000","last_became_active":"2024-09-10T10:05:31.194909+0000","last_became_peered":"2024-09-10T10:05:31.194909+0000","last_unstale":"2024-09-10T10:05:33.194922+0000","last_undegraded":"2024-09-10T10:05:33.194922+0000","last_fullsized":"2024-09-10T10:05:33.194922+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:34.530341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.385981+0000","last_change":"2024-09-10T10:05:32.198620+0000","last_active":"2024-09-10T10:05:34.385981+0000","last_peered":"2024-09-10T10:05:34.385981+0000","last_clean":"2024-09-10T10:05:34.385981+0000","last_became_active":"2024-09-10T10:05:32.197165+0000","last_became_peered":"2024-09-10T10:05:32.197165+0000","last_unstale":"2024-09-10T10:05:34.385981+0000","last_undegraded":"2024-09-10T10:05:34.385981+0000","last_fullsized":"2024-09-10T10:05:34.385981+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:09.383770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.368912+0000","last_change":"2024-09-10T10:05:31.176701+0000","last_active":"2024-09-10T10:05:35.368912+0000","last_peered":"2024-09-10T10:05:35.368912+0000","last_clean":"2024-09-10T10:05:35.368912+0000","last_became_active":"2024-09-10T10:05:31.176425+0000","last_became_peered":"2024-09-10T10:05:31.176425+0000","last_unstale":"2024-09-10T10:05:35.368912+0000","last_undegraded":"2024-09-10T10:05:35.368912+0000","last_fullsized":"2024-09-10T10:05:35.368912+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:37.745872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194579+0000","last_change":"2024-09-10T10:05:32.190495+0000","last_active":"2024-09-10T10:05:33.194579+0000","last_peered":"2024-09-10T10:05:33.194579+0000","last_clean":"2024-09-10T10:05:33.194579+0000","last_became_active":"2024-09-10T10:05:32.189347+0000","last_became_peered":"2024-09-10T10:05:32.189347+0000","last_unstale":"2024-09-10T10:05:33.194579+0000","last_undegraded":"2024-09-10T10:05:33.194579+0000","last_fullsized":"2024-09-10T10:05:33.194579+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:07.432644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"152'8","reported_seq":22,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370869+0000","last_change":"2024-09-10T10:05:31.193087+0000","last_active":"2024-09-10T10:05:35.370869+0000","last_peered":"2024-09-10T10:05:35.370869+0000","last_clean":"2024-09-10T10:05:35.370869+0000","last_became_active":"2024-09-10T10:05:31.192788+0000","last_became_peered":"2024-09-10T10:05:31.192788+0000","last_unstale":"2024-09-10T10:05:35.370869+0000","last_undegraded":"2024-09-10T10:05:35.370869+0000","last_fullsized":"2024-09-10T10:05:35.370869+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:33.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195323+0000","last_change":"2024-09-10T10:05:32.187973+0000","last_active":"2024-09-10T10:05:33.195323+0000","last_peered":"2024-09-10T10:05:33.195323+0000","last_clean":"2024-09-10T10:05:33.195323+0000","last_became_active":"2024-09-10T10:05:32.187672+0000","last_became_peered":"2024-09-10T10:05:32.187672+0000","last_unstale":"2024-09-10T10:05:33.195323+0000","last_undegraded":"2024-09-10T10:05:33.195323+0000","last_fullsized":"2024-09-10T10:05:33.195323+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:11.967689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519472+0000","last_change":"2024-09-10T10:05:31.189469+0000","last_active":"2024-09-10T10:05:34.519472+0000","last_peered":"2024-09-10T10:05:34.519472+0000","last_clean":"2024-09-10T10:05:34.519472+0000","last_became_active":"2024-09-10T10:05:31.189175+0000","last_became_peered":"2024-09-10T10:05:31.189175+0000","last_unstale":"2024-09-10T10:05:34.519472+0000","last_undegraded":"2024-09-10T10:05:34.519472+0000","last_fullsized":"2024-09-10T10:05:34.519472+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:51.865168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386581+0000","last_change":"2024-09-10T10:05:32.205086+0000","last_active":"2024-09-10T10:05:34.386581+0000","last_peered":"2024-09-10T10:05:34.386581+0000","last_clean":"2024-09-10T10:05:34.386581+0000","last_became_active":"2024-09-10T10:05:32.204687+0000","last_became_peered":"2024-09-10T10:05:32.204687+0000","last_unstale":"2024-09-10T10:05:34.386581+0000","last_undegraded":"2024-09-10T10:05:34.386581+0000","last_fullsized":"2024-09-10T10:05:34.386581+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:57.699063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369353+0000","last_change":"2024-09-10T10:05:31.176322+0000","last_active":"2024-09-10T10:05:35.369353+0000","last_peered":"2024-09-10T10:05:35.369353+0000","last_clean":"2024-09-10T10:05:35.369353+0000","last_became_active":"2024-09-10T10:05:31.175852+0000","last_became_peered":"2024-09-10T10:05:31.175852+0000","last_unstale":"2024-09-10T10:05:35.369353+0000","last_undegraded":"2024-09-10T10:05:35.369353+0000","last_fullsized":"2024-09-10T10:05:35.369353+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:18.556500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386471+0000","last_change":"2024-09-10T10:05:32.210506+0000","last_active":"2024-09-10T10:05:34.386471+0000","last_peered":"2024-09-10T10:05:34.386471+0000","last_clean":"2024-09-10T10:05:34.386471+0000","last_became_active":"2024-09-10T10:05:32.210029+0000","last_became_peered":"2024-09-10T10:05:32.210029+0000","last_unstale":"2024-09-10T10:05:34.386471+0000","last_undegraded":"2024-09-10T10:05:34.386471+0000","last_fullsized":"2024-09-10T10:05:34.386471+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:51.917046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"152'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371111+0000","last_change":"2024-09-10T10:05:31.193098+0000","last_active":"2024-09-10T10:05:35.371111+0000","last_peered":"2024-09-10T10:05:35.371111+0000","last_clean":"2024-09-10T10:05:35.371111+0000","last_became_active":"2024-09-10T10:05:31.192779+0000","last_became_peered":"2024-09-10T10:05:31.192779+0000","last_unstale":"2024-09-10T10:05:35.371111+0000","last_undegraded":"2024-09-10T10:05:35.371111+0000","last_fullsized":"2024-09-10T10:05:35.371111+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:50.441112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198508+0000","last_change":"2024-09-10T10:05:32.200610+0000","last_active":"2024-09-10T10:05:33.198508+0000","last_peered":"2024-09-10T10:05:33.198508+0000","last_clean":"2024-09-10T10:05:33.198508+0000","last_became_active":"2024-09-10T10:05:32.198412+0000","last_became_peered":"2024-09-10T10:05:32.198412+0000","last_unstale":"2024-09-10T10:05:33.198508+0000","last_undegraded":"2024-09-10T10:05:33.198508+0000","last_fullsized":"2024-09-10T10:05:33.198508+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:25.293334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198525+0000","last_change":"2024-09-10T10:05:31.193283+0000","last_active":"2024-09-10T10:05:33.198525+0000","last_peered":"2024-09-10T10:05:33.198525+0000","last_clean":"2024-09-10T10:05:33.198525+0000","last_became_active":"2024-09-10T10:05:31.192470+0000","last_became_peered":"2024-09-10T10:05:31.192470+0000","last_unstale":"2024-09-10T10:05:33.198525+0000","last_undegraded":"2024-09-10T10:05:33.198525+0000","last_fullsized":"2024-09-10T10:05:33.198525+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:26.997421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194954+0000","last_change":"2024-09-10T10:05:32.198896+0000","last_active":"2024-09-10T10:05:33.194954+0000","last_peered":"2024-09-10T10:05:33.194954+0000","last_clean":"2024-09-10T10:05:33.194954+0000","last_became_active":"2024-09-10T10:05:32.198582+0000","last_became_peered":"2024-09-10T10:05:32.198582+0000","last_unstale":"2024-09-10T10:05:33.194954+0000","last_undegraded":"2024-09-10T10:05:33.194954+0000","last_fullsized":"2024-09-10T10:05:33.194954+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:51.447729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202606+0000","last_change":"2024-09-10T10:05:31.194554+0000","last_active":"2024-09-10T10:05:33.202606+0000","last_peered":"2024-09-10T10:05:33.202606+0000","last_clean":"2024-09-10T10:05:33.202606+0000","last_became_active":"2024-09-10T10:05:31.194265+0000","last_became_peered":"2024-09-10T10:05:31.194265+0000","last_unstale":"2024-09-10T10:05:33.202606+0000","last_undegraded":"2024-09-10T10:05:33.202606+0000","last_fullsized":"2024-09-10T10:05:33.202606+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:03.612793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198134+0000","last_change":"2024-09-10T10:05:32.195045+0000","last_active":"2024-09-10T10:05:33.198134+0000","last_peered":"2024-09-10T10:05:33.198134+0000","last_clean":"2024-09-10T10:05:33.198134+0000","last_became_active":"2024-09-10T10:05:32.194735+0000","last_became_peered":"2024-09-10T10:05:32.194735+0000","last_unstale":"2024-09-10T10:05:33.198134+0000","last_undegraded":"2024-09-10T10:05:33.198134+0000","last_fullsized":"2024-09-10T10:05:33.198134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:43.449510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519177+0000","last_change":"2024-09-10T10:05:31.188898+0000","last_active":"2024-09-10T10:05:34.519177+0000","last_peered":"2024-09-10T10:05:34.519177+0000","last_clean":"2024-09-10T10:05:34.519177+0000","last_became_active":"2024-09-10T10:05:31.186075+0000","last_became_peered":"2024-09-10T10:05:31.186075+0000","last_unstale":"2024-09-10T10:05:34.519177+0000","last_undegraded":"2024-09-10T10:05:34.519177+0000","last_fullsized":"2024-09-10T10:05:34.519177+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:01.924390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197595+0000","last_change":"2024-09-10T10:05:32.198142+0000","last_active":"2024-09-10T10:05:33.197595+0000","last_peered":"2024-09-10T10:05:33.197595+0000","last_clean":"2024-09-10T10:05:33.197595+0000","last_became_active":"2024-09-10T10:05:32.197887+0000","last_became_peered":"2024-09-10T10:05:32.197887+0000","last_unstale":"2024-09-10T10:05:33.197595+0000","last_undegraded":"2024-09-10T10:05:33.197595+0000","last_fullsized":"2024-09-10T10:05:33.197595+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:12.558158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202472+0000","last_change":"2024-09-10T10:05:31.187311+0000","last_active":"2024-09-10T10:05:33.202472+0000","last_peered":"2024-09-10T10:05:33.202472+0000","last_clean":"2024-09-10T10:05:33.202472+0000","last_became_active":"2024-09-10T10:05:31.186827+0000","last_became_peered":"2024-09-10T10:05:31.186827+0000","last_unstale":"2024-09-10T10:05:33.202472+0000","last_undegraded":"2024-09-10T10:05:33.202472+0000","last_fullsized":"2024-09-10T10:05:33.202472+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:50.910428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519067+0000","last_change":"2024-09-10T10:05:32.209255+0000","last_active":"2024-09-10T10:05:34.519067+0000","last_peered":"2024-09-10T10:05:34.519067+0000","last_clean":"2024-09-10T10:05:34.519067+0000","last_became_active":"2024-09-10T10:05:32.209021+0000","last_became_peered":"2024-09-10T10:05:32.209021+0000","last_unstale":"2024-09-10T10:05:34.519067+0000","last_undegraded":"2024-09-10T10:05:34.519067+0000","last_fullsized":"2024-09-10T10:05:34.519067+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:00.267164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386022+0000","last_change":"2024-09-10T10:05:31.187284+0000","last_active":"2024-09-10T10:05:34.386022+0000","last_peered":"2024-09-10T10:05:34.386022+0000","last_clean":"2024-09-10T10:05:34.386022+0000","last_became_active":"2024-09-10T10:05:31.186766+0000","last_became_peered":"2024-09-10T10:05:31.186766+0000","last_unstale":"2024-09-10T10:05:34.386022+0000","last_undegraded":"2024-09-10T10:05:34.386022+0000","last_fullsized":"2024-09-10T10:05:34.386022+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:18.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519346+0000","last_change":"2024-09-10T10:05:32.210555+0000","last_active":"2024-09-10T10:05:34.519346+0000","last_peered":"2024-09-10T10:05:34.519346+0000","last_clean":"2024-09-10T10:05:34.519346+0000","last_became_active":"2024-09-10T10:05:32.210243+0000","last_became_peered":"2024-09-10T10:05:32.210243+0000","last_unstale":"2024-09-10T10:05:34.519346+0000","last_undegraded":"2024-09-10T10:05:34.519346+0000","last_fullsized":"2024-09-10T10:05:34.519346+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:54.156101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"152'2","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371182+0000","last_change":"2024-09-10T10:05:31.188601+0000","last_active":"2024-09-10T10:05:35.371182+0000","last_peered":"2024-09-10T10:05:35.371182+0000","last_clean":"2024-09-10T10:05:35.371182+0000","last_became_active":"2024-09-10T10:05:31.188289+0000","last_became_peered":"2024-09-10T10:05:31.188289+0000","last_unstale":"2024-09-10T10:05:35.371182+0000","last_undegraded":"2024-09-10T10:05:35.371182+0000","last_fullsized":"2024-09-10T10:05:35.371182+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:31.276538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195571+0000","last_change":"2024-09-10T10:05:32.197359+0000","last_active":"2024-09-10T10:05:33.195571+0000","last_peered":"2024-09-10T10:05:33.195571+0000","last_clean":"2024-09-10T10:05:33.195571+0000","last_became_active":"2024-09-10T10:05:32.197045+0000","last_became_peered":"2024-09-10T10:05:32.197045+0000","last_unstale":"2024-09-10T10:05:33.195571+0000","last_undegraded":"2024-09-10T10:05:33.195571+0000","last_fullsized":"2024-09-10T10:05:33.195571+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:19.617660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"153'1","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195419+0000","last_change":"2024-09-10T10:05:31.196820+0000","last_active":"2024-09-10T10:05:33.195419+0000","last_peered":"2024-09-10T10:05:33.195419+0000","last_clean":"2024-09-10T10:05:33.195419+0000","last_became_active":"2024-09-10T10:05:31.196557+0000","last_became_peered":"2024-09-10T10:05:31.196557+0000","last_unstale":"2024-09-10T10:05:33.195419+0000","last_undegraded":"2024-09-10T10:05:33.195419+0000","last_fullsized":"2024-09-10T10:05:33.195419+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:28.197728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369302+0000","last_change":"2024-09-10T10:05:32.215523+0000","last_active":"2024-09-10T10:05:35.369302+0000","last_peered":"2024-09-10T10:05:35.369302+0000","last_clean":"2024-09-10T10:05:35.369302+0000","last_became_active":"2024-09-10T10:05:32.215239+0000","last_became_peered":"2024-09-10T10:05:32.215239+0000","last_unstale":"2024-09-10T10:05:35.369302+0000","last_undegraded":"2024-09-10T10:05:35.369302+0000","last_fullsized":"2024-09-10T10:05:35.369302+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:22.289745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.195118+0000","last_change":"2024-09-10T10:05:31.197033+0000","last_active":"2024-09-10T10:05:33.195118+0000","last_peered":"2024-09-10T10:05:33.195118+0000","last_clean":"2024-09-10T10:05:33.195118+0000","last_became_active":"2024-09-10T10:05:31.196739+0000","last_became_peered":"2024-09-10T10:05:31.196739+0000","last_unstale":"2024-09-10T10:05:33.195118+0000","last_undegraded":"2024-09-10T10:05:33.195118+0000","last_fullsized":"2024-09-10T10:05:33.195118+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:02.511972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369913+0000","last_change":"2024-09-10T10:05:32.199610+0000","last_active":"2024-09-10T10:05:35.369913+0000","last_peered":"2024-09-10T10:05:35.369913+0000","last_clean":"2024-09-10T10:05:35.369913+0000","last_became_active":"2024-09-10T10:05:32.199339+0000","last_became_peered":"2024-09-10T10:05:32.199339+0000","last_unstale":"2024-09-10T10:05:35.369913+0000","last_undegraded":"2024-09-10T10:05:35.369913+0000","last_fullsized":"2024-09-10T10:05:35.369913+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:31.653256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.369395+0000","last_change":"2024-09-10T10:05:31.179485+0000","last_active":"2024-09-10T10:05:35.369395+0000","last_peered":"2024-09-10T10:05:35.369395+0000","last_clean":"2024-09-10T10:05:35.369395+0000","last_became_active":"2024-09-10T10:05:31.178050+0000","last_became_peered":"2024-09-10T10:05:31.178050+0000","last_unstale":"2024-09-10T10:05:35.369395+0000","last_undegraded":"2024-09-10T10:05:35.369395+0000","last_fullsized":"2024-09-10T10:05:35.369395+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:54.607588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370973+0000","last_change":"2024-09-10T10:05:32.203836+0000","last_active":"2024-09-10T10:05:35.370973+0000","last_peered":"2024-09-10T10:05:35.370973+0000","last_clean":"2024-09-10T10:05:35.370973+0000","last_became_active":"2024-09-10T10:05:32.203166+0000","last_became_peered":"2024-09-10T10:05:32.203166+0000","last_unstale":"2024-09-10T10:05:35.370973+0000","last_undegraded":"2024-09-10T10:05:35.370973+0000","last_fullsized":"2024-09-10T10:05:35.370973+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:43.104902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386182+0000","last_change":"2024-09-10T10:05:31.193385+0000","last_active":"2024-09-10T10:05:34.386182+0000","last_peered":"2024-09-10T10:05:34.386182+0000","last_clean":"2024-09-10T10:05:34.386182+0000","last_became_active":"2024-09-10T10:05:31.193085+0000","last_became_peered":"2024-09-10T10:05:31.193085+0000","last_unstale":"2024-09-10T10:05:34.386182+0000","last_undegraded":"2024-09-10T10:05:34.386182+0000","last_fullsized":"2024-09-10T10:05:34.386182+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:10.299735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.371137+0000","last_change":"2024-09-10T10:05:32.211776+0000","last_active":"2024-09-10T10:05:35.371137+0000","last_peered":"2024-09-10T10:05:35.371137+0000","last_clean":"2024-09-10T10:05:35.371137+0000","last_became_active":"2024-09-10T10:05:32.210887+0000","last_became_peered":"2024-09-10T10:05:32.210887+0000","last_unstale":"2024-09-10T10:05:35.371137+0000","last_undegraded":"2024-09-10T10:05:35.371137+0000","last_fullsized":"2024-09-10T10:05:35.371137+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:47.203644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.386316+0000","last_change":"2024-09-10T10:05:31.186964+0000","last_active":"2024-09-10T10:05:34.386316+0000","last_peered":"2024-09-10T10:05:34.386316+0000","last_clean":"2024-09-10T10:05:34.386316+0000","last_became_active":"2024-09-10T10:05:31.186613+0000","last_became_peered":"2024-09-10T10:05:31.186613+0000","last_unstale":"2024-09-10T10:05:34.386316+0000","last_undegraded":"2024-09-10T10:05:34.386316+0000","last_fullsized":"2024-09-10T10:05:34.386316+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:28.486954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.197580+0000","last_change":"2024-09-10T10:05:32.216197+0000","last_active":"2024-09-10T10:05:33.197580+0000","last_peered":"2024-09-10T10:05:33.197580+0000","last_clean":"2024-09-10T10:05:33.197580+0000","last_became_active":"2024-09-10T10:05:32.215922+0000","last_became_peered":"2024-09-10T10:05:32.215922+0000","last_unstale":"2024-09-10T10:05:33.197580+0000","last_undegraded":"2024-09-10T10:05:33.197580+0000","last_fullsized":"2024-09-10T10:05:33.197580+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:51.678475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.198145+0000","last_change":"2024-09-10T10:05:31.187417+0000","last_active":"2024-09-10T10:05:33.198145+0000","last_peered":"2024-09-10T10:05:33.198145+0000","last_clean":"2024-09-10T10:05:33.198145+0000","last_became_active":"2024-09-10T10:05:31.187030+0000","last_became_peered":"2024-09-10T10:05:31.187030+0000","last_unstale":"2024-09-10T10:05:33.198145+0000","last_undegraded":"2024-09-10T10:05:33.198145+0000","last_fullsized":"2024-09-10T10:05:33.198145+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:48.710671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:34.519141+0000","last_change":"2024-09-10T10:05:32.193628+0000","last_active":"2024-09-10T10:05:34.519141+0000","last_peered":"2024-09-10T10:05:34.519141+0000","last_clean":"2024-09-10T10:05:34.519141+0000","last_became_active":"2024-09-10T10:05:32.193327+0000","last_became_peered":"2024-09-10T10:05:32.193327+0000","last_unstale":"2024-09-10T10:05:34.519141+0000","last_undegraded":"2024-09-10T10:05:34.519141+0000","last_fullsized":"2024-09-10T10:05:34.519141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.619372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.202360+0000","last_change":"2024-09-10T10:05:31.194388+0000","last_active":"2024-09-10T10:05:33.202360+0000","last_peered":"2024-09-10T10:05:33.202360+0000","last_clean":"2024-09-10T10:05:33.202360+0000","last_became_active":"2024-09-10T10:05:31.193967+0000","last_became_peered":"2024-09-10T10:05:31.193967+0000","last_unstale":"2024-09-10T10:05:33.202360+0000","last_undegraded":"2024-09-10T10:05:33.202360+0000","last_fullsized":"2024-09-10T10:05:33.202360+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:34.854768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:33.194489+0000","last_change":"2024-09-10T10:05:32.186936+0000","last_active":"2024-09-10T10:05:33.194489+0000","last_peered":"2024-09-10T10:05:33.194489+0000","last_clean":"2024-09-10T10:05:33.194489+0000","last_became_active":"2024-09-10T10:05:32.186582+0000","last_became_peered":"2024-09-10T10:05:32.186582+0000","last_unstale":"2024-09-10T10:05:33.194489+0000","last_undegraded":"2024-09-10T10:05:33.194489+0000","last_fullsized":"2024-09-10T10:05:33.194489+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:31.158515+0000","last_clean_scrub_stamp":"2024-09-10T10:05:31.158515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:47.849004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370868+0000","last_change":"2024-09-10T10:05:31.191205+0000","last_active":"2024-09-10T10:05:35.370868+0000","last_peered":"2024-09-10T10:05:35.370868+0000","last_clean":"2024-09-10T10:05:35.370868+0000","last_became_active":"2024-09-10T10:05:31.190885+0000","last_became_peered":"2024-09-10T10:05:31.190885+0000","last_unstale":"2024-09-10T10:05:35.370868+0000","last_undegraded":"2024-09-10T10:05:35.370868+0000","last_fullsized":"2024-09-10T10:05:35.370868+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:05:30.151898+0000","last_clean_scrub_stamp":"2024-09-10T10:05:30.151898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:18.985524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":448,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T10:05:35.370902+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:05:35.370902+0000","last_peered":"2024-09-10T10:05:35.370902+0000","last_clean":"2024-09-10T10:05:35.370902+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:05:35.370902+0000","last_undegraded":"2024-09-10T10:05:35.370902+0000","last_fullsized":"2024-09-10T10:05:35.370902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2523136,"data_stored":1438395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1921024,"data_stored":847923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1921024,"data_stored":847923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":850117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542237,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1933312,"data_stored":848117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1933312,"data_stored":848617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1933312,"data_stored":849535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509532,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37812,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334028,"statfs":{"total":96636764160,"available":96598044672,"internally_reserved":0,"allocated":2519040,"data_stored":1438313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:05:41.248 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:05:41.248 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:05:41.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c21600640 1 Processor -- start 2024-09-10T10:05:41.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c21600640 1 -- start start 2024-09-10T10:05:41.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c21600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3c1c15b170 0x7f3c1c073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c21600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c0cf2e0 0x7f3c1c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c21600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1c0b3b40 con 0x7f3c1c15ed40 2024-09-10T10:05:41.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c21600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3c1c0b3910 con 0x7f3c1c15b170 2024-09-10T10:05:41.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c21600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c0cfc60 con 0x7f3c1c0cf2e0 2024-09-10T10:05:41.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c1be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3c1c15b170 0x7f3c1c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c1be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3c1c15b170 0x7f3c1c073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39772/0 (socket says 172.21.15.110:39772) 2024-09-10T10:05:41.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.389+0000 7f3c1be00640 1 -- 172.21.15.110:0/1997021253 learned_addr learned my addr 172.21.15.110:0/1997021253 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:41.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c13400640 1 --2- 172.21.15.110:0/1997021253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c0cf2e0 0x7f3c1c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c1be00640 1 -- 172.21.15.110:0/1997021253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c0cf2e0 msgr2=0x7f3c1c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:41.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c1be00640 1 --2- 172.21.15.110:0/1997021253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c0cf2e0 0x7f3c1c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c1be00640 1 -- 172.21.15.110:0/1997021253 >> v1:172.21.15.110:6789/0 conn(0x7f3c1c15ed40 legacy=0x7f3c1c15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c1be00640 1 -- 172.21.15.110:0/1997021253 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c0b3e20 con 0x7f3c1c15b170 2024-09-10T10:05:41.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c13400640 1 --2- 172.21.15.110:0/1997021253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c0cf2e0 0x7f3c1c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:41.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c1be00640 1 --2- 172.21.15.110:0/1997021253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15b170 0x7f3c1c073390 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00bb10 tx=0x7f3c0c01ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8981d5532fe05c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:41.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c1b400640 1 -- 172.21.15.110:0/1997021253 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0b4070 con 0x7f3c1c15b170 2024-09-10T10:05:41.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c1b400640 1 -- 172.21.15.110:0/1997021253 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0a7bd0 con 0x7f3c1c15b170 2024-09-10T10:05:41.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.393+0000 7f3c1b400640 1 -- 172.21.15.110:0/1997021253 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0af440 con 0x7f3c1c15b170 2024-09-10T10:05:41.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- 172.21.15.110:0/1997021253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15b170 msgr2=0x7f3c1c073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 --2- 172.21.15.110:0/1997021253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15b170 0x7f3c1c073390 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00bb10 tx=0x7f3c0c01ec00 comp rx=0 tx=0).stop 2024-09-10T10:05:41.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- 172.21.15.110:0/1997021253 shutdown_connections 2024-09-10T10:05:41.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 --2- 172.21.15.110:0/1997021253 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c0cf2e0 0x7f3c1c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 --2- 172.21.15.110:0/1997021253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15b170 0x7f3c1c073390 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- 172.21.15.110:0/1997021253 >> 172.21.15.110:0/1997021253 conn(0x7f3c1c06d0f0 msgr2=0x7f3c1c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:41.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- 172.21.15.110:0/1997021253 shutdown_connections 2024-09-10T10:05:41.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- 172.21.15.110:0/1997021253 wait complete. 2024-09-10T10:05:41.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 Processor -- start 2024-09-10T10:05:41.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- start start 2024-09-10T10:05:41.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c1c0cf2e0 0x7f3c1c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c15b170 0x7f3c1c1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15ed40 0x7f3c1c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c072680 con 0x7f3c1c15ed40 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c15b550 con 0x7f3c1c0cf2e0 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c21600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c072800 con 0x7f3c1c15b170 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15ed40 0x7f3c1c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15ed40 0x7f3c1c15a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39774/0 (socket says 172.21.15.110:39774) 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 -- 172.21.15.110:0/3903256037 learned_addr learned my addr 172.21.15.110:0/3903256037 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c13400640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c15b170 0x7f3c1c1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 -- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c15b170 msgr2=0x7f3c1c1f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c15b170 0x7f3c1c1f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 -- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c1c0cf2e0 msgr2=0x7f3c1c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c1c0cf2e0 0x7f3c1c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 -- 172.21.15.110:0/3903256037 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c141980 con 0x7f3c1c15ed40 2024-09-10T10:05:41.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c13400640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c15b170 0x7f3c1c1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:41.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.397+0000 7f3c20c00640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15ed40 0x7f3c1c15a0e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400c730 tx=0x7f3c1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:41.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c18c00640 1 -- 172.21.15.110:0/3903256037 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14003b80 con 0x7f3c1c15ed40 2024-09-10T10:05:41.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c18c00640 1 -- 172.21.15.110:0/3903256037 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c14003d20 con 0x7f3c1c15ed40 2024-09-10T10:05:41.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c18c00640 1 -- 172.21.15.110:0/3903256037 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400aa30 con 0x7f3c1c15ed40 2024-09-10T10:05:41.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c141c60 con 0x7f3c1c15ed40 2024-09-10T10:05:41.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c142220 con 0x7f3c1c15ed40 2024-09-10T10:05:41.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c0cf720 con 0x7f3c1c15ed40 2024-09-10T10:05:41.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c18c00640 1 -- 172.21.15.110:0/3903256037 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f3c1400abd0 con 0x7f3c1c15ed40 2024-09-10T10:05:41.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c18c00640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bf80680a0 0x7f3bf806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c18c00640 1 -- 172.21.15.110:0/3903256037 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f3c14086dd0 con 0x7f3c1c15ed40 2024-09-10T10:05:41.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.401+0000 7f3c1be00640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bf80680a0 0x7f3bf806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.405+0000 7f3c1be00640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bf80680a0 0x7f3bf806a560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0b1000 tx=0x7f3c0c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:41.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.405+0000 7f3c18c00640 1 -- 172.21.15.110:0/3903256037 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3c14056940 con 0x7f3c1c15ed40 2024-09-10T10:05:41.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.621+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3c1c165840 con 0x7f3c1c15ed40 2024-09-10T10:05:41.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c18c00640 1 -- 172.21.15.110:0/3903256037 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f3c14056240 con 0x7f3c1c15ed40 2024-09-10T10:05:41.626 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:05:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:05:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:05:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:05:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:05:41.626 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bf80680a0 msgr2=0x7f3bf806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bf80680a0 0x7f3bf806a560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0b1000 tx=0x7f3c0c004740 comp rx=0 tx=0).stop 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15ed40 msgr2=0x7f3c1c15a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15ed40 0x7f3c1c15a0e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400c730 tx=0x7f3c1400cbf0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 shutdown_connections 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3bf80680a0 0x7f3bf806a560 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c1c15ed40 0x7f3c1c15a0e0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3c1c15b170 0x7f3c1c1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 --2- 172.21.15.110:0/3903256037 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c1c0cf2e0 0x7f3c1c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 >> 172.21.15.110:0/3903256037 conn(0x7f3c1c06d0f0 msgr2=0x7f3c1c0b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:41.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 shutdown_connections 2024-09-10T10:05:41.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.625+0000 7f3c21600640 1 -- 172.21.15.110:0/3903256037 wait complete. 2024-09-10T10:05:41.647 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f6765400640 1 Processor -- start 2024-09-10T10:05:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f6765400640 1 -- start start 2024-09-10T10:05:41.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f6765400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f6760163000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f6765400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6760073290 0x7f6760073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f6765400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67600706a0 con 0x7f676006eb30 2024-09-10T10:05:41.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f6765400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f676015ee90 con 0x7f6760073290 2024-09-10T10:05:41.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f6765400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f67600710d0 con 0x7f67600d0cb0 2024-09-10T10:05:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f675de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f6760163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f675de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f6760163000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:52556/0 (socket says 172.21.15.110:52556) 2024-09-10T10:05:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f675de00640 1 -- 172.21.15.110:0/1164161317 learned_addr learned my addr 172.21.15.110:0/1164161317 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f675f200640 1 --2- 172.21.15.110:0/1164161317 >> v2:172.21.15.145:3300/0 conn(0x7f6760073290 0x7f6760073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f675de00640 1 -- 172.21.15.110:0/1164161317 >> v2:172.21.15.145:3300/0 conn(0x7f6760073290 msgr2=0x7f6760073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f675de00640 1 --2- 172.21.15.110:0/1164161317 >> v2:172.21.15.145:3300/0 conn(0x7f6760073290 0x7f6760073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f675de00640 1 -- 172.21.15.110:0/1164161317 >> v1:172.21.15.145:6789/0 conn(0x7f676006eb30 legacy=0x7f6760071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.781+0000 7f675de00640 1 -- 172.21.15.110:0/1164161317 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67600d10f0 con 0x7f67600d0cb0 2024-09-10T10:05:41.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f675de00640 1 --2- 172.21.15.110:0/1164161317 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f6760163000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f674c009870 tx=0x7f674c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47239200c3ffc536 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:41.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f675d400640 1 -- 172.21.15.110:0/1164161317 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0b0070 con 0x7f67600d0cb0 2024-09-10T10:05:41.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f675d400640 1 -- 172.21.15.110:0/1164161317 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674c0ab600 con 0x7f67600d0cb0 2024-09-10T10:05:41.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f675d400640 1 -- 172.21.15.110:0/1164161317 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0ab9d0 con 0x7f67600d0cb0 2024-09-10T10:05:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f6765400640 1 -- 172.21.15.110:0/1164161317 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 msgr2=0x7f6760163000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f6765400640 1 --2- 172.21.15.110:0/1164161317 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f6760163000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f674c009870 tx=0x7f674c01c4d0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f6765400640 1 -- 172.21.15.110:0/1164161317 shutdown_connections 2024-09-10T10:05:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f6765400640 1 --2- 172.21.15.110:0/1164161317 >> v2:172.21.15.145:3300/0 conn(0x7f6760073290 0x7f6760073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f6765400640 1 --2- 172.21.15.110:0/1164161317 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f6760163000 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f6765400640 1 -- 172.21.15.110:0/1164161317 >> 172.21.15.110:0/1164161317 conn(0x7f67600d25c0 msgr2=0x7f67600d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:41.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f6765400640 1 -- 172.21.15.110:0/1164161317 shutdown_connections 2024-09-10T10:05:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.785+0000 7f6765400640 1 -- 172.21.15.110:0/1164161317 wait complete. 2024-09-10T10:05:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 Processor -- start 2024-09-10T10:05:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 -- start start 2024-09-10T10:05:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f676006eb30 0x7f6760155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6760073290 0x7f67600cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f676015b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6760070d60 con 0x7f6760073290 2024-09-10T10:05:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6760071c70 con 0x7f676006eb30 2024-09-10T10:05:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f676006dd60 con 0x7f67600d0cb0 2024-09-10T10:05:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6760073290 0x7f67600cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6760073290 0x7f67600cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39786/0 (socket says 172.21.15.110:39786) 2024-09-10T10:05:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 -- 172.21.15.110:0/1614828776 learned_addr learned my addr 172.21.15.110:0/1614828776 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:41.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675e800640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f676006eb30 0x7f6760155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675f200640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f676015b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 -- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 msgr2=0x7f676015b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f676015b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 -- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f676006eb30 msgr2=0x7f6760155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f676006eb30 0x7f6760155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 -- 172.21.15.110:0/1614828776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67601561e0 con 0x7f6760073290 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675f200640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f676015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f675de00640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6760073290 0x7f67600cbed0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f674c0ae040 tx=0x7f674c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6756a00640 1 -- 172.21.15.110:0/1614828776 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0ba020 con 0x7f6760073290 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6756a00640 1 -- 172.21.15.110:0/1614828776 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674c0b6420 con 0x7f6760073290 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67600ce400 con 0x7f6760073290 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6756a00640 1 -- 172.21.15.110:0/1614828776 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0b0040 con 0x7f6760073290 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.789+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67600ce9c0 con 0x7f6760073290 2024-09-10T10:05:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.793+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6760073740 con 0x7f6760073290 2024-09-10T10:05:41.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.793+0000 7f6756a00640 1 -- 172.21.15.110:0/1614828776 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f674c0bd020 con 0x7f6760073290 2024-09-10T10:05:41.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.793+0000 7f6756a00640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f67400680a0 0x7f674006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:41.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.793+0000 7f6756a00640 1 -- 172.21.15.110:0/1614828776 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f674c120e30 con 0x7f6760073290 2024-09-10T10:05:41.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.793+0000 7f675e800640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f67400680a0 0x7f674006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:41.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.793+0000 7f675e800640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f67400680a0 0x7f674006a560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6750001d50 tx=0x7f6750009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:41.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.797+0000 7f6756a00640 1 -- 172.21.15.110:0/1614828776 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f674c0c4030 con 0x7f6760073290 2024-09-10T10:05:41.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.985+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67600b5fb0 con 0x7f6760073290 2024-09-10T10:05:41.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.985+0000 7f6756a00640 1 -- 172.21.15.110:0/1614828776 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5245 (secure 0 0 0) 0x7f67601561e0 con 0x7f6760073290 2024-09-10T10:05:41.987 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:41.987 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":106,"btime":"2024-09-10T10:05:35:199222+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3244721906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3244721906},{"type":"v1","addr":"172.21.15.145:6834","nonce":3244721906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/1342717567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1342717567},{"type":"v1","addr":"172.21.15.110:6834","nonce":1342717567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1495976904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1495976904},{"type":"v1","addr":"172.21.15.110:6835","nonce":1495976904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:31.551366+0000","modified":"2024-09-10T10:05:34.580103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1869854747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1869854747},{"type":"v1","addr":"172.21.15.145:6835","nonce":1869854747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":12}]} 2024-09-10T10:05:41.987 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 106 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f67400680a0 msgr2=0x7f674006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f67400680a0 0x7f674006a560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6750001d50 tx=0x7f6750009040 comp rx=0 tx=0).stop 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6760073290 msgr2=0x7f67600cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6760073290 0x7f67600cbed0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f674c0ae040 tx=0x7f674c01ad10 comp rx=0 tx=0).stop 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 shutdown_connections 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f67400680a0 0x7f674006a560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f67600d0cb0 0x7f676015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6760073290 0x7f67600cbed0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 --2- 172.21.15.110:0/1614828776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f676006eb30 0x7f6760155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 >> 172.21.15.110:0/1614828776 conn(0x7f67600d25c0 msgr2=0x7f67600cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 shutdown_connections 2024-09-10T10:05:41.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:41.989+0000 7f6765400640 1 -- 172.21.15.110:0/1614828776 wait complete. 2024-09-10T10:05:42.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:05:31.551366+0000', 'modified': '2024-09-10T10:05:34.580103+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8526': {'gid': 8526, 'name': 'b', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.145:6835/1869854747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6833', 'nonce': 1869854747}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 1869854747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8526, 'qdb_cluster': [8526]} 2024-09-10T10:05:42.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:05:42.411 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T10:05:42.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.545+0000 7f97fa800640 1 Processor -- start 2024-09-10T10:05:42.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.545+0000 7f97fa800640 1 -- start start 2024-09-10T10:05:42.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97fa800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f97f40726e0 0x7f97f40cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97fa800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40b5600 0x7f97f40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97fa800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f406f410 con 0x7f97f406efa0 2024-09-10T10:05:42.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97fa800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f97f40cee50 con 0x7f97f40726e0 2024-09-10T10:05:42.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97fa800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f97f415fda0 con 0x7f97f40b5600 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f3400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f97f40726e0 0x7f97f40cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f2a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f97f406efa0 0x7f97f40b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:45202/0 (socket says 172.21.15.110:45202) 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40b5600 0x7f97f40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f2a00640 1 -- 172.21.15.110:0/333540610 learned_addr learned my addr 172.21.15.110:0/333540610 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f3400640 1 -- 172.21.15.110:0/333540610 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40b5600 msgr2=0x7f97f40b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f3400640 1 --2- 172.21.15.110:0/333540610 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40b5600 0x7f97f40b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f3400640 1 -- 172.21.15.110:0/333540610 >> v1:172.21.15.145:6789/0 conn(0x7f97f406efa0 legacy=0x7f97f40b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f3400640 1 -- 172.21.15.110:0/333540610 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f40ce520 con 0x7f97f40726e0 2024-09-10T10:05:42.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.549+0000 7f97f2000640 1 -- 172.21.15.110:0/333540610 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 738864369 0 0) 0x7f97e0001120 con 0x7f97f406efa0 2024-09-10T10:05:42.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97f3400640 1 --2- 172.21.15.110:0/333540610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40726e0 0x7f97f40cf2e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009d90 tx=0x7f97e401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3614375ffa596e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97f2000640 1 -- 172.21.15.110:0/333540610 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40b2070 con 0x7f97f40726e0 2024-09-10T10:05:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97f2000640 1 -- 172.21.15.110:0/333540610 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e40a5a00 con 0x7f97f40726e0 2024-09-10T10:05:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97f2000640 1 -- 172.21.15.110:0/333540610 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40a5dd0 con 0x7f97f40726e0 2024-09-10T10:05:42.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 -- 172.21.15.110:0/333540610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40726e0 msgr2=0x7f97f40cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 --2- 172.21.15.110:0/333540610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40726e0 0x7f97f40cf2e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009d90 tx=0x7f97e401c940 comp rx=0 tx=0).stop 2024-09-10T10:05:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 -- 172.21.15.110:0/333540610 shutdown_connections 2024-09-10T10:05:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 --2- 172.21.15.110:0/333540610 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40b5600 0x7f97f40b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 --2- 172.21.15.110:0/333540610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40726e0 0x7f97f40cf2e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 -- 172.21.15.110:0/333540610 >> 172.21.15.110:0/333540610 conn(0x7f97f4062d10 msgr2=0x7f97f406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 -- 172.21.15.110:0/333540610 shutdown_connections 2024-09-10T10:05:42.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 -- 172.21.15.110:0/333540610 wait complete. 2024-09-10T10:05:42.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.553+0000 7f97fa800640 1 Processor -- start 2024-09-10T10:05:42.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 -- start start 2024-09-10T10:05:42.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f406efa0 0x7f97f4151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40726e0 0x7f97f41512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40b5600 0x7f97f40ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4072c40 con 0x7f97f406efa0 2024-09-10T10:05:42.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4072ac0 con 0x7f97f40b5600 2024-09-10T10:05:42.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f97f415f950 con 0x7f97f40726e0 2024-09-10T10:05:42.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f406efa0 0x7f97f4151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f406efa0 0x7f97f4151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39794/0 (socket says 172.21.15.110:39794) 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 -- 172.21.15.110:0/1215462966 learned_addr learned my addr 172.21.15.110:0/1215462966 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f2a00640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40726e0 0x7f97f41512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3e00640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40b5600 0x7f97f40ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 -- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40726e0 msgr2=0x7f97f41512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40726e0 0x7f97f41512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 -- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40b5600 msgr2=0x7f97f40ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40b5600 0x7f97f40ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 -- 172.21.15.110:0/1215462966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f4151a60 con 0x7f97f406efa0 2024-09-10T10:05:42.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97f3400640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f406efa0 0x7f97f4151d40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f97e40b0040 tx=0x7f97e40196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97eb400640 1 -- 172.21.15.110:0/1215462966 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40b2050 con 0x7f97f406efa0 2024-09-10T10:05:42.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97eb400640 1 -- 172.21.15.110:0/1215462966 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e40be070 con 0x7f97f406efa0 2024-09-10T10:05:42.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 -- 172.21.15.110:0/1215462966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f4153b90 con 0x7f97f406efa0 2024-09-10T10:05:42.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97eb400640 1 -- 172.21.15.110:0/1215462966 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40adb90 con 0x7f97f406efa0 2024-09-10T10:05:42.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.557+0000 7f97fa800640 1 -- 172.21.15.110:0/1215462966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f4154150 con 0x7f97f406efa0 2024-09-10T10:05:42.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97eb400640 1 -- 172.21.15.110:0/1215462966 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f97e40c3460 con 0x7f97f406efa0 2024-09-10T10:05:42.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97eb400640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f97d40680a0 0x7f97d406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97eb400640 1 -- 172.21.15.110:0/1215462966 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f97e411fa50 con 0x7f97f406efa0 2024-09-10T10:05:42.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97fdb03000 1 Processor -- start 2024-09-10T10:05:42.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97fdb03000 1 -- start start 2024-09-10T10:05:42.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f97d40680a0 0x7f97d406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 0x558697c8ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c90440 0x558697c45d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c46290 0x558697c46670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f97d40680a0 0x7f97d406a560 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97f4072370 tx=0x7f97e0007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97fdb03000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558697b2db60 con 0x558697c90440 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97fdb03000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558697a94c70 con 0x558697c46290 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97fdb03000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x558697a3aed0 con 0x558697c8fb20 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c90440 0x558697c45d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f3400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 0x558697c8ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c90440 0x558697c45d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39810/0 (socket says 172.21.15.110:39810) 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c46290 0x558697c46670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 -- 172.21.15.110:0/3743416479 learned_addr learned my addr 172.21.15.110:0/3743416479 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 -- 172.21.15.110:0/3743416479 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 msgr2=0x558697c8ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 --2- 172.21.15.110:0/3743416479 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 0x558697c8ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 -- 172.21.15.110:0/3743416479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c46290 msgr2=0x558697c46670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 --2- 172.21.15.110:0/3743416479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c46290 0x558697c46670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.561+0000 7f97f2a00640 1 -- 172.21.15.110:0/3743416479 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558697a4ce50 con 0x558697c90440 2024-09-10T10:05:42.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f2a00640 1 --2- 172.21.15.110:0/3743416479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c90440 0x558697c45d50 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005d90 tx=0x7f97e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284a6b32ba5e80b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97dfe00640 1 -- 172.21.15.110:0/3743416479 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0015110 con 0x558697c90440 2024-09-10T10:05:42.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97dfe00640 1 -- 172.21.15.110:0/3743416479 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e00159c0 con 0x558697c90440 2024-09-10T10:05:42.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97dfe00640 1 -- 172.21.15.110:0/3743416479 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0015d70 con 0x558697c90440 2024-09-10T10:05:42.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- 172.21.15.110:0/3743416479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c90440 msgr2=0x558697c45d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 --2- 172.21.15.110:0/3743416479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c90440 0x558697c45d50 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005d90 tx=0x7f97e0012000 comp rx=0 tx=0).stop 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- 172.21.15.110:0/3743416479 shutdown_connections 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 --2- 172.21.15.110:0/3743416479 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c46290 0x558697c46670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 --2- 172.21.15.110:0/3743416479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c90440 0x558697c45d50 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 --2- 172.21.15.110:0/3743416479 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 0x558697c8ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- 172.21.15.110:0/3743416479 >> 172.21.15.110:0/3743416479 conn(0x558697c4c600 msgr2=0x558697c44630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- 172.21.15.110:0/3743416479 shutdown_connections 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- 172.21.15.110:0/3743416479 wait complete. 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- start start 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 Processor -- start 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697c4ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697c50cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697c51230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558697af6b00 con 0x558697c8fb20 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558697a6f360 con 0x558697c90440 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x558697a6c2d0 con 0x558697c4b390 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97fdb03000 10 client.0 _ll_register_callbacks cb 0x558697c31820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697c4ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697c4ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39818/0 (socket says 172.21.15.110:39818) 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 -- 172.21.15.110:0/935775838 learned_addr learned my addr 172.21.15.110:0/935775838 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3e00640 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697c51230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f2a00640 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697c50cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 -- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 msgr2=0x558697c51230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697c51230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 -- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 msgr2=0x558697c50cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697c50cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 -- 172.21.15.110:0/935775838 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558697a3b120 con 0x558697c8fb20 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97f3400640 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697c4ae50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f97e4098520 tx=0x7f97e40f1e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b5c67acd1d59ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.565+0000 7f97dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 1 -- 172.21.15.110:0/935775838 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40fa050 con 0x558697c8fb20 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 1 -- 172.21.15.110:0/935775838 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f97e4105b90 con 0x558697c8fb20 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 -- 172.21.15.110:0/935775838 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e00159c0 con 0x558697c8fb20 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 1 -- 172.21.15.110:0/935775838 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f97e40fa2e0 con 0x558697c8fb20 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 10 client.8658 fetch_fsmap learned FSMap version 106 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 -- 172.21.15.110:0/935775838 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x558697a34450 con 0x558697c8fb20 2024-09-10T10:05:42.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 1 -- 172.21.15.110:0/935775838 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e00159c0 con 0x558697c8fb20 2024-09-10T10:05:42.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 1 -- 172.21.15.110:0/935775838 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 106) ==== 3502+0+0 (secure 0 0 0) 0x7f97e40f0b50 con 0x558697c8fb20 2024-09-10T10:05:42.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 10 client.8658 fetch_fsmap finished waiting for FSMap version 106 2024-09-10T10:05:42.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 10 client.8658 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:05:42.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 20 client.8658 populate_metadata read hostname 'smithi110' 2024-09-10T10:05:42.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] conn(0x558697c9e280 0x558697ca0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 4 client.8658 mds_command: new command op to 8526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:05:42.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 -- 172.21.15.110:0/935775838 --> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558697a2ec00 con 0x558697c9e280 2024-09-10T10:05:42.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97f2a00640 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] conn(0x558697c9e280 0x558697ca0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97f2a00640 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] conn(0x558697c9e280 0x558697ca0670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7750b038c7b5698 server_cookie=f4e43a0513ad21c7 in_seq=0 out_seq=0 2024-09-10T10:05:42.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 10 client.8658 ms_handle_connect on v2:172.21.15.145:6833/1869854747 2024-09-10T10:05:42.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 1 -- 172.21.15.110:0/935775838 <== mds.0 v2:172.21.15.145:6833/1869854747 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558697a2ec00 con 0x558697c9e280 2024-09-10T10:05:42.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 10 client.8658 handle_command_reply: tid=0 2024-09-10T10:05:42.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 client.8658 shutdown 2024-09-10T10:05:42.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 -- 172.21.15.110:0/935775838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 msgr2=0x558697c4ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697c4ae50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f97e4098520 tx=0x7f97e40f1e00 comp rx=0 tx=0).stop 2024-09-10T10:05:42.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 -- 172.21.15.110:0/935775838 shutdown_connections 2024-09-10T10:05:42.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] conn(0x558697c9e280 0x558697ca0670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697c51230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697c50cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97dcc00640 0 client.8658 ms_handle_reset on v2:172.21.15.145:6833/1869854747 2024-09-10T10:05:42.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 --2- 172.21.15.110:0/935775838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697c4ae50 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 -- 172.21.15.110:0/935775838 >> 172.21.15.110:0/935775838 conn(0x558697c46290 msgr2=0x558697c44d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:42.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 -- 172.21.15.110:0/935775838 shutdown_connections 2024-09-10T10:05:42.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 1 -- 172.21.15.110:0/935775838 wait complete. 2024-09-10T10:05:42.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.569+0000 7f97fdb03000 20 client.8658 trim_cache size 0 max 16384 2024-09-10T10:05:42.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97fdb03000 1 Processor -- start 2024-09-10T10:05:42.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97fdb03000 1 -- start start 2024-09-10T10:05:42.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c4b390 0x558697c8ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 0x558697c8f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697ca0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97fdb03000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4202930 con 0x558697c4b390 2024-09-10T10:05:42.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97fdb03000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41550a0 con 0x558697c90440 2024-09-10T10:05:42.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97fdb03000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f97f41556f0 con 0x558697c8fb20 2024-09-10T10:05:42.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c4b390 0x558697c8ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c4b390 0x558697c8ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39828/0 (socket says 172.21.15.110:39828) 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 -- 172.21.15.110:0/1710021071 learned_addr learned my addr 172.21.15.110:0/1710021071 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3e00640 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697ca0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f2a00640 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 0x558697c8f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 -- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 msgr2=0x558697c8f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 0x558697c8f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 -- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 msgr2=0x558697ca0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697ca0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 -- 172.21.15.110:0/1710021071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558697a1bfd0 con 0x558697c4b390 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3e00640 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697ca0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97f3400640 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c4b390 0x558697c8ecf0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f97e401ce20 tx=0x7f97e40f3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fd4fc6730f1b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97df400640 1 -- 172.21.15.110:0/1710021071 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4105b90 con 0x558697c4b390 2024-09-10T10:05:42.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97df400640 1 -- 172.21.15.110:0/1710021071 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e40b3040 con 0x558697c4b390 2024-09-10T10:05:42.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.589+0000 7f97df400640 1 -- 172.21.15.110:0/1710021071 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4105de0 con 0x558697c4b390 2024-09-10T10:05:42.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- 172.21.15.110:0/1710021071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c4b390 msgr2=0x558697c8ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c4b390 0x558697c8ecf0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f97e401ce20 tx=0x7f97e40f3000 comp rx=0 tx=0).stop 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- 172.21.15.110:0/1710021071 shutdown_connections 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697ca0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c8fb20 0x558697c8f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 --2- 172.21.15.110:0/1710021071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c4b390 0x558697c8ecf0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- 172.21.15.110:0/1710021071 >> 172.21.15.110:0/1710021071 conn(0x558697c46290 msgr2=0x558697c4a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- 172.21.15.110:0/1710021071 shutdown_connections 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- 172.21.15.110:0/1710021071 wait complete. 2024-09-10T10:05:42.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- start start 2024-09-10T10:05:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 Processor -- start 2024-09-10T10:05:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697ca3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697cbe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697cbeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558697a93f80 con 0x558697c8fb20 2024-09-10T10:05:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558697a4cca0 con 0x558697c90440 2024-09-10T10:05:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x558697a992a0 con 0x558697c4b390 2024-09-10T10:05:42.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 10 client.0 _ll_register_callbacks cb 0x7f97e40f1240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697cbe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697cbe960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39842/0 (socket says 172.21.15.110:39842) 2024-09-10T10:05:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 -- 172.21.15.110:0/2102753624 learned_addr learned my addr 172.21.15.110:0/2102753624 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f3400640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697ca3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 -- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 msgr2=0x558697ca3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f3e00640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697cbeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697ca3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 -- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 msgr2=0x558697cbeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697cbeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 -- 172.21.15.110:0/2102753624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558697b17750 con 0x558697c8fb20 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f3400640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697ca3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f3e00640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697cbeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97f2a00640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697cbe960 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x558697b4cac0 tx=0x7f97e0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c486494b0655eac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97dcc00640 1 -- 172.21.15.110:0/2102753624 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0007da0 con 0x558697c8fb20 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97dcc00640 1 -- 172.21.15.110:0/2102753624 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f97e0010040 con 0x558697c8fb20 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- 172.21.15.110:0/2102753624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e40b3040 con 0x558697c8fb20 2024-09-10T10:05:42.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97dcc00640 1 -- 172.21.15.110:0/2102753624 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f97e003b050 con 0x558697c8fb20 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 10 client.8664 fetch_fsmap learned FSMap version 106 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- 172.21.15.110:0/2102753624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x558697a22a30 con 0x558697c8fb20 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97dcc00640 1 -- 172.21.15.110:0/2102753624 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e40b3040 con 0x558697c8fb20 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97dcc00640 1 -- 172.21.15.110:0/2102753624 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 106) ==== 3502+0+0 (secure 0 0 0) 0x7f97e003e020 con 0x558697c8fb20 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 10 client.8664 fetch_fsmap finished waiting for FSMap version 106 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 10 client.8664 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 20 client.8664 populate_metadata read hostname 'smithi110' 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] conn(0x558697cbf3e0 0x558697d02cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 4 client.8664 mds_command: new command op to 8526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.593+0000 7f97fdb03000 1 -- 172.21.15.110:0/2102753624 --> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] -- command(tid 0: {"prefix": "status"}) -- 0x558697aa23a0 con 0x558697cbf3e0 2024-09-10T10:05:42.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97f3400640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] conn(0x558697cbf3e0 0x558697d02cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97f3400640 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] conn(0x558697cbf3e0 0x558697d02cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69022a6a76893cb server_cookie=5300556a29cf4a60 in_seq=0 out_seq=0 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97dcc00640 10 client.8664 ms_handle_connect on v2:172.21.15.145:6833/1869854747 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97dcc00640 1 -- 172.21.15.110:0/2102753624 <== mds.0 v2:172.21.15.145:6833/1869854747 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x558697aa23a0 con 0x558697cbf3e0 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97dcc00640 10 client.8664 handle_command_reply: tid=0 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 client.8664 shutdown 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 -- 172.21.15.110:0/2102753624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 msgr2=0x558697cbe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697cbe960 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x558697b4cac0 tx=0x7f97e0011000 comp rx=0 tx=0).stop 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 -- 172.21.15.110:0/2102753624 shutdown_connections 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:6833/1869854747,v1:172.21.15.145:6835/1869854747] conn(0x558697cbf3e0 0x558697d02cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558697c90440 0x558697cbeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558697c8fb20 0x558697cbe960 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 --2- 172.21.15.110:0/2102753624 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x558697c4b390 0x558697ca3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97dcc00640 0 client.8664 ms_handle_reset on v2:172.21.15.145:6833/1869854747 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 -- 172.21.15.110:0/2102753624 >> 172.21.15.110:0/2102753624 conn(0x558697c46290 msgr2=0x558697c8dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 -- 172.21.15.110:0/2102753624 shutdown_connections 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 1 -- 172.21.15.110:0/2102753624 wait complete. 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fdb03000 20 client.8664 trim_cache size 0 max 16384 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 8526, 2024-09-10T10:05:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:05:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:05:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:05:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 10.40836331, 2024-09-10T10:05:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 106, 2024-09-10T10:05:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 153, 2024-09-10T10:05:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 153, 2024-09-10T10:05:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 15.265160307 2024-09-10T10:05:42.606 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:05:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 -- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f97d40680a0 msgr2=0x7f97d406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f97d40680a0 0x7f97d406a560 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97f4072370 tx=0x7f97e0007c10 comp rx=0 tx=0).stop 2024-09-10T10:05:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 -- 172.21.15.110:0/1215462966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f406efa0 msgr2=0x7f97f4151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f406efa0 0x7f97f4151d40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f97e40b0040 tx=0x7f97e40196e0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 -- 172.21.15.110:0/1215462966 shutdown_connections 2024-09-10T10:05:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f97d40680a0 0x7f97d406a560 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f97f40b5600 0x7f97f40ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f97f40726e0 0x7f97f41512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 --2- 172.21.15.110:0/1215462966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f406efa0 0x7f97f4151d40 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 -- 172.21.15.110:0/1215462966 >> 172.21.15.110:0/1215462966 conn(0x7f97f4062d10 msgr2=0x7f97f4155e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 -- 172.21.15.110:0/1215462966 shutdown_connections 2024-09-10T10:05:42.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.597+0000 7f97fa800640 1 -- 172.21.15.110:0/1215462966 wait complete. 2024-09-10T10:05:42.617 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:05:42.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba7600640 1 Processor -- start 2024-09-10T10:05:42.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba7600640 1 -- start start 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba7600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6ba00cfe30 0x7f6ba0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba7600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6ba00cd510 0x7f6ba00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba7600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba006e090 0x7f6ba006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba7600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba015d9a0 con 0x7f6ba00cd510 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba7600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba006ea60 con 0x7f6ba00cfe30 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba7600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba006d2d0 con 0x7f6ba006e090 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6b9fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6ba00cd510 0x7f6ba00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6b9fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6ba00cd510 0x7f6ba00cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39858/0 (socket says 172.21.15.110:39858) 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6b9fe00640 1 -- 172.21.15.110:0/3056592503 learned_addr learned my addr 172.21.15.110:0/3056592503 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba4c00640 1 --2- 172.21.15.110:0/3056592503 >> v2:172.21.15.145:3300/0 conn(0x7f6ba00cfe30 0x7f6ba0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6ba5600640 1 --2- 172.21.15.110:0/3056592503 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba006e090 0x7f6ba006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6b9fe00640 1 -- 172.21.15.110:0/3056592503 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba006e090 msgr2=0x7f6ba006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.753+0000 7f6b9fe00640 1 --2- 172.21.15.110:0/3056592503 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba006e090 0x7f6ba006e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6b9fe00640 1 -- 172.21.15.110:0/3056592503 >> v2:172.21.15.145:3300/0 conn(0x7f6ba00cfe30 msgr2=0x7f6ba0071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:42.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6b9fe00640 1 --2- 172.21.15.110:0/3056592503 >> v2:172.21.15.145:3300/0 conn(0x7f6ba00cfe30 0x7f6ba0071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6b9fe00640 1 -- 172.21.15.110:0/3056592503 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba006d5b0 con 0x7f6ba00cd510 2024-09-10T10:05:42.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba4c00640 1 --2- 172.21.15.110:0/3056592503 >> v2:172.21.15.145:3300/0 conn(0x7f6ba00cfe30 0x7f6ba0071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:42.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6b9fe00640 1 --2- 172.21.15.110:0/3056592503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba00cd510 0x7f6ba00cd9a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00ce120 tx=0x7f6b8c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73f554420f3deef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6b9f400640 1 -- 172.21.15.110:0/3056592503 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0b2070 con 0x7f6ba00cd510 2024-09-10T10:05:42.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6b9f400640 1 -- 172.21.15.110:0/3056592503 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8c0a9070 con 0x7f6ba00cd510 2024-09-10T10:05:42.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6b9f400640 1 -- 172.21.15.110:0/3056592503 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0ad880 con 0x7f6ba00cd510 2024-09-10T10:05:42.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 -- 172.21.15.110:0/3056592503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba00cd510 msgr2=0x7f6ba00cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 --2- 172.21.15.110:0/3056592503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba00cd510 0x7f6ba00cd9a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00ce120 tx=0x7f6b8c01c490 comp rx=0 tx=0).stop 2024-09-10T10:05:42.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 -- 172.21.15.110:0/3056592503 shutdown_connections 2024-09-10T10:05:42.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 --2- 172.21.15.110:0/3056592503 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba006e090 0x7f6ba006e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 --2- 172.21.15.110:0/3056592503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba00cd510 0x7f6ba00cd9a0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 --2- 172.21.15.110:0/3056592503 >> v2:172.21.15.145:3300/0 conn(0x7f6ba00cfe30 0x7f6ba0071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 -- 172.21.15.110:0/3056592503 >> 172.21.15.110:0/3056592503 conn(0x7f6ba006bc50 msgr2=0x7f6ba006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:42.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 -- 172.21.15.110:0/3056592503 shutdown_connections 2024-09-10T10:05:42.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.757+0000 7f6ba7600640 1 -- 172.21.15.110:0/3056592503 wait complete. 2024-09-10T10:05:42.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 Processor -- start 2024-09-10T10:05:42.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 -- start start 2024-09-10T10:05:42.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba006e090 0x7f6ba00b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba00cd510 0x7f6ba0146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ba00cfe30 0x7f6ba00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0072190 con 0x7f6ba006e090 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba015d680 con 0x7f6ba00cfe30 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba0160010 con 0x7f6ba00cd510 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba006e090 0x7f6ba00b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba006e090 0x7f6ba00b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39862/0 (socket says 172.21.15.110:39862) 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 -- 172.21.15.110:0/1629941831 learned_addr learned my addr 172.21.15.110:0/1629941831 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6b9fe00640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba00cd510 0x7f6ba0146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba5600640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ba00cfe30 0x7f6ba00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 -- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba00cd510 msgr2=0x7f6ba0146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba00cd510 0x7f6ba0146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 -- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ba00cfe30 msgr2=0x7f6ba00b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ba00cfe30 0x7f6ba00b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 -- 172.21.15.110:0/1629941831 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba01470e0 con 0x7f6ba006e090 2024-09-10T10:05:42.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba4c00640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba006e090 0x7f6ba00b48c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007ef0 tx=0x7f6b9000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6b9cc00640 1 -- 172.21.15.110:0/1629941831 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9001a070 con 0x7f6ba006e090 2024-09-10T10:05:42.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6b9cc00640 1 -- 172.21.15.110:0/1629941831 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b90003ba0 con 0x7f6ba006e090 2024-09-10T10:05:42.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6b9cc00640 1 -- 172.21.15.110:0/1629941831 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90004030 con 0x7f6ba006e090 2024-09-10T10:05:42.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba0155d10 con 0x7f6ba006e090 2024-09-10T10:05:42.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.761+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba00b3b00 con 0x7f6ba006e090 2024-09-10T10:05:42.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.765+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba00cd1b0 con 0x7f6ba006e090 2024-09-10T10:05:42.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.765+0000 7f6b9cc00640 1 -- 172.21.15.110:0/1629941831 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f6b9001d070 con 0x7f6ba006e090 2024-09-10T10:05:42.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.765+0000 7f6b9cc00640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b800680a0 0x7f6b8006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:42.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.765+0000 7f6b9cc00640 1 -- 172.21.15.110:0/1629941831 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f6b900871e0 con 0x7f6ba006e090 2024-09-10T10:05:42.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.765+0000 7f6b9fe00640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b800680a0 0x7f6b8006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:42.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.765+0000 7f6b9fe00640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b800680a0 0x7f6b8006a560 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0af000 tx=0x7f6b8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:42.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.769+0000 7f6b9cc00640 1 -- 172.21.15.110:0/1629941831 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6ba01470e0 con 0x7f6ba006e090 2024-09-10T10:05:42.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.957+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6ba015a5b0 con 0x7f6ba006e090 2024-09-10T10:05:42.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6b9cc00640 1 -- 172.21.15.110:0/1629941831 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v106) ==== 53+0+73 (secure 0 0 0) 0x7f6b90010070 con 0x7f6ba006e090 2024-09-10T10:05:42.961 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:05:42.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b800680a0 msgr2=0x7f6b8006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b800680a0 0x7f6b8006a560 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0af000 tx=0x7f6b8c008040 comp rx=0 tx=0).stop 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba006e090 msgr2=0x7f6ba00b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba006e090 0x7f6ba00b48c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007ef0 tx=0x7f6b9000d4a0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 shutdown_connections 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6b800680a0 0x7f6b8006a560 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6ba00cfe30 0x7f6ba00b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6ba00cd510 0x7f6ba0146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 --2- 172.21.15.110:0/1629941831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ba006e090 0x7f6ba00b48c0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 >> 172.21.15.110:0/1629941831 conn(0x7f6ba006bc50 msgr2=0x7f6ba015b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 shutdown_connections 2024-09-10T10:05:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:42.961+0000 7f6ba7600640 1 -- 172.21.15.110:0/1629941831 wait complete. 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:05:42.982 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:05:42.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:42.983 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:05:42.991 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:05:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:05:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:05:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42032sec preferred_lft 42032sec 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:05:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:42.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:42.995 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:05:43.053 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:05:43.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:43.055 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:05:43.107 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:05:43.107 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:05:43.107 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:05:43.107 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:05:43.107 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:05:43.107 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:05:43.108 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:05:43.108 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:43.108 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:43.108 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:43.108 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:05:43.108 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:05:43.108 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:05:43.108 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:05:43.108 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:05:43.109 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:05:43.109 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:05:43.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:05:43.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:43.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:43.246 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:05:43.255 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:05:43.255 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:05:43.255 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:05:43.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:43.255 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:05:43.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:05:43.315 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:43.364 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:05:43.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:05:43.364 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:43.419 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:05:43.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:43.419 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:43.507 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:43.508 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:05:43.521 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:43.521 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:05:43.521 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:05:43.521 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:05:43.522 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:05:43.525 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:05:43.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:43.526 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:43.591 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:05:43.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:05:43.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:43.698 INFO:teuthology.orchestra.run.smithi110.stdout:client7100 2024-09-10T10:05:43.698 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:43.698 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:43.707 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:05:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.881+0000 7f2127200640 1 Processor -- start 2024-09-10T10:05:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.881+0000 7f2127200640 1 -- start start 2024-09-10T10:05:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.881+0000 7f2127200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 0x7f21200cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.881+0000 7f2127200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f212015c560 0x7f212006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.881+0000 7f2127200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f212006f010 0x7f212015eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.881+0000 7f2127200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f21200b4570 con 0x7f212015c560 2024-09-10T10:05:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.881+0000 7f2127200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f212006f450 con 0x7f212006f010 2024-09-10T10:05:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.881+0000 7f2127200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2120155d50 con 0x7f2120155970 2024-09-10T10:05:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f212015c560 0x7f212006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f212015c560 0x7f212006ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39870/0 (socket says 172.21.15.110:39870) 2024-09-10T10:05:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 -- 172.21.15.110:0/3069590358 learned_addr learned my addr 172.21.15.110:0/3069590358 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211fe00640 1 --2- 172.21.15.110:0/3069590358 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 0x7f21200cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f2125200640 1 --2- 172.21.15.110:0/3069590358 >> v2:172.21.15.145:3300/0 conn(0x7f212006f010 0x7f212015eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 -- 172.21.15.110:0/3069590358 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 msgr2=0x7f21200cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 --2- 172.21.15.110:0/3069590358 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 0x7f21200cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 -- 172.21.15.110:0/3069590358 >> v2:172.21.15.145:3300/0 conn(0x7f212006f010 msgr2=0x7f212015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 --2- 172.21.15.110:0/3069590358 >> v2:172.21.15.145:3300/0 conn(0x7f212006f010 0x7f212015eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 -- 172.21.15.110:0/3069590358 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21200b34d0 con 0x7f212015c560 2024-09-10T10:05:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211f400640 1 --2- 172.21.15.110:0/3069590358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212015c560 0x7f212006ead0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f212006d4d0 tx=0x7f210c01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1a91f89863bd2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211ea00640 1 -- 172.21.15.110:0/3069590358 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c0b0070 con 0x7f212015c560 2024-09-10T10:05:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211ea00640 1 -- 172.21.15.110:0/3069590358 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f210c0ab600 con 0x7f212015c560 2024-09-10T10:05:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.885+0000 7f211ea00640 1 -- 172.21.15.110:0/3069590358 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c0ab9d0 con 0x7f212015c560 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- 172.21.15.110:0/3069590358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212015c560 msgr2=0x7f212006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 --2- 172.21.15.110:0/3069590358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212015c560 0x7f212006ead0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f212006d4d0 tx=0x7f210c01c1a0 comp rx=0 tx=0).stop 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- 172.21.15.110:0/3069590358 shutdown_connections 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 --2- 172.21.15.110:0/3069590358 >> v2:172.21.15.145:3300/0 conn(0x7f212006f010 0x7f212015eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 --2- 172.21.15.110:0/3069590358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212015c560 0x7f212006ead0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 --2- 172.21.15.110:0/3069590358 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 0x7f21200cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- 172.21.15.110:0/3069590358 >> 172.21.15.110:0/3069590358 conn(0x7f212006bc50 msgr2=0x7f212006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- 172.21.15.110:0/3069590358 shutdown_connections 2024-09-10T10:05:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- 172.21.15.110:0/3069590358 wait complete. 2024-09-10T10:05:43.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 Processor -- start 2024-09-10T10:05:43.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- start start 2024-09-10T10:05:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212006f010 0x7f2120153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 0x7f212014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f212015c560 0x7f212006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21200cce00 con 0x7f212006f010 2024-09-10T10:05:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f21200b4050 con 0x7f212015c560 2024-09-10T10:05:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2127200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f21200ce420 con 0x7f2120155970 2024-09-10T10:05:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212006f010 0x7f2120153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212006f010 0x7f2120153ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39876/0 (socket says 172.21.15.110:39876) 2024-09-10T10:05:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211fe00640 1 -- 172.21.15.110:0/2949938672 learned_addr learned my addr 172.21.15.110:0/2949938672 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211f400640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 0x7f212014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f2125200640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f212015c560 0x7f212006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211fe00640 1 -- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 msgr2=0x7f212014f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211fe00640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 0x7f212014f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211fe00640 1 -- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f212015c560 msgr2=0x7f212006d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211fe00640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f212015c560 0x7f212006d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.889+0000 7f211fe00640 1 -- 172.21.15.110:0/2949938672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21200ab4e0 con 0x7f212006f010 2024-09-10T10:05:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f211fe00640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212006f010 0x7f2120153ea0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f211000e9b0 tx=0x7f211000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2117e00640 1 -- 172.21.15.110:0/2949938672 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110017070 con 0x7f212006f010 2024-09-10T10:05:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2117e00640 1 -- 172.21.15.110:0/2949938672 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211000c9a0 con 0x7f212006f010 2024-09-10T10:05:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2117e00640 1 -- 172.21.15.110:0/2949938672 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211000cd70 con 0x7f212006f010 2024-09-10T10:05:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21200ac5a0 con 0x7f212006f010 2024-09-10T10:05:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21200abd40 con 0x7f212006f010 2024-09-10T10:05:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21200cc8c0 con 0x7f212006f010 2024-09-10T10:05:43.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2117e00640 1 -- 172.21.15.110:0/2949938672 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f211001a070 con 0x7f212006f010 2024-09-10T10:05:43.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2117e00640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21000680a0 0x7f210006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:43.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f2117e00640 1 -- 172.21.15.110:0/2949938672 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f2110086fc0 con 0x7f212006f010 2024-09-10T10:05:43.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.893+0000 7f211f400640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21000680a0 0x7f210006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:43.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.897+0000 7f211f400640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21000680a0 0x7f210006a560 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ad000 tx=0x7f210c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:43.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:43.897+0000 7f2117e00640 1 -- 172.21.15.110:0/2949938672 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2110056450 con 0x7f212006f010 2024-09-10T10:05:44.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f21200cf4a0 con 0x7f212006f010 2024-09-10T10:05:44.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2117e00640 1 -- 172.21.15.110:0/2949938672 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v106) ==== 53+0+73 (secure 0 0 0) 0x7f211005a750 con 0x7f212006f010 2024-09-10T10:05:44.089 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:05:44.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21000680a0 msgr2=0x7f210006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:44.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21000680a0 0x7f210006a560 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ad000 tx=0x7f210c008040 comp rx=0 tx=0).stop 2024-09-10T10:05:44.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212006f010 msgr2=0x7f2120153ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:44.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212006f010 0x7f2120153ea0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f211000e9b0 tx=0x7f211000ee70 comp rx=0 tx=0).stop 2024-09-10T10:05:44.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 shutdown_connections 2024-09-10T10:05:44.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f21000680a0 0x7f210006a560 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f212015c560 0x7f212006d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2120155970 0x7f212014f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 --2- 172.21.15.110:0/2949938672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f212006f010 0x7f2120153ea0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 >> 172.21.15.110:0/2949938672 conn(0x7f212006bc50 msgr2=0x7f212015ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:44.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 shutdown_connections 2024-09-10T10:05:44.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.089+0000 7f2127200640 1 -- 172.21.15.110:0/2949938672 wait complete. 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:05:44.110 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:05:44.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:44.111 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:44.121 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42027sec preferred_lft 42027sec 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:44.122 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:05:44.123 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:05:44.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:44.124 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:05:44.192 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:05:44.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:44.194 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:05:44.250 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:05:44.250 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:05:44.250 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:05:44.250 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:05:44.250 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:05:44.250 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:05:44.251 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:05:44.251 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:44.251 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:44.251 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:05:44.251 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:05:44.251 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:05:44.251 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:05:44.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:05:44.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:05:44.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:05:44.252 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:05:44.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:05:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:05:44.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:44.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:44.395 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:05:44.403 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:05:44.403 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:05:44.404 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:05:44.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:44.404 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:05:44.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:05:44.468 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:44.520 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:05:44.520 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:05:44.520 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:44.571 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:05:44.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:44.571 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:44.661 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:44.661 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:05:44.674 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:05:44.674 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:05:44.674 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:05:44.674 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:05:44.674 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:05:44.679 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:05:44.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:44.679 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:44.698 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:05:44.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:05:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:05:44.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:44.795 INFO:teuthology.orchestra.run.smithi145.stdout:client8065 2024-09-10T10:05:44.796 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:44.796 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:44.806 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:05:44.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbfde00640 1 Processor -- start 2024-09-10T10:05:44.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbfde00640 1 -- start start 2024-09-10T10:05:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbfde00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdbf8155970 0x7fdbf80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbfde00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf815c560 0x7fdbf806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbfde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdbf806f010 0x7fdbf815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbfde00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf80b4570 con 0x7fdbf806f010 2024-09-10T10:05:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbfde00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf806f430 con 0x7fdbf8155970 2024-09-10T10:05:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbfde00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf8155d50 con 0x7fdbf815c560 2024-09-10T10:05:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbf7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdbf806f010 0x7fdbf815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:44.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbf7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdbf806f010 0x7fdbf815eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39882/0 (socket says 172.21.15.110:39882) 2024-09-10T10:05:44.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.945+0000 7fdbf7400640 1 -- 172.21.15.110:0/2648388790 learned_addr learned my addr 172.21.15.110:0/2648388790 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:44.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf6a00640 1 --2- 172.21.15.110:0/2648388790 >> v2:172.21.15.145:3300/0 conn(0x7fdbf8155970 0x7fdbf80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:44.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf6000640 1 --2- 172.21.15.110:0/2648388790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf815c560 0x7fdbf806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:44.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf7400640 1 -- 172.21.15.110:0/2648388790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf815c560 msgr2=0x7fdbf806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:44.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf7400640 1 --2- 172.21.15.110:0/2648388790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf815c560 0x7fdbf806ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf7400640 1 -- 172.21.15.110:0/2648388790 >> v2:172.21.15.145:3300/0 conn(0x7fdbf8155970 msgr2=0x7fdbf80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:44.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf7400640 1 --2- 172.21.15.110:0/2648388790 >> v2:172.21.15.145:3300/0 conn(0x7fdbf8155970 0x7fdbf80cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf7400640 1 -- 172.21.15.110:0/2648388790 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf80b34d0 con 0x7fdbf806f010 2024-09-10T10:05:44.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf6a00640 1 --2- 172.21.15.110:0/2648388790 >> v2:172.21.15.145:3300/0 conn(0x7fdbf8155970 0x7fdbf80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:44.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf6000640 1 --2- 172.21.15.110:0/2648388790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf815c560 0x7fdbf806ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:44.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf7400640 1 --2- 172.21.15.110:0/2648388790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf806f010 0x7fdbf815eea0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdbf806cc30 tx=0x7fdbf001e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ae1c2732bece62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:44.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf5600640 1 -- 172.21.15.110:0/2648388790 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00b4070 con 0x7fdbf806f010 2024-09-10T10:05:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf5600640 1 -- 172.21.15.110:0/2648388790 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf00a7900 con 0x7fdbf806f010 2024-09-10T10:05:44.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbf5600640 1 -- 172.21.15.110:0/2648388790 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00a7cd0 con 0x7fdbf806f010 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbfde00640 1 -- 172.21.15.110:0/2648388790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf806f010 msgr2=0x7fdbf815eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.949+0000 7fdbfde00640 1 --2- 172.21.15.110:0/2648388790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf806f010 0x7fdbf815eea0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdbf806cc30 tx=0x7fdbf001e870 comp rx=0 tx=0).stop 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 -- 172.21.15.110:0/2648388790 shutdown_connections 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 --2- 172.21.15.110:0/2648388790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf806f010 0x7fdbf815eea0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 --2- 172.21.15.110:0/2648388790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf815c560 0x7fdbf806ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 --2- 172.21.15.110:0/2648388790 >> v2:172.21.15.145:3300/0 conn(0x7fdbf8155970 0x7fdbf80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 -- 172.21.15.110:0/2648388790 >> 172.21.15.110:0/2648388790 conn(0x7fdbf806bc50 msgr2=0x7fdbf806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 -- 172.21.15.110:0/2648388790 shutdown_connections 2024-09-10T10:05:44.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 -- 172.21.15.110:0/2648388790 wait complete. 2024-09-10T10:05:44.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 Processor -- start 2024-09-10T10:05:44.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 -- start start 2024-09-10T10:05:44.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf806f010 0x7fdbf8158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:44.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf8155970 0x7fdbf8153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf815c560 0x7fdbf80b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf80cce00 con 0x7fdbf815c560 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf80b4050 con 0x7fdbf806f010 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbfde00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf806d9c0 con 0x7fdbf8155970 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf815c560 0x7fdbf80b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf815c560 0x7fdbf80b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39896/0 (socket says 172.21.15.110:39896) 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 -- 172.21.15.110:0/831483520 learned_addr learned my addr 172.21.15.110:0/831483520 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf806f010 0x7fdbf8158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:44.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf6000640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf8155970 0x7fdbf8153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:44.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 -- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf8155970 msgr2=0x7fdbf8153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:44.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf8155970 0x7fdbf8153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 -- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf806f010 msgr2=0x7fdbf8158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf806f010 0x7fdbf8158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 -- 172.21.15.110:0/831483520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf8157ab0 con 0x7fdbf815c560 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.953+0000 7fdbf7400640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf815c560 0x7fdbf80b0140 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000b2c0 tx=0x7fdbf001b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbef400640 1 -- 172.21.15.110:0/831483520 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00b4070 con 0x7fdbf815c560 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbef400640 1 -- 172.21.15.110:0/831483520 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf00b6070 con 0x7fdbf815c560 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbef400640 1 -- 172.21.15.110:0/831483520 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00afd20 con 0x7fdbf815c560 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf8157d90 con 0x7fdbf815c560 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf8158350 con 0x7fdbf815c560 2024-09-10T10:05:44.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf815eea0 con 0x7fdbf815c560 2024-09-10T10:05:44.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbef400640 1 -- 172.21.15.110:0/831483520 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fdbf00c43f0 con 0x7fdbf815c560 2024-09-10T10:05:44.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbef400640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdbd80680a0 0x7fdbd806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:44.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbef400640 1 -- 172.21.15.110:0/831483520 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7fdbf0120e20 con 0x7fdbf815c560 2024-09-10T10:05:44.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.957+0000 7fdbf6a00640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdbd80680a0 0x7fdbd806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:44.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.961+0000 7fdbf6a00640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdbd80680a0 0x7fdbd806a560 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8007130 tx=0x7fdbe80077c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:44.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:44.961+0000 7fdbef400640 1 -- 172.21.15.110:0/831483520 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdbf00187e0 con 0x7fdbf815c560 2024-09-10T10:05:45.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.153+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fdbf8165c10 con 0x7fdbf815c560 2024-09-10T10:05:45.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.245+0000 7fdbef400640 1 -- 172.21.15.110:0/831483520 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v107) ==== 136+0+0 (secure 0 0 0) 0x7fdbf00f06c0 con 0x7fdbf815c560 2024-09-10T10:05:45.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdbd80680a0 msgr2=0x7fdbd806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdbd80680a0 0x7fdbd806a560 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8007130 tx=0x7fdbe80077c0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf815c560 msgr2=0x7fdbf80b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf815c560 0x7fdbf80b0140 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000b2c0 tx=0x7fdbf001b610 comp rx=0 tx=0).stop 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 shutdown_connections 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdbd80680a0 0x7fdbd806a560 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf815c560 0x7fdbf80b0140 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdbf8155970 0x7fdbf8153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 --2- 172.21.15.110:0/831483520 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbf806f010 0x7fdbf8158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 >> 172.21.15.110:0/831483520 conn(0x7fdbf806bc50 msgr2=0x7fdbf80ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:45.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 shutdown_connections 2024-09-10T10:05:45.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.249+0000 7fdbfde00640 1 -- 172.21.15.110:0/831483520 wait complete. 2024-09-10T10:05:45.269 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:05:45.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467ec00640 1 Processor -- start 2024-09-10T10:05:45.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467ec00640 1 -- start start 2024-09-10T10:05:45.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467ec00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46780d2ec0 0x7f4678071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467ec00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4678073380 con 0x7f467815e450 2024-09-10T10:05:45.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467ec00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4678073660 con 0x7f46780b71e0 2024-09-10T10:05:45.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467ec00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4678074140 con 0x7f46780d2ec0 2024-09-10T10:05:45.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467d800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f467815e450 0x7f46780b6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37330/0 (socket says 172.21.15.110:37330) 2024-09-10T10:05:45.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467d800640 1 -- 172.21.15.110:0/2246185438 learned_addr learned my addr 172.21.15.110:0/2246185438 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:45.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.405+0000 7f467ce00640 1 --2- 172.21.15.110:0/2246185438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46780d2ec0 0x7f4678071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f4677e00640 1 -- 172.21.15.110:0/2246185438 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2510774050 0 0) 0x7f4678073380 con 0x7f467815e450 2024-09-10T10:05:45.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f4677e00640 1 -- 172.21.15.110:0/2246185438 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4660003530 con 0x7f467815e450 2024-09-10T10:05:45.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f4677e00640 1 -- 172.21.15.110:0/2246185438 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121247604 0 0) 0x7f4678073660 con 0x7f46780b71e0 2024-09-10T10:05:45.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f4677e00640 1 -- 172.21.15.110:0/2246185438 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4678073380 con 0x7f46780b71e0 2024-09-10T10:05:45.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f467ce00640 1 -- 172.21.15.110:0/2246185438 >> v1:172.21.15.145:6789/0 conn(0x7f46780b71e0 legacy=0x7f467806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f467ce00640 1 -- 172.21.15.110:0/2246185438 >> v1:172.21.15.110:6789/0 conn(0x7f467815e450 legacy=0x7f46780b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f467ce00640 1 -- 172.21.15.110:0/2246185438 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4678073940 con 0x7f46780d2ec0 2024-09-10T10:05:45.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f467ce00640 1 --2- 172.21.15.110:0/2246185438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46780d2ec0 0x7f4678071280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46640097f0 tx=0x7f466401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab6e7cd117fb4f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:45.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f4677e00640 1 -- 172.21.15.110:0/2246185438 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640b0070 con 0x7f46780d2ec0 2024-09-10T10:05:45.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f4677e00640 1 -- 172.21.15.110:0/2246185438 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46640ab600 con 0x7f46780d2ec0 2024-09-10T10:05:45.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.409+0000 7f4677e00640 1 -- 172.21.15.110:0/2246185438 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640ab9b0 con 0x7f46780d2ec0 2024-09-10T10:05:45.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- 172.21.15.110:0/2246185438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46780d2ec0 msgr2=0x7f4678071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 --2- 172.21.15.110:0/2246185438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46780d2ec0 0x7f4678071280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46640097f0 tx=0x7f466401c4e0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- 172.21.15.110:0/2246185438 shutdown_connections 2024-09-10T10:05:45.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 --2- 172.21.15.110:0/2246185438 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f46780d2ec0 0x7f4678071280 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- 172.21.15.110:0/2246185438 >> 172.21.15.110:0/2246185438 conn(0x7f46780d23a0 msgr2=0x7f46780d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:45.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- 172.21.15.110:0/2246185438 shutdown_connections 2024-09-10T10:05:45.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- 172.21.15.110:0/2246185438 wait complete. 2024-09-10T10:05:45.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 Processor -- start 2024-09-10T10:05:45.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- start start 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46780b71e0 0x7f467815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46780d2ec0 0x7f46780cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f467815e450 0x7f4678159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f467816c1e0 con 0x7f46780d2ec0 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f467806de90 con 0x7f46780b71e0 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ec00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f467816c360 con 0x7f467815e450 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46780d2ec0 0x7f46780cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46780d2ec0 0x7f46780cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39908/0 (socket says 172.21.15.110:39908) 2024-09-10T10:05:45.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ce00640 1 -- 172.21.15.110:0/168730347 learned_addr learned my addr 172.21.15.110:0/168730347 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:45.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467d800640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46780b71e0 0x7f467815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467e200640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f467815e450 0x7f4678159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ce00640 1 -- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f467815e450 msgr2=0x7f4678159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ce00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f467815e450 0x7f4678159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ce00640 1 -- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46780b71e0 msgr2=0x7f467815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ce00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46780b71e0 0x7f467815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.413+0000 7f467ce00640 1 -- 172.21.15.110:0/168730347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4678154800 con 0x7f46780d2ec0 2024-09-10T10:05:45.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f467ce00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46780d2ec0 0x7f46780cb890 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f46640ae040 tx=0x7f4664019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:45.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f4675600640 1 -- 172.21.15.110:0/168730347 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640b0040 con 0x7f46780d2ec0 2024-09-10T10:05:45.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f4675600640 1 -- 172.21.15.110:0/168730347 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46640ab400 con 0x7f46780d2ec0 2024-09-10T10:05:45.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f4675600640 1 -- 172.21.15.110:0/168730347 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640ab9b0 con 0x7f46780d2ec0 2024-09-10T10:05:45.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4678154ae0 con 0x7f46780d2ec0 2024-09-10T10:05:45.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467815a5b0 con 0x7f46780d2ec0 2024-09-10T10:05:45.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4678071280 con 0x7f46780d2ec0 2024-09-10T10:05:45.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f4675600640 1 -- 172.21.15.110:0/168730347 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f46640bb030 con 0x7f46780d2ec0 2024-09-10T10:05:45.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f4675600640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f46580680a0 0x7f465806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f4675600640 1 -- 172.21.15.110:0/168730347 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f466411fe60 con 0x7f46780d2ec0 2024-09-10T10:05:45.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.417+0000 7f467d800640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f46580680a0 0x7f465806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.421+0000 7f467d800640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f46580680a0 0x7f465806a560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f466c0012f0 tx=0x7f466c001ae0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:45.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.421+0000 7f4675600640 1 -- 172.21.15.110:0/168730347 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f46640c2020 con 0x7f46780d2ec0 2024-09-10T10:05:45.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f46781545b0 con 0x7f46580680a0 2024-09-10T10:05:45.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f4675600640 1 -- 172.21.15.110:0/168730347 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f46781545b0 con 0x7f46580680a0 2024-09-10T10:05:45.602 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:05:45.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:05:45.602 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:05:45.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:05:45.602 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:05:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f46580680a0 msgr2=0x7f465806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f46580680a0 0x7f465806a560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f466c0012f0 tx=0x7f466c001ae0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46780d2ec0 msgr2=0x7f46780cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46780d2ec0 0x7f46780cb890 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f46640ae040 tx=0x7f4664019280 comp rx=0 tx=0).stop 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 shutdown_connections 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f46580680a0 0x7f465806a560 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f467815e450 0x7f4678159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46780d2ec0 0x7f46780cb890 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 --2- 172.21.15.110:0/168730347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46780b71e0 0x7f467815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 >> 172.21.15.110:0/168730347 conn(0x7f46780d23a0 msgr2=0x7f4678070600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 shutdown_connections 2024-09-10T10:05:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.601+0000 7f467ec00640 1 -- 172.21.15.110:0/168730347 wait complete. 2024-09-10T10:05:45.624 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:05:45.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.761+0000 7f8149000640 1 Processor -- start 2024-09-10T10:05:45.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.761+0000 7f8149000640 1 -- start start 2024-09-10T10:05:45.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.761+0000 7f8149000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8144145f30 0x7f81440b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.761+0000 7f8149000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81440cfd90 0x7f81440b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.761+0000 7f8149000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81440b37c0 con 0x7f81440b29e0 2024-09-10T10:05:45.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.761+0000 7f8149000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f81440b2e50 con 0x7f81440cfd90 2024-09-10T10:05:45.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.761+0000 7f8149000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f81440d0170 con 0x7f8144145f30 2024-09-10T10:05:45.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8142800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81440cfd90 0x7f81440b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8142800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f81440cfd90 0x7f81440b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39922/0 (socket says 172.21.15.110:39922) 2024-09-10T10:05:45.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8142800640 1 -- 172.21.15.110:0/3176438088 learned_addr learned my addr 172.21.15.110:0/3176438088 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:45.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8141e00640 1 --2- 172.21.15.110:0/3176438088 >> v2:172.21.15.145:3300/0 conn(0x7f8144145f30 0x7f81440b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8142800640 1 -- 172.21.15.110:0/3176438088 >> v2:172.21.15.145:3300/0 conn(0x7f8144145f30 msgr2=0x7f81440b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:45.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8142800640 1 --2- 172.21.15.110:0/3176438088 >> v2:172.21.15.145:3300/0 conn(0x7f8144145f30 0x7f81440b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8142800640 1 -- 172.21.15.110:0/3176438088 >> v1:172.21.15.145:6789/0 conn(0x7f81440b29e0 legacy=0x7f81440cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8142800640 1 -- 172.21.15.110:0/3176438088 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81440b3aa0 con 0x7f81440cfd90 2024-09-10T10:05:45.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8142800640 1 --2- 172.21.15.110:0/3176438088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440cfd90 0x7f81440b4580 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f81440b3000 tx=0x7f813c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02271b28a25f6f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:45.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8140a00640 1 -- 172.21.15.110:0/3176438088 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0a56c0 con 0x7f81440cfd90 2024-09-10T10:05:45.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8140a00640 1 -- 172.21.15.110:0/3176438088 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813c0a5860 con 0x7f81440cfd90 2024-09-10T10:05:45.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.765+0000 7f8140a00640 1 -- 172.21.15.110:0/3176438088 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0a5c30 con 0x7f81440cfd90 2024-09-10T10:05:45.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- 172.21.15.110:0/3176438088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440cfd90 msgr2=0x7f81440b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 --2- 172.21.15.110:0/3176438088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440cfd90 0x7f81440b4580 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f81440b3000 tx=0x7f813c01c9f0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- 172.21.15.110:0/3176438088 shutdown_connections 2024-09-10T10:05:45.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 --2- 172.21.15.110:0/3176438088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440cfd90 0x7f81440b4580 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 --2- 172.21.15.110:0/3176438088 >> v2:172.21.15.145:3300/0 conn(0x7f8144145f30 0x7f81440b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- 172.21.15.110:0/3176438088 >> 172.21.15.110:0/3176438088 conn(0x7f8144062e40 msgr2=0x7f814406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:45.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- 172.21.15.110:0/3176438088 shutdown_connections 2024-09-10T10:05:45.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- 172.21.15.110:0/3176438088 wait complete. 2024-09-10T10:05:45.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 Processor -- start 2024-09-10T10:05:45.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- start start 2024-09-10T10:05:45.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440b29e0 0x7f81440ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81440cfd90 0x7f8144140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8144145f30 0x7f814415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81440b56a0 con 0x7f81440b29e0 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81440b5520 con 0x7f8144145f30 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8149000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81440b5820 con 0x7f81440cfd90 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440b29e0 0x7f81440ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440b29e0 0x7f81440ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39930/0 (socket says 172.21.15.110:39930) 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 -- 172.21.15.110:0/3921584647 learned_addr learned my addr 172.21.15.110:0/3921584647 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8142800640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8144145f30 0x7f814415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141400640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81440cfd90 0x7f8144140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 -- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81440cfd90 msgr2=0x7f8144140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81440cfd90 0x7f8144140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 -- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8144145f30 msgr2=0x7f814415ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8144145f30 0x7f814415ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 -- 172.21.15.110:0/3921584647 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81441f78e0 con 0x7f81440b29e0 2024-09-10T10:05:45.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8142800640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8144145f30 0x7f814415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:45.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.769+0000 7f8141e00640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440b29e0 0x7f81440ae840 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f813400d5f0 tx=0x7f813400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:45.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f813a000640 1 -- 172.21.15.110:0/3921584647 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340125f0 con 0x7f81440b29e0 2024-09-10T10:05:45.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f813a000640 1 -- 172.21.15.110:0/3921584647 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8134019070 con 0x7f81440b29e0 2024-09-10T10:05:45.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f813a000640 1 -- 172.21.15.110:0/3921584647 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340129b0 con 0x7f81440b29e0 2024-09-10T10:05:45.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8144140760 con 0x7f81440b29e0 2024-09-10T10:05:45.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81440aeed0 con 0x7f81440b29e0 2024-09-10T10:05:45.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81440b4580 con 0x7f81440b29e0 2024-09-10T10:05:45.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f813a000640 1 -- 172.21.15.110:0/3921584647 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f8134012b50 con 0x7f81440b29e0 2024-09-10T10:05:45.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f813a000640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81240680d0 0x7f812406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:45.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f813a000640 1 -- 172.21.15.110:0/3921584647 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f8134087340 con 0x7f81440b29e0 2024-09-10T10:05:45.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.773+0000 7f8141400640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81240680d0 0x7f812406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:45.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.777+0000 7f8141400640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81240680d0 0x7f812406a590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8144167160 tx=0x7f8130009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:45.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.777+0000 7f813a000640 1 -- 172.21.15.110:0/3921584647 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8134056e60 con 0x7f81440b29e0 2024-09-10T10:05:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.933+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f814413faf0 con 0x7f81440b29e0 2024-09-10T10:05:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f813a000640 1 -- 172.21.15.110:0/3921584647 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f8134056760 con 0x7f81440b29e0 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81240680d0 msgr2=0x7f812406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81240680d0 0x7f812406a590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8144167160 tx=0x7f8130009040 comp rx=0 tx=0).stop 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440b29e0 msgr2=0x7f81440ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440b29e0 0x7f81440ae840 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f813400d5f0 tx=0x7f813400dab0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 shutdown_connections 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f81240680d0 0x7f812406a590 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8144145f30 0x7f814415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f81440cfd90 0x7f8144140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 --2- 172.21.15.110:0/3921584647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f81440b29e0 0x7f81440ae840 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:45.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 >> 172.21.15.110:0/3921584647 conn(0x7f8144062e40 msgr2=0x7f814415cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:45.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 shutdown_connections 2024-09-10T10:05:45.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:45.937+0000 7f8149000640 1 -- 172.21.15.110:0/3921584647 wait complete. 2024-09-10T10:05:45.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:45.958 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol40290/cd5716c0-7584-4d68-8c16-41c94b9d7606) 2024-09-10T10:05:45.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:45.987 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T10:05:46.038 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:05:46.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:46.039 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol40290/cd5716c0-7584-4d68-8c16-41c94b9d7606) 2024-09-10T10:05:46.104 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:46.105 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol40290/.meta 2024-09-10T10:05:46.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:46.171 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp66060/subvol70021/e3fe5128-595e-4787-9bd5-d83276c80016) 2024-09-10T10:05:46.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:46.235 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T10:05:46.285 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:05:46.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:46.285 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp66060/subvol70021/e3fe5128-595e-4787-9bd5-d83276c80016) 2024-09-10T10:05:46.353 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:46.353 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp66060/subvol70021/.meta 2024-09-10T10:05:46.420 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol40290 2024-09-10T10:05:46.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f42200640 1 Processor -- start 2024-09-10T10:05:46.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f42200640 1 -- start start 2024-09-10T10:05:46.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f42200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7f3c15e9b0 0x7f7f3c06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f42200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c0d2b70 0x7f7f3c162a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f42200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3c0b5970 con 0x7f7f3c0734d0 2024-09-10T10:05:46.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f42200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f7f3c162600 con 0x7f7f3c15e9b0 2024-09-10T10:05:46.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f42200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3c06e910 con 0x7f7f3c0d2b70 2024-09-10T10:05:46.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3aa00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f7f3c0734d0 0x7f7f3c0d2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37338/0 (socket says 172.21.15.110:37338) 2024-09-10T10:05:46.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3aa00640 1 -- 172.21.15.110:0/1272570550 learned_addr learned my addr 172.21.15.110:0/1272570550 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:46.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3b400640 1 --2- 172.21.15.110:0/1272570550 >> v2:172.21.15.145:3300/0 conn(0x7f7f3c15e9b0 0x7f7f3c06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3be00640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c0d2b70 0x7f7f3c162a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2136130092 0 0) 0x7f7f3c0b5970 con 0x7f7f3c0734d0 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f20003530 con 0x7f7f3c0734d0 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4091776223 0 0) 0x7f7f20003530 con 0x7f7f3c0734d0 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c0d2b70 msgr2=0x7f7f3c162a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c0d2b70 0x7f7f3c162a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 >> v2:172.21.15.145:3300/0 conn(0x7f7f3c15e9b0 msgr2=0x7f7f3c06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 --2- 172.21.15.110:0/1272570550 >> v2:172.21.15.145:3300/0 conn(0x7f7f3c15e9b0 0x7f7f3c06cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c06ebf0 con 0x7f7f3c0734d0 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f7f28001680 con 0x7f7f3c0734d0 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3b400640 1 --2- 172.21.15.110:0/1272570550 >> v2:172.21.15.145:3300/0 conn(0x7f7f3c15e9b0 0x7f7f3c06cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:46.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 >> v1:172.21.15.110:6789/0 conn(0x7f7f3c0734d0 legacy=0x7f7f3c0d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f200039c0 0x7f7f20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c0d2b70 0x7f7f20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f20004d50 0x7f7f200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c162600 con 0x7f7f20004d50 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c06e910 con 0x7f7f200039c0 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.601+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2000b9d0 con 0x7f7f3c0d2b70 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3be00640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f20004d50 0x7f7f200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3b400640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f200039c0 0x7f7f20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3aa00640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c0d2b70 0x7f7f20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3be00640 1 -- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c0d2b70 msgr2=0x7f7f20004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:46.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3be00640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c0d2b70 0x7f7f20004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3be00640 1 -- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f200039c0 msgr2=0x7f7f20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:46.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3be00640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f200039c0 0x7f7f20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3be00640 1 -- 172.21.15.110:0/1272570550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f20003530 con 0x7f7f20004d50 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3be00640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f20004d50 0x7f7f200051d0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009ae0 tx=0x7f7f3401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba0afc1b6657f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340b2070 con 0x7f7f20004d50 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f340a5620 con 0x7f7f20004d50 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.605+0000 7f7f3a000640 1 -- 172.21.15.110:0/1272570550 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340096b0 con 0x7f7f20004d50 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- 172.21.15.110:0/1272570550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f20004d50 msgr2=0x7f7f200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 --2- 172.21.15.110:0/1272570550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f20004d50 0x7f7f200051d0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009ae0 tx=0x7f7f3401ca30 comp rx=0 tx=0).stop 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 -- 172.21.15.110:0/1272570550 reap_dead start 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- 172.21.15.110:0/1272570550 shutdown_connections 2024-09-10T10:05:46.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- 172.21.15.110:0/1272570550 >> 172.21.15.110:0/1272570550 conn(0x7f7f3c06f420 msgr2=0x7f7f3c06f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:46.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- 172.21.15.110:0/1272570550 shutdown_connections 2024-09-10T10:05:46.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- 172.21.15.110:0/1272570550 wait complete. 2024-09-10T10:05:46.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 Processor -- start 2024-09-10T10:05:46.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- start start 2024-09-10T10:05:46.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f20004d50 0x7f7f3c1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c151140 0x7f7f3c1528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f3c140430 0x7f7f3c1408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c06d2a0 con 0x7f7f3c140430 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c15ed90 con 0x7f7f20004d50 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f42200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3c06d420 con 0x7f7f3c151140 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f3c140430 0x7f7f3c1408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f3c140430 0x7f7f3c1408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39960/0 (socket says 172.21.15.110:39960) 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 -- 172.21.15.110:0/1343876926 learned_addr learned my addr 172.21.15.110:0/1343876926 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3aa00640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c151140 0x7f7f3c1528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3b400640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f20004d50 0x7f7f3c1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 -- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c151140 msgr2=0x7f7f3c1528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c151140 0x7f7f3c1528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 -- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f20004d50 msgr2=0x7f7f3c1f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:46.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f20004d50 0x7f7f3c1f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 -- 172.21.15.110:0/1343876926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c152e30 con 0x7f7f3c140430 2024-09-10T10:05:46.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3b400640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f20004d50 0x7f7f3c1f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:46.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.609+0000 7f7f3be00640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f3c140430 0x7f7f3c1408b0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009ae0 tx=0x7f7f20005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:46.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f33400640 1 -- 172.21.15.110:0/1343876926 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340b2040 con 0x7f7f3c140430 2024-09-10T10:05:46.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f33400640 1 -- 172.21.15.110:0/1343876926 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f34000e50 con 0x7f7f3c140430 2024-09-10T10:05:46.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f33400640 1 -- 172.21.15.110:0/1343876926 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340096b0 con 0x7f7f3c140430 2024-09-10T10:05:46.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c1515c0 con 0x7f7f3c140430 2024-09-10T10:05:46.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c151b30 con 0x7f7f3c140430 2024-09-10T10:05:46.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c0b1a60 con 0x7f7f3c140430 2024-09-10T10:05:46.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f33400640 1 -- 172.21.15.110:0/1343876926 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f7f34004260 con 0x7f7f3c140430 2024-09-10T10:05:46.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f33400640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7f1c0680a0 0x7f7f1c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:46.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f33400640 1 -- 172.21.15.110:0/1343876926 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f7f34120260 con 0x7f7f3c140430 2024-09-10T10:05:46.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f3b400640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7f1c0680a0 0x7f7f1c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:46.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.613+0000 7f7f3b400640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7f1c0680a0 0x7f7f1c06a560 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c007660 tx=0x7f7f2c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:46.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.617+0000 7f7f33400640 1 -- 172.21.15.110:0/1343876926 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7f3c152e30 con 0x7f7f3c140430 2024-09-10T10:05:46.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.805+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol40290", "target": ["mon-mgr", ""]}) -- 0x7f7f3c0706a0 con 0x7f7f1c0680a0 2024-09-10T10:05:46.812 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:46.813+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:46.812 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:46.813+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:46.812 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:46.813+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:46.812 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:46.813+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:46.812 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:46.813+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:46.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.865+0000 7f7f33400640 1 -- 172.21.15.110:0/1343876926 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f7f3c0706a0 con 0x7f7f1c0680a0 2024-09-10T10:05:46.867 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/_nogroup/subvol40290/cd5716c0-7584-4d68-8c16-41c94b9d7606 2024-09-10T10:05:46.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7f1c0680a0 msgr2=0x7f7f1c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:46.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7f1c0680a0 0x7f7f1c06a560 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c007660 tx=0x7f7f2c00b040 comp rx=0 tx=0).stop 2024-09-10T10:05:46.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f3c140430 msgr2=0x7f7f3c1408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:46.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f3c140430 0x7f7f3c1408b0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009ae0 tx=0x7f7f20005840 comp rx=0 tx=0).stop 2024-09-10T10:05:46.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 shutdown_connections 2024-09-10T10:05:46.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7f1c0680a0 0x7f7f1c06a560 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f3c140430 0x7f7f3c1408b0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7f3c151140 0x7f7f3c1528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 --2- 172.21.15.110:0/1343876926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7f20004d50 0x7f7f3c1f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:46.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 >> 172.21.15.110:0/1343876926 conn(0x7f7f3c06f420 msgr2=0x7f7f3c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:46.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 shutdown_connections 2024-09-10T10:05:46.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:46.869+0000 7f7f42200640 1 -- 172.21.15.110:0/1343876926 wait complete. 2024-09-10T10:05:46.889 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol70021 subvol_grp66060 2024-09-10T10:05:47.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f377e000640 1 Processor -- start 2024-09-10T10:05:47.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f377e000640 1 -- start start 2024-09-10T10:05:47.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f377e000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f37780cfe30 0x7f3778071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f377e000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f377806e7a0 0x7f37780cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f377e000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377806d2d0 con 0x7f37780cd510 2024-09-10T10:05:47.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f377e000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f377815d9a0 con 0x7f37780cfe30 2024-09-10T10:05:47.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f377e000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f377806d5b0 con 0x7f377806e7a0 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f3777400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f37780cfe30 0x7f3778071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f3777400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f37780cfe30 0x7f3778071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39974/0 (socket says 172.21.15.110:39974) 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f3777400640 1 -- 172.21.15.110:0/704519051 learned_addr learned my addr 172.21.15.110:0/704519051 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f376fe00640 1 --1- 172.21.15.110:0/704519051 >> v1:172.21.15.110:6789/0 conn(0x7f37780cd510 0x7f377806e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37350/0 (socket says 172.21.15.110:37350) 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f3777e00640 1 --2- 172.21.15.110:0/704519051 >> v2:172.21.15.145:3300/0 conn(0x7f377806e7a0 0x7f37780cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f3777400640 1 -- 172.21.15.110:0/704519051 >> v2:172.21.15.145:3300/0 conn(0x7f377806e7a0 msgr2=0x7f37780cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f3777400640 1 --2- 172.21.15.110:0/704519051 >> v2:172.21.15.145:3300/0 conn(0x7f377806e7a0 0x7f37780cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f3777400640 1 -- 172.21.15.110:0/704519051 >> v1:172.21.15.110:6789/0 conn(0x7f37780cd510 legacy=0x7f377806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.029+0000 7f3777400640 1 -- 172.21.15.110:0/704519051 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377806d890 con 0x7f37780cfe30 2024-09-10T10:05:47.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f3777400640 1 --2- 172.21.15.110:0/704519051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778071a40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f377806cb00 tx=0x7f376801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cff0883631b8fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f3776a00640 1 -- 172.21.15.110:0/704519051 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680b1070 con 0x7f37780cfe30 2024-09-10T10:05:47.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f3776a00640 1 -- 172.21.15.110:0/704519051 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37680ac600 con 0x7f37780cfe30 2024-09-10T10:05:47.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f3776a00640 1 -- 172.21.15.110:0/704519051 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680ac9d0 con 0x7f37780cfe30 2024-09-10T10:05:47.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f377e000640 1 -- 172.21.15.110:0/704519051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 msgr2=0x7f3778071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f377e000640 1 --2- 172.21.15.110:0/704519051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778071a40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f377806cb00 tx=0x7f376801c490 comp rx=0 tx=0).stop 2024-09-10T10:05:47.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f377e000640 1 -- 172.21.15.110:0/704519051 shutdown_connections 2024-09-10T10:05:47.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f377e000640 1 --2- 172.21.15.110:0/704519051 >> v2:172.21.15.145:3300/0 conn(0x7f377806e7a0 0x7f37780cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f377e000640 1 --2- 172.21.15.110:0/704519051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778071a40 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f377e000640 1 -- 172.21.15.110:0/704519051 >> 172.21.15.110:0/704519051 conn(0x7f377806bc50 msgr2=0x7f377806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:47.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f377e000640 1 -- 172.21.15.110:0/704519051 shutdown_connections 2024-09-10T10:05:47.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.033+0000 7f377e000640 1 -- 172.21.15.110:0/704519051 wait complete. 2024-09-10T10:05:47.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 Processor -- start 2024-09-10T10:05:47.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 -- start start 2024-09-10T10:05:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f377806e7a0 0x7f37780af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37780cd510 0x7f3778154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3778072190 con 0x7f37780cfe30 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f377815d6c0 con 0x7f377806e7a0 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3778169180 con 0x7f37780cd510 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778147b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39982/0 (socket says 172.21.15.110:39982) 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 -- 172.21.15.110:0/2077348607 learned_addr learned my addr 172.21.15.110:0/2077348607 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777400640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f377806e7a0 0x7f37780af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f376fe00640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37780cd510 0x7f3778154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 -- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37780cd510 msgr2=0x7f3778154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37780cd510 0x7f3778154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 -- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f377806e7a0 msgr2=0x7f37780af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f377806e7a0 0x7f37780af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 -- 172.21.15.110:0/2077348607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377806ecd0 con 0x7f37780cfe30 2024-09-10T10:05:47.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f3777e00640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778147b40 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3770007e60 tx=0x7f377000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f376f400640 1 -- 172.21.15.110:0/2077348607 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770014420 con 0x7f37780cfe30 2024-09-10T10:05:47.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f376f400640 1 -- 172.21.15.110:0/2077348607 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3770019070 con 0x7f37780cfe30 2024-09-10T10:05:47.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f376f400640 1 -- 172.21.15.110:0/2077348607 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770014820 con 0x7f37780cfe30 2024-09-10T10:05:47.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3778154b60 con 0x7f37780cfe30 2024-09-10T10:05:47.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377814cfc0 con 0x7f37780cfe30 2024-09-10T10:05:47.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.037+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37780b3950 con 0x7f37780cfe30 2024-09-10T10:05:47.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.041+0000 7f376f400640 1 -- 172.21.15.110:0/2077348607 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f37700149c0 con 0x7f37780cfe30 2024-09-10T10:05:47.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.041+0000 7f376f400640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37540680a0 0x7f375406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.041+0000 7f376f400640 1 -- 172.21.15.110:0/2077348607 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f37700876d0 con 0x7f37780cfe30 2024-09-10T10:05:47.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.041+0000 7f3777400640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37540680a0 0x7f375406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.041+0000 7f3777400640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37540680a0 0x7f375406a560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f37680ae000 tx=0x7f37680a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.045+0000 7f376f400640 1 -- 172.21.15.110:0/2077348607 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3770056bd0 con 0x7f37780cfe30 2024-09-10T10:05:47.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.229+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol70021", "group_name": "subvol_grp66060", "target": ["mon-mgr", ""]}) -- 0x7f3778148080 con 0x7f37540680a0 2024-09-10T10:05:47.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f376f400640 1 -- 172.21.15.110:0/2077348607 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7f3778148080 con 0x7f37540680a0 2024-09-10T10:05:47.285 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/subvol_grp66060/subvol70021/e3fe5128-595e-4787-9bd5-d83276c80016 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37540680a0 msgr2=0x7f375406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37540680a0 0x7f375406a560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f37680ae000 tx=0x7f37680a7000 comp rx=0 tx=0).stop 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 msgr2=0x7f3778147b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778147b40 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3770007e60 tx=0x7f377000e4b0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 shutdown_connections 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f37540680a0 0x7f375406a560 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37780cfe30 0x7f3778147b40 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f37780cd510 0x7f3778154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 --2- 172.21.15.110:0/2077348607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f377806e7a0 0x7f37780af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 >> 172.21.15.110:0/2077348607 conn(0x7f377806bc50 msgr2=0x7f377806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:47.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 shutdown_connections 2024-09-10T10:05:47.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.285+0000 7f377e000640 1 -- 172.21.15.110:0/2077348607 wait complete. 2024-09-10T10:05:47.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:47.306 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol40290/.meta) 2024-09-10T10:05:47.327 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:05:47.327 INFO:teuthology.orchestra.run.smithi110.stdout:version = 2 2024-09-10T10:05:47.327 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:05:47.327 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/_nogroup/subvol40290/cd5716c0-7584-4d68-8c16-41c94b9d7606 2024-09-10T10:05:47.327 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:05:47.327 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:47.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:47.329 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp66060/subvol70021/.meta) 2024-09-10T10:05:47.391 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:05:47.391 INFO:teuthology.orchestra.run.smithi110.stdout:version = 2 2024-09-10T10:05:47.391 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:05:47.391 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/subvol_grp66060/subvol70021/e3fe5128-595e-4787-9bd5-d83276c80016 2024-09-10T10:05:47.391 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:05:47.391 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:47.393 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol40290 2024-09-10T10:05:47.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0442800640 1 Processor -- start 2024-09-10T10:05:47.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0442800640 1 -- start start 2024-09-10T10:05:47.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0442800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f043c0b1d10 0x7f043c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0442800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b4b90 0x7f043c06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0442800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f043c06cf00 con 0x7f043c0b1930 2024-09-10T10:05:47.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0442800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f043c0b7870 con 0x7f043c0b1d10 2024-09-10T10:05:47.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0442800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f043c0b72b0 con 0x7f043c0b4b90 2024-09-10T10:05:47.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0440a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f043c0b1d10 0x7f043c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0440a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f043c0b1d10 0x7f043c0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39998/0 (socket says 172.21.15.110:39998) 2024-09-10T10:05:47.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0440a00640 1 -- 172.21.15.110:0/448612023 learned_addr learned my addr 172.21.15.110:0/448612023 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:47.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0441e00640 1 --2- 172.21.15.110:0/448612023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b4b90 0x7f043c06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0440a00640 1 -- 172.21.15.110:0/448612023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b4b90 msgr2=0x7f043c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:47.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0440a00640 1 --2- 172.21.15.110:0/448612023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b4b90 0x7f043c06d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0440a00640 1 -- 172.21.15.110:0/448612023 >> v1:172.21.15.145:6789/0 conn(0x7f043c0b1930 legacy=0x7f043c0b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.577+0000 7f0440a00640 1 -- 172.21.15.110:0/448612023 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c0b7590 con 0x7f043c0b1d10 2024-09-10T10:05:47.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0440a00640 1 --2- 172.21.15.110:0/448612023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b1d10 0x7f043c0b4650 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f0428009870 tx=0x7f042801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1f3dad1b9956f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f043be00640 1 -- 172.21.15.110:0/448612023 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04280b1070 con 0x7f043c0b1d10 2024-09-10T10:05:47.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f043be00640 1 -- 172.21.15.110:0/448612023 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04280ac410 con 0x7f043c0b1d10 2024-09-10T10:05:47.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f043be00640 1 -- 172.21.15.110:0/448612023 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04280ac7e0 con 0x7f043c0b1d10 2024-09-10T10:05:47.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0442800640 1 -- 172.21.15.110:0/448612023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b1d10 msgr2=0x7f043c0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0442800640 1 --2- 172.21.15.110:0/448612023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b1d10 0x7f043c0b4650 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f0428009870 tx=0x7f042801c560 comp rx=0 tx=0).stop 2024-09-10T10:05:47.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0442800640 1 -- 172.21.15.110:0/448612023 shutdown_connections 2024-09-10T10:05:47.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0442800640 1 --2- 172.21.15.110:0/448612023 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b4b90 0x7f043c06d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0442800640 1 --2- 172.21.15.110:0/448612023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b1d10 0x7f043c0b4650 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0442800640 1 -- 172.21.15.110:0/448612023 >> 172.21.15.110:0/448612023 conn(0x7f043c0b7be0 msgr2=0x7f043c0b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:47.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0442800640 1 -- 172.21.15.110:0/448612023 shutdown_connections 2024-09-10T10:05:47.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.581+0000 7f0442800640 1 -- 172.21.15.110:0/448612023 wait complete. 2024-09-10T10:05:47.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 Processor -- start 2024-09-10T10:05:47.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 -- start start 2024-09-10T10:05:47.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b1930 0x7f043c0aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f043c0b1d10 0x7f043c164920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b4b90 0x7f043c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f043c06e410 con 0x7f043c0b4b90 2024-09-10T10:05:47.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f043c15ee10 con 0x7f043c0b1d10 2024-09-10T10:05:47.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f043c06e590 con 0x7f043c0b1930 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b4b90 0x7f043c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b1930 0x7f043c0aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b4b90 0x7f043c14f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40006/0 (socket says 172.21.15.110:40006) 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 -- 172.21.15.110:0/813966656 learned_addr learned my addr 172.21.15.110:0/813966656 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0440a00640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f043c0b1d10 0x7f043c164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 -- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b1930 msgr2=0x7f043c0aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b1930 0x7f043c0aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 -- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f043c0b1d10 msgr2=0x7f043c164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f043c0b1d10 0x7f043c164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 -- 172.21.15.110:0/813966656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c164e60 con 0x7f043c0b4b90 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0441e00640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b4b90 0x7f043c14f4a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f043400e7f0 tx=0x7f043400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0439600640 1 -- 172.21.15.110:0/813966656 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434017070 con 0x7f043c0b4b90 2024-09-10T10:05:47.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0439600640 1 -- 172.21.15.110:0/813966656 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043400c940 con 0x7f043c0b4b90 2024-09-10T10:05:47.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043c165110 con 0x7f043c0b4b90 2024-09-10T10:05:47.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0439600640 1 -- 172.21.15.110:0/813966656 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043400eea0 con 0x7f043c0b4b90 2024-09-10T10:05:47.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.585+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f043c1656d0 con 0x7f043c0b4b90 2024-09-10T10:05:47.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.589+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043c0b4650 con 0x7f043c0b4b90 2024-09-10T10:05:47.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.589+0000 7f0439600640 1 -- 172.21.15.110:0/813966656 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7f043401a070 con 0x7f043c0b4b90 2024-09-10T10:05:47.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.589+0000 7f0439600640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c0680a0 0x7f041c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.589+0000 7f0439600640 1 -- 172.21.15.110:0/813966656 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f0434086e10 con 0x7f043c0b4b90 2024-09-10T10:05:47.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.589+0000 7f0441400640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c0680a0 0x7f041c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.589+0000 7f0441400640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c0680a0 0x7f041c06a560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0430001ea0 tx=0x7f0430009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.593+0000 7f0439600640 1 -- 172.21.15.110:0/813966656 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f04340562a0 con 0x7f043c0b4b90 2024-09-10T10:05:47.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.777+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol40290", "target": ["mon-mgr", ""]}) -- 0x7f043c0008e0 con 0x7f041c0680a0 2024-09-10T10:05:47.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0439600640 1 -- 172.21.15.110:0/813966656 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f043c0008e0 con 0x7f041c0680a0 2024-09-10T10:05:47.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c0680a0 msgr2=0x7f041c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c0680a0 0x7f041c06a560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0430001ea0 tx=0x7f0430009040 comp rx=0 tx=0).stop 2024-09-10T10:05:47.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b4b90 msgr2=0x7f043c14f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b4b90 0x7f043c14f4a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f043400e7f0 tx=0x7f043400ecb0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 shutdown_connections 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f041c0680a0 0x7f041c06a560 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f043c0b4b90 0x7f043c14f4a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f043c0b1d10 0x7f043c164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 --2- 172.21.15.110:0/813966656 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f043c0b1930 0x7f043c0aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 >> 172.21.15.110:0/813966656 conn(0x7f043c0b7be0 msgr2=0x7f043c0b7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 shutdown_connections 2024-09-10T10:05:47.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.793+0000 7f0442800640 1 -- 172.21.15.110:0/813966656 wait complete. 2024-09-10T10:05:47.801 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.801+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.802 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.801+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.802 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.801+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.802 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.801+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.802 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.801+0000 7f95e1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.818 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol70021 subvol_grp66060 2024-09-10T10:05:47.819 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.817+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.819 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.817+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.819 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.817+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.819 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.817+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.819 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:05:47.817+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:05:47.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda58e00640 1 Processor -- start 2024-09-10T10:05:47.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda58e00640 1 -- start start 2024-09-10T10:05:47.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda58e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda54145c70 0x7fda54073f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda58e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fda540b5da0 0x7fda5415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda58e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5415ccf0 con 0x7fda540712f0 2024-09-10T10:05:47.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda58e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fda5415cab0 con 0x7fda540b5da0 2024-09-10T10:05:47.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda58e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fda54072040 con 0x7fda54145c70 2024-09-10T10:05:47.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda53e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fda540b5da0 0x7fda5415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda53e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fda540b5da0 0x7fda5415d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40018/0 (socket says 172.21.15.110:40018) 2024-09-10T10:05:47.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda53e00640 1 -- 172.21.15.110:0/333697565 learned_addr learned my addr 172.21.15.110:0/333697565 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:47.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.957+0000 7fda53400640 1 --2- 172.21.15.110:0/333697565 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda54145c70 0x7fda54073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda53e00640 1 -- 172.21.15.110:0/333697565 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda54145c70 msgr2=0x7fda54073f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:47.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda53e00640 1 --2- 172.21.15.110:0/333697565 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda54145c70 0x7fda54073f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda53e00640 1 -- 172.21.15.110:0/333697565 >> v1:172.21.15.110:6789/0 conn(0x7fda540712f0 legacy=0x7fda540b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda53e00640 1 -- 172.21.15.110:0/333697565 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda54071a80 con 0x7fda540b5da0 2024-09-10T10:05:47.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda53e00640 1 --2- 172.21.15.110:0/333697565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5415d4d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009e50 tx=0x7fda4c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b0fc8a7866418c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda52000640 1 -- 172.21.15.110:0/333697565 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0a5660 con 0x7fda540b5da0 2024-09-10T10:05:47.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda52000640 1 -- 172.21.15.110:0/333697565 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c0a5800 con 0x7fda540b5da0 2024-09-10T10:05:47.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda52000640 1 -- 172.21.15.110:0/333697565 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0a5bd0 con 0x7fda540b5da0 2024-09-10T10:05:47.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.961+0000 7fda58e00640 1 -- 172.21.15.110:0/333697565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 msgr2=0x7fda5415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 --2- 172.21.15.110:0/333697565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5415d4d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009e50 tx=0x7fda4c01c9a0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 -- 172.21.15.110:0/333697565 shutdown_connections 2024-09-10T10:05:47.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 --2- 172.21.15.110:0/333697565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5415d4d0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 --2- 172.21.15.110:0/333697565 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda54145c70 0x7fda54073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 -- 172.21.15.110:0/333697565 >> 172.21.15.110:0/333697565 conn(0x7fda5406d0f0 msgr2=0x7fda5406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:47.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 -- 172.21.15.110:0/333697565 shutdown_connections 2024-09-10T10:05:47.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 -- 172.21.15.110:0/333697565 wait complete. 2024-09-10T10:05:47.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 Processor -- start 2024-09-10T10:05:47.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 -- start start 2024-09-10T10:05:47.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda540712f0 0x7fda540b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5413fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda54145c70 0x7fda54140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fda5415e220 con 0x7fda540b5da0 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fda54146260 con 0x7fda54145c70 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda58e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fda5415e3a0 con 0x7fda540712f0 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5413fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5413fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40024/0 (socket says 172.21.15.110:40024) 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 -- 172.21.15.110:0/2828397343 learned_addr learned my addr 172.21.15.110:0/2828397343 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda53400640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda540712f0 0x7fda540b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda53e00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda54145c70 0x7fda54140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 -- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda540712f0 msgr2=0x7fda540b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda540712f0 0x7fda540b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 -- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda54145c70 msgr2=0x7fda54140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda54145c70 0x7fda54140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 -- 172.21.15.110:0/2828397343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda541386c0 con 0x7fda540b5da0 2024-09-10T10:05:47.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda53e00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda54145c70 0x7fda54140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:47.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.965+0000 7fda52a00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5413fb50 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fda4000c450 tx=0x7fda4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40016020 con 0x7fda540b5da0 2024-09-10T10:05:47.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda400052d0 con 0x7fda540b5da0 2024-09-10T10:05:47.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40007260 con 0x7fda540b5da0 2024-09-10T10:05:47.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda541389a0 con 0x7fda540b5da0 2024-09-10T10:05:47.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda540b5790 con 0x7fda540b5da0 2024-09-10T10:05:47.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5415e730 con 0x7fda540b5da0 2024-09-10T10:05:47.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 88159+0+0 (secure 0 0 0) 0x7fda40007400 con 0x7fda540b5da0 2024-09-10T10:05:47.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda4b400640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda340680a0 0x7fda3406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:47.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7fda400868d0 con 0x7fda540b5da0 2024-09-10T10:05:47.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.969+0000 7fda53400640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda340680a0 0x7fda3406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:47.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.973+0000 7fda53400640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda340680a0 0x7fda3406a560 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fda44009a50 tx=0x7fda44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:47.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:47.973+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fda40020020 con 0x7fda540b5da0 2024-09-10T10:05:48.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:48.153+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol70021", "group_name": "subvol_grp66060", "target": ["mon-mgr", ""]}) -- 0x7fda54139ec0 con 0x7fda340680a0 2024-09-10T10:05:48.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:48.265+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 23) ==== 88210+0+0 (secure 0 0 0) 0x7fda40019d80 con 0x7fda540b5da0 2024-09-10T10:05:49.489 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:05:49.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:05:50.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:50.277+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fda400523b0 con 0x7fda540b5da0 2024-09-10T10:05:57.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.345+0000 7fda4b400640 1 -- 172.21.15.110:0/2828397343 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda54139ec0 con 0x7fda340680a0 2024-09-10T10:05:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda340680a0 msgr2=0x7fda3406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda340680a0 0x7fda3406a560 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fda44009a50 tx=0x7fda44008040 comp rx=0 tx=0).stop 2024-09-10T10:05:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 msgr2=0x7fda5413fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5413fb50 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fda4000c450 tx=0x7fda4000c910 comp rx=0 tx=0).stop 2024-09-10T10:05:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 shutdown_connections 2024-09-10T10:05:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fda340680a0 0x7fda3406a560 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda54145c70 0x7fda54140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda540b5da0 0x7fda5413fb50 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 --2- 172.21.15.110:0/2828397343 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fda540712f0 0x7fda540b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 >> 172.21.15.110:0/2828397343 conn(0x7fda5406d0f0 msgr2=0x7fda541417f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 shutdown_connections 2024-09-10T10:05:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.349+0000 7fda58e00640 1 -- 172.21.15.110:0/2828397343 wait complete. 2024-09-10T10:05:57.374 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T10:05:57.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:05:57.453 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-10T10:05:57.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:05:57.460 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:05:57.460 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:05:57.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:05:57.647 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:05:57.647 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:05:57.647 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:57.647 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:57.656 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:57.656 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:57.704 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:05:57.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.885+0000 7f481ba00640 1 Processor -- start 2024-09-10T10:05:57.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.885+0000 7f481ba00640 1 -- start start 2024-09-10T10:05:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f481ba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cfb90 0x7f481415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f481ba00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f48140cdcb0 0x7f481406db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f481ba00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481415d4d0 con 0x7f48140b3f40 2024-09-10T10:05:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f481ba00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f481406d130 con 0x7f48140cdcb0 2024-09-10T10:05:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f481ba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f481415d7b0 con 0x7f48140cfb90 2024-09-10T10:05:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4813e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f48140b3f40 0x7f48140cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43370/0 (socket says 172.21.15.110:43370) 2024-09-10T10:05:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4813e00640 1 -- 172.21.15.110:0/633399571 learned_addr learned my addr 172.21.15.110:0/633399571 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4819000640 1 --2- 172.21.15.110:0/633399571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cfb90 0x7f481415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4819a00640 1 --2- 172.21.15.110:0/633399571 >> v2:172.21.15.145:3300/0 conn(0x7f48140cdcb0 0x7f481406db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4813400640 1 -- 172.21.15.110:0/633399571 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948694928 0 0) 0x7f481415d4d0 con 0x7f48140b3f40 2024-09-10T10:05:57.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4813400640 1 -- 172.21.15.110:0/633399571 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47f8003530 con 0x7f48140b3f40 2024-09-10T10:05:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4819000640 1 -- 172.21.15.110:0/633399571 >> v2:172.21.15.145:3300/0 conn(0x7f48140cdcb0 msgr2=0x7f481406db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4819000640 1 --2- 172.21.15.110:0/633399571 >> v2:172.21.15.145:3300/0 conn(0x7f48140cdcb0 0x7f481406db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4819000640 1 -- 172.21.15.110:0/633399571 >> v1:172.21.15.110:6789/0 conn(0x7f48140b3f40 legacy=0x7f48140cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4813400640 1 -- 172.21.15.110:0/633399571 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 897488072 0 0) 0x7f47f8003530 con 0x7f48140b3f40 2024-09-10T10:05:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4819000640 1 -- 172.21.15.110:0/633399571 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481406d380 con 0x7f48140cfb90 2024-09-10T10:05:57.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.889+0000 7f4819a00640 1 --2- 172.21.15.110:0/633399571 >> v2:172.21.15.145:3300/0 conn(0x7f48140cdcb0 0x7f481406db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:57.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f4819000640 1 --2- 172.21.15.110:0/633399571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cfb90 0x7f481415ca10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4804009d90 tx=0x7f480401c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=973c37ac07e78264 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:57.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f4813400640 1 -- 172.21.15.110:0/633399571 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040ac070 con 0x7f48140cfb90 2024-09-10T10:05:57.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f4813400640 1 -- 172.21.15.110:0/633399571 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48040a6410 con 0x7f48140cfb90 2024-09-10T10:05:57.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f4813400640 1 -- 172.21.15.110:0/633399571 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800001680 con 0x7f48140cfb90 2024-09-10T10:05:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 -- 172.21.15.110:0/633399571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cfb90 msgr2=0x7f481415ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 --2- 172.21.15.110:0/633399571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cfb90 0x7f481415ca10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4804009d90 tx=0x7f480401c800 comp rx=0 tx=0).stop 2024-09-10T10:05:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 -- 172.21.15.110:0/633399571 shutdown_connections 2024-09-10T10:05:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 --2- 172.21.15.110:0/633399571 >> v2:172.21.15.145:3300/0 conn(0x7f48140cdcb0 0x7f481406db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 --2- 172.21.15.110:0/633399571 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cfb90 0x7f481415ca10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 -- 172.21.15.110:0/633399571 >> 172.21.15.110:0/633399571 conn(0x7f4814062d10 msgr2=0x7f481406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 -- 172.21.15.110:0/633399571 shutdown_connections 2024-09-10T10:05:57.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 -- 172.21.15.110:0/633399571 wait complete. 2024-09-10T10:05:57.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 Processor -- start 2024-09-10T10:05:57.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.893+0000 7f481ba00640 1 -- start start 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48140b3f40 0x7f481414dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cdcb0 0x7f481414f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48140cfb90 0x7f4814161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4814169170 con 0x7f48140b3f40 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f48141692f0 con 0x7f48140cfb90 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4814169470 con 0x7f48140cdcb0 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48140b3f40 0x7f481414dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48140b3f40 0x7f481414dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41518/0 (socket says 172.21.15.110:41518) 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 -- 172.21.15.110:0/837559192 learned_addr learned my addr 172.21.15.110:0/837559192 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4813e00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cdcb0 0x7f481414f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:57.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 -- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cdcb0 msgr2=0x7f481414f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cdcb0 0x7f481414f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 -- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48140cfb90 msgr2=0x7f4814161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48140cfb90 0x7f4814161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 -- 172.21.15.110:0/837559192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4814163c70 con 0x7f48140b3f40 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4813e00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cdcb0 0x7f481414f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4819000640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48140b3f40 0x7f481414dd40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f480401aeb0 tx=0x7f48040195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4810c00640 1 -- 172.21.15.110:0/837559192 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040ac070 con 0x7f48140b3f40 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4810c00640 1 -- 172.21.15.110:0/837559192 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48040b1880 con 0x7f48140b3f40 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4810c00640 1 -- 172.21.15.110:0/837559192 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804009450 con 0x7f48140b3f40 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4814163f50 con 0x7f48140b3f40 2024-09-10T10:05:57.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4814164510 con 0x7f48140b3f40 2024-09-10T10:05:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481406db80 con 0x7f48140b3f40 2024-09-10T10:05:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.897+0000 7f4810c00640 1 -- 172.21.15.110:0/837559192 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f48040af070 con 0x7f48140b3f40 2024-09-10T10:05:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.901+0000 7f4810c00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47f4068250 0x7f47f406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:57.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.901+0000 7f4810c00640 1 -- 172.21.15.110:0/837559192 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f480411ee10 con 0x7f48140b3f40 2024-09-10T10:05:57.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.901+0000 7f4813e00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47f4068250 0x7f47f406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:57.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.901+0000 7f4813e00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47f4068250 0x7f47f406a710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f481414f300 tx=0x7f4800009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:57.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:57.905+0000 7f4810c00640 1 -- 172.21.15.110:0/837559192 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4804016480 con 0x7f48140b3f40 2024-09-10T10:05:58.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4814166400 con 0x7f48140b3f40 2024-09-10T10:05:58.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f4810c00640 1 -- 172.21.15.110:0/837559192 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v153) ==== 80+0+0 (secure 0 0 0) 0x7f48040f1550 con 0x7f48140b3f40 2024-09-10T10:05:58.073 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:05:58.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47f4068250 msgr2=0x7f47f406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47f4068250 0x7f47f406a710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f481414f300 tx=0x7f4800009040 comp rx=0 tx=0).stop 2024-09-10T10:05:58.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48140b3f40 msgr2=0x7f481414dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48140b3f40 0x7f481414dd40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f480401aeb0 tx=0x7f48040195a0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 shutdown_connections 2024-09-10T10:05:58.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f47f4068250 0x7f47f406a710 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f48140cfb90 0x7f4814161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f48140cdcb0 0x7f481414f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 --2- 172.21.15.110:0/837559192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48140b3f40 0x7f481414dd40 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 >> 172.21.15.110:0/837559192 conn(0x7f4814062d10 msgr2=0x7f48140b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:58.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 shutdown_connections 2024-09-10T10:05:58.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.073+0000 7f481ba00640 1 -- 172.21.15.110:0/837559192 wait complete. 2024-09-10T10:05:58.094 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:05:58.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:58.095 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:58.154 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:05:58.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:58.155 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:05:58.172 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:05:58.172 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:58.172 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:58.181 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:58.182 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:58.232 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:05:58.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359f400640 1 Processor -- start 2024-09-10T10:05:58.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359f400640 1 -- start start 2024-09-10T10:05:58.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f35980726e0 0x7f35980cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f359806efa0 0x7f35980b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f35980b5430 0x7f35980b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359f400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f35980b5880 con 0x7f35980726e0 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359f400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f35980cee50 con 0x7f35980b5430 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359f400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f359806f3e0 con 0x7f359806efa0 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.369+0000 7f359ca00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f35980726e0 0x7f35980cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359ca00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f35980726e0 0x7f35980cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52322/0 (socket says 172.21.15.110:52322) 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359ca00640 1 -- 172.21.15.110:0/3848460718 learned_addr learned my addr 172.21.15.110:0/3848460718 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359d400640 1 --2- 172.21.15.110:0/3848460718 >> v2:172.21.15.145:3300/0 conn(0x7f35980b5430 0x7f35980b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f3597e00640 1 --2- 172.21.15.110:0/3848460718 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f359806efa0 0x7f35980b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359ca00640 1 -- 172.21.15.110:0/3848460718 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f359806efa0 msgr2=0x7f35980b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359ca00640 1 --2- 172.21.15.110:0/3848460718 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f359806efa0 0x7f35980b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359ca00640 1 -- 172.21.15.110:0/3848460718 >> v2:172.21.15.145:3300/0 conn(0x7f35980b5430 msgr2=0x7f35980b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359ca00640 1 --2- 172.21.15.110:0/3848460718 >> v2:172.21.15.145:3300/0 conn(0x7f35980b5430 0x7f35980b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359ca00640 1 -- 172.21.15.110:0/3848460718 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359815fda0 con 0x7f35980726e0 2024-09-10T10:05:58.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f359ca00640 1 --2- 172.21.15.110:0/3848460718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35980726e0 0x7f35980cf2e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f35980b00f0 tx=0x7f358801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de250b3d2b9cd1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f3597400640 1 -- 172.21.15.110:0/3848460718 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880a5840 con 0x7f35980726e0 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f3597400640 1 -- 172.21.15.110:0/3848460718 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35880a59e0 con 0x7f35980726e0 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.373+0000 7f3597400640 1 -- 172.21.15.110:0/3848460718 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880a5db0 con 0x7f35980726e0 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- 172.21.15.110:0/3848460718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35980726e0 msgr2=0x7f35980cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 --2- 172.21.15.110:0/3848460718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35980726e0 0x7f35980cf2e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f35980b00f0 tx=0x7f358801c9c0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- 172.21.15.110:0/3848460718 shutdown_connections 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 --2- 172.21.15.110:0/3848460718 >> v2:172.21.15.145:3300/0 conn(0x7f35980b5430 0x7f35980b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 --2- 172.21.15.110:0/3848460718 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f359806efa0 0x7f35980b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 --2- 172.21.15.110:0/3848460718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35980726e0 0x7f35980cf2e0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- 172.21.15.110:0/3848460718 >> 172.21.15.110:0/3848460718 conn(0x7f3598062d10 msgr2=0x7f359806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- 172.21.15.110:0/3848460718 shutdown_connections 2024-09-10T10:05:58.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- 172.21.15.110:0/3848460718 wait complete. 2024-09-10T10:05:58.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 Processor -- start 2024-09-10T10:05:58.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- start start 2024-09-10T10:05:58.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f359806efa0 0x7f35981413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35980726e0 0x7f35980ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35980b5430 0x7f35980aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3598072ac0 con 0x7f359806efa0 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3598073270 con 0x7f35980726e0 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359f400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3598072c40 con 0x7f35980b5430 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f359806efa0 0x7f35981413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f359806efa0 0x7f35981413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52330/0 (socket says 172.21.15.110:52330) 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 -- 172.21.15.110:0/1480977225 learned_addr learned my addr 172.21.15.110:0/1480977225 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f3597e00640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35980726e0 0x7f35980ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359d400640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35980b5430 0x7f35980aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 -- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35980b5430 msgr2=0x7f35980aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35980b5430 0x7f35980aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 -- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35980726e0 msgr2=0x7f35980ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35980726e0 0x7f35980ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 -- 172.21.15.110:0/1480977225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35980b0e30 con 0x7f359806efa0 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f3597e00640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35980726e0 0x7f35980ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359d400640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35980b5430 0x7f35980aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f359ca00640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f359806efa0 0x7f35981413f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f35880af040 tx=0x7f3588019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.377+0000 7f3594c00640 1 -- 172.21.15.110:0/1480977225 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880b8670 con 0x7f359806efa0 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f3594c00640 1 -- 172.21.15.110:0/1480977225 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35880be070 con 0x7f359806efa0 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f3594c00640 1 -- 172.21.15.110:0/1480977225 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880b8a70 con 0x7f359806efa0 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3598141df0 con 0x7f359806efa0 2024-09-10T10:05:58.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35981423b0 con 0x7f359806efa0 2024-09-10T10:05:58.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359815f950 con 0x7f359806efa0 2024-09-10T10:05:58.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f3594c00640 1 -- 172.21.15.110:0/1480977225 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f35880b8c10 con 0x7f359806efa0 2024-09-10T10:05:58.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f3594c00640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3578068250 0x7f357806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f3594c00640 1 -- 172.21.15.110:0/1480977225 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f358811f5d0 con 0x7f359806efa0 2024-09-10T10:05:58.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.381+0000 7f3597e00640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3578068250 0x7f357806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.385+0000 7f3597e00640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3578068250 0x7f357806a710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f35981533e0 tx=0x7f3584008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.385+0000 7f3594c00640 1 -- 172.21.15.110:0/1480977225 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f35880165e0 con 0x7f359806efa0 2024-09-10T10:05:58.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3598159fb0 con 0x7f359806efa0 2024-09-10T10:05:58.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f3594c00640 1 -- 172.21.15.110:0/1480977225 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v153) ==== 80+0+0 (secure 0 0 0) 0x7f35880f1d10 con 0x7f359806efa0 2024-09-10T10:05:58.557 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:05:58.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3578068250 msgr2=0x7f357806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3578068250 0x7f357806a710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f35981533e0 tx=0x7f3584008040 comp rx=0 tx=0).stop 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f359806efa0 msgr2=0x7f35981413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f359806efa0 0x7f35981413f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f35880af040 tx=0x7f3588019760 comp rx=0 tx=0).stop 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 shutdown_connections 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3578068250 0x7f357806a710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f35980b5430 0x7f35980aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35980726e0 0x7f35980ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 --2- 172.21.15.110:0/1480977225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f359806efa0 0x7f35981413f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 >> 172.21.15.110:0/1480977225 conn(0x7f3598062d10 msgr2=0x7f359815bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:58.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 shutdown_connections 2024-09-10T10:05:58.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.557+0000 7f359f400640 1 -- 172.21.15.110:0/1480977225 wait complete. 2024-09-10T10:05:58.580 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:05:58.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:58.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:05:58.610 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:05:58.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:58.611 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:05:58.665 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:05:58.665 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:05:58.665 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:58.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:05:58.675 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:05:58.675 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:05:58.675 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:58.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:05:58.717 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:58.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.849+0000 7f2e85600640 1 Processor -- start 2024-09-10T10:05:58.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.849+0000 7f2e85600640 1 -- start start 2024-09-10T10:05:58.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e85600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2e80155970 0x7f2e800cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e85600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8015c560 0x7f2e8006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e85600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8006f430 con 0x7f2e8006f010 2024-09-10T10:05:58.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e85600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2e800b4570 con 0x7f2e80155970 2024-09-10T10:05:58.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e85600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2e80155d50 con 0x7f2e8015c560 2024-09-10T10:05:58.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2e80155970 0x7f2e800cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7e000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8015c560 0x7f2e8006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7f400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f2e8006f010 0x7f2e8015eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:36862/0 (socket says 172.21.15.110:36862) 2024-09-10T10:05:58.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7f400640 1 -- 172.21.15.110:0/4180237972 learned_addr learned my addr 172.21.15.110:0/4180237972 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:05:58.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2e80155970 0x7f2e800cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:58976/0 (socket says 172.21.15.110:58976) 2024-09-10T10:05:58.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7ea00640 1 -- 172.21.15.110:0/4180237972 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8015c560 msgr2=0x7f2e8006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7ea00640 1 --2- 172.21.15.110:0/4180237972 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8015c560 0x7f2e8006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7ea00640 1 -- 172.21.15.110:0/4180237972 >> v1:172.21.15.145:6789/0 conn(0x7f2e8006f010 legacy=0x7f2e8015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7ea00640 1 -- 172.21.15.110:0/4180237972 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e800b34d0 con 0x7f2e80155970 2024-09-10T10:05:58.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7d600640 1 -- 172.21.15.110:0/4180237972 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3291290133 0 0) 0x7f2e8006f430 con 0x7f2e8006f010 2024-09-10T10:05:58.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.853+0000 7f2e7e000640 1 --2- 172.21.15.110:0/4180237972 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8015c560 0x7f2e8006ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:58.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e7ea00640 1 --2- 172.21.15.110:0/4180237972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 0x7f2e800cc8c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2e700097c0 tx=0x7f2e7001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4754ea6d1282da46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e7d600640 1 -- 172.21.15.110:0/4180237972 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700b1070 con 0x7f2e80155970 2024-09-10T10:05:58.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e7d600640 1 -- 172.21.15.110:0/4180237972 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e700a8050 con 0x7f2e80155970 2024-09-10T10:05:58.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e7d600640 1 -- 172.21.15.110:0/4180237972 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700ac860 con 0x7f2e80155970 2024-09-10T10:05:58.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 -- 172.21.15.110:0/4180237972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 msgr2=0x7f2e800cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 --2- 172.21.15.110:0/4180237972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 0x7f2e800cc8c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2e700097c0 tx=0x7f2e7001c3e0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 -- 172.21.15.110:0/4180237972 shutdown_connections 2024-09-10T10:05:58.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 --2- 172.21.15.110:0/4180237972 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8015c560 0x7f2e8006ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 --2- 172.21.15.110:0/4180237972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 0x7f2e800cc8c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 -- 172.21.15.110:0/4180237972 >> 172.21.15.110:0/4180237972 conn(0x7f2e8006bc50 msgr2=0x7f2e8006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:58.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 -- 172.21.15.110:0/4180237972 shutdown_connections 2024-09-10T10:05:58.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 -- 172.21.15.110:0/4180237972 wait complete. 2024-09-10T10:05:58.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 Processor -- start 2024-09-10T10:05:58.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.857+0000 7f2e85600640 1 -- start start 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8006f010 0x7f2e80151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 0x7f2e80155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e8015c560 0x7f2e801578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e800cce00 con 0x7f2e8015c560 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2e800b4050 con 0x7f2e80155970 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8006d9c0 con 0x7f2e8006f010 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e8015c560 0x7f2e801578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e8015c560 0x7f2e801578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52346/0 (socket says 172.21.15.110:52346) 2024-09-10T10:05:58.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 -- 172.21.15.110:0/897983865 learned_addr learned my addr 172.21.15.110:0/897983865 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7ea00640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8006f010 0x7f2e80151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7e000640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 0x7f2e80155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 -- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8006f010 msgr2=0x7f2e80151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8006f010 0x7f2e80151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 -- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 msgr2=0x7f2e80155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 0x7f2e80155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 -- 172.21.15.110:0/897983865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e801524e0 con 0x7f2e8015c560 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e7f400640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e8015c560 0x7f2e801578e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800c7a0 tx=0x7f2e7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e76a00640 1 -- 172.21.15.110:0/897983865 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78018070 con 0x7f2e8015c560 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e76a00640 1 -- 172.21.15.110:0/897983865 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7800a640 con 0x7f2e8015c560 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e76a00640 1 -- 172.21.15.110:0/897983865 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800aab0 con 0x7f2e8015c560 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e80156840 con 0x7f2e8015c560 2024-09-10T10:05:58.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e80156e00 con 0x7f2e8015c560 2024-09-10T10:05:58.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e800cc8c0 con 0x7f2e8015c560 2024-09-10T10:05:58.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.861+0000 7f2e76a00640 1 -- 172.21.15.110:0/897983865 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f2e7800ac50 con 0x7f2e8015c560 2024-09-10T10:05:58.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.865+0000 7f2e76a00640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2e60068250 0x7f2e6006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.865+0000 7f2e76a00640 1 -- 172.21.15.110:0/897983865 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f2e780839d0 con 0x7f2e8015c560 2024-09-10T10:05:58.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.865+0000 7f2e7ea00640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2e60068250 0x7f2e6006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.865+0000 7f2e7ea00640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2e60068250 0x7f2e6006a710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e700ae000 tx=0x7f2e700a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:58.869+0000 7f2e76a00640 1 -- 172.21.15.110:0/897983865 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2e78055ac0 con 0x7f2e8015c560 2024-09-10T10:05:59.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.057+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e80074f90 con 0x7f2e8015c560 2024-09-10T10:05:59.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.057+0000 7f2e76a00640 1 -- 172.21.15.110:0/897983865 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5245 (secure 0 0 0) 0x7f2e78059dc0 con 0x7f2e8015c560 2024-09-10T10:05:59.059 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:59.059 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":107,"btime":"2024-09-10T10:05:45:243547+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3244721906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3244721906},{"type":"v1","addr":"172.21.15.145:6834","nonce":3244721906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/1342717567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1342717567},{"type":"v1","addr":"172.21.15.110:6834","nonce":1342717567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1495976904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1495976904},{"type":"v1","addr":"172.21.15.110:6835","nonce":1495976904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:31.551366+0000","modified":"2024-09-10T10:05:34.580103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1869854747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1869854747},{"type":"v1","addr":"172.21.15.145:6835","nonce":1869854747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":12}]} 2024-09-10T10:05:59.059 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 107 2024-09-10T10:05:59.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.057+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2e60068250 msgr2=0x7f2e6006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.057+0000 7f2e85600640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2e60068250 0x7f2e6006a710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e700ae000 tx=0x7f2e700a7000 comp rx=0 tx=0).stop 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.057+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e8015c560 msgr2=0x7f2e801578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.057+0000 7f2e85600640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e8015c560 0x7f2e801578e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800c7a0 tx=0x7f2e7800cc60 comp rx=0 tx=0).stop 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.061+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 shutdown_connections 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.061+0000 7f2e85600640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2e60068250 0x7f2e6006a710 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.061+0000 7f2e85600640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e8015c560 0x7f2e801578e0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.061+0000 7f2e85600640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e80155970 0x7f2e80155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.061+0000 7f2e85600640 1 --2- 172.21.15.110:0/897983865 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2e8006f010 0x7f2e80151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.061+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 >> 172.21.15.110:0/897983865 conn(0x7f2e8006bc50 msgr2=0x7f2e80154c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.061+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 shutdown_connections 2024-09-10T10:05:59.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.061+0000 7f2e85600640 1 -- 172.21.15.110:0/897983865 wait complete. 2024-09-10T10:05:59.080 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:59.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d8da00640 1 Processor -- start 2024-09-10T10:05:59.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d8da00640 1 -- start start 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d8da00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d8815c960 0x7f0d880b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d8da00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0d881603c0 0x7f0d880d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d8da00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d880706a0 con 0x7f0d880d04c0 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d8da00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8815cd40 con 0x7f0d881603c0 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d8da00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0d880b6370 con 0x7f0d8815c960 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d86000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0d881603c0 0x7f0d880d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d86000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0d881603c0 0x7f0d880d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52354/0 (socket says 172.21.15.110:52354) 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d86000640 1 -- 172.21.15.110:0/2708814097 learned_addr learned my addr 172.21.15.110:0/2708814097 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d86a00640 1 --2- 172.21.15.110:0/2708814097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d8815c960 0x7f0d880b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d86000640 1 -- 172.21.15.110:0/2708814097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d8815c960 msgr2=0x7f0d880b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:59.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d86000640 1 --2- 172.21.15.110:0/2708814097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d8815c960 0x7f0d880b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d86000640 1 -- 172.21.15.110:0/2708814097 >> v1:172.21.15.145:6789/0 conn(0x7f0d880d04c0 legacy=0x7f0d88070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.217+0000 7f0d86000640 1 -- 172.21.15.110:0/2708814097 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d88070a80 con 0x7f0d881603c0 2024-09-10T10:05:59.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.221+0000 7f0d86000640 1 --2- 172.21.15.110:0/2708814097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d881603c0 0x7f0d880d0cd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009870 tx=0x7f0d7401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccadcf6a1b649514 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.221+0000 7f0d85600640 1 -- 172.21.15.110:0/2708814097 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740b0070 con 0x7f0d881603c0 2024-09-10T10:05:59.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.221+0000 7f0d85600640 1 -- 172.21.15.110:0/2708814097 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d740ab600 con 0x7f0d881603c0 2024-09-10T10:05:59.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.221+0000 7f0d85600640 1 -- 172.21.15.110:0/2708814097 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740ab9d0 con 0x7f0d881603c0 2024-09-10T10:05:59.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.221+0000 7f0d8da00640 1 -- 172.21.15.110:0/2708814097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d881603c0 msgr2=0x7f0d880d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.221+0000 7f0d8da00640 1 --2- 172.21.15.110:0/2708814097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d881603c0 0x7f0d880d0cd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009870 tx=0x7f0d7401c410 comp rx=0 tx=0).stop 2024-09-10T10:05:59.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 -- 172.21.15.110:0/2708814097 shutdown_connections 2024-09-10T10:05:59.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 --2- 172.21.15.110:0/2708814097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d881603c0 0x7f0d880d0cd0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 --2- 172.21.15.110:0/2708814097 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d8815c960 0x7f0d880b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 -- 172.21.15.110:0/2708814097 >> 172.21.15.110:0/2708814097 conn(0x7f0d880d25c0 msgr2=0x7f0d880d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:59.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 -- 172.21.15.110:0/2708814097 shutdown_connections 2024-09-10T10:05:59.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 -- 172.21.15.110:0/2708814097 wait complete. 2024-09-10T10:05:59.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 Processor -- start 2024-09-10T10:05:59.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 -- start start 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d880d04c0 0x7f0d88159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d8815c960 0x7f0d8806e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881603c0 0x7f0d88157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880d3150 con 0x7f0d8815c960 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881611b0 con 0x7f0d881603c0 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d8da00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0d880d32d0 con 0x7f0d880d04c0 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d8815c960 0x7f0d8806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d8815c960 0x7f0d8806e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52362/0 (socket says 172.21.15.110:52362) 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86000640 1 -- 172.21.15.110:0/945854527 learned_addr learned my addr 172.21.15.110:0/945854527 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86a00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d880d04c0 0x7f0d88159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d87400640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881603c0 0x7f0d88157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86000640 1 -- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d880d04c0 msgr2=0x7f0d88159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86000640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d880d04c0 0x7f0d88159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86000640 1 -- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881603c0 msgr2=0x7f0d88157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86000640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881603c0 0x7f0d88157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d86000640 1 -- 172.21.15.110:0/945854527 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d88153a70 con 0x7f0d8815c960 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.225+0000 7f0d87400640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881603c0 0x7f0d88157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d86000640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d8815c960 0x7f0d8806e570 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0d740ae040 tx=0x7f0d740191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d7ea00640 1 -- 172.21.15.110:0/945854527 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740b0040 con 0x7f0d8815c960 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d7ea00640 1 -- 172.21.15.110:0/945854527 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d740ab400 con 0x7f0d8815c960 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d88153d50 con 0x7f0d8815c960 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d7ea00640 1 -- 172.21.15.110:0/945854527 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740b4680 con 0x7f0d8815c960 2024-09-10T10:05:59.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d88154310 con 0x7f0d8815c960 2024-09-10T10:05:59.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d88071a80 con 0x7f0d8815c960 2024-09-10T10:05:59.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d7ea00640 1 -- 172.21.15.110:0/945854527 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f0d740bb020 con 0x7f0d8815c960 2024-09-10T10:05:59.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d7ea00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0d68068280 0x7f0d6806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d7ea00640 1 -- 172.21.15.110:0/945854527 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f0d7411d7a0 con 0x7f0d8815c960 2024-09-10T10:05:59.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.229+0000 7f0d86a00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0d68068280 0x7f0d6806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.233+0000 7f0d86a00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0d68068280 0x7f0d6806a740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0d780073f0 tx=0x7f0d7800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.233+0000 7f0d7ea00640 1 -- 172.21.15.110:0/945854527 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0d740c2020 con 0x7f0d8815c960 2024-09-10T10:05:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d88167270 con 0x7f0d8815c960 2024-09-10T10:05:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d7ea00640 1 -- 172.21.15.110:0/945854527 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5245 (secure 0 0 0) 0x7f0d74016150 con 0x7f0d8815c960 2024-09-10T10:05:59.430 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:59.430 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":107,"btime":"2024-09-10T10:05:45:243547+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3244721906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3244721906},{"type":"v1","addr":"172.21.15.145:6834","nonce":3244721906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/1342717567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1342717567},{"type":"v1","addr":"172.21.15.110:6834","nonce":1342717567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1495976904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1495976904},{"type":"v1","addr":"172.21.15.110:6835","nonce":1495976904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:31.551366+0000","modified":"2024-09-10T10:05:34.580103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1869854747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1869854747},{"type":"v1","addr":"172.21.15.145:6835","nonce":1869854747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":12}]} 2024-09-10T10:05:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 107 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0d68068280 msgr2=0x7f0d6806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0d68068280 0x7f0d6806a740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0d780073f0 tx=0x7f0d7800b040 comp rx=0 tx=0).stop 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d8815c960 msgr2=0x7f0d8806e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d8815c960 0x7f0d8806e570 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0d740ae040 tx=0x7f0d740191b0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 shutdown_connections 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0d68068280 0x7f0d6806a740 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d881603c0 0x7f0d88157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d8815c960 0x7f0d8806e570 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 --2- 172.21.15.110:0/945854527 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0d880d04c0 0x7f0d88159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 >> 172.21.15.110:0/945854527 conn(0x7f0d880d25c0 msgr2=0x7f0d88073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:59.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 shutdown_connections 2024-09-10T10:05:59.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.429+0000 7f0d8da00640 1 -- 172.21.15.110:0/945854527 wait complete. 2024-09-10T10:05:59.451 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:05:59.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.585+0000 7fdfd8a00640 1 Processor -- start 2024-09-10T10:05:59.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.585+0000 7fdfd8a00640 1 -- start start 2024-09-10T10:05:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd8a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdfd40b4bd0 0x7fdfd406e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd8a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdfd4163a60 0x7fdfd4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd8a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd406d770 con 0x7fdfd4071ff0 2024-09-10T10:05:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd8a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd4072440 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd8a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd406da50 con 0x7fdfd4163a60 2024-09-10T10:05:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd1e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdfd40b4bd0 0x7fdfd406e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd1e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdfd40b4bd0 0x7fdfd406e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52372/0 (socket says 172.21.15.110:52372) 2024-09-10T10:05:59.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd1e00640 1 -- 172.21.15.110:0/2436052026 learned_addr learned my addr 172.21.15.110:0/2436052026 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:59.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfc9400640 1 --2- 172.21.15.110:0/2436052026 >> v2:172.21.15.145:3300/0 conn(0x7fdfd4163a60 0x7fdfd4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd1e00640 1 -- 172.21.15.110:0/2436052026 >> v2:172.21.15.145:3300/0 conn(0x7fdfd4163a60 msgr2=0x7fdfd4071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:59.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd1e00640 1 --2- 172.21.15.110:0/2436052026 >> v2:172.21.15.145:3300/0 conn(0x7fdfd4163a60 0x7fdfd4071ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd1e00640 1 -- 172.21.15.110:0/2436052026 >> v1:172.21.15.145:6789/0 conn(0x7fdfd4071ff0 legacy=0x7fdfd415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:59.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfd1e00640 1 -- 172.21.15.110:0/2436052026 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd406dd30 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.589+0000 7fdfc9400640 1 --2- 172.21.15.110:0/2436052026 >> v2:172.21.15.145:3300/0 conn(0x7fdfd4163a60 0x7fdfd4071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:59.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd1e00640 1 --2- 172.21.15.110:0/2436052026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd406e810 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40731c0 tx=0x7fdfc401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a81afbe6698b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd1400640 1 -- 172.21.15.110:0/2436052026 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40a5840 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd1400640 1 -- 172.21.15.110:0/2436052026 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40a59e0 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd1400640 1 -- 172.21.15.110:0/2436052026 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40a5db0 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 -- 172.21.15.110:0/2436052026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 msgr2=0x7fdfd406e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/2436052026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd406e810 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40731c0 tx=0x7fdfc401c980 comp rx=0 tx=0).stop 2024-09-10T10:05:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 -- 172.21.15.110:0/2436052026 shutdown_connections 2024-09-10T10:05:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/2436052026 >> v2:172.21.15.145:3300/0 conn(0x7fdfd4163a60 0x7fdfd4071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/2436052026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd406e810 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 -- 172.21.15.110:0/2436052026 >> 172.21.15.110:0/2436052026 conn(0x7fdfd406bc50 msgr2=0x7fdfd406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 -- 172.21.15.110:0/2436052026 shutdown_connections 2024-09-10T10:05:59.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 -- 172.21.15.110:0/2436052026 wait complete. 2024-09-10T10:05:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 Processor -- start 2024-09-10T10:05:59.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 -- start start 2024-09-10T10:05:59.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfd4071ff0 0x7fdfd414dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd41586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.593+0000 7fdfd8a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4163a60 0x7fdfd415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd8a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd406d530 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd406d3b0 con 0x7fdfd4071ff0 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd8a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40b4fb0 con 0x7fdfd4163a60 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd41586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd41586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52376/0 (socket says 172.21.15.110:52376) 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 -- 172.21.15.110:0/1504003504 learned_addr learned my addr 172.21.15.110:0/1504003504 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd1e00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfd4071ff0 0x7fdfd414dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd2800640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4163a60 0x7fdfd415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 -- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4163a60 msgr2=0x7fdfd415c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4163a60 0x7fdfd415c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 -- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfd4071ff0 msgr2=0x7fdfd414dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfd4071ff0 0x7fdfd414dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 -- 172.21.15.110:0/1504003504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd415b2c0 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd2800640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4163a60 0x7fdfd415c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfc9400640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd41586b0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c7c0 tx=0x7fdfbc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfcaa00640 1 -- 172.21.15.110:0/1504003504 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc007a30 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfcaa00640 1 -- 172.21.15.110:0/1504003504 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc007bd0 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfcaa00640 1 -- 172.21.15.110:0/1504003504 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0113f0 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd415b5a0 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd415bb60 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4163e70 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.597+0000 7fdfcaa00640 1 -- 172.21.15.110:0/1504003504 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fdfbc011590 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.601+0000 7fdfcaa00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfb0068250 0x7fdfb006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.601+0000 7fdfcaa00640 1 -- 172.21.15.110:0/1504003504 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7fdfbc084550 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.601+0000 7fdfd1e00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfb0068250 0x7fdfb006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.601+0000 7fdfd1e00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfb0068250 0x7fdfb006a710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40ae000 tx=0x7fdfc40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.605+0000 7fdfcaa00640 1 -- 172.21.15.110:0/1504003504 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdfbc015020 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.761+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfd4163780 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfcaa00640 1 -- 172.21.15.110:0/1504003504 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16864 (secure 0 0 0) 0x7fdfbc056640 con 0x7fdfd40b4bd0 2024-09-10T10:05:59.765 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:05:59.765 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":153,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:33.186967+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:29.268211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:30.557102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfb0068250 msgr2=0x7fdfb006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfb0068250 0x7fdfb006a710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40ae000 tx=0x7fdfc40a7070 comp rx=0 tx=0).stop 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 msgr2=0x7fdfd41586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd41586b0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c7c0 tx=0x7fdfbc00cc80 comp rx=0 tx=0).stop 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 shutdown_connections 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdfb0068250 0x7fdfb006a710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdfd4163a60 0x7fdfd415c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd40b4bd0 0x7fdfd41586b0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 --2- 172.21.15.110:0/1504003504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfd4071ff0 0x7fdfd414dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 >> 172.21.15.110:0/1504003504 conn(0x7fdfd406bc50 msgr2=0x7fdfd4164a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:59.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 shutdown_connections 2024-09-10T10:05:59.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.765+0000 7fdfd8a00640 1 -- 172.21.15.110:0/1504003504 wait complete. 2024-09-10T10:05:59.787 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:05:59.787 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:05:59.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f7c200640 1 Processor -- start 2024-09-10T10:05:59.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f7c200640 1 -- start start 2024-09-10T10:05:59.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f7c200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6f7415e700 0x7f6f7406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f7c200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6f7406ff60 0x7f6f740af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f7c200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f740703a0 con 0x7f6f740729b0 2024-09-10T10:05:59.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f7c200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7415eae0 con 0x7f6f7415e700 2024-09-10T10:05:59.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f7c200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7406eec0 con 0x7f6f7406ff60 2024-09-10T10:05:59.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f79800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6f7415e700 0x7f6f7406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f79800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6f7415e700 0x7f6f7406e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52392/0 (socket says 172.21.15.110:52392) 2024-09-10T10:05:59.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f79800640 1 -- 172.21.15.110:0/2526281381 learned_addr learned my addr 172.21.15.110:0/2526281381 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:59.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f79800640 1 -- 172.21.15.110:0/2526281381 >> v2:172.21.15.145:3300/0 conn(0x7f6f7406ff60 msgr2=0x7f6f740af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:59.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f79800640 1 --2- 172.21.15.110:0/2526281381 >> v2:172.21.15.145:3300/0 conn(0x7f6f7406ff60 0x7f6f740af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f79800640 1 -- 172.21.15.110:0/2526281381 >> v1:172.21.15.145:6789/0 conn(0x7f6f740729b0 legacy=0x7f6f74072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:59.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.921+0000 7f6f79800640 1 -- 172.21.15.110:0/2526281381 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7406f1a0 con 0x7f6f7415e700 2024-09-10T10:05:59.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f79800640 1 --2- 172.21.15.110:0/2526281381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f7406e890 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6f7406e040 tx=0x7f6f6801c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e20bbc7f5900d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f73e00640 1 -- 172.21.15.110:0/2526281381 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f680a5660 con 0x7f6f7415e700 2024-09-10T10:05:59.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f73e00640 1 -- 172.21.15.110:0/2526281381 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f680a5800 con 0x7f6f7415e700 2024-09-10T10:05:59.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f73e00640 1 -- 172.21.15.110:0/2526281381 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f680a5bd0 con 0x7f6f7415e700 2024-09-10T10:05:59.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f7c200640 1 -- 172.21.15.110:0/2526281381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 msgr2=0x7f6f7406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f7c200640 1 --2- 172.21.15.110:0/2526281381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f7406e890 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6f7406e040 tx=0x7f6f6801c7a0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f7c200640 1 -- 172.21.15.110:0/2526281381 shutdown_connections 2024-09-10T10:05:59.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f7c200640 1 --2- 172.21.15.110:0/2526281381 >> v2:172.21.15.145:3300/0 conn(0x7f6f7406ff60 0x7f6f740af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f7c200640 1 --2- 172.21.15.110:0/2526281381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f7406e890 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f7c200640 1 -- 172.21.15.110:0/2526281381 >> 172.21.15.110:0/2526281381 conn(0x7f6f7406cb90 msgr2=0x7f6f7406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:59.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f7c200640 1 -- 172.21.15.110:0/2526281381 shutdown_connections 2024-09-10T10:05:59.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.925+0000 7f6f7c200640 1 -- 172.21.15.110:0/2526281381 wait complete. 2024-09-10T10:05:59.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 Processor -- start 2024-09-10T10:05:59.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 -- start start 2024-09-10T10:05:59.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f7406ff60 0x7f6f7414bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f740729b0 0x7f6f7415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f74148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740af930 con 0x7f6f7415e700 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740af7b0 con 0x7f6f7406ff60 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6f740b4660 con 0x7f6f740729b0 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f74148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f74148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52408/0 (socket says 172.21.15.110:52408) 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 -- 172.21.15.110:0/3342732767 learned_addr learned my addr 172.21.15.110:0/3342732767 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f78e00640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f740729b0 0x7f6f7415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f79800640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f7406ff60 0x7f6f7414bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 -- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f740729b0 msgr2=0x7f6f7415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f740729b0 0x7f6f7415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 -- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f7406ff60 msgr2=0x7f6f7414bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f7406ff60 0x7f6f7414bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 -- 172.21.15.110:0/3342732767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7414b730 con 0x7f6f7415e700 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f78e00640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f740729b0 0x7f6f7415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f79800640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f7406ff60 0x7f6f7414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7a200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f74148eb0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c440 tx=0x7f6f6c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f71600640 1 -- 172.21.15.110:0/3342732767 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c017070 con 0x7f6f7415e700 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f71600640 1 -- 172.21.15.110:0/3342732767 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f6c003600 con 0x7f6f7415e700 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741486d0 con 0x7f6f7415e700 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f71600640 1 -- 172.21.15.110:0/3342732767 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0039f0 con 0x7f6f7415e700 2024-09-10T10:05:59.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7414b9b0 con 0x7f6f7415e700 2024-09-10T10:05:59.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.929+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7414bbb0 con 0x7f6f7415e700 2024-09-10T10:05:59.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.933+0000 7f6f71600640 1 -- 172.21.15.110:0/3342732767 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f6f6c01a070 con 0x7f6f7415e700 2024-09-10T10:05:59.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.933+0000 7f6f71600640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f54068250 0x7f6f5406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:59.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.933+0000 7f6f71600640 1 -- 172.21.15.110:0/3342732767 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f6f6c083780 con 0x7f6f7415e700 2024-09-10T10:05:59.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.933+0000 7f6f79800640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f54068250 0x7f6f5406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:59.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.933+0000 7f6f79800640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f54068250 0x7f6f5406a710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f680ae000 tx=0x7f6f680a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:59.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:05:59.937+0000 7f6f71600640 1 -- 172.21.15.110:0/3342732767 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6f7414bbb0 con 0x7f6f7415e700 2024-09-10T10:06:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.125+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f740b4a10 con 0x7f6f7415e700 2024-09-10T10:06:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.125+0000 7f6f71600640 1 -- 172.21.15.110:0/3342732767 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5245 (secure 0 0 0) 0x7f6f7414b730 con 0x7f6f7415e700 2024-09-10T10:06:00.128 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:00.128 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":107,"btime":"2024-09-10T10:05:45:243547+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3244721906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3244721906},{"type":"v1","addr":"172.21.15.145:6834","nonce":3244721906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/1342717567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1342717567},{"type":"v1","addr":"172.21.15.110:6834","nonce":1342717567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1495976904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1495976904},{"type":"v1","addr":"172.21.15.110:6835","nonce":1495976904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:31.551366+0000","modified":"2024-09-10T10:05:34.580103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1869854747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1869854747},{"type":"v1","addr":"172.21.15.145:6835","nonce":1869854747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":12}]} 2024-09-10T10:06:00.129 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 107 2024-09-10T10:06:00.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f54068250 msgr2=0x7f6f5406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f54068250 0x7f6f5406a710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f680ae000 tx=0x7f6f680a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:00.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 msgr2=0x7f6f74148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f74148eb0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c440 tx=0x7f6f6c00c900 comp rx=0 tx=0).stop 2024-09-10T10:06:00.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 shutdown_connections 2024-09-10T10:06:00.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6f54068250 0x7f6f5406a710 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f7415e700 0x7f6f74148eb0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6f740729b0 0x7f6f7415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 --2- 172.21.15.110:0/3342732767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f7406ff60 0x7f6f7414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 >> 172.21.15.110:0/3342732767 conn(0x7f6f7406cb90 msgr2=0x7f6f7415ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:00.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 shutdown_connections 2024-09-10T10:06:00.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.129+0000 7f6f7c200640 1 -- 172.21.15.110:0/3342732767 wait complete. 2024-09-10T10:06:00.148 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:06:00.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.281+0000 7f5a4ae00640 1 Processor -- start 2024-09-10T10:06:00.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a4ae00640 1 -- start start 2024-09-10T10:06:00.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a4ae00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5a44148120 0x7f5a4406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a4ae00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5a440ae840 0x7f5a440aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a4ae00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a440702b0 con 0x7f5a440b0890 2024-09-10T10:06:00.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a4ae00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5a440af1f0 con 0x7f5a44148120 2024-09-10T10:06:00.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a4ae00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4406eb00 con 0x7f5a440ae840 2024-09-10T10:06:00.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a43e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5a44148120 0x7f5a4406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a43e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5a44148120 0x7f5a4406e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52422/0 (socket says 172.21.15.110:52422) 2024-09-10T10:06:00.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a43e00640 1 -- 172.21.15.110:0/2751921743 learned_addr learned my addr 172.21.15.110:0/2751921743 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:00.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a48e00640 1 --2- 172.21.15.110:0/2751921743 >> v2:172.21.15.145:3300/0 conn(0x7f5a440ae840 0x7f5a440aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a43e00640 1 -- 172.21.15.110:0/2751921743 >> v2:172.21.15.145:3300/0 conn(0x7f5a440ae840 msgr2=0x7f5a440aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:00.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a43e00640 1 --2- 172.21.15.110:0/2751921743 >> v2:172.21.15.145:3300/0 conn(0x7f5a440ae840 0x7f5a440aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a43e00640 1 -- 172.21.15.110:0/2751921743 >> v1:172.21.15.145:6789/0 conn(0x7f5a440b0890 legacy=0x7f5a4406fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:00.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a43e00640 1 -- 172.21.15.110:0/2751921743 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4406ede0 con 0x7f5a44148120 2024-09-10T10:06:00.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.285+0000 7f5a48e00640 1 --2- 172.21.15.110:0/2751921743 >> v2:172.21.15.145:3300/0 conn(0x7f5a440ae840 0x7f5a440aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:00.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a43e00640 1 --2- 172.21.15.110:0/2751921743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a44148120 0x7f5a4406e1b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5a440d13c0 tx=0x7f5a3401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725bd05b8a2aca7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:00.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a42a00640 1 -- 172.21.15.110:0/2751921743 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340b1070 con 0x7f5a44148120 2024-09-10T10:06:00.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a42a00640 1 -- 172.21.15.110:0/2751921743 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a340ac630 con 0x7f5a44148120 2024-09-10T10:06:00.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a42a00640 1 -- 172.21.15.110:0/2751921743 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340a8050 con 0x7f5a44148120 2024-09-10T10:06:00.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 -- 172.21.15.110:0/2751921743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a44148120 msgr2=0x7f5a4406e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/2751921743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a44148120 0x7f5a4406e1b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5a440d13c0 tx=0x7f5a3401c4e0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 -- 172.21.15.110:0/2751921743 shutdown_connections 2024-09-10T10:06:00.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/2751921743 >> v2:172.21.15.145:3300/0 conn(0x7f5a440ae840 0x7f5a440aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/2751921743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a44148120 0x7f5a4406e1b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 -- 172.21.15.110:0/2751921743 >> 172.21.15.110:0/2751921743 conn(0x7f5a4406cf60 msgr2=0x7f5a4406d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:00.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 -- 172.21.15.110:0/2751921743 shutdown_connections 2024-09-10T10:06:00.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 -- 172.21.15.110:0/2751921743 wait complete. 2024-09-10T10:06:00.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 Processor -- start 2024-09-10T10:06:00.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.289+0000 7f5a4ae00640 1 -- start start 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a4ae00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a440ae840 0x7f5a441451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a4ae00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a440b0890 0x7f5a44140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a4ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a44148120 0x7f5a44146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a4ae00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4416b340 con 0x7f5a440ae840 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a4ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4416b1c0 con 0x7f5a44148120 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a4ae00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4416b4c0 con 0x7f5a440b0890 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a440ae840 0x7f5a441451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a440ae840 0x7f5a441451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52438/0 (socket says 172.21.15.110:52438) 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 -- 172.21.15.110:0/4282704776 learned_addr learned my addr 172.21.15.110:0/4282704776 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a48e00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a44148120 0x7f5a44146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43400640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a440b0890 0x7f5a44140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 -- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a440b0890 msgr2=0x7f5a44140030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a440b0890 0x7f5a44140030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 -- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a44148120 msgr2=0x7f5a44146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a44148120 0x7f5a44146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 -- 172.21.15.110:0/4282704776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a441420d0 con 0x7f5a440ae840 2024-09-10T10:06:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43400640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a440b0890 0x7f5a44140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a48e00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a44148120 0x7f5a44146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a43e00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a440ae840 0x7f5a441451d0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5a3401ab90 tx=0x7f5a34019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:00.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a3be00640 1 -- 172.21.15.110:0/4282704776 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340bc070 con 0x7f5a440ae840 2024-09-10T10:06:00.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a3be00640 1 -- 172.21.15.110:0/4282704776 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a340ab800 con 0x7f5a440ae840 2024-09-10T10:06:00.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a3be00640 1 -- 172.21.15.110:0/4282704776 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340b1070 con 0x7f5a440ae840 2024-09-10T10:06:00.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a44140d90 con 0x7f5a440ae840 2024-09-10T10:06:00.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.293+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a44141350 con 0x7f5a440ae840 2024-09-10T10:06:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.297+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a440aecb0 con 0x7f5a440ae840 2024-09-10T10:06:00.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.297+0000 7f5a3be00640 1 -- 172.21.15.110:0/4282704776 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f5a340ac4f0 con 0x7f5a440ae840 2024-09-10T10:06:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.297+0000 7f5a3be00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a24068280 0x7f5a2406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.297+0000 7f5a3be00640 1 -- 172.21.15.110:0/4282704776 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f5a3411de80 con 0x7f5a440ae840 2024-09-10T10:06:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.297+0000 7f5a43400640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a24068280 0x7f5a2406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.297+0000 7f5a43400640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a24068280 0x7f5a2406a740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5a440b3e60 tx=0x7f5a30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:00.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.305+0000 7f5a3be00640 1 -- 172.21.15.110:0/4282704776 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5a441420d0 con 0x7f5a440ae840 2024-09-10T10:06:00.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.493+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a44168f90 con 0x7f5a440ae840 2024-09-10T10:06:00.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.493+0000 7f5a3be00640 1 -- 172.21.15.110:0/4282704776 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5245 (secure 0 0 0) 0x7f5a340160d0 con 0x7f5a440ae840 2024-09-10T10:06:00.496 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:00.496 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":107,"btime":"2024-09-10T10:05:45:243547+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3244721906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3244721906},{"type":"v1","addr":"172.21.15.145:6834","nonce":3244721906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/1342717567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1342717567},{"type":"v1","addr":"172.21.15.110:6834","nonce":1342717567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1495976904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1495976904},{"type":"v1","addr":"172.21.15.110:6835","nonce":1495976904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:05:31.551366+0000","modified":"2024-09-10T10:05:34.580103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/1869854747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":1869854747},{"type":"v1","addr":"172.21.15.145:6835","nonce":1869854747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":12}]} 2024-09-10T10:06:00.497 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 107 2024-09-10T10:06:00.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a24068280 msgr2=0x7f5a2406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a24068280 0x7f5a2406a740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5a440b3e60 tx=0x7f5a30008040 comp rx=0 tx=0).stop 2024-09-10T10:06:00.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a440ae840 msgr2=0x7f5a441451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a440ae840 0x7f5a441451d0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5a3401ab90 tx=0x7f5a34019280 comp rx=0 tx=0).stop 2024-09-10T10:06:00.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 shutdown_connections 2024-09-10T10:06:00.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5a24068280 0x7f5a2406a740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a44148120 0x7f5a44146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5a440b0890 0x7f5a44140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 --2- 172.21.15.110:0/4282704776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a440ae840 0x7f5a441451d0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 >> 172.21.15.110:0/4282704776 conn(0x7f5a4406cf60 msgr2=0x7f5a440b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:00.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 shutdown_connections 2024-09-10T10:06:00.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.497+0000 7f5a4ae00640 1 -- 172.21.15.110:0/4282704776 wait complete. 2024-09-10T10:06:00.518 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:06:00.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa2600640 1 Processor -- start 2024-09-10T10:06:00.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa2600640 1 -- start start 2024-09-10T10:06:00.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa2600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdf9c15c620 0x7fdf9c0cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa2600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdf9c0b4900 0x7fdf9c0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa2600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c071640 con 0x7fdf9c15ca00 2024-09-10T10:06:00.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa2600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c0cf6f0 con 0x7fdf9c15c620 2024-09-10T10:06:00.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa2600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c071c00 con 0x7fdf9c0b4900 2024-09-10T10:06:00.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa1200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdf9c15c620 0x7fdf9c0cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa1200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdf9c15c620 0x7fdf9c0cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52442/0 (socket says 172.21.15.110:52442) 2024-09-10T10:06:00.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa1200640 1 -- 172.21.15.110:0/1507028084 learned_addr learned my addr 172.21.15.110:0/1507028084 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:00.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa1200640 1 -- 172.21.15.110:0/1507028084 >> v2:172.21.15.145:3300/0 conn(0x7fdf9c0b4900 msgr2=0x7fdf9c0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:00.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa1200640 1 --2- 172.21.15.110:0/1507028084 >> v2:172.21.15.145:3300/0 conn(0x7fdf9c0b4900 0x7fdf9c0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa1200640 1 -- 172.21.15.110:0/1507028084 >> v1:172.21.15.145:6789/0 conn(0x7fdf9c15ca00 legacy=0x7fdf9c0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:00.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.653+0000 7fdfa1200640 1 -- 172.21.15.110:0/1507028084 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c071920 con 0x7fdf9c15c620 2024-09-10T10:06:00.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa1200640 1 --2- 172.21.15.110:0/1507028084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c15c620 0x7fdf9c0cfaa0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0d2020 tx=0x7fdf8c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204085b6d713aea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:00.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdf9b400640 1 -- 172.21.15.110:0/1507028084 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0b0070 con 0x7fdf9c15c620 2024-09-10T10:06:00.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdf9b400640 1 -- 172.21.15.110:0/1507028084 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c0ab600 con 0x7fdf9c15c620 2024-09-10T10:06:00.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdf9b400640 1 -- 172.21.15.110:0/1507028084 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0a4e60 con 0x7fdf9c15c620 2024-09-10T10:06:00.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 -- 172.21.15.110:0/1507028084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c15c620 msgr2=0x7fdf9c0cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1507028084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c15c620 0x7fdf9c0cfaa0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0d2020 tx=0x7fdf8c01c3c0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 -- 172.21.15.110:0/1507028084 shutdown_connections 2024-09-10T10:06:00.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1507028084 >> v2:172.21.15.145:3300/0 conn(0x7fdf9c0b4900 0x7fdf9c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1507028084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c15c620 0x7fdf9c0cfaa0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 -- 172.21.15.110:0/1507028084 >> 172.21.15.110:0/1507028084 conn(0x7fdf9c06f440 msgr2=0x7fdf9c06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:00.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 -- 172.21.15.110:0/1507028084 shutdown_connections 2024-09-10T10:06:00.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 -- 172.21.15.110:0/1507028084 wait complete. 2024-09-10T10:06:00.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.657+0000 7fdfa2600640 1 Processor -- start 2024-09-10T10:06:00.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 -- start start 2024-09-10T10:06:00.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c0b4900 0x7fdf9c1fb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdf9c15c620 0x7fdf9c14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf9c15ca00 0x7fdf9c14ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c0b68b0 con 0x7fdf9c0b4900 2024-09-10T10:06:00.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c0b6730 con 0x7fdf9c15ca00 2024-09-10T10:06:00.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c0b6a30 con 0x7fdf9c15c620 2024-09-10T10:06:00.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c0b4900 0x7fdf9c1fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c0b4900 0x7fdf9c1fb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52450/0 (socket says 172.21.15.110:52450) 2024-09-10T10:06:00.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 -- 172.21.15.110:0/1360868483 learned_addr learned my addr 172.21.15.110:0/1360868483 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:00.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1c00640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf9c15ca00 0x7fdf9c14ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdf9be00640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdf9c15c620 0x7fdf9c14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 -- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdf9c15c620 msgr2=0x7fdf9c14e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdf9c15c620 0x7fdf9c14e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 -- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf9c15ca00 msgr2=0x7fdf9c14ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf9c15ca00 0x7fdf9c14ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 -- 172.21.15.110:0/1360868483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c0b24f0 con 0x7fdf9c0b4900 2024-09-10T10:06:00.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1c00640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf9c15ca00 0x7fdf9c14ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:00.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdf9be00640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdf9c15c620 0x7fdf9c14e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:00.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa1200640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c0b4900 0x7fdf9c1fb020 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0ae040 tx=0x7fdf8c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:00.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdf98c00640 1 -- 172.21.15.110:0/1360868483 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0ba020 con 0x7fdf9c0b4900 2024-09-10T10:06:00.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdf98c00640 1 -- 172.21.15.110:0/1360868483 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c0b6420 con 0x7fdf9c0b4900 2024-09-10T10:06:00.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c0b2790 con 0x7fdf9c0b4900 2024-09-10T10:06:00.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdf98c00640 1 -- 172.21.15.110:0/1360868483 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0b0040 con 0x7fdf9c0b4900 2024-09-10T10:06:00.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c0b2d50 con 0x7fdf9c0b4900 2024-09-10T10:06:00.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.661+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c0cf1b0 con 0x7fdf9c0b4900 2024-09-10T10:06:00.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.665+0000 7fdf98c00640 1 -- 172.21.15.110:0/1360868483 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fdf8c0bd020 con 0x7fdf9c0b4900 2024-09-10T10:06:00.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.665+0000 7fdf98c00640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdf7c068250 0x7fdf7c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.665+0000 7fdf98c00640 1 -- 172.21.15.110:0/1360868483 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7fdf8c11e6c0 con 0x7fdf9c0b4900 2024-09-10T10:06:00.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.665+0000 7fdf9be00640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdf7c068250 0x7fdf7c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.665+0000 7fdf9be00640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdf7c068250 0x7fdf7c06a710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c15bfe0 tx=0x7fdf88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:00.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.669+0000 7fdf98c00640 1 -- 172.21.15.110:0/1360868483 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdf8c0c4030 con 0x7fdf9c0b4900 2024-09-10T10:06:00.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.825+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf9c156060 con 0x7fdf9c0b4900 2024-09-10T10:06:00.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.825+0000 7fdf98c00640 1 -- 172.21.15.110:0/1360868483 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16864 (secure 0 0 0) 0x7fdf8c0160d0 con 0x7fdf9c0b4900 2024-09-10T10:06:00.828 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:00.828 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":153,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:05:33.186967+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:05:29.268211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T10:05:30.557102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:06:00.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdf7c068250 msgr2=0x7fdf7c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdf7c068250 0x7fdf7c06a710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c15bfe0 tx=0x7fdf88008040 comp rx=0 tx=0).stop 2024-09-10T10:06:00.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c0b4900 msgr2=0x7fdf9c1fb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c0b4900 0x7fdf9c1fb020 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0ae040 tx=0x7fdf8c01ac00 comp rx=0 tx=0).stop 2024-09-10T10:06:00.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 shutdown_connections 2024-09-10T10:06:00.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fdf7c068250 0x7fdf7c06a710 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdf9c15ca00 0x7fdf9c14ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fdf9c15c620 0x7fdf9c14e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 --2- 172.21.15.110:0/1360868483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf9c0b4900 0x7fdf9c1fb020 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 >> 172.21.15.110:0/1360868483 conn(0x7fdf9c06f440 msgr2=0x7fdf9c070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:00.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 shutdown_connections 2024-09-10T10:06:00.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.829+0000 7fdfa2600640 1 -- 172.21.15.110:0/1360868483 wait complete. 2024-09-10T10:06:00.849 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:06:00.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1514000640 1 Processor -- start 2024-09-10T10:06:00.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1514000640 1 -- start start 2024-09-10T10:06:00.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1514000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f150c0726e0 0x7f150c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1514000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c06efa0 0x7f150c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1514000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150c06f3e0 con 0x7f150c0b5430 2024-09-10T10:06:00.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1514000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f150c0b5880 con 0x7f150c0726e0 2024-09-10T10:06:00.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1514000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f150c0cee50 con 0x7f150c06efa0 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1511600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f150c0726e0 0x7f150c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1510c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c06efa0 0x7f150c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1512000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f150c0b5430 0x7f150c0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:36932/0 (socket says 172.21.15.110:36932) 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1512000640 1 -- 172.21.15.110:0/2378765459 learned_addr learned my addr 172.21.15.110:0/2378765459 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1511600640 1 -- 172.21.15.110:0/2378765459 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c06efa0 msgr2=0x7f150c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1511600640 1 --2- 172.21.15.110:0/2378765459 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c06efa0 0x7f150c0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1511600640 1 -- 172.21.15.110:0/2378765459 >> v1:172.21.15.145:6789/0 conn(0x7f150c0b5430 legacy=0x7f150c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1511600640 1 -- 172.21.15.110:0/2378765459 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c15fda0 con 0x7f150c0726e0 2024-09-10T10:06:00.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.985+0000 7f1510c00640 1 --2- 172.21.15.110:0/2378765459 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c06efa0 0x7f150c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:00.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1511600640 1 --2- 172.21.15.110:0/2378765459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c0726e0 0x7f150c0cf2e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1500009d90 tx=0x7f150001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a16f820a39c967ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:00.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f150be00640 1 -- 172.21.15.110:0/2378765459 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15000b2070 con 0x7f150c0726e0 2024-09-10T10:06:00.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f150be00640 1 -- 172.21.15.110:0/2378765459 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15000a5a00 con 0x7f150c0726e0 2024-09-10T10:06:00.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f150be00640 1 -- 172.21.15.110:0/2378765459 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15000a5dd0 con 0x7f150c0726e0 2024-09-10T10:06:00.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1514000640 1 -- 172.21.15.110:0/2378765459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c0726e0 msgr2=0x7f150c0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1514000640 1 --2- 172.21.15.110:0/2378765459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c0726e0 0x7f150c0cf2e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1500009d90 tx=0x7f150001c940 comp rx=0 tx=0).stop 2024-09-10T10:06:00.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1514000640 1 -- 172.21.15.110:0/2378765459 shutdown_connections 2024-09-10T10:06:00.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1514000640 1 --2- 172.21.15.110:0/2378765459 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c06efa0 0x7f150c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1514000640 1 --2- 172.21.15.110:0/2378765459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c0726e0 0x7f150c0cf2e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1514000640 1 -- 172.21.15.110:0/2378765459 >> 172.21.15.110:0/2378765459 conn(0x7f150c062d10 msgr2=0x7f150c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:00.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1514000640 1 -- 172.21.15.110:0/2378765459 shutdown_connections 2024-09-10T10:06:00.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.989+0000 7f1514000640 1 -- 172.21.15.110:0/2378765459 wait complete. 2024-09-10T10:06:00.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 Processor -- start 2024-09-10T10:06:00.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 -- start start 2024-09-10T10:06:00.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c06efa0 0x7f150c0af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c0726e0 0x7f150c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f150c0b5430 0x7f150c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0b3ff0 con 0x7f150c0b5430 2024-09-10T10:06:00.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f150c073270 con 0x7f150c06efa0 2024-09-10T10:06:00.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f150c0b4170 con 0x7f150c0726e0 2024-09-10T10:06:00.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f150c0b5430 0x7f150c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f150c0b5430 0x7f150c0b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52458/0 (socket says 172.21.15.110:52458) 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 -- 172.21.15.110:0/3736688699 learned_addr learned my addr 172.21.15.110:0/3736688699 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1510c00640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c0726e0 0x7f150c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1511600640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c06efa0 0x7f150c0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 -- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c0726e0 msgr2=0x7f150c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c0726e0 0x7f150c154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 -- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c06efa0 msgr2=0x7f150c0af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c06efa0 0x7f150c0af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 -- 172.21.15.110:0/3736688699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c0b2140 con 0x7f150c0b5430 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1512000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f150c0b5430 0x7f150c0b0e70 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f150400c400 tx=0x7f150400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1509600640 1 -- 172.21.15.110:0/3736688699 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504017070 con 0x7f150c0b5430 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1509600640 1 -- 172.21.15.110:0/3736688699 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1504005cf0 con 0x7f150c0b5430 2024-09-10T10:06:00.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1509600640 1 -- 172.21.15.110:0/3736688699 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040072d0 con 0x7f150c0b5430 2024-09-10T10:06:00.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.993+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c0af950 con 0x7f150c0b5430 2024-09-10T10:06:00.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.997+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c0b0670 con 0x7f150c0b5430 2024-09-10T10:06:00.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.997+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c072ac0 con 0x7f150c0b5430 2024-09-10T10:06:00.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.997+0000 7f1509600640 1 -- 172.21.15.110:0/3736688699 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f150401a070 con 0x7f150c0b5430 2024-09-10T10:06:00.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.997+0000 7f1509600640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f14ec068250 0x7f14ec06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:00.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.997+0000 7f1509600640 1 -- 172.21.15.110:0/3736688699 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5765+0+0 (secure 0 0 0) 0x7f1504083630 con 0x7f150c0b5430 2024-09-10T10:06:00.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:00.997+0000 7f1511600640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f14ec068250 0x7f14ec06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:01.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.001+0000 7f1511600640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f14ec068250 0x7f14ec06a710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f15000af000 tx=0x7f15000a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:01.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.001+0000 7f1509600640 1 -- 172.21.15.110:0/3736688699 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1504055720 con 0x7f150c0b5430 2024-09-10T10:06:01.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.189+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f150c155380 con 0x7f150c0b5430 2024-09-10T10:06:01.192 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:06:01.189+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:06:01.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.193+0000 7f1509600640 1 -- 172.21.15.110:0/3736688699 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) ==== 161+0+0 (secure 0 0 0) 0x7f1504059a20 con 0x7f150c0b5430 2024-09-10T10:06:01.194 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:06:01.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f14ec068250 msgr2=0x7f14ec06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:01.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f14ec068250 0x7f14ec06a710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f15000af000 tx=0x7f15000a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:01.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f150c0b5430 msgr2=0x7f150c0b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:01.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f150c0b5430 0x7f150c0b0e70 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f150400c400 tx=0x7f150400c8c0 comp rx=0 tx=0).stop 2024-09-10T10:06:01.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 shutdown_connections 2024-09-10T10:06:01.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f14ec068250 0x7f14ec06a710 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:01.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f150c0b5430 0x7f150c0b0e70 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:01.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f150c0726e0 0x7f150c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:01.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 --2- 172.21.15.110:0/3736688699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f150c06efa0 0x7f150c0af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:01.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 >> 172.21.15.110:0/3736688699 conn(0x7f150c062d10 msgr2=0x7f150c0b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:01.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 shutdown_connections 2024-09-10T10:06:01.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.197+0000 7f1514000640 1 -- 172.21.15.110:0/3736688699 wait complete. 2024-09-10T10:06:01.216 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:06:01.266 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:06:01.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5be1600640 1 Processor -- start 2024-09-10T10:06:01.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5be1600640 1 -- start start 2024-09-10T10:06:01.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5be1600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5bdc0b46c0 0x7f5bdc0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:01.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5be1600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bdc15d370 con 0x7f5bdc15c6c0 2024-09-10T10:06:01.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5be1600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bdc15d650 con 0x7f5bdc15c2e0 2024-09-10T10:06:01.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5be1600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5bdc15d930 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5bdb400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5bdc0b46c0 0x7f5bdc0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:01.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5be0c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5bdc15c6c0 0x7f5bdc15cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:34616/0 (socket says 172.21.15.110:34616) 2024-09-10T10:06:01.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5bdb400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5bdc0b46c0 0x7f5bdc0b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52466/0 (socket says 172.21.15.110:52466) 2024-09-10T10:06:01.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.353+0000 7f5be0c00640 1 -- 172.21.15.110:0/1832280635 learned_addr learned my addr 172.21.15.110:0/1832280635 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:01.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.357+0000 7f5bdb400640 1 -- 172.21.15.110:0/1832280635 >> v1:172.21.15.145:6789/0 conn(0x7f5bdc15c2e0 legacy=0x7f5bdc0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:01.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.357+0000 7f5bdb400640 1 -- 172.21.15.110:0/1832280635 >> v1:172.21.15.110:6789/0 conn(0x7f5bdc15c6c0 legacy=0x7f5bdc15cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:01.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.357+0000 7f5bdb400640 1 -- 172.21.15.110:0/1832280635 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc0b5690 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.357+0000 7f5bdaa00640 1 -- 172.21.15.110:0/1832280635 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3207531769 0 0) 0x7f5bdc15d370 con 0x7f5bdc15c6c0 2024-09-10T10:06:01.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.357+0000 7f5bdb400640 1 --2- 172.21.15.110:0/1832280635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc0b4b50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80097c0 tx=0x7f5bc801cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60542e74ec1cb63a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:01.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.357+0000 7f5bdaa00640 1 -- 172.21.15.110:0/1832280635 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80b1070 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.357+0000 7f5bdaa00640 1 -- 172.21.15.110:0/1832280635 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc80ac410 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.357+0000 7f5bdaa00640 1 -- 172.21.15.110:0/1832280635 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80ac7e0 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- 172.21.15.110:0/1832280635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 msgr2=0x7f5bdc0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:01.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 --2- 172.21.15.110:0/1832280635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc0b4b50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80097c0 tx=0x7f5bc801cf00 comp rx=0 tx=0).stop 2024-09-10T10:06:01.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- 172.21.15.110:0/1832280635 shutdown_connections 2024-09-10T10:06:01.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 --2- 172.21.15.110:0/1832280635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc0b4b50 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:01.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- 172.21.15.110:0/1832280635 >> 172.21.15.110:0/1832280635 conn(0x7f5bdc0b7be0 msgr2=0x7f5bdc0b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:01.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- 172.21.15.110:0/1832280635 shutdown_connections 2024-09-10T10:06:01.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- 172.21.15.110:0/1832280635 wait complete. 2024-09-10T10:06:01.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 Processor -- start 2024-09-10T10:06:01.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- start start 2024-09-10T10:06:01.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc14cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:01.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc15c2e0 0x7f5bdc159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:01.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc15c6c0 0x7f5bdc157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:01.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc0d0910 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc0d0790 con 0x7f5bdc15c6c0 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be1600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc158d40 con 0x7f5bdc15c2e0 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc14cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc14cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52482/0 (socket says 172.21.15.110:52482) 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 -- 172.21.15.110:0/2157881450 learned_addr learned my addr 172.21.15.110:0/2157881450 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5be0c00640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc15c6c0 0x7f5bdc157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdb400640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc15c2e0 0x7f5bdc159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 -- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc15c2e0 msgr2=0x7f5bdc159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc15c2e0 0x7f5bdc159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 -- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc15c6c0 msgr2=0x7f5bdc157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc15c6c0 0x7f5bdc157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 -- 172.21.15.110:0/2157881450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc159020 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.361+0000 7f5bdbe00640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc14cfe0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c500 tx=0x7f5bcc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:01.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5bd3e00640 1 -- 172.21.15.110:0/2157881450 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc016020 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5bd3e00640 1 -- 172.21.15.110:0/2157881450 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc005220 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5bd3e00640 1 -- 172.21.15.110:0/2157881450 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc007030 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bdc159300 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bdc14d520 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bdc0b4b50 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5bd3e00640 1 -- 172.21.15.110:0/2157881450 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f5bcc0071f0 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5bd3e00640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5bbc068250 0x7f5bbc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:01.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5bd3e00640 1 -- 172.21.15.110:0/2157881450 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5851+0+0 (secure 0 0 0) 0x7f5bcc083ce0 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.365+0000 7f5bdb400640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5bbc068250 0x7f5bbc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:01.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.369+0000 7f5bdb400640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5bbc068250 0x7f5bbc06a710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80ae000 tx=0x7f5bc8004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:01.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.369+0000 7f5bd3e00640 1 -- 172.21.15.110:0/2157881450 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5bcc020020 con 0x7f5bdc0b46c0 2024-09-10T10:06:01.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:01.561+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5bdc073580 con 0x7f5bdc0b46c0 2024-09-10T10:06:02.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.193+0000 7f5bd3e00640 1 -- 172.21.15.110:0/2157881450 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v109) ==== 104+0+0 (secure 0 0 0) 0x7f5bcc055d50 con 0x7f5bdc0b46c0 2024-09-10T10:06:02.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5bbc068250 msgr2=0x7f5bbc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5bbc068250 0x7f5bbc06a710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80ae000 tx=0x7f5bc8004740 comp rx=0 tx=0).stop 2024-09-10T10:06:02.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 msgr2=0x7f5bdc14cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc14cfe0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c500 tx=0x7f5bcc00c9c0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 shutdown_connections 2024-09-10T10:06:02.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5bbc068250 0x7f5bbc06a710 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bdc15c6c0 0x7f5bdc157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5bdc15c2e0 0x7f5bdc159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 --2- 172.21.15.110:0/2157881450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bdc0b46c0 0x7f5bdc14cfe0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 >> 172.21.15.110:0/2157881450 conn(0x7f5bdc0b7be0 msgr2=0x7f5bdc0d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:02.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 shutdown_connections 2024-09-10T10:06:02.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.197+0000 7f5be1600640 1 -- 172.21.15.110:0/2157881450 wait complete. 2024-09-10T10:06:02.221 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:06:02.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.365+0000 7f3622200640 1 Processor -- start 2024-09-10T10:06:02.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.365+0000 7f3622200640 1 -- start start 2024-09-10T10:06:02.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.365+0000 7f3622200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f361c15c560 0x7f361c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:02.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.365+0000 7f3622200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361c06f430 con 0x7f361c06f010 2024-09-10T10:06:02.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.365+0000 7f3622200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361c155d50 con 0x7f361c155970 2024-09-10T10:06:02.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.365+0000 7f3622200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f361c0b4570 con 0x7f361c15c560 2024-09-10T10:06:02.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361be00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f361c06f010 0x7f361c0ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:34626/0 (socket says 172.21.15.110:34626) 2024-09-10T10:06:02.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361be00640 1 -- 172.21.15.110:0/2620571949 learned_addr learned my addr 172.21.15.110:0/2620571949 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:02.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f3612a00640 1 --2- 172.21.15.110:0/2620571949 >> v2:172.21.15.145:3300/0 conn(0x7f361c15c560 0x7f361c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006483738 0 0) 0x7f361c06f430 con 0x7f361c06f010 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3600003530 con 0x7f361c06f010 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3933797507 0 0) 0x7f361c155d50 con 0x7f361c155970 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f361c06f430 con 0x7f361c155970 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 387606642 0 0) 0x7f3600003530 con 0x7f361c06f010 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 >> v2:172.21.15.145:3300/0 conn(0x7f361c15c560 msgr2=0x7f361c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 --2- 172.21.15.110:0/2620571949 >> v2:172.21.15.145:3300/0 conn(0x7f361c15c560 0x7f361c06ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 >> v1:172.21.15.145:6789/0 conn(0x7f361c155970 legacy=0x7f361c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c0b34d0 con 0x7f361c06f010 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f3614001680 con 0x7f361c06f010 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f3612a00640 1 --2- 172.21.15.110:0/2620571949 >> v2:172.21.15.145:3300/0 conn(0x7f361c15c560 0x7f361c06ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:02.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 >> v1:172.21.15.110:6789/0 conn(0x7f361c06f010 legacy=0x7f361c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3600003ba0 0x7f3600003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3600004810 0x7f3600004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36000051d0 0x7f360000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f360000e2b0 con 0x7f3600003ba0 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f360000e430 con 0x7f36000051d0 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f360000e630 con 0x7f3600004810 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361b400640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3600003ba0 0x7f3600003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361be00640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36000051d0 0x7f360000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f3612a00640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3600004810 0x7f3600004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361b400640 1 -- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3600004810 msgr2=0x7f3600004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361b400640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3600004810 0x7f3600004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361b400640 1 -- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36000051d0 msgr2=0x7f360000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:02.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361b400640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36000051d0 0x7f360000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361b400640 1 -- 172.21.15.110:0/2620571949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3600003530 con 0x7f3600003ba0 2024-09-10T10:06:02.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f361be00640 1 -- 172.21.15.110:0/2620571949 reap_dead start 2024-09-10T10:06:02.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.369+0000 7f3612a00640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3600004810 0x7f3600004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f361b400640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3600003ba0 0x7f3600003fa0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f360c001fd0 tx=0x7f360c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c33a4da1449d53e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c001c20 con 0x7f3600003ba0 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c003ba0 con 0x7f3600003ba0 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f361aa00640 1 -- 172.21.15.110:0/2620571949 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c0050d0 con 0x7f3600003ba0 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 -- 172.21.15.110:0/2620571949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3600003ba0 msgr2=0x7f3600003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3600003ba0 0x7f3600003fa0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f360c001fd0 tx=0x7f360c01c8b0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 -- 172.21.15.110:0/2620571949 shutdown_connections 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 --2- 172.21.15.110:0/2620571949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3600003ba0 0x7f3600003fa0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 -- 172.21.15.110:0/2620571949 >> 172.21.15.110:0/2620571949 conn(0x7f361c06bc50 msgr2=0x7f361c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:02.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 -- 172.21.15.110:0/2620571949 shutdown_connections 2024-09-10T10:06:02.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 -- 172.21.15.110:0/2620571949 wait complete. 2024-09-10T10:06:02.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 Processor -- start 2024-09-10T10:06:02.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.373+0000 7f3622200640 1 -- start start 2024-09-10T10:06:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3600003ba0 0x7f361c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f361c152ed0 0x7f361c148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f361c15e1f0 0x7f361c15ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f361c0b4050 con 0x7f361c15e1f0 2024-09-10T10:06:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f361c0b3d70 con 0x7f3600003ba0 2024-09-10T10:06:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f361c0cccd0 con 0x7f361c152ed0 2024-09-10T10:06:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f361c15e1f0 0x7f361c15ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:02.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f361c15e1f0 0x7f361c15ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52502/0 (socket says 172.21.15.110:52502) 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 -- 172.21.15.110:0/1176289835 learned_addr learned my addr 172.21.15.110:0/1176289835 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361b400640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3600003ba0 0x7f361c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 -- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f361c152ed0 msgr2=0x7f361c148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3612a00640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f361c152ed0 0x7f361c148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f361c152ed0 0x7f361c148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 -- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3600003ba0 msgr2=0x7f361c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3600003ba0 0x7f361c15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 -- 172.21.15.110:0/1176289835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c15d150 con 0x7f361c15e1f0 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361b400640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3600003ba0 0x7f361c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3612a00640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f361c152ed0 0x7f361c148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f361be00640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f361c15e1f0 0x7f361c15ef60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f361c0b5890 tx=0x7f361400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:02.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3613e00640 1 -- 172.21.15.110:0/1176289835 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614019070 con 0x7f361c15e1f0 2024-09-10T10:06:02.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3613e00640 1 -- 172.21.15.110:0/1176289835 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3614010510 con 0x7f361c15e1f0 2024-09-10T10:06:02.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c15d430 con 0x7f361c15e1f0 2024-09-10T10:06:02.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3613e00640 1 -- 172.21.15.110:0/1176289835 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614010900 con 0x7f361c15e1f0 2024-09-10T10:06:02.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c15d9f0 con 0x7f361c15e1f0 2024-09-10T10:06:02.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361c0ceb60 con 0x7f361c15e1f0 2024-09-10T10:06:02.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3613e00640 1 -- 172.21.15.110:0/1176289835 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f361401c070 con 0x7f361c15e1f0 2024-09-10T10:06:02.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.377+0000 7f3613e00640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35f8068280 0x7f35f806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:02.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.381+0000 7f3613e00640 1 -- 172.21.15.110:0/1176289835 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5851+0+0 (secure 0 0 0) 0x7f3614083930 con 0x7f361c15e1f0 2024-09-10T10:06:02.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.381+0000 7f361b400640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35f8068280 0x7f35f806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:02.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.381+0000 7f361b400640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35f8068280 0x7f35f806a740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f360c0ad000 tx=0x7f360c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:02.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.385+0000 7f3613e00640 1 -- 172.21.15.110:0/1176289835 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3614056070 con 0x7f361c15e1f0 2024-09-10T10:06:02.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:02.541+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f361c1540b0 con 0x7f361c15e1f0 2024-09-10T10:06:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.313+0000 7f3613e00640 1 -- 172.21.15.110:0/1176289835 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v155) ==== 181+0+0 (secure 0 0 0) 0x7f3614055970 con 0x7f361c15e1f0 2024-09-10T10:06:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35f8068280 msgr2=0x7f35f806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35f8068280 0x7f35f806a740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f360c0ad000 tx=0x7f360c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f361c15e1f0 msgr2=0x7f361c15ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f361c15e1f0 0x7f361c15ef60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f361c0b5890 tx=0x7f361400e890 comp rx=0 tx=0).stop 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 shutdown_connections 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f35f8068280 0x7f35f806a740 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f361c15e1f0 0x7f361c15ef60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f361c152ed0 0x7f361c148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 --2- 172.21.15.110:0/1176289835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3600003ba0 0x7f361c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 >> 172.21.15.110:0/1176289835 conn(0x7f361c06bc50 msgr2=0x7f361c074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 shutdown_connections 2024-09-10T10:06:03.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.317+0000 7f3622200640 1 -- 172.21.15.110:0/1176289835 wait complete. 2024-09-10T10:06:03.340 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:06:03.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c6ca00640 1 Processor -- start 2024-09-10T10:06:03.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c6ca00640 1 -- start start 2024-09-10T10:06:03.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c6ca00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c6815e690 0x7f0c6806e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:03.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c6ca00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0c680aef50 0x7f0c6806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c6ca00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c6806c710 con 0x7f0c6806bc50 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c6ca00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f0c680b4680 con 0x7f0c680aef50 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c6ca00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0c680b4960 con 0x7f0c6815e690 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c66800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0c680aef50 0x7f0c6806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c66800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0c680aef50 0x7f0c6806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52506/0 (socket says 172.21.15.110:52506) 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c66800640 1 -- 172.21.15.110:0/3137252429 learned_addr learned my addr 172.21.15.110:0/3137252429 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c65e00640 1 --2- 172.21.15.110:0/3137252429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c6815e690 0x7f0c6806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c66800640 1 -- 172.21.15.110:0/3137252429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c6815e690 msgr2=0x7f0c6806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c66800640 1 --2- 172.21.15.110:0/3137252429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c6815e690 0x7f0c6806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c66800640 1 -- 172.21.15.110:0/3137252429 >> v1:172.21.15.145:6789/0 conn(0x7f0c6806bc50 legacy=0x7f0c680ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:03.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c66800640 1 -- 172.21.15.110:0/3137252429 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c680b4c40 con 0x7f0c680aef50 2024-09-10T10:06:03.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.481+0000 7f0c65e00640 1 --2- 172.21.15.110:0/3137252429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c6815e690 0x7f0c6806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:03.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c66800640 1 --2- 172.21.15.110:0/3137252429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680aef50 0x7f0c6806c090 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009e50 tx=0x7f0c6001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df28c2ec5d8ad342 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c65400640 1 -- 172.21.15.110:0/3137252429 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c600a58d0 con 0x7f0c680aef50 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c65400640 1 -- 172.21.15.110:0/3137252429 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c600a5a70 con 0x7f0c680aef50 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c65400640 1 -- 172.21.15.110:0/3137252429 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c600a5e40 con 0x7f0c680aef50 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c6ca00640 1 -- 172.21.15.110:0/3137252429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680aef50 msgr2=0x7f0c6806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/3137252429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680aef50 0x7f0c6806c090 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009e50 tx=0x7f0c6001ca00 comp rx=0 tx=0).stop 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c6ca00640 1 -- 172.21.15.110:0/3137252429 shutdown_connections 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/3137252429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c680aef50 0x7f0c6806c090 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/3137252429 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c6815e690 0x7f0c6806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c6ca00640 1 -- 172.21.15.110:0/3137252429 >> 172.21.15.110:0/3137252429 conn(0x7f0c680d1530 msgr2=0x7f0c680d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c6ca00640 1 -- 172.21.15.110:0/3137252429 shutdown_connections 2024-09-10T10:06:03.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.485+0000 7f0c6ca00640 1 -- 172.21.15.110:0/3137252429 wait complete. 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 Processor -- start 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 -- start start 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c6806bc50 0x7f0c68161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c680aef50 0x7f0c68160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c6815e690 0x7f0c68159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6815ebf0 con 0x7f0c6815e690 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6815ea70 con 0x7f0c6806bc50 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0c680b5c50 con 0x7f0c680aef50 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c6815e690 0x7f0c68159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c6815e690 0x7f0c68159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52520/0 (socket says 172.21.15.110:52520) 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 -- 172.21.15.110:0/2860470671 learned_addr learned my addr 172.21.15.110:0/2860470671 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c65e00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c6806bc50 0x7f0c68161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:03.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c5d400640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c680aef50 0x7f0c68160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:03.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 -- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c680aef50 msgr2=0x7f0c68160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:03.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c680aef50 0x7f0c68160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 -- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c6806bc50 msgr2=0x7f0c68161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:03.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c6806bc50 0x7f0c68161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:03.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 -- 172.21.15.110:0/2860470671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c68160660 con 0x7f0c6815e690 2024-09-10T10:06:03.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c66800640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c6815e690 0x7f0c68159d50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0c600af040 tx=0x7f0c600197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:03.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c5ea00640 1 -- 172.21.15.110:0/2860470671 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c600bc020 con 0x7f0c6815e690 2024-09-10T10:06:03.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c5ea00640 1 -- 172.21.15.110:0/2860470671 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c600b8470 con 0x7f0c6815e690 2024-09-10T10:06:03.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c5ea00640 1 -- 172.21.15.110:0/2860470671 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c600a58d0 con 0x7f0c6815e690 2024-09-10T10:06:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c68154230 con 0x7f0c6815e690 2024-09-10T10:06:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.489+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c68160f70 con 0x7f0c6815e690 2024-09-10T10:06:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.493+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6806c090 con 0x7f0c6815e690 2024-09-10T10:06:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.493+0000 7f0c5ea00640 1 -- 172.21.15.110:0/2860470671 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f0c600bf020 con 0x7f0c6815e690 2024-09-10T10:06:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.493+0000 7f0c5ea00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c44068280 0x7f0c4406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.493+0000 7f0c5ea00640 1 -- 172.21.15.110:0/2860470671 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5413+0+0 (secure 0 0 0) 0x7f0c6011f850 con 0x7f0c6815e690 2024-09-10T10:06:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.493+0000 7f0c65e00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c44068280 0x7f0c4406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:03.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.493+0000 7f0c65e00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c44068280 0x7f0c4406a740 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0c6815a8e0 tx=0x7f0c58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:03.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.497+0000 7f0c5ea00640 1 -- 172.21.15.110:0/2860470671 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0c60126070 con 0x7f0c6815e690 2024-09-10T10:06:03.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:03.653+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0c68150130 con 0x7f0c6815e690 2024-09-10T10:06:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.317+0000 7f0c5ea00640 1 -- 172.21.15.110:0/2860470671 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v156) ==== 169+0+0 (secure 0 0 0) 0x7f0c600c6030 con 0x7f0c6815e690 2024-09-10T10:06:04.320 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:06:04.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c44068280 msgr2=0x7f0c4406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c44068280 0x7f0c4406a740 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0c6815a8e0 tx=0x7f0c58008040 comp rx=0 tx=0).stop 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c6815e690 msgr2=0x7f0c68159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c6815e690 0x7f0c68159d50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0c600af040 tx=0x7f0c600197a0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 shutdown_connections 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0c44068280 0x7f0c4406a740 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c6815e690 0x7f0c68159d50 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0c680aef50 0x7f0c68160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 --2- 172.21.15.110:0/2860470671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c6806bc50 0x7f0c68161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 >> 172.21.15.110:0/2860470671 conn(0x7f0c680d1530 msgr2=0x7f0c6806e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 shutdown_connections 2024-09-10T10:06:04.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.321+0000 7f0c6ca00640 1 -- 172.21.15.110:0/2860470671 wait complete. 2024-09-10T10:06:04.344 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:06:04.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.481+0000 7f4061c00640 1 Processor -- start 2024-09-10T10:06:04.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.481+0000 7f4061c00640 1 -- start start 2024-09-10T10:06:04.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.481+0000 7f4061c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f405c15b330 0x7f405c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.481+0000 7f4061c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f405c0ce4d0 0x7f405c06e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.481+0000 7f4061c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c0b1c60 0x7f405c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.481+0000 7f4061c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f405c0b5570 con 0x7f405c15b330 2024-09-10T10:06:04.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.481+0000 7f4061c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f405c0b4960 con 0x7f405c0ce4d0 2024-09-10T10:06:04.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.481+0000 7f4061c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f405c0b4c40 con 0x7f405c0b1c60 2024-09-10T10:06:04.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.485+0000 7f405aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f405c15b330 0x7f405c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.485+0000 7f405aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f405c15b330 0x7f405c06c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52530/0 (socket says 172.21.15.110:52530) 2024-09-10T10:06:04.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.485+0000 7f405aa00640 1 -- 172.21.15.110:0/3161155086 learned_addr learned my addr 172.21.15.110:0/3161155086 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:04.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.485+0000 7f405aa00640 1 -- 172.21.15.110:0/3161155086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c0b1c60 msgr2=0x7f405c06e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:04.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.485+0000 7f405aa00640 1 --2- 172.21.15.110:0/3161155086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c0b1c60 0x7f405c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.485+0000 7f405aa00640 1 -- 172.21.15.110:0/3161155086 >> v2:172.21.15.145:3300/0 conn(0x7f405c0ce4d0 msgr2=0x7f405c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:04.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.485+0000 7f405aa00640 1 --2- 172.21.15.110:0/3161155086 >> v2:172.21.15.145:3300/0 conn(0x7f405c0ce4d0 0x7f405c06e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.485+0000 7f405aa00640 1 -- 172.21.15.110:0/3161155086 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c0b4f20 con 0x7f405c15b330 2024-09-10T10:06:04.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.493+0000 7f405aa00640 1 --2- 172.21.15.110:0/3161155086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c15b330 0x7f405c06c6a0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f405c15eea0 tx=0x7f404c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd74e7a2367dbbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:04.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.493+0000 7f4059600640 1 -- 172.21.15.110:0/3161155086 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c0a5780 con 0x7f405c15b330 2024-09-10T10:06:04.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.493+0000 7f4059600640 1 -- 172.21.15.110:0/3161155086 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c0a5920 con 0x7f405c15b330 2024-09-10T10:06:04.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.493+0000 7f4059600640 1 -- 172.21.15.110:0/3161155086 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c0a5cf0 con 0x7f405c15b330 2024-09-10T10:06:04.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.493+0000 7f4061c00640 1 -- 172.21.15.110:0/3161155086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c15b330 msgr2=0x7f405c06c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.493+0000 7f4061c00640 1 --2- 172.21.15.110:0/3161155086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c15b330 0x7f405c06c6a0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f405c15eea0 tx=0x7f404c01c9f0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.497+0000 7f4061c00640 1 -- 172.21.15.110:0/3161155086 shutdown_connections 2024-09-10T10:06:04.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.497+0000 7f4061c00640 1 --2- 172.21.15.110:0/3161155086 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c0b1c60 0x7f405c06e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.497+0000 7f4061c00640 1 --2- 172.21.15.110:0/3161155086 >> v2:172.21.15.145:3300/0 conn(0x7f405c0ce4d0 0x7f405c06e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.497+0000 7f4061c00640 1 --2- 172.21.15.110:0/3161155086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c15b330 0x7f405c06c6a0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.497+0000 7f4061c00640 1 -- 172.21.15.110:0/3161155086 >> 172.21.15.110:0/3161155086 conn(0x7f405c062e40 msgr2=0x7f405c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:04.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.497+0000 7f4061c00640 1 -- 172.21.15.110:0/3161155086 shutdown_connections 2024-09-10T10:06:04.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.497+0000 7f4061c00640 1 -- 172.21.15.110:0/3161155086 wait complete. 2024-09-10T10:06:04.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f4061c00640 1 Processor -- start 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f4061c00640 1 -- start start 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f4061c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c0b1c60 0x7f405c14af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f4061c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c15b330 0x7f405c14b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f4061c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f405c14bb80 0x7f405c14d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f4061c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f405c15fb20 con 0x7f405c0b1c60 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f4061c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f405c15f9a0 con 0x7f405c14bb80 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f4061c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f405c15fca0 con 0x7f405c15b330 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405a000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c15b330 0x7f405c14b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f405c14bb80 0x7f405c14d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c0b1c60 0x7f405c14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f405c14bb80 0x7f405c14d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:59144/0 (socket says 172.21.15.110:59144) 2024-09-10T10:06:04.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405b400640 1 -- 172.21.15.110:0/1475371653 learned_addr learned my addr 172.21.15.110:0/1475371653 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:04.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405b400640 1 -- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c15b330 msgr2=0x7f405c14b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405b400640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c15b330 0x7f405c14b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405b400640 1 -- 172.21.15.110:0/1475371653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c0b1c60 msgr2=0x7f405c14af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405b400640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c0b1c60 0x7f405c14af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.501+0000 7f405b400640 1 -- 172.21.15.110:0/1475371653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c14c030 con 0x7f405c14bb80 2024-09-10T10:06:04.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f405b400640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f405c14bb80 0x7f405c14d7f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f405400b440 tx=0x7f405400b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:04.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4052a00640 1 -- 172.21.15.110:0/1475371653 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054015020 con 0x7f405c14bb80 2024-09-10T10:06:04.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4052a00640 1 -- 172.21.15.110:0/1475371653 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4054011850 con 0x7f405c14bb80 2024-09-10T10:06:04.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4052a00640 1 -- 172.21.15.110:0/1475371653 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054011c40 con 0x7f405c14bb80 2024-09-10T10:06:04.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405c14c2a0 con 0x7f405c14bb80 2024-09-10T10:06:04.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405c14c860 con 0x7f405c14bb80 2024-09-10T10:06:04.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405c06c6a0 con 0x7f405c14bb80 2024-09-10T10:06:04.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4052a00640 1 -- 172.21.15.110:0/1475371653 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f4054018020 con 0x7f405c14bb80 2024-09-10T10:06:04.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4052a00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f403c068280 0x7f403c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.505+0000 7f4052a00640 1 -- 172.21.15.110:0/1475371653 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4999+0+0 (secure 0 0 0) 0x7f40540847f0 con 0x7f405c14bb80 2024-09-10T10:06:04.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.509+0000 7f405aa00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f403c068280 0x7f403c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.509+0000 7f405aa00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f403c068280 0x7f403c06a740 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f404c0ae000 tx=0x7f404c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:04.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.513+0000 7f4052a00640 1 -- 172.21.15.110:0/1475371653 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f405401f020 con 0x7f405c14bb80 2024-09-10T10:06:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.673+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f405c0cefa0 con 0x7f405c14bb80 2024-09-10T10:06:04.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4052a00640 1 -- 172.21.15.110:0/1475371653 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f4054056850 con 0x7f405c14bb80 2024-09-10T10:06:04.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f403c068280 msgr2=0x7f403c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f403c068280 0x7f403c06a740 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f404c0ae000 tx=0x7f404c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:04.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f405c14bb80 msgr2=0x7f405c14d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f405c14bb80 0x7f405c14d7f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f405400b440 tx=0x7f405400b900 comp rx=0 tx=0).stop 2024-09-10T10:06:04.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 shutdown_connections 2024-09-10T10:06:04.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f403c068280 0x7f403c06a740 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f405c14bb80 0x7f405c14d7f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f405c15b330 0x7f405c14b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 --2- 172.21.15.110:0/1475371653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f405c0b1c60 0x7f405c14af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 >> 172.21.15.110:0/1475371653 conn(0x7f405c062e40 msgr2=0x7f405c14fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:04.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 shutdown_connections 2024-09-10T10:06:04.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.681+0000 7f4061c00640 1 -- 172.21.15.110:0/1475371653 wait complete. 2024-09-10T10:06:04.702 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-10T10:06:04.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe62400640 1 Processor -- start 2024-09-10T10:06:04.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe62400640 1 -- start start 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe62400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fbe5c0cfe30 0x7fbe5c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe62400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbe5c06e7a0 0x7fbe5c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe62400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe5c06d2d0 con 0x7fbe5c0cd510 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe62400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fbe5c15d9a0 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe62400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fbe5c06d5b0 con 0x7fbe5c0cfe30 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbe5c06e7a0 0x7fbe5c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbe5c06e7a0 0x7fbe5c0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52560/0 (socket says 172.21.15.110:52560) 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5be00640 1 -- 172.21.15.110:0/4053341305 learned_addr learned my addr 172.21.15.110:0/4053341305 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5b400640 1 --2- 172.21.15.110:0/4053341305 >> v2:172.21.15.145:3300/0 conn(0x7fbe5c0cfe30 0x7fbe5c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5be00640 1 -- 172.21.15.110:0/4053341305 >> v2:172.21.15.145:3300/0 conn(0x7fbe5c0cfe30 msgr2=0x7fbe5c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5be00640 1 --2- 172.21.15.110:0/4053341305 >> v2:172.21.15.145:3300/0 conn(0x7fbe5c0cfe30 0x7fbe5c071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5be00640 1 -- 172.21.15.110:0/4053341305 >> v1:172.21.15.145:6789/0 conn(0x7fbe5c0cd510 legacy=0x7fbe5c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5be00640 1 -- 172.21.15.110:0/4053341305 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c06d890 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.837+0000 7fbe5b400640 1 --2- 172.21.15.110:0/4053341305 >> v2:172.21.15.145:3300/0 conn(0x7fbe5c0cfe30 0x7fbe5c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:04.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe5be00640 1 --2- 172.21.15.110:0/4053341305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c0cd9a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0ce120 tx=0x7fbe5401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b895f97d1a07dfa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:04.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe5aa00640 1 -- 172.21.15.110:0/4053341305 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540a56f0 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe5aa00640 1 -- 172.21.15.110:0/4053341305 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe540a5890 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe5aa00640 1 -- 172.21.15.110:0/4053341305 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540a5c60 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe62400640 1 -- 172.21.15.110:0/4053341305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 msgr2=0x7fbe5c0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe62400640 1 --2- 172.21.15.110:0/4053341305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c0cd9a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0ce120 tx=0x7fbe5401c920 comp rx=0 tx=0).stop 2024-09-10T10:06:04.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe62400640 1 -- 172.21.15.110:0/4053341305 shutdown_connections 2024-09-10T10:06:04.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe62400640 1 --2- 172.21.15.110:0/4053341305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c0cd9a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe62400640 1 --2- 172.21.15.110:0/4053341305 >> v2:172.21.15.145:3300/0 conn(0x7fbe5c0cfe30 0x7fbe5c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe62400640 1 -- 172.21.15.110:0/4053341305 >> 172.21.15.110:0/4053341305 conn(0x7fbe5c06bc50 msgr2=0x7fbe5c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:04.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe62400640 1 -- 172.21.15.110:0/4053341305 shutdown_connections 2024-09-10T10:06:04.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.841+0000 7fbe62400640 1 -- 172.21.15.110:0/4053341305 wait complete. 2024-09-10T10:06:04.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 Processor -- start 2024-09-10T10:06:04.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 -- start start 2024-09-10T10:06:04.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe5c0cd510 0x7fbe5c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe5c0cfe30 0x7fbe5c1f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c072190 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c15d6c0 con 0x7fbe5c0cfe30 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbe5c169180 con 0x7fbe5c0cd510 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52574/0 (socket says 172.21.15.110:52574) 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 -- 172.21.15.110:0/3487762879 learned_addr learned my addr 172.21.15.110:0/3487762879 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5be00640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe5c0cfe30 0x7fbe5c1f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe53e00640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe5c0cd510 0x7fbe5c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 -- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe5c0cd510 msgr2=0x7fbe5c148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe5c0cd510 0x7fbe5c148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 -- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe5c0cfe30 msgr2=0x7fbe5c1f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe5c0cfe30 0x7fbe5c1f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 -- 172.21.15.110:0/3487762879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c0b3820 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe5b400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c148ab0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00c7f0 tx=0x7fbe4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe53400640 1 -- 172.21.15.110:0/3487762879 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c015070 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe53400640 1 -- 172.21.15.110:0/3487762879 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4c007820 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c0b3b00 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe53400640 1 -- 172.21.15.110:0/3487762879 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c007c10 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.845+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c0b40c0 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.849+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c06c370 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.849+0000 7fbe53400640 1 -- 172.21.15.110:0/3487762879 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fbe4c018070 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.849+0000 7fbe53400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbe38068250 0x7fbe3806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:04.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.849+0000 7fbe53400640 1 -- 172.21.15.110:0/3487762879 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4999+0+0 (secure 0 0 0) 0x7fbe4c084590 con 0x7fbe5c06e7a0 2024-09-10T10:06:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.849+0000 7fbe53e00640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbe38068250 0x7fbe3806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.849+0000 7fbe53e00640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbe38068250 0x7fbe3806a710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c1483d0 tx=0x7fbe44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:04.853+0000 7fbe53400640 1 -- 172.21.15.110:0/3487762879 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbe4c056cd0 con 0x7fbe5c06e7a0 2024-09-10T10:06:05.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.065+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7fbe5c0719c0 con 0x7fbe5c06e7a0 2024-09-10T10:06:05.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe53400640 1 -- 172.21.15.110:0/3487762879 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v424) ==== 145+0+0 (secure 0 0 0) 0x7fbe4c0565d0 con 0x7fbe5c06e7a0 2024-09-10T10:06:05.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbe38068250 msgr2=0x7fbe3806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbe38068250 0x7fbe3806a710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c1483d0 tx=0x7fbe44008040 comp rx=0 tx=0).stop 2024-09-10T10:06:05.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 msgr2=0x7fbe5c148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c148ab0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00c7f0 tx=0x7fbe4c00ccb0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 shutdown_connections 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbe38068250 0x7fbe3806a710 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe5c0cfe30 0x7fbe5c1f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbe5c0cd510 0x7fbe5c148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 --2- 172.21.15.110:0/3487762879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c06e7a0 0x7fbe5c148ab0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 >> 172.21.15.110:0/3487762879 conn(0x7fbe5c06bc50 msgr2=0x7fbe5c15a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 shutdown_connections 2024-09-10T10:06:05.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.321+0000 7fbe62400640 1 -- 172.21.15.110:0/3487762879 wait complete. 2024-09-10T10:06:05.344 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-10T10:06:05.344 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-10T10:06:05.344 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T10:06:05.345 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:06:05.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.477+0000 7feca1a00640 1 Processor -- start 2024-09-10T10:06:05.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.477+0000 7feca1a00640 1 -- start start 2024-09-10T10:06:05.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7feca1a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c155970 0x7fec9c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7feca1a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fec9c06f1e0 0x7fec9c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7feca1a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9c155d50 con 0x7fec9c15c560 2024-09-10T10:06:05.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7feca1a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fec9c0b4570 con 0x7fec9c06f1e0 2024-09-10T10:06:05.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7feca1a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fec9c0b34d0 con 0x7fec9c155970 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c155970 0x7fec9c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fec9c06f1e0 0x7fec9c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9aa00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c155970 0x7fec9c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:51666/0 (socket says 172.21.15.110:51666) 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9a000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fec9c15c560 0x7fec9c06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:36966/0 (socket says 172.21.15.110:36966) 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9aa00640 1 -- 172.21.15.110:0/2110840094 learned_addr learned my addr 172.21.15.110:0/2110840094 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9aa00640 1 -- 172.21.15.110:0/2110840094 >> v2:172.21.15.145:3300/0 conn(0x7fec9c06f1e0 msgr2=0x7fec9c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9aa00640 1 --2- 172.21.15.110:0/2110840094 >> v2:172.21.15.145:3300/0 conn(0x7fec9c06f1e0 0x7fec9c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9aa00640 1 -- 172.21.15.110:0/2110840094 >> v1:172.21.15.145:6789/0 conn(0x7fec9c15c560 legacy=0x7fec9c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9aa00640 1 -- 172.21.15.110:0/2110840094 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c0b37b0 con 0x7fec9c155970 2024-09-10T10:06:05.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.481+0000 7fec9b400640 1 --2- 172.21.15.110:0/2110840094 >> v2:172.21.15.145:3300/0 conn(0x7fec9c06f1e0 0x7fec9c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:05.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7fec9aa00640 1 --2- 172.21.15.110:0/2110840094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c155970 0x7fec9c0cc8c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00bb10 tx=0x7fec8c01eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d800e124e4b03a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:05.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7fec99600640 1 -- 172.21.15.110:0/2110840094 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0b4070 con 0x7fec9c155970 2024-09-10T10:06:05.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7fec99600640 1 -- 172.21.15.110:0/2110840094 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c0a7c40 con 0x7fec9c155970 2024-09-10T10:06:05.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7fec99600640 1 -- 172.21.15.110:0/2110840094 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0af440 con 0x7fec9c155970 2024-09-10T10:06:05.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 -- 172.21.15.110:0/2110840094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c155970 msgr2=0x7fec9c0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 --2- 172.21.15.110:0/2110840094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c155970 0x7fec9c0cc8c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00bb10 tx=0x7fec8c01eb70 comp rx=0 tx=0).stop 2024-09-10T10:06:05.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 -- 172.21.15.110:0/2110840094 shutdown_connections 2024-09-10T10:06:05.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 --2- 172.21.15.110:0/2110840094 >> v2:172.21.15.145:3300/0 conn(0x7fec9c06f1e0 0x7fec9c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 --2- 172.21.15.110:0/2110840094 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c155970 0x7fec9c0cc8c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 -- 172.21.15.110:0/2110840094 >> 172.21.15.110:0/2110840094 conn(0x7fec9c06bc50 msgr2=0x7fec9c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:05.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 -- 172.21.15.110:0/2110840094 shutdown_connections 2024-09-10T10:06:05.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 -- 172.21.15.110:0/2110840094 wait complete. 2024-09-10T10:06:05.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.485+0000 7feca1a00640 1 Processor -- start 2024-09-10T10:06:05.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 -- start start 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06f1e0 0x7fec9c0b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec9c155970 0x7fec9c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c15c560 0x7fec9c1643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c0cf170 con 0x7fec9c06f1e0 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c0cce00 con 0x7fec9c155970 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fec9c0cf2f0 con 0x7fec9c15c560 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06f1e0 0x7fec9c0b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06f1e0 0x7fec9c0b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52586/0 (socket says 172.21.15.110:52586) 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 -- 172.21.15.110:0/37378421 learned_addr learned my addr 172.21.15.110:0/37378421 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:05.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9b400640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c15c560 0x7fec9c1643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9a000640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec9c155970 0x7fec9c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 -- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c15c560 msgr2=0x7fec9c1643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c15c560 0x7fec9c1643e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 -- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec9c155970 msgr2=0x7fec9c151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec9c155970 0x7fec9c151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 -- 172.21.15.110:0/37378421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c15a810 con 0x7fec9c06f1e0 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9a000640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec9c155970 0x7fec9c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec9aa00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06f1e0 0x7fec9c0b1610 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b2c0 tx=0x7fec8c01d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec92a00640 1 -- 172.21.15.110:0/37378421 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0bf070 con 0x7fec9c06f1e0 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec92a00640 1 -- 172.21.15.110:0/37378421 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c0afc10 con 0x7fec9c06f1e0 2024-09-10T10:06:05.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c150130 con 0x7fec9c06f1e0 2024-09-10T10:06:05.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7fec92a00640 1 -- 172.21.15.110:0/37378421 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0b4070 con 0x7fec9c06f1e0 2024-09-10T10:06:05.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c1506b0 con 0x7fec9c06f1e0 2024-09-10T10:06:05.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.489+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9c0ce420 con 0x7fec9c06f1e0 2024-09-10T10:06:05.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.493+0000 7fec92a00640 1 -- 172.21.15.110:0/37378421 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fec8c0c2070 con 0x7fec9c06f1e0 2024-09-10T10:06:05.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.493+0000 7fec92a00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fec7c068250 0x7fec7c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.493+0000 7fec92a00640 1 -- 172.21.15.110:0/37378421 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4999+0+0 (secure 0 0 0) 0x7fec8c11e7b0 con 0x7fec9c06f1e0 2024-09-10T10:06:05.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.493+0000 7fec9a000640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fec7c068250 0x7fec7c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.493+0000 7fec9a000640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fec7c068250 0x7fec7c06a710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fec9c15ff60 tx=0x7fec88007d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:05.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.497+0000 7fec92a00640 1 -- 172.21.15.110:0/37378421 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fec9c15a810 con 0x7fec9c06f1e0 2024-09-10T10:06:05.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec9c151050 con 0x7fec9c06f1e0 2024-09-10T10:06:05.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7fec92a00640 1 -- 172.21.15.110:0/37378421 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+13169 (secure 0 0 0) 0x7fec8c018820 con 0x7fec9c06f1e0 2024-09-10T10:06:05.657 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:05.657 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":156,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:06:04.314704+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:6835/1869854747":"2024-09-11T10:06:01.193207+0000","172.21.15.145:6833/1869854747":"2024-09-11T10:06:01.193207+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:06:05.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fec7c068250 msgr2=0x7fec7c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fec7c068250 0x7fec7c06a710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fec9c15ff60 tx=0x7fec88007d70 comp rx=0 tx=0).stop 2024-09-10T10:06:05.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06f1e0 msgr2=0x7fec9c0b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06f1e0 0x7fec9c0b1610 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b2c0 tx=0x7fec8c01d3b0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 shutdown_connections 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fec7c068250 0x7fec7c06a710 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fec9c15c560 0x7fec9c1643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fec9c155970 0x7fec9c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 --2- 172.21.15.110:0/37378421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06f1e0 0x7fec9c0b1610 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 >> 172.21.15.110:0/37378421 conn(0x7fec9c06bc50 msgr2=0x7fec9c14e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 shutdown_connections 2024-09-10T10:06:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.657+0000 7feca1a00640 1 -- 172.21.15.110:0/37378421 wait complete. 2024-09-10T10:06:05.678 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:58:13.418450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:06:05.678 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:06:05.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed25000640 1 Processor -- start 2024-09-10T10:06:05.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed25000640 1 -- start start 2024-09-10T10:06:05.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed25000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed200d20b0 0x7fed2006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed25000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed20071470 con 0x7fed200b71c0 2024-09-10T10:06:05.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed25000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed20071710 con 0x7fed2015e940 2024-09-10T10:06:05.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed25000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fed2006cd60 con 0x7fed200d20b0 2024-09-10T10:06:05.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1e800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fed200b71c0 0x7fed2006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:34640/0 (socket says 172.21.15.110:34640) 2024-09-10T10:06:05.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1e800640 1 -- 172.21.15.110:0/752367755 learned_addr learned my addr 172.21.15.110:0/752367755 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:05.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed15400640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed200d20b0 0x7fed2006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344531379 0 0) 0x7fed20071470 con 0x7fed200b71c0 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed04003530 con 0x7fed200b71c0 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805280285 0 0) 0x7fed20071710 con 0x7fed2015e940 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed20071470 con 0x7fed2015e940 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1284500198 0 0) 0x7fed04003530 con 0x7fed200b71c0 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed200d20b0 msgr2=0x7fed2006d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed200d20b0 0x7fed2006d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 >> v1:172.21.15.145:6789/0 conn(0x7fed2015e940 legacy=0x7fed200b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed200719f0 con 0x7fed200b71c0 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fed18001a00 con 0x7fed200b71c0 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 >> v1:172.21.15.110:6789/0 conn(0x7fed200b71c0 legacy=0x7fed2006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fed04003b80 0x7fed04003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed04004810 0x7fed04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed2006cd60 con 0x7fed040051d0 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fed0400c100 con 0x7fed04003b80 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fed0400c300 con 0x7fed04004810 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.817+0000 7fed15400640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed200d20b0 0x7fed2006d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1e800640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1de00640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fed04003b80 0x7fed04003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed15400640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed04004810 0x7fed04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1e800640 1 -- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed04004810 msgr2=0x7fed04004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:05.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1e800640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed04004810 0x7fed04004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1e800640 1 -- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fed04003b80 msgr2=0x7fed04003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:05.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1e800640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fed04003b80 0x7fed04003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1e800640 1 -- 172.21.15.110:0/752367755 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed04003530 con 0x7fed040051d0 2024-09-10T10:06:05.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1e800640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed0400b9d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fed1800a7f0 tx=0x7fed1801d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43a479aeb80d075 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:05.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1800a410 con 0x7fed040051d0 2024-09-10T10:06:05.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed180ad650 con 0x7fed040051d0 2024-09-10T10:06:05.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed1d400640 1 -- 172.21.15.110:0/752367755 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed180ada40 con 0x7fed040051d0 2024-09-10T10:06:05.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed25000640 1 -- 172.21.15.110:0/752367755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 msgr2=0x7fed0400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.821+0000 7fed25000640 1 --2- 172.21.15.110:0/752367755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed0400b9d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fed1800a7f0 tx=0x7fed1801d300 comp rx=0 tx=0).stop 2024-09-10T10:06:05.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1e800640 1 -- 172.21.15.110:0/752367755 reap_dead start 2024-09-10T10:06:05.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- 172.21.15.110:0/752367755 shutdown_connections 2024-09-10T10:06:05.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- 172.21.15.110:0/752367755 >> 172.21.15.110:0/752367755 conn(0x7fed200d25a0 msgr2=0x7fed200d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:05.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- 172.21.15.110:0/752367755 shutdown_connections 2024-09-10T10:06:05.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- 172.21.15.110:0/752367755 wait complete. 2024-09-10T10:06:05.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 Processor -- start 2024-09-10T10:06:05.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- start start 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed2015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed20159670 0x7fed201623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fed201628e0 0x7fed20167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed2016a6c0 con 0x7fed040051d0 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fed2016a540 con 0x7fed201628e0 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fed2016a840 con 0x7fed20159670 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed2015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed2015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52616/0 (socket says 172.21.15.110:52616) 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 -- 172.21.15.110:0/2893636103 learned_addr learned my addr 172.21.15.110:0/2893636103 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 -- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed20159670 msgr2=0x7fed201623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed20159670 0x7fed201623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 -- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fed201628e0 msgr2=0x7fed20167c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fed201628e0 0x7fed20167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 -- 172.21.15.110:0/2893636103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2015a9f0 con 0x7fed040051d0 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed1de00640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed2015a4b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fed1000f770 tx=0x7fed1000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed16a00640 1 -- 172.21.15.110:0/2893636103 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed100050d0 con 0x7fed040051d0 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed16a00640 1 -- 172.21.15.110:0/2893636103 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed10018070 con 0x7fed040051d0 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed20157800 con 0x7fed040051d0 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed16a00640 1 -- 172.21.15.110:0/2893636103 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1000d650 con 0x7fed040051d0 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.825+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed20164ae0 con 0x7fed040051d0 2024-09-10T10:06:05.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.829+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed20162d00 con 0x7fed040051d0 2024-09-10T10:06:05.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.829+0000 7fed16a00640 1 -- 172.21.15.110:0/2893636103 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fed1000d7f0 con 0x7fed040051d0 2024-09-10T10:06:05.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.829+0000 7fed16a00640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecfc068250 0x7fecfc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.829+0000 7fed16a00640 1 -- 172.21.15.110:0/2893636103 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4999+0+0 (secure 0 0 0) 0x7fed10084200 con 0x7fed040051d0 2024-09-10T10:06:05.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.829+0000 7fed15400640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecfc068250 0x7fecfc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:05.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.829+0000 7fed15400640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecfc068250 0x7fecfc06a710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed201fae30 tx=0x7fed08009710 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:05.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.833+0000 7fed16a00640 1 -- 172.21.15.110:0/2893636103 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fed10056940 con 0x7fed040051d0 2024-09-10T10:06:05.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed201579d0 con 0x7fed040051d0 2024-09-10T10:06:05.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed16a00640 1 -- 172.21.15.110:0/2893636103 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7fed10056240 con 0x7fed040051d0 2024-09-10T10:06:05.993 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-09-10T10:06:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecfc068250 msgr2=0x7fecfc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecfc068250 0x7fecfc06a710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed201fae30 tx=0x7fed08009710 comp rx=0 tx=0).stop 2024-09-10T10:06:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 msgr2=0x7fed2015a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed2015a4b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fed1000f770 tx=0x7fed1000fc30 comp rx=0 tx=0).stop 2024-09-10T10:06:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 shutdown_connections 2024-09-10T10:06:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fecfc068250 0x7fecfc06a710 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fed201628e0 0x7fed20167c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fed20159670 0x7fed201623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 --2- 172.21.15.110:0/2893636103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed040051d0 0x7fed2015a4b0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:05.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 >> 172.21.15.110:0/2893636103 conn(0x7fed200d25a0 msgr2=0x7fed20073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:05.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 shutdown_connections 2024-09-10T10:06:05.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:05.993+0000 7fed25000640 1 -- 172.21.15.110:0/2893636103 wait complete. 2024-09-10T10:06:06.015 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-10T10:06:06.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8c5600640 1 Processor -- start 2024-09-10T10:06:06.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8c5600640 1 -- start start 2024-09-10T10:06:06.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8c5600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd8c0155970 0x7fd8c00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:06.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8c5600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd8c006f1e0 0x7fd8c00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:06.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8c5600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c0155d50 con 0x7fd8c015c560 2024-09-10T10:06:06.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8c5600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c00b4570 con 0x7fd8c0155970 2024-09-10T10:06:06.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8c5600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c00b34d0 con 0x7fd8c006f1e0 2024-09-10T10:06:06.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8bea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd8c0155970 0x7fd8c00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:06.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8bea00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd8c0155970 0x7fd8c00cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52628/0 (socket says 172.21.15.110:52628) 2024-09-10T10:06:06.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8bea00640 1 -- 172.21.15.110:0/4024532302 learned_addr learned my addr 172.21.15.110:0/4024532302 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:06.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8bf400640 1 --2- 172.21.15.110:0/4024532302 >> v2:172.21.15.145:3300/0 conn(0x7fd8c006f1e0 0x7fd8c00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:06.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8bea00640 1 -- 172.21.15.110:0/4024532302 >> v2:172.21.15.145:3300/0 conn(0x7fd8c006f1e0 msgr2=0x7fd8c00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:06.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8bea00640 1 --2- 172.21.15.110:0/4024532302 >> v2:172.21.15.145:3300/0 conn(0x7fd8c006f1e0 0x7fd8c00ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:06.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.149+0000 7fd8bea00640 1 -- 172.21.15.110:0/4024532302 >> v1:172.21.15.110:6789/0 conn(0x7fd8c015c560 legacy=0x7fd8c006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:06.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8bd600640 1 -- 172.21.15.110:0/4024532302 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2700305647 0 0) 0x7fd8c0155d50 con 0x7fd8c015c560 2024-09-10T10:06:06.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8bea00640 1 -- 172.21.15.110:0/4024532302 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c00b37b0 con 0x7fd8c0155970 2024-09-10T10:06:06.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8bea00640 1 --2- 172.21.15.110:0/4024532302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c0155970 0x7fd8c00cc8c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00735e0 tx=0x7fd8b001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2f813054d984dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:06.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8bd600640 1 -- 172.21.15.110:0/4024532302 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00a5880 con 0x7fd8c0155970 2024-09-10T10:06:06.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8bd600640 1 -- 172.21.15.110:0/4024532302 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b00a5a20 con 0x7fd8c0155970 2024-09-10T10:06:06.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8bd600640 1 -- 172.21.15.110:0/4024532302 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00a5df0 con 0x7fd8c0155970 2024-09-10T10:06:06.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8c5600640 1 -- 172.21.15.110:0/4024532302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c0155970 msgr2=0x7fd8c00cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:06.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8c5600640 1 --2- 172.21.15.110:0/4024532302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c0155970 0x7fd8c00cc8c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00735e0 tx=0x7fd8b001c9c0 comp rx=0 tx=0).stop 2024-09-10T10:06:06.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8c5600640 1 -- 172.21.15.110:0/4024532302 shutdown_connections 2024-09-10T10:06:06.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8c5600640 1 --2- 172.21.15.110:0/4024532302 >> v2:172.21.15.145:3300/0 conn(0x7fd8c006f1e0 0x7fd8c00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:06.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8c5600640 1 --2- 172.21.15.110:0/4024532302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c0155970 0x7fd8c00cc8c0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:06.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8c5600640 1 -- 172.21.15.110:0/4024532302 >> 172.21.15.110:0/4024532302 conn(0x7fd8c006bc50 msgr2=0x7fd8c006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:06.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8c5600640 1 -- 172.21.15.110:0/4024532302 shutdown_connections 2024-09-10T10:06:06.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.153+0000 7fd8c5600640 1 -- 172.21.15.110:0/4024532302 wait complete. 2024-09-10T10:06:06.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 Processor -- start 2024-09-10T10:06:06.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 -- start start 2024-09-10T10:06:06.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c006f1e0 0x7fd8c01f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:06.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8c0155970 0x7fd8c0141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:06.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c015c560 0x7fd8c0157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:06.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00cf170 con 0x7fd8c006f1e0 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00cce00 con 0x7fd8c015c560 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c00cf2f0 con 0x7fd8c0155970 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c006f1e0 0x7fd8c01f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c006f1e0 0x7fd8c01f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52632/0 (socket says 172.21.15.110:52632) 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 -- 172.21.15.110:0/1486458524 learned_addr learned my addr 172.21.15.110:0/1486458524 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bf400640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c015c560 0x7fd8c0157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 -- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8c0155970 msgr2=0x7fd8c0141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8be000640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8c0155970 0x7fd8c0141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8c0155970 0x7fd8c0141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 -- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c015c560 msgr2=0x7fd8c0157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c015c560 0x7fd8c0157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:06.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 -- 172.21.15.110:0/1486458524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c01f74f0 con 0x7fd8c006f1e0 2024-09-10T10:06:06.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bf400640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c015c560 0x7fd8c0157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:06.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8be000640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8c0155970 0x7fd8c0141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:06.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8bea00640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c006f1e0 0x7fd8c01f6fb0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00af040 tx=0x7fd8b0019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:06.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8b6a00640 1 -- 172.21.15.110:0/1486458524 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00bc020 con 0x7fd8c006f1e0 2024-09-10T10:06:06.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8b6a00640 1 -- 172.21.15.110:0/1486458524 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b00b8670 con 0x7fd8c006f1e0 2024-09-10T10:06:06.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8b6a00640 1 -- 172.21.15.110:0/1486458524 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00b8a60 con 0x7fd8c006f1e0 2024-09-10T10:06:06.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c014e9f0 con 0x7fd8c006f1e0 2024-09-10T10:06:06.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.157+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c014efb0 con 0x7fd8c006f1e0 2024-09-10T10:06:06.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.161+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c00ce420 con 0x7fd8c006f1e0 2024-09-10T10:06:06.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.161+0000 7fd8b6a00640 1 -- 172.21.15.110:0/1486458524 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fd8b00bf020 con 0x7fd8c006f1e0 2024-09-10T10:06:06.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.161+0000 7fd8b6a00640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8a0068250 0x7fd8a006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:06.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.161+0000 7fd8b6a00640 1 -- 172.21.15.110:0/1486458524 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4999+0+0 (secure 0 0 0) 0x7fd8b011f5c0 con 0x7fd8c006f1e0 2024-09-10T10:06:06.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.161+0000 7fd8be000640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8a0068250 0x7fd8a006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:06.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.165+0000 7fd8be000640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8a0068250 0x7fd8a006a710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00af3b0 tx=0x7fd8ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:06.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.165+0000 7fd8b6a00640 1 -- 172.21.15.110:0/1486458524 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd8b00c6030 con 0x7fd8c006f1e0 2024-09-10T10:06:06.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:06.381+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fd8c0072f20 con 0x7fd8c006f1e0 2024-09-10T10:06:07.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.329+0000 7fd8b6a00640 1 -- 172.21.15.110:0/1486458524 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v426) ==== 166+0+0 (secure 0 0 0) 0x7fd8b00165e0 con 0x7fd8c006f1e0 2024-09-10T10:06:07.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8a0068250 msgr2=0x7fd8a006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:07.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8a0068250 0x7fd8a006a710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00af3b0 tx=0x7fd8ac009040 comp rx=0 tx=0).stop 2024-09-10T10:06:07.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c006f1e0 msgr2=0x7fd8c01f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c006f1e0 0x7fd8c01f6fb0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00af040 tx=0x7fd8b0019760 comp rx=0 tx=0).stop 2024-09-10T10:06:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 shutdown_connections 2024-09-10T10:06:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8a0068250 0x7fd8a006a710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c015c560 0x7fd8c0157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8c0155970 0x7fd8c0141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 --2- 172.21.15.110:0/1486458524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8c006f1e0 0x7fd8c01f6fb0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 >> 172.21.15.110:0/1486458524 conn(0x7fd8c006bc50 msgr2=0x7fd8c015b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 shutdown_connections 2024-09-10T10:06:07.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.333+0000 7fd8c5600640 1 -- 172.21.15.110:0/1486458524 wait complete. 2024-09-10T10:06:07.356 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:06:07.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f9000640 1 Processor -- start 2024-09-10T10:06:07.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f9000640 1 -- start start 2024-09-10T10:06:07.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f9000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f04f4145f30 0x7f04f40b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:07.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f9000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04f40cfd90 0x7f04f40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:07.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f9000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f40b37c0 con 0x7f04f40b29e0 2024-09-10T10:06:07.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f9000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f04f40b2e50 con 0x7f04f40cfd90 2024-09-10T10:06:07.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f9000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f04f40d0170 con 0x7f04f4145f30 2024-09-10T10:06:07.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f2800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04f40cfd90 0x7f04f40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:07.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f2800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f04f40cfd90 0x7f04f40b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52640/0 (socket says 172.21.15.110:52640) 2024-09-10T10:06:07.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.497+0000 7f04f2800640 1 -- 172.21.15.110:0/1677337676 learned_addr learned my addr 172.21.15.110:0/1677337676 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:07.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f2800640 1 -- 172.21.15.110:0/1677337676 >> v2:172.21.15.145:3300/0 conn(0x7f04f4145f30 msgr2=0x7f04f40b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:07.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f1e00640 1 --2- 172.21.15.110:0/1677337676 >> v2:172.21.15.145:3300/0 conn(0x7f04f4145f30 0x7f04f40b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:07.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f2800640 1 --2- 172.21.15.110:0/1677337676 >> v2:172.21.15.145:3300/0 conn(0x7f04f4145f30 0x7f04f40b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f2800640 1 -- 172.21.15.110:0/1677337676 >> v1:172.21.15.145:6789/0 conn(0x7f04f40b29e0 legacy=0x7f04f40cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:07.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f2800640 1 -- 172.21.15.110:0/1677337676 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f40b3aa0 con 0x7f04f40cfd90 2024-09-10T10:06:07.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f1e00640 1 --2- 172.21.15.110:0/1677337676 >> v2:172.21.15.145:3300/0 conn(0x7f04f4145f30 0x7f04f40b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:07.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f2800640 1 --2- 172.21.15.110:0/1677337676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f40b4580 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f04f40b5c50 tx=0x7f04ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08971bddcf1dd32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:07.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f0a00640 1 -- 172.21.15.110:0/1677337676 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0b0070 con 0x7f04f40cfd90 2024-09-10T10:06:07.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f0a00640 1 -- 172.21.15.110:0/1677337676 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec0ab600 con 0x7f04f40cfd90 2024-09-10T10:06:07.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f0a00640 1 -- 172.21.15.110:0/1677337676 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0ab9d0 con 0x7f04f40cfd90 2024-09-10T10:06:07.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f9000640 1 -- 172.21.15.110:0/1677337676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 msgr2=0x7f04f40b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:07.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.501+0000 7f04f9000640 1 --2- 172.21.15.110:0/1677337676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f40b4580 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f04f40b5c50 tx=0x7f04ec01c4a0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 -- 172.21.15.110:0/1677337676 shutdown_connections 2024-09-10T10:06:07.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 --2- 172.21.15.110:0/1677337676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f40b4580 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 --2- 172.21.15.110:0/1677337676 >> v2:172.21.15.145:3300/0 conn(0x7f04f4145f30 0x7f04f40b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 -- 172.21.15.110:0/1677337676 >> 172.21.15.110:0/1677337676 conn(0x7f04f4062e40 msgr2=0x7f04f406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:07.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 -- 172.21.15.110:0/1677337676 shutdown_connections 2024-09-10T10:06:07.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 -- 172.21.15.110:0/1677337676 wait complete. 2024-09-10T10:06:07.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 Processor -- start 2024-09-10T10:06:07.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 -- start start 2024-09-10T10:06:07.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04f40b29e0 0x7f04f4161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:07.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f413fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:07.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04f4145f30 0x7f04f413ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04f40b5660 con 0x7f04f40cfd90 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f04f40b54e0 con 0x7f04f4145f30 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f9000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f04f40b57e0 con 0x7f04f40b29e0 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f413fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f413fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52642/0 (socket says 172.21.15.110:52642) 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 -- 172.21.15.110:0/3848698106 learned_addr learned my addr 172.21.15.110:0/3848698106 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1e00640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04f40b29e0 0x7f04f4161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f2800640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04f4145f30 0x7f04f413ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 -- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04f40b29e0 msgr2=0x7f04f4161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04f40b29e0 0x7f04f4161ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 -- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04f4145f30 msgr2=0x7f04f413ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04f4145f30 0x7f04f413ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 -- 172.21.15.110:0/3848698106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f4163ef0 con 0x7f04f40cfd90 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.505+0000 7f04f1400640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f413fa40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f04e000b410 tx=0x7f04e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04ea000640 1 -- 172.21.15.110:0/3848698106 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0015030 con 0x7f04f40cfd90 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04ea000640 1 -- 172.21.15.110:0/3848698106 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e0004020 con 0x7f04f40cfd90 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04ea000640 1 -- 172.21.15.110:0/3848698106 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0004450 con 0x7f04f40cfd90 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f415dfb0 con 0x7f04f40cfd90 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f415efb0 con 0x7f04f40cfd90 2024-09-10T10:06:07.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f40b4580 con 0x7f04f40cfd90 2024-09-10T10:06:07.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04ea000640 1 -- 172.21.15.110:0/3848698106 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f04e0018070 con 0x7f04f40cfd90 2024-09-10T10:06:07.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04ea000640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04d4068250 0x7f04d406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:07.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04ea000640 1 -- 172.21.15.110:0/3848698106 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4999+0+0 (secure 0 0 0) 0x7f04e0083e50 con 0x7f04f40cfd90 2024-09-10T10:06:07.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.509+0000 7f04f1e00640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04d4068250 0x7f04d406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:07.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.513+0000 7f04f1e00640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04d4068250 0x7f04d406a710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f04f415eb30 tx=0x7f04e4007770 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:07.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.513+0000 7f04ea000640 1 -- 172.21.15.110:0/3848698106 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f04e0055f40 con 0x7f04f40cfd90 2024-09-10T10:06:07.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.669+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f04f4145610 con 0x7f04f40cfd90 2024-09-10T10:06:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.673+0000 7f04ea000640 1 -- 172.21.15.110:0/3848698106 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f04e005a240 con 0x7f04f40cfd90 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04d4068250 msgr2=0x7f04d406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04d4068250 0x7f04d406a710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f04f415eb30 tx=0x7f04e4007770 comp rx=0 tx=0).stop 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 msgr2=0x7f04f413fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f413fa40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f04e000b410 tx=0x7f04e000b8d0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 shutdown_connections 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f04d4068250 0x7f04d406a710 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f04f4145f30 0x7f04f413ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04f40cfd90 0x7f04f413fa40 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 --2- 172.21.15.110:0/3848698106 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f04f40b29e0 0x7f04f4161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:07.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 >> 172.21.15.110:0/3848698106 conn(0x7f04f4062e40 msgr2=0x7f04f40ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:07.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 shutdown_connections 2024-09-10T10:06:07.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:07.677+0000 7f04f9000640 1 -- 172.21.15.110:0/3848698106 wait complete. 2024-09-10T10:06:07.696 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:06:07.696 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:06:07.791 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:06:07.887 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:06:07.887 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:07.887 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:07.896 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:06:07.896 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:06:07.896 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:07.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:06:07.906 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:07.906 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:07.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:06:07.948 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:06:07.948 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:07.956 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:06:07.957 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:06:07.957 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:06:08.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d81200640 1 Processor -- start 2024-09-10T10:06:08.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d81200640 1 -- start start 2024-09-10T10:06:08.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d81200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1d7c155970 0x7f1d7c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d81200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c06f1e0 0x7f1d7c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d81200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d7c155d50 con 0x7f1d7c15c560 2024-09-10T10:06:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d81200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f1d7c0b4570 con 0x7f1d7c155970 2024-09-10T10:06:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d81200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7c0b34d0 con 0x7f1d7c06f1e0 2024-09-10T10:06:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d79e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1d7c155970 0x7f1d7c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d79e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1d7c155970 0x7f1d7c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52658/0 (socket says 172.21.15.110:52658) 2024-09-10T10:06:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d79e00640 1 -- 172.21.15.110:0/357507373 learned_addr learned my addr 172.21.15.110:0/357507373 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d79400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f1d7c15c560 0x7f1d7c06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:34658/0 (socket says 172.21.15.110:34658) 2024-09-10T10:06:08.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d7a800640 1 --2- 172.21.15.110:0/357507373 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c06f1e0 0x7f1d7c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d79e00640 1 -- 172.21.15.110:0/357507373 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c06f1e0 msgr2=0x7f1d7c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d79e00640 1 --2- 172.21.15.110:0/357507373 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c06f1e0 0x7f1d7c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d79e00640 1 -- 172.21.15.110:0/357507373 >> v1:172.21.15.110:6789/0 conn(0x7f1d7c15c560 legacy=0x7f1d7c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d79e00640 1 -- 172.21.15.110:0/357507373 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c0b37b0 con 0x7f1d7c155970 2024-09-10T10:06:08.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.125+0000 7f1d7a800640 1 --2- 172.21.15.110:0/357507373 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c06f1e0 0x7f1d7c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:08.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d79e00640 1 --2- 172.21.15.110:0/357507373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c155970 0x7f1d7c0cc8c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009e50 tx=0x7f1d6c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f772c8c14475cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:08.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d78a00640 1 -- 172.21.15.110:0/357507373 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0a58d0 con 0x7f1d7c155970 2024-09-10T10:06:08.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d78a00640 1 -- 172.21.15.110:0/357507373 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d6c0a5a70 con 0x7f1d7c155970 2024-09-10T10:06:08.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d78a00640 1 -- 172.21.15.110:0/357507373 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0a5e40 con 0x7f1d7c155970 2024-09-10T10:06:08.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d81200640 1 -- 172.21.15.110:0/357507373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c155970 msgr2=0x7f1d7c0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d81200640 1 --2- 172.21.15.110:0/357507373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c155970 0x7f1d7c0cc8c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009e50 tx=0x7f1d6c01ca00 comp rx=0 tx=0).stop 2024-09-10T10:06:08.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d81200640 1 -- 172.21.15.110:0/357507373 shutdown_connections 2024-09-10T10:06:08.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d81200640 1 --2- 172.21.15.110:0/357507373 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c06f1e0 0x7f1d7c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d81200640 1 --2- 172.21.15.110:0/357507373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c155970 0x7f1d7c0cc8c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d81200640 1 -- 172.21.15.110:0/357507373 >> 172.21.15.110:0/357507373 conn(0x7f1d7c06bc50 msgr2=0x7f1d7c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:08.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d81200640 1 -- 172.21.15.110:0/357507373 shutdown_connections 2024-09-10T10:06:08.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.129+0000 7f1d81200640 1 -- 172.21.15.110:0/357507373 wait complete. 2024-09-10T10:06:08.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 Processor -- start 2024-09-10T10:06:08.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 -- start start 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d7c06f1e0 0x7f1d7c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c155970 0x7f1d7c15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c15c560 0x7f1d7c071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c0cf170 con 0x7f1d7c15c560 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c0cce00 con 0x7f1d7c06f1e0 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7c0cf2f0 con 0x7f1d7c155970 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c15c560 0x7f1d7c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c15c560 0x7f1d7c071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52674/0 (socket says 172.21.15.110:52674) 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 -- 172.21.15.110:0/2162858779 learned_addr learned my addr 172.21.15.110:0/2162858779 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d79400640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c155970 0x7f1d7c15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d79e00640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d7c06f1e0 0x7f1d7c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 -- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c155970 msgr2=0x7f1d7c15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c155970 0x7f1d7c15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 -- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d7c06f1e0 msgr2=0x7f1d7c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d7c06f1e0 0x7f1d7c15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 -- 172.21.15.110:0/2162858779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c15cf90 con 0x7f1d7c15c560 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d7a800640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c15c560 0x7f1d7c071660 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400c820 tx=0x7f1d7400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d72000640 1 -- 172.21.15.110:0/2162858779 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74015070 con 0x7f1d7c15c560 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d72000640 1 -- 172.21.15.110:0/2162858779 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d74007a70 con 0x7f1d7c15c560 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d72000640 1 -- 172.21.15.110:0/2162858779 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74007e60 con 0x7f1d7c15c560 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7c15d210 con 0x7f1d7c15c560 2024-09-10T10:06:08.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.133+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7c15d7d0 con 0x7f1d7c15c560 2024-09-10T10:06:08.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.137+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d7c0ce420 con 0x7f1d7c15c560 2024-09-10T10:06:08.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.137+0000 7f1d72000640 1 -- 172.21.15.110:0/2162858779 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f1d74018070 con 0x7f1d7c15c560 2024-09-10T10:06:08.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.137+0000 7f1d72000640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1d5c068250 0x7f1d5c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.137+0000 7f1d72000640 1 -- 172.21.15.110:0/2162858779 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4999+0+0 (secure 0 0 0) 0x7f1d74084340 con 0x7f1d7c15c560 2024-09-10T10:06:08.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.137+0000 7f1d79e00640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1d5c068250 0x7f1d5c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.137+0000 7f1d79e00640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1d5c068250 0x7f1d5c06a710 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0ae000 tx=0x7f1d6c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:08.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.141+0000 7f1d72000640 1 -- 172.21.15.110:0/2162858779 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1d7c15cf90 con 0x7f1d7c15c560 2024-09-10T10:06:08.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.329+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d7c06c340 con 0x7f1d7c15c560 2024-09-10T10:06:08.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.329+0000 7f1d72000640 1 -- 172.21.15.110:0/2162858779 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+3925 (secure 0 0 0) 0x7f1d74056a80 con 0x7f1d7c15c560 2024-09-10T10:06:08.330 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:08.331 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":109,"btime":"2024-09-10T10:06:02:195632+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/616347927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":616347927},{"type":"v1","addr":"172.21.15.145:6835","nonce":616347927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":7918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/3244721906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3244721906},{"type":"v1","addr":"172.21.15.145:6834","nonce":3244721906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/1342717567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":1342717567},{"type":"v1","addr":"172.21.15.110:6834","nonce":1342717567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1495976904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":1495976904},{"type":"v1","addr":"172.21.15.110:6835","nonce":1495976904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[]} 2024-09-10T10:06:08.331 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 109 2024-09-10T10:06:08.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1d5c068250 msgr2=0x7f1d5c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1d5c068250 0x7f1d5c06a710 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0ae000 tx=0x7f1d6c0a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c15c560 msgr2=0x7f1d7c071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c15c560 0x7f1d7c071660 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400c820 tx=0x7f1d7400cce0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 shutdown_connections 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1d5c068250 0x7f1d5c06a710 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d7c15c560 0x7f1d7c071660 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f1d7c155970 0x7f1d7c15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 --2- 172.21.15.110:0/2162858779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d7c06f1e0 0x7f1d7c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 >> 172.21.15.110:0/2162858779 conn(0x7f1d7c06bc50 msgr2=0x7f1d7c15ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 shutdown_connections 2024-09-10T10:06:08.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.333+0000 7f1d81200640 1 -- 172.21.15.110:0/2162858779 wait complete. 2024-09-10T10:06:08.352 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:06:08.352 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:06:08.352 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:06:08.352 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:06:08.353 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:06:08.353 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:06:08.353 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:06:08.353 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:06:08.353 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:06:08.353 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:06:08.353 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:06:08.353 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:06:08.354 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:06:08.354 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:06:08.354 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:06:08.354 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:06:08.364 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:06:08.364 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:06:08.365 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:06:08.365 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:06:08.367 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:06:08.367 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:06:08.368 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:06:08.368 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:06:08.414 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:06:08.414 DEBUG:teuthology.parallel:result is None 2024-09-10T10:06:08.415 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:06:08.415 DEBUG:teuthology.parallel:result is None 2024-09-10T10:06:08.416 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:06:08.416 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:06:08.417 DEBUG:teuthology.parallel:result is None 2024-09-10T10:06:08.417 DEBUG:teuthology.parallel:result is None 2024-09-10T10:06:08.417 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:06:08.500 INFO:tasks.ceph.mds.a.smithi110.stdout:starting mds.a at 2024-09-10T10:06:08.501 INFO:tasks.ceph.mds.c.smithi110.stdout:starting mds.c at 2024-09-10T10:06:08.504 INFO:tasks.ceph.mds.d.smithi145.stdout:starting mds.d at 2024-09-10T10:06:08.505 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:06:08.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f8800640 1 Processor -- start 2024-09-10T10:06:08.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f8800640 1 -- start start 2024-09-10T10:06:08.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f8800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f19f015c570 0x7f19f00d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f8800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00d1650 0x7f19f0073200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f8800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f19f00b1b80 0x7f19f0073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f8800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f19f00d2d10 con 0x7f19f00b1b80 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f8800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f19f00d2ff0 con 0x7f19f015c570 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f8800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00d0630 con 0x7f19f00d1650 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f6800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f19f00b1b80 0x7f19f0073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f6800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f19f00b1b80 0x7f19f0073740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47006/0 (socket says 172.21.15.110:47006) 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f5400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00d1650 0x7f19f0073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f6800640 1 -- 172.21.15.110:0/1156595745 learned_addr learned my addr 172.21.15.110:0/1156595745 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:08.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f5e00640 1 --2- 172.21.15.110:0/1156595745 >> v2:172.21.15.145:3300/0 conn(0x7f19f015c570 0x7f19f00d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f6800640 1 -- 172.21.15.110:0/1156595745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00d1650 msgr2=0x7f19f0073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:08.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f6800640 1 --2- 172.21.15.110:0/1156595745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00d1650 0x7f19f0073200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f6800640 1 -- 172.21.15.110:0/1156595745 >> v2:172.21.15.145:3300/0 conn(0x7f19f015c570 msgr2=0x7f19f00d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:08.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f6800640 1 --2- 172.21.15.110:0/1156595745 >> v2:172.21.15.145:3300/0 conn(0x7f19f015c570 0x7f19f00d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f6800640 1 -- 172.21.15.110:0/1156595745 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00d0910 con 0x7f19f00b1b80 2024-09-10T10:06:08.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.561+0000 7f19f5400640 1 --2- 172.21.15.110:0/1156595745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00d1650 0x7f19f0073200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:08.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f6800640 1 --2- 172.21.15.110:0/1156595745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00b1b80 0x7f19f0073740 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f19f00b7c50 tx=0x7f19ec01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa66230505746a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:08.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f4a00640 1 -- 172.21.15.110:0/1156595745 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0a5750 con 0x7f19f00b1b80 2024-09-10T10:06:08.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f4a00640 1 -- 172.21.15.110:0/1156595745 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec0a58f0 con 0x7f19f00b1b80 2024-09-10T10:06:08.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f4a00640 1 -- 172.21.15.110:0/1156595745 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0a5cc0 con 0x7f19f00b1b80 2024-09-10T10:06:08.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 -- 172.21.15.110:0/1156595745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00b1b80 msgr2=0x7f19f0073740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 --2- 172.21.15.110:0/1156595745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00b1b80 0x7f19f0073740 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f19f00b7c50 tx=0x7f19ec01c9c0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 -- 172.21.15.110:0/1156595745 shutdown_connections 2024-09-10T10:06:08.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 --2- 172.21.15.110:0/1156595745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00b1b80 0x7f19f0073740 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 --2- 172.21.15.110:0/1156595745 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00d1650 0x7f19f0073200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 --2- 172.21.15.110:0/1156595745 >> v2:172.21.15.145:3300/0 conn(0x7f19f015c570 0x7f19f00d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 -- 172.21.15.110:0/1156595745 >> 172.21.15.110:0/1156595745 conn(0x7f19f006f440 msgr2=0x7f19f006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:08.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 -- 172.21.15.110:0/1156595745 shutdown_connections 2024-09-10T10:06:08.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.565+0000 7f19f8800640 1 -- 172.21.15.110:0/1156595745 wait complete. 2024-09-10T10:06:08.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 Processor -- start 2024-09-10T10:06:08.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 -- start start 2024-09-10T10:06:08.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00b1b80 0x7f19f01f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00d1650 0x7f19f01e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f015c570 0x7f19f015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0160270 con 0x7f19f00d1650 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f19f015cb60 con 0x7f19f015c570 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f19f01603f0 con 0x7f19f00b1b80 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00d1650 0x7f19f01e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00d1650 0x7f19f01e7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47012/0 (socket says 172.21.15.110:47012) 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 -- 172.21.15.110:0/2134762534 learned_addr learned my addr 172.21.15.110:0/2134762534 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f6800640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f015c570 0x7f19f015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5e00640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00b1b80 0x7f19f01f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 -- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00b1b80 msgr2=0x7f19f01f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00b1b80 0x7f19f01f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 -- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f015c570 msgr2=0x7f19f015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f015c570 0x7f19f015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 -- 172.21.15.110:0/2134762534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f0156e70 con 0x7f19f00d1650 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f5400640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00d1650 0x7f19f01e7d40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9d0 tx=0x7f19e000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19ea000640 1 -- 172.21.15.110:0/2134762534 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0017070 con 0x7f19f00d1650 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19ea000640 1 -- 172.21.15.110:0/2134762534 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e000ca60 con 0x7f19f00d1650 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f01e8690 con 0x7f19f00d1650 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19ea000640 1 -- 172.21.15.110:0/2134762534 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000ce50 con 0x7f19f00d1650 2024-09-10T10:06:08.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01f3940 con 0x7f19f00d1650 2024-09-10T10:06:08.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.569+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f01f3e30 con 0x7f19f00d1650 2024-09-10T10:06:08.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.573+0000 7f19ea000640 1 -- 172.21.15.110:0/2134762534 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f19e001a070 con 0x7f19f00d1650 2024-09-10T10:06:08.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.573+0000 7f19ea000640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19d40683f0 0x7f19d406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:08.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.573+0000 7f19ea000640 1 -- 172.21.15.110:0/2134762534 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4999+0+0 (secure 0 0 0) 0x7f19e0083fb0 con 0x7f19f00d1650 2024-09-10T10:06:08.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.573+0000 7f19f5e00640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19d40683f0 0x7f19d406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:08.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.573+0000 7f19f5e00640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19d40683f0 0x7f19d406a8b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f19e400b3f0 tx=0x7f19e400f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:08.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.577+0000 7f19ea000640 1 -- 172.21.15.110:0/2134762534 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f19f0156e70 con 0x7f19f00d1650 2024-09-10T10:06:08.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:08.765+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f19f00d1140 con 0x7f19f00d1650 2024-09-10T10:06:09.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19ea000640 1 -- 172.21.15.110:0/2134762534 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v161) ==== 97+0+0 (secure 0 0 0) 0x7f19f00d1140 con 0x7f19f00d1650 2024-09-10T10:06:09.697 INFO:teuthology.orchestra.run.smithi110.stderr: removed all blocklist entries 2024-09-10T10:06:09.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19d40683f0 msgr2=0x7f19d406a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:09.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19d40683f0 0x7f19d406a8b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f19e400b3f0 tx=0x7f19e400f040 comp rx=0 tx=0).stop 2024-09-10T10:06:09.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00d1650 msgr2=0x7f19f01e7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00d1650 0x7f19f01e7d40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9d0 tx=0x7f19e000ee90 comp rx=0 tx=0).stop 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 shutdown_connections 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f19d40683f0 0x7f19d406a8b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19f015c570 0x7f19f015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19f00d1650 0x7f19f01e7d40 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 --2- 172.21.15.110:0/2134762534 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f19f00b1b80 0x7f19f01f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 >> 172.21.15.110:0/2134762534 conn(0x7f19f006f440 msgr2=0x7f19f00b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 shutdown_connections 2024-09-10T10:06:09.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.697+0000 7f19f8800640 1 -- 172.21.15.110:0/2134762534 wait complete. 2024-09-10T10:06:09.721 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:06:09.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362c200640 1 Processor -- start 2024-09-10T10:06:09.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362c200640 1 -- start start 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362c200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f36240cd510 0x7f36240cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362c200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f362406e090 0x7f362406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362c200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3624072150 con 0x7f36240cfe30 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362c200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f362415d9a0 con 0x7f36240cd510 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362c200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f362406d2d0 con 0x7f362406e090 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f3628e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f36240cd510 0x7f36240cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f3628e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f36240cd510 0x7f36240cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47014/0 (socket says 172.21.15.110:47014) 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f3628e00640 1 -- 172.21.15.110:0/427454400 learned_addr learned my addr 172.21.15.110:0/427454400 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362a200640 1 --2- 172.21.15.110:0/427454400 >> v2:172.21.15.145:3300/0 conn(0x7f362406e090 0x7f362406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:09.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f3628e00640 1 -- 172.21.15.110:0/427454400 >> v2:172.21.15.145:3300/0 conn(0x7f362406e090 msgr2=0x7f362406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:09.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f3628e00640 1 --2- 172.21.15.110:0/427454400 >> v2:172.21.15.145:3300/0 conn(0x7f362406e090 0x7f362406e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f3628e00640 1 -- 172.21.15.110:0/427454400 >> v1:172.21.15.110:6789/0 conn(0x7f36240cfe30 legacy=0x7f3624071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:09.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f3628e00640 1 -- 172.21.15.110:0/427454400 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362406d5b0 con 0x7f36240cd510 2024-09-10T10:06:09.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.857+0000 7f362a200640 1 --2- 172.21.15.110:0/427454400 >> v2:172.21.15.145:3300/0 conn(0x7f362406e090 0x7f362406e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:09.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f3628e00640 1 --2- 172.21.15.110:0/427454400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f36240cd9a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f362406cb00 tx=0x7f361001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf6bba0cd59731a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:09.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f3623e00640 1 -- 172.21.15.110:0/427454400 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100b1070 con 0x7f36240cd510 2024-09-10T10:06:09.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f3623e00640 1 -- 172.21.15.110:0/427454400 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36100ac600 con 0x7f36240cd510 2024-09-10T10:06:09.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f3623e00640 1 -- 172.21.15.110:0/427454400 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100ac9d0 con 0x7f36240cd510 2024-09-10T10:06:09.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f362c200640 1 -- 172.21.15.110:0/427454400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 msgr2=0x7f36240cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:09.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f362c200640 1 --2- 172.21.15.110:0/427454400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f36240cd9a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f362406cb00 tx=0x7f361001c490 comp rx=0 tx=0).stop 2024-09-10T10:06:09.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f362c200640 1 -- 172.21.15.110:0/427454400 shutdown_connections 2024-09-10T10:06:09.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f362c200640 1 --2- 172.21.15.110:0/427454400 >> v2:172.21.15.145:3300/0 conn(0x7f362406e090 0x7f362406e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f362c200640 1 --2- 172.21.15.110:0/427454400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f36240cd9a0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f362c200640 1 -- 172.21.15.110:0/427454400 >> 172.21.15.110:0/427454400 conn(0x7f362406bc50 msgr2=0x7f362406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:09.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f362c200640 1 -- 172.21.15.110:0/427454400 shutdown_connections 2024-09-10T10:06:09.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.861+0000 7f362c200640 1 -- 172.21.15.110:0/427454400 wait complete. 2024-09-10T10:06:09.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 Processor -- start 2024-09-10T10:06:09.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 -- start start 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f362406e090 0x7f3624150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f3624150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36240cfe30 0x7f362414e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3624169180 con 0x7f36240cd510 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f362415d680 con 0x7f36240cfe30 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3624169300 con 0x7f362406e090 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f3624150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f3624150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47028/0 (socket says 172.21.15.110:47028) 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 -- 172.21.15.110:0/2414789467 learned_addr learned my addr 172.21.15.110:0/2414789467 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:09.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362a200640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36240cfe30 0x7f362414e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3629800640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f362406e090 0x7f3624150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 -- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f362406e090 msgr2=0x7f3624150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f362406e090 0x7f3624150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 -- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36240cfe30 msgr2=0x7f362414e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36240cfe30 0x7f362414e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 -- 172.21.15.110:0/2414789467 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36241585f0 con 0x7f36240cd510 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3628e00640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f3624150c60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f36100190a0 tx=0x7f361001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3621600640 1 -- 172.21.15.110:0/2414789467 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100b1040 con 0x7f36240cd510 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3621600640 1 -- 172.21.15.110:0/2414789467 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36100bc070 con 0x7f36240cd510 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f3621600640 1 -- 172.21.15.110:0/2414789467 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100b78d0 con 0x7f36240cd510 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36241588d0 con 0x7f36240cd510 2024-09-10T10:06:09.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.865+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3624158e90 con 0x7f36240cd510 2024-09-10T10:06:09.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.869+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f362415c6c0 con 0x7f36240cd510 2024-09-10T10:06:09.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.869+0000 7f3621600640 1 -- 172.21.15.110:0/2414789467 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f36100b7a70 con 0x7f36240cd510 2024-09-10T10:06:09.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.869+0000 7f3621600640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3604068250 0x7f360406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:09.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.869+0000 7f3621600640 1 -- 172.21.15.110:0/2414789467 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4913+0+0 (secure 0 0 0) 0x7f361011e560 con 0x7f36240cd510 2024-09-10T10:06:09.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.869+0000 7f3629800640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3604068250 0x7f360406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:09.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.869+0000 7f3629800640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3604068250 0x7f360406a710 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3624166ce0 tx=0x7f3618009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:09.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:09.873+0000 7f3621600640 1 -- 172.21.15.110:0/2414789467 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3610016110 con 0x7f36240cd510 2024-09-10T10:06:10.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.085+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f36241512f0 con 0x7f36240cd510 2024-09-10T10:06:10.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f3621600640 1 -- 172.21.15.110:0/2414789467 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7f36100f0770 con 0x7f36240cd510 2024-09-10T10:06:10.089 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:10.089 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:10.089 INFO:teuthology.orchestra.run.smithi110.stdout: "auth_dump": [ 2024-09-10T10:06:10.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.089 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.a", 2024-09-10T10:06:10.089 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBm5Pq5ORAAL6mqSM3AUFVTs161gG0t0A==", 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.b", 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmNY4BChAAQpBjw5L3NYmLaojQ5SJTbw==", 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:06:10.090 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.c", 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmDJ8pBRAACVR5UrRlgyQFr2iIdNnmFw==", 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mds.d", 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmgPk1ERAAKfNpGFI+u2o1axw3SXuX+w==", 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.0", 2024-09-10T10:06:10.092 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQACGOBm2RDiMBAAEOmrSWDBH11mmZm972RNxQ==", 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.1", 2024-09-10T10:06:10.093 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAGGOBmB5puKRAASdH7PIOhUSAvj3x+DmSvEw==", 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.2", 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAKGOBmX8D/HhAAEbO1RzhI4QHkxsQ2brnBHg==", 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:06:10.094 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.3", 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAOGOBmZhwIJhAAkn2P0HTbWA7VfJzRCZGTkg==", 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.4", 2024-09-10T10:06:10.095 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAUGOBmcCCPJhAANVDc0gcN4UF8KNrIYUeXfg==", 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.5", 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAYGOBmkp93KxAAfbhFcB69EDtJu3k6anh8cA==", 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:06:10.096 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.6", 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAcGOBmxFv8ERAA/EnoSXOMQRMXlSNhp1PdXg==", 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.097 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "osd.7", 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAfGOBmNOAoNBAATYDT5083t98iJtoZ0xmHjA==", 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow profile osd", 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile osd", 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.0", 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q==", 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow r", 2024-09-10T10:06:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.1", 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBm8CyIHhAATSZmHZpbttnTn52FgRhU7g==", 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.2", 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmopMaGxAAQg00OU3Fe0zHJijYZKLVTg==", 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.3", 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQABGOBmJyWuJBAAJvqgKEYcPvjI6gJ/7w2vng==", 2024-09-10T10:06:10.100 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow", 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow r", 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow rw", 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow rwx" 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "client.admin", 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmDVMXHhAAk/yFcMmxqjXGmAQ/2uCeIw==", 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": "allow *", 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow *", 2024-09-10T10:06:10.101 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.x", 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmsgG/NRAAKoyf7NkzvujJMhvxzN23tQ==", 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: "entity": "mgr.y", 2024-09-10T10:06:10.102 INFO:teuthology.orchestra.run.smithi110.stdout: "key": "AQAAGOBmFfPbMRAAE2JuDzh9hkgtyTU53FpFUw==", 2024-09-10T10:06:10.103 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": { 2024-09-10T10:06:10.103 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": "allow *", 2024-09-10T10:06:10.103 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": "allow profile mgr", 2024-09-10T10:06:10.103 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": "allow *" 2024-09-10T10:06:10.103 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.103 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.103 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-10T10:06:10.103 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:10.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3604068250 msgr2=0x7f360406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3604068250 0x7f360406a710 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3624166ce0 tx=0x7f3618009040 comp rx=0 tx=0).stop 2024-09-10T10:06:10.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 msgr2=0x7f3624150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f3624150c60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f36100190a0 tx=0x7f361001acd0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 shutdown_connections 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3604068250 0x7f360406a710 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36240cfe30 0x7f362414e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36240cd510 0x7f3624150c60 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 --2- 172.21.15.110:0/2414789467 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f362406e090 0x7f3624150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 >> 172.21.15.110:0/2414789467 conn(0x7f362406bc50 msgr2=0x7f362406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 shutdown_connections 2024-09-10T10:06:10.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.089+0000 7f362c200640 1 -- 172.21.15.110:0/2414789467 wait complete. 2024-09-10T10:06:10.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:06:10.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4028e00640 1 Processor -- start 2024-09-10T10:06:10.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4028e00640 1 -- start start 2024-09-10T10:06:10.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4028e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4024155970 0x7f40240cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4028e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f402415c560 0x7f402406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4028e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 0x7f402415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4028e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f40240b4570 con 0x7f402415c560 2024-09-10T10:06:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4028e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f402406f430 con 0x7f4024155970 2024-09-10T10:06:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4028e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4024155d50 con 0x7f402406f010 2024-09-10T10:06:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f401be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f402415c560 0x7f402406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f401be00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f402415c560 0x7f402406ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47034/0 (socket says 172.21.15.110:47034) 2024-09-10T10:06:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4021e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4024155970 0x7f40240cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f401be00640 1 -- 172.21.15.110:0/4211221031 learned_addr learned my addr 172.21.15.110:0/4211221031 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:10.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4022800640 1 --2- 172.21.15.110:0/4211221031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 0x7f402415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f401be00640 1 -- 172.21.15.110:0/4211221031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 msgr2=0x7f402415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f401be00640 1 --2- 172.21.15.110:0/4211221031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 0x7f402415eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f401be00640 1 -- 172.21.15.110:0/4211221031 >> v2:172.21.15.145:3300/0 conn(0x7f4024155970 msgr2=0x7f40240cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f401be00640 1 --2- 172.21.15.110:0/4211221031 >> v2:172.21.15.145:3300/0 conn(0x7f4024155970 0x7f40240cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f401be00640 1 -- 172.21.15.110:0/4211221031 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40240b34d0 con 0x7f402415c560 2024-09-10T10:06:10.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.245+0000 7f4021e00640 1 --2- 172.21.15.110:0/4211221031 >> v2:172.21.15.145:3300/0 conn(0x7f4024155970 0x7f40240cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:10.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f401be00640 1 --2- 172.21.15.110:0/4211221031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f402406ead0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f402406cc30 tx=0x7f400c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a72981bc8f6a8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4021400640 1 -- 172.21.15.110:0/4211221031 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0b0070 con 0x7f402415c560 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4021400640 1 -- 172.21.15.110:0/4211221031 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c0ab600 con 0x7f402415c560 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4021400640 1 -- 172.21.15.110:0/4211221031 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0ab9d0 con 0x7f402415c560 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 -- 172.21.15.110:0/4211221031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 msgr2=0x7f402406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 --2- 172.21.15.110:0/4211221031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f402406ead0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f402406cc30 tx=0x7f400c01c230 comp rx=0 tx=0).stop 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 -- 172.21.15.110:0/4211221031 shutdown_connections 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 --2- 172.21.15.110:0/4211221031 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 0x7f402415eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 --2- 172.21.15.110:0/4211221031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f402406ead0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 --2- 172.21.15.110:0/4211221031 >> v2:172.21.15.145:3300/0 conn(0x7f4024155970 0x7f40240cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 -- 172.21.15.110:0/4211221031 >> 172.21.15.110:0/4211221031 conn(0x7f402406bc50 msgr2=0x7f402406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:10.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 -- 172.21.15.110:0/4211221031 shutdown_connections 2024-09-10T10:06:10.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.249+0000 7f4028e00640 1 -- 172.21.15.110:0/4211221031 wait complete. 2024-09-10T10:06:10.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 Processor -- start 2024-09-10T10:06:10.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 -- start start 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 0x7f4024157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4024155970 0x7f4024156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f4024141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40240cce00 con 0x7f402415c560 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40240b4050 con 0x7f4024155970 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f40240ce420 con 0x7f402406f010 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f4024141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f4024141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47050/0 (socket says 172.21.15.110:47050) 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 -- 172.21.15.110:0/3546776413 learned_addr learned my addr 172.21.15.110:0/3546776413 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:10.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f401be00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4024155970 0x7f4024156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4021e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 0x7f4024157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 -- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 msgr2=0x7f4024157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 0x7f4024157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 -- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4024155970 msgr2=0x7f4024156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4024155970 0x7f4024156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 -- 172.21.15.110:0/3546776413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40241579f0 con 0x7f402415c560 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f401be00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4024155970 0x7f4024156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4022800640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f4024141430 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f401c00eed0 tx=0x7f401c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f401a000640 1 -- 172.21.15.110:0/3546776413 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c018020 con 0x7f402415c560 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f401a000640 1 -- 172.21.15.110:0/3546776413 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401c014400 con 0x7f402415c560 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f401a000640 1 -- 172.21.15.110:0/3546776413 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c0147f0 con 0x7f402415c560 2024-09-10T10:06:10.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4024157cd0 con 0x7f402415c560 2024-09-10T10:06:10.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40241421c0 con 0x7f402415c560 2024-09-10T10:06:10.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.253+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40240cc8c0 con 0x7f402415c560 2024-09-10T10:06:10.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.257+0000 7f401a000640 1 -- 172.21.15.110:0/3546776413 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f401c01b020 con 0x7f402415c560 2024-09-10T10:06:10.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.257+0000 7f401a000640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4000068250 0x7f400006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.257+0000 7f401a000640 1 -- 172.21.15.110:0/3546776413 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4913+0+0 (secure 0 0 0) 0x7f401c084a90 con 0x7f402415c560 2024-09-10T10:06:10.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.257+0000 7f4021e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4000068250 0x7f400006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.257+0000 7f4021e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4000068250 0x7f400006a710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4014009580 tx=0x7f4014008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:10.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.261+0000 7f401a000640 1 -- 172.21.15.110:0/3546776413 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f401c022070 con 0x7f402415c560 2024-09-10T10:06:10.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.421+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f40240715e0 con 0x7f402415c560 2024-09-10T10:06:10.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.421+0000 7f401a000640 1 -- 172.21.15.110:0/3546776413 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v161) ==== 84+0+65 (secure 0 0 0) 0x7f401c0571d0 con 0x7f402415c560 2024-09-10T10:06:10.423 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:10.423 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:06:10.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:10.423 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-09-10T10:06:10.423 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-09-10T10:06:10.423 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:10.423 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:06:10.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4000068250 msgr2=0x7f400006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4000068250 0x7f400006a710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4014009580 tx=0x7f4014008040 comp rx=0 tx=0).stop 2024-09-10T10:06:10.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 msgr2=0x7f4024141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f4024141430 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f401c00eed0 tx=0x7f401c00b480 comp rx=0 tx=0).stop 2024-09-10T10:06:10.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 shutdown_connections 2024-09-10T10:06:10.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4000068250 0x7f400006a710 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f402415c560 0x7f4024141430 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4024155970 0x7f4024156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 --2- 172.21.15.110:0/3546776413 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f402406f010 0x7f4024157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 >> 172.21.15.110:0/3546776413 conn(0x7f402406bc50 msgr2=0x7f402415ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:10.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 shutdown_connections 2024-09-10T10:06:10.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.425+0000 7f4028e00640 1 -- 172.21.15.110:0/3546776413 wait complete. 2024-09-10T10:06:10.443 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:06:10.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:06:10.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.581+0000 7f53e6c00640 1 Processor -- start 2024-09-10T10:06:10.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.581+0000 7f53e6c00640 1 -- start start 2024-09-10T10:06:10.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.581+0000 7f53e6c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f53e00cfe30 0x7f53e0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.581+0000 7f53e6c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e006d2d0 con 0x7f53e006e7a0 2024-09-10T10:06:10.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.581+0000 7f53e6c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e006d5b0 con 0x7f53e00cd510 2024-09-10T10:06:10.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.581+0000 7f53e6c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f53e015d9a0 con 0x7f53e00cfe30 2024-09-10T10:06:10.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53e4c00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f53e006e7a0 0x7f53e015cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37232/0 (socket says 172.21.15.110:37232) 2024-09-10T10:06:10.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53e4c00640 1 -- 172.21.15.110:0/591426892 learned_addr learned my addr 172.21.15.110:0/591426892 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:10.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dfe00640 1 --2- 172.21.15.110:0/591426892 >> v2:172.21.15.145:3300/0 conn(0x7f53e00cfe30 0x7f53e0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dea00640 1 -- 172.21.15.110:0/591426892 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 526877670 0 0) 0x7f53e006d2d0 con 0x7f53e006e7a0 2024-09-10T10:06:10.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dea00640 1 -- 172.21.15.110:0/591426892 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53c4003530 con 0x7f53e006e7a0 2024-09-10T10:06:10.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dfe00640 1 -- 172.21.15.110:0/591426892 >> v1:172.21.15.145:6789/0 conn(0x7f53e00cd510 legacy=0x7f53e006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dfe00640 1 -- 172.21.15.110:0/591426892 >> v1:172.21.15.110:6789/0 conn(0x7f53e006e7a0 legacy=0x7f53e015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dfe00640 1 -- 172.21.15.110:0/591426892 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e006d890 con 0x7f53e00cfe30 2024-09-10T10:06:10.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dfe00640 1 --2- 172.21.15.110:0/591426892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e00cfe30 0x7f53e0071a40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f53d00097c0 tx=0x7f53d001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3ccdea52f4ff73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:10.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dea00640 1 -- 172.21.15.110:0/591426892 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00b0070 con 0x7f53e00cfe30 2024-09-10T10:06:10.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dea00640 1 -- 172.21.15.110:0/591426892 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d00ab600 con 0x7f53e00cfe30 2024-09-10T10:06:10.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.585+0000 7f53dea00640 1 -- 172.21.15.110:0/591426892 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00ab9d0 con 0x7f53e00cfe30 2024-09-10T10:06:10.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- 172.21.15.110:0/591426892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e00cfe30 msgr2=0x7f53e0071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 --2- 172.21.15.110:0/591426892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e00cfe30 0x7f53e0071a40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f53d00097c0 tx=0x7f53d001c410 comp rx=0 tx=0).stop 2024-09-10T10:06:10.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- 172.21.15.110:0/591426892 shutdown_connections 2024-09-10T10:06:10.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 --2- 172.21.15.110:0/591426892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e00cfe30 0x7f53e0071a40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- 172.21.15.110:0/591426892 >> 172.21.15.110:0/591426892 conn(0x7f53e006bc50 msgr2=0x7f53e006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:10.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- 172.21.15.110:0/591426892 shutdown_connections 2024-09-10T10:06:10.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- 172.21.15.110:0/591426892 wait complete. 2024-09-10T10:06:10.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 Processor -- start 2024-09-10T10:06:10.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- start start 2024-09-10T10:06:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e006e7a0 0x7f53e0157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e00cd510 0x7f53e0147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e00cfe30 0x7f53e01599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53e015d740 con 0x7f53e00cfe30 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f53e0072190 con 0x7f53e006e7a0 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e6c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f53e016abb0 con 0x7f53e00cd510 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53e4c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e00cfe30 0x7f53e01599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e006e7a0 0x7f53e0157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e006e7a0 0x7f53e0157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:58206/0 (socket says 172.21.15.110:58206) 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53dfe00640 1 -- 172.21.15.110:0/1516107589 learned_addr learned my addr 172.21.15.110:0/1516107589 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.589+0000 7f53df400640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e00cd510 0x7f53e0147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e4c00640 1 -- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e00cd510 msgr2=0x7f53e0147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e4c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e00cd510 0x7f53e0147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e4c00640 1 -- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e006e7a0 msgr2=0x7f53e0157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e4c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e006e7a0 0x7f53e0157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e4c00640 1 -- 172.21.15.110:0/1516107589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e006d2a0 con 0x7f53e00cfe30 2024-09-10T10:06:10.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53dfe00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e006e7a0 0x7f53e0157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e4c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e00cfe30 0x7f53e01599d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f53d800caa0 tx=0x7f53d800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53d7e00640 1 -- 172.21.15.110:0/1516107589 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8017070 con 0x7f53e00cfe30 2024-09-10T10:06:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53d7e00640 1 -- 172.21.15.110:0/1516107589 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d800ab40 con 0x7f53e00cfe30 2024-09-10T10:06:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e01576c0 con 0x7f53e00cfe30 2024-09-10T10:06:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53d7e00640 1 -- 172.21.15.110:0/1516107589 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8012400 con 0x7f53e00cfe30 2024-09-10T10:06:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e0157c80 con 0x7f53e00cfe30 2024-09-10T10:06:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e00b3d30 con 0x7f53e00cfe30 2024-09-10T10:06:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53d7e00640 1 -- 172.21.15.110:0/1516107589 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f53d801a070 con 0x7f53e00cfe30 2024-09-10T10:06:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.593+0000 7f53d7e00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53c0068250 0x7f53c006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.597+0000 7f53d7e00640 1 -- 172.21.15.110:0/1516107589 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 4913+0+0 (secure 0 0 0) 0x7f53d8083f60 con 0x7f53e00cfe30 2024-09-10T10:06:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.597+0000 7f53dfe00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53c0068250 0x7f53c006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:10.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.597+0000 7f53dfe00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53c0068250 0x7f53c006a710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f53d001aac0 tx=0x7f53d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:10.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.601+0000 7f53d7e00640 1 -- 172.21.15.110:0/1516107589 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f53d8056050 con 0x7f53e00cfe30 2024-09-10T10:06:10.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:10.761+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f53e00b3a50 con 0x7f53e00cfe30 2024-09-10T10:06:11.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53d7e00640 1 -- 172.21.15.110:0/1516107589 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v162) ==== 138+0+0 (secure 0 0 0) 0x7f53d805a350 con 0x7f53e00cfe30 2024-09-10T10:06:11.701 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-09-10T10:06:11.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53c0068250 msgr2=0x7f53c006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:11.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53c0068250 0x7f53c006a710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f53d001aac0 tx=0x7f53d0008040 comp rx=0 tx=0).stop 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e00cfe30 msgr2=0x7f53e01599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e00cfe30 0x7f53e01599d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f53d800caa0 tx=0x7f53d800cea0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 shutdown_connections 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f53c0068250 0x7f53c006a710 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53e00cfe30 0x7f53e01599d0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f53e00cd510 0x7f53e0147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 --2- 172.21.15.110:0/1516107589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53e006e7a0 0x7f53e0157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 >> 172.21.15.110:0/1516107589 conn(0x7f53e006bc50 msgr2=0x7f53e015c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 shutdown_connections 2024-09-10T10:06:11.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.701+0000 7f53e6c00640 1 -- 172.21.15.110:0/1516107589 wait complete. 2024-09-10T10:06:11.733 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:06:11.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8f2c00640 1 Processor -- start 2024-09-10T10:06:11.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8f2c00640 1 -- start start 2024-09-10T10:06:11.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8f2c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd8ec0cd510 0x7fd8ec0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:11.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8f2c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd8ec06e090 0x7fd8ec06e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:11.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8f2c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ec072150 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8f2c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ec15d9a0 con 0x7fd8ec06e090 2024-09-10T10:06:11.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8f2c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ec06d2d0 con 0x7fd8ec0cd510 2024-09-10T10:06:11.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8ebe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fd8ec0cfe30 0x7fd8ec071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37234/0 (socket says 172.21.15.110:37234) 2024-09-10T10:06:11.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8ebe00640 1 -- 172.21.15.110:0/2557762004 learned_addr learned my addr 172.21.15.110:0/2557762004 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:11.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8f0c00640 1 --2- 172.21.15.110:0/2557762004 >> v2:172.21.15.110:3300/0 conn(0x7fd8ec06e090 0x7fd8ec06e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:11.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.877+0000 7fd8e3400640 1 --2- 172.21.15.110:0/2557762004 >> v2:172.21.15.145:3300/0 conn(0x7fd8ec0cd510 0x7fd8ec0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:11.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f0c00640 1 -- 172.21.15.110:0/2557762004 >> v2:172.21.15.145:3300/0 conn(0x7fd8ec0cd510 msgr2=0x7fd8ec0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:11.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f0c00640 1 --2- 172.21.15.110:0/2557762004 >> v2:172.21.15.145:3300/0 conn(0x7fd8ec0cd510 0x7fd8ec0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f0c00640 1 -- 172.21.15.110:0/2557762004 >> v1:172.21.15.110:6789/0 conn(0x7fd8ec0cfe30 legacy=0x7fd8ec071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:11.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f0c00640 1 -- 172.21.15.110:0/2557762004 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec06d5b0 con 0x7fd8ec06e090 2024-09-10T10:06:11.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8eb400640 1 -- 172.21.15.110:0/2557762004 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615455658 0 0) 0x7fd8ec072150 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f0c00640 1 --2- 172.21.15.110:0/2557762004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec06e090 0x7fd8ec06e4e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0ce0f0 tx=0x7fd8e401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607c66618c5b27e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8eb400640 1 -- 172.21.15.110:0/2557762004 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40b0070 con 0x7fd8ec06e090 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8eb400640 1 -- 172.21.15.110:0/2557762004 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e40ab600 con 0x7fd8ec06e090 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8eb400640 1 -- 172.21.15.110:0/2557762004 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40ab9d0 con 0x7fd8ec06e090 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f2c00640 1 -- 172.21.15.110:0/2557762004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec06e090 msgr2=0x7fd8ec06e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/2557762004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec06e090 0x7fd8ec06e4e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0ce0f0 tx=0x7fd8e401c3f0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f2c00640 1 -- 172.21.15.110:0/2557762004 shutdown_connections 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/2557762004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec06e090 0x7fd8ec06e4e0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/2557762004 >> v2:172.21.15.145:3300/0 conn(0x7fd8ec0cd510 0x7fd8ec0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f2c00640 1 -- 172.21.15.110:0/2557762004 >> 172.21.15.110:0/2557762004 conn(0x7fd8ec06bc50 msgr2=0x7fd8ec06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f2c00640 1 -- 172.21.15.110:0/2557762004 shutdown_connections 2024-09-10T10:06:11.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.881+0000 7fd8f2c00640 1 -- 172.21.15.110:0/2557762004 wait complete. 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 Processor -- start 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 -- start start 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8ec06e090 0x7fd8ec14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ec0cd510 0x7fd8ec0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec0cfe30 0x7fd8ec0b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec169180 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec15d6c0 con 0x7fd8ec0cd510 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec169300 con 0x7fd8ec06e090 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec0cfe30 0x7fd8ec0b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:11.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec0cfe30 0x7fd8ec0b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47086/0 (socket says 172.21.15.110:47086) 2024-09-10T10:06:11.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 -- 172.21.15.110:0/1806885476 learned_addr learned my addr 172.21.15.110:0/1806885476 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:11.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8ebe00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8ec06e090 0x7fd8ec14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:11.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8e3400640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ec0cd510 0x7fd8ec0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:11.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 -- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8ec06e090 msgr2=0x7fd8ec14a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:11.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8ec06e090 0x7fd8ec14a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 -- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ec0cd510 msgr2=0x7fd8ec0b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:11.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ec0cd510 0x7fd8ec0b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:11.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 -- 172.21.15.110:0/1806885476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec15a820 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8e3400640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ec0cd510 0x7fd8ec0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:11.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f0c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec0cfe30 0x7fd8ec0b0040 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd8e401a020 tx=0x7fd8e401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:11.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8e8c00640 1 -- 172.21.15.110:0/1806885476 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40bc070 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8e8c00640 1 -- 172.21.15.110:0/1806885476 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e40b1040 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec0b0ac0 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.885+0000 7fd8e8c00640 1 -- 172.21.15.110:0/1806885476 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40b0050 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.889+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec0b0dd0 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.889+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec06e4e0 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.889+0000 7fd8e8c00640 1 -- 172.21.15.110:0/1806885476 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fd8e40bf070 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.889+0000 7fd8e8c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8c8068250 0x7fd8c806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:11.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.889+0000 7fd8e8c00640 1 -- 172.21.15.110:0/1806885476 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5283+0+0 (secure 0 0 0) 0x7fd8e411de20 con 0x7fd8ec0cfe30 2024-09-10T10:06:11.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.889+0000 7fd8ebe00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8c8068250 0x7fd8c806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:11.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.889+0000 7fd8ebe00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8c8068250 0x7fd8c806a710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0012f0 tx=0x7fd8dc005ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:11.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:11.893+0000 7fd8e8c00640 1 -- 172.21.15.110:0/1806885476 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd8e4016070 con 0x7fd8ec0cfe30 2024-09-10T10:06:12.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.053+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd8ec0731d0 con 0x7fd8ec0cfe30 2024-09-10T10:06:12.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8e8c00640 1 -- 172.21.15.110:0/1806885476 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v163) ==== 170+0+0 (secure 0 0 0) 0x7fd8e40f0560 con 0x7fd8ec0cfe30 2024-09-10T10:06:12.709 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-09-10T10:06:12.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8c8068250 msgr2=0x7fd8c806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8c8068250 0x7fd8c806a710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0012f0 tx=0x7fd8dc005ec0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec0cfe30 msgr2=0x7fd8ec0b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec0cfe30 0x7fd8ec0b0040 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd8e401a020 tx=0x7fd8e401ac30 comp rx=0 tx=0).stop 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 shutdown_connections 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd8c8068250 0x7fd8c806a710 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8ec0cfe30 0x7fd8ec0b0040 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8ec0cd510 0x7fd8ec0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 --2- 172.21.15.110:0/1806885476 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd8ec06e090 0x7fd8ec14a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.709+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 >> 172.21.15.110:0/1806885476 conn(0x7fd8ec06bc50 msgr2=0x7fd8ec071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:12.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.713+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 shutdown_connections 2024-09-10T10:06:12.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.713+0000 7fd8f2c00640 1 -- 172.21.15.110:0/1806885476 wait complete. 2024-09-10T10:06:12.752 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:06:12.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba2a600640 1 Processor -- start 2024-09-10T10:06:12.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba2a600640 1 -- start start 2024-09-10T10:06:12.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba2a600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 0x7fba240cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:12.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba2a600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fba240cfbc0 0x7fba240b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:12.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba2a600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba240b37c0 con 0x7fba24145f30 2024-09-10T10:06:12.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba2a600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fba240b3aa0 con 0x7fba240cfbc0 2024-09-10T10:06:12.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba2a600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fba240b3d80 con 0x7fba240b2bb0 2024-09-10T10:06:12.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba23400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fba24145f30 0x7fba240b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37244/0 (socket says 172.21.15.110:37244) 2024-09-10T10:06:12.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba23400640 1 -- 172.21.15.110:0/2722934731 learned_addr learned my addr 172.21.15.110:0/2722934731 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:12.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba23e00640 1 --2- 172.21.15.110:0/2722934731 >> v2:172.21.15.110:3300/0 conn(0x7fba240cfbc0 0x7fba240b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:12.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba23e00640 1 -- 172.21.15.110:0/2722934731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 msgr2=0x7fba240cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:12.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba22a00640 1 --2- 172.21.15.110:0/2722934731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 0x7fba240cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:12.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba22000640 1 -- 172.21.15.110:0/2722934731 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541933912 0 0) 0x7fba240b37c0 con 0x7fba24145f30 2024-09-10T10:06:12.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba23e00640 1 --2- 172.21.15.110:0/2722934731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 0x7fba240cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba23e00640 1 -- 172.21.15.110:0/2722934731 >> v1:172.21.15.110:6789/0 conn(0x7fba24145f30 legacy=0x7fba240b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:12.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba23e00640 1 -- 172.21.15.110:0/2722934731 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba24146310 con 0x7fba240cfbc0 2024-09-10T10:06:12.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.893+0000 7fba22a00640 1 --2- 172.21.15.110:0/2722934731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 0x7fba240cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:12.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba23e00640 1 --2- 172.21.15.110:0/2722934731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba240b4580 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0097c0 tx=0x7fba1c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5e0780db090451 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:12.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba22000640 1 -- 172.21.15.110:0/2722934731 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0b0070 con 0x7fba240cfbc0 2024-09-10T10:06:12.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba22000640 1 -- 172.21.15.110:0/2722934731 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1c0ab600 con 0x7fba240cfbc0 2024-09-10T10:06:12.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba22000640 1 -- 172.21.15.110:0/2722934731 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0ab9d0 con 0x7fba240cfbc0 2024-09-10T10:06:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba2a600640 1 -- 172.21.15.110:0/2722934731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 msgr2=0x7fba240b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba2a600640 1 --2- 172.21.15.110:0/2722934731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba240b4580 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0097c0 tx=0x7fba1c01c4a0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba2a600640 1 -- 172.21.15.110:0/2722934731 shutdown_connections 2024-09-10T10:06:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba2a600640 1 --2- 172.21.15.110:0/2722934731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba240b4580 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba2a600640 1 --2- 172.21.15.110:0/2722934731 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 0x7fba240cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba2a600640 1 -- 172.21.15.110:0/2722934731 >> 172.21.15.110:0/2722934731 conn(0x7fba24062e40 msgr2=0x7fba2406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:12.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba2a600640 1 -- 172.21.15.110:0/2722934731 shutdown_connections 2024-09-10T10:06:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.897+0000 7fba2a600640 1 -- 172.21.15.110:0/2722934731 wait complete. 2024-09-10T10:06:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 Processor -- start 2024-09-10T10:06:12.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 -- start start 2024-09-10T10:06:12.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 0x7fba2415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:12.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba241432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:12.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24145f30 0x7fba24143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:12.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fba240b56a0 con 0x7fba240cfbc0 2024-09-10T10:06:12.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba240b5520 con 0x7fba24145f30 2024-09-10T10:06:12.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fba240b5820 con 0x7fba240b2bb0 2024-09-10T10:06:12.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba241432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:12.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba241432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47096/0 (socket says 172.21.15.110:47096) 2024-09-10T10:06:12.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 -- 172.21.15.110:0/1001769267 learned_addr learned my addr 172.21.15.110:0/1001769267 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:12.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 -- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 msgr2=0x7fba2415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:12.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 0x7fba2415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 -- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24145f30 msgr2=0x7fba24143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:12.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24145f30 0x7fba24143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:12.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 -- 172.21.15.110:0/1001769267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2413d8f0 con 0x7fba240cfbc0 2024-09-10T10:06:12.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba22a00640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba241432d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fba1000c820 tx=0x7fba1000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:12.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba1b400640 1 -- 172.21.15.110:0/1001769267 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10015070 con 0x7fba240cfbc0 2024-09-10T10:06:12.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba1b400640 1 -- 172.21.15.110:0/1001769267 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba10007a30 con 0x7fba240cfbc0 2024-09-10T10:06:12.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba1b400640 1 -- 172.21.15.110:0/1001769267 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10007e40 con 0x7fba240cfbc0 2024-09-10T10:06:12.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2413dbd0 con 0x7fba240cfbc0 2024-09-10T10:06:12.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2413e190 con 0x7fba240cfbc0 2024-09-10T10:06:12.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.901+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba240b37c0 con 0x7fba240cfbc0 2024-09-10T10:06:12.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.905+0000 7fba1b400640 1 -- 172.21.15.110:0/1001769267 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fba10018070 con 0x7fba240cfbc0 2024-09-10T10:06:12.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.905+0000 7fba1b400640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba04068280 0x7fba0406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:12.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.905+0000 7fba1b400640 1 -- 172.21.15.110:0/1001769267 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 5665+0+0 (secure 0 0 0) 0x7fba10084800 con 0x7fba240cfbc0 2024-09-10T10:06:12.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.905+0000 7fba23400640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba04068280 0x7fba0406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:12.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.905+0000 7fba23400640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba04068280 0x7fba0406a740 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fba14001ca0 tx=0x7fba14007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:12.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:12.909+0000 7fba1b400640 1 -- 172.21.15.110:0/1001769267 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fba10056f40 con 0x7fba240cfbc0 2024-09-10T10:06:13.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.097+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fba24141480 con 0x7fba240cfbc0 2024-09-10T10:06:13.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.709+0000 7fba1b400640 1 -- 172.21.15.110:0/1001769267 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v111) ==== 174+0+0 (secure 0 0 0) 0x7fba10056840 con 0x7fba240cfbc0 2024-09-10T10:06:13.711 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 26 and data pool 27 2024-09-10T10:06:13.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba04068280 msgr2=0x7fba0406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:13.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba04068280 0x7fba0406a740 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fba14001ca0 tx=0x7fba14007cf0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 msgr2=0x7fba241432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:13.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba241432d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fba1000c820 tx=0x7fba1000cce0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 shutdown_connections 2024-09-10T10:06:13.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fba04068280 0x7fba0406a740 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24145f30 0x7fba24143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba240cfbc0 0x7fba241432d0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 --2- 172.21.15.110:0/1001769267 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fba240b2bb0 0x7fba2415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 >> 172.21.15.110:0/1001769267 conn(0x7fba24062e40 msgr2=0x7fba24071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:13.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 shutdown_connections 2024-09-10T10:06:13.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.713+0000 7fba2a600640 1 -- 172.21.15.110:0/1001769267 wait complete. 2024-09-10T10:06:13.758 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:06:13.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.893+0000 7ff37f200640 1 Processor -- start 2024-09-10T10:06:13.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.893+0000 7ff37f200640 1 -- start start 2024-09-10T10:06:13.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37f200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3800b5090 0x7ff3800b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:13.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37f200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5470 0x7ff38015c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:13.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37f200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3800ce1d0 con 0x7ff3800cef70 2024-09-10T10:06:13.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37f200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff3800cfb30 con 0x7ff3800b5090 2024-09-10T10:06:13.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37f200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff3800b4e70 con 0x7ff3800b5470 2024-09-10T10:06:13.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37e800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff3800cef70 0x7ff3800cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37258/0 (socket says 172.21.15.110:37258) 2024-09-10T10:06:13.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37e800640 1 -- 172.21.15.110:0/2894550162 learned_addr learned my addr 172.21.15.110:0/2894550162 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:13.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37d400640 1 --2- 172.21.15.110:0/2894550162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5470 0x7ff38015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:13.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37de00640 1 --2- 172.21.15.110:0/2894550162 >> v2:172.21.15.145:3300/0 conn(0x7ff3800b5090 0x7ff3800b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:13.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37ca00640 1 -- 172.21.15.110:0/2894550162 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410283794 0 0) 0x7ff3800ce1d0 con 0x7ff3800cef70 2024-09-10T10:06:13.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37ca00640 1 -- 172.21.15.110:0/2894550162 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff364003530 con 0x7ff3800cef70 2024-09-10T10:06:13.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37d400640 1 -- 172.21.15.110:0/2894550162 >> v2:172.21.15.145:3300/0 conn(0x7ff3800b5090 msgr2=0x7ff3800b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:13.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37d400640 1 --2- 172.21.15.110:0/2894550162 >> v2:172.21.15.145:3300/0 conn(0x7ff3800b5090 0x7ff3800b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37d400640 1 -- 172.21.15.110:0/2894550162 >> v1:172.21.15.110:6789/0 conn(0x7ff3800cef70 legacy=0x7ff3800cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:13.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37d400640 1 -- 172.21.15.110:0/2894550162 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38015be60 con 0x7ff3800b5470 2024-09-10T10:06:13.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.897+0000 7ff37de00640 1 --2- 172.21.15.110:0/2894550162 >> v2:172.21.15.145:3300/0 conn(0x7ff3800b5090 0x7ff3800b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:13.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37d400640 1 --2- 172.21.15.110:0/2894550162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5470 0x7ff38015c180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0097c0 tx=0x7ff36c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2f19003d68edb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:13.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37ca00640 1 -- 172.21.15.110:0/2894550162 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0b1070 con 0x7ff3800b5470 2024-09-10T10:06:13.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37ca00640 1 -- 172.21.15.110:0/2894550162 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c0ac640 con 0x7ff3800b5470 2024-09-10T10:06:13.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37ca00640 1 -- 172.21.15.110:0/2894550162 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0aca10 con 0x7ff3800b5470 2024-09-10T10:06:13.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 -- 172.21.15.110:0/2894550162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5470 msgr2=0x7ff38015c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:13.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 --2- 172.21.15.110:0/2894550162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5470 0x7ff38015c180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0097c0 tx=0x7ff36c01c4a0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 -- 172.21.15.110:0/2894550162 shutdown_connections 2024-09-10T10:06:13.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 --2- 172.21.15.110:0/2894550162 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5470 0x7ff38015c180 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 --2- 172.21.15.110:0/2894550162 >> v2:172.21.15.145:3300/0 conn(0x7ff3800b5090 0x7ff3800b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 -- 172.21.15.110:0/2894550162 >> 172.21.15.110:0/2894550162 conn(0x7ff38006d0f0 msgr2=0x7ff38006d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:13.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 -- 172.21.15.110:0/2894550162 shutdown_connections 2024-09-10T10:06:13.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 -- 172.21.15.110:0/2894550162 wait complete. 2024-09-10T10:06:13.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 Processor -- start 2024-09-10T10:06:13.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.901+0000 7ff37f200640 1 -- start start 2024-09-10T10:06:13.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5090 0x7ff38013e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:13.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3800b5470 0x7ff380072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:13.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800cef70 0x7ff380073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:13.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801588e0 con 0x7ff3800b5470 2024-09-10T10:06:13.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3800cd890 con 0x7ff3800cef70 2024-09-10T10:06:13.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff380158a60 con 0x7ff3800b5090 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3800b5470 0x7ff380072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3800b5470 0x7ff380072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47100/0 (socket says 172.21.15.110:47100) 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 -- 172.21.15.110:0/1504165746 learned_addr learned my addr 172.21.15.110:0/1504165746 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37e800640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800cef70 0x7ff380073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37de00640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5090 0x7ff38013e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 -- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5090 msgr2=0x7ff38013e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5090 0x7ff38013e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 -- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800cef70 msgr2=0x7ff380073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800cef70 0x7ff380073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 -- 172.21.15.110:0/1504165746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3800aa990 con 0x7ff3800b5470 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37e800640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800cef70 0x7ff380073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:13.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37de00640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5090 0x7ff38013e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:13.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37d400640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3800b5470 0x7ff380072d10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0af040 tx=0x7ff36c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:13.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff376000640 1 -- 172.21.15.110:0/1504165746 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0b1050 con 0x7ff3800b5470 2024-09-10T10:06:13.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff376000640 1 -- 172.21.15.110:0/1504165746 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c0bd070 con 0x7ff3800b5470 2024-09-10T10:06:13.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff380141ac0 con 0x7ff3800b5470 2024-09-10T10:06:13.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff376000640 1 -- 172.21.15.110:0/1504165746 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c006e20 con 0x7ff3800b5470 2024-09-10T10:06:13.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38013f180 con 0x7ff3800b5470 2024-09-10T10:06:13.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.905+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3800b5890 con 0x7ff3800b5470 2024-09-10T10:06:13.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.909+0000 7ff376000640 1 -- 172.21.15.110:0/1504165746 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7ff36c0b6870 con 0x7ff3800b5470 2024-09-10T10:06:13.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.909+0000 7ff376000640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff360068250 0x7ff36006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:13.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.909+0000 7ff376000640 1 -- 172.21.15.110:0/1504165746 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5765+0+0 (secure 0 0 0) 0x7ff36c11ec50 con 0x7ff3800b5470 2024-09-10T10:06:13.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.909+0000 7ff37de00640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff360068250 0x7ff36006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:13.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.909+0000 7ff37de00640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff360068250 0x7ff36006a710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff37000ad50 tx=0x7ff37000a860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:13.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:13.913+0000 7ff376000640 1 -- 172.21.15.110:0/1504165746 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff3800aa990 con 0x7ff3800b5470 2024-09-10T10:06:14.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.069+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff38015a720 con 0x7ff3800b5470 2024-09-10T10:06:14.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff376000640 1 -- 172.21.15.110:0/1504165746 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) ==== 74+0+16881 (secure 0 0 0) 0x7ff36c016090 con 0x7ff3800b5470 2024-09-10T10:06:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:14.073 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":164,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:06:13.100275+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:06:10.807237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T10:06:12.100049+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:06:14.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff360068250 msgr2=0x7ff36006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff360068250 0x7ff36006a710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff37000ad50 tx=0x7ff37000a860 comp rx=0 tx=0).stop 2024-09-10T10:06:14.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3800b5470 msgr2=0x7ff380072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3800b5470 0x7ff380072d10 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0af040 tx=0x7ff36c01ace0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 shutdown_connections 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff360068250 0x7ff36006a710 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3800cef70 0x7ff380073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3800b5470 0x7ff380072d10 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 --2- 172.21.15.110:0/1504165746 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff3800b5090 0x7ff38013e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 >> 172.21.15.110:0/1504165746 conn(0x7ff38006d0f0 msgr2=0x7ff38015d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 shutdown_connections 2024-09-10T10:06:14.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.073+0000 7ff37f200640 1 -- 172.21.15.110:0/1504165746 wait complete. 2024-09-10T10:06:14.095 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:06:14.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.229+0000 7ff9ad600640 1 Processor -- start 2024-09-10T10:06:14.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.229+0000 7ff9ad600640 1 -- start start 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.229+0000 7ff9ad600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a8072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.229+0000 7ff9ad600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff9a8140c30 0x7ff9a815b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.229+0000 7ff9ad600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a80ccf40 con 0x7ff9a8072670 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.229+0000 7ff9ad600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff9a8072ad0 con 0x7ff9a8140c30 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.229+0000 7ff9ad600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a81412a0 con 0x7ff9a80ccb60 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a6000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff9a8072670 0x7ff9a8140520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37270/0 (socket says 172.21.15.110:37270) 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a6000640 1 -- 172.21.15.110:0/2121157404 learned_addr learned my addr 172.21.15.110:0/2121157404 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a6a00640 1 --2- 172.21.15.110:0/2121157404 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a8072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a7400640 1 --2- 172.21.15.110:0/2121157404 >> v2:172.21.15.145:3300/0 conn(0x7ff9a8140c30 0x7ff9a815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a5600640 1 -- 172.21.15.110:0/2121157404 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1582105919 0 0) 0x7ff9a80ccf40 con 0x7ff9a8072670 2024-09-10T10:06:14.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a5600640 1 -- 172.21.15.110:0/2121157404 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff98c003530 con 0x7ff9a8072670 2024-09-10T10:06:14.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a6a00640 1 -- 172.21.15.110:0/2121157404 >> v2:172.21.15.145:3300/0 conn(0x7ff9a8140c30 msgr2=0x7ff9a815b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:14.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a6a00640 1 --2- 172.21.15.110:0/2121157404 >> v2:172.21.15.145:3300/0 conn(0x7ff9a8140c30 0x7ff9a815b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a6a00640 1 -- 172.21.15.110:0/2121157404 >> v1:172.21.15.110:6789/0 conn(0x7ff9a8072670 legacy=0x7ff9a8140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a6a00640 1 -- 172.21.15.110:0/2121157404 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a815e760 con 0x7ff9a80ccb60 2024-09-10T10:06:14.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a7400640 1 --2- 172.21.15.110:0/2121157404 >> v2:172.21.15.145:3300/0 conn(0x7ff9a8140c30 0x7ff9a815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:14.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.233+0000 7ff9a6a00640 1 --2- 172.21.15.110:0/2121157404 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a8072130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff998009d90 tx=0x7ff99801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5859eb338d8b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:14.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a5600640 1 -- 172.21.15.110:0/2121157404 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980b2070 con 0x7ff9a80ccb60 2024-09-10T10:06:14.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a5600640 1 -- 172.21.15.110:0/2121157404 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9980a5920 con 0x7ff9a80ccb60 2024-09-10T10:06:14.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a5600640 1 -- 172.21.15.110:0/2121157404 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980a5cf0 con 0x7ff9a80ccb60 2024-09-10T10:06:14.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- 172.21.15.110:0/2121157404 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 msgr2=0x7ff9a8072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 --2- 172.21.15.110:0/2121157404 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a8072130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff998009d90 tx=0x7ff99801c970 comp rx=0 tx=0).stop 2024-09-10T10:06:14.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- 172.21.15.110:0/2121157404 shutdown_connections 2024-09-10T10:06:14.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 --2- 172.21.15.110:0/2121157404 >> v2:172.21.15.145:3300/0 conn(0x7ff9a8140c30 0x7ff9a815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 --2- 172.21.15.110:0/2121157404 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a8072130 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- 172.21.15.110:0/2121157404 >> 172.21.15.110:0/2121157404 conn(0x7ff9a806bc50 msgr2=0x7ff9a806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:14.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- 172.21.15.110:0/2121157404 shutdown_connections 2024-09-10T10:06:14.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- 172.21.15.110:0/2121157404 wait complete. 2024-09-10T10:06:14.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 Processor -- start 2024-09-10T10:06:14.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- start start 2024-09-10T10:06:14.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a8072670 0x7ff9a815d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a8140c30 0x7ff9a8164930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a815f520 con 0x7ff9a8072670 2024-09-10T10:06:14.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a815b660 con 0x7ff9a8140c30 2024-09-10T10:06:14.240 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9ad600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a815d860 con 0x7ff9a80ccb60 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a8072670 0x7ff9a815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a8072670 0x7ff9a815d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47110/0 (socket says 172.21.15.110:47110) 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6a00640 1 -- 172.21.15.110:0/878287089 learned_addr learned my addr 172.21.15.110:0/878287089 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6000640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a7400640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a8140c30 0x7ff9a8164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6a00640 1 -- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 msgr2=0x7ff9a80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6a00640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a80cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6a00640 1 -- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a8140c30 msgr2=0x7ff9a8164930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6a00640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a8140c30 0x7ff9a8164930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a6a00640 1 -- 172.21.15.110:0/878287089 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a815db40 con 0x7ff9a8072670 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.237+0000 7ff9a7400640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a8140c30 0x7ff9a8164930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:14.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff9a6a00640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a8072670 0x7ff9a815d130 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7ff99801a5a0 tx=0x7ff998019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:14.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff99ea00640 1 -- 172.21.15.110:0/878287089 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980bd020 con 0x7ff9a8072670 2024-09-10T10:06:14.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff99ea00640 1 -- 172.21.15.110:0/878287089 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9980b8670 con 0x7ff9a8072670 2024-09-10T10:06:14.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a815de20 con 0x7ff9a8072670 2024-09-10T10:06:14.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff99ea00640 1 -- 172.21.15.110:0/878287089 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980b2040 con 0x7ff9a8072670 2024-09-10T10:06:14.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a8164e70 con 0x7ff9a8072670 2024-09-10T10:06:14.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a8072130 con 0x7ff9a8072670 2024-09-10T10:06:14.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff99ea00640 1 -- 172.21.15.110:0/878287089 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7ff9980c0020 con 0x7ff9a8072670 2024-09-10T10:06:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff99ea00640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff988068250 0x7ff98806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff99ea00640 1 -- 172.21.15.110:0/878287089 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5765+0+0 (secure 0 0 0) 0x7ff99811f2a0 con 0x7ff9a8072670 2024-09-10T10:06:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.241+0000 7ff9a6000640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff988068250 0x7ff98806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.245+0000 7ff9a6000640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff988068250 0x7ff98806a710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80cfde0 tx=0x7ff994009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:14.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.245+0000 7ff99ea00640 1 -- 172.21.15.110:0/878287089 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff9980c7030 con 0x7ff9a8072670 2024-09-10T10:06:14.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9a80b3850 con 0x7ff9a8072670 2024-09-10T10:06:14.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff99ea00640 1 -- 172.21.15.110:0/878287089 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) ==== 74+0+16881 (secure 0 0 0) 0x7ff998016560 con 0x7ff9a8072670 2024-09-10T10:06:14.418 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:14.418 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":164,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:06:13.100275+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:06:10.807237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T10:06:12.100049+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff988068250 msgr2=0x7ff98806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff988068250 0x7ff98806a710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80cfde0 tx=0x7ff994009040 comp rx=0 tx=0).stop 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a8072670 msgr2=0x7ff9a815d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a8072670 0x7ff9a815d130 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7ff99801a5a0 tx=0x7ff998019710 comp rx=0 tx=0).stop 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 shutdown_connections 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff988068250 0x7ff98806a710 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff9a8140c30 0x7ff9a8164930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff9a80ccb60 0x7ff9a80cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 --2- 172.21.15.110:0/878287089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9a8072670 0x7ff9a815d130 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 >> 172.21.15.110:0/878287089 conn(0x7ff9a806bc50 msgr2=0x7ff9a806eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 shutdown_connections 2024-09-10T10:06:14.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.417+0000 7ff9ad600640 1 -- 172.21.15.110:0/878287089 wait complete. 2024-09-10T10:06:14.440 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:06:14.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.577+0000 7fb7f6a00640 1 Processor -- start 2024-09-10T10:06:14.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.577+0000 7fb7f6a00640 1 -- start start 2024-09-10T10:06:14.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.577+0000 7fb7f6a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f015b330 0x7fb7f006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.577+0000 7fb7f6a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb7f006e850 0x7fb7f00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.577+0000 7fb7f6a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f00b4960 con 0x7fb7f00ce4d0 2024-09-10T10:06:14.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.577+0000 7fb7f6a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f00b5570 con 0x7fb7f006e850 2024-09-10T10:06:14.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.577+0000 7fb7f6a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f00b4c40 con 0x7fb7f015b330 2024-09-10T10:06:14.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7efe00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f015b330 0x7fb7f006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7ef400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fb7f00ce4d0 0x7fb7f006e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:41444/0 (socket says 172.21.15.110:41444) 2024-09-10T10:06:14.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7f4a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb7f006e850 0x7fb7f00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7ef400640 1 -- 172.21.15.110:0/1624235451 learned_addr learned my addr 172.21.15.110:0/1624235451 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:14.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7efe00640 1 -- 172.21.15.110:0/1624235451 >> v2:172.21.15.145:3300/0 conn(0x7fb7f006e850 msgr2=0x7fb7f00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:14.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7efe00640 1 --2- 172.21.15.110:0/1624235451 >> v2:172.21.15.145:3300/0 conn(0x7fb7f006e850 0x7fb7f00b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7efe00640 1 -- 172.21.15.110:0/1624235451 >> v1:172.21.15.145:6789/0 conn(0x7fb7f00ce4d0 legacy=0x7fb7f006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7efe00640 1 -- 172.21.15.110:0/1624235451 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f00b4f20 con 0x7fb7f015b330 2024-09-10T10:06:14.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7efe00640 1 --2- 172.21.15.110:0/1624235451 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f015b330 0x7fb7f006c6a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009d90 tx=0x7fb7e001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=533358bf534301cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7eea00640 1 -- 172.21.15.110:0/1624235451 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00b2070 con 0x7fb7f015b330 2024-09-10T10:06:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7eea00640 1 -- 172.21.15.110:0/1624235451 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e00a5940 con 0x7fb7f015b330 2024-09-10T10:06:14.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.581+0000 7fb7eea00640 1 -- 172.21.15.110:0/1624235451 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00a5d10 con 0x7fb7f015b330 2024-09-10T10:06:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- 172.21.15.110:0/1624235451 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f015b330 msgr2=0x7fb7f006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/1624235451 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f015b330 0x7fb7f006c6a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009d90 tx=0x7fb7e001c970 comp rx=0 tx=0).stop 2024-09-10T10:06:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- 172.21.15.110:0/1624235451 shutdown_connections 2024-09-10T10:06:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/1624235451 >> v2:172.21.15.145:3300/0 conn(0x7fb7f006e850 0x7fb7f00b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/1624235451 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f015b330 0x7fb7f006c6a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- 172.21.15.110:0/1624235451 >> 172.21.15.110:0/1624235451 conn(0x7fb7f0062e40 msgr2=0x7fb7f006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- 172.21.15.110:0/1624235451 shutdown_connections 2024-09-10T10:06:14.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- 172.21.15.110:0/1624235451 wait complete. 2024-09-10T10:06:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 Processor -- start 2024-09-10T10:06:14.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- start start 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f006e850 0x7fb7f013eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7f015b330 0x7fb7f01521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7f0152710 0x7fb7f015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f00b36b0 con 0x7fb7f015b330 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f00b17c0 con 0x7fb7f0152710 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f6a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f00b3830 con 0x7fb7f006e850 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7ef400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7f015b330 0x7fb7f01521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7ef400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7f015b330 0x7fb7f01521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47124/0 (socket says 172.21.15.110:47124) 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7ef400640 1 -- 172.21.15.110:0/3736231077 learned_addr learned my addr 172.21.15.110:0/3736231077 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7efe00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f006e850 0x7fb7f013eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.585+0000 7fb7f4a00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7f0152710 0x7fb7f015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7ef400640 1 -- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f006e850 msgr2=0x7fb7f013eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7ef400640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f006e850 0x7fb7f013eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7ef400640 1 -- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7f0152710 msgr2=0x7fb7f015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7ef400640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7f0152710 0x7fb7f015d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7ef400640 1 -- 172.21.15.110:0/3736231077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f013f400 con 0x7fb7f015b330 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7efe00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f006e850 0x7fb7f013eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7ef400640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7f015b330 0x7fb7f01521d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c400 tx=0x7fb7dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7e7e00640 1 -- 172.21.15.110:0/3736231077 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc017070 con 0x7fb7f015b330 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7e7e00640 1 -- 172.21.15.110:0/3736231077 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc005cf0 con 0x7fb7f015b330 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f015e460 con 0x7fb7f015b330 2024-09-10T10:06:14.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7e7e00640 1 -- 172.21.15.110:0/3736231077 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc007310 con 0x7fb7f015b330 2024-09-10T10:06:14.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f014fd30 con 0x7fb7f015b330 2024-09-10T10:06:14.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f0152b20 con 0x7fb7f015b330 2024-09-10T10:06:14.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7e7e00640 1 -- 172.21.15.110:0/3736231077 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fb7dc01a070 con 0x7fb7f015b330 2024-09-10T10:06:14.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7e7e00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7d0068280 0x7fb7d006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:14.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7e7e00640 1 -- 172.21.15.110:0/3736231077 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5765+0+0 (secure 0 0 0) 0x7fb7dc0836e0 con 0x7fb7f015b330 2024-09-10T10:06:14.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.589+0000 7fb7efe00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7d0068280 0x7fb7d006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:14.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.593+0000 7fb7efe00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7d0068280 0x7fb7d006a740 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00af000 tx=0x7fb7e00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:14.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.593+0000 7fb7e7e00640 1 -- 172.21.15.110:0/3736231077 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb7f0152b20 con 0x7fb7f015b330 2024-09-10T10:06:14.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:14.789+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb7f00b0840 con 0x7fb7f015b330 2024-09-10T10:06:15.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.741+0000 7fb7e7e00640 1 -- 172.21.15.110:0/3736231077 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v114) ==== 118+0+0 (secure 0 0 0) 0x7fb7dc0557d0 con 0x7fb7f015b330 2024-09-10T10:06:15.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7d0068280 msgr2=0x7fb7d006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7d0068280 0x7fb7d006a740 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00af000 tx=0x7fb7e00a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:15.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7f015b330 msgr2=0x7fb7f01521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7f015b330 0x7fb7f01521d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c400 tx=0x7fb7dc00c8c0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 shutdown_connections 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb7d0068280 0x7fb7d006a740 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7f0152710 0x7fb7f015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7f015b330 0x7fb7f01521d0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 --2- 172.21.15.110:0/3736231077 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb7f006e850 0x7fb7f013eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 >> 172.21.15.110:0/3736231077 conn(0x7fb7f0062e40 msgr2=0x7fb7f00aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 shutdown_connections 2024-09-10T10:06:15.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.745+0000 7fb7f6a00640 1 -- 172.21.15.110:0/3736231077 wait complete. 2024-09-10T10:06:15.767 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:06:15.768 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:06:15.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0c1800640 1 Processor -- start 2024-09-10T10:06:15.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0c1800640 1 -- start start 2024-09-10T10:06:15.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0c1800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc0cf0a0 0x7fc0bc0cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0c1800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc0bc13f9c0 0x7fc0bc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0c1800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0bc0cfa00 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0c1800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc0bc13f740 con 0x7fc0bc13f9c0 2024-09-10T10:06:15.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0c1800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc0bc0c6010 con 0x7fc0bc0cf0a0 2024-09-10T10:06:15.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0bbe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc0bc0b2df0 0x7fc0bc0c5900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37276/0 (socket says 172.21.15.110:37276) 2024-09-10T10:06:15.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.909+0000 7fc0bbe00640 1 -- 172.21.15.110:0/1610475735 learned_addr learned my addr 172.21.15.110:0/1610475735 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:15.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0bb400640 1 --2- 172.21.15.110:0/1610475735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc0cf0a0 0x7fc0bc0cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0c0e00640 1 --2- 172.21.15.110:0/1610475735 >> v2:172.21.15.145:3300/0 conn(0x7fc0bc13f9c0 0x7fc0bc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0baa00640 1 -- 172.21.15.110:0/1610475735 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1254862543 0 0) 0x7fc0bc0cfa00 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0baa00640 1 -- 172.21.15.110:0/1610475735 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0a0003530 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0bb400640 1 -- 172.21.15.110:0/1610475735 >> v2:172.21.15.145:3300/0 conn(0x7fc0bc13f9c0 msgr2=0x7fc0bc13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:15.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0bb400640 1 --2- 172.21.15.110:0/1610475735 >> v2:172.21.15.145:3300/0 conn(0x7fc0bc13f9c0 0x7fc0bc13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0bb400640 1 -- 172.21.15.110:0/1610475735 >> v1:172.21.15.110:6789/0 conn(0x7fc0bc0b2df0 legacy=0x7fc0bc0c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0bb400640 1 -- 172.21.15.110:0/1610475735 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc15c9a0 con 0x7fc0bc0cf0a0 2024-09-10T10:06:15.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0bb400640 1 --2- 172.21.15.110:0/1610475735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc0cf0a0 0x7fc0bc0cf4c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80097c0 tx=0x7fc0a801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b4230bf403aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:15.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0baa00640 1 -- 172.21.15.110:0/1610475735 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80b2070 con 0x7fc0bc0cf0a0 2024-09-10T10:06:15.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0baa00640 1 -- 172.21.15.110:0/1610475735 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a80ad640 con 0x7fc0bc0cf0a0 2024-09-10T10:06:15.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.913+0000 7fc0baa00640 1 -- 172.21.15.110:0/1610475735 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80ad9f0 con 0x7fc0bc0cf0a0 2024-09-10T10:06:15.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- 172.21.15.110:0/1610475735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc0cf0a0 msgr2=0x7fc0bc0cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 --2- 172.21.15.110:0/1610475735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc0cf0a0 0x7fc0bc0cf4c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80097c0 tx=0x7fc0a801c4a0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- 172.21.15.110:0/1610475735 shutdown_connections 2024-09-10T10:06:15.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 --2- 172.21.15.110:0/1610475735 >> v2:172.21.15.145:3300/0 conn(0x7fc0bc13f9c0 0x7fc0bc13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 --2- 172.21.15.110:0/1610475735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc0cf0a0 0x7fc0bc0cf4c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- 172.21.15.110:0/1610475735 >> 172.21.15.110:0/1610475735 conn(0x7fc0bc06d710 msgr2=0x7fc0bc0b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:15.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- 172.21.15.110:0/1610475735 shutdown_connections 2024-09-10T10:06:15.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- 172.21.15.110:0/1610475735 wait complete. 2024-09-10T10:06:15.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 Processor -- start 2024-09-10T10:06:15.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- start start 2024-09-10T10:06:15.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0bc0b2df0 0x7fc0bc13b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0bc0cf0a0 0x7fc0bc142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc13f9c0 0x7fc0bc1429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc0b60d0 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc0b5f50 con 0x7fc0bc0cf0a0 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c1800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc0bc169010 con 0x7fc0bc13f9c0 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0bc0b2df0 0x7fc0bc13b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bbe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0bc0b2df0 0x7fc0bc13b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47138/0 (socket says 172.21.15.110:47138) 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bbe00640 1 -- 172.21.15.110:0/2262338468 learned_addr learned my addr 172.21.15.110:0/2262338468 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bb400640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0bc0cf0a0 0x7fc0bc142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c0e00640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc13f9c0 0x7fc0bc1429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bbe00640 1 -- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc13f9c0 msgr2=0x7fc0bc1429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bbe00640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc13f9c0 0x7fc0bc1429a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bbe00640 1 -- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0bc0cf0a0 msgr2=0x7fc0bc142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bbe00640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0bc0cf0a0 0x7fc0bc142460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0bbe00640 1 -- 172.21.15.110:0/2262338468 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc13bd50 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.917+0000 7fc0c0e00640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc13f9c0 0x7fc0bc1429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0bbe00640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0bc0b2df0 0x7fc0bc13b5a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00c3f0 tx=0x7fc0ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0b3e00640 1 -- 172.21.15.110:0/2262338468 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac017070 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0b3e00640 1 -- 172.21.15.110:0/2262338468 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ac005cf0 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0b3e00640 1 -- 172.21.15.110:0/2262338468 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0072f0 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc13c030 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc13c5f0 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0bc13fe70 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0b3e00640 1 -- 172.21.15.110:0/2262338468 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fc0ac01a070 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0b3e00640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc09c068250 0x7fc09c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0b3e00640 1 -- 172.21.15.110:0/2262338468 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7fc0ac0837d0 con 0x7fc0bc0b2df0 2024-09-10T10:06:15.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.921+0000 7fc0bb400640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc09c068250 0x7fc09c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.925+0000 7fc0bb400640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc09c068250 0x7fc09c06a710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80af000 tx=0x7fc0a80a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:15.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:15.925+0000 7fc0b3e00640 1 -- 172.21.15.110:0/2262338468 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc0ac0558c0 con 0x7fc0bc0b2df0 2024-09-10T10:06:16.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.113+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc0bc141830 con 0x7fc0bc0b2df0 2024-09-10T10:06:16.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.745+0000 7fc0b3e00640 1 -- 172.21.15.110:0/2262338468 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v115) ==== 122+0+0 (secure 0 0 0) 0x7fc0ac059bc0 con 0x7fc0bc0b2df0 2024-09-10T10:06:16.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc09c068250 msgr2=0x7fc09c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:16.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc09c068250 0x7fc09c06a710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80af000 tx=0x7fc0a80a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:16.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0bc0b2df0 msgr2=0x7fc0bc13b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:16.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0bc0b2df0 0x7fc0bc13b5a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00c3f0 tx=0x7fc0ac00c8b0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 shutdown_connections 2024-09-10T10:06:16.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc09c068250 0x7fc09c06a710 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc0bc13f9c0 0x7fc0bc1429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0bc0cf0a0 0x7fc0bc142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 --2- 172.21.15.110:0/2262338468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0bc0b2df0 0x7fc0bc13b5a0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 >> 172.21.15.110:0/2262338468 conn(0x7fc0bc06d710 msgr2=0x7fc0bc15dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:16.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 shutdown_connections 2024-09-10T10:06:16.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.749+0000 7fc0c1800640 1 -- 172.21.15.110:0/2262338468 wait complete. 2024-09-10T10:06:16.772 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:06:16.772 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:06:16.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2368c00640 1 Processor -- start 2024-09-10T10:06:16.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2368c00640 1 -- start start 2024-09-10T10:06:16.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2368c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f236415c560 0x7f236406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2368c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f236406f430 con 0x7f2364155970 2024-09-10T10:06:16.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2368c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2364155d50 con 0x7f236406f010 2024-09-10T10:06:16.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2368c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f23640b4570 con 0x7f236415c560 2024-09-10T10:06:16.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2361e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2364155970 0x7f23640cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37288/0 (socket says 172.21.15.110:37288) 2024-09-10T10:06:16.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2361e00640 1 -- 172.21.15.110:0/807959867 learned_addr learned my addr 172.21.15.110:0/807959867 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:16.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.909+0000 7f2361400640 1 --2- 172.21.15.110:0/807959867 >> v2:172.21.15.110:3300/0 conn(0x7f236415c560 0x7f236406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2360a00640 1 -- 172.21.15.110:0/807959867 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425852093 0 0) 0x7f236406f430 con 0x7f2364155970 2024-09-10T10:06:16.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2361400640 1 -- 172.21.15.110:0/807959867 >> v1:172.21.15.145:6789/0 conn(0x7f236406f010 legacy=0x7f23640ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:16.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2361400640 1 -- 172.21.15.110:0/807959867 >> v1:172.21.15.110:6789/0 conn(0x7f2364155970 legacy=0x7f23640cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:16.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2361400640 1 -- 172.21.15.110:0/807959867 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23640b34d0 con 0x7f236415c560 2024-09-10T10:06:16.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2361400640 1 --2- 172.21.15.110:0/807959867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f236406ead0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f23500097c0 tx=0x7f235001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f79ca242bce64f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:16.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2360a00640 1 -- 172.21.15.110:0/807959867 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23500a7070 con 0x7f236415c560 2024-09-10T10:06:16.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2360a00640 1 -- 172.21.15.110:0/807959867 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23500b0070 con 0x7f236415c560 2024-09-10T10:06:16.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2360a00640 1 -- 172.21.15.110:0/807959867 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23500ab890 con 0x7f236415c560 2024-09-10T10:06:16.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2368c00640 1 -- 172.21.15.110:0/807959867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 msgr2=0x7f236406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:16.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2368c00640 1 --2- 172.21.15.110:0/807959867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f236406ead0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f23500097c0 tx=0x7f235001c470 comp rx=0 tx=0).stop 2024-09-10T10:06:16.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2368c00640 1 -- 172.21.15.110:0/807959867 shutdown_connections 2024-09-10T10:06:16.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2368c00640 1 --2- 172.21.15.110:0/807959867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f236406ead0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.913+0000 7f2368c00640 1 -- 172.21.15.110:0/807959867 >> 172.21.15.110:0/807959867 conn(0x7f236406bc50 msgr2=0x7f236406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:16.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 -- 172.21.15.110:0/807959867 shutdown_connections 2024-09-10T10:06:16.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 -- 172.21.15.110:0/807959867 wait complete. 2024-09-10T10:06:16.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 Processor -- start 2024-09-10T10:06:16.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 -- start start 2024-09-10T10:06:16.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f236406f010 0x7f236414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2364155970 0x7f236414ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f2364072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23640cf340 con 0x7f236415c560 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f23640b4050 con 0x7f236406f010 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f23640cf4c0 con 0x7f2364155970 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f2364072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f2364072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47150/0 (socket says 172.21.15.110:47150) 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 -- 172.21.15.110:0/1819272100 learned_addr learned my addr 172.21.15.110:0/1819272100 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2361400640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2364155970 0x7f236414ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2361e00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f236406f010 0x7f236414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 -- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2364155970 msgr2=0x7f236414ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2364155970 0x7f236414ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 -- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f236406f010 msgr2=0x7f236414bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:16.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f236406f010 0x7f236414bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 -- 172.21.15.110:0/1819272100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236414c4f0 con 0x7f236415c560 2024-09-10T10:06:16.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2362800640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f2364072270 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c770 tx=0x7f235c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:16.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f235a000640 1 -- 172.21.15.110:0/1819272100 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c003b80 con 0x7f236415c560 2024-09-10T10:06:16.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f235a000640 1 -- 172.21.15.110:0/1819272100 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235c003d20 con 0x7f236415c560 2024-09-10T10:06:16.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236414c7d0 con 0x7f236415c560 2024-09-10T10:06:16.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.917+0000 7f235a000640 1 -- 172.21.15.110:0/1819272100 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c00aa90 con 0x7f236415c560 2024-09-10T10:06:16.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.921+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236414cd90 con 0x7f236415c560 2024-09-10T10:06:16.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.921+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236406ead0 con 0x7f236415c560 2024-09-10T10:06:16.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.921+0000 7f235a000640 1 -- 172.21.15.110:0/1819272100 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f235c00ac30 con 0x7f236415c560 2024-09-10T10:06:16.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.921+0000 7f235a000640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2344068280 0x7f234406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.921+0000 7f235a000640 1 -- 172.21.15.110:0/1819272100 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f235c0841b0 con 0x7f236415c560 2024-09-10T10:06:16.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.921+0000 7f2361e00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2344068280 0x7f234406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.921+0000 7f2361e00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2344068280 0x7f234406a740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2354007a90 tx=0x7f23540079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:16.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:16.925+0000 7f235a000640 1 -- 172.21.15.110:0/1819272100 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f235c0562a0 con 0x7f236415c560 2024-09-10T10:06:17.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.113+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2364073120 con 0x7f236415c560 2024-09-10T10:06:17.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.113+0000 7f235a000640 1 -- 172.21.15.110:0/1819272100 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5242 (secure 0 0 0) 0x7f235c05a5a0 con 0x7f236415c560 2024-09-10T10:06:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:17.116 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":115,"btime":"2024-09-10T10:06:16:743404+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1276880772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1276880772},{"type":"v1","addr":"172.21.15.145:6834","nonce":1276880772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2843226954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2843226954},{"type":"v1","addr":"172.21.15.110:6834","nonce":2843226954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8877,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/878298806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":878298806},{"type":"v1","addr":"172.21.15.110:6835","nonce":878298806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:06:13.100588+0000","modified":"2024-09-10T10:06:16.117620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/4014830481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4014830481},{"type":"v1","addr":"172.21.15.145:6835","nonce":4014830481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8883,"qdb_cluster":[8883]},"id":13}]} 2024-09-10T10:06:17.116 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 115 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2344068280 msgr2=0x7f234406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2344068280 0x7f234406a740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2354007a90 tx=0x7f23540079e0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 msgr2=0x7f2364072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f2364072270 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c770 tx=0x7f235c00cc30 comp rx=0 tx=0).stop 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 shutdown_connections 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2344068280 0x7f234406a740 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f236415c560 0x7f2364072270 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2364155970 0x7f236414ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 --2- 172.21.15.110:0/1819272100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f236406f010 0x7f236414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 >> 172.21.15.110:0/1819272100 conn(0x7f236406bc50 msgr2=0x7f23640b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:17.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 shutdown_connections 2024-09-10T10:06:17.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.117+0000 7f2368c00640 1 -- 172.21.15.110:0/1819272100 wait complete. 2024-09-10T10:06:17.136 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:06:17.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.269+0000 7f15bac00640 1 Processor -- start 2024-09-10T10:06:17.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.269+0000 7f15bac00640 1 -- start start 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15bac00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f15b40cfe30 0x7f15b4071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15bac00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 0x7f15b40cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15bac00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f15b406e090 0x7f15b406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15bac00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f15b40cdeb0 con 0x7f15b406e090 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15bac00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f15b415d9a0 con 0x7f15b40cfe30 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15bac00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f15b406ea60 con 0x7f15b40cd510 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b8c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f15b406e090 0x7f15b406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b8c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f15b406e090 0x7f15b406e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47162/0 (socket says 172.21.15.110:47162) 2024-09-10T10:06:17.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b8c00640 1 -- 172.21.15.110:0/2891361790 learned_addr learned my addr 172.21.15.110:0/2891361790 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b3400640 1 --2- 172.21.15.110:0/2891361790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 0x7f15b40cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b3e00640 1 --2- 172.21.15.110:0/2891361790 >> v2:172.21.15.145:3300/0 conn(0x7f15b40cfe30 0x7f15b4071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b8c00640 1 -- 172.21.15.110:0/2891361790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 msgr2=0x7f15b40cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b8c00640 1 --2- 172.21.15.110:0/2891361790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 0x7f15b40cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b8c00640 1 -- 172.21.15.110:0/2891361790 >> v2:172.21.15.145:3300/0 conn(0x7f15b40cfe30 msgr2=0x7f15b4071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b8c00640 1 --2- 172.21.15.110:0/2891361790 >> v2:172.21.15.145:3300/0 conn(0x7f15b40cfe30 0x7f15b4071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.273+0000 7f15b8c00640 1 -- 172.21.15.110:0/2891361790 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b406d2d0 con 0x7f15b406e090 2024-09-10T10:06:17.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15b8c00640 1 --2- 172.21.15.110:0/2891361790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b406e090 0x7f15b406e520 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f15b40ce120 tx=0x7f15ac01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d24e27b4c8f09bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:17.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15b2a00640 1 -- 172.21.15.110:0/2891361790 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0a56f0 con 0x7f15b406e090 2024-09-10T10:06:17.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15b2a00640 1 -- 172.21.15.110:0/2891361790 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac0a5890 con 0x7f15b406e090 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15b2a00640 1 -- 172.21.15.110:0/2891361790 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0a5c60 con 0x7f15b406e090 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 -- 172.21.15.110:0/2891361790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b406e090 msgr2=0x7f15b406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 --2- 172.21.15.110:0/2891361790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b406e090 0x7f15b406e520 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f15b40ce120 tx=0x7f15ac01c960 comp rx=0 tx=0).stop 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 -- 172.21.15.110:0/2891361790 shutdown_connections 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 --2- 172.21.15.110:0/2891361790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b406e090 0x7f15b406e520 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 --2- 172.21.15.110:0/2891361790 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 0x7f15b40cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 --2- 172.21.15.110:0/2891361790 >> v2:172.21.15.145:3300/0 conn(0x7f15b40cfe30 0x7f15b4071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 -- 172.21.15.110:0/2891361790 >> 172.21.15.110:0/2891361790 conn(0x7f15b406bc50 msgr2=0x7f15b406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 -- 172.21.15.110:0/2891361790 shutdown_connections 2024-09-10T10:06:17.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 -- 172.21.15.110:0/2891361790 wait complete. 2024-09-10T10:06:17.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 Processor -- start 2024-09-10T10:06:17.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.277+0000 7f15bac00640 1 -- start start 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15b406e090 0x7f15b414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 0x7f15b4150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b40cfe30 0x7f15b4150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15b415d430 con 0x7f15b40cfe30 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4072190 con 0x7f15b406e090 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f15b415d5b0 con 0x7f15b40cd510 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b40cfe30 0x7f15b4150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b40cfe30 0x7f15b4150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47166/0 (socket says 172.21.15.110:47166) 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 -- 172.21.15.110:0/3492067723 learned_addr learned my addr 172.21.15.110:0/3492067723 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b3400640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 0x7f15b4150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b3e00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15b406e090 0x7f15b414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 -- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 msgr2=0x7f15b4150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 0x7f15b4150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 -- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15b406e090 msgr2=0x7f15b414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15b406e090 0x7f15b414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 -- 172.21.15.110:0/3492067723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b414f3b0 con 0x7f15b40cfe30 2024-09-10T10:06:17.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15b8c00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b40cfe30 0x7f15b4150d60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0af040 tx=0x7f15ac019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:17.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15abe00640 1 -- 172.21.15.110:0/3492067723 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0bc020 con 0x7f15b40cfe30 2024-09-10T10:06:17.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15abe00640 1 -- 172.21.15.110:0/3492067723 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac0b8670 con 0x7f15b40cfe30 2024-09-10T10:06:17.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b414f690 con 0x7f15b40cfe30 2024-09-10T10:06:17.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15abe00640 1 -- 172.21.15.110:0/3492067723 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac009510 con 0x7f15b40cfe30 2024-09-10T10:06:17.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b414fc50 con 0x7f15b40cfe30 2024-09-10T10:06:17.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b406e520 con 0x7f15b40cfe30 2024-09-10T10:06:17.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.281+0000 7f15abe00640 1 -- 172.21.15.110:0/3492067723 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f15ac0bf020 con 0x7f15b40cfe30 2024-09-10T10:06:17.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.285+0000 7f15abe00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1594068250 0x7f159406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.285+0000 7f15abe00640 1 -- 172.21.15.110:0/3492067723 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f15ac11f100 con 0x7f15b40cfe30 2024-09-10T10:06:17.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.285+0000 7f15b3e00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1594068250 0x7f159406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.285+0000 7f15b3e00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1594068250 0x7f159406a710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009ac0 tx=0x7f15a4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:17.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.289+0000 7f15abe00640 1 -- 172.21.15.110:0/3492067723 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f15ac0c6030 con 0x7f15b40cfe30 2024-09-10T10:06:17.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.445+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15b415aff0 con 0x7f15b40cfe30 2024-09-10T10:06:17.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.445+0000 7f15abe00640 1 -- 172.21.15.110:0/3492067723 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+16868 (secure 0 0 0) 0x7f15ac0165e0 con 0x7f15b40cfe30 2024-09-10T10:06:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:17.448 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":165,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:06:14.723170+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:06:10.807237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T10:06:12.100049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:06:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1594068250 msgr2=0x7f159406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1594068250 0x7f159406a710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009ac0 tx=0x7f15a4009190 comp rx=0 tx=0).stop 2024-09-10T10:06:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b40cfe30 msgr2=0x7f15b4150d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b40cfe30 0x7f15b4150d60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0af040 tx=0x7f15ac019700 comp rx=0 tx=0).stop 2024-09-10T10:06:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 shutdown_connections 2024-09-10T10:06:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1594068250 0x7f159406a710 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b40cfe30 0x7f15b4150d60 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15b40cd510 0x7f15b4150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 --2- 172.21.15.110:0/3492067723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15b406e090 0x7f15b414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 >> 172.21.15.110:0/3492067723 conn(0x7f15b406bc50 msgr2=0x7f15b4072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 shutdown_connections 2024-09-10T10:06:17.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.449+0000 7f15bac00640 1 -- 172.21.15.110:0/3492067723 wait complete. 2024-09-10T10:06:17.470 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:06:17.470 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:06:17.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4f400640 1 Processor -- start 2024-09-10T10:06:17.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4f400640 1 -- start start 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cd510 0x7ffa480cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffa4806e090 0x7ffa4806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4f400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa48072150 con 0x7ffa480cfe30 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4f400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ffa480cdeb0 con 0x7ffa4806e090 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4f400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4815d9a0 con 0x7ffa480cd510 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4ca00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ffa480cfe30 0x7ffa48071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37296/0 (socket says 172.21.15.110:37296) 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4ca00640 1 -- 172.21.15.110:0/708095844 learned_addr learned my addr 172.21.15.110:0/708095844 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4d400640 1 --2- 172.21.15.110:0/708095844 >> v2:172.21.15.145:3300/0 conn(0x7ffa4806e090 0x7ffa4806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa47400640 1 -- 172.21.15.110:0/708095844 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 23412087 0 0) 0x7ffa48072150 con 0x7ffa480cfe30 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa47e00640 1 --2- 172.21.15.110:0/708095844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cd510 0x7ffa480cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa47400640 1 -- 172.21.15.110:0/708095844 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa2c003530 con 0x7ffa480cfe30 2024-09-10T10:06:17.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4d400640 1 -- 172.21.15.110:0/708095844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cd510 msgr2=0x7ffa480cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:17.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4d400640 1 --2- 172.21.15.110:0/708095844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cd510 0x7ffa480cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa47400640 1 -- 172.21.15.110:0/708095844 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 515393463 0 0) 0x7ffa2c003530 con 0x7ffa480cfe30 2024-09-10T10:06:17.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4d400640 1 -- 172.21.15.110:0/708095844 >> v1:172.21.15.110:6789/0 conn(0x7ffa480cfe30 legacy=0x7ffa48071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.605+0000 7ffa4d400640 1 -- 172.21.15.110:0/708095844 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4806ea60 con 0x7ffa4806e090 2024-09-10T10:06:17.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4d400640 1 --2- 172.21.15.110:0/708095844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 0x7ffa4806e520 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009d90 tx=0x7ffa4001c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc7d95737fcac08b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:17.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa47400640 1 -- 172.21.15.110:0/708095844 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400b2070 con 0x7ffa4806e090 2024-09-10T10:06:17.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa47400640 1 -- 172.21.15.110:0/708095844 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa400a5510 con 0x7ffa4806e090 2024-09-10T10:06:17.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa47400640 1 -- 172.21.15.110:0/708095844 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38001680 con 0x7ffa4806e090 2024-09-10T10:06:17.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4f400640 1 -- 172.21.15.110:0/708095844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 msgr2=0x7ffa4806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4f400640 1 --2- 172.21.15.110:0/708095844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 0x7ffa4806e520 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009d90 tx=0x7ffa4001c800 comp rx=0 tx=0).stop 2024-09-10T10:06:17.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4f400640 1 -- 172.21.15.110:0/708095844 shutdown_connections 2024-09-10T10:06:17.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4f400640 1 --2- 172.21.15.110:0/708095844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 0x7ffa4806e520 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4f400640 1 --2- 172.21.15.110:0/708095844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cd510 0x7ffa480cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4f400640 1 -- 172.21.15.110:0/708095844 >> 172.21.15.110:0/708095844 conn(0x7ffa4806bc50 msgr2=0x7ffa4806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:17.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4f400640 1 -- 172.21.15.110:0/708095844 shutdown_connections 2024-09-10T10:06:17.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.609+0000 7ffa4f400640 1 -- 172.21.15.110:0/708095844 wait complete. 2024-09-10T10:06:17.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 Processor -- start 2024-09-10T10:06:17.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 -- start start 2024-09-10T10:06:17.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 0x7ffa480b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa480cd510 0x7ffa48150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cfe30 0x7ffa48147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4815d5b0 con 0x7ffa480cd510 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4815d430 con 0x7ffa4806e090 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4815d730 con 0x7ffa480cfe30 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa480cd510 0x7ffa48150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa480cd510 0x7ffa48150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47168/0 (socket says 172.21.15.110:47168) 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 -- 172.21.15.110:0/2798699416 learned_addr learned my addr 172.21.15.110:0/2798699416 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4ca00640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 0x7ffa480b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4d400640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cfe30 0x7ffa48147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 -- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cfe30 msgr2=0x7ffa48147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cfe30 0x7ffa48147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 -- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 msgr2=0x7ffa480b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 0x7ffa480b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 -- 172.21.15.110:0/2798699416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa48069bb0 con 0x7ffa480cd510 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa47e00640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa480cd510 0x7ffa48150a00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400d5f0 tx=0x7ffa3400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa44c00640 1 -- 172.21.15.110:0/2798699416 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340125f0 con 0x7ffa480cd510 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa44c00640 1 -- 172.21.15.110:0/2798699416 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa34019070 con 0x7ffa480cd510 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa480b1030 con 0x7ffa480cd510 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa44c00640 1 -- 172.21.15.110:0/2798699416 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340129d0 con 0x7ffa480cd510 2024-09-10T10:06:17.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.613+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa480b18d0 con 0x7ffa480cd510 2024-09-10T10:06:17.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.617+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa480b1420 con 0x7ffa480cd510 2024-09-10T10:06:17.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.617+0000 7ffa44c00640 1 -- 172.21.15.110:0/2798699416 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7ffa34012b70 con 0x7ffa480cd510 2024-09-10T10:06:17.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.617+0000 7ffa44c00640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa28068250 0x7ffa2806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.617+0000 7ffa44c00640 1 -- 172.21.15.110:0/2798699416 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7ffa340848f0 con 0x7ffa480cd510 2024-09-10T10:06:17.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.617+0000 7ffa4ca00640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa28068250 0x7ffa2806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.617+0000 7ffa4ca00640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa28068250 0x7ffa2806a710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffa38007950 tx=0x7ffa38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:17.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.621+0000 7ffa44c00640 1 -- 172.21.15.110:0/2798699416 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ffa34057030 con 0x7ffa480cd510 2024-09-10T10:06:17.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.777+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa481558d0 con 0x7ffa28068250 2024-09-10T10:06:17.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.785+0000 7ffa44c00640 1 -- 172.21.15.110:0/2798699416 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252488 (secure 0 0 0) 0x7ffa481558d0 con 0x7ffa28068250 2024-09-10T10:06:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:17.789 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:06:17.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.785+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa28068250 msgr2=0x7ffa2806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.785+0000 7ffa4f400640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa28068250 0x7ffa2806a710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffa38007950 tx=0x7ffa38009040 comp rx=0 tx=0).stop 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.785+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa480cd510 msgr2=0x7ffa48150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.785+0000 7ffa4f400640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa480cd510 0x7ffa48150a00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400d5f0 tx=0x7ffa3400dab0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.789+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 shutdown_connections 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.789+0000 7ffa4f400640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ffa28068250 0x7ffa2806a710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.789+0000 7ffa4f400640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ffa480cfe30 0x7ffa48147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.789+0000 7ffa4f400640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffa480cd510 0x7ffa48150a00 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.789+0000 7ffa4f400640 1 --2- 172.21.15.110:0/2798699416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4806e090 0x7ffa480b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.789+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 >> 172.21.15.110:0/2798699416 conn(0x7ffa4806bc50 msgr2=0x7ffa48154c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.789+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 shutdown_connections 2024-09-10T10:06:17.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.789+0000 7ffa4f400640 1 -- 172.21.15.110:0/2798699416 wait complete. 2024-09-10T10:06:17.807 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-10T10:06:17.751392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283624,"kb_used_data":17448,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691096,"statfs":{"total":773094113280,"available":772803682304,"internally_reserved":0,"allocated":17866752,"data_stored":8522436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001365"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727742+0000","last_change":"2024-09-10T10:06:13.734465+0000","last_active":"2024-09-10T10:06:14.727742+0000","last_peered":"2024-09-10T10:06:14.727742+0000","last_clean":"2024-09-10T10:06:14.727742+0000","last_became_active":"2024-09-10T10:06:13.733857+0000","last_became_peered":"2024-09-10T10:06:13.733857+0000","last_unstale":"2024-09-10T10:06:14.727742+0000","last_undegraded":"2024-09-10T10:06:14.727742+0000","last_fullsized":"2024-09-10T10:06:14.727742+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:39.733024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734854+0000","last_change":"2024-09-10T10:06:13.741950+0000","last_active":"2024-09-10T10:06:14.734854+0000","last_peered":"2024-09-10T10:06:14.734854+0000","last_clean":"2024-09-10T10:06:14.734854+0000","last_became_active":"2024-09-10T10:06:13.741687+0000","last_became_peered":"2024-09-10T10:06:13.741687+0000","last_unstale":"2024-09-10T10:06:14.734854+0000","last_undegraded":"2024-09-10T10:06:14.734854+0000","last_fullsized":"2024-09-10T10:06:14.734854+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:06.371322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734819+0000","last_change":"2024-09-10T10:06:13.732325+0000","last_active":"2024-09-10T10:06:14.734819+0000","last_peered":"2024-09-10T10:06:14.734819+0000","last_clean":"2024-09-10T10:06:14.734819+0000","last_became_active":"2024-09-10T10:06:13.732030+0000","last_became_peered":"2024-09-10T10:06:13.732030+0000","last_unstale":"2024-09-10T10:06:14.734819+0000","last_undegraded":"2024-09-10T10:06:14.734819+0000","last_fullsized":"2024-09-10T10:06:14.734819+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:22.199408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735806+0000","last_change":"2024-09-10T10:06:13.746925+0000","last_active":"2024-09-10T10:06:14.735806+0000","last_peered":"2024-09-10T10:06:14.735806+0000","last_clean":"2024-09-10T10:06:14.735806+0000","last_became_active":"2024-09-10T10:06:13.746720+0000","last_became_peered":"2024-09-10T10:06:13.746720+0000","last_unstale":"2024-09-10T10:06:14.735806+0000","last_undegraded":"2024-09-10T10:06:14.735806+0000","last_fullsized":"2024-09-10T10:06:14.735806+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:36.904490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733251+0000","last_change":"2024-09-10T10:06:13.745046+0000","last_active":"2024-09-10T10:06:14.733251+0000","last_peered":"2024-09-10T10:06:14.733251+0000","last_clean":"2024-09-10T10:06:14.733251+0000","last_became_active":"2024-09-10T10:06:13.744405+0000","last_became_peered":"2024-09-10T10:06:13.744405+0000","last_unstale":"2024-09-10T10:06:14.733251+0000","last_undegraded":"2024-09-10T10:06:14.733251+0000","last_fullsized":"2024-09-10T10:06:14.733251+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:53.511879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733545+0000","last_change":"2024-09-10T10:06:13.731004+0000","last_active":"2024-09-10T10:06:14.733545+0000","last_peered":"2024-09-10T10:06:14.733545+0000","last_clean":"2024-09-10T10:06:14.733545+0000","last_became_active":"2024-09-10T10:06:13.730745+0000","last_became_peered":"2024-09-10T10:06:13.730745+0000","last_unstale":"2024-09-10T10:06:14.733545+0000","last_undegraded":"2024-09-10T10:06:14.733545+0000","last_fullsized":"2024-09-10T10:06:14.733545+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:24.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727612+0000","last_change":"2024-09-10T10:06:13.739547+0000","last_active":"2024-09-10T10:06:14.727612+0000","last_peered":"2024-09-10T10:06:14.727612+0000","last_clean":"2024-09-10T10:06:14.727612+0000","last_became_active":"2024-09-10T10:06:13.737781+0000","last_became_peered":"2024-09-10T10:06:13.737781+0000","last_unstale":"2024-09-10T10:06:14.727612+0000","last_undegraded":"2024-09-10T10:06:14.727612+0000","last_fullsized":"2024-09-10T10:06:14.727612+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:55.245814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733108+0000","last_change":"2024-09-10T10:06:13.731689+0000","last_active":"2024-09-10T10:06:14.733108+0000","last_peered":"2024-09-10T10:06:14.733108+0000","last_clean":"2024-09-10T10:06:14.733108+0000","last_became_active":"2024-09-10T10:06:13.731420+0000","last_became_peered":"2024-09-10T10:06:13.731420+0000","last_unstale":"2024-09-10T10:06:14.733108+0000","last_undegraded":"2024-09-10T10:06:14.733108+0000","last_fullsized":"2024-09-10T10:06:14.733108+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:00.681262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735233+0000","last_change":"2024-09-10T10:06:13.733998+0000","last_active":"2024-09-10T10:06:14.735233+0000","last_peered":"2024-09-10T10:06:14.735233+0000","last_clean":"2024-09-10T10:06:14.735233+0000","last_became_active":"2024-09-10T10:06:13.733678+0000","last_became_peered":"2024-09-10T10:06:13.733678+0000","last_unstale":"2024-09-10T10:06:14.735233+0000","last_undegraded":"2024-09-10T10:06:14.735233+0000","last_fullsized":"2024-09-10T10:06:14.735233+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:47.413666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733603+0000","last_change":"2024-09-10T10:06:13.743834+0000","last_active":"2024-09-10T10:06:14.733603+0000","last_peered":"2024-09-10T10:06:14.733603+0000","last_clean":"2024-09-10T10:06:14.733603+0000","last_became_active":"2024-09-10T10:06:13.743561+0000","last_became_peered":"2024-09-10T10:06:13.743561+0000","last_unstale":"2024-09-10T10:06:14.733603+0000","last_undegraded":"2024-09-10T10:06:14.733603+0000","last_fullsized":"2024-09-10T10:06:14.733603+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:33.533674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734851+0000","last_change":"2024-09-10T10:06:13.747512+0000","last_active":"2024-09-10T10:06:14.734851+0000","last_peered":"2024-09-10T10:06:14.734851+0000","last_clean":"2024-09-10T10:06:14.734851+0000","last_became_active":"2024-09-10T10:06:13.746425+0000","last_became_peered":"2024-09-10T10:06:13.746425+0000","last_unstale":"2024-09-10T10:06:14.734851+0000","last_undegraded":"2024-09-10T10:06:14.734851+0000","last_fullsized":"2024-09-10T10:06:14.734851+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:49.846210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730561+0000","last_change":"2024-09-10T10:06:13.731888+0000","last_active":"2024-09-10T10:06:14.730561+0000","last_peered":"2024-09-10T10:06:14.730561+0000","last_clean":"2024-09-10T10:06:14.730561+0000","last_became_active":"2024-09-10T10:06:13.731616+0000","last_became_peered":"2024-09-10T10:06:13.731616+0000","last_unstale":"2024-09-10T10:06:14.730561+0000","last_undegraded":"2024-09-10T10:06:14.730561+0000","last_fullsized":"2024-09-10T10:06:14.730561+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:21.893885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731114+0000","last_change":"2024-09-10T10:06:13.730415+0000","last_active":"2024-09-10T10:06:14.731114+0000","last_peered":"2024-09-10T10:06:14.731114+0000","last_clean":"2024-09-10T10:06:14.731114+0000","last_became_active":"2024-09-10T10:06:13.730136+0000","last_became_peered":"2024-09-10T10:06:13.730136+0000","last_unstale":"2024-09-10T10:06:14.731114+0000","last_undegraded":"2024-09-10T10:06:14.731114+0000","last_fullsized":"2024-09-10T10:06:14.731114+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:22.090794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727334+0000","last_change":"2024-09-10T10:06:13.739028+0000","last_active":"2024-09-10T10:06:14.727334+0000","last_peered":"2024-09-10T10:06:14.727334+0000","last_clean":"2024-09-10T10:06:14.727334+0000","last_became_active":"2024-09-10T10:06:13.738713+0000","last_became_peered":"2024-09-10T10:06:13.738713+0000","last_unstale":"2024-09-10T10:06:14.727334+0000","last_undegraded":"2024-09-10T10:06:14.727334+0000","last_fullsized":"2024-09-10T10:06:14.727334+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:28.990348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727266+0000","last_change":"2024-09-10T10:06:13.747898+0000","last_active":"2024-09-10T10:06:14.727266+0000","last_peered":"2024-09-10T10:06:14.727266+0000","last_clean":"2024-09-10T10:06:14.727266+0000","last_became_active":"2024-09-10T10:06:13.747563+0000","last_became_peered":"2024-09-10T10:06:13.747563+0000","last_unstale":"2024-09-10T10:06:14.727266+0000","last_undegraded":"2024-09-10T10:06:14.727266+0000","last_fullsized":"2024-09-10T10:06:14.727266+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:24.958103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732359+0000","last_change":"2024-09-10T10:06:13.744389+0000","last_active":"2024-09-10T10:06:14.732359+0000","last_peered":"2024-09-10T10:06:14.732359+0000","last_clean":"2024-09-10T10:06:14.732359+0000","last_became_active":"2024-09-10T10:06:13.744101+0000","last_became_peered":"2024-09-10T10:06:13.744101+0000","last_unstale":"2024-09-10T10:06:14.732359+0000","last_undegraded":"2024-09-10T10:06:14.732359+0000","last_fullsized":"2024-09-10T10:06:14.732359+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:49.444482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727904+0000","last_change":"2024-09-10T10:06:13.733778+0000","last_active":"2024-09-10T10:06:14.727904+0000","last_peered":"2024-09-10T10:06:14.727904+0000","last_clean":"2024-09-10T10:06:14.727904+0000","last_became_active":"2024-09-10T10:06:13.733516+0000","last_became_peered":"2024-09-10T10:06:13.733516+0000","last_unstale":"2024-09-10T10:06:14.727904+0000","last_undegraded":"2024-09-10T10:06:14.727904+0000","last_fullsized":"2024-09-10T10:06:14.727904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:01.857442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733737+0000","last_change":"2024-09-10T10:06:13.733795+0000","last_active":"2024-09-10T10:06:14.733737+0000","last_peered":"2024-09-10T10:06:14.733737+0000","last_clean":"2024-09-10T10:06:14.733737+0000","last_became_active":"2024-09-10T10:06:13.733506+0000","last_became_peered":"2024-09-10T10:06:13.733506+0000","last_unstale":"2024-09-10T10:06:14.733737+0000","last_undegraded":"2024-09-10T10:06:14.733737+0000","last_fullsized":"2024-09-10T10:06:14.733737+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:30.381657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733040+0000","last_change":"2024-09-10T10:06:13.745764+0000","last_active":"2024-09-10T10:06:14.733040+0000","last_peered":"2024-09-10T10:06:14.733040+0000","last_clean":"2024-09-10T10:06:14.733040+0000","last_became_active":"2024-09-10T10:06:13.744136+0000","last_became_peered":"2024-09-10T10:06:13.744136+0000","last_unstale":"2024-09-10T10:06:14.733040+0000","last_undegraded":"2024-09-10T10:06:14.733040+0000","last_fullsized":"2024-09-10T10:06:14.733040+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:46.932364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733447+0000","last_change":"2024-09-10T10:06:13.745345+0000","last_active":"2024-09-10T10:06:14.733447+0000","last_peered":"2024-09-10T10:06:14.733447+0000","last_clean":"2024-09-10T10:06:14.733447+0000","last_became_active":"2024-09-10T10:06:13.745067+0000","last_became_peered":"2024-09-10T10:06:13.745067+0000","last_unstale":"2024-09-10T10:06:14.733447+0000","last_undegraded":"2024-09-10T10:06:14.733447+0000","last_fullsized":"2024-09-10T10:06:14.733447+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:01.161851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735876+0000","last_change":"2024-09-10T10:06:13.744740+0000","last_active":"2024-09-10T10:06:14.735876+0000","last_peered":"2024-09-10T10:06:14.735876+0000","last_clean":"2024-09-10T10:06:14.735876+0000","last_became_active":"2024-09-10T10:06:13.744401+0000","last_became_peered":"2024-09-10T10:06:13.744401+0000","last_unstale":"2024-09-10T10:06:14.735876+0000","last_undegraded":"2024-09-10T10:06:14.735876+0000","last_fullsized":"2024-09-10T10:06:14.735876+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:45.035476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.729902+0000","last_change":"2024-09-10T10:06:13.750736+0000","last_active":"2024-09-10T10:06:14.729902+0000","last_peered":"2024-09-10T10:06:14.729902+0000","last_clean":"2024-09-10T10:06:14.729902+0000","last_became_active":"2024-09-10T10:06:13.750272+0000","last_became_peered":"2024-09-10T10:06:13.750272+0000","last_unstale":"2024-09-10T10:06:14.729902+0000","last_undegraded":"2024-09-10T10:06:14.729902+0000","last_fullsized":"2024-09-10T10:06:14.729902+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:53.511879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730061+0000","last_change":"2024-09-10T10:06:13.742519+0000","last_active":"2024-09-10T10:06:14.730061+0000","last_peered":"2024-09-10T10:06:14.730061+0000","last_clean":"2024-09-10T10:06:14.730061+0000","last_became_active":"2024-09-10T10:06:13.740224+0000","last_became_peered":"2024-09-10T10:06:13.740224+0000","last_unstale":"2024-09-10T10:06:14.730061+0000","last_undegraded":"2024-09-10T10:06:14.730061+0000","last_fullsized":"2024-09-10T10:06:14.730061+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:55.226969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735500+0000","last_change":"2024-09-10T10:06:13.746333+0000","last_active":"2024-09-10T10:06:14.735500+0000","last_peered":"2024-09-10T10:06:14.735500+0000","last_clean":"2024-09-10T10:06:14.735500+0000","last_became_active":"2024-09-10T10:06:13.745638+0000","last_became_peered":"2024-09-10T10:06:13.745638+0000","last_unstale":"2024-09-10T10:06:14.735500+0000","last_undegraded":"2024-09-10T10:06:14.735500+0000","last_fullsized":"2024-09-10T10:06:14.735500+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:05.945907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735014+0000","last_change":"2024-09-10T10:06:13.748004+0000","last_active":"2024-09-10T10:06:14.735014+0000","last_peered":"2024-09-10T10:06:14.735014+0000","last_clean":"2024-09-10T10:06:14.735014+0000","last_became_active":"2024-09-10T10:06:13.747687+0000","last_became_peered":"2024-09-10T10:06:13.747687+0000","last_unstale":"2024-09-10T10:06:14.735014+0000","last_undegraded":"2024-09-10T10:06:14.735014+0000","last_fullsized":"2024-09-10T10:06:14.735014+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.316000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727697+0000","last_change":"2024-09-10T10:06:13.741284+0000","last_active":"2024-09-10T10:06:14.727697+0000","last_peered":"2024-09-10T10:06:14.727697+0000","last_clean":"2024-09-10T10:06:14.727697+0000","last_became_active":"2024-09-10T10:06:13.740588+0000","last_became_peered":"2024-09-10T10:06:13.740588+0000","last_unstale":"2024-09-10T10:06:14.727697+0000","last_undegraded":"2024-09-10T10:06:14.727697+0000","last_fullsized":"2024-09-10T10:06:14.727697+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:21.426914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731411+0000","last_change":"2024-09-10T10:06:13.734915+0000","last_active":"2024-09-10T10:06:14.731411+0000","last_peered":"2024-09-10T10:06:14.731411+0000","last_clean":"2024-09-10T10:06:14.731411+0000","last_became_active":"2024-09-10T10:06:13.734595+0000","last_became_peered":"2024-09-10T10:06:13.734595+0000","last_unstale":"2024-09-10T10:06:14.731411+0000","last_undegraded":"2024-09-10T10:06:14.731411+0000","last_fullsized":"2024-09-10T10:06:14.731411+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:51.143945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732720+0000","last_change":"2024-09-10T10:06:13.736977+0000","last_active":"2024-09-10T10:06:14.732720+0000","last_peered":"2024-09-10T10:06:14.732720+0000","last_clean":"2024-09-10T10:06:14.732720+0000","last_became_active":"2024-09-10T10:06:13.736612+0000","last_became_peered":"2024-09-10T10:06:13.736612+0000","last_unstale":"2024-09-10T10:06:14.732720+0000","last_undegraded":"2024-09-10T10:06:14.732720+0000","last_fullsized":"2024-09-10T10:06:14.732720+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:44.911704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730687+0000","last_change":"2024-09-10T10:06:13.732529+0000","last_active":"2024-09-10T10:06:14.730687+0000","last_peered":"2024-09-10T10:06:14.730687+0000","last_clean":"2024-09-10T10:06:14.730687+0000","last_became_active":"2024-09-10T10:06:13.732257+0000","last_became_peered":"2024-09-10T10:06:13.732257+0000","last_unstale":"2024-09-10T10:06:14.730687+0000","last_undegraded":"2024-09-10T10:06:14.730687+0000","last_fullsized":"2024-09-10T10:06:14.730687+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:28.747834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734859+0000","last_change":"2024-09-10T10:06:13.748754+0000","last_active":"2024-09-10T10:06:14.734859+0000","last_peered":"2024-09-10T10:06:14.734859+0000","last_clean":"2024-09-10T10:06:14.734859+0000","last_became_active":"2024-09-10T10:06:13.748364+0000","last_became_peered":"2024-09-10T10:06:13.748364+0000","last_unstale":"2024-09-10T10:06:14.734859+0000","last_undegraded":"2024-09-10T10:06:14.734859+0000","last_fullsized":"2024-09-10T10:06:14.734859+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:41.757207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732396+0000","last_change":"2024-09-10T10:06:13.735742+0000","last_active":"2024-09-10T10:06:14.732396+0000","last_peered":"2024-09-10T10:06:14.732396+0000","last_clean":"2024-09-10T10:06:14.732396+0000","last_became_active":"2024-09-10T10:06:13.735403+0000","last_became_peered":"2024-09-10T10:06:13.735403+0000","last_unstale":"2024-09-10T10:06:14.732396+0000","last_undegraded":"2024-09-10T10:06:14.732396+0000","last_fullsized":"2024-09-10T10:06:14.732396+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:52.996895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727325+0000","last_change":"2024-09-10T10:06:13.737537+0000","last_active":"2024-09-10T10:06:14.727325+0000","last_peered":"2024-09-10T10:06:14.727325+0000","last_clean":"2024-09-10T10:06:14.727325+0000","last_became_active":"2024-09-10T10:06:13.737227+0000","last_became_peered":"2024-09-10T10:06:13.737227+0000","last_unstale":"2024-09-10T10:06:14.727325+0000","last_undegraded":"2024-09-10T10:06:14.727325+0000","last_fullsized":"2024-09-10T10:06:14.727325+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:05.945907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735009+0000","last_change":"2024-09-10T10:06:13.732228+0000","last_active":"2024-09-10T10:06:14.735009+0000","last_peered":"2024-09-10T10:06:14.735009+0000","last_clean":"2024-09-10T10:06:14.735009+0000","last_became_active":"2024-09-10T10:06:13.731884+0000","last_became_peered":"2024-09-10T10:06:13.731884+0000","last_unstale":"2024-09-10T10:06:14.735009+0000","last_undegraded":"2024-09-10T10:06:14.735009+0000","last_fullsized":"2024-09-10T10:06:14.735009+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:49.846210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"164'4","reported_seq":18,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731207+0000","last_change":"2024-09-10T10:06:12.735303+0000","last_active":"2024-09-10T10:06:14.731207+0000","last_peered":"2024-09-10T10:06:14.731207+0000","last_clean":"2024-09-10T10:06:14.731207+0000","last_became_active":"2024-09-10T10:06:12.735001+0000","last_became_peered":"2024-09-10T10:06:12.735001+0000","last_unstale":"2024-09-10T10:06:14.731207+0000","last_undegraded":"2024-09-10T10:06:14.731207+0000","last_fullsized":"2024-09-10T10:06:14.731207+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:54.658689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730083+0000","last_change":"2024-09-10T10:06:13.746703+0000","last_active":"2024-09-10T10:06:14.730083+0000","last_peered":"2024-09-10T10:06:14.730083+0000","last_clean":"2024-09-10T10:06:14.730083+0000","last_became_active":"2024-09-10T10:06:13.746294+0000","last_became_peered":"2024-09-10T10:06:13.746294+0000","last_unstale":"2024-09-10T10:06:14.730083+0000","last_undegraded":"2024-09-10T10:06:14.730083+0000","last_fullsized":"2024-09-10T10:06:14.730083+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:01.161851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735202+0000","last_change":"2024-09-10T10:06:12.735541+0000","last_active":"2024-09-10T10:06:14.735202+0000","last_peered":"2024-09-10T10:06:14.735202+0000","last_clean":"2024-09-10T10:06:14.735202+0000","last_became_active":"2024-09-10T10:06:12.735242+0000","last_became_peered":"2024-09-10T10:06:12.735242+0000","last_unstale":"2024-09-10T10:06:14.735202+0000","last_undegraded":"2024-09-10T10:06:14.735202+0000","last_fullsized":"2024-09-10T10:06:14.735202+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:32.219825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727475+0000","last_change":"2024-09-10T10:06:13.738536+0000","last_active":"2024-09-10T10:06:14.727475+0000","last_peered":"2024-09-10T10:06:14.727475+0000","last_clean":"2024-09-10T10:06:14.727475+0000","last_became_active":"2024-09-10T10:06:13.738251+0000","last_became_peered":"2024-09-10T10:06:13.738251+0000","last_unstale":"2024-09-10T10:06:14.727475+0000","last_undegraded":"2024-09-10T10:06:14.727475+0000","last_fullsized":"2024-09-10T10:06:14.727475+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:45.035476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734941+0000","last_change":"2024-09-10T10:06:12.734080+0000","last_active":"2024-09-10T10:06:14.734941+0000","last_peered":"2024-09-10T10:06:14.734941+0000","last_clean":"2024-09-10T10:06:14.734941+0000","last_became_active":"2024-09-10T10:06:12.733770+0000","last_became_peered":"2024-09-10T10:06:12.733770+0000","last_unstale":"2024-09-10T10:06:14.734941+0000","last_undegraded":"2024-09-10T10:06:14.734941+0000","last_fullsized":"2024-09-10T10:06:14.734941+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:53.099507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732771+0000","last_change":"2024-09-10T10:06:13.745616+0000","last_active":"2024-09-10T10:06:14.732771+0000","last_peered":"2024-09-10T10:06:14.732771+0000","last_clean":"2024-09-10T10:06:14.732771+0000","last_became_active":"2024-09-10T10:06:13.743958+0000","last_became_peered":"2024-09-10T10:06:13.743958+0000","last_unstale":"2024-09-10T10:06:14.732771+0000","last_undegraded":"2024-09-10T10:06:14.732771+0000","last_fullsized":"2024-09-10T10:06:14.732771+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:26.369523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735924+0000","last_change":"2024-09-10T10:06:12.736958+0000","last_active":"2024-09-10T10:06:14.735924+0000","last_peered":"2024-09-10T10:06:14.735924+0000","last_clean":"2024-09-10T10:06:14.735924+0000","last_became_active":"2024-09-10T10:06:12.736631+0000","last_became_peered":"2024-09-10T10:06:12.736631+0000","last_unstale":"2024-09-10T10:06:14.735924+0000","last_undegraded":"2024-09-10T10:06:14.735924+0000","last_fullsized":"2024-09-10T10:06:14.735924+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:32.460556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735859+0000","last_change":"2024-09-10T10:06:13.746857+0000","last_active":"2024-09-10T10:06:14.735859+0000","last_peered":"2024-09-10T10:06:14.735859+0000","last_clean":"2024-09-10T10:06:14.735859+0000","last_became_active":"2024-09-10T10:06:13.746525+0000","last_became_peered":"2024-09-10T10:06:13.746525+0000","last_unstale":"2024-09-10T10:06:14.735859+0000","last_undegraded":"2024-09-10T10:06:14.735859+0000","last_fullsized":"2024-09-10T10:06:14.735859+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.137086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733773+0000","last_change":"2024-09-10T10:06:12.730590+0000","last_active":"2024-09-10T10:06:14.733773+0000","last_peered":"2024-09-10T10:06:14.733773+0000","last_clean":"2024-09-10T10:06:14.733773+0000","last_became_active":"2024-09-10T10:06:12.730331+0000","last_became_peered":"2024-09-10T10:06:12.730331+0000","last_unstale":"2024-09-10T10:06:14.733773+0000","last_undegraded":"2024-09-10T10:06:14.733773+0000","last_fullsized":"2024-09-10T10:06:14.733773+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:28.390353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733253+0000","last_change":"2024-09-10T10:06:13.725498+0000","last_active":"2024-09-10T10:06:14.733253+0000","last_peered":"2024-09-10T10:06:14.733253+0000","last_clean":"2024-09-10T10:06:14.733253+0000","last_became_active":"2024-09-10T10:06:13.725221+0000","last_became_peered":"2024-09-10T10:06:13.725221+0000","last_unstale":"2024-09-10T10:06:14.733253+0000","last_undegraded":"2024-09-10T10:06:14.733253+0000","last_fullsized":"2024-09-10T10:06:14.733253+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:55.226969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733093+0000","last_change":"2024-09-10T10:06:12.737164+0000","last_active":"2024-09-10T10:06:14.733093+0000","last_peered":"2024-09-10T10:06:14.733093+0000","last_clean":"2024-09-10T10:06:14.733093+0000","last_became_active":"2024-09-10T10:06:12.736918+0000","last_became_peered":"2024-09-10T10:06:12.736918+0000","last_unstale":"2024-09-10T10:06:14.733093+0000","last_undegraded":"2024-09-10T10:06:14.733093+0000","last_fullsized":"2024-09-10T10:06:14.733093+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:21.196567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735515+0000","last_change":"2024-09-10T10:06:13.745196+0000","last_active":"2024-09-10T10:06:14.735515+0000","last_peered":"2024-09-10T10:06:14.735515+0000","last_clean":"2024-09-10T10:06:14.735515+0000","last_became_active":"2024-09-10T10:06:13.744918+0000","last_became_peered":"2024-09-10T10:06:13.744918+0000","last_unstale":"2024-09-10T10:06:14.735515+0000","last_undegraded":"2024-09-10T10:06:14.735515+0000","last_fullsized":"2024-09-10T10:06:14.735515+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:28.990348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730367+0000","last_change":"2024-09-10T10:06:12.733832+0000","last_active":"2024-09-10T10:06:14.730367+0000","last_peered":"2024-09-10T10:06:14.730367+0000","last_clean":"2024-09-10T10:06:14.730367+0000","last_became_active":"2024-09-10T10:06:12.733522+0000","last_became_peered":"2024-09-10T10:06:12.733522+0000","last_unstale":"2024-09-10T10:06:14.730367+0000","last_undegraded":"2024-09-10T10:06:14.730367+0000","last_fullsized":"2024-09-10T10:06:14.730367+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:55.935424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.736239+0000","last_change":"2024-09-10T10:06:13.739573+0000","last_active":"2024-09-10T10:06:14.736239+0000","last_peered":"2024-09-10T10:06:14.736239+0000","last_clean":"2024-09-10T10:06:14.736239+0000","last_became_active":"2024-09-10T10:06:13.738317+0000","last_became_peered":"2024-09-10T10:06:13.738317+0000","last_unstale":"2024-09-10T10:06:14.736239+0000","last_undegraded":"2024-09-10T10:06:14.736239+0000","last_fullsized":"2024-09-10T10:06:14.736239+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:55.245814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733053+0000","last_change":"2024-09-10T10:06:12.732876+0000","last_active":"2024-09-10T10:06:14.733053+0000","last_peered":"2024-09-10T10:06:14.733053+0000","last_clean":"2024-09-10T10:06:14.733053+0000","last_became_active":"2024-09-10T10:06:12.732612+0000","last_became_peered":"2024-09-10T10:06:12.732612+0000","last_unstale":"2024-09-10T10:06:14.733053+0000","last_undegraded":"2024-09-10T10:06:14.733053+0000","last_fullsized":"2024-09-10T10:06:14.733053+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:48.843369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734519+0000","last_change":"2024-09-10T10:06:13.745115+0000","last_active":"2024-09-10T10:06:14.734519+0000","last_peered":"2024-09-10T10:06:14.734519+0000","last_clean":"2024-09-10T10:06:14.734519+0000","last_became_active":"2024-09-10T10:06:13.744806+0000","last_became_peered":"2024-09-10T10:06:13.744806+0000","last_unstale":"2024-09-10T10:06:14.734519+0000","last_undegraded":"2024-09-10T10:06:14.734519+0000","last_fullsized":"2024-09-10T10:06:14.734519+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:41.757207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730398+0000","last_change":"2024-09-10T10:06:12.732909+0000","last_active":"2024-09-10T10:06:14.730398+0000","last_peered":"2024-09-10T10:06:14.730398+0000","last_clean":"2024-09-10T10:06:14.730398+0000","last_became_active":"2024-09-10T10:06:12.732407+0000","last_became_peered":"2024-09-10T10:06:12.732407+0000","last_unstale":"2024-09-10T10:06:14.730398+0000","last_undegraded":"2024-09-10T10:06:14.730398+0000","last_fullsized":"2024-09-10T10:06:14.730398+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:47.973993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.736025+0000","last_change":"2024-09-10T10:06:13.743575+0000","last_active":"2024-09-10T10:06:14.736025+0000","last_peered":"2024-09-10T10:06:14.736025+0000","last_clean":"2024-09-10T10:06:14.736025+0000","last_became_active":"2024-09-10T10:06:13.743039+0000","last_became_peered":"2024-09-10T10:06:13.743039+0000","last_unstale":"2024-09-10T10:06:14.736025+0000","last_undegraded":"2024-09-10T10:06:14.736025+0000","last_fullsized":"2024-09-10T10:06:14.736025+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:42.985695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"164'3","reported_seq":17,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733134+0000","last_change":"2024-09-10T10:06:12.737610+0000","last_active":"2024-09-10T10:06:14.733134+0000","last_peered":"2024-09-10T10:06:14.733134+0000","last_clean":"2024-09-10T10:06:14.733134+0000","last_became_active":"2024-09-10T10:06:12.737350+0000","last_became_peered":"2024-09-10T10:06:12.737350+0000","last_unstale":"2024-09-10T10:06:14.733134+0000","last_undegraded":"2024-09-10T10:06:14.733134+0000","last_fullsized":"2024-09-10T10:06:14.733134+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:58.009495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727663+0000","last_change":"2024-09-10T10:06:13.750099+0000","last_active":"2024-09-10T10:06:14.727663+0000","last_peered":"2024-09-10T10:06:14.727663+0000","last_clean":"2024-09-10T10:06:14.727663+0000","last_became_active":"2024-09-10T10:06:13.747826+0000","last_became_peered":"2024-09-10T10:06:13.747826+0000","last_unstale":"2024-09-10T10:06:14.727663+0000","last_undegraded":"2024-09-10T10:06:14.727663+0000","last_fullsized":"2024-09-10T10:06:14.727663+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:28:54.423544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"164'6","reported_seq":20,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.736117+0000","last_change":"2024-09-10T10:06:12.730656+0000","last_active":"2024-09-10T10:06:14.736117+0000","last_peered":"2024-09-10T10:06:14.736117+0000","last_clean":"2024-09-10T10:06:14.736117+0000","last_became_active":"2024-09-10T10:06:12.730333+0000","last_became_peered":"2024-09-10T10:06:12.730333+0000","last_unstale":"2024-09-10T10:06:14.736117+0000","last_undegraded":"2024-09-10T10:06:14.736117+0000","last_fullsized":"2024-09-10T10:06:14.736117+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:08.591001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733676+0000","last_change":"2024-09-10T10:06:13.746041+0000","last_active":"2024-09-10T10:06:14.733676+0000","last_peered":"2024-09-10T10:06:14.733676+0000","last_clean":"2024-09-10T10:06:14.733676+0000","last_became_active":"2024-09-10T10:06:13.745591+0000","last_became_peered":"2024-09-10T10:06:13.745591+0000","last_unstale":"2024-09-10T10:06:14.733676+0000","last_undegraded":"2024-09-10T10:06:14.733676+0000","last_fullsized":"2024-09-10T10:06:14.733676+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:21.893885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730871+0000","last_change":"2024-09-10T10:06:12.725249+0000","last_active":"2024-09-10T10:06:14.730871+0000","last_peered":"2024-09-10T10:06:14.730871+0000","last_clean":"2024-09-10T10:06:14.730871+0000","last_became_active":"2024-09-10T10:06:12.724875+0000","last_became_peered":"2024-09-10T10:06:12.724875+0000","last_unstale":"2024-09-10T10:06:14.730871+0000","last_undegraded":"2024-09-10T10:06:14.730871+0000","last_fullsized":"2024-09-10T10:06:14.730871+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:28.390353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734314+0000","last_change":"2024-09-10T10:06:13.735749+0000","last_active":"2024-09-10T10:06:14.734314+0000","last_peered":"2024-09-10T10:06:14.734314+0000","last_clean":"2024-09-10T10:06:14.734314+0000","last_became_active":"2024-09-10T10:06:13.735312+0000","last_became_peered":"2024-09-10T10:06:13.735312+0000","last_unstale":"2024-09-10T10:06:14.734314+0000","last_undegraded":"2024-09-10T10:06:14.734314+0000","last_fullsized":"2024-09-10T10:06:14.734314+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:32.270063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.729865+0000","last_change":"2024-09-10T10:06:12.729245+0000","last_active":"2024-09-10T10:06:14.729865+0000","last_peered":"2024-09-10T10:06:14.729865+0000","last_clean":"2024-09-10T10:06:14.729865+0000","last_became_active":"2024-09-10T10:06:12.728911+0000","last_became_peered":"2024-09-10T10:06:12.728911+0000","last_unstale":"2024-09-10T10:06:14.729865+0000","last_undegraded":"2024-09-10T10:06:14.729865+0000","last_fullsized":"2024-09-10T10:06:14.729865+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:00.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732450+0000","last_change":"2024-09-10T10:06:13.745599+0000","last_active":"2024-09-10T10:06:14.732450+0000","last_peered":"2024-09-10T10:06:14.732450+0000","last_clean":"2024-09-10T10:06:14.732450+0000","last_became_active":"2024-09-10T10:06:13.744797+0000","last_became_peered":"2024-09-10T10:06:13.744797+0000","last_unstale":"2024-09-10T10:06:14.732450+0000","last_undegraded":"2024-09-10T10:06:14.732450+0000","last_fullsized":"2024-09-10T10:06:14.732450+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:10.748536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732555+0000","last_change":"2024-09-10T10:06:12.728387+0000","last_active":"2024-09-10T10:06:14.732555+0000","last_peered":"2024-09-10T10:06:14.732555+0000","last_clean":"2024-09-10T10:06:14.732555+0000","last_became_active":"2024-09-10T10:06:12.727050+0000","last_became_peered":"2024-09-10T10:06:12.727050+0000","last_unstale":"2024-09-10T10:06:14.732555+0000","last_undegraded":"2024-09-10T10:06:14.732555+0000","last_fullsized":"2024-09-10T10:06:14.732555+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:12:44.215834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727419+0000","last_change":"2024-09-10T10:06:13.742641+0000","last_active":"2024-09-10T10:06:14.727419+0000","last_peered":"2024-09-10T10:06:14.727419+0000","last_clean":"2024-09-10T10:06:14.727419+0000","last_became_active":"2024-09-10T10:06:13.737362+0000","last_became_peered":"2024-09-10T10:06:13.737362+0000","last_unstale":"2024-09-10T10:06:14.727419+0000","last_undegraded":"2024-09-10T10:06:14.727419+0000","last_fullsized":"2024-09-10T10:06:14.727419+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:36.904490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"164'8","reported_seq":22,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732684+0000","last_change":"2024-09-10T10:06:12.734036+0000","last_active":"2024-09-10T10:06:14.732684+0000","last_peered":"2024-09-10T10:06:14.732684+0000","last_clean":"2024-09-10T10:06:14.732684+0000","last_became_active":"2024-09-10T10:06:12.733733+0000","last_became_peered":"2024-09-10T10:06:12.733733+0000","last_unstale":"2024-09-10T10:06:14.732684+0000","last_undegraded":"2024-09-10T10:06:14.732684+0000","last_fullsized":"2024-09-10T10:06:14.732684+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:23.990859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735052+0000","last_change":"2024-09-10T10:06:13.734316+0000","last_active":"2024-09-10T10:06:14.735052+0000","last_peered":"2024-09-10T10:06:14.735052+0000","last_clean":"2024-09-10T10:06:14.735052+0000","last_became_active":"2024-09-10T10:06:13.733984+0000","last_became_peered":"2024-09-10T10:06:13.733984+0000","last_unstale":"2024-09-10T10:06:14.735052+0000","last_undegraded":"2024-09-10T10:06:14.735052+0000","last_fullsized":"2024-09-10T10:06:14.735052+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:56.938265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733160+0000","last_change":"2024-09-10T10:06:12.727110+0000","last_active":"2024-09-10T10:06:14.733160+0000","last_peered":"2024-09-10T10:06:14.733160+0000","last_clean":"2024-09-10T10:06:14.733160+0000","last_became_active":"2024-09-10T10:06:12.726876+0000","last_became_peered":"2024-09-10T10:06:12.726876+0000","last_unstale":"2024-09-10T10:06:14.733160+0000","last_undegraded":"2024-09-10T10:06:14.733160+0000","last_fullsized":"2024-09-10T10:06:14.733160+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:59.313159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735042+0000","last_change":"2024-09-10T10:06:13.748998+0000","last_active":"2024-09-10T10:06:14.735042+0000","last_peered":"2024-09-10T10:06:14.735042+0000","last_clean":"2024-09-10T10:06:14.735042+0000","last_became_active":"2024-09-10T10:06:13.748036+0000","last_became_peered":"2024-09-10T10:06:13.748036+0000","last_unstale":"2024-09-10T10:06:14.735042+0000","last_undegraded":"2024-09-10T10:06:14.735042+0000","last_fullsized":"2024-09-10T10:06:14.735042+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:41.037988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735203+0000","last_change":"2024-09-10T10:06:12.737826+0000","last_active":"2024-09-10T10:06:14.735203+0000","last_peered":"2024-09-10T10:06:14.735203+0000","last_clean":"2024-09-10T10:06:14.735203+0000","last_became_active":"2024-09-10T10:06:12.737371+0000","last_became_peered":"2024-09-10T10:06:12.737371+0000","last_unstale":"2024-09-10T10:06:14.735203+0000","last_undegraded":"2024-09-10T10:06:14.735203+0000","last_fullsized":"2024-09-10T10:06:14.735203+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:53.099507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732859+0000","last_change":"2024-09-10T10:06:13.736506+0000","last_active":"2024-09-10T10:06:14.732859+0000","last_peered":"2024-09-10T10:06:14.732859+0000","last_clean":"2024-09-10T10:06:14.732859+0000","last_became_active":"2024-09-10T10:06:13.735978+0000","last_became_peered":"2024-09-10T10:06:13.735978+0000","last_unstale":"2024-09-10T10:06:14.732859+0000","last_undegraded":"2024-09-10T10:06:14.732859+0000","last_fullsized":"2024-09-10T10:06:14.732859+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:31.507046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733227+0000","last_change":"2024-09-10T10:06:12.724585+0000","last_active":"2024-09-10T10:06:14.733227+0000","last_peered":"2024-09-10T10:06:14.733227+0000","last_clean":"2024-09-10T10:06:14.733227+0000","last_became_active":"2024-09-10T10:06:12.724329+0000","last_became_peered":"2024-09-10T10:06:12.724329+0000","last_unstale":"2024-09-10T10:06:14.733227+0000","last_undegraded":"2024-09-10T10:06:14.733227+0000","last_fullsized":"2024-09-10T10:06:14.733227+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:43.007873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732905+0000","last_change":"2024-09-10T10:06:13.744391+0000","last_active":"2024-09-10T10:06:14.732905+0000","last_peered":"2024-09-10T10:06:14.732905+0000","last_clean":"2024-09-10T10:06:14.732905+0000","last_became_active":"2024-09-10T10:06:13.743509+0000","last_became_peered":"2024-09-10T10:06:13.743509+0000","last_unstale":"2024-09-10T10:06:14.732905+0000","last_undegraded":"2024-09-10T10:06:14.732905+0000","last_fullsized":"2024-09-10T10:06:14.732905+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:50.918252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734260+0000","last_change":"2024-09-10T10:06:12.736724+0000","last_active":"2024-09-10T10:06:14.734260+0000","last_peered":"2024-09-10T10:06:14.734260+0000","last_clean":"2024-09-10T10:06:14.734260+0000","last_became_active":"2024-09-10T10:06:12.736469+0000","last_became_peered":"2024-09-10T10:06:12.736469+0000","last_unstale":"2024-09-10T10:06:14.734260+0000","last_undegraded":"2024-09-10T10:06:14.734260+0000","last_fullsized":"2024-09-10T10:06:14.734260+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:16.719617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732827+0000","last_change":"2024-09-10T10:06:13.748174+0000","last_active":"2024-09-10T10:06:14.732827+0000","last_peered":"2024-09-10T10:06:14.732827+0000","last_clean":"2024-09-10T10:06:14.732827+0000","last_became_active":"2024-09-10T10:06:13.747903+0000","last_became_peered":"2024-09-10T10:06:13.747903+0000","last_unstale":"2024-09-10T10:06:14.732827+0000","last_undegraded":"2024-09-10T10:06:14.732827+0000","last_fullsized":"2024-09-10T10:06:14.732827+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:06.793474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734408+0000","last_change":"2024-09-10T10:06:12.733519+0000","last_active":"2024-09-10T10:06:14.734408+0000","last_peered":"2024-09-10T10:06:14.734408+0000","last_clean":"2024-09-10T10:06:14.734408+0000","last_became_active":"2024-09-10T10:06:12.732717+0000","last_became_peered":"2024-09-10T10:06:12.732717+0000","last_unstale":"2024-09-10T10:06:14.734408+0000","last_undegraded":"2024-09-10T10:06:14.734408+0000","last_fullsized":"2024-09-10T10:06:14.734408+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:54.805925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734096+0000","last_change":"2024-09-10T10:06:13.733484+0000","last_active":"2024-09-10T10:06:14.734096+0000","last_peered":"2024-09-10T10:06:14.734096+0000","last_clean":"2024-09-10T10:06:14.734096+0000","last_became_active":"2024-09-10T10:06:13.733237+0000","last_became_peered":"2024-09-10T10:06:13.733237+0000","last_unstale":"2024-09-10T10:06:14.734096+0000","last_undegraded":"2024-09-10T10:06:14.734096+0000","last_fullsized":"2024-09-10T10:06:14.734096+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:06.371322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727528+0000","last_change":"2024-09-10T10:06:12.730275+0000","last_active":"2024-09-10T10:06:14.727528+0000","last_peered":"2024-09-10T10:06:14.727528+0000","last_clean":"2024-09-10T10:06:14.727528+0000","last_became_active":"2024-09-10T10:06:12.730014+0000","last_became_peered":"2024-09-10T10:06:12.730014+0000","last_unstale":"2024-09-10T10:06:14.727528+0000","last_undegraded":"2024-09-10T10:06:14.727528+0000","last_fullsized":"2024-09-10T10:06:14.727528+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:47.434806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734715+0000","last_change":"2024-09-10T10:06:13.735906+0000","last_active":"2024-09-10T10:06:14.734715+0000","last_peered":"2024-09-10T10:06:14.734715+0000","last_clean":"2024-09-10T10:06:14.734715+0000","last_became_active":"2024-09-10T10:06:13.735631+0000","last_became_peered":"2024-09-10T10:06:13.735631+0000","last_unstale":"2024-09-10T10:06:14.734715+0000","last_undegraded":"2024-09-10T10:06:14.734715+0000","last_fullsized":"2024-09-10T10:06:14.734715+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:31.507046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727514+0000","last_change":"2024-09-10T10:06:12.726846+0000","last_active":"2024-09-10T10:06:14.727514+0000","last_peered":"2024-09-10T10:06:14.727514+0000","last_clean":"2024-09-10T10:06:14.727514+0000","last_became_active":"2024-09-10T10:06:12.726284+0000","last_became_peered":"2024-09-10T10:06:12.726284+0000","last_unstale":"2024-09-10T10:06:14.727514+0000","last_undegraded":"2024-09-10T10:06:14.727514+0000","last_fullsized":"2024-09-10T10:06:14.727514+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:57.450534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730134+0000","last_change":"2024-09-10T10:06:13.743268+0000","last_active":"2024-09-10T10:06:14.730134+0000","last_peered":"2024-09-10T10:06:14.730134+0000","last_clean":"2024-09-10T10:06:14.730134+0000","last_became_active":"2024-09-10T10:06:13.742937+0000","last_became_peered":"2024-09-10T10:06:13.742937+0000","last_unstale":"2024-09-10T10:06:14.730134+0000","last_undegraded":"2024-09-10T10:06:14.730134+0000","last_fullsized":"2024-09-10T10:06:14.730134+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:24.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733360+0000","last_change":"2024-09-10T10:06:12.732634+0000","last_active":"2024-09-10T10:06:14.733360+0000","last_peered":"2024-09-10T10:06:14.733360+0000","last_clean":"2024-09-10T10:06:14.733360+0000","last_became_active":"2024-09-10T10:06:12.732205+0000","last_became_peered":"2024-09-10T10:06:12.732205+0000","last_unstale":"2024-09-10T10:06:14.733360+0000","last_undegraded":"2024-09-10T10:06:14.733360+0000","last_fullsized":"2024-09-10T10:06:14.733360+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:40.035147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731094+0000","last_change":"2024-09-10T10:06:13.748008+0000","last_active":"2024-09-10T10:06:14.731094+0000","last_peered":"2024-09-10T10:06:14.731094+0000","last_clean":"2024-09-10T10:06:14.731094+0000","last_became_active":"2024-09-10T10:06:13.747690+0000","last_became_peered":"2024-09-10T10:06:13.747690+0000","last_unstale":"2024-09-10T10:06:14.731094+0000","last_undegraded":"2024-09-10T10:06:14.731094+0000","last_fullsized":"2024-09-10T10:06:14.731094+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:52.811453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734622+0000","last_change":"2024-09-10T10:06:12.734762+0000","last_active":"2024-09-10T10:06:14.734622+0000","last_peered":"2024-09-10T10:06:14.734622+0000","last_clean":"2024-09-10T10:06:14.734622+0000","last_became_active":"2024-09-10T10:06:12.734501+0000","last_became_peered":"2024-09-10T10:06:12.734501+0000","last_unstale":"2024-09-10T10:06:14.734622+0000","last_undegraded":"2024-09-10T10:06:14.734622+0000","last_fullsized":"2024-09-10T10:06:14.734622+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:12.180453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727916+0000","last_change":"2024-09-10T10:06:13.742496+0000","last_active":"2024-09-10T10:06:14.727916+0000","last_peered":"2024-09-10T10:06:14.727916+0000","last_clean":"2024-09-10T10:06:14.727916+0000","last_became_active":"2024-09-10T10:06:13.742177+0000","last_became_peered":"2024-09-10T10:06:13.742177+0000","last_unstale":"2024-09-10T10:06:14.727916+0000","last_undegraded":"2024-09-10T10:06:14.727916+0000","last_fullsized":"2024-09-10T10:06:14.727916+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:26.996263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"165'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:15.725493+0000","last_change":"2024-09-10T10:06:12.730901+0000","last_active":"2024-09-10T10:06:15.725493+0000","last_peered":"2024-09-10T10:06:15.725493+0000","last_clean":"2024-09-10T10:06:15.725493+0000","last_became_active":"2024-09-10T10:06:12.730559+0000","last_became_peered":"2024-09-10T10:06:12.730559+0000","last_unstale":"2024-09-10T10:06:15.725493+0000","last_undegraded":"2024-09-10T10:06:15.725493+0000","last_fullsized":"2024-09-10T10:06:15.725493+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:19.774789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734980+0000","last_change":"2024-09-10T10:06:13.735913+0000","last_active":"2024-09-10T10:06:14.734980+0000","last_peered":"2024-09-10T10:06:14.734980+0000","last_clean":"2024-09-10T10:06:14.734980+0000","last_became_active":"2024-09-10T10:06:13.735642+0000","last_became_peered":"2024-09-10T10:06:13.735642+0000","last_unstale":"2024-09-10T10:06:14.734980+0000","last_undegraded":"2024-09-10T10:06:14.734980+0000","last_fullsized":"2024-09-10T10:06:14.734980+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:22.199408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733934+0000","last_change":"2024-09-10T10:06:12.735001+0000","last_active":"2024-09-10T10:06:14.733934+0000","last_peered":"2024-09-10T10:06:14.733934+0000","last_clean":"2024-09-10T10:06:14.733934+0000","last_became_active":"2024-09-10T10:06:12.734717+0000","last_became_peered":"2024-09-10T10:06:12.734717+0000","last_unstale":"2024-09-10T10:06:14.733934+0000","last_undegraded":"2024-09-10T10:06:14.733934+0000","last_fullsized":"2024-09-10T10:06:14.733934+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:55.935424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730477+0000","last_change":"2024-09-10T10:06:13.741394+0000","last_active":"2024-09-10T10:06:14.730477+0000","last_peered":"2024-09-10T10:06:14.730477+0000","last_clean":"2024-09-10T10:06:14.730477+0000","last_became_active":"2024-09-10T10:06:13.740773+0000","last_became_peered":"2024-09-10T10:06:13.740773+0000","last_unstale":"2024-09-10T10:06:14.730477+0000","last_undegraded":"2024-09-10T10:06:14.730477+0000","last_fullsized":"2024-09-10T10:06:14.730477+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:21.893885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734346+0000","last_change":"2024-09-10T10:06:12.735766+0000","last_active":"2024-09-10T10:06:14.734346+0000","last_peered":"2024-09-10T10:06:14.734346+0000","last_clean":"2024-09-10T10:06:14.734346+0000","last_became_active":"2024-09-10T10:06:12.735135+0000","last_became_peered":"2024-09-10T10:06:12.735135+0000","last_unstale":"2024-09-10T10:06:14.734346+0000","last_undegraded":"2024-09-10T10:06:14.734346+0000","last_fullsized":"2024-09-10T10:06:14.734346+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:32.219825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727472+0000","last_change":"2024-09-10T10:06:13.738401+0000","last_active":"2024-09-10T10:06:14.727472+0000","last_peered":"2024-09-10T10:06:14.727472+0000","last_clean":"2024-09-10T10:06:14.727472+0000","last_became_active":"2024-09-10T10:06:13.738135+0000","last_became_peered":"2024-09-10T10:06:13.738135+0000","last_unstale":"2024-09-10T10:06:14.727472+0000","last_undegraded":"2024-09-10T10:06:14.727472+0000","last_fullsized":"2024-09-10T10:06:14.727472+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:42.985695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735999+0000","last_change":"2024-09-10T10:06:12.740866+0000","last_active":"2024-09-10T10:06:14.735999+0000","last_peered":"2024-09-10T10:06:14.735999+0000","last_clean":"2024-09-10T10:06:14.735999+0000","last_became_active":"2024-09-10T10:06:12.740591+0000","last_became_peered":"2024-09-10T10:06:12.740591+0000","last_unstale":"2024-09-10T10:06:14.735999+0000","last_undegraded":"2024-09-10T10:06:14.735999+0000","last_fullsized":"2024-09-10T10:06:14.735999+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:32.316892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730894+0000","last_change":"2024-09-10T10:06:13.737237+0000","last_active":"2024-09-10T10:06:14.730894+0000","last_peered":"2024-09-10T10:06:14.730894+0000","last_clean":"2024-09-10T10:06:14.730894+0000","last_became_active":"2024-09-10T10:06:13.736815+0000","last_became_peered":"2024-09-10T10:06:13.736815+0000","last_unstale":"2024-09-10T10:06:14.730894+0000","last_undegraded":"2024-09-10T10:06:14.730894+0000","last_fullsized":"2024-09-10T10:06:14.730894+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:18.783891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733912+0000","last_change":"2024-09-10T10:06:12.725002+0000","last_active":"2024-09-10T10:06:14.733912+0000","last_peered":"2024-09-10T10:06:14.733912+0000","last_clean":"2024-09-10T10:06:14.733912+0000","last_became_active":"2024-09-10T10:06:12.724743+0000","last_became_peered":"2024-09-10T10:06:12.724743+0000","last_unstale":"2024-09-10T10:06:14.733912+0000","last_undegraded":"2024-09-10T10:06:14.733912+0000","last_fullsized":"2024-09-10T10:06:14.733912+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.410825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732582+0000","last_change":"2024-09-10T10:06:13.745814+0000","last_active":"2024-09-10T10:06:14.732582+0000","last_peered":"2024-09-10T10:06:14.732582+0000","last_clean":"2024-09-10T10:06:14.732582+0000","last_became_active":"2024-09-10T10:06:13.745233+0000","last_became_peered":"2024-09-10T10:06:13.745233+0000","last_unstale":"2024-09-10T10:06:14.732582+0000","last_undegraded":"2024-09-10T10:06:14.732582+0000","last_fullsized":"2024-09-10T10:06:14.732582+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:39.830795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730253+0000","last_change":"2024-09-10T10:06:12.727463+0000","last_active":"2024-09-10T10:06:14.730253+0000","last_peered":"2024-09-10T10:06:14.730253+0000","last_clean":"2024-09-10T10:06:14.730253+0000","last_became_active":"2024-09-10T10:06:12.727028+0000","last_became_peered":"2024-09-10T10:06:12.727028+0000","last_unstale":"2024-09-10T10:06:14.730253+0000","last_undegraded":"2024-09-10T10:06:14.730253+0000","last_fullsized":"2024-09-10T10:06:14.730253+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.410825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734745+0000","last_change":"2024-09-10T10:06:13.732978+0000","last_active":"2024-09-10T10:06:14.734745+0000","last_peered":"2024-09-10T10:06:14.734745+0000","last_clean":"2024-09-10T10:06:14.734745+0000","last_became_active":"2024-09-10T10:06:13.732689+0000","last_became_peered":"2024-09-10T10:06:13.732689+0000","last_unstale":"2024-09-10T10:06:14.734745+0000","last_undegraded":"2024-09-10T10:06:14.734745+0000","last_fullsized":"2024-09-10T10:06:14.734745+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:44.911704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730853+0000","last_change":"2024-09-10T10:06:12.736869+0000","last_active":"2024-09-10T10:06:14.730853+0000","last_peered":"2024-09-10T10:06:14.730853+0000","last_clean":"2024-09-10T10:06:14.730853+0000","last_became_active":"2024-09-10T10:06:12.736569+0000","last_became_peered":"2024-09-10T10:06:12.736569+0000","last_unstale":"2024-09-10T10:06:14.730853+0000","last_undegraded":"2024-09-10T10:06:14.730853+0000","last_fullsized":"2024-09-10T10:06:14.730853+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:00.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727445+0000","last_change":"2024-09-10T10:06:13.747623+0000","last_active":"2024-09-10T10:06:14.727445+0000","last_peered":"2024-09-10T10:06:14.727445+0000","last_clean":"2024-09-10T10:06:14.727445+0000","last_became_active":"2024-09-10T10:06:13.747320+0000","last_became_peered":"2024-09-10T10:06:13.747320+0000","last_unstale":"2024-09-10T10:06:14.727445+0000","last_undegraded":"2024-09-10T10:06:14.727445+0000","last_fullsized":"2024-09-10T10:06:14.727445+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.137086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731239+0000","last_change":"2024-09-10T10:06:12.728139+0000","last_active":"2024-09-10T10:06:14.731239+0000","last_peered":"2024-09-10T10:06:14.731239+0000","last_clean":"2024-09-10T10:06:14.731239+0000","last_became_active":"2024-09-10T10:06:12.727855+0000","last_became_peered":"2024-09-10T10:06:12.727855+0000","last_unstale":"2024-09-10T10:06:14.731239+0000","last_undegraded":"2024-09-10T10:06:14.731239+0000","last_fullsized":"2024-09-10T10:06:14.731239+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:31.989079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":471,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735353+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:06:14.735353+0000","last_peered":"2024-09-10T10:06:14.735353+0000","last_clean":"2024-09-10T10:06:14.735353+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:06:14.735353+0000","last_undegraded":"2024-09-10T10:06:14.735353+0000","last_fullsized":"2024-09-10T10:06:14.735353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33808,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338032,"statfs":{"total":96636764160,"available":96602144768,"internally_reserved":0,"allocated":2682880,"data_stored":1509090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2080768,"data_stored":917692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2080768,"data_stored":917692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2097152,"data_stored":918826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542245,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2088960,"data_stored":917292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2076672,"data_stored":917110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":917166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":2612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":2674688,"data_stored":1507568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:06:17.812 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:06:17.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.985+0000 7f8119e00640 1 Processor -- start 2024-09-10T10:06:17.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.985+0000 7f8119e00640 1 -- start start 2024-09-10T10:06:17.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.985+0000 7f8119e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8114155970 0x7f81140cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.985+0000 7f8119e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f811406f1e0 0x7f81140ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.985+0000 7f8119e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8114155d50 con 0x7f811415c560 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.985+0000 7f8119e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f81140b4570 con 0x7f811406f1e0 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.985+0000 7f8119e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f81140b34d0 con 0x7f8114155970 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8113400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f811406f1e0 0x7f81140ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8113400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f811406f1e0 0x7f81140ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47184/0 (socket says 172.21.15.110:47184) 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8113400640 1 -- 172.21.15.110:0/348034944 learned_addr learned my addr 172.21.15.110:0/348034944 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f810a000640 1 --1- 172.21.15.110:0/348034944 >> v1:172.21.15.110:6789/0 conn(0x7f811415c560 0x7f811406ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37306/0 (socket says 172.21.15.110:37306) 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8112a00640 1 --2- 172.21.15.110:0/348034944 >> v2:172.21.15.145:3300/0 conn(0x7f8114155970 0x7f81140cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8113400640 1 -- 172.21.15.110:0/348034944 >> v2:172.21.15.145:3300/0 conn(0x7f8114155970 msgr2=0x7f81140cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8113400640 1 --2- 172.21.15.110:0/348034944 >> v2:172.21.15.145:3300/0 conn(0x7f8114155970 0x7f81140cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8113400640 1 -- 172.21.15.110:0/348034944 >> v1:172.21.15.110:6789/0 conn(0x7f811415c560 legacy=0x7f811406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8113400640 1 -- 172.21.15.110:0/348034944 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81140b37b0 con 0x7f811406f1e0 2024-09-10T10:06:17.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8113400640 1 --2- 172.21.15.110:0/348034944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f811406f1e0 0x7f81140ce420 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f811406cc30 tx=0x7f810c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730ff5be6d5f42d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:17.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8112000640 1 -- 172.21.15.110:0/348034944 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0a5880 con 0x7f811406f1e0 2024-09-10T10:06:17.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8112000640 1 -- 172.21.15.110:0/348034944 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c0a5a20 con 0x7f811406f1e0 2024-09-10T10:06:17.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.989+0000 7f8112000640 1 -- 172.21.15.110:0/348034944 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0a5df0 con 0x7f811406f1e0 2024-09-10T10:06:17.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- 172.21.15.110:0/348034944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f811406f1e0 msgr2=0x7f81140ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 --2- 172.21.15.110:0/348034944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f811406f1e0 0x7f81140ce420 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f811406cc30 tx=0x7f810c01c9c0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- 172.21.15.110:0/348034944 shutdown_connections 2024-09-10T10:06:17.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 --2- 172.21.15.110:0/348034944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f811406f1e0 0x7f81140ce420 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 --2- 172.21.15.110:0/348034944 >> v2:172.21.15.145:3300/0 conn(0x7f8114155970 0x7f81140cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- 172.21.15.110:0/348034944 >> 172.21.15.110:0/348034944 conn(0x7f811406bc50 msgr2=0x7f811406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:17.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- 172.21.15.110:0/348034944 shutdown_connections 2024-09-10T10:06:17.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- 172.21.15.110:0/348034944 wait complete. 2024-09-10T10:06:17.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 Processor -- start 2024-09-10T10:06:17.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- start start 2024-09-10T10:06:17.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f811406f1e0 0x7f8114160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8114155970 0x7f8114157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f811415c560 0x7f8114144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81140cf170 con 0x7f8114155970 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81140cce00 con 0x7f811415c560 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8119e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f81140cf2f0 con 0x7f811406f1e0 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8114155970 0x7f8114157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8114155970 0x7f8114157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47186/0 (socket says 172.21.15.110:47186) 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 -- 172.21.15.110:0/1518299052 learned_addr learned my addr 172.21.15.110:0/1518299052 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8112a00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f811406f1e0 0x7f8114160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8113400640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f811415c560 0x7f8114144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 -- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f811406f1e0 msgr2=0x7f8114160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f811406f1e0 0x7f8114160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 -- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f811415c560 msgr2=0x7f8114144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:17.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f811415c560 0x7f8114144840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:17.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 -- 172.21.15.110:0/1518299052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8114158ea0 con 0x7f8114155970 2024-09-10T10:06:17.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f8113400640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f811415c560 0x7f8114144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:17.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810a000640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8114155970 0x7f8114157130 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00c370 tx=0x7f80fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:17.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.993+0000 7f810be00640 1 -- 172.21.15.110:0/1518299052 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0074e0 con 0x7f8114155970 2024-09-10T10:06:17.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f810be00640 1 -- 172.21.15.110:0/1518299052 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80fc007680 con 0x7f8114155970 2024-09-10T10:06:17.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f810be00640 1 -- 172.21.15.110:0/1518299052 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc004a30 con 0x7f8114155970 2024-09-10T10:06:17.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8114159180 con 0x7f8114155970 2024-09-10T10:06:17.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8114159740 con 0x7f8114155970 2024-09-10T10:06:17.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8114069bb0 con 0x7f8114155970 2024-09-10T10:06:17.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f810be00640 1 -- 172.21.15.110:0/1518299052 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f80fc01a020 con 0x7f8114155970 2024-09-10T10:06:17.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f810be00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f80f0068250 0x7f80f006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:17.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f810be00640 1 -- 172.21.15.110:0/1518299052 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f80fc083ba0 con 0x7f8114155970 2024-09-10T10:06:17.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f8112a00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f80f0068250 0x7f80f006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:17.997+0000 7f8112a00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f80f0068250 0x7f80f006a710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8104009af0 tx=0x7f8104008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:18.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.001+0000 7f810be00640 1 -- 172.21.15.110:0/1518299052 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f80fc0562e0 con 0x7f8114155970 2024-09-10T10:06:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.153+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81140cf680 con 0x7f80f0068250 2024-09-10T10:06:18.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.161+0000 7f810be00640 1 -- 172.21.15.110:0/1518299052 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252488 (secure 0 0 0) 0x7f81140cf680 con 0x7f80f0068250 2024-09-10T10:06:18.165 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:18.166 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:06:18.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f80f0068250 msgr2=0x7f80f006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f80f0068250 0x7f80f006a710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8104009af0 tx=0x7f8104008040 comp rx=0 tx=0).stop 2024-09-10T10:06:18.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8114155970 msgr2=0x7f8114157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8114155970 0x7f8114157130 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00c370 tx=0x7f80fc00c830 comp rx=0 tx=0).stop 2024-09-10T10:06:18.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 shutdown_connections 2024-09-10T10:06:18.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f80f0068250 0x7f80f006a710 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f811415c560 0x7f8114144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8114155970 0x7f8114157130 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 --2- 172.21.15.110:0/1518299052 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f811406f1e0 0x7f8114160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 >> 172.21.15.110:0/1518299052 conn(0x7f811406bc50 msgr2=0x7f81140cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:18.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 shutdown_connections 2024-09-10T10:06:18.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.165+0000 7f8119e00640 1 -- 172.21.15.110:0/1518299052 wait complete. 2024-09-10T10:06:18.185 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-10T10:06:17.751392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283624,"kb_used_data":17448,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691096,"statfs":{"total":773094113280,"available":772803682304,"internally_reserved":0,"allocated":17866752,"data_stored":8522436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001365"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727742+0000","last_change":"2024-09-10T10:06:13.734465+0000","last_active":"2024-09-10T10:06:14.727742+0000","last_peered":"2024-09-10T10:06:14.727742+0000","last_clean":"2024-09-10T10:06:14.727742+0000","last_became_active":"2024-09-10T10:06:13.733857+0000","last_became_peered":"2024-09-10T10:06:13.733857+0000","last_unstale":"2024-09-10T10:06:14.727742+0000","last_undegraded":"2024-09-10T10:06:14.727742+0000","last_fullsized":"2024-09-10T10:06:14.727742+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:39.733024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734854+0000","last_change":"2024-09-10T10:06:13.741950+0000","last_active":"2024-09-10T10:06:14.734854+0000","last_peered":"2024-09-10T10:06:14.734854+0000","last_clean":"2024-09-10T10:06:14.734854+0000","last_became_active":"2024-09-10T10:06:13.741687+0000","last_became_peered":"2024-09-10T10:06:13.741687+0000","last_unstale":"2024-09-10T10:06:14.734854+0000","last_undegraded":"2024-09-10T10:06:14.734854+0000","last_fullsized":"2024-09-10T10:06:14.734854+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:06.371322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734819+0000","last_change":"2024-09-10T10:06:13.732325+0000","last_active":"2024-09-10T10:06:14.734819+0000","last_peered":"2024-09-10T10:06:14.734819+0000","last_clean":"2024-09-10T10:06:14.734819+0000","last_became_active":"2024-09-10T10:06:13.732030+0000","last_became_peered":"2024-09-10T10:06:13.732030+0000","last_unstale":"2024-09-10T10:06:14.734819+0000","last_undegraded":"2024-09-10T10:06:14.734819+0000","last_fullsized":"2024-09-10T10:06:14.734819+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:22.199408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735806+0000","last_change":"2024-09-10T10:06:13.746925+0000","last_active":"2024-09-10T10:06:14.735806+0000","last_peered":"2024-09-10T10:06:14.735806+0000","last_clean":"2024-09-10T10:06:14.735806+0000","last_became_active":"2024-09-10T10:06:13.746720+0000","last_became_peered":"2024-09-10T10:06:13.746720+0000","last_unstale":"2024-09-10T10:06:14.735806+0000","last_undegraded":"2024-09-10T10:06:14.735806+0000","last_fullsized":"2024-09-10T10:06:14.735806+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:36.904490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733251+0000","last_change":"2024-09-10T10:06:13.745046+0000","last_active":"2024-09-10T10:06:14.733251+0000","last_peered":"2024-09-10T10:06:14.733251+0000","last_clean":"2024-09-10T10:06:14.733251+0000","last_became_active":"2024-09-10T10:06:13.744405+0000","last_became_peered":"2024-09-10T10:06:13.744405+0000","last_unstale":"2024-09-10T10:06:14.733251+0000","last_undegraded":"2024-09-10T10:06:14.733251+0000","last_fullsized":"2024-09-10T10:06:14.733251+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:53.511879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733545+0000","last_change":"2024-09-10T10:06:13.731004+0000","last_active":"2024-09-10T10:06:14.733545+0000","last_peered":"2024-09-10T10:06:14.733545+0000","last_clean":"2024-09-10T10:06:14.733545+0000","last_became_active":"2024-09-10T10:06:13.730745+0000","last_became_peered":"2024-09-10T10:06:13.730745+0000","last_unstale":"2024-09-10T10:06:14.733545+0000","last_undegraded":"2024-09-10T10:06:14.733545+0000","last_fullsized":"2024-09-10T10:06:14.733545+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:24.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727612+0000","last_change":"2024-09-10T10:06:13.739547+0000","last_active":"2024-09-10T10:06:14.727612+0000","last_peered":"2024-09-10T10:06:14.727612+0000","last_clean":"2024-09-10T10:06:14.727612+0000","last_became_active":"2024-09-10T10:06:13.737781+0000","last_became_peered":"2024-09-10T10:06:13.737781+0000","last_unstale":"2024-09-10T10:06:14.727612+0000","last_undegraded":"2024-09-10T10:06:14.727612+0000","last_fullsized":"2024-09-10T10:06:14.727612+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:55.245814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733108+0000","last_change":"2024-09-10T10:06:13.731689+0000","last_active":"2024-09-10T10:06:14.733108+0000","last_peered":"2024-09-10T10:06:14.733108+0000","last_clean":"2024-09-10T10:06:14.733108+0000","last_became_active":"2024-09-10T10:06:13.731420+0000","last_became_peered":"2024-09-10T10:06:13.731420+0000","last_unstale":"2024-09-10T10:06:14.733108+0000","last_undegraded":"2024-09-10T10:06:14.733108+0000","last_fullsized":"2024-09-10T10:06:14.733108+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:00.681262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735233+0000","last_change":"2024-09-10T10:06:13.733998+0000","last_active":"2024-09-10T10:06:14.735233+0000","last_peered":"2024-09-10T10:06:14.735233+0000","last_clean":"2024-09-10T10:06:14.735233+0000","last_became_active":"2024-09-10T10:06:13.733678+0000","last_became_peered":"2024-09-10T10:06:13.733678+0000","last_unstale":"2024-09-10T10:06:14.735233+0000","last_undegraded":"2024-09-10T10:06:14.735233+0000","last_fullsized":"2024-09-10T10:06:14.735233+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:47.413666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733603+0000","last_change":"2024-09-10T10:06:13.743834+0000","last_active":"2024-09-10T10:06:14.733603+0000","last_peered":"2024-09-10T10:06:14.733603+0000","last_clean":"2024-09-10T10:06:14.733603+0000","last_became_active":"2024-09-10T10:06:13.743561+0000","last_became_peered":"2024-09-10T10:06:13.743561+0000","last_unstale":"2024-09-10T10:06:14.733603+0000","last_undegraded":"2024-09-10T10:06:14.733603+0000","last_fullsized":"2024-09-10T10:06:14.733603+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:33.533674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734851+0000","last_change":"2024-09-10T10:06:13.747512+0000","last_active":"2024-09-10T10:06:14.734851+0000","last_peered":"2024-09-10T10:06:14.734851+0000","last_clean":"2024-09-10T10:06:14.734851+0000","last_became_active":"2024-09-10T10:06:13.746425+0000","last_became_peered":"2024-09-10T10:06:13.746425+0000","last_unstale":"2024-09-10T10:06:14.734851+0000","last_undegraded":"2024-09-10T10:06:14.734851+0000","last_fullsized":"2024-09-10T10:06:14.734851+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:49.846210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730561+0000","last_change":"2024-09-10T10:06:13.731888+0000","last_active":"2024-09-10T10:06:14.730561+0000","last_peered":"2024-09-10T10:06:14.730561+0000","last_clean":"2024-09-10T10:06:14.730561+0000","last_became_active":"2024-09-10T10:06:13.731616+0000","last_became_peered":"2024-09-10T10:06:13.731616+0000","last_unstale":"2024-09-10T10:06:14.730561+0000","last_undegraded":"2024-09-10T10:06:14.730561+0000","last_fullsized":"2024-09-10T10:06:14.730561+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:21.893885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731114+0000","last_change":"2024-09-10T10:06:13.730415+0000","last_active":"2024-09-10T10:06:14.731114+0000","last_peered":"2024-09-10T10:06:14.731114+0000","last_clean":"2024-09-10T10:06:14.731114+0000","last_became_active":"2024-09-10T10:06:13.730136+0000","last_became_peered":"2024-09-10T10:06:13.730136+0000","last_unstale":"2024-09-10T10:06:14.731114+0000","last_undegraded":"2024-09-10T10:06:14.731114+0000","last_fullsized":"2024-09-10T10:06:14.731114+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:00:22.090794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727334+0000","last_change":"2024-09-10T10:06:13.739028+0000","last_active":"2024-09-10T10:06:14.727334+0000","last_peered":"2024-09-10T10:06:14.727334+0000","last_clean":"2024-09-10T10:06:14.727334+0000","last_became_active":"2024-09-10T10:06:13.738713+0000","last_became_peered":"2024-09-10T10:06:13.738713+0000","last_unstale":"2024-09-10T10:06:14.727334+0000","last_undegraded":"2024-09-10T10:06:14.727334+0000","last_fullsized":"2024-09-10T10:06:14.727334+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:28.990348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727266+0000","last_change":"2024-09-10T10:06:13.747898+0000","last_active":"2024-09-10T10:06:14.727266+0000","last_peered":"2024-09-10T10:06:14.727266+0000","last_clean":"2024-09-10T10:06:14.727266+0000","last_became_active":"2024-09-10T10:06:13.747563+0000","last_became_peered":"2024-09-10T10:06:13.747563+0000","last_unstale":"2024-09-10T10:06:14.727266+0000","last_undegraded":"2024-09-10T10:06:14.727266+0000","last_fullsized":"2024-09-10T10:06:14.727266+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:24.958103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732359+0000","last_change":"2024-09-10T10:06:13.744389+0000","last_active":"2024-09-10T10:06:14.732359+0000","last_peered":"2024-09-10T10:06:14.732359+0000","last_clean":"2024-09-10T10:06:14.732359+0000","last_became_active":"2024-09-10T10:06:13.744101+0000","last_became_peered":"2024-09-10T10:06:13.744101+0000","last_unstale":"2024-09-10T10:06:14.732359+0000","last_undegraded":"2024-09-10T10:06:14.732359+0000","last_fullsized":"2024-09-10T10:06:14.732359+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:49.444482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727904+0000","last_change":"2024-09-10T10:06:13.733778+0000","last_active":"2024-09-10T10:06:14.727904+0000","last_peered":"2024-09-10T10:06:14.727904+0000","last_clean":"2024-09-10T10:06:14.727904+0000","last_became_active":"2024-09-10T10:06:13.733516+0000","last_became_peered":"2024-09-10T10:06:13.733516+0000","last_unstale":"2024-09-10T10:06:14.727904+0000","last_undegraded":"2024-09-10T10:06:14.727904+0000","last_fullsized":"2024-09-10T10:06:14.727904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:01.857442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733737+0000","last_change":"2024-09-10T10:06:13.733795+0000","last_active":"2024-09-10T10:06:14.733737+0000","last_peered":"2024-09-10T10:06:14.733737+0000","last_clean":"2024-09-10T10:06:14.733737+0000","last_became_active":"2024-09-10T10:06:13.733506+0000","last_became_peered":"2024-09-10T10:06:13.733506+0000","last_unstale":"2024-09-10T10:06:14.733737+0000","last_undegraded":"2024-09-10T10:06:14.733737+0000","last_fullsized":"2024-09-10T10:06:14.733737+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:30.381657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733040+0000","last_change":"2024-09-10T10:06:13.745764+0000","last_active":"2024-09-10T10:06:14.733040+0000","last_peered":"2024-09-10T10:06:14.733040+0000","last_clean":"2024-09-10T10:06:14.733040+0000","last_became_active":"2024-09-10T10:06:13.744136+0000","last_became_peered":"2024-09-10T10:06:13.744136+0000","last_unstale":"2024-09-10T10:06:14.733040+0000","last_undegraded":"2024-09-10T10:06:14.733040+0000","last_fullsized":"2024-09-10T10:06:14.733040+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:46.932364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733447+0000","last_change":"2024-09-10T10:06:13.745345+0000","last_active":"2024-09-10T10:06:14.733447+0000","last_peered":"2024-09-10T10:06:14.733447+0000","last_clean":"2024-09-10T10:06:14.733447+0000","last_became_active":"2024-09-10T10:06:13.745067+0000","last_became_peered":"2024-09-10T10:06:13.745067+0000","last_unstale":"2024-09-10T10:06:14.733447+0000","last_undegraded":"2024-09-10T10:06:14.733447+0000","last_fullsized":"2024-09-10T10:06:14.733447+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:01.161851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735876+0000","last_change":"2024-09-10T10:06:13.744740+0000","last_active":"2024-09-10T10:06:14.735876+0000","last_peered":"2024-09-10T10:06:14.735876+0000","last_clean":"2024-09-10T10:06:14.735876+0000","last_became_active":"2024-09-10T10:06:13.744401+0000","last_became_peered":"2024-09-10T10:06:13.744401+0000","last_unstale":"2024-09-10T10:06:14.735876+0000","last_undegraded":"2024-09-10T10:06:14.735876+0000","last_fullsized":"2024-09-10T10:06:14.735876+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:45.035476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.729902+0000","last_change":"2024-09-10T10:06:13.750736+0000","last_active":"2024-09-10T10:06:14.729902+0000","last_peered":"2024-09-10T10:06:14.729902+0000","last_clean":"2024-09-10T10:06:14.729902+0000","last_became_active":"2024-09-10T10:06:13.750272+0000","last_became_peered":"2024-09-10T10:06:13.750272+0000","last_unstale":"2024-09-10T10:06:14.729902+0000","last_undegraded":"2024-09-10T10:06:14.729902+0000","last_fullsized":"2024-09-10T10:06:14.729902+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:53.511879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730061+0000","last_change":"2024-09-10T10:06:13.742519+0000","last_active":"2024-09-10T10:06:14.730061+0000","last_peered":"2024-09-10T10:06:14.730061+0000","last_clean":"2024-09-10T10:06:14.730061+0000","last_became_active":"2024-09-10T10:06:13.740224+0000","last_became_peered":"2024-09-10T10:06:13.740224+0000","last_unstale":"2024-09-10T10:06:14.730061+0000","last_undegraded":"2024-09-10T10:06:14.730061+0000","last_fullsized":"2024-09-10T10:06:14.730061+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:55.226969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735500+0000","last_change":"2024-09-10T10:06:13.746333+0000","last_active":"2024-09-10T10:06:14.735500+0000","last_peered":"2024-09-10T10:06:14.735500+0000","last_clean":"2024-09-10T10:06:14.735500+0000","last_became_active":"2024-09-10T10:06:13.745638+0000","last_became_peered":"2024-09-10T10:06:13.745638+0000","last_unstale":"2024-09-10T10:06:14.735500+0000","last_undegraded":"2024-09-10T10:06:14.735500+0000","last_fullsized":"2024-09-10T10:06:14.735500+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:05.945907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735014+0000","last_change":"2024-09-10T10:06:13.748004+0000","last_active":"2024-09-10T10:06:14.735014+0000","last_peered":"2024-09-10T10:06:14.735014+0000","last_clean":"2024-09-10T10:06:14.735014+0000","last_became_active":"2024-09-10T10:06:13.747687+0000","last_became_peered":"2024-09-10T10:06:13.747687+0000","last_unstale":"2024-09-10T10:06:14.735014+0000","last_undegraded":"2024-09-10T10:06:14.735014+0000","last_fullsized":"2024-09-10T10:06:14.735014+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.316000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727697+0000","last_change":"2024-09-10T10:06:13.741284+0000","last_active":"2024-09-10T10:06:14.727697+0000","last_peered":"2024-09-10T10:06:14.727697+0000","last_clean":"2024-09-10T10:06:14.727697+0000","last_became_active":"2024-09-10T10:06:13.740588+0000","last_became_peered":"2024-09-10T10:06:13.740588+0000","last_unstale":"2024-09-10T10:06:14.727697+0000","last_undegraded":"2024-09-10T10:06:14.727697+0000","last_fullsized":"2024-09-10T10:06:14.727697+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:13:21.426914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731411+0000","last_change":"2024-09-10T10:06:13.734915+0000","last_active":"2024-09-10T10:06:14.731411+0000","last_peered":"2024-09-10T10:06:14.731411+0000","last_clean":"2024-09-10T10:06:14.731411+0000","last_became_active":"2024-09-10T10:06:13.734595+0000","last_became_peered":"2024-09-10T10:06:13.734595+0000","last_unstale":"2024-09-10T10:06:14.731411+0000","last_undegraded":"2024-09-10T10:06:14.731411+0000","last_fullsized":"2024-09-10T10:06:14.731411+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:51.143945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732720+0000","last_change":"2024-09-10T10:06:13.736977+0000","last_active":"2024-09-10T10:06:14.732720+0000","last_peered":"2024-09-10T10:06:14.732720+0000","last_clean":"2024-09-10T10:06:14.732720+0000","last_became_active":"2024-09-10T10:06:13.736612+0000","last_became_peered":"2024-09-10T10:06:13.736612+0000","last_unstale":"2024-09-10T10:06:14.732720+0000","last_undegraded":"2024-09-10T10:06:14.732720+0000","last_fullsized":"2024-09-10T10:06:14.732720+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:44.911704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730687+0000","last_change":"2024-09-10T10:06:13.732529+0000","last_active":"2024-09-10T10:06:14.730687+0000","last_peered":"2024-09-10T10:06:14.730687+0000","last_clean":"2024-09-10T10:06:14.730687+0000","last_became_active":"2024-09-10T10:06:13.732257+0000","last_became_peered":"2024-09-10T10:06:13.732257+0000","last_unstale":"2024-09-10T10:06:14.730687+0000","last_undegraded":"2024-09-10T10:06:14.730687+0000","last_fullsized":"2024-09-10T10:06:14.730687+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:28.747834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734859+0000","last_change":"2024-09-10T10:06:13.748754+0000","last_active":"2024-09-10T10:06:14.734859+0000","last_peered":"2024-09-10T10:06:14.734859+0000","last_clean":"2024-09-10T10:06:14.734859+0000","last_became_active":"2024-09-10T10:06:13.748364+0000","last_became_peered":"2024-09-10T10:06:13.748364+0000","last_unstale":"2024-09-10T10:06:14.734859+0000","last_undegraded":"2024-09-10T10:06:14.734859+0000","last_fullsized":"2024-09-10T10:06:14.734859+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:41.757207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732396+0000","last_change":"2024-09-10T10:06:13.735742+0000","last_active":"2024-09-10T10:06:14.732396+0000","last_peered":"2024-09-10T10:06:14.732396+0000","last_clean":"2024-09-10T10:06:14.732396+0000","last_became_active":"2024-09-10T10:06:13.735403+0000","last_became_peered":"2024-09-10T10:06:13.735403+0000","last_unstale":"2024-09-10T10:06:14.732396+0000","last_undegraded":"2024-09-10T10:06:14.732396+0000","last_fullsized":"2024-09-10T10:06:14.732396+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:52.996895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727325+0000","last_change":"2024-09-10T10:06:13.737537+0000","last_active":"2024-09-10T10:06:14.727325+0000","last_peered":"2024-09-10T10:06:14.727325+0000","last_clean":"2024-09-10T10:06:14.727325+0000","last_became_active":"2024-09-10T10:06:13.737227+0000","last_became_peered":"2024-09-10T10:06:13.737227+0000","last_unstale":"2024-09-10T10:06:14.727325+0000","last_undegraded":"2024-09-10T10:06:14.727325+0000","last_fullsized":"2024-09-10T10:06:14.727325+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:05.945907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735009+0000","last_change":"2024-09-10T10:06:13.732228+0000","last_active":"2024-09-10T10:06:14.735009+0000","last_peered":"2024-09-10T10:06:14.735009+0000","last_clean":"2024-09-10T10:06:14.735009+0000","last_became_active":"2024-09-10T10:06:13.731884+0000","last_became_peered":"2024-09-10T10:06:13.731884+0000","last_unstale":"2024-09-10T10:06:14.735009+0000","last_undegraded":"2024-09-10T10:06:14.735009+0000","last_fullsized":"2024-09-10T10:06:14.735009+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:49.846210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"164'4","reported_seq":18,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731207+0000","last_change":"2024-09-10T10:06:12.735303+0000","last_active":"2024-09-10T10:06:14.731207+0000","last_peered":"2024-09-10T10:06:14.731207+0000","last_clean":"2024-09-10T10:06:14.731207+0000","last_became_active":"2024-09-10T10:06:12.735001+0000","last_became_peered":"2024-09-10T10:06:12.735001+0000","last_unstale":"2024-09-10T10:06:14.731207+0000","last_undegraded":"2024-09-10T10:06:14.731207+0000","last_fullsized":"2024-09-10T10:06:14.731207+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:54.658689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730083+0000","last_change":"2024-09-10T10:06:13.746703+0000","last_active":"2024-09-10T10:06:14.730083+0000","last_peered":"2024-09-10T10:06:14.730083+0000","last_clean":"2024-09-10T10:06:14.730083+0000","last_became_active":"2024-09-10T10:06:13.746294+0000","last_became_peered":"2024-09-10T10:06:13.746294+0000","last_unstale":"2024-09-10T10:06:14.730083+0000","last_undegraded":"2024-09-10T10:06:14.730083+0000","last_fullsized":"2024-09-10T10:06:14.730083+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:01.161851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735202+0000","last_change":"2024-09-10T10:06:12.735541+0000","last_active":"2024-09-10T10:06:14.735202+0000","last_peered":"2024-09-10T10:06:14.735202+0000","last_clean":"2024-09-10T10:06:14.735202+0000","last_became_active":"2024-09-10T10:06:12.735242+0000","last_became_peered":"2024-09-10T10:06:12.735242+0000","last_unstale":"2024-09-10T10:06:14.735202+0000","last_undegraded":"2024-09-10T10:06:14.735202+0000","last_fullsized":"2024-09-10T10:06:14.735202+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:32.219825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727475+0000","last_change":"2024-09-10T10:06:13.738536+0000","last_active":"2024-09-10T10:06:14.727475+0000","last_peered":"2024-09-10T10:06:14.727475+0000","last_clean":"2024-09-10T10:06:14.727475+0000","last_became_active":"2024-09-10T10:06:13.738251+0000","last_became_peered":"2024-09-10T10:06:13.738251+0000","last_unstale":"2024-09-10T10:06:14.727475+0000","last_undegraded":"2024-09-10T10:06:14.727475+0000","last_fullsized":"2024-09-10T10:06:14.727475+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:45.035476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734941+0000","last_change":"2024-09-10T10:06:12.734080+0000","last_active":"2024-09-10T10:06:14.734941+0000","last_peered":"2024-09-10T10:06:14.734941+0000","last_clean":"2024-09-10T10:06:14.734941+0000","last_became_active":"2024-09-10T10:06:12.733770+0000","last_became_peered":"2024-09-10T10:06:12.733770+0000","last_unstale":"2024-09-10T10:06:14.734941+0000","last_undegraded":"2024-09-10T10:06:14.734941+0000","last_fullsized":"2024-09-10T10:06:14.734941+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:53.099507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732771+0000","last_change":"2024-09-10T10:06:13.745616+0000","last_active":"2024-09-10T10:06:14.732771+0000","last_peered":"2024-09-10T10:06:14.732771+0000","last_clean":"2024-09-10T10:06:14.732771+0000","last_became_active":"2024-09-10T10:06:13.743958+0000","last_became_peered":"2024-09-10T10:06:13.743958+0000","last_unstale":"2024-09-10T10:06:14.732771+0000","last_undegraded":"2024-09-10T10:06:14.732771+0000","last_fullsized":"2024-09-10T10:06:14.732771+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:22:26.369523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735924+0000","last_change":"2024-09-10T10:06:12.736958+0000","last_active":"2024-09-10T10:06:14.735924+0000","last_peered":"2024-09-10T10:06:14.735924+0000","last_clean":"2024-09-10T10:06:14.735924+0000","last_became_active":"2024-09-10T10:06:12.736631+0000","last_became_peered":"2024-09-10T10:06:12.736631+0000","last_unstale":"2024-09-10T10:06:14.735924+0000","last_undegraded":"2024-09-10T10:06:14.735924+0000","last_fullsized":"2024-09-10T10:06:14.735924+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:32.460556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735859+0000","last_change":"2024-09-10T10:06:13.746857+0000","last_active":"2024-09-10T10:06:14.735859+0000","last_peered":"2024-09-10T10:06:14.735859+0000","last_clean":"2024-09-10T10:06:14.735859+0000","last_became_active":"2024-09-10T10:06:13.746525+0000","last_became_peered":"2024-09-10T10:06:13.746525+0000","last_unstale":"2024-09-10T10:06:14.735859+0000","last_undegraded":"2024-09-10T10:06:14.735859+0000","last_fullsized":"2024-09-10T10:06:14.735859+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.137086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733773+0000","last_change":"2024-09-10T10:06:12.730590+0000","last_active":"2024-09-10T10:06:14.733773+0000","last_peered":"2024-09-10T10:06:14.733773+0000","last_clean":"2024-09-10T10:06:14.733773+0000","last_became_active":"2024-09-10T10:06:12.730331+0000","last_became_peered":"2024-09-10T10:06:12.730331+0000","last_unstale":"2024-09-10T10:06:14.733773+0000","last_undegraded":"2024-09-10T10:06:14.733773+0000","last_fullsized":"2024-09-10T10:06:14.733773+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:28.390353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733253+0000","last_change":"2024-09-10T10:06:13.725498+0000","last_active":"2024-09-10T10:06:14.733253+0000","last_peered":"2024-09-10T10:06:14.733253+0000","last_clean":"2024-09-10T10:06:14.733253+0000","last_became_active":"2024-09-10T10:06:13.725221+0000","last_became_peered":"2024-09-10T10:06:13.725221+0000","last_unstale":"2024-09-10T10:06:14.733253+0000","last_undegraded":"2024-09-10T10:06:14.733253+0000","last_fullsized":"2024-09-10T10:06:14.733253+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:55.226969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733093+0000","last_change":"2024-09-10T10:06:12.737164+0000","last_active":"2024-09-10T10:06:14.733093+0000","last_peered":"2024-09-10T10:06:14.733093+0000","last_clean":"2024-09-10T10:06:14.733093+0000","last_became_active":"2024-09-10T10:06:12.736918+0000","last_became_peered":"2024-09-10T10:06:12.736918+0000","last_unstale":"2024-09-10T10:06:14.733093+0000","last_undegraded":"2024-09-10T10:06:14.733093+0000","last_fullsized":"2024-09-10T10:06:14.733093+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:21.196567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735515+0000","last_change":"2024-09-10T10:06:13.745196+0000","last_active":"2024-09-10T10:06:14.735515+0000","last_peered":"2024-09-10T10:06:14.735515+0000","last_clean":"2024-09-10T10:06:14.735515+0000","last_became_active":"2024-09-10T10:06:13.744918+0000","last_became_peered":"2024-09-10T10:06:13.744918+0000","last_unstale":"2024-09-10T10:06:14.735515+0000","last_undegraded":"2024-09-10T10:06:14.735515+0000","last_fullsized":"2024-09-10T10:06:14.735515+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:28.990348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730367+0000","last_change":"2024-09-10T10:06:12.733832+0000","last_active":"2024-09-10T10:06:14.730367+0000","last_peered":"2024-09-10T10:06:14.730367+0000","last_clean":"2024-09-10T10:06:14.730367+0000","last_became_active":"2024-09-10T10:06:12.733522+0000","last_became_peered":"2024-09-10T10:06:12.733522+0000","last_unstale":"2024-09-10T10:06:14.730367+0000","last_undegraded":"2024-09-10T10:06:14.730367+0000","last_fullsized":"2024-09-10T10:06:14.730367+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:55.935424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.736239+0000","last_change":"2024-09-10T10:06:13.739573+0000","last_active":"2024-09-10T10:06:14.736239+0000","last_peered":"2024-09-10T10:06:14.736239+0000","last_clean":"2024-09-10T10:06:14.736239+0000","last_became_active":"2024-09-10T10:06:13.738317+0000","last_became_peered":"2024-09-10T10:06:13.738317+0000","last_unstale":"2024-09-10T10:06:14.736239+0000","last_undegraded":"2024-09-10T10:06:14.736239+0000","last_fullsized":"2024-09-10T10:06:14.736239+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:55.245814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733053+0000","last_change":"2024-09-10T10:06:12.732876+0000","last_active":"2024-09-10T10:06:14.733053+0000","last_peered":"2024-09-10T10:06:14.733053+0000","last_clean":"2024-09-10T10:06:14.733053+0000","last_became_active":"2024-09-10T10:06:12.732612+0000","last_became_peered":"2024-09-10T10:06:12.732612+0000","last_unstale":"2024-09-10T10:06:14.733053+0000","last_undegraded":"2024-09-10T10:06:14.733053+0000","last_fullsized":"2024-09-10T10:06:14.733053+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:48.843369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734519+0000","last_change":"2024-09-10T10:06:13.745115+0000","last_active":"2024-09-10T10:06:14.734519+0000","last_peered":"2024-09-10T10:06:14.734519+0000","last_clean":"2024-09-10T10:06:14.734519+0000","last_became_active":"2024-09-10T10:06:13.744806+0000","last_became_peered":"2024-09-10T10:06:13.744806+0000","last_unstale":"2024-09-10T10:06:14.734519+0000","last_undegraded":"2024-09-10T10:06:14.734519+0000","last_fullsized":"2024-09-10T10:06:14.734519+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:41.757207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730398+0000","last_change":"2024-09-10T10:06:12.732909+0000","last_active":"2024-09-10T10:06:14.730398+0000","last_peered":"2024-09-10T10:06:14.730398+0000","last_clean":"2024-09-10T10:06:14.730398+0000","last_became_active":"2024-09-10T10:06:12.732407+0000","last_became_peered":"2024-09-10T10:06:12.732407+0000","last_unstale":"2024-09-10T10:06:14.730398+0000","last_undegraded":"2024-09-10T10:06:14.730398+0000","last_fullsized":"2024-09-10T10:06:14.730398+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:47.973993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.736025+0000","last_change":"2024-09-10T10:06:13.743575+0000","last_active":"2024-09-10T10:06:14.736025+0000","last_peered":"2024-09-10T10:06:14.736025+0000","last_clean":"2024-09-10T10:06:14.736025+0000","last_became_active":"2024-09-10T10:06:13.743039+0000","last_became_peered":"2024-09-10T10:06:13.743039+0000","last_unstale":"2024-09-10T10:06:14.736025+0000","last_undegraded":"2024-09-10T10:06:14.736025+0000","last_fullsized":"2024-09-10T10:06:14.736025+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:42.985695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"164'3","reported_seq":17,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733134+0000","last_change":"2024-09-10T10:06:12.737610+0000","last_active":"2024-09-10T10:06:14.733134+0000","last_peered":"2024-09-10T10:06:14.733134+0000","last_clean":"2024-09-10T10:06:14.733134+0000","last_became_active":"2024-09-10T10:06:12.737350+0000","last_became_peered":"2024-09-10T10:06:12.737350+0000","last_unstale":"2024-09-10T10:06:14.733134+0000","last_undegraded":"2024-09-10T10:06:14.733134+0000","last_fullsized":"2024-09-10T10:06:14.733134+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:58.009495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727663+0000","last_change":"2024-09-10T10:06:13.750099+0000","last_active":"2024-09-10T10:06:14.727663+0000","last_peered":"2024-09-10T10:06:14.727663+0000","last_clean":"2024-09-10T10:06:14.727663+0000","last_became_active":"2024-09-10T10:06:13.747826+0000","last_became_peered":"2024-09-10T10:06:13.747826+0000","last_unstale":"2024-09-10T10:06:14.727663+0000","last_undegraded":"2024-09-10T10:06:14.727663+0000","last_fullsized":"2024-09-10T10:06:14.727663+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:28:54.423544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"164'6","reported_seq":20,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.736117+0000","last_change":"2024-09-10T10:06:12.730656+0000","last_active":"2024-09-10T10:06:14.736117+0000","last_peered":"2024-09-10T10:06:14.736117+0000","last_clean":"2024-09-10T10:06:14.736117+0000","last_became_active":"2024-09-10T10:06:12.730333+0000","last_became_peered":"2024-09-10T10:06:12.730333+0000","last_unstale":"2024-09-10T10:06:14.736117+0000","last_undegraded":"2024-09-10T10:06:14.736117+0000","last_fullsized":"2024-09-10T10:06:14.736117+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:08.591001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733676+0000","last_change":"2024-09-10T10:06:13.746041+0000","last_active":"2024-09-10T10:06:14.733676+0000","last_peered":"2024-09-10T10:06:14.733676+0000","last_clean":"2024-09-10T10:06:14.733676+0000","last_became_active":"2024-09-10T10:06:13.745591+0000","last_became_peered":"2024-09-10T10:06:13.745591+0000","last_unstale":"2024-09-10T10:06:14.733676+0000","last_undegraded":"2024-09-10T10:06:14.733676+0000","last_fullsized":"2024-09-10T10:06:14.733676+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:21.893885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730871+0000","last_change":"2024-09-10T10:06:12.725249+0000","last_active":"2024-09-10T10:06:14.730871+0000","last_peered":"2024-09-10T10:06:14.730871+0000","last_clean":"2024-09-10T10:06:14.730871+0000","last_became_active":"2024-09-10T10:06:12.724875+0000","last_became_peered":"2024-09-10T10:06:12.724875+0000","last_unstale":"2024-09-10T10:06:14.730871+0000","last_undegraded":"2024-09-10T10:06:14.730871+0000","last_fullsized":"2024-09-10T10:06:14.730871+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:28.390353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734314+0000","last_change":"2024-09-10T10:06:13.735749+0000","last_active":"2024-09-10T10:06:14.734314+0000","last_peered":"2024-09-10T10:06:14.734314+0000","last_clean":"2024-09-10T10:06:14.734314+0000","last_became_active":"2024-09-10T10:06:13.735312+0000","last_became_peered":"2024-09-10T10:06:13.735312+0000","last_unstale":"2024-09-10T10:06:14.734314+0000","last_undegraded":"2024-09-10T10:06:14.734314+0000","last_fullsized":"2024-09-10T10:06:14.734314+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:32.270063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.729865+0000","last_change":"2024-09-10T10:06:12.729245+0000","last_active":"2024-09-10T10:06:14.729865+0000","last_peered":"2024-09-10T10:06:14.729865+0000","last_clean":"2024-09-10T10:06:14.729865+0000","last_became_active":"2024-09-10T10:06:12.728911+0000","last_became_peered":"2024-09-10T10:06:12.728911+0000","last_unstale":"2024-09-10T10:06:14.729865+0000","last_undegraded":"2024-09-10T10:06:14.729865+0000","last_fullsized":"2024-09-10T10:06:14.729865+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:00.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732450+0000","last_change":"2024-09-10T10:06:13.745599+0000","last_active":"2024-09-10T10:06:14.732450+0000","last_peered":"2024-09-10T10:06:14.732450+0000","last_clean":"2024-09-10T10:06:14.732450+0000","last_became_active":"2024-09-10T10:06:13.744797+0000","last_became_peered":"2024-09-10T10:06:13.744797+0000","last_unstale":"2024-09-10T10:06:14.732450+0000","last_undegraded":"2024-09-10T10:06:14.732450+0000","last_fullsized":"2024-09-10T10:06:14.732450+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:10.748536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732555+0000","last_change":"2024-09-10T10:06:12.728387+0000","last_active":"2024-09-10T10:06:14.732555+0000","last_peered":"2024-09-10T10:06:14.732555+0000","last_clean":"2024-09-10T10:06:14.732555+0000","last_became_active":"2024-09-10T10:06:12.727050+0000","last_became_peered":"2024-09-10T10:06:12.727050+0000","last_unstale":"2024-09-10T10:06:14.732555+0000","last_undegraded":"2024-09-10T10:06:14.732555+0000","last_fullsized":"2024-09-10T10:06:14.732555+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:12:44.215834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727419+0000","last_change":"2024-09-10T10:06:13.742641+0000","last_active":"2024-09-10T10:06:14.727419+0000","last_peered":"2024-09-10T10:06:14.727419+0000","last_clean":"2024-09-10T10:06:14.727419+0000","last_became_active":"2024-09-10T10:06:13.737362+0000","last_became_peered":"2024-09-10T10:06:13.737362+0000","last_unstale":"2024-09-10T10:06:14.727419+0000","last_undegraded":"2024-09-10T10:06:14.727419+0000","last_fullsized":"2024-09-10T10:06:14.727419+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:36.904490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"164'8","reported_seq":22,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732684+0000","last_change":"2024-09-10T10:06:12.734036+0000","last_active":"2024-09-10T10:06:14.732684+0000","last_peered":"2024-09-10T10:06:14.732684+0000","last_clean":"2024-09-10T10:06:14.732684+0000","last_became_active":"2024-09-10T10:06:12.733733+0000","last_became_peered":"2024-09-10T10:06:12.733733+0000","last_unstale":"2024-09-10T10:06:14.732684+0000","last_undegraded":"2024-09-10T10:06:14.732684+0000","last_fullsized":"2024-09-10T10:06:14.732684+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:23.990859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735052+0000","last_change":"2024-09-10T10:06:13.734316+0000","last_active":"2024-09-10T10:06:14.735052+0000","last_peered":"2024-09-10T10:06:14.735052+0000","last_clean":"2024-09-10T10:06:14.735052+0000","last_became_active":"2024-09-10T10:06:13.733984+0000","last_became_peered":"2024-09-10T10:06:13.733984+0000","last_unstale":"2024-09-10T10:06:14.735052+0000","last_undegraded":"2024-09-10T10:06:14.735052+0000","last_fullsized":"2024-09-10T10:06:14.735052+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:56.938265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733160+0000","last_change":"2024-09-10T10:06:12.727110+0000","last_active":"2024-09-10T10:06:14.733160+0000","last_peered":"2024-09-10T10:06:14.733160+0000","last_clean":"2024-09-10T10:06:14.733160+0000","last_became_active":"2024-09-10T10:06:12.726876+0000","last_became_peered":"2024-09-10T10:06:12.726876+0000","last_unstale":"2024-09-10T10:06:14.733160+0000","last_undegraded":"2024-09-10T10:06:14.733160+0000","last_fullsized":"2024-09-10T10:06:14.733160+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:59.313159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735042+0000","last_change":"2024-09-10T10:06:13.748998+0000","last_active":"2024-09-10T10:06:14.735042+0000","last_peered":"2024-09-10T10:06:14.735042+0000","last_clean":"2024-09-10T10:06:14.735042+0000","last_became_active":"2024-09-10T10:06:13.748036+0000","last_became_peered":"2024-09-10T10:06:13.748036+0000","last_unstale":"2024-09-10T10:06:14.735042+0000","last_undegraded":"2024-09-10T10:06:14.735042+0000","last_fullsized":"2024-09-10T10:06:14.735042+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:41.037988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735203+0000","last_change":"2024-09-10T10:06:12.737826+0000","last_active":"2024-09-10T10:06:14.735203+0000","last_peered":"2024-09-10T10:06:14.735203+0000","last_clean":"2024-09-10T10:06:14.735203+0000","last_became_active":"2024-09-10T10:06:12.737371+0000","last_became_peered":"2024-09-10T10:06:12.737371+0000","last_unstale":"2024-09-10T10:06:14.735203+0000","last_undegraded":"2024-09-10T10:06:14.735203+0000","last_fullsized":"2024-09-10T10:06:14.735203+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:53.099507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732859+0000","last_change":"2024-09-10T10:06:13.736506+0000","last_active":"2024-09-10T10:06:14.732859+0000","last_peered":"2024-09-10T10:06:14.732859+0000","last_clean":"2024-09-10T10:06:14.732859+0000","last_became_active":"2024-09-10T10:06:13.735978+0000","last_became_peered":"2024-09-10T10:06:13.735978+0000","last_unstale":"2024-09-10T10:06:14.732859+0000","last_undegraded":"2024-09-10T10:06:14.732859+0000","last_fullsized":"2024-09-10T10:06:14.732859+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:31.507046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733227+0000","last_change":"2024-09-10T10:06:12.724585+0000","last_active":"2024-09-10T10:06:14.733227+0000","last_peered":"2024-09-10T10:06:14.733227+0000","last_clean":"2024-09-10T10:06:14.733227+0000","last_became_active":"2024-09-10T10:06:12.724329+0000","last_became_peered":"2024-09-10T10:06:12.724329+0000","last_unstale":"2024-09-10T10:06:14.733227+0000","last_undegraded":"2024-09-10T10:06:14.733227+0000","last_fullsized":"2024-09-10T10:06:14.733227+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:43.007873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732905+0000","last_change":"2024-09-10T10:06:13.744391+0000","last_active":"2024-09-10T10:06:14.732905+0000","last_peered":"2024-09-10T10:06:14.732905+0000","last_clean":"2024-09-10T10:06:14.732905+0000","last_became_active":"2024-09-10T10:06:13.743509+0000","last_became_peered":"2024-09-10T10:06:13.743509+0000","last_unstale":"2024-09-10T10:06:14.732905+0000","last_undegraded":"2024-09-10T10:06:14.732905+0000","last_fullsized":"2024-09-10T10:06:14.732905+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:50.918252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734260+0000","last_change":"2024-09-10T10:06:12.736724+0000","last_active":"2024-09-10T10:06:14.734260+0000","last_peered":"2024-09-10T10:06:14.734260+0000","last_clean":"2024-09-10T10:06:14.734260+0000","last_became_active":"2024-09-10T10:06:12.736469+0000","last_became_peered":"2024-09-10T10:06:12.736469+0000","last_unstale":"2024-09-10T10:06:14.734260+0000","last_undegraded":"2024-09-10T10:06:14.734260+0000","last_fullsized":"2024-09-10T10:06:14.734260+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:16.719617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732827+0000","last_change":"2024-09-10T10:06:13.748174+0000","last_active":"2024-09-10T10:06:14.732827+0000","last_peered":"2024-09-10T10:06:14.732827+0000","last_clean":"2024-09-10T10:06:14.732827+0000","last_became_active":"2024-09-10T10:06:13.747903+0000","last_became_peered":"2024-09-10T10:06:13.747903+0000","last_unstale":"2024-09-10T10:06:14.732827+0000","last_undegraded":"2024-09-10T10:06:14.732827+0000","last_fullsized":"2024-09-10T10:06:14.732827+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:06.793474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734408+0000","last_change":"2024-09-10T10:06:12.733519+0000","last_active":"2024-09-10T10:06:14.734408+0000","last_peered":"2024-09-10T10:06:14.734408+0000","last_clean":"2024-09-10T10:06:14.734408+0000","last_became_active":"2024-09-10T10:06:12.732717+0000","last_became_peered":"2024-09-10T10:06:12.732717+0000","last_unstale":"2024-09-10T10:06:14.734408+0000","last_undegraded":"2024-09-10T10:06:14.734408+0000","last_fullsized":"2024-09-10T10:06:14.734408+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:54.805925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734096+0000","last_change":"2024-09-10T10:06:13.733484+0000","last_active":"2024-09-10T10:06:14.734096+0000","last_peered":"2024-09-10T10:06:14.734096+0000","last_clean":"2024-09-10T10:06:14.734096+0000","last_became_active":"2024-09-10T10:06:13.733237+0000","last_became_peered":"2024-09-10T10:06:13.733237+0000","last_unstale":"2024-09-10T10:06:14.734096+0000","last_undegraded":"2024-09-10T10:06:14.734096+0000","last_fullsized":"2024-09-10T10:06:14.734096+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:54:06.371322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727528+0000","last_change":"2024-09-10T10:06:12.730275+0000","last_active":"2024-09-10T10:06:14.727528+0000","last_peered":"2024-09-10T10:06:14.727528+0000","last_clean":"2024-09-10T10:06:14.727528+0000","last_became_active":"2024-09-10T10:06:12.730014+0000","last_became_peered":"2024-09-10T10:06:12.730014+0000","last_unstale":"2024-09-10T10:06:14.727528+0000","last_undegraded":"2024-09-10T10:06:14.727528+0000","last_fullsized":"2024-09-10T10:06:14.727528+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:47.434806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734715+0000","last_change":"2024-09-10T10:06:13.735906+0000","last_active":"2024-09-10T10:06:14.734715+0000","last_peered":"2024-09-10T10:06:14.734715+0000","last_clean":"2024-09-10T10:06:14.734715+0000","last_became_active":"2024-09-10T10:06:13.735631+0000","last_became_peered":"2024-09-10T10:06:13.735631+0000","last_unstale":"2024-09-10T10:06:14.734715+0000","last_undegraded":"2024-09-10T10:06:14.734715+0000","last_fullsized":"2024-09-10T10:06:14.734715+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:31.507046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727514+0000","last_change":"2024-09-10T10:06:12.726846+0000","last_active":"2024-09-10T10:06:14.727514+0000","last_peered":"2024-09-10T10:06:14.727514+0000","last_clean":"2024-09-10T10:06:14.727514+0000","last_became_active":"2024-09-10T10:06:12.726284+0000","last_became_peered":"2024-09-10T10:06:12.726284+0000","last_unstale":"2024-09-10T10:06:14.727514+0000","last_undegraded":"2024-09-10T10:06:14.727514+0000","last_fullsized":"2024-09-10T10:06:14.727514+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:57.450534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730134+0000","last_change":"2024-09-10T10:06:13.743268+0000","last_active":"2024-09-10T10:06:14.730134+0000","last_peered":"2024-09-10T10:06:14.730134+0000","last_clean":"2024-09-10T10:06:14.730134+0000","last_became_active":"2024-09-10T10:06:13.742937+0000","last_became_peered":"2024-09-10T10:06:13.742937+0000","last_unstale":"2024-09-10T10:06:14.730134+0000","last_undegraded":"2024-09-10T10:06:14.730134+0000","last_fullsized":"2024-09-10T10:06:14.730134+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:24.649092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733360+0000","last_change":"2024-09-10T10:06:12.732634+0000","last_active":"2024-09-10T10:06:14.733360+0000","last_peered":"2024-09-10T10:06:14.733360+0000","last_clean":"2024-09-10T10:06:14.733360+0000","last_became_active":"2024-09-10T10:06:12.732205+0000","last_became_peered":"2024-09-10T10:06:12.732205+0000","last_unstale":"2024-09-10T10:06:14.733360+0000","last_undegraded":"2024-09-10T10:06:14.733360+0000","last_fullsized":"2024-09-10T10:06:14.733360+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:40.035147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731094+0000","last_change":"2024-09-10T10:06:13.748008+0000","last_active":"2024-09-10T10:06:14.731094+0000","last_peered":"2024-09-10T10:06:14.731094+0000","last_clean":"2024-09-10T10:06:14.731094+0000","last_became_active":"2024-09-10T10:06:13.747690+0000","last_became_peered":"2024-09-10T10:06:13.747690+0000","last_unstale":"2024-09-10T10:06:14.731094+0000","last_undegraded":"2024-09-10T10:06:14.731094+0000","last_fullsized":"2024-09-10T10:06:14.731094+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:50:52.811453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"164'2","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734622+0000","last_change":"2024-09-10T10:06:12.734762+0000","last_active":"2024-09-10T10:06:14.734622+0000","last_peered":"2024-09-10T10:06:14.734622+0000","last_clean":"2024-09-10T10:06:14.734622+0000","last_became_active":"2024-09-10T10:06:12.734501+0000","last_became_peered":"2024-09-10T10:06:12.734501+0000","last_unstale":"2024-09-10T10:06:14.734622+0000","last_undegraded":"2024-09-10T10:06:14.734622+0000","last_fullsized":"2024-09-10T10:06:14.734622+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:12.180453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727916+0000","last_change":"2024-09-10T10:06:13.742496+0000","last_active":"2024-09-10T10:06:14.727916+0000","last_peered":"2024-09-10T10:06:14.727916+0000","last_clean":"2024-09-10T10:06:14.727916+0000","last_became_active":"2024-09-10T10:06:13.742177+0000","last_became_peered":"2024-09-10T10:06:13.742177+0000","last_unstale":"2024-09-10T10:06:14.727916+0000","last_undegraded":"2024-09-10T10:06:14.727916+0000","last_fullsized":"2024-09-10T10:06:14.727916+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:26.996263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"165'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:15.725493+0000","last_change":"2024-09-10T10:06:12.730901+0000","last_active":"2024-09-10T10:06:15.725493+0000","last_peered":"2024-09-10T10:06:15.725493+0000","last_clean":"2024-09-10T10:06:15.725493+0000","last_became_active":"2024-09-10T10:06:12.730559+0000","last_became_peered":"2024-09-10T10:06:12.730559+0000","last_unstale":"2024-09-10T10:06:15.725493+0000","last_undegraded":"2024-09-10T10:06:15.725493+0000","last_fullsized":"2024-09-10T10:06:15.725493+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:19.774789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734980+0000","last_change":"2024-09-10T10:06:13.735913+0000","last_active":"2024-09-10T10:06:14.734980+0000","last_peered":"2024-09-10T10:06:14.734980+0000","last_clean":"2024-09-10T10:06:14.734980+0000","last_became_active":"2024-09-10T10:06:13.735642+0000","last_became_peered":"2024-09-10T10:06:13.735642+0000","last_unstale":"2024-09-10T10:06:14.734980+0000","last_undegraded":"2024-09-10T10:06:14.734980+0000","last_fullsized":"2024-09-10T10:06:14.734980+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:22.199408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733934+0000","last_change":"2024-09-10T10:06:12.735001+0000","last_active":"2024-09-10T10:06:14.733934+0000","last_peered":"2024-09-10T10:06:14.733934+0000","last_clean":"2024-09-10T10:06:14.733934+0000","last_became_active":"2024-09-10T10:06:12.734717+0000","last_became_peered":"2024-09-10T10:06:12.734717+0000","last_unstale":"2024-09-10T10:06:14.733934+0000","last_undegraded":"2024-09-10T10:06:14.733934+0000","last_fullsized":"2024-09-10T10:06:14.733934+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:55.935424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730477+0000","last_change":"2024-09-10T10:06:13.741394+0000","last_active":"2024-09-10T10:06:14.730477+0000","last_peered":"2024-09-10T10:06:14.730477+0000","last_clean":"2024-09-10T10:06:14.730477+0000","last_became_active":"2024-09-10T10:06:13.740773+0000","last_became_peered":"2024-09-10T10:06:13.740773+0000","last_unstale":"2024-09-10T10:06:14.730477+0000","last_undegraded":"2024-09-10T10:06:14.730477+0000","last_fullsized":"2024-09-10T10:06:14.730477+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:21.893885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734346+0000","last_change":"2024-09-10T10:06:12.735766+0000","last_active":"2024-09-10T10:06:14.734346+0000","last_peered":"2024-09-10T10:06:14.734346+0000","last_clean":"2024-09-10T10:06:14.734346+0000","last_became_active":"2024-09-10T10:06:12.735135+0000","last_became_peered":"2024-09-10T10:06:12.735135+0000","last_unstale":"2024-09-10T10:06:14.734346+0000","last_undegraded":"2024-09-10T10:06:14.734346+0000","last_fullsized":"2024-09-10T10:06:14.734346+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:32.219825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727472+0000","last_change":"2024-09-10T10:06:13.738401+0000","last_active":"2024-09-10T10:06:14.727472+0000","last_peered":"2024-09-10T10:06:14.727472+0000","last_clean":"2024-09-10T10:06:14.727472+0000","last_became_active":"2024-09-10T10:06:13.738135+0000","last_became_peered":"2024-09-10T10:06:13.738135+0000","last_unstale":"2024-09-10T10:06:14.727472+0000","last_undegraded":"2024-09-10T10:06:14.727472+0000","last_fullsized":"2024-09-10T10:06:14.727472+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:42.985695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735999+0000","last_change":"2024-09-10T10:06:12.740866+0000","last_active":"2024-09-10T10:06:14.735999+0000","last_peered":"2024-09-10T10:06:14.735999+0000","last_clean":"2024-09-10T10:06:14.735999+0000","last_became_active":"2024-09-10T10:06:12.740591+0000","last_became_peered":"2024-09-10T10:06:12.740591+0000","last_unstale":"2024-09-10T10:06:14.735999+0000","last_undegraded":"2024-09-10T10:06:14.735999+0000","last_fullsized":"2024-09-10T10:06:14.735999+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:32.316892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730894+0000","last_change":"2024-09-10T10:06:13.737237+0000","last_active":"2024-09-10T10:06:14.730894+0000","last_peered":"2024-09-10T10:06:14.730894+0000","last_clean":"2024-09-10T10:06:14.730894+0000","last_became_active":"2024-09-10T10:06:13.736815+0000","last_became_peered":"2024-09-10T10:06:13.736815+0000","last_unstale":"2024-09-10T10:06:14.730894+0000","last_undegraded":"2024-09-10T10:06:14.730894+0000","last_fullsized":"2024-09-10T10:06:14.730894+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:18.783891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.733912+0000","last_change":"2024-09-10T10:06:12.725002+0000","last_active":"2024-09-10T10:06:14.733912+0000","last_peered":"2024-09-10T10:06:14.733912+0000","last_clean":"2024-09-10T10:06:14.733912+0000","last_became_active":"2024-09-10T10:06:12.724743+0000","last_became_peered":"2024-09-10T10:06:12.724743+0000","last_unstale":"2024-09-10T10:06:14.733912+0000","last_undegraded":"2024-09-10T10:06:14.733912+0000","last_fullsized":"2024-09-10T10:06:14.733912+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.410825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.732582+0000","last_change":"2024-09-10T10:06:13.745814+0000","last_active":"2024-09-10T10:06:14.732582+0000","last_peered":"2024-09-10T10:06:14.732582+0000","last_clean":"2024-09-10T10:06:14.732582+0000","last_became_active":"2024-09-10T10:06:13.745233+0000","last_became_peered":"2024-09-10T10:06:13.745233+0000","last_unstale":"2024-09-10T10:06:14.732582+0000","last_undegraded":"2024-09-10T10:06:14.732582+0000","last_fullsized":"2024-09-10T10:06:14.732582+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:39.830795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730253+0000","last_change":"2024-09-10T10:06:12.727463+0000","last_active":"2024-09-10T10:06:14.730253+0000","last_peered":"2024-09-10T10:06:14.730253+0000","last_clean":"2024-09-10T10:06:14.730253+0000","last_became_active":"2024-09-10T10:06:12.727028+0000","last_became_peered":"2024-09-10T10:06:12.727028+0000","last_unstale":"2024-09-10T10:06:14.730253+0000","last_undegraded":"2024-09-10T10:06:14.730253+0000","last_fullsized":"2024-09-10T10:06:14.730253+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:46.410825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.734745+0000","last_change":"2024-09-10T10:06:13.732978+0000","last_active":"2024-09-10T10:06:14.734745+0000","last_peered":"2024-09-10T10:06:14.734745+0000","last_clean":"2024-09-10T10:06:14.734745+0000","last_became_active":"2024-09-10T10:06:13.732689+0000","last_became_peered":"2024-09-10T10:06:13.732689+0000","last_unstale":"2024-09-10T10:06:14.734745+0000","last_undegraded":"2024-09-10T10:06:14.734745+0000","last_fullsized":"2024-09-10T10:06:14.734745+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:44.911704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.730853+0000","last_change":"2024-09-10T10:06:12.736869+0000","last_active":"2024-09-10T10:06:14.730853+0000","last_peered":"2024-09-10T10:06:14.730853+0000","last_clean":"2024-09-10T10:06:14.730853+0000","last_became_active":"2024-09-10T10:06:12.736569+0000","last_became_peered":"2024-09-10T10:06:12.736569+0000","last_unstale":"2024-09-10T10:06:14.730853+0000","last_undegraded":"2024-09-10T10:06:14.730853+0000","last_fullsized":"2024-09-10T10:06:14.730853+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:00.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.727445+0000","last_change":"2024-09-10T10:06:13.747623+0000","last_active":"2024-09-10T10:06:14.727445+0000","last_peered":"2024-09-10T10:06:14.727445+0000","last_clean":"2024-09-10T10:06:14.727445+0000","last_became_active":"2024-09-10T10:06:13.747320+0000","last_became_peered":"2024-09-10T10:06:13.747320+0000","last_unstale":"2024-09-10T10:06:14.727445+0000","last_undegraded":"2024-09-10T10:06:14.727445+0000","last_fullsized":"2024-09-10T10:06:14.727445+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:12.702706+0000","last_clean_scrub_stamp":"2024-09-10T10:06:12.702706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:15.137086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"164'1","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.731239+0000","last_change":"2024-09-10T10:06:12.728139+0000","last_active":"2024-09-10T10:06:14.731239+0000","last_peered":"2024-09-10T10:06:14.731239+0000","last_clean":"2024-09-10T10:06:14.731239+0000","last_became_active":"2024-09-10T10:06:12.727855+0000","last_became_peered":"2024-09-10T10:06:12.727855+0000","last_unstale":"2024-09-10T10:06:14.731239+0000","last_undegraded":"2024-09-10T10:06:14.731239+0000","last_fullsized":"2024-09-10T10:06:14.731239+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:06:11.699865+0000","last_clean_scrub_stamp":"2024-09-10T10:06:11.699865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:31.989079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":471,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T10:06:14.735353+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:06:14.735353+0000","last_peered":"2024-09-10T10:06:14.735353+0000","last_clean":"2024-09-10T10:06:14.735353+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:06:14.735353+0000","last_undegraded":"2024-09-10T10:06:14.735353+0000","last_fullsized":"2024-09-10T10:06:14.735353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33808,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338032,"statfs":{"total":96636764160,"available":96602144768,"internally_reserved":0,"allocated":2682880,"data_stored":1509090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2080768,"data_stored":917692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2080768,"data_stored":917692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2097152,"data_stored":918826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542245,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2088960,"data_stored":917292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2076672,"data_stored":917110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":917166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":2612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":2674688,"data_stored":1507568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:06:18.189 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:06:18.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:06:18.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0df6600640 1 Processor -- start 2024-09-10T10:06:18.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0df6600640 1 -- start start 2024-09-10T10:06:18.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0df6600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0df015c560 0x7f0df006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0df6600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df006f430 con 0x7f0df006f010 2024-09-10T10:06:18.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0df6600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df0155d50 con 0x7f0df0155970 2024-09-10T10:06:18.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0df6600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0df00b4570 con 0x7f0df015c560 2024-09-10T10:06:18.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0defe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f0df006f010 0x7f0df00ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37068/0 (socket says 172.21.15.110:37068) 2024-09-10T10:06:18.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0defe00640 1 -- 172.21.15.110:0/1330130114 learned_addr learned my addr 172.21.15.110:0/1330130114 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:18.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0deea00640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0df015c560 0x7f0df006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2478221882 0 0) 0x7f0df006f430 con 0x7f0df006f010 2024-09-10T10:06:18.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dd4003530 con 0x7f0df006f010 2024-09-10T10:06:18.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2303632655 0 0) 0x7f0dd4003530 con 0x7f0df006f010 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0df015c560 msgr2=0x7f0df006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0df015c560 0x7f0df006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 >> v1:172.21.15.145:6789/0 conn(0x7f0df0155970 legacy=0x7f0df00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df00b34d0 con 0x7f0df006f010 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f0de8001a00 con 0x7f0df006f010 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 >> v1:172.21.15.110:6789/0 conn(0x7f0df006f010 legacy=0x7f0df00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dd40039f0 0x7f0dd4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0df015c560 0x7f0dd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd4004d50 0x7f0dd40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd400bb60 con 0x7f0dd40039f0 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd400bce0 con 0x7f0dd4004d50 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.369+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd400bee0 con 0x7f0df015c560 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0def400640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dd40039f0 0x7f0dd4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0deea00640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0df015c560 0x7f0dd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0defe00640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd4004d50 0x7f0dd40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0def400640 1 -- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0df015c560 msgr2=0x7f0dd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0def400640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0df015c560 0x7f0dd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0def400640 1 -- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd4004d50 msgr2=0x7f0dd40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0def400640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0dd4004d50 0x7f0dd40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0def400640 1 -- 172.21.15.110:0/1330130114 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd4003530 con 0x7f0dd40039f0 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0def400640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dd40039f0 0x7f0dd4003e10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0de0001780 tx=0x7f0de001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a196f9faf799116 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de00013a0 con 0x7f0dd40039f0 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de00052f0 con 0x7f0dd40039f0 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.373+0000 7f0dee000640 1 -- 172.21.15.110:0/1330130114 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0003840 con 0x7f0dd40039f0 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- 172.21.15.110:0/1330130114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dd40039f0 msgr2=0x7f0dd4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 --2- 172.21.15.110:0/1330130114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dd40039f0 0x7f0dd4003e10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0de0001780 tx=0x7f0de001c3f0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0defe00640 1 -- 172.21.15.110:0/1330130114 reap_dead start 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- 172.21.15.110:0/1330130114 shutdown_connections 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- 172.21.15.110:0/1330130114 >> 172.21.15.110:0/1330130114 conn(0x7f0df006bc50 msgr2=0x7f0df006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:18.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- 172.21.15.110:0/1330130114 shutdown_connections 2024-09-10T10:06:18.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- 172.21.15.110:0/1330130114 wait complete. 2024-09-10T10:06:18.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 Processor -- start 2024-09-10T10:06:18.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- start start 2024-09-10T10:06:18.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0dd40039f0 0x7f0df014a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0df015a520 0x7f0df015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df015e660 0x7f0df015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0df00cf340 con 0x7f0df015a520 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0df00b4050 con 0x7f0df015e660 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0df6600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f0df00cf4c0 con 0x7f0dd40039f0 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0df015a520 0x7f0df015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0df015a520 0x7f0df015a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35922/0 (socket says 172.21.15.110:35922) 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 -- 172.21.15.110:0/3901235510 learned_addr learned my addr 172.21.15.110:0/3901235510 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0def400640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0dd40039f0 0x7f0df014a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0defe00640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df015e660 0x7f0df015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 -- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0dd40039f0 msgr2=0x7f0df014a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0dd40039f0 0x7f0df014a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 -- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df015e660 msgr2=0x7f0df015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df015e660 0x7f0df015eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 -- 172.21.15.110:0/3901235510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df0152f90 con 0x7f0df015a520 2024-09-10T10:06:18.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.377+0000 7f0deea00640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0df015a520 0x7f0df015a9a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc011b60 tx=0x7f0ddc010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:18.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0de7400640 1 -- 172.21.15.110:0/3901235510 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00f7e0 con 0x7f0df015a520 2024-09-10T10:06:18.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0de7400640 1 -- 172.21.15.110:0/3901235510 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ddc00f980 con 0x7f0df015a520 2024-09-10T10:06:18.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df0153270 con 0x7f0df015a520 2024-09-10T10:06:18.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0de7400640 1 -- 172.21.15.110:0/3901235510 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00fd70 con 0x7f0df015a520 2024-09-10T10:06:18.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df0153830 con 0x7f0df015a520 2024-09-10T10:06:18.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df00ce420 con 0x7f0df015a520 2024-09-10T10:06:18.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0de7400640 1 -- 172.21.15.110:0/3901235510 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f0ddc023440 con 0x7f0df015a520 2024-09-10T10:06:18.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0de7400640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0dd0068250 0x7f0dd006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0de7400640 1 -- 172.21.15.110:0/3901235510 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f0ddc085240 con 0x7f0df015a520 2024-09-10T10:06:18.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.381+0000 7f0def400640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0dd0068250 0x7f0dd006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.385+0000 7f0def400640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0dd0068250 0x7f0dd006a710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0de0001750 tx=0x7f0de00011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.389+0000 7f0de7400640 1 -- 172.21.15.110:0/3901235510 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0ddc0573f0 con 0x7f0df015a520 2024-09-10T10:06:18.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.601+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0df015f050 con 0x7f0df015a520 2024-09-10T10:06:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.601+0000 7f0de7400640 1 -- 172.21.15.110:0/3901235510 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f0ddc05b6f0 con 0x7f0df015a520 2024-09-10T10:06:18.603 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-10T10:06:18.603 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQABGOBm7J0ZFRAAr3TyQ9SjUyoljfVbGIDl/Q== 2024-09-10T10:06:18.603 INFO:teuthology.orchestra.run.smithi110.stdout: caps mds = "allow" 2024-09-10T10:06:18.603 INFO:teuthology.orchestra.run.smithi110.stdout: caps mon = "allow r" 2024-09-10T10:06:18.603 INFO:teuthology.orchestra.run.smithi110.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:06:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:updated caps for client.0 2024-09-10T10:06:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0dd0068250 msgr2=0x7f0dd006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0dd0068250 0x7f0dd006a710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0de0001750 tx=0x7f0de00011a0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0df015a520 msgr2=0x7f0df015a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0df015a520 0x7f0df015a9a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc011b60 tx=0x7f0ddc010810 comp rx=0 tx=0).stop 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 shutdown_connections 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f0dd0068250 0x7f0dd006a710 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df015e660 0x7f0df015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0df015a520 0x7f0df015a9a0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 --2- 172.21.15.110:0/3901235510 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f0dd40039f0 0x7f0df014a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 >> 172.21.15.110:0/3901235510 conn(0x7f0df006bc50 msgr2=0x7f0df0161740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 shutdown_connections 2024-09-10T10:06:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.605+0000 7f0df6600640 1 -- 172.21.15.110:0/3901235510 wait complete. 2024-09-10T10:06:18.629 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:06:18.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723f200640 1 Processor -- start 2024-09-10T10:06:18.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723f200640 1 -- start start 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723f200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f724015e470 0x7f72400b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723f200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7240162020 0x7f72400731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723f200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724006d020 con 0x7f7240073720 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723f200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f72400b6870 con 0x7f7240162020 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723f200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f72400b5c60 con 0x7f724015e470 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723d400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7240162020 0x7f72400731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f724015e470 0x7f72400b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723d400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7240162020 0x7f72400731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:42364/0 (socket says 172.21.15.110:42364) 2024-09-10T10:06:18.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723de00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f724015e470 0x7f72400b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:54768/0 (socket says 172.21.15.110:54768) 2024-09-10T10:06:18.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723d400640 1 -- 172.21.15.110:0/3314593659 learned_addr learned my addr 172.21.15.110:0/3314593659 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:18.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723de00640 1 -- 172.21.15.110:0/3314593659 >> v2:172.21.15.145:3300/0 conn(0x7f7240162020 msgr2=0x7f72400731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:18.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723de00640 1 --2- 172.21.15.110:0/3314593659 >> v2:172.21.15.145:3300/0 conn(0x7f7240162020 0x7f72400731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723de00640 1 -- 172.21.15.110:0/3314593659 >> v1:172.21.15.145:6789/0 conn(0x7f7240073720 legacy=0x7f7240070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.769+0000 7f723de00640 1 -- 172.21.15.110:0/3314593659 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72400b5f40 con 0x7f724015e470 2024-09-10T10:06:18.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723de00640 1 --2- 172.21.15.110:0/3314593659 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f724015e470 0x7f72400b55d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f72300097c0 tx=0x7f723001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89c558a0cf6818c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:18.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723ca00640 1 -- 172.21.15.110:0/3314593659 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72300b5070 con 0x7f724015e470 2024-09-10T10:06:18.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723ca00640 1 -- 172.21.15.110:0/3314593659 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72300a8990 con 0x7f724015e470 2024-09-10T10:06:18.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723ca00640 1 -- 172.21.15.110:0/3314593659 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72300a8d60 con 0x7f724015e470 2024-09-10T10:06:18.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723f200640 1 -- 172.21.15.110:0/3314593659 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f724015e470 msgr2=0x7f72400b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723f200640 1 --2- 172.21.15.110:0/3314593659 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f724015e470 0x7f72400b55d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f72300097c0 tx=0x7f723001ec90 comp rx=0 tx=0).stop 2024-09-10T10:06:18.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723f200640 1 -- 172.21.15.110:0/3314593659 shutdown_connections 2024-09-10T10:06:18.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723f200640 1 --2- 172.21.15.110:0/3314593659 >> v2:172.21.15.145:3300/0 conn(0x7f7240162020 0x7f72400731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723f200640 1 --2- 172.21.15.110:0/3314593659 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f724015e470 0x7f72400b55d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723f200640 1 -- 172.21.15.110:0/3314593659 >> 172.21.15.110:0/3314593659 conn(0x7f72400b42d0 msgr2=0x7f72400b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:18.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723f200640 1 -- 172.21.15.110:0/3314593659 shutdown_connections 2024-09-10T10:06:18.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.773+0000 7f723f200640 1 -- 172.21.15.110:0/3314593659 wait complete. 2024-09-10T10:06:18.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 Processor -- start 2024-09-10T10:06:18.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 -- start start 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7240073720 0x7f7240151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f724015e470 0x7f724014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7240162020 0x7f7240154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f724015e850 con 0x7f7240073720 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f72400b6500 con 0x7f724015e470 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f724006e730 con 0x7f7240162020 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7240073720 0x7f7240151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7240073720 0x7f7240151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35930/0 (socket says 172.21.15.110:35930) 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f724015e470 0x7f724014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723e800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7240162020 0x7f7240154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 -- 172.21.15.110:0/3408455486 learned_addr learned my addr 172.21.15.110:0/3408455486 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 -- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7240162020 msgr2=0x7f7240154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7240162020 0x7f7240154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 -- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f724015e470 msgr2=0x7f724014c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f724015e470 0x7f724014c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 -- 172.21.15.110:0/3408455486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7240161a80 con 0x7f7240073720 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723d400640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f724015e470 0x7f724014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723de00640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7240073720 0x7f7240151cd0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f72300b3040 tx=0x7f723001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f7236000640 1 -- 172.21.15.110:0/3408455486 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72300bf070 con 0x7f7240073720 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f7236000640 1 -- 172.21.15.110:0/3408455486 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72300b07a0 con 0x7f7240073720 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7240153110 con 0x7f7240073720 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f7236000640 1 -- 172.21.15.110:0/3408455486 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72300b5040 con 0x7f7240073720 2024-09-10T10:06:18.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72401511f0 con 0x7f7240073720 2024-09-10T10:06:18.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.777+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72400731c0 con 0x7f7240073720 2024-09-10T10:06:18.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.781+0000 7f7236000640 1 -- 172.21.15.110:0/3408455486 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f72300c2070 con 0x7f7240073720 2024-09-10T10:06:18.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.781+0000 7f7236000640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7220068250 0x7f722006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:18.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.781+0000 7f7236000640 1 -- 172.21.15.110:0/3408455486 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f7230120b20 con 0x7f7240073720 2024-09-10T10:06:18.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.781+0000 7f723d400640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7220068250 0x7f722006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:18.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.781+0000 7f723d400640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7220068250 0x7f722006a710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f72401500b0 tx=0x7f722c009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:18.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.785+0000 7f7236000640 1 -- 172.21.15.110:0/3408455486 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f72300187f0 con 0x7f7240073720 2024-09-10T10:06:18.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.973+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72400008e0 con 0x7f7240073720 2024-09-10T10:06:18.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.973+0000 7f7236000640 1 -- 172.21.15.110:0/3408455486 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5242 (secure 0 0 0) 0x7f72300f2c10 con 0x7f7240073720 2024-09-10T10:06:18.975 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:18.975 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":115,"btime":"2024-09-10T10:06:16:743404+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1276880772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1276880772},{"type":"v1","addr":"172.21.15.145:6834","nonce":1276880772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2843226954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2843226954},{"type":"v1","addr":"172.21.15.110:6834","nonce":2843226954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8877,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/878298806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":878298806},{"type":"v1","addr":"172.21.15.110:6835","nonce":878298806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:06:13.100588+0000","modified":"2024-09-10T10:06:16.117620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/4014830481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4014830481},{"type":"v1","addr":"172.21.15.145:6835","nonce":4014830481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8883,"qdb_cluster":[8883]},"id":13}]} 2024-09-10T10:06:18.975 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 115 2024-09-10T10:06:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7220068250 msgr2=0x7f722006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7220068250 0x7f722006a710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f72401500b0 tx=0x7f722c009240 comp rx=0 tx=0).stop 2024-09-10T10:06:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7240073720 msgr2=0x7f7240151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7240073720 0x7f7240151cd0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f72300b3040 tx=0x7f723001ba30 comp rx=0 tx=0).stop 2024-09-10T10:06:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 shutdown_connections 2024-09-10T10:06:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f7220068250 0x7f722006a710 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f7240162020 0x7f7240154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f724015e470 0x7f724014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 --2- 172.21.15.110:0/3408455486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7240073720 0x7f7240151cd0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:18.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 >> 172.21.15.110:0/3408455486 conn(0x7f72400b42d0 msgr2=0x7f72400b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:18.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 shutdown_connections 2024-09-10T10:06:18.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:18.977+0000 7f723f200640 1 -- 172.21.15.110:0/3408455486 wait complete. 2024-09-10T10:06:19.342 INFO:tasks.ceph.mds.b.smithi145.stderr:2024-09-10T10:06:19.341+0000 7f8050800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:06:19.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 115, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:06:13.100588+0000', 'modified': '2024-09-10T10:06:16.117620+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8883}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8883': {'gid': 8883, 'name': 'b', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.145:6835/4014830481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6833', 'nonce': 4014830481}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 4014830481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8883, 'qdb_cluster': [8883]} 2024-09-10T10:06:19.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:06:19.399 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T10:06:19.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.533+0000 7fa77e200640 1 Processor -- start 2024-09-10T10:06:19.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.533+0000 7fa77e200640 1 -- start start 2024-09-10T10:06:19.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa77e200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa7780cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa77e200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77806f3e0 con 0x7fa77806efa0 2024-09-10T10:06:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa77e200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77815fda0 con 0x7fa7780b5600 2024-09-10T10:06:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa77e200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa7780cee50 con 0x7fa7780726e0 2024-09-10T10:06:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa76fe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa77806efa0 0x7fa7780b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37076/0 (socket says 172.21.15.110:37076) 2024-09-10T10:06:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa76fe00640 1 -- 172.21.15.110:0/1729453273 learned_addr learned my addr 172.21.15.110:0/1729453273 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa777400640 1 --2- 172.21.15.110:0/1729453273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa7780cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa776a00640 1 -- 172.21.15.110:0/1729453273 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2943986968 0 0) 0x7fa77806f3e0 con 0x7fa77806efa0 2024-09-10T10:06:19.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa776a00640 1 -- 172.21.15.110:0/1729453273 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa75c003530 con 0x7fa77806efa0 2024-09-10T10:06:19.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa776a00640 1 -- 172.21.15.110:0/1729453273 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2204872297 0 0) 0x7fa77815fda0 con 0x7fa7780b5600 2024-09-10T10:06:19.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa776a00640 1 -- 172.21.15.110:0/1729453273 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa77806f3e0 con 0x7fa7780b5600 2024-09-10T10:06:19.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa776a00640 1 -- 172.21.15.110:0/1729453273 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1639819520 0 0) 0x7fa75c003530 con 0x7fa77806efa0 2024-09-10T10:06:19.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa777400640 1 -- 172.21.15.110:0/1729453273 >> v1:172.21.15.145:6789/0 conn(0x7fa7780b5600 legacy=0x7fa7780b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa777400640 1 -- 172.21.15.110:0/1729453273 >> v1:172.21.15.110:6789/0 conn(0x7fa77806efa0 legacy=0x7fa7780b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.537+0000 7fa777400640 1 -- 172.21.15.110:0/1729453273 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7780ce520 con 0x7fa7780726e0 2024-09-10T10:06:19.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa777400640 1 --2- 172.21.15.110:0/1729453273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa7780cf2e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7680097f0 tx=0x7fa76801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f84e9f5e0264e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa776a00640 1 -- 172.21.15.110:0/1729453273 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680b0070 con 0x7fa7780726e0 2024-09-10T10:06:19.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa776a00640 1 -- 172.21.15.110:0/1729453273 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7680ab400 con 0x7fa7780726e0 2024-09-10T10:06:19.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa776a00640 1 -- 172.21.15.110:0/1729453273 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760001680 con 0x7fa7780726e0 2024-09-10T10:06:19.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 -- 172.21.15.110:0/1729453273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 msgr2=0x7fa7780cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 --2- 172.21.15.110:0/1729453273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa7780cf2e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7680097f0 tx=0x7fa76801c3d0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 -- 172.21.15.110:0/1729453273 shutdown_connections 2024-09-10T10:06:19.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 --2- 172.21.15.110:0/1729453273 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa7780cf2e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 -- 172.21.15.110:0/1729453273 >> 172.21.15.110:0/1729453273 conn(0x7fa778062d10 msgr2=0x7fa77806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 -- 172.21.15.110:0/1729453273 shutdown_connections 2024-09-10T10:06:19.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 -- 172.21.15.110:0/1729453273 wait complete. 2024-09-10T10:06:19.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 Processor -- start 2024-09-10T10:06:19.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.541+0000 7fa77e200640 1 -- start start 2024-09-10T10:06:19.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa77e200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77806efa0 0x7fa77814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa77e200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa778152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa77e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7780b5600 0x7fa778138af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa77e200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780b4170 con 0x7fa77806efa0 2024-09-10T10:06:19.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa77e200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780b3ff0 con 0x7fa7780b5600 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa77e200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa778072ac0 con 0x7fa7780726e0 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77806efa0 0x7fa77814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77806efa0 0x7fa77814dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35940/0 (socket says 172.21.15.110:35940) 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 -- 172.21.15.110:0/2217894978 learned_addr learned my addr 172.21.15.110:0/2217894978 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa76fe00640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa778152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777e00640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7780b5600 0x7fa778138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 -- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 msgr2=0x7fa778152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa778152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 -- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7780b5600 msgr2=0x7fa778138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7780b5600 0x7fa778138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 -- 172.21.15.110:0/2217894978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa778139320 con 0x7fa77806efa0 2024-09-10T10:06:19.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa777400640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77806efa0 0x7fa77814dce0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa768000c10 tx=0x7fa76801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa76f400640 1 -- 172.21.15.110:0/2217894978 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680b0050 con 0x7fa77806efa0 2024-09-10T10:06:19.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa76f400640 1 -- 172.21.15.110:0/2217894978 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7680b1040 con 0x7fa77806efa0 2024-09-10T10:06:19.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa77e200640 1 -- 172.21.15.110:0/2217894978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa778139600 con 0x7fa77806efa0 2024-09-10T10:06:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa76f400640 1 -- 172.21.15.110:0/2217894978 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680b7650 con 0x7fa77806efa0 2024-09-10T10:06:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa77e200640 1 -- 172.21.15.110:0/2217894978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa778139030 con 0x7fa77806efa0 2024-09-10T10:06:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.545+0000 7fa76f400640 1 -- 172.21.15.110:0/2217894978 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fa7680be030 con 0x7fa77806efa0 2024-09-10T10:06:19.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa76f400640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa754068250 0x7fa75406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa76f400640 1 -- 172.21.15.110:0/2217894978 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7fa76811dfb0 con 0x7fa77806efa0 2024-09-10T10:06:19.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa76fe00640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa754068250 0x7fa75406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa781634000 1 Processor -- start 2024-09-10T10:06:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa781634000 1 -- start start 2024-09-10T10:06:19.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa76fe00640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa754068250 0x7fa75406a710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa7781fa7d0 tx=0x7fa7600016c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa781634000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa781634000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 0x5620f5460d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa781634000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5461290 0x5620f5461670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa781634000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5620f5348b60 con 0x5620f5461290 2024-09-10T10:06:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa781634000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5620f52afc70 con 0x5620f54aab20 2024-09-10T10:06:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa781634000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5620f5255ed0 con 0x5620f54ab440 2024-09-10T10:06:19.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5461290 0x5620f5461670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5461290 0x5620f5461670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35950/0 (socket says 172.21.15.110:35950) 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 -- 172.21.15.110:0/83953163 learned_addr learned my addr 172.21.15.110:0/83953163 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777400640 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa76fe00640 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 0x5620f5460d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 -- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 msgr2=0x5620f5460d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 0x5620f5460d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 -- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 msgr2=0x5620f54aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 -- 172.21.15.110:0/83953163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f5267e50 con 0x5620f5461290 2024-09-10T10:06:19.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777400640 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:19.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa777e00640 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5461290 0x5620f5461670 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa770001ec0 tx=0x7fa770001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11466eef0920b175 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa767e00640 1 -- 172.21.15.110:0/83953163 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700020d0 con 0x5620f5461290 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa767e00640 1 -- 172.21.15.110:0/83953163 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa770018070 con 0x5620f5461290 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.549+0000 7fa767e00640 1 -- 172.21.15.110:0/83953163 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770003ba0 con 0x5620f5461290 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- 172.21.15.110:0/83953163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5461290 msgr2=0x5620f5461670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5461290 0x5620f5461670 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa770001ec0 tx=0x7fa770001640 comp rx=0 tx=0).stop 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- 172.21.15.110:0/83953163 shutdown_connections 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5461290 0x5620f5461670 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 0x5620f5460d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 --2- 172.21.15.110:0/83953163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- 172.21.15.110:0/83953163 >> 172.21.15.110:0/83953163 conn(0x5620f5467600 msgr2=0x5620f545f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- 172.21.15.110:0/83953163 shutdown_connections 2024-09-10T10:06:19.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- 172.21.15.110:0/83953163 wait complete. 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- start start 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 Processor -- start 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 0x5620f5465e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f546bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f546c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5620f5311b00 con 0x5620f5466390 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5620f528a360 con 0x5620f54ab440 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5620f52872d0 con 0x5620f54aab20 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 10 client.0 _ll_register_callbacks cb 0x5620f544c820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f546c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f546c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35956/0 (socket says 172.21.15.110:35956) 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 -- 172.21.15.110:0/2557129700 learned_addr learned my addr 172.21.15.110:0/2557129700 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777400640 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 0x5620f5465e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa76fe00640 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f546bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 -- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 msgr2=0x5620f5465e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 0x5620f5465e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 -- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 msgr2=0x5620f546bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f546bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 -- 172.21.15.110:0/2557129700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f5256120 con 0x5620f5466390 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa777e00640 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f546c230 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa770000f90 tx=0x7fa770001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3f334375833cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa764c00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa764c00640 1 -- 172.21.15.110:0/2557129700 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700052d0 con 0x5620f5466390 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa764c00640 1 -- 172.21.15.110:0/2557129700 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7fa770005470 con 0x5620f5466390 2024-09-10T10:06:19.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- 172.21.15.110:0/2557129700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa770018070 con 0x5620f5466390 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa764c00640 1 -- 172.21.15.110:0/2557129700 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7fa77002e050 con 0x5620f5466390 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 10 client.9000 fetch_fsmap learned FSMap version 115 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.553+0000 7fa781634000 1 -- 172.21.15.110:0/2557129700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x5620f524f450 con 0x5620f5466390 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa764c00640 1 -- 172.21.15.110:0/2557129700 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa770018070 con 0x5620f5466390 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa764c00640 1 -- 172.21.15.110:0/2557129700 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 115) ==== 3502+0+0 (secure 0 0 0) 0x7fa770031020 con 0x5620f5466390 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 10 client.9000 fetch_fsmap finished waiting for FSMap version 115 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 10 client.9000 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 20 client.9000 populate_metadata read hostname 'smithi110' 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] conn(0x5620f54b9280 0x5620f54bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 4 client.9000 mds_command: new command op to 8883 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 -- 172.21.15.110:0/2557129700 --> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620f5249c00 con 0x5620f54b9280 2024-09-10T10:06:19.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa777400640 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] conn(0x5620f54b9280 0x5620f54bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.561 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:06:19.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa777400640 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] conn(0x5620f54b9280 0x5620f54bb670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3eee235c0ebb1e server_cookie=ab8d8ba18560be58 in_seq=0 out_seq=0 2024-09-10T10:06:19.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa764c00640 10 client.9000 ms_handle_connect on v2:172.21.15.145:6833/4014830481 2024-09-10T10:06:19.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa764c00640 1 -- 172.21.15.110:0/2557129700 <== mds.0 v2:172.21.15.145:6833/4014830481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620f5249c00 con 0x5620f54b9280 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa764c00640 10 client.9000 handle_command_reply: tid=0 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 client.9000 shutdown 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 -- 172.21.15.110:0/2557129700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 msgr2=0x5620f546c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f546c230 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa770000f90 tx=0x7fa770001260 comp rx=0 tx=0).stop 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 -- 172.21.15.110:0/2557129700 shutdown_connections 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] conn(0x5620f54b9280 0x5620f54bb670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f546c230 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f546bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa764c00640 0 client.9000 ms_handle_reset on v2:172.21.15.145:6833/4014830481 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 --2- 172.21.15.110:0/2557129700 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 0x5620f5465e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 -- 172.21.15.110:0/2557129700 >> 172.21.15.110:0/2557129700 conn(0x5620f5461290 msgr2=0x5620f545fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 -- 172.21.15.110:0/2557129700 shutdown_connections 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 1 -- 172.21.15.110:0/2557129700 wait complete. 2024-09-10T10:06:19.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.557+0000 7fa781634000 20 client.9000 trim_cache size 0 max 16384 2024-09-10T10:06:19.566 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:06:19.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 Processor -- start 2024-09-10T10:06:19.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- start start 2024-09-10T10:06:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 0x5620f54bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa778202b60 con 0x5620f5466390 2024-09-10T10:06:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780aec70 con 0x5620f54aab20 2024-09-10T10:06:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa7780af2c0 con 0x5620f54ab440 2024-09-10T10:06:19.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54a9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35962/0 (socket says 172.21.15.110:35962) 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 -- 172.21.15.110:0/3331456065 learned_addr learned my addr 172.21.15.110:0/3331456065 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777e00640 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 0x5620f54bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa76fe00640 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 -- 172.21.15.110:0/3331456065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 msgr2=0x5620f54bb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 0x5620f54bb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 -- 172.21.15.110:0/3331456065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 msgr2=0x5620f54aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 -- 172.21.15.110:0/3331456065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f5236fd0 con 0x5620f5466390 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa777400640 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54a9cf0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa76801a000 tx=0x7fa768106040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e52f6f7d85e0519 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa767400640 1 -- 172.21.15.110:0/3331456065 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680f43a0 con 0x5620f5466390 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa767400640 1 -- 172.21.15.110:0/3331456065 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7680f8bd0 con 0x5620f5466390 2024-09-10T10:06:19.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa767400640 1 -- 172.21.15.110:0/3331456065 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76811e3e0 con 0x5620f5466390 2024-09-10T10:06:19.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- 172.21.15.110:0/3331456065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 msgr2=0x5620f54a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54a9cf0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa76801a000 tx=0x7fa768106040 comp rx=0 tx=0).stop 2024-09-10T10:06:19.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- 172.21.15.110:0/3331456065 shutdown_connections 2024-09-10T10:06:19.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54ab440 0x5620f54bb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54aab20 0x5620f54aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 --2- 172.21.15.110:0/3331456065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54a9cf0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- 172.21.15.110:0/3331456065 >> 172.21.15.110:0/3331456065 conn(0x5620f5461290 msgr2=0x5620f5465650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- 172.21.15.110:0/3331456065 shutdown_connections 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.577+0000 7fa781634000 1 -- 172.21.15.110:0/3331456065 wait complete. 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 -- start start 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 Processor -- start 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54be3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 0x5620f54be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f54d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5620f52aef80 con 0x5620f5466390 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5620f5267ca0 con 0x5620f54ab440 2024-09-10T10:06:19.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x5620f52b42a0 con 0x5620f54aab20 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 10 client.0 _ll_register_callbacks cb 0x7fa760001b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54be3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54be3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35974/0 (socket says 172.21.15.110:35974) 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 -- 172.21.15.110:0/684097009 learned_addr learned my addr 172.21.15.110:0/684097009 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa76fe00640 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 0x5620f54be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777e00640 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f54d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 -- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 msgr2=0x5620f54be8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 0x5620f54be8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 -- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 msgr2=0x5620f54d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f54d9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 -- 172.21.15.110:0/684097009 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f5332750 con 0x5620f5466390 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777e00640 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f54d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa777400640 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54be3b0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa7680ab580 tx=0x7fa77000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f29532b2783bbe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa764c00640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa764c00640 1 -- 172.21.15.110:0/684097009 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680f43a0 con 0x5620f5466390 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa764c00640 1 -- 172.21.15.110:0/684097009 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7fa7680ee140 con 0x5620f5466390 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 -- 172.21.15.110:0/684097009 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7680f8bd0 con 0x5620f5466390 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa764c00640 1 -- 172.21.15.110:0/684097009 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7fa7680fe050 con 0x5620f5466390 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 10 client.9006 fetch_fsmap learned FSMap version 115 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 -- 172.21.15.110:0/684097009 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x5620f523da30 con 0x5620f5466390 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa764c00640 1 -- 172.21.15.110:0/684097009 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7680f8bd0 con 0x5620f5466390 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa764c00640 1 -- 172.21.15.110:0/684097009 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 115) ==== 3502+0+0 (secure 0 0 0) 0x7fa768101080 con 0x5620f5466390 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 10 client.9006 fetch_fsmap finished waiting for FSMap version 115 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 10 client.9006 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 20 client.9006 populate_metadata read hostname 'smithi110' 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] conn(0x5620f54da320 0x5620f551dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 4 client.9006 mds_command: new command op to 8883 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:06:19.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa781634000 1 -- 172.21.15.110:0/684097009 --> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] -- command(tid 0: {"prefix": "status"}) -- 0x5620f52bd3a0 con 0x5620f54da320 2024-09-10T10:06:19.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.581+0000 7fa76fe00640 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] conn(0x5620f54da320 0x5620f551dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa764c00640 10 client.9006 ms_handle_connect on v2:172.21.15.145:6833/4014830481 2024-09-10T10:06:19.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa76fe00640 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] conn(0x5620f54da320 0x5620f551dc00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71cdf2404880c520 server_cookie=58c78cdac6c429f8 in_seq=0 out_seq=0 2024-09-10T10:06:19.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa764c00640 1 -- 172.21.15.110:0/684097009 <== mds.0 v2:172.21.15.145:6833/4014830481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5620f52bd3a0 con 0x5620f54da320 2024-09-10T10:06:19.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa764c00640 10 client.9006 handle_command_reply: tid=0 2024-09-10T10:06:19.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 client.9006 shutdown 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 -- 172.21.15.110:0/684097009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 msgr2=0x5620f54be3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54be3b0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa7680ab580 tx=0x7fa77000a000 comp rx=0 tx=0).stop 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 -- 172.21.15.110:0/684097009 shutdown_connections 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:6833/4014830481,v1:172.21.15.145:6835/4014830481] conn(0x5620f54da320 0x5620f551dc00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620f54ab440 0x5620f54d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x5620f54aab20 0x5620f54be8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 --2- 172.21.15.110:0/684097009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5620f5466390 0x5620f54be3b0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 -- 172.21.15.110:0/684097009 >> 172.21.15.110:0/684097009 conn(0x5620f5461290 msgr2=0x5620f54a8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa764c00640 0 client.9006 ms_handle_reset on v2:172.21.15.145:6833/4014830481 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 -- 172.21.15.110:0/684097009 shutdown_connections 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 1 -- 172.21.15.110:0/684097009 wait complete. 2024-09-10T10:06:19.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa781634000 20 client.9006 trim_cache size 0 max 16384 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "af5c1104-e6d8-4fa7-9635-0686bce1580a", 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 8883, 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 5.8731128379999999, 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 115, 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 165, 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 165, 2024-09-10T10:06:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 11.075443634999999 2024-09-10T10:06:19.591 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:19.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 -- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa754068250 msgr2=0x7fa75406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa754068250 0x7fa75406a710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa7781fa7d0 tx=0x7fa7600016c0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 -- 172.21.15.110:0/2217894978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77806efa0 msgr2=0x7fa77814dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77806efa0 0x7fa77814dce0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa768000c10 tx=0x7fa76801ac10 comp rx=0 tx=0).stop 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 -- 172.21.15.110:0/2217894978 shutdown_connections 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa754068250 0x7fa75406a710 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7780b5600 0x7fa778138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa7780726e0 0x7fa778152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 --2- 172.21.15.110:0/2217894978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa77806efa0 0x7fa77814dce0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 -- 172.21.15.110:0/2217894978 >> 172.21.15.110:0/2217894978 conn(0x7fa778062d10 msgr2=0x7fa7780b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 -- 172.21.15.110:0/2217894978 shutdown_connections 2024-09-10T10:06:19.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.585+0000 7fa77e200640 1 -- 172.21.15.110:0/2217894978 wait complete. 2024-09-10T10:06:19.605 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:06:19.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.793+0000 7fd3cde00640 1 Processor -- start 2024-09-10T10:06:19.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.793+0000 7fd3cde00640 1 -- start start 2024-09-10T10:06:19.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3cde00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 0x7fd3c80b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3cde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd3c80b5470 0x7fd3c815c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3cde00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd3c80cef70 0x7fd3c80cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3cde00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c80b4e70 con 0x7fd3c80b5470 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3cde00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c80ceae0 con 0x7fd3c80cef70 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3cde00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c80ce1d0 con 0x7fd3c80b5090 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3c7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd3c80b5470 0x7fd3c815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3c7e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fd3c80b5470 0x7fd3c815c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35976/0 (socket says 172.21.15.110:35976) 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3c7e00640 1 -- 172.21.15.110:0/3039772051 learned_addr learned my addr 172.21.15.110:0/3039772051 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3cca00640 1 --2- 172.21.15.110:0/3039772051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 0x7fd3c80b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3cd400640 1 --2- 172.21.15.110:0/3039772051 >> v2:172.21.15.145:3300/0 conn(0x7fd3c80cef70 0x7fd3c80cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3c7e00640 1 -- 172.21.15.110:0/3039772051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 msgr2=0x7fd3c80b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3c7e00640 1 --2- 172.21.15.110:0/3039772051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 0x7fd3c80b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3c7e00640 1 -- 172.21.15.110:0/3039772051 >> v2:172.21.15.145:3300/0 conn(0x7fd3c80cef70 msgr2=0x7fd3c80cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:19.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3c7e00640 1 --2- 172.21.15.110:0/3039772051 >> v2:172.21.15.145:3300/0 conn(0x7fd3c80cef70 0x7fd3c80cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.797+0000 7fd3c7e00640 1 -- 172.21.15.110:0/3039772051 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c815be60 con 0x7fd3c80b5470 2024-09-10T10:06:19.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3c7e00640 1 --2- 172.21.15.110:0/3039772051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c815c180 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80b4420 tx=0x7fd3b401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852bf2e0fbffac2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3c7400640 1 -- 172.21.15.110:0/3039772051 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40b4070 con 0x7fd3c80b5470 2024-09-10T10:06:19.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3c7400640 1 -- 172.21.15.110:0/3039772051 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b40a7bd0 con 0x7fd3c80b5470 2024-09-10T10:06:19.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3c7400640 1 -- 172.21.15.110:0/3039772051 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40af440 con 0x7fd3c80b5470 2024-09-10T10:06:19.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 -- 172.21.15.110:0/3039772051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 msgr2=0x7fd3c815c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 --2- 172.21.15.110:0/3039772051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c815c180 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80b4420 tx=0x7fd3b401ebc0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 -- 172.21.15.110:0/3039772051 shutdown_connections 2024-09-10T10:06:19.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 --2- 172.21.15.110:0/3039772051 >> v2:172.21.15.145:3300/0 conn(0x7fd3c80cef70 0x7fd3c80cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 --2- 172.21.15.110:0/3039772051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c815c180 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 --2- 172.21.15.110:0/3039772051 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 0x7fd3c80b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 -- 172.21.15.110:0/3039772051 >> 172.21.15.110:0/3039772051 conn(0x7fd3c806d0f0 msgr2=0x7fd3c806d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 -- 172.21.15.110:0/3039772051 shutdown_connections 2024-09-10T10:06:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 -- 172.21.15.110:0/3039772051 wait complete. 2024-09-10T10:06:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 Processor -- start 2024-09-10T10:06:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.801+0000 7fd3cde00640 1 -- start start 2024-09-10T10:06:19.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 0x7fd3c8146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c80ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3c80cef70 0x7fd3c80aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81588e0 con 0x7fd3c80b5470 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80cd890 con 0x7fd3c80cef70 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c8158a60 con 0x7fd3c80b5090 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c80ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cca00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 0x7fd3c8146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c80ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35990/0 (socket says 172.21.15.110:35990) 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 -- 172.21.15.110:0/4236750490 learned_addr learned my addr 172.21.15.110:0/4236750490 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cd400640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3c80cef70 0x7fd3c80aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 -- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 msgr2=0x7fd3c8146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 0x7fd3c8146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 -- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3c80cef70 msgr2=0x7fd3c80aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3c80cef70 0x7fd3c80aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 -- 172.21.15.110:0/4236750490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c8147d50 con 0x7fd3c80b5470 2024-09-10T10:06:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c7e00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c80ae840 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400b2c0 tx=0x7fd3b401b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c4c00640 1 -- 172.21.15.110:0/4236750490 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40bf070 con 0x7fd3c80b5470 2024-09-10T10:06:19.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c4c00640 1 -- 172.21.15.110:0/4236750490 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b40b6050 con 0x7fd3c80b5470 2024-09-10T10:06:19.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c8148030 con 0x7fd3c80b5470 2024-09-10T10:06:19.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c4c00640 1 -- 172.21.15.110:0/4236750490 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40b4030 con 0x7fd3c80b5470 2024-09-10T10:06:19.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c814f7c0 con 0x7fd3c80b5470 2024-09-10T10:06:19.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c80cf610 con 0x7fd3c80b5470 2024-09-10T10:06:19.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.805+0000 7fd3c4c00640 1 -- 172.21.15.110:0/4236750490 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fd3b40c2070 con 0x7fd3c80b5470 2024-09-10T10:06:19.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.809+0000 7fd3c4c00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd3a8068250 0x7fd3a806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:19.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.809+0000 7fd3c4c00640 1 -- 172.21.15.110:0/4236750490 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7fd3b40afe00 con 0x7fd3c80b5470 2024-09-10T10:06:19.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.809+0000 7fd3cca00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd3a8068250 0x7fd3a806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:19.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.809+0000 7fd3cca00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd3a8068250 0x7fd3a806a710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80af400 tx=0x7fd3b8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:19.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:19.813+0000 7fd3c4c00640 1 -- 172.21.15.110:0/4236750490 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd3b40187e0 con 0x7fd3c80b5470 2024-09-10T10:06:20.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd3c8158f60 con 0x7fd3c80b5470 2024-09-10T10:06:20.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3c4c00640 1 -- 172.21.15.110:0/4236750490 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v115) ==== 53+0+73 (secure 0 0 0) 0x7fd3b40f07f0 con 0x7fd3c80b5470 2024-09-10T10:06:20.006 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd3a8068250 msgr2=0x7fd3a806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd3a8068250 0x7fd3a806a710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80af400 tx=0x7fd3b8009190 comp rx=0 tx=0).stop 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 msgr2=0x7fd3c80ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c80ae840 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400b2c0 tx=0x7fd3b401b960 comp rx=0 tx=0).stop 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 shutdown_connections 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fd3a8068250 0x7fd3a806a710 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3c80cef70 0x7fd3c80aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80b5470 0x7fd3c80ae840 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 --2- 172.21.15.110:0/4236750490 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fd3c80b5090 0x7fd3c8146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 >> 172.21.15.110:0/4236750490 conn(0x7fd3c806d0f0 msgr2=0x7fd3c8155450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:20.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 shutdown_connections 2024-09-10T10:06:20.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.005+0000 7fd3cde00640 1 -- 172.21.15.110:0/4236750490 wait complete. 2024-09-10T10:06:20.027 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:06:20.027 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:06:20.027 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-09-10T10:06:20.027 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:06:20.027 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:06:20.027 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:06:20.027 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:06:20.028 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:06:20.028 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:06:20.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:20.028 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-09-10T10:06:20.037 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:06:20.037 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:06:20.037 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:06:20.037 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:20.037 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-09-10T10:06:20.037 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:06:20.038 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 41995sec preferred_lft 41995sec 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ec4:7aff:fe8f:cc35/64 scope link 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 02:42:10:55:34:4a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:20.039 INFO:teuthology.orchestra.run.smithi110.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:06:20.040 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:20.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:20.041 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:06:20.101 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-10T10:06:20.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:20.103 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-09-10T10:06:20.159 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-09-10T10:06:20.159 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:06:20.159 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:06:20.160 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:06:20.160 INFO:teuthology.orchestra.run.smithi110.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:06:20.160 INFO:teuthology.orchestra.run.smithi110.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:06:20.160 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:06:20.160 INFO:teuthology.orchestra.run.smithi110.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:06:20.160 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:06:20.160 INFO:teuthology.orchestra.run.smithi110.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:06:20.160 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:06:20.160 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:06:20.160 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-09-10T10:06:20.161 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:06:20.161 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:06:20.161 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:06:20.161 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-09-10T10:06:20.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:06:20.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:06:20.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:20.292 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:06:20.301 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:06:20.301 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:06:20.301 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:06:20.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:20.302 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-09-10T10:06:20.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:06:20.362 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:06:20.411 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:06:20.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:06:20.412 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:06:20.459 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T10:06:20.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:20.459 DEBUG:teuthology.orchestra.run.smithi110:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:06:20.546 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:06:20.547 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:06:20.560 INFO:teuthology.orchestra.run.smithi110.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:06:20.560 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:06:20.560 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:06:20.560 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: trying mount with new device syntax: 0@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:06:20.560 INFO:teuthology.orchestra.run.smithi110.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:06:20.564 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:06:20.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:20.564 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:06:20.627 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:06:20.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:06:20.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:06:20.738 INFO:teuthology.orchestra.run.smithi110.stdout:client9015 2024-09-10T10:06:20.738 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:20.738 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:20.747 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:06:20.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.921+0000 7f3f97200640 1 Processor -- start 2024-09-10T10:06:20.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.921+0000 7f3f97200640 1 -- start start 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.921+0000 7f3f97200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3f900cfe30 0x7f3f90071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.921+0000 7f3f97200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3f900cd510 0x7f3f900cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.921+0000 7f3f97200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f9006d2d0 con 0x7f3f9006e090 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.921+0000 7f3f97200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9015d9a0 con 0x7f3f900cfe30 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.921+0000 7f3f97200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9006d5b0 con 0x7f3f900cd510 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3f900cfe30 0x7f3f90071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3f900cfe30 0x7f3f90071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36000/0 (socket says 172.21.15.110:36000) 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8fe00640 1 -- 172.21.15.110:0/111603975 learned_addr learned my addr 172.21.15.110:0/111603975 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8f400640 1 --2- 172.21.15.110:0/111603975 >> v2:172.21.15.145:3300/0 conn(0x7f3f900cd510 0x7f3f900cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8fe00640 1 -- 172.21.15.110:0/111603975 >> v2:172.21.15.145:3300/0 conn(0x7f3f900cd510 msgr2=0x7f3f900cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8fe00640 1 --2- 172.21.15.110:0/111603975 >> v2:172.21.15.145:3300/0 conn(0x7f3f900cd510 0x7f3f900cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8fe00640 1 -- 172.21.15.110:0/111603975 >> v1:172.21.15.145:6789/0 conn(0x7f3f9006e090 legacy=0x7f3f9006e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:20.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8fe00640 1 -- 172.21.15.110:0/111603975 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9006d890 con 0x7f3f900cfe30 2024-09-10T10:06:20.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8fe00640 1 --2- 172.21.15.110:0/111603975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cfe30 0x7f3f90071a40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3f900ce0f0 tx=0x7f3f8001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd7e039ec285755 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8ea00640 1 -- 172.21.15.110:0/111603975 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f800b2070 con 0x7f3f900cfe30 2024-09-10T10:06:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8ea00640 1 -- 172.21.15.110:0/111603975 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f800a9070 con 0x7f3f900cfe30 2024-09-10T10:06:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.925+0000 7f3f8ea00640 1 -- 172.21.15.110:0/111603975 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f800ad880 con 0x7f3f900cfe30 2024-09-10T10:06:20.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- 172.21.15.110:0/111603975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cfe30 msgr2=0x7f3f90071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 --2- 172.21.15.110:0/111603975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cfe30 0x7f3f90071a40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3f900ce0f0 tx=0x7f3f8001c450 comp rx=0 tx=0).stop 2024-09-10T10:06:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- 172.21.15.110:0/111603975 shutdown_connections 2024-09-10T10:06:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 --2- 172.21.15.110:0/111603975 >> v2:172.21.15.145:3300/0 conn(0x7f3f900cd510 0x7f3f900cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 --2- 172.21.15.110:0/111603975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cfe30 0x7f3f90071a40 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- 172.21.15.110:0/111603975 >> 172.21.15.110:0/111603975 conn(0x7f3f9006bc50 msgr2=0x7f3f9006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- 172.21.15.110:0/111603975 shutdown_connections 2024-09-10T10:06:20.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- 172.21.15.110:0/111603975 wait complete. 2024-09-10T10:06:20.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 Processor -- start 2024-09-10T10:06:20.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- start start 2024-09-10T10:06:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f9006e090 0x7f3f90165590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cd510 0x7f3f90163b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f900cfe30 0x7f3f90166460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90072190 con 0x7f3f900cd510 2024-09-10T10:06:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9015d6c0 con 0x7f3f9006e090 2024-09-10T10:06:20.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f97200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3f90169180 con 0x7f3f900cfe30 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cd510 0x7f3f90163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cd510 0x7f3f90163b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36014/0 (socket says 172.21.15.110:36014) 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 -- 172.21.15.110:0/4149084901 learned_addr learned my addr 172.21.15.110:0/4149084901 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8fe00640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f9006e090 0x7f3f90165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f95200640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f900cfe30 0x7f3f90166460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 -- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f900cfe30 msgr2=0x7f3f90166460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f900cfe30 0x7f3f90166460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 -- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f9006e090 msgr2=0x7f3f90165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f9006e090 0x7f3f90165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 -- 172.21.15.110:0/4149084901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f90162ae0 con 0x7f3f900cd510 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.929+0000 7f3f8f400640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cd510 0x7f3f90163b80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d5f0 tx=0x7f3f7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:20.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f87e00640 1 -- 172.21.15.110:0/4149084901 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0125f0 con 0x7f3f900cd510 2024-09-10T10:06:20.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f87e00640 1 -- 172.21.15.110:0/4149084901 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f7c019070 con 0x7f3f900cd510 2024-09-10T10:06:20.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f90162dc0 con 0x7f3f900cd510 2024-09-10T10:06:20.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f87e00640 1 -- 172.21.15.110:0/4149084901 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0129d0 con 0x7f3f900cd510 2024-09-10T10:06:20.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f90163380 con 0x7f3f900cd510 2024-09-10T10:06:20.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f90163870 con 0x7f3f900cd510 2024-09-10T10:06:20.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f87e00640 1 -- 172.21.15.110:0/4149084901 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f3f7c012b70 con 0x7f3f900cd510 2024-09-10T10:06:20.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f87e00640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f70068250 0x7f3f7006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:20.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f87e00640 1 -- 172.21.15.110:0/4149084901 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f3f7c084750 con 0x7f3f900cd510 2024-09-10T10:06:20.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.933+0000 7f3f8fe00640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f70068250 0x7f3f7006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:20.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.937+0000 7f3f8fe00640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f70068250 0x7f3f7006a710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3f800af000 tx=0x7f3f800a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:20.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:20.937+0000 7f3f87e00640 1 -- 172.21.15.110:0/4149084901 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3f7c056840 con 0x7f3f900cd510 2024-09-10T10:06:21.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.129+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f901564e0 con 0x7f3f900cd510 2024-09-10T10:06:21.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f87e00640 1 -- 172.21.15.110:0/4149084901 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v115) ==== 53+0+73 (secure 0 0 0) 0x7f3f7c05ab40 con 0x7f3f900cd510 2024-09-10T10:06:21.133 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:06:21.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f70068250 msgr2=0x7f3f7006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f70068250 0x7f3f7006a710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3f800af000 tx=0x7f3f800a7000 comp rx=0 tx=0).stop 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cd510 msgr2=0x7f3f90163b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cd510 0x7f3f90163b80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d5f0 tx=0x7f3f7c00dab0 comp rx=0 tx=0).stop 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 shutdown_connections 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3f70068250 0x7f3f7006a710 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3f900cfe30 0x7f3f90166460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f900cd510 0x7f3f90163b80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 --2- 172.21.15.110:0/4149084901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f9006e090 0x7f3f90165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 >> 172.21.15.110:0/4149084901 conn(0x7f3f9006bc50 msgr2=0x7f3f90074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 shutdown_connections 2024-09-10T10:06:21.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:21.133+0000 7f3f97200640 1 -- 172.21.15.110:0/4149084901 wait complete. 2024-09-10T10:06:21.153 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:06:21.153 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:06:21.153 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-09-10T10:06:21.153 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:06:21.154 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:06:21.154 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:06:21.154 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:06:21.154 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:06:21.154 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:06:21.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:21.154 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-09-10T10:06:21.164 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T10:06:21.165 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 41990sec preferred_lft 41990sec 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 02:42:9c:49:6b:b1 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:06:21.166 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T10:06:21.167 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:06:21.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:21.168 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:06:21.230 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-09-10T10:06:21.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:21.233 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-09-10T10:06:21.288 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-09-10T10:06:21.288 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:06:21.288 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T10:06:21.288 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:06:21.288 INFO:teuthology.orchestra.run.smithi145.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T10:06:21.289 INFO:teuthology.orchestra.run.smithi145.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T10:06:21.289 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:06:21.289 INFO:teuthology.orchestra.run.smithi145.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:06:21.289 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:06:21.289 INFO:teuthology.orchestra.run.smithi145.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T10:06:21.289 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:06:21.289 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:06:21.289 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-09-10T10:06:21.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:06:21.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:06:21.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:06:21.289 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-09-10T10:06:21.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:06:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:06:21.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:06:21.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:21.434 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:06:21.443 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:06:21.443 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:06:21.444 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:06:21.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:21.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-09-10T10:06:21.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:06:21.507 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:06:21.560 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:06:21.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:06:21.560 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:06:21.611 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T10:06:21.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:21.611 DEBUG:teuthology.orchestra.run.smithi145:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:06:21.701 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:06:21.702 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:06:21.714 INFO:teuthology.orchestra.run.smithi145.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T10:06:21.714 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T10:06:21.715 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: resolved to: "172.21.15.110:3300,172.21.15.145:3300,172.21.15.145:3301" 2024-09-10T10:06:21.715 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: trying mount with new device syntax: 1@af5c1104-e6d8-4fa7-9635-0686bce1580a.cephfs=/ 2024-09-10T10:06:21.715 INFO:teuthology.orchestra.run.smithi145.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.110:3300/172.21.15.145:3300/172.21.15.145:3301" will pass to kernel 2024-09-10T10:06:21.719 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:06:21.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:21.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:06:21.741 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:06:21.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:06:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:06:21.833 INFO:teuthology.orchestra.run.smithi145.stderr:2024-09-10T10:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:06:21.839 INFO:teuthology.orchestra.run.smithi145.stdout:client7352 2024-09-10T10:06:21.839 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:06:21.839 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:21.892 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T10:06:22.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe641600640 1 Processor -- start 2024-09-10T10:06:22.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe641600640 1 -- start start 2024-09-10T10:06:22.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe641600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe63c155970 0x7fe63c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe641600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe63c06f1e0 0x7fe63c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe641600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe63c155d50 con 0x7fe63c15c560 2024-09-10T10:06:22.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe641600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe63c0b4570 con 0x7fe63c155970 2024-09-10T10:06:22.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe641600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe63c0b34d0 con 0x7fe63c06f1e0 2024-09-10T10:06:22.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe63aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe63c155970 0x7fe63c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe63aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe63c155970 0x7fe63c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36016/0 (socket says 172.21.15.110:36016) 2024-09-10T10:06:22.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.041+0000 7fe63aa00640 1 -- 172.21.15.110:0/3711429531 learned_addr learned my addr 172.21.15.110:0/3711429531 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe63b400640 1 --2- 172.21.15.110:0/3711429531 >> v2:172.21.15.145:3300/0 conn(0x7fe63c06f1e0 0x7fe63c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe63aa00640 1 -- 172.21.15.110:0/3711429531 >> v2:172.21.15.145:3300/0 conn(0x7fe63c06f1e0 msgr2=0x7fe63c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe63aa00640 1 --2- 172.21.15.110:0/3711429531 >> v2:172.21.15.145:3300/0 conn(0x7fe63c06f1e0 0x7fe63c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe63aa00640 1 -- 172.21.15.110:0/3711429531 >> v1:172.21.15.110:6789/0 conn(0x7fe63c15c560 legacy=0x7fe63c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe63aa00640 1 -- 172.21.15.110:0/3711429531 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c0b37b0 con 0x7fe63c155970 2024-09-10T10:06:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe639600640 1 -- 172.21.15.110:0/3711429531 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3058848405 0 0) 0x7fe63c155d50 con 0x7fe63c15c560 2024-09-10T10:06:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe63b400640 1 --2- 172.21.15.110:0/3711429531 >> v2:172.21.15.145:3300/0 conn(0x7fe63c06f1e0 0x7fe63c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:22.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe63aa00640 1 --2- 172.21.15.110:0/3711429531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c155970 0x7fe63c0cc8c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0735e0 tx=0x7fe62c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42522e5bb97b8393 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:22.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe639600640 1 -- 172.21.15.110:0/3711429531 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0a5880 con 0x7fe63c155970 2024-09-10T10:06:22.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe639600640 1 -- 172.21.15.110:0/3711429531 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c0a5a20 con 0x7fe63c155970 2024-09-10T10:06:22.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe639600640 1 -- 172.21.15.110:0/3711429531 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0a5df0 con 0x7fe63c155970 2024-09-10T10:06:22.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe641600640 1 -- 172.21.15.110:0/3711429531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c155970 msgr2=0x7fe63c0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe641600640 1 --2- 172.21.15.110:0/3711429531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c155970 0x7fe63c0cc8c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0735e0 tx=0x7fe62c01c9c0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe641600640 1 -- 172.21.15.110:0/3711429531 shutdown_connections 2024-09-10T10:06:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe641600640 1 --2- 172.21.15.110:0/3711429531 >> v2:172.21.15.145:3300/0 conn(0x7fe63c06f1e0 0x7fe63c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe641600640 1 --2- 172.21.15.110:0/3711429531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c155970 0x7fe63c0cc8c0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.045+0000 7fe641600640 1 -- 172.21.15.110:0/3711429531 >> 172.21.15.110:0/3711429531 conn(0x7fe63c06bc50 msgr2=0x7fe63c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 -- 172.21.15.110:0/3711429531 shutdown_connections 2024-09-10T10:06:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 -- 172.21.15.110:0/3711429531 wait complete. 2024-09-10T10:06:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 Processor -- start 2024-09-10T10:06:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 -- start start 2024-09-10T10:06:22.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c06f1e0 0x7fe63c143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe63c155970 0x7fe63c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63c15c560 0x7fe63c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c0cf170 con 0x7fe63c06f1e0 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c0cce00 con 0x7fe63c155970 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe63c0cf2f0 con 0x7fe63c15c560 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c06f1e0 0x7fe63c143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c06f1e0 0x7fe63c143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36018/0 (socket says 172.21.15.110:36018) 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 -- 172.21.15.110:0/3335030889 learned_addr learned my addr 172.21.15.110:0/3335030889 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63b400640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63c15c560 0x7fe63c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63a000640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe63c155970 0x7fe63c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 -- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63c15c560 msgr2=0x7fe63c157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63c15c560 0x7fe63c157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 -- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe63c155970 msgr2=0x7fe63c1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe63c155970 0x7fe63c1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 -- 172.21.15.110:0/3335030889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c1505f0 con 0x7fe63c06f1e0 2024-09-10T10:06:22.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe63aa00640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c06f1e0 0x7fe63c143cc0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0af040 tx=0x7fe62c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:22.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe632a00640 1 -- 172.21.15.110:0/3335030889 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0b8680 con 0x7fe63c06f1e0 2024-09-10T10:06:22.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe632a00640 1 -- 172.21.15.110:0/3335030889 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c0b8820 con 0x7fe63c06f1e0 2024-09-10T10:06:22.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe632a00640 1 -- 172.21.15.110:0/3335030889 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0b8bf0 con 0x7fe63c06f1e0 2024-09-10T10:06:22.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.049+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63c144200 con 0x7fe63c06f1e0 2024-09-10T10:06:22.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.053+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63c158810 con 0x7fe63c06f1e0 2024-09-10T10:06:22.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.053+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63c0ce420 con 0x7fe63c06f1e0 2024-09-10T10:06:22.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.053+0000 7fe632a00640 1 -- 172.21.15.110:0/3335030889 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fe62c0c0020 con 0x7fe63c06f1e0 2024-09-10T10:06:22.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.053+0000 7fe632a00640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe61c068250 0x7fe61c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.053+0000 7fe632a00640 1 -- 172.21.15.110:0/3335030889 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7fe62c11efe0 con 0x7fe63c06f1e0 2024-09-10T10:06:22.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.053+0000 7fe63a000640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe61c068250 0x7fe61c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.057+0000 7fe63a000640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe61c068250 0x7fe61c06a710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe63c157950 tx=0x7fe628009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:22.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.057+0000 7fe632a00640 1 -- 172.21.15.110:0/3335030889 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe62c0c7020 con 0x7fe63c06f1e0 2024-09-10T10:06:22.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.249+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fe63c071260 con 0x7fe63c06f1e0 2024-09-10T10:06:22.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.773+0000 7fe632a00640 1 -- 172.21.15.110:0/3335030889 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v116) ==== 136+0+0 (secure 0 0 0) 0x7fe62c0165e0 con 0x7fe63c06f1e0 2024-09-10T10:06:22.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe61c068250 msgr2=0x7fe61c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe61c068250 0x7fe61c06a710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe63c157950 tx=0x7fe628009040 comp rx=0 tx=0).stop 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c06f1e0 msgr2=0x7fe63c143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c06f1e0 0x7fe63c143cc0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0af040 tx=0x7fe62c019760 comp rx=0 tx=0).stop 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 shutdown_connections 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe61c068250 0x7fe61c06a710 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63c15c560 0x7fe63c157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe63c155970 0x7fe63c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 --2- 172.21.15.110:0/3335030889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe63c06f1e0 0x7fe63c143cc0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 >> 172.21.15.110:0/3335030889 conn(0x7fe63c06bc50 msgr2=0x7fe63c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:22.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 shutdown_connections 2024-09-10T10:06:22.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.777+0000 7fe641600640 1 -- 172.21.15.110:0/3335030889 wait complete. 2024-09-10T10:06:22.800 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T10:06:22.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.933+0000 7feba7400640 1 Processor -- start 2024-09-10T10:06:22.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.933+0000 7feba7400640 1 -- start start 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba7400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feba00cf2e0 0x7feba006efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba7400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba015fda0 con 0x7feba00b5600 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba7400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba00ce520 con 0x7feba00726e0 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba7400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7feba00cee50 con 0x7feba00cf2e0 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba5400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feba00cf2e0 0x7feba006efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba5400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feba00cf2e0 0x7feba006efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36020/0 (socket says 172.21.15.110:36020) 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba5400640 1 -- 172.21.15.110:0/2494241290 learned_addr learned my addr 172.21.15.110:0/2494241290 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feb9fe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7feba00b5600 0x7feba00b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37090/0 (socket says 172.21.15.110:37090) 2024-09-10T10:06:22.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba5400640 1 -- 172.21.15.110:0/2494241290 >> v1:172.21.15.145:6789/0 conn(0x7feba00726e0 legacy=0x7feba00b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:22.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba5400640 1 -- 172.21.15.110:0/2494241290 >> v1:172.21.15.110:6789/0 conn(0x7feba00b5600 legacy=0x7feba00b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.937+0000 7feba5400640 1 -- 172.21.15.110:0/2494241290 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba00ce800 con 0x7feba00cf2e0 2024-09-10T10:06:22.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba5400640 1 --2- 172.21.15.110:0/2494241290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00cf2e0 0x7feba006efa0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d150 tx=0x7feb9801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba8a785cc7d708a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:22.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feb9f400640 1 -- 172.21.15.110:0/2494241290 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9800b2d0 con 0x7feba00cf2e0 2024-09-10T10:06:22.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feb9f400640 1 -- 172.21.15.110:0/2494241290 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9800b470 con 0x7feba00cf2e0 2024-09-10T10:06:22.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feb9f400640 1 -- 172.21.15.110:0/2494241290 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980a79e0 con 0x7feba00cf2e0 2024-09-10T10:06:22.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 -- 172.21.15.110:0/2494241290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00cf2e0 msgr2=0x7feba006efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 --2- 172.21.15.110:0/2494241290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00cf2e0 0x7feba006efa0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d150 tx=0x7feb9801ebf0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 -- 172.21.15.110:0/2494241290 shutdown_connections 2024-09-10T10:06:22.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 --2- 172.21.15.110:0/2494241290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00cf2e0 0x7feba006efa0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 -- 172.21.15.110:0/2494241290 >> 172.21.15.110:0/2494241290 conn(0x7feba0062d10 msgr2=0x7feba006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:22.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 -- 172.21.15.110:0/2494241290 shutdown_connections 2024-09-10T10:06:22.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 -- 172.21.15.110:0/2494241290 wait complete. 2024-09-10T10:06:22.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 Processor -- start 2024-09-10T10:06:22.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.941+0000 7feba7400640 1 -- start start 2024-09-10T10:06:22.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00726e0 0x7feba014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feba00b5600 0x7feba014f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feba00cf2e0 0x7feba0159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feba0072c40 con 0x7feba00726e0 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feba0072ac0 con 0x7feba00b5600 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7feba015f950 con 0x7feba00cf2e0 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00726e0 0x7feba014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00726e0 0x7feba014de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36034/0 (socket says 172.21.15.110:36034) 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 -- 172.21.15.110:0/490754735 learned_addr learned my addr 172.21.15.110:0/490754735 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 -- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feba00cf2e0 msgr2=0x7feba0159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feba00cf2e0 0x7feba0159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 -- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feba00b5600 msgr2=0x7feba014f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feba00b5600 0x7feba014f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 -- 172.21.15.110:0/490754735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba014e380 con 0x7feba00726e0 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba4a00640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00726e0 0x7feba014de40 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7feb9000b410 tx=0x7feb9000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:22.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feb9cc00640 1 -- 172.21.15.110:0/490754735 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90015030 con 0x7feba00726e0 2024-09-10T10:06:22.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feb9cc00640 1 -- 172.21.15.110:0/490754735 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb90004020 con 0x7feba00726e0 2024-09-10T10:06:22.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feb9cc00640 1 -- 172.21.15.110:0/490754735 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90004450 con 0x7feba00726e0 2024-09-10T10:06:22.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba00cce80 con 0x7feba00726e0 2024-09-10T10:06:22.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba0152ad0 con 0x7feba00726e0 2024-09-10T10:06:22.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba006efa0 con 0x7feba00726e0 2024-09-10T10:06:22.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.945+0000 7feb9cc00640 1 -- 172.21.15.110:0/490754735 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7feb90018070 con 0x7feba00726e0 2024-09-10T10:06:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.949+0000 7feb9cc00640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb80068250 0x7feb8006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.949+0000 7feb9cc00640 1 -- 172.21.15.110:0/490754735 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7feb90083ef0 con 0x7feba00726e0 2024-09-10T10:06:22.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.949+0000 7feb9fe00640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb80068250 0x7feb8006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:22.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.949+0000 7feb9fe00640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb80068250 0x7feb8006a710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feb8c000f70 tx=0x7feb8c007a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:22.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:22.953+0000 7feb9cc00640 1 -- 172.21.15.110:0/490754735 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feba00cce80 con 0x7feba00726e0 2024-09-10T10:06:23.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.129+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7feba00b4940 con 0x7feb80068250 2024-09-10T10:06:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feb9cc00640 1 -- 172.21.15.110:0/490754735 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7feba00b4940 con 0x7feb80068250 2024-09-10T10:06:23.132 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-09-10T10:06:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-10T10:06:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephfs" 2024-09-10T10:06:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-10T10:06:23.133 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-09-10T10:06:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb80068250 msgr2=0x7feb8006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb80068250 0x7feb8006a710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feb8c000f70 tx=0x7feb8c007a10 comp rx=0 tx=0).stop 2024-09-10T10:06:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00726e0 msgr2=0x7feba014de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00726e0 0x7feba014de40 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7feb9000b410 tx=0x7feb9000b8d0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 shutdown_connections 2024-09-10T10:06:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7feb80068250 0x7feb8006a710 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7feba00cf2e0 0x7feba0159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feba00b5600 0x7feba014f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 --2- 172.21.15.110:0/490754735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feba00726e0 0x7feba014de40 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 >> 172.21.15.110:0/490754735 conn(0x7feba0062d10 msgr2=0x7feba0157f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 shutdown_connections 2024-09-10T10:06:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.133+0000 7feba7400640 1 -- 172.21.15.110:0/490754735 wait complete. 2024-09-10T10:06:23.154 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:06:23.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.289+0000 7f9a75400640 1 Processor -- start 2024-09-10T10:06:23.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.289+0000 7f9a75400640 1 -- start start 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.289+0000 7f9a75400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9a7015b330 0x7f9a7006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a75400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7006e850 0x7f9a700b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a75400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a700b4960 con 0x7f9a700ce4d0 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a75400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9a700b5570 con 0x7f9a7015b330 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a75400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9a700b4c40 con 0x7f9a7006e850 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9a7015b330 0x7f9a7006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6e800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9a7015b330 0x7f9a7006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36048/0 (socket says 172.21.15.110:36048) 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6de00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f9a700ce4d0 0x7f9a7006e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37104/0 (socket says 172.21.15.110:37104) 2024-09-10T10:06:23.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6e800640 1 -- 172.21.15.110:0/446711893 learned_addr learned my addr 172.21.15.110:0/446711893 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6f200640 1 --2- 172.21.15.110:0/446711893 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7006e850 0x7f9a700b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6e800640 1 -- 172.21.15.110:0/446711893 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7006e850 msgr2=0x7f9a700b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6e800640 1 --2- 172.21.15.110:0/446711893 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7006e850 0x7f9a700b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6e800640 1 -- 172.21.15.110:0/446711893 >> v1:172.21.15.110:6789/0 conn(0x7f9a700ce4d0 legacy=0x7f9a7006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6e800640 1 -- 172.21.15.110:0/446711893 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a700b4f20 con 0x7f9a7015b330 2024-09-10T10:06:23.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6f200640 1 --2- 172.21.15.110:0/446711893 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7006e850 0x7f9a700b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6e800640 1 --2- 172.21.15.110:0/446711893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7015b330 0x7f9a7006c6a0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009870 tx=0x7f9a6001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3885621297be8e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6d400640 1 -- 172.21.15.110:0/446711893 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600a7070 con 0x7f9a7015b330 2024-09-10T10:06:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6d400640 1 -- 172.21.15.110:0/446711893 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a600b0070 con 0x7f9a7015b330 2024-09-10T10:06:23.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.293+0000 7f9a6d400640 1 -- 172.21.15.110:0/446711893 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600ab890 con 0x7f9a7015b330 2024-09-10T10:06:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- 172.21.15.110:0/446711893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7015b330 msgr2=0x7f9a7006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 --2- 172.21.15.110:0/446711893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7015b330 0x7f9a7006c6a0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009870 tx=0x7f9a6001c560 comp rx=0 tx=0).stop 2024-09-10T10:06:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- 172.21.15.110:0/446711893 shutdown_connections 2024-09-10T10:06:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 --2- 172.21.15.110:0/446711893 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7006e850 0x7f9a700b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 --2- 172.21.15.110:0/446711893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7015b330 0x7f9a7006c6a0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- 172.21.15.110:0/446711893 >> 172.21.15.110:0/446711893 conn(0x7f9a70062e40 msgr2=0x7f9a7006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- 172.21.15.110:0/446711893 shutdown_connections 2024-09-10T10:06:23.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- 172.21.15.110:0/446711893 wait complete. 2024-09-10T10:06:23.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 Processor -- start 2024-09-10T10:06:23.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- start start 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7006e850 0x7f9a70142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7015b330 0x7f9a7015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a7015d8e0 0x7f9a7015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9a700b36b0 con 0x7f9a7006e850 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9a700b1800 con 0x7f9a7015d8e0 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a75400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9a700b3830 con 0x7f9a7015b330 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7006e850 0x7f9a70142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6e800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7006e850 0x7f9a70142d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36060/0 (socket says 172.21.15.110:36060) 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6e800640 1 -- 172.21.15.110:0/178268824 learned_addr learned my addr 172.21.15.110:0/178268824 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6de00640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7015b330 0x7f9a7015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6f200640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a7015d8e0 0x7f9a7015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6e800640 1 -- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7015b330 msgr2=0x7f9a7015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6e800640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7015b330 0x7f9a7015d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6e800640 1 -- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a7015d8e0 msgr2=0x7f9a7015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6e800640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a7015d8e0 0x7f9a7015df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.297+0000 7f9a6e800640 1 -- 172.21.15.110:0/178268824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a70151720 con 0x7f9a7006e850 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a6e800640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7006e850 0x7f9a70142d70 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9a600ad040 tx=0x7f9a6001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a66a00640 1 -- 172.21.15.110:0/178268824 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600abe60 con 0x7f9a7006e850 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a66a00640 1 -- 172.21.15.110:0/178268824 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a600b4820 con 0x7f9a7006e850 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a70151a00 con 0x7f9a7006e850 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a66a00640 1 -- 172.21.15.110:0/178268824 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600b4c10 con 0x7f9a7006e850 2024-09-10T10:06:23.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a70151fc0 con 0x7f9a7006e850 2024-09-10T10:06:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a700b0860 con 0x7f9a7006e850 2024-09-10T10:06:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a66a00640 1 -- 172.21.15.110:0/178268824 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f9a600c7390 con 0x7f9a7006e850 2024-09-10T10:06:23.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a66a00640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a50068250 0x7f9a5006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a66a00640 1 -- 172.21.15.110:0/178268824 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f9a6011eb60 con 0x7f9a7006e850 2024-09-10T10:06:23.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.301+0000 7f9a6de00640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a50068250 0x7f9a5006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.305+0000 7f9a6de00640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a50068250 0x7f9a5006a710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9a700d0040 tx=0x7f9a5c007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:23.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.305+0000 7f9a66a00640 1 -- 172.21.15.110:0/178268824 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9a600c4060 con 0x7f9a7006e850 2024-09-10T10:06:23.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9a700cd830 con 0x7f9a7006e850 2024-09-10T10:06:23.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a66a00640 1 -- 172.21.15.110:0/178268824 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f9a60016150 con 0x7f9a7006e850 2024-09-10T10:06:23.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a50068250 msgr2=0x7f9a5006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a50068250 0x7f9a5006a710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9a700d0040 tx=0x7f9a5c007e90 comp rx=0 tx=0).stop 2024-09-10T10:06:23.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7006e850 msgr2=0x7f9a70142d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7006e850 0x7f9a70142d70 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9a600ad040 tx=0x7f9a6001ada0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 shutdown_connections 2024-09-10T10:06:23.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9a50068250 0x7f9a5006a710 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a7015d8e0 0x7f9a7015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9a7015b330 0x7f9a7015d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 --2- 172.21.15.110:0/178268824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a7006e850 0x7f9a70142d70 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 >> 172.21.15.110:0/178268824 conn(0x7f9a70062e40 msgr2=0x7f9a7006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:23.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 shutdown_connections 2024-09-10T10:06:23.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.465+0000 7f9a75400640 1 -- 172.21.15.110:0/178268824 wait complete. 2024-09-10T10:06:23.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:23.487 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp15380/subvol84070) 2024-09-10T10:06:23.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:23.514 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T10:06:23.569 INFO:teuthology.orchestra.run.smithi110.stdout:cephfs_data 2024-09-10T10:06:23.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:23.570 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp15380/subvol84070) 2024-09-10T10:06:23.636 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:23.636 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp15380/subvol84070/.meta 2024-09-10T10:06:23.707 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol84070 subvol_grp15380 2024-09-10T10:06:23.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff8a6a00640 1 Processor -- start 2024-09-10T10:06:23.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff8a6a00640 1 -- start start 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff8a6a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 0x7ff8a00b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff8a6a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff8a015ec10 0x7ff8a00d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff8a6a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a006d500 con 0x7ff8a00cf380 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff8a6a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a00d12c0 con 0x7ff8a015ec10 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff8a6a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a006d7a0 con 0x7ff8a015e830 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff89f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff8a015ec10 0x7ff8a00d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff89f400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff8a015ec10 0x7ff8a00d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36062/0 (socket says 172.21.15.110:36062) 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff89f400640 1 -- 172.21.15.110:0/2900313298 learned_addr learned my addr 172.21.15.110:0/2900313298 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff89fe00640 1 --2- 172.21.15.110:0/2900313298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 0x7ff8a00b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff89f400640 1 -- 172.21.15.110:0/2900313298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 msgr2=0x7ff8a00b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:23.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff89f400640 1 --2- 172.21.15.110:0/2900313298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 0x7ff8a00b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff89f400640 1 -- 172.21.15.110:0/2900313298 >> v1:172.21.15.145:6789/0 conn(0x7ff8a00cf380 legacy=0x7ff8a006dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.889+0000 7ff89f400640 1 -- 172.21.15.110:0/2900313298 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a006da80 con 0x7ff8a015ec10 2024-09-10T10:06:23.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff89f400640 1 --2- 172.21.15.110:0/2900313298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a00d2530 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009870 tx=0x7ff88c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892d9445fa0f067d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:23.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff89ea00640 1 -- 172.21.15.110:0/2900313298 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0b2070 con 0x7ff8a015ec10 2024-09-10T10:06:23.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff89ea00640 1 -- 172.21.15.110:0/2900313298 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88c0ad680 con 0x7ff8a015ec10 2024-09-10T10:06:23.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff89ea00640 1 -- 172.21.15.110:0/2900313298 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0ada50 con 0x7ff8a015ec10 2024-09-10T10:06:23.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff8a6a00640 1 -- 172.21.15.110:0/2900313298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 msgr2=0x7ff8a00d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/2900313298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a00d2530 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009870 tx=0x7ff88c01c470 comp rx=0 tx=0).stop 2024-09-10T10:06:23.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff8a6a00640 1 -- 172.21.15.110:0/2900313298 shutdown_connections 2024-09-10T10:06:23.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/2900313298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a00d2530 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/2900313298 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 0x7ff8a00b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff8a6a00640 1 -- 172.21.15.110:0/2900313298 >> 172.21.15.110:0/2900313298 conn(0x7ff8a006f440 msgr2=0x7ff8a006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:23.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff8a6a00640 1 -- 172.21.15.110:0/2900313298 shutdown_connections 2024-09-10T10:06:23.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.893+0000 7ff8a6a00640 1 -- 172.21.15.110:0/2900313298 wait complete. 2024-09-10T10:06:23.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 Processor -- start 2024-09-10T10:06:23.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 -- start start 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8a00cf380 0x7ff8a0157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 0x7ff8a01fab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a0157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a016b750 con 0x7ff8a015ec10 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a00d2e80 con 0x7ff8a00cf380 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a016b8d0 con 0x7ff8a015e830 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a0157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a0157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36072/0 (socket says 172.21.15.110:36072) 2024-09-10T10:06:23.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 -- 172.21.15.110:0/3677763679 learned_addr learned my addr 172.21.15.110:0/3677763679 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff89f400640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 0x7ff8a01fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff89fe00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8a00cf380 0x7ff8a0157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 -- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 msgr2=0x7ff8a01fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 0x7ff8a01fab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 -- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8a00cf380 msgr2=0x7ff8a0157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8a00cf380 0x7ff8a0157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 -- 172.21.15.110:0/3677763679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a01f9110 con 0x7ff8a015ec10 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a4a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a0157e20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c740 tx=0x7ff89800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff897e00640 1 -- 172.21.15.110:0/3677763679 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898016020 con 0x7ff8a015ec10 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff897e00640 1 -- 172.21.15.110:0/3677763679 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89800ce50 con 0x7ff8a015ec10 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff897e00640 1 -- 172.21.15.110:0/3677763679 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898003980 con 0x7ff8a015ec10 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a0157860 con 0x7ff8a015ec10 2024-09-10T10:06:23.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a00cbab0 con 0x7ff8a015ec10 2024-09-10T10:06:23.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.897+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a00b5590 con 0x7ff8a015ec10 2024-09-10T10:06:23.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.901+0000 7ff897e00640 1 -- 172.21.15.110:0/3677763679 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7ff898003b20 con 0x7ff8a015ec10 2024-09-10T10:06:23.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.901+0000 7ff897e00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff880068250 0x7ff88006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:23.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.901+0000 7ff897e00640 1 -- 172.21.15.110:0/3677763679 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7ff898083f20 con 0x7ff8a015ec10 2024-09-10T10:06:23.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.901+0000 7ff89fe00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff880068250 0x7ff88006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:23.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.901+0000 7ff89fe00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff880068250 0x7ff88006a710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff890006aa0 tx=0x7ff890008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:23.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:23.905+0000 7ff897e00640 1 -- 172.21.15.110:0/3677763679 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff898020020 con 0x7ff8a015ec10 2024-09-10T10:06:24.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.085+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol84070", "group_name": "subvol_grp15380", "target": ["mon-mgr", ""]}) -- 0x7ff8a00b6dc0 con 0x7ff880068250 2024-09-10T10:06:24.094 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.093+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.095 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.093+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.095 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.093+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.095 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.093+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.095 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.093+0000 7f95dda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.133+0000 7ff897e00640 1 -- 172.21.15.110:0/3677763679 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff8a00b6dc0 con 0x7ff880068250 2024-09-10T10:06:24.136 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/subvol_grp15380/subvol84070 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff880068250 msgr2=0x7ff88006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff880068250 0x7ff88006a710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff890006aa0 tx=0x7ff890008040 comp rx=0 tx=0).stop 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 msgr2=0x7ff8a0157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a0157e20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c740 tx=0x7ff89800cc00 comp rx=0 tx=0).stop 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 shutdown_connections 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff880068250 0x7ff88006a710 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8a015ec10 0x7ff8a0157e20 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff8a015e830 0x7ff8a01fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 --2- 172.21.15.110:0/3677763679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8a00cf380 0x7ff8a0157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 >> 172.21.15.110:0/3677763679 conn(0x7ff8a006f440 msgr2=0x7ff8a00b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:24.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 shutdown_connections 2024-09-10T10:06:24.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.137+0000 7ff8a6a00640 1 -- 172.21.15.110:0/3677763679 wait complete. 2024-09-10T10:06:24.156 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol84070 subvol_grp15380 2024-09-10T10:06:24.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.293+0000 7fbf9be00640 1 Processor -- start 2024-09-10T10:06:24.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.293+0000 7fbf9be00640 1 -- start start 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf9be00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf9415c560 0x7fbf9406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf9be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fbf9406f010 0x7fbf940ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf9be00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf9406f430 con 0x7fbf94155970 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf9be00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fbf940b4570 con 0x7fbf9406f010 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf9be00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbf94155d50 con 0x7fbf9415c560 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf98a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf9415c560 0x7fbf9406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf99400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fbf94155970 0x7fbf940cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:38206/0 (socket says 172.21.15.110:38206) 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf99400640 1 -- 172.21.15.110:0/3890876991 learned_addr learned my addr 172.21.15.110:0/3890876991 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf98a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf9415c560 0x7fbf9406ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:54936/0 (socket says 172.21.15.110:54936) 2024-09-10T10:06:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf99e00640 1 --2- 172.21.15.110:0/3890876991 >> v2:172.21.15.145:3300/0 conn(0x7fbf9406f010 0x7fbf940ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf93e00640 1 -- 172.21.15.110:0/3890876991 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902266334 0 0) 0x7fbf9406f430 con 0x7fbf94155970 2024-09-10T10:06:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf99e00640 1 -- 172.21.15.110:0/3890876991 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf9415c560 msgr2=0x7fbf9406ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf99e00640 1 --2- 172.21.15.110:0/3890876991 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf9415c560 0x7fbf9406ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf99e00640 1 -- 172.21.15.110:0/3890876991 >> v1:172.21.15.145:6789/0 conn(0x7fbf94155970 legacy=0x7fbf940cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf99e00640 1 -- 172.21.15.110:0/3890876991 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf940b34d0 con 0x7fbf9406f010 2024-09-10T10:06:24.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.297+0000 7fbf98a00640 1 --2- 172.21.15.110:0/3890876991 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf9415c560 0x7fbf9406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:24.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf99e00640 1 --2- 172.21.15.110:0/3890876991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9406f010 0x7fbf940ce420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0097c0 tx=0x7fbf8c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f82d80bf061efa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:24.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf93e00640 1 -- 172.21.15.110:0/3890876991 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0b1070 con 0x7fbf9406f010 2024-09-10T10:06:24.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf93e00640 1 -- 172.21.15.110:0/3890876991 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8c0a8050 con 0x7fbf9406f010 2024-09-10T10:06:24.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf93e00640 1 -- 172.21.15.110:0/3890876991 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0ac860 con 0x7fbf9406f010 2024-09-10T10:06:24.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf9be00640 1 -- 172.21.15.110:0/3890876991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9406f010 msgr2=0x7fbf940ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf9be00640 1 --2- 172.21.15.110:0/3890876991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9406f010 0x7fbf940ce420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0097c0 tx=0x7fbf8c01c3e0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf9be00640 1 -- 172.21.15.110:0/3890876991 shutdown_connections 2024-09-10T10:06:24.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf9be00640 1 --2- 172.21.15.110:0/3890876991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9406f010 0x7fbf940ce420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf9be00640 1 --2- 172.21.15.110:0/3890876991 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf9415c560 0x7fbf9406ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf9be00640 1 -- 172.21.15.110:0/3890876991 >> 172.21.15.110:0/3890876991 conn(0x7fbf9406bc50 msgr2=0x7fbf9406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:24.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf9be00640 1 -- 172.21.15.110:0/3890876991 shutdown_connections 2024-09-10T10:06:24.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.301+0000 7fbf9be00640 1 -- 172.21.15.110:0/3890876991 wait complete. 2024-09-10T10:06:24.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 Processor -- start 2024-09-10T10:06:24.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 -- start start 2024-09-10T10:06:24.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf9406f010 0x7fbf9415f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf94155970 0x7fbf9414aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9415c560 0x7fbf9414afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf940cf170 con 0x7fbf9406f010 2024-09-10T10:06:24.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbf940b4050 con 0x7fbf9415c560 2024-09-10T10:06:24.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fbf940cf2f0 con 0x7fbf94155970 2024-09-10T10:06:24.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf9406f010 0x7fbf9415f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf9406f010 0x7fbf9415f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36082/0 (socket says 172.21.15.110:36082) 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 -- 172.21.15.110:0/2301493808 learned_addr learned my addr 172.21.15.110:0/2301493808 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99e00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9415c560 0x7fbf9414afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf98a00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf94155970 0x7fbf9414aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 -- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf94155970 msgr2=0x7fbf9414aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf94155970 0x7fbf9414aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 -- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9415c560 msgr2=0x7fbf9414afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9415c560 0x7fbf9414afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 -- 172.21.15.110:0/2301493808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9414cf40 con 0x7fbf9406f010 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99e00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9415c560 0x7fbf9414afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf99400640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf9406f010 0x7fbf9415f020 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c890 tx=0x7fbf8800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf91600640 1 -- 172.21.15.110:0/2301493808 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88018070 con 0x7fbf9406f010 2024-09-10T10:06:24.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf91600640 1 -- 172.21.15.110:0/2301493808 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8800a640 con 0x7fbf9406f010 2024-09-10T10:06:24.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf91600640 1 -- 172.21.15.110:0/2301493808 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8800aa30 con 0x7fbf9406f010 2024-09-10T10:06:24.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9415df80 con 0x7fbf9406f010 2024-09-10T10:06:24.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9415e540 con 0x7fbf9406f010 2024-09-10T10:06:24.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.305+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf940ce420 con 0x7fbf9406f010 2024-09-10T10:06:24.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.309+0000 7fbf91600640 1 -- 172.21.15.110:0/2301493808 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7fbf8800abd0 con 0x7fbf9406f010 2024-09-10T10:06:24.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.309+0000 7fbf91600640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf74068250 0x7fbf7406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.309+0000 7fbf91600640 1 -- 172.21.15.110:0/2301493808 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7fbf88083a20 con 0x7fbf9406f010 2024-09-10T10:06:24.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.309+0000 7fbf98a00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf74068250 0x7fbf7406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.309+0000 7fbf98a00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf74068250 0x7fbf7406a710 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbf94072610 tx=0x7fbf80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:24.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.313+0000 7fbf91600640 1 -- 172.21.15.110:0/2301493808 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbf88056160 con 0x7fbf9406f010 2024-09-10T10:06:24.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.497+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol84070", "group_name": "subvol_grp15380", "target": ["mon-mgr", ""]}) -- 0x7fbf940b0830 con 0x7fbf74068250 2024-09-10T10:06:24.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf91600640 1 -- 172.21.15.110:0/2301493808 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf940b0830 con 0x7fbf74068250 2024-09-10T10:06:24.502 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/subvol_grp15380/subvol84070 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf74068250 msgr2=0x7fbf7406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf74068250 0x7fbf7406a710 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbf94072610 tx=0x7fbf80008040 comp rx=0 tx=0).stop 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf9406f010 msgr2=0x7fbf9415f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf9406f010 0x7fbf9415f020 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c890 tx=0x7fbf8800cd50 comp rx=0 tx=0).stop 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 shutdown_connections 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fbf74068250 0x7fbf7406a710 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbf9415c560 0x7fbf9414afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fbf94155970 0x7fbf9414aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 --2- 172.21.15.110:0/2301493808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf9406f010 0x7fbf9415f020 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 >> 172.21.15.110:0/2301493808 conn(0x7fbf9406bc50 msgr2=0x7fbf9414e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:24.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 shutdown_connections 2024-09-10T10:06:24.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.501+0000 7fbf9be00640 1 -- 172.21.15.110:0/2301493808 wait complete. 2024-09-10T10:06:24.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:24.524 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/926ad3211d1e711906c9f4d27acca961.meta) 2024-09-10T10:06:24.550 INFO:teuthology.orchestra.run.smithi110.stdout:[GLOBAL] 2024-09-10T10:06:24.550 INFO:teuthology.orchestra.run.smithi110.stdout:version = 1 2024-09-10T10:06:24.550 INFO:teuthology.orchestra.run.smithi110.stdout:type = subvolume 2024-09-10T10:06:24.550 INFO:teuthology.orchestra.run.smithi110.stdout:path = /volumes/subvol_grp15380/subvol84070 2024-09-10T10:06:24.550 INFO:teuthology.orchestra.run.smithi110.stdout:state = complete 2024-09-10T10:06:24.551 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:24.552 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol84070 subvol_grp15380 2024-09-10T10:06:24.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff498a00640 1 Processor -- start 2024-09-10T10:06:24.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff498a00640 1 -- start start 2024-09-10T10:06:24.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff498a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff49415b170 0x7ff494073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff498a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff4940b5bf0 0x7ff4940cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff498a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4940b3b40 con 0x7ff4940cf2e0 2024-09-10T10:06:24.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff498a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7ff4940cfc90 con 0x7ff49415b170 2024-09-10T10:06:24.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff498a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff4940b3910 con 0x7ff4940b5bf0 2024-09-10T10:06:24.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff492800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff49415b170 0x7ff494073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff492800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff49415b170 0x7ff494073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36094/0 (socket says 172.21.15.110:36094) 2024-09-10T10:06:24.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff492800640 1 -- 172.21.15.110:0/1291389172 learned_addr learned my addr 172.21.15.110:0/1291389172 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:24.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff492800640 1 -- 172.21.15.110:0/1291389172 >> v2:172.21.15.145:3300/0 conn(0x7ff4940b5bf0 msgr2=0x7ff4940cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:24.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff492800640 1 --2- 172.21.15.110:0/1291389172 >> v2:172.21.15.145:3300/0 conn(0x7ff4940b5bf0 0x7ff4940cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff492800640 1 -- 172.21.15.110:0/1291389172 >> v1:172.21.15.145:6789/0 conn(0x7ff4940cf2e0 legacy=0x7ff49415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:24.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.741+0000 7ff492800640 1 -- 172.21.15.110:0/1291389172 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4940b3e20 con 0x7ff49415b170 2024-09-10T10:06:24.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff492800640 1 --2- 172.21.15.110:0/1291389172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff494073390 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff494071100 tx=0x7ff48401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b62108118e6132 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:24.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff491400640 1 -- 172.21.15.110:0/1291389172 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840b1070 con 0x7ff49415b170 2024-09-10T10:06:24.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff491400640 1 -- 172.21.15.110:0/1291389172 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4840ac630 con 0x7ff49415b170 2024-09-10T10:06:24.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff491400640 1 -- 172.21.15.110:0/1291389172 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840a8050 con 0x7ff49415b170 2024-09-10T10:06:24.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 -- 172.21.15.110:0/1291389172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 msgr2=0x7ff494073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 --2- 172.21.15.110:0/1291389172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff494073390 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff494071100 tx=0x7ff48401c4b0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 -- 172.21.15.110:0/1291389172 shutdown_connections 2024-09-10T10:06:24.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 --2- 172.21.15.110:0/1291389172 >> v2:172.21.15.145:3300/0 conn(0x7ff4940b5bf0 0x7ff4940cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 --2- 172.21.15.110:0/1291389172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff494073390 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 -- 172.21.15.110:0/1291389172 >> 172.21.15.110:0/1291389172 conn(0x7ff49406d0f0 msgr2=0x7ff49406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:24.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 -- 172.21.15.110:0/1291389172 shutdown_connections 2024-09-10T10:06:24.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 -- 172.21.15.110:0/1291389172 wait complete. 2024-09-10T10:06:24.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 Processor -- start 2024-09-10T10:06:24.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.745+0000 7ff498a00640 1 -- start start 2024-09-10T10:06:24.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4940b5bf0 0x7ff494149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4940cf2e0 0x7ff49415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff49415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff494072680 con 0x7ff49415b170 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415b550 con 0x7ff4940cf2e0 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff494072800 con 0x7ff4940b5bf0 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff49415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff49415a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36106/0 (socket says 172.21.15.110:36106) 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 -- 172.21.15.110:0/3934585104 learned_addr learned my addr 172.21.15.110:0/3934585104 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff491e00640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4940cf2e0 0x7ff49415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff492800640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4940b5bf0 0x7ff494149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 -- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4940b5bf0 msgr2=0x7ff494149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4940b5bf0 0x7ff494149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 -- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4940cf2e0 msgr2=0x7ff49415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4940cf2e0 0x7ff49415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 -- 172.21.15.110:0/3934585104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4941f7210 con 0x7ff49415b170 2024-09-10T10:06:24.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff493200640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff49415a640 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00b970 tx=0x7ff48c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:24.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff48aa00640 1 -- 172.21.15.110:0/3934585104 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c019070 con 0x7ff49415b170 2024-09-10T10:06:24.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff48aa00640 1 -- 172.21.15.110:0/3934585104 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c00cb60 con 0x7ff49415b170 2024-09-10T10:06:24.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff494158620 con 0x7ff49415b170 2024-09-10T10:06:24.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff48aa00640 1 -- 172.21.15.110:0/3934585104 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c014440 con 0x7ff49415b170 2024-09-10T10:06:24.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff494158ec0 con 0x7ff49415b170 2024-09-10T10:06:24.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.749+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff494073960 con 0x7ff49415b170 2024-09-10T10:06:24.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.753+0000 7ff48aa00640 1 -- 172.21.15.110:0/3934585104 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7ff48c01c070 con 0x7ff49415b170 2024-09-10T10:06:24.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.753+0000 7ff48aa00640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff474068250 0x7ff47406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:24.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.753+0000 7ff48aa00640 1 -- 172.21.15.110:0/3934585104 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7ff48c084140 con 0x7ff49415b170 2024-09-10T10:06:24.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.753+0000 7ff492800640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff474068250 0x7ff47406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:24.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.753+0000 7ff492800640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff474068250 0x7ff47406a710 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff4840ae000 tx=0x7ff484008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:24.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.757+0000 7ff48aa00640 1 -- 172.21.15.110:0/3934585104 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff48c056880 con 0x7ff49415b170 2024-09-10T10:06:24.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.941+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol84070", "group_name": "subvol_grp15380", "target": ["mon-mgr", ""]}) -- 0x7ff4940af2f0 con 0x7ff474068250 2024-09-10T10:06:24.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff48aa00640 1 -- 172.21.15.110:0/3934585104 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff4940af2f0 con 0x7ff474068250 2024-09-10T10:06:24.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff474068250 msgr2=0x7ff47406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff474068250 0x7ff47406a710 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff4840ae000 tx=0x7ff484008040 comp rx=0 tx=0).stop 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 msgr2=0x7ff49415a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff49415a640 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00b970 tx=0x7ff48c00be30 comp rx=0 tx=0).stop 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 shutdown_connections 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff474068250 0x7ff47406a710 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff49415b170 0x7ff49415a640 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4940cf2e0 0x7ff49415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 --2- 172.21.15.110:0/3934585104 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff4940b5bf0 0x7ff494149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 >> 172.21.15.110:0/3934585104 conn(0x7ff49406d0f0 msgr2=0x7ff4940ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 shutdown_connections 2024-09-10T10:06:24.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:24.973+0000 7ff498a00640 1 -- 172.21.15.110:0/3934585104 wait complete. 2024-09-10T10:06:24.979 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.977+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.979 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.977+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.979 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.977+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.979 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.977+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.979 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.977+0000 7f95e0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.993 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T10:06:24.997 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.997+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.997 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.997+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.997 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.997+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.997 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.997+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:24.997 INFO:tasks.ceph.mgr.x.smithi145.stderr:2024-09-10T10:06:24.997+0000 7f95e0200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:06:25.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:06:25.058 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-10T10:06:25.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:06:25.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp15380 2024-09-10T10:06:25.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.201+0000 7f57dce00640 1 Processor -- start 2024-09-10T10:06:25.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.201+0000 7f57dce00640 1 -- start start 2024-09-10T10:06:25.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57dce00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f57d80ce4d0 0x7f57d806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57dce00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d80b4960 con 0x7f57d815b330 2024-09-10T10:06:25.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57dce00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d80b4c40 con 0x7f57d806e850 2024-09-10T10:06:25.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57dce00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f57d80b5570 con 0x7f57d80ce4d0 2024-09-10T10:06:25.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d5e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f57d815b330 0x7f57d806e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37120/0 (socket says 172.21.15.110:37120) 2024-09-10T10:06:25.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d5e00640 1 -- 172.21.15.110:0/2636084702 learned_addr learned my addr 172.21.15.110:0/2636084702 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:25.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d5400640 1 --2- 172.21.15.110:0/2636084702 >> v2:172.21.15.145:3300/0 conn(0x7f57d80ce4d0 0x7f57d806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d4a00640 1 -- 172.21.15.110:0/2636084702 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800899187 0 0) 0x7f57d80b4960 con 0x7f57d815b330 2024-09-10T10:06:25.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d4a00640 1 -- 172.21.15.110:0/2636084702 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57bc005740 con 0x7f57d815b330 2024-09-10T10:06:25.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d5400640 1 -- 172.21.15.110:0/2636084702 >> v1:172.21.15.145:6789/0 conn(0x7f57d806e850 legacy=0x7f57d80b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d5400640 1 -- 172.21.15.110:0/2636084702 >> v1:172.21.15.110:6789/0 conn(0x7f57d815b330 legacy=0x7f57d806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d4a00640 1 -- 172.21.15.110:0/2636084702 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1816713300 0 0) 0x7f57bc005740 con 0x7f57d815b330 2024-09-10T10:06:25.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.205+0000 7f57d5400640 1 -- 172.21.15.110:0/2636084702 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d80b4f20 con 0x7f57d80ce4d0 2024-09-10T10:06:25.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57d5400640 1 --2- 172.21.15.110:0/2636084702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d80ce4d0 0x7f57d806c6a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f57c40097c0 tx=0x7f57c401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d375fc06805365 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:25.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57d4a00640 1 -- 172.21.15.110:0/2636084702 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40b0070 con 0x7f57d80ce4d0 2024-09-10T10:06:25.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57d4a00640 1 -- 172.21.15.110:0/2636084702 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c40ab400 con 0x7f57d80ce4d0 2024-09-10T10:06:25.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57d4a00640 1 -- 172.21.15.110:0/2636084702 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8001a00 con 0x7f57d80ce4d0 2024-09-10T10:06:25.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57dce00640 1 -- 172.21.15.110:0/2636084702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d80ce4d0 msgr2=0x7f57d806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57dce00640 1 --2- 172.21.15.110:0/2636084702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d80ce4d0 0x7f57d806c6a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f57c40097c0 tx=0x7f57c401c3c0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57dce00640 1 -- 172.21.15.110:0/2636084702 shutdown_connections 2024-09-10T10:06:25.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57dce00640 1 --2- 172.21.15.110:0/2636084702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d80ce4d0 0x7f57d806c6a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57dce00640 1 -- 172.21.15.110:0/2636084702 >> 172.21.15.110:0/2636084702 conn(0x7f57d8062e40 msgr2=0x7f57d806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:25.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57dce00640 1 -- 172.21.15.110:0/2636084702 shutdown_connections 2024-09-10T10:06:25.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57dce00640 1 -- 172.21.15.110:0/2636084702 wait complete. 2024-09-10T10:06:25.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.209+0000 7f57dce00640 1 Processor -- start 2024-09-10T10:06:25.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 -- start start 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57d806e850 0x7f57d81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57d815b330 0x7f57d81f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d815a0d0 0x7f57d815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f57d80b3b00 con 0x7f57d815b330 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f57d80b3980 con 0x7f57d815a0d0 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f57d80b3c80 con 0x7f57d806e850 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57d815b330 0x7f57d81f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57d815b330 0x7f57d81f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36122/0 (socket says 172.21.15.110:36122) 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 -- 172.21.15.110:0/2789304014 learned_addr learned my addr 172.21.15.110:0/2789304014 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:25.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5e00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57d806e850 0x7f57d81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 -- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57d806e850 msgr2=0x7f57d81543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d6800640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d815a0d0 0x7f57d815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57d806e850 0x7f57d81543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 -- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d815a0d0 msgr2=0x7f57d815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d815a0d0 0x7f57d815a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 -- 172.21.15.110:0/2789304014 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d8159720 con 0x7f57d815b330 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5e00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57d806e850 0x7f57d81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d6800640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d815a0d0 0x7f57d815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:25.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57d5400640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57d815b330 0x7f57d81f7600 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000c10 tx=0x7f57c4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:25.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57ce000640 1 -- 172.21.15.110:0/2789304014 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40b0040 con 0x7f57d815b330 2024-09-10T10:06:25.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57ce000640 1 -- 172.21.15.110:0/2789304014 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c4003720 con 0x7f57d815b330 2024-09-10T10:06:25.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d8156120 con 0x7f57d815b330 2024-09-10T10:06:25.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57ce000640 1 -- 172.21.15.110:0/2789304014 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4003ad0 con 0x7f57d815b330 2024-09-10T10:06:25.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81566e0 con 0x7f57d815b330 2024-09-10T10:06:25.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.213+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d806c6a0 con 0x7f57d815b330 2024-09-10T10:06:25.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.217+0000 7f57ce000640 1 -- 172.21.15.110:0/2789304014 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 88312+0+0 (secure 0 0 0) 0x7f57c40bd020 con 0x7f57d815b330 2024-09-10T10:06:25.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.217+0000 7f57ce000640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f57b8068250 0x7f57b806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.217+0000 7f57ce000640 1 -- 172.21.15.110:0/2789304014 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f57c411dd80 con 0x7f57d815b330 2024-09-10T10:06:25.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.217+0000 7f57d5e00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f57b8068250 0x7f57b806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.217+0000 7f57d5e00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f57b8068250 0x7f57b806a710 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f57c8001710 tx=0x7f57c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:25.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.221+0000 7f57ce000640 1 -- 172.21.15.110:0/2789304014 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f57c4015fb0 con 0x7f57d815b330 2024-09-10T10:06:25.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.397+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp15380", "target": ["mon-mgr", ""]}) -- 0x7f57d81f7b40 con 0x7f57b8068250 2024-09-10T10:06:25.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.401+0000 7f57ce000640 1 -- 172.21.15.110:0/2789304014 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f57d81f7b40 con 0x7f57b8068250 2024-09-10T10:06:25.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f57b8068250 msgr2=0x7f57b806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f57b8068250 0x7f57b806a710 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f57c8001710 tx=0x7f57c8009040 comp rx=0 tx=0).stop 2024-09-10T10:06:25.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57d815b330 msgr2=0x7f57d81f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57d815b330 0x7f57d81f7600 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000c10 tx=0x7f57c4019160 comp rx=0 tx=0).stop 2024-09-10T10:06:25.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 shutdown_connections 2024-09-10T10:06:25.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f57b8068250 0x7f57b806a710 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f57d815a0d0 0x7f57d815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57d815b330 0x7f57d81f7600 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 --2- 172.21.15.110:0/2789304014 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f57d806e850 0x7f57d81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 >> 172.21.15.110:0/2789304014 conn(0x7f57d8062e40 msgr2=0x7f57d80ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:25.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 shutdown_connections 2024-09-10T10:06:25.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.405+0000 7f57dce00640 1 -- 172.21.15.110:0/2789304014 wait complete. 2024-09-10T10:06:25.425 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:06:25.425 DEBUG:teuthology.orchestra.run.smithi110:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:06:25.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:06:25.614 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:06:25.615 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:06:25.615 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:25.615 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:25.623 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:25.624 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:25.672 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:06:25.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f154d800640 1 Processor -- start 2024-09-10T10:06:25.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f154d800640 1 -- start start 2024-09-10T10:06:25.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f154d800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f15480b4ce0 0x7f1548165690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f154d800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154815ae90 con 0x7f15480b4170 2024-09-10T10:06:25.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f154d800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154815b170 con 0x7f154815aab0 2024-09-10T10:06:25.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f154d800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f154815b450 con 0x7f15480b4ce0 2024-09-10T10:06:25.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f1547400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f15480b4ce0 0x7f1548165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f1547400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f15480b4ce0 0x7f1548165690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36124/0 (socket says 172.21.15.110:36124) 2024-09-10T10:06:25.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f1547400640 1 -- 172.21.15.110:0/3819316388 learned_addr learned my addr 172.21.15.110:0/3819316388 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:25.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.852+0000 7f1546000640 1 --1- 172.21.15.110:0/3819316388 >> v1:172.21.15.110:6789/0 conn(0x7f15480b4170 0x7f15480b45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37136/0 (socket says 172.21.15.110:37136) 2024-09-10T10:06:25.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f1547400640 1 -- 172.21.15.110:0/3819316388 >> v1:172.21.15.145:6789/0 conn(0x7f154815aab0 legacy=0x7f1548072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:25.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f1547400640 1 -- 172.21.15.110:0/3819316388 >> v1:172.21.15.110:6789/0 conn(0x7f15480b4170 legacy=0x7f15480b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f1547400640 1 -- 172.21.15.110:0/3819316388 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154815b730 con 0x7f15480b4ce0 2024-09-10T10:06:25.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f1545600640 1 -- 172.21.15.110:0/3819316388 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2360865005 0 0) 0x7f154815ae90 con 0x7f15480b4170 2024-09-10T10:06:25.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f1547400640 1 --2- 172.21.15.110:0/3819316388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4ce0 0x7f1548165690 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1540009e50 tx=0x7f154001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16a1ed6b6780463 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:25.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f1545600640 1 -- 172.21.15.110:0/3819316388 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400a57a0 con 0x7f15480b4ce0 2024-09-10T10:06:25.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f1545600640 1 -- 172.21.15.110:0/3819316388 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15400a5940 con 0x7f15480b4ce0 2024-09-10T10:06:25.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f1545600640 1 -- 172.21.15.110:0/3819316388 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400a5d10 con 0x7f15480b4ce0 2024-09-10T10:06:25.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f154d800640 1 -- 172.21.15.110:0/3819316388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4ce0 msgr2=0x7f1548165690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f154d800640 1 --2- 172.21.15.110:0/3819316388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4ce0 0x7f1548165690 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1540009e50 tx=0x7f154001c9f0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f154d800640 1 -- 172.21.15.110:0/3819316388 shutdown_connections 2024-09-10T10:06:25.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f154d800640 1 --2- 172.21.15.110:0/3819316388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4ce0 0x7f1548165690 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.856+0000 7f154d800640 1 -- 172.21.15.110:0/3819316388 >> 172.21.15.110:0/3819316388 conn(0x7f1548062d10 msgr2=0x7f154806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:25.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 -- 172.21.15.110:0/3819316388 shutdown_connections 2024-09-10T10:06:25.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 -- 172.21.15.110:0/3819316388 wait complete. 2024-09-10T10:06:25.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 Processor -- start 2024-09-10T10:06:25.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 -- start start 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4170 0x7f15481501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15480b4ce0 0x7f1548159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f154815aab0 0x7f1548156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f154806d4c0 con 0x7f15480b4170 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f154806d340 con 0x7f154815aab0 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f154806d640 con 0x7f15480b4ce0 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4170 0x7f15481501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4170 0x7f15481501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36126/0 (socket says 172.21.15.110:36126) 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 -- 172.21.15.110:0/3404686517 learned_addr learned my addr 172.21.15.110:0/3404686517 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1547400640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f154815aab0 0x7f1548156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546000640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15480b4ce0 0x7f1548159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 -- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15480b4ce0 msgr2=0x7f1548159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15480b4ce0 0x7f1548159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 -- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f154815aab0 msgr2=0x7f1548156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f154815aab0 0x7f1548156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 -- 172.21.15.110:0/3404686517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1548151760 con 0x7f15480b4170 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546000640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15480b4ce0 0x7f1548159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f1546a00640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4170 0x7f15481501f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f153800b410 tx=0x7f153800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f153ea00640 1 -- 172.21.15.110:0/3404686517 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538015030 con 0x7f15480b4170 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f153ea00640 1 -- 172.21.15.110:0/3404686517 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1538004020 con 0x7f15480b4170 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f153ea00640 1 -- 172.21.15.110:0/3404686517 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538004450 con 0x7f15480b4170 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1548151ce0 con 0x7f15480b4170 2024-09-10T10:06:25.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.860+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15481522a0 con 0x7f15480b4170 2024-09-10T10:06:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.864+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15480b45d0 con 0x7f15480b4170 2024-09-10T10:06:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.864+0000 7f153ea00640 1 -- 172.21.15.110:0/3404686517 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f1538018070 con 0x7f15480b4170 2024-09-10T10:06:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.864+0000 7f153ea00640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1528068430 0x7f152806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.864+0000 7f153ea00640 1 -- 172.21.15.110:0/3404686517 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f1538084370 con 0x7f15480b4170 2024-09-10T10:06:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.864+0000 7f1546000640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1528068430 0x7f152806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.864+0000 7f1546000640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1528068430 0x7f152806a8f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1534007fd0 tx=0x7f1534007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:25.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:25.868+0000 7f153ea00640 1 -- 172.21.15.110:0/3404686517 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f153808a030 con 0x7f15480b4170 2024-09-10T10:06:26.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.036+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f15480b30a0 con 0x7f15480b4170 2024-09-10T10:06:26.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.036+0000 7f153ea00640 1 -- 172.21.15.110:0/3404686517 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v165) ==== 80+0+0 (secure 0 0 0) 0x7f1538056620 con 0x7f15480b4170 2024-09-10T10:06:26.039 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:06:26.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1528068430 msgr2=0x7f152806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1528068430 0x7f152806a8f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1534007fd0 tx=0x7f1534007c20 comp rx=0 tx=0).stop 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4170 msgr2=0x7f15481501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4170 0x7f15481501f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f153800b410 tx=0x7f153800b8d0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 shutdown_connections 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f1528068430 0x7f152806a8f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f154815aab0 0x7f1548156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f15480b4ce0 0x7f1548159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 --2- 172.21.15.110:0/3404686517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15480b4170 0x7f15481501f0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 >> 172.21.15.110:0/3404686517 conn(0x7f1548062d10 msgr2=0x7f15480b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 shutdown_connections 2024-09-10T10:06:26.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.040+0000 7f154d800640 1 -- 172.21.15.110:0/3404686517 wait complete. 2024-09-10T10:06:26.061 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:06:26.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:26.061 DEBUG:teuthology.orchestra.run.smithi110:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:06:26.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:06:26.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:26.121 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:06:26.136 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:06:26.137 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:06:26.137 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:26.146 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:06:26.146 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:26.196 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:06:26.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f267b600640 1 Processor -- start 2024-09-10T10:06:26.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f267b600640 1 -- start start 2024-09-10T10:06:26.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f267b600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cdcb0 0x7f2674167160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f267b600640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267415d4d0 con 0x7f26740b3f40 2024-09-10T10:06:26.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f267b600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267415d7b0 con 0x7f26740cfb90 2024-09-10T10:06:26.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f267b600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f267406d130 con 0x7f26740cdcb0 2024-09-10T10:06:26.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2673e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f26740b3f40 0x7f26740cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37148/0 (socket says 172.21.15.110:37148) 2024-09-10T10:06:26.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2673e00640 1 -- 172.21.15.110:0/3387742318 learned_addr learned my addr 172.21.15.110:0/3387742318 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:26.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2679600640 1 --2- 172.21.15.110:0/3387742318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cdcb0 0x7f2674167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2673400640 1 -- 172.21.15.110:0/3387742318 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1328292267 0 0) 0x7f267415d4d0 con 0x7f26740b3f40 2024-09-10T10:06:26.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2673400640 1 -- 172.21.15.110:0/3387742318 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f265c003530 con 0x7f26740b3f40 2024-09-10T10:06:26.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2673400640 1 -- 172.21.15.110:0/3387742318 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1534919864 0 0) 0x7f267415d7b0 con 0x7f26740cfb90 2024-09-10T10:06:26.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2673400640 1 -- 172.21.15.110:0/3387742318 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f267415d4d0 con 0x7f26740cfb90 2024-09-10T10:06:26.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2679600640 1 -- 172.21.15.110:0/3387742318 >> v1:172.21.15.145:6789/0 conn(0x7f26740cfb90 legacy=0x7f267415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2679600640 1 -- 172.21.15.110:0/3387742318 >> v1:172.21.15.110:6789/0 conn(0x7f26740b3f40 legacy=0x7f26740cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.332+0000 7f2679600640 1 -- 172.21.15.110:0/3387742318 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267406d380 con 0x7f26740cdcb0 2024-09-10T10:06:26.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f2679600640 1 --2- 172.21.15.110:0/3387742318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cdcb0 0x7f2674167160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f267406d2b0 tx=0x7f266c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35ec0637dd692ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:26.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f2673400640 1 -- 172.21.15.110:0/3387742318 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c0b2070 con 0x7f26740cdcb0 2024-09-10T10:06:26.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f2673400640 1 -- 172.21.15.110:0/3387742318 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266c0a57c0 con 0x7f26740cdcb0 2024-09-10T10:06:26.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f2673400640 1 -- 172.21.15.110:0/3387742318 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c0a5b90 con 0x7f26740cdcb0 2024-09-10T10:06:26.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f267b600640 1 -- 172.21.15.110:0/3387742318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cdcb0 msgr2=0x7f2674167160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f267b600640 1 --2- 172.21.15.110:0/3387742318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cdcb0 0x7f2674167160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f267406d2b0 tx=0x7f266c01c900 comp rx=0 tx=0).stop 2024-09-10T10:06:26.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f267b600640 1 -- 172.21.15.110:0/3387742318 shutdown_connections 2024-09-10T10:06:26.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f267b600640 1 --2- 172.21.15.110:0/3387742318 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cdcb0 0x7f2674167160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f267b600640 1 -- 172.21.15.110:0/3387742318 >> 172.21.15.110:0/3387742318 conn(0x7f2674062d10 msgr2=0x7f267406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:26.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f267b600640 1 -- 172.21.15.110:0/3387742318 shutdown_connections 2024-09-10T10:06:26.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.336+0000 7f267b600640 1 -- 172.21.15.110:0/3387742318 wait complete. 2024-09-10T10:06:26.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 Processor -- start 2024-09-10T10:06:26.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 -- start start 2024-09-10T10:06:26.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26740b3f40 0x7f26741f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26740cdcb0 0x7f26741550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cfb90 0x7f2674155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f267416a250 con 0x7f26740b3f40 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f267416a0d0 con 0x7f26740cdcb0 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f267416a3d0 con 0x7f26740cfb90 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26740b3f40 0x7f26741f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26740b3f40 0x7f26741f5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36136/0 (socket says 172.21.15.110:36136) 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 -- 172.21.15.110:0/798990376 learned_addr learned my addr 172.21.15.110:0/798990376 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2673e00640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26740cdcb0 0x7f26741550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2679600640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cfb90 0x7f2674155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 -- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cfb90 msgr2=0x7f2674155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cfb90 0x7f2674155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 -- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26740cdcb0 msgr2=0x7f26741550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26740cdcb0 0x7f26741550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 -- 172.21.15.110:0/798990376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26741f5e20 con 0x7f26740b3f40 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2673e00640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26740cdcb0 0x7f26741550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2679600640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cfb90 0x7f2674155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2678c00640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26740b3f40 0x7f26741f5330 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f266800c490 tx=0x7f266800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2670c00640 1 -- 172.21.15.110:0/798990376 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26680052d0 con 0x7f26740b3f40 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2670c00640 1 -- 172.21.15.110:0/798990376 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2668005470 con 0x7f26740b3f40 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26741f6100 con 0x7f26740b3f40 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f2670c00640 1 -- 172.21.15.110:0/798990376 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668012870 con 0x7f26740b3f40 2024-09-10T10:06:26.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26741f69a0 con 0x7f26740b3f40 2024-09-10T10:06:26.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.340+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26740cd5a0 con 0x7f26740b3f40 2024-09-10T10:06:26.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.344+0000 7f2670c00640 1 -- 172.21.15.110:0/798990376 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f2668012a10 con 0x7f26740b3f40 2024-09-10T10:06:26.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.344+0000 7f2670c00640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2654068400 0x7f265406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.344+0000 7f2670c00640 1 -- 172.21.15.110:0/798990376 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f2668084550 con 0x7f26740b3f40 2024-09-10T10:06:26.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.344+0000 7f2673e00640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2654068400 0x7f265406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.344+0000 7f2673e00640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2654068400 0x7f265406a8c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2674153d00 tx=0x7f2660009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:26.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.348+0000 7f2670c00640 1 -- 172.21.15.110:0/798990376 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2668056890 con 0x7f26740b3f40 2024-09-10T10:06:26.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.516+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f267415e6d0 con 0x7f26740b3f40 2024-09-10T10:06:26.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.516+0000 7f2670c00640 1 -- 172.21.15.110:0/798990376 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v165) ==== 80+0+0 (secure 0 0 0) 0x7f2668056190 con 0x7f26740b3f40 2024-09-10T10:06:26.520 INFO:teuthology.orchestra.run.smithi110.stderr:listed 0 entries 2024-09-10T10:06:26.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2654068400 msgr2=0x7f265406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2654068400 0x7f265406a8c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2674153d00 tx=0x7f2660009040 comp rx=0 tx=0).stop 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26740b3f40 msgr2=0x7f26741f5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26740b3f40 0x7f26741f5330 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f266800c490 tx=0x7f266800c950 comp rx=0 tx=0).stop 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 shutdown_connections 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2654068400 0x7f265406a8c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f26740cfb90 0x7f2674155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26740cdcb0 0x7f26741550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 --2- 172.21.15.110:0/798990376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26740b3f40 0x7f26741f5330 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 >> 172.21.15.110:0/798990376 conn(0x7f2674062d10 msgr2=0x7f267415ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 shutdown_connections 2024-09-10T10:06:26.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.520+0000 7f267b600640 1 -- 172.21.15.110:0/798990376 wait complete. 2024-09-10T10:06:26.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:06:26.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:26.541 DEBUG:teuthology.orchestra.run.smithi145:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T10:06:26.569 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:06:26.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:26.569 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:06:26.625 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:06:26.625 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:26.625 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:26.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:06:26.634 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:06:26.634 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:06:26.634 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:26.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:06:26.680 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:06:26.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.816+0000 7ff5ee800640 1 Processor -- start 2024-09-10T10:06:26.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.816+0000 7ff5ee800640 1 -- start start 2024-09-10T10:06:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.816+0000 7ff5ee800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff5e815c560 0x7ff5e806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5ee800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5ee800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e806f430 con 0x7ff5e8155970 2024-09-10T10:06:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5ee800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e80b4570 con 0x7ff5e815c560 2024-09-10T10:06:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5ee800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e8155d50 con 0x7ff5e806f010 2024-09-10T10:06:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5dea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff5e815c560 0x7ff5e806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:54984/0 (socket says 172.21.15.110:54984) 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ff5e8155970 0x7ff5e80cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:38260/0 (socket says 172.21.15.110:38260) 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5dea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff5e815c560 0x7ff5e806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.110:42592/0 (socket says 172.21.15.110:42592) 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7e00640 1 -- 172.21.15.110:0/1115534775 learned_addr learned my addr 172.21.15.110:0/1115534775 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7e00640 1 -- 172.21.15.110:0/1115534775 >> v2:172.21.15.145:3300/0 conn(0x7ff5e815c560 msgr2=0x7ff5e806ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7e00640 1 --2- 172.21.15.110:0/1115534775 >> v2:172.21.15.145:3300/0 conn(0x7ff5e815c560 0x7ff5e806ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7e00640 1 -- 172.21.15.110:0/1115534775 >> v1:172.21.15.145:6789/0 conn(0x7ff5e8155970 legacy=0x7ff5e80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7e00640 1 -- 172.21.15.110:0/1115534775 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e80b34d0 con 0x7ff5e806f010 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5dea00640 1 --2- 172.21.15.110:0/1115534775 >> v2:172.21.15.145:3300/0 conn(0x7ff5e815c560 0x7ff5e806ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:26.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e6a00640 1 -- 172.21.15.110:0/1115534775 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367031582 0 0) 0x7ff5e806f430 con 0x7ff5e8155970 2024-09-10T10:06:26.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.820+0000 7ff5e7e00640 1 --2- 172.21.15.110:0/1115534775 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ce420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009870 tx=0x7ff5e001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc08a16f6546e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:26.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5e6a00640 1 -- 172.21.15.110:0/1115534775 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00a95f0 con 0x7ff5e806f010 2024-09-10T10:06:26.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5e6a00640 1 -- 172.21.15.110:0/1115534775 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e00a9790 con 0x7ff5e806f010 2024-09-10T10:06:26.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5e6a00640 1 -- 172.21.15.110:0/1115534775 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00a9b60 con 0x7ff5e806f010 2024-09-10T10:06:26.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- 172.21.15.110:0/1115534775 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 msgr2=0x7ff5e80ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1115534775 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ce420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009870 tx=0x7ff5e001c530 comp rx=0 tx=0).stop 2024-09-10T10:06:26.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- 172.21.15.110:0/1115534775 shutdown_connections 2024-09-10T10:06:26.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1115534775 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ce420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1115534775 >> v2:172.21.15.145:3300/0 conn(0x7ff5e815c560 0x7ff5e806ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- 172.21.15.110:0/1115534775 >> 172.21.15.110:0/1115534775 conn(0x7ff5e806bc50 msgr2=0x7ff5e806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:26.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- 172.21.15.110:0/1115534775 shutdown_connections 2024-09-10T10:06:26.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- 172.21.15.110:0/1115534775 wait complete. 2024-09-10T10:06:26.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 Processor -- start 2024-09-10T10:06:26.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- start start 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5e8155970 0x7ff5e80aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e815c560 0x7ff5e8167030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e80cf170 con 0x7ff5e8155970 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e80b4050 con 0x7ff5e815c560 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5ee800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e80cf2f0 con 0x7ff5e806f010 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5dea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5e8155970 0x7ff5e80aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5dea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5e8155970 0x7ff5e80aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36138/0 (socket says 172.21.15.110:36138) 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5dea00640 1 -- 172.21.15.110:0/1836559056 learned_addr learned my addr 172.21.15.110:0/1836559056 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:26.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5e7400640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.824+0000 7ff5e7e00640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e815c560 0x7ff5e8167030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dea00640 1 -- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 msgr2=0x7ff5e80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dea00640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dea00640 1 -- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e815c560 msgr2=0x7ff5e8167030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dea00640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e815c560 0x7ff5e8167030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dea00640 1 -- 172.21.15.110:0/1836559056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e8150f10 con 0x7ff5e8155970 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dea00640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5e8155970 0x7ff5e80aed80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000e980 tx=0x7ff5d000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dfe00640 1 -- 172.21.15.110:0/1836559056 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0017070 con 0x7ff5e8155970 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dfe00640 1 -- 172.21.15.110:0/1836559056 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d000ca00 con 0x7ff5e8155970 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e81511f0 con 0x7ff5e8155970 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dfe00640 1 -- 172.21.15.110:0/1836559056 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000cdd0 con 0x7ff5e8155970 2024-09-10T10:06:26.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e81517b0 con 0x7ff5e8155970 2024-09-10T10:06:26.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e80ce420 con 0x7ff5e8155970 2024-09-10T10:06:26.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dfe00640 1 -- 172.21.15.110:0/1836559056 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7ff5d001a070 con 0x7ff5e8155970 2024-09-10T10:06:26.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dfe00640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5c4068400 0x7ff5c406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:26.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5dfe00640 1 -- 172.21.15.110:0/1836559056 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7ff5d0084740 con 0x7ff5e8155970 2024-09-10T10:06:26.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.828+0000 7ff5e7400640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5c4068400 0x7ff5c406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:26.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.832+0000 7ff5e7400640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5c4068400 0x7ff5c406a8c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80012f0 tx=0x7ff5d8004d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:26.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:26.832+0000 7ff5dfe00640 1 -- 172.21.15.110:0/1836559056 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff5d0056430 con 0x7ff5e8155970 2024-09-10T10:06:27.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.024+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5e8152420 con 0x7ff5e8155970 2024-09-10T10:06:27.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.024+0000 7ff5dfe00640 1 -- 172.21.15.110:0/1836559056 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5242 (secure 0 0 0) 0x7ff5d005a730 con 0x7ff5e8155970 2024-09-10T10:06:27.028 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:27.028 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":116,"btime":"2024-09-10T10:06:22:769093+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1276880772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1276880772},{"type":"v1","addr":"172.21.15.145:6834","nonce":1276880772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2843226954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2843226954},{"type":"v1","addr":"172.21.15.110:6834","nonce":2843226954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8877,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/878298806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":878298806},{"type":"v1","addr":"172.21.15.110:6835","nonce":878298806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:06:13.100588+0000","modified":"2024-09-10T10:06:16.117620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/4014830481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4014830481},{"type":"v1","addr":"172.21.15.145:6835","nonce":4014830481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8883,"qdb_cluster":[8883]},"id":13}]} 2024-09-10T10:06:27.029 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 116 2024-09-10T10:06:27.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5c4068400 msgr2=0x7ff5c406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5c4068400 0x7ff5c406a8c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80012f0 tx=0x7ff5d8004d50 comp rx=0 tx=0).stop 2024-09-10T10:06:27.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5e8155970 msgr2=0x7ff5e80aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5e8155970 0x7ff5e80aed80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000e980 tx=0x7ff5d000ee40 comp rx=0 tx=0).stop 2024-09-10T10:06:27.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 shutdown_connections 2024-09-10T10:06:27.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7ff5c4068400 0x7ff5c406a8c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5e815c560 0x7ff5e8167030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5e8155970 0x7ff5e80aed80 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 --2- 172.21.15.110:0/1836559056 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7ff5e806f010 0x7ff5e80ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 >> 172.21.15.110:0/1836559056 conn(0x7ff5e806bc50 msgr2=0x7ff5e806d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:27.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 shutdown_connections 2024-09-10T10:06:27.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.028+0000 7ff5ee800640 1 -- 172.21.15.110:0/1836559056 wait complete. 2024-09-10T10:06:27.050 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:06:27.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.184+0000 7f2b20400640 1 Processor -- start 2024-09-10T10:06:27.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.184+0000 7f2b20400640 1 -- start start 2024-09-10T10:06:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.184+0000 7f2b20400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2b180cfe30 0x7f2b18071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.184+0000 7f2b20400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2b180cd510 0x7f2b180cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.184+0000 7f2b20400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.184+0000 7f2b20400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2b1815d9a0 con 0x7f2b180cd510 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.184+0000 7f2b20400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2b1806ea60 con 0x7f2b180cfe30 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.184+0000 7f2b20400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1806d2d0 con 0x7f2b1806e090 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2b180cd510 0x7f2b180cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2b180cd510 0x7f2b180cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36152/0 (socket says 172.21.15.110:36152) 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 -- 172.21.15.110:0/2525169516 learned_addr learned my addr 172.21.15.110:0/2525169516 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1e400640 1 --2- 172.21.15.110:0/2525169516 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1da00640 1 --2- 172.21.15.110:0/2525169516 >> v2:172.21.15.145:3300/0 conn(0x7f2b180cfe30 0x7f2b18071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 -- 172.21.15.110:0/2525169516 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 msgr2=0x7f2b1806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 --2- 172.21.15.110:0/2525169516 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1806e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 -- 172.21.15.110:0/2525169516 >> v2:172.21.15.145:3300/0 conn(0x7f2b180cfe30 msgr2=0x7f2b18071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:27.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 --2- 172.21.15.110:0/2525169516 >> v2:172.21.15.145:3300/0 conn(0x7f2b180cfe30 0x7f2b18071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 -- 172.21.15.110:0/2525169516 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1806d5b0 con 0x7f2b180cd510 2024-09-10T10:06:27.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b1d000640 1 --2- 172.21.15.110:0/2525169516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cd510 0x7f2b180cd9a0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2b180ce120 tx=0x7f2b0401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b28cabdb99b60f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b17e00640 1 -- 172.21.15.110:0/2525169516 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b040b2070 con 0x7f2b180cd510 2024-09-10T10:06:27.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b17e00640 1 -- 172.21.15.110:0/2525169516 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b040a9070 con 0x7f2b180cd510 2024-09-10T10:06:27.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.188+0000 7f2b17e00640 1 -- 172.21.15.110:0/2525169516 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b040ad880 con 0x7f2b180cd510 2024-09-10T10:06:27.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- 172.21.15.110:0/2525169516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cd510 msgr2=0x7f2b180cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 --2- 172.21.15.110:0/2525169516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cd510 0x7f2b180cd9a0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2b180ce120 tx=0x7f2b0401c490 comp rx=0 tx=0).stop 2024-09-10T10:06:27.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- 172.21.15.110:0/2525169516 shutdown_connections 2024-09-10T10:06:27.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 --2- 172.21.15.110:0/2525169516 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1806e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 --2- 172.21.15.110:0/2525169516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cd510 0x7f2b180cd9a0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 --2- 172.21.15.110:0/2525169516 >> v2:172.21.15.145:3300/0 conn(0x7f2b180cfe30 0x7f2b18071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- 172.21.15.110:0/2525169516 >> 172.21.15.110:0/2525169516 conn(0x7f2b1806bc50 msgr2=0x7f2b1806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:27.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- 172.21.15.110:0/2525169516 shutdown_connections 2024-09-10T10:06:27.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- 172.21.15.110:0/2525169516 wait complete. 2024-09-10T10:06:27.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 Processor -- start 2024-09-10T10:06:27.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- start start 2024-09-10T10:06:27.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1813eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b180cd510 0x7f2b180b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cfe30 0x7f2b180b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18072190 con 0x7f2b180cfe30 2024-09-10T10:06:27.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1815d680 con 0x7f2b180cd510 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b20400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2b18160010 con 0x7f2b1806e090 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cfe30 0x7f2b180b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cfe30 0x7f2b180b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36168/0 (socket says 172.21.15.110:36168) 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 -- 172.21.15.110:0/478986499 learned_addr learned my addr 172.21.15.110:0/478986499 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1da00640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1813eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1d000640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b180cd510 0x7f2b180b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 -- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 msgr2=0x7f2b1813eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1813eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 -- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b180cd510 msgr2=0x7f2b180b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b180cd510 0x7f2b180b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 -- 172.21.15.110:0/478986499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1815a0d0 con 0x7f2b180cfe30 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1da00640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1813eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:27.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.192+0000 7f2b1e400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cfe30 0x7f2b180b22e0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000e850 tx=0x7f2b1000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b15600640 1 -- 172.21.15.110:0/478986499 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10017070 con 0x7f2b180cfe30 2024-09-10T10:06:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b15600640 1 -- 172.21.15.110:0/478986499 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1000c940 con 0x7f2b180cfe30 2024-09-10T10:06:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b15600640 1 -- 172.21.15.110:0/478986499 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000cd10 con 0x7f2b180cfe30 2024-09-10T10:06:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b18157180 con 0x7f2b180cfe30 2024-09-10T10:06:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b18157740 con 0x7f2b180cfe30 2024-09-10T10:06:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b18166410 con 0x7f2b180cfe30 2024-09-10T10:06:27.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b15600640 1 -- 172.21.15.110:0/478986499 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f2b1001a070 con 0x7f2b180cfe30 2024-09-10T10:06:27.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b15600640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2af8068400 0x7f2af806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b15600640 1 -- 172.21.15.110:0/478986499 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f2b10084840 con 0x7f2b180cfe30 2024-09-10T10:06:27.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.196+0000 7f2b1da00640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2af8068400 0x7f2af806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.200+0000 7f2b1da00640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2af8068400 0x7f2af806a8c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009fd0 tx=0x7f2b0c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.200+0000 7f2b15600640 1 -- 172.21.15.110:0/478986499 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2b10056b80 con 0x7f2b180cfe30 2024-09-10T10:06:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.388+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b180719c0 con 0x7f2b180cfe30 2024-09-10T10:06:27.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.388+0000 7f2b15600640 1 -- 172.21.15.110:0/478986499 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5242 (secure 0 0 0) 0x7f2b10056480 con 0x7f2b180cfe30 2024-09-10T10:06:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:27.392 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":116,"btime":"2024-09-10T10:06:22:769093+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1276880772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1276880772},{"type":"v1","addr":"172.21.15.145:6834","nonce":1276880772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2843226954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2843226954},{"type":"v1","addr":"172.21.15.110:6834","nonce":2843226954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8877,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/878298806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":878298806},{"type":"v1","addr":"172.21.15.110:6835","nonce":878298806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:06:13.100588+0000","modified":"2024-09-10T10:06:16.117620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/4014830481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4014830481},{"type":"v1","addr":"172.21.15.145:6835","nonce":4014830481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8883,"qdb_cluster":[8883]},"id":13}]} 2024-09-10T10:06:27.392 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 116 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2af8068400 msgr2=0x7f2af806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2af8068400 0x7f2af806a8c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009fd0 tx=0x7f2b0c00b040 comp rx=0 tx=0).stop 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cfe30 msgr2=0x7f2b180b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cfe30 0x7f2b180b22e0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000e850 tx=0x7f2b1000ed10 comp rx=0 tx=0).stop 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 shutdown_connections 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2af8068400 0x7f2af806a8c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2b180cfe30 0x7f2b180b22e0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b180cd510 0x7f2b180b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 --2- 172.21.15.110:0/478986499 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2b1806e090 0x7f2b1813eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 >> 172.21.15.110:0/478986499 conn(0x7f2b1806bc50 msgr2=0x7f2b180b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:27.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 shutdown_connections 2024-09-10T10:06:27.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.392+0000 7f2b20400640 1 -- 172.21.15.110:0/478986499 wait complete. 2024-09-10T10:06:27.414 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:06:27.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e68200640 1 Processor -- start 2024-09-10T10:06:27.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e68200640 1 -- start start 2024-09-10T10:06:27.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e68200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5e600aef50 0x7f5e6006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e68200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e6006c710 con 0x7f5e6006bc50 2024-09-10T10:06:27.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e68200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e600b4680 con 0x7f5e6015e690 2024-09-10T10:06:27.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e68200640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f5e600b4960 con 0x7f5e600aef50 2024-09-10T10:06:27.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e66200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5e600aef50 0x7f5e6006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e66200640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5e600aef50 0x7f5e6006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36182/0 (socket says 172.21.15.110:36182) 2024-09-10T10:06:27.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e64e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5e6006bc50 0x7f5e600ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37156/0 (socket says 172.21.15.110:37156) 2024-09-10T10:06:27.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e66200640 1 -- 172.21.15.110:0/1017202831 learned_addr learned my addr 172.21.15.110:0/1017202831 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:27.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e66200640 1 -- 172.21.15.110:0/1017202831 >> v1:172.21.15.145:6789/0 conn(0x7f5e6015e690 legacy=0x7f5e6006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e66200640 1 -- 172.21.15.110:0/1017202831 >> v1:172.21.15.110:6789/0 conn(0x7f5e6006bc50 legacy=0x7f5e600ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.548+0000 7f5e66200640 1 -- 172.21.15.110:0/1017202831 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e600b4c40 con 0x7f5e600aef50 2024-09-10T10:06:27.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e66200640 1 --2- 172.21.15.110:0/1017202831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6006c090 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009870 tx=0x7f5e5801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e734d3eede1a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e5fe00640 1 -- 172.21.15.110:0/1017202831 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580a9610 con 0x7f5e600aef50 2024-09-10T10:06:27.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e5fe00640 1 -- 172.21.15.110:0/1017202831 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e580a97b0 con 0x7f5e600aef50 2024-09-10T10:06:27.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e5fe00640 1 -- 172.21.15.110:0/1017202831 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580a9b80 con 0x7f5e600aef50 2024-09-10T10:06:27.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e68200640 1 -- 172.21.15.110:0/1017202831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 msgr2=0x7f5e6006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e68200640 1 --2- 172.21.15.110:0/1017202831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6006c090 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009870 tx=0x7f5e5801c530 comp rx=0 tx=0).stop 2024-09-10T10:06:27.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e68200640 1 -- 172.21.15.110:0/1017202831 shutdown_connections 2024-09-10T10:06:27.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e68200640 1 --2- 172.21.15.110:0/1017202831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6006c090 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e68200640 1 -- 172.21.15.110:0/1017202831 >> 172.21.15.110:0/1017202831 conn(0x7f5e600d1530 msgr2=0x7f5e600d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:27.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e68200640 1 -- 172.21.15.110:0/1017202831 shutdown_connections 2024-09-10T10:06:27.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.552+0000 7f5e68200640 1 -- 172.21.15.110:0/1017202831 wait complete. 2024-09-10T10:06:27.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 Processor -- start 2024-09-10T10:06:27.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 -- start start 2024-09-10T10:06:27.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e6006bc50 0x7f5e6014e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6013ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e6015e690 0x7f5e6014dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e600b5dd0 con 0x7f5e600aef50 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5e600b5c50 con 0x7f5e6006bc50 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f5e600b5f50 con 0x7f5e6015e690 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6013ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6013ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36184/0 (socket says 172.21.15.110:36184) 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 -- 172.21.15.110:0/2886742630 learned_addr learned my addr 172.21.15.110:0/2886742630 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e65800640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e6006bc50 0x7f5e6014e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e66200640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e6015e690 0x7f5e6014dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 -- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e6015e690 msgr2=0x7f5e6014dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e6015e690 0x7f5e6014dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 -- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e6006bc50 msgr2=0x7f5e6014e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e6006bc50 0x7f5e6014e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 -- 172.21.15.110:0/2886742630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e60161c50 con 0x7f5e600aef50 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e66200640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e6015e690 0x7f5e6014dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e64e00640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6013ada0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00c460 tx=0x7f5e4c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e5d600640 1 -- 172.21.15.110:0/2886742630 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c018070 con 0x7f5e600aef50 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e5d600640 1 -- 172.21.15.110:0/2886742630 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e4c007480 con 0x7f5e600aef50 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e5d600640 1 -- 172.21.15.110:0/2886742630 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0047f0 con 0x7f5e600aef50 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e60161970 con 0x7f5e600aef50 2024-09-10T10:06:27.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e60152290 con 0x7f5e600aef50 2024-09-10T10:06:27.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.556+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6006c090 con 0x7f5e600aef50 2024-09-10T10:06:27.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.560+0000 7f5e5d600640 1 -- 172.21.15.110:0/2886742630 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f5e4c0049b0 con 0x7f5e600aef50 2024-09-10T10:06:27.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.560+0000 7f5e5d600640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e40068400 0x7f5e4006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.560+0000 7f5e5d600640 1 -- 172.21.15.110:0/2886742630 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f5e4c0839f0 con 0x7f5e600aef50 2024-09-10T10:06:27.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.560+0000 7f5e65800640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e40068400 0x7f5e4006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.560+0000 7f5e65800640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e40068400 0x7f5e4006a8c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5e54001d50 tx=0x7f5e54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.564+0000 7f5e5d600640 1 -- 172.21.15.110:0/2886742630 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5e4c055cf0 con 0x7f5e600aef50 2024-09-10T10:06:27.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e60072fc0 con 0x7f5e600aef50 2024-09-10T10:06:27.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e5d600640 1 -- 172.21.15.110:0/2886742630 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+16868 (secure 0 0 0) 0x7f5e4c0555f0 con 0x7f5e600aef50 2024-09-10T10:06:27.726 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:27.726 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":165,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:06:14.723170+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:06:10.807237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T10:06:12.100049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:06:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e40068400 msgr2=0x7f5e4006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e40068400 0x7f5e4006a8c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5e54001d50 tx=0x7f5e54009040 comp rx=0 tx=0).stop 2024-09-10T10:06:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 msgr2=0x7f5e6013ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6013ada0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00c460 tx=0x7f5e4c00c920 comp rx=0 tx=0).stop 2024-09-10T10:06:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 shutdown_connections 2024-09-10T10:06:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f5e40068400 0x7f5e4006a8c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f5e6015e690 0x7f5e6014dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e600aef50 0x7f5e6013ada0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 --2- 172.21.15.110:0/2886742630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5e6006bc50 0x7f5e6014e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.724+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 >> 172.21.15.110:0/2886742630 conn(0x7f5e600d1530 msgr2=0x7f5e6006e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:27.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.728+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 shutdown_connections 2024-09-10T10:06:27.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.728+0000 7f5e68200640 1 -- 172.21.15.110:0/2886742630 wait complete. 2024-09-10T10:06:27.749 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:06:27.749 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:06:27.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.884+0000 7f6452400640 1 Processor -- start 2024-09-10T10:06:27.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.884+0000 7f6452400640 1 -- start start 2024-09-10T10:06:27.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.884+0000 7f6452400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f644c15c560 0x7f644c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.884+0000 7f6452400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f644c06f010 0x7f644c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f6452400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644c06f430 con 0x7f644c155970 2024-09-10T10:06:27.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f6452400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f644c0b4570 con 0x7f644c15c560 2024-09-10T10:06:27.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f6452400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f644c155d50 con 0x7f644c06f010 2024-09-10T10:06:27.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f644c15c560 0x7f644c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644b400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f644c155970 0x7f644c0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:37172/0 (socket says 172.21.15.110:37172) 2024-09-10T10:06:27.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644b400640 1 -- 172.21.15.110:0/1610754550 learned_addr learned my addr 172.21.15.110:0/1610754550 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:27.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644be00640 1 --2- 172.21.15.110:0/1610754550 >> v2:172.21.15.145:3300/0 conn(0x7f644c06f010 0x7f644c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644aa00640 1 -- 172.21.15.110:0/1610754550 >> v2:172.21.15.145:3300/0 conn(0x7f644c06f010 msgr2=0x7f644c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:27.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644aa00640 1 --2- 172.21.15.110:0/1610754550 >> v2:172.21.15.145:3300/0 conn(0x7f644c06f010 0x7f644c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644aa00640 1 -- 172.21.15.110:0/1610754550 >> v1:172.21.15.110:6789/0 conn(0x7f644c155970 legacy=0x7f644c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644aa00640 1 -- 172.21.15.110:0/1610754550 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c0b34d0 con 0x7f644c15c560 2024-09-10T10:06:27.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644a000640 1 -- 172.21.15.110:0/1610754550 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316733103 0 0) 0x7f644c06f430 con 0x7f644c155970 2024-09-10T10:06:27.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644be00640 1 --2- 172.21.15.110:0/1610754550 >> v2:172.21.15.145:3300/0 conn(0x7f644c06f010 0x7f644c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:27.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644aa00640 1 --2- 172.21.15.110:0/1610754550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c15c560 0x7f644c06ead0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f644c0735e0 tx=0x7f643801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82808f120261e29c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644a000640 1 -- 172.21.15.110:0/1610754550 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64380ab650 con 0x7f644c15c560 2024-09-10T10:06:27.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644a000640 1 -- 172.21.15.110:0/1610754550 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64380b0070 con 0x7f644c15c560 2024-09-10T10:06:27.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.888+0000 7f644a000640 1 -- 172.21.15.110:0/1610754550 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64380aba10 con 0x7f644c15c560 2024-09-10T10:06:27.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- 172.21.15.110:0/1610754550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c15c560 msgr2=0x7f644c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 --2- 172.21.15.110:0/1610754550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c15c560 0x7f644c06ead0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f644c0735e0 tx=0x7f643801c450 comp rx=0 tx=0).stop 2024-09-10T10:06:27.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- 172.21.15.110:0/1610754550 shutdown_connections 2024-09-10T10:06:27.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 --2- 172.21.15.110:0/1610754550 >> v2:172.21.15.145:3300/0 conn(0x7f644c06f010 0x7f644c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 --2- 172.21.15.110:0/1610754550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c15c560 0x7f644c06ead0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- 172.21.15.110:0/1610754550 >> 172.21.15.110:0/1610754550 conn(0x7f644c06bc50 msgr2=0x7f644c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:27.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- 172.21.15.110:0/1610754550 shutdown_connections 2024-09-10T10:06:27.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- 172.21.15.110:0/1610754550 wait complete. 2024-09-10T10:06:27.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 Processor -- start 2024-09-10T10:06:27.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- start start 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f644c06f010 0x7f644c152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c155970 0x7f644c0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f644c15c560 0x7f644c0aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f644c0cf170 con 0x7f644c155970 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f644c0b4050 con 0x7f644c15c560 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f6452400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f644c0cf2f0 con 0x7f644c06f010 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c155970 0x7f644c0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644aa00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c155970 0x7f644c0ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36202/0 (socket says 172.21.15.110:36202) 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644aa00640 1 -- 172.21.15.110:0/3141875505 learned_addr learned my addr 172.21.15.110:0/3141875505 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644be00640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f644c15c560 0x7f644c0aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644b400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f644c06f010 0x7f644c152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644aa00640 1 -- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f644c06f010 msgr2=0x7f644c152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644aa00640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f644c06f010 0x7f644c152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644aa00640 1 -- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f644c15c560 msgr2=0x7f644c0aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:27.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644aa00640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f644c15c560 0x7f644c0aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:27.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644aa00640 1 -- 172.21.15.110:0/3141875505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c153500 con 0x7f644c155970 2024-09-10T10:06:27.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.892+0000 7f644be00640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f644c15c560 0x7f644c0aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:27.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f644aa00640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c155970 0x7f644c0ae840 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f64380ad040 tx=0x7f643801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6443400640 1 -- 172.21.15.110:0/3141875505 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438004220 con 0x7f644c155970 2024-09-10T10:06:27.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6443400640 1 -- 172.21.15.110:0/3141875505 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64380043c0 con 0x7f644c155970 2024-09-10T10:06:27.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6443400640 1 -- 172.21.15.110:0/3141875505 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438003820 con 0x7f644c155970 2024-09-10T10:06:27.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c15acf0 con 0x7f644c155970 2024-09-10T10:06:27.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c15b2b0 con 0x7f644c155970 2024-09-10T10:06:27.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c15b4f0 con 0x7f644c155970 2024-09-10T10:06:27.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6443400640 1 -- 172.21.15.110:0/3141875505 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f64380039c0 con 0x7f644c155970 2024-09-10T10:06:27.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6443400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f642c068400 0x7f642c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:27.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f6443400640 1 -- 172.21.15.110:0/3141875505 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f643811f030 con 0x7f644c155970 2024-09-10T10:06:27.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.896+0000 7f644b400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f642c068400 0x7f642c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:27.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.900+0000 7f644b400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f642c068400 0x7f642c06a8c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f643c007fd0 tx=0x7f643c007ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:27.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:27.900+0000 7f6443400640 1 -- 172.21.15.110:0/3141875505 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6438016070 con 0x7f644c155970 2024-09-10T10:06:28.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f644c069bb0 con 0x7f644c155970 2024-09-10T10:06:28.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6443400640 1 -- 172.21.15.110:0/3141875505 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5242 (secure 0 0 0) 0x7f64380f15c0 con 0x7f644c155970 2024-09-10T10:06:28.094 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:28.094 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":116,"btime":"2024-09-10T10:06:22:769093+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1276880772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1276880772},{"type":"v1","addr":"172.21.15.145:6834","nonce":1276880772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2843226954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2843226954},{"type":"v1","addr":"172.21.15.110:6834","nonce":2843226954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8877,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/878298806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":878298806},{"type":"v1","addr":"172.21.15.110:6835","nonce":878298806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:06:13.100588+0000","modified":"2024-09-10T10:06:16.117620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/4014830481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4014830481},{"type":"v1","addr":"172.21.15.145:6835","nonce":4014830481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8883,"qdb_cluster":[8883]},"id":13}]} 2024-09-10T10:06:28.094 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 116 2024-09-10T10:06:28.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f642c068400 msgr2=0x7f642c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f642c068400 0x7f642c06a8c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f643c007fd0 tx=0x7f643c007ee0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c155970 msgr2=0x7f644c0ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c155970 0x7f644c0ae840 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f64380ad040 tx=0x7f643801ac90 comp rx=0 tx=0).stop 2024-09-10T10:06:28.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 shutdown_connections 2024-09-10T10:06:28.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f642c068400 0x7f642c06a8c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f644c15c560 0x7f644c0aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f644c155970 0x7f644c0ae840 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 --2- 172.21.15.110:0/3141875505 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f644c06f010 0x7f644c152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 >> 172.21.15.110:0/3141875505 conn(0x7f644c06bc50 msgr2=0x7f644c0710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:28.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 shutdown_connections 2024-09-10T10:06:28.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.092+0000 7f6452400640 1 -- 172.21.15.110:0/3141875505 wait complete. 2024-09-10T10:06:28.115 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:06:28.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9404000640 1 Processor -- start 2024-09-10T10:06:28.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9404000640 1 -- start start 2024-09-10T10:06:28.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9404000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f93fc06f1e0 0x7f93fc166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9404000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93fc155d50 con 0x7f93fc155970 2024-09-10T10:06:28.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9404000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93fc0b34d0 con 0x7f93fc15c560 2024-09-10T10:06:28.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9404000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f93fc0b4570 con 0x7f93fc06f1e0 2024-09-10T10:06:28.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9402000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f93fc06f1e0 0x7f93fc166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9401600640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f93fc155970 0x7f93fc0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41274/0 (socket says 172.21.15.110:41274) 2024-09-10T10:06:28.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9402000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f93fc06f1e0 0x7f93fc166020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51412/0 (socket says 172.21.15.110:51412) 2024-09-10T10:06:28.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.248+0000 7f9402000640 1 -- 172.21.15.110:0/1047870290 learned_addr learned my addr 172.21.15.110:0/1047870290 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:28.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9402000640 1 -- 172.21.15.110:0/1047870290 >> v1:172.21.15.145:6789/0 conn(0x7f93fc15c560 legacy=0x7f93fc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9402000640 1 -- 172.21.15.110:0/1047870290 >> v1:172.21.15.110:6789/0 conn(0x7f93fc155970 legacy=0x7f93fc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9402000640 1 -- 172.21.15.110:0/1047870290 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc0b37b0 con 0x7f93fc06f1e0 2024-09-10T10:06:28.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9402000640 1 --2- 172.21.15.110:0/1047870290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc166020 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009e50 tx=0x7f93f401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ce816fcfef2860 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f93fbe00640 1 -- 172.21.15.110:0/1047870290 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40a79e0 con 0x7f93fc06f1e0 2024-09-10T10:06:28.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f93fbe00640 1 -- 172.21.15.110:0/1047870290 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f40a7b80 con 0x7f93fc06f1e0 2024-09-10T10:06:28.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f93fbe00640 1 -- 172.21.15.110:0/1047870290 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40af3f0 con 0x7f93fc06f1e0 2024-09-10T10:06:28.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9404000640 1 -- 172.21.15.110:0/1047870290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 msgr2=0x7f93fc166020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9404000640 1 --2- 172.21.15.110:0/1047870290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc166020 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009e50 tx=0x7f93f401ec10 comp rx=0 tx=0).stop 2024-09-10T10:06:28.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9404000640 1 -- 172.21.15.110:0/1047870290 shutdown_connections 2024-09-10T10:06:28.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9404000640 1 --2- 172.21.15.110:0/1047870290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc166020 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.252+0000 7f9404000640 1 -- 172.21.15.110:0/1047870290 >> 172.21.15.110:0/1047870290 conn(0x7f93fc06bc50 msgr2=0x7f93fc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:28.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 -- 172.21.15.110:0/1047870290 shutdown_connections 2024-09-10T10:06:28.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 -- 172.21.15.110:0/1047870290 wait complete. 2024-09-10T10:06:28.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 Processor -- start 2024-09-10T10:06:28.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 -- start start 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93fc155970 0x7f93fc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93fc15c560 0x7f93fc1fa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc06e350 con 0x7f93fc06f1e0 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc06e1d0 con 0x7f93fc15c560 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f93fc06e4d0 con 0x7f93fc155970 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51418/0 (socket says 172.21.15.110:51418) 2024-09-10T10:06:28.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 -- 172.21.15.110:0/3827662756 learned_addr learned my addr 172.21.15.110:0/3827662756 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9402000640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93fc15c560 0x7f93fc1fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9400c00640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93fc155970 0x7f93fc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 -- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93fc155970 msgr2=0x7f93fc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93fc155970 0x7f93fc1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 -- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93fc15c560 msgr2=0x7f93fc1fa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93fc15c560 0x7f93fc1fa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 -- 172.21.15.110:0/3827662756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc156cf0 con 0x7f93fc06f1e0 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9402000640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93fc15c560 0x7f93fc1fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9400c00640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93fc155970 0x7f93fc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9401600640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc15e8a0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f93f000c490 tx=0x7f93f000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f93f9600640 1 -- 172.21.15.110:0/3827662756 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0018070 con 0x7f93fc06f1e0 2024-09-10T10:06:28.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f93f9600640 1 -- 172.21.15.110:0/3827662756 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f0007480 con 0x7f93fc06f1e0 2024-09-10T10:06:28.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc156fd0 con 0x7f93fc06f1e0 2024-09-10T10:06:28.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f93f9600640 1 -- 172.21.15.110:0/3827662756 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0004a10 con 0x7f93fc06f1e0 2024-09-10T10:06:28.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.256+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc157590 con 0x7f93fc06f1e0 2024-09-10T10:06:28.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.260+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93fc0cc8c0 con 0x7f93fc06f1e0 2024-09-10T10:06:28.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.260+0000 7f93f9600640 1 -- 172.21.15.110:0/3827662756 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f93f001b070 con 0x7f93fc06f1e0 2024-09-10T10:06:28.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.260+0000 7f93f9600640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f93dc068400 0x7f93dc06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.260+0000 7f93f9600640 1 -- 172.21.15.110:0/3827662756 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f93f0083b10 con 0x7f93fc06f1e0 2024-09-10T10:06:28.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.260+0000 7f9400c00640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f93dc068400 0x7f93dc06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.260+0000 7f9400c00640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f93dc068400 0x7f93dc06a8c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f93fc072f90 tx=0x7f93e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.264+0000 7f93f9600640 1 -- 172.21.15.110:0/3827662756 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f93f0055e50 con 0x7f93fc06f1e0 2024-09-10T10:06:28.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93fc155350 con 0x7f93fc06f1e0 2024-09-10T10:06:28.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f93f9600640 1 -- 172.21.15.110:0/3827662756 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5242 (secure 0 0 0) 0x7f93f0055750 con 0x7f93fc06f1e0 2024-09-10T10:06:28.457 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:28.457 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":116,"btime":"2024-09-10T10:06:22:769093+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1276880772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1276880772},{"type":"v1","addr":"172.21.15.145:6834","nonce":1276880772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2843226954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2843226954},{"type":"v1","addr":"172.21.15.110:6834","nonce":2843226954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8877,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/878298806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":878298806},{"type":"v1","addr":"172.21.15.110:6835","nonce":878298806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:06:13.100588+0000","modified":"2024-09-10T10:06:16.117620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"b","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6835/4014830481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":4014830481},{"type":"v1","addr":"172.21.15.145:6835","nonce":4014830481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8883,"qdb_cluster":[8883]},"id":13}]} 2024-09-10T10:06:28.457 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 116 2024-09-10T10:06:28.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f93dc068400 msgr2=0x7f93dc06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f93dc068400 0x7f93dc06a8c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f93fc072f90 tx=0x7f93e8009040 comp rx=0 tx=0).stop 2024-09-10T10:06:28.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 msgr2=0x7f93fc15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc15e8a0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f93f000c490 tx=0x7f93f000c950 comp rx=0 tx=0).stop 2024-09-10T10:06:28.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 shutdown_connections 2024-09-10T10:06:28.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f93dc068400 0x7f93dc06a8c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93fc15c560 0x7f93fc1fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f93fc155970 0x7f93fc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 --2- 172.21.15.110:0/3827662756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc06f1e0 0x7f93fc15e8a0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 >> 172.21.15.110:0/3827662756 conn(0x7f93fc06bc50 msgr2=0x7f93fc0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:28.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 shutdown_connections 2024-09-10T10:06:28.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.456+0000 7f9404000640 1 -- 172.21.15.110:0/3827662756 wait complete. 2024-09-10T10:06:28.477 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:06:28.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe6a0c00640 1 Processor -- start 2024-09-10T10:06:28.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe6a0c00640 1 -- start start 2024-09-10T10:06:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe6a0c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe6a0c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69c0714b0 con 0x7fe69c15e710 2024-09-10T10:06:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe6a0c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69c071790 con 0x7fe69c0d13d0 2024-09-10T10:06:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe6a0c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe69c06d6b0 con 0x7fe69c0b5590 2024-09-10T10:06:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe699e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe69c15e710 0x7fe69c070a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41284/0 (socket says 172.21.15.110:41284) 2024-09-10T10:06:28.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe699e00640 1 -- 172.21.15.110:0/2640957329 learned_addr learned my addr 172.21.15.110:0/2640957329 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:28.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe691400640 1 --2- 172.21.15.110:0/2640957329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe699400640 1 -- 172.21.15.110:0/2640957329 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2316901472 0 0) 0x7fe69c0714b0 con 0x7fe69c15e710 2024-09-10T10:06:28.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe699400640 1 -- 172.21.15.110:0/2640957329 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe680003530 con 0x7fe69c15e710 2024-09-10T10:06:28.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe699400640 1 -- 172.21.15.110:0/2640957329 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594002891 0 0) 0x7fe69c071790 con 0x7fe69c0d13d0 2024-09-10T10:06:28.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe699400640 1 -- 172.21.15.110:0/2640957329 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe69c0714b0 con 0x7fe69c0d13d0 2024-09-10T10:06:28.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe691400640 1 -- 172.21.15.110:0/2640957329 >> v1:172.21.15.145:6789/0 conn(0x7fe69c0d13d0 legacy=0x7fe69c06db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe691400640 1 -- 172.21.15.110:0/2640957329 >> v1:172.21.15.110:6789/0 conn(0x7fe69c15e710 legacy=0x7fe69c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.612+0000 7fe691400640 1 -- 172.21.15.110:0/2640957329 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c071a70 con 0x7fe69c0b5590 2024-09-10T10:06:28.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe691400640 1 --2- 172.21.15.110:0/2640957329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0d0e90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe6840097f0 tx=0x7fe68401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813ac3e1ed3b53c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe699400640 1 -- 172.21.15.110:0/2640957329 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840b0070 con 0x7fe69c0b5590 2024-09-10T10:06:28.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe699400640 1 -- 172.21.15.110:0/2640957329 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6840ab600 con 0x7fe69c0b5590 2024-09-10T10:06:28.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe699400640 1 -- 172.21.15.110:0/2640957329 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840ab9b0 con 0x7fe69c0b5590 2024-09-10T10:06:28.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2640957329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 msgr2=0x7fe69c0d0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe6a0c00640 1 --2- 172.21.15.110:0/2640957329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0d0e90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe6840097f0 tx=0x7fe68401c4c0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2640957329 shutdown_connections 2024-09-10T10:06:28.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe6a0c00640 1 --2- 172.21.15.110:0/2640957329 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0d0e90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2640957329 >> 172.21.15.110:0/2640957329 conn(0x7fe69c0b4290 msgr2=0x7fe69c0b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:28.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2640957329 shutdown_connections 2024-09-10T10:06:28.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.616+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2640957329 wait complete. 2024-09-10T10:06:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 Processor -- start 2024-09-10T10:06:28.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 -- start start 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69c0d13d0 0x7fe69c150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c15e710 0x7fe69c1617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c06d400 con 0x7fe69c15e710 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c06d280 con 0x7fe69c0d13d0 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe69c16c680 con 0x7fe69c0b5590 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c15e710 0x7fe69c1617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c15e710 0x7fe69c1617c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51422/0 (socket says 172.21.15.110:51422) 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 -- 172.21.15.110:0/2990788060 learned_addr learned my addr 172.21.15.110:0/2990788060 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe691400640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69c0d13d0 0x7fe69c150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe699e00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 -- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 msgr2=0x7fe69c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 -- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69c0d13d0 msgr2=0x7fe69c150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69c0d13d0 0x7fe69c150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 -- 172.21.15.110:0/2990788060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c0cc030 con 0x7fe69c15e710 2024-09-10T10:06:28.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe69a800640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c15e710 0x7fe69c1617c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c790 tx=0x7fe69400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe692a00640 1 -- 172.21.15.110:0/2990788060 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694003b80 con 0x7fe69c15e710 2024-09-10T10:06:28.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe692a00640 1 -- 172.21.15.110:0/2990788060 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe694003d20 con 0x7fe69c15e710 2024-09-10T10:06:28.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c0cc310 con 0x7fe69c15e710 2024-09-10T10:06:28.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe692a00640 1 -- 172.21.15.110:0/2990788060 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69400aa50 con 0x7fe69c15e710 2024-09-10T10:06:28.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.620+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c0cc8d0 con 0x7fe69c15e710 2024-09-10T10:06:28.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.624+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69c0d0e90 con 0x7fe69c15e710 2024-09-10T10:06:28.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.624+0000 7fe692a00640 1 -- 172.21.15.110:0/2990788060 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fe69400abf0 con 0x7fe69c15e710 2024-09-10T10:06:28.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.624+0000 7fe692a00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe678068400 0x7fe67806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.624+0000 7fe692a00640 1 -- 172.21.15.110:0/2990788060 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7fe6940847b0 con 0x7fe69c15e710 2024-09-10T10:06:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.624+0000 7fe699e00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe678068400 0x7fe67806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.624+0000 7fe699e00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe678068400 0x7fe67806a8c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0011f0 tx=0x7fe68c007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.628+0000 7fe692a00640 1 -- 172.21.15.110:0/2990788060 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe694056410 con 0x7fe69c15e710 2024-09-10T10:06:28.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.792+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe69c0b4ef0 con 0x7fe69c15e710 2024-09-10T10:06:28.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.792+0000 7fe692a00640 1 -- 172.21.15.110:0/2990788060 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+16868 (secure 0 0 0) 0x7fe69405a710 con 0x7fe69c15e710 2024-09-10T10:06:28.796 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:28.796 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":165,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:06:14.723170+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:06:10.807237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T10:06:12.100049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:06:28.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe678068400 msgr2=0x7fe67806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe678068400 0x7fe67806a8c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0011f0 tx=0x7fe68c007d40 comp rx=0 tx=0).stop 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c15e710 msgr2=0x7fe69c1617c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c15e710 0x7fe69c1617c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c790 tx=0x7fe69400cc50 comp rx=0 tx=0).stop 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 shutdown_connections 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe678068400 0x7fe67806a8c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe69c15e710 0x7fe69c1617c0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe69c0d13d0 0x7fe69c150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 --2- 172.21.15.110:0/2990788060 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe69c0b5590 0x7fe69c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 >> 172.21.15.110:0/2990788060 conn(0x7fe69c0b4290 msgr2=0x7fe69c15f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 shutdown_connections 2024-09-10T10:06:28.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.796+0000 7fe6a0c00640 1 -- 172.21.15.110:0/2990788060 wait complete. 2024-09-10T10:06:28.818 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:06:28.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.952+0000 7f6c06e00640 1 Processor -- start 2024-09-10T10:06:28.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.952+0000 7f6c06e00640 1 -- start start 2024-09-10T10:06:28.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c06e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000b3f40 0x7f6c000cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c06e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6c000cff70 0x7f6c000cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c06e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0015d4d0 con 0x7f6c000cfb90 2024-09-10T10:06:28.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c06e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6c0006d130 con 0x7f6c000cff70 2024-09-10T10:06:28.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c06e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0015d7b0 con 0x7f6c000b3f40 2024-09-10T10:06:28.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6bffe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6c000cfb90 0x7f6c0015ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41292/0 (socket says 172.21.15.110:41292) 2024-09-10T10:06:28.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6bffe00640 1 -- 172.21.15.110:0/655416634 learned_addr learned my addr 172.21.15.110:0/655416634 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c04e00640 1 --2- 172.21.15.110:0/655416634 >> v2:172.21.15.145:3300/0 conn(0x7f6c000cff70 0x7f6c000cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6bff400640 1 --2- 172.21.15.110:0/655416634 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000b3f40 0x7f6c000cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6bfea00640 1 -- 172.21.15.110:0/655416634 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1379289541 0 0) 0x7f6c0015d4d0 con 0x7f6c000cfb90 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6bfea00640 1 -- 172.21.15.110:0/655416634 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6be4003530 con 0x7f6c000cfb90 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c04e00640 1 -- 172.21.15.110:0/655416634 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000b3f40 msgr2=0x7f6c000cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c04e00640 1 --2- 172.21.15.110:0/655416634 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000b3f40 0x7f6c000cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6bfea00640 1 -- 172.21.15.110:0/655416634 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1714218317 0 0) 0x7f6be4003530 con 0x7f6c000cfb90 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c04e00640 1 -- 172.21.15.110:0/655416634 >> v1:172.21.15.110:6789/0 conn(0x7f6c000cfb90 legacy=0x7f6c0015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.956+0000 7f6c04e00640 1 -- 172.21.15.110:0/655416634 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0006d380 con 0x7f6c000cff70 2024-09-10T10:06:28.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c04e00640 1 --2- 172.21.15.110:0/655416634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c000cff70 0x7f6c000cdae0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80097c0 tx=0x7f6bf801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952a8535e0778829 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6bfea00640 1 -- 172.21.15.110:0/655416634 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80b0070 con 0x7f6c000cff70 2024-09-10T10:06:28.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6bfea00640 1 -- 172.21.15.110:0/655416634 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf80ab400 con 0x7f6c000cff70 2024-09-10T10:06:28.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6bfea00640 1 -- 172.21.15.110:0/655416634 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0001a00 con 0x7f6c000cff70 2024-09-10T10:06:28.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 -- 172.21.15.110:0/655416634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c000cff70 msgr2=0x7f6c000cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 --2- 172.21.15.110:0/655416634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c000cff70 0x7f6c000cdae0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80097c0 tx=0x7f6bf801c330 comp rx=0 tx=0).stop 2024-09-10T10:06:28.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 -- 172.21.15.110:0/655416634 shutdown_connections 2024-09-10T10:06:28.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 --2- 172.21.15.110:0/655416634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c000cff70 0x7f6c000cdae0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 --2- 172.21.15.110:0/655416634 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000b3f40 0x7f6c000cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 -- 172.21.15.110:0/655416634 >> 172.21.15.110:0/655416634 conn(0x7f6c00062d10 msgr2=0x7f6c0006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:28.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 -- 172.21.15.110:0/655416634 shutdown_connections 2024-09-10T10:06:28.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 -- 172.21.15.110:0/655416634 wait complete. 2024-09-10T10:06:28.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 Processor -- start 2024-09-10T10:06:28.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 -- start start 2024-09-10T10:06:28.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.960+0000 7f6c06e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c000b3f40 0x7f6c00162a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6c06e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c000cfb90 0x7f6c000b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6c06e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000cff70 0x7f6c00152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6c06e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0006e720 con 0x7f6c000b3f40 2024-09-10T10:06:28.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6c06e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0006e5a0 con 0x7f6c000cfb90 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6c06e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0006e8a0 con 0x7f6c000cff70 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c000b3f40 0x7f6c00162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c000b3f40 0x7f6c00162a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51424/0 (socket says 172.21.15.110:51424) 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 -- 172.21.15.110:0/846846277 learned_addr learned my addr 172.21.15.110:0/846846277 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 -- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000cff70 msgr2=0x7f6c00152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000cff70 0x7f6c00152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 -- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c000cfb90 msgr2=0x7f6c000b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c000cfb90 0x7f6c000b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 -- 172.21.15.110:0/846846277 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c00165da0 con 0x7f6c000b3f40 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bffe00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c000b3f40 0x7f6c00162a50 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0001880 tx=0x7f6bf000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bf7e00640 1 -- 172.21.15.110:0/846846277 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0012400 con 0x7f6c000b3f40 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bf7e00640 1 -- 172.21.15.110:0/846846277 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf00125c0 con 0x7f6c000b3f40 2024-09-10T10:06:28.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bf7e00640 1 -- 172.21.15.110:0/846846277 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00129b0 con 0x7f6c000b3f40 2024-09-10T10:06:28.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c00165500 con 0x7f6c000b3f40 2024-09-10T10:06:28.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c00165ac0 con 0x7f6c000b3f40 2024-09-10T10:06:28.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c0015ca10 con 0x7f6c000b3f40 2024-09-10T10:06:28.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bf7e00640 1 -- 172.21.15.110:0/846846277 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f6bf0012b50 con 0x7f6c000b3f40 2024-09-10T10:06:28.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.964+0000 7f6bf7e00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6be0068400 0x7f6be006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:28.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.968+0000 7f6bf7e00640 1 -- 172.21.15.110:0/846846277 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5765+0+0 (secure 0 0 0) 0x7f6bf0085100 con 0x7f6c000b3f40 2024-09-10T10:06:28.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.968+0000 7f6bff400640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6be0068400 0x7f6be006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:28.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.968+0000 7f6bff400640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6be0068400 0x7f6be006a8c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006aa0 tx=0x7f6bec006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:28.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:28.972+0000 7f6bf7e00640 1 -- 172.21.15.110:0/846846277 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6bf0057440 con 0x7f6c000b3f40 2024-09-10T10:06:29.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.172+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6c0015f8c0 con 0x7f6c000b3f40 2024-09-10T10:06:29.175 INFO:tasks.ceph.mon.a.smithi110.stderr:2024-09-10T10:06:29.172+0000 7f6ed0400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:06:29.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.176+0000 7f6bf7e00640 1 -- 172.21.15.110:0/846846277 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) ==== 161+0+0 (secure 0 0 0) 0x7f6bf0056d40 con 0x7f6c000b3f40 2024-09-10T10:06:29.177 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6be0068400 msgr2=0x7f6be006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6be0068400 0x7f6be006a8c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006aa0 tx=0x7f6bec006a30 comp rx=0 tx=0).stop 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c000b3f40 msgr2=0x7f6c00162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c000b3f40 0x7f6c00162a50 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0001880 tx=0x7f6bf000a430 comp rx=0 tx=0).stop 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 shutdown_connections 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6be0068400 0x7f6be006a8c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6c000cff70 0x7f6c00152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c000cfb90 0x7f6c000b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 --2- 172.21.15.110:0/846846277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c000b3f40 0x7f6c00162a50 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 >> 172.21.15.110:0/846846277 conn(0x7f6c00062d10 msgr2=0x7f6c000ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:29.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 shutdown_connections 2024-09-10T10:06:29.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.180+0000 7f6c06e00640 1 -- 172.21.15.110:0/846846277 wait complete. 2024-09-10T10:06:29.199 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:06:29.250 INFO:tasks.ceph.mds.b.smithi145.stdout:starting mds.b at 2024-09-10T10:06:29.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.336+0000 7f2bfe200640 1 Processor -- start 2024-09-10T10:06:29.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.336+0000 7f2bfe200640 1 -- start start 2024-09-10T10:06:29.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bfe200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2bf815b330 0x7f2bf806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:29.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bfe200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf806e850 0x7f2bf80b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:29.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bfe200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf80b4960 con 0x7f2bf80ce4d0 2024-09-10T10:06:29.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bfe200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf80b5570 con 0x7f2bf815b330 2024-09-10T10:06:29.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bfe200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf80b4c40 con 0x7f2bf806e850 2024-09-10T10:06:29.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf6a00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2bf80ce4d0 0x7f2bf806e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41308/0 (socket says 172.21.15.110:41308) 2024-09-10T10:06:29.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf6a00640 1 -- 172.21.15.110:0/4086899984 learned_addr learned my addr 172.21.15.110:0/4086899984 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:29.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf7400640 1 --2- 172.21.15.110:0/4086899984 >> v2:172.21.15.145:3300/0 conn(0x7f2bf815b330 0x7f2bf806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:29.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf7e00640 1 --2- 172.21.15.110:0/4086899984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf806e850 0x7f2bf80b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:29.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf6000640 1 -- 172.21.15.110:0/4086899984 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983042822 0 0) 0x7f2bf80b4960 con 0x7f2bf80ce4d0 2024-09-10T10:06:29.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf6000640 1 -- 172.21.15.110:0/4086899984 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bdc003530 con 0x7f2bf80ce4d0 2024-09-10T10:06:29.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf7e00640 1 -- 172.21.15.110:0/4086899984 >> v2:172.21.15.145:3300/0 conn(0x7f2bf815b330 msgr2=0x7f2bf806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:29.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf7e00640 1 --2- 172.21.15.110:0/4086899984 >> v2:172.21.15.145:3300/0 conn(0x7f2bf815b330 0x7f2bf806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf7e00640 1 -- 172.21.15.110:0/4086899984 >> v1:172.21.15.110:6789/0 conn(0x7f2bf80ce4d0 legacy=0x7f2bf806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:29.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.340+0000 7f2bf7e00640 1 -- 172.21.15.110:0/4086899984 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf80b4f20 con 0x7f2bf806e850 2024-09-10T10:06:29.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bf7e00640 1 --2- 172.21.15.110:0/4086899984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf806e850 0x7f2bf80b0860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00097c0 tx=0x7f2bf001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cccf005ea109d963 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:29.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bf6000640 1 -- 172.21.15.110:0/4086899984 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00b1070 con 0x7f2bf806e850 2024-09-10T10:06:29.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bf6000640 1 -- 172.21.15.110:0/4086899984 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf00ac410 con 0x7f2bf806e850 2024-09-10T10:06:29.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bf6000640 1 -- 172.21.15.110:0/4086899984 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00ac7c0 con 0x7f2bf806e850 2024-09-10T10:06:29.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 -- 172.21.15.110:0/4086899984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf806e850 msgr2=0x7f2bf80b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:29.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 --2- 172.21.15.110:0/4086899984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf806e850 0x7f2bf80b0860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00097c0 tx=0x7f2bf001c4a0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 -- 172.21.15.110:0/4086899984 shutdown_connections 2024-09-10T10:06:29.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 --2- 172.21.15.110:0/4086899984 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf806e850 0x7f2bf80b0860 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 --2- 172.21.15.110:0/4086899984 >> v2:172.21.15.145:3300/0 conn(0x7f2bf815b330 0x7f2bf806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 -- 172.21.15.110:0/4086899984 >> 172.21.15.110:0/4086899984 conn(0x7f2bf8062e40 msgr2=0x7f2bf806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:29.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 -- 172.21.15.110:0/4086899984 shutdown_connections 2024-09-10T10:06:29.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 -- 172.21.15.110:0/4086899984 wait complete. 2024-09-10T10:06:29.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 Processor -- start 2024-09-10T10:06:29.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.344+0000 7f2bfe200640 1 -- start start 2024-09-10T10:06:29.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bf806e850 0x7f2bf81f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:29.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bf815b330 0x7f2bf8154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:29.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf8154dd0 0x7f2bf80ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:29.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf81680b0 con 0x7f2bf815b330 2024-09-10T10:06:29.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf80b1800 con 0x7f2bf806e850 2024-09-10T10:06:29.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf8168230 con 0x7f2bf8154dd0 2024-09-10T10:06:29.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bf815b330 0x7f2bf8154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:29.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bf815b330 0x7f2bf8154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51438/0 (socket says 172.21.15.110:51438) 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 -- 172.21.15.110:0/3983831951 learned_addr learned my addr 172.21.15.110:0/3983831951 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf7e00640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf8154dd0 0x7f2bf80ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf7400640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bf806e850 0x7f2bf81f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 -- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf8154dd0 msgr2=0x7f2bf80ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf8154dd0 0x7f2bf80ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 -- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bf806e850 msgr2=0x7f2bf81f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bf806e850 0x7f2bf81f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 -- 172.21.15.110:0/3983831951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf8154170 con 0x7f2bf815b330 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf7400640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bf806e850 0x7f2bf81f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bf6a00640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bf815b330 0x7f2bf8154890 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c450 tx=0x7f2be400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:29.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bef400640 1 -- 172.21.15.110:0/3983831951 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4017070 con 0x7f2bf815b330 2024-09-10T10:06:29.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bef400640 1 -- 172.21.15.110:0/3983831951 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be4005cf0 con 0x7f2bf815b330 2024-09-10T10:06:29.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf81f7150 con 0x7f2bf815b330 2024-09-10T10:06:29.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bef400640 1 -- 172.21.15.110:0/3983831951 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4007290 con 0x7f2bf815b330 2024-09-10T10:06:29.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf81f7710 con 0x7f2bf815b330 2024-09-10T10:06:29.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf81551b0 con 0x7f2bf815b330 2024-09-10T10:06:29.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.348+0000 7f2bef400640 1 -- 172.21.15.110:0/3983831951 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f2be401a070 con 0x7f2bf815b330 2024-09-10T10:06:29.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.352+0000 7f2bef400640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bd8068400 0x7f2bd806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:29.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.352+0000 7f2bef400640 1 -- 172.21.15.110:0/3983831951 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5851+0+0 (secure 0 0 0) 0x7f2be4083c10 con 0x7f2bf815b330 2024-09-10T10:06:29.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.352+0000 7f2bf7400640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bd8068400 0x7f2bd806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:29.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.352+0000 7f2bf7400640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bd8068400 0x7f2bd806a8c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80aff00 tx=0x7f2be800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:29.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.356+0000 7f2bef400640 1 -- 172.21.15.110:0/3983831951 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2be4055880 con 0x7f2bf815b330 2024-09-10T10:06:29.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:29.544+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2bf80b0860 con 0x7f2bf815b330 2024-09-10T10:06:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bef400640 1 -- 172.21.15.110:0/3983831951 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v118) ==== 104+0+0 (secure 0 0 0) 0x7f2be4059b80 con 0x7f2bf815b330 2024-09-10T10:06:30.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bd8068400 msgr2=0x7f2bd806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:30.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bd8068400 0x7f2bd806a8c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80aff00 tx=0x7f2be800b040 comp rx=0 tx=0).stop 2024-09-10T10:06:30.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bf815b330 msgr2=0x7f2bf8154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:30.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bf815b330 0x7f2bf8154890 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c450 tx=0x7f2be400c910 comp rx=0 tx=0).stop 2024-09-10T10:06:30.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 shutdown_connections 2024-09-10T10:06:30.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2bd8068400 0x7f2bd806a8c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:30.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2bf8154dd0 0x7f2bf80ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:30.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2bf815b330 0x7f2bf8154890 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:30.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 --2- 172.21.15.110:0/3983831951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bf806e850 0x7f2bf81f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:30.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 >> 172.21.15.110:0/3983831951 conn(0x7f2bf8062e40 msgr2=0x7f2bf80cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:30.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 shutdown_connections 2024-09-10T10:06:30.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.184+0000 7f2bfe200640 1 -- 172.21.15.110:0/3983831951 wait complete. 2024-09-10T10:06:30.211 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:06:30.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d71000640 1 Processor -- start 2024-09-10T10:06:30.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d71000640 1 -- start start 2024-09-10T10:06:30.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d71000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6d6c15e470 0x7f6d6c0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:30.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d71000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d6c06d020 con 0x7f6d6c0738f0 2024-09-10T10:06:30.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d71000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d6c0b5c60 con 0x7f6d6c162020 2024-09-10T10:06:30.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d71000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f6d6c0b6870 con 0x7f6d6c15e470 2024-09-10T10:06:30.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d6b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6d6c15e470 0x7f6d6c0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:30.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d6b400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6d6c15e470 0x7f6d6c0b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51442/0 (socket says 172.21.15.110:51442) 2024-09-10T10:06:30.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d6b400640 1 -- 172.21.15.110:0/675140680 learned_addr learned my addr 172.21.15.110:0/675140680 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:30.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d6be00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f6d6c0738f0 0x7f6d6c06d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41320/0 (socket says 172.21.15.110:41320) 2024-09-10T10:06:30.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d6b400640 1 -- 172.21.15.110:0/675140680 >> v1:172.21.15.145:6789/0 conn(0x7f6d6c162020 legacy=0x7f6d6c0731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:30.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d6b400640 1 -- 172.21.15.110:0/675140680 >> v1:172.21.15.110:6789/0 conn(0x7f6d6c0738f0 legacy=0x7f6d6c06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:30.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d6b400640 1 -- 172.21.15.110:0/675140680 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c0b5f40 con 0x7f6d6c15e470 2024-09-10T10:06:30.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.352+0000 7f6d6a000640 1 -- 172.21.15.110:0/675140680 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3779342953 0 0) 0x7f6d6c06d020 con 0x7f6d6c0738f0 2024-09-10T10:06:30.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d6b400640 1 --2- 172.21.15.110:0/675140680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c15e470 0x7f6d6c0b55d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009870 tx=0x7f6d5c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30a746ada9bd0e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:30.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d6a000640 1 -- 172.21.15.110:0/675140680 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0b5070 con 0x7f6d6c15e470 2024-09-10T10:06:30.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d6a000640 1 -- 172.21.15.110:0/675140680 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5c0a8a10 con 0x7f6d6c15e470 2024-09-10T10:06:30.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d6a000640 1 -- 172.21.15.110:0/675140680 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0a8de0 con 0x7f6d6c15e470 2024-09-10T10:06:30.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d71000640 1 -- 172.21.15.110:0/675140680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c15e470 msgr2=0x7f6d6c0b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:30.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d71000640 1 --2- 172.21.15.110:0/675140680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c15e470 0x7f6d6c0b55d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009870 tx=0x7f6d5c01ed10 comp rx=0 tx=0).stop 2024-09-10T10:06:30.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d71000640 1 -- 172.21.15.110:0/675140680 shutdown_connections 2024-09-10T10:06:30.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d71000640 1 --2- 172.21.15.110:0/675140680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c15e470 0x7f6d6c0b55d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:30.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d71000640 1 -- 172.21.15.110:0/675140680 >> 172.21.15.110:0/675140680 conn(0x7f6d6c0b42d0 msgr2=0x7f6d6c0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:30.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d71000640 1 -- 172.21.15.110:0/675140680 shutdown_connections 2024-09-10T10:06:30.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.356+0000 7f6d71000640 1 -- 172.21.15.110:0/675140680 wait complete. 2024-09-10T10:06:30.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 Processor -- start 2024-09-10T10:06:30.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 -- start start 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c0738f0 0x7f6d6c144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d6c15e470 0x7f6d6c154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d6c162020 0x7f6d6c154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c15e850 con 0x7f6d6c0738f0 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c0b6500 con 0x7f6d6c162020 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6c06de30 con 0x7f6d6c15e470 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c0738f0 0x7f6d6c144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c0738f0 0x7f6d6c144e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51454/0 (socket says 172.21.15.110:51454) 2024-09-10T10:06:30.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 -- 172.21.15.110:0/3804391540 learned_addr learned my addr 172.21.15.110:0/3804391540 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6be00640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d6c162020 0x7f6d6c154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6aa00640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d6c15e470 0x7f6d6c154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 -- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d6c15e470 msgr2=0x7f6d6c154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d6c15e470 0x7f6d6c154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 -- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d6c162020 msgr2=0x7f6d6c154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d6c162020 0x7f6d6c154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 -- 172.21.15.110:0/3804391540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c160820 con 0x7f6d6c0738f0 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d6b400640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c0738f0 0x7f6d6c144e90 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009870 tx=0x7f6d5c01d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d63400640 1 -- 172.21.15.110:0/3804391540 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0c1070 con 0x7f6d6c0738f0 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d63400640 1 -- 172.21.15.110:0/3804391540 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5c004230 con 0x7f6d6c0738f0 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c153a90 con 0x7f6d6c0738f0 2024-09-10T10:06:30.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d63400640 1 -- 172.21.15.110:0/3804391540 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0b5040 con 0x7f6d6c0738f0 2024-09-10T10:06:30.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c160440 con 0x7f6d6c0738f0 2024-09-10T10:06:30.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.360+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6c0b55d0 con 0x7f6d6c0738f0 2024-09-10T10:06:30.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.364+0000 7f6d63400640 1 -- 172.21.15.110:0/3804391540 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f6d5c0c4070 con 0x7f6d6c0738f0 2024-09-10T10:06:30.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.364+0000 7f6d63400640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6d4c068400 0x7f6d4c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:30.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.364+0000 7f6d63400640 1 -- 172.21.15.110:0/3804391540 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5851+0+0 (secure 0 0 0) 0x7f6d5c121910 con 0x7f6d6c0738f0 2024-09-10T10:06:30.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.364+0000 7f6d6aa00640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6d4c068400 0x7f6d4c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:30.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.364+0000 7f6d6aa00640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6d4c068400 0x7f6d4c06a8c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0d1c50 tx=0x7f6d58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:30.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.368+0000 7f6d63400640 1 -- 172.21.15.110:0/3804391540 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6d5c018870 con 0x7f6d6c0738f0 2024-09-10T10:06:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:30.528+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6d6c0b1e00 con 0x7f6d6c0738f0 2024-09-10T10:06:31.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.184+0000 7f6d63400640 1 -- 172.21.15.110:0/3804391540 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v167) ==== 181+0+0 (secure 0 0 0) 0x7f6d5c0f3bd0 con 0x7f6d6c0738f0 2024-09-10T10:06:31.187 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:06:31.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6d4c068400 msgr2=0x7f6d4c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6d4c068400 0x7f6d4c06a8c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0d1c50 tx=0x7f6d58008040 comp rx=0 tx=0).stop 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c0738f0 msgr2=0x7f6d6c144e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c0738f0 0x7f6d6c144e90 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009870 tx=0x7f6d5c01d550 comp rx=0 tx=0).stop 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 shutdown_connections 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f6d4c068400 0x7f6d4c06a8c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d6c162020 0x7f6d6c154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f6d6c15e470 0x7f6d6c154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 --2- 172.21.15.110:0/3804391540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d6c0738f0 0x7f6d6c144e90 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 >> 172.21.15.110:0/3804391540 conn(0x7f6d6c0b42d0 msgr2=0x7f6d6c159020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 shutdown_connections 2024-09-10T10:06:31.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.188+0000 7f6d71000640 1 -- 172.21.15.110:0/3804391540 wait complete. 2024-09-10T10:06:31.213 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:06:31.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16c200640 1 Processor -- start 2024-09-10T10:06:31.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16c200640 1 -- start start 2024-09-10T10:06:31.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16c200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe1640cfe30 0x7fe164071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:31.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16c200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe16406e7a0 0x7fe1640cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16c200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe16406d2d0 con 0x7fe1640cd510 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16c200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe16415d9a0 con 0x7fe1640cfe30 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16c200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe16406d5b0 con 0x7fe16406e7a0 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe168e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe1640cd510 0x7fe16406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41328/0 (socket says 172.21.15.110:41328) 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe168e00640 1 -- 172.21.15.110:0/1512626421 learned_addr learned my addr 172.21.15.110:0/1512626421 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe169800640 1 --2- 172.21.15.110:0/1512626421 >> v2:172.21.15.145:3300/0 conn(0x7fe1640cfe30 0x7fe164071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16a200640 1 --2- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe16406e7a0 0x7fe1640cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe163e00640 1 -- 172.21.15.110:0/1512626421 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996364290 0 0) 0x7fe16406d2d0 con 0x7fe1640cd510 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe163e00640 1 -- 172.21.15.110:0/1512626421 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe14c003530 con 0x7fe1640cd510 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe169800640 1 -- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe16406e7a0 msgr2=0x7fe1640cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe169800640 1 --2- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe16406e7a0 0x7fe1640cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe169800640 1 -- 172.21.15.110:0/1512626421 >> v1:172.21.15.110:6789/0 conn(0x7fe1640cd510 legacy=0x7fe16406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:31.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe169800640 1 -- 172.21.15.110:0/1512626421 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16406d890 con 0x7fe1640cfe30 2024-09-10T10:06:31.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.356+0000 7fe16a200640 1 --2- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe16406e7a0 0x7fe1640cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:31.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe169800640 1 --2- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 0x7fe164071a40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe1580097c0 tx=0x7fe15801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ff2cf5b7c54dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:31.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe163e00640 1 -- 172.21.15.110:0/1512626421 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b0070 con 0x7fe1640cfe30 2024-09-10T10:06:31.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe163e00640 1 -- 172.21.15.110:0/1512626421 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1580ab600 con 0x7fe1640cfe30 2024-09-10T10:06:31.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe163e00640 1 -- 172.21.15.110:0/1512626421 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580ab9d0 con 0x7fe1640cfe30 2024-09-10T10:06:31.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe16c200640 1 -- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 msgr2=0x7fe164071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:31.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe16c200640 1 --2- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 0x7fe164071a40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe1580097c0 tx=0x7fe15801c410 comp rx=0 tx=0).stop 2024-09-10T10:06:31.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe16c200640 1 -- 172.21.15.110:0/1512626421 shutdown_connections 2024-09-10T10:06:31.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe16c200640 1 --2- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe16406e7a0 0x7fe1640cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe16c200640 1 --2- 172.21.15.110:0/1512626421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 0x7fe164071a40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe16c200640 1 -- 172.21.15.110:0/1512626421 >> 172.21.15.110:0/1512626421 conn(0x7fe16406bc50 msgr2=0x7fe16406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:31.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe16c200640 1 -- 172.21.15.110:0/1512626421 shutdown_connections 2024-09-10T10:06:31.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.360+0000 7fe16c200640 1 -- 172.21.15.110:0/1512626421 wait complete. 2024-09-10T10:06:31.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 Processor -- start 2024-09-10T10:06:31.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 -- start start 2024-09-10T10:06:31.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe16406e7a0 0x7fe1641f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:31.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1640cd510 0x7fe1640b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 0x7fe164152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe164072190 con 0x7fe16406e7a0 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe16415d6c0 con 0x7fe1640cfe30 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe164169180 con 0x7fe1640cd510 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe16406e7a0 0x7fe1641f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe16406e7a0 0x7fe1641f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51458/0 (socket says 172.21.15.110:51458) 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 -- 172.21.15.110:0/3531261464 learned_addr learned my addr 172.21.15.110:0/3531261464 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe168e00640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1640cd510 0x7fe1640b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16a200640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 0x7fe164152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 -- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1640cd510 msgr2=0x7fe1640b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1640cd510 0x7fe1640b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 -- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 msgr2=0x7fe164152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:31.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 0x7fe164152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:31.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 -- 172.21.15.110:0/3531261464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1640b1f80 con 0x7fe16406e7a0 2024-09-10T10:06:31.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe169800640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe16406e7a0 0x7fe1641f78c0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe15801a040 tx=0x7fe15801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:31.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe161600640 1 -- 172.21.15.110:0/3531261464 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b0050 con 0x7fe16406e7a0 2024-09-10T10:06:31.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe161600640 1 -- 172.21.15.110:0/3531261464 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1580b1040 con 0x7fe16406e7a0 2024-09-10T10:06:31.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1640b16e0 con 0x7fe16406e7a0 2024-09-10T10:06:31.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe161600640 1 -- 172.21.15.110:0/3531261464 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b5850 con 0x7fe16406e7a0 2024-09-10T10:06:31.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1640b1ca0 con 0x7fe16406e7a0 2024-09-10T10:06:31.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.364+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16415ab30 con 0x7fe16406e7a0 2024-09-10T10:06:31.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.368+0000 7fe161600640 1 -- 172.21.15.110:0/3531261464 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fe1580be030 con 0x7fe16406e7a0 2024-09-10T10:06:31.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.368+0000 7fe161600640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe144068400 0x7fe14406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:31.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.368+0000 7fe161600640 1 -- 172.21.15.110:0/3531261464 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5413+0+0 (secure 0 0 0) 0x7fe15811e4c0 con 0x7fe16406e7a0 2024-09-10T10:06:31.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.368+0000 7fe168e00640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe144068400 0x7fe14406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:31.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.368+0000 7fe168e00640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe144068400 0x7fe14406a8c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe164166470 tx=0x7fe150009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:31.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.372+0000 7fe161600640 1 -- 172.21.15.110:0/3531261464 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe1580c5060 con 0x7fe16406e7a0 2024-09-10T10:06:31.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:31.532+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe1640719c0 con 0x7fe16406e7a0 2024-09-10T10:06:32.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.192+0000 7fe161600640 1 -- 172.21.15.110:0/3531261464 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v168) ==== 169+0+0 (secure 0 0 0) 0x7fe158016090 con 0x7fe16406e7a0 2024-09-10T10:06:32.193 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-09-10T10:06:32.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.192+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe144068400 msgr2=0x7fe14406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.192+0000 7fe16c200640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe144068400 0x7fe14406a8c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe164166470 tx=0x7fe150009040 comp rx=0 tx=0).stop 2024-09-10T10:06:32.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.192+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe16406e7a0 msgr2=0x7fe1641f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.192+0000 7fe16c200640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe16406e7a0 0x7fe1641f78c0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe15801a040 tx=0x7fe15801ac50 comp rx=0 tx=0).stop 2024-09-10T10:06:32.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.196+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 shutdown_connections 2024-09-10T10:06:32.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.196+0000 7fe16c200640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe144068400 0x7fe14406a8c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.196+0000 7fe16c200640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1640cfe30 0x7fe164152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.196+0000 7fe16c200640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe1640cd510 0x7fe1640b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.196+0000 7fe16c200640 1 --2- 172.21.15.110:0/3531261464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe16406e7a0 0x7fe1641f78c0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.196+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 >> 172.21.15.110:0/3531261464 conn(0x7fe16406bc50 msgr2=0x7fe1640b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:32.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.196+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 shutdown_connections 2024-09-10T10:06:32.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.196+0000 7fe16c200640 1 -- 172.21.15.110:0/3531261464 wait complete. 2024-09-10T10:06:32.222 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:06:32.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.364+0000 7f391bc00640 1 Processor -- start 2024-09-10T10:06:32.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.364+0000 7f391bc00640 1 -- start start 2024-09-10T10:06:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.364+0000 7f391bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39140b2bb0 0x7f39140cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.364+0000 7f391bc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f39140cfbc0 0x7f39140b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.364+0000 7f391bc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39140b37c0 con 0x7f3914145f30 2024-09-10T10:06:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.364+0000 7f391bc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f39140b3aa0 con 0x7f39140cfbc0 2024-09-10T10:06:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.364+0000 7f391bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f39140b3d80 con 0x7f39140b2bb0 2024-09-10T10:06:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3919200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3914145f30 0x7f39140b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41344/0 (socket says 172.21.15.110:41344) 2024-09-10T10:06:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3919200640 1 -- 172.21.15.110:0/2018492260 learned_addr learned my addr 172.21.15.110:0/2018492260 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:32.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3919c00640 1 --2- 172.21.15.110:0/2018492260 >> v2:172.21.15.145:3300/0 conn(0x7f39140cfbc0 0x7f39140b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3913e00640 1 --2- 172.21.15.110:0/2018492260 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39140b2bb0 0x7f39140cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3913400640 1 -- 172.21.15.110:0/2018492260 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751020956 0 0) 0x7f39140b37c0 con 0x7f3914145f30 2024-09-10T10:06:32.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3913400640 1 -- 172.21.15.110:0/2018492260 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38f8003530 con 0x7f3914145f30 2024-09-10T10:06:32.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3919c00640 1 -- 172.21.15.110:0/2018492260 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39140b2bb0 msgr2=0x7f39140cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:32.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3919c00640 1 --2- 172.21.15.110:0/2018492260 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39140b2bb0 0x7f39140cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3919c00640 1 -- 172.21.15.110:0/2018492260 >> v1:172.21.15.110:6789/0 conn(0x7f3914145f30 legacy=0x7f39140b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3919c00640 1 -- 172.21.15.110:0/2018492260 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3914146310 con 0x7f39140cfbc0 2024-09-10T10:06:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.368+0000 7f3919c00640 1 --2- 172.21.15.110:0/2018492260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 0x7f39140b4580 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f390c009d90 tx=0x7f390c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c31da72094bafa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3913400640 1 -- 172.21.15.110:0/2018492260 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c0b2070 con 0x7f39140cfbc0 2024-09-10T10:06:32.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3913400640 1 -- 172.21.15.110:0/2018492260 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390c0a5830 con 0x7f39140cfbc0 2024-09-10T10:06:32.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3913400640 1 -- 172.21.15.110:0/2018492260 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c0a5be0 con 0x7f39140cfbc0 2024-09-10T10:06:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- 172.21.15.110:0/2018492260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 msgr2=0x7f39140b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 --2- 172.21.15.110:0/2018492260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 0x7f39140b4580 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f390c009d90 tx=0x7f390c01c970 comp rx=0 tx=0).stop 2024-09-10T10:06:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- 172.21.15.110:0/2018492260 shutdown_connections 2024-09-10T10:06:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 --2- 172.21.15.110:0/2018492260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 0x7f39140b4580 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 --2- 172.21.15.110:0/2018492260 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f39140b2bb0 0x7f39140cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- 172.21.15.110:0/2018492260 >> 172.21.15.110:0/2018492260 conn(0x7f3914062e40 msgr2=0x7f391406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:32.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- 172.21.15.110:0/2018492260 shutdown_connections 2024-09-10T10:06:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- 172.21.15.110:0/2018492260 wait complete. 2024-09-10T10:06:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 Processor -- start 2024-09-10T10:06:32.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- start start 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39140b2bb0 0x7f391413f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 0x7f39141425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3914145f30 0x7f3914142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39140b56a0 con 0x7f39140b2bb0 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f39140b5520 con 0x7f39140cfbc0 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f391bc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f39140b5820 con 0x7f3914145f30 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39140b2bb0 0x7f391413f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39140b2bb0 0x7f391413f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51464/0 (socket says 172.21.15.110:51464) 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919200640 1 -- 172.21.15.110:0/4083316844 learned_addr learned my addr 172.21.15.110:0/4083316844 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:32.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3913e00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 0x7f39141425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919c00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3914145f30 0x7f3914142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919200640 1 -- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3914145f30 msgr2=0x7f3914142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919200640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3914145f30 0x7f3914142b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919200640 1 -- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 msgr2=0x7f39141425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919200640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 0x7f39141425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.372+0000 7f3919200640 1 -- 172.21.15.110:0/4083316844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3914140e10 con 0x7f39140b2bb0 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3919c00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3914145f30 0x7f3914142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3913e00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 0x7f39141425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3919200640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39140b2bb0 0x7f391413f3a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f390400c3c0 tx=0x7f390400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3910c00640 1 -- 172.21.15.110:0/4083316844 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904017070 con 0x7f39140b2bb0 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3910c00640 1 -- 172.21.15.110:0/4083316844 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3904005cf0 con 0x7f39140b2bb0 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3914140850 con 0x7f39140b2bb0 2024-09-10T10:06:32.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3910c00640 1 -- 172.21.15.110:0/4083316844 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39040072f0 con 0x7f39140b2bb0 2024-09-10T10:06:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39140abfb0 con 0x7f39140b2bb0 2024-09-10T10:06:32.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39140cf680 con 0x7f39140b2bb0 2024-09-10T10:06:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3910c00640 1 -- 172.21.15.110:0/4083316844 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f390401a070 con 0x7f39140b2bb0 2024-09-10T10:06:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3910c00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f38f4068400 0x7f38f406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3910c00640 1 -- 172.21.15.110:0/4083316844 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f3904083c40 con 0x7f39140b2bb0 2024-09-10T10:06:32.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.376+0000 7f3913e00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f38f4068400 0x7f38f406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.380+0000 7f3913e00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f38f4068400 0x7f38f406a8c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3914076570 tx=0x7f3900008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.380+0000 7f3910c00640 1 -- 172.21.15.110:0/4083316844 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3914140e10 con 0x7f39140b2bb0 2024-09-10T10:06:32.543 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.540+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f39140afea0 con 0x7f39140b2bb0 2024-09-10T10:06:32.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.544+0000 7f3910c00640 1 -- 172.21.15.110:0/4083316844 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f3904055f80 con 0x7f39140b2bb0 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f38f4068400 msgr2=0x7f38f406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f38f4068400 0x7f38f406a8c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3914076570 tx=0x7f3900008040 comp rx=0 tx=0).stop 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39140b2bb0 msgr2=0x7f391413f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39140b2bb0 0x7f391413f3a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f390400c3c0 tx=0x7f390400c880 comp rx=0 tx=0).stop 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 shutdown_connections 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f38f4068400 0x7f38f406a8c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3914145f30 0x7f3914142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f39140cfbc0 0x7f39141425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 --2- 172.21.15.110:0/4083316844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39140b2bb0 0x7f391413f3a0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 >> 172.21.15.110:0/4083316844 conn(0x7f3914062e40 msgr2=0x7f391415f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 shutdown_connections 2024-09-10T10:06:32.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.548+0000 7f391bc00640 1 -- 172.21.15.110:0/4083316844 wait complete. 2024-09-10T10:06:32.568 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-10T10:06:32.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.700+0000 7f40b2e00640 1 Processor -- start 2024-09-10T10:06:32.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.700+0000 7f40b2e00640 1 -- start start 2024-09-10T10:06:32.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40b2e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f40ac0cd510 0x7f40ac0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40b2e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac06e090 0x7f40ac06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40b2e00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40ac072150 con 0x7f40ac0cfe30 2024-09-10T10:06:32.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40b2e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f40ac15d9a0 con 0x7f40ac0cd510 2024-09-10T10:06:32.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40b2e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f40ac06ea60 con 0x7f40ac06e090 2024-09-10T10:06:32.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40abe00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f40ac0cfe30 0x7f40ac071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41352/0 (socket says 172.21.15.110:41352) 2024-09-10T10:06:32.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40abe00640 1 -- 172.21.15.110:0/246358288 learned_addr learned my addr 172.21.15.110:0/246358288 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:32.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40a3e00640 1 --2- 172.21.15.110:0/246358288 >> v2:172.21.15.145:3300/0 conn(0x7f40ac0cd510 0x7f40ac0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40b0e00640 1 --2- 172.21.15.110:0/246358288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac06e090 0x7f40ac06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40ab400640 1 -- 172.21.15.110:0/246358288 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3067537259 0 0) 0x7f40ac072150 con 0x7f40ac0cfe30 2024-09-10T10:06:32.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40ab400640 1 -- 172.21.15.110:0/246358288 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4090003530 con 0x7f40ac0cfe30 2024-09-10T10:06:32.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40a3e00640 1 -- 172.21.15.110:0/246358288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac06e090 msgr2=0x7f40ac06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:32.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40a3e00640 1 --2- 172.21.15.110:0/246358288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac06e090 0x7f40ac06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40a3e00640 1 -- 172.21.15.110:0/246358288 >> v1:172.21.15.110:6789/0 conn(0x7f40ac0cfe30 legacy=0x7f40ac071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40ab400640 1 -- 172.21.15.110:0/246358288 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2267857934 0 0) 0x7f4090003530 con 0x7f40ac0cfe30 2024-09-10T10:06:32.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.704+0000 7f40a3e00640 1 -- 172.21.15.110:0/246358288 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac06d2d0 con 0x7f40ac0cd510 2024-09-10T10:06:32.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40a3e00640 1 --2- 172.21.15.110:0/246358288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 0x7f40ac0cd9a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f40940097c0 tx=0x7f409401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8f630dab259bae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40ab400640 1 -- 172.21.15.110:0/246358288 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40940b0070 con 0x7f40ac0cd510 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40ab400640 1 -- 172.21.15.110:0/246358288 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40940ab400 con 0x7f40ac0cd510 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40ab400640 1 -- 172.21.15.110:0/246358288 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c001680 con 0x7f40ac0cd510 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 -- 172.21.15.110:0/246358288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 msgr2=0x7f40ac0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 --2- 172.21.15.110:0/246358288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 0x7f40ac0cd9a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f40940097c0 tx=0x7f409401c330 comp rx=0 tx=0).stop 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 -- 172.21.15.110:0/246358288 shutdown_connections 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 --2- 172.21.15.110:0/246358288 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac06e090 0x7f40ac06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 --2- 172.21.15.110:0/246358288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 0x7f40ac0cd9a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 -- 172.21.15.110:0/246358288 >> 172.21.15.110:0/246358288 conn(0x7f40ac06bc50 msgr2=0x7f40ac06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 -- 172.21.15.110:0/246358288 shutdown_connections 2024-09-10T10:06:32.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 -- 172.21.15.110:0/246358288 wait complete. 2024-09-10T10:06:32.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 Processor -- start 2024-09-10T10:06:32.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.708+0000 7f40b2e00640 1 -- start start 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40ac06e090 0x7f40ac147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 0x7f40ac0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac0cfe30 0x7f40ac153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac169180 con 0x7f40ac06e090 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac15d6c0 con 0x7f40ac0cd510 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f40ac169300 con 0x7f40ac0cfe30 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40ac06e090 0x7f40ac147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40ac06e090 0x7f40ac147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51476/0 (socket says 172.21.15.110:51476) 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 -- 172.21.15.110:0/3772672657 learned_addr learned my addr 172.21.15.110:0/3772672657 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40a3e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 0x7f40ac0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b0e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac0cfe30 0x7f40ac153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 -- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac0cfe30 msgr2=0x7f40ac153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac0cfe30 0x7f40ac153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 -- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 msgr2=0x7f40ac0b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 0x7f40ac0b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 -- 172.21.15.110:0/3772672657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac155730 con 0x7f40ac06e090 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b0e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac0cfe30 0x7f40ac153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40abe00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40ac06e090 0x7f40ac147ab0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c6d0 tx=0x7f409c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40a8c00640 1 -- 172.21.15.110:0/3772672657 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c017070 con 0x7f40ac06e090 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40a8c00640 1 -- 172.21.15.110:0/3772672657 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f409c005ae0 con 0x7f40ac06e090 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40a8c00640 1 -- 172.21.15.110:0/3772672657 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c007070 con 0x7f40ac06e090 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ac0b2770 con 0x7f40ac06e090 2024-09-10T10:06:32.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ac0b2d30 con 0x7f40ac06e090 2024-09-10T10:06:32.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ac0b2f70 con 0x7f40ac06e090 2024-09-10T10:06:32.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.712+0000 7f40a8c00640 1 -- 172.21.15.110:0/3772672657 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f409c01a070 con 0x7f40ac06e090 2024-09-10T10:06:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.716+0000 7f40a8c00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4088068400 0x7f408806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.716+0000 7f40a8c00640 1 -- 172.21.15.110:0/3772672657 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f409c083d50 con 0x7f40ac06e090 2024-09-10T10:06:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.716+0000 7f40a3e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4088068400 0x7f408806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.716+0000 7f40a3e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4088068400 0x7f408806a8c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f40940ad000 tx=0x7f40940a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.720+0000 7f40a8c00640 1 -- 172.21.15.110:0/3772672657 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f40ac155730 con 0x7f40ac06e090 2024-09-10T10:06:32.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:32.936+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f40ac06e500 con 0x7f40ac06e090 2024-09-10T10:06:33.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.192+0000 7f40a8c00640 1 -- 172.21.15.110:0/3772672657 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v451) ==== 163+0+0 (secure 0 0 0) 0x7f409c0559c0 con 0x7f40ac06e090 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4088068400 msgr2=0x7f408806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4088068400 0x7f408806a8c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f40940ad000 tx=0x7f40940a6000 comp rx=0 tx=0).stop 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40ac06e090 msgr2=0x7f40ac147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40ac06e090 0x7f40ac147ab0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c6d0 tx=0x7f409c00cb90 comp rx=0 tx=0).stop 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 shutdown_connections 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4088068400 0x7f408806a8c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f40ac0cfe30 0x7f40ac153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f40ac0cd510 0x7f40ac0b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 --2- 172.21.15.110:0/3772672657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40ac06e090 0x7f40ac147ab0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 >> 172.21.15.110:0/3772672657 conn(0x7f40ac06bc50 msgr2=0x7f40ac06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:33.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 shutdown_connections 2024-09-10T10:06:33.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:33.196+0000 7f40b2e00640 1 -- 172.21.15.110:0/3772672657 wait complete. 2024-09-10T10:06:33.218 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-10T10:06:33.218 INFO:tasks.cephfs_test_runner: 2024-09-10T10:06:33.218 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T10:06:33.218 INFO:tasks.cephfs_test_runner:Ran 11 tests in 466.703s 2024-09-10T10:06:33.218 INFO:tasks.cephfs_test_runner: 2024-09-10T10:06:33.219 INFO:tasks.cephfs_test_runner:OK 2024-09-10T10:06:33.219 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T10:06:33.228 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T10:06:33.269 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T10:06:33.270 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:33.270 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:33.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:06:33.279 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-10T10:06:33.279 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:33.332 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4170 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:06:33.332 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:06:33.332 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:33.342 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:06:33.342 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-10T10:06:33.342 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:06:33.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350188k,nr_inodes=4087547,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=890 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:06:33.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:33.392 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:06:33.400 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:06:33.400 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:06:33.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:33.401 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:06:33.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:33.465 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:06:33.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:33.930 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-09-10T10:06:34.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:34.045 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-09-10T10:06:34.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:34.055 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-09-10T10:06:34.482 INFO:teuthology.orchestra.run.smithi110.stderr:Cannot find device "ceph-brx" 2024-09-10T10:06:34.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:06:34.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:34.485 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:06:34.496 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:06:34.496 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:06:34.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:34.498 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:06:34.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:34.562 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:06:34.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:34.963 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2024-09-10T10:06:35.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:35.135 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-09-10T10:06:35.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:06:35.145 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2024-09-10T10:06:35.625 INFO:teuthology.orchestra.run.smithi145.stderr:Cannot find device "ceph-brx" 2024-09-10T10:06:35.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:06:35.627 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T10:06:35.638 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:06:35.639 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:06:35.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f711ec00640 1 Processor -- start 2024-09-10T10:06:35.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f711ec00640 1 -- start start 2024-09-10T10:06:35.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f711ec00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f711815e690 0x7f711806e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:35.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f711ec00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f711806bc50 0x7f711806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:35.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f711ec00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f711806c710 con 0x7f71180ae840 2024-09-10T10:06:35.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f711ec00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f71180b4680 con 0x7f711815e690 2024-09-10T10:06:35.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f711ec00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f71180b4960 con 0x7f711806bc50 2024-09-10T10:06:35.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f7117e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f711815e690 0x7f711806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:35.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f7117e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f711815e690 0x7f711806e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51478/0 (socket says 172.21.15.110:51478) 2024-09-10T10:06:35.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f7117e00640 1 -- 172.21.15.110:0/2833308221 learned_addr learned my addr 172.21.15.110:0/2833308221 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:35.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f7117e00640 1 -- 172.21.15.110:0/2833308221 >> v2:172.21.15.145:3300/0 conn(0x7f711806bc50 msgr2=0x7f711806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:35.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f7117e00640 1 --2- 172.21.15.110:0/2833308221 >> v2:172.21.15.145:3300/0 conn(0x7f711806bc50 0x7f711806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f7117e00640 1 -- 172.21.15.110:0/2833308221 >> v1:172.21.15.145:6789/0 conn(0x7f71180ae840 legacy=0x7f71180b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:35.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.784+0000 7f7117e00640 1 -- 172.21.15.110:0/2833308221 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71180b4c40 con 0x7f711815e690 2024-09-10T10:06:35.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f7117e00640 1 --2- 172.21.15.110:0/2833308221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f711815e690 0x7f711806e240 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7118069820 tx=0x7f710001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787d8fe614f00e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:35.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f7116a00640 1 -- 172.21.15.110:0/2833308221 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000aa610 con 0x7f711815e690 2024-09-10T10:06:35.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f7116a00640 1 -- 172.21.15.110:0/2833308221 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71000af070 con 0x7f711815e690 2024-09-10T10:06:35.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f7116a00640 1 -- 172.21.15.110:0/2833308221 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000aa9d0 con 0x7f711815e690 2024-09-10T10:06:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f711ec00640 1 -- 172.21.15.110:0/2833308221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f711815e690 msgr2=0x7f711806e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f711ec00640 1 --2- 172.21.15.110:0/2833308221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f711815e690 0x7f711806e240 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7118069820 tx=0x7f710001c4b0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f711ec00640 1 -- 172.21.15.110:0/2833308221 shutdown_connections 2024-09-10T10:06:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f711ec00640 1 --2- 172.21.15.110:0/2833308221 >> v2:172.21.15.145:3300/0 conn(0x7f711806bc50 0x7f711806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f711ec00640 1 --2- 172.21.15.110:0/2833308221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f711815e690 0x7f711806e240 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f711ec00640 1 -- 172.21.15.110:0/2833308221 >> 172.21.15.110:0/2833308221 conn(0x7f71180d1530 msgr2=0x7f71180d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f711ec00640 1 -- 172.21.15.110:0/2833308221 shutdown_connections 2024-09-10T10:06:35.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.788+0000 7f711ec00640 1 -- 172.21.15.110:0/2833308221 wait complete. 2024-09-10T10:06:35.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 Processor -- start 2024-09-10T10:06:35.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 -- start start 2024-09-10T10:06:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f711806bc50 0x7f711815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:35.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71180ae840 0x7f7118154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711815e690 0x7f7118152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71180af0a0 con 0x7f71180ae840 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f71180b0aa0 con 0x7f711806bc50 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f711815ea70 con 0x7f711815e690 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71180ae840 0x7f7118154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71180ae840 0x7f7118154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51490/0 (socket says 172.21.15.110:51490) 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 -- 172.21.15.110:0/1921746326 learned_addr learned my addr 172.21.15.110:0/1921746326 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 -- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711815e690 msgr2=0x7f7118152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711815e690 0x7f7118152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 -- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f711806bc50 msgr2=0x7f711815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f711806bc50 0x7f711815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 -- 172.21.15.110:0/1921746326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711815fac0 con 0x7f71180ae840 2024-09-10T10:06:35.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f7117400640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71180ae840 0x7f7118154100 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f710800b550 tx=0x7f710800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:35.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f710fe00640 1 -- 172.21.15.110:0/1921746326 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108014070 con 0x7f71180ae840 2024-09-10T10:06:35.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f710fe00640 1 -- 172.21.15.110:0/1921746326 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7108010890 con 0x7f71180ae840 2024-09-10T10:06:35.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f710fe00640 1 -- 172.21.15.110:0/1921746326 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108010c80 con 0x7f71180ae840 2024-09-10T10:06:35.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7118154640 con 0x7f71180ae840 2024-09-10T10:06:35.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.792+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711815f7e0 con 0x7f71180ae840 2024-09-10T10:06:35.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.796+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711806c090 con 0x7f71180ae840 2024-09-10T10:06:35.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.796+0000 7f710fe00640 1 -- 172.21.15.110:0/1921746326 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f7108017070 con 0x7f71180ae840 2024-09-10T10:06:35.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.796+0000 7f710fe00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4068400 0x7f70f406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:35.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.796+0000 7f710fe00640 1 -- 172.21.15.110:0/1921746326 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f7108084660 con 0x7f71180ae840 2024-09-10T10:06:35.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.796+0000 7f7117e00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4068400 0x7f70f406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:35.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.796+0000 7f7117e00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4068400 0x7f70f406a8c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f71000ae000 tx=0x7f7100004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:35.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.800+0000 7f710fe00640 1 -- 172.21.15.110:0/1921746326 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f71080569a0 con 0x7f71180ae840 2024-09-10T10:06:35.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.956+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7118070910 con 0x7f70f4068400 2024-09-10T10:06:35.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.956+0000 7f710fe00640 1 -- 172.21.15.110:0/1921746326 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7f7118070910 con 0x7f70f4068400 2024-09-10T10:06:35.960 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:35.960 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4068400 msgr2=0x7f70f406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4068400 0x7f70f406a8c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f71000ae000 tx=0x7f7100004740 comp rx=0 tx=0).stop 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71180ae840 msgr2=0x7f7118154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71180ae840 0x7f7118154100 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f710800b550 tx=0x7f710800ba10 comp rx=0 tx=0).stop 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 shutdown_connections 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f70f4068400 0x7f70f406a8c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f711815e690 0x7f7118152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71180ae840 0x7f7118154100 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 --2- 172.21.15.110:0/1921746326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f711806bc50 0x7f711815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 >> 172.21.15.110:0/1921746326 conn(0x7f71180d1530 msgr2=0x7f71180b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 shutdown_connections 2024-09-10T10:06:35.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:35.960+0000 7f711ec00640 1 -- 172.21.15.110:0/1921746326 wait complete. 2024-09-10T10:06:35.980 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-10T10:06:35.755680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283712,"kb_used_data":17568,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691008,"statfs":{"total":773094113280,"available":772803592192,"internally_reserved":0,"allocated":17989632,"data_stored":8592597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53492,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-73,"num_write_kb":-79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.567465"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":476,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T10:06:31.348497+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:06:31.348497+0000","last_peered":"2024-09-10T10:06:31.348497+0000","last_clean":"2024-09-10T10:06:31.348497+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:06:31.348497+0000","last_undegraded":"2024-09-10T10:06:31.348497+0000","last_fullsized":"2024-09-10T10:06:31.348497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2699264,"data_stored":1519674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2097152,"data_stored":924132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2093056,"data_stored":923829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":929306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542248,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2101248,"data_stored":923429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509543,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2088960,"data_stored":923247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37404,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334436,"statfs":{"total":96636764160,"available":96598462464,"internally_reserved":0,"allocated":2105344,"data_stored":929306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509543,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37984,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333856,"statfs":{"total":96636764160,"available":96597868544,"internally_reserved":0,"allocated":2699264,"data_stored":1519674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:06:35.981 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:06:36.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe336200640 1 Processor -- start 2024-09-10T10:06:36.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe336200640 1 -- start start 2024-09-10T10:06:36.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe336200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe3300d0870 0x7fe3301672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe336200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3300b3dd0 con 0x7fe33015c6d0 2024-09-10T10:06:36.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe336200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3300d0d40 con 0x7fe33006c360 2024-09-10T10:06:36.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe336200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe33006e6f0 con 0x7fe3300d0870 2024-09-10T10:06:36.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32f400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe33015c6d0 0x7fe33006bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41356/0 (socket says 172.21.15.110:41356) 2024-09-10T10:06:36.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32f400640 1 -- 172.21.15.110:0/103600096 learned_addr learned my addr 172.21.15.110:0/103600096 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:36.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32fe00640 1 --2- 172.21.15.110:0/103600096 >> v2:172.21.15.145:3300/0 conn(0x7fe3300d0870 0x7fe3301672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370298969 0 0) 0x7fe320001120 con 0x7fe33015c6d0 2024-09-10T10:06:36.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe314003530 con 0x7fe33015c6d0 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 <== mon.1 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823056266 0 0) 0x7fe3300d0d40 con 0x7fe33006c360 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe320001120 con 0x7fe33006c360 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3096675074 0 0) 0x7fe314003530 con 0x7fe33015c6d0 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 >> v2:172.21.15.145:3300/0 conn(0x7fe3300d0870 msgr2=0x7fe3301672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 --2- 172.21.15.110:0/103600096 >> v2:172.21.15.145:3300/0 conn(0x7fe3300d0870 0x7fe3301672d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 >> v1:172.21.15.145:6789/0 conn(0x7fe33006c360 legacy=0x7fe3300d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3300d1020 con 0x7fe33015c6d0 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fe320001850 con 0x7fe33015c6d0 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 >> v1:172.21.15.110:6789/0 conn(0x7fe33015c6d0 legacy=0x7fe33006bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314003b50 0x7fe314003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe314004810 0x7fe314004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe314005240 0x7fe31400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe33006e6f0 con 0x7fe314005240 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe31400c1a0 con 0x7fe314004810 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe31400c3a0 con 0x7fe314003b50 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32fe00640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe314005240 0x7fe31400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32f400640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314003b50 0x7fe314003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe327e00640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe314004810 0x7fe314004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32fe00640 1 -- 172.21.15.110:0/103600096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314003b50 msgr2=0x7fe314003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32fe00640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314003b50 0x7fe314003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32fe00640 1 -- 172.21.15.110:0/103600096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe314004810 msgr2=0x7fe314004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32fe00640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe314004810 0x7fe314004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.116+0000 7fe32fe00640 1 -- 172.21.15.110:0/103600096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe314003530 con 0x7fe314005240 2024-09-10T10:06:36.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe32fe00640 1 -- 172.21.15.110:0/103600096 reap_dead start 2024-09-10T10:06:36.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe32fe00640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe314005240 0x7fe31400b9d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe328009fc0 tx=0x7fe32801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a842dc32dc9dae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328009c10 con 0x7fe314005240 2024-09-10T10:06:36.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3280a5c00 con 0x7fe314005240 2024-09-10T10:06:36.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe32ea00640 1 -- 172.21.15.110:0/103600096 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3280039a0 con 0x7fe314005240 2024-09-10T10:06:36.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe336200640 1 -- 172.21.15.110:0/103600096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe314005240 msgr2=0x7fe31400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe336200640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe314005240 0x7fe31400b9d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe328009fc0 tx=0x7fe32801ccc0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe336200640 1 -- 172.21.15.110:0/103600096 shutdown_connections 2024-09-10T10:06:36.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe336200640 1 --2- 172.21.15.110:0/103600096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe314005240 0x7fe31400b9d0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe336200640 1 -- 172.21.15.110:0/103600096 >> 172.21.15.110:0/103600096 conn(0x7fe33006d360 msgr2=0x7fe3300b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:36.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe336200640 1 -- 172.21.15.110:0/103600096 shutdown_connections 2024-09-10T10:06:36.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.120+0000 7fe336200640 1 -- 172.21.15.110:0/103600096 wait complete. 2024-09-10T10:06:36.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 Processor -- start 2024-09-10T10:06:36.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 -- start start 2024-09-10T10:06:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314005240 0x7fe330152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3301540e0 0x7fe330155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe330156010 0x7fe330159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3300d04f0 con 0x7fe330156010 2024-09-10T10:06:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe3300d0370 con 0x7fe3301540e0 2024-09-10T10:06:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe3300d0670 con 0x7fe314005240 2024-09-10T10:06:36.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe330156010 0x7fe330159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe330156010 0x7fe330159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51508/0 (socket says 172.21.15.110:51508) 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 -- 172.21.15.110:0/2398886286 learned_addr learned my addr 172.21.15.110:0/2398886286 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe327e00640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3301540e0 0x7fe330155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32f400640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314005240 0x7fe330152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 -- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314005240 msgr2=0x7fe330152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314005240 0x7fe330152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 -- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3301540e0 msgr2=0x7fe330155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3301540e0 0x7fe330155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 -- 172.21.15.110:0/2398886286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3300b4600 con 0x7fe330156010 2024-09-10T10:06:36.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe32fe00640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe330156010 0x7fe330159010 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe3280ae930 tx=0x7fe328009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe327400640 1 -- 172.21.15.110:0/2398886286 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328009c10 con 0x7fe330156010 2024-09-10T10:06:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe327400640 1 -- 172.21.15.110:0/2398886286 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3280bd070 con 0x7fe330156010 2024-09-10T10:06:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe327400640 1 -- 172.21.15.110:0/2398886286 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328003ba0 con 0x7fe330156010 2024-09-10T10:06:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3300b6030 con 0x7fe330156010 2024-09-10T10:06:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.124+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3300b48a0 con 0x7fe330156010 2024-09-10T10:06:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.128+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33006bc50 con 0x7fe330156010 2024-09-10T10:06:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.128+0000 7fe327400640 1 -- 172.21.15.110:0/2398886286 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fe3280ae990 con 0x7fe330156010 2024-09-10T10:06:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.128+0000 7fe327400640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe30c068400 0x7fe30c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.128+0000 7fe327400640 1 -- 172.21.15.110:0/2398886286 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fe32811fc90 con 0x7fe330156010 2024-09-10T10:06:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.128+0000 7fe32f400640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe30c068400 0x7fe30c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.128+0000 7fe32f400640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe30c068400 0x7fe30c06a8c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe320000c10 tx=0x7fe320007a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.132+0000 7fe327400640 1 -- 172.21.15.110:0/2398886286 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe3280169d0 con 0x7fe330156010 2024-09-10T10:06:36.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.288+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe3301530e0 con 0x7fe30c068400 2024-09-10T10:06:36.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe327400640 1 -- 172.21.15.110:0/2398886286 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7fe3301530e0 con 0x7fe30c068400 2024-09-10T10:06:36.293 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:36.293 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:06:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe30c068400 msgr2=0x7fe30c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe30c068400 0x7fe30c06a8c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe320000c10 tx=0x7fe320007a30 comp rx=0 tx=0).stop 2024-09-10T10:06:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe330156010 msgr2=0x7fe330159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe330156010 0x7fe330159010 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe3280ae930 tx=0x7fe328009750 comp rx=0 tx=0).stop 2024-09-10T10:06:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 shutdown_connections 2024-09-10T10:06:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe30c068400 0x7fe30c06a8c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe330156010 0x7fe330159010 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3301540e0 0x7fe330155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 --2- 172.21.15.110:0/2398886286 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe314005240 0x7fe330152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 >> 172.21.15.110:0/2398886286 conn(0x7fe33006d360 msgr2=0x7fe3300d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 shutdown_connections 2024-09-10T10:06:36.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.292+0000 7fe336200640 1 -- 172.21.15.110:0/2398886286 wait complete. 2024-09-10T10:06:36.314 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-10T10:06:35.755680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283712,"kb_used_data":17568,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691008,"statfs":{"total":773094113280,"available":772803592192,"internally_reserved":0,"allocated":17989632,"data_stored":8592597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53492,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-73,"num_write_kb":-79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.567465"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":476,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T10:06:31.348497+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:06:31.348497+0000","last_peered":"2024-09-10T10:06:31.348497+0000","last_clean":"2024-09-10T10:06:31.348497+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:06:31.348497+0000","last_undegraded":"2024-09-10T10:06:31.348497+0000","last_fullsized":"2024-09-10T10:06:31.348497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2699264,"data_stored":1519674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2097152,"data_stored":924132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2093056,"data_stored":923829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":929306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542248,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2101248,"data_stored":923429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509543,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2088960,"data_stored":923247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37404,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334436,"statfs":{"total":96636764160,"available":96598462464,"internally_reserved":0,"allocated":2105344,"data_stored":929306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509543,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37984,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333856,"statfs":{"total":96636764160,"available":96597868544,"internally_reserved":0,"allocated":2699264,"data_stored":1519674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:06:36.314 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:06:36.315 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:06:36.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62efc00640 1 Processor -- start 2024-09-10T10:06:36.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62efc00640 1 -- start start 2024-09-10T10:06:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62efc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f62e815c560 0x7f62e806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62efc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e806f430 con 0x7f62e8155970 2024-09-10T10:06:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62efc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e8155d50 con 0x7f62e806f010 2024-09-10T10:06:36.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62efc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f62e80b4570 con 0x7f62e815c560 2024-09-10T10:06:36.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62ed200640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f62e8155970 0x7f62e80cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41372/0 (socket says 172.21.15.110:41372) 2024-09-10T10:06:36.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62ed200640 1 -- 172.21.15.110:0/1412705442 learned_addr learned my addr 172.21.15.110:0/1412705442 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:36.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62e7e00640 1 --2- 172.21.15.110:0/1412705442 >> v2:172.21.15.145:3300/0 conn(0x7f62e815c560 0x7f62e806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.448+0000 7f62e7400640 1 -- 172.21.15.110:0/1412705442 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249724172 0 0) 0x7f62e806f430 con 0x7f62e8155970 2024-09-10T10:06:36.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7400640 1 -- 172.21.15.110:0/1412705442 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62cc005740 con 0x7f62e8155970 2024-09-10T10:06:36.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7400640 1 -- 172.21.15.110:0/1412705442 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2399242971 0 0) 0x7f62cc005740 con 0x7f62e8155970 2024-09-10T10:06:36.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7e00640 1 -- 172.21.15.110:0/1412705442 >> v1:172.21.15.145:6789/0 conn(0x7f62e806f010 legacy=0x7f62e80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7e00640 1 -- 172.21.15.110:0/1412705442 >> v1:172.21.15.110:6789/0 conn(0x7f62e8155970 legacy=0x7f62e80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7e00640 1 -- 172.21.15.110:0/1412705442 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e80b34d0 con 0x7f62e815c560 2024-09-10T10:06:36.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7e00640 1 --2- 172.21.15.110:0/1412705442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 0x7f62e806ead0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f62d40097c0 tx=0x7f62d401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c2aacd91acd07d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7400640 1 -- 172.21.15.110:0/1412705442 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40b1070 con 0x7f62e815c560 2024-09-10T10:06:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7400640 1 -- 172.21.15.110:0/1412705442 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d40ac440 con 0x7f62e815c560 2024-09-10T10:06:36.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.452+0000 7f62e7400640 1 -- 172.21.15.110:0/1412705442 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8001a00 con 0x7f62e815c560 2024-09-10T10:06:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- 172.21.15.110:0/1412705442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 msgr2=0x7f62e806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 --2- 172.21.15.110:0/1412705442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 0x7f62e806ead0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f62d40097c0 tx=0x7f62d401c390 comp rx=0 tx=0).stop 2024-09-10T10:06:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- 172.21.15.110:0/1412705442 shutdown_connections 2024-09-10T10:06:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 --2- 172.21.15.110:0/1412705442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 0x7f62e806ead0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- 172.21.15.110:0/1412705442 >> 172.21.15.110:0/1412705442 conn(0x7f62e806bc50 msgr2=0x7f62e806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- 172.21.15.110:0/1412705442 shutdown_connections 2024-09-10T10:06:36.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- 172.21.15.110:0/1412705442 wait complete. 2024-09-10T10:06:36.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 Processor -- start 2024-09-10T10:06:36.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- start start 2024-09-10T10:06:36.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62e806f010 0x7f62e8145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e8155970 0x7f62e813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 0x7f62e8140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f62e80cf340 con 0x7f62e8155970 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62e80b4050 con 0x7f62e815c560 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62efc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f62e80cf4c0 con 0x7f62e806f010 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62e7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e8155970 0x7f62e813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62e7e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e8155970 0x7f62e813faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51518/0 (socket says 172.21.15.110:51518) 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62e7e00640 1 -- 172.21.15.110:0/2438370828 learned_addr learned my addr 172.21.15.110:0/2438370828 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62ed200640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62e806f010 0x7f62e8145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62edc00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 0x7f62e8140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62e7e00640 1 -- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62e806f010 msgr2=0x7f62e8145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62e7e00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62e806f010 0x7f62e8145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62e7e00640 1 -- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 msgr2=0x7f62e8140030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62e7e00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 0x7f62e8140030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62e7e00640 1 -- 172.21.15.110:0/2438370828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e8140660 con 0x7f62e8155970 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62ed200640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62e806f010 0x7f62e8145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.456+0000 7f62edc00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 0x7f62e8140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62e7e00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e8155970 0x7f62e813faf0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f62d4018fa0 tx=0x7f62d401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62e4c00640 1 -- 172.21.15.110:0/2438370828 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40b1050 con 0x7f62e8155970 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62e4c00640 1 -- 172.21.15.110:0/2438370828 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d40bd070 con 0x7f62e8155970 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62e4c00640 1 -- 172.21.15.110:0/2438370828 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4006e20 con 0x7f62e8155970 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e8140940 con 0x7f62e8155970 2024-09-10T10:06:36.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e8140f00 con 0x7f62e8155970 2024-09-10T10:06:36.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e80ce420 con 0x7f62e8155970 2024-09-10T10:06:36.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62e4c00640 1 -- 172.21.15.110:0/2438370828 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f62d40b8850 con 0x7f62e8155970 2024-09-10T10:06:36.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62e4c00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62c8068400 0x7f62c806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.460+0000 7f62e4c00640 1 -- 172.21.15.110:0/2438370828 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f62d411f020 con 0x7f62e8155970 2024-09-10T10:06:36.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.464+0000 7f62ed200640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62c8068400 0x7f62c806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.464+0000 7f62ed200640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62c8068400 0x7f62c806a8c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f62d8001b70 tx=0x7f62d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.468+0000 7f62e4c00640 1 -- 172.21.15.110:0/2438370828 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f62d4015fb0 con 0x7f62e8155970 2024-09-10T10:06:36.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.620+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62e80cf980 con 0x7f62c8068400 2024-09-10T10:06:36.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.620+0000 7f62e4c00640 1 -- 172.21.15.110:0/2438370828 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13482 (secure 0 0 0) 0x7f62e80cf980 con 0x7f62c8068400 2024-09-10T10:06:36.623 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:06:36.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62c8068400 msgr2=0x7f62c806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62c8068400 0x7f62c806a8c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f62d8001b70 tx=0x7f62d8009040 comp rx=0 tx=0).stop 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e8155970 msgr2=0x7f62e813faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e8155970 0x7f62e813faf0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f62d4018fa0 tx=0x7f62d401abd0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 shutdown_connections 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f62c8068400 0x7f62c806a8c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62e815c560 0x7f62e8140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62e8155970 0x7f62e813faf0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 --2- 172.21.15.110:0/2438370828 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f62e806f010 0x7f62e8145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 >> 172.21.15.110:0/2438370828 conn(0x7f62e806bc50 msgr2=0x7f62e80710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:36.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 shutdown_connections 2024-09-10T10:06:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.624+0000 7f62efc00640 1 -- 172.21.15.110:0/2438370828 wait complete. 2024-09-10T10:06:36.644 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-10T10:06:35.755680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283712,"kb_used_data":17568,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691008,"statfs":{"total":773094113280,"available":772803592192,"internally_reserved":0,"allocated":17989632,"data_stored":8592597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53492,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-73,"num_write_kb":-79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.567465"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":476,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T10:06:31.348497+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:06:31.348497+0000","last_peered":"2024-09-10T10:06:31.348497+0000","last_clean":"2024-09-10T10:06:31.348497+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:06:31.348497+0000","last_undegraded":"2024-09-10T10:06:31.348497+0000","last_fullsized":"2024-09-10T10:06:31.348497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2699264,"data_stored":1519674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2097152,"data_stored":924132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2093056,"data_stored":923829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":929306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542248,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2101248,"data_stored":923429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509543,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2088960,"data_stored":923247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37404,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334436,"statfs":{"total":96636764160,"available":96598462464,"internally_reserved":0,"allocated":2105344,"data_stored":929306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509543,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37984,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333856,"statfs":{"total":96636764160,"available":96597868544,"internally_reserved":0,"allocated":2699264,"data_stored":1519674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:06:36.644 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:06:36.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.780+0000 7fb1c4400640 1 Processor -- start 2024-09-10T10:06:36.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.780+0000 7fb1c4400640 1 -- start start 2024-09-10T10:06:36.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c4400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 0x7fb1bc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c4400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb1bc06f1e0 0x7fb1bc0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c4400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1bc155d50 con 0x7fb1bc15c560 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c4400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fb1bc0b4570 con 0x7fb1bc06f1e0 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c4400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb1bc0b34d0 con 0x7fb1bc155970 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c2400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb1bc06f1e0 0x7fb1bc0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c1000640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fb1bc15c560 0x7fb1bc06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41388/0 (socket says 172.21.15.110:41388) 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c2400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb1bc06f1e0 0x7fb1bc0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51520/0 (socket says 172.21.15.110:51520) 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c1000640 1 -- 172.21.15.110:0/2301314509 learned_addr learned my addr 172.21.15.110:0/2301314509 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c1a00640 1 --2- 172.21.15.110:0/2301314509 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 0x7fb1bc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c2400640 1 -- 172.21.15.110:0/2301314509 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 msgr2=0x7fb1bc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c2400640 1 --2- 172.21.15.110:0/2301314509 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 0x7fb1bc0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c2400640 1 -- 172.21.15.110:0/2301314509 >> v1:172.21.15.110:6789/0 conn(0x7fb1bc15c560 legacy=0x7fb1bc06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.784+0000 7fb1c2400640 1 -- 172.21.15.110:0/2301314509 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1bc0b37b0 con 0x7fb1bc06f1e0 2024-09-10T10:06:36.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c2400640 1 --2- 172.21.15.110:0/2301314509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc06f1e0 0x7fb1bc0ce420 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009d90 tx=0x7fb1b401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a3b9095303b21f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1bbe00640 1 -- 172.21.15.110:0/2301314509 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40b2070 con 0x7fb1bc06f1e0 2024-09-10T10:06:36.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1bbe00640 1 -- 172.21.15.110:0/2301314509 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b40a5970 con 0x7fb1bc06f1e0 2024-09-10T10:06:36.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1bbe00640 1 -- 172.21.15.110:0/2301314509 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40a5d40 con 0x7fb1bc06f1e0 2024-09-10T10:06:36.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 -- 172.21.15.110:0/2301314509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc06f1e0 msgr2=0x7fb1bc0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 --2- 172.21.15.110:0/2301314509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc06f1e0 0x7fb1bc0ce420 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009d90 tx=0x7fb1b401c8b0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 -- 172.21.15.110:0/2301314509 shutdown_connections 2024-09-10T10:06:36.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 --2- 172.21.15.110:0/2301314509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc06f1e0 0x7fb1bc0ce420 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 --2- 172.21.15.110:0/2301314509 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 0x7fb1bc0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 -- 172.21.15.110:0/2301314509 >> 172.21.15.110:0/2301314509 conn(0x7fb1bc06bc50 msgr2=0x7fb1bc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:36.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 -- 172.21.15.110:0/2301314509 shutdown_connections 2024-09-10T10:06:36.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 -- 172.21.15.110:0/2301314509 wait complete. 2024-09-10T10:06:36.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 Processor -- start 2024-09-10T10:06:36.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.788+0000 7fb1c4400640 1 -- start start 2024-09-10T10:06:36.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1bc06f1e0 0x7fb1bc155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 0x7fb1bc0ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc15c560 0x7fb1bc0aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc0cf170 con 0x7fb1bc15c560 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc0cce00 con 0x7fb1bc06f1e0 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fb1bc0cf2f0 con 0x7fb1bc155970 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc15c560 0x7fb1bc0aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc15c560 0x7fb1bc0aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51532/0 (socket says 172.21.15.110:51532) 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 -- 172.21.15.110:0/3245744935 learned_addr learned my addr 172.21.15.110:0/3245744935 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 -- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 msgr2=0x7fb1bc0ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 0x7fb1bc0ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 -- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1bc06f1e0 msgr2=0x7fb1bc155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1bc06f1e0 0x7fb1bc155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 -- 172.21.15.110:0/3245744935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1bc0adbc0 con 0x7fb1bc15c560 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c2400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc15c560 0x7fb1bc0aba30 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009d90 tx=0x7fb1b4019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1b9600640 1 -- 172.21.15.110:0/3245744935 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40b2050 con 0x7fb1bc15c560 2024-09-10T10:06:36.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1b9600640 1 -- 172.21.15.110:0/3245744935 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b40be070 con 0x7fb1bc15c560 2024-09-10T10:06:36.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1b9600640 1 -- 172.21.15.110:0/3245744935 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40adaf0 con 0x7fb1bc15c560 2024-09-10T10:06:36.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1bc0ac1a0 con 0x7fb1bc15c560 2024-09-10T10:06:36.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1bc0ae140 con 0x7fb1bc15c560 2024-09-10T10:06:36.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1bc0ce420 con 0x7fb1bc15c560 2024-09-10T10:06:36.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.792+0000 7fb1b9600640 1 -- 172.21.15.110:0/3245744935 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fb1b40adc90 con 0x7fb1bc15c560 2024-09-10T10:06:36.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.796+0000 7fb1b9600640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb19c068400 0x7fb19c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:36.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.796+0000 7fb1b9600640 1 -- 172.21.15.110:0/3245744935 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fb1b411fa30 con 0x7fb1bc15c560 2024-09-10T10:06:36.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.796+0000 7fb1c1a00640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb19c068400 0x7fb19c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:36.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.796+0000 7fb1c1a00640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb19c068400 0x7fb19c06a8c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00098f0 tx=0x7fb1b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:36.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.800+0000 7fb1b9600640 1 -- 172.21.15.110:0/3245744935 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb1b4016560 con 0x7fb1bc15c560 2024-09-10T10:06:36.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1bc1548b0 con 0x7fb1bc15c560 2024-09-10T10:06:36.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1b9600640 1 -- 172.21.15.110:0/3245744935 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+13169 (secure 0 0 0) 0x7fb1b40ab340 con 0x7fb1bc15c560 2024-09-10T10:06:36.958 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:36.958 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":168,"fsid":"af5c1104-e6d8-4fa7-9635-0686bce1580a","created":"2024-09-10T09:57:58.312942+0000","modified":"2024-09-10T10:06:32.189184+0000","last_up_change":"2024-09-10T09:58:12.198411+0000","last_in_change":"2024-09-10T09:58:00.807874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:58:13.418450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"936d59a8-3d51-4d57-b195-7d9187cf8116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6815","nonce":321203351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6817","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6819","nonce":321203351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6823","nonce":321203351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":321203351},{"type":"v1","addr":"172.21.15.110:6821","nonce":321203351}]},"public_addr":"172.21.15.110:6815/321203351","cluster_addr":"172.21.15.110:6819/321203351","heartbeat_back_addr":"172.21.15.110:6823/321203351","heartbeat_front_addr":"172.21.15.110:6821/321203351","state":["exists","up"]},{"osd":1,"uuid":"80b34968-15c0-4a88-864f-d59bb573fc12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6809","nonce":3662700555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6811","nonce":3662700555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6818","nonce":3662700555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3662700555},{"type":"v1","addr":"172.21.15.110:6813","nonce":3662700555}]},"public_addr":"172.21.15.110:6809/3662700555","cluster_addr":"172.21.15.110:6811/3662700555","heartbeat_back_addr":"172.21.15.110:6818/3662700555","heartbeat_front_addr":"172.21.15.110:6813/3662700555","state":["exists","up"]},{"osd":2,"uuid":"5319529e-6347-4786-9daf-ad1857d2912b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6801","nonce":1201840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6803","nonce":1201840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6807","nonce":1201840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1201840781},{"type":"v1","addr":"172.21.15.110:6805","nonce":1201840781}]},"public_addr":"172.21.15.110:6801/1201840781","cluster_addr":"172.21.15.110:6803/1201840781","heartbeat_back_addr":"172.21.15.110:6807/1201840781","heartbeat_front_addr":"172.21.15.110:6805/1201840781","state":["exists","up"]},{"osd":3,"uuid":"4a19b2ea-bfc7-4cec-a6d2-e837a89b2260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6825","nonce":4112166746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6827","nonce":4112166746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6831","nonce":4112166746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4112166746},{"type":"v1","addr":"172.21.15.110:6829","nonce":4112166746}]},"public_addr":"172.21.15.110:6825/4112166746","cluster_addr":"172.21.15.110:6827/4112166746","heartbeat_back_addr":"172.21.15.110:6831/4112166746","heartbeat_front_addr":"172.21.15.110:6829/4112166746","state":["exists","up"]},{"osd":4,"uuid":"2ff23641-e17c-4abe-a494-ab882bf527b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6817","nonce":927410831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6819","nonce":927410831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6823","nonce":927410831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":927410831},{"type":"v1","addr":"172.21.15.145:6821","nonce":927410831}]},"public_addr":"172.21.15.145:6817/927410831","cluster_addr":"172.21.15.145:6819/927410831","heartbeat_back_addr":"172.21.15.145:6823/927410831","heartbeat_front_addr":"172.21.15.145:6821/927410831","state":["exists","up"]},{"osd":5,"uuid":"ddbaf2e3-cfba-4c75-a56d-0c32e415dd17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6809","nonce":1843342183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6811","nonce":1843342183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6815","nonce":1843342183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":1843342183},{"type":"v1","addr":"172.21.15.145:6813","nonce":1843342183}]},"public_addr":"172.21.15.145:6809/1843342183","cluster_addr":"172.21.15.145:6811/1843342183","heartbeat_back_addr":"172.21.15.145:6815/1843342183","heartbeat_front_addr":"172.21.15.145:6813/1843342183","state":["exists","up"]},{"osd":6,"uuid":"5150a649-1978-44bc-a1ee-8efa95bad937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6825","nonce":3155222893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6827","nonce":3155222893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6831","nonce":3155222893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":3155222893},{"type":"v1","addr":"172.21.15.145:6829","nonce":3155222893}]},"public_addr":"172.21.15.145:6825/3155222893","cluster_addr":"172.21.15.145:6827/3155222893","heartbeat_back_addr":"172.21.15.145:6831/3155222893","heartbeat_front_addr":"172.21.15.145:6829/3155222893","state":["exists","up"]},{"osd":7,"uuid":"1c218083-28ff-425a-bb60-86a19ed07a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6801","nonce":938429327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6803","nonce":938429327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6807","nonce":938429327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":938429327},{"type":"v1","addr":"172.21.15.145:6805","nonce":938429327}]},"public_addr":"172.21.15.145:6801/938429327","cluster_addr":"172.21.15.145:6803/938429327","heartbeat_back_addr":"172.21.15.145:6807/938429327","heartbeat_front_addr":"172.21.15.145:6805/938429327","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.981436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:09.027323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.986124+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.706199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.857763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.840483+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.954325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:08.813669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:6835/4014830481":"2024-09-11T10:06:29.176997+0000","172.21.15.145:6833/4014830481":"2024-09-11T10:06:29.176997+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:06:36.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb19c068400 msgr2=0x7fb19c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb19c068400 0x7fb19c06a8c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00098f0 tx=0x7fb1b0008040 comp rx=0 tx=0).stop 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc15c560 msgr2=0x7fb1bc0aba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc15c560 0x7fb1bc0aba30 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009d90 tx=0x7fb1b4019650 comp rx=0 tx=0).stop 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 shutdown_connections 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fb19c068400 0x7fb19c06a8c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1bc15c560 0x7fb1bc0aba30 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fb1bc155970 0x7fb1bc0ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 --2- 172.21.15.110:0/3245744935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1bc06f1e0 0x7fb1bc155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:36.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 >> 172.21.15.110:0/3245744935 conn(0x7fb1bc06bc50 msgr2=0x7fb1bc154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:36.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 shutdown_connections 2024-09-10T10:06:36.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:36.956+0000 7fb1c4400640 1 -- 172.21.15.110:0/3245744935 wait complete. 2024-09-10T10:06:37.979 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T10:06:37.980 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:06:38.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.128+0000 7f3efcc00640 1 Processor -- start 2024-09-10T10:06:38.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.128+0000 7f3efcc00640 1 -- start start 2024-09-10T10:06:38.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3efcc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3ef80d05c0 0x7f3ef80d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3efcc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef815b970 con 0x7f3ef80cf540 2024-09-10T10:06:38.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3efcc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef80d0ee0 con 0x7f3ef80b59c0 2024-09-10T10:06:38.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3efcc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef815bc10 con 0x7f3ef80d05c0 2024-09-10T10:06:38.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3ef7e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f3ef80cf540 0x7f3ef80cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:41392/0 (socket says 172.21.15.110:41392) 2024-09-10T10:06:38.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3ef7e00640 1 -- 172.21.15.110:0/1446596265 learned_addr learned my addr 172.21.15.110:0/1446596265 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:38.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3ef7400640 1 --2- 172.21.15.110:0/1446596265 >> v2:172.21.15.145:3300/0 conn(0x7f3ef80d05c0 0x7f3ef80d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3ef6000640 1 -- 172.21.15.110:0/1446596265 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773254080 0 0) 0x7f3ef815b970 con 0x7f3ef80cf540 2024-09-10T10:06:38.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3ef6000640 1 -- 172.21.15.110:0/1446596265 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3edc005740 con 0x7f3ef80cf540 2024-09-10T10:06:38.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3ef7400640 1 -- 172.21.15.110:0/1446596265 >> v1:172.21.15.145:6789/0 conn(0x7f3ef80b59c0 legacy=0x7f3ef806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3ef7400640 1 -- 172.21.15.110:0/1446596265 >> v1:172.21.15.110:6789/0 conn(0x7f3ef80cf540 legacy=0x7f3ef80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.132+0000 7f3ef7400640 1 -- 172.21.15.110:0/1446596265 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef815bef0 con 0x7f3ef80d05c0 2024-09-10T10:06:38.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3ef7400640 1 --2- 172.21.15.110:0/1446596265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80d05c0 0x7f3ef80d09a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80097c0 tx=0x7f3ee801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f83f93104b3e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3ef6000640 1 -- 172.21.15.110:0/1446596265 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80b0070 con 0x7f3ef80d05c0 2024-09-10T10:06:38.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3ef6000640 1 -- 172.21.15.110:0/1446596265 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee80ab600 con 0x7f3ef80d05c0 2024-09-10T10:06:38.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3ef6000640 1 -- 172.21.15.110:0/1446596265 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80ab9d0 con 0x7f3ef80d05c0 2024-09-10T10:06:38.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 -- 172.21.15.110:0/1446596265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80d05c0 msgr2=0x7f3ef80d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 --2- 172.21.15.110:0/1446596265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80d05c0 0x7f3ef80d09a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80097c0 tx=0x7f3ee801c470 comp rx=0 tx=0).stop 2024-09-10T10:06:38.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 -- 172.21.15.110:0/1446596265 shutdown_connections 2024-09-10T10:06:38.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 --2- 172.21.15.110:0/1446596265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80d05c0 0x7f3ef80d09a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 -- 172.21.15.110:0/1446596265 >> 172.21.15.110:0/1446596265 conn(0x7f3ef8062ce0 msgr2=0x7f3ef80b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:38.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 -- 172.21.15.110:0/1446596265 shutdown_connections 2024-09-10T10:06:38.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 -- 172.21.15.110:0/1446596265 wait complete. 2024-09-10T10:06:38.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 Processor -- start 2024-09-10T10:06:38.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.136+0000 7f3efcc00640 1 -- start start 2024-09-10T10:06:38.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80b59c0 0x7f3ef814e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ef80cf540 0x7f3ef814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef80d05c0 0x7f3ef8150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef806e6b0 con 0x7f3ef80cf540 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef806e530 con 0x7f3ef80b59c0 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef80cd940 con 0x7f3ef80d05c0 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ef80cf540 0x7f3ef814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ef80cf540 0x7f3ef814ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51536/0 (socket says 172.21.15.110:51536) 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 -- 172.21.15.110:0/838046960 learned_addr learned my addr 172.21.15.110:0/838046960 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef7400640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80b59c0 0x7f3ef814e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef7e00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef80d05c0 0x7f3ef8150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 -- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef80d05c0 msgr2=0x7f3ef8150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f3ef80d05c0 0x7f3ef8150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 -- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80b59c0 msgr2=0x7f3ef814e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80b59c0 0x7f3ef814e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 -- 172.21.15.110:0/838046960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef814cf40 con 0x7f3ef80cf540 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef7400640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ef80b59c0 0x7f3ef814e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3ef6a00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ef80cf540 0x7f3ef814ca00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c450 tx=0x7f3ee400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4016020 con 0x7f3ef80cf540 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee40052d0 con 0x7f3ef80cf540 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4007260 con 0x7f3ef80cf540 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef8150880 con 0x7f3ef80cf540 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef8150de0 con 0x7f3ef80cf540 2024-09-10T10:06:38.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ef80cdd00 con 0x7f3ef80cf540 2024-09-10T10:06:38.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.140+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f3ee4007400 con 0x7f3ef80cf540 2024-09-10T10:06:38.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3eef400640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ed8068400 0x7f3ed806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f3ee40873d0 con 0x7f3ef80cf540 2024-09-10T10:06:38.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3eef400640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f3ed806e2c0 0x7f3ed8070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 --> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ee4017e70 con 0x7f3ed806e2c0 2024-09-10T10:06:38.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3ef7400640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ed8068400 0x7f3ed806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=168) ==== 24+0+0 (secure 0 0 0) 0x7f3ee40875f0 con 0x7f3ef80cf540 2024-09-10T10:06:38.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3ef7e00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f3ed806e2c0 0x7f3ed8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3ef7e00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f3ed806e2c0 0x7f3ed8070700 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3ef7400640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ed8068400 0x7f3ed806a8c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80ad000 tx=0x7f3ee80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.144+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 <== osd.0 v2:172.21.15.110:6814/321203351 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ee4017e70 con 0x7f3ed806e2c0 2024-09-10T10:06:38.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 --> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3ef80b4800 con 0x7f3ed806e2c0 2024-09-10T10:06:38.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3eef400640 1 -- 172.21.15.110:0/838046960 <== osd.0 v2:172.21.15.110:6814/321203351 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3ef80b4800 con 0x7f3ed806e2c0 2024-09-10T10:06:38.159 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:38.160 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:06:38.160 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f3ed806e2c0 msgr2=0x7f3ed8070700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.110:6814/321203351,v1:172.21.15.110:6815/321203351] conn(0x7f3ed806e2c0 0x7f3ed8070700 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ed8068400 msgr2=0x7f3ed806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f3ed8068400 0x7f3ed806a8c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80ad000 tx=0x7f3ee80a6000 comp rx=0 tx=0).stop 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ef80cf540 msgr2=0x7f3ef814ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 --2- 172.21.15.110:0/838046960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ef80cf540 0x7f3ef814ca00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c450 tx=0x7f3ee400c910 comp rx=0 tx=0).stop 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3ef7e00640 1 -- 172.21.15.110:0/838046960 reap_dead start 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 shutdown_connections 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.156+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 >> 172.21.15.110:0/838046960 conn(0x7f3ef8062ce0 msgr2=0x7f3ef8072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.160+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 shutdown_connections 2024-09-10T10:06:38.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.160+0000 7f3efcc00640 1 -- 172.21.15.110:0/838046960 wait complete. 2024-09-10T10:06:38.178 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T10:06:38.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.312+0000 7fc5ccc00640 1 Processor -- start 2024-09-10T10:06:38.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.312+0000 7fc5ccc00640 1 -- start start 2024-09-10T10:06:38.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5ccc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc5c80cfb90 0x7fc5c815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5ccc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c80cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5ccc00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c815d4d0 con 0x7fc5c80cff70 2024-09-10T10:06:38.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5ccc00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c806d130 con 0x7fc5c80cfb90 2024-09-10T10:06:38.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5ccc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c815d7b0 con 0x7fc5c80b3f40 2024-09-10T10:06:38.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c5e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc5c80cfb90 0x7fc5c815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c6800640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc5c80cff70 0x7fc5c8160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33700/0 (socket says 172.21.15.110:33700) 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c6800640 1 -- 172.21.15.110:0/2901199854 learned_addr learned my addr 172.21.15.110:0/2901199854 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c5400640 1 --2- 172.21.15.110:0/2901199854 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c80cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c5e00640 1 -- 172.21.15.110:0/2901199854 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 msgr2=0x7fc5c80cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c5e00640 1 --2- 172.21.15.110:0/2901199854 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c80cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c5e00640 1 -- 172.21.15.110:0/2901199854 >> v1:172.21.15.110:6789/0 conn(0x7fc5c80cff70 legacy=0x7fc5c8160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c5e00640 1 -- 172.21.15.110:0/2901199854 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c806d380 con 0x7fc5c80cfb90 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c4a00640 1 -- 172.21.15.110:0/2901199854 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856301280 0 0) 0x7fc5c815d4d0 con 0x7fc5c80cff70 2024-09-10T10:06:38.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.316+0000 7fc5c5400640 1 --2- 172.21.15.110:0/2901199854 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c80cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:38.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5c5e00640 1 --2- 172.21.15.110:0/2901199854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cfb90 0x7fc5c815ca10 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097c0 tx=0x7fc5b801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a952734c22f3a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5c4a00640 1 -- 172.21.15.110:0/2901199854 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80b0070 con 0x7fc5c80cfb90 2024-09-10T10:06:38.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5c4a00640 1 -- 172.21.15.110:0/2901199854 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b80ab600 con 0x7fc5c80cfb90 2024-09-10T10:06:38.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5c4a00640 1 -- 172.21.15.110:0/2901199854 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80ab9d0 con 0x7fc5c80cfb90 2024-09-10T10:06:38.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2901199854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cfb90 msgr2=0x7fc5c815ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2901199854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cfb90 0x7fc5c815ca10 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097c0 tx=0x7fc5b801c410 comp rx=0 tx=0).stop 2024-09-10T10:06:38.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2901199854 shutdown_connections 2024-09-10T10:06:38.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2901199854 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c80cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2901199854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cfb90 0x7fc5c815ca10 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2901199854 >> 172.21.15.110:0/2901199854 conn(0x7fc5c8062d10 msgr2=0x7fc5c806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:38.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2901199854 shutdown_connections 2024-09-10T10:06:38.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2901199854 wait complete. 2024-09-10T10:06:38.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 Processor -- start 2024-09-10T10:06:38.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- start start 2024-09-10T10:06:38.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c81f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5c80cfb90 0x7fc5c8150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cff70 0x7fc5c8150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8169500 con 0x7fc5c80cff70 2024-09-10T10:06:38.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8169380 con 0x7fc5c80cfb90 2024-09-10T10:06:38.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5ccc00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c8169680 con 0x7fc5c80b3f40 2024-09-10T10:06:38.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5c6800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cff70 0x7fc5c8150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5c6800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cff70 0x7fc5c8150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55048/0 (socket says 172.21.15.110:55048) 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.320+0000 7fc5c6800640 1 -- 172.21.15.110:0/2982877944 learned_addr learned my addr 172.21.15.110:0/2982877944 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5c5e00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c81f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5c5400640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5c80cfb90 0x7fc5c8150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5c6800640 1 -- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 msgr2=0x7fc5c81f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5c6800640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c81f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5c6800640 1 -- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5c80cfb90 msgr2=0x7fc5c8150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5c6800640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5c80cfb90 0x7fc5c8150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5c6800640 1 -- 172.21.15.110:0/2982877944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c80abf80 con 0x7fc5c80cff70 2024-09-10T10:06:38.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5c6800640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cff70 0x7fc5c8150ff0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000c700 tx=0x7fc5c000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5be000640 1 -- 172.21.15.110:0/2982877944 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0018070 con 0x7fc5c80cff70 2024-09-10T10:06:38.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5be000640 1 -- 172.21.15.110:0/2982877944 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c000ce10 con 0x7fc5c80cff70 2024-09-10T10:06:38.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c80ac260 con 0x7fc5c80cff70 2024-09-10T10:06:38.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5be000640 1 -- 172.21.15.110:0/2982877944 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c000a860 con 0x7fc5c80cff70 2024-09-10T10:06:38.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c80ac820 con 0x7fc5c80cff70 2024-09-10T10:06:38.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c80cd5a0 con 0x7fc5c80cff70 2024-09-10T10:06:38.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5be000640 1 -- 172.21.15.110:0/2982877944 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fc5c001b070 con 0x7fc5c80cff70 2024-09-10T10:06:38.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5be000640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc5a8068400 0x7fc5a806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.324+0000 7fc5be000640 1 -- 172.21.15.110:0/2982877944 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fc5c0083db0 con 0x7fc5c80cff70 2024-09-10T10:06:38.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.328+0000 7fc5c5e00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc5a8068400 0x7fc5a806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.328+0000 7fc5c5e00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc5a8068400 0x7fc5a806a8c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ad000 tx=0x7fc5b80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.332+0000 7fc5be000640 1 -- 172.21.15.110:0/2982877944 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc5c0055aa0 con 0x7fc5c80cff70 2024-09-10T10:06:38.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.488+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc5c80af510 con 0x7fc5a8068400 2024-09-10T10:06:38.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.488+0000 7fc5be000640 1 -- 172.21.15.110:0/2982877944 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5c80af510 con 0x7fc5a8068400 2024-09-10T10:06:38.491 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T10:06:38.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc5a8068400 msgr2=0x7fc5a806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc5a8068400 0x7fc5a806a8c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ad000 tx=0x7fc5b80a6000 comp rx=0 tx=0).stop 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cff70 msgr2=0x7fc5c8150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cff70 0x7fc5c8150ff0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000c700 tx=0x7fc5c000cbc0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 shutdown_connections 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc5a8068400 0x7fc5a806a8c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5c80cff70 0x7fc5c8150ff0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc5c80cfb90 0x7fc5c8150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 --2- 172.21.15.110:0/2982877944 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc5c80b3f40 0x7fc5c81f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 >> 172.21.15.110:0/2982877944 conn(0x7fc5c8062d10 msgr2=0x7fc5c80b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 shutdown_connections 2024-09-10T10:06:38.494 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.492+0000 7fc5ccc00640 1 -- 172.21.15.110:0/2982877944 wait complete. 2024-09-10T10:06:38.513 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T10:06:38.513 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:06:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4167e00640 1 Processor -- start 2024-09-10T10:06:38.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4167e00640 1 -- start start 2024-09-10T10:06:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4167e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f416815b170 0x7f4168073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4167e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4167e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41680b3b40 con 0x7f416815ed40 2024-09-10T10:06:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4167e00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f41680b3910 con 0x7f416815b170 2024-09-10T10:06:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4167e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f41680cfc60 con 0x7f41680cf2e0 2024-09-10T10:06:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f416815b170 0x7f4168073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f416815b170 0x7f4168073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55054/0 (socket says 172.21.15.110:55054) 2024-09-10T10:06:38.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166a00640 1 -- 172.21.15.110:0/1565378006 learned_addr learned my addr 172.21.15.110:0/1565378006 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:38.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166a00640 1 -- 172.21.15.110:0/1565378006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 msgr2=0x7f41680cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:38.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166000640 1 --2- 172.21.15.110:0/1565378006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166a00640 1 --2- 172.21.15.110:0/1565378006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166a00640 1 -- 172.21.15.110:0/1565378006 >> v1:172.21.15.145:6789/0 conn(0x7f416815ed40 legacy=0x7f416815f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:38.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166a00640 1 -- 172.21.15.110:0/1565378006 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41680b3e20 con 0x7f416815b170 2024-09-10T10:06:38.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.652+0000 7f4166000640 1 --2- 172.21.15.110:0/1565378006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:38.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4166a00640 1 --2- 172.21.15.110:0/1565378006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815b170 0x7f4168073390 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f4158009e50 tx=0x7f415801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a0e5bcce1327e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4165600640 1 -- 172.21.15.110:0/1565378006 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580a5840 con 0x7f416815b170 2024-09-10T10:06:38.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4165600640 1 -- 172.21.15.110:0/1565378006 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41580a59e0 con 0x7f416815b170 2024-09-10T10:06:38.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4165600640 1 -- 172.21.15.110:0/1565378006 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580a5db0 con 0x7f416815b170 2024-09-10T10:06:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 -- 172.21.15.110:0/1565378006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815b170 msgr2=0x7f4168073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 --2- 172.21.15.110:0/1565378006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815b170 0x7f4168073390 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f4158009e50 tx=0x7f415801c9c0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 -- 172.21.15.110:0/1565378006 shutdown_connections 2024-09-10T10:06:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 --2- 172.21.15.110:0/1565378006 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 --2- 172.21.15.110:0/1565378006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815b170 0x7f4168073390 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 -- 172.21.15.110:0/1565378006 >> 172.21.15.110:0/1565378006 conn(0x7f416806d0f0 msgr2=0x7f416806d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 -- 172.21.15.110:0/1565378006 shutdown_connections 2024-09-10T10:06:38.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 -- 172.21.15.110:0/1565378006 wait complete. 2024-09-10T10:06:38.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 Processor -- start 2024-09-10T10:06:38.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.656+0000 7f4167e00640 1 -- start start 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416815b170 0x7f4168153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815ed40 0x7f41680ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4168072680 con 0x7f416815ed40 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f416815b550 con 0x7f416815b170 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f4168072800 con 0x7f41680cf2e0 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815ed40 0x7f41680ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815ed40 0x7f41680ae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55062/0 (socket says 172.21.15.110:55062) 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 -- 172.21.15.110:0/3362501198 learned_addr learned my addr 172.21.15.110:0/3362501198 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:38.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4166000640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416815b170 0x7f4168153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4166a00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 -- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 msgr2=0x7f41680ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 -- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416815b170 msgr2=0x7f4168153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416815b170 0x7f4168153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 -- 172.21.15.110:0/3362501198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41681536b0 con 0x7f416815ed40 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4166000640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416815b170 0x7f4168153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4166a00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f41680cf2e0 0x7f41680ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167400640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815ed40 0x7f41680ae510 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f416000b410 tx=0x7f416000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160015020 con 0x7f416815ed40 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4160011870 con 0x7f416815ed40 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160011c60 con 0x7f416815ed40 2024-09-10T10:06:38.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4168160230 con 0x7f416815ed40 2024-09-10T10:06:38.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4168160ca0 con 0x7f416815ed40 2024-09-10T10:06:38.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41680cf720 con 0x7f416815ed40 2024-09-10T10:06:38.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.660+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f4160018020 con 0x7f416815ed40 2024-09-10T10:06:38.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f415ea00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4148068430 0x7f414806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f4160084200 con 0x7f416815ed40 2024-09-10T10:06:38.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f415ea00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f414806e2f0 0x7f4148070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 --> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4148070e00 con 0x7f414806e2f0 2024-09-10T10:06:38.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=168) ==== 24+0+0 (secure 0 0 0) 0x7f416008b2a0 con 0x7f416815ed40 2024-09-10T10:06:38.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f4166a00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4148068430 0x7f414806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f4166000640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f414806e2f0 0x7f4148070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f4166000640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f414806e2f0 0x7f4148070730 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f4166a00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4148068430 0x7f414806a8f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f41580ae000 tx=0x7f41580a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.664+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 <== osd.1 v2:172.21.15.110:6808/3662700555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4148070e00 con 0x7f414806e2f0 2024-09-10T10:06:38.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 --> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f41680afaf0 con 0x7f414806e2f0 2024-09-10T10:06:38.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f415ea00640 1 -- 172.21.15.110:0/3362501198 <== osd.1 v2:172.21.15.110:6808/3662700555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f41680afaf0 con 0x7f414806e2f0 2024-09-10T10:06:38.679 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:38.680 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:06:38.680 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:38.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f414806e2f0 msgr2=0x7f4148070730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.110:6808/3662700555,v1:172.21.15.110:6809/3662700555] conn(0x7f414806e2f0 0x7f4148070730 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4148068430 msgr2=0x7f414806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f4148068430 0x7f414806a8f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f41580ae000 tx=0x7f41580a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:38.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815ed40 msgr2=0x7f41680ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 --2- 172.21.15.110:0/3362501198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416815ed40 0x7f41680ae510 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f416000b410 tx=0x7f416000b8d0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167400640 1 -- 172.21.15.110:0/3362501198 reap_dead start 2024-09-10T10:06:38.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 shutdown_connections 2024-09-10T10:06:38.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.676+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 >> 172.21.15.110:0/3362501198 conn(0x7f416806d0f0 msgr2=0x7f41680ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:38.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.680+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 shutdown_connections 2024-09-10T10:06:38.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.680+0000 7f4167e00640 1 -- 172.21.15.110:0/3362501198 wait complete. 2024-09-10T10:06:38.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T10:06:38.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0bb400640 1 Processor -- start 2024-09-10T10:06:38.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0bb400640 1 -- start start 2024-09-10T10:06:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0bb400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa0b40b3f40 0x7fa0b40cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0bb400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b415d4d0 con 0x7fa0b40cff70 2024-09-10T10:06:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0bb400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b415d7b0 con 0x7fa0b40cfb90 2024-09-10T10:06:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0bb400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b406d130 con 0x7fa0b40b3f40 2024-09-10T10:06:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0b3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa0b40b3f40 0x7fa0b40cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0b9400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa0b40cff70 0x7fa0b406db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33708/0 (socket says 172.21.15.110:33708) 2024-09-10T10:06:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0b3e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa0b40b3f40 0x7fa0b40cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55072/0 (socket says 172.21.15.110:55072) 2024-09-10T10:06:38.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0b9400640 1 -- 172.21.15.110:0/2961246281 learned_addr learned my addr 172.21.15.110:0/2961246281 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:38.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0b3e00640 1 -- 172.21.15.110:0/2961246281 >> v1:172.21.15.145:6789/0 conn(0x7fa0b40cfb90 legacy=0x7fa0b415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:38.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0b3e00640 1 -- 172.21.15.110:0/2961246281 >> v1:172.21.15.110:6789/0 conn(0x7fa0b40cff70 legacy=0x7fa0b406db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.832+0000 7fa0b3e00640 1 -- 172.21.15.110:0/2961246281 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b406d380 con 0x7fa0b40b3f40 2024-09-10T10:06:38.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0b3e00640 1 --2- 172.21.15.110:0/2961246281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40b3f40 0x7fa0b40cd5a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00097c0 tx=0x7fa0a001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89108eed97ef314a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0b3400640 1 -- 172.21.15.110:0/2961246281 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00b0070 con 0x7fa0b40b3f40 2024-09-10T10:06:38.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0b3400640 1 -- 172.21.15.110:0/2961246281 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a00ab600 con 0x7fa0b40b3f40 2024-09-10T10:06:38.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0b3400640 1 -- 172.21.15.110:0/2961246281 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00a4e60 con 0x7fa0b40b3f40 2024-09-10T10:06:38.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0bb400640 1 -- 172.21.15.110:0/2961246281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40b3f40 msgr2=0x7fa0b40cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0bb400640 1 --2- 172.21.15.110:0/2961246281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40b3f40 0x7fa0b40cd5a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00097c0 tx=0x7fa0a001c3d0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0bb400640 1 -- 172.21.15.110:0/2961246281 shutdown_connections 2024-09-10T10:06:38.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0bb400640 1 --2- 172.21.15.110:0/2961246281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40b3f40 0x7fa0b40cd5a0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0bb400640 1 -- 172.21.15.110:0/2961246281 >> 172.21.15.110:0/2961246281 conn(0x7fa0b4062d10 msgr2=0x7fa0b406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:38.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0bb400640 1 -- 172.21.15.110:0/2961246281 shutdown_connections 2024-09-10T10:06:38.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.836+0000 7fa0bb400640 1 -- 172.21.15.110:0/2961246281 wait complete. 2024-09-10T10:06:38.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 Processor -- start 2024-09-10T10:06:38.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 -- start start 2024-09-10T10:06:38.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0b40b3f40 0x7fa0b41f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0b40cfb90 0x7fa0b4158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40cff70 0x7fa0b415f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b416a510 con 0x7fa0b40cff70 2024-09-10T10:06:38.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b416a390 con 0x7fa0b40b3f40 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b416a690 con 0x7fa0b40cfb90 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40cff70 0x7fa0b415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40cff70 0x7fa0b415f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55076/0 (socket says 172.21.15.110:55076) 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 -- 172.21.15.110:0/3642260066 learned_addr learned my addr 172.21.15.110:0/3642260066 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b3e00640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0b40cfb90 0x7fa0b4158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b8a00640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0b40b3f40 0x7fa0b41f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 -- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0b40cfb90 msgr2=0x7fa0b4158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0b40cfb90 0x7fa0b4158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 -- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0b40b3f40 msgr2=0x7fa0b41f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0b40b3f40 0x7fa0b41f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 -- 172.21.15.110:0/3642260066 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b415a390 con 0x7fa0b40cff70 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b9400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40cff70 0x7fa0b415f420 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00ca80 tx=0x7fa0ac00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b0c00640 1 -- 172.21.15.110:0/3642260066 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac017070 con 0x7fa0b40cff70 2024-09-10T10:06:38.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b0c00640 1 -- 172.21.15.110:0/3642260066 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac00ac30 con 0x7fa0b40cff70 2024-09-10T10:06:38.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b4155800 con 0x7fa0b40cff70 2024-09-10T10:06:38.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0b0c00640 1 -- 172.21.15.110:0/3642260066 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac012400 con 0x7fa0b40cff70 2024-09-10T10:06:38.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.840+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b415a910 con 0x7fa0b40cff70 2024-09-10T10:06:38.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.844+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b40cd5a0 con 0x7fa0b40cff70 2024-09-10T10:06:38.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.844+0000 7fa0b0c00640 1 -- 172.21.15.110:0/3642260066 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fa0ac01a070 con 0x7fa0b40cff70 2024-09-10T10:06:38.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.844+0000 7fa0b0c00640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa094068400 0x7fa09406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:38.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.844+0000 7fa0b0c00640 1 -- 172.21.15.110:0/3642260066 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fa0ac084440 con 0x7fa0b40cff70 2024-09-10T10:06:38.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.844+0000 7fa0b8a00640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa094068400 0x7fa09406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:38.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.844+0000 7fa0b8a00640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa094068400 0x7fa09406a8c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4004e30 tx=0x7fa0a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:38.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:38.848+0000 7fa0b0c00640 1 -- 172.21.15.110:0/3642260066 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa0ac056130 con 0x7fa0b40cff70 2024-09-10T10:06:39.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.004+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fa0b4075560 con 0x7fa094068400 2024-09-10T10:06:39.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.004+0000 7fa0b0c00640 1 -- 172.21.15.110:0/3642260066 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa0b4075560 con 0x7fa094068400 2024-09-10T10:06:39.007 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T10:06:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa094068400 msgr2=0x7fa09406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa094068400 0x7fa09406a8c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4004e30 tx=0x7fa0a4009040 comp rx=0 tx=0).stop 2024-09-10T10:06:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40cff70 msgr2=0x7fa0b415f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40cff70 0x7fa0b415f420 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00ca80 tx=0x7fa0ac00cf40 comp rx=0 tx=0).stop 2024-09-10T10:06:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 shutdown_connections 2024-09-10T10:06:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa094068400 0x7fa09406a8c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0b40cff70 0x7fa0b415f420 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa0b40cfb90 0x7fa0b4158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 --2- 172.21.15.110:0/3642260066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0b40b3f40 0x7fa0b41f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 >> 172.21.15.110:0/3642260066 conn(0x7fa0b4062d10 msgr2=0x7fa0b40ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 shutdown_connections 2024-09-10T10:06:39.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.008+0000 7fa0bb400640 1 -- 172.21.15.110:0/3642260066 wait complete. 2024-09-10T10:06:39.028 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T10:06:39.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:06:39.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.164+0000 7f98f1000640 1 Processor -- start 2024-09-10T10:06:39.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.164+0000 7f98f1000640 1 -- start start 2024-09-10T10:06:39.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98f1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98ec155970 0x7f98ec0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98f1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f98ec06f1e0 0x7f98ec0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98f1000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec155d50 con 0x7f98ec15c560 2024-09-10T10:06:39.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98f1000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f98ec0b4570 con 0x7f98ec06f1e0 2024-09-10T10:06:39.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98f1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec0b34d0 con 0x7f98ec155970 2024-09-10T10:06:39.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98ea800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f98ec06f1e0 0x7f98ec0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98ea800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f98ec06f1e0 0x7f98ec0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55080/0 (socket says 172.21.15.110:55080) 2024-09-10T10:06:39.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98e9400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f98ec15c560 0x7f98ec06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33724/0 (socket says 172.21.15.110:33724) 2024-09-10T10:06:39.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98ea800640 1 -- 172.21.15.110:0/3409364020 learned_addr learned my addr 172.21.15.110:0/3409364020 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:39.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98e9e00640 1 --2- 172.21.15.110:0/3409364020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98ec155970 0x7f98ec0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98ea800640 1 -- 172.21.15.110:0/3409364020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98ec155970 msgr2=0x7f98ec0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:39.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98ea800640 1 --2- 172.21.15.110:0/3409364020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98ec155970 0x7f98ec0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98ea800640 1 -- 172.21.15.110:0/3409364020 >> v1:172.21.15.110:6789/0 conn(0x7f98ec15c560 legacy=0x7f98ec06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:39.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.168+0000 7f98ea800640 1 -- 172.21.15.110:0/3409364020 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec0b37b0 con 0x7f98ec06f1e0 2024-09-10T10:06:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98ea800640 1 --2- 172.21.15.110:0/3409364020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 0x7f98ec0ce420 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009870 tx=0x7f98e401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c97bb7c48cdfb01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98e8a00640 1 -- 172.21.15.110:0/3409364020 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40ab620 con 0x7f98ec06f1e0 2024-09-10T10:06:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98e8a00640 1 -- 172.21.15.110:0/3409364020 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e40b0070 con 0x7f98ec06f1e0 2024-09-10T10:06:39.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98e8a00640 1 -- 172.21.15.110:0/3409364020 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40ab9e0 con 0x7f98ec06f1e0 2024-09-10T10:06:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 -- 172.21.15.110:0/3409364020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 msgr2=0x7f98ec0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 --2- 172.21.15.110:0/3409364020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 0x7f98ec0ce420 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009870 tx=0x7f98e401c460 comp rx=0 tx=0).stop 2024-09-10T10:06:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 -- 172.21.15.110:0/3409364020 shutdown_connections 2024-09-10T10:06:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 --2- 172.21.15.110:0/3409364020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 0x7f98ec0ce420 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 --2- 172.21.15.110:0/3409364020 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98ec155970 0x7f98ec0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 -- 172.21.15.110:0/3409364020 >> 172.21.15.110:0/3409364020 conn(0x7f98ec06bc50 msgr2=0x7f98ec06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 -- 172.21.15.110:0/3409364020 shutdown_connections 2024-09-10T10:06:39.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 -- 172.21.15.110:0/3409364020 wait complete. 2024-09-10T10:06:39.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 Processor -- start 2024-09-10T10:06:39.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.172+0000 7f98f1000640 1 -- start start 2024-09-10T10:06:39.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 0x7f98ec15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98ec155970 0x7f98ec15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98ec15c560 0x7f98ec153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec0cf170 con 0x7f98ec06f1e0 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec0cce00 con 0x7f98ec15c560 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec0cf2f0 con 0x7f98ec155970 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 0x7f98ec15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 0x7f98ec15e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55096/0 (socket says 172.21.15.110:55096) 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 -- 172.21.15.110:0/1737542620 learned_addr learned my addr 172.21.15.110:0/1737542620 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 -- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98ec155970 msgr2=0x7f98ec15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f98ec155970 0x7f98ec15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 -- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98ec15c560 msgr2=0x7f98ec153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98ec15c560 0x7f98ec153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 -- 172.21.15.110:0/1737542620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec1544a0 con 0x7f98ec06f1e0 2024-09-10T10:06:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e9e00640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 0x7f98ec15e1f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f98dc007ec0 tx=0x7f98dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc014620 con 0x7f98ec06f1e0 2024-09-10T10:06:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc019070 con 0x7f98ec06f1e0 2024-09-10T10:06:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec15d150 con 0x7f98ec06f1e0 2024-09-10T10:06:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc014a20 con 0x7f98ec06f1e0 2024-09-10T10:06:39.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec15d710 con 0x7f98ec06f1e0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98ec0ce420 con 0x7f98ec06f1e0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.176+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f98dc014bc0 con 0x7f98ec06f1e0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98e2000640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f98cc068400 0x7f98cc06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f98dc084fd0 con 0x7f98ec06f1e0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98e2000640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f98cc06e2c0 0x7f98cc070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 --> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98cc070dd0 con 0x7f98cc06e2c0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=168) ==== 24+0+0 (secure 0 0 0) 0x7f98dc08e050 con 0x7f98ec06f1e0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98e9400640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f98cc068400 0x7f98cc06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98ea800640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f98cc06e2c0 0x7f98cc070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98ea800640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f98cc06e2c0 0x7f98cc070700 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98e9400640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f98cc068400 0x7f98cc06a8c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f98ec157180 tx=0x7f98d8006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.180+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 <== osd.2 v2:172.21.15.110:6800/1201840781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98cc070dd0 con 0x7f98cc06e2c0 2024-09-10T10:06:39.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 --> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f98ec0b0550 con 0x7f98cc06e2c0 2024-09-10T10:06:39.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98e2000640 1 -- 172.21.15.110:0/1737542620 <== osd.2 v2:172.21.15.110:6800/1201840781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f98ec0b0550 con 0x7f98cc06e2c0 2024-09-10T10:06:39.195 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:39.195 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:06:39.196 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:39.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f98cc06e2c0 msgr2=0x7f98cc070700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.110:6800/1201840781,v1:172.21.15.110:6801/1201840781] conn(0x7f98cc06e2c0 0x7f98cc070700 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f98cc068400 msgr2=0x7f98cc06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f98cc068400 0x7f98cc06a8c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f98ec157180 tx=0x7f98d8006bd0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 msgr2=0x7f98ec15e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 --2- 172.21.15.110:0/1737542620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98ec06f1e0 0x7f98ec15e1f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f98dc007ec0 tx=0x7f98dc00e4b0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98ea800640 1 -- 172.21.15.110:0/1737542620 reap_dead start 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 shutdown_connections 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 >> 172.21.15.110:0/1737542620 conn(0x7f98ec06bc50 msgr2=0x7f98ec0ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 shutdown_connections 2024-09-10T10:06:39.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.192+0000 7f98f1000640 1 -- 172.21.15.110:0/1737542620 wait complete. 2024-09-10T10:06:39.214 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T10:06:39.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.348+0000 7fc211000640 1 Processor -- start 2024-09-10T10:06:39.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.348+0000 7fc211000640 1 -- start start 2024-09-10T10:06:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc211000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc20c0cfe50 0x7fc20c0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc211000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc20c0ae840 0x7fc20c0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc211000640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc20c0d0230 con 0x7fc20c15f9f0 2024-09-10T10:06:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc211000640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fc20c0b4710 con 0x7fc20c0ae840 2024-09-10T10:06:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc211000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc20c072d10 con 0x7fc20c0cfe50 2024-09-10T10:06:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc20a800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc20c0ae840 0x7fc20c0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc20a800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc20c0ae840 0x7fc20c0aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55100/0 (socket says 172.21.15.110:55100) 2024-09-10T10:06:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc20a800640 1 -- 172.21.15.110:0/3941472523 learned_addr learned my addr 172.21.15.110:0/3941472523 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:39.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc201400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc20c15f9f0 0x7fc20c072600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33732/0 (socket says 172.21.15.110:33732) 2024-09-10T10:06:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc209e00640 1 --2- 172.21.15.110:0/3941472523 >> v2:172.21.15.145:3300/0 conn(0x7fc20c0cfe50 0x7fc20c0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc20a800640 1 -- 172.21.15.110:0/3941472523 >> v2:172.21.15.145:3300/0 conn(0x7fc20c0cfe50 msgr2=0x7fc20c0b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc20a800640 1 --2- 172.21.15.110:0/3941472523 >> v2:172.21.15.145:3300/0 conn(0x7fc20c0cfe50 0x7fc20c0b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc20a800640 1 -- 172.21.15.110:0/3941472523 >> v1:172.21.15.110:6789/0 conn(0x7fc20c15f9f0 legacy=0x7fc20c072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc20a800640 1 -- 172.21.15.110:0/3941472523 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c072f50 con 0x7fc20c0ae840 2024-09-10T10:06:39.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.352+0000 7fc209e00640 1 --2- 172.21.15.110:0/3941472523 >> v2:172.21.15.145:3300/0 conn(0x7fc20c0cfe50 0x7fc20c0b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:39.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc20a800640 1 --2- 172.21.15.110:0/3941472523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0ae840 0x7fc20c0aec70 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fc20c06cb00 tx=0x7fc20401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc463b7f7cc2d361 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc209400640 1 -- 172.21.15.110:0/3941472523 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2040a5620 con 0x7fc20c0ae840 2024-09-10T10:06:39.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc209400640 1 -- 172.21.15.110:0/3941472523 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2040a57c0 con 0x7fc20c0ae840 2024-09-10T10:06:39.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc209400640 1 -- 172.21.15.110:0/3941472523 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2040a5b90 con 0x7fc20c0ae840 2024-09-10T10:06:39.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- 172.21.15.110:0/3941472523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0ae840 msgr2=0x7fc20c0aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 --2- 172.21.15.110:0/3941472523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0ae840 0x7fc20c0aec70 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fc20c06cb00 tx=0x7fc20401c960 comp rx=0 tx=0).stop 2024-09-10T10:06:39.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- 172.21.15.110:0/3941472523 shutdown_connections 2024-09-10T10:06:39.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 --2- 172.21.15.110:0/3941472523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0ae840 0x7fc20c0aec70 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 --2- 172.21.15.110:0/3941472523 >> v2:172.21.15.145:3300/0 conn(0x7fc20c0cfe50 0x7fc20c0b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- 172.21.15.110:0/3941472523 >> 172.21.15.110:0/3941472523 conn(0x7fc20c062d10 msgr2=0x7fc20c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:39.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- 172.21.15.110:0/3941472523 shutdown_connections 2024-09-10T10:06:39.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- 172.21.15.110:0/3941472523 wait complete. 2024-09-10T10:06:39.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 Processor -- start 2024-09-10T10:06:39.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- start start 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc20c0ae840 0x7fc20c148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0cfe50 0x7fc20c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc20c15f9f0 0x7fc20c157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c0718c0 con 0x7fc20c0cfe50 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c0b5960 con 0x7fc20c0ae840 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc211000640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fc20c071a40 con 0x7fc20c15f9f0 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc201400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0cfe50 0x7fc20c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc201400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0cfe50 0x7fc20c1577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55104/0 (socket says 172.21.15.110:55104) 2024-09-10T10:06:39.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc201400640 1 -- 172.21.15.110:0/4059617727 learned_addr learned my addr 172.21.15.110:0/4059617727 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.356+0000 7fc209e00640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc20c0ae840 0x7fc20c148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc20a800640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc20c15f9f0 0x7fc20c157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc201400640 1 -- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc20c15f9f0 msgr2=0x7fc20c157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc201400640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc20c15f9f0 0x7fc20c157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc201400640 1 -- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc20c0ae840 msgr2=0x7fc20c148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc201400640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc20c0ae840 0x7fc20c148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc201400640 1 -- 172.21.15.110:0/4059617727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c14f2c0 con 0x7fc20c0cfe50 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc201400640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0cfe50 0x7fc20c1577a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c420 tx=0x7fc1f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc202a00640 1 -- 172.21.15.110:0/4059617727 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40074e0 con 0x7fc20c0cfe50 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc202a00640 1 -- 172.21.15.110:0/4059617727 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f4007680 con 0x7fc20c0cfe50 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc202a00640 1 -- 172.21.15.110:0/4059617727 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004a30 con 0x7fc20c0cfe50 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c14f5a0 con 0x7fc20c0cfe50 2024-09-10T10:06:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c148ff0 con 0x7fc20c0cfe50 2024-09-10T10:06:39.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20c0aec70 con 0x7fc20c0cfe50 2024-09-10T10:06:39.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.360+0000 7fc202a00640 1 -- 172.21.15.110:0/4059617727 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fc1f401a020 con 0x7fc20c0cfe50 2024-09-10T10:06:39.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.364+0000 7fc202a00640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc1e8068400 0x7fc1e806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.364+0000 7fc202a00640 1 -- 172.21.15.110:0/4059617727 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fc1f4083d20 con 0x7fc20c0cfe50 2024-09-10T10:06:39.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.364+0000 7fc209e00640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc1e8068400 0x7fc1e806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.364+0000 7fc209e00640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc1e8068400 0x7fc1e806a8c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009950 tx=0x7fc1fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.368+0000 7fc202a00640 1 -- 172.21.15.110:0/4059617727 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc1f4055c60 con 0x7fc20c0cfe50 2024-09-10T10:06:39.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc20c136c00 con 0x7fc1e8068400 2024-09-10T10:06:39.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc202a00640 1 -- 172.21.15.110:0/4059617727 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc20c136c00 con 0x7fc1e8068400 2024-09-10T10:06:39.529 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T10:06:39.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc1e8068400 msgr2=0x7fc1e806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc1e8068400 0x7fc1e806a8c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009950 tx=0x7fc1fc008040 comp rx=0 tx=0).stop 2024-09-10T10:06:39.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0cfe50 msgr2=0x7fc20c1577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0cfe50 0x7fc20c1577a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c420 tx=0x7fc1f400c8e0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 shutdown_connections 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fc1e8068400 0x7fc1e806a8c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fc20c15f9f0 0x7fc20c157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc20c0cfe50 0x7fc20c1577a0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 --2- 172.21.15.110:0/4059617727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc20c0ae840 0x7fc20c148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 >> 172.21.15.110:0/4059617727 conn(0x7fc20c062d10 msgr2=0x7fc20c0b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 shutdown_connections 2024-09-10T10:06:39.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.528+0000 7fc211000640 1 -- 172.21.15.110:0/4059617727 wait complete. 2024-09-10T10:06:39.550 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T10:06:39.551 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:06:39.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9de4a00640 1 Processor -- start 2024-09-10T10:06:39.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9de4a00640 1 -- start start 2024-09-10T10:06:39.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9de4a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9de0155970 0x7f9de00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9de4a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de0155d50 con 0x7f9de015c560 2024-09-10T10:06:39.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9de4a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de00b34d0 con 0x7f9de006f1e0 2024-09-10T10:06:39.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9de4a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f9de00b4570 con 0x7f9de0155970 2024-09-10T10:06:39.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9ddde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9de0155970 0x7f9de00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9ddde00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9de0155970 0x7f9de00cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55112/0 (socket says 172.21.15.110:55112) 2024-09-10T10:06:39.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9ddde00640 1 -- 172.21.15.110:0/3954829159 learned_addr learned my addr 172.21.15.110:0/3954829159 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:39.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9ddde00640 1 -- 172.21.15.110:0/3954829159 >> v1:172.21.15.145:6789/0 conn(0x7f9de006f1e0 legacy=0x7f9de00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:39.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9ddde00640 1 -- 172.21.15.110:0/3954829159 >> v1:172.21.15.110:6789/0 conn(0x7f9de015c560 legacy=0x7f9de006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9ddde00640 1 -- 172.21.15.110:0/3954829159 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de00b37b0 con 0x7f9de0155970 2024-09-10T10:06:39.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.688+0000 7f9ddca00640 1 -- 172.21.15.110:0/3954829159 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2265656862 0 0) 0x7f9de0155d50 con 0x7f9de015c560 2024-09-10T10:06:39.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9ddde00640 1 --2- 172.21.15.110:0/3954829159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 0x7f9de00cc8c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009e50 tx=0x7f9dd001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a1b12b9e6a36c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9ddca00640 1 -- 172.21.15.110:0/3954829159 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00a5880 con 0x7f9de0155970 2024-09-10T10:06:39.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9ddca00640 1 -- 172.21.15.110:0/3954829159 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd00a5a20 con 0x7f9de0155970 2024-09-10T10:06:39.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9ddca00640 1 -- 172.21.15.110:0/3954829159 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00a5df0 con 0x7f9de0155970 2024-09-10T10:06:39.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9de4a00640 1 -- 172.21.15.110:0/3954829159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 msgr2=0x7f9de00cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9de4a00640 1 --2- 172.21.15.110:0/3954829159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 0x7f9de00cc8c0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009e50 tx=0x7f9dd001c9c0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9de4a00640 1 -- 172.21.15.110:0/3954829159 shutdown_connections 2024-09-10T10:06:39.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9de4a00640 1 --2- 172.21.15.110:0/3954829159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 0x7f9de00cc8c0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9de4a00640 1 -- 172.21.15.110:0/3954829159 >> 172.21.15.110:0/3954829159 conn(0x7f9de006bc50 msgr2=0x7f9de006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9de4a00640 1 -- 172.21.15.110:0/3954829159 shutdown_connections 2024-09-10T10:06:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.692+0000 7f9de4a00640 1 -- 172.21.15.110:0/3954829159 wait complete. 2024-09-10T10:06:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 Processor -- start 2024-09-10T10:06:39.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 -- start start 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de006f1e0 0x7f9de015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 0x7f9de015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9de015c560 0x7f9de0153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9de00cf340 con 0x7f9de0155970 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9de00cce00 con 0x7f9de006f1e0 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9de00cf4c0 con 0x7f9de015c560 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 0x7f9de015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 0x7f9de015f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55116/0 (socket says 172.21.15.110:55116) 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 -- 172.21.15.110:0/1242163518 learned_addr learned my addr 172.21.15.110:0/1242163518 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddde00640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de006f1e0 0x7f9de015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9dde800640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9de015c560 0x7f9de0153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 -- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9de015c560 msgr2=0x7f9de0153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9de015c560 0x7f9de0153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 -- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de006f1e0 msgr2=0x7f9de015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de006f1e0 0x7f9de015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 -- 172.21.15.110:0/1242163518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de0151c00 con 0x7f9de0155970 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddde00640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9de006f1e0 0x7f9de015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9ddd400640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 0x7f9de015f2d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc001ec0 tx=0x7f9dcc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc017070 con 0x7f9de0155970 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc003600 con 0x7f9de0155970 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0039f0 con 0x7f9de0155970 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de015d590 con 0x7f9de0155970 2024-09-10T10:06:39.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de0152140 con 0x7f9de0155970 2024-09-10T10:06:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.696+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9de0151ee0 con 0x7f9de0155970 2024-09-10T10:06:39.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f9dcc01a070 con 0x7f9de0155970 2024-09-10T10:06:39.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dd6000640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dc0068400 0x7f9dc006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f9dcc083c90 con 0x7f9de0155970 2024-09-10T10:06:39.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dd6000640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7f9dc006e2c0 0x7f9dc0070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 --> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9dcc014ec0 con 0x7f9dc006e2c0 2024-09-10T10:06:39.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=168) ==== 24+0+0 (secure 0 0 0) 0x7f9dcc08d050 con 0x7f9de0155970 2024-09-10T10:06:39.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9ddde00640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dc0068400 0x7f9dc006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dde800640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7f9dc006e2c0 0x7f9dc0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9ddde00640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dc0068400 0x7f9dc006a8c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00ae000 tx=0x7f9dd00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dde800640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7f9dc006e2c0 0x7f9dc0070700 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.700+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 <== osd.3 v2:172.21.15.110:6824/4112166746 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9dcc014ec0 con 0x7f9dc006e2c0 2024-09-10T10:06:39.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 --> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9de00cc8c0 con 0x7f9dc006e2c0 2024-09-10T10:06:39.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9dd6000640 1 -- 172.21.15.110:0/1242163518 <== osd.3 v2:172.21.15.110:6824/4112166746 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9de00cc8c0 con 0x7f9dc006e2c0 2024-09-10T10:06:39.717 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:39.717 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:06:39.717 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:39.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7f9dc006e2c0 msgr2=0x7f9dc0070700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.110:6824/4112166746,v1:172.21.15.110:6825/4112166746] conn(0x7f9dc006e2c0 0x7f9dc0070700 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dc0068400 msgr2=0x7f9dc006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dc0068400 0x7f9dc006a8c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00ae000 tx=0x7f9dd00a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 msgr2=0x7f9de015f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 --2- 172.21.15.110:0/1242163518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9de0155970 0x7f9de015f2d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc001ec0 tx=0x7f9dcc00c8e0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9dde800640 1 -- 172.21.15.110:0/1242163518 reap_dead start 2024-09-10T10:06:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 shutdown_connections 2024-09-10T10:06:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 >> 172.21.15.110:0/1242163518 conn(0x7f9de006bc50 msgr2=0x7f9de006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 shutdown_connections 2024-09-10T10:06:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.716+0000 7f9de4a00640 1 -- 172.21.15.110:0/1242163518 wait complete. 2024-09-10T10:06:39.735 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T10:06:39.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.868+0000 7fe756c00640 1 Processor -- start 2024-09-10T10:06:39.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.868+0000 7fe756c00640 1 -- start start 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe756c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe7500626e0 0x7fe7500af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe756c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7500afce0 0x7fe75006f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe756c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7500ae9e0 con 0x7fe75015e540 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe756c00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fe7500aecc0 con 0x7fe7500afce0 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe756c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe7500aefa0 con 0x7fe7500626e0 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe754c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7500afce0 0x7fe75006f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe754c00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe7500afce0 0x7fe75006f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55122/0 (socket says 172.21.15.110:55122) 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe754c00640 1 -- 172.21.15.110:0/1699355156 learned_addr learned my addr 172.21.15.110:0/1699355156 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:39.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe74f400640 1 --2- 172.21.15.110:0/1699355156 >> v2:172.21.15.145:3300/0 conn(0x7fe7500626e0 0x7fe7500af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe754c00640 1 -- 172.21.15.110:0/1699355156 >> v2:172.21.15.145:3300/0 conn(0x7fe7500626e0 msgr2=0x7fe7500af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe754c00640 1 --2- 172.21.15.110:0/1699355156 >> v2:172.21.15.145:3300/0 conn(0x7fe7500626e0 0x7fe7500af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe754c00640 1 -- 172.21.15.110:0/1699355156 >> v1:172.21.15.145:6789/0 conn(0x7fe75015e540 legacy=0x7fe7500b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe754c00640 1 -- 172.21.15.110:0/1699355156 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7500b0120 con 0x7fe7500afce0 2024-09-10T10:06:39.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.872+0000 7fe74f400640 1 --2- 172.21.15.110:0/1699355156 >> v2:172.21.15.145:3300/0 conn(0x7fe7500626e0 0x7fe7500af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:39.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe754c00640 1 --2- 172.21.15.110:0/1699355156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7500afce0 0x7fe75006f450 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe748009e50 tx=0x7fe74801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2c868b77ba0f77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe74ea00640 1 -- 172.21.15.110:0/1699355156 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480a9470 con 0x7fe7500afce0 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe74ea00640 1 -- 172.21.15.110:0/1699355156 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7480ae070 con 0x7fe7500afce0 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe74ea00640 1 -- 172.21.15.110:0/1699355156 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480a9830 con 0x7fe7500afce0 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 -- 172.21.15.110:0/1699355156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7500afce0 msgr2=0x7fe75006f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 --2- 172.21.15.110:0/1699355156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7500afce0 0x7fe75006f450 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe748009e50 tx=0x7fe74801c6a0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 -- 172.21.15.110:0/1699355156 shutdown_connections 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 --2- 172.21.15.110:0/1699355156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7500afce0 0x7fe75006f450 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 --2- 172.21.15.110:0/1699355156 >> v2:172.21.15.145:3300/0 conn(0x7fe7500626e0 0x7fe7500af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 -- 172.21.15.110:0/1699355156 >> 172.21.15.110:0/1699355156 conn(0x7fe7500b0970 msgr2=0x7fe7500b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 -- 172.21.15.110:0/1699355156 shutdown_connections 2024-09-10T10:06:39.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 -- 172.21.15.110:0/1699355156 wait complete. 2024-09-10T10:06:39.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 Processor -- start 2024-09-10T10:06:39.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.876+0000 7fe756c00640 1 -- start start 2024-09-10T10:06:39.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7500626e0 0x7fe750138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7500afce0 0x7fe750161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75015e540 0x7fe75015f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe75016b500 con 0x7fe75015e540 2024-09-10T10:06:39.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe75016b380 con 0x7fe7500626e0 2024-09-10T10:06:39.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe75016b680 con 0x7fe7500afce0 2024-09-10T10:06:39.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75015e540 0x7fe75015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75015e540 0x7fe75015f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55128/0 (socket says 172.21.15.110:55128) 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 -- 172.21.15.110:0/3569032598 learned_addr learned my addr 172.21.15.110:0/3569032598 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe74fe00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7500626e0 0x7fe750138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe74f400640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7500afce0 0x7fe750161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 -- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7500afce0 msgr2=0x7fe750161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7500afce0 0x7fe750161550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 -- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7500626e0 msgr2=0x7fe750138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7500626e0 0x7fe750138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 -- 172.21.15.110:0/3569032598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe750138750 con 0x7fe75015e540 2024-09-10T10:06:39.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe74f400640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7500afce0 0x7fe750161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:39.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe754c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75015e540 0x7fe75015f080 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ab040 tx=0x7fe748019440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe747e00640 1 -- 172.21.15.110:0/3569032598 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480b8070 con 0x7fe75015e540 2024-09-10T10:06:39.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe747e00640 1 -- 172.21.15.110:0/3569032598 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7480a98a0 con 0x7fe75015e540 2024-09-10T10:06:39.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7501389f0 con 0x7fe75015e540 2024-09-10T10:06:39.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe747e00640 1 -- 172.21.15.110:0/3569032598 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480a9c90 con 0x7fe75015e540 2024-09-10T10:06:39.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe750138fb0 con 0x7fe75015e540 2024-09-10T10:06:39.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7500af7a0 con 0x7fe75015e540 2024-09-10T10:06:39.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.880+0000 7fe747e00640 1 -- 172.21.15.110:0/3569032598 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fe7480c3060 con 0x7fe75015e540 2024-09-10T10:06:39.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.884+0000 7fe747e00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe730068400 0x7fe73006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:39.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.884+0000 7fe747e00640 1 -- 172.21.15.110:0/3569032598 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fe74811e610 con 0x7fe75015e540 2024-09-10T10:06:39.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.884+0000 7fe74fe00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe730068400 0x7fe73006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:39.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.884+0000 7fe74fe00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe730068400 0x7fe73006a8c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe740001770 tx=0x7fe740009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:39.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:39.888+0000 7fe747e00640 1 -- 172.21.15.110:0/3569032598 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe7480165e0 con 0x7fe75015e540 2024-09-10T10:06:40.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe75015dd00 con 0x7fe730068400 2024-09-10T10:06:40.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe747e00640 1 -- 172.21.15.110:0/3569032598 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe75015dd00 con 0x7fe730068400 2024-09-10T10:06:40.046 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T10:06:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe730068400 msgr2=0x7fe73006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe730068400 0x7fe73006a8c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe740001770 tx=0x7fe740009040 comp rx=0 tx=0).stop 2024-09-10T10:06:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75015e540 msgr2=0x7fe75015f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75015e540 0x7fe75015f080 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ab040 tx=0x7fe748019440 comp rx=0 tx=0).stop 2024-09-10T10:06:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 shutdown_connections 2024-09-10T10:06:40.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe730068400 0x7fe73006a8c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe75015e540 0x7fe75015f080 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe7500afce0 0x7fe750161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 --2- 172.21.15.110:0/3569032598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7500626e0 0x7fe750138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 >> 172.21.15.110:0/3569032598 conn(0x7fe7500b0970 msgr2=0x7fe7500b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 shutdown_connections 2024-09-10T10:06:40.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.044+0000 7fe756c00640 1 -- 172.21.15.110:0/3569032598 wait complete. 2024-09-10T10:06:40.067 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T10:06:40.067 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:06:40.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0ec00640 1 Processor -- start 2024-09-10T10:06:40.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0ec00640 1 -- start start 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0ec00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae08072840 0x7fae08140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0ec00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fae08140a60 0x7fae0815b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0ec00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae080ccf40 con 0x7fae080ccb60 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0ec00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fae081412a0 con 0x7fae08140a60 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0ec00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fae0815e760 con 0x7fae08072840 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0cc00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fae08140a60 0x7fae0815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae07e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fae080ccb60 0x7fae08072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33752/0 (socket says 172.21.15.110:33752) 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae07e00640 1 -- 172.21.15.110:0/1359829946 learned_addr learned my addr 172.21.15.110:0/1359829946 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:40.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.204+0000 7fae0cc00640 1 --2- 172.21.15.110:0/1359829946 >> v2:172.21.15.110:3300/0 conn(0x7fae08140a60 0x7fae0815b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55136/0 (socket says 172.21.15.110:55136) 2024-09-10T10:06:40.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae07400640 1 --2- 172.21.15.110:0/1359829946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae08072840 0x7fae08140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae0cc00640 1 -- 172.21.15.110:0/1359829946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae08072840 msgr2=0x7fae08140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae0cc00640 1 --2- 172.21.15.110:0/1359829946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae08072840 0x7fae08140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae0cc00640 1 -- 172.21.15.110:0/1359829946 >> v1:172.21.15.110:6789/0 conn(0x7fae080ccb60 legacy=0x7fae08072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae0cc00640 1 -- 172.21.15.110:0/1359829946 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0815ea40 con 0x7fae08140a60 2024-09-10T10:06:40.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae06a00640 1 -- 172.21.15.110:0/1359829946 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370228719 0 0) 0x7fae080ccf40 con 0x7fae080ccb60 2024-09-10T10:06:40.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae07400640 1 --2- 172.21.15.110:0/1359829946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae08072840 0x7fae08140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:40.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae0cc00640 1 --2- 172.21.15.110:0/1359829946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 0x7fae0815b120 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fae0000bfa0 tx=0x7fae0001eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7f0c6616127766 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae06a00640 1 -- 172.21.15.110:0/1359829946 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0000b2d0 con 0x7fae08140a60 2024-09-10T10:06:40.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae06a00640 1 -- 172.21.15.110:0/1359829946 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae000a7a10 con 0x7fae08140a60 2024-09-10T10:06:40.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae06a00640 1 -- 172.21.15.110:0/1359829946 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae000a7de0 con 0x7fae08140a60 2024-09-10T10:06:40.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae0ec00640 1 -- 172.21.15.110:0/1359829946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 msgr2=0x7fae0815b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.208+0000 7fae0ec00640 1 --2- 172.21.15.110:0/1359829946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 0x7fae0815b120 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fae0000bfa0 tx=0x7fae0001eb40 comp rx=0 tx=0).stop 2024-09-10T10:06:40.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- 172.21.15.110:0/1359829946 shutdown_connections 2024-09-10T10:06:40.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 --2- 172.21.15.110:0/1359829946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 0x7fae0815b120 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 --2- 172.21.15.110:0/1359829946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae08072840 0x7fae08140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- 172.21.15.110:0/1359829946 >> 172.21.15.110:0/1359829946 conn(0x7fae0806bc50 msgr2=0x7fae0806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:40.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- 172.21.15.110:0/1359829946 shutdown_connections 2024-09-10T10:06:40.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- 172.21.15.110:0/1359829946 wait complete. 2024-09-10T10:06:40.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 Processor -- start 2024-09-10T10:06:40.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- start start 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae08072840 0x7fae081310a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080ccb60 0x7fae080b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 0x7fae080ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae08140ef0 con 0x7fae08140a60 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae0815b660 con 0x7fae08072840 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fae08130c10 con 0x7fae080ccb60 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 0x7fae080ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 0x7fae080ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55140/0 (socket says 172.21.15.110:55140) 2024-09-10T10:06:40.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 -- 172.21.15.110:0/2621790102 learned_addr learned my addr 172.21.15.110:0/2621790102 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae07e00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae08072840 0x7fae081310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae07400640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080ccb60 0x7fae080b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 -- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080ccb60 msgr2=0x7fae080b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080ccb60 0x7fae080b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 -- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae08072840 msgr2=0x7fae081310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae08072840 0x7fae081310a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 -- 172.21.15.110:0/2621790102 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae080b2cc0 con 0x7fae08140a60 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae07e00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae08072840 0x7fae081310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae07400640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fae080ccb60 0x7fae080b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0cc00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 0x7fae080ae840 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fae000b1040 tx=0x7fae0001b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae000bd020 con 0x7fae08140a60 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae00003ba0 con 0x7fae08140a60 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00009750 con 0x7fae08140a60 2024-09-10T10:06:40.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.212+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae081305c0 con 0x7fae08140a60 2024-09-10T10:06:40.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.216+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae080b2fa0 con 0x7fae08140a60 2024-09-10T10:06:40.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.216+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae08140520 con 0x7fae08140a60 2024-09-10T10:06:40.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.216+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fae000b6070 con 0x7fae08140a60 2024-09-10T10:06:40.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.216+0000 7fadffe00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fade8068430 0x7fade806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.216+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fae00120360 con 0x7fae08140a60 2024-09-10T10:06:40.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.216+0000 7fadffe00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7fade806e2f0 0x7fade8070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.216+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 --> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae000a8e50 con 0x7fade806e2f0 2024-09-10T10:06:40.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.216+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=168) ==== 24+0+0 (secure 0 0 0) 0x7fae00122ab0 con 0x7fae08140a60 2024-09-10T10:06:40.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.220+0000 7fae07e00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fade8068430 0x7fade806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.220+0000 7fae07400640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7fade806e2f0 0x7fade8070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.220+0000 7fae07e00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fade8068430 0x7fade806a8f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fadf8000f70 tx=0x7fadf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.220+0000 7fae07400640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7fade806e2f0 0x7fade8070730 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.220+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 <== osd.4 v2:172.21.15.145:6816/927410831 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae000a8e50 con 0x7fade806e2f0 2024-09-10T10:06:40.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.232+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 --> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fae080b2a50 con 0x7fade806e2f0 2024-09-10T10:06:40.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.232+0000 7fadffe00640 1 -- 172.21.15.110:0/2621790102 <== osd.4 v2:172.21.15.145:6816/927410831 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fae080b2a50 con 0x7fade806e2f0 2024-09-10T10:06:40.236 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:40.236 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:06:40.236 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:40.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7fade806e2f0 msgr2=0x7fade8070730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6816/927410831,v1:172.21.15.145:6817/927410831] conn(0x7fade806e2f0 0x7fade8070730 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fade8068430 msgr2=0x7fade806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fade8068430 0x7fade806a8f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fadf8000f70 tx=0x7fadf8009040 comp rx=0 tx=0).stop 2024-09-10T10:06:40.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 msgr2=0x7fae080ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 --2- 172.21.15.110:0/2621790102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08140a60 0x7fae080ae840 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fae000b1040 tx=0x7fae0001b8e0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0cc00640 1 -- 172.21.15.110:0/2621790102 reap_dead start 2024-09-10T10:06:40.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 shutdown_connections 2024-09-10T10:06:40.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 >> 172.21.15.110:0/2621790102 conn(0x7fae0806bc50 msgr2=0x7fae080b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:40.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 shutdown_connections 2024-09-10T10:06:40.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.236+0000 7fae0ec00640 1 -- 172.21.15.110:0/2621790102 wait complete. 2024-09-10T10:06:40.254 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T10:06:40.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.388+0000 7fa445c00640 1 Processor -- start 2024-09-10T10:06:40.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.388+0000 7fa445c00640 1 -- start start 2024-09-10T10:06:40.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.388+0000 7fa445c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa4400726e0 0x7fa4400cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.388+0000 7fa445c00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44006f410 con 0x7fa44006efa0 2024-09-10T10:06:40.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.388+0000 7fa445c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44015fda0 con 0x7fa4400b5600 2024-09-10T10:06:40.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.388+0000 7fa445c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa4400cee50 con 0x7fa4400726e0 2024-09-10T10:06:40.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa437e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fa44006efa0 0x7fa4400b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33760/0 (socket says 172.21.15.110:33760) 2024-09-10T10:06:40.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa437e00640 1 -- 172.21.15.110:0/4199122978 learned_addr learned my addr 172.21.15.110:0/4199122978 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43ea00640 1 --2- 172.21.15.110:0/4199122978 >> v2:172.21.15.145:3300/0 conn(0x7fa4400726e0 0x7fa4400cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3674315750 0 0) 0x7fa428001120 con 0x7fa44006efa0 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa424005740 con 0x7fa44006efa0 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4234843009 0 0) 0x7fa424005740 con 0x7fa44006efa0 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 >> v2:172.21.15.145:3300/0 conn(0x7fa4400726e0 msgr2=0x7fa4400cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 --2- 172.21.15.110:0/4199122978 >> v2:172.21.15.145:3300/0 conn(0x7fa4400726e0 0x7fa4400cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 >> v1:172.21.15.145:6789/0 conn(0x7fa4400b5600 legacy=0x7fa4400b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4400ce520 con 0x7fa44006efa0 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7fa428001850 con 0x7fa44006efa0 2024-09-10T10:06:40.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 >> v1:172.21.15.110:6789/0 conn(0x7fa44006efa0 legacy=0x7fa4400b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa424005bd0 0x7fa424006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa424006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4240073e0 0x7fa42400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4400cee50 con 0x7fa424006a20 2024-09-10T10:06:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa42400e2b0 con 0x7fa424005bd0 2024-09-10T10:06:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa42400e4b0 con 0x7fa4240073e0 2024-09-10T10:06:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa437e00640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa424006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43ea00640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa424005bd0 0x7fa424006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43f400640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4240073e0 0x7fa42400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa437e00640 1 -- 172.21.15.110:0/4199122978 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4240073e0 msgr2=0x7fa42400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa437e00640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa4240073e0 0x7fa42400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa437e00640 1 -- 172.21.15.110:0/4199122978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa424005bd0 msgr2=0x7fa424006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa437e00640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa424005bd0 0x7fa424006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa437e00640 1 -- 172.21.15.110:0/4199122978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa424005740 con 0x7fa424006a20 2024-09-10T10:06:40.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.392+0000 7fa43f400640 1 -- 172.21.15.110:0/4199122978 reap_dead start 2024-09-10T10:06:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa437e00640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa424006ea0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fa42800c4c0 tx=0x7fa42801f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4662bdfae0249a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42800c0e0 con 0x7fa424006a20 2024-09-10T10:06:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa428005b00 con 0x7fa424006a20 2024-09-10T10:06:40.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa43e000640 1 -- 172.21.15.110:0/4199122978 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428007070 con 0x7fa424006a20 2024-09-10T10:06:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 -- 172.21.15.110:0/4199122978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 msgr2=0x7fa424006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa424006ea0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fa42800c4c0 tx=0x7fa42801f3e0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 -- 172.21.15.110:0/4199122978 shutdown_connections 2024-09-10T10:06:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 --2- 172.21.15.110:0/4199122978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa424006ea0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 -- 172.21.15.110:0/4199122978 >> 172.21.15.110:0/4199122978 conn(0x7fa440062d10 msgr2=0x7fa44006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 -- 172.21.15.110:0/4199122978 shutdown_connections 2024-09-10T10:06:40.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 -- 172.21.15.110:0/4199122978 wait complete. 2024-09-10T10:06:40.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 Processor -- start 2024-09-10T10:06:40.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.396+0000 7fa445c00640 1 -- start start 2024-09-10T10:06:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa440156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa440146cd0 0x7fa4401544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa440154a10 0x7fa440156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa440073270 con 0x7fa424006a20 2024-09-10T10:06:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4400ceae0 con 0x7fa440146cd0 2024-09-10T10:06:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa440072ac0 con 0x7fa440154a10 2024-09-10T10:06:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa440156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa440156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55158/0 (socket says 172.21.15.110:55158) 2024-09-10T10:06:40.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 -- 172.21.15.110:0/894639279 learned_addr learned my addr 172.21.15.110:0/894639279 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:40.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43f400640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa440154a10 0x7fa440156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa437e00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa440146cd0 0x7fa4401544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 -- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa440154a10 msgr2=0x7fa440156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa440154a10 0x7fa440156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 -- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa440146cd0 msgr2=0x7fa4401544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa440146cd0 0x7fa4401544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 -- 172.21.15.110:0/894639279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4400cc7e0 con 0x7fa424006a20 2024-09-10T10:06:40.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa437e00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa440146cd0 0x7fa4401544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:40.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa43ea00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa440156010 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fa430011aa0 tx=0x7fa430011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa436a00640 1 -- 172.21.15.110:0/894639279 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43000fb20 con 0x7fa424006a20 2024-09-10T10:06:40.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa436a00640 1 -- 172.21.15.110:0/894639279 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43000fcc0 con 0x7fa424006a20 2024-09-10T10:06:40.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa436a00640 1 -- 172.21.15.110:0/894639279 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430014680 con 0x7fa424006a20 2024-09-10T10:06:40.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa440154f70 con 0x7fa424006a20 2024-09-10T10:06:40.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa440155530 con 0x7fa424006a20 2024-09-10T10:06:40.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44015f950 con 0x7fa424006a20 2024-09-10T10:06:40.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.400+0000 7fa436a00640 1 -- 172.21.15.110:0/894639279 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fa430012040 con 0x7fa424006a20 2024-09-10T10:06:40.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.404+0000 7fa436a00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa41c068430 0x7fa41c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.404+0000 7fa436a00640 1 -- 172.21.15.110:0/894639279 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fa430084eb0 con 0x7fa424006a20 2024-09-10T10:06:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.404+0000 7fa437e00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa41c068430 0x7fa41c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.404+0000 7fa437e00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa41c068430 0x7fa41c06a8f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa42800c4c0 tx=0x7fa428007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.408+0000 7fa436a00640 1 -- 172.21.15.110:0/894639279 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa430019030 con 0x7fa424006a20 2024-09-10T10:06:40.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fa4400009c0 con 0x7fa41c068430 2024-09-10T10:06:40.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa436a00640 1 -- 172.21.15.110:0/894639279 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa4400009c0 con 0x7fa41c068430 2024-09-10T10:06:40.574 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T10:06:40.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa41c068430 msgr2=0x7fa41c06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa41c068430 0x7fa41c06a8f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa42800c4c0 tx=0x7fa428007040 comp rx=0 tx=0).stop 2024-09-10T10:06:40.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 msgr2=0x7fa440156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa440156010 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fa430011aa0 tx=0x7fa430011f60 comp rx=0 tx=0).stop 2024-09-10T10:06:40.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 shutdown_connections 2024-09-10T10:06:40.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa41c068430 0x7fa41c06a8f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa440154a10 0x7fa440156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa440146cd0 0x7fa4401544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 --2- 172.21.15.110:0/894639279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa424006a20 0x7fa440156010 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 >> 172.21.15.110:0/894639279 conn(0x7fa440062d10 msgr2=0x7fa4400718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:40.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 shutdown_connections 2024-09-10T10:06:40.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.572+0000 7fa445c00640 1 -- 172.21.15.110:0/894639279 wait complete. 2024-09-10T10:06:40.595 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T10:06:40.595 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:06:40.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2ce9600640 1 Processor -- start 2024-09-10T10:06:40.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2ce9600640 1 -- start start 2024-09-10T10:06:40.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2ce9600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ce4155970 0x7f2ce40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2ce9600640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2ce415c560 0x7f2ce406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2ce9600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce4155d50 con 0x7f2ce406f010 2024-09-10T10:06:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2ce9600640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce40b4570 con 0x7f2ce415c560 2024-09-10T10:06:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2ce9600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce406f450 con 0x7f2ce4155970 2024-09-10T10:06:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2cda000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2ce415c560 0x7f2ce406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2cda000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2ce415c560 0x7f2ce406ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55168/0 (socket says 172.21.15.110:55168) 2024-09-10T10:06:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2cda000640 1 -- 172.21.15.110:0/635029753 learned_addr learned my addr 172.21.15.110:0/635029753 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:40.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2ce2a00640 1 --2- 172.21.15.110:0/635029753 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ce4155970 0x7f2ce40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2cda000640 1 -- 172.21.15.110:0/635029753 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ce4155970 msgr2=0x7f2ce40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2cda000640 1 --2- 172.21.15.110:0/635029753 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ce4155970 0x7f2ce40cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2cda000640 1 -- 172.21.15.110:0/635029753 >> v1:172.21.15.145:6789/0 conn(0x7f2ce406f010 legacy=0x7f2ce415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.732+0000 7f2cda000640 1 -- 172.21.15.110:0/635029753 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce40b34d0 con 0x7f2ce415c560 2024-09-10T10:06:40.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2cda000640 1 --2- 172.21.15.110:0/635029753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce415c560 0x7f2ce406ead0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f2ce406f5d0 tx=0x7f2ccc01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d665bcea092391e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce2000640 1 -- 172.21.15.110:0/635029753 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0a5810 con 0x7f2ce415c560 2024-09-10T10:06:40.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce2000640 1 -- 172.21.15.110:0/635029753 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc0a59b0 con 0x7f2ce415c560 2024-09-10T10:06:40.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce2000640 1 -- 172.21.15.110:0/635029753 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0a5d80 con 0x7f2ce415c560 2024-09-10T10:06:40.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce9600640 1 -- 172.21.15.110:0/635029753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce415c560 msgr2=0x7f2ce406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce9600640 1 --2- 172.21.15.110:0/635029753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce415c560 0x7f2ce406ead0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f2ce406f5d0 tx=0x7f2ccc01c950 comp rx=0 tx=0).stop 2024-09-10T10:06:40.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce9600640 1 -- 172.21.15.110:0/635029753 shutdown_connections 2024-09-10T10:06:40.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce9600640 1 --2- 172.21.15.110:0/635029753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce415c560 0x7f2ce406ead0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce9600640 1 --2- 172.21.15.110:0/635029753 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ce4155970 0x7f2ce40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce9600640 1 -- 172.21.15.110:0/635029753 >> 172.21.15.110:0/635029753 conn(0x7f2ce406bc50 msgr2=0x7f2ce406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:40.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce9600640 1 -- 172.21.15.110:0/635029753 shutdown_connections 2024-09-10T10:06:40.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.736+0000 7f2ce9600640 1 -- 172.21.15.110:0/635029753 wait complete. 2024-09-10T10:06:40.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 Processor -- start 2024-09-10T10:06:40.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 -- start start 2024-09-10T10:06:40.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce406f010 0x7f2ce4140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ce4155970 0x7f2ce4143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ce415c560 0x7f2ce41441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce40cce00 con 0x7f2ce406f010 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce40b4050 con 0x7f2ce4155970 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce4168030 con 0x7f2ce415c560 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce406f010 0x7f2ce4140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce406f010 0x7f2ce4140680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55184/0 (socket says 172.21.15.110:55184) 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 -- 172.21.15.110:0/3452695709 learned_addr learned my addr 172.21.15.110:0/3452695709 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:40.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 -- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ce415c560 msgr2=0x7f2ce41441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2ce415c560 0x7f2ce41441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2cda000640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ce4155970 0x7f2ce4143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 -- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ce4155970 msgr2=0x7f2ce4143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ce4155970 0x7f2ce4143710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 -- 172.21.15.110:0/3452695709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce4143c50 con 0x7f2ce406f010 2024-09-10T10:06:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2cda000640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2ce4155970 0x7f2ce4143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce2a00640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce406f010 0x7f2ce4140680 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e930 tx=0x7f2cd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4017070 con 0x7f2ce406f010 2024-09-10T10:06:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd400c9a0 con 0x7f2ce406f010 2024-09-10T10:06:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce4140ff0 con 0x7f2ce406f010 2024-09-10T10:06:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400cd50 con 0x7f2ce406f010 2024-09-10T10:06:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce41415b0 con 0x7f2ce406f010 2024-09-10T10:06:40.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.740+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ce40cc8c0 con 0x7f2ce406f010 2024-09-10T10:06:40.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f2cd401a070 con 0x7f2ce406f010 2024-09-10T10:06:40.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2cdb400640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2cc0068400 0x7f2cc006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f2cd4085060 con 0x7f2ce406f010 2024-09-10T10:06:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2cdb400640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7f2cc006e2c0 0x7f2cc0070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 --> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2cd4026e50 con 0x7f2cc006e2c0 2024-09-10T10:06:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=168) ==== 24+0+0 (secure 0 0 0) 0x7f2cd40877a0 con 0x7f2ce406f010 2024-09-10T10:06:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2cda000640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2cc0068400 0x7f2cc006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2ce3400640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7f2cc006e2c0 0x7f2cc0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2cda000640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2cc0068400 0x7f2cc006a8c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0ae000 tx=0x7f2ccc0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.744+0000 7f2ce3400640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7f2cc006e2c0 0x7f2cc0070700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.748+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 <== osd.5 v2:172.21.15.145:6808/1843342183 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cd4026e50 con 0x7f2cc006e2c0 2024-09-10T10:06:40.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 --> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2ce406c310 con 0x7f2cc006e2c0 2024-09-10T10:06:40.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2cdb400640 1 -- 172.21.15.110:0/3452695709 <== osd.5 v2:172.21.15.145:6808/1843342183 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2ce406c310 con 0x7f2cc006e2c0 2024-09-10T10:06:40.762 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:40.762 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:06:40.762 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:40.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7f2cc006e2c0 msgr2=0x7f2cc0070700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6808/1843342183,v1:172.21.15.145:6809/1843342183] conn(0x7f2cc006e2c0 0x7f2cc0070700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2cc0068400 msgr2=0x7f2cc006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2cc0068400 0x7f2cc006a8c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0ae000 tx=0x7f2ccc0a7070 comp rx=0 tx=0).stop 2024-09-10T10:06:40.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce406f010 msgr2=0x7f2ce4140680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 --2- 172.21.15.110:0/3452695709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ce406f010 0x7f2ce4140680 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e930 tx=0x7f2cd400edf0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce3400640 1 -- 172.21.15.110:0/3452695709 reap_dead start 2024-09-10T10:06:40.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 shutdown_connections 2024-09-10T10:06:40.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 >> 172.21.15.110:0/3452695709 conn(0x7f2ce406bc50 msgr2=0x7f2ce40b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:40.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 shutdown_connections 2024-09-10T10:06:40.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.760+0000 7f2ce9600640 1 -- 172.21.15.110:0/3452695709 wait complete. 2024-09-10T10:06:40.781 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T10:06:40.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.916+0000 7f8638200640 1 Processor -- start 2024-09-10T10:06:40.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8638200640 1 -- start start 2024-09-10T10:06:40.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8638200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f86300cfb90 0x7f863015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8638200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86300cdcb0 0x7f863006db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8638200640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f863015d4d0 con 0x7f86300b3f40 2024-09-10T10:06:40.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8638200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f863006d130 con 0x7f86300cfb90 2024-09-10T10:06:40.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8638200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f863015d7b0 con 0x7f86300cdcb0 2024-09-10T10:06:40.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8634e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f86300b3f40 0x7f86300cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33766/0 (socket says 172.21.15.110:33766) 2024-09-10T10:06:40.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8634e00640 1 -- 172.21.15.110:0/2821691632 learned_addr learned my addr 172.21.15.110:0/2821691632 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:40.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8636200640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86300cdcb0 0x7f863006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8635800640 1 --2- 172.21.15.110:0/2821691632 >> v2:172.21.15.145:3300/0 conn(0x7f86300cfb90 0x7f863015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4041755282 0 0) 0x7f863015d4d0 con 0x7f86300b3f40 2024-09-10T10:06:40.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8618003530 con 0x7f86300b3f40 2024-09-10T10:06:40.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2483629523 0 0) 0x7f8618003530 con 0x7f86300b3f40 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86300cdcb0 msgr2=0x7f863006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86300cdcb0 0x7f863006db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 >> v2:172.21.15.145:3300/0 conn(0x7f86300cfb90 msgr2=0x7f863015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 --2- 172.21.15.110:0/2821691632 >> v2:172.21.15.145:3300/0 conn(0x7f86300cfb90 0x7f863015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863006d380 con 0x7f86300b3f40 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2673182657 0 0) 0x7f861c001680 con 0x7f86300b3f40 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 >> v1:172.21.15.110:6789/0 conn(0x7f86300b3f40 legacy=0x7f86300cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86180039c0 0x7f8618003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86180043b0 0x7f8618004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8618004fa0 0x7f861800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f863006d130 con 0x7f8618004fa0 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f863015d7b0 con 0x7f86180043b0 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f861800bf40 con 0x7f86180039c0 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8636200640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8618004fa0 0x7f861800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8635800640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86180039c0 0x7f8618003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8634e00640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86180043b0 0x7f8618004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8636200640 1 -- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86180039c0 msgr2=0x7f8618003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8636200640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f86180039c0 0x7f8618003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8636200640 1 -- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86180043b0 msgr2=0x7f8618004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8636200640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86180043b0 0x7f8618004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8636200640 1 -- 172.21.15.110:0/2821691632 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8618003530 con 0x7f8618004fa0 2024-09-10T10:06:40.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.920+0000 7f8634e00640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86180043b0 0x7f8618004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:40.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f8636200640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8618004fa0 0x7f861800b9d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f8628009f00 tx=0x7f862801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da58cab28d675f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86280b2070 con 0x7f8618004fa0 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86280a5c10 con 0x7f8618004fa0 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f862fe00640 1 -- 172.21.15.110:0/2821691632 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628009bd0 con 0x7f8618004fa0 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f8638200640 1 -- 172.21.15.110:0/2821691632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8618004fa0 msgr2=0x7f861800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f8638200640 1 --2- 172.21.15.110:0/2821691632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8618004fa0 0x7f861800b9d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f8628009f00 tx=0x7f862801cb60 comp rx=0 tx=0).stop 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f8636200640 1 -- 172.21.15.110:0/2821691632 reap_dead start 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f8638200640 1 -- 172.21.15.110:0/2821691632 shutdown_connections 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f8638200640 1 -- 172.21.15.110:0/2821691632 >> 172.21.15.110:0/2821691632 conn(0x7f8630062d10 msgr2=0x7f863006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f8638200640 1 -- 172.21.15.110:0/2821691632 shutdown_connections 2024-09-10T10:06:40.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.924+0000 7f8638200640 1 -- 172.21.15.110:0/2821691632 wait complete. 2024-09-10T10:06:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 Processor -- start 2024-09-10T10:06:40.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 -- start start 2024-09-10T10:06:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8618004fa0 0x7f86300ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8630072e00 0x7f86300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863015b6b0 0x7f86300af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86300d00f0 con 0x7f863015b6b0 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86300cff70 con 0x7f8618004fa0 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f86300d0270 con 0x7f8630072e00 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863015b6b0 0x7f86300af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863015b6b0 0x7f86300af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55192/0 (socket says 172.21.15.110:55192) 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 -- 172.21.15.110:0/3099145064 learned_addr learned my addr 172.21.15.110:0/3099145064 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8634e00640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8630072e00 0x7f86300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8635800640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8618004fa0 0x7f86300ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 -- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8630072e00 msgr2=0x7f86300710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8630072e00 0x7f86300710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 -- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8618004fa0 msgr2=0x7f86300ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:40.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8618004fa0 0x7f86300ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:40.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 -- 172.21.15.110:0/3099145064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86300719b0 con 0x7f863015b6b0 2024-09-10T10:06:40.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8636200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863015b6b0 0x7f86300af1f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8628009f00 tx=0x7f8628009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f862d600640 1 -- 172.21.15.110:0/3099145064 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86280b2040 con 0x7f863015b6b0 2024-09-10T10:06:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f862d600640 1 -- 172.21.15.110:0/3099145064 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86280bb070 con 0x7f863015b6b0 2024-09-10T10:06:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f862d600640 1 -- 172.21.15.110:0/3099145064 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628009bd0 con 0x7f863015b6b0 2024-09-10T10:06:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8630071c90 con 0x7f863015b6b0 2024-09-10T10:06:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.928+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8630152430 con 0x7f863015b6b0 2024-09-10T10:06:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.932+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8630152630 con 0x7f863015b6b0 2024-09-10T10:06:40.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.932+0000 7f862d600640 1 -- 172.21.15.110:0/3099145064 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f86280b6650 con 0x7f863015b6b0 2024-09-10T10:06:40.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.932+0000 7f862d600640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8610068430 0x7f861006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:40.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.932+0000 7f862d600640 1 -- 172.21.15.110:0/3099145064 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f862811f4f0 con 0x7f863015b6b0 2024-09-10T10:06:40.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.932+0000 7f8635800640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8610068430 0x7f861006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:40.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.932+0000 7f8635800640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8610068430 0x7f861006a8f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8624009b20 tx=0x7f8624008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:40.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:40.936+0000 7f862d600640 1 -- 172.21.15.110:0/3099145064 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8630152630 con 0x7f863015b6b0 2024-09-10T10:06:41.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f86300cf0a0 con 0x7f8610068430 2024-09-10T10:06:41.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f862d600640 1 -- 172.21.15.110:0/3099145064 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f86300cf0a0 con 0x7f8610068430 2024-09-10T10:06:41.098 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T10:06:41.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8610068430 msgr2=0x7f861006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8610068430 0x7f861006a8f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8624009b20 tx=0x7f8624008040 comp rx=0 tx=0).stop 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863015b6b0 msgr2=0x7f86300af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863015b6b0 0x7f86300af1f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8628009f00 tx=0x7f8628009750 comp rx=0 tx=0).stop 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 shutdown_connections 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8610068430 0x7f861006a8f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f863015b6b0 0x7f86300af1f0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f8630072e00 0x7f86300710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 --2- 172.21.15.110:0/3099145064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8618004fa0 0x7f86300ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 >> 172.21.15.110:0/3099145064 conn(0x7f8630062d10 msgr2=0x7f863015efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 shutdown_connections 2024-09-10T10:06:41.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.096+0000 7f8638200640 1 -- 172.21.15.110:0/3099145064 wait complete. 2024-09-10T10:06:41.118 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T10:06:41.118 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:06:41.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.256+0000 7f9e23a00640 1 Processor -- start 2024-09-10T10:06:41.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.256+0000 7f9e23a00640 1 -- start start 2024-09-10T10:06:41.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e23a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 0x7f9e1c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e23a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e1c06e7a0 0x7f9e1c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e23a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e1c06d2d0 con 0x7f9e1c0cd510 2024-09-10T10:06:41.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e23a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9e1c0cdeb0 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e23a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c15d9a0 con 0x7f9e1c0cfe30 2024-09-10T10:06:41.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9e1c06e7a0 0x7f9e1c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 0x7f9e1c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21000640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 0x7f9e1c071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:47388/0 (socket says 172.21.15.110:47388) 2024-09-10T10:06:41.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21000640 1 -- 172.21.15.110:0/1448130547 learned_addr learned my addr 172.21.15.110:0/1448130547 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:41.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e1be00640 1 --1- 172.21.15.110:0/1448130547 >> v1:172.21.15.145:6789/0 conn(0x7f9e1c0cd510 0x7f9e1c06e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:48790/0 (socket says 172.21.15.110:48790) 2024-09-10T10:06:41.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21000640 1 -- 172.21.15.110:0/1448130547 >> v2:172.21.15.145:3300/0 conn(0x7f9e1c06e7a0 msgr2=0x7f9e1c0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:41.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21000640 1 --2- 172.21.15.110:0/1448130547 >> v2:172.21.15.145:3300/0 conn(0x7f9e1c06e7a0 0x7f9e1c0cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21000640 1 -- 172.21.15.110:0/1448130547 >> v1:172.21.15.145:6789/0 conn(0x7f9e1c0cd510 legacy=0x7f9e1c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21000640 1 -- 172.21.15.110:0/1448130547 --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c06d5b0 con 0x7f9e1c0cfe30 2024-09-10T10:06:41.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.260+0000 7f9e21a00640 1 --2- 172.21.15.110:0/1448130547 >> v2:172.21.15.145:3300/0 conn(0x7f9e1c06e7a0 0x7f9e1c0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:06:41.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e21000640 1 --2- 172.21.15.110:0/1448130547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 0x7f9e1c071a40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009e50 tx=0x7f9e0c01c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a95fc5a7f7d6d57d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e1b400640 1 -- 172.21.15.110:0/1448130547 <== mon.2 v2:172.21.15.145:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0a5770 con 0x7f9e1c0cfe30 2024-09-10T10:06:41.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e1b400640 1 -- 172.21.15.110:0/1448130547 <== mon.2 v2:172.21.15.145:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0c0a5910 con 0x7f9e1c0cfe30 2024-09-10T10:06:41.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e1b400640 1 -- 172.21.15.110:0/1448130547 <== mon.2 v2:172.21.15.145:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0a5ce0 con 0x7f9e1c0cfe30 2024-09-10T10:06:41.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 -- 172.21.15.110:0/1448130547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 msgr2=0x7f9e1c071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 --2- 172.21.15.110:0/1448130547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 0x7f9e1c071a40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009e50 tx=0x7f9e0c01c9a0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 -- 172.21.15.110:0/1448130547 shutdown_connections 2024-09-10T10:06:41.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 --2- 172.21.15.110:0/1448130547 >> v2:172.21.15.145:3300/0 conn(0x7f9e1c06e7a0 0x7f9e1c0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 --2- 172.21.15.110:0/1448130547 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 0x7f9e1c071a40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 -- 172.21.15.110:0/1448130547 >> 172.21.15.110:0/1448130547 conn(0x7f9e1c06bc50 msgr2=0x7f9e1c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:41.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 -- 172.21.15.110:0/1448130547 shutdown_connections 2024-09-10T10:06:41.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 -- 172.21.15.110:0/1448130547 wait complete. 2024-09-10T10:06:41.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 Processor -- start 2024-09-10T10:06:41.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 -- start start 2024-09-10T10:06:41.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1c06e7a0 0x7f9e1c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1c0cd510 0x7f9e1c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.264+0000 7f9e23a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 0x7f9e1c165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e23a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c072190 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e23a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c15d640 con 0x7f9e1c0cd510 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e23a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c169180 con 0x7f9e1c0cfe30 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1c06e7a0 0x7f9e1c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1c06e7a0 0x7f9e1c0b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55200/0 (socket says 172.21.15.110:55200) 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 -- 172.21.15.110:0/1553780706 learned_addr learned my addr 172.21.15.110:0/1553780706 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 -- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 msgr2=0x7f9e1c165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f9e1c0cfe30 0x7f9e1c165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 -- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1c0cd510 msgr2=0x7f9e1c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1c0cd510 0x7f9e1c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 -- 172.21.15.110:0/1553780706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c0b48e0 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e21000640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1c06e7a0 0x7f9e1c0b5000 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c01a5d0 tx=0x7f9e0c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0bc070 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0c0b8640 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c0b4bc0 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0b8a10 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c157b20 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e1c0b4690 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.268+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f9e0c0b8bb0 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e18c00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dfc068400 0x7f9dfc06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f9e0c11ef20 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e18c00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f9dfc06e2c0 0x7f9dfc070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 --> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9dfc070dd0 con 0x7f9dfc06e2c0 2024-09-10T10:06:41.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e1be00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dfc068400 0x7f9dfc06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=168) ==== 24+0+0 (secure 0 0 0) 0x7f9e0c128050 con 0x7f9e1c06e7a0 2024-09-10T10:06:41.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e21a00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f9dfc06e2c0 0x7f9dfc070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e1be00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dfc068400 0x7f9dfc06a8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9e08000f70 tx=0x7f9e08007a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e21a00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f9dfc06e2c0 0x7f9dfc070700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.272+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 <== osd.6 v2:172.21.15.145:6824/3155222893 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9dfc070dd0 con 0x7f9dfc06e2c0 2024-09-10T10:06:41.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.284+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 --> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9e1c0cd970 con 0x7f9dfc06e2c0 2024-09-10T10:06:41.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e18c00640 1 -- 172.21.15.110:0/1553780706 <== osd.6 v2:172.21.15.145:6824/3155222893 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9e1c0cd970 con 0x7f9dfc06e2c0 2024-09-10T10:06:41.288 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:41.288 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:06:41.289 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:41.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f9dfc06e2c0 msgr2=0x7f9dfc070700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6824/3155222893,v1:172.21.15.145:6825/3155222893] conn(0x7f9dfc06e2c0 0x7f9dfc070700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dfc068400 msgr2=0x7f9dfc06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f9dfc068400 0x7f9dfc06a8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9e08000f70 tx=0x7f9e08007a10 comp rx=0 tx=0).stop 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1c06e7a0 msgr2=0x7f9e1c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 --2- 172.21.15.110:0/1553780706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e1c06e7a0 0x7f9e1c0b5000 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c01a5d0 tx=0x7f9e0c019740 comp rx=0 tx=0).stop 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e21a00640 1 -- 172.21.15.110:0/1553780706 reap_dead start 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 shutdown_connections 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 >> 172.21.15.110:0/1553780706 conn(0x7f9e1c06bc50 msgr2=0x7f9e1c166290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 shutdown_connections 2024-09-10T10:06:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.288+0000 7f9e23a00640 1 -- 172.21.15.110:0/1553780706 wait complete. 2024-09-10T10:06:41.306 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T10:06:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.440+0000 7f2c6ce00640 1 Processor -- start 2024-09-10T10:06:41.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.440+0000 7f2c6ce00640 1 -- start start 2024-09-10T10:06:41.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c6ce00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2c680b5600 0x7f2c680cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c6ce00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 0x7f2c680b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c6ce00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c6806f3e0 con 0x7f2c680726e0 2024-09-10T10:06:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c6ce00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f2c680cee50 con 0x7f2c680b5600 2024-09-10T10:06:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c6ce00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2c6815fda0 con 0x7f2c6806efa0 2024-09-10T10:06:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c65e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f2c680726e0 0x7f2c680b4ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33768/0 (socket says 172.21.15.110:33768) 2024-09-10T10:06:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c65400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f2c680b5600 0x7f2c680cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c65e00640 1 -- 172.21.15.110:0/3198175932 learned_addr learned my addr 172.21.15.110:0/3198175932 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:41.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c66800640 1 --2- 172.21.15.110:0/3198175932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 0x7f2c680b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c65400640 1 -- 172.21.15.110:0/3198175932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 msgr2=0x7f2c680b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:41.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c65400640 1 --2- 172.21.15.110:0/3198175932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 0x7f2c680b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c65400640 1 -- 172.21.15.110:0/3198175932 >> v1:172.21.15.110:6789/0 conn(0x7f2c680726e0 legacy=0x7f2c680b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c64a00640 1 -- 172.21.15.110:0/3198175932 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510282070 0 0) 0x7f2c6806f3e0 con 0x7f2c680726e0 2024-09-10T10:06:41.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c65400640 1 -- 172.21.15.110:0/3198175932 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c680ce520 con 0x7f2c680b5600 2024-09-10T10:06:41.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.444+0000 7f2c65400640 1 --2- 172.21.15.110:0/3198175932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c680cf2e0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2c540097c0 tx=0x7f2c5401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f99300565dc0a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c64a00640 1 -- 172.21.15.110:0/3198175932 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c540a7070 con 0x7f2c680b5600 2024-09-10T10:06:41.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c64a00640 1 -- 172.21.15.110:0/3198175932 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c540b0070 con 0x7f2c680b5600 2024-09-10T10:06:41.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c64a00640 1 -- 172.21.15.110:0/3198175932 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c540ab890 con 0x7f2c680b5600 2024-09-10T10:06:41.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- 172.21.15.110:0/3198175932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 msgr2=0x7f2c680cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/3198175932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c680cf2e0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2c540097c0 tx=0x7f2c5401c470 comp rx=0 tx=0).stop 2024-09-10T10:06:41.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- 172.21.15.110:0/3198175932 shutdown_connections 2024-09-10T10:06:41.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/3198175932 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 0x7f2c680b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/3198175932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c680cf2e0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- 172.21.15.110:0/3198175932 >> 172.21.15.110:0/3198175932 conn(0x7f2c68062d10 msgr2=0x7f2c6806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:41.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- 172.21.15.110:0/3198175932 shutdown_connections 2024-09-10T10:06:41.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- 172.21.15.110:0/3198175932 wait complete. 2024-09-10T10:06:41.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 Processor -- start 2024-09-10T10:06:41.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- start start 2024-09-10T10:06:41.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 0x7f2c68152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c680726e0 0x7f2c680ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c68154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68072c40 con 0x7f2c680b5600 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68072ac0 con 0x7f2c680726e0 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c6ce00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f2c6815f950 con 0x7f2c6806efa0 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c66800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c68154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c66800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c68154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55216/0 (socket says 172.21.15.110:55216) 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c66800640 1 -- 172.21.15.110:0/1005015500 learned_addr learned my addr 172.21.15.110:0/1005015500 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c65e00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 0x7f2c68152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.448+0000 7f2c65400640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c680726e0 0x7f2c680ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c66800640 1 -- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 msgr2=0x7f2c68152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c66800640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 0x7f2c68152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c66800640 1 -- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c680726e0 msgr2=0x7f2c680ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c66800640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c680726e0 0x7f2c680ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c66800640 1 -- 172.21.15.110:0/1005015500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c68152c50 con 0x7f2c680b5600 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c66800640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c68154cf0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000d560 tx=0x7f2c6000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c5e000640 1 -- 172.21.15.110:0/1005015500 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c600125f0 con 0x7f2c680b5600 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c5e000640 1 -- 172.21.15.110:0/1005015500 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c60019070 con 0x7f2c680b5600 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c68153490 con 0x7f2c680b5600 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c5e000640 1 -- 172.21.15.110:0/1005015500 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c600129f0 con 0x7f2c680b5600 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c680aa710 con 0x7f2c680b5600 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c680b34d0 con 0x7f2c680b5600 2024-09-10T10:06:41.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c5e000640 1 -- 172.21.15.110:0/1005015500 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f2c60012b90 con 0x7f2c680b5600 2024-09-10T10:06:41.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c5e000640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2c48068400 0x7f2c4806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c5e000640 1 -- 172.21.15.110:0/1005015500 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f2c600848c0 con 0x7f2c680b5600 2024-09-10T10:06:41.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.452+0000 7f2c65e00640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2c48068400 0x7f2c4806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.456+0000 7f2c65e00640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2c48068400 0x7f2c4806a8c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2c580012f0 tx=0x7f2c58001770 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.456+0000 7f2c5e000640 1 -- 172.21.15.110:0/1005015500 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2c60056800 con 0x7f2c680b5600 2024-09-10T10:06:41.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f2c68156610 con 0x7f2c48068400 2024-09-10T10:06:41.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c5e000640 1 -- 172.21.15.110:0/1005015500 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2c68156610 con 0x7f2c48068400 2024-09-10T10:06:41.618 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2c48068400 msgr2=0x7f2c4806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2c48068400 0x7f2c4806a8c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2c580012f0 tx=0x7f2c58001770 comp rx=0 tx=0).stop 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 msgr2=0x7f2c68154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c68154cf0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000d560 tx=0x7f2c6000da20 comp rx=0 tx=0).stop 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 shutdown_connections 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2c48068400 0x7f2c4806a8c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c680b5600 0x7f2c68154cf0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c680726e0 0x7f2c680ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 --2- 172.21.15.110:0/1005015500 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2c6806efa0 0x7f2c68152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 >> 172.21.15.110:0/1005015500 conn(0x7f2c68062d10 msgr2=0x7f2c6815edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 shutdown_connections 2024-09-10T10:06:41.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.616+0000 7f2c6ce00640 1 -- 172.21.15.110:0/1005015500 wait complete. 2024-09-10T10:06:41.639 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T10:06:41.639 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:06:41.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.776+0000 7f89f3a00640 1 Processor -- start 2024-09-10T10:06:41.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.776+0000 7f89f3a00640 1 -- start start 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f3a00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f89ec0cfe30 0x7f89ec071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f3a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cd510 0x7f89ec0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f3a00640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ec06d2d0 con 0x7f89ec06e090 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f3a00640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec0cdeb0 con 0x7f89ec0cfe30 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f3a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f89ec15d9a0 con 0x7f89ec0cd510 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f89ec0cfe30 0x7f89ec071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f1000640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f89ec0cfe30 0x7f89ec071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55226/0 (socket says 172.21.15.110:55226) 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f1000640 1 -- 172.21.15.110:0/243566376 learned_addr learned my addr 172.21.15.110:0/243566376 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:41.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f1a00640 1 --1- 172.21.15.110:0/243566376 >> v1:172.21.15.110:6789/0 conn(0x7f89ec06e090 0x7f89ec06e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33784/0 (socket says 172.21.15.110:33784) 2024-09-10T10:06:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89ebe00640 1 --2- 172.21.15.110:0/243566376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cd510 0x7f89ec0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f1000640 1 -- 172.21.15.110:0/243566376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cd510 msgr2=0x7f89ec0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f1000640 1 --2- 172.21.15.110:0/243566376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cd510 0x7f89ec0cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f1000640 1 -- 172.21.15.110:0/243566376 >> v1:172.21.15.110:6789/0 conn(0x7f89ec06e090 legacy=0x7f89ec06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89f1000640 1 -- 172.21.15.110:0/243566376 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec06d5b0 con 0x7f89ec0cfe30 2024-09-10T10:06:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89eb400640 1 -- 172.21.15.110:0/243566376 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099809907 0 0) 0x7f89ec06d2d0 con 0x7f89ec06e090 2024-09-10T10:06:41.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.780+0000 7f89ebe00640 1 --2- 172.21.15.110:0/243566376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cd510 0x7f89ec0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f1000640 1 --2- 172.21.15.110:0/243566376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cfe30 0x7f89ec071a40 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009870 tx=0x7f89e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec495d65d816ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89eb400640 1 -- 172.21.15.110:0/243566376 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e00b1070 con 0x7f89ec0cfe30 2024-09-10T10:06:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89eb400640 1 -- 172.21.15.110:0/243566376 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e00a8050 con 0x7f89ec0cfe30 2024-09-10T10:06:41.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89eb400640 1 -- 172.21.15.110:0/243566376 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e00ac860 con 0x7f89ec0cfe30 2024-09-10T10:06:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- 172.21.15.110:0/243566376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cfe30 msgr2=0x7f89ec071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 --2- 172.21.15.110:0/243566376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cfe30 0x7f89ec071a40 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009870 tx=0x7f89e001c4d0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- 172.21.15.110:0/243566376 shutdown_connections 2024-09-10T10:06:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 --2- 172.21.15.110:0/243566376 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cd510 0x7f89ec0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 --2- 172.21.15.110:0/243566376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cfe30 0x7f89ec071a40 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- 172.21.15.110:0/243566376 >> 172.21.15.110:0/243566376 conn(0x7f89ec06bc50 msgr2=0x7f89ec06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:41.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- 172.21.15.110:0/243566376 shutdown_connections 2024-09-10T10:06:41.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- 172.21.15.110:0/243566376 wait complete. 2024-09-10T10:06:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 Processor -- start 2024-09-10T10:06:41.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- start start 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89ec06e090 0x7f89ec154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cd510 0x7f89ec1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cfe30 0x7f89ec160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec072190 con 0x7f89ec0cd510 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec15d640 con 0x7f89ec06e090 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89f3a00640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f89ec0b35d0 con 0x7f89ec0cfe30 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cd510 0x7f89ec1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cd510 0x7f89ec1567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55228/0 (socket says 172.21.15.110:55228) 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.784+0000 7f89ebe00640 1 -- 172.21.15.110:0/982051545 learned_addr learned my addr 172.21.15.110:0/982051545 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89f1000640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89ec06e090 0x7f89ec154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89f1a00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cfe30 0x7f89ec160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89ebe00640 1 -- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cfe30 msgr2=0x7f89ec160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89ebe00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f89ec0cfe30 0x7f89ec160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89ebe00640 1 -- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89ec06e090 msgr2=0x7f89ec154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89ebe00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89ec06e090 0x7f89ec154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89ebe00640 1 -- 172.21.15.110:0/982051545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec1495a0 con 0x7f89ec0cd510 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89f1000640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89ec06e090 0x7f89ec154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89ebe00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cd510 0x7f89ec1567e0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c7c0 tx=0x7f89d800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8007a30 con 0x7f89ec0cd510 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d8007bd0 con 0x7f89ec0cd510 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec1531a0 con 0x7f89ec0cd510 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80113f0 con 0x7f89ec0cd510 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec1562c0 con 0x7f89ec0cd510 2024-09-10T10:06:41.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89ec0cd970 con 0x7f89ec0cd510 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f89d8011590 con 0x7f89ec0cd510 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f89cc068400 0x7f89cc06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f89d8084a50 con 0x7f89ec0cd510 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7f89cc06e2c0 0x7f89cc070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 --> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89d8006820 con 0x7f89cc06e2c0 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=168) ==== 24+0+0 (secure 0 0 0) 0x7f89d808e050 con 0x7f89ec0cd510 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.788+0000 7f89f1000640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f89cc068400 0x7f89cc06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.792+0000 7f89f1a00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7f89cc06e2c0 0x7f89cc070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.792+0000 7f89f1000640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f89cc068400 0x7f89cc06a8c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f89e00ae000 tx=0x7f89e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.792+0000 7f89f1a00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7f89cc06e2c0 0x7f89cc070700 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.792+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 <== osd.7 v2:172.21.15.145:6800/938429327 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89d8006820 con 0x7f89cc06e2c0 2024-09-10T10:06:41.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.804+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 --> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f89ec160740 con 0x7f89cc06e2c0 2024-09-10T10:06:41.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.804+0000 7f89e8c00640 1 -- 172.21.15.110:0/982051545 <== osd.7 v2:172.21.15.145:6800/938429327 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f89ec160740 con 0x7f89cc06e2c0 2024-09-10T10:06:41.808 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-10T10:06:41.808 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:06:41.808 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.804+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7f89cc06e2c0 msgr2=0x7f89cc070700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.804+0000 7f89f3a00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6800/938429327,v1:172.21.15.145:6801/938429327] conn(0x7f89cc06e2c0 0x7f89cc070700 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f89cc068400 msgr2=0x7f89cc06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f3a00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f89cc068400 0x7f89cc06a8c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f89e00ae000 tx=0x7f89e0008040 comp rx=0 tx=0).stop 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cd510 msgr2=0x7f89ec1567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f3a00640 1 --2- 172.21.15.110:0/982051545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89ec0cd510 0x7f89ec1567e0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c7c0 tx=0x7f89d800cc80 comp rx=0 tx=0).stop 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f1a00640 1 -- 172.21.15.110:0/982051545 reap_dead start 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 shutdown_connections 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 >> 172.21.15.110:0/982051545 conn(0x7f89ec06bc50 msgr2=0x7f89ec15e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 shutdown_connections 2024-09-10T10:06:41.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.808+0000 7f89f3a00640 1 -- 172.21.15.110:0/982051545 wait complete. 2024-09-10T10:06:41.826 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T10:06:41.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa511400640 1 Processor -- start 2024-09-10T10:06:41.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa511400640 1 -- start start 2024-09-10T10:06:41.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa511400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa50c15aab0 0x7fa50c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa511400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c0b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa511400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa50c0731c0 0x7fa50c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa511400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fa50c15ae90 con 0x7fa50c15aab0 2024-09-10T10:06:41.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa511400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa50c15b170 con 0x7fa50c0731c0 2024-09-10T10:06:41.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa511400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c15b450 con 0x7fa50c0b4170 2024-09-10T10:06:41.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa50a800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa50c15aab0 0x7fa50c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa50a800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa50c15aab0 0x7fa50c072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55232/0 (socket says 172.21.15.110:55232) 2024-09-10T10:06:41.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.960+0000 7fa50a800640 1 -- 172.21.15.110:0/505653946 learned_addr learned my addr 172.21.15.110:0/505653946 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa509e00640 1 --2- 172.21.15.110:0/505653946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c0b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa50a800640 1 -- 172.21.15.110:0/505653946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 msgr2=0x7fa50c0b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa50b200640 1 --2- 172.21.15.110:0/505653946 >> v2:172.21.15.145:3300/0 conn(0x7fa50c0731c0 0x7fa50c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa50a800640 1 --2- 172.21.15.110:0/505653946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c0b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa50a800640 1 -- 172.21.15.110:0/505653946 >> v2:172.21.15.145:3300/0 conn(0x7fa50c0731c0 msgr2=0x7fa50c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa50a800640 1 --2- 172.21.15.110:0/505653946 >> v2:172.21.15.145:3300/0 conn(0x7fa50c0731c0 0x7fa50c0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa50a800640 1 -- 172.21.15.110:0/505653946 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c15b730 con 0x7fa50c15aab0 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa509e00640 1 --2- 172.21.15.110:0/505653946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:41.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa50b200640 1 --2- 172.21.15.110:0/505653946 >> v2:172.21.15.145:3300/0 conn(0x7fa50c0731c0 0x7fa50c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:41.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa50a800640 1 --2- 172.21.15.110:0/505653946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c15aab0 0x7fa50c072c80 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fa50c15c1b0 tx=0x7fa4fc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9a98988cc7eb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa509400640 1 -- 172.21.15.110:0/505653946 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0a5700 con 0x7fa50c15aab0 2024-09-10T10:06:41.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa509400640 1 -- 172.21.15.110:0/505653946 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c06ece0 con 0x7fa50c15aab0 2024-09-10T10:06:41.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa509400640 1 -- 172.21.15.110:0/505653946 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0a5ac0 con 0x7fa50c15aab0 2024-09-10T10:06:41.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa511400640 1 -- 172.21.15.110:0/505653946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c15aab0 msgr2=0x7fa50c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa511400640 1 --2- 172.21.15.110:0/505653946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c15aab0 0x7fa50c072c80 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fa50c15c1b0 tx=0x7fa4fc01c9f0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa511400640 1 -- 172.21.15.110:0/505653946 shutdown_connections 2024-09-10T10:06:41.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa511400640 1 --2- 172.21.15.110:0/505653946 >> v2:172.21.15.145:3300/0 conn(0x7fa50c0731c0 0x7fa50c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa511400640 1 --2- 172.21.15.110:0/505653946 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa511400640 1 --2- 172.21.15.110:0/505653946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c15aab0 0x7fa50c072c80 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.964+0000 7fa511400640 1 -- 172.21.15.110:0/505653946 >> 172.21.15.110:0/505653946 conn(0x7fa50c062d10 msgr2=0x7fa50c06bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:41.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 -- 172.21.15.110:0/505653946 shutdown_connections 2024-09-10T10:06:41.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 -- 172.21.15.110:0/505653946 wait complete. 2024-09-10T10:06:41.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 Processor -- start 2024-09-10T10:06:41.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 -- start start 2024-09-10T10:06:41.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c0731c0 0x7fa50c14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c15aab0 0x7fa50c152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c166a20 con 0x7fa50c0731c0 2024-09-10T10:06:41.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1668a0 con 0x7fa50c15aab0 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c166ba0 con 0x7fa50c0b4170 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c0731c0 0x7fa50c14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c0731c0 0x7fa50c14e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55242/0 (socket says 172.21.15.110:55242) 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 -- 172.21.15.110:0/591222209 learned_addr learned my addr 172.21.15.110:0/591222209 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa509e00640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50b200640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c15aab0 0x7fa50c152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 -- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 msgr2=0x7fa50c1527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c1527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 -- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c15aab0 msgr2=0x7fa50c152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c15aab0 0x7fa50c152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 -- 172.21.15.110:0/591222209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c150de0 con 0x7fa50c0731c0 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50b200640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c15aab0 0x7fa50c152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:06:41.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa50a800640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c0731c0 0x7fa50c14e1f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0af040 tx=0x7fa4fc019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa502a00640 1 -- 172.21.15.110:0/591222209 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0bc020 con 0x7fa50c0731c0 2024-09-10T10:06:41.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa502a00640 1 -- 172.21.15.110:0/591222209 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc0b8690 con 0x7fa50c0731c0 2024-09-10T10:06:41.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c1510c0 con 0x7fa50c0731c0 2024-09-10T10:06:41.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa502a00640 1 -- 172.21.15.110:0/591222209 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0b8a80 con 0x7fa50c0731c0 2024-09-10T10:06:41.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.968+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c151680 con 0x7fa50c0731c0 2024-09-10T10:06:41.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.972+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50c072c80 con 0x7fa50c0731c0 2024-09-10T10:06:41.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.972+0000 7fa502a00640 1 -- 172.21.15.110:0/591222209 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fa4fc0bf020 con 0x7fa50c0731c0 2024-09-10T10:06:41.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.972+0000 7fa502a00640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec068400 0x7fa4ec06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:41.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.972+0000 7fa502a00640 1 -- 172.21.15.110:0/591222209 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fa4fc11f670 con 0x7fa50c0731c0 2024-09-10T10:06:41.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.972+0000 7fa509e00640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec068400 0x7fa4ec06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:41.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.972+0000 7fa509e00640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec068400 0x7fa4ec06a8c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa50c153fa0 tx=0x7fa4f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:41.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:41.976+0000 7fa502a00640 1 -- 172.21.15.110:0/591222209 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa4fc0c6030 con 0x7fa50c0731c0 2024-09-10T10:06:42.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.132+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa50c06ece0 con 0x7fa4ec068400 2024-09-10T10:06:42.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa502a00640 1 -- 172.21.15.110:0/591222209 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa50c06ece0 con 0x7fa4ec068400 2024-09-10T10:06:42.137 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T10:06:42.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec068400 msgr2=0x7fa4ec06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:42.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec068400 0x7fa4ec06a8c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa50c153fa0 tx=0x7fa4f8008040 comp rx=0 tx=0).stop 2024-09-10T10:06:42.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c0731c0 msgr2=0x7fa50c14e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:42.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c0731c0 0x7fa50c14e1f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0af040 tx=0x7fa4fc019790 comp rx=0 tx=0).stop 2024-09-10T10:06:42.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 shutdown_connections 2024-09-10T10:06:42.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fa4ec068400 0x7fa4ec06a8c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa50c15aab0 0x7fa50c152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fa50c0b4170 0x7fa50c1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 --2- 172.21.15.110:0/591222209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50c0731c0 0x7fa50c14e1f0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 >> 172.21.15.110:0/591222209 conn(0x7fa50c062d10 msgr2=0x7fa50c06e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:42.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 shutdown_connections 2024-09-10T10:06:42.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.136+0000 7fa511400640 1 -- 172.21.15.110:0/591222209 wait complete. 2024-09-10T10:06:42.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:06:42.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf81800640 1 Processor -- start 2024-09-10T10:06:42.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf81800640 1 -- start start 2024-09-10T10:06:42.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf81800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcf7c15c960 0x7fcf7c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:42.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf81800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcf7c1603c0 0x7fcf7c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf81800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7c0706a0 con 0x7fcf7c0d04c0 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf81800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7c15cd40 con 0x7fcf7c15c960 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf81800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7c0b6370 con 0x7fcf7c1603c0 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7aa00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fcf7c15c960 0x7fcf7c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7b400640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcf7c0d04c0 0x7fcf7c070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:33796/0 (socket says 172.21.15.110:33796) 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7b400640 1 -- 172.21.15.110:0/655844295 learned_addr learned my addr 172.21.15.110:0/655844295 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7a000640 1 --2- 172.21.15.110:0/655844295 >> v2:172.21.15.145:3300/0 conn(0x7fcf7c1603c0 0x7fcf7c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7aa00640 1 -- 172.21.15.110:0/655844295 >> v2:172.21.15.145:3300/0 conn(0x7fcf7c1603c0 msgr2=0x7fcf7c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7aa00640 1 --2- 172.21.15.110:0/655844295 >> v2:172.21.15.145:3300/0 conn(0x7fcf7c1603c0 0x7fcf7c0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7aa00640 1 -- 172.21.15.110:0/655844295 >> v1:172.21.15.110:6789/0 conn(0x7fcf7c0d04c0 legacy=0x7fcf7c070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7aa00640 1 -- 172.21.15.110:0/655844295 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c070a80 con 0x7fcf7c15c960 2024-09-10T10:06:42.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.296+0000 7fcf7a000640 1 --2- 172.21.15.110:0/655844295 >> v2:172.21.15.145:3300/0 conn(0x7fcf7c1603c0 0x7fcf7c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:06:42.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf7aa00640 1 --2- 172.21.15.110:0/655844295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c0b5e30 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0b64f0 tx=0x7fcf6c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41c4a9e4a601761 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:42.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf79600640 1 -- 172.21.15.110:0/655844295 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0b0070 con 0x7fcf7c15c960 2024-09-10T10:06:42.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf79600640 1 -- 172.21.15.110:0/655844295 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6c0ab600 con 0x7fcf7c15c960 2024-09-10T10:06:42.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf79600640 1 -- 172.21.15.110:0/655844295 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0ab9d0 con 0x7fcf7c15c960 2024-09-10T10:06:42.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf81800640 1 -- 172.21.15.110:0/655844295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 msgr2=0x7fcf7c0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:42.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf81800640 1 --2- 172.21.15.110:0/655844295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c0b5e30 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0b64f0 tx=0x7fcf6c01c310 comp rx=0 tx=0).stop 2024-09-10T10:06:42.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf81800640 1 -- 172.21.15.110:0/655844295 shutdown_connections 2024-09-10T10:06:42.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf81800640 1 --2- 172.21.15.110:0/655844295 >> v2:172.21.15.145:3300/0 conn(0x7fcf7c1603c0 0x7fcf7c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf81800640 1 --2- 172.21.15.110:0/655844295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c0b5e30 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf81800640 1 -- 172.21.15.110:0/655844295 >> 172.21.15.110:0/655844295 conn(0x7fcf7c0d25c0 msgr2=0x7fcf7c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:42.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf81800640 1 -- 172.21.15.110:0/655844295 shutdown_connections 2024-09-10T10:06:42.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.300+0000 7fcf81800640 1 -- 172.21.15.110:0/655844295 wait complete. 2024-09-10T10:06:42.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 Processor -- start 2024-09-10T10:06:42.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 -- start start 2024-09-10T10:06:42.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf7c0d04c0 0x7fcf7c14a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:42.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7c1603c0 0x7fcf7c1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c0d3150 con 0x7fcf7c15c960 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1611b0 con 0x7fcf7c0d04c0 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7c0d32d0 con 0x7fcf7c1603c0 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55258/0 (socket says 172.21.15.110:55258) 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 -- 172.21.15.110:0/22291385 learned_addr learned my addr 172.21.15.110:0/22291385 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7aa00640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf7c0d04c0 0x7fcf7c14a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7b400640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7c1603c0 0x7fcf7c1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 -- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7c1603c0 msgr2=0x7fcf7c1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7c1603c0 0x7fcf7c1594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 -- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf7c0d04c0 msgr2=0x7fcf7c14a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:42.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf7c0d04c0 0x7fcf7c14a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 -- 172.21.15.110:0/22291385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c14b000 con 0x7fcf7c15c960 2024-09-10T10:06:42.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf7a000640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c158f80 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c7f0 tx=0x7fcf6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:42.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf72a00640 1 -- 172.21.15.110:0/22291385 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68015070 con 0x7fcf7c15c960 2024-09-10T10:06:42.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf72a00640 1 -- 172.21.15.110:0/22291385 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf68007a30 con 0x7fcf7c15c960 2024-09-10T10:06:42.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf72a00640 1 -- 172.21.15.110:0/22291385 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68007e20 con 0x7fcf7c15c960 2024-09-10T10:06:42.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c14b2e0 con 0x7fcf7c15c960 2024-09-10T10:06:42.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c14b8a0 con 0x7fcf7c15c960 2024-09-10T10:06:42.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.304+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7c071a80 con 0x7fcf7c15c960 2024-09-10T10:06:42.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.308+0000 7fcf72a00640 1 -- 172.21.15.110:0/22291385 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fcf68018070 con 0x7fcf7c15c960 2024-09-10T10:06:42.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.308+0000 7fcf72a00640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf5c068400 0x7fcf5c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:42.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.308+0000 7fcf72a00640 1 -- 172.21.15.110:0/22291385 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fcf68084770 con 0x7fcf7c15c960 2024-09-10T10:06:42.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.308+0000 7fcf7aa00640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf5c068400 0x7fcf5c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:42.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.308+0000 7fcf7aa00640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf5c068400 0x7fcf5c06a8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0ad000 tx=0x7fcf6c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:42.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.312+0000 7fcf72a00640 1 -- 172.21.15.110:0/22291385 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcf68056460 con 0x7fcf7c15c960 2024-09-10T10:06:42.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf7c0d34d0 con 0x7fcf5c068400 2024-09-10T10:06:42.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf72a00640 1 -- 172.21.15.110:0/22291385 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13374 (secure 0 0 0) 0x7fcf7c0d34d0 con 0x7fcf5c068400 2024-09-10T10:06:42.474 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:06:42.474 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:06:42.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf5c068400 msgr2=0x7fcf5c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:42.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf5c068400 0x7fcf5c06a8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0ad000 tx=0x7fcf6c0a6000 comp rx=0 tx=0).stop 2024-09-10T10:06:42.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 msgr2=0x7fcf7c158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c158f80 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c7f0 tx=0x7fcf6800ccb0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 shutdown_connections 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fcf5c068400 0x7fcf5c06a8c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fcf7c1603c0 0x7fcf7c1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf7c15c960 0x7fcf7c158f80 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 --2- 172.21.15.110:0/22291385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf7c0d04c0 0x7fcf7c14a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 >> 172.21.15.110:0/22291385 conn(0x7fcf7c0d25c0 msgr2=0x7fcf7c152410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 shutdown_connections 2024-09-10T10:06:42.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:06:42.472+0000 7fcf81800640 1 -- 172.21.15.110:0/22291385 wait complete. 2024-09-10T10:06:42.494 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-10T10:06:41.756971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":283900,"kb_used_data":17736,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690820,"statfs":{"total":773094113280,"available":772803399680,"internally_reserved":0,"allocated":18161664,"data_stored":8660152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-4,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002050"},"pg_stats":[{"pgid":"1.0","version":"72'128","reported_seq":478,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-10T10:06:33.096190+0000","last_change":"2024-09-10T09:58:15.479463+0000","last_active":"2024-09-10T10:06:33.096190+0000","last_peered":"2024-09-10T10:06:33.096190+0000","last_clean":"2024-09-10T10:06:33.096190+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:06:33.096190+0000","last_undegraded":"2024-09-10T10:06:33.096190+0000","last_fullsized":"2024-09-10T10:06:33.096190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:58:14.203455+0000","last_clean_scrub_stamp":"2024-09-10T09:58:14.203455+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.830055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33840,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338000,"statfs":{"total":96636764160,"available":96602112000,"internally_reserved":0,"allocated":2715648,"data_stored":1525295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33264,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33264,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33264,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37420,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37424,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37424,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38000,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333840,"statfs":{"total":96636764160,"available":96597852160,"internally_reserved":0,"allocated":2715648,"data_stored":1525295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:06:42.497 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T09:58:14.203455+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=9, tm_min=58, tm_sec=14, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=6, tm_sec=36, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:06:42.497 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T10:06:49.617 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:06:49.621 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:07:02.498 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:07:02.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f2265400640 1 Processor -- start 2024-09-10T10:07:02.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f2265400640 1 -- start start 2024-09-10T10:07:02.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f2265400640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f226015e470 0x7f22600b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f2265400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2260162020 0x7f22600731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f2265400640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f226006d020 con 0x7f2260073720 2024-09-10T10:07:02.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f2265400640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f22600b6870 con 0x7f226015e470 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f2265400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f22600b5c60 con 0x7f2260162020 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f226015e470 0x7f22600b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225fe00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f226015e470 0x7f22600b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50714/0 (socket says 172.21.15.110:50714) 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225fe00640 1 -- 172.21.15.110:0/149669285 learned_addr learned my addr 172.21.15.110:0/149669285 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225f400640 1 --2- 172.21.15.110:0/149669285 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2260162020 0x7f22600731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225fe00640 1 -- 172.21.15.110:0/149669285 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2260162020 msgr2=0x7f22600731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225fe00640 1 --2- 172.21.15.110:0/149669285 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2260162020 0x7f22600731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225fe00640 1 -- 172.21.15.110:0/149669285 >> v1:172.21.15.110:6789/0 conn(0x7f2260073720 legacy=0x7f2260070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225ea00640 1 -- 172.21.15.110:0/149669285 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858490149 0 0) 0x7f226006d020 con 0x7f2260073720 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225fe00640 1 -- 172.21.15.110:0/149669285 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600b5f40 con 0x7f226015e470 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.644+0000 7f225f400640 1 --2- 172.21.15.110:0/149669285 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2260162020 0x7f22600731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:07:02.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f225fe00640 1 --2- 172.21.15.110:0/149669285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f226015e470 0x7f22600b55d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f2250009e50 tx=0x7f225001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f72c59ab264e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:02.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f225ea00640 1 -- 172.21.15.110:0/149669285 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500b4070 con 0x7f226015e470 2024-09-10T10:07:02.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f225ea00640 1 -- 172.21.15.110:0/149669285 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22500af400 con 0x7f226015e470 2024-09-10T10:07:02.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f225ea00640 1 -- 172.21.15.110:0/149669285 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500af7d0 con 0x7f226015e470 2024-09-10T10:07:02.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 -- 172.21.15.110:0/149669285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f226015e470 msgr2=0x7f22600b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 --2- 172.21.15.110:0/149669285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f226015e470 0x7f22600b55d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f2250009e50 tx=0x7f225001f210 comp rx=0 tx=0).stop 2024-09-10T10:07:02.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 -- 172.21.15.110:0/149669285 shutdown_connections 2024-09-10T10:07:02.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 --2- 172.21.15.110:0/149669285 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f2260162020 0x7f22600731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 --2- 172.21.15.110:0/149669285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f226015e470 0x7f22600b55d0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 -- 172.21.15.110:0/149669285 >> 172.21.15.110:0/149669285 conn(0x7f22600b42d0 msgr2=0x7f22600b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:02.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 -- 172.21.15.110:0/149669285 shutdown_connections 2024-09-10T10:07:02.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 -- 172.21.15.110:0/149669285 wait complete. 2024-09-10T10:07:02.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 Processor -- start 2024-09-10T10:07:02.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.648+0000 7f2265400640 1 -- start start 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2260073720 0x7f2260151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f226015e470 0x7f22600b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2260162020 0x7f226014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f226015e850 con 0x7f2260073720 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22600b6500 con 0x7f2260162020 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f226006e730 con 0x7f226015e470 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2260073720 0x7f2260151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2260073720 0x7f2260151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50718/0 (socket says 172.21.15.110:50718) 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 -- 172.21.15.110:0/3729429083 learned_addr learned my addr 172.21.15.110:0/3729429083 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225f400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f226015e470 0x7f22600b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2264a00640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2260162020 0x7f226014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 -- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f226015e470 msgr2=0x7f22600b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f226015e470 0x7f22600b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 -- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2260162020 msgr2=0x7f226014fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2260162020 0x7f226014fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 -- 172.21.15.110:0/3729429083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2260150990 con 0x7f2260073720 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2264a00640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2260162020 0x7f226014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f225fe00640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2260073720 0x7f2260151a30 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f225001d8c0 tx=0x7f225001bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2257e00640 1 -- 172.21.15.110:0/3729429083 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500b4040 con 0x7f2260073720 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2257e00640 1 -- 172.21.15.110:0/3729429083 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22500afe10 con 0x7f2260073720 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2257e00640 1 -- 172.21.15.110:0/3729429083 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250009510 con 0x7f2260073720 2024-09-10T10:07:02.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2260150c70 con 0x7f2260073720 2024-09-10T10:07:02.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2260151230 con 0x7f2260073720 2024-09-10T10:07:02.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226015dc60 con 0x7f2260073720 2024-09-10T10:07:02.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.652+0000 7f2257e00640 1 -- 172.21.15.110:0/3729429083 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f22500a8d10 con 0x7f2260073720 2024-09-10T10:07:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.656+0000 7f2257e00640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2240068400 0x7f224006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.656+0000 7f2257e00640 1 -- 172.21.15.110:0/3729429083 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f22501225f0 con 0x7f2260073720 2024-09-10T10:07:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.656+0000 7f225f400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2240068400 0x7f224006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.656+0000 7f225f400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2240068400 0x7f224006a8c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f224c006fd0 tx=0x7f224c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:02.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.660+0000 7f2257e00640 1 -- 172.21.15.110:0/3729429083 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2250018d70 con 0x7f2260073720 2024-09-10T10:07:02.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.812+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 --> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2260161d90 con 0x7f2240068400 2024-09-10T10:07:02.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2257e00640 1 -- 172.21.15.110:0/3729429083 <== mgr.4100 v2:172.21.15.145:6836/1978165248 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f2260161d90 con 0x7f2240068400 2024-09-10T10:07:02.817 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:07:02.817 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2240068400 msgr2=0x7f224006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2240068400 0x7f224006a8c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f224c006fd0 tx=0x7f224c008040 comp rx=0 tx=0).stop 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2260073720 msgr2=0x7f2260151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2260073720 0x7f2260151a30 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f225001d8c0 tx=0x7f225001bfb0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 shutdown_connections 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f2240068400 0x7f224006a8c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2260162020 0x7f226014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f226015e470 0x7f22600b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 --2- 172.21.15.110:0/3729429083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2260073720 0x7f2260151a30 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 >> 172.21.15.110:0/3729429083 conn(0x7f22600b42d0 msgr2=0x7f226015a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:02.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 shutdown_connections 2024-09-10T10:07:02.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.816+0000 7f2265400640 1 -- 172.21.15.110:0/3729429083 wait complete. 2024-09-10T10:07:02.837 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-10T10:07:01.761491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":283892,"kb_used_data":17736,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690828,"statfs":{"total":773094113280,"available":772803407872,"internally_reserved":0,"allocated":18161664,"data_stored":8660152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12756,"internal_metadata":272354860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002647"},"pg_stats":[{"pgid":"1.0","version":"168'130","reported_seq":489,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-10T10:06:42.253339+0000","last_change":"2024-09-10T10:06:42.253369+0000","last_active":"2024-09-10T10:06:42.253339+0000","last_peered":"2024-09-10T10:06:42.253339+0000","last_clean":"2024-09-10T10:06:42.253339+0000","last_became_active":"2024-09-10T09:58:15.478374+0000","last_became_peered":"2024-09-10T09:58:15.478374+0000","last_unstale":"2024-09-10T10:06:42.253339+0000","last_undegraded":"2024-09-10T10:06:42.253339+0000","last_fullsized":"2024-09-10T10:06:42.253339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"168'130","last_scrub_stamp":"2024-09-10T10:06:42.253264+0000","last_deep_scrub":"168'130","last_deep_scrub_stamp":"2024-09-10T10:06:42.253264+0000","last_clean_scrub_stamp":"2024-09-10T10:06:42.253264+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:38:45.094344+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2715648,"data_stored":1525295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33264,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33264,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33264,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37420,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37424,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37424,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2121728,"data_stored":934927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37996,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333844,"statfs":{"total":96636764160,"available":96597856256,"internally_reserved":0,"allocated":2715648,"data_stored":1525295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:07:02.838 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T10:07:02.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88bb800640 1 Processor -- start 2024-09-10T10:07:02.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88bb800640 1 -- start start 2024-09-10T10:07:02.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88bb800640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f88b4145f30 0x7f88b40b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88bb800640 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b40b37c0 con 0x7f88b40b29e0 2024-09-10T10:07:02.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88bb800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b40b3aa0 con 0x7f88b40cfd90 2024-09-10T10:07:02.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88bb800640 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 -- 0x7f88b40b2e50 con 0x7f88b4145f30 2024-09-10T10:07:02.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88b8e00640 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f88b4145f30 0x7f88b40b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88b3e00640 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f88b40b29e0 0x7f88b40cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:54308/0 (socket says 172.21.15.110:54308) 2024-09-10T10:07:02.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88b3e00640 1 -- 172.21.15.110:0/3809634921 learned_addr learned my addr 172.21.15.110:0/3809634921 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:07:02.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88b8e00640 1 -- 172.21.15.110:0/3809634921 >> v1:172.21.15.145:6789/0 conn(0x7f88b40cfd90 legacy=0x7f88b40b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:02.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88b8e00640 1 -- 172.21.15.110:0/3809634921 >> v1:172.21.15.110:6789/0 conn(0x7f88b40b29e0 legacy=0x7f88b40cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88b8e00640 1 -- 172.21.15.110:0/3809634921 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b40b3d80 con 0x7f88b4145f30 2024-09-10T10:07:02.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.972+0000 7f88b3400640 1 -- 172.21.15.110:0/3809634921 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444297932 0 0) 0x7f88b40b37c0 con 0x7f88b40b29e0 2024-09-10T10:07:02.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88b8e00640 1 --2- 172.21.15.110:0/3809634921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b24a0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f88a80097c0 tx=0x7f88a801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d71d2a441b33c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:02.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88b3400640 1 -- 172.21.15.110:0/3809634921 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80b0070 con 0x7f88b4145f30 2024-09-10T10:07:02.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88b3400640 1 -- 172.21.15.110:0/3809634921 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a80ab600 con 0x7f88b4145f30 2024-09-10T10:07:02.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88b3400640 1 -- 172.21.15.110:0/3809634921 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80ab9d0 con 0x7f88b4145f30 2024-09-10T10:07:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 -- 172.21.15.110:0/3809634921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 msgr2=0x7f88b40b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 --2- 172.21.15.110:0/3809634921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b24a0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f88a80097c0 tx=0x7f88a801c460 comp rx=0 tx=0).stop 2024-09-10T10:07:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 -- 172.21.15.110:0/3809634921 shutdown_connections 2024-09-10T10:07:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 --2- 172.21.15.110:0/3809634921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b24a0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 -- 172.21.15.110:0/3809634921 >> 172.21.15.110:0/3809634921 conn(0x7f88b4062e40 msgr2=0x7f88b406bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 -- 172.21.15.110:0/3809634921 shutdown_connections 2024-09-10T10:07:02.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 -- 172.21.15.110:0/3809634921 wait complete. 2024-09-10T10:07:02.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 Processor -- start 2024-09-10T10:07:02.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.976+0000 7f88bb800640 1 -- start start 2024-09-10T10:07:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88b40b29e0 0x7f88b413cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b40cfd90 0x7f88b414dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88b40b56b0 con 0x7f88b4145f30 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f88b40d0170 con 0x7f88b40cfd90 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7f88b40b5830 con 0x7f88b40b29e0 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50736/0 (socket says 172.21.15.110:50736) 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 -- 172.21.15.110:0/3470325882 learned_addr learned my addr 172.21.15.110:0/3470325882 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b3e00640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b40cfd90 0x7f88b414dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b8e00640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88b40b29e0 0x7f88b413cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 -- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88b40b29e0 msgr2=0x7f88b413cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88b40b29e0 0x7f88b413cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 -- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b40cfd90 msgr2=0x7f88b414dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:02.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b40cfd90 0x7f88b414dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 -- 172.21.15.110:0/3470325882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b413f4b0 con 0x7f88b4145f30 2024-09-10T10:07:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b3e00640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b40cfd90 0x7f88b414dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:07:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b9800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b0830 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00b970 tx=0x7f88ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b0c00640 1 -- 172.21.15.110:0/3470325882 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac019070 con 0x7f88b4145f30 2024-09-10T10:07:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b0c00640 1 -- 172.21.15.110:0/3470325882 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ac00cb40 con 0x7f88b4145f30 2024-09-10T10:07:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b413bbb0 con 0x7f88b4145f30 2024-09-10T10:07:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88b0c00640 1 -- 172.21.15.110:0/3470325882 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac014440 con 0x7f88b4145f30 2024-09-10T10:07:02.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b413c450 con 0x7f88b4145f30 2024-09-10T10:07:02.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.980+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b40b24a0 con 0x7f88b4145f30 2024-09-10T10:07:02.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.984+0000 7f88b0c00640 1 -- 172.21.15.110:0/3470325882 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7f88ac01c070 con 0x7f88b4145f30 2024-09-10T10:07:02.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.984+0000 7f88b0c00640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8894068400 0x7f889406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:02.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.984+0000 7f88b0c00640 1 -- 172.21.15.110:0/3470325882 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7f88ac0844c0 con 0x7f88b4145f30 2024-09-10T10:07:02.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.984+0000 7f88b8e00640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8894068400 0x7f889406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:02.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.984+0000 7f88b8e00640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8894068400 0x7f889406a8c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f88a80ad000 tx=0x7f88a80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:02.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:02.988+0000 7f88b0c00640 1 -- 172.21.15.110:0/3470325882 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f88ac056800 con 0x7f88b4145f30 2024-09-10T10:07:03.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.144+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f88b4140830 con 0x7f88b4145f30 2024-09-10T10:07:03.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.148+0000 7f88b0c00640 1 -- 172.21.15.110:0/3470325882 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) ==== 123+0+0 (secure 0 0 0) 0x7f88ac056100 con 0x7f88b4145f30 2024-09-10T10:07:03.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88b0c00640 1 -- 172.21.15.110:0/3470325882 <== mon.0 v2:172.21.15.110:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f88ac08c080 con 0x7f88b4145f30 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8894068400 msgr2=0x7f889406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8894068400 0x7f889406a8c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f88a80ad000 tx=0x7f88a80a6000 comp rx=0 tx=0).stop 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 msgr2=0x7f88b40b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b0830 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00b970 tx=0x7f88ac00be30 comp rx=0 tx=0).stop 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 shutdown_connections 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7f8894068400 0x7f889406a8c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88b4145f30 0x7f88b40b0830 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88b40cfd90 0x7f88b414dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 --2- 172.21.15.110:0/3470325882 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7f88b40b29e0 0x7f88b413cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 >> 172.21.15.110:0/3470325882 conn(0x7f88b4062e40 msgr2=0x7f88b40721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:03.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 shutdown_connections 2024-09-10T10:07:03.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.152+0000 7f88bb800640 1 -- 172.21.15.110:0/3470325882 wait complete. 2024-09-10T10:07:03.173 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:07:03.173 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:07:03.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe63f600640 1 Processor -- start 2024-09-10T10:07:03.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe63f600640 1 -- start start 2024-09-10T10:07:03.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe63f600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe6380cd510 0x7fe6380cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:03.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe63f600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:03.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe63f600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe638072150 con 0x7fe6380cfe30 2024-09-10T10:07:03.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe63f600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe63815d9a0 con 0x7fe6380cd510 2024-09-10T10:07:03.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe63f600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe63806ea60 con 0x7fe63806e090 2024-09-10T10:07:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe63d600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.308+0000 7fe637e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe6380cd510 0x7fe6380cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe63cc00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fe6380cfe30 0x7fe638071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.110:42250/0 (socket says 172.21.15.110:42250) 2024-09-10T10:07:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe63cc00640 1 -- 172.21.15.110:0/187701863 learned_addr learned my addr 172.21.15.110:0/187701863 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-10T10:07:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe637e00640 1 -- 172.21.15.110:0/187701863 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 msgr2=0x7fe63806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe637e00640 1 --2- 172.21.15.110:0/187701863 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63806e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe637e00640 1 -- 172.21.15.110:0/187701863 >> v1:172.21.15.145:6789/0 conn(0x7fe6380cfe30 legacy=0x7fe638071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:03.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe637e00640 1 -- 172.21.15.110:0/187701863 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63806d2d0 con 0x7fe6380cd510 2024-09-10T10:07:03.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe637e00640 1 --2- 172.21.15.110:0/187701863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cd510 0x7fe6380cd9a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe6240097c0 tx=0x7fe62401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=555fad7ea371403c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:03.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe637400640 1 -- 172.21.15.110:0/187701863 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6240b0070 con 0x7fe6380cd510 2024-09-10T10:07:03.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe637400640 1 -- 172.21.15.110:0/187701863 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe6240ab600 con 0x7fe6380cd510 2024-09-10T10:07:03.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.312+0000 7fe637400640 1 -- 172.21.15.110:0/187701863 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6240ab9f0 con 0x7fe6380cd510 2024-09-10T10:07:03.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- 172.21.15.110:0/187701863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cd510 msgr2=0x7fe6380cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 --2- 172.21.15.110:0/187701863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cd510 0x7fe6380cd9a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe6240097c0 tx=0x7fe62401c410 comp rx=0 tx=0).stop 2024-09-10T10:07:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- 172.21.15.110:0/187701863 shutdown_connections 2024-09-10T10:07:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 --2- 172.21.15.110:0/187701863 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63806e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 --2- 172.21.15.110:0/187701863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cd510 0x7fe6380cd9a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- 172.21.15.110:0/187701863 >> 172.21.15.110:0/187701863 conn(0x7fe63806bc50 msgr2=0x7fe63806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- 172.21.15.110:0/187701863 shutdown_connections 2024-09-10T10:07:03.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- 172.21.15.110:0/187701863 wait complete. 2024-09-10T10:07:03.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 Processor -- start 2024-09-10T10:07:03.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- start start 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63814ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6380cd510 0x7fe6381491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cfe30 0x7fe6380b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe638169180 con 0x7fe6380cd510 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe63815d6c0 con 0x7fe6380cfe30 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63f600640 1 -- --> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] -- mon_getmap magic: 0 -- 0x7fe638169300 con 0x7fe63806e090 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe637e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6380cd510 0x7fe6381491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63cc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63814ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cfe30 0x7fe6380b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:03.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63cc00640 1 --2- >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63814ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3301/0 says I am v2:172.21.15.110:60722/0 (socket says 172.21.15.110:60722) 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63cc00640 1 -- 172.21.15.110:0/4108388910 learned_addr learned my addr 172.21.15.110:0/4108388910 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe637e00640 1 -- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 msgr2=0x7fe63814ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe637e00640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63814ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe637e00640 1 -- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cfe30 msgr2=0x7fe6380b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe637e00640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cfe30 0x7fe6380b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe637e00640 1 -- 172.21.15.110:0/4108388910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380cc620 con 0x7fe6380cd510 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63cc00640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63814ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.316+0000 7fe63d600640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cfe30 0x7fe6380b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:07:03.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe637e00640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6380cd510 0x7fe6381491a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fe6240ab000 tx=0x7fe6240191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:03.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe634c00640 1 -- 172.21.15.110:0/4108388910 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6240b8070 con 0x7fe6380cd510 2024-09-10T10:07:03.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe634c00640 1 -- 172.21.15.110:0/4108388910 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe6240b4410 con 0x7fe6380cd510 2024-09-10T10:07:03.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63814db40 con 0x7fe6380cd510 2024-09-10T10:07:03.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe634c00640 1 -- 172.21.15.110:0/4108388910 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6240b0040 con 0x7fe6380cd510 2024-09-10T10:07:03.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63814e100 con 0x7fe6380cd510 2024-09-10T10:07:03.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63806e520 con 0x7fe6380cd510 2024-09-10T10:07:03.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe634c00640 1 -- 172.21.15.110:0/4108388910 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 88465+0+0 (secure 0 0 0) 0x7fe6240bb070 con 0x7fe6380cd510 2024-09-10T10:07:03.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe634c00640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe618068400 0x7fe61806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:03.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe634c00640 1 -- 172.21.15.110:0/4108388910 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 4999+0+0 (secure 0 0 0) 0x7fe62411e770 con 0x7fe6380cd510 2024-09-10T10:07:03.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.320+0000 7fe63cc00640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe618068400 0x7fe61806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:03.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.324+0000 7fe63cc00640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe618068400 0x7fe61806a8c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe628005c10 tx=0x7fe628009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:03.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.328+0000 7fe634c00640 1 -- 172.21.15.110:0/4108388910 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe624016090 con 0x7fe6380cd510 2024-09-10T10:07:03.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.516+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe638071b70 con 0x7fe6380cd510 2024-09-10T10:07:03.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.516+0000 7fe634c00640 1 -- 172.21.15.110:0/4108388910 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+3925 (secure 0 0 0) 0x7fe6240f03d0 con 0x7fe6380cd510 2024-09-10T10:07:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-10T10:07:03.519 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":118,"btime":"2024-09-10T10:06:30:179660+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6834/1276880772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1276880772},{"type":"v1","addr":"172.21.15.145:6834","nonce":1276880772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8446,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2050499076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6833","nonce":2050499076},{"type":"v1","addr":"172.21.15.145:6835","nonce":2050499076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":118},{"gid":8874,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6834/2843226954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2843226954},{"type":"v1","addr":"172.21.15.110:6834","nonce":2843226954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8877,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/878298806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6833","nonce":878298806},{"type":"v1","addr":"172.21.15.110:6835","nonce":878298806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[]} 2024-09-10T10:07:03.520 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 118 2024-09-10T10:07:03.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe618068400 msgr2=0x7fe61806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:03.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe618068400 0x7fe61806a8c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe628005c10 tx=0x7fe628009040 comp rx=0 tx=0).stop 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6380cd510 msgr2=0x7fe6381491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6380cd510 0x7fe6381491a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fe6240ab000 tx=0x7fe6240191b0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 shutdown_connections 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:6836/1978165248,v1:172.21.15.145:6837/1978165248] conn(0x7fe618068400 0x7fe61806a8c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6380cfe30 0x7fe6380b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6380cd510 0x7fe6381491a0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 --2- 172.21.15.110:0/4108388910 >> [v2:172.21.15.145:3301/0,v1:172.21.15.145:6790/0] conn(0x7fe63806e090 0x7fe63814ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 >> 172.21.15.110:0/4108388910 conn(0x7fe63806bc50 msgr2=0x7fe6380b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 shutdown_connections 2024-09-10T10:07:03.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-10T10:07:03.520+0000 7fe63f600640 1 -- 172.21.15.110:0/4108388910 wait complete. 2024-09-10T10:07:03.540 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T10:07:03.541 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T10:07:03.541 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:07:03.541 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:03.553 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:07:03.553 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:07:03.553 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:03.607 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:07:03.607 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:07:03.607 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:03.621 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:07:03.622 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:07:03.622 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:03.681 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:07:03.682 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T10:07:03.682 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T10:07:03.682 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:03.775 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T10:07:03.775 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T10:07:03.775 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:03.857 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T10:07:03.858 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T10:07:03.858 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:03.956 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T10:07:03.957 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T10:07:03.957 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:10.021 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-10T10:07:10.059 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T10:07:10.059 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T10:07:10.059 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:10.138 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T10:07:10.138 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T10:07:10.138 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:10.237 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T10:07:10.238 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T10:07:10.238 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:15.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-10T10:07:15.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-10T10:07:16.339 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T10:07:16.339 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T10:07:16.339 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:16.435 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T10:07:16.435 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T10:07:16.435 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T10:07:16.435 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:16.497 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T10:07:16.497 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T10:07:16.497 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:16.561 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T10:07:16.561 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T10:07:16.561 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T10:07:16.561 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:16.581 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T10:07:16.581 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T10:07:16.581 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:16.599 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T10:07:16.600 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T10:07:16.600 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:07:16.664 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T10:07:16.664 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T10:07:16.665 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-10T10:07:16.703 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:07:16.703 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T10:07:16.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:07:16.953 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T10:07:17.132 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:07:17.132 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T10:07:17.347 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:07:17.347 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T10:07:17.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:07:17.531 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T10:07:17.735 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:07:17.735 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T10:07:17.888 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:07:17.888 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T10:07:18.046 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:07:18.046 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T10:07:18.206 INFO:tasks.ceph:Archiving mon data... 2024-09-10T10:07:18.207 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/data/mon.a.tgz 2024-09-10T10:07:18.208 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-10T10:07:18.217 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.unxaOk8cL4 2024-09-10T10:07:18.217 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.unxaOk8cL4 2024-09-10T10:07:18.433 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.unxaOk8cL4 2024-09-10T10:07:18.511 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.unxaOk8cL4 is 923KB 2024-09-10T10:07:18.582 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.unxaOk8cL4 2024-09-10T10:07:18.591 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/data/mon.b.tgz 2024-09-10T10:07:18.591 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-09-10T10:07:18.599 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.bleSV4fq2f 2024-09-10T10:07:18.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.bleSV4fq2f 2024-09-10T10:07:18.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.bleSV4fq2f 2024-09-10T10:07:18.887 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.bleSV4fq2f is 923KB 2024-09-10T10:07:18.954 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.bleSV4fq2f 2024-09-10T10:07:18.962 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/data/mon.c.tgz 2024-09-10T10:07:18.962 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-09-10T10:07:19.010 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.wAxijwBusJ 2024-09-10T10:07:19.010 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.wAxijwBusJ 2024-09-10T10:07:19.216 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.wAxijwBusJ 2024-09-10T10:07:19.291 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.wAxijwBusJ is 923KB 2024-09-10T10:07:19.358 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.wAxijwBusJ 2024-09-10T10:07:19.366 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T10:07:19.366 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T10:07:19.369 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T10:07:19.436 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T10:07:19.474 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T10:07:19.475 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T10:07:19.476 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/remote/smithi110/crash 2024-09-10T10:07:19.477 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T10:07:19.499 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/remote/smithi145/crash 2024-09-10T10:07:19.500 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T10:07:19.532 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T10:07:19.533 DEBUG:teuthology.orchestra.run.smithi110:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:07:19.550 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:07:19.620 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T10:07:19.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T10:07:19.681 INFO:tasks.ceph:Compressing logs... 2024-09-10T10:07:19.681 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:07:19.690 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:07:19.722 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45128.log 2024-09-10T10:07:19.723 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35584.log 2024-09-10T10:07:19.724 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44537.log 2024-09-10T10:07:19.724 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45128.log.gz 2024-09-10T10:07:19.724 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45503.log 2024-09-10T10:07:19.724 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35584.log.gz 2024-09-10T10:07:19.726 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40103.log 2024-09-10T10:07:19.726 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44537.log.gz 2024-09-10T10:07:19.726 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41582.log 2024-09-10T10:07:19.727 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45503.log.gz 2024-09-10T10:07:19.727 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39971.log 2024-09-10T10:07:19.727 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40103.log.gz 2024-09-10T10:07:19.728 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43688.log 2024-09-10T10:07:19.728 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41582.log.gz 2024-09-10T10:07:19.728 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45633.log 2024-09-10T10:07:19.729 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39971.log.gz 2024-09-10T10:07:19.729 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.43688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43688.log.gz 2024-09-10T10:07:19.729 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34760.log 2024-09-10T10:07:19.730 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50946.log 2024-09-10T10:07:19.730 INFO:teuthology.orchestra.run.smithi110.stderr: 91.2%/var/log/ceph/ceph-client.admin.34760.log: -- replaced with /var/log/ceph/ceph-client.admin.45633.log.gz 2024-09-10T10:07:19.731 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44991.log 2024-09-10T10:07:19.731 INFO:teuthology.orchestra.run.smithi110.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34760.log.gz 2024-09-10T10:07:19.732 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38658.log 2024-09-10T10:07:19.732 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50946.log.gz 2024-09-10T10:07:19.733 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47046.log 2024-09-10T10:07:19.733 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44991.log.gz 2024-09-10T10:07:19.733 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49717.log 2024-09-10T10:07:19.733 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38658.log.gz 2024-09-10T10:07:19.734 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40899.log 2024-09-10T10:07:19.734 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47046.log.gz 2024-09-10T10:07:19.735 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41660.log 2024-09-10T10:07:19.735 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49717.log.gz 2024-09-10T10:07:19.736 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48714.log 2024-09-10T10:07:19.736 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40899.log.gz 2024-09-10T10:07:19.737 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41899.log 2024-09-10T10:07:19.737 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41660.log.gz 2024-09-10T10:07:19.738 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-09-10T10:07:19.738 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48714.log.gz 2024-09-10T10:07:19.738 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34323.log 2024-09-10T10:07:19.739 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41899.log.gz 2024-09-10T10:07:19.739 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33595.log 2024-09-10T10:07:19.739 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-09-10T10:07:19.740 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51102.log 2024-09-10T10:07:19.740 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34323.log.gz 2024-09-10T10:07:19.741 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2024-09-10T10:07:19.741 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33595.log.gz 2024-09-10T10:07:19.742 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45277.log 2024-09-10T10:07:19.742 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2024-09-10T10:07:19.742 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41452.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T10:07:19.743 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2024-09-10T10:07:19.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46050.log 2024-09-10T10:07:19.744 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45277.log.gz 2024-09-10T10:07:19.744 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48007.log 2024-09-10T10:07:19.744 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48511.log 2024-09-10T10:07:19.745 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46050.log.gz 2024-09-10T10:07:19.745 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39312.log 2024-09-10T10:07:19.745 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48007.log.gz 2024-09-10T10:07:19.747 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52817.log 2024-09-10T10:07:19.747 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48511.log.gz 2024-09-10T10:07:19.747 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52895.log 2024-09-10T10:07:19.747 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39312.log.gz 2024-09-10T10:07:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38431.log 2024-09-10T10:07:19.748 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52817.log.gz 2024-09-10T10:07:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46848.log 2024-09-10T10:07:19.749 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52895.log.gz 2024-09-10T10:07:19.750 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39120.log 2024-09-10T10:07:19.750 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38431.log.gz 2024-09-10T10:07:19.750 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48479.log 2024-09-10T10:07:19.751 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46848.log.gz 2024-09-10T10:07:19.751 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42896.log 2024-09-10T10:07:19.752 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35938.log 2024-09-10T10:07:19.752 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39120.log.gz 2024-09-10T10:07:19.752 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34409.log 2024-09-10T10:07:19.752 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48479.log.gz 2024-09-10T10:07:19.754 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35281.log 2024-09-10T10:07:19.754 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.35938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35058.log 2024-09-10T10:07:19.754 INFO:teuthology.orchestra.run.smithi145.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35938.log.gz 2024-09-10T10:07:19.754 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.35281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29142.log 2024-09-10T10:07:19.754 INFO:teuthology.orchestra.run.smithi145.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35281.log.gz 2024-09-10T10:07:19.755 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47494.log 2024-09-10T10:07:19.755 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42896.log.gz 2024-09-10T10:07:19.755 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50086.log 2024-09-10T10:07:19.755 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34409.log.gz 2024-09-10T10:07:19.755 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48948.log 2024-09-10T10:07:19.755 INFO:teuthology.orchestra.run.smithi110.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47494.log.gz 2024-09-10T10:07:19.758 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50972.log 2024-09-10T10:07:19.758 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50086.log.gz 2024-09-10T10:07:19.758 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48688.log 2024-09-10T10:07:19.758 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48948.log.gz 2024-09-10T10:07:19.758 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35612.log 2024-09-10T10:07:19.759 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50972.log.gz 2024-09-10T10:07:19.759 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45069.log 2024-09-10T10:07:19.759 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48688.log.gz 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.35058.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35058.log.gz 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.29142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35766.log 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.29142.log.gz 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.35766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36219.log 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35766.log.gz 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35400.log 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.36219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29046.log 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.36219.log.gz 2024-09-10T10:07:19.760 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.35400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35400.log.gz 2024-09-10T10:07:19.761 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29022.log 2024-09-10T10:07:19.761 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.29046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35655.log 2024-09-10T10:07:19.764 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29046.log.gz 2024-09-10T10:07:19.764 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.29022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29070.log 2024-09-10T10:07:19.764 INFO:teuthology.orchestra.run.smithi145.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29022.log.gz 2024-09-10T10:07:19.764 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.35655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35655.log.gz 2024-09-10T10:07:19.764 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T10:07:19.764 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.29070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36574.log 2024-09-10T10:07:19.764 INFO:teuthology.orchestra.run.smithi145.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29070.log.gz 2024-09-10T10:07:19.764 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29094.log 2024-09-10T10:07:19.767 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46876.log 2024-09-10T10:07:19.767 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35612.log.gz 2024-09-10T10:07:19.767 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38088.log 2024-09-10T10:07:19.767 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45069.log.gz 2024-09-10T10:07:19.767 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53105.log 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46876.log.gz 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33569.log 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38088.log.gz 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.53105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53129.log 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53105.log.gz 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47176.log 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33569.log.gz 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.53129.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53129.log.gz 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44866.log 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.0.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47176.log.gz 2024-09-10T10:07:19.768 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40784.log 2024-09-10T10:07:19.769 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50526.log 2024-09-10T10:07:19.769 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44866.log.gz 2024-09-10T10:07:19.770 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.36574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.36574.log.gz 2024-09-10T10:07:19.770 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T10:07:19.770 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28974.log 2024-09-10T10:07:19.770 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.29094.log: /var/log/ceph/ceph-mgr.x.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29094.log.gz 2024-09-10T10:07:19.770 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T10:07:19.770 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.28974.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T10:07:19.771 INFO:teuthology.orchestra.run.smithi145.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28974.log.gz 2024-09-10T10:07:19.771 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29118.log 2024-09-10T10:07:19.771 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36393.log 2024-09-10T10:07:19.772 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.29118.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29118.log.gz 2024-09-10T10:07:19.772 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.36393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.36393.log.gz 2024-09-10T10:07:19.774 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40524.log 2024-09-10T10:07:19.774 INFO:teuthology.orchestra.run.smithi110.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40784.log.gz 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48114.log 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50526.log.gz 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45306.log 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40524.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40524.log.gz 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44031.log 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48114.log.gz 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35530.log 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45306.log.gz 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46024.log 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44031.log.gz 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35530.log: gzip -5 88.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.35530.log.gz -- 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr: /var/log/ceph/ceph-client.admin.50164.log 2024-09-10T10:07:19.775 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37170.log 2024-09-10T10:07:19.778 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T10:07:19.778 INFO:teuthology.orchestra.run.smithi145.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T10:07:19.778 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28998.log 2024-09-10T10:07:19.778 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T10:07:19.778 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.28998.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28998.log.gz 2024-09-10T10:07:19.778 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36081.log 2024-09-10T10:07:19.779 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46024.log.gz 2024-09-10T10:07:19.779 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33771.log 2024-09-10T10:07:19.779 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50164.log.gz 2024-09-10T10:07:19.779 INFO:teuthology.orchestra.run.smithi110.stderr:gzip/var/log/ceph/ceph-client.admin.37170.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49141.log 2024-09-10T10:07:19.779 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37170.log.gz 2024-09-10T10:07:19.779 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33413.log 2024-09-10T10:07:19.779 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33771.log.gz 2024-09-10T10:07:19.780 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49141.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49141.log.gz 2024-09-10T10:07:19.780 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35860.log 2024-09-10T10:07:19.780 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 2024-09-10T10:07:19.780 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.1.36081.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.36081.log.gz 2024-09-10T10:07:19.780 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T10:07:19.781 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35860.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33413.log.gz 2024-09-10T10:07:19.781 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35860.log.gz 2024-09-10T10:07:19.781 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47694.log 2024-09-10T10:07:19.781 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37382.log 2024-09-10T10:07:19.781 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2024-09-10T10:07:19.782 INFO:teuthology.orchestra.run.smithi145.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T10:07:19.783 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35729.log 2024-09-10T10:07:19.783 INFO:teuthology.orchestra.run.smithi110.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47694.log.gz 2024-09-10T10:07:19.784 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T10:07:19.784 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37382.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.35198.log 2024-09-10T10:07:19.784 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37382.log.gz 2024-09-10T10:07:19.784 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39198.log 2024-09-10T10:07:19.785 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35729.log.gz 2024-09-10T10:07:19.785 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34786.log 2024-09-10T10:07:19.785 INFO:teuthology.orchestra.run.smithi110.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35198.log.gz 2024-09-10T10:07:19.786 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33981.log 2024-09-10T10:07:19.786 INFO:teuthology.orchestra.run.smithi110.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39198.log.gz 2024-09-10T10:07:19.787 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33725.log 2024-09-10T10:07:19.787 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34786.log.gz 2024-09-10T10:07:19.788 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T10:07:19.788 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34923.log 2024-09-10T10:07:19.788 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.3.34923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.34923.log.gz 2024-09-10T10:07:19.788 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34821.log 2024-09-10T10:07:19.789 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41005.log 2024-09-10T10:07:19.789 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33981.log.gz 2024-09-10T10:07:19.789 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48140.log 2024-09-10T10:07:19.789 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33725.log.gz 2024-09-10T10:07:19.790 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50668.log 2024-09-10T10:07:19.790 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41005.log.gz 2024-09-10T10:07:19.791 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50346.log 2024-09-10T10:07:19.791 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48140.log.gz 2024-09-10T10:07:19.791 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51758.log 2024-09-10T10:07:19.792 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50668.log.gz 2024-09-10T10:07:19.792 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48394.log 2024-09-10T10:07:19.793 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.1.34821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34821.log.gz 2024-09-10T10:07:19.793 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50346.log.gz 2024-09-10T10:07:19.794 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52423.log 2024-09-10T10:07:19.794 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51758.log.gz 2024-09-10T10:07:19.795 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52157.log 2024-09-10T10:07:19.795 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48394.log.gz 2024-09-10T10:07:19.795 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35164.log 2024-09-10T10:07:19.796 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37600.log 2024-09-10T10:07:19.796 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52423.log.gz 2024-09-10T10:07:19.796 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34245.log 2024-09-10T10:07:19.797 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52157.log.gz 2024-09-10T10:07:19.797 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48193.log 2024-09-10T10:07:19.797 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37600.log.gz 2024-09-10T10:07:19.798 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37109.log 2024-09-10T10:07:19.798 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34245.log.gz 2024-09-10T10:07:19.799 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48193.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48193.log.gz -5 2024-09-10T10:07:19.799 INFO:teuthology.orchestra.run.smithi110.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52267.log 2024-09-10T10:07:19.800 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50476.log 2024-09-10T10:07:19.801 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37109.log.gz 2024-09-10T10:07:19.801 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.1.35164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35164.log.gz 2024-09-10T10:07:19.802 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41424.log 2024-09-10T10:07:19.802 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52267.log.gz 2024-09-10T10:07:19.802 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2024-09-10T10:07:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50476.log: /var/log/ceph/ceph-client.admin.41424.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50476.log.gz 2024-09-10T10:07:19.803 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40680.log 2024-09-10T10:07:19.803 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41424.log.gz 2024-09-10T10:07:19.804 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49559.log 2024-09-10T10:07:19.805 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2024-09-10T10:07:19.805 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38554.log 2024-09-10T10:07:19.805 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40680.log.gz 2024-09-10T10:07:19.806 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45251.log 2024-09-10T10:07:19.806 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49559.log.gz 2024-09-10T10:07:19.807 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33987.log 2024-09-10T10:07:19.807 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38554.log.gz 2024-09-10T10:07:19.808 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42825.log 2024-09-10T10:07:19.808 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45251.log.gz 2024-09-10T10:07:19.809 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45421.log 2024-09-10T10:07:19.809 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33987.log.gz 2024-09-10T10:07:19.809 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45225.log 2024-09-10T10:07:19.810 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42825.log.gz 2024-09-10T10:07:19.811 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45683.log 2024-09-10T10:07:19.811 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45421.log.gz 2024-09-10T10:07:19.811 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37226.log 2024-09-10T10:07:19.812 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45225.log.gz 2024-09-10T10:07:19.812 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50029.log 2024-09-10T10:07:19.813 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45683.log.gz 2024-09-10T10:07:19.813 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40550.log 2024-09-10T10:07:19.814 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37226.log.gz 2024-09-10T10:07:19.814 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36740.log 2024-09-10T10:07:19.814 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50029.log.gz 2024-09-10T10:07:19.815 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50242.log 2024-09-10T10:07:19.815 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40550.log.gz 2024-09-10T10:07:19.816 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-09-10T10:07:19.816 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36740.log.gz 2024-09-10T10:07:19.817 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34219.log 2024-09-10T10:07:19.817 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50242.log.gz 2024-09-10T10:07:19.818 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50003.log 2024-09-10T10:07:19.818 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-09-10T10:07:19.819 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44657.log 2024-09-10T10:07:19.819 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34219.log.gz 2024-09-10T10:07:19.820 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49089.log 2024-09-10T10:07:19.820 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50003.log.gz 2024-09-10T10:07:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41338.log 2024-09-10T10:07:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44657.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44657.log.gz 2024-09-10T10:07:19.822 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41634.log 2024-09-10T10:07:19.822 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49089.log.gz 2024-09-10T10:07:19.823 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38960.log 2024-09-10T10:07:19.823 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41634.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41338.log.gz 2024-09-10T10:07:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43352.log 2024-09-10T10:07:19.824 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41634.log.gz 2024-09-10T10:07:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49199.log 2024-09-10T10:07:19.825 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38960.log.gz 2024-09-10T10:07:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35347.log 2024-09-10T10:07:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49199.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43352.log.gz 2024-09-10T10:07:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46168.log 2024-09-10T10:07:19.826 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49199.log.gz 2024-09-10T10:07:19.827 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50894.log 2024-09-10T10:07:19.827 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35347.log.gz 2024-09-10T10:07:19.828 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43981.log 2024-09-10T10:07:19.828 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46168.log.gz 2024-09-10T10:07:19.829 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36066.log 2024-09-10T10:07:19.829 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50894.log.gz 2024-09-10T10:07:19.830 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43981.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43981.log.gz 2024-09-10T10:07:19.830 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-09-10T10:07:19.831 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50573.log 2024-09-10T10:07:19.831 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36066.log.gz 2024-09-10T10:07:19.832 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45940.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-09-10T10:07:19.832 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42948.log 2024-09-10T10:07:19.833 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35172.log 2024-09-10T10:07:19.834 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.50573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50573.log.gz 2024-09-10T10:07:19.834 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43641.log 2024-09-10T10:07:19.834 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42948.log.gz 2024-09-10T10:07:19.834 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42669.log 2024-09-10T10:07:19.835 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35172.log.gz 2024-09-10T10:07:19.835 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45773.log 2024-09-10T10:07:19.836 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43641.log.gz 2024-09-10T10:07:19.836 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39691.log 2024-09-10T10:07:19.837 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42669.log.gz 2024-09-10T10:07:19.837 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34960.log 2024-09-10T10:07:19.837 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45773.log.gz 2024-09-10T10:07:19.838 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39691.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39691.log.gz 2024-09-10T10:07:19.838 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46453.log 2024-09-10T10:07:19.839 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34960.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T10:07:19.839 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34960.log.gz 2024-09-10T10:07:19.840 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44788.log 2024-09-10T10:07:19.840 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46453.log.gz 2024-09-10T10:07:19.841 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-09-10T10:07:19.842 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33227.log 2024-09-10T10:07:19.842 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44788.log.gz 2024-09-10T10:07:19.843 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46819.log 2024-09-10T10:07:19.843 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-09-10T10:07:19.844 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33491.log 2024-09-10T10:07:19.844 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33227.log.gz 2024-09-10T10:07:19.845 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38986.log 2024-09-10T10:07:19.846 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46819.log.gz 2024-09-10T10:07:19.846 INFO:teuthology.orchestra.run.smithi145.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T10:07:19.847 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51952.log 2024-09-10T10:07:19.847 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33491.log.gz 2024-09-10T10:07:19.847 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38986.log: /var/log/ceph/ceph-client.admin.35016.log 2024-09-10T10:07:19.848 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5%/var/log/ceph/ceph-client.admin.51952.log: -- replaced with /var/log/ceph/ceph-client.admin.38986.log.gz 2024-09-10T10:07:19.848 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46767.log 2024-09-10T10:07:19.848 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51952.log.gz 2024-09-10T10:07:19.849 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44590.log 2024-09-10T10:07:19.849 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35016.log.gz 2024-09-10T10:07:19.850 INFO:teuthology.orchestra.run.smithi110.stderr:gzip/var/log/ceph/ceph-client.admin.46767.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34931.log 2024-09-10T10:07:19.850 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46767.log.gz 2024-09-10T10:07:19.851 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49691.log 2024-09-10T10:07:19.851 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44590.log.gz 2024-09-10T10:07:19.852 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42747.log 2024-09-10T10:07:19.852 INFO:teuthology.orchestra.run.smithi110.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34931.log.gz 2024-09-10T10:07:19.853 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52345.log 2024-09-10T10:07:19.854 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49691.log: /var/log/ceph/ceph-client.admin.42747.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49691.log.gz 2024-09-10T10:07:19.854 INFO:teuthology.orchestra.run.smithi110.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T10:07:19.855 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49665.log 2024-09-10T10:07:19.855 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42747.log.gz 2024-09-10T10:07:19.855 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36565.log 2024-09-10T10:07:19.855 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52345.log.gz 2024-09-10T10:07:19.856 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49309.log 2024-09-10T10:07:19.856 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49665.log.gz 2024-09-10T10:07:19.857 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38248.log 2024-09-10T10:07:19.857 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36565.log.gz 2024-09-10T10:07:19.858 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35886.log 2024-09-10T10:07:19.858 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2024-09-10T10:07:19.859 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40951.log 2024-09-10T10:07:19.859 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38248.log.gz 2024-09-10T10:07:19.860 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45914.log 2024-09-10T10:07:19.860 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35886.log.gz 2024-09-10T10:07:19.861 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50294.log 2024-09-10T10:07:19.861 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40951.log.gz 2024-09-10T10:07:19.862 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45451.log 2024-09-10T10:07:19.862 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45914.log.gz 2024-09-10T10:07:19.862 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-09-10T10:07:19.863 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50294.log.gz 2024-09-10T10:07:19.863 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39660.log 2024-09-10T10:07:19.863 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45451.log.gz 2024-09-10T10:07:19.864 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52124.log 2024-09-10T10:07:19.864 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-09-10T10:07:19.865 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36016.log 2024-09-10T10:07:19.865 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39660.log.gz 2024-09-10T10:07:19.866 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40420.log 2024-09-10T10:07:19.866 INFO:teuthology.orchestra.run.smithi110.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52124.log.gz 2024-09-10T10:07:19.867 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-09-10T10:07:19.867 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40420.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36016.log.gz 2024-09-10T10:07:19.868 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44814.log 2024-09-10T10:07:19.868 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40420.log.gz 2024-09-10T10:07:19.869 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2024-09-10T10:07:19.869 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-09-10T10:07:19.870 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38180.log 2024-09-10T10:07:19.870 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44814.log.gz 2024-09-10T10:07:19.871 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38326.log 2024-09-10T10:07:19.871 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49428.log: /var/log/ceph/ceph-client.admin.38180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-09-10T10:07:19.872 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz 2024-09-10T10:07:19.872 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38180.log.gz 2024-09-10T10:07:19.873 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51550.log 2024-09-10T10:07:19.873 INFO:teuthology.orchestra.run.smithi110.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38326.log.gz 2024-09-10T10:07:19.873 INFO:teuthology.orchestra.run.smithi110.stderr:gzip/var/log/ceph/ceph-client.admin.46132.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40602.log 2024-09-10T10:07:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51550.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-09-10T10:07:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39068.log 2024-09-10T10:07:19.875 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51550.log.gz 2024-09-10T10:07:19.875 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40602.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T10:07:19.875 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40602.log.gz 2024-09-10T10:07:19.876 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44710.log 2024-09-10T10:07:19.876 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39068.log.gz 2024-09-10T10:07:19.877 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52293.log 2024-09-10T10:07:19.878 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47744.log 2024-09-10T10:07:19.878 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44710.log.gz 2024-09-10T10:07:19.879 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50450.log 2024-09-10T10:07:19.879 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52293.log.gz 2024-09-10T10:07:19.880 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35426.log 2024-09-10T10:07:19.881 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47744.log: /var/log/ceph/ceph-client.admin.50450.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47744.log.gz 2024-09-10T10:07:19.881 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42974.log 2024-09-10T10:07:19.881 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50450.log.gz 2024-09-10T10:07:19.882 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47944.log 2024-09-10T10:07:19.882 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35426.log.gz 2024-09-10T10:07:19.883 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42974.log: /var/log/ceph/ceph-client.admin.40472.log 2024-09-10T10:07:19.883 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8%/var/log/ceph/ceph-client.admin.47944.log: -- replaced with /var/log/ceph/ceph-client.admin.42974.log.gz 2024-09-10T10:07:19.884 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35699.log 2024-09-10T10:07:19.884 INFO:teuthology.orchestra.run.smithi110.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47944.log.gz 2024-09-10T10:07:19.885 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37844.log 2024-09-10T10:07:19.885 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40472.log.gz 2024-09-10T10:07:19.886 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48766.log 2024-09-10T10:07:19.886 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35699.log: /var/log/ceph/ceph-client.admin.37844.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35699.log.gz 2024-09-10T10:07:19.887 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40628.log 2024-09-10T10:07:19.887 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37844.log.gz 2024-09-10T10:07:19.888 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40732.log 2024-09-10T10:07:19.888 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48766.log.gz 2024-09-10T10:07:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23970.log 2024-09-10T10:07:19.889 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40628.log: /var/log/ceph/ceph-client.admin.40732.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40628.log.gz 2024-09-10T10:07:19.890 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33775.log 2024-09-10T10:07:19.890 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40732.log.gz 2024-09-10T10:07:19.891 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph.tmp-client.admin.23970.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23970.log.gz 2024-09-10T10:07:19.891 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50058.log 2024-09-10T10:07:19.892 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49847.log 2024-09-10T10:07:19.892 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33775.log: /var/log/ceph/ceph-client.admin.50058.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33775.log.gz 2024-09-10T10:07:19.893 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39146.log 2024-09-10T10:07:19.893 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50058.log.gz 2024-09-10T10:07:19.894 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51784.log 2024-09-10T10:07:19.894 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49847.log.gz 2024-09-10T10:07:19.895 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50616.log 2024-09-10T10:07:19.895 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39146.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39146.log.gz 2024-09-10T10:07:19.896 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52791.log 2024-09-10T10:07:19.896 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51784.log.gz 2024-09-10T10:07:19.897 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37658.log 2024-09-10T10:07:19.897 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50616.log.gz 2024-09-10T10:07:19.898 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47226.log 2024-09-10T10:07:19.899 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52791.log.gz 2024-09-10T10:07:19.899 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45334.log 2024-09-10T10:07:19.899 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37658.log.gz 2024-09-10T10:07:19.900 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42799.log 2024-09-10T10:07:19.900 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47226.log.gz 2024-09-10T10:07:19.901 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37144.log 2024-09-10T10:07:19.902 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-09-10T10:07:19.902 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47976.log 2024-09-10T10:07:19.902 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42799.log.gz 2024-09-10T10:07:19.903 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43026.log 2024-09-10T10:07:19.903 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37144.log.gz 2024-09-10T10:07:19.904 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35556.log 2024-09-10T10:07:19.905 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47976.log: /var/log/ceph/ceph-client.admin.43026.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.47976.log.gz 2024-09-10T10:07:19.905 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42364.log 2024-09-10T10:07:19.905 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43026.log.gz 2024-09-10T10:07:19.906 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33120.log 2024-09-10T10:07:19.906 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35556.log.gz 2024-09-10T10:07:19.909 INFO:teuthology.orchestra.run.smithi110.stderr:gzip/var/log/ceph/ceph-client.admin.42364.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45966.log 2024-09-10T10:07:19.909 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42364.log.gz 2024-09-10T10:07:19.909 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34297.log 2024-09-10T10:07:19.909 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33120.log.gz 2024-09-10T10:07:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41177.log 2024-09-10T10:07:19.911 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45966.log.gz 2024-09-10T10:07:19.911 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53025.log 2024-09-10T10:07:19.912 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34297.log.gz 2024-09-10T10:07:19.913 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50867.log 2024-09-10T10:07:19.913 INFO:teuthology.orchestra.run.smithi110.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41177.log.gz 2024-09-10T10:07:19.914 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.53025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37330.log 2024-09-10T10:07:19.914 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53025.log.gz 2024-09-10T10:07:19.917 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52449.log 2024-09-10T10:07:19.917 INFO:teuthology.orchestra.run.smithi110.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50867.log.gz 2024-09-10T10:07:19.918 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34734.log 2024-09-10T10:07:19.918 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37330.log.gz 2024-09-10T10:07:19.918 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40498.log 2024-09-10T10:07:19.918 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52449.log.gz 2024-09-10T10:07:19.918 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41125.log 2024-09-10T10:07:19.918 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34734.log.gz 2024-09-10T10:07:19.923 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2024-09-10T10:07:19.924 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41125.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40498.log.gz 2024-09-10T10:07:19.924 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41125.log.gz 2024-09-10T10:07:19.924 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40129.log 2024-09-10T10:07:19.924 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51258.log 2024-09-10T10:07:19.924 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50920.log: /var/log/ceph/ceph-client.admin.40129.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2024-09-10T10:07:19.924 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40259.log 2024-09-10T10:07:19.925 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40129.log.gz 2024-09-10T10:07:19.925 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36766.log 2024-09-10T10:07:19.925 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51258.log.gz 2024-09-10T10:07:19.925 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49335.log 2024-09-10T10:07:19.925 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40259.log: /var/log/ceph/ceph-client.admin.36766.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40259.log.gz 2024-09-10T10:07:19.930 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36208.log 2024-09-10T10:07:19.930 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36766.log.gz 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36688.log 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49335.log.gz 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48844.log 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36208.log: /var/log/ceph/ceph-client.admin.36688.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36208.log.gz 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35120.log 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36688.log.gz 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48844.log.gz 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43487.log 2024-09-10T10:07:19.931 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35120.log: /var/log/ceph/ceph-client.admin.45825.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35120.log.gz 2024-09-10T10:07:19.932 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29069.log 2024-09-10T10:07:19.932 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-09-10T10:07:19.935 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40576.log 2024-09-10T10:07:19.935 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43487.log.gz 2024-09-10T10:07:19.936 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33774.log 2024-09-10T10:07:19.936 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.29069.log: /var/log/ceph/ceph-client.admin.40576.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29069.log.gz 2024-09-10T10:07:19.936 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2024-09-10T10:07:19.936 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40576.log.gz 2024-09-10T10:07:19.936 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43291.log 2024-09-10T10:07:19.936 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33774.log.gz 2024-09-10T10:07:19.936 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48629.log 2024-09-10T10:07:19.936 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51180.log: /var/log/ceph/ceph-client.admin.43291.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2024-09-10T10:07:19.939 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43434.log 2024-09-10T10:07:19.939 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43291.log.gz 2024-09-10T10:07:19.939 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43929.log 2024-09-10T10:07:19.939 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48629.log.gz 2024-09-10T10:07:19.939 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51602.log 2024-09-10T10:07:19.939 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43434.log: /var/log/ceph/ceph-client.admin.43929.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43434.log.gz 2024-09-10T10:07:19.941 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40706.log 2024-09-10T10:07:19.941 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43929.log.gz 2024-09-10T10:07:19.941 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44762.log 2024-09-10T10:07:19.941 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51602.log.gz 2024-09-10T10:07:19.945 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47544.log 2024-09-10T10:07:19.945 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40706.log: /var/log/ceph/ceph-client.admin.44762.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40706.log.gz 2024-09-10T10:07:19.945 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36818.log 2024-09-10T10:07:19.945 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44762.log.gz 2024-09-10T10:07:19.945 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47020.log 2024-09-10T10:07:19.946 INFO:teuthology.orchestra.run.smithi110.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.47544.log.gz 2024-09-10T10:07:19.946 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32542.log 2024-09-10T10:07:19.946 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36818.log: /var/log/ceph/ceph-client.admin.47020.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36818.log.gz 2024-09-10T10:07:19.951 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41397.log 2024-09-10T10:07:19.951 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47020.log.gz 2024-09-10T10:07:19.951 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.32542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33765.log 2024-09-10T10:07:19.951 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32542.log.gz 2024-09-10T10:07:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49167.log 2024-09-10T10:07:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41397.log: /var/log/ceph/ceph-client.admin.33765.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41397.log.gz 2024-09-10T10:07:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34193.log 2024-09-10T10:07:19.952 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33765.log.gz 2024-09-10T10:07:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33764.log 2024-09-10T10:07:19.952 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49167.log.gz 2024-09-10T10:07:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38120.log 2024-09-10T10:07:19.952 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34193.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34193.log.gz 2024-09-10T10:07:19.953 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35755.log 2024-09-10T10:07:19.955 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33764.log.gz 2024-09-10T10:07:19.956 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48290.log 2024-09-10T10:07:19.956 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38120.log.gz 2024-09-10T10:07:19.956 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45095.log 2024-09-10T10:07:19.956 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35755.log.gz 2024-09-10T10:07:19.956 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33762.log 2024-09-10T10:07:19.956 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48290.log.gz 2024-09-10T10:07:19.956 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39295.log 2024-09-10T10:07:19.956 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45095.log.gz 2024-09-10T10:07:19.957 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52241.log 2024-09-10T10:07:19.957 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33762.log.gz 2024-09-10T10:07:19.957 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.39295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39295.log.gz 2024-09-10T10:07:19.958 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38378.log 2024-09-10T10:07:19.959 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36461.log 2024-09-10T10:07:19.959 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52241.log.gz 2024-09-10T10:07:19.960 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38378.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38378.log.gz 2024-09-10T10:07:19.960 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37874.log 2024-09-10T10:07:19.961 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37252.log 2024-09-10T10:07:19.961 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36461.log: /var/log/ceph/ceph-client.admin.37874.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36461.log.gz 2024-09-10T10:07:19.962 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46481.log 2024-09-10T10:07:19.962 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37874.log.gz 2024-09-10T10:07:19.963 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37458.log 2024-09-10T10:07:19.963 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37252.log.gz 2024-09-10T10:07:19.964 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49397.log 2024-09-10T10:07:19.964 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46481.log: /var/log/ceph/ceph-client.admin.37458.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46481.log.gz 2024-09-10T10:07:19.965 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47819.log 2024-09-10T10:07:19.965 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37458.log.gz 2024-09-10T10:07:19.966 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47594.log 2024-09-10T10:07:19.966 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49397.log.gz 2024-09-10T10:07:19.967 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42773.log 2024-09-10T10:07:19.967 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47819.log: /var/log/ceph/ceph-client.admin.47594.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47819.log.gz 2024-09-10T10:07:19.968 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35647.log 2024-09-10T10:07:19.968 INFO:teuthology.orchestra.run.smithi110.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47594.log.gz 2024-09-10T10:07:19.969 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40162.log 2024-09-10T10:07:19.969 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42773.log.gz 2024-09-10T10:07:19.970 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51654.log 2024-09-10T10:07:19.970 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35647.log: /var/log/ceph/ceph-client.admin.40162.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35647.log.gz 2024-09-10T10:07:19.971 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39548.log 2024-09-10T10:07:19.971 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40162.log.gz 2024-09-10T10:07:19.972 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38274.log 2024-09-10T10:07:19.972 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51654.log.gz 2024-09-10T10:07:19.973 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51732.log 2024-09-10T10:07:19.973 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39548.log.gz 2024-09-10T10:07:19.974 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36714.log 2024-09-10T10:07:19.974 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38274.log.gz 2024-09-10T10:07:19.975 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39718.log 2024-09-10T10:07:19.975 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51732.log.gz 2024-09-10T10:07:19.976 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52635.log 2024-09-10T10:07:19.976 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36714.log: /var/log/ceph/ceph-client.admin.39718.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36714.log.gz 2024-09-10T10:07:19.977 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50836.log 2024-09-10T10:07:19.977 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39718.log.gz 2024-09-10T10:07:19.978 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33361.log 2024-09-10T10:07:19.978 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52635.log.gz 2024-09-10T10:07:19.979 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49115.log 2024-09-10T10:07:19.979 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50836.log: /var/log/ceph/ceph-client.admin.33361.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50836.log.gz 2024-09-10T10:07:19.980 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T10:07:19.980 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33361.log.gz 2024-09-10T10:07:19.981 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39744.log 2024-09-10T10:07:19.981 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49115.log.gz 2024-09-10T10:07:19.982 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51926.log 2024-09-10T10:07:19.983 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46356.log 2024-09-10T10:07:19.983 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39744.log.gz 2024-09-10T10:07:19.984 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38632.log 2024-09-10T10:07:19.984 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51926.log.gz 2024-09-10T10:07:19.985 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43000.log 2024-09-10T10:07:19.986 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46356.log.gz 2024-09-10T10:07:19.986 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42922.log 2024-09-10T10:07:19.986 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38632.log.gz 2024-09-10T10:07:19.987 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52947.log 2024-09-10T10:07:19.987 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43000.log.gz 2024-09-10T10:07:19.988 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38934.log 2024-09-10T10:07:19.989 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42922.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42922.log.gz 2024-09-10T10:07:19.989 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41312.log 2024-09-10T10:07:19.989 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52947.log.gz 2024-09-10T10:07:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36156.log 2024-09-10T10:07:19.990 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38934.log.gz 2024-09-10T10:07:19.991 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2024-09-10T10:07:19.992 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41312.log.gz 2024-09-10T10:07:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42338.log 2024-09-10T10:07:19.993 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36156.log.gz 2024-09-10T10:07:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43851.log 2024-09-10T10:07:19.993 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2024-09-10T10:07:19.994 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39624.log 2024-09-10T10:07:19.995 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42338.log.gz 2024-09-10T10:07:19.995 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42721.log 2024-09-10T10:07:19.996 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43851.log.gz 2024-09-10T10:07:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43565.log 2024-09-10T10:07:19.996 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39624.log.gz 2024-09-10T10:07:19.997 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39900.log 2024-09-10T10:07:19.998 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42721.log.gz 2024-09-10T10:07:19.998 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52503.log 2024-09-10T10:07:19.999 INFO:teuthology.orchestra.run.smithi110.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43565.log.gz 2024-09-10T10:07:20.000 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40446.log 2024-09-10T10:07:20.000 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39900.log.gz 2024-09-10T10:07:20.000 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37574.log 2024-09-10T10:07:20.001 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52503.log.gz 2024-09-10T10:07:20.002 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38871.log 2024-09-10T10:07:20.003 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40446.log.gz 2024-09-10T10:07:20.003 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35990.log 2024-09-10T10:07:20.003 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37574.log.gz 2024-09-10T10:07:20.004 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45888.log 2024-09-10T10:07:20.004 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38871.log.gz 2024-09-10T10:07:20.005 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33621.log 2024-09-10T10:07:20.005 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35990.log.gz 2024-09-10T10:07:20.006 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47290.log 2024-09-10T10:07:20.006 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45888.log.gz 2024-09-10T10:07:20.007 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47769.log 2024-09-10T10:07:20.007 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33621.log.gz 2024-09-10T10:07:20.008 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47290.log: -- /var/log/ceph/ceph-client.admin.43052.log 2024-09-10T10:07:20.009 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47290.log.gz 2024-09-10T10:07:20.009 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35146.log 2024-09-10T10:07:20.009 INFO:teuthology.orchestra.run.smithi110.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47769.log.gz 2024-09-10T10:07:20.010 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42068.log 2024-09-10T10:07:20.011 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43052.log.gz 2024-09-10T10:07:20.011 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48577.log 2024-09-10T10:07:20.011 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35146.log.gz 2024-09-10T10:07:20.012 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37548.log 2024-09-10T10:07:20.013 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42068.log: /var/log/ceph/ceph-client.admin.48577.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42068.log.gz 2024-09-10T10:07:20.013 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37905.log 88.8% 2024-09-10T10:07:20.014 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48577.log.gz 2024-09-10T10:07:20.014 INFO:teuthology.orchestra.run.smithi110.stderr:gzip/var/log/ceph/ceph-client.admin.37548.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44170.log 2024-09-10T10:07:20.015 INFO:teuthology.orchestra.run.smithi110.stderr: 89.5%/var/log/ceph/ceph-client.admin.37905.log: -- replaced with /var/log/ceph/ceph-client.admin.37548.log.gz 2024-09-10T10:07:20.015 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37905.log.gz 2024-09-10T10:07:20.015 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44430.log 2024-09-10T10:07:20.016 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44840.log 2024-09-10T10:07:20.017 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44170.log.gz 2024-09-10T10:07:20.018 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39424.log 2024-09-10T10:07:20.018 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44430.log.gz 2024-09-10T10:07:20.018 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35807.log 2024-09-10T10:07:20.019 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44840.log.gz 2024-09-10T10:07:20.019 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39016.log 2024-09-10T10:07:20.020 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39424.log.gz 2024-09-10T10:07:20.020 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38840.log 2024-09-10T10:07:20.021 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35807.log.gz 2024-09-10T10:07:20.022 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46278.log 2024-09-10T10:07:20.022 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39016.log.gz 2024-09-10T10:07:20.023 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41871.log 2024-09-10T10:07:20.023 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38840.log.gz 2024-09-10T10:07:20.024 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36870.log 2024-09-10T10:07:20.024 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46278.log.gz 2024-09-10T10:07:20.024 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41871.log: /var/log/ceph/ceph-client.admin.41986.log 2024-09-10T10:07:20.025 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36870.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41871.log.gz 2024-09-10T10:07:20.026 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33153.log 2024-09-10T10:07:20.026 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36870.log.gz 2024-09-10T10:07:20.027 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43117.log 2024-09-10T10:07:20.027 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41986.log.gz 2024-09-10T10:07:20.028 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49880.log 2024-09-10T10:07:20.029 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33153.log.gz 2024-09-10T10:07:20.029 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44140.log 2024-09-10T10:07:20.029 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43117.log.gz 2024-09-10T10:07:20.030 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51706.log 2024-09-10T10:07:20.030 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49880.log.gz 2024-09-10T10:07:20.031 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52609.log 2024-09-10T10:07:20.031 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44140.log.gz 2024-09-10T10:07:20.032 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46330.log 2024-09-10T10:07:20.032 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51706.log.gz 2024-09-10T10:07:20.033 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42172.log 2024-09-10T10:07:20.033 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52609.log.gz 2024-09-10T10:07:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37408.log 2024-09-10T10:07:20.034 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46330.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46330.log.gz 2024-09-10T10:07:20.035 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33465.log 2024-09-10T10:07:20.035 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42172.log.gz 2024-09-10T10:07:20.036 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46226.log 2024-09-10T10:07:20.037 INFO:teuthology.orchestra.run.smithi110.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.37408.log.gz 2024-09-10T10:07:20.037 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37278.log 2024-09-10T10:07:20.038 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33465.log.gz 2024-09-10T10:07:20.038 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40194.log 2024-09-10T10:07:20.038 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46226.log.gz 2024-09-10T10:07:20.039 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37278.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T10:07:20.039 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37278.log.gz 2024-09-10T10:07:20.040 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37052.log 2024-09-10T10:07:20.041 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40194.log.gz 2024-09-10T10:07:20.042 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45992.log 2024-09-10T10:07:20.043 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43825.log 2024-09-10T10:07:20.043 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37052.log.gz 2024-09-10T10:07:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41843.log 2024-09-10T10:07:20.044 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45992.log.gz 2024-09-10T10:07:20.045 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43825.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz 2024-09-10T10:07:20.045 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2024-09-10T10:07:20.047 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41843.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41843.log.gz 2024-09-10T10:07:20.047 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48420.log 2024-09-10T10:07:20.049 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49977.log 2024-09-10T10:07:20.049 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2024-09-10T10:07:20.049 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48420.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48420.log.gz 2024-09-10T10:07:20.049 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50590.log 2024-09-10T10:07:20.050 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49977.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49977.log.gz 2024-09-10T10:07:20.050 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47316.log 2024-09-10T10:07:20.052 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37782.log 2024-09-10T10:07:20.052 INFO:teuthology.orchestra.run.smithi110.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50590.log.gz 2024-09-10T10:07:20.053 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47316.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47316.log.gz 2024-09-10T10:07:20.054 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37782.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37782.log.gz 2024-09-10T10:07:20.054 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36591.log 2024-09-10T10:07:20.055 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33778.log 2024-09-10T10:07:20.057 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41934.log 2024-09-10T10:07:20.057 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36591.log.gz 2024-09-10T10:07:20.058 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41478.log 2024-09-10T10:07:20.058 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33778.log.gz 2024-09-10T10:07:20.059 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43326.log 2024-09-10T10:07:20.059 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41934.log.gz 2024-09-10T10:07:20.061 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41478.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41478.log.gz 2024-09-10T10:07:20.061 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35068.log 2024-09-10T10:07:20.062 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43326.log.gz 2024-09-10T10:07:20.062 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43591.log 2024-09-10T10:07:20.063 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46533.log 2024-09-10T10:07:20.064 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35068.log.gz 2024-09-10T10:07:20.067 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43591.log: /var/log/ceph/ceph-client.admin.46533.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43591.log.gz 2024-09-10T10:07:20.067 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46533.log.gz 2024-09-10T10:07:20.067 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35938.log 2024-09-10T10:07:20.068 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35399.log 2024-09-10T10:07:20.069 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40311.log 2024-09-10T10:07:20.069 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35938.log.gz 2024-09-10T10:07:20.070 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44965.log 2024-09-10T10:07:20.070 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35399.log.gz 2024-09-10T10:07:20.071 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40311.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40311.log.gz 2024-09-10T10:07:20.072 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52066.log 2024-09-10T10:07:20.072 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44965.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44965.log.gz 2024-09-10T10:07:20.073 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52319.log 2024-09-10T10:07:20.074 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33439.log 2024-09-10T10:07:20.074 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52066.log.gz 2024-09-10T10:07:20.076 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52319.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52319.log.gz 2024-09-10T10:07:20.076 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47150.log 2024-09-10T10:07:20.077 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33439.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33439.log.gz 2024-09-10T10:07:20.077 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39338.log 2024-09-10T10:07:20.078 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35912.log 2024-09-10T10:07:20.078 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47150.log.gz 2024-09-10T10:07:20.079 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51468.log 2024-09-10T10:07:20.080 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39338.log.gz 2024-09-10T10:07:20.080 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35912.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35912.log.gz 2024-09-10T10:07:20.081 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38210.log 2024-09-10T10:07:20.082 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51468.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51468.log.gzgzip 2024-09-10T10:07:20.082 INFO:teuthology.orchestra.run.smithi110.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-09-10T10:07:20.083 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49533.log 2024-09-10T10:07:20.083 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38210.log.gz 2024-09-10T10:07:20.084 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48870.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz 2024-09-10T10:07:20.084 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48342.log 2024-09-10T10:07:20.085 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49533.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49533.log.gz 2024-09-10T10:07:20.086 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52371.log 2024-09-10T10:07:20.087 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48316.log 2024-09-10T10:07:20.087 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48342.log.gz 2024-09-10T10:07:20.088 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52371.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52371.log.gz 2024-09-10T10:07:20.089 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39848.log 2024-09-10T10:07:20.089 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48316.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48316.log.gz 2024-09-10T10:07:20.090 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42295.log 2024-09-10T10:07:20.091 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46637.log 2024-09-10T10:07:20.091 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39848.log.gz 2024-09-10T10:07:20.092 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.42295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42295.log.gz 2024-09-10T10:07:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46637.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46637.log.gz 2024-09-10T10:07:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49455.log 2024-09-10T10:07:20.095 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34866.log 2024-09-10T10:07:20.096 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52843.log 2024-09-10T10:07:20.097 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49455.log: 89.1%/var/log/ceph/ceph-client.admin.34866.log: -- replaced with /var/log/ceph/ceph-client.admin.49455.log.gz 2024-09-10T10:07:20.097 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34866.log.gz 2024-09-10T10:07:20.098 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52843.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52843.log.gz 2024-09-10T10:07:20.099 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51349.log 2024-09-10T10:07:20.100 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51628.log 2024-09-10T10:07:20.102 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43705.log 2024-09-10T10:07:20.102 INFO:teuthology.orchestra.run.smithi110.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51349.log.gz 2024-09-10T10:07:20.102 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51317.log 2024-09-10T10:07:20.103 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51628.log.gz 2024-09-10T10:07:20.103 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43705.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43705.log.gz 2024-09-10T10:07:20.104 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42120.log 2024-09-10T10:07:20.105 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42870.log 2024-09-10T10:07:20.105 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51317.log.gz 2024-09-10T10:07:20.106 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44563.log 2024-09-10T10:07:20.106 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42120.log.gz 2024-09-10T10:07:20.107 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42870.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42870.log.gz 2024-09-10T10:07:20.108 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36643.log 2024-09-10T10:07:20.109 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53077.log 2024-09-10T10:07:20.109 INFO:teuthology.orchestra.run.smithi110.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44563.log.gz 2024-09-10T10:07:20.110 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46702.log 2024-09-10T10:07:20.110 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36643.log.gz 2024-09-10T10:07:20.111 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.53077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34900.log 2024-09-10T10:07:20.111 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53077.log.gz 2024-09-10T10:07:20.113 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46702.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46702.log.gz 2024-09-10T10:07:20.113 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39467.log 2024-09-10T10:07:20.113 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34900.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34900.log.gz 2024-09-10T10:07:20.114 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34271.log 2024-09-10T10:07:20.115 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50772.log 2024-09-10T10:07:20.116 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39467.log.gz 2024-09-10T10:07:20.116 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41608.log 2024-09-10T10:07:20.116 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34271.log.gz 2024-09-10T10:07:20.118 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48446.log 2024-09-10T10:07:20.118 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50772.log.gz 2024-09-10T10:07:20.119 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41608.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41608.log.gz 2024-09-10T10:07:20.119 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36182.log 2024-09-10T10:07:20.120 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48446.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48446.log.gz 2024-09-10T10:07:20.120 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33543.log 2024-09-10T10:07:20.121 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52713.log 2024-09-10T10:07:20.122 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36182.log.gz 2024-09-10T10:07:20.123 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33543.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33543.log.gz 2024-09-10T10:07:20.124 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39796.log 2024-09-10T10:07:20.124 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52713.log.gz 2024-09-10T10:07:20.125 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40023.log 2024-09-10T10:07:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42312.log 2024-09-10T10:07:20.127 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39796.log.gz 2024-09-10T10:07:20.127 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42583.log 2024-09-10T10:07:20.128 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40023.log.gz 2024-09-10T10:07:20.128 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42312.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42312.log.gz 2024-09-10T10:07:20.129 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50372.log 2024-09-10T10:07:20.130 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50424.log 2024-09-10T10:07:20.130 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42583.log.gz 2024-09-10T10:07:20.131 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47644.log 2024-09-10T10:07:20.131 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50372.log.gz 2024-09-10T10:07:20.132 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50424.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50424.log.gz 2024-09-10T10:07:20.133 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35833.log 2024-09-10T10:07:20.133 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47644.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47644.log.gz 2024-09-10T10:07:20.134 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34349.log 2024-09-10T10:07:20.135 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-09-10T10:07:20.135 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35833.log.gz 2024-09-10T10:07:20.137 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34349.log: /var/log/ceph/ceph-client.admin.42390.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34349.log.gz 2024-09-10T10:07:20.137 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-09-10T10:07:20.137 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35964.log 2024-09-10T10:07:20.138 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T10:07:20.140 INFO:teuthology.orchestra.run.smithi145.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T10:07:20.141 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47519.log 2024-09-10T10:07:20.141 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35964.log.gz 2024-09-10T10:07:20.143 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.47519.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47519.log.gz 2024-09-10T10:07:20.143 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44626.log 2024-09-10T10:07:20.143 INFO:teuthology.orchestra.run.smithi110.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T10:07:20.144 INFO:teuthology.orchestra.run.smithi145.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T10:07:20.144 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33986.log 2024-09-10T10:07:20.146 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33977.log 2024-09-10T10:07:20.146 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44626.log.gz 2024-09-10T10:07:20.146 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33986.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33986.log.gz 2024-09-10T10:07:20.148 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33977.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33977.log.gz 2024-09-10T10:07:20.148 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33968.log 2024-09-10T10:07:20.149 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52999.log 2024-09-10T10:07:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46106.log 2024-09-10T10:07:20.151 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33968.log.gz 2024-09-10T10:07:20.152 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44508.log 2024-09-10T10:07:20.152 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52999.log.gz 2024-09-10T10:07:20.153 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37752.log 2024-09-10T10:07:20.153 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46106.log.gz 2024-09-10T10:07:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44508.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44508.log.gz 2024-09-10T10:07:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43513.log 2024-09-10T10:07:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37752.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37752.log.gz 2024-09-10T10:07:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41286.log 2024-09-10T10:07:20.157 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37714.log 2024-09-10T10:07:20.157 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43513.log.gz 2024-09-10T10:07:20.158 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41286.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41286.log.gz 2024-09-10T10:07:20.159 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41693.log 2024-09-10T10:07:20.159 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37714.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37714.log.gzgzip 2024-09-10T10:07:20.160 INFO:teuthology.orchestra.run.smithi110.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41504.log 2024-09-10T10:07:20.161 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47669.log 2024-09-10T10:07:20.161 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41693.log.gz 2024-09-10T10:07:20.162 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41504.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41504.log.gz 2024-09-10T10:07:20.163 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47669.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47669.log.gz 2024-09-10T10:07:20.164 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-09-10T10:07:20.165 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43903.log 2024-09-10T10:07:20.166 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47444.log 2024-09-10T10:07:20.166 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2024-09-10T10:07:20.167 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35504.log 2024-09-10T10:07:20.167 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43903.log.gz 2024-09-10T10:07:20.168 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47444.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47444.log.gz 2024-09-10T10:07:20.169 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46611.log 2024-09-10T10:07:20.169 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35504.log.gz 2024-09-10T10:07:20.170 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45747.log 2024-09-10T10:07:20.172 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42479.log 2024-09-10T10:07:20.172 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46611.log.gz 2024-09-10T10:07:20.172 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45043.log 2024-09-10T10:07:20.173 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45747.log.gz 2024-09-10T10:07:20.173 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42479.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42479.log.gz 2024-09-10T10:07:20.174 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46937.log 2024-09-10T10:07:20.175 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-09-10T10:07:20.175 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45043.log.gz 2024-09-10T10:07:20.176 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36244.log 2024-09-10T10:07:20.176 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46937.log.gz 2024-09-10T10:07:20.178 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52765.log 2024-09-10T10:07:20.178 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-09-10T10:07:20.178 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45395.log 2024-09-10T10:07:20.179 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36244.log.gz 2024-09-10T10:07:20.180 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45607.log 2024-09-10T10:07:20.180 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52765.log.gz 2024-09-10T10:07:20.181 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45395.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45395.log.gz 2024-09-10T10:07:20.182 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45607.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45607.log.gz 2024-09-10T10:07:20.183 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44348.log 2024-09-10T10:07:20.184 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47072.log 2024-09-10T10:07:20.185 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36487.log 2024-09-10T10:07:20.186 INFO:teuthology.orchestra.run.smithi110.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44348.log.gz 2024-09-10T10:07:20.186 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-09-10T10:07:20.187 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47072.log.gz 2024-09-10T10:07:20.187 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48740.log 2024-09-10T10:07:20.187 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36487.log.gz 2024-09-10T10:07:20.189 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35042.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-09-10T10:07:20.189 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40077.log 2024-09-10T10:07:20.190 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48740.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48740.log.gz 2024-09-10T10:07:20.190 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49225.log 2024-09-10T10:07:20.192 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52661.log 2024-09-10T10:07:20.192 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40077.log.gz 2024-09-10T10:07:20.193 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49225.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49225.log.gz 2024-09-10T10:07:20.193 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47794.log 2024-09-10T10:07:20.194 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44085.log 2024-09-10T10:07:20.194 INFO:teuthology.orchestra.run.smithi110.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52661.log.gz 2024-09-10T10:07:20.195 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-09-10T10:07:20.195 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47794.log.gz 2024-09-10T10:07:20.197 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37935.log 2024-09-10T10:07:20.197 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44085.log.gz 2024-09-10T10:07:20.198 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50268.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-09-10T10:07:20.198 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41061.log 2024-09-10T10:07:20.199 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37935.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37935.log.gz 2024-09-10T10:07:20.200 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39248.log 2024-09-10T10:07:20.201 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41061.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41061.log.gz 2024-09-10T10:07:20.201 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51284.log 2024-09-10T10:07:20.202 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49795.log 2024-09-10T10:07:20.202 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39248.log.gz 2024-09-10T10:07:20.203 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49743.log 2024-09-10T10:07:20.204 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51284.log.gz 2024-09-10T10:07:20.204 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-09-10T10:07:20.205 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49795.log.gz 2024-09-10T10:07:20.206 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36403.log 2024-09-10T10:07:20.206 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49743.log.gz 2024-09-10T10:07:20.207 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42527.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-09-10T10:07:20.208 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41725.log 2024-09-10T10:07:20.208 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36403.log.gz 2024-09-10T10:07:20.209 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36617.log 2024-09-10T10:07:20.210 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48219.log 2024-09-10T10:07:20.211 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41725.log.gz 2024-09-10T10:07:20.211 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36617.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36617.log.gz 2024-09-10T10:07:20.212 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28970.log 2024-09-10T10:07:20.213 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48219.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48219.log.gz 2024-09-10T10:07:20.213 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37505.log 2024-09-10T10:07:20.214 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.28970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2024-09-10T10:07:20.214 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28970.log.gz 2024-09-10T10:07:20.215 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.37505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37505.log.gz 2024-09-10T10:07:20.215 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51496.log 2024-09-10T10:07:20.216 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34990.log 2024-09-10T10:07:20.217 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2024-09-10T10:07:20.217 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47368.log 2024-09-10T10:07:20.217 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51496.log.gz 2024-09-10T10:07:20.219 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34990.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34990.log.gz 2024-09-10T10:07:20.220 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46304.log 2024-09-10T10:07:20.220 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47368.log.gz 2024-09-10T10:07:20.221 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51909.log 2024-09-10T10:07:20.222 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28854.log 2024-09-10T10:07:20.222 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gz 2024-09-10T10:07:20.223 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.51909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51909.log.gz 2024-09-10T10:07:20.225 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.28854.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28854.log.gz 2024-09-10T10:07:20.225 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45730.log 2024-09-10T10:07:20.226 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39574.log 2024-09-10T10:07:20.227 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.45730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38788.log 2024-09-10T10:07:20.227 INFO:teuthology.orchestra.run.smithi110.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45730.log.gz 2024-09-10T10:07:20.229 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46252.log 2024-09-10T10:07:20.229 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39574.log.gz 2024-09-10T10:07:20.229 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38788.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38788.log.gz 2024-09-10T10:07:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46252.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46252.log.gz 2024-09-10T10:07:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50138.log 2024-09-10T10:07:20.232 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35094.log 2024-09-10T10:07:20.234 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44911.log 2024-09-10T10:07:20.234 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50138.log.gz 2024-09-10T10:07:20.235 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35094.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35094.log.gz 2024-09-10T10:07:20.236 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44911.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44911.log.gz 2024-09-10T10:07:20.236 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45555.log 2024-09-10T10:07:20.237 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37304.log 2024-09-10T10:07:20.239 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52040.log 2024-09-10T10:07:20.239 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45555.log.gz 2024-09-10T10:07:20.240 INFO:teuthology.orchestra.run.smithi145.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T10:07:20.240 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33335.log 2024-09-10T10:07:20.241 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37304.log.gz 2024-09-10T10:07:20.241 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40977.log 2024-09-10T10:07:20.241 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52040.log.gz 2024-09-10T10:07:20.242 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33335.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33335.log.gz 2024-09-10T10:07:20.243 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-09-10T10:07:20.243 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40977.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40977.log.gz 2024-09-10T10:07:20.244 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47569.log 2024-09-10T10:07:20.245 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49912.log 2024-09-10T10:07:20.245 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gz 2024-09-10T10:07:20.246 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47569.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.47569.log.gz 2024-09-10T10:07:20.247 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46382.log 2024-09-10T10:07:20.248 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49912.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.50740.log 2024-09-10T10:07:20.248 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49912.log.gz 2024-09-10T10:07:20.249 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33387.log 2024-09-10T10:07:20.249 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46382.log.gz 2024-09-10T10:07:20.250 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50740.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50740.log.gz 2024-09-10T10:07:20.251 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33387.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33387.log.gz 2024-09-10T10:07:20.251 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43085.log 2024-09-10T10:07:20.253 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45529.log 2024-09-10T10:07:20.256 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43085.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43085.log.gz 2024-09-10T10:07:20.257 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45529.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45529.log.gz 2024-09-10T10:07:20.257 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39042.log 2024-09-10T10:07:20.257 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39770.log 2024-09-10T10:07:20.258 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41960.log 2024-09-10T10:07:20.258 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39042.log.gz 2024-09-10T10:07:20.259 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50798.log 2024-09-10T10:07:20.260 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39770.log.gz 2024-09-10T10:07:20.260 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42416.log 2024-09-10T10:07:20.260 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41960.log.gz 2024-09-10T10:07:20.262 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50798.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50798.log.gz 2024-09-10T10:07:20.262 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2024-09-10T10:07:20.262 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42416.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42416.log.gz 2024-09-10T10:07:20.263 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52583.log 2024-09-10T10:07:20.265 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47844.log 2024-09-10T10:07:20.265 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2024-09-10T10:07:20.265 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52583.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52583.log.gz 2024-09-10T10:07:20.266 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35452.log 2024-09-10T10:07:20.267 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47844.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.47844.log.gz 2024-09-10T10:07:20.267 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36130.log 2024-09-10T10:07:20.268 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39493.log 2024-09-10T10:07:20.269 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35452.log.gz 2024-09-10T10:07:20.269 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36130.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36130.log.gz 2024-09-10T10:07:20.270 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39493.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39493.log.gz 2024-09-10T10:07:20.271 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49280.log 2024-09-10T10:07:20.271 INFO:teuthology.orchestra.run.smithi145.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T10:07:20.272 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47394.log 2024-09-10T10:07:20.274 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39522.log 2024-09-10T10:07:20.274 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49280.log.gz 2024-09-10T10:07:20.274 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47394.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47394.log.gz 2024-09-10T10:07:20.275 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39522.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39522.log.gz 2024-09-10T10:07:20.276 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35250.log 2024-09-10T10:07:20.277 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48922.log 2024-09-10T10:07:20.279 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47098.log 2024-09-10T10:07:20.279 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35250.log.gz 2024-09-10T10:07:20.279 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37081.log 2024-09-10T10:07:20.280 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48922.log.gz 2024-09-10T10:07:20.281 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41371.log 2024-09-10T10:07:20.281 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47098.log.gz 2024-09-10T10:07:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37081.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37081.log.gz 2024-09-10T10:07:20.282 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T10:07:20.283 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41371.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41371.log.gz 2024-09-10T10:07:20.283 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T10:07:20.284 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40285.log 2024-09-10T10:07:20.287 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.40285.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40285.log.gz 2024-09-10T10:07:20.288 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50216.log 2024-09-10T10:07:20.289 INFO:teuthology.orchestra.run.smithi110.stderr: 89.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T10:07:20.289 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37814.log 2024-09-10T10:07:20.290 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50216.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50216.log.gz 2024-09-10T10:07:20.292 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48792.log 2024-09-10T10:07:20.292 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37814.log.gz 2024-09-10T10:07:20.293 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50642.log 2024-09-10T10:07:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48792.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48792.log.gz 2024-09-10T10:07:20.295 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50642.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50642.log.gz 2024-09-10T10:07:20.296 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34570.log 2024-09-10T10:07:20.297 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2024-09-10T10:07:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34570.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34570.log.gz 2024-09-10T10:07:20.299 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42016.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2024-09-10T10:07:20.300 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46199.log 2024-09-10T10:07:20.301 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T10:07:20.303 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46199.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46199.log.gz 2024-09-10T10:07:20.304 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52397.log 2024-09-10T10:07:20.306 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40834.log 2024-09-10T10:07:20.307 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35673.log 2024-09-10T10:07:20.307 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52397.log.gz 2024-09-10T10:07:20.307 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38606.log 2024-09-10T10:07:20.312 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35673.log: /var/log/ceph/ceph-client.admin.38606.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.38606.log.gz 2024-09-10T10:07:20.312 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52095.log 2024-09-10T10:07:20.312 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35673.log.gz 2024-09-10T10:07:20.313 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40834.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40834.log.gz 2024-09-10T10:07:20.314 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52095.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52095.log.gz 2024-09-10T10:07:20.315 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44401.log 2024-09-10T10:07:20.322 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36903.log 2024-09-10T10:07:20.323 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44401.log.gz 2024-09-10T10:07:20.325 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48975.log 2024-09-10T10:07:20.330 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48975.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48975.log.gz 2024-09-10T10:07:20.330 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36903.log.gz 2024-09-10T10:07:20.330 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51680.log 2024-09-10T10:07:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51680.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51680.log.gz 2024-09-10T10:07:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46963.log 2024-09-10T10:07:20.339 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46963.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46963.log.gz 2024-09-10T10:07:20.339 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50998.log 2024-09-10T10:07:20.339 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50998.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50998.log.gz 2024-09-10T10:07:20.339 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47894.log 2024-09-10T10:07:20.342 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47894.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47894.log.gz 2024-09-10T10:07:20.342 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34682.log 2024-09-10T10:07:20.343 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43877.log 2024-09-10T10:07:20.345 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34682.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34682.log.gz 2024-09-10T10:07:20.345 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33980.log 2024-09-10T10:07:20.345 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43877.log.gz 2024-09-10T10:07:20.348 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33980.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33980.log.gz 2024-09-10T10:07:20.348 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49251.log 2024-09-10T10:07:20.354 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49251.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49251.log.gz 2024-09-10T10:07:20.354 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42440.log 2024-09-10T10:07:20.354 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42440.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42440.log.gz 2024-09-10T10:07:20.355 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-09-10T10:07:20.357 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45799.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-09-10T10:07:20.357 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-09-10T10:07:20.359 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-09-10T10:07:20.360 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43460.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-09-10T10:07:20.362 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37000.log 2024-09-10T10:07:20.364 INFO:teuthology.orchestra.run.smithi110.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-09-10T10:07:20.366 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37000.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37000.log.gz 2024-09-10T10:07:20.366 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42695.log 2024-09-10T10:07:20.369 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42695.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42695.log.gz 2024-09-10T10:07:20.369 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49769.log 2024-09-10T10:07:20.372 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49769.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49769.log.gz 2024-09-10T10:07:20.372 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43539.log 2024-09-10T10:07:20.375 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-09-10T10:07:20.375 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43539.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43539.log.gz 2024-09-10T10:07:20.378 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44374.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2024-09-10T10:07:20.378 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46077.log 2024-09-10T10:07:20.381 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46077.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46077.log.gz 2024-09-10T10:07:20.381 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45477.log 2024-09-10T10:07:20.384 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45477.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45477.log.gz 2024-09-10T10:07:20.384 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42094.log 2024-09-10T10:07:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33779.log 2024-09-10T10:07:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42094.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42094.log.gz 2024-09-10T10:07:20.390 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33779.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33779.log.gz 2024-09-10T10:07:20.390 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41530.log 2024-09-10T10:07:20.393 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41530.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41530.log.gz 2024-09-10T10:07:20.393 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53051.log 2024-09-10T10:07:20.396 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.53051.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53051.log.gz 2024-09-10T10:07:20.396 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51978.log 2024-09-10T10:07:20.399 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51978.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51978.log.gz 2024-09-10T10:07:20.399 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36539.log 2024-09-10T10:07:20.401 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36539.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36539.log.gz 2024-09-10T10:07:20.401 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48034.log 2024-09-10T10:07:20.403 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52188.log 2024-09-10T10:07:20.404 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48034.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48034.log.gz 2024-09-10T10:07:20.405 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36317.log 2024-09-10T10:07:20.408 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36317.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36317.log.gz 2024-09-10T10:07:20.408 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49507.log 2024-09-10T10:07:20.409 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52188.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52188.log.gz 2024-09-10T10:07:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49507.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49507.log.gz 2024-09-10T10:07:20.412 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35224.log 2024-09-10T10:07:20.415 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35224.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35224.log.gz 2024-09-10T10:07:20.415 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48368.log 2024-09-10T10:07:20.418 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48368.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48368.log.gz 2024-09-10T10:07:20.418 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39874.log 2024-09-10T10:07:20.421 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51076.log 2024-09-10T10:07:20.421 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39874.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39874.log.gz 2024-09-10T10:07:20.422 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34708.log 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51076.log.gz 2024-09-10T10:07:20.422 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-10T10:07:20.424 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34708.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34708.log.gz 2024-09-10T10:07:20.425 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46507.log 2024-09-10T10:07:20.427 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46507.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46507.log.gz 2024-09-10T10:07:20.427 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38814.log 2024-09-10T10:07:20.430 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38814.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38814.log.gz 2024-09-10T10:07:20.430 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44736.log 2024-09-10T10:07:20.433 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44736.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44736.log.gz 2024-09-10T10:07:20.433 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43378.log 2024-09-10T10:07:20.436 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43378.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43378.log.gz 2024-09-10T10:07:20.436 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40881.log 2024-09-10T10:07:20.439 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51442.log 2024-09-10T10:07:20.439 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.40881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40881.log.gz 2024-09-10T10:07:20.442 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51442.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51442.log.gz 2024-09-10T10:07:20.442 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51050.log 2024-09-10T10:07:20.445 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51050.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51050.log.gz 2024-09-10T10:07:20.445 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35478.log 2024-09-10T10:07:20.448 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35478.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35478.log.gz 2024-09-10T10:07:20.448 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37196.log 2024-09-10T10:07:20.449 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40338.log 2024-09-10T10:07:20.451 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37196.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37196.log.gz 2024-09-10T10:07:20.454 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40338.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40338.log.gz 2024-09-10T10:07:20.454 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40366.log 2024-09-10T10:07:20.456 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38300.log 2024-09-10T10:07:20.457 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40366.log.gz 2024-09-10T10:07:20.458 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41556.log 2024-09-10T10:07:20.463 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38300.log: /var/log/ceph/ceph-client.admin.41556.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41556.log.gz 2024-09-10T10:07:20.463 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38300.log.gz 2024-09-10T10:07:20.463 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49821.log 2024-09-10T10:07:20.465 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49821.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49821.log.gz 2024-09-10T10:07:20.466 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41035.log 2024-09-10T10:07:20.469 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41035.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41035.log.gz 2024-09-10T10:07:20.469 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33984.log 2024-09-10T10:07:20.472 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33984.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33984.log.gz 2024-09-10T10:07:20.472 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35282.log 2024-09-10T10:07:20.475 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36288.log 2024-09-10T10:07:20.475 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35282.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35282.log.gz 2024-09-10T10:07:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36288.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36288.log.gz 2024-09-10T10:07:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39364.log 2024-09-10T10:07:20.481 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39364.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39364.log.gz 2024-09-10T10:07:20.481 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39094.log 2024-09-10T10:07:20.486 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39094.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39094.log.gz 2024-09-10T10:07:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37026.log 2024-09-10T10:07:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37026.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37026.log.gz 2024-09-10T10:07:20.487 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50190.log 2024-09-10T10:07:20.489 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50190.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50190.log.gz 2024-09-10T10:07:20.490 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37684.log 2024-09-10T10:07:20.491 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33673.log 2024-09-10T10:07:20.492 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37684.log: /var/log/ceph/ceph-client.admin.33673.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37684.log.gz 2024-09-10T10:07:20.493 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52739.log 2024-09-10T10:07:20.493 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33673.log.gz 2024-09-10T10:07:20.495 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52739.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52739.log.gz 2024-09-10T10:07:20.496 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33969.log 2024-09-10T10:07:20.499 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33969.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33969.log.gz 2024-09-10T10:07:20.499 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51416.log 2024-09-10T10:07:20.500 INFO:teuthology.orchestra.run.smithi145.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T10:07:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51416.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51416.log.gz 2024-09-10T10:07:20.502 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52004.log 2024-09-10T10:07:20.506 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-09-10T10:07:20.506 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52004.log: /var/log/ceph/ceph-client.admin.43955.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52004.log.gz 2024-09-10T10:07:20.506 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43235.log 2024-09-10T10:07:20.506 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-09-10T10:07:20.508 INFO:teuthology.orchestra.run.smithi145.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T10:07:20.508 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43235.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43235.log.gz 2024-09-10T10:07:20.509 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37522.log 2024-09-10T10:07:20.514 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37522.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37522.log.gz 2024-09-10T10:07:20.514 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37356.log 2024-09-10T10:07:20.514 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37356.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37356.log.gz 2024-09-10T10:07:20.515 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-09-10T10:07:20.518 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50714.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-09-10T10:07:20.518 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38028.log 2024-09-10T10:07:20.520 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38028.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz 2024-09-10T10:07:20.521 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40394.log 2024-09-10T10:07:20.523 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41087.log 2024-09-10T10:07:20.524 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40394.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40394.log.gz 2024-09-10T10:07:20.526 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41087.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41087.log.gz 2024-09-10T10:07:20.526 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39822.log 2024-09-10T10:07:20.529 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39822.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39822.log.gz 2024-09-10T10:07:20.530 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35781.log 2024-09-10T10:07:20.532 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35781.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35781.log.gz 2024-09-10T10:07:20.533 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41791.log 2024-09-10T10:07:20.535 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48166.log 2024-09-10T10:07:20.535 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41791.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41791.log.gz 2024-09-10T10:07:20.538 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48166.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48166.log.gz 2024-09-10T10:07:20.538 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51862.log 2024-09-10T10:07:20.541 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51862.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51862.log.gz 2024-09-10T10:07:20.541 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44684.log 2024-09-10T10:07:20.544 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46793.log 2024-09-10T10:07:20.546 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44684.log: /var/log/ceph/ceph-client.admin.46793.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46793.log.gz 2024-09-10T10:07:20.547 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49481.log -- replaced with /var/log/ceph/ceph-client.admin.44684.log.gz 2024-09-10T10:07:20.547 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-10T10:07:20.548 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52477.log 2024-09-10T10:07:20.555 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49481.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49481.log.gz 2024-09-10T10:07:20.556 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52477.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52477.log.gz 2024-09-10T10:07:20.556 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45160.log 2024-09-10T10:07:20.556 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45160.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45160.log.gz 2024-09-10T10:07:20.556 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52687.log 2024-09-10T10:07:20.557 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52687.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52687.log.gz 2024-09-10T10:07:20.557 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33307.log 2024-09-10T10:07:20.560 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33699.log 2024-09-10T10:07:20.561 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33307.log.gz 2024-09-10T10:07:20.561 INFO:teuthology.orchestra.run.smithi145.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T10:07:20.562 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33517.log 2024-09-10T10:07:20.566 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33699.log: /var/log/ceph/ceph-client.admin.33517.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33253.log 2024-09-10T10:07:20.567 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33699.log.gz 2024-09-10T10:07:20.567 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33517.log.gz 2024-09-10T10:07:20.569 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42643.log 2024-09-10T10:07:20.571 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33253.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33253.log.gz 2024-09-10T10:07:20.573 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48896.log 2024-09-10T10:07:20.574 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42643.log.gz 2024-09-10T10:07:20.575 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41203.log 2024-09-10T10:07:20.578 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48896.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48896.log.gz 2024-09-10T10:07:20.580 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33647.log 2024-09-10T10:07:20.581 INFO:teuthology.orchestra.run.smithi110.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41203.log.gz 2024-09-10T10:07:20.584 INFO:teuthology.orchestra.run.smithi110.stderr:gzip/var/log/ceph/ceph-client.admin.33647.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33647.log.gz 2024-09-10T10:07:20.584 INFO:teuthology.orchestra.run.smithi110.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44456.log 2024-09-10T10:07:20.587 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44456.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44456.log.gz 2024-09-10T10:07:20.587 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40051.log 2024-09-10T10:07:20.590 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40051.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40051.log.gz 2024-09-10T10:07:20.590 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41260.log 2024-09-10T10:07:20.591 INFO:teuthology.orchestra.run.smithi145.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T10:07:20.593 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45581.log 2024-09-10T10:07:20.593 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41260.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41260.log.gz 2024-09-10T10:07:20.593 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-09-10T10:07:20.594 INFO:teuthology.orchestra.run.smithi145.stderr:real 0m0.865s 2024-09-10T10:07:20.594 INFO:teuthology.orchestra.run.smithi145.stderr:user 0m4.499s 2024-09-10T10:07:20.594 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m0.390s 2024-09-10T10:07:20.596 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45581.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45581.log.gz 2024-09-10T10:07:20.596 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2024-09-10T10:07:20.599 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43783.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2024-09-10T10:07:20.599 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49025.log 2024-09-10T10:07:20.603 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47124.log 2024-09-10T10:07:20.604 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49025.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49025.log.gz 2024-09-10T10:07:20.604 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47273.log 2024-09-10T10:07:20.604 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47124.log.gz 2024-09-10T10:07:20.607 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.47273.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47273.log.gz 2024-09-10T10:07:20.607 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38058.log 2024-09-10T10:07:20.609 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38476.log 2024-09-10T10:07:20.610 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38058.log.gz 2024-09-10T10:07:20.613 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5/var/log/ceph/ceph-client.admin.38476.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38476.log.gz 2024-09-10T10:07:20.613 INFO:teuthology.orchestra.run.smithi110.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53155.log 2024-09-10T10:07:20.616 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.53155.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53155.log.gz 2024-09-10T10:07:20.616 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49585.log 2024-09-10T10:07:20.617 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49613.log 2024-09-10T10:07:20.619 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52921.log/var/log/ceph/ceph-client.admin.49613.log: 2024-09-10T10:07:20.623 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49613.log.gz 2024-09-10T10:07:20.623 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49585.log.gz 2024-09-10T10:07:20.624 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52921.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52921.log.gz 2024-09-10T10:07:20.624 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42248.log 2024-09-10T10:07:20.626 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42248.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-09-10T10:07:20.626 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40654.log 2024-09-10T10:07:20.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-10T10:07:20.628 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39997.log 2024-09-10T10:07:20.629 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40654.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40654.log.gz 2024-09-10T10:07:20.631 INFO:teuthology.orchestra.run.smithi110.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T10:07:20.632 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34656.log 2024-09-10T10:07:20.632 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39997.log.gz 2024-09-10T10:07:20.633 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42042.log 2024-09-10T10:07:20.634 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.2.34656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.34656.log.gz 2024-09-10T10:07:20.635 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36935.log 2024-09-10T10:07:20.636 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42042.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42042.log.gz 2024-09-10T10:07:20.637 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36935.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36935.log.gz 2024-09-10T10:07:20.637 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42146.log 2024-09-10T10:07:20.639 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39945.log 2024-09-10T10:07:20.640 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42146.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42146.log.gz 2024-09-10T10:07:20.641 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39945.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39945.log.gz 2024-09-10T10:07:20.641 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34528.log 2024-09-10T10:07:20.643 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-09-10T10:07:20.644 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.34528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34528.log.gz 2024-09-10T10:07:20.645 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49364.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-09-10T10:07:20.646 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47342.log 2024-09-10T10:07:20.647 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46993.log 2024-09-10T10:07:20.648 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47342.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47342.log.gz 2024-09-10T10:07:20.649 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46993.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46993.log.gz 2024-09-10T10:07:20.650 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47619.log 2024-09-10T10:07:20.651 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37628.log 2024-09-10T10:07:20.653 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47619.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47619.log.gz 2024-09-10T10:07:20.653 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37628.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37628.log.gz 2024-09-10T10:07:20.654 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44482.log 2024-09-10T10:07:20.655 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38352.log 2024-09-10T10:07:20.657 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44482.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44482.log.gz 2024-09-10T10:07:20.658 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38352.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38352.log.gz 2024-09-10T10:07:20.658 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47919.log 2024-09-10T10:07:20.659 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47719.log 2024-09-10T10:07:20.664 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47919.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47919.log.gz 2024-09-10T10:07:20.665 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47719.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47719.log.gz 2024-09-10T10:07:20.665 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38723.log 2024-09-10T10:07:20.665 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36113.log 2024-09-10T10:07:20.665 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38723.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38723.log.gz 2024-09-10T10:07:20.666 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.36113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36113.log.gz 2024-09-10T10:07:20.666 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50398.log 2024-09-10T10:07:20.667 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34838.log 2024-09-10T10:07:20.668 INFO:teuthology.orchestra.run.smithi110.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T10:07:20.668 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50398.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50398.log.gz 2024-09-10T10:07:20.670 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34838.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34838.log.gz 2024-09-10T10:07:20.670 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41817.log 2024-09-10T10:07:20.671 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41151.log 2024-09-10T10:07:20.672 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51524.log 2024-09-10T10:07:20.673 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41817.log.gz 2024-09-10T10:07:20.673 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.41151.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41151.log.gz 2024-09-10T10:07:20.675 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51524.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51524.log.gz 2024-09-10T10:07:20.675 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49639.log 2024-09-10T10:07:20.677 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49072.log 2024-09-10T10:07:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.49639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39172.log 2024-09-10T10:07:20.678 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49639.log.gz 2024-09-10T10:07:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.49072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49072.log.gz 2024-09-10T10:07:20.680 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39172.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39172.log.gz 2024-09-10T10:07:20.680 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36370.log 2024-09-10T10:07:20.681 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39390.log 2024-09-10T10:07:20.683 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48264.log 2024-09-10T10:07:20.683 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36370.log.gz 2024-09-10T10:07:20.684 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.39390.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39390.log.gz 2024-09-10T10:07:20.685 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48264.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48264.log.gz 2024-09-10T10:07:20.686 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38580.log 2024-09-10T10:07:20.687 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52973.log 2024-09-10T10:07:20.688 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43408.log 2024-09-10T10:07:20.688 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38580.log.gz 2024-09-10T10:07:20.690 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52973.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52973.log.gz 2024-09-10T10:07:20.690 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43408.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43408.log.gz 2024-09-10T10:07:20.691 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38899.log 2024-09-10T10:07:20.692 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44937.log 2024-09-10T10:07:20.693 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48603.log 2024-09-10T10:07:20.694 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38899.log.gz 2024-09-10T10:07:20.694 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44937.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44937.log.gz 2024-09-10T10:07:20.696 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48603.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48603.log.gz 2024-09-10T10:07:20.696 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43209.log 2024-09-10T10:07:20.697 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51576.log 2024-09-10T10:07:20.699 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47869.log 2024-09-10T10:07:20.699 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43209.log.gz 2024-09-10T10:07:20.700 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51576.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51576.log.gz 2024-09-10T10:07:20.701 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47869.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47869.log.gz 2024-09-10T10:07:20.701 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2024-09-10T10:07:20.702 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-09-10T10:07:20.704 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35373.log 2024-09-10T10:07:20.704 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2024-09-10T10:07:20.705 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43183.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-09-10T10:07:20.706 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.35373.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35373.log.gz 2024-09-10T10:07:20.706 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2024-09-10T10:07:20.708 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42503.log 2024-09-10T10:07:20.709 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45369.log 2024-09-10T10:07:20.709 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz 2024-09-10T10:07:20.710 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.42503.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42503.log.gz 2024-09-10T10:07:20.711 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.45369.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45369.log.gz 2024-09-10T10:07:20.712 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36434.log 2024-09-10T10:07:20.713 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2024-09-10T10:07:20.714 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38502.log 2024-09-10T10:07:20.715 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36434.log.gz 2024-09-10T10:07:20.715 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38691.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2024-09-10T10:07:20.716 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38502.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38502.log.gz 2024-09-10T10:07:20.717 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46408.log 2024-09-10T10:07:20.717 INFO:teuthology.orchestra.run.smithi110.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T10:07:20.718 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34812.log 2024-09-10T10:07:20.719 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48060.log 2024-09-10T10:07:20.719 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46408.log.gz 2024-09-10T10:07:20.720 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37998.log 2024-09-10T10:07:20.720 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34812.log.gz 2024-09-10T10:07:20.721 INFO:teuthology.orchestra.run.smithi110.stderr:gzip/var/log/ceph/ceph-client.admin.48060.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50320.log 2024-09-10T10:07:20.721 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48060.log.gz 2024-09-10T10:07:20.722 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.37998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47469.log 2024-09-10T10:07:20.722 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37998.log.gz 2024-09-10T10:07:20.723 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.50320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46585.log 2024-09-10T10:07:20.723 INFO:teuthology.orchestra.run.smithi110.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50320.log.gz 2024-09-10T10:07:20.724 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.47469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36844.log 2024-09-10T10:07:20.724 INFO:teuthology.orchestra.run.smithi110.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47469.log.gz 2024-09-10T10:07:20.725 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-09-10T10:07:20.725 INFO:teuthology.orchestra.run.smithi110.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46585.log.gz 2024-09-10T10:07:20.726 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52215.log 2024-09-10T10:07:20.726 INFO:teuthology.orchestra.run.smithi110.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36844.log.gz 2024-09-10T10:07:20.727 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.44059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46670.log 2024-09-10T10:07:20.727 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2024-09-10T10:07:20.728 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34167.log 2024-09-10T10:07:20.729 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52215.log.gz 2024-09-10T10:07:20.729 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.46670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43263.log 2024-09-10T10:07:20.729 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46670.log.gz 2024-09-10T10:07:20.730 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.34167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36792.log 2024-09-10T10:07:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43263.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34167.log.gz 2024-09-10T10:07:20.731 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38404.log 2024-09-10T10:07:20.731 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43263.log.gz 2024-09-10T10:07:20.732 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48086.log 2024-09-10T10:07:20.732 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36792.log.gz 2024-09-10T10:07:20.733 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38528.log 2024-09-10T10:07:20.734 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38404.log.gz 2024-09-10T10:07:20.734 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40925.log 2024-09-10T10:07:20.735 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48086.log.gz 2024-09-10T10:07:20.735 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48818.log 2024-09-10T10:07:20.735 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38528.log.gz 2024-09-10T10:07:20.736 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.40925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43731.log 2024-09-10T10:07:20.737 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40925.log.gz 2024-09-10T10:07:20.738 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.48818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38150.log 2024-09-10T10:07:20.738 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48818.log.gz 2024-09-10T10:07:20.738 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33279.log 2024-09-10T10:07:20.739 INFO:teuthology.orchestra.run.smithi110.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43731.log.gz 2024-09-10T10:07:20.739 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51810.log 2024-09-10T10:07:20.740 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.38150.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38150.log.gz 2024-09-10T10:07:20.741 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.33279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52869.log 2024-09-10T10:07:20.741 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33279.log.gz 2024-09-10T10:07:20.742 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.51810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36513.log 2024-09-10T10:07:20.742 INFO:teuthology.orchestra.run.smithi110.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51810.log.gz 2024-09-10T10:07:20.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.52869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43757.log 2024-09-10T10:07:20.743 INFO:teuthology.orchestra.run.smithi110.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52869.log.gz 2024-09-10T10:07:20.744 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.36513.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36513.log.gz 2024-09-10T10:07:20.745 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.43757.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43757.log.gz 2024-09-10T10:07:20.764 INFO:teuthology.orchestra.run.smithi110.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T10:07:20.884 INFO:teuthology.orchestra.run.smithi110.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T10:07:20.901 INFO:teuthology.orchestra.run.smithi110.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T10:07:21.028 INFO:teuthology.orchestra.run.smithi110.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T10:07:21.030 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-10T10:07:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:real 0m1.337s 2024-09-10T10:07:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:user 0m5.888s 2024-09-10T10:07:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m1.621s 2024-09-10T10:07:21.031 INFO:tasks.ceph:Archiving logs... 2024-09-10T10:07:21.032 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/remote/smithi110/log 2024-09-10T10:07:21.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:07:22.149 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/remote/smithi145/log 2024-09-10T10:07:22.150 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:07:22.623 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T10:07:22.631 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T10:07:22.631 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:07:22.634 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:07:22.719 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T10:07:22.727 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T10:07:22.727 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:07:22.731 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:07:22.743 INFO:teuthology.orchestra.run.smithi110.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T10:07:22.743 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================== 2024-09-10T10:07:22.743 INFO:teuthology.orchestra.run.smithi110.stdout:+hv01.front.sepi 67.205.162.81 3 u 47 64 377 0.096 -0.069 0.904 2024-09-10T10:07:22.743 INFO:teuthology.orchestra.run.smithi110.stdout:*hv02.front.sepi 63.231.80.2 3 u 39 64 377 0.081 -1.656 0.465 2024-09-10T10:07:22.743 INFO:teuthology.orchestra.run.smithi110.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.072 -0.703 0.576 2024-09-10T10:07:22.743 INFO:teuthology.orchestra.run.smithi110.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-10T10:07:22.746 INFO:teuthology.orchestra.run.smithi145.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T10:07:22.746 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================== 2024-09-10T10:07:22.746 INFO:teuthology.orchestra.run.smithi145.stdout:*hv01.front.sepi 67.205.162.81 3 u 42 64 377 0.110 -0.661 1.291 2024-09-10T10:07:22.746 INFO:teuthology.orchestra.run.smithi145.stdout:+hv02.front.sepi 63.231.80.2 3 u 46 64 377 0.109 -1.571 1.239 2024-09-10T10:07:22.746 INFO:teuthology.orchestra.run.smithi145.stdout:+hv03.front.sepi 74.6.168.72 3 u 37 64 377 0.064 -1.156 1.176 2024-09-10T10:07:22.746 INFO:teuthology.orchestra.run.smithi145.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-10T10:07:22.747 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T10:07:22.755 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T10:07:22.755 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T10:07:22.806 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T10:07:22.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T10:07:22.906 INFO:teuthology.task.internal:Duration was 1259.508377 seconds 2024-09-10T10:07:22.907 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T10:07:22.957 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T10:07:22.957 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:07:22.960 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:07:23.057 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T10:07:23.057 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-10T10:07:23.057 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:07:23.078 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-09-10T10:07:23.078 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:07:23.098 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T10:07:23.098 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:07:23.126 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:07:23.177 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T10:07:23.177 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:07:23.210 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:07:23.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T10:07:23.305 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T10:07:23.305 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:07:23.325 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:07:23.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T10:07:23.354 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:07:23.370 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:07:23.384 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-10T10:07:23.403 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-09-10T10:07:23.420 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:07:23.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:07:23.454 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:07:23.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:07:23.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T10:07:23.478 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T10:07:23.479 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/remote/smithi110 2024-09-10T10:07:23.479 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:07:23.521 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899008/remote/smithi145 2024-09-10T10:07:23.521 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:07:23.546 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T10:07:23.547 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:07:23.566 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:07:23.594 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T10:07:23.604 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T10:07:23.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T10:07:23.632 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T10:07:23.632 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:07:23.635 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:07:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: 658135 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:07 /home/ubuntu/cephtest 2024-09-10T10:07:23.646 INFO:teuthology.orchestra.run.smithi145.stdout: 655441 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:07 /home/ubuntu/cephtest 2024-09-10T10:07:23.648 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T10:07:23.683 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T10:07:23.791 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1259.5083770751953 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T10:07:23.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:07:23.876 INFO:teuthology.run:pass